New User Special Price Expires in

Let's log you in.

Sign in with Facebook


Don't have a StudySoup account? Create one here!


Create a StudySoup account

Be part of our community, it's free to join!

Sign up with Facebook


Create your account
By creating an account you agree to StudySoup's terms and conditions and privacy policy

Already have a StudySoup account? Login here


by: Gaetano Dooley


Gaetano Dooley
University of Central Florida
GPA 3.6


Almost Ready


These notes were just uploaded, and will be ready to view shortly.

Purchase these notes here, or revisit this page.

Either way, we'll remind you when they're ready :)

Preview These Notes for FREE

Get a free preview of these Notes, just enter your email below.

Unlock Preview
Unlock Preview

Preview these materials now for free

Why put in your email? Get access to more of this material and other relevant free materials for your school

View Preview

About this Document

Class Notes
25 ?




Popular in Course

Popular in ENGLISH (ENG)

This 23 page Class Notes was uploaded by Gaetano Dooley on Thursday October 22, 2015. The Class Notes belongs to ESL READING at University of Central Florida taught by Staff in Fall. Since its upload, it has received 30 views. For similar materials see /class/227534/esl-reading-university-of-central-florida in ENGLISH (ENG) at University of Central Florida.

Similar to ESL READING at University of Central Florida

Popular in ENGLISH (ENG)


Reviews for ESL READING


Report this Material


What is Karma?


Karma is the currency of StudySoup.

You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/22/15
Multiagent models for partially observable environments Matthijs Spaan Institute for Systems and Robotics Instituto Superior T cnico Lisbon Portugal Reading group meeting March 26 2007 118 Overview 0 Multiagent models for partially observable environments gt Noncommunicative models gt Communicative models gt Gametheoretic models gt Some algorithms 0 Talk based on survey by Frans Oliehoek 2006 218 m INSTITUTO surzuon ricmco The DecTiger problem A toy problem decentralized tiger Nair et al 2003 Two agents two doors Opening correct door both receive treasure Opening wrong door both get attacked by a tiger Agents can open a door or listen Two noisy observations hear tiger left or light Don t know the other s actions or observations 318 Multiagent planning frameworks Aspects 0 communication 0 online vs offline o centralized vs distributed 0 cooperative vs selfinterested o observability o factored reward 418 Partially observable stochastic games Partially observable stochastic games POSGs Hansen et a1 2004 0 Extension of stochastic games Shapley 1953 0 Hence selfinterested 0 Agents do not observe each other s observations or actions 518 m INSTITUTO surzuon ricmco POSGs de nition A set 1 n ofn agents Ai is the set of actions for agent 139 01 is the set of observations for agent 139 Transition model 125 5 d where 2 6 A1 gtlt gtlt A Observation mode1p6 5 2 where 6 E 01 gtlt gtlt On Reward function Ri S gtlt A1 gtlt gtlt A gt R Each agents maximizes E 20 39thf Policy 7r 7H 7rn with m gtltt1Ai gtlt Oi gt Ai 618 m INSTITUTO surzuon rzcmco Decentralized POMDPs Decentralized partially observable Markov decision processes DecPOMDPs Bernstein et al 2002 o Cooperative version of POSGs 0 Only one reward ie reward functions are identical for each agent 0 Reward function R S gtlt A1 gtlt gtlt A gt R DecMDPs o Jointly observable DecPOMDP joint observation 6 01 on identi es the state 0 But each agents only observes 01 I R MTDP Pynadath and Tambe 2002 essentially identical to Dec POMDP 718 m Interactive POMDPs INSTITUTO surmon rscmco Interactive POMDPs Gmytrasiewicz and Doshi 2005 o For selfinterested agents 0 Each agents keeps a belief over world states and other agents models 0 An agent s model local observation history policy observation function 0 Leads to in nite hierarchy of beliefs 818 m INSTITUTO surmon ncmco Communication 0 Implicit or explicit o Implicit communication can be modeled in noncommunicative frameworks o Explicit communication Goldman and Zilberstein 2004 V informative messages V commitments V rewardspunishments o Semantics V Fixed optimize joint policy given semantics V General case optimize meanings as well 0 Potential assumptions instantaneous noisefree broadcast communication 918 m INSTITUTO surmon TECNICO DecPOMDPs with communication DecPOMDP Com Goldman and Zilberstein 2004 DecPOMDP plus 2 is the alphabet of all possible messages O39i is a message sent by agent 139 CE E a R is the cost of sending a message Reward depends on message sent Rs 2101 an an 5 Instantaneous broadcast communication Fixed semantics Two policies for domainlevel actions and for communicating Closely related model ComMTDP Pynadath and Tambe 2002 a 1018 YYYYY 0 Extensive form games 1118 Extensive form games 1 Extensive form games 0 View a POSG as a game tree 0 Agents act on information sets 0 Actions are taken in turns 0 POSGS are de ned over world states extensive form games over nodes in the game tree 1218 DecPOMDP complexity results Observability Communication fully jointly partial none none P NEXP NEXP NP general P NEXP NEXP NP free instantaneous P P PSPACE NP 1318 m INSTITUTO surmon rzcmco Dynamic programming for POSGs Dynamic programming for POSGs Hansen et al 2004 Uncertainty over state and the other agent s future conditional plans De ne value function Vt over state and other agent s deptht policy trees a 3 vector for each pair of policy trees Computing the t 1 value function requires backing up all combinations of all agents deptht policy trees Prune very weakly dominated strategies Optimal for cooperative settings DECPOMDP Still infeasible for all but the smallest problems 1418 m INSTITUTO surzuon rzcmco Approximate DECPOMDP solving Extra assumptions eg independent observations factored state representation local full observability DECMDP structure in the reward function Optimize one agent while keeping others xed and iterate Settle for locally optimal solutions Free communication turns problem into a big POMDP Find good online communication policy Add synchronization action Nair et a1 2004 Belief over belief tree Roth et a1 2005 1518 m IN STITUTO surzuon ricmco Some algorithms Joint Equilibrium based Search for Policies Nair et a1 2003 Use alternating maximization Converges to Nash equilibrium which is a local optimum Keeps belief over state and other agents observation histories This POMDP is transformed to an MDP over the belief states and solved using value iteration 1618 m Some algorithms 1 SetCoverage algorithm Becker et a1 2004 o For transitionindependent DecMDPs with a particular joint reward structure Bounded Policy Iteration for DecPOMDPs Bernstein et a1 2005 o Optimize a nitestate controller with a bounded size 0 Alternating maximization 1718 m INSTITUTO surnmn TECNICO References R Becker S leberstem V Lesser and C V Guldmzn Imellxgence Rexearch 22 4234355 ZUEI4 D s Bemstan R Gwen N Immermzn ands lebersLem Thecumplexltyufdecentxahzedcuntxul ufMarkuv densun prunesses Wthemancxaf Operanan Rexearrh 274 alww znuz D s Bemstan E A Hansen ands leba39stan In Pm In A 4 2779 znus F J c v Guldmzn and s was Lem Rexeamh 22 1437174 znm Susan D BemsLEm and s lebersLem Arn cml Imellxgence zn n4 R Nan M szbe M Yukuu D Pynada ands Marsella In Pm 1m Jam Conf on Arn cml Imellxgwce znnz R Nan M szbe M Ruth andM Yu Du Amanamm Agent ade Agent system znm D v Pynadann and M szbe Imellxgence Rexeamh 15 389423 znnz M Ruth R Slmmuns andM Velusu Decentralized cummumcanun strateges fur cuurdmated multiragentpuhnes an Schultz L parka and F Schnada dw r M 7 mm v L Sthley Stuchasn games memhg aftheNanamlAmdemyafSnencex 39 mssinnn 1953 m vquot CanEmmy on 1818 Hancock Mental Workload I MENTAL WORKLOAD Instructor Dr Peter Hancock Lecture Overview Just how hard are you working right now I am assuming that you re reading these lecture notes and looking to understand the information they contain The action of reading takes some effort and if I look at your eyes I could tell they were moving but other than that I would have some rather severe dif culties in measuring the demands currently imposed on you and your reaction to them But this is not so for physical work Here I can use all the methods of physics biochemistry biomechanics indeed even ergonomics and the like to ask simple and soluble questions about your current physical workload and your muscular response And herein lies the problem Traditional work measurement in Industrial Engineering and Ergonomics has been predominantly about physical effort where the muscle is the engine of action Now when we move on to the brain as the major source of work we deal with a very different form of measurement challenge What might be surprising to the uninitiated is that the brain takes about a third of the resting metabolic energy produced by the body and this can increase during especially intense mental work Thus although specific signals within the brain are faint and dif th to distinguish the mass action of the brain itself is extensive As we shall see recent brain imaging techniques each try to evaluate certain aspects of this mass action and use various indicators to achieve this aim However the four primary methods of mental workload assessment have held sway for the last two decades and continue to dominate even as these new techniques emerge Thus we shall here deal with these four techniques but with your awareness that brain measurement technologies are a volatile and changing enterprise in which new developments are constantly emerging Primary Task Performance If one wishes to know how well an individual is performing cognitive work the rst and most obvious method is to measure their outcome ef ciency 7 in short their Primary task performance In many practical situations these measures which emerge from the very origin of tiIne and motion studies are the sole representation that the assessor requires For example in piece work the rate ofproduction combined with the rate of item rejection is used to calculate the remuneration for the individual The faster and more accurately the individual works the higher the pay level and presuInably the higher the level of cognitive Hancock Mental Workload 2 demand However this remains an issue for employers For example what if an individual is highly productive in cognitive work but is working with much spare capacity Does this mean the employer could get more out of that individual What happens when the cognitive work is creative and not repetitive How many great ideas equals a number of units of rote work These are dif cult questions to answer and actually underlie the intrinsic contract that work creates between the worker and the employer No wonder the question mental of workload is not one of scientific definition and interest alone However the fundamental shortfall of priInary task measures is that they re ect work as is being presently accomplished Let s suppose we are trying to use priInary task measures to assess a critical process We know that if we iInpose too much cognitive load on the individual say 25 aircraft on an airitraffic controller they will not be able to accomplish the task and catastrophe may follow However this failure is nonlinear see Hancock amp Warm 1989 and so as we add aircraft we will not see the failure coming ifpriInary task performance is all that we have In essence priInary task re ections are good for the present moment but can be very dangerous ifwe want to use them to predict future cognitive performance limits In some cases the process is not one that will suffer excessively from this handicap in other processes it is the difference between life and death Thus while primary task performance is a useful and obvious measure it cannot be our whole picture unless we are operating in very benign conditions with few if any consequences for failure Secondary Task Performance If primary task measures can be used for the current performance level but are ofrestricted use in predicting behavior then perhaps there are other approaches that can help with the latter problem Indeed there are One of the rst of these measures is labeled the secondary task technique This technique emanates from research efforts on attention and relies on the following logic Let us suppose that a priInary task takes a certain proportion of one s attention but under 100 This leaves some residual attention which can be directed to another task Assuming the priInary task is defended against the change in demand the efficiency of the secondary task then represents a re ection of primary task demand That is if the primary task takes 80 it leaves 20 for the secondary task Now ifthe demand increases and the priInary task takes 90 the reduction in secondary task performance to 10 re ects this change in overall demand Note here that the primary task is expected to be performed successfully at all tiInes and operators must understand this priority via instructions or feedback Now the secondary task level is diagnostic without the potential for catastrophic failure in priInary task performance As you should understand this logic contains Hancock Mental Workload 3 many assumptions and listing these assumptions will help in discussion in class of the utility of secondary task measures Wickens 1984 Subjective Assessment So far we have discussed the two methods of primary and secondary task performance and some of their advantages and disadvantages However it is evident that if prediction is an important issue and if the task is being performed in important realiworld contexts neither of the previously discussed methods will suffice In respect of this we need to proceed to an additional form which is composed of subjective measures The rationale here is simple 7 if you want to know how hard someone is working simply ask them There are many legitimate concerns about these subjective responses How do you scale between individuals How do you know someone is telling the truth How do you turn perceptions into numbers Indeed these are all important issues but they are not insuperable barriers and as we among others have sought to show subjective measures have their place The major such methods The NASAiTLX and the SWAT are fundamentally similar in that they try to present descriptive adjectives relating to cognitive work and then provide a numerical representation on these scales In this sense they follow a great tradition in psychology of trying to render the mind transparent To the degree that any such endeavors are successful subjective measures of mental workload are successful and often they are simple and convenient to collect making them attractive for researcher and practitioner alike As with all the other methods we shall examine the advantages and disadvantages of this method further in discussion and see Meshkati Hancock Rahimi amp Dawes 1995 Physiological Re ections There are many situations in which primary task performance is sacrosanct In these performance crucial situations such as aviation vehicle control surgery combat etc any method which serves to interrupt the ongoing performance can itself induce catastrophic failures Further in these circumstances such as emergency response law enforcement and the like operators typically will not react to external questions and interruptions and data from techniques like the secondary task paradigm or subjective response simply cannot derive useful information As a consequence in these crucial realiworld situations in which we would really like the most reliable and diagnostic information our major methods actually fail to function effectively As a result we have to seek an alternative avenues through which to derived mental workload values Becoming more and more popular as techniques evolve in sophistication and reliability Hancock Mental Workload 4 physiological measures are currently in vogue As indicated by Hancock Meshkati and Robertson 1985 one can either measure re ections in the peripheral or the central nervous system The degree to which one gets accurate and reliable data often depends upon the proximity of the measurement both physically and systemically to the site of action That is measuring memory demands may be done via toeinail growth rate but this is a remote site and has poor resolution It is much better to examine brain activation in the memoryistimulated regions In class we shall discuss several such measurement techniques Iwould ask you to identify one and be prepared to talk about its relative advantages and disadvantages Application Areas Understanding mental workload and being able to provide a reliable and accurate measure of this form of workload on an individual basis may be a very satisfying scienti c achievement However the realization of such a goal goes well beyond the realm of academics There are an almost limitless vista of potential applications and here we consider two recent and highly pertinent examples with your recognition that there are many many others Earlier in our class we talked about human interaction with automated and semiiautomated systems One of the major advances in that realm was the idea of adaptive hurnanimachine systems This conception seeks to understand the state of the machine and the state of the human and then reconcile these respective states with the ongoing needs of the combined hurnanimachine system toward some mutual goal Obviously to accomplish this goal we need to know about the machine and need to be able to express this status in human terms However we also need to be able to capture the operator state and express their situation in machine terms largely quantitative assessment Accurate mental workload measures are thus absolutely vital here In a more recent conceptual advance Parasurarnan 2003 has suggested the possibility of generalizing this basic conception by tying together the principles of ergonomics with those of neuroscience This neuroniergonomics initiative takes the idea of direct connections between brain and machine to the next level Here the diagnostic capabilities of modern neuroscience are matched to the machine mediation of ergonomics to allow direct brain control of complex systems while permitting the recursive loop of direct brain input from the external environment This is a particularly exciting development and mental workload assessment as the measure of mass action holds much promise to help push such ideas forward In class we will also consider other practical uses ofreliable workload measures Hancock Mental Workload 5 Current Learning Objectives After the evident change in the currency of work from ergs to bytes the comparable change in work measurement has gone from assessment of muscular action to the comprehension of mental or cognitive load As is clear from our present lecture the brain poses many more problems for such assessment compared to the muscle and we have had to search for an adapt any number ofmethodologies to achieve this goal Following the present lecture you should now understand the four major methods of such assessment and be able to draw from a number of examples in each method You should understand the iInportance of mental workload assessment together with a number of barriers that prevent our complete comprehension at present Finally you should now be familiar with some recent brain imaging and assessment techniques which hold particular future promise and you should be cognizant of the advanced uses that such measures can be put to including adaptive automation systems and neuroer gonomics applications LECTURE READINGS Hancock PA amp Meshkati N Eds 1988 Human mental workload Amsterdam NorthHolland Hancock PA Meshkati N amp Robertson MM 1985 Physiological reflections of mental workload AViation Space andEnVironmentalAedicine 56 11101114 Hancock PA amp Chignell MH 1988 Mental workload dynamics in adaptive interface design IEEE Transactions on systems Mn and cybernetics 16 647658 Hancock PA amp Szalma IL 2003 The future of Neuroergonomics Theoretical Issues in Ergonomic Science 4 I 238249 Meshkati N Hancock PA amp Rahimi M 1989 Techniques of mental workload assessment In J Wilson Ed Evaluation ofliuman work practical ergonomics methodology pp 605627 London Taylor and Francis Meshkati N Hancock PA Rahimi M amp Dawes SM 1995 Techniques of mental workload assessment In J Wilson and EN Corlett Eds Evaluation oflzuman work A practical ergonomics me iodology pp 749782 Second Edition London Taylor and Francis Parasur am an R 2003 Neur oer gonom ics Resear ch and practice Theoretical Issues in Ergonomic Science 4 1 520 Wickens CD 1984 Processing resources in attention In R Parasuraman and DR Davies Eds Varieties of attention pp 63102 Orlando Academic Press


Buy Material

Are you sure you want to buy this material for

25 Karma

Buy Material

BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.


You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

Why people love StudySoup

Bentley McCaw University of Florida

"I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

Amaris Trozzo George Washington University

"I made $350 in just two days after posting my first study guide."

Jim McGreen Ohio University

"Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."


"Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!

Refund Policy


All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email


StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here:

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.