Special Topics CS 8802
Popular in Course
Popular in ComputerScienence
This 0 page Class Notes was uploaded by Alayna Veum on Monday November 2, 2015. The Class Notes belongs to CS 8802 at Georgia Institute of Technology - Main Campus taught by Staff in Fall. Since its upload, it has received 14 views. For similar materials see /class/234157/cs-8802-georgia-institute-of-technology-main-campus in ComputerScienence at Georgia Institute of Technology - Main Campus.
Reviews for Special Topics
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 11/02/15
BEN SHNEIDERMAN en Shneiderman is a longtime proponent of direct manipulation for user interfaces Direct manipulation affords the user control and predictability in their interfaces Pattie Maes believes direct manipulation will have to give way to some form of delegation namely software agents Should users give up complete control of their interaction with interfaces Will users want to risk depending on quotagentsquot that learn their likes and dislikes and act on a user39s behalf Ben and Pattie debated these issues and more at both U 97 Intelligent User Interfaces conference January 6 9 1997 and again at CHI 97 in Atlanta March 22 27 1997 Read on and decide for yourself where the future of interfaces should be headed and why interactions novcmbcr dcccmbcr1997 PATTIE MAES Excerpts from debates at U 97 and CHI 97 I think we would do best to focus on the remarkable human capabilities in the visual domain which I think are largely underutilized by the current designs with 40 icons in 23 windows I think we should have two or three orders of magnitude more 4000 or more items on the screen in an orderly way that enables people to see all of the possibilities and navigate among them Jim Okay welcome all to the afternoon ses sion My name is Jim Alty I think I am sup posed to be the moderator whatever that means for this session I hope things don t get too rough The debate topic this afternoon as you all know is direct manipulation versus intelligent agents On my right I ve got Ben Shneidermanifresh from his triumphs this morning when he s been gnawing away at the agent peopleifrom the University of Mary land On my left ofcourse fresh for a fight is Pattie Maes from the MIT Media Laboratory Let me just explain It s 15 minutes from each speaker to place their position and then 5 minutes allowed each for rebuttal Then we open it up to the floor for about 30 minutes or so Could you please use the microphones If you want to line up behind the micro phones I will select you to make your com ments Then at the end there will be two 5minute summing ups So let us commence the debate Ben First my thanks to the organizers of this Intelligent User Interfaces workshop for dar ing to arrange this debate It was Angel Puer ta s careful attention to my concerns that made me feel comfortable in coming to speak here and so I want to offer him a souvenir from our labithe usual Tshirt And to Jim Alty a cup of tea for when he needs to relax from our group at the lab I am pleased to represent the University of Maryland s HumanComputer Interaction Lab which is now 14 years young Over the years we ve explored a host of userinterface design issues in an interdisciplinary way involving computer science psychology and the library school Our goal is to create envi ronments where users comprehend the dis play where they feel in control where the system is predictable and where they are will ing to take responsibility for their actions To me responsibility will be the central issues in this debate My pitch over 20 years has been to make a science out of user interface research I want to get past the notion of userfriendly a vague and misleading term and to be really clear about specifying who the users are and what their tasks are Then we can make a theory that allows us to predict the time it takes for a specific user to learn a specific task the speed of performance on benchmark tasks the rate and distribution of errors and the retention of those tasks over time We look at subjective satisfaction as a secondary measure and have developed a standardized Questionnaire of User Interaction Satisfaction that the university has licensed to more than 100 orga nizations around the world QUIS consists of 71 items from lowlevel details to higher level concepts in the interface Of ce of Technolo gy Liaison 13014054210 We think accommodating individual dif ferences is importantinot just mentioning experts and novices but understanding quan titatively what performance differences we anticipate Do we expect experts to perform twice as fast or twenty times as fast as novices Would men perform differently from women in the use of interfaces or prefer different interfaces And then we try to deal with broader cultural issues that are even more dif ficult to measure For me the future is most clearly moving in the direction of information visualiza tion I think we would do best to focus on the remarkable human capabilities in the visu al domain which I think are largely underuti lized by the current designs with 40 icons in 23 windows I think we should have two or three orders of magnitude more 4000 or more items on the screen in an orderly way that enables people to see all of the possibili ties and navigate among them I will show you three brief videotaped examples They all show applications and extensions of the strategy of direct manipula tion a term I coined in 1982 to describe the existing successful systems These systems all had rapid incremental and reversible actions selection by pointing and immediate feedback 100millisecond updates for all actions I believe that this strategy reduces errors and encourages exploration The current manifes tations of direct manipulation are the visual ways of searching in databases and on the Web accompanied by visual presentation of results Let s take a look at an example that goes back 4 years in the first videotape called the FilmFiwlex interactionsnovember december1997 Popularity Is 3 39 39 3 32 on quot quot quotCE 8 39 a39 395 J 39 13 1 I nd 2 iw 7 r 39 quot a395 r 5 5 39239 39 aha E r quot quot Zi u 5ir 6 pi1 39P 2 quot 39 quot 5s 39 quot E39Er 5 T 3 2quot 1 339 2 39 quotz 3939 4 n 1 J 39 quot 39 5 925quot 39 39 quot 71 Equotquot 3 39 39 whoquot 915quotquot 3 I 39 39 In 39quot 5 La 2 39 39 gar59 39 f f ii quot quot 39 39 I quot3 31 quot 39 xquot 4 2 o 1920 1925 1930 1935 1940 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 I M Year of Product Action Title ALL Ml l Alit lsf 114anth 1 ll Actor ALL M l M tgtr5umur Far 1390 Actress ALL an J A l C CI 11311 ML 54 i l1 5 TWS Director ALL 5 C b 73H 1quot H FFLE THE 0 Length 450 1 J 0 450 Ratings i G i PG i PG 13 i R Films Shown 1455 Ell Crx39yI rl C I T 933 quotl l 1995 ion quot 395 Horror arranged on the X aXis and on the y axis from low to high populari ty We can use the length slider to trim the set of lms by the number of minutes in the lm so we do not have to see lms that are too FilmFinderVideo see Figures 1a c CHI94 Videotape or HCIL 1995 Video tape reports This display shows thousands of lms Popularity 39 3 Indiana Jones 8 Ithe Last Crusade Gold ngcr 1 Name of the Rose The Thlmdcrball rlurder o 39 Mammogram Highlander F Fled Tent The Untouchables The 103919931 D 39s39 The rear Train Robbery The u and G 39 I r me Russ39a mm Lam IElan Who Would Be King The 39 Robin 3 Marion Zard oz Cub Offence The 5 39 SBword ofthe Valiant 1122 Meteor 4 1960 1965 19339 195 1980 1985 1990 to show only drama from 1920 to 1993 only the more popul l l lzl Your of Production Title ALL n r l l39 39 l 391 quotc 397 Actor Connery Scan an l J m 3 3r r s In Actress ALL l a c rrrr l tn 3 Director ALL an E AB 3 24149 s M 60 Length 289 el39 1 l 450 Rulings l G l PG 139 PG13 I R Films Shown 24 IE 1JI 239 Pk 391 I All 1995 long and then we can use the category button or only action lms We can zoom in on more recent pictures and take ar ones And when there are fewer than 25 the titles will appear auto matically When we select one of them we get a description of the lm and information about the actress and actor We can also hunt interactionsnovember december 1997 debate Figure 1a FilmFinder shows 1500 films in a starfield display where the location of each point is determined by the year of the film x axis and its popularity in video store rentals yaxis The color encodes the film type Ahlberg amp Shneider man 1994 pzhuww cs umd edup 70 jettybciZScrem dumpsFiZm m alldotsgif Figure 1b FilmFinder after zooming in on recent popular films When fewer than 25 films remain the titles appear automatically pzhuww cs umd edup 70 jettybciZScrem dumpsFilmtitesng Users have great control over the display and as they select items the details appear in windows on the sides Popularity 319 3 Indiana Jones sign Last Crusade Goldfingcr 1 Name ofthe Rose The Thunderball 4 durder or 1 a A 39 n Jm39 39rm w Highlander PutterI rrrr1erz Sean Red Tent Tlrc Jail I Z Urrteucl1ubles The quot quotquot 39quot quot395 7 Long DW Thc Great Truin Robbery The mums ALL Er 39 I 39Hul39 l Y 3939 Murder on the Orient Expre Director ALL 3 Director Lunret Sidney Year 194 a Country USA Language English I r M rm M Actors Actresses so Lengm 259 Balsam Martin Bacall Lauren J 5 Cnesel JennPierre Bergman Ingrid 0 450 Perkins An iony Bisset Jacquelin Rulings 39 G PG Connery Scan HlllEf Wendy S PG13 I R Gicluud John Films Shown 24 4 Fm 1960 1965 19m 19 1960 1985 1990 1995 L l bJ Year of Production LamaJunta 11 watery m won a Figure 1c FilmFinder after selecting a single film The info card pops up with detailsondemand PMwwu rc umr I I 7 th interface IIJL i1 Screen Vbquot 171 flu scan ng for lms organized by actors In this case you might be interested in Sean Connery and his lms will appear on the screen Ben Okay I think you get an idea that the controls are visually apparent as you drag them The updates occur in 100 milliseconds and users get a clear understanding of what the situation is This work goes back to 1993 and the 1994 CHI conference has a couple of papers describing it 1 2 A general purpose version of that software was used for the Department ofJuvenile Justice project which you will hear about shortly Here is a FilmFinder done in the UNIX version of the product called SpotFire Figure 2 Chris Ahlberg has made a commercial product out of this and you can download the demo version off of the Web http wwwiveecom interactions It would be hard to see how to program any kind of agent tool to anticipate all of the possibilities that your eye would pick up We show the age of the youthful offenders on the X axis There are 4700 of them from 10 to 19 years old The number of days until a deci sion was made on their treatment plan is shown on the y aXis The rules of this organi zation say that decisions must be made with in 25 days but you can see a lot more than they anticipated are well above the 25 day limit Interesting things pop up whenever you try a visualization I hope you will spot these yel low lines those were a surprise We thought there was a bug in the program but it turns out that if you start clicking on them to get the details on demand you ll find out they all occur in a Hartford County They were all brought in on a certain day They all have the november december 1997 same charge which is narcotics possession These were drug busts and they were all put on treatment plans at the same time Those kind of patterns happen to anyone who tries Visualization programs and it would be hard to see how you could program an agent to anticipate all of the possibilities that your eye can pick up in 1 10th of a second Another problem we have dealt with in the second Videotape has to do with Visualizing per sonal histories It is called szeLz39nes Figure 3 The placement line shows this youth is currently placed at WaXter a detention center We also see a placement in a drug abuse pro gram and a placement at Cheltenham This one is thicker because the court found him guilty of auto theft When I click on the line of the drug abuse program all of the related information is highlighted I can tell that the placement was for a breaking and entering debate 01ij W EEEE PM uutwomcm quot10313611 Inga quotL 39 TIL O AD c DKYGXJLH mic v 39 Comm 0 0 0 11012 O t 0 war H I rpm L v 26 JED a 39Y 39 r L Aciu H 5 C DID anus Biti S 39339 AL che M c cuequot Him r F Hquot il Diana AB C own man H an 7v 3 3 Peanut 39 I 5 I fun Avudl Dash 1 l 11 1771 02 Manor Figure 2 Spotfire version of FilmFinder provides increased user controls Users can set axes set to length in minues and year and glyph attributes color is set to subject and larger size indicates award winning film Spot fire is available in Unix Windows and Java versions Jtzgowwwiveecom r o l I I O v a File Edit View Summary Options Help nw u Stanton rr r V imtrn F J I a39vs39Cl we nuns73 quot General A5 I an My 1 um IiiH 39zmJ Leave I an mu mm was um wu mu w a nu 4quot awn was mat mu 3quot i I I I l l I l I I I I a Cases Attempted Huron HOUR039 Auto The 1 email DElOAII u T39he MSAP lCVSJlOLOi Brit Intermix 030301 E HM Mio39mnl 5 g LDIJIJ lms Chitool M Walter R lAsslgmenul 8510501 0 039 Brown 39 5 Jcmes SAO S HIIH ammo r I James Brow391 QJOLO39L qu39mrs Brcwu 5 Reviews 0 O 0 AH rave mm 20 LMMMWWMM 0 ltltltlt ltlt lt ltltltgtgtgt3ngtgtgtgtgtgtgtgt 0 39 I I I l I l i l 1 t I I I H MN MW 93996 VP H39N 2351 1 97 23997 11939 44939 3quot37 1 339 TI39Q39 12 939 9quot y l Figure 3 Youth Record prototype using the Lifelines display to show a case history for the Maryland Department of Juvenile Justice CHI96 videotape or HClL 1996 Video Tape Reports 4 pwwwcsumd eduprojectsbcilResearcbI 997104tientrecordbtml interactionsnovember december 1997 Ben During the CHI97 debate Pattie made a point about my use of autofoms cam eras suggesting that they were some sort of agent As we were speaking Alan Wexelblat was taking pic tures of the events using my camera When I went through the color slides I found that most of the pictures he took were out of foms As with most autofocus cameras there is a narrow area in the center that is used for fomsing and this must be placed on the intended subjects of the photo Unfortunately Alan didn39t know this and simply pointed at the stage area but the focus point was on the badltground poster Almost all the photos were unusable We can all find this amus ing and leave it at that but I think there is a seri ous point which is that agents don39t always do what we expect them to do and it takes some knowledge to make effective use of agents or autofocus cameras If we were to assess responsi bility I would take part of it in failing to give ade quate instruction to Alan he might take part because he was the direct user and maybe the manufacturer has another part for a poor design which fails to provide appropriate feedbad case and was requested by Brown A click on Brown s name gives the contact information Here are the reviews which were written A click on the code brings the text of the review In the same way I can get more details about each case and placement by clicking on the labels Those screens are all for the old system showing that Li diner can be used as a front end and acts as a large menu for all of the screens The top buttons can access general information but any critical information will appear on the overview screen For example here s a mention of suicide risk Seeing the overview gives the user a better sense of how much information is available and what type of information it is Of course this implies that all of the information can be presented in one screen Ben We think that Li diner can also be applied for medical records and we are now applying it in a project with IBM We give an overview of patient histories that contains the consultations conditions documents hospi talizations and medications Users have great control over the display and as they select items the details appear in windows on the sides We think this strategy has great power in providing convenient access to large and complex databases in a way that gives the users an overview of what is happening and an appreciation of where the details t into the overall context The visual presentation gives users enormous bandwidth and there are potentially thousands of selectable items on the screen at once offering rapid access to the item that you are seeking The third and nal example is a visual database of the human body called the Vixible Human Our role was to develop a browsing user interface to the 15 gigabytes of images at the National Library of Medicine 3 CHI96 videotape and HCIL 1996 Video Reports free software available for Sun workstations from httpwwwnlmnihgov and select the Vixible Human links till you nd our Vixible Human Explorer This direct manipulation interface presents a thumbnail image of a coronal cross section of the body reconstructed directly from the axial cryosections Figure 4 This coronal Interactionsnovember image acts as an overview giving a visual rep resentation of the entire body The axial cryosections are a local view showing a thumbnail corresponding to the slider posi tion on the overview We can explore the body by simply dragging the slider It updates in real time giving an experience of ying through the human body Here we see the brain the shoulders the torso the abdomen the thighs the knees and all of the way down to the toes We press the retrieve button and the corresponding fullsize image is retrieved over the network from the NLM archive in a couple of minutes We provide several useful alternative ovewiews and also the ability to generate any coronal section overview for example near the back ofthe body or near the front Ben Other labs are working on related ideas of information visualization From the Paci c Northwest National Labs this textual data base that has been presented in a twodimen sional mo untainsandclusters visualization to give users an idea of the volume and interrela tionship of items Steve Eick at ATampT Labs has these wonderful visual overviews of large textual documents Here the characters in a children s book are color coded so that you can see the progress of the story as it moves on to different characters Departmental email networks and richer information such as 3D network representa tions are part of the things he likes to show with a variety of user controls to lter the traf c and reveal patterns of usage that might be dif cult to see with other data presentation strategies The closing slide says that the overview is the most important It gives users a sense of context of what to look atithe big picture Then they zoom in on what they want lter out what they don t want and nally go for detailsondemand My claim is that this gives users the feeling of being in control and there fore they can be responsible for the decisions they make Thank you Jim Okay thanks very much Ben That was perfect timing We now hand it over to Pattie december1997 Lllil gluiwl 1ucu Int gun 39n39ngr Salaam kelr39itw 39I ring 1 41 Intrl hrg mortal y I39rpl39l 39 mgr Raw 3 lwl 1 390 Irma 339 M lm Fu39l S39r39 lmg39s I tazgrza39 3 L935 u lawCum Jar In rst 4 Le Lhi yer i 94 of 1rr5 lrrl 1quot I nllrg39 Fun Bates 22quot Eil Figure 4 Visible Human Explorer user interface showing a reconstructed coronal section overview on the left and an axial preview image of the upper abdominal region on the upper right Drag ging the sliders animates the crosssections through the body North et al 1996 p39www cs umd edu v rojectsbcilResearcbI995vi5ible buman him Pattie I m not going to bribe the moderator with tea or T shirts or anything I hope that the work will speak for itself The word agent is used in a lot of different ways I want to start this presentation by explaining what I mean by the word agent and in a particular software agent Basically software agents are a new approach to user software a new way of think ing about software that end users have to use In particular the way in which agents differ from the software that we use today is that a software agent is personalized A software agent knows the individual user s habits preferences and interests Second a software agent is proactive It can take initia tive because it knows what your interests are It can for example tell you about something that you may want to know about based on the fact that you have particular interests Current software again is not at all proactive It does n t take any initiative All of the initiative has to come from the user A third difference with current software is that software agents are more long lived They keep running and they can run autonomously while the user goes about and does other things Finally software agents are adaptive in that they track the user s interests as they change over time So you can ask well why do you call it an agent Why call it an agent given that the term interactionsnovember december1997 debate anipulation l t 9 21 Ff Why do we need software agents Take a look at the World Wide Web for exam ple You couldn39t possibly try to visualize the World Wide Web in any way because it is completely unstructured and because it has been built by so many different people and is continu ously changing I believe that the dominant metaphor that we have today is a mismatch for the computer environment we are dealing with tomorrow ct anlpulation is already so overloaded and given the fact that it s really software that is slightly different from existing software Well we call it an agent to emphasize the fact that agent soft ware can act on your behalf while you are doing other things We also want to empha size that it does this based on its knowledge of your preferences just like a travel agent will act on your behalfby buying you a travel tick et based on the information that the travel agent has about your preferences Note that I prefer not to use the term intelligent agents nor autonomous agents because those terms have even more problems associated with them Now why do we need software agents Why does our software need to become more personalized Why does our software need to take initiative to help us as a user This needs to happen because our current computer envi ronment is getting more and more complex and the users are becoming more and more naive Finally the number of tasks to take care of and the number ofissues to keep track of are continuously increasing Let me tell you more about this First ofall the nature ofour computer environment is radically different today from 20 years ago back when the cur rent style of computer interaction was invent ed Twenty years ago one typically had one user using one computer and everything in that computer every le every object was in a particular place because the user put it there There was a limit to the amount of informa tion on that computer It was completely stat ic Nothing changed unless the user made it change It was completely structured and well organized Today our computer environments are completely different More and more the World Wide Web and our browser is becoming the one and only interface It s not quite the case yet today but it will be a year from now In that situation our computer is no longer this closed environ ment that we have complete control over Instead our computer or the screen is a win dow onto this vast network this vast network of information and other people That net work is continuously changing It is dynamic Something may be in one place today and the next day it may be in another place or may be lnteractlonsnovember gone Continuously new information is being created new versions of software are being added Also that environment is completely unstructured Take a look at the World Wide Web for example You couldn t possibly try to visualize the World Vide Web in any way because it is completely unstructured and because it has been built by so many different people and is continuously changing I believe that the dominant metaphor that we have today is a mismatch for the computer envi ronment we are dealing with tomorrow Second the user 20 years ago was different from the typical user today Twenty years ago we mostly dealt with professional users of computers Today and tomorrow the con sumer electronics market is going to be the one that dominates and those users do not even know how to program their VCRs How are they going to deal with user interfaces Third the number of things that people have to keep track of and the number oftasks that they use their computers for is huge and is increasing all of the time As we know from other domains whenever workload or infor mation load gets too high there is a point where a person has to delegate There is no other solution than to delegate For example many of you may have students that you del egate certain tasks to or you may have per sonal assistants that you delegate certain tasks to not because you can t deal with those tasks yourself but because you are overloaded with work and information I think the same will happen with our computer environments that they become just so complex and we use them for so many different things that we need to be able to delegate We need to be able to delegate to what could be thought of as extra eyes or extra ears that are on the lookout for things that you may be interested in We also need extra hands or extra brains so to speak because there will be tasks that we just cannot deal with because of our limited attention span or limited time and we need other entities to be able to represent us and act on our behalf Some examples to make this more concrete I didn t bring any videos because of the limited amount oftime but most ofyou have seen at least one of these agents These are some of december1997 1UELL I u 1 LI I r31 SUEZ H n ng or You should check out these artists An asterisk indicates your camem retire not rated yet Recommended E 39not rated yet Suggested not rated yet Suggested mg 39not rated yet 1 I Sugarcuhes l I Huger Waters l I Maul Kiit39upl39ler l I load The Wet Sprocket Suggested law quotnot rated yet Suggested ma 39 Submit and Send More I I Jilll llll jl Ten mny what yau re looking tor the ones we built in our lab Letizia built by Henry Lieberman who is here at the confer ence is an agent which continuously watches you as you browse the Web analyzing and memorizing all of the pages that you visit It extracts from those pages the common key words Whenever you are using your Web brows er Letizia always looks ahead and checks whether within a certain depth of the current page there happen to be any pages that you may be interested in So for example if I am interested in scuba diving my agent may have picked it up because I look at a lot of pages about scuba diving If I go to a particular entertainment site it may look ahead and say hey did you realize that if you follow that link that there are some pages about scuba diving in the Florida area The Remem mnce Agentis another agent that continuously tracks the behavior of the user It helps you remember certain things It helps you remember who sent e mail or whether you already replied to a certain e mail message It may proactively remind you of information related to the information you are currently looking at It works in EMACS When I am for example looking at an e mail message from a particular person it proactively reminds me of the previ ous e mail messages from that same person which is very useful because I may have for gotten to reply to one of them Fire y some of you may have tried that agent is basically a personal filterer for enter tainment not unlike the movie application that Ben talked about except that this agent will again keep track of your interests your preferences and proactively tell you about new movies that you may be interested in which you even forgot to ask about in the first place Yem iz is another agent that we built which tracks what the user s interests are by looking at your e mail and files and extracting key words It talks to other Ymm agents belonging interactionsnovember december1997 debate wwwfireflycom We need to be able to delegate to what could he thought of as quotextra eyes or extra earsquot that are on the lookout for things that you may be interested in gangrs rile as quotn1 as a user collects a large database of private knowledge his Rs becomes an expert on that knowledge base through constant retraining R goal 0 the RH is to aIIOw coworkers to convenientlg access the public portions 0 this database without interrupting the user Thus 1 a colleague wants to know about augmented realitg he simplg sends a message to the user s Remembrance agent For example thad raEmediamitedu The Rn can then return its best guess at an appropriate Pile Thus the user is never bothered by the querg never has to Format his knowledge ie some mark up language and the colleague Peels Free to use the resource as opposed to knocking on an oPFice door Knowledge transPer mag occur in a similar Tashion when an engineer trains his replacement he can also transPer his RH database 0 knowledge on the subject so that his replacement mag continually get the beneFit of his experience even aFter he has leFt Finallg i a large collective of people use Remembrance agents queries can be sent to communities not Just individuals This allows questions 0 the orm How do I reboot a Sun workstation7 rw macs wearables a flevt Eemenhranne Filll Hnt 31 Boston local wearable Computing talk take 2 quot 25 mobile Linux web page 51 rebootine workstations in the a FEmEm diSJlJLh ents area wouldn t have thought of yourself K4564 is another set of agents that buy and sell on behalf of users We are currently setting up this experiment MIT wide meaning for 15000 people We have already done tests with 200 people It s basically a marketplace where you can create an agent who will buy or sell a second hand book or a second hand music CD for you You just tell the agent I want to sell The foxJud 77496 by U2 1 want to ask 12 for it at rst You are allowed to go as low as 9 You have two months to sell this CI Ybu dun d beiedbrtough and onhr change the price all the way at the end near when the 2 months are over That agent will represent you in that marketplace negotiating to other users and if it notices that another user shares some of your interests especially if those interests are very rare then it introduces you to that other user It may say hey did you realize that at this IUI conference there is another person who is interested in going scuba diving in Florida so that maybe then we can decide to go scuba diving together Again it s suggesting something that you interactionsnovember december on your behalf with other people or other agents who may be interested in buying that CD from you Again it is sort of acting on your behalf You don t have to waste any time trying to make 10 bucks but the agent will do this for you I think it s important to address some com mon misconceptions about agents First of all sorry to say so but agents are not an alterna 1997 tive for direct manipulation A lot of confer ences and magazines pitch agents against direct manipulation They are actually com plementary metaphors Whenever you have an agent interface typically you also need a very good application interface because an agent is not a substitute for an interface An agent basically interacts with the application just like you interact with the application It s as if you had someone else looking over your shoulder as you are using the application noticing some of your preferences and habits and then offering to automate some of the tasks for you So you still need a very good direct manipulation interface visualiza tion all of these wonderful tools so that the user can personally interact with the applica tion An agent can never predict all of the movies that I may possibly be interested in It may be able to make some interesting sugges tions to me but I will still need to look up particular movies myself A second misconception is that some peo ple think that agents are necessarily personi fled or anthropomorphized In fact most agents are not Most of them don t even deal with a natural language interaction interface A third misconception is that agents necessar ily rely on traditional AI artificial intelligence techniques like knowledge representation and inferencing In fact most of the agents that are commercially available and have proven success ful with large numbers of users rely on either user programming or on machine learning rather than traditional AI techniques I want to conclude by addressing some crit icisms of software agents that Ben has come up with as well as people like Jaron Lanier see A Conversation with Jaron Lanier interactions ii5 July 1995 pp 46 65 who is also very vocal about all of this Opponents of agents typically argue that well designed visualization interfaces are better Like I said before you still need a well designed interface when incorporating agents in an application However some tasks I may just not want to do myself even if the interface was perfect If my car had a perfect interface for xing the engine I still wouldn t fix it I just don t want to bother with xing cars I want someone else to do it debate Pattie Maes is an Asso ciate Professor at MIT39s Media Laboratory where she founded and directs the Software 7 Agents Group She cur rently holds the Sony Corporation Career Development Chair Previously she was a visiting Professor and a Research Scientist at the MIT Artificial Intelligence Labora tory She holds a Bachelor39s degree and PhD degree in Computer Science from the Vrije Universiteit Brussel in Bel gium Her areas of expertise are Human Computer Interac tion Electronic Publishing and Electronic Commerce Artificial Intelligence and Artificial Life Pattie is one of the pioneers of a new research area called Software Agents that is semiintelligent computer programs which assist a user with the overload of infor mation and the complexity of the online world She is one of the organizers for the leading conferences in this area such as the annual Autonomous Agents conference and the annual Practical Applications of Agents and Multi Agent Systems conference She is a founder and board member of the Agent Society an international industry and professional organization established to assist in the widespread development and emergence of intelligent agent technologies and markets Pattie is a consultant in the area of Software Agents for several major companies such as Apple Computer Hughes Research etc She is the editor of three books and is an editorial board member and reviewer for numerous profes sional journals and conferences such as the User Modeling journal the Personal Computing journal and the Artificial Life journal Her work has achieved several prizes includ ing the IBM best Bachelor39s thesis award 1984 the OOP SLA1987 best paper award the Ars Electronica 1995 award the Interactive Media Festival Award 1995 the ArcTec 1995 award and so on Finally she is a founder of Firefly Network Inc in Boston Massachusetts one of the first companies to com mercialize software agent technology interactionsnovember december1997 a Opponents of agents typically argue that well designed visual ization interfaces are better Like I said before you still need a well designed inter face when incorporating agents in an application However some tasks I may just not want to do myself even if the interface was perfect If my car had a perfect interface for fixing the engine I still wouldn39t fix it I just don39t want to bother with fixing cars I want someone else to do it ct anlpulation interface agents A second criticism is that agents make the user dumb That s actually more one of Jaron s objections rather than Ben s I think To some extent it s true If I don t x my car then I m not going to learn about xing cars However this does not constitute a problem As long as there s always an agent available or I can call one by a motor association like AAA then that s fine It s too bad that I will never learn about cars but I want to learn about other things instead A third criticism expressed is that using agents implies giving up all control That s incor rect I think you do give up some control when you deal with an agent I tell the car mechanic to x my car or to x this or that part of the car I don t know how exactly he or she is going to do that I don t mind giving up some control actu ally and giving up control over the details as long as the job is done in a moreorless satisfactory way and it saves me a lot of time Okay just very brie y I want to say that I think where the true challenge lies is in designing the right useragent interface In particular we need to take care of these two issues understanding and control Under standing means that the agentuser collabora tion can only be successful if the user can understand and trust the agent and control means that users must be able to turn over control of tasks to agents but users must never feel out of control I believe that this is a won derful interfacedesign challenge and we have come up with a lot of solutions to actually make sure that the agent s user interface has these two properties that the user feels in con trol or has control when he or she wants it as well as that the user understands what the agent does and what its limitations are Let me save that for later maybe Thanks Jim Thanks very much Okay Ben s going to go up to 5 minutes to say whatever he likes Ben How interesting We are debating but part of me is drawn to the idea of celebrating Pattie Maes and encouraging you to follow her example I want to draw the audience s attention to her transformation during the months we ve had these discussions As I go back to Pattie Maes s work and I read her ear lier papers and her Web sites she promotes autonomous agents and presents an anthropo morphic vision Even in the current proceed ings her article is titled Intelligent Software so I was delighted with her opening remarks that rejected intelligent and anthropomorphic designs The old Pattie Maes wrote agents will appear as living entities on the screen conveying their current state of behavior with animated facial expression or body language rather than windows text graphics and g ures So we ve got two Pattie Maes I will choose the newer one that demonstrates movement in my direction including her last slide which might have been written by me User understanding is central and user con trol is vital for people to be successful In fact I have other ways of celebrating Pattie Maes I encourage you to look at her Fire y Web site which is an interesting appli cation Collaborative ltering I think will become an important approach for many domains But as a user I can t find the agents on the Fire y Web site In fact as I searched to find the agents all I came up with was that the company had previously been called Agents Inc and is now called Fire y If you read the Fire y Web site you will not find the word agents in the description of this sys tem In fact the interface is a quite lovely direct manipulation environment allowing users to make choices by clicking in a very direct manipulation way So I think we ve made progress in clarify ing the issues in the past year of our ongoing discussions For example I think we can sepa rate out the issue of natural language interac tion which as far as I can see has not been a success The systems that were offered com mercially even a few years ago like QampA from Symantec or Intellect from AI Corporation to do database query and Lotus HAL for spread sheets are gone and direct manipulation is the sunriving technology A second issue is anthropomorphic inter faces such as chatty bank tellers and the Postal Buddy or the proposed Knowledge Navigator of Apple s 1988 video Microsoft s playful attempt at a social interface in BOB is also a failed product As far as I can see the anthro pomorphic or social interface is not to be the future of computing interactionsnovember december1997 debate Transactions Kasbah 0 saw t39m bm h 139 l1quot 39lllfl tnunnr NJ Manage turnquot Nyntl RHNUI up W I homelmessagesx qkl new dog Hi pattic now you can create your Now Selling Agent I ll an tin 5111 M Mam rn39lhg 1ch that w lull 3 ma a gnu rst nun a quotimam munImer r prm n mam ymr 33quot v m 4quot mch rand 5 w mu can Renard In kirk quotCm Agul bm 3 lb Mk mi up an all you w lude A third issue of adaptive interfaces is quite interesting I would concede half a point and say that we now see two levels the user inter face level which users want to be predictable and controllable as Pattie has stated and the Matultosn Cl level below the table where there may be Deadine March 10th 1997 some interesting algorithms such as collabora Start price tive filtering If those can be adaptive there may be benefits in the way that Pattie Min price 70000 describes This is related to other adaptations Strategy tou h baroainer such as when I save a file to disk I see it saved 39 g and it is retrievable by me Under the table Location local there s a great deal of adaptation dealing with space allocation disk fragmentation and Kasbah example selling agent Level of Autonomy check before transaction compression strategies but from the user s point of view there s no adaptation It s quite Reporting Method event driven predictable The same goes for engines in interactionsnovember december 1997 I am concerned about the confusion of human and machine capabil39t make the basic ies I assertion that people are not machines and machines are not people I do not think that humantohuman interaction is a good model for the design of user interfaces ct anlpulation interface Is automobiles I turn the key I step on the gas it goes Underneath the hood below the user s concern there are adaptive algorithms that will set the engine speed based on many fac tors such as the temperature gas mixture etc That level of adaptivity is important as long as it does Lot interfere with the user s prediction of behavior I am concerned about the anthropomor phic representation it misleads the designers it deceives the users it increases anxiety about computer usage interferes with predictability reduces user control and undermines users responsibilityiwhich I think is central 1 think destroy the users sense of accomplishment l anthropomorphic representations think users want to have the feeling they did the jobinot some magical agent Finally I am concerned about the confu sion of human and machine capabilities I make the basic assertion that people are not machines and machines are not people I do not think that humantohuman interaction is a good model for the design of user interfaces Jim Okay Pattie would you like to respond to that Pattie First of all I should clarify that autonomous agents or the word agents has a much broader meaning than the words soft ware agent and my group at the Media Lab does research on autonomous agents more generally as well as software agents So when Ben was quoting from our Web site he s actu ally quoting other work that we do for exarn ple work on synthetic characters that can interact with people in a virtual environment which doesn t have anything to do with the software agent s work In fact it has less and less to do with it than it may have at one point So ifyou go to autonomous agent con ferences for example the Agents Conference in Marina del Rey in February First Interna tional Conference on Autonomous Agents see interartiom i 6 Conference Preview you ll see work being discussed that relates to robots autonomous robots You ll see work about synthetic believable characters and you ll see work about software agents So that s one thing I wanted to respond to It s impor 1nreracnonsnovember tant to distinguish these different types of agents and not lump them all together Second I absolutely agree with Ben that so far the most successful interfaces are the ones where the agents are pretty much invisible They are not visualized as a little anthropo morphic character For example in Firefly that is the case That doesn t mean that there isn t an agent operating For example in Fire y the Firefly agent will proactively tell you about other users that you may want to talk to It will warn you when there is something that has changed somewhere that you may be interested in There is still an agent there monitoring all of your preferences and proac tively making recommendations to you but that doesn t mean that there has to be this lit tle cute character on the screen Now I think one of the reasons that Ben and I disagree is actually that we are focusing on completely different problem domains In pretty much all of the problem domains that Ben looks at we are dealing with a user who is a professional user and we are dealing with a task domain that is very well structured and an information domain that is very well orga nized so that it lends itself to visualizing all of the different dimensions The kind of prob lems that we have typically been dealing with are very different because we are dealing with end users who are not necessarily very trained They may use the Web for a couple of hours per week but that is about it We are dealing with a very different information domain an information domain that may be very ill struc tured and very dynamic For example the World Wide Web is actually sort of one of the key domains that we do all of our research on Finally to illustrate that these approaches aren t necessarily incompatible I could envi sion a version of Ben s movie finder which uses Ben s nice visualization interface where an agent is continuously monitoring what movies you seem to be interested in That agent may for example highlight particular areas in the interface which it thinks you will be specifi cally interested in That kind of interface would actually combine an agent that learns about your preferences and proactively makes suggestions to you with a nice visualization interface The reason why you want that kind december1997 of proactive software is that the user does not necessarily always want to have all of that con trol when searching for a movie I believe that users sometimes want to be couch potatoes and wait for an agent to suggest a movie to them to look at rather than using 4000 slid ers or however many it is to come up with a movie that they may want to see Question Okay I have a remark to Ben which then transitions into a question for hoth speakers Ben I was a little hit irritated think Pattie was too with your lovely presentation on information visualization which seems to he entirely heside the point As Pattie said we will take the hest direct I quot and 39 quotI39 in as we can possihly get It seems to me the contrihution to this discussion would he some negative examples of where agentlike things are had At some level I take the thrust of your position Ben as heing reactionary to put it in simpler words sort of fear driven I woukl like to test my theory hy ask ing the following question which is we have now a new medium on the interface of playing which is speech Speech is now practical BM makes speech systems that are heing used and so on My question actually to hoth of you is to see what your reactions are to this new technology I predict that you are going to have speech with a computer rst of all research at Stanford recently has shown that once you have a comput er talking you cannot prevent people from anthropomorphizing the computer I do not see how you are going to have a coherent speech interface without using human communication principles So predict that you will say just don t do it I also want to hear Patties comments ahout speech technology Ben Do we have another half an hour here I thought I said very positive things about Fire fly and its agent and adaptation and I cer tainly like to see automaticity built into interfaces that amplify the user s capabilities I have trouble with the words like agents and expert and smart and intelligent because they mislead the designer and designers wind up leaving out important things In fact I love Pattie s slide up here If the agent oriented community would adopt debate Ben Shneiderman is a Professor in the Department of Computer Science Head of the Human Computer Interac tion Laboratory and Member of the Institute for Systems Research all at the University of Maryland at College Park He has taught previously at the State University of New York and at Indiana University He regularly teaches popu lar short courses and organizes an annual satellite television presentation on User Interface Strategies seen by thousands of professionals since 1987 The third edition of his book Designing the User Inter facetion 1987 has recently been published by Addi sonWesley Longman Ben is the author of Software Psychology Human Factors in Computer and Information Systems 1980 AddisonWesley Longman Reading MA and his 1989 book coauthored with Greg Kearsley Hypertext HandsOn contains a hypertext version on two disks He is the originator of the Hyperties hypermedia sys tem now produced by Cognetics Corp Princeton Junction NJ In addition he has coauthored two text books edited three technical books published more than 190 technical papers and book chapters His 1993 edited book Sparks of Innovation in HumanComputer Interaction collects 25 papers from the past 10 years of research at the University of Maryland Ben has been on the Editorial Advisory Boards of nine journals including the ACM Transactions on Cam puter Human Interaction and the ACM interactions He edits the Ablex Publishing Co book series on quotHumanComputer Interactionquot He has consulted and lectured for many organizations including Apple ATampT Citicorp GE Honeywell IBM Intel Library of Congress NASA NCR and university research groups interactionsnovember december1997 User interfaces should be predictable so that users trust them User interfaces should be thor oughly tested and users should be u these principles it would go a long way in making me sympathetic For example Pattie writes Make the user model available to the user I don t see that being done in most of the work about agents Explanations should be available and methods of operation should be understandable to the user So much ofthe work in agentry goes against the principles I like the new Pattie I am ready to be partners and collaborate with the new Pattie Now to focus on speech We have heard for 25 years the great hope and dream that speech is going to solve our user interface problems Dreamers prophecy that the Star Trek scenario is going to take over and we will talk to our computers I do not believe that speech will be a generally usable tool It has important niches opportunities for disabled users for certain handsbusy eyesbusy and mobilityrequired applications In preparing the third edition of my book I worked hard to find speech applications that do recognition effectively I am quite happy with speech f 1 r 1 I trained for all emergencies In emergency situations people cannot solve problems They can only do what39s rehearsed and predictable J J by telephone but the recognition paradigm is not being widely accepted even for minor tasks such as voice dialing Speech output except by tele phone is also a problem because speech is very slow and disruptive of cognitive process ing I think what annoys me the most about the devotees of speech is their failure to take in the scientific evidence that speaking com mands is cognitively more demanding than pointing Speech uses your shortterm memo ry and working memory By contrast hand eye coordination can be conducted in parallel with problem solving by another part of your brain and therefore does not degrade your per formance as much as speaking Question You ean do both at the tame time Ben Yes you can do handeye tasks in paral lel with problem solving more easily that you can speak while problem solving This fact is not a barrier to use ofspeech but it is a hur dle that designers of speech systems must rec ognize if they are to find ways to overcome it Pattie I must admit I actually agree with a lot of what Ben says I haven t used speech at all in my research the main reason being com pletely personalithat these systems often don t understand my accent but apart from that I do agree that it is not a very high band width kind of connection There is also a lot of ambiguity So I personally would like to see speech being used just for situations where the hands are not available like in your car or as an addi tional channel actually for example giving an agent some additional advice while you are also pointing at something For example Henry Lieberman sitting here in the audi ence did some interesting work where he taught an agent a particular procedure and while he was performing actions with the mouse he would give speech inputs to tell the agent what it had to pay attention to For example pay attention to this corner here that I am dragging or to this side of the rectangle So in that situation it is very useful because your hands are already doing something else and you need that additional channel to con vey some more information in parallel Question Thir quertion it for both of you Both ofyou reem to be concerned ahoutproteeting the merit eontrol of the environment but the one thing rtudier have rhown time and time again it that merx are very good at making mixtaker So how do yourporitionr relate to timeeritieal deei nonrapport enoironmentr rueh a medieal y temr or eoehpit ryxtemr Pattie I have actually been focusing on a completely different kind of application a type of application that is not as critical For example if yo ur World Wide Web agent gives you a wrong Web page to look atiit assumes that you are interested in a Web page and you are notithat is not at all critical It is not a big deal I have been focusing on that kind of situation and those kinds of problems the ones where if there is an error it is not very costly I ve been doing that because I believe that it will be very hard to make agents that always come up with the right answer always do the right thing I believe that there is a very large set of these kind of applications where things don t have to be completely precise or interactionsnovember december1997 100 percent correct in order for the agent to be very very useful to a user Ben I like your question I think it is an extremely important research area There is a long history of work often called supervisory control Tom Sheridan is a key player in the area for nuclearreactor control rooms cock pits and so on I think the design of these sys tems is most effective when the users have a clear predictive model of what their actions will produce If they do not know or are uncertain about what the results of their actions are they will disengage the automatic system as is the evidence with cockpit systems or nuclearcontrol rooms So one danger is that in complex controlroom environments when an emergency occurs users are uncer tain about its behavior In these situations they are likely to disengage a potentially help ful system and do what makes sense to them Therefore as Jim Foley said keep it simple very simple User interfaces should be predictable so that users trust them User interfaces should be thoroughly tested and users should be thor oughly trained for all emergencies In emer gency situations people cannot solve problems They can only do what s rehearsed and pre dictable It is a good topic that I would love to see more attention to it by this community Question In the interet ofbreoity I wa going to bring up eoeral human limitation or eon traint on human that make direot manipula tion a little more intereting and wanted to ab you both zr eomment not thatIhaoe all ofthe anwer butIthinb Iwouldjut limit it to your baio bind oflaw that the more thing you have to eroll through or the more thing you have to eareh through the longer it take you to eareh The idea ofhow do you deal with thi particu larly for many people erolling i not a particu larly uable thing I have particularly een thi with olderpeople and ofooure ifyou want a good example jut go to a fat food retaurant and wateh a new peron at the eah regiter try to nd how to ring up your hamburger It i pretty ter rible You will be therefor day That kind of idea and particularly we are lnEeraCElons talking about the wonder tl oiual proeeing and how important oiualization i57thi eem to be auming that everybody ha perfeet oiion What ifI am blind ifI am over ixty oe and I have a very mall ue tl eld ofoiew and I don t notiee thing o mueh on theperiphery or Iam in my oae aperon itting right here who i haoingaproblem with my contact lene how could you Ben you were ayingiwell peeeh i okay for diabled uerihow are you going to render that diagram into peeeh for a blind per on and what i the role of that Ben That is a legitimate concern Direct manipulation does bene t from and depends heavily on visual representations For those who are vision challenged or blind alterna tives to visual displays are important What surprises me is there are great supporters of direct manipulation in the visuallychallenged community because direct manipulation depends on spatial relationships Blind users often are strong at spatial processing If you can provide movement left down backward forward they can navigate fairly rich spaces in efficient ways I would say also you have been a little too quick about criticizing menu selection The question is what would the alternative be and how might those menus be better designed I do believe that fast and vast menus are a great bene t in many applications Jim Okay There are two more questions if you could be very brief please Question Thi i direoted more toward Ben than toward Pattie Will thi debate between direet manipulation and ageney alway exit in inter zoe deign or will it eventually be replaeed by ome bind of nion ofthe two approaehe In other word are we going to ee new Ben and new Pattie every day or i there going to be ome kind of quotShneiderMaef Ben I think it has been interesting to see how the debates evolve I certainly will point you to the new Pattie whom I am ready to cele brate and be partners with as I said but I think the debate will move on I think it has november december1997 debate I have been focusing on that kind of situation and those kinds of problems the ones where if there is an error it is not very costly I39ve been doing that because I believe that it will be very hard to make agents that always come up with the right answer always do the right thing I believe that there is a very large set of these kind of applications where things don39t have to be completely precise or 100 percent correct in order for the agent to be very very useful to a user I am here to promote direct manipulation with compre hensible pre dictable and controllable actions Direct manipulation designs promote rapid learning It supports rapid performance and low error rates while supporting exploratory usage in posi ive ways ct anipulation interface agents matured in interesting ways from where we were a year ago I think we are all at the edge of looking at new interfaces and so as we push that envelope back we are getting better understanding of the territory of the strengths and weakness of direct manipulation of the strengths and weaknesses of agents and where they are appropriate I am pleased by the progress in the discussion Pattie I think we both have changed Would you agree to that or not Jim I think before they start kissing let s move on Last question Question Thir rtartr out at leart a a elari ea tion quertion for Dr Shneiderman but it may go plaeer om there 7 what extent doe direet manipulation in your de nition and in your view admit autonomour ryrtem behavior Beeaure it reemx to me that a roon a you admit anything unexpeeted uneontrolled potentially anthropomorphizahle out of the iyrtem that you are interaeting with you have opened the door you have taken a rtep down the rlippery rlope toward agentnerr Ben Yeah I am in favor of increased automa tion that amplifies the productivity of users and gives them increased capabilities in carry ing out their tasks while preserving their sense of control and their responsibility responsibil ity responsibility I am sort of not answering your question because I don t want to work in the language you are dealing with We should be thinking about productivity improvement tools for users whether they are graphical macros dynamic queries star eld displays or other things Question Then what it it ahoutagentx that you dirlihe Ben Can I go to my closing slide I want to reassert the importance of scientific evalua tions We must get past the argumentation about my system being more friendly than yours or more natural or intuitive and talk lnteractlonsnovember about user performance We can deal with sat isfaction also but please focus on user perfor mance and realistic tasks Please please please do your studiesiwhether they are controlled scientific experiments usability studies or simply obsewations and get past the wishful thinking and be a scientist and report on real users doing real tasks with these systems That is my number one takeaway message I am here to promote direct manipulation with comprehensible predictable and con trollable actions Direct manipulation designs promote rapid learning It supports rapid per formance and low error rates while supporting exploratory usage in positive ways Direct manipulation is a youthful concept which is still emerging in wonderful ways Our current work leans to information visual ization with dynamic queries but there are people doing fascinating things with enriched control panels style sheets and enduser pro gramming Graphical macros would be my favorite project to advance the design of gen eral computing tools It is embarrassing that after 15 years of graphic user interface being widely available we have no graphical macros tools What is going on This is the greatest opportunity for visual programming Thirdiand I am answering your question hereiI think the intelligent agent notion limits the imagination of the designer and it avoids dealing with interface issues That s my view of the agent literatureithere is insuffi cient attention to the interface Maybe the way agents will mature is as Pattie is suggest ing that the agents take care of the processes below the table and there is a nice direct manipulation interface that the user sees A leading AI researcher commented to me that the 30 years of planning work in AI is essentially down the tubes because of lack of attention to the user interface The designers deliver a system and the first thing that the users say is This is great but what we really want to do is change these parameters The designers say Well you know we didn t put that in the interface They just haven t thought adequately about the interface nor done testing early enough I believe that this language of intelligent autonomous agents undermines human december1997
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'