1、Intelligent User Interfaces,Frank Shipman Department of Computer Science Texas A&M University E-mail: shipmancs.tamu.edu,What this is about,Designing, building, and evaluating intelligent user interfaces Particular technologies used to create intelligent user interfaces Issues concerning applicabili
2、ty of intelligent user interfaces,Intelligent user interfaces (IUI)?,Systems that provide interactive support based on embedded AI mechanisms Interfaces to AI functionality and knowledge representations Adaptive systems Cooperative problem-solving systems,Technologies I,User models, Situation models
3、, and Programming by Demonstration,Adaptive Interfaces,Requirements: interface that can be adapted user or situation model adaptation strategyFrequently used for providing assistance or training to user,User Model,“any information which a program has which is specific to a particular user. The infor
4、mation itself could range from a simple count of errors, to some complicated data structure which purports to represent relevant parts of the users knowledge of the problem domain.”,Stereotyping vs. Individual,Stereotyping (canonical user modeling) provide interfaces for classes of users classes mig
5、ht be based on skill (novices, midrange users, experts) or role Individual approach dynamically adapt to suit each user can be based on observed use of system or self assessment,Representations for User Models,Descriptive method modeling the users observed behavior describes what system has seen use
6、r do Skill-based or cognitive method attempt to model the internal cognitive models and processes of the user represents background knowledge, goals, plans, preferences, misconceptions,Acquisition of User Model,Model based on a combination of: Observations of system use statistical history chronolog
7、ical history Self-assessment Testing How can model evolve over time? Any of the above,Berkeley UNIX Consultant,Goal: Provide help to new UNIX usersGenerates user model based on “successful” use of UNIX commands Explanations of difficult commands can make use of students knowledge.,Intelligent Tutori
8、ng Systems,Task is generally well-known assignments given to student by system systems track partial completion Systems keep record of students success and failure. used to determine future assignments used to determine how to help when student has difficulties,Situation Models,Components of situati
9、on: users, system, environment Users multiple user models System hardware constraints and load device / resource availability,Representing the Environment,Identifying environmental influences anticipating use situations classes of use vs. detailed model of environment Monitoring environment direct i
10、nput devices user description,Mars Medical Assistant,Goal: Provide medical support for astronauts on three year trip to Mars Consider educational, consultation, and emergency situations Models of user and patient limited highly-trained user community no new users joining during mission,Other Adaptiv
11、e Systems,Typing completion suggests completions for partial terms based on prior use Emacs suggestions notifies user when more efficient method available to complete task Computer Chess Game determines quality of own play based on perceived level of opponent,Programming by Demonstration,Generalizin
12、g from demonstrated action and situation sequences to programs Difficulties: knowing what must stay the same knowing what are variables and their types connecting to programmed application code,Programming by Demonstration Systems,Peridot - demonstration of simple interface Marquise - demonstration
13、of graphical editors including palettes and modes DEMO - demonstrating dynamically created objects DEMO2 - refinement by system based on multiple demonstrations,Pavlov,Focus on programming animation Includes: graphical objects models of motion and time Stimulus-response demonstration modes for creat
14、ing objects and behaviors mode for demonstrating interaction,Technologies II,Presentation generation, Design Environments, and Interface agents,Presentation Generation,Generating dynamic links to information enabling user-controlled flow Generating presentations based on current situation and/or use
15、r Use of user or situation model Generating rhetorical structure/transition Scripting events Media-based decisions,Presentations and Explanations,Examples: Explainer (Redmiles) Explainable Expert System (Moore) Story Presentation System (Sgouros, ),Explainer,Domain: Graphical program explanation for
16、 software reuse Creates links between perspectives on software including source code, documentation, execution information, application domain view Provides user multiple points of access to better inform about software,Explainable Expert System (EES),Explains different outcomes in an expert system
17、/ planner Generates natural language to answer users questions Keeps dialog history to provide differential descriptions,EES Architecture for Explanation,Query analyzer,Text planner,Sentence generator,User goals,Dialog history,Focus Information,Knowledge Base,Plan Operators,User Question,Response,EE
18、S Example,User: “Describe Inderal” System: “Inderal a drug that ” User: “Describe Elavil” System: “Like Inderal, Elavil is used ” User: “Describe Cafergot” System: “Cafergot is very different from the drugs we have been talking about. ”,Story Presentation System,A dynamic dramatization method for na
19、rrative presentations Architecture:,Plot Analysis,Dramatization,Presentation Manager,Dramatic Effects Library,Original Story Material,Symbolic Plot Description,Story Presentation,Story Presentation Analysis,Plot analysis models: physical and emotional state changes positive and negative interference
20、 among characters Dramatization uses plot analysis to determine dramatic events in story Lifeline, Rising complication, Reversal of fortune, Dramatic irony, Happy end,Story Presentation Results,Presentation manager adds dramatic effects to original story material to emphasize dramatic events in stor
21、y Effects include audio: selection of noises or music images and video: flashbacks, flashforwards, images of other characters,Design Environments,Provide a software environment supporting the activities part of design. specification, construction argumentation, documentation, communication Examples:
22、 Framer (Lemke, Fischer) JANUS (Morch, McCall, Fischer , .),Framer,Knowledge-based support for interface design Approach: direct manipulation interface builder Framer 1 - construction kit approach Framer 2 - design environment,Design Environment Components (1),Checklist system provides decomposition
23、 of task, user identifies current focus Palette & Workspace system provides primitive components user identifies components used and organization of components in design,Design Environment Components (2),Specification sheets system brings design issues to users attention, presents potential answers,
24、 and explains significance and consequences of design choices user symbolically specifies answers to design issues,Design Environment Components (3),Critics system points out sub-optimal design decisions, explains why this is believed, and provides heuristics for making decisions users may accept or
25、 reject the systems critique,Design Environment Components (4),Catalog system provides examples user selects designs to reuse and modify Code generator system generates an executable representation of designed interface,Other Design Environments,JANUS - kitchen design designed for non-technical user
26、s XNetwork - computer network design identified need for simulation component VDDE - voice dialog design another type of interface design with interesting constraints,Software Agents,One view: Software processes that have non-trivial tasks delegated to them which require independent action and a rep
27、ort on the results.,Issues for Software Agents (1),Personification Should agents be represented as a living or animated character? Does it improve adoption of software? Does it create inflated expectations? Is it just too annoying?,Issues for Software Agents (2),Trust and Competence How does user de
28、velop an informed level of trust? Can agent give self-assessment on likely outcome of task? Delegation How can user delegate tasks? How can user check on status of delegated tasks?,Issues for Software Agents (3),Control How does user set limits on the agents activity? When does the agent get to inte
29、rrupt the user (mixed-initiative dialog)? Dealing with multiple agents How can the user manage many agents? How can interactions between agents be predicted?,Information Retrieval Agents,Watch user patterns to infer interests or goals which are used to classify, rank, or suggest new information Exam
30、ples: INFOSCOPE: patterns in Netnews use BASAR: patterns in Web access Issue: the “cold start” problem must watch a while before suggesting,Social Filtering,Finding elements liked by others (with similar preferences) requires some notion of preferences improves with more users Examples Tapestry - ra
31、ting of documents GroupLens - collaboration & user profiles A and CD-NOW,Technologies III,Knowledge manipulation and Using recognized structure,Interacting with Knowledge,User tasks Adding knowledge Editing rule bases and object hierarchies Examples HITS Knowledge Editor (Terveen) Modifier (Girgenso
32、hn) Hyper-Object Substrate (Shipman),Knowledge Representations,Informal text, graphics, audio, video Semi-formal hypertext, argumentation Formal frames, semantic nets, scripts, rules, inheritance hierarchies,HITS Knowledge Editor,Knowledge editor for CYC project Difficulties of knowledge representat
33、ion formalization - articulation in precise detail comprehension - complex vocabulary, size modification - location and consistency,Terveens Design Principle #1,Provide a workspace in which users and systems can jointly construct and manipulate a context for problem-solving, and in which the state o
34、f the problem-solving is represented visibly.,Terveens Design Principle #2,Deliver intelligent assistance through critics.,Terveens Design Principle #3,Exploit the interactive potential of computational media to manage the user-system interaction according to conventions that are appropriate to the
35、role of each party in the interaction.,Support Provided by HKE,Inferences - information inferred from workspace and existing KB Troubles - inconsistencies between workspace and KB Suggestions - relevant representational issues for users to consider,Modifier,Support for End-User Modifiability Users a
36、re not knowledge engineers Example: Adding new object class to existing system Support: suggestions based on similarity of features and efficiency of representation,Hyper-Object Substrate,Formalization as difficulty cognitive load tacit knowledge premature structure Supporting incremental formalizat
37、ion Flexible knowledge representation Suggestions based on informal content Automating knowledge acquisition?,Using Recognized Structure,When can it be used without user acknowledging correctness? Examples: TileBars (Hearst) graphically presents results of text analysis inferring topic changes withi
38、n text corpus Data Mining search for patterns within database,Non-Verbal Representations,Non-verbal communication allows expression of emergent thoughts Systems need to recognize implicit structure Examples: VIKI (Marshall, Shipman, ) PerSketch (Saund, Moran) Tivoli (Moran, van Melle, Chui, ),VIKI:
39、An Analysts Workspace,Task: collect, organize, and interpret information Characteristics: evolving understanding ephemeral structures emerging visual languages Goal: Recognize visual structures and languages,PerSketch: Adaptive Drawing Tool,Recognize multiple potential connections between strokes in
40、 drawing Enables cleaning up drawings, editing drawn objects Example:,TIVOLI: Meeting Room Support,Manipulating structures on an electronic whiteboard Recognize gestures, characters, etc. Recognize lists, groupings Map changes in visual structure to underlying database,Design Considerations,Issues o
41、f applicability, Evaluating interfaces, and Discussion,IUI: Appearing Too Smart?,A central danger of creating IUI is, like ELIZA, appearing too smart Creating unrealistic expectations Communication difficulties false alarm garden path Design guideline: do not hide systems knowledge or lack thereof,E
42、valuation Methods,Interviews Cognitive Walkthroughs Questionnaires Videotapes of use Usage data from software monitoring Comparative evaluations Pre- and post- use testing,Evaluating Interfaces,Deciding between observational and empirical methods Is there a specific question being asked? How mature
43、is the technology? Is having real tasks in a natural setting important?,Expectation Agents,Support for greater user participation in software design Software designers encode expectations of use which are matched against actual use. When expectations are not met, a dialog can be started between designers and users.,
