4/12/12  

1  

Introduction

HCI DTIC Masters

Josep Blat

Two motivating statements Consider what the computer of the future might look like. Suppose I told you it wouldn’t even be

visible, that you wouldn’t even know you were using one?

Don Norman (The Design of Everyday Things, 1988) p. 185

The Web makes the whole world accessible to

the user; but it is only a window in a computer Tim Berners-Lee (Weaving the Web)

•  Both statements indicate how far the computer

is from being really usable

4/12/12  

2  

HCI applicability

•  Research in HCI should provides knowledge and methods for the design of usable computer systems

•  How to ensure that this knowledge is applied successfully by those who design and build computer systems?

•  No consensus for the research strategy and methods by which this can be achieved

Viewing HCI as a … •  science: emphasis on theory and what is

happening, hypothesis-testing experiments •  design science: artifact theory,

interpretation and evaluation of these artifacts in the context of real use, observation-invention

•  engineering discipline: engineering/human factors. Aims at optimizing the man-machine fit –  ability to apply this research to the design of

novel products? –  accuracy of the observation? –  unscientific?

4/12/12  

3  

Methodologies

quan%ta%ve   qualita%ve   mixed  

cause  and  effect  thinking  

theory  or  pa8ern  building  

consequence-­‐oriented,  problem-­‐centered  

measured  observa@ons  with  experiments  and  surveys  

open-­‐ended  data  collec@on.  narra@ves,  ethnography,  case  studies  

both  qualita@ve  and  quan@ta@ve  informa@on  

Viewing HCI as a … •  Interdisciplinary

–  computer science –  psychology (e.g. human perception, human

memory, cognitive sciences) –  sociology (e.g. anthropology) –  artificial intelligence (e.g. machine learning) –  computer graphics (e.g. infovis, knowledge

vis) •  flirting with...

–  artists –  designers –  engineers –  hackers

4/12/12  

4  

Context: history

From punched cards … Brad A. Myers: A Brief History of Human Computer Interaction Technology, ACM Interactions, 5 (2), March, 1998, pp. 44-54

From human factors to user experience

Computers in 1945: past

Harvard Mark I: 5 tons

4/12/12  

5  

Present; but the future ...? n Computers weren't always like this… n How will they be in the future!

The early days … n At the beginning: perforated cards,

switches (with lights) n Then modified typewriters, line-terminals;

command lines is the typical interface; it is also called the 1D interface

n Then CRT use, with a full screen interface, it is already 2D interface, but mainly alphanumeric

4/12/12  

6  

Graphical User Interface n  Pioneer work by Sutherland (Sketchpad, 1963);

then Xerox PARC (Alan Kay, Dynabook – early 70’s – Star – late 70’s) and Apple’s Lisa-Macintosh (early 80’s); much later even Microsoft adopts it.

n  It is essentially the WIMP interaction (windows, icons, mouse, pointer), with a series of actions: drag and drop; point and click; move, select, copy, paste; … the desktop metaphor

n  It comes with a generation of new applications making use of new paradigms such as WYSIWYG (What You See Is What You Get): document editors, spreadsheets, CAD programs, …

Now (but the future?) n Gesture and voice recognition n Natural language interfaces

n Multimedia and 3D

n Virtual (or augmented) reality

n Cooperative environments

n And tools: UIMS (User Interface Management Systems) and toolkits, Interface builders, ...

4/12/12  

7  

HCI today

Context: history

From punched cards … From human factors to user experience

Bødker, S. When second wave HCI meets third wave challenges. Proc. 4th Nordic HCI. (2006), 1-8

4/12/12  

8  

First HCI wave: Human Factors

•  Focus on individuals and single applications •  Cognitive modelling and information

processing as paradigms •  Rationalism / positivism as philosophical

background

15  

First HCI wave Human Factors

•  Interface qualities: efficiency, learnability, memorability, … in summary, pragmatic ones

•  User centered design •  What to improve?

•  user’s tasks; and parts of tasks •  guidelines …

•  Evaluation •  Short time user’s interactions

16  

4/12/12  

9  

Second HCI wave Human Actors

•  Focus on groups working with a collection

of applications •  Situated action, distributed cognition and

activity theory as paradigms •  Phenomenology as philosophical

background •  Work settings; interaction within

communities of practice. •  Importance of context •  Participatory design workshops,

prototyping and contextual inquiries Acting with technology : activity theory and interaction design / Victor Kaptelinin and Bonnie A. Nardi, Cambridge, Mass. : MIT Press, 2006

17  

Third HCI wave name? UX?

•  Cultural level (esthetics) •  Expansion of the cognitive to the

emotional •  Pragmatic/cultural-historical focus on

experience •  User Experience (UX)

18  

4/12/12  

10  

More on the third wave

Roots in part of second wave

•  Understanding users and the use context (contextual inquiry)

•  From cognition to interpretation (activity theory as epistemology)

•  Participatory design Wright, Peter; McCarthy, John: Experience-Centered Design. Designers, Users and Communities in Dialogue, Morgan & Claypool, 2010

20  

4/12/12  

11  

Experience is

•  Subjective (vs. objective) •  Holistic (vs. instrumental) see next

slide •  Situated (vs. abstract) •  Dynamic (vs. static) •  Positive (worth) Hassenzahl, Marc: Experience Design. Technology for all the right reasons, Morgan & Claypool, 2010

21  

Beyond what and how

22  

4/12/12  

12  

Analyzing motivation: needs

23  

!""#$ %"&'()*+),-$./+,-,01$2$3-#"*"-#"-'"$

!""#$%&'#$("')*+',-"'./"'0,+1"'*2')*+-'*3%',0.$*%1'-,./"-'./,%'2""#$%&'./,.'"4."-%,#'2*-0"1'*2'5-"11+-"',-"'./"'0,+1"'*2')*+-',0.$*%'

4,0*"+"-'"$2$"55"'+6-'"$

!""#$%&'./,.')*+',-"'6"-)'0,5,7#"',%8'"22"0.$6"'$%')*+-',0.$*%'-,./"-'./,%'2""#$%&'$%0*95"."%.'*-'$%"22"0.$6"'

7"86+"#-"&&$2$9"8,-:)-:-"&&$

!""#$%&'./,.')*+'/,6"'-"&+#,-'$%.$9,."'0*%.,0.'3$./'5"*5#"'3/*'0,-"',7*+.')*+'-,./"-'./,%'2""#$%&'#*%"#)',%8'+%0,-"8'*2'

;"85<6'+/68)=)-:$2$0"6-)-:$

!""#$%&'./,.')*+',-"'8"6"#*5$%&')*+-'6"1.'5*."%.$,#1'-,./"-'./,%'2""#$%&'1.,&%,%.',%8'./,.'#$2"'8*"1'%*.'/,6"'9+0/'9",%$%&'

;"'/()+1$2$',-+(,8$ !""#$%&'1,2"',%8'$%'0*%.-*#'*2')*+-'#$2"'-,./"-'./,%'2""#$%&'+%0"-.,$%',%8'./-",."%"8'7)')*+-'0$-0+91.,%0"1'

>,-"1$2$8/?/(1$ !""#$%&'./,.')*+'/,6"'5#"%.)'*2'9*%")'.*'7+)'9*1.'*2'3/,.')*+'3,%.'-,./"-'./,%'2""#$%&'#$("','5**-'5"-1*%'3/*'/,1'%*'%$0"'5*11"11$*%1'

3-58/"-'"$2$*,*/86()+1$

!""#$%&'./,.')*+',-"'#$("8:'-"15"0."8:',%8'/,6"'$%2#+"%0"'*6"-'*./"-1'-,./"-'./,%'2""#$%&'#$("','5"-1*%'3/*1"',86$0"'*-'*5$%$*%'%*7*8)'$1'$%."-"1."8'$%'

@A1&)'68$+A()B)-:$2$9,#)81$

!""#$%&'./,.')*+-'7*8)'$1'/",#./)',%8'3"##'.,("'0,-"'*2'-,./"-'./,%'2""#$%&'*+.'*2'1/,5"',%8'+%/",#./)'

;"85<"&+""0$2$&"85<("&*"'+$

!""#$%&'./,.')*+',-"','3*-./)'5"-1*%'3/*'$1',1'&**8',1',%)*%"'"#1"'-,./"-'./,%'2""#$%&',';#**1"-<'

@8"6&/("$2$&+)0/86+),-$

!""#$%&'./,.')*+'&".'5#"%.)'*2'"%=*)9"%.',%8'5#",1+-"'-,./"-'./,%'2""#$%&'7*-"8',%8'+%8"-1.$9+#,."8'7)'#$2"'

'

Important unsolved issues

•  How to describe use, experience? related to

•  Which are the design implications?

24  

4/12/12  

13  

Flash-back to basics

The Psychology of Everyday Things (POET)

The Design of Everyday Things n How many everyday things?

Don Norman suggests twenty thousand n 20000 user manuals? No: things

should tell you what to do with them n Design, interaction should be

“obvious”: back to basics is key

4/12/12  

14  

Objects can (should?) have psychology

n  Well-designed objects have good affordances: visible clues to their operation n  like the holes of the scissors: they are there for

the fingers to be inserted n  false clues: teapot with handle & spout on the

same side n  Another visible structure are constraints:

n  the big hole of the scissors indicates that several fingers can be inserted

Objects can (should?) have psychology (II)

n Norman also talks about mappings: n Mapping is the mathematical term for

relationship between two sets n  In this case the relationship of controls

(in the interface) and results (in the world) is the mapping.

n Example: a control for an automobile seat could be of the form of the seat; by moving the corresponding shape we get the good result (reclining …).

4/12/12  

15  

Conceptual models: a starting point

n  The conceptual model is a mental representation of how objects work and how interface controls act

n  Q: Where do the things in the trash go when I empty the trash? n  People have preconceived models that you may not

be able to change n  We usually have three models

n  The user’s model: the mental model n  The designer’s model: the design model n  The system image which is what the interface

communicates (they should match!!)

Consequence: good design guides

n  1) Provide a good conceptual model n  Users imagine how things work in order to

understand, and avoid to remember blindly n  Interface should provide a basis for building the

model n  2) Make things visible

n  if object has function, interface should show it n  controls in a car are very visible; in old watches

too; not in new watches, not in telephones.

4/12/12  

16  

Consequence: good design guides (II)

n 3) Develop a good map of interface controls to user’s model

n 4) Provide feedback n The old telephone system (when

engaged, when you could dial, sounds when dialing, … what you do)

n People learn by trial and error, feedback consistent with model!

n WYSYWYG is good feedback

Two examples n  Lots of visible

controls n  One button for

each functionality n  Usually sound

feedback

How to transfer a call? Did you succeed?

Are the calls diverted?

4/12/12  

17  

And a mapping …

n  Loudspeaker balance in a car n  try to mirror the real world n  left-right; front-rear

Action and interaction n  Interaction is a complex process n  Norman distinguishes two main aspects

n  Execution (doing something)

n  Evaluation (comparing what happened with what was expected)

n  And seven stages for the action: n  Forming the goal: setting the goals

n  Forming the intention, Specifying the action, Executing the action: for execution

n  Perceving the state of the world, Interpreting the state of the world, Evaluating the outcome: for evaluation

4/12/12  

18  

And the corresponding questions for an interface

Seven design principles n  Use both knowledge in the world and knowledge in the

head n  Simplify the structure of the tasks n  Make things visible: bridge the gulfs of execution and

evaluation n  Get the mappings right

n  Exploit the powers of constraints both natural and artificial

n  Design for error n  When all else fails, standardize

4/12/12  

19  

And 10 usability slogans n  Your best idea is not good enough n  The user is always right n  The user is not always right n  The users are not designers n  The designers are not users n  The bosses are not users n  Less is more n  Details are important n  Help is not n  Usability engineering is a process (Nielsen, Usability Engineering)

User cognitive models: The Human Information Processor

4/12/12  

20  

The Model •  Two components

–  The set memories and processors –  The set of operational principles

•  Three subsystems (interacting) –  The perceptive system, with sensors, buffer memories

and transforms inputs into symbolic codes which are stpred

–  The cognitive system, which places the symbols in the working memory, and uses this and the long-term memory to take decisions

–  The motor system, which executes the decisions

The Model

4/12/12  

21  

The model parameters •  Three processors (perceptive, cognitive, motor)

working sequentially or in parallel, described by the cycle, t.

•  Memories parametrised by –  Storage capacity (number of items, m)

–  Item decaying time (d)

–  Main code type (visual, physical, …) k

The operational principles (1/2)

•  P0 Cycle recognise - act of the cognitive processor: actions triggered by associations to long-time memory; actions modify working memory

•  P1 Variable time cycle of the perceptual processor: inversely to stimulus intensity

•  P2 Coding specificity: what is stored and how. •  P3 Discrimination: difficulty of retrieval

proportional to number of candidates •  P4 Variable time cycle of the cognitive processor:

larger when overloaded; diminishing with practice •  P5 Fitts law

4/12/12  

22  

The operational principles (2/2)

•  P6 Power law of practice: Tn to perform a task in the n-th attempt follows a power law (na, a = 0.4).

•  P7 Uncertainty: decision time grows with uncertainty (number of alternatives)

•  P8 Rationality: Goals + Task + Inputs + Knowledge + Restrictions = Behaviour.

•  P9 Problem space: in terms of knowledge states, operators to change state, restrictions, control knowledge

Card, Stuart K, Thomas P Moran, and Allen Newell. 1983. The Psychology of Human-Computer Interaction. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers.

Ubiquitous computing – Intelligent agents

4/12/12  

23  

Ubiquitous Computing (1/2) •  Weiser, M: Some Computer Science issues in Ubiquitous Computing, in

Communications of the ACM, 36(7), 75-84, 1993 n  Ubiquitous computing is the method of enhancing computer use by making

many computers available throughout the physical environment, but making them effectively invisible to the user

n  ... a possible next generation computing environment in which each person is continually interacting with hundreds of nearby wirelessly interconnected computers

n  ... anthropological studies of work life teach us that people primarily work in a world of shared situations and unexamined technological skills. However the computer today is isolated and isolating from the overall situation, and fails to get out of the way of the work ... rather than being a tool through which we work, and so which disappears from our awareness, the computer too often remains the focus of attention.

Ubiquitous Computing (2/2) n  The problem is not one of “interface” ... it is a property of the whole

context of usage of the machine and the affordances of its physical properties: the keyboard, the weight and desktop position of screens, and so on.

n  ... the approach I took was the definition and construction of new computing artifacts for use in everyday life ... three sizes of computers, boards, pads, and tabs, ... for each person in an office, there should be hundreds of tabs, tens of pads, and one or two boards. n  boards are wall-sized interactive surfaces

n  pads are analogous to scrap paper to be grabbed and used easily n  tabs are analogous to tiny individual notes or PostIts

4/12/12  

24  

Ubiquitous Computing / interface agents (1/2)

n  The interface agent metaphor: a personalized computer as something like a well-trained, long-standing English butler - someone intimately aware of your idiosyncrasies, your habits, your friends, your goals, and who you deal with. n  You talk to it

n  It watches us and learns our needs

n  It has "knowledge", is "aware", or has a personality

n  It is an assistant

n  Limitations n  it stops at the notion of assistant instead of promoting intuitive or anticipatory

computer, that needs no commands aim for an extension of our body, or integration of mind/body/world

n  It keeps the computer in the foreground personal computer is the wrong idea, intimate computer even worse invisible computer is best

n  perfect, all-powerful, slave (ancient prejudice)

Ubiquitous Computing / interface agents (2/2)

Interface agents •  1) single locus of

information about me •  2) command the

computer •  3) personal, intimate,

computer •  4) filtering •  5) DWIM (do what I

mean) •  6) user interface •  7) I interact with agent

Ubiquitous computing •  1) distributed, partial

information by place, time and situation

•  2) what computer? •  3) personal, intimate people •  4) breathing, living, strolling •  5) WIWYHIAFI (when I want

your help I'll ask for it) •  6) no boundary between you

and machine •  7) I interact with the world

4/12/12  

25  

Tangible User Interfaces

Tangible User Interfaces (Tangible Bits) (1/3)

•  H Ishii, B Ullmer: Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms, in CHI 97, 234-241, 1997

n  the future of HCI … : the locus of computation is now shifting from the desktop into: n  onto our skins/bodies (recent activities in the new field of “wearable

computers”)

n  into the physical environments we inhabit, through integration of computational augmentations into the physical environment ... Our intention is to take advantage of natural physical affordances to achieve a heightened legibility and seamlessness of interaction between people and information.

4/12/12  

26  

Tangible User Interfaces (Tangible Bits) (2/3)

n  Key concepts to make bits accessible through the physical environment: n  Interactive Surfaces: Transformation of each surface within architectural space (e.

g., walls, desktops, ceilings, doors, windows) into an active interface between the physical and virtual worlds;

n  Coupling of Bits and Atoms: Seamless coupling of everyday graspable objects (e.g., cards, books, models) with the digital information that pertains to them

n  Ambient Media: Use of ambient media such as sound, light, airflow, and water movement for background interfaces with cyberspace at the periphery of human perception.

ways of both improving the quality and broadening the bandwidth of interaction between people and digital information by allowing users to “grasp & manipulate” foreground bits by coupling bits with physical objects, and enabling users to be aware of background bits at the periphery using ambient media in an augmented space

Tangible User Interfaces (Tangible Bits) (3/3)

n  computers ubiquitous? … rather awakening richly-afforded physical objects, instruments, surfaces, and spaces to computational mediation

n  Augmented Reality: visual overlay of digital information onto real-world imagery … Tangible Bits strong focus on graspable physical objects as input

n  Design prototypes n  metaDESK: a nearly horizontal back-projected graphical surface; an “activeLENS, ”

an arm-mounted LCD screen which is a physically instantiated window which allows haptic interaction with 3D digital information bound to physical objects; a “passiveLENS,” a passive optically transparent “lens” actively mediated by the desk; “phicons,” physical icons; and instruments which are used on the surface of the desk.

n  transBOARD : is a networked digitally-enhanced physical whiteboard which absorbs information from the physical world, transforming this data into bits and distributing it into cyberspace

4/12/12  

27  

Wearable, Situated, Context aware (computing)

Augmented Reality (AR)

Wearable computing

Walter Van de Velde, Starlab COMRIS (parrott), wearable networked autonomous personal assistant competing for attention ...

4/12/12  

28  

What is a wearable? n Bradley Rhodes (MIT)

n  portable while operational n  needing minimal manual input n  aware of the environment n  always on and able to attract attention

n Steve Mann (Toronto) n  part of the user n  controlled by the user n  negligible operational delays

Context awareness n  The ability of a device or program to sense,

react or adapt to its environment of use n  Uses:

n  Improving interaction with a device •  better defaults, automated choices,...

n  Improving quality of existing services •  communication, information,...

n  Extended sensing and monitoring •  security, health and safety,…

n  Enabling of new services •  tourist services •  health services ...

Examples: Active and smart badges, Media Cups, Aware GSM, Intelligent Jogging Suit

4/12/12  

29  

Wearable with GPS/GIS (Augmented Reality)

n  Location based context has immediate applications such as the tourist guide systems

n  Example prototype system from Columbia University

Prototype tour guide Here’s what the user sees

Context: more than location

(taken from Walter van de Velde)

4/12/12  

30  

Situated computing

Context Awareness Meta Information

Software Agents

Situated Computing

Shallow situated computing: computation in context Deep situated computing: computation of context

Information: pull or push? n Web: passive store of information

waiting to be explored (pull) n Web: massive collection of active

agents, trying to get to you (push) n Pull has become impossible without

computers supporting humans n Push is problematic, but might be

the future