emma l clayes university of glasgow supervisor: prof. anne anderson co-supervisor: jim mullin

The effects of relevance of on-screen The effects of relevance of on-screen information on gaze behaviour and information on gaze behaviour and communication in 3-party groupscommunication in 3-party groups

Emma L Clayes

University of Glasgow

Supervisor: Prof. Anne Anderson

Co-Supervisor: Jim Mullin

BT Supervisor: Dr. David Hands

Sponsored by UK ESRC and BT

OutlineOutline

Background & HypothesesDesign and MethodResultsSummary Future workConclusion

Representations: form and locationRepresentations: form and location

Many studies concerning mediated communication (Finn et al, 1997) with different technologies, measures and tasks

Little research on the impact of different representations (e.g Sellen, 1995; Parise et al, 1996)

Representations and relevanceRepresentations and relevance

How does the form and location of representations impact on gaze and communication?

How does this interact with the relevance of on screen information ( e.g. task related information and representations in terms of the task role)

Evaluation techniques in CMCEvaluation techniques in CMC

Many different measures used to examine computer mediated communication (e.g. dialogue analysis, task performance, questionnaires)

Additional factors (e.g. task, no of participants, technology) and different methods have led to conflicting results

Methodological issuesMethodological issues

Three-party groupsProblem solving and social tasksCommunication analysis,

questionnaire responses and task performance

Eye-tracking as an evaluation technique

Eye-tracking in Psychology and HCIEye-tracking in Psychology and HCI

Well established measure of human information processing (Rayner, 1998)

Eye-gaze computer interfaces (Jacob, 1991)

Recent studies examining gaze and CMC-Velichkovsky et al (1997), Mullin et al (2001)

Communication and AttentionCommunication and Attention

Eye-tracking as an evaluation technique: What do users attend to when using

remote communication systems?

Does the relevance of on screen information impact on gaze behaviour?

Are patterns of gaze related to patterns of interaction within the group?

Eye-tracking in CMCEye-tracking in CMC

Exploratory questions:

Is it possible to obtain significant amounts of eye data during a non-restricted interaction?

Are participants consistent in their patterns of gaze across a screen?

Are these patterns meaningful?

Communication and RelevanceCommunication and Relevance

Relevance of on screen information (e.g. video data and video links)

Shared visual data more useful than video conference links (Daly-Jones et al 1998)

Communication and RelevanceCommunication and Relevance

Relevance of other collaborators (e.g. task role and status)

Status effects in mediated communication (Dubrovsky et al 1991, France et al, 2000)

Study 1Study 1

What on-screen features attract most gaze?

Does the position of representations on screen impact on mediated communication?

Does this interact with the type of task and relevance of information presented on screen?

Task 1-Problem-solving taskTask 1-Problem-solving task

Map task-collaborative problem-solving task

Two Instruction Givers (IG1 and IG2) have to instruct one other person (Instruction follower-IF) on how to draw a route on their map

Therefore, relationship between IG and IF more relevant to task success than IG1 and IG2

Task 1-Display ScreenTask 1-Display Screen

Task 1-Hypothesis ITask 1-Hypothesis I

Hypothesis I- Users will gaze more often at shared visual data ( i.e. the map) than video links of remote collaborators

Task 1-Hypothesis IITask 1-Hypothesis II

Do participants look equally often at the IF and IG video images?

Does the location of the video image affect gaze behaviour?

Task 1-Hypothesis IIITask 1-Hypothesis III

Do patterns of gaze reflect patterns of interaction within the group?

I.e. Do participants talk more often to the person they look most often at?

DesignDesign

2 factor mixed design

Relevance (map, video-IF, video-IG) within subjects

Location (video-IF top, video-IF bottom) between subjects

MethodMethod

Subjects- 10 groups of 3, 20 eye-tracked

2 maps, order of maps and location of IF video balanced

3 participants in different rooms-high quality audio and video links-

Eye tracked participant always an IG

ResultsResults

Screen divided into areas of interest for eye gaze analysis

Percentage analysis on different areas of interest and different fixations

Eye Data-Pictorial analysisEye Data-Pictorial analysis

Results-Hypothesis IResults-Hypothesis I

Participants spent significantly more time looking at the map (72.4%) than video image of IF (12%) and IG (10%)

Relevance of on-screen information impacts on gaze behaviour

F(2,36)=258.15, p<.001

Results-Hypothesis IIResults-Hypothesis II

Do participants look more often at IF compared to IG and is this affected by location?

Main effect of relevance (IF 12%, IG 10%) p<.05

No effect of location p>.05.

Results-Hypothesis IIResults-Hypothesis II

Significant interaction between relevance of video image and location of video image F(1,18)=5.73,p<.05

Participants looked more often at the instruction follower - only significant when the IF video was located in the top left of the screen (F=10.48, p<.05)

Results-Hypothesis IIIResults-Hypothesis III

Do patterns of gaze reflect patterns of interaction within the group?

Does the eye tracked participant direct more turns of speech to the IF or IG?

Turn combination analysis: IG1-IG2, IG1-IF No effects of relevance (task role-IG2/IF),

location of IF or an interaction between the two.

Results-Hypothesis IIIResults-Hypothesis III

Turn Combination resultsIG1-IF IG1-IG2

IF top left 20.7 22.6IF bottom left 19.5 23.2

20.1 22.9

Average no of turns by IG1-43

Task 1-SummaryTask 1-Summary

Users look more often at shared visual data (map 72.4%) than video images (22%)

Users gaze more often at IF than IG-only significant when IF video is located in top left of screen

Task 1-SummaryTask 1-Summary

Communication patterns reflect that IG directs almost equal number of turns to IG and IF- does not reflect patterns of gaze

Positional effect for salient video images?

Task 2-Relevance and StatusTask 2-Relevance and Status

Realistic task-mediated business meeting

Two confederates- one high status (boss) and one low status (marketing assistant)

Video data presented not essential to complete task

Task 2-Relevance and StatusTask 2-Relevance and Status

Hypothesis I- Do participants gaze more often at shared data compared to video images when the information is relevant, but not essential to complete the task.

Task 2-Relevance and StatusTask 2-Relevance and Status

Hypothesis II- Do participants look more often at a high-status collaborator?

Does the location of the high status video image impact on gaze behaviour?

Task 2-Relevance and StatusTask 2-Relevance and Status

Hypothesis III- do patterns of gaze reflect patterns of interaction within the group?

I.e do participants talk more often to the person they look most often at?

Task 2-DesignTask 2-Design

2 factor mixed design

Relevance (visual graphic, video-high status, video-low status) within subjects

Location (video-high status top, video-high status bottom) between subjects

Task 2-MethodTask 2-Method

2 confederates, 20 eye-tracked participants

3 participants in different rooms-high quality audio and video links

Task 2-Display ScreenTask 2-Display Screen

Results-Hypothesis IResults-Hypothesis I

Participants spent significantly less time looking at the low status video image (18%) than both the task feature (24.5%) and the high status video image (35%)

Relevance of on-screen information impacts on gaze behaviour (F(2, 36)=8.19, p<.05), no effect of location or an interaction.

Results-Hypothesis IIResults-Hypothesis II

Do participants gaze more often at the high status confederate compared to the low status confederate and does this interact with the location of video images?

Only sig main effect of status F(1,18)=22.19,p<.05, no effect of location or an interaction

Participants gazed more often at the high status confederate (35%) compared to the low status confederate (24.5%)

Results-Hypothesis IIIResults-Hypothesis III

Do patterns of gaze reflect patterns of interaction within the group?

Do participants direct more turns of speech to the high status confederate compared to the low status confederate?

Results-Hypothesis IIIResults-Hypothesis III

Turn combination analysis: sig effect of status, no effect of location or an interaction

Participant directs more turns of speech to high status confederate and this reflects patterns of gaze

Results-Hypothesis IIIResults-Hypothesis III

Turn Combination resultsParticipant-LS Participant-HS

HS top left 2.9 10.5HS bottom left 2.7 9.3

2.8 9.9

Average no of turns by Participant-13

Task 2-SummaryTask 2-Summary

Participants gaze less often at low status video compared to high status video and task feature

Participants gaze more often at high status video regardless of location

Interaction in Task 1 not replicated- video images attract more gaze and main effect of status found

Patterns of gaze reflect patterns of communication

Study 1-SummaryStudy 1-Summary

Participants gaze more often at shared data in task 1, not in task 2

Significant interaction between location and relevance of video images in task 1, not in task 2

Overwhelming impact of status on gaze and communication in task 2

Communication patterns reflect patterns of gaze in Task 2, not Task 1

Study 1-SummaryStudy 1-Summary

Eye tracking provides valuable information about mediated interaction

Distribution of attention related to experimental task manipulations

Positional effect for salient video images?

Study 1-Eye-tracking in CMCStudy 1-Eye-tracking in CMC

Exploratory questions about eye tracking answered

Data capture rate high –managed to track on average 70% of participants who took part

Average of 80% of fixations directed on screen during task

Patterns of gaze consistent and meaningful

Future WorkFuture Work

Same tasks, different design Video images placed in 4 corners of the screen,

therefore 4 conditions for each task

Task 1:Provisional results suggest interaction not replicated- always gaze more often at the map

Task 2:Difference in gaze distribution to videos smaller when video images are placed on same side of the screen than when they are placed on opposite sides of the screen

ConclusionConclusion

Eye tracking can be used successfully as an evaluation technique

Task differences and status effects robust- reflected in both patterns of gaze and communication

May be positional effect for salient video images?

ConclusionConclusion

Implications for the design of remote communication systems

Context in which system is to be applied very important e.g. social or problem solving

Further research required on positional effects-may be used to enhance or reduce amount of gaze directed to representations