1

MonicaMonica TavantiTavanti

Experiments & contexts

Ethnography and field studies• Lab is not like the real world• Context• Artificial tasks

Experimental approach• Realistic conditions can be created• Variables can have applied meaning• Personal biases /statistics/ general results

Old debate

Experiments

“Experimental approach”

Performing experiments Testing hypotheses In laboratory Controlling variables

Example from Air Traffic Control (ATC)

Radar displayATC en-route

Every colored “bit” = an aircraft:

Goal: keep them separated

3D stereo device Explore applicability for ATC

Literature review

Interview with former controllers

Devise an operational scenario

Select a task

…hypothesis…..

Test the hypothesis

Example ATC I

2D versus 3D stereo

Example ATC II

2

Example ATC III Design several scenes of traffic Cross-check with controllers if realistic• Flight levels, etc.

Then, design the experiment:• Within subject design• One independent variable -interface type with 2

levels 2D vs. 3D-• Measure accuracy and time• Task:

• Determine if any aircraft at the same flight level• If so, how many and what level

3D not better but faster than 2D

Anything wrong???Example ATC IV

What is the right unit of analysis???

Single userStand alone applicationSingle-oversimplified task

Users in context

Expand the unit of analysis There is more than single user, doing

single task There is “context”

The unit of analysis Augmented Reality for Control TowerNew ATC project I

Augment “real” information with additional data

Correct bad “real” information

Bad weatherNew ATC project II

Occlusion

Head-Down-Time ProblemNew ATC project III

Lack of optimal attention splitting (looking down instead oflooking up, also in good visibility conditions)

And in order to look outside: stand up!

3

New ATC project IV

No….different approach: field studies

Using radar information for gathering data Highlight and profiling information

Forget about Augmented Reality Focus on the *tower* Going on the field….

Again ATC but different approach

Drawback….Very short studies •:-(

Ethnography requires longer times (weeks, months…..…)

And yet….interesting findings

First: know a bit your users….

Look at the physical space I

Different aerodromes design Different procedures

• E.g backtrack• Vs parallel rolling• What if landing?• Workload / task demand

Definition of “standard roles” inside the tower Ground (GND), Tower (TWR), Clearance Delivery, etc

• “Real” controllers: GND and TWR

• Everybody is in charge of something specific• NOT REALLY STANDARD• Flexible….

• E.g one controller “playing” GND+TWR

• Every tower has tailored staff to its own requirements

Look at the physical space II

Speak, but don’t speakPhysical space and collaboration

“Gaze direction”

Hand over

Collaboration: not a single user

Paper strips/strip bay

You’re on the phone, I cannot hear, but I understand, hence Igive you the info

“Smell the situation” “Feel what is going on” “Knowing yourcolleague, the mood too”

4

The company responsible of the airport operations executes RWY inspections• Check lighting system• Remove debris

Forgetting about the presence of vehicles on the RWY is a serious threat tosafety (although not uncommon!!).

A pre-requisite for the execution of landings and takeoffs is the verification ofthe Runway (RWY) status (occupied or vacated).

How to remember RWY status???

Look at the artifacts I Different solutions during the years…

• Initially a luminous indicator had to be turned on by thecontrollers all the times a vehicle occupied the RWY

So, how come that the led is not there anymore???

Look at the artifacts II

“Despite the clear visibility of the led, sometimes we were simplyforgetting about it ... we were also forgetting of turning it off when the

RWY was vacated”.

Pay attention when users get angry I

First 2 towers, after a couple of days• One controller started to “scream” against a pilot

• (when radio off)

• Use expression as that was not a singleoccurrence but something “recurring”

• E.g. They are all the same…they always behave likethis…..impossible to trust them….

• Investigate, and asked more questions….• Different levels of trusts towards pilots• Quite common in “those 2” towers

Pay attention when users get angry II

Second “round” of observation• Also questionnaire, hence occasion to understand

whether “trust in pilots” is an issue

31 controllers from two very different towers• Big commercial airport• Small airport

According to your experience, are there some pilots thatyou trust less than others?

Tower I, yes: 85% Tower II, yes: 80%

Pay attention when users get angry III

Trust to pilots “verbal”• Poor English, poor standard phraseology…

But also “visual”• Taxiing with authority

• Quick and smooth motions of the aircraft• Rapid and accurate execution of the instructions

• Allocation of attention across aircraft

• “An extra eye to the ones I trust less”

Learn the rules I

During the second “round” of observation(questionnaire)

• Understand separation rules in the big airport

• Distance from DME• Outer-marker: the mental gate• …….what is that supposed to mean?…..

5

Learn the rules II

Understand the rule …through the artifact….

Sometimes difficult to translate field findings into design However….

What’s the meaning of all this?

Check aerodrome design (external physical configuration)• Specific design implies different task demand• E.g. Backtrack procedure

• Backtrack speed in relation to landing: projections needed

Check tower “internal” physical configuration• Need for information sharing• Support “silent” collaboration• Intrusive technology as HMD is not appropriate

What’s the meaning of all this?

Check artifacts, especially modification-creation• Revealing “troubles”- “needs”• Flaws in the system that must be “fixed”

• Do they say something about what should be displayed(in AR or any other display)???

Trust issues……• Radio communication (verbal info) but also visual

• Taxiing with authority…..quick… smooth…accurate….

Using radar information for gathering data

What’s the meaning of all this?

NO !! Radar gives delayed info!!! Alternative technologies for feeding

information …

What’s the meaning of all this?

Check and understand the rules Outermarker-DME for separation

• Alerting system if spacing according to these rules isviolated

And much more……

More detailed requirements…..

Scenarios……

Collection of *real* operational occurrences Episodes (stories) telling about:

• Users, in a context, doing something, facing problems,having some needs, etc…..

Support elicitation of more specific requirements Provide basis for future evaluations

Example

6

Error and wings The AFR123 is exiting the Main TXY via R4, then starts following the follow

me car, directed towards the NPA. An AZA 567 is parked just below thetower, at the stand 431. As soon the AFR approaches the AZA and startsmaking its turn, the GND controller stands up. She suspects that the wings ofthe two A/C may collide. Visibly worried she asks to the colleagues: “Will hemake it? Are they going to hit each other?”. The CRD replies, that, in hisopinion, they will not collide, as it is just a parallax error. As the AFR hasfinished turning (without any problem), the GND says: “You’re right, it was aparallax error”.

TowerNPA

The green A/C isparked at the standThe black A/C isexiting from R4(multiple black AA/CCdenote A/C motion)NP is North ParkingArea

In brief… Field studies are very useful

• Give real insights on real users, real needs, problemsand real tasks

• Provide basis for design requirements• Both general - specific level

• Expensive, require strong commitment• Take time

• Do results generalize?