user interface engineering hs 2013 - ait lab · research perspective overview of own research as...
TRANSCRIPT
||
Input Fundamentals
29.09.2013 1
User Interface Engineering – HS 2013
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Brief Overview of HCI as a discipline
History of the UI
Product perspective
Research perspective
Overview of own research as motivation
29.09.2013 2
Last Week
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Design
PrototypeEvaluation
||
Admin
Input fundamentals
Buttons, switches
Keyboards
Mice
Basics of human capabilities and performance
29.09.2013 3
Overview
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 4
Exercise location (for hands on work)
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
CNB D101
|| 29.09.2013 5ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
https://www1.ethz.ch/immobilien/gmis/do_enter
• Lab access
• Predefined hours
(more details in
today’s exercise)
• Electronic building
access
• Download form
• Print, sign and return
• Barbara von Allmen
• Me
• Matthias Kovatsch
• CNB H floor
||
Date still tba – We will communicate this before the next lecture.
Will be roughly same date as last homework assignment's due date (30.10.)
29.09.2013 6
Midterm exam date
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
In-Class vote
Possible dates:
Last lecture: 18.12.13
After Christmas: 15.1.14
During exam period: 22.1.14
29.09.2013 7
Final Project Date?
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 8
Today
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 9ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 10ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 11
Bop It – What’s in it?
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 12
Bop It – What’s in it?
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 13ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
https://www.sparkfun.com/products/9136
||
Buttons are everywhere
29.09.2013 14
Buttons & Switches
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Especially on interactive devices
||
What’s in a Switch?
29.09.2013 15ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 16
Switches
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
+
-
Interrupter Switch
|| 29.09.2013 17
Switches
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
+
-
Two way diverter switch
||
Number of
Poles
Throws
Default Behavior for Push Buttons
Normally Open (NO)
Normally Closed (NC)
Duration of contact
Long-term (Switch)
Momentary (Button)
29.09.2013 18
Types of Switches
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 19
Magnetic Reed Switch
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 20
Mercury Tilt-Over Switch
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Designing with Switches - Essentials
29.09.2013 21ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 22
An interactive system – Conceptual Overview
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Mechanical
elements
Enclosure
Circuits Microcontroller Host-PC
Sensors
Actuators
ADC
DAC
|| 29.09.2013 23
Microcontroller
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Atmel ATmega328
Used in Arduino and other electronics
frameworks
|| 29.09.2013 24
Arduino Board
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 25
Arduino Pin Mapping
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 26
Interactivity
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
D
LED
1K
Output 12
5V
SP
ST
Input 13
10K
|| 29.09.2013 27ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
/* * Switch and LED test program */
int ledPin = 12; // LED is connected to pin 12
int switchPin = 13; // switch is connected to pin 13
int val; // variable for reading the pin status
void setup()
pinMode(ledPin, OUTPUT); // Set the LED pin as output
pinMode(switchPin, INPUT); // Set the switch pin as input
void loop()
val = digitalRead(switchPin); // read input value and store it in val
If (val == LOW) // check if the button is pressed
digitalWrite(ledPin, HIGH); // turn LED on
if (val == HIGH) // check if the button is not pressed
digitalWrite(ledPin, LOW); // turn LED off
|| 29.09.2013 28ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 29ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 30
Key cap construction
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Key Switch Membrane
Conductive Layer
Conductive Layer
Spacer Non Conductive
|| 29.09.2013 31
Key cap construction
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Key Switch Membrane
Conductive Layer
Conductive Layer
Spacer Non Conductive
||
IBM compatible keyboard has 104(+) keys
To expensive to read each key individually
Depending on controller 20-100 IO pins
Needed for many different tasks
Solution:
Connect buttons in matrix fashion
Keyboard: 16x8 matrix requires 24 pins
29.09.2013 32
Keyboard Switch Matrix
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 33ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Rows & Columns connected to
microcontroller
Controller drives circuitry and generates key
events (key codes)
Each row and column pin
Can be used as in- or output pin
Output pins are driven (low / high)
Input pins are read (low / high)
29.09.2013 34
Key Matrix
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
A B
C D
Mic
rocontr
olle
r
||
Drive rows, read columns
Drive columns, read rows
29.09.2013 35
Single Key
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
A B
E F
C D
G H
I J
M N
K L
O P
||
Drive columns sequentially
Read rows simultaneously
29.09.2013 36
Multiple Keys
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
A B
E F
C D
G H
I J
M N
K L
O P
||
Certain constellations may lead to erroneous key press/release events
“C” is detected – pressed or not
Release of “B” can’t be detected
29.09.2013 37
Multiple Keys - Ghosting and Masking
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
A B
C D
A B
D
||
Diodes restrict flow direction of current
29.09.2013 38
Eliminating Ghosting and Masking
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
D
|| 29.09.2013 39ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 40ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Many other devices (can) produce binary events just like buttons
Proximity detector
Touch sensor
Light barrier
…
29.09.2013 41
Things that act like switches
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 42ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Optoelectronic sensor
essentially a super fast low-res camera
Typically 18x18 pixel
Sometimes >3000Hz (gaming mice)
LED at grazing angle
Onboard microprocessor computes 2D
coordinates
29.09.2013 43
Optical Mice
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Mouse sensor operates under very
constraint circumstances
Translation only
Fixed illumination
Simple flow methods are sufficient
29.09.2013 44
Optical Flow
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Cross-Correlation
Computes similarity between signals (offset in
some dimension(s))
CC-based Optical Flow computation
Shift frame t0 relative to frame t1
n times in all directions
Highest correlation score signifies motion
direction
29.09.2013 45
Onboard Optical Flow
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Implemented on ASIC
Integrates imaging sensor and processor
Compute optical flow via correlation
Store previous frame
(Row- and column-wise) sums of XORs of
single pixels
Integrate decisions to track movement
Massive parallel implementation
Very fast in hardware
29.09.2013 46
Optical Flow in Hardware
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Uses infrared laser diode rather than LED
for illumination.
Exploit laser speckle
Pattern produced by mutual interference of
several wavefronts of the same wavelength
but different phase and amplitudes
Observable when coherent lightsource (i.e.
laser) is shone onto a surface (each point on
the surface acts as secondary lightsource)
Increases observable structure in the
environment
Allows for much higher sensing fidelity and
faster update rate
Works on more surfaces (even glass)
29.09.2013 47
Laser Optical Mice
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 48ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 49ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 50ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
IR Emitter
Photo Diode
|| 29.09.2013 51ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Forward / Backward rotation is ambiguous!
|| 29.09.2013 52
Quadrature Encoding
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Clockwise
Rotation
Phase A B
1 0 0
2 0 1
3 1 1
4 1 0
C-Clockwise
Rotation
Phase A B
1 1 0
2 1 1
3 0 1
4 0 0
||
15 Minute Break
29.09.2013 53ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Continuous Input
29.09.2013 54ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Convert variable resistance to measurable signal
Resistance-varying and fixed resistor in series,
measure voltage between them
No discrete event
Polling required
ADC necessary
29.09.2013 55
Voltage divider
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
𝑉𝑂𝑢𝑡 = 𝑉𝑠𝑢𝑝𝑝𝑙𝑦 ×𝑅2
𝑅1 + 𝑅2
||
Resistive element
Strip or arc
Sliding contact (Wiper)
Electrical terminals at each end of resistive element
Electrical terminal at the wiper
29.09.2013 56
Example: Potentiometer
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
R
|| 29.09.2013 57
Example: Rotary Encoder
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Force Sensing Resistor
29.09.2013 58
Further examples
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
http://www.openmusiclabs.com/learning/sensors/fsr/
|| 29.09.2013 59ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Resistance straight: 10kΩ
Resistance bent: 40kΩ
29.09.2013 60
Further examples: Flex sensor
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 61ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 62ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 63
Basic Human Cognition and Perception
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
How fast / accurate can we perform a task?
Depends on relatively stable and well
understood human capabilities
Can be modelled using basic cognitive
processes
Allows us to predict task completion time
Can complement / replace usability studies
Allows to compare different design choices
29.09.2013 64
Human Cognition and Targeting Performance
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Developed by S. Card, T. Moran, A. Newell (The Psychology of Human-
Computer Interaction, 1983)
Basic model that gives us an abstracted understanding of
Perception
Memory
Motor system
Can be applied to estimate:
Execution time
Error rates
Training effects for simple stimulation / reaction interactions
29.09.2013 65
Model Human Processor
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Three processors with associated memory
Perceptual system
Sensors and buffers
Cognitive system
Working memory content symbolically coded
Cognitive processes
Motor system
Movements
Overall ‘runtime’ determines task completion
time
29.09.2013 66
Model Human Processor - Overview
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
29.09.2013ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges 67
29.09.2013ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges 68
||
Receives sensor signals and stores in buffers
One buffer for each sensory channel
Perception time 𝜏𝑝 ≈ 100 𝑚𝑠
Range from 50 − 200 𝑚𝑠
Bloch’s Law: 𝑅 = 𝐼 ∙ 𝑡 𝑅 is response, 𝐼 is Intensity and 𝑡 is exposure time
𝑅 is constant for 𝑡 < 100 𝑚𝑠
Explains why movies appear as fluid animation
29.09.2013 69
Perceptual processor
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Operates on chunks of information
Processing time 𝜏𝐶 ≈ 70 𝑚𝑠
Short Term Memory as working memory
Conscious control of cognitive processes
Serial “Tunnel of consciousness”
Examples: multiply, read
Automatic cognitive processes
Parallel
Example: automatic visual searching
Repetition and building of connections stores
information in long-term memory
29.09.2013 70
Cognitive processor
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
0446321111
29.09.2013 71
Chunking Example
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
vs
044 632 11 11
||
Short-term storage of limited data
Maximum capacity: 7 [5,9] units, duration: 2 sec
Typical capacity: 3 [2,4] units, duration: 7 sec
Storage capacity & duration depend on cognitive load
Even modest cognitive load increases error rate
STM limited by number but not complexity of units
Chunking (grouping of symbols)
29.09.2013 72
Cognitive processor
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 73
Long-term Memory
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Memory of the past
Declarative memory
Associative network of representations of facts
Episodic memory: experienced events
Semantic memory: facts, knowledge
Procedural memory
Operative knowledge, typically not conscious
Skills (type writing, piano playing, skiing)
Memory capacity practically unlimited
Problem is organization for retrieval
Duration depends on quality and intensity of elaboration
Fast read, slow write => commiting to LTM maxes out at 7s / chunk
|| 29.09.2013 74ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 75
Motor System (Processing time)
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Experiment: Move pen between lines as fast as possible for 5 sec
Open loop
Without perceptual control
ca. 70 pen reversals in 5 sec
𝜏𝑀 ≈ 70 𝑚𝑠
Closed loop
Perceptual system controls movement
ca. 20 corrections in 5 sec
Processing time
𝜏𝑃 + 𝜏𝐶 + 𝜏𝑀 ≈ 240 𝑚𝑠
|| 29.09.2013 76
Motor System: Fitt’s Law
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Directed movement as an information processing task
Not limited by muscles, but by ability to process sensory input
Predict time to hit a target (to press a button) as function of distance and size
Robust performance model (Fitts, 1954): 𝑇𝑝𝑜𝑠 = 𝐼𝑀 ∙ 𝐼𝐷
Index of difficulty 𝐼𝐷 = 𝑙𝑜𝑔2(2𝐷
𝑊)
Index of Movement 𝐼𝑀 = 100 ms / bit (approx. constant)
W
D
|| 29.09.2013 77
Fitts’ Law: Experiment
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
4 cm
16 cm
8 cm
For each target width & distance:
tap for 10s, count taps afterwards
||
Doubling the distance adds roughly a constant to execution time
Logarithmic nature of the law
Doubling the target width (within limits) is roughly equal to halving the distance
Clear by D/W term in the formulation
29.09.2013 78
Fitts’ Law Summary
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 79
Interpreting Results
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Draw graph with 𝐼𝐷 values on the x-axis and average 𝐼𝑀 values on the y-axis
Perform a linear regression (e.g., spreadsheet program)
Fixed distance D
Varying width W
Slope = 𝐼𝑀
MT = -0.4595 + 0.8092 ID
R2 = 0.93
0
0.5
1
1.5
2
2.5
3
3.5
4
0 1 2 3 4 5 6
ID
MT
[sec]
||
Shannon’s Formulation:
𝑇𝑝𝑜𝑠 = 𝑎 + 𝑏 ∗ log2𝐷
𝑊+ 1
a,b depend on device, need to be empirically derived
Use 𝑎 = 0, 𝑏 = 𝐼𝑀 = 100 𝑚𝑠 for quick and dirty estimates
Improved curve fit
And?
29.09.2013 80
Improvements
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Find uses of Fitts’ Law in modern software design.
Email screenshot and a one paragraph explanation
on how Fitts’ Law (and Shannon’s Formulation in
particular) might have influenced design decisions.
We will discuss your examples next week.
29.09.2013 81
Homework (not for credit)
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
|| 29.09.2013 82
Summary
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
Overview of input devices built from simple buttons and
switches
Discrete (Game controller buttons, Keyboards)
Analog(-ish) (Mouse-Wheel, Data Glove)
Working principle of several fundamental input devices
Mouse
Keyboard
Basic model of human cognitive processes and pointing
performance
|| 29.09.2013 83
Next Week
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges
||
Scherz, Paul (2006). Practical Electronics for Inventors. McGraw-Hill
http://www.andyholtin.com/links/Practical_Electronics_for_Inventors.pdf
Tucker, Allen B. (2004). Computer Science Handbook (Fundamental Input and Output Devices).
Chapman and Hall. 2nd Ed.
Lyon, Richard F. (1981). The Optical Mouse and an Architectural Methodology for Smart Digital
Sensors. Xerox PARC TechReport
Card, S.K; Moran, T. P; and Newell, A. The Model Human Processor: An Engineering Model of
Human Performance. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of Perception
and Human Performance. Vol. 2: Cognitive Processes and Performance, 1986, pages 1–35.
29.09.2013 84
Reading suggestions
ETH Zürich – Departement Computer Science – User Interface Engineering – FS 2013 – Prof. Dr. Otmar Hilliges