interaction devices devices 1 tactile: keyboard, mice, stylus
TRANSCRIPT
devices 1
Tactile: keyboard, mice, stylus / tablets (chord keyboards, 3D gloves)
Visual: monitors, printers (helmets, see-through, microscreens)
Acoustic: speech synthesis & recognition, music
Smell: burn chemicals, control air flow, in movies across head rest
Taste ?
Neurological Implants, “R U wired?”, Sci-Fi
cochlear implants
Virtual Retinal Display (VRD)
Laser-based projected image
onto the retina
Visual Cortical Implant
Interaction Devices
devices 2
QWERTY designed for manual (slow) typewriters
Dvoark minimizes finger travel distance
Dvoark takes a week or so to learn:
Benefits not greater than effort to learn?
Resistance to change?
Chord Keyboards
Several keys are pressed to enter a symbol.
Speed up to 300w/m with high accuracy.
Steep learning curve, constant practice.
E.g.: court recorders, piano keyboards...
Keyboards
QWERTY Dvoark
Words / minute 150 200
Error more less
qaz
wsx
edc
r tf gv b
space
u yj hn b
ik,
o l.
p;/
.458.2
.957.9
.6818.4
.5422.9
.42
.5421.6
1.177.2 .41
12.6.601.3
% Error% Work Load
symbols
devices 3
QUERTY
Work Load and Errors
common device performance
metrics / analysis
task performance
devices 4
Special keys often require "homing" or positioning of hands off
keyboard's "home row".
Homing is a source of error (description) and often requires a change in
visual focus from task to keyboard.
Function keys require homing. Side function keys were operated by
single hand.
Control sequence (shortcuts / accelerator) commands require least
homing like function key use.
Cursor positioning keys vary both home position and key layout across
keyboards.
Homing Position
devices 5
3rd hand - homing problem
Direct Pointers: fingers, stylus / pen (lightpen, touchscreen)
+ User selects and manipulates objects on display directly with pointer
device. More natural (touchscreen).
+ Fast positioning
- Pointing on vertical display causes fatigue.
- Less accurate, pointing (hand) can block view of display. In-Out
touchscreen positioning can help accuracy.
Touchscreens useful in novice user and environmental critical areas.
automated tellers, information kiosk, factory floor computers
controlling manufacturing, civil engineers on construction sites,
survey workers in the field.
phones, tablets, laptops (vertical fatique)
Pointing Devices
devices 6
- User positions a "pointer" icon on screen with a pointer device.
The pointer device does not contact the display. After indirectly
positioning icon the user can select and manipulate displayed
objects.
- Positioning less "natural", a learned skill
+ View not blocked, less fatigue
Indirect pointers have a resolution – sensitivity to movement
("pixels"/inch, ballistic mode)
Mouse is dominant indirect pointer.
Card et.al. showed arrow key faster
than mouse for very short distances only.
Fitt's law applies to the study of pointing tasks.
Distance
0
1
2
3
4
1 2 4 8
Centimeters
Se
co
nd
s
Arrow Mouse
Indirect Pointers (mice, joystick)
devices 7
Direct pointers are faster than keyboard cursor keys in most cases.
Indirect pointers excel at fine resolution position of on screen targets and
far positioning tasks.
Cursor keys excel at low resolution position of on screen targets (form
fill in tasks) and close position tasks.
Cursor keys preferred on tasks that mix text processing (keyboard
required) and positioning
Less 3rd hand homing problem with cursor keys and keyboard
commands.
Indirect pointer are often "preferred" position and selection devices in
"easy to use" oriented software.
Are small mobile devices “easy” or “convenient” to use?
User satisfaction due to closure, control, direct manipulation interface.
Keyboard VS. Pointer Devices
devices 8
Feet based pointer devices (no 3rd hand)
Head (eye) based pointers -- infrared headbands, glasses
IR head tracker
Trackir
3D Manipulators
Data Glove: manipulates objects, has sense of contact with object. No
"mass/resistance" feedback.
Joystring: manipulates objects, no sense
of contact with object, "mass/resistance" feedback.
3D “mouse”: eg. 3DConnexion.
Dimensions = (x, y ,z, pitch, yaw, and roll).
Interfaceboard
trackingsensor
tactilefeedbackflexing
sensors
Dataglove
foot pointerUncommon pointers
devices 9
Interfaceboard
trackingsensor
tactilefeedbackflexing
sensors
Dataglove
foot pointer
Joystring
hand grabs "inverted T"
wires attached to
stepper motors provide
resistance.
devices 10
Natural I/O interface: selection, input
Metaphor to existing tools
Expressive and portable
Smart Paper & Electronic Ink
Smart paper (use awareness in the media)
text OCR convert, auto complete, spell (grammar) check, justify,
evaluate math
draw CAD conversion, snap, stretch...
Electronic writing - pen strokes (vs bitmaps) can be edited, layered
(“post-its”), attached to objects in smart paper.
Pen I/O
devices 11PenPoint -- eg. pen commands, common edit cmds
brackets, selects pieces of text
caret, insert text
check, display options for selected text, objects, icons, tools, etc
circle, opens edit pad to modify text fields and labels
cross out deletes object beneath it
scratch out deletes sfsff
flick, scrolls document in flick direction {up, down, right, left}
pigtail deletes a character
tap selects or invokes
press - hold initiates drag (move)
tap - press initiates drag (copy)
devices 12
Palm Pilot's graffiti reduced stroke input language
write letters here write numbers here
Graffiti
Help screen
Division marks
Start
stroke at
heavy dot
Lift
Stylus
here
devices 14
3 interacting variables of color vision:
Hue color
Brightness intensity (bright - dull)
Saturation % color in field
Opponent process theory of color vision
These colors can't be seen in same patch of light.
They produce shadows and edges.
Avoid use of opponent (opposite) colors.
Color is very useful to have user selected items stand out in a display.
Spatial and temporal representations of blue colors is worse than
other colors.
Selection / Applicability
Color can also be used to indicate whether a menu option is valid in
the current state or not ("greyed options").
blue / yellow
green / red
white / black
Color Vision
devices 15
Alert / Attention.
Change of color represents change in state (green, yellow, red).
• Use few colors that are easy to discriminate
• Use warning colors sparingly.
• Consistent system wide analysis of color use.
Element Discrimination.
Color provides contrast and improves discrimination. Need high
contrast difference. Contrast a function of luminance or hue.
Category grouping & field definition. Color can help group display
elements and facilitate visual search. Visual search is affected by:
• number of items
• color separation of categories
• legibility of coded symbols
• relationship between color coding and targets
Color Usage
devices 16As screen density increases color effect increase.
Color can define visual fields on display - weather maps
Size & Visual Acuity
As number of colors increase size of text should also increase.
Color can't be assumed! Redundantly code display.
Designer's color perception != user's color perception
color & text codes (categorization)
color, size & text
color, size, text & icon ....
Color Memory:
5 - 7 color memory for codes.
Don't tax Working Memory use around 4 colors!
devices 17Strong color connotations:
red danger, error, hot, revolution
green OK, go on, well, alive, healing
yellow slow down, caution, sun light
Weak color connotations:
blue fluid / liquid, wet, calm, hidden
black empty, death, anarchy
brown earth, warm
Color Preferences
• Children prefer warm colors: reds, browns
• Adults prefer cool colors: blues, greens
• Occupations (degrees) have color associations:
green health
orange engineering
devices 18
No more than 6 colors (including black) should be used on one screen.
Backgrounds should not be brighter than foregrounds. Grid lines should
be half intensity
Do not have extreme color contrasts between foreground and
background colors -- causes afterimages due to rod fatigue.
Use white for critical / important dynamic information (color gun fails)
A monochrome flash of twice intensity is as effective as color use.
Separate significant information on display by size, distance, intensity
or highlighting.
Use screen position consistently. Time / Date info or page number in
same place.
Groups screen elements. Provide title for elements. Hard to title than
grouping is poor.
Rules of Thumb for Visual Displays
devices 19
US Government suggested color use in (critical) monitoring systems
Color State Result
Flashing red Emergency immediate action
red Alert corrective action
magenta Emergency out of limit indicator
warning
yellow Advise Caution, recheck
Blue Advisory ( use only as
background / filler )
devices 20
Pixels (picture elements) / inch - density measure. For text higher
resolution implies larger font dimensions. (Text same visual size)
Pixel shape (saturation)
Display Fields
Layout (tiled)
center in fields
uncluttered fields
Resolution
devices 21
Tiled vs Overlapping Windows.
Titled: Applications with static window sizes and little or no window
manipulation. E.g.: form entry, message windows, popups.
Overlapped: Applications with dynamically sized windows with user
window manipulation.More robust - let user control display rather than
application. E.g.: text or graphic editors.
Experts tend to prefer overlapped (control ,customization).
Novices tend to prefer tiled (full screen / switch apps) when applicable.
Preference
full screen Vs overlapped
mobile devices ?
tiles Vs full screen
Windowed Displays
devices 22Refresh rate
CRT Vs LCD / LED displays
CRT (85 Hz)
+ faster refresh rate, less motion blur, less pixel persistence
+ independent of age and temperature
? better color resolution, better black / white
- flicker, eye fatique
- burn in, constant display burns phospors
LCD / LED (60 or 75Hz, set @ 75Hz)
+ less eye fatique, slower refresh rate (60hz), pixel persistence
120Hz / 144 Hz LCD availability
+ text display
+ LED black / white
- degrades with age and temperature
- LCD black / white
Older users have greater eye fatique with CRT.
devices 23
Display glasses user sees a full-sized
display floating in front of their eyes.
Projected Displays - displays projected
transparent surfaces: glass windows, cockpits, visors
opaque surfaces: SixthSense, Cave
Enable user to view displays w/o moving eyes in critical environments.
Proposed for displays in cars. (dashboard controls, in car TV / movies
Wearable displays
devices 24
Visual display can appear 360 degrees, or at least always be present
regardless of head position.
HMD w/ trackers can update display based on head movement (inside a
virtual world).
Sound (and possibly smell) can also be presented (3D sound effects).
Complete control of user's visual and acoustic interface. No external
environment interference.
Comfort?
Head Mounted Displays
devices 25
Speech Synthesis (generation)
+ Good quality - issue of digitized sampling resolution and complexity.
+ Basic speech phonemes can be edited to make comprehensible
speech.
? Speaking devices: cars, computers, houses...
Acoustic Interfaces
devices 26
hearing & understanding limited w/o training
Trained systems. User speech commands are sampled and stored for
pattern matching.
Production Systems:
spoken cmd pattern1 : action1
spoken cmd pattern2 : action2
: :
- Sensitive to surrounding noise
- Low transference across users
- High storage overhead for representing spoken commands and
searching command pattern set. (Can be hierarchical in nature).
Speech Recognition
devices 27
+ High Quality, actual sound
? capacity function of sampling
rate / compression
CD R/W provides large capacity
Using Recorded (or Digitized) Sound
Sampling & manipulations (music -rap)
Sound Bites (commercials, film, TV) HCI Applications:
Voice Mail -- Computer Phone surveys - digitized voice
production with user input by touch tones.
Acoustic menu system...
Acoustic Desktop
Sonic Icons: pilots, blind
Desktop becomes a spherical environment.
Digital Sound Processing
devices 28
(Reality) = = the product of our 3 lb. universe (brain)
Reality consists of a person's perception of the world around them. That
perception is the product of their sensory inputs and expectations.
Illusions, "magic" are the interactions of our sensations and
expectations. We see and hear what we want, what we expect.
Current computer controlled interactive devices have the rudimentary
capability to generate a truly artificial reality for the user:
Vision and Acoustic control of environment via helmets, goggles,
headphones, ...
Tactile interaction and feedback via data glove like devices for
hands, feet, body glove ...
Existing, accepted artificial realities: movies, books, television, games,
simulators for training, sensory deprivation, dreams...
Virtual Reality
devices 29Uses of VR, AR (augmented reality)
Multimedia interfaces like interactive CD-ROM for graphics,
animations, and sound, helmets, and data gloves enable complex
adventure/fantasy environments (caves) for entertainment.
Will books and movies become interactive ?
Scientific Visualization - engineers and scientist can move through
dangerous or theoretical worlds.
Boeing Dreamliner first virtual (VR assisted) designed aircraft
Medicine - surgeons walk through accurate 3D representations of
patients gathered w/ CAT or MRI scans. (Robotic/Waldo surgery)
Waldo enhancements for workers in dangerous environments.
Programming as a visual/intuitive process - looking at data and processes
aka Neuromancer. Software walkthrough gets a new meaning!
Is software modeling 2D (UML...) or can it be 3D
devices 30
"We try to move our designs closer to the
human being and blend the machine with
the body." - Hideji Takemasa, NEC
Wearable Data Terminal an optical scanner
worn on forearm enables reading bar codes,
OCR, graphics, with writable optical disk and
CD-ROM database computer located in
terminal worn on neck.
The MIThril hardware platform combines
body-worn computation, sensing, and
networking in a clothing-integrated design.
The MIThril software platform is a
combination of user interface elements and
machine learning tools built on the Linux
operating system.
Wearable Computers
devices 31
annual wearable computing conference urls
http://iswc.gatech.edu/archives.htm
http://www.media.mit.edu/wearables/
http://wearables.gatech.edu/
"A person's computer should be worn, much
as eyeglasses or clothing are worn, and
interact with the user based on the context of
the situation. With heads-up displays,
unobtrusive input devices, personal wireless
local area networks, and a host of other
context sensing and communication tools, the
wearable computer can act as an intelligent
assistant, whether it be through a
Remembrance Agent, augmented reality, or
intellectual collectives." -- MIThril
devices 32I, Cyborg
In 1998 Porfessor K. Warwick surgically implant a silicon chip
transponder surgically implanted into his forearm.
http://www.kevinwarwick.com/index.asp
He can be monitored using a signals
emitted by the chip. He could operate
doors, lights, heaters and other computers
without lifting a finger...
In 2002 a 100 electrode array was
implanted into the median nerve fibres of
his left arm. The implant can send signals
back and forth between Warwick's
nervous system and a computer.
This bi-directional functionality was
demonstrated with the aid of Kevin’s
wife Irena and a second, less complex
implant with her nervous system...