human measurement technologies– compression changes capacitance by moving diaphragm • need power...

01/09/2018 CPEN541: Copyright from 2019, Sidney Fels

Human Measurement Technologies

!1

01/09/2019 EECE518: Copyright from 2002, Sidney Fels

• HMT: Introduction – categorizations – physical properties

• Introduction to Sensors • Tracking Technologies

– magnetic – optical – mechanical – video based – other

• Primary user input – head tracking – eye tracking – hand tracking – pointing and selecting – other

HMT: Overview

!2


• How to categorize? – By human sense organs?

• Visual, audio, proprioception, taste, smell • doesn’t relate to which physical property is being measured

– By physical properties/physical principle? • Electrical sensor, optical sensor, magnetic sensor, mechanical sensor • one effect can be used to measure many attributes

– By relation of source to reference? • Inside-in

– sensor and source on body • Inside-out

– sensors on body which sense off-body source (artificial or natural) • Outside-in

– external sensor measures artificial or natural markers • no relation to what is being measured

HMT: Introduction

!3


• For this class: – start with discussion of some physical sensors – task separation

• general (large object) body tracking • primary user input techniques

– pointing and selecting • mouse+

– head tracking – eye tracking – hand tracking

• Other interesting devices

HMT: Introduction

!4


• General HMT framework:

• For HMT we’ll focus on: – input/data acquisition – some pattern/gesture recognition

• these will be good topics for student lectures

HMT: Introduction

!5


• Physical quantities available for transduction – electrical

• voltage • resistance • impedance

– optical • colour • intensity

– magnetic • induced current • field direction

– mechanical force

HMT: Sensors

!6


• How to pick a sensor? – Decide which human part/action you want to measure – determine which attributes are related to the part – check what sensors you have available

• possibly modify it to suit your needs – if none available

• determine how to transduce quantity • build sensor

HMT: Sensors

!7


• Transduced physical quantity may need – conversion to voltage – filter

• band limiting • outliers removed

– reduce complexity

HMT: Signal Conditioning

!8


• Possibly need to do some data acquisition – use a data acquisition board plugged into

your computer • e.g. National Instruments DAQ

– Up to 16 analog inputs; 12-bit resolution; up to 500 kS/s sampling rate

– Two 12-bit analog outputs; 8 digital I/O lines; two 24-bit counters

-Icube (voltage->MIDI signal) -Arduino board - audio port (single analog channel) - video capture board (frame grabber)

HMT: DAQ

!9


• Piezoelectric Sensors • Force Sensing Resistors • Accelerometer (Analog Devices ADXL50) • Biopotential Sensors • Microphones • Photodetectors • CCDs and CMOS cameras • Electric Field Sensors (Fish) • More Sensors

HMT: Sensors

!10


• Piezoelectric effect – energy converted between mechanical

force /pressure and electrical form • piezoelectric microphone • piezoelectric speaker

– deformation of piezoelectric causes charge build up on surface

– put material in capacitor and measure voltage change

• Q_f is charge change due to pressure

HMT: Piezoelectric Sensors

!11


• About 10-15% error • useful for many applications

HMT: Force Sensing Resistors

!12


• Measure acceleration of sensor • several techniques available to do this

– use a mass with some springs – use gyroscopes – use vibrating tuning forks – heated air cooling

• Basic principle: – Hooke’s law: F = kx – Newton: F = ma – displacement is proportional to acceleration – single axis accelerometer

• Analog Devices ADXL50 accelerometer •

HMT: Accelerometer

!13


• Popular model – Analog Devices ADXL50 accelerometer

• new models available: ADXL150 and 250 ($15-$24) • 10 mg resolution, 10mg to 50g

– dual axis – micromachined silcon bar – capacitance used to measure displacement – intended application:

• airbags

HMT: Accelerometer

!14


• Other techniques used measuring acceleration: – Electromechanical – Piezo-Film – Piezoelectric – Piezo Resistive Bulk Micromachined – Capacitive Bulk Micromachined

• Applications: – tilt sensing – inertial sensing

HMT: Accelerometer

!15


• Carbon: – lightly packed carbon – compression increases conduction

• needs power supply

• Capacitor (condenser): – capacitor between a stationary metal plate, and a light metallic

diaphragm – compression changes capacitance by moving diaphragm

• need power supply

• Electret and Piezoelectric: – already talked about these – no external power needed

• Magnetic (moving coil): – induction - moving conductor in magnetic field

HMT: Microphones

!16


• Output voltage proportional to light – photodiodes (use voltage bias, light affects

concentration of electron/hole pairs -> current ) – photovoltaic effect (P-N junction, light produces

voltage, electron/hole pairs separate) • also used for solar panels

• can arrange in array – photodiode array (PDA)

• used in Optotrak

Current changes with light

HMT: Photodectors

!17


• CCD - charge coupled device – 2D array of capacitors on silicon substrate – light creates electron/hole pairs and accumulates charge – read charge after some interval

• similar to PDA – more sensitive to low light

HMT: CCD and CMOS cameras

!18


• Some cameras use different sizes of CCD element • some use 3 CCDs

– red, green and blue separated • Lots of different styles and products • Very versatile

– low light sensitivity • astronomy applications, microscopes

– Example: Marshall Electronics, V1055 • 0.05 lux, HAD sensor technology • 510 * 492 pixels (60 fields/sec) • 380 lines of resolution • Automatic electronic iris • 12 VDC, 100 mA, 1.2W • 1/3” CCD

• Consumes lots of power – need various sub-voltages

Can be board mounted

HMT: CCD Cameras

!19


• CCDs have to transfer charge rows and columns one at a time

• CMOS photodiode arrays put amplifier at each pixel – about 3 transistors (photodiode version) – 1 transistor (photogate version)

• amplifier circuitry takes up a lot of room – only viable recently as sub-micron tech. gets better – only useful for low-end still

• cheap (<$100), low power (10-50mW vs. 1-2W) • offer single chip solution

HMT: CMOS Cameras

!20


• Example: Marshall Electronics – Single chip 1/3" format video camera – 310 TV line resolution – Composite NTSC/PAL video output – On-chip auto exposure – Automatic gain control – Auto white balance – 7 VDC-12 VDC, 20mA @ 9 VDC

HMT: CMOS Cameras

!21


• Sense electric field – shunting

• external EF grounded by person – transmitting

• low frequency energy coupled to person • person is emitter

• uses: – to transmit data, Personal Area Network (PAN) – position sensing

• some problems – proximity sensing

• easy to make • cheap

HMT: Electric Field Sensing

!22


• Here’s a few that find common usage: – Bend - piezo-resistive – Close - IR reflection, 1-7” – FarReach - ultrasonic (50Hz update) – Flash - phototransistor – Gforce - piezo-electric single axis accelerometer – Hot - zener effect (thermocouple)

• -40 to 100deg C – Light - photo-resistive

HMT: More Sensors

!23


• Reach - EMF disturbance • Slide - resistive • TapTile - Force senstive resistor • Tilt

– electrolytic, single axis (-70-+70 deg) • Touch - 0 travel FSR • TouchGlove

– several touch sensors • TouchStrip

– long touch sensor • Turn

– potentiometer

HMT: More Sensors

!24


• Tracking user location is very important for many applications – want 3 dof position

• X, Y, and Z – want 3 dof orientation

• roll, pitch, yaw (Euler angles) – sometime better to use:

• quaternions (or angle axis) • rotation matrix

– Trackers used to measure 3-6 dof typically.

HMT: Tracking Interfaces

!25


• Head • Eyes • Hand • Body • Other

HMT: Things to Track

!26


• Main characteristics – resolution – accuracy – range – System responsiveness

• sample rate - sensors checked (frequency) • data rate - positions computed (frequency) • update rate - system reports new positions (frequency) • latency (lag) - hopefully in msec • repeatability - same bend, same data value (variance) • linearity - response characteristic • drift - temperature, mechanical force (% of max)

HMT: Head Tracking Interfaces

!27


• Some Head Tracking Technologies also good for tracking: – Hand – Body

• but, some only good for heads • Uses:

– viewpoint tracking • immersive VR with HMDs • attention

– motion parallax systems • fish tank VR, Cubby,

HIT: Head Tracking Interfaces

!28


• In head tracking: – latency is probably most important system attribute

• >10msec may cause sickness (Durlach, 1994) • >60msec impair adaptation and immersion (Durlach, 1994) • >500msec are not interactive (Bryson, 1993)

– situation determines importance • I.e. hand tracking latency more critical in non-immersive VEs

– (Ware and Balakrishnan, 1994)

– Sources: • tracker, communication delays, computation delays, graphic

rendering, (Bryson, 1993) • video sync.


!29


• Head movements – max: 1000 deg/sec (33 deg/33msec) – typical:

• 600 deg/sec (20 deg/33msec) yaw • 300 deg/sec (10 deg/33msec) pitch and roll

– most energy below 8 Hz and all below 15 Hz • -> update rate of 30 Hz is sufficient


!30


• Polhemus Fastrak is very popular sensor – see also Ascension for

competition – many different instantiations

• long range, cheap version, small, etc.

• Uses alternating magnetic field • Receiver has three wire loops • As loops move in field voltage

is induced • Measure voltage and determine

HMT: Polhemus Fastrak

!31


• Specs ($5000 with one receiver, $1000/receiver) – full accuracy 30”, reduced accuracy 10” – latency, 4msec - really?

• 8.5 msec, unfiltered (Adelstein, Johnston, Ellis, 1995) – Update Rate

• 120 Hz / number of receivers – Linear Accuracy

• 0.03 in – Angular Accuracy

• 0.15 degrees – Resolution

• 0.0002 in/in, 0.025 degrees orientation – RS232 interface, – worst user manual in the world – only works in one hemisphere


!32


• Optotrak 3020 from Northern Digital Inc. (about $100,000)

• Uses IR based receivers with IR LED markers on body • three 1-D CCD track IR LEDs

– 3 dof of markers calculated from CCD readings • can use off line or in real time mode • Specs:

– data rate: • 3500Hz (raw), 600Hz (real-time 3D)

– Accuracy • 0.1mm for x, y, 0.15 for z at 2.25m distance

– x,y Resolution • 0.01 mm at 2.25m distance

– Max markers: 256 – FOV: 1.28m x 1.34m at 2.25m

• Good for very accurate position detection at high data rates

HMT: Optotrak

!33

01/09/2019

• OptiTrack • Vicon • others

Many other optical motion tracking system out there

!34

01/09/2019

• Motion tracking with body model – head, arms and feet – body geometry

• 20 joints per person – face recognition

• RGB camera – 30 Hz

• depth sensor – Kinect: Infrared pattern light + camera – Kinect 2: infrared time-of-flight camera

• microphone array – directional sound localization, speech

recognition and noise cancelation • Cheap!

Kinect (2011) and Kinect 2 (2014)

!35


• Eye tracking is important in: – motion parallax based systems

• fish-tank VR • Cubby

– attention – collaboration systems – camera focus and stabilization

• Techniques: – optical (reflection) – electrical (electrooculogram EOG) – magnetic (wire in lens on eye)

HMT: Eye Tracking

!36


• Many different attempts – Levine, 1981; Bolt, 1982l; Ware and Mikaelian, 1987; Jacob, 1990 – Neural network technique, Baluja and Pomerleau, 1993 – MAGIC (Manual and Gaze Input Cascaded Pointing), Zhai,

Morimoto, Ihde, CHI99 • Main problems are

– fovea is 1 degree and there is jitter • 25” viewing distance, 0.44 in • not precise enough for UI widgets

– Eye has not evolved as control organ • considerable non-volitional movement • select techniques not natural to eye

– dwell time, blink

HMT: Eye Tracking

!37


• Solution – separate what eye is good at from what it is bad at

• use gaze to indicate region for cursor • manual input for target acquisition and selection

– Liberal positioning • move cursor to any new location more than 120 pixels from current

location – Conservative

• move cursor only when manual device activated – cursor appears moving toward target in same direction as manual device – interaction now with input device dynamics

– Is eye tracking a Fitts’ Law task? • Pointing time is proportional to log(A/W) • Ware and Mikaelian, 1987 - yes • Silbert and Jacob, 1996 - no • if not, time to acquire should be faster

HMT: MAGIC

!38


• Implementation – IBM Almaden Eye Tracker

• tried ASL Model 5K, not good enough • use two light sources for reflection off retina

– one on-axis and one off axis – synchronize two Near-IR sources with odd and even camera

fields – most systems use only one light source – Tomono et al., 1989, Ebisawa and Satoh, 1993

• Use corneal glint plus pupil location to determine gaze direction

• filter to select only fixation points – ignore saccadic movements

HMT: MAGIC

!39


HMT: MAGIC

!40


• Pilot study – didn’t use mouse

• “intelligent” mode is problem since clutching needed • used… isometric pointing stick… surprise! • Did typical Fitts’ pointing tasks in experiments • Manipulated

– target sizes (20 and 60 pixels) – target separation (200 pixels, 500 pixels and 800 pixels) – target directions (horizontal, vertical and diagonal)

• Three conditions – manual only – conservative MAGIC – liberal MAGIC

HMT: MAGIC

!41


• Results: – liberal slightly faster than manual only (6.8%) – conservative slightly slower than manual only (4.3%) – fastest time was achieved with conservative (1.03s) – Both generally liked by test group – subjects noticed engineering problems with tracker – not clear how much of an advantage can be had

• should be able to do much cheaper now

HMT: MAGIC

!42


• Eye tracking will become cheap – will it be useful?

• Hands free environments • disabled users

– probably don’t want to use it as point and select device • attention tracking rather than precise pointing device

HMT: Eye Tracking Summary

!43


• Hands/fingers are main techniques for manipulation • positions and orientations all very valuable • multiple degrees of freedom can be easily manipulated

simultaneously – in 2D and 3D

• coordination of two hand (and feet) can be very good (see Guiard’s work on bimanual control)

• quiet, unobtrusive, embedded mode identification etc. etc. – pointing and selecting (group presentation)

• mouse, touchpad, isometric joystick, isotonic, touch screen, etc. • see Buxton’s discussions on pointing - does 1+1=2?

– The details of pointing devices are critical. • see MacKenzie and Oniszczak, CHI’99 for some comparisons on

touchpads

HMT: Hand/Arm/Body Tracking

!44


• Hands can do other things – see McNeill for example

• deitic and symbolic gesturing • how to use in a CHI?

• Technologies for tracking: – Fastrak, Flock of Birds – Cameras (Utsumi) – Cyberglove, VPL Dataglove, Exos glove

• cheaper versions – LEDs


!45


• Applications – 2D and 3D input devices – Remote control manipulators – puppetry and computer animation – musical performance – surgical simulation – scientific simulation


!46


– gestural interface examples • sign language recog., finger spelling

– Kramer and Leifer, 1989 – Glove-Talk: Fels and Hinton, 1990

• Music controller – foot controller (Paradiso, 1998) – sound sculpting (Mulder, Fels, and Mase, 1999)

• gesture mappings – Glove-TalkII: Fels and Hinton, 1998


!47



!48



!49



!50


HMT: CyberGlove

!51

01/09/2019

Inertial Measurement Unit (IMU)

!52

Xsense


• 5 & 14 sensors

• fibre optic based

• USB/RS232

• 75 Hz sample rate

HMT: 5DT Glove

!53


HMT: Pinch

!54


• New Technologies – Ring interface (Fukumoto and Tonomura - CHI97) – Electric Fish Technology (variation on Theremin)

• Zimmerman et al., CHI95 and now used in PAN – Virtex uses flex sensors for body suit – Electromagnetic Articulograph (EMA) - see Zierdt,

Hoole and Tillmann

Hand/Arm/Body Tracking

!55


• Put ring mounted accelerometers at the base of each finger. – No acquire time

• currently still using wires – use direct coupling

• similar to EF body coupled like PAN

• Chords used to specify characters – novice: 27 symbols/hand, 130 symbols/min – expert: 52 symbols/hand, 210 symbols/min

HMT: FingeRing

!56


HMT: FingeRing

!57


• Besides 6dof trackers

3D Input Devices

!58


HMT: Immersion Probe

!59


HMT: FingerMouse

!60


HMT: SpaceBall

!61


HMT: Toronto EGG

!62


• GTII • SoundSculpting • Bricks • Virtual Chopsticks • Instrumented Footwear

HMT: Gesture I/F Examples

!63


• Bricks: Laying the Foundations for Graspable User Intefaces – Fitzmaurice, Ishii and Buxton, CHI’95

• idea – use physical artifacts (bricks/handles) to attach to

virtual objects • graspable user interface • affordance of handle leveraged

– two-handed, spatial caching, parallel position and orientation control

• objects are on ActiveDesk

HMT Examples: Bricks

!64


• Bridges gap (somewhat) between – time-multiplexed input – space-multiplexed input

• Graspable UIs – two handed interactions – specialized, context sensitive input devices – parallel input – leverage prehensile behaviours – externalizes internal computer representations

• aka tangible bits – interface elements more direct – leverage spatial reasoning – space multiplexing – affords multi-person/collaborative use


!65


• Implementation – Lego sized objects’ position tracked

• use Ascension birds and mock ups – physical object associated/”attached” to virtual object

• i.e. MacDraw objects – One handle Operations

• place brick on object -> attach • lift brick off object -> detach • moving brick moves attached objects • rotating brick rotates object (c.o.g. centre of brick)

– Two handles • stretching, other deformations


!66



!67


• Experiments – used mock-ups for exploratory hand

manipulation tasks • block sorting tasks, stretchable square, MacDraw

comparison and Curve matching – Prototype used for GraspDraw

• physical tray for function selection – select, delete, rectangle, line, circle, colour – use audio feedback to indicate selection

• developed concept of – anchor: origin of frame of reference – actuator: operate in anchors frame of reference


!68



!69


HMT Examples: Bricks Design Space

!70


– Like LEGO behaviour construction kits (Resnick, 1993),

– AlgoBlocks (Suzuki and Kato) – 3-Draw (Sachs, Roberts and Stoops) – Props (Hinkley et. al. - look in readings, 1994) – Digital Desk (Wellner, 1993)


!71


• Many technologies available to measure human activity

• Knowledge of technologies critical for research and development in – pointing and selecting – ubiquitous computing – musical controllers – 2D and 3D direct manipulation interfaces – etc.

HMT Summary

!72


• Important issues: – measurement side

• resolution, accuracy, update rates, lag, range etc. – application side

• continuous vs. discrete • mapping issues • intrusive • matching degree of freedom of control with task • and others

HMT Summary

!73

human measurement technologies– compression changes capacitance by moving diaphragm • need power...

Documents