the user as interface public

8/8/2019 The User as Interface Public

1/29

The User As Interface

Microsoft Research Cambridge, UKMicrosoft Research Cambridge, UK

Lyndsay Williams April 2000Lyndsay Williams April 2000


2/29

Lyndsay Williams Current interests include mobile and wearableCurrent interests include mobile and wearable

computers, and biometric sensingcomputers, and biometric sensing

Graduated 1980 University of Salford, UKGraduated 1980 University of Salford, UK BiomedicalBiomedical

electronics Honselectronics Hons Designed SmartQuill at BT Labs, Ipswich, UKDesigned SmartQuill at BT Labs, Ipswich, UK

Joined Microsoft Research, Cambridge, UK Nov 98 asJoined Microsoft Research, Cambridge, UK Nov 98 asResearcherResearcher


3/29

Analogue sequencer for music synthesisers

designed and published in 1977 by Lyndsay Williams


4/29


5/29

ADXL202 accelerometer


6/29

Inspiration.


7/29

SmartMoveX project goals To determineTo determine locationlocation andand activityactivityof personof person

using biometric sensors, e.g.using biometric sensors, e.g.

Movement, heartrate, skin resistance, ambientMovement, heartrate, skin resistance, ambientlightlight

Aim for very low bandwidth = 4 bytes/minAim for very low bandwidth = 4 bytes/min

Applications for wearable computersApplications for wearable computerscellphonescellphones

Want to use sensing available in environmentWant to use sensing available in environment NOT tied to radio infrastructureNOT tied to radio infrastructure privacyprivacyreqreq


8/29

SmartMoveX prototype 8 bit microcontroller8 bit microcontroller

Wearable on belt in pager caseWearable on belt in pager case

Skin, light, accelerometerSkin, light, accelerometer

HeartHeart

Record length 11 hoursRecord length 11 hours

Connect to pc via RS232Connect to pc via RS232


9/29

Prototype XSkin data loggerPager sized data logger of skin

resistance, acceleration and light


10/29

View inside working prototype ofSmartMoveX, designed & built at MSR

CambridgePIC 16c57 8 bit micro


11/29

The SensorsAccelerometer ADXL202 from Analog Devices

Skin resistance sensorVoice recorder, LDR and heart rate monitor


12/29

Closer view of skin

sensors


13/29

One experiment 11 hour recording of 4 parameters11 hour recording of 4 parameters 1 min intervals1 min intervals

Audio feedback of skin resistance in real timeAudio feedback of skin resistance in real time

Skin resistance sensor attached on upper armSkin resistance sensor attached on upper arm

Synchronised biometric data using a Sony ICDSynchronised biometric data using a Sony ICD--3535digital voice recorder with time taggingdigital voice recorder with time tagging

Downloaded to pc via RS232Downloaded to pc via RS232

Analysed using Excel SpreadsheetAnalysed using Excel Spreadsheet


14/29

Unfiltered

data:11hourrecording2

2000

weatherfromAT&T

020

40

60

80

100

120

140

160

180

06:43:38

07:26:38

08:09:38

08:52:38

09:35:38

10:18:38

11:01:38

11:44:38

12:27:38

13:10:38

13:53:38

14:36:38

15:19:38

16:02:38

16:45:38

17:28:38

18:11:38

accel

heart

light

skin

res


15/29

SmartMoveX record 28 March 00 1615 onwardsseries 1,2,3,4 Acceleration , heart,light,skin res

0

20

40

60

80

100

120

140

160

180

16:

15:

8

16:

20:

8

16:

25:

8

16:

0:

8

16:

5:

8

16:

40:

8

16:

45:

8

16:

50:

8

16:

55:

8

17:

00:

8

17:

05:

8

17:

10:

8

17:

15:

8

17:

20:

8

Series1

Series2

Series3

Series4


16/29

3 hour recording of activity starting at 0643GMT


17/29

A Day In The Office 17th Nov 1999


18/29

Xskin record sync to AutoRoute

Express


19/29

Heartrate/skin resistance correlation?? More

work needed


20/29


21/29


22/29

Screen shot of SmartMoveX connected to Window CETerminal

LIGHT = 2 SKIN = 17 ADDRESS = 413.


23/29

Analysis goalsInterface to Windows CEInterface to Windows CE

Raw data is 4 bytes/min of biometric data, inc timeRaw data is 4 bytes/min of biometric data, inc time

How do signals correlate e.g. subtract algorithm accel fromHow do signals correlate e.g. subtract algorithm accel fromheart?heart?

IdeaIdea -- SmartMoveX system learns behaviour of user e.g. trainingSmartMoveX system learns behaviour of user e.g. trainingphase user tells device their activityphase user tells device their activity e.g. cyclinge.g. cycling invisible to SMXinvisible to SMXwhat parameter is (e.g. heartrate)what parameter is (e.g. heartrate) just a valuejust a value

How accurate is a prediction of user activity?How accurate is a prediction of user activity?

User speaks to SmartMoveXUser speaks to SmartMoveX records voice & logs sensor valuesrecords voice & logs sensor values

What is minimum number of bytes to define a physical activityWhat is minimum number of bytes to define a physical activity 4 +4 +time?time?

Band activity into zonesBand activity into zones predict activity/ location classification?predict activity/ location classification?

Liase with interested Redmond Researchers for software analysis e.g.Liase with interested Redmond Researchers for software analysis e.g.Eric Horvitz, Dan Venolia, Mary CzerwinskiEric Horvitz, Dan Venolia, Mary Czerwinski


24/29

Future XSkin hardware

Currently interfaces to Windows CE Palm PCCurrently interfaces to Windows CE Palm PCCasio E105 Terminal programCasio E105 Terminal program

Sample 4 chans at 100Hz = 1.4Mbyte/hr intoSample 4 chans at 100Hz = 1.4Mbyte/hr intoPalm PCPalm PC

Data log module to integrate all 4 sensorsData log module to integrate all 4 sensors

Simple filtering classification done onSimple filtering classification done onSmartMoveXSmartMoveX

Radio transmit for real time skin resistanceRadio transmit for real time skin resistancetransmit via 418Mhz link to pctransmit via 418Mhz link to pc


25/29

418Mhz AM transmitter from RF Solutions

100Metres, 3V, 2.5mA100Metres, 3V, 2.5mA

1200 Baud1200 Baud


26/29

SmartMoveX - some

conclusions It seems possible to determine user activity andIt seems possible to determine user activity and

location from simple environmental sensorslocation from simple environmental sensors

Use sensors for power management of handheldUse sensors for power management of handheld

computerscomputers


27/29


28/29

Other projects

Haptic penHaptic pen -- tilttilt

SmartQuill below British TelecomSmartQuill below British Telecom


29/29

Any Questions? ContactContact [email protected]@microsoft.com

www.smartquill.comwww.smartquill.com

the user as interface public

Documents