idea lab presentation
DESCRIPTION
IDEA Lab PresentationTRANSCRIPT
Using wearable motion sensors to recognize human gestures
Peng Deng, Qifeng Mao{pdeng,qmao}@students.csse.unimelb.edu.au
CSSE University of MelbourneLabSUM∑
2
Agenda
• Introduction– Wireless sensor network– Normal applications– WSN in HCI
• Our HCI project– Idea– How it works using Sun SPOT– Challenges and solutions
• Demos
• Future apps and comments
3
Introduction: Sensor Networks1. Deploy2. Network setup3. Query and response
User
Event
CommunicationProcessing Element
Sensing Element
P S O U W PE PR L
Y
SENSORS
ADC MICRO
PROCESSOR
MEMORYRADIO
REAL TIME OS
ALGORITHMS
Limited Lifetime
Require Supervision Slow
Processing
Limited Memory
1 kbps – 1 Mbps, 3 – 100 m, Lossy
Transmission
4
Applications
• Environmental monitoring
• Security
• Defence
• Bioinformatics and health
• Transportation management
• Chemical detection and emergency response
Pictures from [3]
5
Accelerometer based HCI
6
Our idea
• “Off the desk” interaction
• Multiple sensor nodes on body– 1*hand held � 2*wrists+1*waist+2*feet
• Recognize gestures � recognize actions
• Use different sensors (not limited to accelerometer) to interact with environment near by
7
Sun SPOT
8
Sun SPOT
• 3-axis accelerometer• Temperature sensor• Light sensor• LEDs• Analog inputs• Switches• General purpose I/O
Embedded sensorsEmbedded sensors
2.4 GHz IEEE 802.15.4 radiowith integrated antenna
RadioRadio
512K RAM/4M FlashMemoryMemory
180 MHz 32 bit ARM920TCPUCPU
32 uADeep sleepDeep sleep
720 mAh lithium-ion batteryBattery capacityBattery capacity
Sun SPOTPlatformPlatform
NetBeans 5.0IDEIDE
JavaProgramming LanguageProgramming Language
Sun Java Squawk VMFrameworkFramework
[7] [8]
9
Sun SPOT Applications [7]
10
3-axis Accelerometer
General purpose. Depends on developers’ implementation
Carefully tuned to optimize performance in determining hand and arm motion
PurposePurpose
250~30050Acceleration Noise DensityAcceleration Noise Density
850 uA @ 3.3 VSleep mode support
300 uA @ ~3VPower consumptionPower consumption
+/- 2G (600 mv/g)+/- 6G (200 mV/g)
+/- 3 G (300 mv/g)Range & SensitivityRange & Sensitivity
USD $10.82USD $8.97PricePrice
ST Microsystems LIS3L02AQAnalog Devices ADXL330 ChipChip
Sun SPOT [18]Wii Remote Controller [17]
Hzg /µ Hzg /µ
11
How it works
Base
Station
CU-
HTKApps
12
How it works cont.
• CU-HTK: a speech recognition toolkit– Supervised machine learning (HMM)
• Training phase– 5 samples per gesture
• Recognition phase– Find most similar trained gesture
13
Challenges and solutions
• Human gestures are inexact and vary every time– HMM
• Real time data stream segmentation– Manually– Threshold level– Machine Learning (DTW, SVM, LDA …)
14
Demos
Pictures from [15] [16]
Q & A
