Bilge Mutlu, Andreas Krause, Jodi Forlizzi, Carlos Guestrin, and Jessica Hodgins

Human-Computer Interaction Institute, Carnegie Mellon University

Robust, Low-cost, Non-Intrusive Sensing and Recognition of Seated

Postures

Why seated postures?

AutomobileAutomobile ClassroomClassroom WheelchairWheelchair

HomeHome OfficeOffice

Using posture information

Today’s talk

Pellegrini and Iocchi., 2006

• KinestheticMotion-capture markers or conductive- elastomer-embedded fabrics

Existing approaches

• KinestheticMotion-capture markers or conductive- elastomer-embedded fabrics

• Vision-basedImage sequences from a single camera or multiple cameras

Tognetti et al., 2005

Existing approaches

• KinestheticMotion-capture markers or conductive- elastomer-embedded fabrics

• Vision-basedImage sequences from a single camera or multiple cameras

• Pressure-sensing-basedPressure readings from the seating surfaces

Existing approaches

Han et al., 2001

•Poor generalizationGood performance in classifying “familiar” subjects, poor performance with “unfamiliar” subjects due to high dimensionality.

•High costHigh-fidelity pressure sensors are expensive.

•Slow performanceProcessing high-fidelity sensor data demands computational power, which leads to slow processing.

ChallengesRobust generalization

Low-cost

Near-real-time performance

Our solution•Robust generalization

Up to 87% accuracy in classifying 10 postures with new subjects.

•Low costUsing 19 pressure sensors instead of 4032. Reducing sensor cost from $3K to ~$100.

•Near-real-time performance10Hz on a standard desktop computer

•Novel methodologyUsing domain knowledge and near-optimal sensor placement.

Methodology

Learning Algorithm•Logistic Regression

Sparse representation

•Cross-validation10-fold, gender-balanced training and testing samples from different subjects

•Separate setsTraining, testing, and reporting samples from 52 people in 5 trials

• Implementation in Java

We would like to thank Hong Tan and Lynne Slivovsky for providing their data set for comparison.

✴

Understanding pressure dataModeling

Understanding pressure dataModeling

Understanding our dataModeling

Domain knowledgeModeling

FeaturesModeling

Size and position of bounding

boxes

Distances to the edges of the

seat

Distance and angle to between bounding

boxes

Parameters of the ellipses that fit the bottom

area

Pressure applied to the bottom

area

FeaturesModeling

Classification accuracy

Separability testModeling

Feature eliminationModeling

Methodology

Dimensionality Reduction

Sensor granularity

Dimensionality Reduction

Sensor granularity

How to place sensors?•F, feature variables

•V, locations and granularities

•A subset A of V that maximizes information gain about F where H is entropy

•NP-Hard optimization problem

•We use near-optimal approximation algorithm

Dimensionality Reduction

IG(A;F) = H(F) - H(F | A)

FV

A ⊆ V

Near-optimal placementDimensionality Reduction

Sensor placementsDimensionality Reduction

Near-optimal placementDimensionality Reduction

Classification accuracy

Methodology

Prototyping

Evaluation of prototype• 20 naive participants

10-fold cross validation testing with %5 of the data

• 78% accuracyIn classifying 10 postures

• 10 Hz real-time performanceOn a standard desktop computer

Methodology

Conclusions•Generalizability

Up to 87% (with a base rate of 10%) achieved with unfamiliar subjects.

•Low costHigher classification accuracy than existing systems using less than 1% of the sensors. ~ $100 sensor cost compared to the commercial sensor for $3K (33 times reduction in price).

•Near-real-time performanceAt 10Hz on a standard desktop computer.

Applications

AutomobileAutomobile ClassroomClassroom WheelchairWheelchair

HomeHome OfficeOffice

Future challenges•Transferring learning across chairs

A “transformation map” could be created

•Only static posturesTemporal dimension needs to be considered

•The set of ten posturesThe set of postures should come from the activity

Next Steps

Summary of Contributions• A non-intrusive, robust, low-cost system that recognizes

seated postures with generalizable, near-real-time performance.

• A novel methodology that uses domain-knowledge and near-optimal sensor placement strategy for classification.

This work was supported by NSF grants IIS-0121426, DGE- 0333420, CNS-0509383, Intel Corporation and Ford Motor Company.

From Postures to Activities• Reading the paper

• Watching TV

• Reading paperwork

• Watching TV + eating

• Sleeping

• Talking on the phone

• Reading a book

• Craftwork

• Reading the paper + watching TV

• Reading the paper + eating

Next Steps