Sensor Analysis – Part II

A literature based explorationThomas Plötz

[material taken from the original papers and John Krumm “Ubiquitous Computing Fundamentals”

• sports: training / reviewing movements after the “fact”• daily routine: efficiency (where do you waste time?) – quantified self movement

vanKasteren2008-AARvanKasteren2010-TKOPatterson2005-FGA Perkowitz2004-MMO

Loga

n200

7-AL

T

Ward2006-ARO

ADL well studied and recognized segmentation + classification variety of application domains (health, smart homes, …)

- no fine-grained analysis (read: trivial? Beyond proof-of-concept?)- no quality assessment (How good? vs. What?)

… your conclusions here!

inexpensive non-obtrusive, privacy respecting (typically …) even wearable

- overkill?

… your conclusions here!

• Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers• classical Activity Recognition – ISWC-style

Sound Intensity Analysis for Segmentation of Activities – analysis of two microphone signals (I1 – wrist ; I2 – upper arm)

Sound Classification (which tool used?) based on spectral analysis and templates (after smoothing)

• Spectral representation of acceleration data, calculated using sliding window technique

• HMM-based classification of acceleration data

• different fusion techniques for integrating hypotheses from acoustic acoustic recognition and accelerator-based processing

pioneering work combination of modalities• explicit segmentation step• focus on classification techniques and fusion (NOT on feature representation)

• Fine-Grained Activity Recognition by Aggregating Abstract Object Usage• object-centered sensing and recognition (in contrast to ISWC-style)

Independent HMMs connected HMMs Object-centered HMMs

[DBN with aggregates]

• Transferring Knowledge of Activity Recognition across Sensor Networks

• What if we change the environment?

• list of ADLs is the same for each house, but sensor networks differ

1. How to deal with differences in sensor networks (due to different layouts of houses)?2. How to learn model parameters that respect differences in behavior of the inhabitants?

• Modeling approach: HMM• “A prior distribution is learned from the source houses and used to provide a sensible initial value for the model parameters of the target house.”• conjugate prior: posterior is of same functional form as prior• procedure:

1. learn model parameters for source house (using ML)2. Learn hyperparameters for initial state, and transition is straightforward using

numerical estimation3. Learn hyperparameters for observations requires feature space mapping and

numerical estimation

• Finally: EM-based estimation of target model parameters exploiting estimated priors

• Mining Models of Human Activities from the Web

• What if we don’t have enough (or none) sample data?

“ […] requires models of the activities of interest, but model construction does not scale well: humans must specify low- level details, such as segmentation and feature selection of sensor data, and high-level structure, such as spatio-temporal relations between states of the model, for each and every activity.”

• “[…] we show how to mine very large libraries of human activities from the web, instead of analyzing sensor data.”• “PROACT assumes that “interesting” objects in the environment contain RFID tags.”• “Users employ RFID tag readers to track tag objects they interact with.”• “As users go about their daily activities, the readers detect tags that (a) users touch, (b) are close to them, or (c) are moved by them, and thereby deduce which objects are currently involved in an activity. PROACT uses the sequence and timing of object involvement to deduce what activity is happening.”

• Modeling based on Hidden (semi) Markov Models

• general model description via web-mining

• extract initial parameters (probabilities)using Google conditional probabilities (GCP)

P(o, i, l) = GoogleCount(“l”+o) / GoogleCount(l)

P(o, i, l): prob. that object o is involved in step i of the activity labeled l.

Problem: Skewed distributions (one very dominant class)

Problem: Fragmentation; or: Does it really make sense to evaluate sample wise?

Idea: Event-based evaluation

Bibliography• van Kasteren et al. Accurate activity recognition in a home setting. Proc. Int. Conf. on Ubiquitous Computing

(2008)

• Ward et al. Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans. Pattern Analysis and Machine Intelligence (2006) vol. 28 (10) pp. 1553--1567

• van Kasteren et al. Transferring Knowledge of Activity Recognition across Sensor Networks. Proc. Int. Conf. on Pervasive Computing (2010) pp. 283--300

• Perkowitz et al. Mining models of human activities from the web. Proc. 13th Int. Conf. on World Wide Web (2004) pp. 573--582

• Ward et al. Performance metrics for activity recognition. ACM Trans. on Intelligent Systems and Technology (2011) vol. 2 (1)

• Logan et al. A long-term evaluation of sensing modalities for activity recognition. Proc. Int. Conf. on Ubiquitous Computing (2007) pp. 483--500

• Patterson et al. Fine-Grained Activity Recognition by Aggregating Abstract Object Usage. Proc. IEEE Int. Symp. on Wearable Computers (2005)