![Page 1: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/1.jpg)
Activity recognition based on a multi-sensor hierarchical-
classifier
IWANN 2013, 12-14 June, Tenerife (Spain)
Oresti Baños, Miguel Damas, Héctor Pomares and Ignacio Rojas Department of Computer Architecture and Computer Technology, CITIC-UGR,
University of Granada, SPAIN
DG-Research Grant #228398
![Page 2: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/2.jpg)
Introduction
• Activity recognition concept
– “Recognize the actions and goals of one or more agents from a series of observations on the agents' actions and the environmental conditions”
• Applications (among others)
– eHealth (AAL, telerehabilation)
– Sports (performance improvement, injury-free pose)
– Industrial (assembly tasks, avoidance of risk situations)
– Gaming (Kinect, Wii Mote, PlayStationMove)
• Categorization by sensor modality
– Ambient
– On-body
2
![Page 3: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/3.jpg)
Sensing Activity
3
• Ambient sensors
![Page 4: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/4.jpg)
Sensing Activity
• Ambient sensors
Limitations*
![Page 5: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/5.jpg)
3rd Generation (and beyond…)
2nd Generation 1st Generation
Sensing Activity
5
• On-body sensors
![Page 6: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/6.jpg)
Activity Recognition Chain (ARC)
6
![Page 7: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/7.jpg)
Activity Recognition Chain (ARC)
7
![Page 8: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/8.jpg)
Activity Recognition Chain (ARC)
8
![Page 9: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/9.jpg)
Activity Recognition Chain (ARC)
9
![Page 10: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/10.jpg)
Activity Recognition Chain (ARC)
10
![Page 11: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/11.jpg)
Activity Recognition Chain (ARC)
11
![Page 12: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/12.jpg)
Activity Recognition Chain (ARC)
12
![Page 13: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/13.jpg)
Activity Recognition Chain (ARC)
13
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S u p s1,s2,…,sk fℝ(s1,s2,…,sk) c
![Page 14: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/14.jpg)
Activity Recognition Chain (ARC)
14
![Page 15: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/15.jpg)
Activity Recognition Chain (ARC)
15
![Page 16: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/16.jpg)
Activity Recognition Chain (ARC)
16
SENSOR FUSION
![Page 17: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/17.jpg)
ARC Fusion: Feature Fusion
17
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
c u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
fℝ(s11,s12,…,s1k,
s21,s22,…,s2k,…,
sM1,sM2,…,sMk)
![Page 18: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/18.jpg)
ARC Fusion: Decision Fusion
18
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k) c1
c=φ(c1,c2,…,cM)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k) c2
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk) cM
![Page 19: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/19.jpg)
Multi-Sensor Hierarchical Classifier
19
SM
S2
S1 α11
∑ C12
C1N
C11
∑
C21
C22
C2N
∑
CM1
CM2
CMN
∑
Decisio
n
Class level Source level Fusion
β11
α12 β12
α1N β1N
α21 β21
α22 β22
α2N β2N
αM1 βM1
αM2 βM2
αMN βMN
γ11,…,1N δ11,…,1N
γ21,…,2N δ21,…,2N
γM1,…,MN δM1,…,MN
[-0
.14
, 3.4
1, 4
,21
, … ,
6.1
1]
[-0
.84
, 3.2
1, 4
.21
, … ,
6.1
1]
[-0
.81
, 5.7
1, 4
.21
, … ,
6.2
2]
[-0
.14
, 3.9
2, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
![Page 20: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/20.jpg)
Multi-Sensor Hierarchical Classifier
20
N activities M sensors & Class level Source level Fusion
![Page 21: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/21.jpg)
Multi-Sensor Hierarchical Classifier
21
N activities M sensors & Class level Source level Fusion
![Page 22: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/22.jpg)
Multi-Sensor Hierarchical Classifier
22
N activities M sensors & Class level Source level Fusion
![Page 23: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/23.jpg)
Multi-Sensor Hierarchical Classifier
23
N activities M sensors & Class level Source level Fusion
![Page 24: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/24.jpg)
Experimental setup: dataset
• Fitness benchmark dataset
• Up to 33 activities
• 9 IMUs (XSENS) ACC, GYR, MAG
• 17 subjects
24 Baños, O., Toth M. A., Damas, M., Pomares, H., Rojas, I., Amft, O.: A benchmark dataset to evaluate sensor displacement in activity recognition. In: 14th International Conference on Ubiquitous Computing (Ubicomp 2012), Pittsburgh, USA, September 5-8, (2012)
![Page 25: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/25.jpg)
Results
• Segmentation: sliding window (6 seconds) • Feature extraction: FS1={mean}, FS2={mean,std}, FS3={mean,std,max,min,cr} • Classification: Decision tree (C4.5) (10-fold cross-validated, 100 repetitions)
25 10 activities 20 activities 33 activities
FS1 FS2 FS3 FS1 FS2 FS3 FS1 FS2 FS360
65
70
75
80
85
90
95
100
Accura
cy (
%)
Feature Fusion Weighted Majority Voting Multi-Sensor Hierarchical Classifier
Experimental Parameters
![Page 26: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/26.jpg)
Conclusions
• We propose a multi-sensor hierarchical classifier that allows data fusion of multiple sensors
– Its assymetric decision weighting (SEinsertions/SPrejections) leverages the potential of the classifiers either for classification/rejection or both
– Specially suited for complex scenarios
• Feature Fusion and MSHC are quite in line in terms of performance however
– Our method outperforms the former when a more informative feature set is used
– Particularly notable for complex recognition scenarios
• Our model is expected to be particularly suited to deal with sensor anomalies (work-in-progress)
26
![Page 27: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/27.jpg)
On-going work…
• Our model is expected to be particularly suited to deal with sensor anomalies (work-in-progress)
27
FEAT-FUSION MSHC0
20
40
60
80
100
Accura
cy (
%)
Ideal Self Induced
![Page 28: Activity recognition based on a multi-sensor meta-classifier](https://reader034.vdocuments.mx/reader034/viewer/2022042602/557e790bd8b42a4d108b4c99/html5/thumbnails/28.jpg)
Thank you for your attention. Questions?
Oresti Baños Legrán Dep. Computer Architecture & Computer Technology
Faculty of Computer & Electrical Engineering (ETSIIT) University of Granada, Granada (SPAIN)
Email: [email protected] Phone: +34 958 241 516 Fax: +34 958 248 993
Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme under grant agreement no. 228398, the Spanish CICYT Project SAF2010-20558, Junta de Andalucia Project P09-TIC-175476 and the FPU Spanish grant AP2009-2244.
28