gymskill - a personal trainer for physical exercises

Technische Universität München Fachgebiet Verteilte Multimodale Informationsverarbeitung Prof. Dr. Matthias Kranz

GymSkill: A Personal Trainer for Physical Exercises

Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr,

Matthias Kranz

Technische Universität München, Germany

Nils Hammerla, Patrick Olivier, Thomas Plötz

Newcastle University,

United Kingdom

March 22 PerCom 2012, Lugano, Switzerland


Outline

•  Motivation

•  Automatic Assessment of Physical Exercises

•  Case Study

•  Conclusion

04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 2


Motivation

•  Physical activity is required for healthy lifestyle •  Problem: people do not exercise enough

–  Lack of knowledge of correct exercise execution for fast improvement –  High level of long-term motivation needed

•  Advantages of a personal trainer: –  Continuous supervision and professional feedback –  Individualized advice and motivation –  Minimization of injury risk

•  Disadvantages of a personal trainer: –  Expensive –  Privacy



•  Scientific focus on –  Activity recognition –  Wearable sensors

•  Health and Fitness Systems –  Heart rate monitor, foot pod, GPS watch, …

•  No skill assessment

–  Fixed activity tracking (Wii balance board, Kinect) •  Gaming aspect •  Dedicated hardware

Related Work



GymSkill


•  Smartphone-based personal trainer •  Monitoring and assessment of physical exercises

–  Based on phone sensor data –  No need for additional sensors


Outline

•  Motivation

•  Personal Health and Fitness Systems


•  Case Study

•  Conclusion



GymSkill: Automatic Assessment of Physical Exercises

•  We look at rocker board exercises •  Example use case for equipment-based training •  Defined quality criteria:

tilt angles, speed, smoothness of movement, touching the ground… •  Phone is attached to the board •  During exercise: Basic situated feedback •  After exercise: Fine-grained analysis of exercise quality and reasons for

quality differences


move back and forth move left and right balance on the center


Skill Assessment Principle


Sensor Data Recording

Sensor Data Processing PCBA Analysis

Skill Assessment

Detailed Skill Report

Real-Time Feedback

Skill Level (Score)

Output Rendering

Log File

AJAX

Smartphone Server“Cloud“

Simple Analysis

HTTP

User Feedback


Direct Feedback While Training

•  Repetition count •  Visual feedback of board movement •  Warning when tilted too far •  Optional acoustic feedback



Post-Exercise Analysis

•  Performed for each exercise run (e.g.: 10 repetitions of tilting back and forth) •  Global Analysis

–  Smoothness and continuity of movement –  Global motion quality –  Usage of board‘s degrees of freedom

•  Local Analysis –  Identify „interesting portions“ of sensor data –  What is „interesting“?

In recurrent data, this means unusual data compared to the rest –  E.g. participant hesitates or gets stuck



Global Analysis

•  Estimation of motion axis (providing the dominant signal) •  Comparison of the empirical distribution

to ideal distribution function („gold standard“) •  Usage of normalized and un-normalized functions to determine

smoothness and utilization of board‘s degrees of freedom •  Transformation into a performance quality metric between 0 and 1


��

��

��

��

��

��

��

��

��

��

��! � "��!�

��#

��

��#

��

�� $�

��

%&��'��


Local Analysis

•  Assumption: sensor data of a movement shares (unknown) statistical properties

•  Unusual portions of a sequence violate this assumption and can be identified •  Extension to PCA: Principal Component Breakdown Analysis •  PCA model is learned from local neighborhood

(using sliding window technique) •  Frames are projected to lower-dimensional subspace using PCA •  Reconstruction errors used as a measure for motion quality •  Problem: ideal window size not known •  Solution: multi-scale comparison (iteratively growing window)



User Feedback

•  Visual feedback –  PCA-based assessment diagram

(red and yellow parts contain irregularities)

•  Textual feedback –  Based on global and local metrics –  Rule-based combination of aspects trigger

textual cues

•  “Thumb” feedback –  Overall assessment at a glance


Try to be more continuous in your motion!


Outline

•  Motivation


•  Case Study

•  Conclusion



Case Study

•  Set of 20 exercises developed by sports medicine specialist

•  6 participants, 5 days of training (20 different exercises twice a day)

•  1200 exercise records •  Goal 1: Collection of training data

–  Identification of criteria and assessment by physician

•  Goal 2: Qualitative evaluation of prototype –  Questionnaire study



Case Study: Trial Assessment Example 1


PCBA: Continuity

Time [s]5 10 15 20

!2 !1 0 1 20

0.05

0.1

0.15

0.2

displacement [std]

frequency

General motion

observedideal

!max 0 +max0

0.05

0.1

0.15

0.2

0.25Angle usage

displacement [°]

frequency

Try to be more continuous in your motion!

You touched the ground 3 times.

Your movement is not ideal.

! Move back and forth in a continuous motion.

Try to move similarly to both sides of the board.

! You do not utilise the full range of angles!

! You lean towards the front!


Case Study: Trial Assessment Example 2


PCBA: Continuity

Time [s]5 10 15

!2 !1 0 1 20

0.05

0.1

0.15

0.2

displacement [std]

frequency

General motion

observedideal

!max 0 +max0

0.05

0.1

0.15

0.2Angle usage

displacement [°]

frequency

Your movement is continuous, nice!

You did not touch the ground!

Overall you perform the movement nicely!

Try to move similarly to both sides of the board.

! You do not utilise the full range of angles!

! You lean towards the front!


Qualitative Feedback


Fully agree

Not agree at all


Qualitative Feedback

•  Feature wishlist


Fully agree

Not agree at all


Outline

•  Motivation


•  Case Study

•  Conclusion



Summary

•  Mobile skill assessment of overall exercise quality •  Identification of typical exercising errors •  “Personal trainer” •  Participant feedback indicates potential for long-term exercising motivation

Further work: •  Long-term study on training progress •  Assessment and feedback entirely on mobile phone •  Generalization of assessment model



GymSkill in Google Play



Thank you for your attention! Questions?

? ? 04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 23

[email protected] www.vmi.ei.tum.de/team/andreas-moeller.html


Paper Reference

•  Please find the associated paper at: https://vmi.lmt.ei.tum.de/publications/2012/percom2012-preprint.pdf

•  Please cite this work as follows: •  Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr, Matthias

Kranz, Nils Hammerla, Patrick Olivier, Thomas Plötz GymSkill: A Personal Trainer for Physical Exercises In: 2012 IEEE International Conference on Pervasive Computing and Communications (PerCom2012), Lugano, Switzerland, March 2012, pp. 213-220



If you use BibTex, please use the following entry to cite this work:


@INPROCEEDINGS{6199869, author={M\"{o}ller, Andreas and Roalter, Luis and Diewald, Stefan and Scherr, Johannes and Kranz, Matthias and Hammerla, Nils and Olivier, Patrick and Pl\"{o}tz, Thomas}, booktitle={Pervasive Computing and Communications (PerCom), 2012 IEEE International Conference on}, title={GymSkill: A personal trainer for physical exercises}, year={2012}, month={march}, volume={}, number={}, pages={213 -220}, doi={10.1109/PerCom.2012.6199869}, ISSN={},}


Image Sources

•  Slide 5 –  gpsreview.net –  amazon.com –  spieleradar.de –  golem.de


gymskill - a personal trainer for physical exercises

Health & Medicine