recruitment of small synergistic movement makes a good …

Recruitment of Small Synergistic Movement Makesa Good Pianist

Beth Jelfs1,2 Shengli Zhou1,2,3 Bernard K.Y. Wong1

Chung Tin2,4 Rosa H.M. Chan1,2

1Department of Electronic EngineeringCity University of Hong Kong

2Centre for Biosystems, Neuroscience &Nanotechnology

City University of Hong Kong

3School of AstronauticsNorthwestern Polytechnical University

4Department of Mechanical & BiomedicalEngineering

City University of Hong Kong

Jelfs, Zhou, Wong, Tin & Chan Recruitment of Small Synergistic Movement Makes a Good Pianist EMBC 2015 1 / 12

What Can We Learn From Pianists?

Interested in coordinationHow does practice manifest inexperienced pianists?

Are There Differences in How the Fingers Move ...Individually? - jointsTogether? - simultaneous movementIndependently? - sequential movement

Aim:Can we identify fundamental differences in movement of novice &experienced pianists

Subjects:5 novices - little or no experience5 experienced - either grade 8 (1-8) or concert level


What Can We Learn From Pianists?

Are There Differences in How the Fingers Move ...Individually? - jointsTogether? - simultaneous movementIndependently? - sequential movement

Aim:Can we identify fundamental differences in movement of novice &experienced pianists

Subjects:5 novices - little or no experience5 experienced - either grade 8 (1-8) or concert level


Movement Synergies

Fundamental patterns of movementServe as building blocksReduce complexity (degrees offreedom) to be controlledRequires few components to explainvariance in movement

KinematicsMovement recorded usingdata glove14 sensors

2 per finger (MCP, PIP)4 abduction

Rate of flexion analyzed


Music

Practical Exercises for Beginners, Op.599Carl Czerny

Taken from Czerny Op. 599 - designed to improve fingering ofnovice pianistsStatic hand using all 5 fingers


Music


For analysis:Select 3 consecutive notesCapture transitions before & after the key press of interest

Identify finger playing middle noteGroup all sets of 3 notes by the finger playing the middle note


Music


For analysis:Select 3 consecutive notesCapture transitions before & after the key press of interestIdentify finger playing middle note

Group all sets of 3 notes by the finger playing the middle note


Music


For analysis:Select 3 consecutive notesCapture transitions before & after the key press of interestIdentify finger playing middle noteGroup all sets of 3 notes by the finger playing the middle note


Synergy Extraction via Principal Component Analysis

Subject 1

Subject 10

Sequence1-3-1

Sequence2-3-1

MiddleFinger

θi(1,1)̇

θi(L,K)̇θi(L,1)̇

θi(1,K)̇

Trial i

Samples k

Joint

s l

θj(1,1)̇

θj(L,K)̇θj(L,1)̇

θj(1,K)̇

Trial j

Samples k

Joint

s l

Normalize�θ(l)

PCACovarianceCij

Extraction performed on all subjects simultaneously1 data set per fingerExtract time varying synergies with fixed weighting coefficients


Variance Accounted For

1 2 3 4 5 6 7 8 9 1030

40

50

60

70

80

90

100

Component Number

VA

F (

%)

ThumbIndexMiddleRingLittle

First 4 components account for more than 60% of the variance forall fingers

What about the differences?We can successfully identify fundamental patterns in the motion ofall subjectsBut what can we learn from the differences between experienced& novice pianists?


Variance Accounted For

1 2 3 4 5 6 7 8 9 1030

40

50

60

70

80

90

100

Component Number

VA

F (

%)

ThumbIndexMiddleRingLittle

What about the differences?We can successfully identify fundamental patterns in the motion ofall subjectsBut what can we learn from the differences between experienced& novice pianists?


Classification

EmptySubset

EMGMM

ImprovedResult?

1st UnselectedComponent

FinalSubset

SelectBest

Repeat EachUnselected Component

Add Componentto Subset

Yes

No

Classify the weightings of the PCsExpectation Maximization algorithm

Fit a 2 class Gaussian Mixture Model

Iterative forward selection of components


Classification AccuracyFinger Accuracy (%) No. Components VAF (%)Thumb 93.68 15/253 1.38

Index 94.81 14/212 0.11Middle 93.40 17/409 2.70Ring 96.52 18/115 5.97Little 98.29 17/175 0.96

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Var

ianc

e A

ccou

nted

For

(%

)

17

25

18

37

231 61 187 97 239 10648

118 253 200 89

15 of 253 componentsVAF = 1.38%


Classification AccuracyFinger Accuracy (%) No. Components VAF (%)Thumb 93.68 15/253 1.38Index 94.81 14/212 0.11

Middle 93.40 17/409 2.70Ring 96.52 18/115 5.97Little 98.29 17/175 0.96

0

0.01

0.02

0.03

0.04

0.05

0.06

Var

ianc

e A

ccou

nted

For

(%

)

211 210 135 195

96158

66

75

14398 107

87

55

160



Classification Accuracy

Finger Accuracy (%) No. Components VAF (%)Thumb 93.68 15/253 1.38Index 94.81 14/212 0.11Middle 93.40 17/409 2.70

Ring 96.52 18/115 5.97Little 98.29 17/175 0.96

0

0.5

1

1.5

2

Var

ianc

e A

ccou

nted

For

(%

)

70 8229

24

43 51 61 302 138 408

8

36042

229 375 84 215



Classification AccuracyFinger Accuracy (%) No. Components VAF (%)Thumb 93.68 15/253 1.38Index 94.81 14/212 0.11Middle 93.40 17/409 2.70Ring 96.52 18/115 5.97

Little 98.29 17/175 0.96

0

0.5

1

1.5

2

2.5

Var

ianc

e A

ccou

nted

For

(%

)

12

111109 83 10710523

104

17

10

27

9

1621

37 47 33 108



Classification AccuracyFinger Accuracy (%) No. Components VAF (%)Thumb 93.68 15/253 1.38Index 94.81 14/212 0.11Middle 93.40 17/409 2.70Ring 96.52 18/115 5.97Little 98.29 17/175 0.96

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Var

ianc

e A

ccou

nted

For

(%

)

164 143

22

53171 175

23

19

110 11947

152 63

37

77 145 166



Reconstructed Signals for Middle Finger

0 0.5 1

−0.010

0.010.02

ThumbP

IP J

oint

0 0.5 1

−0.010

0.010.02

Index

0 0.5 1

−0.010

0.010.02

Middle

0 0.5 1

−0.010

0.010.02

Ring

0 0.5 1

−0.010

0.010.02

Little

0 0.5 1

−0.010

0.010.02

MC

P J

oint

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

Abd

uctio

n

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

NovicesExperienced

Reconstructed signals using only the selected componentsThe movement relating to the greatest separability

Comparison:Focus on four long fingers as movement of thumb is different


Reconstructed Signals for Middle Finger

0 0.5 1

−0.010

0.010.02

ThumbP

IP J

oint

0 0.5 1

−0.010

0.010.02

Index

0 0.5 1

−0.010

0.010.02

Middle

0 0.5 1

−0.010

0.010.02

Ring

0 0.5 1

−0.010

0.010.02

Little

0 0.5 1

−0.010

0.010.02

MC

P J

oint

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

Abd

uctio

n

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

0 0.5 1

−0.010

0.010.02

NovicesExperienced

Comparison:Focus on four long fingers as movement of thumb is different


Variability in Reconstructed SignalsIndex Finger

0

0.005

0.01

0.015

Var

ianc

e

Thum

b M

CP

Thum

b IP

T/I Abd

Inde

x MCP

Inde

x PIP

I/M A

bd

Midd

le M

CP

Midd

le PIP

M/R

Abd

Ring M

CP

Ring P

IP

R/L A

bd

Little

MCP

Little

PIP

NovicesExperienced

Ring Finger

0

0.01

0.02

0.03

0.04

0.05

0.06

Var

ianc

e

Thum

b M

CP

Thum

b IP

T/I Abd

Inde

x MCP

Inde

x PIP

I/M A

bd

Midd

le M

CP

Midd

le PIP

M/R

Abd

Ring M

CP

Ring P

IP

R/L A

bd

Little

MCP

Little

PIP

NovicesExperienced

Middle Finger

0

0.002

0.004

0.006

0.008

0.01

0.012

Var

ianc

e

Thum

b M

CP

Thum

b IP

T/I Abd

Inde

x MCP

Inde

x PIP

I/M A

bd

Midd

le M

CP

Midd

le PIP

M/R

Abd

Ring M

CP

Ring P

IP

R/L A

bd

Little

MCP

Little

PIP

NovicesExperienced

Little Finger

0

0.005

0.01

0.015

0.02

0.025

0.03

Var

ianc

e

Thum

b M

CP

Thum

b IP

T/I Abd

Inde

x MCP

Inde

x PIP

I/M A

bd

Midd

le M

CP

Midd

le PIP

M/R

Abd

Ring M

CP

Ring P

IP

R/L A

bd

Little

MCP

Little

PIP

NovicesExperienced


In summary...

ConclusionsPatterns of movement common to all our subjectsDifferences between novice & experienced pianists in componentsaccounting for smaller varianceDistinctions between groups seem to be in how fingers interactwith each otherIs variance accounted for a sufficient measure for coordinatedmovement?


The End

Thank you for listening


Thumb Reconstruction

0 0.5 1

−0.010

0.010.020.03

Thumb

PIP

Joi

nt

0 0.5 1

−0.010

0.010.020.03

Index

0 0.5 1

−0.010

0.010.020.03

Middle

0 0.5 1

−0.010

0.010.020.03

Ring

0 0.5 1

−0.010

0.010.020.03

Little

0 0.5 1

−0.010

0.010.020.03

MC

P J

oint

0 0.5 1

−0.010

0.010.020.03

0 0.5 1

−0.010

0.010.020.03

0 0.5 1

−0.010

0.010.020.03

0 0.5 1

−0.010

0.010.020.03

0 0.5 1

−0.010

0.010.020.03

Abd

uctio

n

0 0.5 1

−0.010

0.010.020.03

0 0.5 1

−0.010

0.010.020.03

0 0.5 1

−0.010

0.010.020.03

NovicesExperienced


Index Finger Reconstruction

0 0.5 1

−0.05

0

0.05

Thumb

PIP

Joi

nt

0 0.5 1

−0.05

0

0.05

Index

0 0.5 1

−0.05

0

0.05

Middle

0 0.5 1

−0.05

0

0.05

Ring

0 0.5 1

−0.05

0

0.05

Little

0 0.5 1

−0.05

0

0.05

MC

P J

oint

0 0.5 1

−0.05

0

0.05

0 0.5 1

−0.05

0

0.05

0 0.5 1

−0.05

0

0.05

0 0.5 1

−0.05

0

0.05

0 0.5 1

−0.05

0

0.05

Abd

uctio

n

0 0.5 1

−0.05

0

0.05

0 0.5 1

−0.05

0

0.05

0 0.5 1

−0.05

0

0.05

NovicesExperienced


Ring Finger Reconstruction

0 0.5 1

−0.1

0

0.1

Thumb

PIP

Joi

nt

0 0.5 1

−0.1

0

0.1

Index

0 0.5 1

−0.1

0

0.1

Middle

0 0.5 1

−0.1

0

0.1

Ring

0 0.5 1

−0.1

0

0.1

Little

0 0.5 1

−0.1

0

0.1

MC

P J

oint

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

Abd

uctio

n

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

NovicesExperienced


Little Finger Reconstruction

0 0.5 1

−0.1

0

0.1

Thumb

PIP

Joi

nt

0 0.5 1

−0.1

0

0.1

Index

0 0.5 1

−0.1

0

0.1

Middle

0 0.5 1

−0.1

0

0.1

Ring

0 0.5 1

−0.1

0

0.1

Little

0 0.5 1

−0.1

0

0.1

MC

P J

oint

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

Abd

uctio

n

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

0 0.5 1

−0.1

0

0.1

NovicesExperienced


recruitment of small synergistic movement makes a good …

Documents