Handrix: Animating the Human Hand

George ElKoura Karan SinghGeorge ElKoura Karan Singh

University of TorontoUniversity of Toronto

Motivation

How do we get from a piece of music:How do we get from a piece of music:

To finger motions that play the music:To finger motions that play the music:

Overview

• System Architecture OverviewSystem Architecture Overview

• Guitar AlgorithmGuitar Algorithm

• Anatomy of the HandAnatomy of the Hand

• Hand ModelHand Model

• ResultsResults

System Architecture

System Architecture

System Architecture

System Architecture

System Architecture

Related work

• Anatomy Anatomy

– HHäägerger-R-Ross and oss and SSchieberchieber,, 200 20000

• VisionVision

– Lee and Kunii, 1995Lee and Kunii, 1995

• Graphics Graphics

– Rijpkema and GRijpkema and Girardirard, 1991, 1991

• MusicMusic

– Quine, 1990Quine, 1990

A Bit of Anatomy

• Why are fingers interdependent?Why are fingers interdependent?

– ““One to many” muscle insertion sitesOne to many” muscle insertion sites

– Close proximity of tendonsClose proximity of tendons

– Neurological constraintsNeurological constraints

• A clear anatomical understanding is still A clear anatomical understanding is still being developedbeing developed

Hand Model

• Goal:Goal:

– Map partially specified hand positions to realistic hand posturesMap partially specified hand positions to realistic hand postures

• Approach:Approach:

– Motion Capture lots of hand positionsMotion Capture lots of hand positions

– Nearest Neighbors, RBFs to approximate hand-pose spaceNearest Neighbors, RBFs to approximate hand-pose space

                                        

                                      

Interdependence Assumptions• Assumptions to reduce number of parameters:Assumptions to reduce number of parameters:

1.1. Independence of the ThumbIndependence of the Thumb

2.2. Invariance of Hand DominanceInvariance of Hand Dominance

3.3. Independence of Adduction/AbductionIndependence of Adduction/Abduction

4.4. Irrelevance of FrequencyIrrelevance of Frequency

5.5. Locality: Posture of wrist and arm do not affect digit Locality: Posture of wrist and arm do not affect digit independence independence

• These assumptions reduce the problem to 12 DOFs.These assumptions reduce the problem to 12 DOFs.

Reference: Häger-Ross and Schieber, Journal of Neuroscience 20(22), 2000

© Babette Adrian

Interdependence Reconstruction

• Allow the animator/controller to partially specify Allow the animator/controller to partially specify a hand posture (a hand posture (00, , 11, … , … nn))

Interdependence Reconstruction

• Allow the animator/controller to supply weights Allow the animator/controller to supply weights for each joint indicating the importance of the for each joint indicating the importance of the supplied joint angle (supplied joint angle (00, , 11, …, , …, nn))

0 = 11

n = 00

…

Interdependence Reconstruction

• Correct the user-supplied hand posture using the Correct the user-supplied hand posture using the given weights and the database of physically given weights and the database of physically possible posturespossible postures

Reconstruction Method• The database is a collection of joint angles represented by 12D vectors: The database is a collection of joint angles represented by 12D vectors: II

• Compute the weighted distance by:Compute the weighted distance by:

• The The kk-Nearest Neighbors interpolation weights are computed by:-Nearest Neighbors interpolation weights are computed by:

Interdependence Results(No Reconstruction)

Interdependence Results(With Reconstruction)

Interdependence Results(Comparison)

The Guitar

6 strings numbered from 1 to 6 6 strings numbered from 1 to 6 (heavy to light)(heavy to light)

Frets delimit strings lengths Frets delimit strings lengths producing audibly different notesproducing audibly different notes

Frets are numbered in increasing Frets are numbered in increasing order from the nut (fret 0)order from the nut (fret 0)

Tablature

• Common guitar music notationCommon guitar music notation

• Tells musician what frets to holdTells musician what frets to hold

• Does not tell musician what fingers to useDoes not tell musician what fingers to use

• Simple ASCII, readily available (OLGA) Simple ASCII, readily available (OLGA)

• Adaptable to specifying multi reaching goals over Adaptable to specifying multi reaching goals over time time

Fretting Design Objectives

1. Economy of effort1. Economy of effort

2. Read-ahead control2. Read-ahead control

3. Maintain natural motion of fingers3. Maintain natural motion of fingers

4. Capture differences in finger dexterity and 4. Capture differences in finger dexterity and strengthsstrengths

5. Fingertips touch guitar at the correct fret5. Fingertips touch guitar at the correct fret

Reference: Hector Quine, Guitar Technique, Oxford University Press, 1990

Guitar Space

Cost Functions

• Assign a different cost function to each finger, wristAssign a different cost function to each finger, wrist

• Each cost function can be a sum of specific cost functions to Each cost function can be a sum of specific cost functions to separately handle dexterity, strength and so forthseparately handle dexterity, strength and so forth

• Overall cost is computed, and cheapest fingering winsOverall cost is computed, and cheapest fingering wins

Finger Resolution Algorithm

Finger Collisions

Finger collisions are handled in a simple wayFinger collisions are handled in a simple way

• When a collision is detected, the colliding finger is moved When a collision is detected, the colliding finger is moved backwards on the fretbackwards on the fret

Results – C-Major Scale

Results – Chromatic Scale (fails)

Results – Chromatic Scale (passes)

Results – Chord Transitions

Results – More Chords

Results – Bar Chords

Results – C-Major Scale (3 Fingers)

Results – Stairway to Heaven

Future Work

Apply to other areas:Apply to other areas:

Future Work

• Slides, bends, hammersSlides, bends, hammers

• Frequency of motionFrequency of motion

• Motion Dynamics (strumming hand)Motion Dynamics (strumming hand)

• Anatomical Hand ModelAnatomical Hand Model

• Extension to other areas (e.g. Extension to other areas (e.g. locomotion)locomotion)

Summary

System Architecture OverviewSystem Architecture Overview

Guitar AlgorithmGuitar Algorithm

Anatomy of the HandAnatomy of the Hand

Hand ModelHand Model

ResultsResults

For More Information Please Visit:For More Information Please Visit:

http://www.dgp.toronto.edu/~gelkoura/handrix/index.html

Acknowledgements

DGP Lab, University of TorontoDGP Lab, University of Toronto

Chris LandrethChris Landreth

Dave BaasDave Baas

Side Effects Software, Inc.Side Effects Software, Inc.

Alias SystemsAlias Systems

NSERCNSERC

For more information please visit:

http://www.dgp.toronto.edu/~gelkoura/handrix/index.html