Motion Capture in Motion Capture in 3D Animation3D Animation

Animation : MoviesAnimation : Movies Video GamesVideo Games Robot ControlRobot Control

Motion Capture as a Motion Capture as a ToolTool Motion capture (MOCAP) is an Motion capture (MOCAP) is an

effective 3D animation tool for effective 3D animation tool for realistically capturing human motion realistically capturing human motion

OutlineOutline

RotoscopingRotoscoping The MOCAP PipelineThe MOCAP Pipeline Limitations of MOCAPLimitations of MOCAP The Future of MOCAPThe Future of MOCAP

RotoscopingRotoscoping

Captured VideoCaptured Video

HistoryHistory

Johnston, et al., 1995Johnston, et al., 1995

RotoscopingRotoscopingHistoryHistory

TracingTracing

Johnston, et al., 1995Johnston, et al., 1995

RotoscopingRotoscoping

Post ProcessingPost Processing

Johnston, et al., 1995Johnston, et al., 1995

The MOCAP PipelineThe MOCAP Pipeline

CalibrationCalibration CaptureCapture 3D Position Reconstruction3D Position Reconstruction Fitting to the SkeletonFitting to the Skeleton Post Processing: IK Post Processing: IK Inverse KinematicsInverse Kinematics

CaptureCapture

Optical Magnetic Mechanical

Costa-Sousa, M., 2004Costa-Sousa, M., 2004

Infra Red LEDsInfra Red LEDs

Magnetic Transmitter

PotentiometersPotentiometers

MOCAP issuesMOCAP issues

Issues: Issues: data inaccuracy, Expensivedata inaccuracy, Expensive OpticalOptical: : $90’000 – $210’000$90’000 – $210’000 high res cameras, occlusion, no outdoors, high res cameras, occlusion, no outdoors,

more and more markers, camerasmore and more markers, cameras

MagneticMagnetic: : $100’000 – $120’000$100’000 – $120’000 heavier sensors, body wires, 10x accurate < heavier sensors, body wires, 10x accurate <

optical, smaller workspaces, sensitive to metal, optical, smaller workspaces, sensitive to metal, EMI, flooringEMI, flooring

MechanicalMechanical: $20’000 – $30’000: $20’000 – $30’000 US US limited movement, heavy and cumbersome,limited movement, heavy and cumbersome,

must know rigid body at design time must know rigid body at design time

MOCAP issuesMOCAP issues

• Resolution/range of motion• Calibration• Accuracy

– Marker movement– Capture rate

• Occlusion/correspondence

MOCAP issues: MOCAP issues: Marker Placement

• Location should move rigidly with limb

• Stay away from bulging muscles• Shoulders: skeletal motion not

closely tied to skin• Shoulders: not a rigid body chain

MOCAP issues: MOCAP issues: Marker Set

3D points -> joint anglesIt matters!So do assumptions about human kinematics

CalibrationCalibration

Image Image SpaceSpace

World World SpaceSpace

(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)

(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)

(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)

3D Position 3D Position Reconstruction Reconstruction (Utopia)(Utopia)

3D Position 3D Position Reconstruction Reconstruction (Reality)(Reality)

Multiple MarkersMultiple Markers

For small number of markers: SizeFor small number of markers: Size Occlusions are a problemOcclusions are a problem Multiple Hypothesis TrackingMultiple Hypothesis Tracking

Ringer, et al., 2002

C1

C2

Multiple MarkersMultiple Markers

Mechanical Optical HybridsMechanical Optical Hybrids

http://www.animazoo.com/services/gypsyHybrid.htmhttp://www.animazoo.com/services/gypsyHybrid.htm

Fitting to the SkeletonFitting to the Skeleton

Utopian approachUtopian approach– 10 – 20% length 10 – 20% length

changeschanges Markers on both Markers on both

sidessides Joint Joint

DisplacementDisplacement Use Rotation Use Rotation

Angles OnlyAngles Only

Post ProcessingPost Processing

Motion EditingMotion Editing– Cut, Copy, PasteCut, Copy, Paste

Motion WarpingMotion Warping– Speed up or Slow DownSpeed up or Slow Down– Rotate, Scale or TranslateRotate, Scale or Translate

Motion Signal ProcessingMotion Signal Processing– Smoother MotionsSmoother Motions

Limitations of MOCAPLimitations of MOCAP

Only realistic motion capturedOnly realistic motion captured– Cartoony Animations not possibleCartoony Animations not possible

WYSIWYG (what you see is what you get)WYSIWYG (what you see is what you get)– Can’t add more expressionCan’t add more expression– Continally need to recapture motionContinally need to recapture motion

Robertson, B., 2001

Eric Darnell, co-director of Antz

The main problem with motion capture associated with characters has to do with mass distribution, weight, and exaggeration. He says that it is impossible for a performer to produce the kind of motion exaggeration that a cartoon character needs, and the mass and weight of the performer almost never looks good when applied to a character of different proportions.

Richard Chuang, VP at PDI

The mapping of human motion to a character with non-human proportions doesn’t work, because the most important things you get out motion capture are the weight shifts and the subtleties and that balancing act of the human body. If the proportions change, you throw all that out the door, so you might as well animate it.

MOCAP issues: MOCAP issues: Marker Placement

• Location should move rigidly with limb

• Stay away from bulging muscles• Shoulders: skeletal motion not

closely tied to skin• Shoulders: not a rigid body chain

The Future of MOCAPThe Future of MOCAP

Markerless MOCAPMarkerless MOCAP Cheaper MOCAPCheaper MOCAP Retargeting and reuse of mocap dataRetargeting and reuse of mocap data

hollywoodjesus.com

Ngan, S., 2004Bregler et al., 2002