Motion Capture Technology

Overview:

Definition of Motion Capture History of Mocap Rotoscoping Types of Motion Capture Mechanical Optical Electromagnetic Marker Placement Production Pipeline Applications of MoCap Examples

What is Motion Capture?

Motion capture is the process of recording human movement through specialised camera’s, and mapping

them onto a character model.

Also known as “Mo-Cap, Real-Time Animation” or “Performance Capture”

The Motion Capture began in late 1800’s when Etienne Jules Marey and Eadweard Muybridge performed motion studies on various animals and humans

History of Motion Capture

Etienne Jules Marey’sMotion Capture Suit

Eadweard Muybridge

•Historically, Motion

Capture in animated

movies was created

using the extension of "ROTOSCOPING" technique.

Eadweard Muybridge (1830-1904)

“Father of the motion picture” several cameras – successive

pictures photographs of human and animal

motion zoopraxiscope (zoogyroscope,

zoetrope) – a device for playing still images in sequence

Rotoscope

Allow animators to trace cartoon character over

photographed frames of live performances

• invented by Max Fleischer in 1915

RotoscopeAllow animators to trace cartoon character over

photographed frames of live performances

• invented by Max Fleischer in 1915

• 2D manual motion capture

A horse animated by rotoscoping from Muybridge’s photos

Rotoscoping

“rotoscoping can be thought of as a primitive form or precursor to motion capture, where the motion is ‘captured’ painstakingly by hand.” - Sturman

“3D Rotoscoping”: measuring 3D positions, orientations,

velocities or accelerations automatically

Current motion capture systems

• Electromagnetic

• Electromechanical

• Fiber optic

• Optical

Current Motion Capture Technologies

This approach has a central magnet with several receivers attached to the actor’s body. The receivers capture and record the actors movements and save them to the computer. This motion capture method can be hindered by nearby metal objects if they are large enough and depending on the power of magnets being used. A suit with magnetic sensors

Electromagnetic systems

Electromagnetic sensors placed on joints of moving object

Measures orientation and position of sensor relative to electromagnetic field generated by transmitter

No occlusion, multiple subjects

Electromagnetic interference, small volume

Pros

• measure 3D positions and orientations

• no occlusion problems

• can capture multiple subjects simultaneously

Cons

• magnetic perturbations (metal)

• small capture volume

• cannot capture deformation (facial expression)

• hard to capture small bone movement (finger movement)

• not as accurate as optical MoCap systems

Electromagnetic MoCap

Electromechanical Motion Capture uses Potentiometers on the aforementioned plastic exoskeleton that an actor must “wear”, and then act out his or her movements. This is very accurate and transmits real-time data at a far greater range than any other technology.

A gypsy3 prosthetic suite

Pros

• Measure 3D orientations

• No occlusion problems

• Can capture multiple subjects simultaneously

• Large capture volume

• Portable and outdoors capture (e.g. skiing)

Cons

• Getting 3D position info is not easy

• Cannot capture facial expression

• Hard to capture small bone movement (finger motion)

• Not as accurate as optical MoCap system

Electromechanical Mocap

• Reflective markers

• Multiple cameras

digitize different views

of performance

• High speed capture

• High data accuracy

• Occlusion

• Position sensitive

cameras

• Limited volume

Optical capture advantages

freedom of movement high quality capture Can Capture Facial Expression high throughput

fast sampling (200 fps at a high resolution) can capture fast motions can have a large capture space can capture many markers

disadvantages occlusion, markers are can be hidden from the camera

additional performers will increase occlusion may be able to add redundant cameras

marker crossover, which marker are you looking at?

cost $$$ extensive post processing (the marker’s have to be located and identified)

Major Optical PlayersMotion Analysis

Films Lord of the Rings King Kong Matrix Final Fantasy

Games NBA Live 2004 Grand Theft Auto III Mortal Kombat 4 (Midway)

Films Polar Express Harry Potter and the

Prisoner of Azkaban The Hulk Spider Man

Games All-Star Baseball 2002 Buffy the Vampire Slayer Everquest II NHL 2K3 (Mocap by

Red Eye Studio)

Production Pipeline

Sensor Calibration

Subject Calibration

Record Movements

Compute the position and orientation of camera/transmitter

Place markers on body, compute skeleton structure(lengths of links)

Marker positions (and orientations) varying over time

Marker Data Cleanup

Convert to Joint Angles

Occlusion and correspondence problemsOcclusion and

correspondence problems

Mapping to characters

Direct Mapping or retargetting

Process of Motion Capture (1)

• Recording area is constructed (often blue screen or green screen area)

• Cameras are set up to record the actors movement

• Cameras map 360 degree image of the person

{

Process of Motion Capture (2)

• Suit fitted with nodes

• Camera follows nodes

• Triangulates information

• DATA is recorded and ready to be manipulated

{

Process of Motion Capture (3)

• DATA given to an animator

• Manipulation of digital skeleton

• Helps character creation

Applications

Entertainment

Medicine

Arts / Education

Science / Engineering

Entertainment: Live Action Films

Computer generated characters in live action films (e.g. Battle Droids and many others in Star Wars Prequels, Gollum in The Lord of the Rings, King Kong in King Kong , Davy Jones in Pirates of Caribbean)

Entertainment: 3D computer animations

Characters in computer animated files (e.g. Polar Express, Monster House)

Kochadaiyaan (India's first photorealistic performance capture film) //After all he is Rajnikanth!!

Entertainment: Video Games

Video games by Electronic Arts, Gremlin, id, RARE, Square, Konami, Namco, and others, (e.g. Enemy Territory, Devil May Cry,FIFA)

Medicine Medicine (e.g., gait analysis, rehabilitation)

Sports medicine (e.g. injury prevention, performance analyses, performance enhancement)

Gait Analysis Service

Science / Engineering

Engineering (e.g., Biped robot developments)

Ergonomic product design

Military (e.g., field exercises, virtual instructors, and role-playing games)

Any Questions?