diy motion capture with kinecttopin
DESCRIPTION
Slides from the 12/4 Volumetric Society presentation at PrattTRANSCRIPT
DIY Motion Capture12/5 Volumetric Society of NYC
How does the Kinect work?
Photo by y3rdua
Getting Started: Hardware
You will need:• A Kinect, or generic equivalent such as the Asus Xtion• USB adapter cable, if you have a Kinect that was
bundled with the Xbox (don’t buy the Microsoft adapter, the cheap ones work fine)• Enough space to perform
Getting Started: Software
OpenNI driver bundle:simple-openni.googlecode.com
KinectToPin:kinecttopin.fox-gieg.com
KinectToPinKinect Motion Capture for After Effects
What is KinectToPin?
A standalone motion capture application A UI panel for After Effects
+
What can you do with it?
Rig 2D character models in 3D
Setting up your space to record
6 – 10 feet
Capturing
• Record 15-point 3D skeletal tracking data• Capture in sync to dialogue• Remote control with a
standard presentation remote• Open source and cross-
platform: runs on OpenNI, built in Processing, also works with generic sensors like the Asus Xtion
Acting for Kinect
• The Kinect can’t record what it can’t see. • Keep your entire body in the camera’s range• Face the camera when capturing• Watch out for occluded joints (sitting down, putting your
hands behind your back etc. can cause it to lose the track)
• It’s surprisingly good at picking up subtle posture shifts, but not smaller motions like laughter. • Use a different performer for each character if possible
Importing tracking dataNative import or copy & paste
Getting data into After Effects
Data format options:• 3D Point Controls (CS5.5+)• 2D Point Controls (CS4+)• Puppet pins• Raw XML and JSON
UI Panel
• Create character templates• 2D and 3D setups• Native XML import
(SLOW)• Automatically add
expressions to smooth tracking data• Automate rigging
Building a CharacterRigging with the UI panel
The Puppet ToolTHREE OR MORE PINS ON ONE LAYER: Elbows don’t bend this way. Ow.
TWO PINS EACH, MULTIPLE LAYERS:Rotation with stretch and squash!
AE Puppet best practices
• Work BIG. Start high-res, scale down in the puppet comp, then scale back up in the project comp (enable Collapse Transformations).• Precompose if you’re using vectors or shape layers• Keep multiple angles of the same character in syncHigh-res Character
Layers
Rigging Template
Project Comp
End big.Start big.
Retargeting for arbitrary character shapes
Quick auto-lip sync
1. Convert audio amplitude to keyframes2. Connect to mouth animation3. Size the audio data to fit. Use this expression:
linear(source,sourceMin,sourceMax,targetMin,targetMax)
Beyond After EffectsMotion capture for live production
Embrace the limitations of the
format.
One last thought:
Further non-K2P tools
FREEBrekel Kinect
Open-source capture for 3D animation (Windows-only)www.brekel.com
DuikInverse kinematics plugin for AEwww.duduf.net
COMMERCIALiPi Soft
Uses two Kinects to recordwww.ipisoft.com
NI MateKinect for OSC/MIDI/Blenderwww.ni-mate.com
Questions?
kinecttopin.fox-gieg.comDownload the software!
Contact UsNick Fox-Giegwww.fox-gieg.com@N1ckFG
Victoria Necewww.victorianece.com@FakeGreenDress