npr - 2d to 3d, painting and rendering daniel teece walt disney feature animation...

NPR - 2D to 3D, painting NPR - 2D to 3D, painting and renderingand renderingNPR - 2D to 3D, painting NPR - 2D to 3D, painting and renderingand rendering

Daniel TeeceWalt Disney Feature Animation

[email protected]

Daniel TeeceWalt Disney Feature Animation

[email protected]

IntroductionIntroductionIntroductionIntroduction

Coming up…

• Background

• 2D and 3D brush models

• Animation problems and approaches

• Examples of NPR used in film production

• Sable

Coming up…

• Background

• 2D and 3D brush models

• Animation problems and approaches

• Examples of NPR used in film production

• Sable

NPR CharacteristicsNPR CharacteristicsNPR CharacteristicsNPR Characteristics

What defines an NPR system?• “Natural Media Emulation”

• 2D / 3D

• Interactivity / Artistic direction

• Brushes, strokes and marks

• Complications - visibility … animation!

What defines an NPR system?• “Natural Media Emulation”

• 2D / 3D

• Interactivity / Artistic direction

• Brushes, strokes and marks

• Complications - visibility … animation!

NPR Cross-sectionNPR Cross-sectionNPR Cross-sectionNPR Cross-section

3D2D

automatic

interactive

image processors

stroke renderers

2D paint systems

3D paint / renderers

NPR Cross-sectionNPR Cross-sectionNPR Cross-sectionNPR Cross-section

3D2D

automatic

interactive

Litwinowicz97

Hertzmann00

Hertzmann98 Meier96

Haeberli90 DeepCanvas

Teece98

Kalnins02

2D Strokes2D Strokes2D Strokes2D Strokes

2D paint strokes and reference images• Images in, images out

• Stamp-based, procedural or physically-based

• Haeberli’s “Paint by Numbers” [Haeberli90]

• Hertzmann’s subsequent improvements [Hertzmann98]

2D paint strokes and reference images• Images in, images out

• Stamp-based, procedural or physically-based

• Haeberli’s “Paint by Numbers” [Haeberli90]

• Hertzmann’s subsequent improvements [Hertzmann98]

Animating StrokesAnimating StrokesAnimating StrokesAnimating Strokes

Finding frame to frame coherence• Stroke placement methods cause temporal

artifacts

• Locking strokes to canvas improves coherence but can look strange

• Strokes should move smoothly, matching motion of subject

Finding frame to frame coherence• Stroke placement methods cause temporal

artifacts

• Locking strokes to canvas improves coherence but can look strange

• Strokes should move smoothly, matching motion of subject

Image Space Stroke Image Space Stroke AnimationAnimationImage Space Stroke Image Space Stroke AnimationAnimation

Optical flow techniques

• “Processing images and video for an impressionist effect” [Litwinowicz97]

• “Painterly rendering for video and interaction” [Hertzmann00]

• Strokes moved, added or removed based on frame-to-frame optical flow data

• Strokes retain two-dimensional nature

Optical flow techniques

• “Processing images and video for an impressionist effect” [Litwinowicz97]

• “Painterly rendering for video and interaction” [Hertzmann00]

• Strokes moved, added or removed based on frame-to-frame optical flow data

• Strokes retain two-dimensional nature

Brush Models - 3DBrush Models - 3DBrush Models - 3DBrush Models - 3D

Strokes and Surfaces

• Pixel (texture maps)

• Geometry (points & matrices)

• Procedure (paths & width vectors)

From [Katanics,Lappas99]

Strokes and Surfaces

• Pixel (texture maps)

• Geometry (points & matrices)

• Procedure (paths & width vectors)


Painterly Rendering for Animation [Meier96]Painterly Rendering for Animation [Meier96]

A 3D stroke renderer • Particles placed on geometry, based on

surface area

• Strokes placed at particle positions, with parameters driven by reference renders

• Depth sorting and layering for visibility

• Smooth stroke animation, with a painterly appearance

A 3D stroke renderer • Particles placed on geometry, based on

surface area

• Strokes placed at particle positions, with parameters driven by reference renders

• Depth sorting and layering for visibility

• Smooth stroke animation, with a painterly appearance

3D Expressive Paint3D Expressive Paint3D Expressive Paint3D Expressive Paint

A 3D interactive painting system • Strokes and marks in object space

• Developed for animating painterly imagery

• Fully interactive

• Two types of stroke - 3D geometry and screen aligned

A 3D interactive painting system • Strokes and marks in object space

• Developed for animating painterly imagery

• Fully interactive

• Two types of stroke - 3D geometry and screen aligned


3D Strokes are painted interactively by ray-casting in to the stencil scene

3D Strokes are painted interactively by ray-casting in to the stencil scene


• Paint order / depth order (“clump buffer”)

• Silhouettes - coherence & character [Kalnins02]

• Brush model - lighting, textures etc.

• Limited animation support (e.g. deforming surfaces)

• Border or transition strokes between surfaces (back to 2D/3D relationship)

• Paint order / depth order (“clump buffer”)

• Silhouettes - coherence & character [Kalnins02]

• Brush model - lighting, textures etc.

• Limited animation support (e.g. deforming surfaces)

• Border or transition strokes between surfaces (back to 2D/3D relationship)

DeepCanvasDeepCanvasDeepCanvasDeepCanvas

A 3D painting and rendering system • Painters interface uses Digpaint (in-house 2D

paint system)

• BG Painters use a set of reference layers prepared by TD (grey render, outlines …)

• Strokes are stored in stroke script files, and mapped on to 3D geometry

• Renderer ‘re-paints’ scene from new angle

A 3D painting and rendering system • Painters interface uses Digpaint (in-house 2D

paint system)

• BG Painters use a set of reference layers prepared by TD (grey render, outlines …)

• Strokes are stored in stroke script files, and mapped on to 3D geometry

• Renderer ‘re-paints’ scene from new angle


A 2D/3D painting and rendering system A 2D/3D painting and rendering system



Simplified stroke representation -

stroke spine and width vectors Simplified stroke representation -

stroke spine and width vectors


SableSableSableSable

A 2D/3D stroke renderer • Library of specifically scanned & processed

strokes

• Operates as a pipeline component with Inka

• Stroke paths can be any Inka line type

• Workflow designed to be driven by traditional animation

A 2D/3D stroke renderer • Library of specifically scanned & processed

strokes

• Operates as a pipeline component with Inka

• Stroke paths can be any Inka line type

• Workflow designed to be driven by traditional animation


Requirements: • Match look of original artwork (stylized, flat,

few strokes, black background)

• Retain performance seen in pencil animation

• Animate strokes smoothly & efficiently

Requirements: • Match look of original artwork (stylized, flat,

few strokes, black background)

• Retain performance seen in pencil animation

• Animate strokes smoothly & efficiently


First attempt - DeepCanvas brush• Utility to convert ink lines to stroke scripts

• All pre-written software (except converter)

• Problems achieving distinctive look

• DeepCanvas procedural brush sensitive to segment distribution

First attempt - DeepCanvas brush• Utility to convert ink lines to stroke scripts

• All pre-written software (except converter)

• Problems achieving distinctive look

• DeepCanvas procedural brush sensitive to segment distribution


Brush model • Texture-based, extracted from example set

• Library of textures and shapes

• Stroke geometry is grown along spine curve

• Sharp twists are problematic … splitting

Brush model • Texture-based, extracted from example set

• Library of textures and shapes

• Stroke geometry is grown along spine curve

• Sharp twists are problematic … splitting


Animation

Rigged curve set animated to match traditional animation

Animation

Rigged curve set animated to match traditional animation

© Disney


Summary • Animate strokes smoothly & efficiently

• Able to match appearance of strokes

• Integration with other tools in pipeline

• Animators able to work with a pencil

Summary • Animate strokes smoothly & efficiently

• Able to match appearance of strokes

• Integration with other tools in pipeline

• Animators able to work with a pencil

ConclusionsConclusionsConclusionsConclusions

• Combining animation and NPR can be approached with 2D or 3D methods

• With 2D, frame-to-frame coherence can be difficult to maintain

• With 3D, some operations may detract from the painterly look

• Brush models vary, but are important

• Interaction and workflow are also important

• Combining animation and NPR can be approached with 2D or 3D methods

• With 2D, frame-to-frame coherence can be difficult to maintain

• With 3D, some operations may detract from the painterly look

• Brush models vary, but are important

• Interaction and workflow are also important

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements

• Mike Gabriel

• George Katanics

• Jack Brooks

• Mike Gabriel

• George Katanics

• Jack Brooks

npr - 2d to 3d, painting and rendering daniel teece walt disney feature animation...

Documents

d strokes

d paint strokes

lappas99 strokes

strange strokes

animating strokes

d paint renderers

d paint systems

surface area strokes