ink line rendering for film production daniel teece walt disney feature animation...

Ink Line Rendering for Ink Line Rendering for Film ProductionFilm ProductionInk Line Rendering for Ink Line Rendering for Film ProductionFilm Production

Daniel TeeceWalt Disney Feature Animation

daniel.teece@disney.com

Daniel TeeceWalt Disney Feature Animation

daniel.teece@disney.com

IntroductionIntroductionIntroductionIntroduction

Coming up…

• Background

• Four routes to an ink line image

• Production examples

Coming up…

• Background

• Four routes to an ink line image

• Production examples

Ink Lines and NPRInk Lines and NPRInk Lines and NPRInk Lines and NPR

The bigger picture• “Natural Media Emulation”

• Several commercial ‘toon renderers’

• Research papers:– Winkenbach– Elber– Gooch & Gooch– Amongst others

• Books (Gooch & Gooch, Apodaca & Gritz)

The bigger picture• “Natural Media Emulation”

• Several commercial ‘toon renderers’

• Research papers:– Winkenbach– Elber– Gooch & Gooch– Amongst others

• Books (Gooch & Gooch, Apodaca & Gritz)

ApplicationsApplicationsApplicationsApplications

Why do we need to render lines? • Outlines convey information minimally

• Simplicity of a line, variety of styles

• For Disney - merging with traditionally animated artwork

• Various uses, various approaches

Why do we need to render lines? • Outlines convey information minimally

• Simplicity of a line, variety of styles

• For Disney - merging with traditionally animated artwork

• Various uses, various approaches

Surface ShadersSurface ShadersSurface ShadersSurface Shaders

The shortest route• Surface normal tested against view vector,

and dot product compared to threshold value

• No rendering code needed (just a shader)

• Easy integration with rendering pipeline

• Able to take advantage of standard features in scanline renderer

• Difficult to control line width

The shortest route• Surface normal tested against view vector,

and dot product compared to threshold value

• No rendering code needed (just a shader)

• Easy integration with rendering pipeline

• Able to take advantage of standard features in scanline renderer

• Difficult to control line width

Surface ShadersSurface ShadersSurface ShadersSurface Shaders

Shader example:

float angle = abs(normalize(N) . normalize(I));

float border = 1 - step (0.3, angle);

Ci = mix (Ci, color 0, border);

(from [Apodaca, Gritz 00])

Shader example:

float angle = abs(normalize(N) . normalize(I));

float border = 1 - step (0.3, angle);

Ci = mix (Ci, color 0, border);

(from [Apodaca, Gritz 00])

Image Buffers and Edge Image Buffers and Edge DetectionDetectionImage Buffers and Edge Image Buffers and Edge DetectionDetection

An established and proven approach • Sobel edge detection on reference images

• High resolution (often 4K or higher)

• Various render passes:– Z Depth (one float per pixel)– Surface Normal (one 3D vector per pixel) or N.I– Object / Surface IDs (one or more ints per pixel)

An established and proven approach • Sobel edge detection on reference images

• High resolution (often 4K or higher)

• Various render passes:– Z Depth (one float per pixel)– Surface Normal (one 3D vector per pixel) or N.I– Object / Surface IDs (one or more ints per pixel)

Image Buffers and Edge Image Buffers and Edge DetectionDetectionImage Buffers and Edge Image Buffers and Edge DetectionDetection

Strengths and Weaknesses • Proven approach, production tested

• A lot comes “for free” - e.g. intersection lines, visibility

• Slow due to image size / multiple passes

• Finding threshold values can be difficult - limited fine-grain control

Strengths and Weaknesses • Proven approach, production tested

• A lot comes “for free” - e.g. intersection lines, visibility

• Slow due to image size / multiple passes

• Finding threshold values can be difficult - limited fine-grain control

Image Buffers and Edge Image Buffers and Edge DetectionDetectionImage Buffers and Edge Image Buffers and Edge DetectionDetection

Technical illustration using edge detectionTechnical illustration using edge detection

Image Buffers and Edge Image Buffers and Edge DetectionDetectionImage Buffers and Edge Image Buffers and Edge DetectionDetection

Examples of production use• Hercules - the Hydra

• Mulan - the Hun charge

• The Iron Giant - the Giant

• Osmosis Jones - Drix

Examples of production use• Hercules - the Hydra

• Mulan - the Hun charge

• The Iron Giant - the Giant

• Osmosis Jones - Drix

Inverted Surface Inverted Surface Outline RenderingOutline RenderingInverted Surface Inverted Surface Outline RenderingOutline Rendering

A novel alternative • Standard back face culling creates outlines

from surfaces

• Each original surface has two surfaces, one flipped and slightly larger than original

• The offset between the surfaces is the line

A novel alternative • Standard back face culling creates outlines

from surfaces

• Each original surface has two surfaces, one flipped and slightly larger than original

• The offset between the surfaces is the line

Inverted Surface Inverted Surface Outline RenderingOutline RenderingInverted Surface Inverted Surface Outline RenderingOutline Rendering

A simple exampleA simple example

Inverted Surface Inverted Surface Outline RenderingOutline RenderingInverted Surface Inverted Surface Outline RenderingOutline Rendering

© Disney

© Disney

© DisneyImages courtesy of Hiroki Itokazu, George Taylor and Lance Williams

Inverted Surface Inverted Surface Outline RenderingOutline RenderingInverted Surface Inverted Surface Outline RenderingOutline Rendering

• Rendering can be completely off-the-shelf

• Focus shifts to modeling

• No line parameterization

• Surface intersections problematic

• Rendering can be completely off-the-shelf

• Focus shifts to modeling

• No line parameterization

• Surface intersections problematic

Geometry-based Ink Geometry-based Ink Line RenderingLine RenderingGeometry-based Ink Geometry-based Ink Line RenderingLine Rendering

Edge detection in object space• Lines typically generated from polygonal data

• Rendering is more easily separated from line extraction

• NURBS edge extraction has also been explored [Gooch98] [Elber,Cohen90]

• Memory use shifts from image buffers to geometrical data structures

Edge detection in object space• Lines typically generated from polygonal data

• Rendering is more easily separated from line extraction

• NURBS edge extraction has also been explored [Gooch98] [Elber,Cohen90]

• Memory use shifts from image buffers to geometrical data structures

Geometry-based Ink Geometry-based Ink Line RenderingLine RenderingGeometry-based Ink Geometry-based Ink Line RenderingLine Rendering

Silhouette detection refinements• Interpolation (edges across faces)

• Probabilistic Searching [Markosian et al 97]

• Edge buffer [Buchanan, Sousa 00]

• Other approaches:

– Dual Surface [Hertzmann, Zorin 00]

– Gauss Maps [Gooch et al 99]

Silhouette detection refinements• Interpolation (edges across faces)

• Probabilistic Searching [Markosian et al 97]

• Edge buffer [Buchanan, Sousa 00]

• Other approaches:

– Dual Surface [Hertzmann, Zorin 00]

– Gauss Maps [Gooch et al 99]

Geometry-based Ink Geometry-based Ink Line RenderingLine RenderingGeometry-based Ink Geometry-based Ink Line RenderingLine Rendering

Strengths and Weaknesses • Fast, but some computations are costly

• Vectorized lines can be used elsewhere

• Have to compute visibility

• Tessellation quality is critical

• Numerical precision issues

• Software development effort is larger

Strengths and Weaknesses • Fast, but some computations are costly

• Vectorized lines can be used elsewhere

• Have to compute visibility

• Tessellation quality is critical

• Numerical precision issues

• Software development effort is larger

Case Study : InkaCase Study : InkaCase Study : InkaCase Study : Inka

A hybrid ink line renderer• Primarily a geometry-based approach

• Lines derived from tessellated geometry (not from NURBS as in Gooch / Elber)

• High level of localized control over lines

• Part of digital production pipeline

• Used for approvals / roughs and final art

A hybrid ink line renderer• Primarily a geometry-based approach

• Lines derived from tessellated geometry (not from NURBS as in Gooch / Elber)

• High level of localized control over lines

• Part of digital production pipeline

• Used for approvals / roughs and final art

Inka - RequirementsInka - RequirementsInka - RequirementsInka - Requirements

Inka needed to support:• Large models

• Faster render times

• High quality lines

• Controllability for LookDev TDs

Inka needed to support:• Large models

• Faster render times

• High quality lines

• Controllability for LookDev TDs

© Disney

Camera

Attributes

Surface Tessellation

Scan Conversion

Line Segment Generation

Line Refinement

Visibility Determination

Attribute Parsing

Line DrawingOutput Image

Vector Lines

Geometry

Inka - ProcessInka - ProcessInka - ProcessInka - Process

# Set line width and colorwidth *object1* 1.5color *object2* 0.6 0.5 0.4 1.0

# Turn off certain linesnoLine *object1* vmin,vmaxnoLine *object2* trim

# Visibility directivesoutlineZMin *object1* 0.0002selfZBias *object2* 0.08

A text file providing ink line directives

Inka - AttributesInka - AttributesInka - AttributesInka - Attributes

Inka - TessellationInka - TessellationInka - TessellationInka - Tessellation

NURBS / SubDs to triangles• Can have significant impact on results

• View dependent / view independent

• Trims are supported

• Inventor format - pipeline standard

NURBS / SubDs to triangles• Can have significant impact on results

• View dependent / view independent

• Trims are supported

• Inventor format - pipeline standard

Line types:Line types:

SilhouettesSilhouettes

BoundariesBoundaries

TrimsTrims

Surface CurvesSurface Curves

Space CurvesSpace Curves

CreasesCreases

Intersections are Intersections are computed separatelycomputed separately

Inka - Line Segment Inka - Line Segment GenerationGenerationInka - Line Segment Inka - Line Segment GenerationGeneration

Inka - VisibilityInka - VisibilityInka - VisibilityInka - Visibility

High resolution Z-Buffer• Appel’s quantitative visibility [Appel67] was

initial approach

• Z-Buffer is higher resolution than image (typically 2x - 4x)

• Line edges compared with scan-converted surface depth values

• Numerical precision can be an issue

High resolution Z-Buffer• Appel’s quantitative visibility [Appel67] was

initial approach

• Z-Buffer is higher resolution than image (typically 2x - 4x)

• Line edges compared with scan-converted surface depth values

• Numerical precision can be an issue

zBias moves point zBias moves point towards viewertowards viewer

ambiguousambiguousintersectionsintersections

zBiaszBias

Inka - Visibility IssuesInka - Visibility IssuesInka - Visibility IssuesInka - Visibility Issues

• Merged lines are scan converted edge by

edge

• Tapering and other effects may be applied

• Alpha accumulation is controlled by attributes and pixel compositing modes

Inka - Line DrawingInka - Line DrawingInka - Line DrawingInka - Line Drawing

Inka - Line QualityInka - Line QualityInka - Line QualityInka - Line Quality

• Line shaders allow user to control variation

of color, width etc.

• Line tapering lends hand-drawn appearance, using screen-space algorithm

• Further extended by vector output - lines replaced by other drawing primitives (e.g. Sable stroke renderer)

Tapering linesTapering lines

Inka - Line QualityInka - Line QualityInka - Line QualityInka - Line Quality

© Disney

Vector output in interactive viewer

Inka - Vector ViewerInka - Vector ViewerInka - Vector ViewerInka - Vector Viewer

Inka - Extra FeaturesInka - Extra FeaturesInka - Extra FeaturesInka - Extra Features

• Simple paint layers

• Memory management (geometry groups)

• Animated attributes

• SWF export

• Line attenuation based on depth– Width and opacity can vary with distance

• Simple paint layers

• Memory management (geometry groups)

• Animated attributes

• SWF export

• Line attenuation based on depth– Width and opacity can vary with distance

Inka - SummaryInka - SummaryInka - SummaryInka - Summary

• Geometry-based

• Highly controllable

• Part of production pipeline

• Film appearances:– The Emperor’s New Groove (2000)– Atlantis - The Lost Empire (2001)– Lilo and Stitch (2002)– Treasure Planet (2002)

• Geometry-based

• Highly controllable

• Part of production pipeline

• Film appearances:– The Emperor’s New Groove (2000)– Atlantis - The Lost Empire (2001)– Lilo and Stitch (2002)– Treasure Planet (2002)

Inka - SummaryInka - SummaryInka - SummaryInka - Summary

Inka software developers : Yun-Chen Sung

Mike King

Patrick Dalton

Rasmus Tamstorf

Joe Lohmar

Ramon Montoya Vozmediano

Daniel Teece

Inka software developers : Yun-Chen Sung

Mike King

Patrick Dalton

Rasmus Tamstorf

Joe Lohmar

Ramon Montoya Vozmediano

Daniel Teece

ConclusionsConclusionsConclusionsConclusions

• Different approaches, each with merits and pitfalls

• Image quality over speed

• Controllability over interactivity

• Integration is important

• Different approaches, each with merits and pitfalls

• Image quality over speed

• Controllability over interactivity

• Integration is important

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements

• Hiroki Itokazu

• Tad Gielow

• Jack Brooks

• Hiroki Itokazu

• Tad Gielow

• Jack Brooks