NonPhotoRealism and NonPhotoRealism and an Evaluation of Rendering an Evaluation of Rendering

Lecture 4

Painterly Rendering: Painterly Rendering: Scene ReconstructionScene Reconstruction

Scene Data Capture (from ‘What Dreams May Come’): Scan in film stock (analog to digital). Soon to be outmoded.

Used orange balls to help with registration between frames.

Scene also filmed with Lidar (Laser Radar) scanning.

Computer vision used to reconstruct a 3D model.

Before VFX Post-Production After VFX Post-Production

Painterly Rendering: ProcessPainterly Rendering: Process

Combines object and image space algorithms.

Object Space: Particles on the object surface ensure consistency

between animation frames.

Image Space: Reference images of the scene are used to determine the

orientation, colour and size brush attributes.

Reference images of brush strokes are altered according to the brush attributes and composited onto the image.

Painterly Rendering: AlgorithmPainterly Rendering: Algorithm

Create particles to represent geometry

FOR each frame of animation

Create reference pictures using geometry, surface attributes and lighting

Sort particles by distance from viewpoint

FOR each particle (starting with furthest from viewpoint)

[1] Transform particles to screen space

[2] Determine brush stroke attributes from particles and/or reference pictures

[3] Randomly perturb attributes based on user selected attributes

[4] Composite brush stroke into paint buffer

ENDFOR

ENDFOR

Painterly Rendering: AlgorithmPainterly Rendering: Algorithm

Painterly Rendering: ConclusionsPainterly Rendering: Conclusions

Jittering: Some random variation in brush attributes is needed to ensure a

painterly appearance.

Problem: this damages temporal coherence.

Solution: associate a random seed with each particle.

Painterly rendering is an intensive process which requires considerable user intervention (parameter tweaking).

May need several passes to composite different effects: shadows, highlights, etc.

Tip of the Iceberg. There has been much research in this area recently, producing a variety of techniques in both painterly and line-art rendering.

Painterly Rendering: ExamplesPainterly Rendering: Examples

Altering brush stroke and painting parameters to create different effects: pastel, pointilist, marker-pen, woodcut.

Other NonPhotoRealistic StylesOther NonPhotoRealistic Styles

Pen-and-Ink Illustration:Difference Image Algorithm:

1. Draw a hatching stroke in the darkest part of a greyscale image.

2. Subtract a blurred version of the stroke from the image and repeat.

Cartoon Illustration: Place Graftal (procedural)

textures along silhouettes. Evoke the style of Dr. Suess. Effectively indicates complexity,

e.g. grass, fur, trees.

NonPhotoRealism: ReferencesNonPhotoRealism: References Survey Article:

Landsdown, J. and Schofield, S. “Expressive Rendering: A Review of Nonphotorealistic techniques”, IEEE Computer Graphics and Applications, Vol. 15, No. 3, 1994, pp. 29-37.

Painterly Rendering: Meier, B. “Painterly Rendering for Animation”, SIGGRAPH ’96, pp. 477-

484. Haeberly, P. “Paint by Numbers: Abstract image representations”,

SIGGRAPH ’90, pp. 207-214.

Line-Art Rendering (pen-and-ink lines and cross-hatching effects which emulate sketching): Wikenbach, G. and Salesin, D. “Computer Generated Pen-and-Ink

Illustration”, SIGGRAPH ’94, pp. 91-100.

Cartoon Illustration: Kowalski, M. et al. “Art-based Rendering of Fur, Grass and Trees”,

SIGGRAPH ’99, pp. 433-438.

Rendering in Computer GraphicsRendering in Computer Graphics

Principal Rendering Techniques:1. Polygon Scan Conversion

2. Ray Tracing (Basic, Distributed, Recursive)

Adjunct Rendering Techniques:1. Radiosity

2. Non-PhotoRealistic Rendering

3. Surface Detail

Evaluation:1. Realism

2. Speed

3. Memory Consumption

‘Real World’ choices

Realism: Direct IlluminationRealism: Direct Illumination

Light may reflect off many surfaces on its way from the light to the camera.

Standard ray tracing and polygon scan conversion can handle a single diffuse or specular bounce.

Direct Diffuse Reflection Assumes that the object

is a perfect Lambertian (diffuse) reflector.

Direct Specular Reflection Use Phong approximation to

true specular (highlight) reflection.

Realism: Indirect IlluminationRealism: Indirect Illumination

Distributed recursive ray tracing can handle multiple specular bounces. Recursion halts when a diffuse surface is encountered.

Radiosity can handle multiple diffuse bounces.

The general case: Some research into adapting and combining radiosity and ray tracing.

Results tend to be very inefficient.

[1] Diffuse Specular [2] SpecularSpecular [3] DiffuseDiffuse [4] SpecularDiffuse

Realism: EffectsRealism: Effects

1. Motion Blur: [RT] Temporally distributed ray tracing OR

[PSC] Blending between frames with Accumulation Buffer.

2. Refraction: [RT] Recursive ray tracing of refracted rays.

[PSC] Not supported but can do non-refractive transparency.

3. Depth of Field (focus): [RT] Distributed ray tracing over the camera lense OR

[PSC] Use z-values to affect focus

Other Effects: lense flare, antialiasing. Achievable by both RT and PSC.

Realism: Reflection in PSCRealism: Reflection in PSC

[RT] Recursive ray tracing of reflected rays.

[PSC] Environment Mapping Also known as reflection mapping.

Render the scene in all directions from a single viewpoint. Can render spherically or onto a box. Use these rendered images as a texture map.

Prone to distortion. A single central reflection point stands in for multiple reflection points on the object’s surface. Particularly poor for planar surfaces.

[PSC] Scene Duplication Create a darker duplicate of the scene behind the reflective object.

Only really works for reflections off a plane.

Realism: ShadowsRealism: Shadows

Shadows depend on visibility from the light source. Surfaces not visible are in shadow.

[RT] Basic ray tracing, with rays shot to light sources. [PSC] Methods:

1. Scan-line: intersect current scan-line with potential shadowing edges – very expensive.

2. Pre-processed surface detail: place surface detail polygons over surfaces visible from the light source. Done once per scene per light source.

3. Shadow volumes: The volume in shadow behind a polygon is stored. During rendering viewing rays are intersected with the shadow volumes.

4. Two-pass z-buffer: Combine z-buffers rendered from the light source (shadow mask) and viewpoint. Efficient but can have aliasing problems.

Rendering EvaluationRendering Evaluation

PSC is capable of emulating many of the effects but not illumination of RT.

PSC RT

Illumination:

diffuse specular

specular specular

diffuse diffuse (+ Radiosity) (+ Radiosity)

specular diffuse

Effects:

motion blur

refraction

reflection

shadows

Costs: ComputationCosts: Computation

Given:

Recursive and Distributed Ray Tracing: ray-object intersections

Polygon Scan Conversion: polygon scan-line renderings Seemingly independent of , but affects screen size of

polygons. As increases the size of polygons and their scan-line

rendering time tend to decrease.

)( resolutionscreen polygons ofnumber

vhrn

)(dnrOpixelper rays ed/recursivdistribute ofnumber d

)(nOr

n

r

Costs: MemoryCosts: Memory

Both algorithms require: screen buffer

(assuming 16 bits per colour component)

scene storage(assuming exclusively naïve triangle meshes )

Recursive and Distributed Ray Tracing: Negligible storage of bounding objects or cell occupancy

lists. Otherwise no extra costs.

Polygon Scan Conversion: Z-buffer:

(assuming 32 bit depth value)

rr 48bits 163

n n 288bits 32 coords 3 vertices3

rr 32bits 32

What is Actually Used?What is Actually Used?

Visualization: Clarity over realism = PSC Sometimes volumetric rendering is required.

Simulation, 3D Games: Speed over realism = PSC + (Sprites, Ray Traced backdrops,

Radiosity) Extensive pre-processing is acceptable.

Special Effects: Realism over speed BUT high resolution (6000x4000) = Mostly PSC

+ some RT for ‘beauty’ shots. Example: Toy Story (proprietary PSC required 1-3 hours per

frame). Not all SFX are computer generated. Use Models, mattes (hand-

painted backgrounds) and pyrotechnics.