Paint By Numbers• The goal of a visual artist (Hagen):

• Without modeling detail, painters use brush strokes to:– Represent objects– Direct attention

The goal of effective representational image making, whether you paint in oil or in numbers, is to select and manipulate visual information in order to direct the viewer’s attention and determine the viewer’s perception.

Basic Approach• Aim: To take a synthetic or natural scene and

convert it into an abstract representation• The paper describes an interactive tool

– A user drags the mouse over a source image– The image is point sampled, and the color drives a

brush stroke– Avoids explicit color selection common in paint

programs– Controlling the nature of the strokes controls the form

of the picture

Representation• A painting is an ordered list of brush strokes• A brush stroke is described by:

– Location– Color– Size– Direction– Shape

• The aim is to use the strokes to convey:– Surface color– Surface curvature– Focus– Edges

Controlling the Brush Strokes• Position comes from mouse with random noise

added• Color comes from source image, with random

noise added• Size comes from the speed of mouse motion or

explicit user selection• Stroke direction comes from mouse motion

direction or explicit user selection• Shape is chosen by the user (circles, rectangles,

lines, scatterings, user design)

Post Processing Operations

• All brush strokes can be stored and later manipulated

• Operations include:– Add noise to some component

– Simple animation by applying same stroke parameters to sequence of images, choosing color from each image

– Use gradient to set orientation

– Use second derivative to enhance edges

Some Other Things

• Use ray tracing to extract color from a 3d model• Apply simple stochastic gradient decent to

automatically place strokes– Fix the number of strokes

– Repeatedly: Perturb an attribute, of the result is better keep the change

• Using cone shaped brushes produces Voronoi diagrams (cute observation, completely unrelated to NPR)

Computer-Generated Pen-and-Ink Illustration

• Illustrations offer many benefits over traditional rendering:– Detail can be enhanced, or omitted– Vitality and style

• Applications:– Medical illustration, Workshop manuals, Design sketches

• Pen-and-Ink Features:– Easy and cheap to print, and goes well with text– Color and shading must be implied by stroke

combinations

Strokes

• A curve segment traced out by a pen

• Principles:– Too thin is washed out; too thick obscures detail– The pen may be turned or varied during the

stroke– Strokes should look natural - to add life.

Characteristics, such as width, should be varied– Wavy lines indicate immature ideas (sketches)

Tones and Texture

• Combinations of strokes combine to form:– Tone: the brightness of a point

– Texture: an indication of the surface properties

• Principles:– Roughly equal weight and spacing for tones

– Absolute values of tones is not really important, but differences in tone are very important

– Tones should sometimes be forced to disambiguate objects

More Tones and Textures

• Texture principles:– The character of strokes is important for texture

• Crisp straight lines for “glass”

• Horizontal surfaces with horizontal lines

• Absence of detail indicates glare

• Sketchy indicates old, crisp indicates new

– To lend economy to the image, texture should be indicated:

• Only draw some subset of strokes

• Vary the exact method across the illustration

Outlines

• Outlines are completely abstract - photographs don’t have them

• Not just for contours - may also indicate interior detail (veins in a leaf)

• Principles:– Outline supports textures– Thick lines for shadows, or less depth– Haloes where objects pass behind each other– Outlines where tone is omitted to convey shape– Indication is just as important as it is for texture

Input and Preprocessing

• A 3D model– Associate texture with each face– Render with diffuse+specular to get reference tones– Indicate where detail should appear– Indicate where the light is

• Generate shadow polygons using a BSP tree shadow volume algorithm

• Generate a map of the screen, indicating what appears where, and the relative depths

Laying Down Tone and Texture

• Fill each region with “stroke texture”:– Set of strokes with priorities as to which should

be rendered first– Scale independent - appropriate strokes get

drawn depending on resolution and size– User indicated detail area drive placement

• Field describes how much texture should be drawn• Field drops off away from detail edges

– Strokes are clipped to regions, with some noise

Laying Down Outlines

• Outline associated with each texture• If shading indicates an edge, do not drawn outline• Where surfaces meet, use closest texture as outline• Thick or thin lines to indicate shadow and relation

to the light sources• Dependence on viewing direction:

– View direction affects which strokes are drawn

Results

Still There to Solve:

• Automatic emphasis control

• Rendering other types of scenery (such as trees, to be presented later)

• Animating pen-and-ink– Coherence– Adding and dropping strokes as priorities

change

Automatic Technical Illustration

• Concentrates on shading, assuming edge lines are desirable and will be added

• Key techniques:– Cool-to-warm shading

– Metal shading

The underlying assumption in NPR is that artistic techniques developed by human artists have intrinsic merit based on the evolutionary nature of art.

Traditional Shading

• Assume highlights and edge lines are also added• In shadow areas, ambient term dominates:

– Ambient must be high to contrast edge lines– No detail in shadow areas

• In areas facing light source, brightest diffuse color dominates– No contrast with highlights– Have to reduce kd

• Detail is lost!

)ˆˆ,0max( nl dda kkkI

Tone-based shading

• “Tones” come from adding gray to a color– Shifts hue, but leaves luminance roughly intact

• “Temperature” of a color is:– High if color is close to red– Low if color is close to blue– Hot colors appear to be forward, while cold colors

appear to be back

• Algorithm:– Replace cosine term in regular model with a new term

that blends between hot and cold

New Model

• Regardless of base color, resulting tones will be cold or warm

• Then use:

• Variation even in shadow regions• Robust to parameter settings

dyellowwarm

dbluecool

kkk

kkk

Define

warmcool kkI

2

ˆˆ11

2

ˆˆ1 nlnl

Metal Shading

• Traditional methods indicate metal by alternating dark and light bands– Bands are aligned with direction of minimum

curvature (so along the axis of a cylinder)– Derived from anisotropic reflection of milled

objects

• Modeled by adding 20 stripes around the object, and a bright stripe closest to the light source

Real-Time Approximations• Aim: To approximate the cool-to-warm shading

model with standard rendering hardware• Use two lights to get cool-to-warm

– Warm light from one direction, cool light from opposite direction

– Exploits negative intensity light sources

• Stencil buffers or software to get contours and silhouettes

• Texture or environment maps to get metal shading• Rotate the object, keeping the viewer and light fixed

More Open Problems

• They do not exploit cool-to-warm to help with depth perception. Can that be done?

• Related to another problem: Contrast maintenance/enhancement in tone reproduction algorithms

• Basic idea: Adjust the shading to enhance local features, while maintaining some global consistency