03/28/03© 2005 university of wisconsin npr today “comprehensible rendering of 3-d shapes”,...

03/28/03 © 2005 University of Wisconsin

NPR Today

• “Comprehensible Rendering of 3-D Shapes”, Takafumi Saito and Tokiichiro Takahashi, SIGGRAPH 1990

• “Painterly Rendering with Curved Brush Strokes of Multiple Sizes”, Aaron Hertzmann, SIGGRAPH 98

• “An Algorithm For Automatic Painterly Rendering Based On Local Source Image Approximation”, Michio Shiraishi and Yasushi Yamaguchi, NPAR 2000

03/15/03 © 2005 University of Wisconsin

Comprehensible Rendering of 3-D Shapes

Takafumi Saito and Tokiichiro Takahashi

SIGGRAPH 1990

03/15/03 © 2005 University of Wisconsin

Comprehensible Rendering

• Focus on conveying information, rather than realism

• Goal: A range of enhancements to “standard” shaded models– Edge enhancement, contour lines,

hatching, …

03/15/03 © 2005 University of Wisconsin

G-buffers

• Extend color and depth buffers with additional images– Objects identifiers

– Parameters

– World coordinates

– Normal vectors

– …

normals z & uv

03/15/03 © 2005 University of Wisconsin

Getting G-buffers

• Almost any algorithm can be modified to produce G-buffers– Ray-tracing – get info from hit points

– Hardware – use programmable shaders

• Common technique now, but done here first

• PBRT has almost all the required information available

03/15/03 © 2005 University of Wisconsin

Algorithms

• Shading

• Edges/Silhouettes– Find discontinuities in depth

• Hatching– Trace parameter lines

• Contours– Trace lines of constant depth

03/15/03 © 2005 University of Wisconsin

Finding Edges

• Use edge detection filters on the depth image– Look for places with rapid change in

depth

– Tuning the filters provides control of extracted edges

• Then just draw edges, or add back into result

03/15/03 © 2005 University of Wisconsin

Contours and Hatching

• Trace lines of constant z to get screen-space contours

• Or, use world coordinate image to get world-space contours

• Control spacing to get hatching

z=xy

03/15/03 © 2005 University of Wisconsin

Shading

• World coordinates and normals enable shading

• Or, use any procedural shading algorithm

• Note meta-information is required– e.g. Materials associated with polygon

IDs

Shading & contours

03/15/03 © 2005 University of Wisconsin

More Results

03/15/03 © 2005 University of Wisconsin

More Results

03/15/03 © 2005 University of Wisconsin

Limitations

• All results to image precision– May lose contour lines, or they may be hard to track

– Use floating point images to help resolve this

– Use layered images (multiple surfaces at each pixel)

• Not all algorithms are amenable to fast hardware implementation

03/15/03 © 2005 University of Wisconsin

Future Work

• The ideas from this paper re-emerged in many different algorithms– Generating an image of visible surface IDs was most common

(could be done on earliest hardware)

• Hardware fragment shaders have made much of this redundant– For example, can easily get contour images or false-color

03/15/03 © 2005 University of Wisconsin

© 2005 University of Wisconsin

Painterly Rendering with Curved Brush Strokes of Multiple Sizes

:Aaron Hertzmann(SIGGRAPH 98 paper)

Presented by: Arup Dutta

© 2005 University of Wisconsin

Painterly rendering

• Objective: Creating an image with a hand-painted appearance from a photograph

• Pictorially:

From this: Produce this:

© 2005 University of Wisconsin

Key Concept

• Painters use varying brush sizes to capture detail:

© 2005 University of Wisconsin

Algorithm equivalent

• Procedure:– Build painting one layer at a time, from biggest to smallest radius

brushes

– At each layer, add detail missing from previous layer

• Source image= Original photograph

• Reference image= Gaussian blurred image

• 3 functions cover this algorithm

© 2005 University of Wisconsin

function paint(sourceImage, R1…Rn)

© 2005 University of Wisconsin

procedure paintLayer(canvas, referenceImage, R)

• canvas is the image we are updating layer by layer. referenceImage is the Gaussian blurred form of source image

• Determine grid size for given radius

• For each pixel of referenceImage calculate area error for grid size. If it is above a certain value, find pixel within grid with largest difference between canvas and referenceImage– Apply makeStroke() algorithm for this point and store the stroke returned.

• After exiting above ‘for loop’, paint all strokes on the canvas.

Gaussian blurred reference image

Eucledean difference of r,g,b distances

© 2005 University of Wisconsin

function makeStroke(x0,y0,R,refImage)

• Generic procedure for placing stroke on canvas

• Places a circle of radius R at the location x0,y0

• Disadvantage: Circles very visible

© 2005 University of Wisconsin

function makeSplineStroke(x0,y0,R,refImage)

• makeSplineStroke(x0,y0,R,refImage): Makes image closer to what a typical painting would look like, by:– Placing long, curved brush strokes, thus eliminating a regular uniform

stroke appearance

– Placing many small brush strokes in high frequency information region, and placing large brush strokes over areas with little detail.

© 2005 University of Wisconsin

function makeSplineStroke(x0,y0,R,refImage)

• Brush strokes of constant color

• Stroke placement normal to image gradient

• Stroke stops when – color deviates from control point by more than specified threshold

OR

– predetermined stroke length is reached

• Stroke points are separated by brush radius distance R.

• E.g.

Image gradient

Stroke direction

© 2005 University of Wisconsin

function makeSplineStroke(x0,y0,R,refImage)

© 2005 University of Wisconsin

Examples

• Style can be changed by changing parameters of the algorithm such as brush radius, thresholds, adding random jitter etc.

© 2005 University of Wisconsin

Future Work

• No one ‘right’ algorithm. Future work should continue to extend strategies to rendering algorithms.

• Brush strokes could be further developed to convey physical as well as semantic elements

• Real time rendering where rendering style changes with mood or action

© 2005 University of Wisconsin

Questions???

Source: UW Madison- CS 559 Spring ’04 Painterly Rendering Assignment sample solution software

03/15/03 © 2005 University of Wisconsin

03/15/0303/15/03 © 2005 University of Wisconsin© 2005 University of Wisconsin

An Algorithm For Automatic Painterly RenderingBased On Local Source Image Approximation

Authors Michio Shiraishi Yasushi Yamaguchi

Goals:Goals:

Automatic Painterly RenderingAutomatic Painterly Rendering Synthesize an image with a handcrafted Synthesize an image with a handcrafted

look from a source image like a look from a source image like a photograph.photograph.

Intensive use of local region of source Intensive use of local region of source image.image.

High emphasis on color information.High emphasis on color information.

Previous Work:Previous Work:

Paul Haeberli developed interactive system where Paul Haeberli developed interactive system where users could place strokes on a canvas.users could place strokes on a canvas.

Barbara Meier introduced 3D model support, Barbara Meier introduced 3D model support, painting order is based on depth.painting order is based on depth.

Aaron Hertzmann used brush strokes represented Aaron Hertzmann used brush strokes represented by spline curves. Composite image with several by spline curves. Composite image with several layers of strokes.layers of strokes.

Process:Process:

Vary stroke size over the canvas. Details are Vary stroke size over the canvas. Details are represented by small strokes, flat areas with represented by small strokes, flat areas with larger strokes.larger strokes.

Determine location, orientation, width and length Determine location, orientation, width and length of each stroke to approximate the local region of of each stroke to approximate the local region of source image.source image.

Paint strokes on order of size.Paint strokes on order of size.

Color Difference Image:Color Difference Image:

One value set by user is One value set by user is ss which is the length and width which is the length and width of the local region.of the local region.

Grayscale image based on how far local region image is Grayscale image based on how far local region image is from stroke color.from stroke color.

Intensity is larger the closer the image color is to the Intensity is larger the closer the image color is to the stroke color.stroke color.

Stroke color is white in this example.

Color image Resulting Gray-scale image

Brush Stroke Properties:Brush Stroke Properties:

Color: Color: The color The color CC in RGB of stroke. in RGB of stroke.

Location: Location: The location of The location of thethe stroke center stroke center (x(xcc,,yycc).).

Orientation: Orientation: The angle between the longest The angle between the longest axis of stroke and x-axis of canvas.axis of stroke and x-axis of canvas.

Size: Size: The size of stroke, length The size of stroke, length ll and width and width ww..

How to find these values?:How to find these values?:

Image moments are used. Image moments are defined Image moments are used. Image moments are defined on a gray-scale image.on a gray-scale image.

Image moments of 0Image moments of 0thth, 1, 1stst, and 2, and 2ndnd degree are used. degree are used. Use these values to find the length, width, location, and Use these values to find the length, width, location, and

orientation of the stroke.orientation of the stroke.

The result is two images with the same image moments.

Gray-scale image Resulting Stroke

Stroke Painting ProcessStroke Painting Process

Scale texture image to fit length and width of Scale texture image to fit length and width of stroke.stroke.

Rotate scaled image by Rotate scaled image by θθ

Translate scaled and rotated image by Translate scaled and rotated image by (x(xcc,y,ycc).).

Paint the stroke with color Paint the stroke with color CC by alpha by alpha blending.blending.

Algorithm:Algorithm:

Stroke Area Stroke Area Estimation:Estimation: Calculate Calculate MM0000 for each for each

pixel with-in local pixel with-in local region region ss..

Algorithm:Algorithm:

Sampling Points Sampling Points Generation:Generation: Murray space filling Murray space filling

polygon to dither polygon to dither image.image.

For each pixel, 1/For each pixel, 1/MM0000 is is summed up until it summed up until it reaches a certain reaches a certain number.number.

Then that pixel is a Then that pixel is a sample pixel.sample pixel.

Algorithm:Algorithm:

For each sampling point, For each sampling point, calculate the color calculate the color difference image.difference image.

From color difference From color difference image, calculate the image, calculate the stroke values.stroke values.

Paint strokes on white Paint strokes on white canvas, largest ones first. canvas, largest ones first. Larger meaning greater Larger meaning greater area.area.

Results:Results:

2500 Strokes 5000 Strokes 7500 Strokes 10000 Strokes

Original s=25, 11048 Strokes s=15, 9673 Strokes

Future Work:Future Work:

Make the algorithm calculate the Make the algorithm calculate the ss value of the value of the local region size.local region size.

Calculate the minimum number of strokes to Calculate the minimum number of strokes to cover entire canvas.cover entire canvas.