Image Space Rendering and Texturing

©Celine Loscos 2004 Anthony Steed 1999

Introduction

• We are interested in the process of displaying the scene in the drawing window

• In other words, we are interested in choosing a colour for each pixel depending on how each polygon is viewed from the viewpoint

Introduction

• One way is with ray tracing: – trace a ray from each pixel, and intersect it with the

polygons of the scene– Problem: This is expensive

• We are looking for methods that project the polygons on the window, and then decisions on the pixel colour are made in the image space

Overview

• Coping with depth• Gouraud shading• Phong shading• Texturing

Coping with depth

• When all polygons are projected on the screen, they may overlap

• The problem is to know which one is in front of the others• There are several methods to treat this

– Scan-line depth buffering– Z-buffer– Recursive sub-division– Trade-offs

• Z stands for the depth

Scan-line depth buffering: Extending the AET (active edge table)

• Easy enough if polygon do NOT intersect• Put all polygon edges into ET with extra depth

information and proceed as before except …• … now consider overlapping ranges of edges

Representation of the edges

• Each edge is represented by four elements

• pt is a pointer to information on the polygon (plane equation, shading, …)

),,,( 12 ptdydxxy

AET Example

y=iBandC,andC isinfront ofB.Showing

x1 x2 x3 x4 x5 x6

AB

C

Polygons A,B,C are such that A is in front ofB which is in front of C.

AET Example Notes

• Our edge table must contain data which enables us to look up z at at intersection point

• From x1 to x2 only A is considered. • At x2 A and B are considered

– plane equations are solved to get depth at (x2, i)– A is closest so x2 to x3 is filled as A

• At x3 A finishes so we draw B from x3 to x4• At x4 B and C are considered, B in front etc…

Method analysis

• Drawbacks– Sorting is required into the AET– If the number of polygons is high, the z-computation will be costly

• Accelerations• Would usually store the scanning in a 1D BSP tree for large numbers

of polygons!• Exploit coherence (assume similar overlapping at y+1)• Pre-order the polygons (no need to compute the depth

calculations)

Zbuffer

• Don’t bother with per span tests - just test every pixel• Most common usage is a full window sized array ZBUF

(M*N) of 16, 24 or 32 bit “depth” values• Basic idea:

– Initialise Zbuffer to Z_MAX– For each polygon

• Point (x,y,z) of the polygon projects on pixel (xs,ys) and has colour col associated

• If z < ZBUF[x,y] set CBUF[x,y] = colelse do nothing

Use of the polygon scan-line renderer

• We can do this in several ways• 1D z-buffer re-used on each scan line

– Process each polygon with separate AET– Or use as adjunct to extended AET for multiple

polygons• Problems …

– Aliasing on depth (z-buffer tearing)

Scanning Depth into the Zbuffer

• Now we have to write a z-value for each point– directly from plane equation (re-calculate for each point)– incremental across the scan-line (store z_start and dz)– Interpolate

• We see this one in more detail

Interpolating Depth

• Interpolate z along edges AND interpolate between edges on each scan-line (bi-linear interpolation)

(X1,Y1,Z1)

(X2,Y2,Z2)

(XL,YL,ZL) (XR,YR,ZR)

lr

lr

xxzzdz

Zbuffer Fill Example

0 1 2 3 4 5 6 7 8012345

6

a

b

c7• General form of ET

– (y2,x1,dx/dy,z1,dz/dy)

• ET[1] =– ac (7,3,1/6,1,3/6)– ab (4,3,4/3,1,1/3)

• ET[4] =– cb (7,7,-3/3,2,2/3)

a=(3,1,1) b=(7,4,2) c=(4,7,4)

…Contents of AET

• Scanline y=1– ac (7,3,1/6,1,3/6)– ab (4,3,4/3,1,1/3)– zspans 1 to 1

• y=2– ac (7,3.166,1/6,1.5,3/6) – ab (4,4.333,4/3, 1.333, 1/3) – zspans 1.5 to 1.333

• y=3– ac (7,3.333,1/6,2.0,3/6)– ab (4,5.666, 4/3, 1.666, 1/3)– zspans 2 to 1.6660 1 2 3 4 5 6 7 8

012345

6

a

b

c7

Recursive Subdivision Visibility

• Init: Clipping rectangle (CR) = window• The CR is covered by a polygon that is in front of all others

(draw it)• The CR does not intersect a polygon (draw nothing) • The CR overlaps a single polygon (draw it clipped)• The CR contains a single polygon (draw it unclipped)• ELSE quarter CR and repeat

Trade-Offs

• Zbuffer can be inaccurate with few bits– really simple to implement though!

• Scan-line AET good for large polygons– good coherency across lines– requires non-intersecting polygons

• zbuffer good for small, sparse polygons– AET more time consuming to maintain

Choosing the colour

• Gouraud shading• Phong shading• Texture mapping

Gouraud Shading

• Recall simple model for local diffuse reflection

• Gouraud interpolates this colour down edges and across scan-lines in the same manner as we just did for depth

N

i ipidaa lnIkIkI1

Gouraud Details

• ET now contains– (y2,x1,dx,z1,dz,r1,dr,g1,dg,b1,db)

• (we are running out of registers!)

• Problems– not constant colour on rotation of points– misses specular highlights

12

12

xxrrdr

Phong Shading

• Include specular component• Interpolate normals across the scan-line instead of

colours• Recaptures highlights in the centre of polygons

N

i sm

idipiaa knhklnIIkI1

Texture mapping

• We have seen that a colour can be assigned to a polygon, or to each vertex

• Now if we consider small details, we may not want to add polygons to represent every detail

• We prefer to keep a large polygon and use an image to represent the details

Example

Texture Mapping

• Why?– Approximation for surface colouring– Efficient packing of flat detail

• Standard texture mapping modifies diffuse mapping– Pasting a picture onto the polygon

• A texture is a 2D array of texels storing RGB or RGBA components

Difference between pixels and texels

• There can be a different match between the pixels of the display window and the texel of the texture

Overview

• Texture mapping– Inverse and Forward Mapping– Bilinear interpolation– Perspective correction

• Mipmapping• Other forms of mapping

– Environment– Bump mapping

Inverse Mapping

• Each vertex is associated with a point on an image (u,v)

Forward Mapping

• For points in the image, map onto the polygon– much harder to implement correctly, and harder to

model• Inverse mapping is much more commonly used

– Most 3D modelers output U,V co-ordinates for texture application

Quick and Dirty Solution

• Yet more parameters in the ET!• Bilinear interpolation of u&v down polygon edges,

and across scan-lines

• Works, but is very ugly– Does not consider fore-shortening of image in depth

The Problem

• Same problem exists with Phong and Gouraud shading, but it MUCH less noticeable

z correct z incorrect

The Solution

• Need to compensate for fore-shortening by interpolating over 1/z

• Interpolate (u’,v’,q) where q=1/z– At vertices we know (u1,v1,z1), (u2,v2,z2)

– Interpolate between (u1 / z1,v1 / z1,1/z1) and (u2/z2,v2 /

z2,1/z2) to get (u’,v’,q)– Restore u,v, by dividing u’ and v’ by q

Sanity Check

p0=(10,5,10,0,0)

pi=(xi,yi,zi,ui,vi)

p1=(15,15,20,0,1)scanlines

Y=5

Y=10

Y=15

• At p0 (u’,v’,q) = (0,0,0.1)

• At p1 (u’,v’,q) = (0,0.05,0.05)

• On scanline 10 (u’,v’,q) = (0,0.025,0.075) (u,v) = (0,0.333)

z = 1/0.075 = 13.33

Minor Issues

• Now have two divisions per pixel!• Some optimisations

– only do the division at end of the spans and interpolate across spans

– or only do the division every n pixels• Remaining problem

– we have not touched upon how to clip u,v values in 3D or 2D!

Major Issues

• Picking your pixel!

image

Sampling

• A pixel maps to a non-rectangular region• Usually only perform map on centre of pixel• Still have a problem of over-sampling (same texel

maps to several pixels) or under-sampling the image (pixels only sparsely sample the texels)

Filtering

• Nearest neighbour

• Pick pixel with closest centre

• Bilinear

• Weighted average based on distance to pixel centre

Filtering

• Bilinear filtering solves (partially) the oversampling problem since it provides smooth shading between pixels

scanline

Mip-Mapping

• When undersampling we use mippmapping• Resample image at lower resolution• Create a “pyramid” of textures.• Interpolate texture between two adjacent layers

Texture Pyramid

128x12864x64

32x321x1...

Sampling

• Choose two layers based on texel span– Choice is made selecting the two levels where the dv

and dv for dx and dy are closest to one• Interpolate between four pixels in higher layer and

one in lower layer

Environment Mapping

• Don’t modulate diffuse colour modulate colour based on normal

• Effectively creates a reflective object

• The environment map is a surrounding texture

Bump Maps

• Use a map to perturb the normal for specular lighting