illumination and shading

CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics1

Illumination and Shading

LightsDiffuse and Specular IlluminationBasicEffectSetting and Animating LightsShaders and HLSLLambertian IlluminationPixel ShadersTextures

LightsDiffuse and Specular IlluminationBasicEffectSetting and Animating LightsShaders and HLSLLambertian IlluminationPixel ShadersTextures


The XNA graphics pipelineMultiply by

World Matrix

Multiply by View Matrix

Homogenize and Clip

Multiply byProjection Matrix

Compute Color

Shade

Effect

Vertex Shader

Pixel Shader

Vertex positionVertex normalTexture coordinateVertex colorother stuff sometimes…


Some Terms

Illumination – What gives a surface its color. This is what our effect will compute.

Material – Description of the surface. Will include one or more colors. These are parameters set in the effect.

Reflection – The reflection of light from a surface. This is what we will simulate to compute the illumination.

Shading – Setting the pixels to the illumination

We’ll often use reflection and illumination just about interchangeably.


Effects/Shaders


My First Effect: FirstEffect.fxfloat4x4 World;float4x4 View;float4x4 Projection;

struct VertexShaderInput{ float4 Position : POSITION0;};

struct VertexShaderOutput{ float4 Position : POSITION0;};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;

float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection);

return output;}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return float4(1, 0, 0, 1);}

technique FirstShader{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}

This is the default effect Visual Studio will create when you do New/Effect. It does nothing except set pixels to red.


What this does

Anywhere there is something to draw, it draws red

Always something to draw other than through the window.


How to use this… firstEffect = Content.Load<Effect>("FirstEffect");

foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (ModelMeshPart part in mesh.MeshParts) { part.Effect = firstEffect; } }

protected void DrawModel(GraphicsDevice graphics, Camera camera, Model model, Matrix world) { Matrix[] transforms = new Matrix[model.Bones.Count]; model.CopyAbsoluteBoneTransformsTo(transforms);

foreach (ModelMesh mesh in model.Meshes) { foreach (Effect effect in mesh.Effects) { effect.Parameters["World"].SetValue(transforms[mesh.ParentBone.Index] * world); effect.Parameters["View"].SetValue(camera.View); effect.Parameters["Projection"].SetValue(camera.Projection); } mesh.Draw(); } }

Loading/Installingin LoadContent()

Drawing


What does what…

float4x4 World;float4x4 View;float4x4 Projection;

effect.Parameters["World"].SetValue(transforms[mesh.ParentBone.Index] * world);effect.Parameters["View"].SetValue(camera.View);effect.Parameters["Projection"].SetValue(camera.Projection);

Setting the effect parameter sets the equivalent value inside the effect.

Sets


The process

The Vertex Shader runs firstIt converts object vertex coordinates into projected coordinates.

The Pixel shader runs next. It computes the actual pixel

color


float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection);

return output;}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return float4(1, 0, 0, 1);}

Once per pixel

Once per vertex


Adding a Material Property

I added this line to the veretx shader

And changed the pixel shader to this

// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return float4(DiffuseColor, 1);}


Setting this Effect private void SetDiffuseColorEffect() { foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (ModelMeshPart part in mesh.MeshParts) { BasicEffect bEffect = part.Effect as BasicEffect; part.Effect = diffuseColorEffect.Clone(part.Effect.GraphicsDevice);

part.Effect.Parameters["DiffuseColor"].SetValue(bEffect.DiffuseColor); } } }

The default content processing supplied a BasicEffect object with the color set in it. We’re creating our own effect and setting the color to match what was loaded.

Note the “Clone”. This makes a copy of the effect. Since we’re putting a material property into it, we need a unique copy here.


What this does

Each pixel is simply set to the material diffuse color. No lighting is included, yet.


Now let’s do real lightingfloat4x4 World;float4x4 View;float4x4 Projection;


// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);


struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0;};


float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection); float3 color = LightAmbient * DiffuseColor; output.Color = float4(color, 1);

return output;}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color;}


I’ve moved our computation to the vertex shader (why?). Ambient illumination is simply the light ambient color times the surface diffuse color.


Some Things

float4x4 World;float4x4 View;float4x4 Projection;


// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);




float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection); float3 color = LightAmbient * DiffuseColor; output.Color = float4(color, 1);

return output;}

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color;}


Note that the VertexShaderOutput now has a color.


What this looks like

Inset has brightness artificially increased in Photoshop


Now for Diffuse Illumination

What we need to know Light location in space Light color

float3 Light1Location = float3(5, 221, -19);float3 Light1Color = float3(1, 1, 1);


HLSL code

// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);float3 Light1Location = float3(5, 221, -19);float3 Light1Color = float3(1, 1, 1);

struct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0;};


We need to know the normal


Vertex ShaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;

// We need the position and normal in world coordinates float4 position = mul(input.Position, World); float3 normal = normalize(mul(input.Normal, World)); // Ambient lighting hitting the location float3 color = LightAmbient; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add contribution due to this light color += saturate(dot(Light1Direction, normal)) * Light1Color; // Multiply by material color color *= DiffuseColor; output.Color = float4(color.x, color.y, color.z, 1);

float4 viewPosition = mul(position, View); output.Position = mul(viewPosition, Projection); return output;}

position

Light1Location

Ligh

t1Lo

catio

n - p

ositi

on

Ligh

t1Di

recti

on

normal


What this looks like

What is missing?


Texture Mapping

Mapping a picture onto the surface of a triangle.


HLSL – Adding a texture variable

// The texture we usetexture Texture;

Not everything in our scene is texture mapped. If it is texture mapped, we use the texture color as the diffuse color. If not, we use the set diffuse color. We have two different ways we will compute the color.

I’ll move the multiplication by the Diffuse Color to the pixel shader.

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1);;}


Slightly modified version, now.

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1);}


We need something called a “sampler”

sampler Sampler = sampler_state{ Texture = <Texture>;

MinFilter = LINEAR; MagFilter = LINEAR; AddressU = Wrap; AddressV = Wrap; AddressW = Wrap;};

This just parameterizes how we get pixels from our texture. Wrap means tiling will be used. LINEAR means linear interpolation. We’ll do some other options later.


And, we need to have the texture coordinatesstruct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0; float2 TexCoord : TEXCOORD0;};

struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0; float2 TexCoord : TEXCOORD0;};

output.TexCoord = input.TexCoord;

And add this to the vertex shader function:


TechniquesWe have two ways things will be treated. We could create two different effects. But, it’s easier to create one effect with two different “techniques”.

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor.x, DiffuseColor.y, DiffuseColor.z, 1);}

technique NoTexture{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}

Here is the technique we had before.


Techniques

float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ return input.Color * tex2D(Sampler, input.TexCoord);}

technique Textured{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderTexturedFunction(); }}

This is the other technique


float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor.x, DiffuseColor.y, DiffuseColor.z, 1);}




All in one place so you can see it.


Loading This Effect private void SetLambertianTextureEffect() { foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (ModelMeshPart part in mesh.MeshParts) { part.Effect = effectsOrig[part];

BasicEffect bEffect = part.Effect as BasicEffect; part.Effect = lambertianTextureEffect.Clone(part.Effect.GraphicsDevice);

part.Effect.Parameters["DiffuseColor"].SetValue(bEffect.DiffuseColor); part.Effect.Parameters["Texture"].SetValue(bEffect.Texture);

if (bEffect.Texture != null) { part.Effect.CurrentTechnique = part.Effect.Techniques["Textured"]; } else { part.Effect.CurrentTechnique = part.Effect.Techniques["NoTexture"]; } } } }


What we get


The complete effect – Variables and typesfloat4x4 World;float4x4 View;float4x4 Projection;


texture Texture;


struct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0; float2 TexCoord : TEXCOORD0;};

struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0; float2 TexCoord : TEXCOORD0;};

sampler Sampler = sampler_state{ Texture = <Texture>;

MinFilter = LINEAR; MagFilter = LINEAR; AddressU = Wrap; AddressV = Wrap; AddressW = Wrap;};


The complete effect – Vertex shaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.TexCoord = input.TexCoord;

// We need the position and normal in world coordinates float4 position = mul(input.Position, World); float3 normal = normalize(mul(input.Normal, World)); // Ambient lighting hitting the location float3 color = LightAmbient; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color;

output.Color = float4(color.x, color.y, color.z, 1);



The complete effect = Pixel Shader and Techniquesfloat4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor.x, DiffuseColor.y, DiffuseColor.z, 1);}





Specular Illumination

Specular ReflectionsReflections from the surface of the material

Generally a different color than the underlying material

diffuse color


More Specular Examples

Specular Reflection makes things appear shiny

The left Dalek has no specular illumination, the right does


Previous lighting components

AmbientIa=Ambient part of illuminationMd=Diffuse material colorCa=Light ambient color

Diffuse IlluminationId=Diffuse part of illuminationMd=Diffuse material colorCi=Color of light ILi=Vector pointing at light IN=Surface normal

ada CMI float3 color = LightAmbient;

return input.Color * float4(DiffuseColor, 1);

)0,max(1

m

iiidd NLCMI

color += max(dot(Light1Direction, normal), 0) * Light1Color;

return input.Color * float4(DiffuseColor, 1);


Specular Component

Specular ComponentIs=Specular part of illuminationMs=Specular material colorCi=Color of light IN=Surface normalH=“Half vector”n=Shininess or SpecularPower

Basic HLSL code

nm

iiss HNCMI )0,max(

1

VL

VLH

// Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color;


Specular Reflection Highlight Coefficient

• The term n is called the specular reflection highlight coefficient or “Shininess” or “Specular Power”

• This effects how large the spectral highlight is. A larger value makes the highlight smaller and sharper. – This is the “shininess” factor in OpenGL, SpecularPower in

XNA– Matte surfaces has smaller n. – Very shiny surfaces have large n. – A perfect mirror would have infinite n.


Shininess Examples

n=1 n=10 n=35

n=65 n=100

nm

iiss HNCMI )0,max(

1


HLSL// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);float3 SpecularColor = float3(0, 0, 0);float Shininess = 0;

texture Texture;


float3 Eye = float3(0, 0, 0);

struct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0; float2 TexCoord : TEXCOORD0;};

struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0; float4 SColor : COLOR1; float2 TexCoord : TEXCOORD0;};

New shader variables


Vertex ShaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.TexCoord = input.TexCoord;

float4 position = mul(input.Position, World); float3 normal = normalize(mul(input.Normal, World)); // Ambient lighting hitting the location float3 color = LightAmbient; float3 scolor = 0; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add diffuse contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color; // Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; output.Color = float4(color, 1); output.SColor = float4(scolor, 1);


Note that we send the specular illumination separate from the diffuse illumination. Any ideas why?


Pixel Shaders

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1) + input.SColor * float4(SpecularColor, 1);}

float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ return input.Color * tex2D(Sampler, input.TexCoord) + input.SColor * float4(SpecularColor, 1);}

Important: We only multiply the texture color by the diffuse illumination, not the specular illuminations. We can have another texture map for specular illumination sometimes.


Those extra parameters

BasicEffect bEffect = part.Effect as BasicEffect; part.Effect = diffuseSpecularEffect.Clone(part.Effect.GraphicsDevice);

part.Effect.Parameters["DiffuseColor"].SetValue(bEffect.DiffuseColor); part.Effect.Parameters["SpecularColor"].SetValue(bEffect.SpecularColor); part.Effect.Parameters["Shininess"].SetValue(bEffect.SpecularPower);

effect.Parameters["Eye"].SetValue(camera.Eye);

Setting up the effect:

When you draw:


Phong Shading or Per-Pixel Lighting

All of the methods we have used computed the lighting at the vertex and interpolated the color between the vertices.

Phong Shading interpolates the normal over the surface and computes the color at every pixel.

Will always look better and the only way to do some effects light spotlights, but can be costly!


HLSL

struct VertexShaderOutput{ float4 Position : POSITION0; float2 TexCoord : TEXCOORD0; float4 WorldPosition : TEXCOORD1; float3 Normal : TEXCOORD2;};

We put the world position and the normal into “texture coordinates” because these are interpolated for the pixel shader.


Vertex ShaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;

// We need the position and normal in world coordinates output.TexCoord = input.TexCoord;

output.WorldPosition = mul(input.Position, World);

output.Normal = normalize(mul(input.Normal, World));

output.Position = mul(mul(output.WorldPosition, View), Projection);

return output;}

This does surprisingly little!


Pixel Shader (no texture version)float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ float3 normal = input.Normal; float4 position = input.WorldPosition; // Ambient lighting hitting the location float3 color = LightAmbient; float3 scolor = 0; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add diffuse contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color; // Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; return float4(color * DiffuseColor + scolor * SpecularColor, 1);}

Identical to Vertex Shader Version, just moved!


Shader Models


A “shader model” specifies the capabilities we require. 2.0 is required to support using the texture coordinates this way. You also needed 2.0 for the many matrices in the skinned model.


Other extremes of efficiency

Graphics systems such as Maya and 3DS Max can precompute the color for every vertex in advance and save it with the vertex data. We call this Vertex Lighting.


Vertex LightingVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.Color = input.Color; output.TexCoord = input.TexCoord;

return output;}

VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.Color = input.Color; output.TexCoord = input.TexCoord;

return output;} float4 PixelShaderTexture(PixelShaderInput input) : COLOR0

{ float4 color = input.Color; float4 texColor = tex2D(Sampler, input.TexCoord); color.rgba *= texColor; return color;}

float4 PixelShader(PixelShaderInput input) : COLOR0{ return input.Color;}

float4 PixelShaderTexture(PixelShaderInput input) : COLOR0{ float4 color = input.Color; float4 texColor = tex2D(Sampler, input.TexCoord); color.rgba *= texColor; return color;}



The Ultimate Extreme





It’s possible to even avoid this bit of work and keeping the colors around. Instead, the graphics system creates a version of the texture with the lighting pre-multiplied into it. We call this Baked Textures.


Baked Textures Example

float4 PixelShader (PixelShaderInput input) : COLOR0{ return tex2D(Sampler, input.TexCoord);}

float4 PixelShader (PixelShaderInput input) : COLOR0{ return tex2D(Sampler, input.TexCoord);}

VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.TexCoord = input.TexCoord;

return output;}

VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.TexCoord = input.TexCoord;

return output;}


Baked Lighting

We refer to vertex lighting or baked textures as “baked lighting”, meaning the lighting is precomputed and built into the model as vertex colors or into the textures directly. Most all games use baked lighting extensively.

Advantages of baked lightingLighting model can be much more complex including as many lights as we want, complex light falloffs, radiosity, ray-tracing, shadows, anything we like! Baked lighting is as fast as is possible.

Disadvantages of baked lightingOnly works for things that don’t move relative to lights.Diffuse and ambient illumination only, no specular illumination (sometimes we bake the diffuse and ambient, then add the specular at runtime, though).Can’t change the illumination at runtime (no sunsets, etc.)


Incandescence

Incandescence is simply an added term for the color that represents light generated by the surface.

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1) + float4(Incandescence, 1);}


Render to Texture

Instead of rendering to the screen, we can render to a texture image.

Why would we want to?

private RenderTarget2D renderTarget = null;

if (renderTarget == null) { renderTarget = new RenderTarget2D(graphics, 512, 512, 1, graphics.DisplayMode.Format); }

graphics.SetRenderTarget(0, renderTarget); DrawActual(gameTime); graphics.SetRenderTarget(0, null);

Texture2D rendering = renderTarget.GetTexture();


How about something in this scene?

Anything missing?


How about something in this scene?

How would you do something like this?


Mirrors

Eye

Center


Notes

This example is simplified by putting the mirror parallel to the x/y plane (a single value of z).

We’ll assume an upright rectangular mirror.

We’ll assume the mirror is flat.

The first two assumptions can be overcome by just using a bit of math. The last requires a completely different method.


Eye Reflection

Eye

Center

Mirrored Eye

Z=-248

Suppose Eye=(-100, 204, 0). What would be the coordinates of the mirrored eye?


Mirroring the eye around z=-248

// How far are we from the mirror in the Z direction?float zDist = camera.Eye.Z - MirrorPlaneZ;

// Create a mirrored cameraCamera mirrorCamera = new Camera();mirrorCamera.Eye = new Vector3(camera.Eye.X, camera.Eye.Y, MirrorPlaneZ - zDist);

x,y are the same, only z changes. It was zDist from the mirror, it is zDist away in the other direction, now.

Suppose Eye=(-100, 204, 0). What would be the coordinates of the mirrored eye?

Mirrored eye = (-100, 204, -496)


Which way will the camera face?

Eye

Center

Mirrored Eye

Z=-248

Seems like it would be facing a mirrored direction, right? Sorry, it’s not that simple…

?


Which way will the camera face?

Eye

Center

Mirrored Eye

Z=-248

Would not project onto the mirror, but onto a plane at an angle to the mirror!

?


What the mirror looks like

-125 -39

74

192


If we look at it from an angle…


Remember Frustums?

Eye

Center

Mirrored Eye

Z=-248

Make the camera point directly at the wall, then use CreatePerspectiveOffCenter to create a custom camera frustum.


Creating a custom frustum

Front view (from back)

Top view

left right

bottom

top

zNear

public static Matrix CreateOrthographicOffCenter ( float left, float right, float bottom, float top, float zNearPlane, float zFarPlane )

Important, left, right, bottom, top MUST be in view coordinates, not world coordinates.


Putting it all together

// Determine the camera view direction Vector3 cameraView = camera.Center - camera.Eye;

// Are we looking towards the mirror? if (cameraView.Z >= 0) return;

// How far are we from the mirror in the Z direction? float zDist = camera.Eye.Z - MirrorPlaneZ;

// Create a mirror camera Camera mirrorCamera = new Camera(); mirrorCamera.Eye = new Vector3(camera.Eye.X, camera.Eye.Y, MirrorPlaneZ - zDist); mirrorCamera.Center = new Vector3(camera.Eye.X, camera.Eye.Y, MirrorPlaneZ); mirrorCamera.Up = new Vector3(0, 1, 0);


The Custom Projection Matrix // Compute mirror corners in the view coordinate system Vector3 corner1 = new Vector3(-125, 74, -248); Vector3 corner2 = new Vector3(-39, 192, -248);

corner1 = Vector3.Transform(corner1, mirrorCamera.View); corner2 = Vector3.Transform(corner2, mirrorCamera.View);

// Create a projection matrix Matrix projection = Matrix.CreatePerspectiveOffCenter(corner2.X, corner1.X, corner1.Y, corner2.Y, zDist, 10000);

Front view (from back)

left right

bottom

top

corner1

corner2


Rendering and using it if (renderTarget == null) { renderTarget = new RenderTarget2D(graphics, 512, 512, 1, graphics.DisplayMode.Format); }

graphics.SetRenderTarget(0, renderTarget); graphics.Clear(Color.Black); graphics.RenderState.DepthBufferEnable = true; graphics.RenderState.DepthBufferWriteEnable = true;

DrawModels(graphics, mirrorCamera, mirrorCamera.View, projection, gameTime); graphics.SetRenderTarget(0, null);

Texture2D rendering = renderTarget.GetTexture();

foreach (ModelMeshPart part in mirrorMesh.MeshParts) { part.Effect.Parameters["Texture"].SetValue(rendering); part.Effect.CurrentTechnique = part.Effect.Techniques["Textured"]; }


Other Notes

The image will be as viewed from the back of the mirror. You need to “mirror” it to see it from the front of the mirror.

Just ask your artist to create u,v coordinates that will mirror the texture.


Getting Really Fancy…

Javier Cantón Ferrerohttp://www.codeplex.com/XNACommunity/Wiki/View.aspx?title=Reflection

Any Ideas?


See any difference?


ShadowsShadows are a nice effect, plus, the provide an important depth cue.

Are you sure this chair is sitting on the floor?


Shadow Map

Image from light 0 viewpoint

Depth map from light 0 viewpoint

The map tells how far the “lit” point is from the light.


Shadow Map Depth Image

Hard to see, but all we are doing is saving the depth for each pixel.


Creating a depth map

We render the scene from the viewpoint of the light into a depth texture.

1.Create buffers to write into.2.Set up to write to the buffers (writing to a texture).3.Save off the buffers for later use.


Creating a depth texture private RenderTarget2D shadowRenderTarget ; private DepthStencilBuffer shadowDepthBuffer; private Texture2D shadowMap;

SurfaceFormat shadowMapFormat = SurfaceFormat.Unknown;

// Check to see if the device supports a 32 or 16 bit floating point render targetif (GraphicsAdapter.DefaultAdapter.CheckDeviceFormat(DeviceType.Hardware, GraphicsAdapter.DefaultAdapter.CurrentDisplayMode.Format, TextureUsage.Linear, QueryUsages.None, ResourceType.RenderTarget, SurfaceFormat.Single) == true){ shadowMapFormat = SurfaceFormat.Single;}else if (GraphicsAdapter.DefaultAdapter.CheckDeviceFormat( DeviceType.Hardware, GraphicsAdapter.DefaultAdapter.CurrentDisplayMode.Format, TextureUsage.Linear, QueryUsages.None, ResourceType.RenderTarget, SurfaceFormat.HalfSingle) == true){ shadowMapFormat = SurfaceFormat.HalfSingle;}

// Create new floating point render targetshadowRenderTarget = new RenderTarget2D(graphics, shadowMapWidthHeight, shadowMapWidthHeight, 1, shadowMapFormat);

// Create depth buffer to use when rendering to the shadow mapshadowDepthBuffer = new DepthStencilBuffer(graphics, shadowMapWidthHeight, shadowMapWidthHeight, DepthFormat.Depth24);

You usually do this in LoadContent or Activate.


Creating the Shadow Map: Setting up to draw /// <summary> /// Renders the scene to the floating point render target then /// sets the texture for use when drawing the scene. /// </summary> void CreateShadowMap(GraphicsDevice graphics, Camera camera) { // We need a view * projection matrix for the light Matrix lightViewProjection = CreateLightViewProjectionMatrix(camera);

// Set our render target to our floating point render target graphics.SetRenderTarget(0, shadowRenderTarget);

// Save the current stencil buffer DepthStencilBuffer oldDepthBuffer = graphics.DepthStencilBuffer;

// Set the graphics device to use the shadow depth stencil buffer graphics.DepthStencilBuffer = shadowDepthBuffer;

// Clear the render target to white or all 1's // We set the clear to white since that represents the // furthest the object could be away graphics.Clear(Color.White); graphics.RenderState.DepthBufferEnable = true; graphics.RenderState.DepthBufferWriteEnable = true;


Creating the Shadow Map: Drawing foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (Effect effect in mesh.Effects) { effect.CurrentTechnique = effect.Techniques["CreateShadowMap"]; effect.Parameters["LightViewProj"].SetValue(lightViewProjection); } }

// Draw any occluders DrawModel(graphics, camera, Bedroom, Matrix.Identity);

// Set render target back to the back buffer graphics.SetRenderTarget(0, null);

// Reset the depth buffer graphics.DepthStencilBuffer = oldDepthBuffer;

// Return the shadow map as a texture shadowMap = shadowRenderTarget.GetTexture();

Set technique and light matrix.

Obtaining the result.


Setting the effect for regular drawing

foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (Effect effect in mesh.Effects) { Texture2D texture = effect.Parameters["Texture"].GetValueTexture2D(); if (texture != null) { effect.CurrentTechnique = effect.Techniques["Textured"]; } else { effect.CurrentTechnique = effect.Techniques["NoTexture"]; }

effect.Parameters["ShadowMap"].SetValue(shadowMap); } }


Shader Additionsstruct ShadowVertexShaderOutput{ float4 Position : POSITION0; float4 WorldPosition : TEXCOORD1;};

// Transforms the model into light space an renders out the depth of the objectShadowVertexShaderOutput CreateShadowMap_VertexShader(float4 Position: POSITION){ ShadowVertexShaderOutput Out; Out.WorldPosition = mul(Position, World); Out.Position = mul(Out.WorldPosition, LightViewProj); return Out;}

// Saves the depth value out to the 32bit floating point texturefloat4 CreateShadowMap_PixelShader(ShadowVertexShaderOutput input) : COLOR{

float4 pos = mul(input.WorldPosition, LightViewProj); return float4(pos.z / pos.w, 0, 0, 1);}

// Technique for creating the shadow maptechnique CreateShadowMap{ pass Pass1 { VertexShader = compile vs_2_0 CreateShadowMap_VertexShader(); PixelShader = compile ps_2_0 CreateShadowMap_PixelShader(); }}


How do we use this?

Light

Eye

a

b


How do we use this?

Light

Eye

13.4

10.0

14.5

Point a is shadowed, point b is not.Point a is 13.4 from the light, but the depth map says the visible point is 10.0 away, so point a is not seen by the light. Point b is 14.5 from the light and the depth buffer says 14.5, so the point is lit.

a

b


The shadow algorithm

• Determine location of vertex on the shadow map

• d1 the depth stored in the shadow map at that location

• d2 the depth the vertex is relative to the light

• If d1 ≥ d2 the light hits the object


float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ float3 normal = input.Normal; float4 position = input.WorldPosition; // Ambient lighting hitting the location float3 color = LightAmbient; float3 scolor = 0; // Find the position of this pixel in the light space float4 lightingPosition = mul(input.WorldPosition, LightViewProj);

// Find position in the shadow map float2 ShadowTexCoord = 0.5 * lightingPosition.xy / lightingPosition.w + float2(0.5, 0.5); ShadowTexCoord.y = 1.0 - ShadowTexCoord.y;

// Get the depth stored in the shadow map float shadowdepth = tex2D(ShadowMapSampler, ShadowTexCoord).r;

// Calculate the pixel dpeth float ourdepth = (lightingPosition.z / lightingPosition.w) - 0.001f;

if(shadowdepth >= ourdepth) { // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position);

// Add diffuse contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color;

// Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V);

scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; } return float4(color * tex2D(Sampler, input.TexCoord) + scolor * SpecularColor, 1);}

Pixel shader – only works for per-pixel lighting


Only piece left…

How do we figure out where to point the camera when taking a picture from the light’s viewpoint?

Any ideas?


We want to enclose the visible camera frustum

Light

Eye

There are 8 points in a camera frustum. Be sure all 8 are visible.


Creating the view matrix

Matrix CreateLightViewProjectionMatrix(Camera camera) { // Create a bounding frustum for our camera BoundingFrustum frustum = new BoundingFrustum(camera.View * camera.Projection);

Vector3[] frustumCorners = frustum.GetCorners(); Vector3 frustumCenter = frustumCorners[0]; for (int i = 1; i < frustumCorners.Length; i++) { frustumCenter += frustumCorners[i]; }

frustumCenter /= frustumCorners.Length;

Vector3 Light1Location = new Vector3(5, 221, -19);

// Create the view and projection matrix for the light Matrix lightView = Matrix.CreateLookAt(Light1Location, frustumCenter, new Vector3(0, 1, 0));

This just means the center of the shadow map will be the center of the view frustum.


Creating the projection matrix float zNear = 50;

// Determine maximum extents in each direction float left = 0, right = 0, top = 0, bottom = 0; foreach (Vector3 corner in frustumCorners) { // Transform to view coordinate system Vector3 v = Vector3.Transform(corner, lightView);

// Project to the near clipping plane v.X = -v.X / v.Z * zNear; v.Y = -v.Y / v.Z * zNear;

if (v.X < left) left = v.X; if (v.X > right) right = v.X;

if (v.Y < bottom) bottom = v.Y; if (v.Y > top) top = v.Y; }

Matrix lightProjection = Matrix.CreatePerspectiveOffCenter(left, right, bottom, top, zNear, 1000);

return lightView * lightProjection; }

This projects the frustum points onto a view plane, then ensures the frustum for the light is big enough.


The net effect


More Touching…

How can we determine if we have clicked on something?

Especially, how could we determine if we have clicked on Victoria’s hand?

Think about this one, particularly the second point!


Item Buffers

Have your artist create extra meshes with a fixed color. These don’t show normally. But, if you render these to a texture, you can tell what is where.

An image where the pixels tell what the object is (often by color) rather than the color of the image is called an item buffer.


Item Buffers

Rendered image

Item buffer

Create a shader that just sets the pixel to the diffuse color; no lighting or anything. Then the diffuse color tells you what is at the pixel.


Optimizations

Often I’m only interested in what’s at one locationWhere the mouse is right now.

Render a small image (4x4 for example) only around the mouse coordinates.


Doing this…

int wid = device.Viewport.Width; int hit = device.Viewport.Height;

float scaleX = 1 / rangeX; float scaleY = 1 / rangeY;

// Where is the mouse on the screen? float rx = ((float)x / (float)wid - 0.5f) * 2; float ry = -((float)y / (float)hit - 0.5f) * 2; mapViewport = device.Viewport;

projection = camera.Projection * Matrix.CreateTranslation(-rx, -ry, 0) * Matrix.CreateScale(scaleX, scaleY, 1);


Spotlightsfloat4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ float3 color = float3(0, 0, 0); // Compute direction to the light float3 Light1Direction = normalize(Light1Location - input.PositionWorld); float3 Light1Pointing = normalize(Light1Location - Light1Target); if(dot(Light1Pointing, Light1Direction) > 0.98) { // Add contribution due to this light color = max(dot(Light1Direction, input.NormalWorld), 0) * Light1Color; }

return ((input.Color + float4(color, 1)) * float4(DiffuseColor, 1)) ;}

0.98 is cos(11.4o). So, our cone will have that spread.


Any ideas?


Fog

Two things at play here:

1.The farther the point the dimmer it gets.

2.The farther the point the more we see the fog color.

We need a weighted sum of the fog color and the computed color.

)1( dfog

dcomputedresult FCFCC

Distance to pointFog density constantFog colorComputed ColorResult Color


HLSLfloat4 fog = 0.992;float4 fogColor = float4(0.4, 0.5, 0.4, 1);

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ … // Add specular contribution due to this light float3 V = Eye - position; float d = length(V); V /= d; float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; float4 thecolor = float4(color * DiffuseColor + scolor * SpecularColor, 1); float fogW = pow(fog, d); thecolor = thecolor * fogW + fogColor * (1 - fogW); return thecolor;}

illumination and shading

Documents