wilf lalonde ©2012 comp 4501 95.4501 ambient occlusion
TRANSCRIPT
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Ambient Occlusion
Wilf LaLonde ©2012Comp 4501
• Ambient occlusion (AO): a “smart” ambient term that adds shadowing to diffuse objects lit with environment lighting; i.e., shadows in corners and creases in room without lights.
• Screen space ambient occlusion (SSAO), a variation computed directly from information on the screen; e.g., the depth buffer or camera space positional information containing z…
Ambient OcclusionAmbient Occlusion
Corners have more occlusion than flat walls.
Wilf LaLonde ©2012Comp 4501
Ambient Occlusion (AO)Ambient Occlusion (AO)
Diagram from Hoang and Low (Multi-Resolution SSAO)
AO is the lighting phenomenon under the direct illumination of a diffuse, uniform, spherical light
source surrounding the scene.
Wilf LaLonde ©2012Comp 4501
• How much does the environment lighting contribute at a point? It depends how much occlusion there is in the volume visible from the sample point...
The IssueThe Issue
Diagram from Loos and Sloan(Volumetric Obscurance)
Wilf LaLonde ©2012Comp 4501
Technically It’s A Volume ContributionTechnically It’s A Volume Contribution
Occlusion is cosine-weighted fraction of a tangent hemisphere
Minimal occlusion = 0 (no contribution, denoted ) for sample at or outside radius of influence or on negative side of normal)
[1 - (di / dmax)] cos i
Maximal occlusion 1 occurs when occlusion occurs immediately at
p; i.e., di is 0
The AO Definition
Wilf LaLonde ©2012Comp 4501
VariationsVariations
Obscurance SSAO
Horizon-Based SSAO (HSSAO)
Z-Buffer Based SSAO
Multi-Resolution SSAO (MSSAO)
Volumetric Obscurance (VO)
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Brief Overview
Of a Number
of Techniques
Wilf LaLonde ©2012Comp 4501
obscurance
• From a point ALONG THE BASE POINT NORMAL, find how many samples in 3D sphere are obscured.
Technique 1: ObscuranceTechnique 1: Obscurance
The Alchemy Screen-Space Ambient Obscurance Algorithm, McGuire, Osman, Bukowski, Hennessay, High Performance Graphics 2011.
Paper Available
base point in green
COMPLEX (LIKE BUMPMAPPING)
Wilf LaLonde ©2012Comp 4501
• Use 3D sphere of points to sample the z buffer in the neighbourhood of the point (approximate occlusion by ratio of occluded to total points).
Technique 2: CrisisTechnique 2: Crisis
Screen Space Ambient Occlusion, Kajalin (Crysis), pp 413-424, ShaderX7, 2009.
wall 0.5 inside corner 0.75 outside corner 0.25
Z-buffer View 3D View
base point in green
works with z-buffer
SIMPLE
Wilf LaLonde ©2012Comp 4501
Actual occlusion is ignored but you have access to all unoccluded points.
Technique 3: MSSAO (AO Definition in Eye Space)Technique 3: MSSAO (AO Definition in Eye Space)
Uses a position bufferinstead of just z-buffer
sampling via 2D sphere
We ignore the factor 1/.
Distance based Occlusion
Multi-resolution screen-space ambient occlusion,, pp. 101-102, ACM Symposium on VR Software and Technology 2010.
MORE COMPLEX + DEMO LATER
Additionally focuses on how a hierarchy can be used to improve AO for both near
and far...
We will use it as a basis for a reimplementation
Wilf LaLonde ©2012Comp 4501
Technique 4: Volumetric ObscuranceTechnique 4: Volumetric Obscurance
Say it’s fully occluded (1) if on negative side of normal;
otherwise not (0).
We will implement this as a MOD of MSSAO just to see how good/bad it is.
This is a simplistic modification of #3 with only 1 pass (though it
probably uses much larger filters).
Wilf LaLonde ©2012Comp 4501
horizon-based
• How far toward horizon you can rotate before hitting… Quantizes horizon angle + quantizes rotation angle around normal... Requires many samples in 3D sphere...
Technique 5: Horizon-Based SSAOTechnique 5: Horizon-Based SSAO
MORE COMPLEX + DEMOS
NVIDiA SDK 10
NVIDiA SDK 11
Image-Space Horizon-Based Ambient Occlusion (NVIDIA demo), Bavoil and Sainz, pp 425-444, ShaderX7, 2009.
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Preparing To
Implement: Some
Ideas
Wilf LaLonde ©2012Comp 4501
• Standard ratio increases linearly from 0 to 1.
Ratios: We’ll want Ratios: We’ll want
dmax
d
• Squared ratio increases more slowly.dmax
2
d2
makes occlusion nearby (more crispy)
Recall: occlusion 1 (at radius 0), occlusion 0 (at radius dmax).
dmax is radius of influence
[1 - (di / dmax)] cos i
From Bavoil and Sainz (Horizon based…)and Hoang and Low (Multi-Resolution SSAO)
From Bavoil and Sainz (Horizon based…)and Hoang and Low (Multi-Resolution SSAO)
Wilf LaLonde ©2012Comp 4501
• Use per pixel normals (more accurately computed from derivatives)…
Details That Make it BetterDetails That Make it Better
From Bavoil and Sainz (Horizon based…)and Hoang and Low (Multi-Resolution SSAO)
From Bavoil and Sainz (Horizon based…)and Hoang and Low (Multi-Resolution SSAO)
Wilf LaLonde ©2012Comp 4501
• Blurring the result helps… even more if the blurring preserves edges.
Details That Make it BetterDetails That Make it Better
From Kajalin (Crysis)…From Kajalin (Crysis)…
Wilf LaLonde ©2012Comp 4501
• Sample in direction of normal to eliminate those that are on the negative side (so fewer samples can be used). Applies mostly in 3D.
Details That Make it BetterDetails That Make it Better
From Briney et al, Fast Fake Global Illumination, ShaderX7, 2009
From Briney et al, Fast Fake Global Illumination, ShaderX7, 2009
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Hoang and Low
multi-resolution screen-
space ambient occlusion
(MSSAO)
Wilf LaLonde ©2012Comp 4501
• Far (low frequency detail) and near (high frequency detail) integrated together...
• Can be done with smaller filters without resorting to random sampling for lower resolutions.
• Results do not suffer from noise and excessive blur (even though some blurring is used).
AdvantagesAdvantages
Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on
Virtual Reality Software and Technology 2010
Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on
Virtual Reality Software and Technology 2010
Wilf LaLonde ©2012Comp 4501
Ambient Occlusion (AO)Ambient Occlusion (AO)
𝐩
𝐪
Diagram from Hoang and Low
Look at Demo
Wilf LaLonde ©2012Comp 4501
• It’s in OpenGL, so a switch from DirectX.• Some simple demo changes to allow experiments
(including bug fixes) and changing the number of hierarchical steps. Original demo needs at least 3; we want 1 or more...
• First use of poisson disk filter. • First use of downsampling median filter.• First use of gaussian blur filter.• First use of bilateral upsampling filter (what’s that).• Minor use of temporal coherence (what’s that).• Experiment with volumetric obscurance when done to
see how good/bad that is...
What We Will Learn By ReImplementing the MSSAO AlgorithmWhat We Will Learn By ReImplementing the MSSAO Algorithm
Wilf LaLonde ©2012Comp 4501
To approximate integration, we sum the sample contributions and divide by their number.
Remember: We Sample in Eye SpaceRemember: We Sample in Eye Space
So all samples are on the surface
We ignore the factor 1/.
We’ll revisit the full AO definition later.
Wilf LaLonde ©2012Comp 4501
• The idea will be presented in 3 ways:
• In words: a bit verbose but accurate...
• Diagrammatically: via diagrams based on (but not the same as) Hoang and Low’s
slides.
• In code: a piece of code with much less
detail that might give you a better perspective...
How Its DoneHow Its Done
Wilf LaLonde ©2012Comp 4501
• The standard drawing pass uses MRT to create 2 render targets; a position buffer in camera coordinates and a normal buffer... (a buffer of attributes is referred to as a g-buffer; short for geometry buffer – predates the notion of geometry shaders which isn’t used here).
• Draw successive full screen quads to ping pong the position/normal textures into lower and lower resolution textures again using MRT (a simple “median downsample” shader).
• On your way back up from lower resolution levels, compute an AO texture using a diamond grid filter and combine with a lower level AO texture (if there is one) with an additional blur pass. Note to Wilf... Make blur optional.
• For the last pass, use poisson filter and no blur...
How Its Done: In WordsHow Its Done: In Words
Wilf LaLonde ©2012Comp 4501
How it’s Done: Diagrammatically using 3 levels...How it’s Done: Diagrammatically using 3 levels...
Initial Datag-buffer (1024x1024)
Downsampleg-buffer (512x512)
g-buffer (256x256)
Compute AO #1
Blur
g-buffer (1024x1024)
AO buffer (256x256)
AO buffer (512x512)
AO buffer (1024x1024)
Blur AO buffer (256x256)
Blur AO buffer (512x512)
Compute A0 #2 Combined with Blurred AO #1
Blur
Compute A0 #3 2 Combined with Blurred AO #2
poisson filter
diamond grid filter
diamond grid filter
No Blur in last stage
Wilf LaLonde ©2012Comp 4501
#define DownsampleGBuffer DownsampleRender2GBuffer (); //Hoang and Low’s name...
for (int index = 1, size = RESOLUTION / 2; index < LEVEL_COUNT; ++index, size /= 2) {DownsampleGBuffer (size, index); //Each step decreases resolution...
} for (int index = LEVEL_COUNT - 1, size = minResolution; index >= 0; --index, size *= 2) {
RenderAO (size, index); //Each step increases resolution.}
How Its Done: In CodeHow Its Done: In Code
Index (LEVEL_COUNT) strictly decrementing for downsampling g-buffer and strictly incrementing for building AO buffer bottom up...,
Wilf LaLonde ©2012Comp 4501
Combining 5 Levels Into ONE FINAL RESULTCombining 5 Levels Into ONE FINAL RESULT
Each one ½ the resolution of previous one
Wilf LaLonde ©2012Comp 4501
Blizzard (noisier) MSSAO
Comparison With Other TechniquesComparison With Other TechniquesComparison With Other TechniquesComparison With Other Techniques
Wilf LaLonde ©2012Comp 4501
Horizon-based AO (noisier) MSSAO
Comparison With Other TechniquesComparison With Other Techniques
Wilf LaLonde ©2012Comp 4501
Volumetric Obscurance MSSAO
Comparison With Other TechniquesComparison With Other Techniques
Wilf LaLonde ©2012Comp 4501
Ground Truth Comparison With Non-Real-time TechniqueGround Truth Comparison With Non-Real-time Technique
Blender MSSAO
Wilf LaLonde ©2012Comp 4501
Blizzard HBAO MSSAO
Summarizing: Less noise and blur and better high-frequency detailsSummarizing: Less noise and blur and better high-frequency details
Wilf LaLonde ©2012Comp 4501
Performance ResultsPerformance Results
MSSAO VAO Blizzard HBAO
Sibenik Cathedral
21.9 ms 22.9 25.7 50.1
Conference Room
24.0 ms 24.8 24.9 49.5
Sponza Atrium
22.2 ms 24.0 28.9 54.3
• Scenes rendered at 1024x1024 on GeForce GTX 460M
• Exclusive of geometry pass
Wilf LaLonde ©2012Comp 4501
Let’s Attack The Problem in StagesLet’s Attack The Problem in Stages
• Start with a brief look at the code (including simple extensions I made).
• Have a look at the framebuffer creation code including the initial compiling of the shaders (and how we’ll simplify that).
• Deal with the simpler aspects first; namely the downsampling code and shader (DownsampleGBuffer) and the blur code and shader (Blur).
• Deals with the more complex code dealing with AO processing (RenderAO).
• Temporal coherance (a minor aspect).Main Focus
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Looking at the Code
Wilf LaLonde ©2012Comp 4501
inline void detectOpenGLError () {//WILF ADDITION...GLenum code = glGetError ();if (code != GL_NO_ERROR) {
printf ("\nEncountered error %x \"%s\".", code, gluErrorString (code));
}}
Addition To Find Out Why Original Demo Wouldn’t RunAddition To Find Out Why Original Demo Wouldn’t Run
• Implemented detectOpenGLError () and peppered code with it.
//varying out vec4 pos; //wilfout vec4 pos;
• Shader “geometry.vert” wasn’t compiling.
Wilf LaLonde ©2012Comp 4501
//WILF VERSION EXPERIMENTS...#define ORIGINAL_VERSION 0#define WILF_VERSION 1
//#define WILF_version ORIGINAL_VERSION#define WILF_version WILF_VERSION#define LEVEL_COUNT 5 // number of mip-map levels.
//NOTE: ORIGINAL_VERSION needs to be 3 or more...
Additions For Switching Between ImplementationsAdditions For Switching Between Implementations
Wilf LaLonde ©2012Comp 4501
• Shader #include is NOT part of this OpenGL. Added includePreprocessor to “shader.cpp”.
Additions To Support One Shader Include FileAdditions To Support One Shader Include File
GLuint LoadShader(EShaderType type, char *file_name) { printf ("Load shader \"%s\".\n", file_name); char *temp = ReadFile(file_name); temp = includePreprocessor (temp); //WILF const char *shader_text = temp; “Code to compile shader and return handle to it.”}
Wilf LaLonde ©2012Comp 4501
• Looks Like it Used to Run from command line via “xxx.bat”. Took that out and instrumented it so one of 3 supplied models could be run.
Modification To MainModification To Main
enum ModelType {UseSponza, UseSibenik, UseConference};
ModelType modelType = UseSponza; //WILF
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Framebuffer
Creation Code
Wilf LaLonde ©2012Comp 4501
// buffers and textures ([0] = finest resolution)GLuint frameBufs [LEVEL_COUNT];GLuint depthBufs [LEVEL_COUNT];GLuint posTex [LEVEL_COUNT];GLuint normTex [LEVEL_COUNT];GLuint aoTex [LEVEL_COUNT];GLuint aoTexBlur [LEVEL_COUNT]; GLuint lastFrameAOTex;GLuint lastFramePosTex;
DeclarationsDeclarations
All handles
Took out “randRotTex” since not used by any shader.
Wilf LaLonde ©2012Comp 4501
void SetupFBOs () {//WILF MADE SHORTER... #define initializeColorTexture(name,size) \
glBindTexture (GL_TEXTURE_RECTANGLE, name); \glTexImage2D (GL_TEXTURE_RECTANGLE, 0, GL_RGBA32F, size, size, 0, GL_RGBA, GL_FLOAT, NULL); \glTexParameterf (GL_TEXTURE_RECTANGLE, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); \glTexParameterf (GL_TEXTURE_RECTANGLE, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); \glTexParameteri (GL_TEXTURE_RECTANGLE, GL_TEXTURE_MAG_FILTER, GL_NEAREST); \glTexParameteri (GL_TEXTURE_RECTANGLE, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
#define initializeDepthTexture(name,size) \glBindRenderbuffer (GL_RENDERBUFFER, name); \glRenderbufferStorage (GL_RENDERBUFFER, GL_DEPTH_COMPONENT, size, size);
#define attachColorTexture(which,name) \glFramebufferTexture2D (GL_FRAMEBUFFER, which, GL_TEXTURE_RECTANGLE, name, 0);
#define attachDepthTexture(name) \glFramebufferRenderbuffer (GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, name);
#define errorCheck(name) \if (glCheckFramebufferStatus (GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE) \ printf ("Framebuffer created successfully.\n"); \else \ printf ("Framebuffer created unsuccessfully. Error code: %d.\n", glCheckFramebufferStatus (name));
#undef initializeColorTexture#undef initializeDepthTexture#undef attachDepthTexture#undef attachColorTexture#undef errorCheck(name
}
Frame Buffers SetupFrame Buffers Setup
code
Wilf LaLonde ©2012Comp 4501
void SetupFBOs () {//WILF MADE SHORTER...
glGenFramebuffers (LEVEL_COUNT, frameBufs); glGenRenderbuffers (LEVEL_COUNT, depthBufs);
glGenTextures (LEVEL_COUNT, posTex); glGenTextures (LEVEL_COUNT, normTex);glGenTextures (LEVEL_COUNT, aoTex); glGenTextures (LEVEL_COUNT, aoTexBlur);glGenTextures (1, &lastFrameAOTex); glGenTextures (1, &lastFramePosTex);
initializeColorTexture (lastFrameAOTex, RESOLUTION); initializeColorTexture (lastFramePosTex, RESOLUTION);
for (int i = 0, size = RESOLUTION; i < LEVEL_COUNT; ++i, size /= 2) { initializeColorTexture (posTex [i], size); initializeColorTexture (normTex [i], size);initializeColorTexture (aoTex [i], size); initializeColorTexture (aoTexBlur [i], size);
initializeDepthTexture (depthBufs [i], size);
glBindFramebuffer (GL_FRAMEBUFFER, frameBufs [i]);attachDepthTexture (depthBufs [i]);attachColorTexture (GL_COLOR_ATTACHMENT0, posTex [i]);attachColorTexture (GL_COLOR_ATTACHMENT1, normTex [i]);attachColorTexture (GL_COLOR_ATTACHMENT2, aoTex [i]);attachColorTexture (GL_COLOR_ATTACHMENT3, aoTexBlur [i]);if (i == 0) { attachColorTexture (GL_COLOR_ATTACHMENT4, lastFrameAOTex); attachColorTexture (GL_COLOR_ATTACHMENT5, lastFramePosTex);}errorCheck (frameBufs [i]);
}}
Frame Buffers SetupFrame Buffers Setup
Wilf LaLonde ©2012Comp 4501
glPushMatrix(); camera.ApplyCameraTransform ();
Render2GBuffer (); DownsampleGBuffer LOOP in decreasing resolutionRenderAO LOOP in increasing resolution
glPopMatrix();
if (oddFrame)glBindTexture(GL_TEXTURE_RECTANGLE, aoTex[0]);
elseglBindTexture(GL_TEXTURE_RECTANGLE, lastFrameAOTex);
oddFrame = !oddFrame;glCopyTexSubImage2D (GL_TEXTURE_RECTANGLE, 0, 0, 0, 0, 0,
RESOLUTION, RESOLUTION);
glutSwapBuffers();
Main Rendering TaskMain Rendering Task
Optional
temporal c
ohesion
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Setting Up The
Shaders
Wilf LaLonde ©2012Comp 4501
3 Groups of Shaders3 Groups of Shaders
SetupGeometryProgram (initial MRT textures)
geometry.vert geometry.frag
ortho.vertdownsample.frag
SetupDownsamplePrograms (lower resolution MRT textures)
SetupAOPrograms (upsampling into AO texture)ortho.vertaoWilf.frag MRT has a position texture
+ normal texture
Render2GBuffer
DownsampleGBuffer
RenderAO
Wilf LaLonde ©2012Comp 4501
• Easier to look at actual code BUT we made one important design change.
Original version: Used 3 different AO shaders
aoLast.frag ao.frag aoFirst.frag
Setting Up The AO (Upsampling) ShadersSetting Up The AO (Upsampling) Shaders
lowest resolutionmiddle resolutionshighest resolution
• As mentioned on previous slide, we use one
aoWilf.frag
Wilf LaLonde ©2012Comp 4501
• Added 3 shader variables to aoWilf.frag
To Support the Use of JUST ONE SHADERTo Support the Use of JUST ONE SHADER
struct ShaderSetting {bool usePoisson; bool useUpsampling; bool useTemporalSmoothing;
};
ShaderSetting computeShaderSetting (long levelCount) {ShaderSetting setting;setting.usePoisson = levelCount == 0; //highest is 0setting.useUpsampling = levelCount < (LEVEL_COUNT - 1) //lowest
&& LEVEL_COUNT > 1;setting.useTemporalSmoothing = true; //or falsereturn setting;
}
• Created a struct in .cpp code
uniform bool usePoisson; uniform bool useUpsampling; uniform bool useTemporalSmoothing;
Now look at actual codeNote to Wilf... Make blur optional.
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Drawing the INITIAL
MRT Textures
(Render2GBuffer)
Wilf LaLonde ©2012Comp 4501
GLenum bufs01[2] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1};GLenum bufs51[2] = {GL_COLOR_ATTACHMENT5, GL_COLOR_ATTACHMENT1};
void Render2GBuffer () { glUseProgram (geometryProg); glBindFramebuffer (GL_FRAMEBUFFER, frameBufs[0]); glDrawBuffers (2, oddFrame ? Bufs01 : bufs51); glPushAttrib (GL_VIEWPORT_BIT); glViewport (0, 0, RESOLUTION, RESOLUTION); glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); DrawModel (); glPopAttrib (); }
Gbuffer Rendering (INITIAL MRT Textures)Gbuffer Rendering (INITIAL MRT Textures)
0: posTex [i]1: normTex [i])5: lastFramePosTex
All we draw
Switching back and forth is for optional temporal cohesion
USE MRT
first shader (geometry.vert + ...frag”
Wilf LaLonde ©2012Comp 4501
• Geometry.vert
G-Buffer Drawing ShaderG-Buffer Drawing Shader
#version 140out vec4 pos; //varying out vec4 pos; //wilf
void main() {pos = gl_ModelViewMatrix * gl_Vertex;gl_Position = ftransform (); }
• Geometry.frag
#version 140#extension GL_ARB_texture_rectangle : require
in vec4 pos; out vec4 Pos; out vec4 Norm;
void main() {Pos = pos / pos.w;//WILF BUG FIX: On my ATI card, one of the differentials has the //wrong sign (needs a card independent solution to fix this properly), vec3 n = -cross (dFdx(pos.xyz), dFdy(pos.xyz));Norm = vec4(normalize(n), sign(abs(pos.z))); }
Differentials Need An Explanation
0 in 4th component if z is 0
MRT
Wilf LaLonde ©2012Comp 4501
• dFdx gives the change of the function with respect to x... (the slope)
DifferentialsDifferentials
f f2xyz – f1xyz fx, fy, fz
x x x x xf1
f2= =
• The direction of greatest x change can be interpreted as a good direction for an x-axis.
• Similarly, dFdy is a good direction for a y-axis.
Wilf LaLonde ©2012Comp 4501
• Consequently, with dFdx and dFdy both on the surface, their cross product is perpendicular to the surface
DifferentialsDifferentials
is normal to the surface
fx
fy
fx
fy
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Drawing the LOWER
RESOLUTION MRT Textures
(DownsampleGBuffer)
Wilf LaLonde ©2012Comp 4501
• Hoang and Low claim to be using the median of the position and normals.
Downsizing to a Lower Resolution G-Buffer Downsizing to a Lower Resolution G-Buffer
• Definition (modified from Wikipedia version): A median of n numbers is a middle value (IF THEY WERE SORTED). When n is odd, the median is unique; otherwise, there are 2 candidates (often, the average is picked to resolve the issue).
• Example:
Median of 1,3,7,100,101 is 7.
Median of 1,2,3,4 is 2 or 3 (or 2.5).
Wilf LaLonde ©2012Comp 4501
Downsampling; e.g. 1024x 1024 to 512x512Downsampling; e.g. 1024x 1024 to 512x512
𝑝1 𝑝2
𝑝3 𝑝4
𝑝
Given p1.z p2.z p3.z p4.z
If (p4.z – p1.z dmax)p (p2+p3)/2; n (n2+n3)/2;
elsep p2; n n2;
After sorting by deph, pick the median (AVERAGE ONLY IF the Z-SPAN is within the radius of influence dmax).
Wilf LaLonde ©2012Comp 4501
void Downsample (int size, int index) {//Note that index is 1 or more; index – 1 is previous one).glUseProgram (downsampleProg [index]);
glActiveTexture (GL_TEXTURE0);if (index == 1 && !oddFrame)
glBindTexture (GL_TEXTURE_RECTANGLE, lastFramePosTex); else
glBindTexture (GL_TEXTURE_RECTANGLE, posTex [index - 1]);
glActiveTexture (GL_TEXTURE1);glBindTexture( GL_TEXTURE_RECTANGLE, normTex [index - 1]);
glBindFramebuffer(GL_FRAMEBUFFER, frameBufs [index]);glDrawBuffers (2, bufs01);
}
The Downsizing Code for the Lower Res G-BufferThe Downsizing Code for the Lower Res G-Buffer
Optional
temporal c
ohesion
Inputs: position + normal
Code to draw a quad (see next slide)outputs in the frame buffer (2 outputs)
Wilf LaLonde ©2012Comp 4501
glPushAttrib (GL_VIEWPORT_BIT);glViewport (0, 0, size, size);glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);glBegin (GL_QUADS); //WILF FIX: Need vertices counter-clockwise
glVertex2d (0, 0); //start at bottom leftglVertex2d (size, 0); //go rightglVertex2d (size, size); //go upglVertex2d (0, size); //go left
glEnd ();glPopAttrib ();
The Downsizing CodeThe Downsizing Code
This is the code to draw a quad...
Note that original code was clockwise which works ONLY if face culling is turned off…
Wilf LaLonde ©2012Comp 4501
The Downsizing ShaderThe Downsizing Shader
[i,j]
0 1 2 30 1 2 3 4 5 6 7
[2i, 2j] Note that red pixels are
+1 or –1 away from center
Wilf LaLonde ©2012Comp 4501
• Recall: AO G-Buffer uses eye space position and pixel normal (one hint for better behavior).
• Hoang and Low uses the median (another hint) of the position and normals (averages only if within radius of influence).
• When averaging normals, does not normalize.
Summarizing Downsizing DetailsSummarizing Downsizing Details
• Claim: using the this median method helps reduce self-occlusion artifacts and provides better temporal coherence.
Let’s look at the shader code
Wilf LaLonde ©2012Comp 4501
• ortho.vert
G-Buffer Downsizing ShaderG-Buffer Downsizing Shader
#version 140
uniform mat4 gluOrtho; //wilf: orthographic
void main() {gl_Position = gluOrtho * gl_Vertex;}
• downsample.frag
#version 140
#extension GL_ARB_texture_rectangle: enable
uniform sampler2DRect hiResNormTex;uniform sampler2DRect hiResPosTex;
out vec4 Norm; out vec4 Pos;
Higher resolution textures
main (next slide)
Wilf LaLonde ©2012Comp 4501
G-Buffer Downsizing ShaderG-Buffer Downsizing Shader
void main() {
vec2 xy2 = gl_FragCoord.xy * 2.0; vec4 pos[4]; vec4 norm[4];
pos[0] = texture2DRect (hiResPosTex, xy2 + vec2(-1.0, 1.0)); //WILF: Change each 0.5 to 1.0. pos[1] = texture2DRect (hiResPosTex, xy2 + vec2(1.0, 1.0));pos[2] = texture2DRect (hiResPosTex, xy2 + vec2(1.0, -1.0));pos[3] = texture2DRect (hiResPosTex, xy2 + vec2(-1.0, -1.0));
norm[0] = texture2DRect (hiResNormTex, xy2 + vec2(-1.0, 1.0));norm[1] = texture2DRect (hiResNormTex, xy2 + vec2(1.0, 1.0));norm[2] = texture2DRect (hiResNormTex, xy2 + vec2(1.0, -1.0));norm[3] = texture2DRect (hiResNormTex, xy2 + vec2(-1.0, -1.0));
float maxZ = max (max (pos[0].z, pos[1].z), max (pos[2].z, pos[3].z));float minZ = min (min (pos[0].z, pos[1].z), min (pos[2].z, pos[3].z));
int minPos, maxPos;
for (int i = 0; i < 4; ++i) {if (pos[i].z == minZ) minPos = i; if (pos[i].z == maxZ) maxPos = i;
}
float d = distance(pos[minPos].xyz, pos[maxPos].xyz); ivec2 median = ivec2 (0, 0); int index = 0;for (int i = 0; i < 4 && index < 2; ++i)
if (i != minPos && i != maxPos) median [index++] = i;
if (d < 1.0) {Pos = (pos [median.x] + pos [median.y]) / 2.0; Norm = (norm [median.x] + norm [median.y]) / 2.0;
} else {Pos = pos [median.x]; Norm = norm [median.x];
}}
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Blurring
Wilf LaLonde ©2012Comp 4501
• Uses a gaussian blur... with weights computed at setup time...
• This shader (ortho.vert + blur.frag) has been untouched…
BlurBlur
Wilf LaLonde ©2012Comp 4501
• Uses a VARIATION OF A gaussian blur... with weights computed at setup time...
• This shader (ortho.vert + blur.frag) has been untouched…
BlurBlur
This has a special name which we will reveal shortly
Wilf LaLonde ©2012Comp 4501
The Blur ShaderThe Blur Shader#version 140#extension GL_ARB_texture_rectangle: enable#extension GL_EXT_gpu_shader4: enable
uniform sampler2DRect aoTex; uniform sampler2DRect normTex; uniform sampler2DRect posTex;out vec3 AO;
void main () {vec3 n = texture2DRect (normTex, gl_FragCoord.xy).xyz;vec3 p = texture2DRect (posTex, gl_FragCoord.xy).xyz;
vec3 ss = vec3 (0.0); float weight = 0.0;
for (float i = -1.0; i <= 1.0; i += 1.0) {for (float j = -1.0; j <= 1.0; j += 1.0) {
vec2 ij = vec2(i, j);vec3 t = texture2DRect(aoTex, gl_FragCoord.xy + ij).xyz;vec3 norm = texture2DRect(normTex, gl_FragCoord.xy + ij).xyz;float depth = texture2DRect(posTex, gl_FragCoord.xy + ij).z;
float normWeight = (dot (norm, n) + 1.2) / 2.2;normWeight = pow (normWeight, 8.0);
float depthWeight = 1.0 / (1.0 + abs(p.z - depth) * 0.2);depthWeight = pow(depthWeight, 16.0);
float gaussianWeight = 1.0 / ((abs(i) + 1.0) * (abs(j) + 1.0));
weight += normWeight * depthWeight * gaussianWeight;ss += t * normWeight * depthWeight * gaussianWeight;
}}
AO = vec3 (ss / weight);}
1 when equal, less otherwise
1 when equal, less otherwise
1 when i=j=0; else ½, ¼.
Wilf LaLonde ©2012Comp 4501
• It’s called a bilinear filter (although it might also make sense to call it a trilinear filter)…
• A bit about the history of bilinear filters coming up.
What Kind Of Filter Was That What Kind Of Filter Was That
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Bilinear Filters
Wilf LaLonde ©2012Comp 4501
• Upsamples source using context around it...
Bilateral Upsampling Introduced in 2007Bilateral Upsampling Introduced in 2007
Joint Bilateral Upsampling, Kopf et al, SIGGRAPH 2007.
Joint Bilateral Upsampling, Kopf et al, SIGGRAPH 2007.
Nearest neighbor upsampling Bicubic upsampling
Upsampled result Gaussian upsampling Joint bilateral upsampling
Wilf LaLonde ©2012Comp 4501
source context
Intuition: Two Filters (weights) Instead of OneIntuition: Two Filters (weights) Instead of One
Sample (p) = 1
kp q
∑ Sq f(||p-q||) g(||Ip-Iq||)
Normal filtering 1
kp q
∑ Sq W(||p-q||)
Sample at q
Filter kernel support (filter coordinates)
Weight depends on distance to reference
Normalizing factor (sum of weights)
Essentially multiply the 2 weights together
Wilf LaLonde ©2012Comp 4501
High resolution low resolution
This Special ApplicationThis Special Application
• Combining a high res image with a low res version that contains more distant information in the same size filter.
• It is a “natural” edge preserving filter... (one of our hints for obtaining a better result).
Like using two 3x3 filters instead of one 6x6 filter
Wilf LaLonde ©2012Comp 4501
The Formalism looks like this (which we’ll ignore)The Formalism looks like this (which we’ll ignore)
Upsamplep = 1
kp q
∑ Lq f(||p-q||) g(|Hp-Hq||)
Low resolution L
Let p and q denote integer coordinate of
pixels in H
Let p and q denote corresponding fractional coordinate of pixels in L
High resolution HUpsampled
version
Wilf LaLonde ©2012Comp 4501
Bilateral Upsampling in MSSAO (Inventor’s Slides)Bilateral Upsampling in MSSAO (Inventor’s Slides)
• Filter weights wf (pi)
• Depth weights wz
wz(pi) = ( )• Normal weights wn
wn(pi) = ( )
tz1
1 + |zi – z|
tn n.ni + 1
2
Low def p1, p2, p3, p4 for high def p tz = 16, tn = 8
Basic idea: make weights go to 0 as
samples differ more (fraction < 1)
AO (p) = wf (pi) wz(pi) wn(pi) AO (pi) ∑ l = 1
4
Wilf LaLonde ©2012Comp 4501
• We’re done with bilinear filters but not done with filters (we need yet another type).
• Why not have a quick look at some well known filters.
Since We Need Another Filter Type… Since We Need Another Filter Type…
Switch to slides “#02bFilters.ppt”
Wilf LaLonde ©2012Comp 4501
Back to Our 3 Shader GroupsBack to Our 3 Shader Groups
SetupGeometryProgram (initial MRT textures)
geometry.vert geometry.frag
ortho.vertdownsample.frag
SetupDownsamplePrograms (lower resolution MRT textures)
SetupAOPrograms (upsampling into AO texture)ortho.vertaoWilf.frag
MRT has a position texture + normal texture
Seen this one
Seen this one
Let’s look at this one
Render2GBuffer
DownsampleGBuffer
RenderAO
Wilf LaLonde ©2012Comp 4501
95.450195.4501
Upsampling into
an AO texture(RenderAO)
Wilf LaLonde ©2012Comp 4501
#version 140#extension GL_ARB_texture_rectangle: enable#extension GL_EXT_gpu_shader4: enable
#include "wilfAmbientOcclusionLibrary.all"
uniform sampler2DRect normTex;uniform sampler2DRect posTex;//And a host of other variables (not shown)…
uniform bool usePoisson; uniform bool useUpsampling; uniform bool useTemporalSmoothing;
out vec4 AO;
//Local variables accessed by 3 functions highResolutionOcclusion,//lowResolutionOcclusion, temporallyBlendedOcclusion...vec2 uv; vec4 basePosition; vec3 baseNormal; //To be filled in...
//3 functions and main (shown next)…
Initial RenderAOInitial RenderAO
We use one shader aoWilf.frag rather than 3: aoLast.frag, ao.frag, aoFirst.frag
Wilf LaLonde ©2012Comp 4501
//It's convenient for 2 occlusion functions below to return the result in pieces;//namely vec3 (occlusion / occlusionCount, occlusion, occlusionCount)...
vec3 highResolutionOcclusion () {return usePoisson //Fills in basePosition, baseNormal…
? poissonFilterAmbientOcclusion (...): gridFilterAmbientOcclusion (...);
}
vec3 lowResolutionOcclusion () {return (useUpsampling)
? vec3 (0, 0, 0); //edgePreserveUpsample (...): vec3 (0, 0, 0); //None...
}
float temporallyBlendedOcclusion (float occlusion) {return useTemporalSmoothing
? occlusion //temporalBlend (...): occlusion;
}
Initial RenderAOInitial RenderAO
We will shortly override to INITIALLY use simpler
routines
Effectively “do nothing”for now.
Effectively “do nothing”for now.
Wilf LaLonde ©2012Comp 4501
void main () { uv = gl_FragCoord.xy; //Initialize the variable containing the texture
coordinates...
//Sample format: vec3 (occlusion / occlusionCount, occlusion, occlusionCount)... vec3 sample1 = highResolutionOcclusion (); vec3 sample2 = lowResolutionOcclusion ();
float maximum = max (sample1.x, sample2.x); float sum = sample1.y + sample2.y;float count = sample1.z + sample2.z;
if (usePoisson) {//Top levelfloat weightedAverage = sum / count;float ambientOcclusion = (1.0 - maximum) * (1.0 - weightedAverage); AO = vec4 (temporallyBlendedOcclusion (ambientOcclusion));
} else {AO = vec4 (maximum, sum, count, 0.0);
}}
Initial RenderAOInitial RenderAO
Since we are drawing; high occlusion 1 is meant to be black (0) and low occlusion 0 is meant to be white (1) (flip via 1 - occlusion).
AO here is really AL (ambient lighting)
Convert to gray scale (all 4 components the same)
Flip
Wilf LaLonde ©2012Comp 4501
The LibraryThe Library
//Wilf's library for ambient occlusion experiments...
//interpolate (a, b, t) gives a for t = 0 and b for t = 1 and blends otherwise...
#define interpolate mix
#define clamp01(x) clamp (x, 0.0, 1.0)
Let’s look at the first routine for computing ambient occlusion
Wilf LaLonde ©2012Comp 4501
Compute ambient occlusion as “high occlusion if far, low occlusion if near”.
To Get Started With The LibraryTo Get Started With The Library
//Return the result in pieces (occlusion / occlusionCount, occlusion, occlusionCount)...
vec3 HACKambientOcclusion (vec2 uv, out vec4 basePosition, out vec3 baseNormal, sampler2DRect positionTexture, sampler2DRect normalTexture) {
basePosition = texture2DRect (positionTexture, uv);baseNormal = texture2DRect (normalTexture, uv).xyz;
//Compute t = 0 => close when z = 0; t = 1 => far when z = 100.float t = clamp01 (abs (basePosition.z) / 100.0);
float sampleOcclusion = t; //0 => close; 1 => farfloat sampleCount = 1.0;return vec3 (sampleOcclusion / sampleCount, sampleOcclusion, sampleCount);
}
Back to Visual Studio to invoke it in highResolutionOcclusion
Wilf LaLonde ©2012Comp 4501
ResultsResults
Wilf LaLonde ©2012Comp 4501
• Recall: As part of the AO definition which we’ll review next, we’ll compute
Adding More Correct Ambient Occlusion FunctionAdding More Correct Ambient Occlusion Function
float squaredLength (vec3 vector) {return dot (vector, vector);}
float squaredDistance (vec3 point1, vec3 point2) {return squaredLength (point1 - point2);}
float linearDistanceRatio (vec3 distanceVector, float dMaximum) {
float d = length (distanceVector);
return min (d / dMaximum, 1.0); //Clamp at 1 if bigger than dMaximum...
}
float squaredDistanceRatio (vec3 distanceVector, float dMaximum) {
float dSquared = squaredLength (distanceVector);
return min (dSquared / (dMaximum * dMaximum), 1.0); //Clamp at 1 if past dMaximum...
}
[1 - (di / dmax)] cos i
Wilf LaLonde ©2012Comp 4501
Technically It’s A Volume ContributionTechnically It’s A Volume Contribution
Occlusion is cosine-weighted fraction of a tangent hemisphere
Minimal occlusion = 0 (no contribution, denoted ) for sample at or outside radius of influence or on negative side of normal)
[1 - (di / dmax)] cos i
At base point p, di is 0.So we’ll get 0 occlusionif we use the base point
The AO Definition
Wilf LaLonde ©2012Comp 4501
The Definition Based AO FunctionThe Definition Based AO Function
float ambientOcclusion (vec4 basePosition, vec3 baseNormal, vec4 position, vec4 normal, float cameraSpaceRadius) {//AO = ambientOcclusion
//Compute AO of {position, normal} relative to {basePosition, baseNormal}.//Note that normal is a vec4 (4th component 0 means distance to eye is 0)...
//Compute the direction vector from the base point to the other point.vec3 distanceVector = position.xyz - basePosition.xyz;
//Compute the distance ratio clamped at dMaximum...float distanceRatio = linearDistanceRatio (distanceVector, cameraSpaceRadius);
//As the ratio goes from 0 to 1, go from 1 to 0 instead.float occlusionFactor = 1.0 - distanceRatio;
//Need POSITIVE ONLY cosine contribution. Note A.B = |A| |B| cos angle if |A| = |B| = 1...float cosine = max (dot (baseNormal, normalize (distanceVector)), 0.0);return occlusionFactor * cosine * normal.w; //Use 0 for a point at the near plane...
}
[1 - (di / dmax)] cos i
Recall: LAST LINE OF Geometry.frag for creating MRT texture.… out vec4 Pos; out vec4 Norm; …vec3 n = -cross (dFdx(pos.xyz), dFdy(pos.xyz));Norm = vec4(normalize(n), sign(abs(pos.z))); } 0 in 4th component if z
is 0; 1 otherwise
Wilf LaLonde ©2012Comp 4501
Version with Samplers Instead of SamplesVersion with Samplers Instead of Samples
float ambientOcclusion (vec2 uv, vec4 basePosition, vec3 baseNormal, sampler2DRect positionTexture, sampler2DRect normalTexture, float cameraSpaceRadius) {
//Version with samplers instead of {position, normal}...
vec4 position = texture2DRect (positionTexture, uv);vec4 normal = texture2DRect (normalTexture, uv);
return ambientOcclusion (basePosition, baseNormal, position, normal, cameraSpaceRadius);
}
Back to Visual Studio
Note: with 1 probe, if position == basePosition, di will be 0
So occlusion will be 1 which will draw as 1-1 (black)
[1 - (di / dmax)] cos i
Wilf LaLonde ©2012Comp 4501
Result: As Expected, Occlusion is 0Result: As Expected, Occlusion is 0
Wilf LaLonde ©2012Comp 4501
Making a Small ChangeMaking a Small Change
uv += 1; //So sample point is right + up from base uv...
Wilf LaLonde ©2012Comp 4501
• One probe is not enough... But there are 2 issues to resolve.
• How many probes to take. Hoang and Low take 16 probes per sample.
• At the sample point, the probe in camera space has a fixed radius of influence called “cameraSpaceRadius”. We probe a full screen texture and so we need the corresponding “pixelSpaceRadius”.
But We Want A Poisson FilterBut We Want A Poisson Filter
Note: By sampling at the same resolution far away as close by, discontinuities are sharp at all distances.
Wilf LaLonde ©2012Comp 4501
const float dMax = 2.5; //Radius of influence in camera space (meters).
const float rMax = 7.0; //Radius of influence in pixel space (pixels).
const float fovFactor; //To be derived (and to show why needed).
Variables Actually Initialized At Shader Setup TimeVariables Actually Initialized At Shader Setup Time
dMax (meters)rMax (pixels)
Wilf LaLonde ©2012Comp 4501
float pixelFromCameraSpace (float cameraSpaceRadius, float resolution, float fovFactor, float depth) {return (resolution * fovFactor / depth) * cameraSpaceRadius;
}
All We Want Is A Simple Conversion RoutineAll We Want Is A Simple Conversion Routine
Let fovFactor = 0.5 / tan (/2) where is field of view
Slightly different from original version which plugged the resolution INTO the factor and called it
resolutionDependentFactor
Wilf LaLonde ©2012Comp 4501
3 Spaces Involved for RADIUS: Camera/Screen/Pixel Spaces3 Spaces Involved for RADIUS: Camera/Screen/Pixel Spaces
rscreenSpacercameraSpace
n = focal length (near)
= fov (field of view)
h
CAMERA SPACE: in meters
Consider radius relative to center point (more exact result not useful).
SCREEN SPACE: 0 to 1 units
PIXEL SPACE: 0 to 1024 units(related to half RESOLUTION
and therefore )
Wilf LaLonde ©2012Comp 4501
Differences between Camera/Screen/Pixel SpacesDifferences between Camera/Screen/Pixel Spaces
rscreenSpacercameraSpace
n = focal length (near)
= fov (field of view)
hrpixelSpace
How are rpixelSpace, rscreenSpace and rcameraSpace related?
dMax(meters)rMax(pixels)
If sample is very close to screen rpixekSpace can exceed a rMax.
Wilf LaLonde ©2012Comp 4501
Step 1: Camera/Screen SpacesStep 1: Camera/Screen Spaces
rscreenSpacercameraSpace
n = focal length (near)
= fov (field of view)
hrpixelSpace
By similar triangles, rscreenSpace/n = rcameraSpace/|z|.
|z|
rscreenSpace = (n/|z|) * rcameraSpace
Wilf LaLonde ©2012Comp 4501
Derivation: h screen space units = R/2 pixels
1 screen space unit = (R/2)/h pixels
But h/n = tan (/2)
So 1 screen space unit = (R/2)/(n tan (/2)) pixels
So r screen space unit = (R/2)/(n tan (/2)) * r pixels
Step 2: Screen/Pixel SpacesStep 2: Screen/Pixel Spaces
rscreenSpacercameraSpace
n
h
Let R be resolution, h be half screen size.
rpixelSpace = (R/2)/(n tan (/2)) * rscreenSpace
Wilf LaLonde ©2012Comp 4501
Step 3: Everything put togetherStep 3: Everything put together
rscreenSpace = (n/|z|) * rcameraSpace
rpixelSpace = (R/2)/(n tan (/2)) * rscreenSpace
rpixelSpace = (R/2)/(n tan (/2)) * (n/|z|) * rcameraSpace
Let fovFactor = 0.5 / tan (/2)
rpixelSpace = (R fovFactor / |z|) * rcameraSpace
float pixelFromCameraSpace (float cameraSpaceRadius, float resolution, float fovFactor, float depth) {return (resolution * fovFactor / depth) * cameraSpaceRadius;
}
Wilf LaLonde ©2012Comp 4501
The Poisson FilterThe Poisson Filtervec3 poissonFilterAmbientOcclusion (vec2 uv, sampler2DRect positionTexture, sampler2DRect normalTexture, float maximumPixelSpaceRadius,
float cameraSpaceRadius, float fovFactor, float resolution, out vec4 basePosition, out vec3 baseNormal) {
//Filter via a poisson disk and return the result in pieces //i.e., (occlusion / occlusionCount, occlusion, occlusionCount)...
//We need the maximum pixel space radius for the poisson disk and the camera space//radius for the occlusion sphere's radius of influence...
const vec2 poissonDisk [16] = …
basePosition = texture2DRect (positionTexture, uv); baseNormal = texture2DRect (normalTexture, uv).xyz;
float pixelSpaceRadius = pixelFromCameraSpace (cameraSpaceRadius, resolution, fovFactor, abs (basePosition.z));
float radius = min (pixelSpaceRadius, maximumPixelSpaceRadius);
float occlusion = 0.0; for (int i = 0; i < 16; i++) {
vec2 randomUV = uv + poissonDisk [i] * radius;occlusion += ambientOcclusion (randomUV, basePosition, baseNormal,
positionTexture, normalTexture, cameraSpaceRadius);}return vec3 (occlusion * (1.0 / 16.0), occlusion, 16.0);
}
Next slide for details
Note: By sampling at the same resolution far away as close by, discontinuities are sharp at all distances.
Is this a trick?.
Wilf LaLonde ©2012Comp 4501
The Poisson FilterThe Poisson Filter
const vec2 poissonDisk [16] = vec2 [16] (
vec2 (-0.6116678, 0.04548655), vec2 (-0.26605980, -0.6445347),vec2 (-0.4798763, 0.78557830), vec2 (-0.19723210, -0.1348270),vec2 (-0.7351842, -0.58396650), vec2 (-0.35353550, 0.3798947),vec2 ( 0.1423388, 0.39469180), vec2 (-0.01819171, 0.8008046),vec2 ( 0.3313283, -0.04656135), vec2 ( 0.58593510, 0.4467109),vec2 ( 0.8577477, 0.11188750), vec2 ( 0.03690137, -0.9906120),vec2 ( 0.4768903, -0.84335800), vec2 ( 0.13749180, -0.4746810),vec2 ( 0.7814927, -0.48938420), vec2 ( 0.38269190, 0.8695006));
Back to Visual Studio
We now have enough to draw at the high resolution level
Wilf LaLonde ©2012Comp 4501
Using the Poisson FilterUsing the Poisson Filter
Wilf LaLonde ©2012Comp 4501
• We use a diamond grid filter (right and up for simplicity) at odd points 1, 3, 5, ... up to radius of influence.
We Need A Much Simpler Filter To Probe AO at Lower ResolutionsWe Need A Much Simpler Filter To Probe AO at Lower Resolutions
Half the probes of the poisson filter...
Wilf LaLonde ©2012Comp 4501
The Diamond Grid FilterThe Diamond Grid Filtervec3 gridFilterAmbientOcclusion (vec2 uv, sampler2DRect positionTexture,
sampler2DRect normalTexture, float maximumPixelSpaceRadius, float cameraSpaceRadius, float fovFactor, float resolution,out vec4 basePosition, out vec3 baseNormal) {
//Sampling right and up at all odd pixel distances; i.e., 1, 3, 5, ... to the limit allowed//and return the pieces (occlusion / occlusionCount, occlusion, occlusionCount)...
basePosition = texture2DRect (positionTexture, uv); baseNormal = texture2DRect (normalTexture, uv).xyz;
float pixelSpaceRadius = pixelFromCameraSpace (cameraSpaceRadius, resolution, fovFactor, abs (basePosition.z));
float radius = min (pixelSpaceRadius, maximumPixelSpaceRadius);
So far, exactly the same as the poisson filter
Wilf LaLonde ©2012Comp 4501
The Diamond Grid FilterThe Diamond Grid Filter
int occlusionCount = 0; float occlusion = 0.00001; //Prevent divide by 0 below...
for (float x = 1.0; x <= radius; x += 2.0) {for (float y = 1.0; y <= radius; y += 2.0) {
occlusion += ambientOcclusion (uv + vec2 (x, y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);
occlusion += ambientOcclusion (uv + vec2 (-x, y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);
occlusion += ambientOcclusion (uv + vec2 (-x, -y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);
occlusion += ambientOcclusion (uv + vec2 (x, -y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);
occlusionCount += 4;}
}return vec3 (occlusion / occlusionCount, occlusion, occlusionCount);
}
Recall: rMax = 7.0;So x can be 1, 3, 5, 7
Wilf LaLonde ©2012Comp 4501
Need an Upsample HACK To Test This OutNeed an Upsample HACK To Test This Out
vec3 HACKedgePreserveUpsample (vec2 highRESuv, vec4 basePosition, vec3 baseNormal, sampler2DRect lowResNormTex, sampler2DRect lowResPosTex, sampler2DRect lowResAOTex) {
//lowResAOTex is stored as (occlusion / occlusionCount, occlusion, occlusionCount)...
vec2 lowResUV = floor ((highRESuv) * 0.5) + vec2 (0.5, 0.5);vec3 lowResAO = texture2DRect (lowResAOTex, loResUV).xyz;return lowResAO;
}
Back to Visual Studio
Divide high res by 2 + half a pixel to get low res
Note to wilf: Need to adjust “#define LEVEL_COUNT 1”.
Wilf LaLonde ©2012Comp 4501
HACK Upsampling with Different LEVEL_COUNT SettingsHACK Upsampling with Different LEVEL_COUNT Settings
1 2 3 4 5
Very small improvements (need to replace HACK…upsample by better version)
Wilf LaLonde ©2012Comp 4501
• The code is a rewrite with no changes except for merging 4 loops into one... It’s computing a weighted sum of the vec3 lower resolution AO samples; i.e.,
A Better Edge Preserving BILATERAL Upsampler A Better Edge Preserving BILATERAL Upsampler
Vec3 C = vec3 (0);C += sample1 * W1
C += sample2 * W2
...Return C / W
A sample is a triple{occlusion/count, occlusion, count}
float weight = normalWeight * depthWeight * (9.0 / 16.0) /
(abs((highRESuv.x - loResUV.x * 2.0) * (highRESuv.y - loResUV.y * 2.0)) * 4.0);
Each low resolution sample provided it own normalWeight
and depthWeight
Wilf LaLonde ©2012Comp 4501
It’s Based On The Following BUT uses 9/16 everywhereIt’s Based On The Following BUT uses 9/16 everywhere
• Filter weights wf (pi)
• Depth weights wz
wz(pi) = ( )• Normal weights wn
wn(pi) = ( )
tz1
1 + |zi – z|
tn n.ni + 1
2
Low def p1, p2, p3, p4 for high def p tz = 16, tn = 8
Basic idea: make weights go to 0 as
samples differ more (fraction < 1)
AO (p) = wf (pi) wz(pi) wn(pi) AO (pi) ∑ l = 1
4
Wilf LaLonde ©2012Comp 4501
The Edge Preserving UpsamplerThe Edge Preserving Upsamplervec3 edgePreserveUpsample (vec2 highRESuv, vec4 basePosition, vec3 baseNormal,
sampler2DRect loResNormTex, sampler2DRect loResPosTex, sampler2DRect loResAOTex) {
//Downsampled values should be almost equal to basePosition and baseNormal...//So weigths will be approximately 1 most of the time and 0 when they are different...
const vec2 offsets [4] = vec2 [4] (vec2 (-1.0, 1.0), vec2 ( 1.0, 1.0), vec2 (-1.0, -1.0), vec2 ( 1.0, -1.0));
float totalWeight = 0.0; vec3 combinedAO = vec3 (0.0);
for (int i = 0; i < 4; ++i) {vec2 loResUV = floor ((highRESuv + offsets [i]) * 0.5) + vec2 (0.5, 0.5);
vec3 loRESNormal = texture2DRect (loResNormTex, loResUV).xyz;float loResDepth = texture2DRect (loResPosTex, loResUV).z;vec3 loResAO = texture2DRect (loResAOTex, loResUV).xyz;
//COMPUTE normalWeight and depthWeight (see next slide).
float weight = normalWeight * depthWeight * (9.0 / 16.0) /
(abs((highRESuv.x - loResUV.x * 2.0) * (highRESuv.y - loResUV.y * 2.0)) * 4.0);
totalWeight += weight; combinedAO += loResAO * weight; }
return combinedAO / totalWeight;}
Wilf LaLonde ©2012Comp 4501
The Edge Preserving UpsamplerThe Edge Preserving Upsampler
//Compute 1 if almost equal, approaches 0 otherwise...
float normalWeight = (dot (loRESNormal, baseNormal) + 1.1) / 2.1;
normalWeight = pow (normalWeight, 8.0);
//Compute 1 if almost equal, approaches 0 otherwise...
float depthWeight = 1.0 / (1.0 + abs (basePosition.z - loResDepth) * 0.2);
depthWeight = pow (depthWeight, 16.0);
Back to Visual Studio
Wilf LaLonde ©2012Comp 4501
author version
Result Are Different But Hard To See HereResult Are Different But Hard To See Here
hack version
Wilf LaLonde ©2012Comp 4501
author version
Zooming InZooming In
hack version
Wilf LaLonde ©2012Comp 4501
• Now’s the time to try the simplistic volumetric occlusion technique...
Now Everything Should WorkNow Everything Should Work
float simplisticAmbientOcclusion (vec4 basePosition, vec3 baseNormal, vec4 position, vec4 normal, float cameraSpaceRadius) {
//Computes the ambient occlusion of {position, normal} //relative to {basePosition, baseNormal}.
//Note that z is negative in OpenGL.
bool furtherOut = position.z < basePosition.z;
return furtherOut ? 0.0 /*no occlusion */ : 1.0; /* full occlusion */
}
no occlusion full occlusion
Wilf LaLonde ©2012Comp 4501
One Pass Simplistic AlgorithmOne Pass Simplistic Algorithm
Wilf LaLonde ©2012Comp 4501
One Pass Sophisticated AlgorithmOne Pass Sophisticated Algorithm
Wilf LaLonde ©2012Comp 4501
5-Pass Simplistic Algorithm5-Pass Simplistic Algorithm
Wilf LaLonde ©2012Comp 4501
5-Pass Sophisticated Algorithm5-Pass Sophisticated Algorithm
Wilf LaLonde ©2012Comp 4501
• The observation that very little changes from frame to frame is called temporal coherence (TC).
• Rapid changes imply rapid movement... This can be exploited in 2 ways:
Avoid recomputing what never changes to achieve a speedup...
Use previous information to provide temporal blurring...
Temporal CoherenceTemporal Coherence
Wilf LaLonde ©2012Comp 4501
panning left
Temporal Coherence ExamplesTemporal Coherence Examples
A Survey on Temporal Coherence Methods in Real-Time Rendering, Scherzer et al, Eurographics 2011
A Survey on Temporal Coherence Methods in Real-Time Rendering, Scherzer et al, Eurographics 2011
AI walking left
green is visible in previous scene; red is new
Wilf LaLonde ©2012Comp 4501
Applications of Temporal CoherenceApplications of Temporal Coherence
• Bypass all or part of computation of complex shader whenever previous results available.
Wilf LaLonde ©2012Comp 4501
Applications of Temporal CoherenceApplications of Temporal Coherence
• Eliminate flickering artifacts by interpolating old with new in non-photorealistic rendering (which created many moving lines).
Wilf LaLonde ©2012Comp 4501
• An important decision when utilizing TC is how the previously computed data is stored, tracked, retrieved and reused.
• Given a current pixel location, determining the previous location associated with it is termed reverse reprojection.
Temporal CoherenceTemporal Coherence
Wilf LaLonde ©2012Comp 4501
Reverse ReprojectionReverse Reprojection
A Survey on Temporal Coherence Methods in Real-Time Rendering, Scherzer et al, Eurographics 2011
A Survey on Temporal Coherence Methods in Real-Time Rendering, Scherzer et al, Eurographics 2011
reverseReprojected (P1) at time t
gives P1 at time t-1 denoted t-1(P1).
inconsistent depths or offscreen indicates no solution
Wilf LaLonde ©2012Comp 4501
Sample = SampleNew (PNew).interpolate (SampleOld (POld), t)
where confidence t (how confident you are in PNew) is between 0 and 1; e.g, 0.6
Performing Reverse ReprojectionPerforming Reverse Reprojection
inverse
Use weighting t to determine how much of the new sample to use compared with the old...
at time t-1 at time t
POld = MVMatrixt-1 (MVMatrixt (PNew))
Wilf LaLonde ©2012Comp 4501
float temporalBlend (float confidence, float resolution, float ambientOcclusion, vec4 position, sampler2DRect oldPositionTexture, sampler2DRect oldAOTexture, mat4 projection, mat4 modelViewInverseNow, mat4 modelViewOld) {
//Blend ambient occlusion at position with version in old position/AO //textures using the transformations to perform the reverse reprojection... //Note that position in camera (view) space...
//Compute the old position from the model view transformations.vec4 oldPosition = modelViewOld * (modelViewInverseNow * position);
//To get screen space UV, we need to convert to perspective space which //gives coordinate in range -1 to +1...vec4 oldScreenSpacePosition = projection * oldPosition;vec2 oldUV = oldScreenSpacePosition.xy / oldScreenSpacePosition.w;
Temporal CoheranceTemporal Coherance
Wilf LaLonde ©2012Comp 4501
//float temporalBlend (...) CONTINUEDif (oldUV.x < -1.0 || oldUV.x > +1.0 || oldUV.y < -1.0 || oldUV.y > +1.0)
return ambientOcclusion; //If offscreen, return the new value...//These coordinates are from -1 to +1 (not 0 to R-1) WHERE R = resolution...oldUV += 1.0; //Now they range from +0 to +2.oldUV *= (resolution - 1.0) * 0.5; //Now from +0 to +2*(R-1)*0.5; i.e., 0 to R-1.oldUV += 0.5; //Add half a pixel to be at the center of the pixel.
//Use stored position.z which may be more accurate than oldPosition.z.float oldPositionZ = texture2DRect (oldPositionTexture, oldUV).z;if (abs (1.0 - oldPositionZ / position.z) > 0.01)
return ambientOcclusion; //Too different to use...
//Use the confidence t value to determine the blend.. ;1 is all new, 0 all old.float oldAmbientOcclusion = texture2DRect (oldAOTexture, oldUV).x;return interpolate (ambientOcclusion, oldAmbientOcclusion, confidence);
}
Temporal CoheranceTemporal Coherance
Wilf LaLonde ©2012Comp 4501
• The Alchemy Screen-Space Ambient Obscurance Algorithm, McGuire, Osman, Bukowski, Hennessay, High Performance Graphics 2011.
• Temporal Screen-Space Ambient Occlusion, Mattausch, Schernzer, and Wimmer, pp 123-141, GPU Pro 2, 2011.
• Screen Space Ambient Occlusion, Kajalin (Crysis), pp 413-424, ShaderX7, 2009.
• Image-Space Horizon-Based Ambient Occlusion (NVIDIA demo), Bavoil and Sainz, pp 425-444, ShaderX7, 2009.
ReferencesReferences
Wilf LaLonde ©2012Comp 4501
• Volumetric Obscurance, Loos and Sloan, Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, 1010.
• Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on Virtual Reality Software and Technology 2010.
• Joint Bilateral Upsampling, Kopf et al, SIGGRAPH 2007.
ReferencesReferences