J. KrügerJ. KrügerR. WestermannR. Westermann
computer graphics & computer graphics & visualizationvisualization
Technical University MunichTechnical University Munich
Acceleration Techniques for Acceleration Techniques for GPU-based Volume RenderingGPU-based Volume Rendering
computer graphics & computer graphics & visualizationvisualization
MotivationMotivation
Properties of real data setsProperties of real data sets
Surface like Surface like structuresstructuresSurface like Surface like structuresstructures
Images courtesy of Joe Kniss and volren.orgImages courtesy of Joe Kniss and volren.org
OcclusionOcclusionssOcclusionOcclusionss
Surface illuminationSurface illuminationSurface illuminationSurface illumination
computer graphics & computer graphics & visualizationvisualization
MotivationMotivation
Texture based volume rendering today:Texture based volume rendering today:
Hardware accelerated rendering
of totally invisible structures
Hardware accelerated rendering
of totally invisible structures
only 2% of all fragments visibleonly 2% of all fragments visible
computer graphics & computer graphics & visualizationvisualization
Our ContributionOur Contribution
We use new GPU Features to:We use new GPU Features to:
• Discard invisible fragmentsDiscard invisible fragments• early ray terminationearly ray termination• empty space skippingempty space skipping
• Implement a high quality GPU RaycasterImplement a high quality GPU Raycaster
• Accelerate the RaycasterAccelerate the Raycaster
computer graphics & computer graphics & visualizationvisualization
Our ToolkitOur Toolkit
Programmable graphics hardwareProgrammable graphics hardware
• ““Arbitrarily” programmable shadersArbitrarily” programmable shaders• Assembly languageAssembly language• High level languages High level languages
• 32 Bit-Floating-point processing32 Bit-Floating-point processing• Input:Input: fp32 texturesfp32 textures• Arithmetic:Arithmetic: fp24-fp32fp24-fp32• Output:Output: fp32 render targetsfp32 render targets
• Early Z-TestEarly Z-Test• Multiple dependent texture fetchesMultiple dependent texture fetches
computer graphics & computer graphics & visualizationvisualization
Stream ArchitectureStream Architecture
Our ToolkitOur Toolkit
RasterizerRasterizer
Early Z-TestEarly Z-Test
GeometryGeometry
FragmentsFragments
Fragment ShaderFragment Shader
Render TextureRender Texture
User TexturesUser Textures
……
……
……
computer graphics & computer graphics & visualizationvisualization
RaycasterRaycaster
Slicing (SBVR) vs. GPU-Raycaster (RC)Slicing (SBVR) vs. GPU-Raycaster (RC)
SBVR SBVR render proxy geometry (slices) render proxy geometry (slices)
RC RC render proxy geometry (faces) render proxy geometry (faces)
computer graphics & computer graphics & visualizationvisualization
RaycastingRaycasting
Using only Using only oneone main pass for Raymarching main pass for Raymarching
computer graphics & computer graphics & visualizationvisualization
Basic IdeaBasic Idea
Multi-pass approachMulti-pass approach
• Generate entry pointGenerate entry point• first hit in local texture coordinatesfirst hit in local texture coordinates
• Compute ray directionCompute ray direction• ray direction in local texture coordinatesray direction in local texture coordinates
• Ray Marcher (Ray Marcher (simplifiedsimplified))• Main pass (Ray traversal)Main pass (Ray traversal)• Intermediate pass (Stopping criterion)Intermediate pass (Stopping criterion)
computer graphics & computer graphics & visualizationvisualization
Entry PointEntry Point• Render into 2D textureRender into 2D texture
• Front facesFront faces of the volume bounding box of the volume bounding box• Texture coordinates as color componentsTexture coordinates as color components
(1,1,1)
(0,1,0)
(0,0,1)
(1,0,1)
(1,0,0)
computer graphics & computer graphics & visualizationvisualization
Ray DirectionRay Direction• render into second texturerender into second texture
• back facesback faces of the bounding box of the bounding box• subtract from entry-point texture to get ray directionsubtract from entry-point texture to get ray direction
(0,1,0)
(1,0,1)
(0,0,1)
(0,0,0)
(1,0,0)
(1,1,1)
(0,1,0)
(0,0,1)
(1,0,1)
(1,0,0)
subtractsubtract
computer graphics & computer graphics & visualizationvisualization
Basic IdeaBasic Idea
Multi-pass approachMulti-pass approach
• Generate entry pointGenerate entry point• first hit in local texture coordinatesfirst hit in local texture coordinates
• Compute ray directionCompute ray direction• ray direction in local texture coordinatesray direction in local texture coordinates
• Ray MarcherRay Marcher• Main passes (Ray traversal)Main passes (Ray traversal)• Intermediate pass (Stopping criterion)Intermediate pass (Stopping criterion)
computer graphics & computer graphics & visualizationvisualization
RaycastingRaycasting
Using multiple Raymarching passesUsing multiple Raymarching passes
computer graphics & computer graphics & visualizationvisualization
Ray Marcher (Main passes)Ray Marcher (Main passes)
We know: We know: Entry pointEntry point and and ray directionray direction
float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;
float3 fFrontVal = v.Color.rgb;float3 fFrontVal = v.Color.rgb;
float3 fVolCoord = fFrontVal+fDepth*fDirection;float3 fVolCoord = fFrontVal+fDepth*fDirection;
float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;
float3 fFrontVal = v.Color.rgb;float3 fFrontVal = v.Color.rgb;
float3 fVolCoord = fFrontVal+fDepth*fDirection;float3 fVolCoord = fFrontVal+fDepth*fDirection;
Render front-facesRender front-faces• set Color like before set Color like before CC• set Texture coords to NDCset Texture coords to NDC (x,y)(x,y)• activate direction textureactivate direction texture DIRDIR• a global counter a global counter tt
compute ray as: r(t)=compute ray as: r(t)=CC+t*+t*DIR(x,y)DIR(x,y)
computer graphics & computer graphics & visualizationvisualization
Ray Marcher (Main passes)Ray Marcher (Main passes)
[..][..]
for (int i=0;i<N;i++) {for (int i=0;i<N;i++) {
fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);
fVolCoord += fStepSize*fDirection;fVolCoord += fStepSize*fDirection;
}}
return fResultVal; return fResultVal;
[..][..]
for (int i=0;i<N;i++) {for (int i=0;i<N;i++) {
fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);
fVolCoord += fStepSize*fDirection;fVolCoord += fStepSize*fDirection;
}}
return fResultVal; return fResultVal;
Integration along the rayIntegration along the ray
• use use r(t)r(t) to access the volume to access the volume• integrate over multiple steps (N)integrate over multiple steps (N)• combine with frame buffer contentscombine with frame buffer contents
computer graphics & computer graphics & visualizationvisualization
RaycastingRaycasting
RaycastingRaycastingEarly Ray TerminationEarly Ray TerminationEmpty Space SkippingEmpty Space Skipping
Acceleration techniquesAcceleration techniques
computer graphics & computer graphics & visualizationvisualization
Acceleration TechniquesAcceleration Techniques
Terminate Rays on one of the following conditions:Terminate Rays on one of the following conditions:• it has left the volumeit has left the volume• it has reached a certain opacityit has reached a certain opacity
Suspend a ray if:Suspend a ray if:• it‘s traversing an empty regionit‘s traversing an empty region
this needs to be done on a per-fragment basisthis needs to be done on a per-fragment basis
early Z-Test early Z-Test
computer graphics & computer graphics & visualizationvisualization
Acceleration techniquesAcceleration techniques
Early-Z Restrictions:Early-Z Restrictions:
• on current GPUs early Z-Test only works ifon current GPUs early Z-Test only works if• no no clipclip or or texkilltexkill operation is executed operation is executed• the Z-value is not changed in the shaderthe Z-value is not changed in the shader
the raymarching shader can not terminate himselfthe raymarching shader can not terminate himself
• Solution: use intermediate pass to do the ray-Solution: use intermediate pass to do the ray-suspending/terminatingsuspending/terminating
computer graphics & computer graphics & visualizationvisualization
Acceleration techniquesAcceleration techniques• Execute intermediate pass after every main passExecute intermediate pass after every main pass
(= N volume samples)(= N volume samples)• Access z-value only in this passAccess z-value only in this pass• If ray is terminated or suspended set z-value If ray is terminated or suspended set z-value
beforebefore main-pass-geometry main-pass-geometry• Reset depth Reset depth behindbehind it if ray is to be resumed it if ray is to be resumed
computer graphics & computer graphics & visualizationvisualization
Ray Marcher (intermediate pass)Ray Marcher (intermediate pass)
for early ray terminationfor early ray termination : : check accumulated check accumulated alphaalpha
for empty space skippingfor empty space skipping : : check “skip-volume”check “skip-volume”
Skip volume:Skip volume:• volume sampled down to 1/8 in every dimensionvolume sampled down to 1/8 in every dimension• every sample contains the minimum and maximum every sample contains the minimum and maximum
of the 512 corresponding entriesof the 512 corresponding entries
Check TextureCheck Texture::• 2D lookup texture2D lookup texture• lookup(x,y)=1 iff all values between x and y are 0 lookup(x,y)=1 iff all values between x and y are 0
under the current transfer functionunder the current transfer function
computer graphics & computer graphics & visualizationvisualization
Ray Marcher (intermediate pass)Ray Marcher (intermediate pass)
float4 fLastValfloat4 fLastVal = tex2D(sLastFrame, v.TexCoords.rg);= tex2D(sLastFrame, v.TexCoords.rg);
float3 fDirection float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;= tex2D(sDirection, v.TexCoords.rg).rgb;
float3 fFrontVal float3 fFrontVal = v.Color.rgb;= v.Color.rgb;
float3 fVolCoord float3 fVolCoord = fFrontVal+fDepth*fDirection;= fFrontVal+fDepth*fDirection;
float2 fMaxMinfloat2 fMaxMin = tex3D(sEmptySpace, fVolCoord).rg;= tex3D(sEmptySpace, fVolCoord).rg;
float fEmptyfloat fEmpty = tex2D(sEmptyLookUp, fMaxMin);= tex2D(sEmptyLookUp, fMaxMin);
OUT.depthOUT.depth = ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;= ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;
OUT.col OUT.col = fLastVal;= fLastVal;
return OUT;return OUT;
float4 fLastValfloat4 fLastVal = tex2D(sLastFrame, v.TexCoords.rg);= tex2D(sLastFrame, v.TexCoords.rg);
float3 fDirection float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;= tex2D(sDirection, v.TexCoords.rg).rgb;
float3 fFrontVal float3 fFrontVal = v.Color.rgb;= v.Color.rgb;
float3 fVolCoord float3 fVolCoord = fFrontVal+fDepth*fDirection;= fFrontVal+fDepth*fDirection;
float2 fMaxMinfloat2 fMaxMin = tex3D(sEmptySpace, fVolCoord).rg;= tex3D(sEmptySpace, fVolCoord).rg;
float fEmptyfloat fEmpty = tex2D(sEmptyLookUp, fMaxMin);= tex2D(sEmptyLookUp, fMaxMin);
OUT.depthOUT.depth = ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;= ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;
OUT.col OUT.col = fLastVal;= fLastVal;
return OUT;return OUT;
for early ray terminationfor early ray termination : : check accumulated check accumulated alphaalpha
for empty space skippingfor empty space skipping : : check “skip-volume”check “skip-volume”
computer graphics & computer graphics & visualizationvisualization
DEMOSDEMOS
Run onRun on
•Intel Pentium IV 2.4GHz
•ATI Radeon 9800 Pro.
•Microsoft Windows XP
•DirectX 9
•Pixel/Vertex Shader 2.0
computer graphics & computer graphics & visualizationvisualization
Iso-Surface RaycastingIso-Surface Raycasting• first 2 passes and intermediate passes remain first 2 passes and intermediate passes remain
unchangedunchanged• global traversal order still front to backglobal traversal order still front to back• order within the main shader is back to frontorder within the main shader is back to front• keep only last isovaluekeep only last isovalue• if isovalue was found computeif isovalue was found compute
illumination with 3D gradientillumination with 3D gradienttexture and write it to thetexture and write it to thefinal imagefinal image
computer graphics & computer graphics & visualizationvisualization
ConclusionConclusion
Using current GPU features we showed how toUsing current GPU features we showed how to
• Discard invisible fragmentsDiscard invisible fragments• early ray terminationearly ray termination• empty space skippingempty space skipping
• Implement a high quality GPU RaycasterImplement a high quality GPU Raycaster
• Accelerate that RaycasterAccelerate that Raycaster
computer graphics & computer graphics & visualizationvisualization
Thank you!Thank you!
Questions?Questions?
Download Slides at: Download Slides at: http://wwwcg.in.tum.de/Research/Publications/Raycast/http://wwwcg.in.tum.de/Research/Publications/Raycast/
The EndThe End