No more texels, no more facets:Emerging trends in GPU procedural shading

Stefan Gustavson, Linkoping University, ITNstefan.gustavson@liu.se

AbstractProcedural textures have long been a staple of off-linerendering, and impressive creative results have been ac-complished by using procedural methods to their ad-vantage. As GPU speeds and computational capa-bilities continue to increase, procedural texturing willlikely become a useful tool also for real time render-ing. In fact, it is already possible to generate proce-dural patterns of considerable complexity at real timeframe rates on a modern GPU. Even on current (2013)low-end and mobile GPUs, the programmable shadingsystem offers considerable processing power that oftenremains largely unused. Such untapped resources couldbe used for procedural patterns and procedural geome-try computed entirely on the GPU.

This article presents the state of the art in the field.Most of this is yet to be implemented in commercialprojects like games, VR and visualization applications.Code for the shader examples in this article is availableunder permissive open source licenses or as public do-main software and is collected on the address:http://www.itn.liu.se/

˜

stegu76/sigrad13

1 History of procedural shadingThe concept of a surface shader, a small program tocompute the color of a surface point using the viewdirection, surface normal, material properties and in-formation on the scene illumination, has been aroundfor a large portion of the history of computer graphics.The concept was formalized in the commercial prod-uct Photorealistic RenderMan from Pixar, first releasedto the public in 1989 [11, 8]. While the RenderManInterface was made famous by its software implemen-tations, the original intent was to make a hardware ren-derer, and a prototype device named RM-1 was manu-factured and sold by Pixar for a few years in the 1980’s.The Academy award winning Pixar short film ”Luxo

Jr.” was rendered on that custom hardware.After Pixar abandoned their hardware renderer

project, hardware graphics rendering took off in adifferent direction with Silicon Graphics’ IrisGL andOpenGL, where real time performance took priorityover both image quality and flexibility. For a long timethere was a harsh disconnect in terms of image qualitybetween real time graphics, where a frame needs to berendered in fractions of a second, and cinematic pre-rendered graphics, where rendering times are allowedto extend to several hours per frame.

2 GPU procedural shadingGraphics hardware has now come to a point where theimage quality can match off-line rendering from abouta decade ago. Around 2004, programmable shadingwas introduced into the hardware rendering pipeline ofmainstream GPUs, which added considerable flexibil-ity. The line between real time graphics and off-linerendered graphics is blurring, and GPU rendering isnow successfully used not only for real time rendering,but also for acceleration of off-line rendering. In a way,we have now come full circle from Pixar’s RM-1 hard-ware in the 1980’s through a long tradition of softwarerendering, and back to a focus on hardware.

Instead of traditional fixed-function lights and mate-rials, with a predetermined, rigid computational struc-ture and a small set of parameters to control both thesurface appearance and the illumination, programmableshading has finally set real time graphics free of the sim-plistic illumination and surface reflection models fromthe 1980’s, and made it possible to explore more mod-ern concepts for real time lighting, shading and textur-ing.

To date, this new-found freedom has been usedmainly to explore better illumination and reflectionmodels, which was indeed an area in great need of im-

SIGRAD 2013, pp. 41-46T. Ropinski and J. Unger (Editors)

41

provement. However, another fascinating area remainslargely unexplored: procedural patterns. A good reviewand a long standing reference work in the field is [1].This article is an attempt to point to the many fascinat-ing possibilities in that area for real time applications.

3 Procedural patternsA procedural pattern is a function over spatial coordi-nates that for each point (u, v) on a surface describesits color, transparency, normal direction or other sur-face property as f(u, v). This function is defined overa continuous domain (u, v), not a discrete set of samplepoints, and contrary to a texture image it is computed di-rectly for each surface point for each frame, rather thansampled and stored for repeated use. While it mightseem wasteful and contrary to common sense in com-puting to throw away previous hard work and computeeach point anew for each surface point, it offers manyclear advantages:

• No memory is required, and no memory accessesare made. This saves bandwidth on the often con-gested memory buses of today’s massively parallelGPU architectures.

• Patterns can be computed at arbitrary resolution.

• Patterns can be changed by simple parameters.

• Animated patterns come at no extra cost.

• 3D and 4D functions can be used for texturing, re-moving the need for 2D texture coordinates.

• The shader program can perform its own analyticanti-aliasing.

Of course, there are also disadvantages:

• Not all patterns can be described by simple func-tions.

• Creating a good procedural pattern is hard work.

• The pattern must be programmed, which takes avery different skill than painting it or editing a dig-ital photo. Such skill is hard to find.

• Production pipelines for real time content arefirmly set in their ways of using texture images asassets.

• Current GPU hardware is designed to provide hugebandwidth for texture images, making it somewhatcounter-productive not to use them.

Procedural patterns are not a universal tool, and theyare not going to replace sampled texture images alto-gether. However, many animation and special effectsstudios have seen an advantage in using procedural pat-terns for many tasks, not least because of the ability toeasily tweak and change the appearance of any surfacein the scene late in the production process.

A modern GPU has a massively parallel architecturewith lots of distributed computing power, but with thou-sands of computing units sharing a common memoryof limited bandwidth. To counter the imbalance, localmemory is used to implement cache strategies, but factremains that memory-less procedural texturing is be-coming ever more attractive for real time use, at leastas a complement to traditional texturing. In a textureintensive shader, the execution speed is often limitedby memory access bandwidth, and the arithmetic capa-bilites of the GPU processing units are often not fullyutilized. Sometimes the computing power to generatea procedural pattern is available, but remains unused.Moreover, computations in local registers may be morepower efficient than accesses to global memory.

4 Simple patternsSome patterns are very suitable for description as func-tions. Stripes, spots, tiles and other periodic functionscan be described by a combination of modulo functionsand thresholding operations on the surface parameters(u, v) or directly on the object coordinates (x, y, z).Some examples with GLSL code are shown in Figure 1.

5 NoiseThe world around us is not clean and perfect. The natu-ral world is largely a result of stochastic processes, thesame sort of random variation is seen in manufacturedobjects due to process variations, dirt and wear. Rec-ognizing this, Ken Perlin created Perlin noise in the1980’s [10] as a tool for adding complexity and vari-ation to surfaces and models. Variations on his functionhave seen heavy use ever since in most off-line render-ing. Perlin noise has several nice and useful properties:

• It is fairly easy to compute.• It is band limited and reasonably isotropic, which

makes it suitable for spectral synthesis of morecomplex patterns.

• It has a well defined derivative everywhere.

42

S Gustavsson / No more texels, no more facets: Emerging trends in GPU procedural shading

smoothstep(0.4, 0.5, max( abs(fract(8.0*s - 0.5*mod( floor(8.0*t), 2.0)) - 0.5), abs(fract(8.0*t) - 0.5)))

smoothstep(-0.01, 0.01, 0.2 - 0.1*sin(30.0*s) - t)

smoothstep(0.3, 0.32, length(fract(5.0*st)-0.5))

s fract(5.0*s)

abs(fract(5.0*s)*2.0-1.0) mod(floor(10.0*s) + floor(10.0*t), 2.0)

Figure 1: Some simple procedural patterns in GLSL.

Noise by itself does not make a very interesting pat-tern, although it can be used to generate waves andbumps. More interesting patterns are generated by per-forming spectral synthesis (combining noise of severalspatial frequencies), and by using color tables or thresh-olding the result. Noise really shines when it is used toadd variation to other, more regular patterns. Some pat-terns based on Perlin noise are shown in Figure 2.

float perlin = 0.5 + 0.5*snoise(vec3(10.0*st, 0.0));gl_FragColor = vec4(vec3(perlin), 1.0);

float cow = snoise(vec3(10.0*st, 0.0));cow += 0.5*snoise(vec3(20.0*st, 0.0));cow = aastep(0.05, n);gl_FragColor = vec4(vec3(cow), 1.0);

float fbm=snoise(vec3(5.0*st, 0.0)) + 0.5*snoise(vec3(10.0*st, 2.0)) + 0.25*snoise(vec3(20.0*st, 4.0)) + 0.125*snoise(vec3(40.0*st, 6.0)) + 0.0625*snoise(vec3(80.0*st, 8.0));gl_FragColor = vec4(0.4*vec3(fbm) + 0.5, 1.0);

float d = length(fract(st*10.0) - 0.5);float n = snoise(vec3(40.0*st, 0.0)) + 0.5*snoise(vec3(80.0*st, 2.0));float blotches = aastep(0.4, d + 0.1*n);gl_FragColor = vec4(vec3(blotches), 1.0);

Figure 2: Some Perlin noise patterns in GLSL.

Another useful and popular pseudo-random patternfunction is cellular noise, introduced by Stephen Wor-

ley [12]. It is a different kind of function than Perlinnoise, and it generates a different class of patterns: tiles,cells, spots or other distinct features distributed acrossa surface with a seemingly random placement. Somepatterns based on cellular noise are shown in Figure 3.

vec2 F = cellular(st*10.0);gl_FragColor = vec4(vec3(F), 1.0);

vec2 F = cellular(st*10.0);float rings = 1.0 - aastep(0.45, F.x) + aastep(0.55, F.x);gl_FragColor = vec4(vec3(rings), 1.0);

vec2 F; // distances to featuresvec4 d; // vectors to features// F and d are ‘out’ parameterscellular(8.0*st, F, d);// Constant width lines, from// the book “Advanced RenderMan”float t = 0.05 *(length(d.xy - d.zw)) / (F.x + F.y);float f = F.y - F.x;// Add small scale roughnessf += t * (0.5 - cellular(64.0*st).y);gl_FragColor = vec4(vec3(aastep(t, f)), 1.0);

vec2 F = cellular(st*10.0);float blobs = 1.0 - F.x*F.x;gl_FragColor = vec4(vec3(blobs), 1.0);

vec2 F = cellular(st*10.0);float facets = 0.1 + (F.y - F.x);gl_FragColor = vec4(vec3(facets), 1.0);

Figure 3: Some simple procedural patterns in GLSL.

Noise is ubiquitous in off-line rendering, in particularwhen rendering scenes from nature, but it has not beena good fit for real time rendering because of its relativecomplexity. Noise is not a very complicated function,but a typical noise-based texture needs several noisevalues per surface point, and each value takes morework to compute than a simple texture image lookup.When GLSL was designed, noise functions were in-cluded in the language, but to date (2013) they remainunimplemented. The desire to implement hardwaresupport for Perlin noise has to compete for silicon areawith other design goals of a modern GPU. Recent re-search has provided some very hardware friendly noisefunctions, and it might be time to reconsider. How-ever, what GPU manufacturers choose to implement islargely beyond our control. In the meantime, we can getby with shader-implemented software versions of noise.Recent advances in GPU capabilites in combinationwith new variations on the kind of noise functions in-troduced by Perlin and Worley has provided hardware-friendly noise functions implemented in GLSL [9, 6].The functions are licensed with a permissive MIT li-

43

S Gustavsson / No more texels, no more facets: Emerging trends in GPU procedural shading

cense, they are very easy to use and show good realtime performance on modern GPUs. A visual exampleof the kind of patterns that can be generated with thesefunctions, at fully interactive frame rates even on a mid-range GPU, is in Figure 5.

6 General contoursAnother problem with traditional image-based textur-ing is crisp edges. Formally, an edge is not a featurethat is suitable for sampling, because it is not band lim-ited. Other representations have been proposed over theyears to render edges and contours on 2D surfaces, butnone have prevailed. Recently, NVIDIA presented anextension to OpenGL named NV path rendering,which is their take on rendering 2D graphics directlyfrom contour descriptions of the kind that are used inPostScript, PDF, SVG and other modern standards for2D object graphics. While successful, it taxes even atop performing GPU considerably to render contours inthis manner, and it is not useful on lower end hardwareor hardware from other manufacturers.

Another method, providing a significant new take onexisting ideas from 3D shape rendering [3], was pro-posed in 2007 by Chris Green of Valve Software [4]:a distance transform representation of the contour iscomputed and stored as a texture, and a shader is usedto generate the crisp edge. This allows a fully gen-eral, anisotropic analytic anti-aliasing of the edge, andthe shader can be kept simple and fast. Combinedwith recent development in distance transforms [7], thismethod is very suitable for certain important kinds ofsurface textures, like alpha-masked outlines, text, de-cals and printed patterns. Until now, real time graph-ics has had an annoying tendency of making such pat-terns blurry or pixelated in close-up views, but contourrendering based on a distance transform can eliminatethat and make edges crisp and clear in a very widerange of scales from minification to extreme magnifi-cations. Moreover, the edges can be anti-aliased in ananisotropic, analytic manner. An example of this is pre-sented in [5]. A pattern processed and rendered in thatmanner is shown in Figure 4.

7 Anti-aliasingThe texture subsystem of a GPU has built-in mecha-nisms to handle anti-aliasing. Mipmapping has beenpart of OpenGL since its early days, and anisotropic

Figure 4: Shapes rendered by a distance transform. Top:bitmap texture used as input to the distance transform.Bottom: crisp, high resolution shapes rendered by thedistance transform and a shader.

mipmap filtering has been common for years. However,mipmapping only handles minification, and there is alimit to how much anisotropy it can handle. Proceduralpatterns can also fill in detail when the pattern is mag-nified. Edges remain crisp, and there is no pixelization.Finally, because the anti-aliasing is analytic in nature, aprocedural surface pattern can be properly anti-aliasedeven in very oblique views with strong anisotropy in anarbitrary orientation. Anti-aliasing of procedural pat-terns is a problem that requires extra care in designingthe shader, but it’s not magic. Producers of off-line ren-dered graphics have a long tradition of anti-aliasing inshader programming, and the methods and concepts aredirectly applicable for real time use as well.

8 Displacement shadersProcedural shaders in off-line production can not onlyset the properties of a surface point. Another importantclass of shaders is displacement shaders, which set theposition of the surface point in relation to some originalsurface. Displacement mapping is the tangible conceptthat is simulated by bump mapping, normal mapping orparallax mapping. By performing a true displacementof the surface, a displacement shader can act as a mod-eling tool and add true geometric detail to a surface.Because the displacement shader is aware of the view-point and the view direction, it can adapt the level ofdetail to what is really needed, thus striking an impor-tant balance between performance and quality.

44

S Gustavsson / No more texels, no more facets: Emerging trends in GPU procedural shading

A powerful tool in off-line production is the combi-nation of a displacement shader and a matching surfaceshader. By passing information about the position andorientation of the displaced surface from the displace-ment shader to the surface shader, very detailed and re-alistic surfaces can be created. Displacement shadershave been an important part of the toolbox for off-lineproduction for a long time.

When GPU shading was introduced, vertex shadersdid not allow for displacement of arbitrary surfacepoints, only vertices. Thereby an important part of whatdisplacement shaders can do was lost. To simulate apoint by point displacement, dynamic tessellation to anappropriate resolution had to be performed on the CPU.While this has been done for things like real time ter-rain rendering, it is a process that is taxing for the CPU,and the tessellation needs to stop before the trianglesbecome too small and too many for the geometry trans-fer from the CPU to the GPU to handle them at real timeframe rates.

The recent introduction of hardware tessellationshaders changed that. Now, the GPU can execute ashader that dices the geometry into very small pieces ina view dependent manner, without even bothering theCPU. The GPU has a high internal bandwidth and cando this for many triangles in parallel. We have seenthis being put to good use for adaptive tessellation ofcurved surfaces, finally making it possible to use bicu-bic patches, NURBS and subdivision surfaces as thenative model description for real time content. Thisis a welcome thing by itself, as real time graphics hasbeen somewhat hampered by the fixation on polygonmodelling, while content for off-line rendering has beenfree to also use more flexible and adaptively tessellatedcurved surface primitives [2].

However, tessellation shaders can be used for more.They can bridge the gap between the comparably crudevertex shaders for hardware rendering and the muchmore versatile displacement shaders used in softwarerendering. GPUs are not quite yet at the point wherethey can easily dice all geometry in the scene to tri-angles the size of a single pixel, but we are rapidlygetting there, and then the vertex shader stage willhave the same capabilities as a traditional displacementshader for off-line rendering. The REYES renderingalgorithm, which is the traditional scanline renderingmethod for Pixar’s RenderMan, has a very hardwarefriendly structure because it originated as an algorithmfor Pixar’s hardware renderer RM-1. Simply put, itworks by dicing all geometry to micropolygons that

are about the projected size of a pixel, and executingshaders for each micropolygon. The time is now rightto start doing similar things in GPU rendering.

9 ConclusionWe are used to dealing with real time graphics and off-line graphics in different ways, using different meth-ods and different tricks to render the images we want.We have also been forced to accept a low image qual-ity from real time graphics, incapable of matching evenprevious generations of off-line renderings. That gapis now closing. While some methods from softwarerendering, like the more advanced methods for globalillumination, will probably remain out of reach for realtime execution for quite some time longer, there are def-initely possibilities to mimic in a real time setting whatsoftware renderers like RenderMan did ten years ago.

Current research and development in GPU renderingseems to be focusing on adding even more sophisti-cated illumination and surface reflectance models to thehardware rendering pipeline. The two concepts empha-sized in this article, procedural patterns and displace-ment shaders, do not seem to attract an interest nearlyas strong. That is a shame, as there are many fun anduseful things to be done in that area. Looking furtherinto this could have a significant impact on the overallperceived realism in real time graphics, possibly moreso than current incremental improvements in illumina-tion and reflectance. Real time illumination and sur-face reflectance models are now roughly at the pointwhere off-line scanline rendered graphics was ten yearsago. Geometry modelling and surface texturing, how-ever, are still twenty years behind or more. To put itbluntly, real time graphics is basically still pasting pix-els onto polyhedra like we did in the early 1990’s, onlyat a much greater speed.

Imagine if in real time graphics applications we couldeliminate pixelated textures in close-up views, add arbi-trary surface variation programmatically to any texture,model geometry features down to a microscopic scaleusing displacement shaders, and stop worrying aboutpolygon edges showing.

No more texels, no more facets!

45

S Gustavsson / No more texels, no more facets: Emerging trends in GPU procedural shading

Figure 5: Some more complex noise-based procedural patterns in GLSL

References

[1] D. S. Ebert, F. K. Musgrave, D. Peachey, K. Perlin, andS. Worley. Texturing and Modeling: A Procedural Ap-proach. Morgan Kaufmann Publishers Inc., San Fran-cisco, CA, USA, 3rd edition, 2002.

[2] A. R. Fernandes and B. Oliveira. Gpu tessellation: Westill have a LOD of terrain to cover. In P. Cozzi andC. Riccio, editors, OpenGL Insights, pages 145–162.CRC Press, 2012.

[3] S. F. Frisken, R. N. Perry, A. P. Rockwood, and T. R.Jones. Adaptively sampled distance fields: a gen-eral representation of shape for computer graphics. InProceedings of the 27th annual conference on Com-puter graphics and interactive techniques, SIGGRAPH’00, pages 249–254, New York, NY, USA, 2000. ACMPress/Addison-Wesley Publishing Co.

[4] C. Green. Improved alpha-tested magnification for vec-tor textures and special effects. In ACM SIGGRAPH2007 courses, SIGGRAPH ’07, pages 9–18, New York,NY, USA, 2007. ACM.

[5] S. Gustavson. 2D shape rendering by distance fields. InP. Cozzi and C. Riccio, editors, OpenGL Insights, pages173–181. CRC Press, 2012.

[6] S. Gustavson. Procedural textures in GLSL. In P. Cozziand C. Riccio, editors, OpenGL Insights, pages 105–120. CRC Press, 2012.

[7] S. Gustavson and R. Strand. Anti-aliased euclidean dis-tance transform. Pattern Recogn. Lett., 32(2):252–257,Jan. 2011.

[8] P. Hanrahan and J. Lawson. A language for shadingand lighting calculations. In Proceedings of the 17thannual conference on Computer graphics and interac-tive techniques, SIGGRAPH ’90, pages 289–298, NewYork, NY, USA, 1990. ACM.

[9] I. McEwan, D. Sheets, M. Richardson, and S. Gus-tavson. Efficient computational noise in GLSL. Journalof Graphics Tools, 16(2):85–94, 2012.

[10] K. Perlin. An image synthesizer. SIGGRAPH Comput.Graph., 19(3):287–296, July 1985.

[11] S. Upstill. RenderMan Companion: A Programmer’sGuide to Realistic Computer Graphics. Addison-WesleyLongman Publishing Co., Inc., Boston, MA, USA,1989.

[12] S. Worley. A cellular texture basis function. In Proceed-ings of the 23rd annual conference on Computer graph-ics and interactive techniques, SIGGRAPH ’96, pages291–294, New York, NY, USA, 1996. ACM.

46

S Gustavsson / No more texels, no more facets: Emerging trends in GPU procedural shading