glsl applications: 1 of 2 - penn engineeringcis565/lectures2011s/lecture...fan strip max area 9...

1

GLSL Applications:

1 of 2

Patrick Cozzi

University of Pennsylvania

CIS 565 - Spring 2011

Administrivia

� Last day to add/drop classes

� Assignment 1 returned

�Nice work

Really???

Administrivia

� Assignment 2 due Wednesday

�11:59pm on Blackboard

� Assignment 3 handed Wednesday

� Final Project

�Cannot be combined with CIS 497, 563, 660, etc.

�Poster session: Thursday 04/28, 12-3pm

Agenda

� GLSL Applications

�Per-Fragment Lighting

� Image Processing

� Finish last week’s slides

�OpenGL Drawing

�OpenGL Multithreading

2

Per-Fragment Lighting

Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

Per-Fragment Lighting

Images from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

Per-Vertex Lighting Per-Fragment Lighting

Per-Fragment Lighting: Diffuse




3


uniform vec3 u_Color;

in vec3 fs_Incident;

in vec3 fs_Normal;

in vec3 fs_Texcoord;

out vec3 out_Color;

void main(void)

{

vec3 incident = normalize(fs_Incident);

vec3 normal = normalize(fs_Normal);

float diffuse = max(0.0f, dot(-incident, normal));

out_Color = vec4(diffuse * u_Color, 1.0f);

}




in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{





}

• in vectors are not normalized. Why?• Good practice: don’t write to in variables




in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{





}

• Know the graph:

• Why max?

Graph from http://www.wsd1.org/waec/math/pre-calculus%20advanced/Trigonometry/Graphing/graphingintro.htm

Per-Fragment Lighting: Specular


4





uniform vec3 u_SpecColor;

uniform float u_SpecHardness;


in vec3 fs_Viewer;

in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{



vec3 H = normalize(-incident + fs_Viewer);

float specular = pow(max(0.0f, dot(H, normal)), u_SpecHardness);


out_Color = vec4(diffuse*u_Color + specular*u_SpecColor,1.0f);

}






in vec3 fs_Viewer;

in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{







}

“half” vector






in vec3 fs_Viewer;

in vec3 fs_Normal;


out vec3 out_Color;

void main(void)

{







}

Blinn-Phong shading

5



Image Processing

� Our first look at GPGPU� General-Purpose computation on Graphics Processing Units

� Input: Image

� Output: Processed image

� A kernel runs on each pixel

Image Processing Examples

Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

Image Negative



Edge Detection

6



Toon Rendering

Image Processing Questions

� Is the GPU a good fit for image processing?

� Is image processing data-parallel?

� What about bus traffic?

� What type of shader should implement an image processing kernel?

Image Processing: GPU Setup

� Input:

�Texture

�Viewport-aligned quad, a.k.a full-screen quad

� Output: framebuffer

�…for now

� Kernel: fragment shader


1. Render viewport-aligned quad

Fragment

shader

2. A fragment is invoked

for each screen pixel

3. Each fragment shader can access any part of the

image stored as a texel:gather

4. Each fragment shader

executes the kernel and

writes the color to the framebuffer


7



Fragment

shader











Fragment

shader











Fragment

shader











Fragment

shader









8


� How do we model the viewport-aligned quad?

Two triangles? One big triangle?

Screen


� Which has more vertex shader overhead?

�Does it matter?

� Which is simpler to implement?

� Which has less fragment shader overhead?


� Triangle edges are redundantly shaded

� Fragments are processed in 2x2 blocks�Why?


� Triangle edges are redundantly shaded

Image and Chart from http://www.humus.name/index.php?page=News&ID=228

Number of vertices

FPS

Fan Strip Max area

9


� The viewport has the same width and height as the image to be processed

�The texture also has the same dimensions

�How does the fragment shader access

texels?


� Store texture coordinates per vertex

in vec3 Position;

in vec2 Texcoords;

out vec2 fs_Texcoords;

void main(void)

{

fs_Texcoords = Texcoords;

gl_Position = vec4(Position, 1.0);

}

Vertex Shader

uniform sampler2D u_Image;

in vec2 fs_Texcoords;

out vec4 out_Color;

void main(void)

{

out_Color = texture(u_Image,

fs_Texcoords);

}

Fragment Shader


� Store texture coordinates per vertex

�What memory costs does this incur?

� Does it matter?

�What bandwidth costs does this incur?

�What non-obvious optimization does it allow?


� Compute texture coordinate in fragment shader

in vec3 Position;

void main(void)

{


}

Vertex Shader


uniform vec2 u_inverseViewportDimensions;

out vec4 out_Color;

void main(void)

{

vec2 txCoord =

u_inverseViewportDimensions *

gl_FragCoord.xy;

out_Color = texture(u_Image, txCoord);

}

Fragment Shader

10


� Compute texture coordinate in fragment shader

in vec3 Position;

void main(void)

{


}

Vertex Shader



out vec4 out_Color;

void main(void)

{

vec2 txCoord =

u_inverseViewportDimensions *

gl_FragCoord.xy;

out_Color = texture(u_Image, txCoord);

}

Fragment Shader

What is u_inverseViewportDimensions?


� How do you access adjacent texels?uniform sampler2D u_Image;


out vec4 out_Color;

void main(void)

{

vec4 c0 = texture(u_Image, fs_Texcoords);

vec4 c1 = textureOffset(u_Image, fs_Texcoords, ivec2(-1, 0));

vec4 c2 = textureOffset(u_Image, fs_Texcoords, ivec2( 1, 0));

vec4 c3 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, -1));


out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}


� How do you access adjacent texels?uniform sampler2D u_Image;


out vec4 out_Color;

void main(void)

{

vec4 c0 = texture(u_Image, fs_Texcoords);

vec4 c1 = textureOffset(u_Image, fs_Texcoords, ivec2(-1, 0));


vec4 c3 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, -1));


out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}

(-1, 0) (1, 0)

(0, -1)

(0, 1)


� textureOffset requires constants

�E.g. ivec2(x, y) is not allowed

� How else do you access adjacent texels?

11


� How else do you access adjacent texels?uniform sampler2D u_Image;


out vec4 out_Color;

void main(void)

{

vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy;

vec2 delta = 1.0 / textureSize(u_Image);

vec4 c0 = texture(u_Image, txCoord);

vec4 c1 = texture(u_Image, txCoord + (delta * vec2(-1.0, 0.0)));

vec4 c2 = texture(u_Image, txCoord + (delta * vec2( 1.0, 0.0)));

vec4 c3 = texture(u_Image, txCoord + (delta * vec2( 0.0, -1.0)));


out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}




out vec4 out_Color;

void main(void)

{








out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}




out vec4 out_Color;

void main(void)

{








out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2;

}

(-1, 0) (1, 0)

(0, -1)

(0, 1)

Image Processing: Kernel Examples

� Image Negative



out vec4 out_Color;

void main(void)

{

out_Color = vec4(1.0) - texture(u_Image, fs_Texcoords);

}


12


Image from http://www.librow.com/articles/article-9

2D Gaussian� Gaussian Blur � Filter for 3x3 Gaussian Blur:

[1 2 1]

1/16 * [2 4 2]

[1 2 1]

• The elements add to one

• Other filters are also used for• Edge detection

• Sharpen• Emboss

• …



� Gaussian Blur

�How would you implement the fragment

shader?

�How is the memory coherence?

� 3x3, 5x5, etc.


Image from http://www.ozone3d.net/tutorials/image_filtering_p2.php

13


� What does this filter do?

[1 1 1]

1/9 * [1 1 1]

[1 1 1]

Image Processing: Read backs

� How do we get the contents of the framebuffer into system memory?� Print Screen?

� It doesn’t matter if we are using:

� Efficient read backs are important


� glReadPixels

glUseProgram(/* ... */);

glDraw*(/* ... */);

unsigned char rgb = new unsigned char[width * height * 3];

glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb);

// ...

delete [] rgb;


� glReadPixels


glDraw*(/* ... */);



// ...

delete [] rgb;

Use GPU for image processing

14


� glReadPixels


glDraw*(/* ... */);



// ...

delete [] rgb;

Allocate buffer for processed image


� glReadPixels


glDraw*(/* ... */);



// ...

delete [] rgb; Ask for framebuffer’s color buffer


� glReadPixels


glDraw*(/* ... */);



// ...

delete [] rgb; You guys are sharp


� What is the major problem with glReadPixels?

15

Image Processing: Use Cases

� Photoshop-type applications

� Post-processing in games

� On the fly video manipulation

� Augmented reality

These last three don’t even need read backs

glsl applications: 1 of 2 - penn engineeringcis565/lectures2011s/lecture...fan strip max area 9...

Documents