Spectral bases for 3D shapes representation

Spectral bases – reminder

Last week we spoke of Fourier analysis 1D sine/cosine bases for signals:

0 πcos (2 1)

16x

πcos (2 1)

16x

2πcos (2 1)

16x

3πcos (2 1)

16x

4πcos (2 1)

16x

5πcos (2 1)

16x

6πcos (2 1)

16x

7πcos (2 1)

16x

Spectral bases – reminder

Last week we spoke of Fourier analysis 2D bases for 2D signals (images)

cos (2 1) cos (2 1)16 16

u vx y

How about 3D shapes?

Problem: general 3D shapes cannot be described as height functions

Height function, regularly sampled above a 2D domain

General 3D shapes

Irregular meshes

In graphics, shapes are mostly represented by triangle meshes

Irregular meshes

In graphics, shapes are mostly represented by triangle meshes

Geometry: Vertex coordinates

(x1, y1, z1)

(x2, y2, z2) . . .

(xn, yn, zn)

Connectivity: List of triangles

(i1, j1, k1)

(i2, j2, k2) . . .

(im, jm, km)

Irregular meshes

In graphics, shapes are mostly represented by triangle meshes The connectivity (the graph) can be very

efficiently encoded About 2 bits per vertex only

The geometry (x,y,z) is heavy! When stored naively, at least 12 bits per

coordinate are needed, i.e. 36 bits per vertex

How to define efficient bases?

Extension of the 2D DCT basis to a general (irregular) mesh

DCT ???

Basis shapes

We need a collection of basis functions First basis functions will be very smooth, slowly-varying Last basis functions will be high-frequency, oscillating

We will represent our shape (mesh geometry) as a linear combination of the basis functions

We will induce the basis functions from the mesh connectivity! We assume that the connectivity is known to the decoder

side (it is transferred first)

The Mesh Laplacian operator

( ) ( )

1( )i i i j i i j

j N i j N ii

L d dd

v v v v v

Measures the local smoothness at each mesh vertex

Laplacian operator in matrix form

1 1 1

2 2 2

3

1 1 1

1 0 1 0

0 1 1

0 1 1 1

1 1 1n n n

n n n

d

d

d

d

d

v

v

v

v

L matrix

Laplacian matrix

L is constructed based on the connectivity alone

So, if the decoder got the connectivity first, it can construct L

Spectral bases

L is a symmetric nn matrix Perform spectral analysis of L!

L =

T1

2

n

1 2

| | |

| | |nb b b 1 2

| | |

| | |nb b b

Basis vectors Frequencies,sorted in ascending order

How to represent our mesh geometry in the spectral basis?

X, Y, Z Rn. Decompose in the spectral basis: 1

21 1 2 2

1

21 1 2 2

1

21 1 2 2

n n

n

n n

n

n n

n

x

x

x

y

y

y

z

z

z

X b b b

Y b b b

Z b b b

How to represent our mesh geometry in the spectral basis?

Decompose the mesh geometry in the spectral basis:

T T T11 2

21 1 2 2

1 2

n

n n

nn

v

vb b b

v

The first components are low-frequency

The last components are high-frequency

The spectral basis First functions are smooth and slow, last oscillate a lot

horse connectivity

spectral basis of L =the DCT basis

chain connectivity

2nd basis function

10th basis function

100th basis function

Why this works?

Low-frequency basis vectors are smooth and slowly-varying: The first basis vector:

It is the smoothest possible function (Laplacian is constant zero on it)

1

T1

0

(1,1, 1)

L

b

b

Why this works?

Low-frequency basis vectors are smooth and slowly-varying: The first basis vector:

Lb1 = 0 because all the rows of L sum to zero

1

T1

0

(1,1, 1)

L

b

b

Why this works?

Low-frequency basis vectors are smooth and slowly-varying: The following basis vectors:

i i iL b b

The frequency is small the Laplacian operator gives small values the shape of the function is smooth

Why this works?

High-frequency basis vectors are non-smooth, oscillating:

i i iL b b

The frequency is large the Laplacian operator gives large values the shape of the function is less smooth!

Geometry compressionT T T1

1 22

1 1 2 2

1 2

n

n n

nn

v

vb b b

v

As in JPEG compression,drop the high-frequency components!

Geometry compression

Increasing number of participatingbasis vectors.The less coefficients are dropped, the less shape information is lost

Technical algorithm

The encoding side: Compute the spectral bases from mesh connectivity Represent the shape geometry in the spectral basis and

decide how many coeffs. to leave (K) Store the connectivity and the K non-zero coefficients

The decoding side: Compute the first K spectral bases from the connectivity Combine them using the K received coefficients and get

the shape

Challenges and trade-offs

Performing spectral decomposition of a large matrix (n>1000) is prohibitively expensive (O(n3)) Today’s meshes come with 50,000 and more vertices We don’t want the decompressor to work forever!

Solution: like in JPEG, work on small blocks:

Challenges and trade-offs

As in JPEG, blocking artifacts appear for low bit rates, because the compression is lossy (too much information is lost)

Further info

“Spectral Mesh Compression”, Zachi Karni and Craig Gotsman, ACM SIGGRAPH 2000http://www.mpi-inf.mpg.de/~karni/publications/spectral.pdf

“3D Mesh compression using fixed spectral bases”, Z. Karni and C. Gotsman, Graphics Interface, 2001http://www.mpi-inf.mpg.de/~karni/publications/fixed.pdf

See you next time