compressive sensing - examples in image compressionw3.impa.br/~aschulz/cs/lecture4.pdf · (a) dct 0...

Compressive Sensing

1

Compressive SensingExamples in Image Compression

Lecture 4 , July 30, 2009

Luiz VelhoEduardo A. B. da SilvaAdriana Schulz

Compressive Sensing

2

Today’s Lecture

• Discuss applications of CS in image compression• Evaluate CS efficiency• Review important definitions and theorems

Compressive Sensing

3

Software

• Matlab functions for CS imaging• Optimization algorithms• Data• Demos

http://www.impa.br/∼aschulz/CS/course.html

Compressive Sensing

4

Experimental Setup• 4 images of size 256× 256

(a) Phantom (b) Lena

(c) Camera man (d) Text

Compressive Sensing

5

Frequency Distributions

50 100 150 200 250

50

100

150

200

250

(a) Phantom

50 100 150 200 250

50

100

150

200

250

(b) Lena

50 100 150 200 250

50

100

150

200

250

(c) Camera man

50 100 150 200 250

50

100

150

200

250

(d) Text

Compressive Sensing

6

Acquisition

• Images are stored as a matrix of pixels

Compressive Sensing

7

Noiselets

• RIP• orthogonal• self-adjoint

Φ =12·

1 −1 1 1

−1 1 1 11 1 −1 11 1 1 −1

Compressive Sensing

8

Sparsity

50

100

150

200

250

Compressive Sensing

9

Reconstruction

Acquisition:y = ΦΩx

Reconstruction:y = ΘΩs

s = mins ‖s‖l1 s. t.

y = Θs‖y −Θs‖l2 ≤ ε

x = Ψ∗s

Compressive Sensing

10

Basic CS• Images are sparse in the Ψ domain• No noise is added

Ordinary images are only approximately sparse :

50

100

150

200

250

(a)

50

100

150

200

250

(b)

Figure: Different visualizations of Lena’s DCT.

Compressive Sensing

11

Forcing SparsityDCT Example:

(a) Original Image (b) Original DCT (c) Original DCT

(d) Result Image (e) Result DCT (f) Result DCT

Compressive Sensing

12

First Experiment

• x Ψ−→ sforce sparsity−−−−−−−→ sS

Ψ∗

−−→ xS

• Acquisition: y = ΦΩxS

• Reconstruction of x via l1 minimization

• Compare x and xS using PSNR (Peak Signal to Noise Ratio)

Compressive Sensing

13

First Result

10 20 30 40 50 6020

30

40

50

60

70

80

Measurements( × 103)

PS

NR

Figure: Basic CS experiment: 10k-sparse Lena.

Theorem

Reconstruction is exact if

M & S · µ2(Φ,Ψ) · log N

Compressive Sensing

14

Varying S

10 20 30 40 50 6020

25

30

35

40

45

50

55

60

65

70


PS

NR

3.5k−Sparse6k−Sparse10k−Sparse14k−Sparse

Compressive Sensing

15

DCT Linear Compression

We measure M N coefficients on the upper-left corner

Compressive Sensing

16


Reconstruction: x = Ψ∗s

Compressive Sensing

17


Problem: we don’t measure all the significant coefficients andmeasure some zero coefficients!

Compressive Sensing

18

CS: Linear + Noiselets Measurements

Reconstruction: s = mins ‖s‖l1 s. t. Φ′

ΩΨ∗s = yx = Ψ∗s

Compressive Sensing

19

Results: CS × Linear Compression10 20 30 40 50 60

[DCT−l1−N] [DCT−l

2−L] [DCT−l

1−LN]

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(a) 3.5K-Sparse

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(b) 6K-Sparse

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(c) 10K-Sparse

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(d) 14K-Sparse

Compressive Sensing

20

Coherence

Before Boundary: best to use linear compressionAfter Boundary: best to use CS

10 20 30 40 50 6020

30

40

50

60

70

80

90

100

110

120


PS

NR

DCT−l1−N

DCT−l1−LN

Compressive Sensing

21

Different ImagesWhat happens when we consider the other images?

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(e) Linear Compression

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(f) Compressive Sensing

(a) Phantom (b) Lena (c) Camera man (d) Text

Compressive Sensing

21


10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(e) Linear Compression

10 20 30 40 50 6020

30

40

50

60

70

80


PS

NR

(f) Compressive Sensing

0 50 100 150 200 250

0

50

100

150

200

250

(a) Phantom

0 50 100 150 200 250

0

50

100

150

200

250

(b) Lena

0 50 100 150 200 250

0

50

100

150

200

250

(c) Camera man

0 50 100 150 200 250

0

50

100

150

200

250

(d) Text

Compressive Sensing

21


20 40 60 phantom lena cameraman text

10 20 30 40 50 6020

30

40

50

60

70

80P

SN

R


(a) Linear Compression

10 20 30 40 50 6010

20

30

40

50

60

70

80

PS

NR


(b) Compressive Sensing

0 50 100 150 200 250

0

50

100

150

200

250

(a) Phantom

0 50 100 150 200 250

0

50

100

150

200

250

(b) Lena

0 50 100 150 200 250

0

50

100

150

200

250

(c) Camera man

0 50 100 150 200 250

0

50

100

150

200

250

(d) Text

Compressive Sensing

22

Different Sparsity DomainsWhat happens when we consider different sparsity domains?

0 50 100 150 200 250

0

50

100

150

200

250

nz = 10000

(a) DCT

0 50 100 150 200 250

0

50

100

150

200

250

nz = 10000

(b) Block DCT

0 50 100 150 200 250

0

50

100

150

200

250

nz = 10000

(c) Wavelets

10 20 30 40 50 6020

25

30

35

40

45

50

55

60

65

70

PS

NR


DCT−l1−N

B_DCT−l1−N

DWT−l1−N

Compressive Sensing

23

But this is not for real...

All this happens because we FORCE sparsity!

Theorem

Reconstruction is exact if

M & S · µ2(Φ,Ψ) · log N

Important Parameters:• µ - Noiselets are highly incoherent with all considered domains• S - We are tempering with it! - Lets stop and see what happens...

Compressive Sensing

24

Sparsity Errors

When Φ are Noiselet measurements the RIP holds and we can use:

Theorem

Let sS be the best S-sparse approximation of s.If the RIP holds, the solution s to

s = mins‖s‖l1 subject to ΘΩs = y

obeys

‖s − s‖l2 .1√S· ‖s − sS‖l1

Compressive Sensing

25

Let’s verify this theorem empirically

Compressive Sensing

26


10 20 30 40 50 6020

22

24

26

28

30

32

34

36

38

40


PS

NR

3.5k−Sparse

6k−Sparse

10k−Sparse

14k−Sparse

10 20 30 40 50 6020

25

30

35

40

45

50

PS

NR


10 20 30 40 50 6020

25

30

35

40

45

50

55

60

65


PS

NR

Test image LenaMeasurements Sparsity is forced Sparsity is not forced

M = 20k S = 3.5k PSNR = 28.8 PSNR = 26.6M = 25k S = 6k PSNR = 30.7 PSNR = 27.8M = 35k S = 10k PSNR = 33.0 PSNR = 30.2M = 40k S = 14k PSNR = 34.9 PSNR = 31.5

Compressive Sensing

26


10 20 30 40 50 6020

22

24

26

28

30

32

34

36

38

40


PS

NR

3.5k−Sparse

6k−Sparse

10k−Sparse

14k−Sparse

10 20 30 40 50 6020

25

30

35

40

45

50

PS

NR


Test image LenaMeasurements Sparsity is forced Sparsity is not forced

M = 20k S = 3.5k PSNR = 28.8 PSNR = 26.6M = 25k S = 6k PSNR = 30.7 PSNR = 27.8M = 35k S = 10k PSNR = 33.0 PSNR = 30.2M = 40k S = 14k PSNR = 34.9 PSNR = 31.5

Compressive Sensing

27

Different Sparsity Domains

10 20 30 40 50 60 [DCT−l

2−L] [B_DCT−l

2−L] [DCT−l

1−N] [B_DCT−l

1−N] [DWT−l

1−N] [SVD−l

1−N] [TV−l

1−N]

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(b) Phantom

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(c) Lena

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(d) Camera man

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(e) Text

Compressive Sensing

27


Total Variation (TV)

minx‖x‖TV subject to y = ΦΩx

• l1-norm of the (appropriately discretized) gradient.

Compressive Sensing

27


10 20 30 40 50 60 [DCT−l

2−L] [B_DCT−l

2−L] [DCT−l

1−N] [B_DCT−l

1−N] [DWT−l

1−N] [SVD−l

1−N] [TV−l

1−N]

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(b) Phantom

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(c) Lena

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(d) Camera man

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(e) Text

Compressive Sensing

27


Singular Value Decomposition (SVD)

• Calculated for each image!• Upper bounds - insights into performance limitations.

Compressive Sensing

27


10 20 30 40 50 60 [DCT−l

2−L] [B_DCT−l

2−L] [DCT−l

1−N] [B_DCT−l

1−N] [DWT−l

1−N] [SVD−l

1−N] [TV−l

1−N]

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(b) Phantom

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(c) Lena

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(d) Camera man

10 20 30 40 50 6010

20

30

40

50

60

70

80


PS

NR

(e) Text

Compressive Sensing

28

Measurement Errors

Measurements: y = ΦΩx + n

• n is a random variable with normal distribution

Theorem

If the RIP holds and ‖n‖l2 ≤ ε, the solution s to

s = mins‖s‖l1 s.t. ‖ΘΩs − y‖l2 ≤ ε

obeys

‖s − s‖l2 .1√S· ‖s − sS‖l1 + ε

Compressive Sensing

29

Gaussian Errors

10 20 30 40 50 6015

20

25

30

35

40

45

50

55

60

65

70


PS

NR

var = 0.1var = 0.5var = 1var = 2var = 3var = 4var = 5var = 10

Compressive Sensing

30

Compression

Number of Measurements × Number of Bits

• To compress we have to quantize the acquired data!

Compressive Sensing

31

Quantization

−300 −200 −100 0 100 200 3000

5

10

15

−300 −200 −100 0 100 200 3000

200

400

600

800

Each measurement may assume K values.

Compressive Sensing

32

Quantization

• Larger quatization steps: less bits• Smaller quatization steps: less errors

Compressive Sensing

33

Rate

Rate = number of bits per pixel

Original Image:• N pixels• 8 bits/pixel

Acquired Measurements:• M N coefficients• Hy bits/coefficients

Rate =MN

Hy bits/pixel

If Rate < 8 data is being compressed!

Compressive Sensing

34

The Rate × Distortion Criteria

=⇒ which is the best compression scheme?

Compressive Sensing

35

Rate × Distortion

step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5

step = 100

step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50

10 20 30 40 50 6010

20

30

40

50

60

70

80

90

100

Measurements x 103

PS

NR

Compressive Sensing

35

Rate × Distortion

0 2 4 6 8 10 12 14 16 180

10

20

30

40

50

60

70

80

90

100

step = 0.01step = 0.02step = 0.05step = 0.1step = 0.2step = 0.5step = 1step = 2step = 3step = 4step = 5step = 10step = 20step = 50step = 100

Rate

PS

NR

Compressive Sensing

35

Rate × Distortion


0 2 4 6 8 10 12 14 16 180

10

80

90

100

Compressive Sensing

35

Rate × Distortion

0 2 4 6 8 10 12 14 16 180

10

20

30

40

50

60

70

80

90

100


Rate

PS

NR

Compressive Sensing

36

Both Sparsity and Measurement Errors

A real example: real image and quantized measurements.

Recovery Strategies:• Ψ = DCT• Ψ = block DCT• Ψ = DWT• TV- minimization• Ψ = SVD - of each image!

Compressive Sensing

37

Both Sparsity and Measurement Errors

0 1 2 3 4 5 6 7 8 Jpeg2000 DCT−l

1−N B_DCT−l

1−N DWT−l

1−N TV−N SVD−l

1−N

0 1 2 3 4 5 6 7 80

10

20

30

40

50

60

70

80

90

100

Rate

PS

NR

(b) Phantom

0 1 2 3 4 5 6 7 80

10

20

30

40

50

60

70

80

90

100

Rate

PS

NR

(c) Lena

0 1 2 3 4 5 6 7 80

10

20

30

40

50

60

70

80

90

100

Rate

PS

NR

(d) Camera man

0 1 2 3 4 5 6 7 80

10

20

30

40

50

60

70

80

90

100

Rate

PS

NR

(e) Text

Compressive Sensing

38

Sparsity × Quantization Errors

‖x − x‖l2 .

εq +1√S‖xS − x‖l1︸︷︷︸

εs

,

The reconstruction error in CS is of the order of the maximum of thequantization (εq) and sparsity errors (εs).

Compressive Sensing

39

(a) DCT (b) DWT

(c) TV (d) SVD

Figure: Rate × PSNR of image Lena

Compressive Sensing

40

(a) DCT (b) DWT

(c) TV (d) SVD

Figure: Measurements × PSNR of image Lena

Compressive Sensing

41

Lets wrap it up!

CS ApplicationsImage acquisition and compression• Interesting, but• Limitations ...

TomorowApplications in graphics and vision• One Pixel Camera• Dual Photography

compressive sensing - examples in image compressionw3.impa.br/~aschulz/cs/lecture4.pdf · (a) dct 0...

Documents