retinex by two bilateral filters michael elad the cs department the technion – israel institute of...
Post on 24-Jan-2016
219 views
TRANSCRIPT
![Page 1: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/1.jpg)
Retinex by Two Bilateral Filters
Michael EladThe CS Department
The Technion – Israel Institute of technologyHaifa 32000, Israel
Scale-Space 2005The 5th international conference on scale-space
and PDE in computer visionHofgeismar, Germany
April 7-9th , 2005
![Page 2: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/2.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
2
Retinex?
![Page 3: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/3.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
3
Agenda
1. What is Retinex? The Basics
2. Retinex from a Variational Point of View
3. Bilateral Filter? Evolution of Denoising Methods
4. Bilateral Filter For Retinex
5. Examples and What Next?
![Page 4: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/4.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
4
Part 1
What is Retinex? The Basics
![Page 5: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/5.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
5
What is Retinex?
The sensed image is S=LR, where: S – Scene intensity,
L – illumination, R – reflectance.
We can measure S, but not L or R.
Getting R from S is an ill-posed inverse problem.
The human visual system (HVS) sees R. How?
Retinex is a similar image enhancement algorithm.
![Page 6: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/6.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
6
A Typical Retinex System
RLS Rlogr
,Llog
,Slogswhere
rs
STake log
s Extract
+_
r Take exp
R
![Page 7: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/7.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
7
Example: Gamma Correction
Gamma correction is a simple Look-Up- Table operation of the form (γ2.5)
γ/1
255S
255S
SGamma
Correction
S
255logs1
1ˆ
In
Out
![Page 8: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/8.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
8
A new Algorithm
Many Retinex Algorithms …
Various Retinex algorithms out there: Random walk smoothing [Land and MacCan 1971]
Homomorphic filtering [Stockham 1972, Faugeras 1979]
Poisson equation [Horn 1974, Blake 1985, Funt et. al. 1992]
Multi-grid Poisson solver [Terzopoulos 1986]
Bilateral filter for retinex [Durand & Dorsey, 2002]
Sparse representations (Curvelet) [Starck et. al. 2003]
Multi-scale heuristic envelope method [McCann 1999]
Variational method [Kimmel et. al. 2003, Elad et. al. 2003]
IIR envelope smoothing [Shaked 2004]
![Page 9: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/9.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
9
Part 2
Retinex from a A Variational Point
of View
![Page 10: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/10.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
10
Things To Consider
The illumination is supposed to be spatially (piecewise!?) smooth.
ds Minimize 22
s
Since the reflectance is passive, 0≤R≤1, we require S≤L and s≤ .
Trivial solution (L=255) should be avoided - The illumination should be forced to be close to s.
![Page 11: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/11.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
11
Illumination as an Upper Envelope
ds Minimize 22
s
s
Small Large
![Page 12: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/12.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
12
The reflectance, s- should behave like a typical image
The Overall Model
dss Minimize
222
s
[Kimmel, Elad, Shaked, Keshet, & Sobel 2003]
This is a quadratic programming problem
A multi-scale method numerical scheme was proposed
![Page 13: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/13.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
13
Shortcomings?
2y
2x
22y
2x
ssss Minimize
DDDD βα
Smooth illumination envelope smooth reflectance
Requires an iterative solver!
rs Noise is magnified in dark
areas. Forcing works against noise suppression.
Promotes hallows on the boundaries of the illumination.
![Page 14: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/14.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
14
Part 3
Bilateral Filter? Evolution of Denoising Methods
![Page 15: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/15.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
15
Evolution in Denoising (1D)
22
zzsz Minimize Dα
Task: Given a noisy signal s, produce a denoised version of
if, z.
22
zzsz Minimize CI α
Or written differently with a shift-operator C:First Stage
![Page 16: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/16.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
16
Going “Weighted” & “Robust”
Over-smoothing – introduce weights in order to preserve
edges zzsz Minimize T2
zCIWCI αSecond
Stage
Better yet – use signal dependent weights (robust
statistics) zzzsz Minimize T2
zCIWCI αThird Stage
![Page 17: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/17.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
17
P
Pm
2
z
m2
z m
zsz MinimizeW
CIα
The Bilateral Filter
Too many iterations – use wide-support smoothness term
Fourth Stage
The bilateral filter is a weighted average smoothing, with weights inversely proportional to
the (total!) distance between the center pixel and the neighbor [Tomasi and Manduchi, 1998]
zBThe first Jacobi iteration that minimizes the
above function leads to the bilateral filter [Elad, 2002] Fifth Stage
![Page 18: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/18.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
18
Part 4
Bilateral Filter For Retinex
![Page 19: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/19.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
19
Retinex - Restatement
rB)s(rBs Minimize
2r
2
r,s
λαλ
Envelope illumination due to
passive reflectance
Piecewise smooth illumination Piecewise smooth
reflectance
Give an estimate of , if there is no noise, then r=s- .
If there is an additive noise, it is r-s+ , and its norm is the proper likelihood term to use.
Avoid trivial solution by forcing the
illumination to be close to s
![Page 20: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/20.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
20
The New Formulation
rB)s(rBs Minimize
2r
2
r,s
λαλ
With this new formulation: Non-iterative solver can be deployed,
Both the illumination and the reflectance are forced to be piece-wise smooth, thus preventing hallows,
Noise is treated appropriately,
Clear relation to Durand & Dorsey’s method is built.
![Page 21: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/21.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
21
Numerical (Sub-Optimal) Solution
rB)s(rBs Minimize
2r
2
r,s
λαλ
Bs Minimize
2r
sαλλ
Part 1: Find by assuming r=0
Part 2: Given , find r by
rB)s(r Minimize 2
rr
λ
Bilateral filter on s in an envelope mode
Bilateral filter on s- in a regular mode
Both can be speeded up dramatically [Durand & Dorsey
2002]
![Page 22: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/22.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
22
Part 5
Examples and
What Next?
![Page 23: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/23.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
23
Returning Some Illumination
Reflectance
Original
Closing (luminance)
Original
IlluminationResult
Gammacorrection
![Page 24: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/24.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
24
Original Result (γ=3)
Illumination
Reflectance
Example 1 – General
Special thanks to Eyal Gordon for his help in simulating the retinex algorithm(s) on stills and videos
![Page 25: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/25.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
25
Example 2 – Noise Reduction
Original
Result (γ=3)
Result with noise reduction
![Page 26: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/26.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
26
Example 3 – Hallows?
![Page 27: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/27.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
27
Conclusion
Retinex by two bilateral filters – based on [Kimmel et.al. 2003] and [Durand et. al. 2002].
It overcomes hallows, the need for iterations, and handles noise well.
Bilateral retinex – excellent choice for video retinex – Future work.
![Page 28: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/28.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
28
THANK YOU !
! תודה רבה
![Page 29: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/29.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
29
Other slides
![Page 30: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/30.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
30
Example 3 – Gamma Correction?
Computing the effective gamma per pixel and showing as a graph
The same gamma values per location – Retinex could be interpreted as an spatially
adaptive gamma correction
![Page 31: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/31.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
31
What About Video?
When this work started, we planned to develop a retinex algorithm for video sequences.
Things to consider for video:
Causality?
Inter-frame versus Intra-frame modes
Motion estimation and compensation
Run time and speedups
Bilateral Retinex seems like a very good match for video purposes: no need to estimate motion.
Our experiments gave successful retinex results with the simple intra-frame mode of work.
So, what could be the benefit in inter-frame mode for video?
![Page 32: Retinex by Two Bilateral Filters Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Scale-Space 2005 The](https://reader031.vdocuments.mx/reader031/viewer/2022012922/56649d805503460f94a64a9c/html5/thumbnails/32.jpg)
Retinex by TwoBilateral FiltersBy: Michael Elad
32
Example2: Homomorphic Filtering
Homomorphic filtering assumes that the low frequencies in S correspond to the illumination, whereas high frequencies to the reflectance
SHomomorphic
Filter
S
[Stockham 1972, Faugeras 1979]