real-time edge-aware image processing with the bilateral grid
TRANSCRIPT
Real-Time Edge-Aware Image Processing with the Bilateral Grid
SIGGRAPH '07Jiawen Chen, Sylvain Paris, Frédo Durand
CSAIL - MIT
By Orazio Gallo, 01/29/08
Motivation
● Many computational photography applications require piecewise smooth filtering
● Bilateral filtering is effective but slow: minutes per megapixel
● Many approximations and optimizations have already been proposed (~seconds per megapixel)
● Real time?
Paper's Main Contribution
● The bilateral grid: a 3D data structure that naturally embeds edge information and allows for parallelization
● Now everything is real-time!● “Edge-aware” algorithms for different image
processing tasks
Talk Overview
● Super-quick overview of bilateral filtering (?)● The data structure and its use● Example apps
Bilateral Filtering
Edge + noise De-noised edge
x
(Adapted from Durand and Forsey 2002)
More on Bilateral Filtering
Input
Output
space
p
q
p
inte
nsity
inte
nsity
More on Bilateral Filtering
Input
Output
space
p
qinte
nsity
inte
nsity
p
Bilateral Grid
The Bilateral GridStandard 2D Image
x
y
• Nearest neighbor
• Bicubic
2D vs. Bilateral Grid Downsampling
x
y
Downsampling in 2D
x
y
Downsampling in 2D Bilateral Grid
Extra dimension compensates for aggressive down-sampling!!
2D vs. Bilateral Grid Downsampling
(70 x 70 x 10 for an 8 megapixel image)
Processing and Slicing● Processing:
any function that takes a 3D function as input!
● Slicing – getting back an image:use the original image to access the grid. The intensity level is obtained by interpolating along the z direction in the grid.
Bilateral Filter
Bilateral Filter on the Bilateral GridImage scanline
space
inte
nsityBilateral
Grid
space
inte
nsity
Bilateral Filter on the Bilateral GridImage scanline
Filtered scanline
Slice: query gridwith input image
BilateralGrid
Gaussian blur grid values
space
inte
nsity
Bilateral Filter on the Bilateral Grid
s is the geometric spread
r is the photometric spread
Bilateral Filter Performance
Image size: 2 MPixels● CPU
– Brute force: 10 minutes– State of the art ’06: 1 second [Weiss 06, Paris 06]
● Our Bilateral Grid with GPU– 2004 card (NV40): 28 ms (36 Hz)– 2006 card (G80): 9 ms (111 Hz)
Bilateral Filter Performance
Show:✔ bf.wmv
Grid Painting
Grid Painting
Grid PaintingImage scanline
space
inte
nsityBilateral
Grid
space
inte
nsity
Grid PaintingImage scanline
Output brush stroke
Query grid with input image: slicing
BilateralGrid
Grid Painting
Show:✔ gp_3d.wmv✔ gp_recut.wmv
Tone Mapping for HDRI
HDRI and Tone Mapping
● CRT displays 100:1● LCD displays up to ~1'000:1● Photographic prints up to ~1'000:1● Real world scenes: > 100'000:1
HDRI and Tone Mapping
From Wikipedia
HDRI and Tone Mapping
From Wikipedia
HDRI and Tone Mapping
The algorithm [Durand and Dorsey 02]
Color ratiosDetail
Large scale
Preserve
Linear remapping
exp
Log luminance
User-Driven, Local Tone Mapping
User-Driven, Local Tone Mapping
Show:✔ ToneMapping.wmv
Other Applications: Histogram Equalization
Matlab's adapthisteq (1.5s) Their result (100ms)
1760 x 2140, HDR
Other Applications: Transfer of Photographic Look [Bae 06]
● Artistic rendition from model– 2 bilateral filters
● Remove compression artifacts● Split into large-scale and small-scale layers
– 2 histogram transfers● Match statistics on layers
● 1.5 megapixels at 30 Hz
Model
Input
Output
Summary● Definition of the Bilateral Grid:
– Intuition, creation, processing, and slicing● Applications
– Bilateral filtering– Grid painting– Local tone mapping– Local histogram equalization– Transfer of Photographic look– ...
thanks!