let us have some after lunch fun. fast image and video colorization (and beyond) using chrominance...

Let us have some after lunch fun

Fast Image and VideoFast Image and Video CCoolloorriizzaattiioonn (and beyond) (and beyond) using using

Chrominance BlendingChrominance BlendingYatziv and S.Yatziv and S.

Hand-Tinting of PhotosHand-Tinting of Photos1919thth century century

Hand-Tinting of postcardsHand-Tinting of postcardsearly 20early 20thth century century

Computer Assisted Colorization

• Given: a single dimension of luminance

• Output: three dimensions– Luminance– Chrominance

• Colorization is ambiguous in nature and requires some amount of human interaction.

Commercial Software

Adobe Photoshop Tutorial

Computer Assisted Colorization InterfacesComputer Assisted Colorization Interfaces

• Sample Based– One or more color images as a source– In video, use one or more colored frames

sample frame gray-scale result colored image

Example source: Sykora, Burianek, Zara “Colorization of Black-and-White Cartoons”

Computer Assisted Colorization InterfacesComputer Assisted Colorization Interfaces

• Chrominance Scribble Based– Small chrominance markings– Scribble area << image size

gray-scale scribbles result colored image

Example source: Levin, Lischinski, Weiss, “Colorization using optimization,” SIGGRAPH’ 04

Previous WorkPrevious Work

• Luminance Keying– Gonzalez, Woods, ‘Digital Image Processing’.

– Welsh, Ashikhmin, Mueller, “Transferring color to grayscale images”.

– Reinhard, Ashikhmin, Gooch, Shirley, “Color transfer between images”.

– Hertzmann, Jacobs, Oliver, Curless, Salesin, “Image analogies”.

• Segmentation– Chen, Wang, Schillings, Meinel, “Gray-scale matting and colorization”.

– Sykora, Burianek, Zara “Colorization of Black-and-White Cartoons”.

• Geometry Based– Sapiro, “Inpainting the colors”.

– Levin, Lischinski, Weiss, “Colorization using optimization”

Our ApproachOur Approach

1

0 , )(min:),(, r ts

CdrrCYtsd

ts

Fast Image and Video Colorization using Chrominance BlendingFast Image and Video Colorization using Chrominance Blending

Cs,t ts

Intrinsic distance:

Y is the gray-scale channel

c

),(min:)( tsdtdcs

c

Distance from a chrominance c:

dc(t) can be calculated efficiently using improved “Fast Marching” algorithm.

……Our ApproachOur Approach

ic

ici

t tdW

tdWcc

i

i

)(

)(

Fast Image and Video Colorization using Chrominance BlendingFast Image and Video Colorization using Chrominance Blending

tc1

c3

c2

BddW )( Bc

Bc

Bc

Bc

Bc

Bc

ttdtdtd

tdctdctdcc

)()()(

)()()(

321

321 321for

(typically B=4)D W

Shepard interpolation

Examples

Examples

Example - Video

NEAR

FAR

DECOLORIZING

Video Inpainting

Patwardhan and S.

Tracking under severe occlusions

Bartesaghi and S.