Image Fusion by Image Blending

Arthur GoshtasbyWright State UniversityWright State University

Image Registration and Fusion Systems

Image fusionImage fusion

Combining relevant information in two orCombining relevant information in two or more images of a scene into a single highly informative imageinformative image.Fusion methods:

L l l ( i l b d)– Low-level (pixel-based)– Mid-level (region-based)– High-level (description-based)

A. Goshtasby 2

Image compositingImage compositing

Cutting and smoothly stitching together partsCutting and smoothly stitching together parts of different images of a scene into a single highly informative imagehighly informative image.

Image 1 Image 2 Combined image

A. Goshtasby 3

Image fusion as an image blendingImage fusion as an image blending problem

A. Goshtasby 4

1 For each local area in the image1. For each local area in the image domain, find the image providing the most informationproviding the most information.

2. Cut out thef

Image domain

area out of the most informative imageimage.

A. Goshtasby 5

Selected local areas in images

3 Blend the local areas to create the fused3. Blend the local areas to create the fused image.

A. Goshtasby 6

Selecting the best-block imageSelecting the best block image

• Highest contrast• Highest entropy• Highest entropy

255

255

0)log(

iiiG ppE Gray-scale

bgrC EEEE Color

A. Goshtasby 7

The blending functionThe blending function

Assuming (x y ) represents the coordinates ofAssuming (xi,yi) represents the coordinates of the center of the ith block, use

G )(

N

j j

ii

yxGyxGyxW

1),(

),(),(

as the blending function, where

)()( 22 yyxx

is adjusted to the size of the local area

.2

)()(exp),( 2

ii

iyyxxyxG

is adjusted to the size of the local area.A. Goshtasby 8

Image blending algorithmImage blending algorithm

Given images {Ik(x,y): k = 1,…,M}, and assumingGiven images {Ik(x,y): k 1,…,M}, and assuming 1) the image domain is subdivided into N

blocks andblocks, and 2) for block i, the most informative image is

I (x y)Ii.k(x,y),the fused image is computed from:

N

ikii yxIyxWyxO

1. ),(),(),(

A. Goshtasby 9

i 1

Characteristics of the algorithmCharacteristics of the algorithm

• Parameter σ is adjusted to block size andParameter σ is adjusted to block size, and block size is chosen based on image size and/or image resolutionand/or image resolution.

• The blending process does not create information in output that does not exist ininformation in output that does not exist in input.Th h d i il i b d f• The method is versatile, it can be used to fuse various types of images.

A. Goshtasby 10

Application 1:Fusion of multi-exposure imagesOptimizing criterion: Entropy

A. Goshtasby 11

Images courtesy of Paul Debevec, USC

Optimizing criterion: Entropy

Original images courtesy of Shree Nayar Columbia University

A. Goshtasby 12

Original images courtesy of Shree Nayar, Columbia University

Optimizing criterion: Entropy

Original images courtesy of Max Lyons

A. Goshtasby 13

Application 2:Fusion of multi-focus images

Original iimages

Optimizing criterion:

Original images courtesy of Image

Fused image

Contrast

A. Goshtasby 14

Original images courtesy of Image Fusion Systems Research

Optimizing criterion: ContrastOptimizing criterion: Contrast

Originalimages

Fusedimageg

A. Goshtasby 15

Concluding remarksConcluding remarks

• If the type of information to be maximized is known,If the type of information to be maximized is known, this method is guaranteed to produce an image with the highest information content at the desired resolution.

• It can be adapted to image resolution by making block size a function of image resolution.

• By changing the optimization criterion, the method can be used to fuse various types of images.

A. Goshtasby 16

Image fusion referencesImage fusion references1. T. Porter and T. Duff, Compositing digital images, ACM SIGGRAPH Computer

Graphics, vol. 18, no. 3, 1984, pp. 253-259.2. P. Willis, Generalized compositing, ACM SIGGRAPH, 2007, pp. 129-135.3. A. Goshtasby, Fusion of multi-exposure images, Image and Vision Computing,

l 23 2005 611 618vol. 23, 2005, pp. 611-618.4. A. Goshtasby, Fusion of multifocus images to maximize image information,

Defense and Security SYmposium, 17-21 April 2006, Orlando, Florida, 2006.5 Special issue of Information Fusion on Image Fusion A Goshtasby and S5. Special issue of Information Fusion on Image Fusion, A. Goshtasby and S.

Nikolov (Eds.), vol. 8, 2007.6. R. S. Blum and Z. Liu, Multi-sensor Image Fusion and its Applications, CRC

Press, 2005. ,7. F. Sroubek and J. Flusser, Registration and fusion of blurred images, Int’l Conf.

Image Analysis and Recognition, 2004, 124-129.8. C. Genderen and J. L. van Pohl, Multisensor image fusion in remote sensing:

Concepts, methods and applications, Int’l J. Remote Sensing, vol. 19, no. 5, 1998, pp. 823-854.

A. Goshtasby17