Application of fuzzy transform to image fusion

Babii Andrii

Kharkiv National University of RadioelectronicsPh.d. student,

Supervisor: prof. Yerokhin A.L.

andrii.babii@nure.ua

Image Fusion

Process of combining multiple input images into single composite image

New image should contain better description of the scene than the input images

For example: low res multi-spectral image, Pan image and Result – pan-sharpened image

http://www.earthview3d.net/about/

Fusion categories

Multifocus fusion

Multimodal fusion

Multiview fusion

Multitemporal fusion

Multifocus fusion• Input: Set of images where there is at least one region of image is in focus in at least one channel

• Output: Image everywhere in focus

• Method: identify the regions in focus on input images and combine it

+

=

http://dsp.etfbl.net/mif/

Images of different modalities: PET, CT, MRI, visible, infrared, ultraviolet, etc.

Multimodal fusion

Input:

Output:Increase quality of band-specific information for lower dataamount of data

Method:Averaging, fusion in transform domain, object level fusion

https://arxiv.org/pdf/1401.0166.pdfMultimodal fusion in medicine

Multimodal fusion in GIS

http://eijournal.com/print/articles/data-fusion-empowers-geospatial-analysis

Images of same modality, at the same time but under different conditions or from different places

Multiview fusion

Input:

Output:Increase quality by complementary information from views

Method:Averaging, multi-view deconvolution, iterative fusion methods

https://publications.mpi-cbg.de/Swoger_2007_1403.pdf

Images of same scene taken at different time

Multitemporal fusion

Input:

Output:Changes detection

Method:Substraction

https://www.semanticscholar.org/paper/Multi-temporal-optical-VHR-image-fusion-for-Land-Paget-Gressin/8730aed1036c9ad2512724cfda6cc1c43690c720

Multifocus fusion using fuzzy transform

+= =

We want:

What is fuzzy transform?

It’s approximation method based on fuzzy sets, proposed by Perfilieva.It can be seen as fuzzy set-based analogue of the Fourie Transform

http://www.math.sk/fsta2012/presentations/dankova-perfilieva.pdf

The use of F-transform for image fusion algorithms,Perfilieva,Dankova,Hodakova http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=5686496

Fuzzy logic. Membership functions

1D data. Crisp partitions

Fuzzy partitions

http://www.atlantis-press.com/php/download_paper.php?id=8437

1D data. Fuzzy partitions on X

EquationsDirect fuzzy transform

Inverse fuzzy transform

Fuzzy partition approximation.

2D fuzzy partitions

=

For each partition!

Real image

Result of approximationInput

F-transform (5x5, triangle)

Gaussian blur(5x5)

How it works

Find sharp parts (the best parts)

extract

create better image

Fuzzy transform used here

Main idea

Fuzzy transform smoothing edges

So, the more fuzzy image – the lower compression error after fuzzy transform !

1. Recursively transform image via fuzzy transformation

2. Good parts will have high difference between origin and reconstruction

3. Use it to reconstruct sharp image

Steps

Step 1. Direct + Inverse Fuzzy Transform

Step 2. Calculate error for pixels and areas

Apply the threshold to areas

Result

Example

Future work

1. What if input image too fuzzy?

2. How to select “good partition size”?

3. In real world images does not fit. How we can deal with it?

Referenceshttp://www.earthview3d.net/about/

http://dsp.etfbl.net/mif/

http://eijournal.com/print/articles/data-fusion-empowers-geospatial-analysis

https://publications.mpi-cbg.de/Swoger_2007_1403.pdf

https://www.semanticscholar.org/paper/Multi-temporal-optical-VHR-image-fusion-for-Land-Paget-Gressin/8730aed1036c9ad2512724cfda6cc1c43690c720

http://www.atlantis-press.com/php/download_paper.php?id=8437

http://www.hindawi.com/journals/afs/2012/125086/

http://www.math.sk/fsta2012/presentations/dankova-perfilieva.pdf

Direct 2D fuzzy transform

Inverse 2D fuzzy transform

Thank you!