transform methods in image processing: …yaro/recentpublications/ps&pdf...transform methods in...

49
TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel Image Restoration Target Location Image Resampling

Upload: others

Post on 10-Jun-2020

12 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

TRANSFORM METHODSin IMAGE PROCESSING:

L. Yaroslavsky,Dept. of Interdisciplinary Studies, Faculty of Engineering,

Tel Aviv University, Tel Aviv, Israel

Image Restoration

Target Location

Image Resampling

Page 2: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

TRANSFORM DOMAIN ADAPTIVEFILTERS FOR IMAGE RESTORATION

• Sliding Window DCT Filters

• Wavelet shrinkage

• Hybrid SWDCT/Wavelet filters

• Comparison and Interpretation

Page 3: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Sliding window transform domain (SWTD) filtersR. Yu. Vitkus, L.P. Yaroslavsky, Recursive Algorithms for Local Adaptive Linear Filtration, In: Mathematical Research.Computer Analysis of Images and Patterns, ed. by L.P. Yaroslavsky, A. Rosenfeld, W. Wilhelmi, Band 40, Academie Verlag,Berlin, 1987, p. 34-39.

Page 4: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Filters for image deblurring and denoising (signal independent noise):

====

≠≠≠≠

−−−−

====

00

0022

r

rrrroptr

,

,AV/,thrAVmax

λλλλ

λλλλββββλλλλββββηηηη

Filters for image deblurring and denoising (signal dependent noise):

(((( ))))

====

≠≠≠≠

⋅⋅⋅⋅−−−−

====

00

002

02

r

rrrp

roptr

,

,AV/,thrAVmax

λλλλ

λλλλββββλλλλββββββββηηηη

Rejective filters:

>>>>

====otherwise,

thrAVif,/ rroptr

0

12

ββββλλλληηηη .

“Fractional” spectrum filter: 11

0

121

0

−−−−====

>>>>

====

====−−−− N,,r,

otherwise,

thrAVif, rp

rr

ββββββββηηηη

ηηηη

ADAPTIVE FILTERS WITH EMPIRICALESTIMATION OF SPECTRA

Page 5: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

The selection of the transform for the filter implementation is governed by• A priory knowledge about image spectra in the transform domain• Accuracy of empirical spectrum estimation• Transform energy compaction capability• Computational complexity of the filtering in the transform domain

Feasible transforms: DFT, DCT, DST, Haar, Walsh, wavelets.

SELECTION OF THE TRANSFORM

2-D basis functions of (left to right) DCT, Walsh, Haar Transforms

Page 6: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Odds in favor of DCT:

☻Good energy compaction capability

☻Good suitability for signal/image restoration taskswith signaling and imaging system specification interms of their frequency responses

☻Suitability for multi component signal/imageprocessing

☻Low computational complexity: recursivecomputing SWDCT is possible with the complexityof O(number of coefficients required)

Page 7: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Sliding window DCT (odd window size).L.P. Yaroslavsky, Local adaptive image restoration and enhancement with the use of DFT and DCT in a running window,Proceedings, Wavelet Applications in Signal and Image Processing IV, 6-9 August 1996, Denver, Colorado, SPIE Proc.Series, v. 2825, pp. 1-13.

Direct transform:

(((( ))))(((( ))))

(((( ))))∑∑∑∑−−−−

====−−−−++++

++++====

1

02

2121 wN

n/Nfixnk

w

kr r

N/ncosa

Nππππαααα

Inverse transform for the window central pixel:

(((( )))) (((( ))))(((( ))))

++++++++==== ∑∑∑∑

−−−−

====

1

10

212221 wN

r

kr

kk N

r//NfixcosN

a ππππαααααααα

For odd N:

(((( ))))(((( ))))(((( ))))

−−−−++++==== ∑∑∑∑−−−−

====

21

120 12

21

/N

r

rkr

k

wk

w

Na αααααααα

Therefore, only DCT coefficients with even indices are relevant

Page 8: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Sliding window DCT: recursive computation

For adjacent k-th and (k+1)-th window positions, DCT spectra (((( )))){{{{ }}}}krαααα and (((( )))){{{{ }}}}1++++k

rαααα are

(((( ))))(((( ))))

(((( ))))∑∑∑∑−−−−

====−−−−++++

++++====

1

02

2121 wN

n/Nwfixnk

w

kr r

N/ncosa

Nππππαααα

and

(((( ))))(((( ))))

(((( ))))∑∑∑∑−−−−

====−−−−++++++++

++++

++++====

1

021

1 21w

w

N

n/Nfixnk

kr r

N/ncosa ππππαααα .

Introduce auxiliary spectra

(((( ))))(((( ))))

(((( ))))∑∑∑∑−−−−

====−−−−++++

++++====

1

02

21w

w

N

n/Nfixnk

kr r

N/niexpa~ ππππαααα

Spectrum (((( )))){{{{ }}}}1++++kr

~αααα can be represented through spectrum (((( )))){{{{ }}}}kr

~αααα :(((( )))) (((( )))) (((( )))) (((( ))))(((( )))) (((( ))))[[[[ ]]]] (((( ))))N/riexpaaN/riexp~~

/Nfixkr

/NfixNkwk

rk

r www21 22

1 ππππππππαααααααα −−−−−−−−−−−−++++−−−−==== −−−−−−−−++++++++ .

Therefore, axiliary spectra (((( )))){{{{ }}}}kr

~αααα can be computed recursively and local DCT spectra (((( )))){{{{ }}}}krαααα can

then be found from the relationship:(((( )))) (((( ))))(((( ))))k

rk

r~real αααααααα ==== .

Page 9: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD DCT filtering for image restorationL.P. Yaroslavsky, Local adaptive image restoration and enhancement with the use of DFT and DCT in a running window(invited paper), in: Proceedings, Wavelet Applications in Signal and Image Processing IV, 6-9 August 1996, Denver,Colorado, SPIE Proc. Series, v. 2825, pp. 1-13

Page 10: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD DCT filtering for image blind deblurring

Page 11: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD DCT filtering speckle noiseLeonid P. Yaroslavsky, Ben-Zion Shaick Transform Oriented Image ProcessingTechnology for Quantitative Analysis ofFetal Movements in Ultrasound Image Sequences. In: Signal Processing IX. Theories and Applications, Proceedings ofEusipco-98, Rhodes, Greece, 8-11 Sept., 1998, ed. By S. Theodorisdis, I. Pitas, A. Stouraitis, N. Kalouptsidis, TyporamaEditions, 1998, p. 1745-1748

Page 12: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Sliding Window DCT 3x3

Involved transform basis functions:

====

22022000

DCTDCTDCTDCT

−−−−−−−−−−−−

−−−−

−−−−−−−−−−−−

−−−−−−−−−−−−

121242

121

111222

111121121121

111111111

;

Basis functions 20DCT , 02DCT , and 22DCT are kernels of 2-D Laplacians:horizontal, vertical and isotropic ones. One can further decompose basisfunction 22DCT into a sum of two diagonal Laplacians:

−−−−−−−−−−−−−−−−

−−−−−−−−++++

−−−−−−−−−−−−−−−−−−−−−−−−

====112121

211

211121112

22DCT

Page 13: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD DCT3x3: Restoration of high resolutionsatellite images L. Yaroslavsky, High Resolution Satellite Image Restoration with the Use of Local Adaptive Linear Filters, Report onKeshet Program, July, University Dauphine, Ceremade, Paris,1997

Spot-image: before Spot-image: after

Page 14: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Potentials of SWTD DCT filtering

Initial noise levelTest image15 30 60

Rej. Fltr 6.1 8.1 10.8Emp. Wiener 6 8.1 10.8Av.Rej.Fltr 5 7.1 10.4

Aero1

Id.Wiener 3.6 5.6 9.2Rej. Fltr 5.6 6.4 10Emp. Wiener 5.6 6.4 10.4Av.Rej.Fltr 5 6 8.2

Aero2

Id.Wiener 3.3 5 8.2Rej. Fltr 10.4 15 21Emp. Wiener 9.3 14.1 19Av.Rej.Fltr 8 11.9 17.5

Lenna

Id.Wiener 5.8 8.4 13

Standard. deviation of residual noise for 3 test noisy images

(R. Oktem, L. Yaroslavsky, K. Egiazarian, Evaluation of Potentials for Local Transform Domain Filters with varyingParameters, EUSIPCO2000, Tampere, Finland, Sept. 3-8, 2000)

Page 15: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Wavelet Shrinkage

Low passfiltering and

downsamplingInterpolation

-

+ + Soft/hardthresholding

Interpolation

+

+ +

+

+ +

InputOutput

Interpolation

-

+

+

Interpolation

Low passfiltering and

downsampling

Low passfiltering and

downsampling

Interpolation Interpolation

Soft/hardthresholding

Soft/hardthresholding

D. L. Donoho, I. M. Johnstone, Ideal Spatial Adaptation by Wavelet Shrinkage, Biometrica, 81(3), pp. 425-455, 1994

Page 16: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Wavelet Shrinkage and Empirical Wiener Filtering

Empirical Wiener Denoising Filter

(((( ))))2

220

r

rroptr

,max

ββββ

ννννββββηηηη

−−−−====

Rejective filter:

>>>>====

otherwise,thrif, ropt

r0

12

ββββηηηη .

Wavelet Shrinkage Filter (soft threshold)

(((( ))))r

roptr

,thrmaxββββ

ββββηηηη

0−−−−====

Wavelet Shrinkage Filter (hard threshold)

>>>>====

otherwise,thrif, ropt

r 01 ββββηηηη .

Page 17: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Wavelet vs SWDCT denoising: comparisonR. Oktem, L. Yaroslavsky, K. Egiazarian, Signal and Image Denoising in Trandform domain and Wavelet Shrinkage: AComparative Study, In: In: Signal Processing IX. Theories and Applications, Proceedings of Eusipco-98, Rhodes, Greece,8-11 Sept., 1998, ed. By S. Theodorisdis, I. Pitas, A. Stouraitis, N. Kalouptsidis, Typorama Editions, 1998, p. 2269-2272

Signals ImagesFilteringmethod ECG Piece-wise

constantLena Piece-

wisecomst

antimage

Aerophot

o 1

Aerophot

o 2

MRI

RMSE MAE RMSE MAE PSNR

PSNR

PSNR

PSNR

PSNR

WL-Haar 0.06 0.042 0.042 0.024 26.72 30.0 26.9 32.3 30.9

WL-Db4 0.052 0.038 0.052 0.035 26.2 27.8 27.6 33.3 31.5

SWDCT 0.04 0.028 0.055 0.038 28 27.8 29 34.6 32.7

SWHaar 0.052 0.033 0.037 0.022 27.4 29.5 28.1 34.2 31.8

Page 18: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Hybrid WaveLet-SWTD DCT filtering

scale 1

scale 2

scale 3

scale 4

Effective basis functions in multi resolution DCT

Low passfiltering and

downsamplingInterpolation

-

+ + SWDCT3x3filtering

Interpolation

+

+ +

+

+ +

InputOutput

Interpolation

-

+

+

Interpolation

Low passfiltering and

downsampling

Low passfiltering and

downsampling

Interpolation Interpolation

SWDCT3x3filtering

SWDCT3x3filtering

B.Z. Shaick, L. Ridel, L. Yaroslavsky, A hybrid transform method for image denoising, EUSIPCO2000, Tampere, Finland,Sept. 5-8, 2000

Page 19: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD-DCT, Wavelet-Shrinkage andHybrid filtering: Performance comparison

Page 20: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

MWTD-DCT, Wavelet-Shrinkage andHybrid filtering: Performance Comparison

FilterP-W

const.image

Lennaimage MRI Air photo

Hard 8.1(3×3)

9.4(3×3)

6.7(5 ×5)

8.2(3 ×3)DCT

Soft 7.5(3×3)

8.6(5×5)

6.3(7 ×7)

7.7(3 ×3)

Hard 8.6(binom5)

10.1(binom5)

8.5(binom5)

9.3(binom5)WL-

Shrinkage. Soft 8.4

(binom5)9.0

(qmf13)7.8

(binom5)8.1

(binom5)

Hard 8.7(binom5)

9.4(binom5)

6.6(binom5)

8.2(binom5)Hyb

rid Soft 7.9(binom5)

8.6(binom5)

6.2(binom5)

7.5(binom5)

Standard. deviation of residual noise for 4 test noisy images(initial St. Dev. 13; optimized filter parameters)

Page 21: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD “time-frequency” signal representationand subband decomposition

Signal transform {{{{ }}}}ka ( 110 −−−−==== N,,,k ) in a sliding window of width wN over set of basis functions (((( )))){{{{ }}}}nrττττ ( 110110 −−−−====−−−−==== ww N,,,r;N,,,n ),

(((( ))))(((( )))) (((( ))))∑∑∑∑

−−−−

====−−−−++++====

1

02

w

w

N

nr/Nfixnk

kr na τττταααα

Its DFT over index k: (((( )))) (((( )))) ====

====ΑΤΑΤΑΤΑΤ ∑∑∑∑

−−−−

====

1

02exp1 N

k

kr

r

Nkfi

Nfππππαααα (((( )))) (((( ))))∑∑∑∑ ∑∑∑∑

−−−−

====

−−−−

====−−−−++++

1

0

1

02/ 2exp1 w

w

N

n

N

kNfixnkr N

kfianN

ππππττττ

Then:

(((( )))) (((( ))))rf

rff

ΤΤΤΤΑΑΑΑ∝∝∝∝ΑΤΑΤΑΤΑΤ ,

where ∑∑∑∑−−−−

====

====ΑΑΑΑ

1

02exp1 N

kkf N

kfiaN

ππππ ;

(((( )))) (((( ))))∑∑∑∑−−−−

====

−−−−

====ΤΤΤΤ1

02exp1 N

nr

w

r fNnin

Nnrect

Nfππππττττ ,

Page 22: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWDCT7x7 subbands

Map of standard deviations of DCT7x7subbands

0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

1

0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

1

1 . 1

0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1

0 . 2

0 . 4

0 . 6

0 . 8

1

1 . 2

1 . 4

1 . 6

0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 10

1

2

3

4

5

SWDCT7x7 subbands (1-D slice)

Frequencies within the bandwidth

Page 23: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SWTD DCT, Walsh and Haar: subband decompositions

Page 24: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Subbands (1-D) of Binom5 wavelets (left)and hybrid Binom5&SWDCT3x3 (right)

Page 25: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

TARGET LOCATION IN CLUTTER:Optimal Adaptive Correlators

and Transform Methods

Page 26: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Problem formulation

S1

S2

S3

S4

S5

S6

S7Object(x0,y0)

(((( ))))××××==== ∫∫∫∫ ∑∑∑∑ ∫∫∫∫∫∫∫∫∞∞∞∞

∞∞∞∞−−−− ====

K

10000 ,)(

k Skka

k

yxwWdbbpP

(((( ))))(((( )))) 00000

,/ dydxdbyxbhb

s

∫∫∫∫∞∞∞∞

bgAVAVims

(((( )))) 1, 0000 ====∫∫∫∫∫∫∫∫ dydxyxwkS

k ; 11

====∑∑∑∑====

K

kkW

Block diagram of the localization device

Linear filter Device for locating

signal maximum

Input image

Object’s coordinates

L.P. Yaroslavsky, The Theory of Optimal Methods for Localization of Objects in Pictures, In: Progress in Optics, Ed. E.Wolf, v.XXXII, Elsevier Science Publishers, Amsterdam, 1993

Page 27: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Optimal Adaptive Correlators:

Additive model:(((( )))) (((( )))) (((( ))))yxayyxxayxb bg ,,, 00 ++++−−−−−−−−====

(((( )))) (((( )))) (((( ))))αααα ββββ ααααbg x y x y x yf f f f f f, , ,2 2 2

≅≅≅≅ ++++

(((( ))))====yxaopt ffH ,

(((( ))))(((( )))) (((( ))))22

,,

,

yxyx

yx

ffff

ff

ααααββββ

αααα

++++

∗∗∗∗

Implant model

≅≅≅≅),( yximplopt ffH

(((( ))))(((( )))) (((( ))))∫∫∫∫ ∫∫∫∫ −−−−−−−−

≅≅≅≅∗∗∗∗

x yF Fyxyxyyxx

yx

dpdppppfpf

ff22

,,

,

ββββ

αααα

bgW

(((( )))) (((( )))) (((( ))))yxbyyxxwyxa bgbg ,,, 00 −−−−−−−−====

(((( ))))====−−−−−−−−

objecttargetthewithin,0objecttargettheoutside,1

, 00 yyxxwbgIn the assumption of uniformdistribution of target co-ordinates:

⇓⇓⇓⇓⇓⇓⇓⇓

In the assumption of uniformdistribution of target co-ordinates:

(((( )))) (((( )))) (((( ))))∫∫∫∫ ∫∫∫∫ −−−−−−−−≅≅≅≅x yF F

yxyxyyxxyxbg dpdppppfpfff222

,,, ββββαααα bgW

Zero order approximation: imgob SS <<<<<<<<

(((( )))) (((( ))))22, ,,

00 yxyxbgyx ffffAV ββββαααα ≅≅≅≅ (((( )))) (((( ))))(((( ))))2

0

,

,,

yx

yxyxopt

pp

ffffH

ββββ

αααα∗∗∗∗

====

Page 28: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Optimal adaptive correlators: detection of“small” objects

Detection and enhancement of microcalcifications in a mammogram

Input mammogram

Page 29: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Optimal adaptive filter and matched filtercorrelators: sensitivity to object rotation

5 0 1 0 0 1 5 0 2 0 0 2 5 0

0

0 . 2

0 . 4

0 . 6

0 . 8

1

5 0 1 0 0 1 5 0 2 0 0 2 5 0

0

0 . 2

0 . 4

0 . 6

0 . 8

1

5 0 1 0 0 1 5 0 2 0 0 2 5 0

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

1

5 0 1 0 0 1 5 0 2 0 0 2 5 0

0 .6

0 .7

0 .8

0 .9

1

Upper row Left column

Optimal adaptive correlator

Matched filter correlator

Page 30: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Object tracking in video sequencies: examples

Ultrasound movie: fetus movements Video movie: infant movements

For details seehttp://www.eng.tau.ac.il/~yaro

Leonid P. Yaroslavsky, Ben-Zion Shaick Transform Oriented Image Processing Technology for Quantitative Analysis of Fetal Movements in UltrasoundImage Sequences. In: Signal Processing IX. Theories and Applications, Proceedings of Eusipco-98, Rhodes, Greece, 8-11 Sept., 1998, ed. By S.Theodorisdis, I. Pitas, A. Stouraitis, N. Kalouptsidis, Typorama Editions, 1998, p. 1745-1748

Page 31: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Optimal localization in spatial inhomogeneousimages:Local adaptive correlator(L. Yaroslavsky, Local Adaptive Filtering in Transform Domain for Image Restoration, Enhancement and Target Location, in: 6th

Int. Workshop on Digital Image Processing and Computer Graphics (DIP-97), Em. Wenger and L. Dimitrov, eds., 20-22 Oct.,1997, Vienna, Austria, SPIE vol. 3346

T a rg e t (h ig h lig h te d )T h e re s u lt o f lo c a liz a tio n (m a rk e d w ith a c ro s s )

5 0 1 0 0 1 5 0 2 0 0 2 5 0

-3 0

-2 0

-1 0

0

1 0

2 0

3 0

4 0

5 0

C o rre la to r´s o u tp u t c ro s s s e c tio n (c o lu m n )

2 0 4 0 6 0 8 0 1 0 0 1 2 0

-3 0

-2 0

-1 0

0

1 0

2 0

3 0

4 0

5 0

C o rre la to r´s o u tp u t c ro s s s e c tio n (ro w )

Localization of a small (8x8 pixels, highlighted) fragment in stereoscopic images bythe local adaptive filter. Running window size is 32x32 pixels; image size is 128x256pixels. The same filter applied globally rather than locally fails to locate the fragment.

Page 32: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Nonlinear optical correlators: 4F k-th lawcorrelator(L.P. Yaroslavsky, Optical correlators with (-k)th law nonlinearity: optimal and suboptimal solutions, Applied Optics, v. 34,No. 20 (10 July, 1995), pp. 3924-3932)

Input image Fourier lens Fourier lensSpatial filter Correlation plane

Coherentillumination

F F F F

k-th law nonlinearityF(.)=(.)k

Page 33: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Nonlinear optical correlators:Joint Transform Correlators(L. Yaroslavsky, E. Marom, Nonlinearity Optimization in Nonlinear Joint Transform Correlators , Applied Optics , vol. 36,No. 20, 10 July, 1997, pp. 4816-4822)

Inputimage

Referenceobject

TVcamera

Nonlinear

Amplifier

f(.)=log(.);

f(.)=(.)1/k

SLM

interface

Spatial lightmodulator

Spatial lightmodulator

Fourierlens

Fourierlens

Outputplane

Page 34: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Color correlators: 3-channel correlatorwith component conversion(L. Yaroslavsky, Optimal target location in color and multi component images, Asian Journal of Physics, Vol. 8,No 3 (1999) 355-369)

Page 35: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Discrimination capability of the colorcorrelator with component conversion

Page 36: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

FAST SIGNALSINC-INTERPOLATION

Page 37: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Image resampling: basic principle

OutputimageInput

image

Page 38: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Sinc-interpolation: zero padding method

Interpolation functions:

(((( ))))(((( ))))∑∑∑∑−−−−

====

====1

01

1 N

k1kk Lk-kN;M;sincda

La~

1;

(((( )))) (((( ))))(((( ))))N/xsinN

N/KxsinxN;K;sincdππππ

ππππ==== ; 110 −−−−==== LN,...,,k

Page 39: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Shifted Discrete Fourier Transforms as discreterepresentations of the Fourier integral:

x

a(x) Sampled signal

x∆∆∆∆u

f

(((( ))))fααααSampled signal spectrum

f∆∆∆∆

v

(((( )))) (((( ))))(((( ))))xukxaxaN

kreconstr_signk ∆∆∆∆++++−−−−==== ∑∑∑∑

−−−−

====

1

0ϕϕϕϕ

(((( )))) (((( ))))(((( ))))fvrffN

kreconstr_spnr ∆∆∆∆++++−−−−==== ∑∑∑∑

−−−−

====

1

0ϕϕϕϕαααααααα

Continuos signal

Continuous signalspectrum

(((( )))) (((( )))) (((( ))))dxfxiexpxaf ∫∫∫∫∞∞∞∞

∞∞∞∞−−−−

==== ππππαααα 2 (((( ))))(((( ))))∑∑∑∑−−−−

====

++++++++====

1

021 N

kk

v,ur N

vrukiexpaN

ππππαααα

Fourier integral Shifted DFT (canonic form)Signal and spectrum sampling

(((( ))))∑∑∑∑−−−−

====

++++

====

1

0221 N

kk

v,ur N

rukiexpNkviexpa

Nππππππππαααα (((( ))))∑∑∑∑

−−−−

====

++++

−−−−====

1

0221 N

kk

v,ur N

vrkiexpNruiexpa

Nππππππππαααα

Direct and inverse Shifted DFTs (reduced form)

L.P. Yaroslavsky, Shifted Discrete Fourier Transforms, In: Digital Signal Processing, Ed. by V. Cappellini, and A. G.Constantinides, Avademic Press, London, 1980, p. 69- 74.

Page 40: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Block diagram of signal fast sinc-interpolationL. P. Yaroslavsky, Signal sinc-interpolation: a fast computer algorithm, Bioimaging, 4, p. 225-231, 1996

Page 41: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

SDFT versus zero padding sinc-interpolation

Zero padding method SDFT based method

Computational complexity (generaloperations) of L-fold zooming signal of Nsamples with the use of FFT

(((( ))))NLlogNLO (((( ))))NlogNLO

Computational complexity (generaloperations) of L-fold zooming signal of Nsamples in the vicinity of an individualsample (as, for instance, in locating positionof signal maximum with subpixel accuracy )

(((( ))))NLlogNLO ,unless FFT pruned algorithms are used

(((( ))))NLO

Computational complexity (generaloperations) for signal shift by a fraction ofthe discretization interval

(((( ))))NLlogNLO ,unless FFT pruned algorithms are used;shift only by (power of 2)-th fraction ofthe discretization interval are possiblewhen the most wide spread FFTalgorithms are used.

(((( ))))NlogNO ;arbitrary shifts arepossible

Zoom factor Power of 2 for the most widely usedFFT algorithms

Arbitrary

Memory usage Requires an intermediate buffer for NLsamples

Does not need anintermediate buffer

Page 42: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Spectrum analysis with sub-pixel resolution

Page 43: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Three pass image rotation with sinc-interpolation

Initia l image Firs t pass

Second pass Third pass: rotated image

M. Unser, P. Thevenaz, L. Yaroslavsky, Convolution-based Interpolation for Fast,High-Quality Rotation of Images, IEEE Trans. on Image Processing, Oct. 1995, v. 4, No. 10, p. 1371-1382

Page 44: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Fast image rotation algorithm in RadonTransform and tomosynthesis

Radon transform: rotationand directional summation

Tomographicreconstruction: ramp-filtering projections, backprojecting, rotation andsummation

Page 45: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Image geometrical transformations by means ofsinc-interpolated image zooming (oversampling)

Sinc-interpolatedsubsampling

Initial image

Magnified image

Transformed image copies

Page 46: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Image geometrical transformations with sinc-interpolation

Radius

Angle

Cartesianto polar

Polar toCartesian

Page 47: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Signal sinc-interpolation in sliding window

1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0

- 0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0

- 0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 . 6

0 . 7

0 . 8

0 . 9

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.5

1

1.5

2

2.5

3

3.5

Window11 Window15

Interpolation functions for window size 11 and 15 samples Interpolation filter frequency responses

Page 48: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

Image zoom: global versus sliding window sinc-interpolation

Page 49: TRANSFORM METHODS in IMAGE PROCESSING: …yaro/RecentPublications/ps&pdf...TRANSFORM METHODS in IMAGE PROCESSING: L. Yaroslavsky, Dept. of Interdisciplinary Studies, Faculty of Engineering,

www.eng.tau.ac.il/~yaro