pawel forczmanski "dimensionality reduction methods applied to digital image processing and...
TRANSCRIPT
![Page 1: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/1.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
1 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
Dimensionality reduction methods applied to digital image processing
and recognition
Paweł ForczmańskiChair of Multimedia Systems, Faculty of Computer Science and Information
Technology, West Pomeranian University of Technology, Szczecin
Vilnius University, Institute of Mathematics and Informatics, 18/04/2016
![Page 2: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/2.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
2 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
AgendaAgenda
Subspace concept in computer visionSubspace concept in computer vision
Application to image recognition: Eignefaces approach
Application to image recognition: Eignefaces approach
One-dimensional linear dimensionality reduction: PCA/KLT, LDA/KLT
One-dimensional linear dimensionality reduction: PCA/KLT, LDA/KLT
Two-dimensional linear dimensionality re-duction: 2DPCA/2DKLT, 2DLDA /2DKLT
Two-dimensional linear dimensionality re-duction: 2DPCA/2DKLT, 2DLDA /2DKLT
Application to image processing: watermarking, scrambling
Application to image processing: watermarking, scrambling
![Page 3: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/3.jpg)
Algo
rytm
y Roz
pozn
awan
ia W
zorc
ów
3 / 26
1 1 2 2
1 2
ˆ ...
where , ,..., is a basein the -dimensionalsub-space (K<N)K K
K
x b u b u b u
u u u K
= + + +
x̂ x=
1 1 2 2
1 2
...
where , ,..., is a basein theoriginal N-dimensionalspaceN N
n
x a v a v a v
v v v
= + + +
The problem of determining a basis in low-dimensional sub-space:− Approximation of vectors by projecting them into a new, low-dimensional sub-
space:
(1) Initial representation:
(2) Low-dimensional representation:
• Remark: if K==N, then
Subspace? (1/2)Subspace? (1/2)
where is a basis in N-dimensional space
is a basis in K-dimensional subspace (K<N)where
![Page 4: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/4.jpg)
Algo
rytm
y Roz
pozn
awan
ia W
zorc
ów
4 / 26
Subspace? (2/2)Subspace? (2/2)
Example (K==N):
![Page 5: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/5.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
5 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
PCAPCA
●Karhunen-Loève Transform●Principal Component Analysis = Hoteling Transform●How? Data decorrelation●Why? Reduce dimensionality●What for? Many applications...
![Page 6: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/6.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
6 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
introductionintroduction
1998
1902 (Pearson)
1936 (Hoteling)1987(Kirby, Sirowich)
1991 (Turk, Pentland)
One-dimensional
Two-dimensional
PCA(Principal Component Analysis)
1998 (Tsapatsoulis N.,Alexopoulos V. Kollias S.)2000, 2001, 2004
(Kukharev G., Forczmanski P.), Faculty report 2000, PRIP'2001, MG&V 2004
![Page 7: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/7.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
7 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
2DKLT/ PCArc2DKLT/ PCArc
On the input we assume L images X in grayscale of M×N pixels.
1.
2.
3.
Then we calculate a matrices of eigenvalues and a matrices of eigenvectors on the basis of covariance matrices RM i CN :
Transformation is performed as follows, where V(R) and V(C) are submatricesof W(R) and W(C):
Λ(R) ,Λ(C)
W (R ) , W (C)
Vector or matrix repre-sentation is possible
X̄ M ×N=1L∑k=1
L
X M ×N( l)
X̂ M ×N( l)
=X M×N( l )
− X̄ M×N ∀ l=1,2 ,… , L
RM=1L∑l=1
L
X̂ M×N( l ) [ X̂ M ×N
( l ) ] T ; C N=1L∑l=1
L
[ X̂ M×N(l ) ] T X̂ M ×N
( k ) ;
Y p×q l =[V M×p
R ] T X M×N l V N ×q
C
G. Kukharev, P. Forczmański, Data dimensionality reduction for face recognition, Ma-chine Graphics & Vision, vol. 13, no. 1/2, 2005, s. 99-122
![Page 8: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/8.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
8 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
Exemplary application to face Exemplary application to face recognitionrecognition
G. Kukharev, P. Forczmański, Data dimensionality reduction for face recognition, Ma-chine Graphics & Vision, vol. 13, no. 1/2, 2005, s. 99-122
X
![Page 9: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/9.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
9 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
General scheme ofGeneral scheme of2DPCA/2DKLT application2DPCA/2DKLT application
inputimage
blockdecomposition blocks 2DKLT transformed
blocks Eigenvectors
Quantization
Coding
Outputfile/stream
inverse2DKLT
blocks
composition
embedding message
message embedding (1)
rearrangement
P. Forczmański 2DKLT-Based image compression and scrambling, Congress of Young IT Scientists, 2007, s. 86-89 (Polish Journal of Environmental Studies, vol. 16, no. 4a)
P. Forczmański Information Embedding in Remotely sensed images by means of two-Two-dimensional Karhunen-Loeve Transform, Advanced Computer Systems: 14th International Conference: ACS’2007, Ol-sztyn: HARD, 2007, s. 275-279 (Polish Journal of Environmental Studies, vol. 16, no. 5B)
![Page 10: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/10.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
10 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
ArtifactsArtifacts
original 2DKLT
JPEG JPEG 2000
![Page 11: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/11.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
11 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
DCT vs 2DKLTDCT vs 2DKLT
DCT (JPEG)
2DKLT
![Page 12: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/12.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
12 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin Watermarking / steganographyWatermarking / steganography
➔ embedding watermarks and copyright information in multimedia (Digital Rights Management – DRM),
➔ hiding secret information for the safe transfer,➔ protection of data against changes.
➔ All methods work either in spatial or spectral domain (FFT, DFT, DCT, Wavelets).
➔ The most popular, yet the least sophisticated method is "Least Significant Bit (LSB) insertion"
➔ The basic problem with the LSB is a low resistance to typical image manipulations
![Page 13: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/13.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
13 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin Modification in each blockModification in each block
➔ Modification of block after 2DKLT
Message 00010101000101010001...
Original (carier) Modifier block
Bit-wide decomposition
key:
{ 4,1,
2, 3,.
..}
![Page 14: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/14.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
14 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin ExamplesExamples
Carrier image +watermark
P. Forczmański, M. Węgrzyn, Virtual Steganographic Laboratory for Digital Ima-ges, Information systems architecture and technology, Polska 2008, s. 163- 173P. Forczmański, M. Węgrzyn, Open Virtual Steganographic Laboratory Elektronika, nr 11, 2009, s. 60-65
![Page 15: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/15.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
15 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
ExperimentsExperiments
10
0
94
88
82
76
70
64
58
52
46
40
34
28
22
16
10
5
10
15
20
25
30
35
40
0%
20%
40%
60%
80%
100%
120%JPEG Compression
PSNR [dB]
Information [%]
Quality
PS
NR
-10
0-9
0
-80
-70
-60
-50
-40
-30
-20
-10 0 10
20
30
40
50
60
70
80
90
10
0
5
10
15
20
25
30
35
40
0%
20%
40%
60%
80%
100%
120%Brightness
PSNR [dB]
Information [%]
Brightness Coefficient
PS
NR
P. Forczmański Information Embedding in Remotely sensed images by means of twodimensional Karhunen-Lo-eve Transform, Advanced Computer Systems: 14th International Conference: ACS’2007, Olsztyn: HARD, 2007, s. 275-279 (Polish Journal of Environmental Studies, vol. 16, no. 5B)
0,0
2,0
4,0
6,0
8,0
10
,01
2,0
14
,01
6,0
18
,02
0,0
22
,02
4,0
26
,02
8,0
30
,03
2,0
34
,03
6,0
38
,04
0,0
42
,04
4,0
46
,04
8,0
50
,0
5
10
15
20
25
30
35
40
0%
20%
40%
60%
80%
100%
120%Additive noise
PSNR [dB]
Information [%]
Noise Amplitude
PS
NR
0,0
5
0,2
5
0,4
5
0,6
5
0,8
5
1,0
5
1,2
5
1,4
5
1,6
5
1,8
5
2,0
5
2,2
5
2,4
5
2,6
5
2,8
5
3,0
5
3,2
5
3,4
5
3,6
5
3,8
5
5
10
15
20
25
30
35
40
0%
20%
40%
60%
80%
100%
120%Contrast
PSNR [dB]
Information [%]
Contrast Coefficient
PS
NR
![Page 16: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/16.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
16 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
ScramblingScrambling
original
Scrambled image
Recovered image #1
Recovered image #2?
?
![Page 17: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/17.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
17 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
Linear Discriminant Analysis (5/6)Linear Discriminant Analysis (5/6)
D. Swets, J. Weng, "Using Discriminant Eigenfeatures for Image Retrieval", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 831-836, 1996
PCA LDA
![Page 18: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/18.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
18 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
LDA : Algorithm (1)LDA : Algorithm (1)
Let us assume input images X are grayscale, gathered in K classes, each one having L objects.We calculate means for each K class and one common, for all classes:
Then, covariance matrices are calculated:
![Page 19: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/19.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
19 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
LDA : Algorithm (2)LDA : Algorithm (2)
Total covariance matrix is:
Which is decomposed using eigen-approach:
where Ω – diagonal of eigen values and U – orthogonal matrix having eigenvectors
Transform matix is created from U by selecting sub-matrix with s columns related to the highest values in Ω.
Ω pxp
→ F sxs
![Page 20: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/20.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
20 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
LDA : Algorithm (3)LDA : Algorithm (3)
Dimensionality reduction is applied in two-step process:
1. initial reduction (down-sampling, DCT/DFT, PCA)2. final LDA transformation LDA:
![Page 21: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/21.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
21 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
LDA: classification of texturesLDA: classification of textures
K. Okarma, P. Forczmański, 2DLDA-based texture recognition in the aspect of objec-tive image quality assessment Annales Universitatis Mariae Curie-Skłodowska. Sectio AI Informatica, vol. 8, no. 1, 2008, s. 99-110
Distortion Recognition accuracy
Nearest element Centers of classes
Median 3x3 81.33 % 71.59 %
Median 5x5 62.18 % 57.79 %
Low-pass 3x3 71.47 % 63.37 %
Low-pass 5x5 46.83 % 47.32 %
5% impulse noise 64.29 % 60.88 %
10% impulse noise 47.24 % 47.89 %
15% impulse noise 38.47 % 38.80 %
20% impulse noise 30.03 % 31.33 %
JPEG 60% 89.77 % 78.08 %
JPEG 40% 90.10 % 77.11 %
JPEG 20% 89.95 % 76.82 %
JPEG 10% 88.96 % 76.14 %
![Page 22: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/22.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
22 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
LDA : limitationsLDA : limitations
Classical LDA method requires to carry out a preliminary dimensionality re-duction of input data, eg. by means of sampling (down-sampling) or PCA / PCArc. It is required to meet the condition:
where K – no. classes, L- no. Images in class, DIM – dimensionality of fe-ature-space.
G. Kukharev, P. Forczmański, Two-Dimensional LDA Approach to Image Compression and Re-cognition, Computing, Multimedia and Intelligent Techniques, vol.2, no. 1, 2006, s.87-98
G. Kukharev, P. Forczmański, Face Recognition by Means of Two-Dimensional Direct Linear, Discriminant Analysis Pattern recognition and information processing: PRIP ’2005: Proceedings of the Eighth International Conference, 18-20 Maj, Mińsk, Białoruś 2005, s.280-283
![Page 23: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/23.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
23 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
2DLDA/LDArc (1)2DLDA/LDArc (1)
The solution to this problem is to use 2DLDA (LDArc), which involves the decomposition of the image into a set of rows and columns and calculating 2 sets of covariance matrices:
![Page 24: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/24.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
24 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
2DLDA/LDArc (5)2DLDA/LDArc (5)
Transformation is done using the following formula:
Exemplary LDA spectra and the reconstruction is presented below:
![Page 25: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/25.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
25 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
2DLDA/LDArc: Facial recognition2DLDA/LDArc: Facial recognition
G. Kukharev, P. Forczmański, Facial images dimensionality reduction and recognition by means of 2DKLT, Machine Graphics & Vision, vol. 16, no. 3/4, 2007, s. 401-425
![Page 26: Pawel FORCZMANSKI "Dimensionality reduction methods applied to digital image processing and recognition"](https://reader031.vdocuments.mx/reader031/viewer/2022030220/58850af71a28ab7d698b7653/html5/thumbnails/26.jpg)
Eras
mus
+ s
emin
ar, 18
/04/
2016
26 / 26
Faculty of ComputerScience andInformationTechnology
West Pomeranian University of Technology,Szczecin
Thank you for your attenttion!
Any questions?
Paweł ForczmańskiChair of Multimedia Systems, Faculty of Computer Science and Information
Technology, West Pomeranian University of Technology, Szczecin
Vilnius University, Institute of Mathematics and Informatics, 18/04/2016
??