thesis writing - week9
DESCRIPTION
s1160123 Tomoyuki SoetaTRANSCRIPT
![Page 1: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/1.jpg)
1
s1160123 Tomoyuki Soeta
Supervised by Prof. Qiangfu Zhao
System Intelligence Lab
Information Retrieved
- Image based search
![Page 2: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/2.jpg)
2
Outline
Introduction
Information Retrieval
VQ (Vector Quantization)
Divide into the 8x8 block
Making of Code book
K-means algorithm
Extract each image’s feature vector
Result
Image and feature vector
Distance of feature vector
Conclusion
Future work
![Page 3: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/3.jpg)
3
Introduction
I want to aim at the improvement of information retrieval system to search it even if the input data are documents or images.
I have charge of a research on information retrieval based on a image.
To search images using a search engine, we may use the index attached to the image, the file name, etc. as the key-words. We may also use "the contents of an image themselves."
I study a new image search technique based on the code book information.
![Page 4: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/4.jpg)
4
Information Retrieved
Text Image
Word Filtering
Morphological
Analysis
Divide into the block
(1 block 8x8)
Feature Vector Feature Vector
NNTree or SVM
Code book
Code of each block
![Page 5: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/5.jpg)
5
VQ (Vector Quantization)
Compression coding of images
Image compression technology
In my study, I use VQ to translate an image into a bag-of-blocks (BOB)
feature vector
image
Vector
Quantization
(VQ)
same way as document search
![Page 6: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/6.jpg)
6
Divide into the 8x8 block (1)
I used 10 facial images with the size 256x256.
images are converted to gray scale images.
Divided into the block (one-block 8x8 size).
Each image obtains the block of 32×32
pieces severally.
1 block 8x8
32 blocks
32
b
l
o
c
k
s
![Page 7: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/7.jpg)
7
Divide into the 8x8 block (2)
Block’s pixel value is read.
Pixel read value is stored in the array of 1x64.
One image can be divided into 1024 blocks,
and an array of 1024 rows can be obtained.
1 block 8x8
2 3 4 6 8 3 7 2
8 2 8 2 8 2
2 3 4 6 8 3 7 2 8 2 8 2
・・・・
1x64
8x8 1024 rows
![Page 8: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/8.jpg)
8
10 images
Image 0 Image 4 Image 3 Image 2 Image 1
Image 5 Image 6 Image 7 Image 8 Image 9
![Page 9: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/9.jpg)
9
Making of Code book
The array of 10240 that can be done
by reading 10 images is made
The code book is made by using the
k-means method.
Making Code book (size 256)
![Page 10: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/10.jpg)
10
K-means algorithm
Step 1) k initial "means" are randomly selected from
the data set .
Step 2) k clusters are created by associating every
observation with the nearest mean.
Step 3) The centroid of each of the k clusters
becomes the new means.
Step 4) Steps 2 and 3 are repeated until
convergence has been reached.
Step 1 Step 2 Step 3 Step 4
![Page 11: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/11.jpg)
Extract each image’s feature vector (1)
The feature vector are extracted by using code book.
There is arrangement 1024 per one image.
Arranging an individual distance of the array each one and code book is measured
The number of the nearest code is returned.
Which code how many times came out is preserved as an array.
11
・・・
1x64
1024 rows
2 3 4 7 8 9 2 # # # # # # # # # #
Code book
Code 7 Code 38 Code 72 Code 200 Code 7
Code 7
Code 38
Code 72
Code 200
Code 7 0
1
2
3
4
5
1 2567 38 72 200
![Page 12: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/12.jpg)
Result – image and feature vector(1)
12
Image 0 Image 1
![Page 13: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/13.jpg)
Result – image and feature vector(2)
13
Image 2 Image 3
![Page 14: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/14.jpg)
Result – image and feature vector(3)
14
Image 4 Image 5
![Page 15: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/15.jpg)
Result – image and feature vector(4)
15
Image 6 Image 7
![Page 16: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/16.jpg)
Result – image and feature vector(5)
16
Image 8 Image 9
![Page 17: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/17.jpg)
Euclidean distance between feature vectors is measured, and the accuracy of the code book is seen.
17
Result - Distance of feature vector(1)
P and Q are assumed to be two feature vectors.
Data : x = (x1, x2, ..., xn) and y = (y1, y2, ..., yn)
n : size of the feature vector
The distance of P and Q is below.
![Page 18: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/18.jpg)
:minimum distance
18
Result - Distance of feature vector(2)
256 feature0 feature1 feature2 feature3 feature4 feature5 feature6 feature7 feature8 feature9
feature0 0 0.279945 0.280761 0.226158 0.291376 0.322875 0.300502 0.2307 0.23509 0.228708
feature1 0.279945 0 0.19849 0.271927 0.318353 0.352126 0.324807 0.272823 0.269333 0.30847
feature2 0.280761 0.19849 0 0.308124 0.352732 0.378846 0.359333 0.310492 0.316141 0.324054
feature3 0.226158 0.271927 0.308124 0 0.221109 0.276269 0.240734 0.09959 0.086469 0.136439
feature4 0.291376 0.318353 0.352732 0.221109 0 0.222279 0.17478 0.202749 0.210865 0.248531
feature5 0.322875 0.352126 0.378846 0.276269 0.222279 0 0.084866 0.282603 0.270858 0.306136
feature6 0.300502 0.324807 0.359333 0.240734 0.17478 0.084866 0 0.245255 0.232873 0.276931
feature7 0.2307 0.272823 0.310492 0.09959 0.202749 0.282603 0.245255 0 0.105974 0.155957
feature8 0.23509 0.269333 0.316141 0.086469 0.210865 0.270858 0.232873 0.105974 0 0.152093
feature9 0.228708 0.30847 0.324054 0.136439 0.248531 0.306136 0.276931 0.155957 0.152093 0
Image 6 Image 5
The image5 and image6 is the same persons, image5 doesn't wear glasses, and image6 wears glasses.
Between feature5 and feature6 is minimum distance.
![Page 19: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/19.jpg)
Conclusion
In my research, I study a new image search technique based on the code book information. The code book is obtained using the VQ method.
It is thought that an accurate feature vector was able to be extracted about the accuracy of the feature vector because the distance between Feature5 and 6 was short.
19
Information retrieval based on
"the contents of a image themselves."
![Page 20: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/20.jpg)
20
Future work
The background is nullified.
The feature vector is extracted in the block of a different size like the block of not the block of 8x8 size but 16x16 size etc.
Multimedia retrieval that uses SVM.
![Page 21: Thesis writing - week9](https://reader033.vdocuments.mx/reader033/viewer/2022052621/557ea892d8b42ac5658b4e13/html5/thumbnails/21.jpg)
21
Thank you for your attention!