by p. muneesawang and l. guan ieee transactions...

45
by P. Muneesawang and L. Guan IEEE Transactions on Multimedia, 2004 Joseph Lee

Upload: others

Post on 14-Jul-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

by P. Muneesawang and L. GuanIEEE Transactions on Multimedia, 2004

Joseph Lee

Page 2: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

1. Backgroundg

2. General Framework

d l3. Previous Models

4. Proposed RBF Model

5. Learning Strategy

E perimental Res lts6. Experimental Results

7. Conclusion

Page 3: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Content-based Image Retrieval (CBIR)◦ use colors, shapes, textures → rely on image itself◦ without image content → rely on metadata

Page 4: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Content-based Image Retrieval (CBIR)◦ use colors, shapes, textures → rely on image itself◦ without image content → rely on metadata

Relevance Feedback◦ User refines the result by marking images as

relevant or non-relevant and repeats search.

Page 5: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Image Similarity

Page 6: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Image Similarity

Page 7: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Examples of Image Similarity Problems

MARS-1 RBF

Page 8: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Ideal Case

Page 9: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Ideal Case – User Query

Page 10: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Ideal Case – Similarity Measure

Page 11: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Real Case

Page 12: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Real Case – Inaccurate Query

Page 13: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Real Case – Inaccurate Query

Page 14: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

We should refine Query

QueryQuery

Page 15: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

We should refine Query & Metric

Metric

QueryQuery

Page 16: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Query Reformulation Model

◦ Relevance feedback → learn query representation

Adaptive Metric Model

◦ Relevance feedback → learn similarity function

Page 17: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 18: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 19: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 20: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 21: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 22: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

CBIR◦ online learning◦ two-class problem

Page 23: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

CBIR◦ online learning◦ two-class problem

Page 24: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Example Vectors

Page 25: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Voronoi Vectors

Page 26: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Voronoi Cells

Page 27: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 28: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 29: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 30: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 31: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 32: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 33: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 34: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 35: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 2)

Page 36: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 37: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 38: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 39: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 40: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Selection of RBF Width

Page 41: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Selection of RBF Width

Page 42: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Selection of RBF Width

◦◦

Page 43: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Average Precision Rate (%)

Page 44: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Non-linear model for similarity evaluation

Learning from positive and negative samples

Page 45: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

http://videolectures.net/minh_hoai_nguyen/http://www mqasem net/vectorquantization/vq htmlhttp://www.mqasem.net/vectorquantization/vq.html