Download - DE Conferentie 2005 Egon v.d. Broek
![Page 1: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/1.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 2: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/2.jpg)
Multilevel Access to our Cultural Heritage (MACH)
People who participated in the project
• Egon L. van den Broek (AI, VU Amsterdam /
NICI, RU Nijmegen)• Thijs Kok (ICIS, RU Nijmegen)
• Eduard Hoenkamp (NICI, RU Nijmegen)• Theo E. Schouten (ICIS, RU Nijmegen)• Peter J. Petiet (AI, VU Amsterdam)• Louis G. Vuurpijl (NICI, RU Nijmegen)
![Page 3: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/3.jpg)
Multilevel Access to our Cultural Heritage (MACH)
In addition…The people who worked on the foundation of M4ART
• Frans Gremmen (RTOG, RU Nijmegen)• Maarten A. Hendriks (freelance)• Peter Kisters (freelance)• Harco C. Kuppens (ICIS, RU Nijmegen)• Eva M. van Rikxoort (ISI, UMC Utrecht)• Charles M. de Weert (Soc. Fac., RU Nijmegen)
![Page 4: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/4.jpg)
Multilevel Access to our Cultural Heritage (MACH)
NWO research line ToKeN
• Two projects within the ToKeN research line:– Eidetic– VindIT
• A few of the aims of the projects– Access and enhancement of our cultural heritage – Combining Computer Science and Cognitive Science– Combining (text-based) Information Retrieval (IR) and Content-Based
Image Retrieval (CBIR)– Launching a CBIR system for the National Gallery of Arts (Het
Rijksmuseum)
![Page 5: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/5.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Key notions of this talk
• Aim: Access and enhancement of knowledge• Domain: Cultural heritage; more specific: The National
Gallery of Arts (Het Rijksmuseum)• Characteristics:
– Digital photographs of objects– Annotations of these objects; i.e., descriptions of the artist, the
material, the content/theme of the object etc. • For … both laypersons (i.e., non-experts) and experts• How: (text-based) Information Retrieval (IR) and Content-
Based Image Retrieval (CBIR)• Additionally, facilitating:
– The user’s understanding of CBIR– Comparison of IR and CBIR
![Page 6: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/6.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Current state-of-the-art in museum access through the Internet
Four examples:
Virtual Catalogue for Art History
The National Gallery of Art (Het Rijksmuseum)
The hermitage museum (powered by IBM)
![Page 7: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/7.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 8: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/8.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 9: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/9.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 10: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/10.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 11: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/11.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 12: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/12.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 13: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/13.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Why this aim in this domain?
• Attract people to museums through Internet• Make museums better accessible for laypersons• Make museums better accessible for experts• Providing more information through new methods of access• Gaining more understanding of CBIR through using it
• Future aims:– Interaction between users and the system through agent technology– Extracting information of artists directly from their art
![Page 14: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/14.jpg)
Multilevel Access to our Cultural Heritage (MACH)In
form
ati
on
Re
trie
va
l (e
.g.,
as
wit
h G
oo
gle
)
![Page 15: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/15.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Color-Based Image Retrieval:3 methods of querying
• Browsing the catalogue
• Providing an image from the Internet, by way of an URL
• Uploading a local image from your desktop
![Page 16: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/16.jpg)
Multilevel Access to our Cultural Heritage (MACH)
![Page 17: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/17.jpg)
Multilevel Access to our Cultural Heritage (MACH)
From simple to complex user interfaces
M4ART can be described using 2 dimensions:
1) Retrieval method; either IR or CBIR
2) Complexity of querying;
- straightforward queries
- specified/combined queries
![Page 18: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/18.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Complex IR querying (1)
• Exclusion of terms
• Boolean search, using AND and/or OR operators
• Utilizing specific tags (fields) of the annotations
![Page 19: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/19.jpg)
Multilevel Access to our Cultural Heritage (MACH)C
om
ple
x IR
qu
ery
ing
(2
)
![Page 20: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/20.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Complex CBIR querying (1)
• Color space
• Quantization scheme; i.e., how any colors do you want to distinguish?
• Distance measure
• Optionally: include texture
![Page 21: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/21.jpg)
Multilevel Access to our Cultural Heritage (MACH)C
om
ple
x C
BIR
qu
ery
ing
(2
)
![Page 22: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/22.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Gaining understanding …
• Showing the features …
• Compare IR and CBIR techniques
![Page 23: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/23.jpg)
Multilevel Access to our Cultural Heritage (MACH)S
ho
win
g t
he
fe
atu
res
![Page 24: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/24.jpg)
Multilevel Access to our Cultural Heritage (MACH)D
ista
nc
e m
atr
ix
![Page 25: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/25.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Advanced schemes incorporated
• 11 color categories quantization scheme
• Parallel-Sequential Texture Analysis
![Page 26: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/26.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Color spaces (and their quantizations)
• RGB• HSV • YUV/YIQ• CIE XYZ• CIE LUV• 11 colors ...
![Page 27: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/27.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Its all only in 11 color categories
• Most famous: Berlin & Kay (1969) with their book
“Basic color terms: Their universals and evolution”
• But already 15 years before them Brown and Lenneberg (1954) presented their work in the paper: “A study in language and cognition”
• More recently:– Derefeldt, Swartling, Berggrund, and Bodrogi (2004)– Moroney (2004)– Regier, Kay, and Cook (2005)
![Page 28: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/28.jpg)
Multilevel Access to our Cultural Heritage (MACH)
The practical use of fundamental research: Color categories
• People perceive in color categories
• People talk in color categories
• Human memory uses color categories
• Robust to inter and intra personal variability
• Computationally inexpensive
![Page 29: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/29.jpg)
Multilevel Access to our Cultural Heritage (MACH)
THE 11 colors
• Red• Green• Blue
• Yellow• Purple• Pink• Orange• Brown
BlackGrayWhite
![Page 30: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/30.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Texture defined …
• Cross and Jain (1983): There is no universally accepted definition for texture
• Bovik, Clarke, and Geisler (1990): an exact definition of texture either as a surface property or as an image property has never been adequately formulated.
• Gonzales and Woods (2002) state: No formal definition of texture exists, intuitively this descriptor provides measures of properties such as smoothness, coarseness and regularity.
![Page 31: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/31.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Texture Analysis
• Palm (2004) noted: “The integration of color and texture is still exceptional.” Others: Drimbarean and Whelan (2001) and Mäenpää and Pietikäinen (2004)
• Techniques applied in literature, so far:– Global color analysis (e.g., color histograms)– Local gray-scale texture analysis (e.g., co-occurrence matrix)– Local color-based texture analysis (e.g., color correlogram)– Global color analysis combined with local gray-scale texture analysis (e.g.,
parallel approach)
• What is used in M4ART: Parallel-sequential texture analysis– Global and local color analysis in parallel: the color histogram and the color
correlogram combined.
![Page 32: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/32.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Gray versus Color
![Page 33: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/33.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Parallel-Sequential texture analysis*
* More info? Broek, E. L. van den and Rikxoort, E. M. van (2005). Parallel-sequential texture analysis. Lecture Notes in Computer Science (Advances
in Pattern Recognition), 3687, 532-541.
![Page 34: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/34.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Back to C-BAR: Some last remarks
• Technical specifications
• Freely accessible
• Questions?
![Page 35: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/35.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Technical specifications
• W3C HTML 4.01 validated
• W3C CSS validated
• Running on an Apache HTTP server
• PHP 5.1 powered
• Running on a MySQL database
![Page 36: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/36.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Freely accessible
• All feedback = welcome!
• How to get there …
Use Google query on ‘M4ART’ or go through:
http://cai.nici.ru.nl/M4ART/
![Page 37: DE Conferentie 2005 Egon v.d. Broek](https://reader033.vdocuments.mx/reader033/viewer/2022061217/54b4d0654a7959b90f8b45e0/html5/thumbnails/37.jpg)
Multilevel Access to our Cultural Heritage (MACH)
Thank you for your attention!
Questions?