![Page 1: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/1.jpg)
1
Querying Graphics through Analysis and Recognition
INRIA Lorraine
![Page 2: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/2.jpg)
2
Research fields
•Image processing and segmentation
•Structural pattern recognition
•Statistical pattern recognition
•Information spotting and retrieval
In the context of the analysis and recognition of graphics-rich documents
![Page 3: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/3.jpg)
3
Querying Graphics through Analysis and Recognition
![Page 4: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/4.jpg)
4
Querying Graphics through Analysis and Recognition
![Page 5: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/5.jpg)
5
Querying Graphics through Analysis and Recognition
![Page 6: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/6.jpg)
6
Querying Graphics through Analysis and Recognition
![Page 7: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/7.jpg)
7Scientific staff
Suzanne Collin, Assist. Prof. UHP
Philippe Dosch, Assist. Prof. Nancy 2
Bart Lamiroy, Assist. Prof. INPL/Mines
Gérald Masini, CR CNRS
Salvatore Tabbone, Assist. Prof. U. Nancy 2
Karl Tombre, Prof. INPL/Mines
Laurent Wendling, Assist. Prof. UHP
PhD students
Sabine Barrat, CIFRE contract (pending)
Thi Oanh Nguyen, joint supervision with IFI (Hanoi, Vietnam)
Oriol Ramos Terrades, joint supervision with UAB, Barcelona (Spain)
Jan Rendek, CIFRE France Télécom
Jean-Pierre Salmon, FRESH (European project)
Zhang Wan, joint supervision with City U. Hong Kong (pending)
Daniel Zuwala, MESR grant
Technical staff
Yamina Smail, Epeires project
X, Fresh project (pending)
Administrative staff
Isabelle Herlich (part time)
Françoise Laurent (part time)
![Page 8: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/8.jpg)
8
Main results 2004-05Hierarchical binarization
![Page 9: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/9.jpg)
9
Focus on symbol recognition – Symbol spotting combining Radon-based signature and structural approach
Main results 2004-05
![Page 10: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/10.jpg)
10
Improvement of recognition rates through combination of shape descriptors
Main results 2004-05
The set of images I
Recognition rates by descriptors
010
2030
405060
7080
90100
1 2 3 4 5 6 7 8 9
Clusters
Re
co
gn
itio
n r
ate
s
Compactness
Ellipticity Degree
Angular Signature
Generic FourierDescriptors
R-signature
Weighted Sumwithout weighted map
![Page 11: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/11.jpg)
11
Application : extraction of letters in heritage documents
Descriptors C E SA GFD TRf WS
Before 49 41 70 59 50 55
After 52 39 75 69 50 72Recognition rates
Descriptors C E SA GFD TRf WS
Before 85 78 90 89 80 96
After 93 81 97 94 87 100
Ranking
Improvement of recognition rates through combination of shape descriptors
Main results 2004-05
![Page 12: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/12.jpg)
12
Main results 2004-05Raster-to-vector conversion method based on random sampling and parametric fitting
![Page 13: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/13.jpg)
13
Segmenting the skeleton
RANVEC : Random sampling on pairs of vector pointsExtension of primitive as long as it fits arc or segment (linear regression)
![Page 14: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/14.jpg)
14
Simplification and unification of primitives
![Page 15: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/15.jpg)
15
GREC’01 GREC’03 GREC’05
Winner Dave Elliman JiQiang Song Xavier Hilaire
VRI 0,681 0,609 0,803
Arc segmentation contest
![Page 16: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/16.jpg)
16
Application domains/transferElectrical wiring diagrams in aeronautics FRESH project (FP6 STREP Aeronautics program)
![Page 17: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/17.jpg)
17
Application domains/transfer
Cultural heritage documents ACI Madonne, FP6 STREP proposal QUIMERA-Doc submitted 9/05
![Page 18: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/18.jpg)
18QgarLib : library of C++ classes
QgarApps : applications
QgarGUI : user interface
qgar.org, APP
Refactoring to professional standards
Open architecture (XML)
80,000 lines of C++ code (comments not counted)
30 to 40 downloads of code per month
>10 documentation browses per day (robots excluded)
![Page 19: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/19.jpg)
19
Positioning within INRIA
Fully within one of INRIA’s 7 challenges in strategic plan: Developing multimedia data and information processing
Regular partnership with Imadoc (research group at Irisa)
Joint contacts Texmex (Sym-C)/Qgar with industrial partner
Recent contacts with Lear on browsing of large image bases
![Page 20: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/20.jpg)
20
Collaborations
National: informal consortium Nancy, Rennes, La Rochelle, Rouen, Tours, Lyon with several joint projects (ACI Madonne, RNTL past and submission, Techno-Vision Epeires, IST submission) and coordination of actions
CVC/UAB, Barcelona: long lasting relationship, associated team SymbolRec, joint PhD supervisions
City University Hong Kong: associated in Epeires, PAI submission accepted, joint PhD supervision
IFI, Vietnam: joint PhD supervision
University of Auckland (NZ), University of Bern, Carleton University (Canada)
![Page 21: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/21.jpg)
21
Achievements, strengths, weaknesses
Leadership position at international level on graphics recognition
Announced in project and largely addressed:• Symbol recognition and spotting
• Performance evaluation
Strong and adequate applicative backing
Improvement in number of PhD students
Still low on permanent workforce
![Page 22: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/22.jpg)
22
Future work
Scalability of symbol recognition methods• Large number of models
• Variations within the same shape class
– Combining structural and statistical methods
– Hierarchical approach
![Page 23: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/23.jpg)
23
Future workComplex symbols
![Page 24: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/24.jpg)
24
Future workDynamic, on-the fly recognition and spotting: from model-based recognition to freehand recognition
![Page 25: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/25.jpg)
25
Future work
Multi-modal indexation (text / graphics / image / video) in multimedia and document databases (collaborations with Texmex, Lear, …)
Interactivity with user (relevance feedback)
![Page 26: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/26.jpg)
26
Future work
Performance evaluation• International symbol recognition contests 2003 & 2005
• Epeires
– French Techno-Vision program
– 4 universities, FT R&D, 1 company + foreign partners UAB & CityU
– www.epeires.org
• Future research challenges
– Simple and non-biased metrics
– Ground-truth/recognition output matching methods
– Generation of large sets of training and benchmarking data using realistic image degradation models
![Page 27: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/27.jpg)
27Epeires – ground-truthing
![Page 28: Querying Graphics through Analysis and Recognition INRIA Lorraine](https://reader036.vdocuments.mx/reader036/viewer/2022062803/568147e2550346895db5183a/html5/thumbnails/28.jpg)
28
Future work
Software : increase number of applications