mbc07 eck bab 02

8/14/2019 Mbc07 Eck Bab 02

1/22

Book-Adaptive and Book-Dependent Models

to Accelerate Digitization of Early Music

Laurent Pugin, John Ashley Burgoyne,

Douglas Eck & Ichiro Fujinaga

NIPS MBC Workshop, WhistlerDecember 2007

8/14/2019 Mbc07 Eck Bab 02

2/22

OMR on early music sources

Aruspix

8/14/2019 Mbc07 Eck Bab 02

3/22

Font shape variability

8/14/2019 Mbc07 Eck Bab 02

4/22


8/14/2019 Mbc07 Eck Bab 02

5/22


8/14/2019 Mbc07 Eck Bab 02

6/22

Document degradation variability

8/14/2019 Mbc07 Eck Bab 02

7/22

What? Enable Aruspix to be able to handle variabilities

more efficiently

How? Make it adaptable to each book Use a supervised adaptation of trained HMMs

Does involve manual correction

Why? OMR output has to be corrected anyway in most

applications

Goal of this research

8/14/2019 Mbc07 Eck Bab 02

8/22

MAP (maximum a posteriori) adaptation

MAP adaptation in speech

A speaker-independent

model (SI) is built off-lineon a large set of data

During recognition, the

SI model is optimized toobtain the speaker-dependent (SD) modelusing a small set of data

MAP adaptation in OMR

A book-independent

model (BI) is built off-lineon a large set of pages

During recognition, the BI

model is optimized toobtain the book-dependent (BD) modelusing a small set of pages

8/14/2019 Mbc07 Eck Bab 02

9/22

MAP (maximum a posteriori) adaptation

MAP adaptation in speech

A speaker-independent

model (SI) is built off-lineon a large set of data

During recognition, the

SI model is optimized toobtain the speaker-dependent (SD) modelusing a small set of data


A book-independent





A book-independent




8/14/2019 Mbc07 Eck Bab 02

10/22

Experiment data

1 BI model trained from scratch on 457pages from various printed books

5 books to experiment with MAP adaptation 50 pages of ground-truth in each book

5 training sets of 40 pages (first pages of the book)

5 testing sets of 10 pages Cross-validated one of the books

8/14/2019 Mbc07 Eck Bab 02

11/22

(a) RISM M.0579 (R. Amadino, Venice, 1587) baseline rec. rate = 83.91%

(b) RISM M.0580 (G. Vincenti, Venice, 1587) baseline rec. rate = 67.64%

(c) RISM M.0583 (A. Gardano, Venice, 1603) baseline rec. rate = 82.85%

(d) RISM M.0585 (P. Phalese, Antwerp, 1607) baseline rec. rate = 86.07%

What We Observe

feature vector

1 V V3 V4 V5 Vn2 ...V

8/14/2019 Mbc07 Eck Bab 02

12/22

Experiment workflow

8/14/2019 Mbc07 Eck Bab 02

13/22

Experiment workflow

8/14/2019 Mbc07 Eck Bab 02

14/22

Experiment workflow

8/14/2019 Mbc07 Eck Bab 02

15/22

Experiment workflow

8/14/2019 Mbc07 Eck Bab 02

16/22

Experiment workflow

8/14/2019 Mbc07 Eck Bab 02

17/22

Results

0

5

10

15

20

10 20 30 40 50 60 70 80

Editingcostpersymbol

Number of pages

(a) RISM M.0579

0

5

10

15

20

10 20 30 40 50 60 70 80

Editingcostpersymbol

Number of pages

(b) RISM M.0580

0

5

10

15

20

10 20 30 40 50 60 70 80

Editingc

ostpersymbol

Number of pages

(c) RISM M.0583

0

5

10

15

20

10 20 30 40 50 60 70 80

Editingc

ostpersymbol

Number of pages

(d) RISM M.0585

Legend

BI model (baseline)

BA model (MAP adaptation)BD model

8/14/2019 Mbc07 Eck Bab 02

18/22

Analysis of the results

MAP adaptation improves both recalland precision

MAP adaptation is faster than trainingfrom scratch in most cases

Only 5 to 10 pages are needed toachieve optimal performance

MAP adaptation can be used in real-time when alignment is good enough

8/14/2019 Mbc07 Eck Bab 02

19/22

Conclusion and future work

Exploit vertical as well as horizontalinformation

This comes in the form of harmonicstructure

More complex graphical model combining

information from multiple staves

Appropriate for polyphonic music

8/14/2019 Mbc07 Eck Bab 02

20/22

Thank you!

8/14/2019 Mbc07 Eck Bab 02

21/22

Results - Precision

8/14/2019 Mbc07 Eck Bab 02

22/22

Results - Recall

mbc07 eck bab 02

Documents