View
226
Download
0
Category
Preview:
Citation preview
8/14/2019 Mbc07 Eck Bab 02
1/22
Book-Adaptive and Book-Dependent Models
to Accelerate Digitization of Early Music
Laurent Pugin, John Ashley Burgoyne,
Douglas Eck & Ichiro Fujinaga
NIPS MBC Workshop, WhistlerDecember 2007
8/14/2019 Mbc07 Eck Bab 02
2/22
OMR on early music sources
Aruspix
8/14/2019 Mbc07 Eck Bab 02
3/22
Font shape variability
8/14/2019 Mbc07 Eck Bab 02
4/22
Font shape variability
8/14/2019 Mbc07 Eck Bab 02
5/22
Font shape variability
8/14/2019 Mbc07 Eck Bab 02
6/22
Document degradation variability
8/14/2019 Mbc07 Eck Bab 02
7/22
What? Enable Aruspix to be able to handle variabilities
more efficiently
How? Make it adaptable to each book Use a supervised adaptation of trained HMMs
Does involve manual correction
Why? OMR output has to be corrected anyway in most
applications
Goal of this research
8/14/2019 Mbc07 Eck Bab 02
8/22
MAP (maximum a posteriori) adaptation
MAP adaptation in speech
A speaker-independent
model (SI) is built off-lineon a large set of data
During recognition, the
SI model is optimized toobtain the speaker-dependent (SD) modelusing a small set of data
MAP adaptation in OMR
A book-independent
model (BI) is built off-lineon a large set of pages
During recognition, the BI
model is optimized toobtain the book-dependent (BD) modelusing a small set of pages
8/14/2019 Mbc07 Eck Bab 02
9/22
MAP (maximum a posteriori) adaptation
MAP adaptation in speech
A speaker-independent
model (SI) is built off-lineon a large set of data
During recognition, the
SI model is optimized toobtain the speaker-dependent (SD) modelusing a small set of data
MAP adaptation in OMR
A book-independent
model (BI) is built off-lineon a large set of pages
During recognition, the BI
model is optimized toobtain the book-dependent (BD) modelusing a small set of pages
MAP adaptation in OMR
A book-independent
model (BI) is built off-lineon a large set of pages
During recognition, the BI
model is optimized toobtain the book-dependent (BD) modelusing a small set of pages
8/14/2019 Mbc07 Eck Bab 02
10/22
Experiment data
1 BI model trained from scratch on 457pages from various printed books
5 books to experiment with MAP adaptation 50 pages of ground-truth in each book
5 training sets of 40 pages (first pages of the book)
5 testing sets of 10 pages Cross-validated one of the books
8/14/2019 Mbc07 Eck Bab 02
11/22
(a) RISM M.0579 (R. Amadino, Venice, 1587) baseline rec. rate = 83.91%
(b) RISM M.0580 (G. Vincenti, Venice, 1587) baseline rec. rate = 67.64%
(c) RISM M.0583 (A. Gardano, Venice, 1603) baseline rec. rate = 82.85%
(d) RISM M.0585 (P. Phalese, Antwerp, 1607) baseline rec. rate = 86.07%
What We Observe
feature vector
1 V V3 V4 V5 Vn2 ...V
8/14/2019 Mbc07 Eck Bab 02
12/22
Experiment workflow
8/14/2019 Mbc07 Eck Bab 02
13/22
Experiment workflow
8/14/2019 Mbc07 Eck Bab 02
14/22
Experiment workflow
8/14/2019 Mbc07 Eck Bab 02
15/22
Experiment workflow
8/14/2019 Mbc07 Eck Bab 02
16/22
Experiment workflow
8/14/2019 Mbc07 Eck Bab 02
17/22
Results
0
5
10
15
20
10 20 30 40 50 60 70 80
Editingcostpersymbol
Number of pages
(a) RISM M.0579
0
5
10
15
20
10 20 30 40 50 60 70 80
Editingcostpersymbol
Number of pages
(b) RISM M.0580
0
5
10
15
20
10 20 30 40 50 60 70 80
Editingc
ostpersymbol
Number of pages
(c) RISM M.0583
0
5
10
15
20
10 20 30 40 50 60 70 80
Editingc
ostpersymbol
Number of pages
(d) RISM M.0585
Legend
BI model (baseline)
BA model (MAP adaptation)BD model
8/14/2019 Mbc07 Eck Bab 02
18/22
Analysis of the results
MAP adaptation improves both recalland precision
MAP adaptation is faster than trainingfrom scratch in most cases
Only 5 to 10 pages are needed toachieve optimal performance
MAP adaptation can be used in real-time when alignment is good enough
8/14/2019 Mbc07 Eck Bab 02
19/22
Conclusion and future work
Exploit vertical as well as horizontalinformation
This comes in the form of harmonicstructure
More complex graphical model combining
information from multiple staves
Appropriate for polyphonic music
8/14/2019 Mbc07 Eck Bab 02
20/22
Thank you!
8/14/2019 Mbc07 Eck Bab 02
21/22
Results - Precision
8/14/2019 Mbc07 Eck Bab 02
22/22
Results - Recall
Recommended