12th International Society for Music Information Retrieval Conference (ISMIR 2011)

INCREMENTAL BAYESIAN AUDIO-TO-SCORE ALIGNMENTWITH FLEXIBLE HARMONIC STRUCTURE MODELS

Takuma Otsuka†, Kazuhiro Nakadai‡, Tetsuya Ogata†, and Hiroshi G. Okuno†

† Graduate School of Informatics, Kyoto University ‡ Honda Research Institute Japan, Co., Ltd.Sakyo-ku, Kyoto 606-8501 Japan Wako, Saitama 351-0114, Japan

{ohtsuka, ogata, okuno }@kuis.kyoto-u.ac.jp nakadai@jp.honda-ri.com

ABSTRACT

Music information retrieval, especially the audio-to-scorealignment problem, often involves a matching problem be-tween the audio and symbolic representations. We mustcope with uncertainty in the audio signal generated from thescore in a symbolic representation such as the variation inthe timbre or temporal fluctuations. Existing audio-to-scorealignment methods are sometimes vulnerable to the uncer-tainty in which multiple notes are simultaneously playedwith a variety of timbres because these methods rely onstatic observation models. For example, a chroma vectoror a fixed harmonic structure template is used under the as-sumption that musical notes in a chord are all in the samevolume and timbre. This paper presents a particle filter-based audio-to-score alignment method with a flexible ob-servation model based on latent harmonic allocation. Ourmethod adapts to the harmonic structure for the audio-to-score matching based on the observation of the audio signalthrough Bayesian inference. Experimental results with 20polyphonic songs reveal that our method is effective whenmore number of instruments are involved in the ensemble.

1. INTRODUCTIONMusic information retrieval tasks require a robust inferenceunder the uncertainty in musical audio signals. For example,a polyphonic or multi-instrument aspect encumbers the fun-damental frequency estimation [10, 15] or instrument iden-tification [9]. Overcoming the uncertainty in musical audiosignals is a key factor in the machine comprehension of mu-sical information. The audio-to-score alignment technologyshares this uncertainty problem in that an audio signal per-formed by human musicians has a wide range of varietiesgiven a symbolic score due to the musicians’ expressive-ness. For example, the type of instruments and the temporal

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page.c© 2011 International Society for Music Information Retrieval.

Audio:1. various harmonic structures2. unknown mixture ratio

Mismatch!

Score:1. fixed harmonic structures2. equal mixture ratio

Figure 1. The issue: uncertainty in the audio and fixed har-monic templates from the score

or pitch fluctuations affect the resulting audio signals.Incremental audio-to-score alignment, also known as score

following, methods are essential to automatic accompani-ment systems [5], intelligent score viewers [2], and robotmusicians [13] because the alignment synchronizes thesesystems with human performances. We need a probabilis-tic framework for the audio-to-score alignment problem inorder to cope with the uncertainty in the audio signal gener-ated from the score in a symbolic representation.

Existing methods tend to fail the alignment when mul-tiple musical notes are played by multiple musical instru-ments. That is, the audio signal contains various timbres andthe volume ratio of each musical note is unsure. Figure 1illustrates this issue. The observed pitched audio signal in-cludes equally-spaced peaks in frequency domain called aharmonic structure. The observed audio is matched withharmonic structure templates generated from the score. Mu-sical notes written in the score is played with arbitrary musi-cal instruments. The resulting audio harmonic structures canvary from instrument to instrument whereas the templates ofthe score have been set in advance using some heuristics ora parameter learning [4]. In Figure 1, harmonic structuresof a guitar and a violin is shown in blue and red lines, re-spectively. Furthermore, the mixture ratio of each note inthe audio is unknown until the observation while the ratio inthe template is fixed, typically equal.

Thus, the variety of the audio signal causes a mismatch

525

Poster Session 4

Frequency x

Am

plitu

deA

mpl

itude

Am

plitu

de

Frequency x

Prob

abili

tydi

stri

butio

n

Frequency xlµlµ2 lmµL

1lθ2lθ

lmθ

1−Λ l

M harmonics

N

Ld GMMs

dlπ

Figure 2. Audio spectrogram based on LHA

dlπ nz

lΛlµdLM

dN Dnx

lmθFigure 3. Graphical model of LHA

dlπ nz

lmθ lΛlµdLM

dN Dnx

Figure 4. LHA with fixed θlm’s

between the observed harmonic structure and the fixed onegenerated from the score. We need a flexible harmonic struc-ture model to robustly match the audio and score since theaudio signal is almost unknown until we observe it.

Our idea is to employ a Bayesian harmonic structure modelcalled latent harmonic allocation (LHA) [15]. This modelallows us to form harmonic structure templates reflectingthe observed audio with the prior knowledge written in thescore, e.g., fundamental frequencies of musical notes.

1.1 Related work

Two important aspects reside in modeling audio-to-scorealignment: (1) a temporal model of musical notes and (2)an observation model of the input audio signal from thecorresponding score. Although improvements are made re-peatedly for the temporal model, misalignments are oftencaused by static and fixed audio observation models. Theaudio observation model used in the methods introduced inthis section uses static features such as chroma vectors orfixed harmonic structure templates based on Gaussian mix-ture model (GMM). These features are often heuristicallydesigned and therefore lose robustness against uncertain sit-uations in which many instruments are involved and the au-dio is polyphonic.

Most audio-to-score alignment methods employ dynamictime warping (DTW) [2,6], hidden Markov models (HMM) [4,12], or particle filters [7, 11, 13]. DTW or HMM-basedmethods sometimes fails the alignment since the length ofmusical notes is less constrained in the decoding.

The note length corresponds to the length of a state se-quence in the HMM. Cont’s method [3] uses a hidden semi-Markov model (HSMM) to control state lengths. The HSMMrestricts the duration of a stay at one state so that the statelength is limited. While the model refrains from delayedstate transitions, this has no restriction on fast transitions.As a result, the HSMM tends to estimate the audio signalfaster than it is.

Some methods estimate not only the score position butalso the tempo, i.e., the speed of the music for the tempo-ral accuracy. Raphael’s method includes the tempo of themusic as a state [14] to accurately decode the note lengths.Otsuka et al. [13] propose a particle filter-based method fortheir simultaneous estimation. While Raphael’s method ob-serves only harmonic structures as pitch information, Ot-suka et al.’s method observes the periodicity of the onsets todirectly estimate the tempo.

2. AUDIO OBSERVATION MODEL

This Section describes how the audio is generated in termsof LHA. We focus on harmonic structures to associate anaudio signal with a symbolic score. The LHA model flexiblyfits the shape of harmonic structures given an audio signalobservation using variational Bayes inference.

The harmonic peaks are often modeled as a Gaussianmixture model (GMM) by regarding each peak as a sin-gle Gaussian [3, 13, 14]. The black lines in Figure 1 arethe GMM curves. These methods use Kullback-Leibler di-vergence (KL-div) as a matching function between the au-dio harmonics and the GMM template harmonics generatedfrom the score by regarding the harmonic structure as a prob-ability distribution. The mean value of each Gaussian peakis determined by a pitch specified in the score.

LHA [15] is a generative model for harmonic structuresof pitched sounds. A graphical model for LHA is depictedin Figure 3. In the LHA model, the amplitude of audio har-monics is regarded as a histogram over the frequency bins.

Figures 2 and 3 explain how a mixture of harmonic struc-tures is generated. Variables in a circle are random variableswhile those without a circle are parameters. Double circledxn means an observed variable. For each segment d, Nd ,frequencies xn are observed. The audio spectrogram is seg-mented into d by chords, which are sets of musical notes. Tosample each xn, a LdM-dimensional multinomial latent vari-able zn is sampled as follows. A harmonic structure GMM l

526

12th International Society for Music Information Retrieval Conference (ISMIR 2011)

T∆

Tk∆ (sec)

Score position

Bea

t in

terv

al

(a) Particle drawing (b) Weight calculation

kA

q

W

(c) Point estimation & resampling

Audio buffer (time range: )

for observation &proposal dist.

kTkAMost recently

observedaudio signal1−kA

W

Audio-to-score matching

Weighted averagecalculation

Figure 5. Three steps in particle filtering for the audio-to-score alignment

is selected with probability πdl , where ∑Ldl=1 πdl = 1. Among

M Gaussian peaks, m is selected to sample xn with probabil-ity θlm, where ∑M

m=1 θlm = 1. Finally, xn is sampled fromthe Gaussian distribution of which mean and precision aremµl and Λl , respectively. The definitions of each variable inLHA in Figure 3 are summarized below:

p(X|Z,µ,Λ) = ∏dnlm

N (xdn|m×µl ,Λl), (1)

p(Z|π,θ) = ∏dnlm

(πdlθlm)zdnlm , (2)

p(π) = ∏d

Dir(πd |α0), p(θ) = ∏l

Dir(θl |β0),and (3)

p(Λ) = ∏l

Gam(Λl |a0,b0), (4)

where N (·), Dir(·), Gam(·) denote the density functions ofGaussian, Dirichlet, and gamma distribution, respectively.The latent variable zdn = [zdnlm] is LdM-dimensional withone element being 1 and the other being 0. Variables π andθ are conjugate priors for Z, and the precision of GaussianharmonicsΛ is a conjugate prior for X. Here, α0, β0, a0, andb0 are hyperparameters for each distribution. α0 = [α0l ]

Ldl=1

is set as α0l = 1, and β0 = [β0m]Mm=1 is set as β0m = 1 be-cause the mixture ratio of each musical note and the heightof each harmonic are unknown. A “flat” prior knowledgeabout these parameters is preferred to reflect our ignorance.The hyperparameters of the gamma distribution are empir-ically set as a0 = 1 and b0 = 2.4 by considering the widthof harmonics determined by the window function of a short-time Fourier transform (STFT).

The LHA is originally designed for multi-pitch analy-sis [15], and therefore the fundamental frequency µl is arandom variable. However, in our audio-to-score alignmentframework, µl is treated as a parameter because fundamen-tal frequencies are given by the score as musical notes. Thisis why µl is not in a circle in Figure 3.

In general, too flexible model can cause an over-fittingproblem. LHA is flexible in terms of the mixture ratio π andharmonic heights θ. To limit the model complexity, we fixthe harmonic heights θ and only consider the mixture ratioπ as in Figure 4. We refer to the former model in Figure 3as full LHA, and the latter in Figure 4 as mixture LHA.

3. AUDIO-TO-SCORE ALIGNMENT USINGPARTICLE FILTER

This section presents the problem setting and procedures ofour method. The problem is specified as follows:� �

Inputs: incremental audio signal and the correspond-ing whole scoreOutputs: the current score position and tempoAssumptions: (1) The score includes musical notesand the approximate tempos of the music. (2) Musicalnotes are pairs of their pitch and length, e.g., a quar-ter note, (3) Approximate tempos are specified as therange of a tempo, e.g., 90–110 beats per minute (bpm).� �

No prior knowledge about musical instruments is assumed.

3.1 Method overview

Let k be the index of filtering steps and At, f be the ampli-tude of the input audio signal in the time-frequency domain.Here, t and f denote the time (sec) and the frequency (Hz),respectively. Our system is implemented at a sampling rateof 44100 (Hz), a window length of 2048 (pt), and a hop sizeof 441 (pt). At, f denotes a quantized integer amplitude givenby At, f = bAt, f /∆Ac, where ∆A is the quantization factor,and b·c is the flooring function. ∆A = 3.0 in our implemen-tation. This value should be so small that the shape of thespectrum is preserved after the quantization and that suffi-cient observations are provided for the Bayesian inferencein the LHA. Let p (beat) be the score position. The score isdivided into frames whose lengths are equal to 1/12 of onebeat, namely, a quarter note1 . Musical notes are denoted byµp = [µ1

p...µLpp ]T , where Lp is the number of notes at p, and

µ is the fundamental frequency of the note.Figure 5 illustrates the procedures. At every ∆T (sec), the

particle filtering [1] proceeds as: (a) move particles in accor-dance with elapsed ∆T (sec) by drawing particles from theproposal distribution, (b) calculate the weight of each parti-cle, (c) report the point estimation of the score position andbeat interval, and resample the particles. Each particle has

1 p is discretized at 1/12 interval in (beat).

527

Poster Session 4

score frame p

onset frames

Figure 6. Score position proposal. Au-dio frames marked by green rectangles arealigned with score onsets.

GMM harmonic

structure

i

kp)(~ τp

i

kb

τ

Figure 7. Frame-by-frame alignmentof audio using pi

k and bik

i

kp

i

kb

Segmentation by different chords

Figure 8. Segmentation by chords forLHA observation

the following information as a hypothesis: the score posi-tion pi

k, beat interval (sec/beat), i.e., the inverse tempo, bik,

and the weight wik as a fitness to the model.

In the kth filtering step, the particle filter estimates theposterior distribution of the score position pk and beat in-terval bk given the latest audio spectrogram Ak = [Aτ, f ],where τ ∈ Tk, Tk = {t|k∆T −W < t ≤ k∆T}, and W isthe window length for the audio spectrogram. The poste-rior distribution is approximated using many particles as inp(sk|Ak) = ∑I

i=1 wikδ (si

k− sk), where I is the number of par-ticles, and si

k = [pik,b

ik] denotes the state of the ith particle.2

The weight of each particle wik is calculated as:

wik ∝

p(sik|si

k−1)p(Ak|sik)

q(sik|si

k−1,Ak), (5)

where p(sik|si

k−1) and p(Ak|sik) in the numerator are the state

transition model and observation model, respectively. Newscore position and beat interval values are drawn at each stepfrom the proposal distribution q(si

k|sik−1,Ak).

3.2 Drawing particles from the proposal distributionParticles are drawn from the proposal distribution in Eq. (6).First, a new beat interval bi

k is drawn, then a new score po-sition pi

k is drawn depending on the drawn bik. The proposal

is designed to draw (1) a beat interval that lies in the temporange provided by the score and that matches the intervalsamong audio onsets and (2) a score position that matches theincrease of the audio amplitude with the score onset frame.

sik ∼ q(b, p|si

k−1,Ak)∝ R(b;Ak)Ψ(b; b)×Q(p;b,Ak,si

k−1). (6)R(b;Ak) and Ψ(b; b) denote the normalized cross correla-tion of the audio signal and the window function that limitsthe range of the beat interval, respectively. Q(p;b,Ak,si

k−1)denotes the onset matching function.Detailed equations areexplained in [13].

The onset matching function Q(p;b,Ak,sik−1) in Eq. (6)

represents how well the audio and score are aligned in termsof the onsets. Figure 6 explains the design. The top case in

2 δ (x) = 1 iff x = 0, otherwise δ (x) = 0.

which audio frames with a peak power is aligned with scoreonsets results in the larger Q, where as the bottom case Q isa small value since the onsets are misaligned. The detailedmathematical expressions are presented in [13].

3.3 Weight calculationThe weight for each particle is calculated with the sampledvalue si

k in Eq. (5) by using the state transition model,p(si

k|sik−1) = N (pi

k|pik,σ

2p)×N (bi

k|bik−1,σ

2b ), (7)

and the observation model,p(Ak|si

k) ∝ p(Ak|pik)×R(bi

k;Ak). (8)The score position transition conforms to a linear Gaussianmodel with the transition pi

k = pik−1+∆T/bi

k−1 and the vari-ance σ2

p (beat2). The beat interval transition is a randomwalk model with the variance σ2

b (sec2/beat2). The variancesare empirically set as σ2

p = 0.25 and σ2b = 0.1, respectively.

For the observation of the beat interval, the normalizedcross-correlation of the audio spectrogram, R(bi

k;Ak), is againused in Eq. (8). The other factor p(Ak|pi

k) is the likeli-hood corresponding to the pitch information. As explainedin Section 2, the GMM-based harmonic structures are usedto match the audio and score. First, the matching with KL-div is presented as a baseline where all the GMM parame-ters, the chord mixture ratio π or harmonic heights θ, andthe Gaussian width λ, are fixed. Then, we explain two typesof LHA-based audio-to-score matchings, the full LHA andmixture LHA, where the GMM parameters are probabilityvariables that flexibly adapt to the observed audio harmonicstructure. Full LHA adapts all π, θ, and λ whereas mixtureLHA adapts only π and λ to the audio. Further discussionof the difference is in Section 4.

The KL-div matching uses a normalized amplitude spec-trogram Ak while the LHA models use the quantized spec-trogram Ak. To match the buffered audio, the audio spectro-gram Ak or Ak is aligned with the score shown as Figure 7.As the time k∆T is assigned to pi

k with the beat interval bik,

the audio frame τ is linearly assigned to the score frame asgiven by

p(τ) = pik − (k∆T − τ)/bi

k. (9)

528

12th International Society for Music Information Retrieval Conference (ISMIR 2011)

3.3.1 Harmonic structure observation based on KL-divFor each score frame p, the GMM template of the harmonicstructure is generated from the musical notes µp as:

Ap, f =Lp

∑l=1

M

∑m=1

CharmπlθmN ( f |gµ lp,σ2

KL)+Cfloor,(10)

where Lp is the number of notes at p and the number of har-monic structures M is 10. The ratio of each note is equallyset as πl = 1/Lp. The height of the mth harmonic is setas θm = 0.2m−1. The variance is set as σ2

KL = 2.4, derivedfrom the window function used in STFT. Cfloor is a floor-ing constant to ensure Ap, f > 0 and avoid zero-divides inEq. (11). Charm = 0.95 and Cfloor is set such that the har-monic structure template is normalized as Ap,· = 1. Thesubscript · means a summation over the replaced index.

Here, the audio likelihood using KL-div is defined as

log p(Ak|sik) = − ∑

τ∈Tk

∑f

Aτ, f logAτ, f

Ap(τ), f, (11)

where Aτ, f = Aτ, f /Aτ,· is the normalized amplitude. Theright-hand side of Eq. (11) is a negative KL-div between theaudio harmonic structure and the GMM harmonic template.

3.3.2 LHA-based likelihood calculationWe first explain how LHA is used as the likelihood, thenshow the iterations for both full and mixture LHA infer-ences. The quantized amplitudes Ak are regarded as a his-togram of amplitudes over frequency bins X illustrated asgray bars in Figure 2. The rigorous likelihood in Eq. (5) is

p(X|sik) = ∑

Z

∫∫∫p(X,Z,π,θ ,Λ|pi

k,µ)dπdθdΛ. (12)

Since this analytical summation over Z is intractable 3 , weinfer the latent variables Z,π,θ, and Λ by variational Bayes(VB) method under the factorization assumption q(Z,π,θ,Λ)=q(Z)q(π,θ,Λ). We use the variational lower bound for theweight calculation as an approximate observation model in-stead of Eq. (12),

log p(X|sik)≈ L(q) = EZ,π,θ,Λ

[log

p(X,Z,π,θ,Λ|pik,µ)

q(Z,π,θ ,Λ)

], (13)

where EZ,π,θ,Λ[·] denotes an expectation over q(Z,π,θ,Λ).For the inference of LHA, the audio is segmented by thechord in the score as shown in Figure 8. This segmentationd is made on the basis of the alignment by Eq. (9).

The variational lower bound in Eq. (13) is maximizedwith the following variational posteriors:

q(Z) = ∏dnlm

γzdnlmdnlm , q(π) = ∏

dDir(πd |αd),

q(θ) = ∏l

Dir(θl |βl), q(Λ) = ∏l

Gam(Λl |al ,bl),

the parameters of which are updated asγdnlm = ρdnlm/ρdn··, (14)

logρdnlm=ψ(αdl)−ψ(αd·)+ψ(βlm)−ψ(βl·)

+ψ(al)/2−(logbl)/2− (xn−mµl)2al/2bl ,

(15)

3 The integration over π, θ, and Λ is tractable thanks to their conjugacy.

αdl =α0l + γd·l·, βlm = β0m + γ··lm,

al = a0 +γ··l·2

, bl = b0+∑dnm γdnlm(xdn−mµl)

2

2,

(16)

where ψ(·) in Eq. (15) denotes the digamma function. Eqs. (14,15)and Eqs. (16) are iteratively calculated until the lower boundin Eq. (13) converges. Note that Eq. (14) is the normaliza-tion of ρ over indices l and m.

Mixture LHA update: In the update for the mixture LHAmodel, the harmonic height parameter is set as θlm = 0.2m−1.Thus, the update equations are modified as:

logρdnlm=ψ(αdl)−ψ(αd·)+ logθlm

ψ(al)/2−(logbl)/2− (xn−mµl)2al/2bl ,

(17)

αdl =α0l + γd·l·,

al = a0 +γ··l·2

, bl = b0+∑dnm γdnlm(xdn−mµl)

2

2.

(18)

Relationship with the KL-div likelihood: Remember thenegative KL-div is used as the log-likelihood in Eq. (11).The following equation always holds during the iterations:

L(q)+KL(q||p) = log p(X|sik) (const wrt. q).

The KL-div is defined between the approximate distribu-tion q(Z,π,θ,Λ) and the true posterior p(Z,π,θ,Λ|X, pi

k,µ).Note that maximizing L(q) is equivalent to minimizing KL-div, namely, maximizing the negative KL-div due to theequation above. Thus, the LHA-based likelihood is inter-preted as an extension of Eq. (11) in that the harmonic tem-plates adapt to the audio observation to minimize the KL-divand maximize the log-likelihood.

3.4 Point estimation and efficient computingAfter the weights of all particles are calculated, the pointestimation is reported as sk = ∑I

i=1 wiksi

k

/∑I

i=1 wik. Particles

are resampled after the point estimation procedure to elimi-nate zero-weight particles. The resampling probability is inproportion to the weight of each particle [1].

4. EXPERIMENTAL RESULTSThis section presents the alignment error of three observa-tion models; conventional KL-div [13], full LHA, and mix-ture LHA. Twenty songs from RWC Jazz music database [8]is used for this experiment. This test set includes variouscompositions of musical instruments from solo performanceto big band ensembles. Our system is implemented on LinuxOS and a 2.4 (GHz) processor. Experiments are carried outwith the following parameter settings; the filtering interval∆T = 0.5 (sec), the window length for the audio processingW = 1.5 (sec), and the number of particles I = 300.

Figure 9 shows the error percentiles of 20 songs for threemethods. Black, red, and blue bars represent the percentilesof KL-div, full LHA, and mixture LHA, respectively. Thedarkest bars are the 50% percentiles, middle bars are the75%, and the lightest segments are the 100% percentiles.The less values indicate the better performance. Songs witha larger ID tend to involve more instruments. Both of the

529

Poster Session 4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

2.5

5

7.5

10

12.5

15

17.5

Song ID

Err

or

per

cen

tile

(se

c)

KL−div

full LHA

mixture LHA

Figure 9. Error percentiles for 20 songs

LHA-based methods outperform the KL-div for 10 songs,and either full or mixture LHA shows less errors than KL-div for 3 songs. In particular, LHA-based methods tend toreport less errors when the song consists of a larger numberof instruments (larger ID songs). This is what we expectfrom the LHA models.

Two major reasons are given why LHA-based observa-tion models still accumulate the alignment error. First, LHAis vulnerable to rest notes where no musical note is speci-fied. This is because the LHA model penalizes unspecifiedharmonic peaks. When the score provides a rest, LHA pe-nalizes any audio observation. The error caused by these restnotes is seen in songs 2, 5, 8, and 9, where we have morechances to have rest notes because the number of musicalinstruments is relatively small. The second reason is thenon-harmonic feature of percussions and drums. Becausedrum sounds are loud and outstanding in the ensemble, thesesounds interfere the harmonic structures of pitched soundsassumed by LHA. This case applies in songs 11, 16, and 17where drums are included in the ensemble.

Here we discuss the difference between the full and mix-ture LHAs. Since mixture LHA has less variables to in-fer, we can expect more accurate inference as long as thefixed parameters θ fit the observation. The fixed θ declinesas the frequency becomes larger. This descending height iswell observed in stringed instruments such as guitar or pianodominantly used in songs 1-6; whereas wind instrumentssuch as saxophone or flute or bowed instruments such as vi-olin show rather different peaks. When these instrumentsare dominant in a song, e.g., songs 16 and 17 which are in abig band style, the full LHA will be the better choice.

5. CONCLUSION AND FUTURE WORKS

The experiment has shown that LHA is especially effec-tive in a large-ensemble situation where more musical notesare simultaneously performed. However, LHA-based audioobservation models is disturbed by (1) rest notes and (2)drum sounds. To make the best use of the LHA model, onepromising solution is to examine the musical score in ad-vance of the alignment whether the expecting audio signalis suitable for LHA. The development of this top-level deci-

sion making process will be one of the future works.Another future work includes an accelerated calculation

of LHA iterations for such real-time applications as auto-matic accompaniment systems. Current implementation re-quires approximately 10 seconds to process one-second au-dio data.

6. REFERENCES[1] M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A Tutorial on

Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking.IEEE Transactions on Signal Proc., 50(2):174–189, 2002.

[2] A. Arzt, G. Widmer, and S. Dixon. Automatic Page Turning for Mu-sicians via Real-Time Machine Listening. In Proc. of the EuropeanConference on Artificial Intelligence, pages 241–245, 2008.

[3] A. Cont. A Coupled Duration-Focused Architecture for Realtime Mu-sic to Score Alignment. IEEE Transactions on Pattern Analysis andMachine Intelligence, 32(6):974–987, 2010.

[4] A. Cont, D. Schwarz, and N. Schnell. Training IRCAM’s score fol-lower. In AAAI Fall Symposium on Style and Meaning in Art, Languageand Music, 2004.

[5] R. Dannenberg and C. Raphael. Music Score Alignment and ComputerAccompaniment. Comm. ACM, 49(8):38–43, 2006.

[6] S. Dixon. An On-line Time Warping Algorithm for Tracking MusicalPerformances. In Proc. of the IJCAI, pages 1727–1728, 2005.

[7] Z. Duan and B. Pardo. A STATE SPACE MODEL FOR ONLINEPOLYPHONIC AUDIO-SCORE ALIGNMENT. In Proc. of Int’l Conf.on Acoustics, Speech and Signal Processing, pages 197–200, 2011.

[8] M. Goto. AIST Annotation for RWC Music Database. In Proc. of IS-MIR, pages 359–360, 2006.

[9] T. Kitahara, M. Goto, K. Komatani, T. Ogata, and H. G. Okuno. Instru-ment Identification in Polyphonic Music: Feature Weighting to Mini-mize Influence of Sound Overlaps. EURASIP Journal on Applied Sig-nal Processing, vol. 2007, 2007. Article ID 51979.

[10] A. Kulapuri. Multipitch Analysis of Polyphonic Music and Speech Sig-nals Using an Auditory Model. IEEE Transactions on Audio, Speechand Language Processing, 16(2):255–266, 2007.

[11] N. Montecchio and A. Cont. A UNIFIED APPROACH TO REALTIME AUDIO-TO-SCORE AND AUDIO-TO-AUDIO ALIGNMENTUSING SEQUENTIAL MONTECARLO INFERENCE TECH-NIQUES. In Proc. of Int’l Conf. on Acoustics, Speech and Signal Pro-cessing, pages 193–196, 2011.

[12] N. Orio, S. Lemouton, and D. Schwarz. Score Following: State of theArt and New Developments. In Proc. of Int’l Conf. on New Interfacesfor Musical Expression, pages 36–41, 2003.

[13] T. Otsuka, K. Nakadai, T. Takahashi, K. Komatani, T. Ogata, and H. G.Okuno. Real-Time Audio-to-Score Alignment using Particle Filter forCo-player Music Robots. EURASIP Journal of Advances in Signal Pro-cessing, vol. 2011, 2011. Article ID 384651.

[14] C. Raphael. Aligning music audio with symbolic scores using a hybridgraphical model. Machine Learning, 65(2–3):389–409, 2006.

[15] K. Yoshii and M. Goto. Infinite Latent Harmonic Allocation: A Non-parametric Bayesian Approach to Multipich Analysis. In Proc. of IS-MIR, pages 309–314, 2010.

530