Download - Rhythm related MIR tasks
Rhythm related MIR tasks
Ajay Srinivasamurthy1, André Holzapfel1
1MTG, Universitat Pompeu Fabra, Barcelona, Spain
10 July, 2012
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 1 / 23
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 2 / 23
Outline
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 3 / 23
Rhythm
Clayton (1996)Cooper and Meyer (1960)Rhythm: the way one or moreunaccented events are grouped inrelation to an accented one.
London (2001)Rhythm: pattern of durations that isphenomenally present in the music.
There is rhythm without meter, periodicity, or even without pulse!Examples: Turkish taksim, Beijing opera, Indian alapMIR research concentrated on rhythm in music with meter (mainlyWestern music, typically 4/4)It is not only “us”: Clayton (1996) reports the same tendency formusicology.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 4 / 23
Outline
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 5 / 23
Onset detection
Bello (2005) The figure depicts theflowchart of a standard onsetdetection algorithm.Evaluation happens usingeither manually annotatedonsets, or onsets derived frominstruments with MIDI outputs.
(a) Cello Spectr.Magn.
time/s
freq
/kH
z
0.15 0.29 0.44 0.58 0.73 0.87 1.02 1.16
2.75
1.38
4.1
(b) Guitar Spectr.Magn.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 6 / 23
Onset Detection
Pre-processingMultiband processingSeparation of percussive/harmonic content
ReductionUsing temporal, spectral magnitude, phase, or F0 information.Feature fusion was proposed to improve performance.Probabilistic models and neural networks were also applied forreduction.
Rhythmic structure was used to improve onset detection recently.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 7 / 23
Outline
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 8 / 23
What is Beat Tracking ?
“Tapping one’s foot in time to music" [Davies-07]
Extract a sequence of beat instants and the correspondinginter-beat intervals given an audio filePerceptually accurate beat timesLocally constant inter-beat intervals (IBI)Causal v/s non-causal beat trackingBeat Tracking on Symbolic v/s audio data
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 9 / 23
What is Beat Tracking ? (contd...)
ChallengesTempo variationOn-beat and off-beatGenre variationDifferent time signatures
Before we begin - Fundamental QuestionsIs the problem well defined? - Metrical levelsIs it an intuitive and an easy task for humans ?Ground truth for evaluation ?
“All (most) beats occur at onsets, but not all onsets are beats"
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 10 / 23
Components of a Beat Tracking System
Rhythmic Feature ExtractionExtract relevant rhythm featuresOnset detectionOnset Salience estimation
Tempo InductionDetermine the basic tempo/tempo hypothesesTempo definition for Indian music ?
Beat TrackingEstimate Beat timesBeat phase ?
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 11 / 23
Approaches to Beat Tracking
Tempo InductionPulse selection methods e.g BeatRoot [Dixon-06]Periodicity functions e.g. [Klapuri-06, Davies-07, Ellis-07]
Beat TrackingMultiple agent architecture e.g BeatRootStatistical Model [Klapuri-06]Dynamic Programming [Ellis-07]Context Dependent Tracking [Davies-07]
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 12 / 23
Beat Tracking Systems
Davies (2007)Dixon (2006) - BeatRoot
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 13 / 23
Examples of Beat Tracking
Money (Pink Floyd)
Clip Beats
Charleston Dance piece
Clip Beats
MahaganapatimNaata raga, Chaturashra Eka taala
Clip Beats
Light Indian classicalShivaranjani raga, Jhap ‘like’ taal (pentuple meter)
Clip Beats
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 14 / 23
Outline
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 15 / 23
Meter and Time Signature Recognition
Long-term, high level rhythm descriptionBeat similarity based approachOnset detection, Tempo estimation and Beat Tracking trackingnecessaryBeat Similarity matrix based approach: [Gainza-09]Recent work in Indian Music: [Gulati-11], [Miron-11]
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 16 / 23
Outline
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 17 / 23
Rhythm Similarity
Figure: Example for periodicitydescriptors (J.H.Jensen)
Periodicity descriptorsCan be considered state-of-the artin MIR.No beat estimation necessary →can be applied to all types ofsignalsTempo robustness can beachieved in various degreesHowever, phase information is lost.Usually evaluated on dance music,or in music similarity tasks.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 18 / 23
Rhythm Similarity: Sequential Description
General PropertiesNo information loss because not based on transform magnitudes.Disadvantage: Music must have a pulse which can be estimated.
Paulus and Klapuri (2002)Using spectral centroid and loudness to derive pattern description.Dynamic Time Warping is used to compare patterns.
Whiteley et al.(2007)
Probabilistic framework is proposedto infer tempo and rhythmic patternsOnly applied to MIDI signals in thispaper, rhythmic patterns are given.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 19 / 23
Outline
1 Rhythm
2 Onset detection
3 Tempo Estimation and Beat Tracking
4 Meter and Time Signature Recognition
5 Rhythmic Similarity/Classification
6 Structural Analysis
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 20 / 23
Structural Analysis
Chorus detectionMain goal: Detect repetitions of parts of a songMost common: Self-similarity matrix analysis
Cover song detectionIdentify if two songs are different interpretations of the samecompositionFeatures: e.g. chroma features, chord estimations.Similarity measures e.g. dynamic programming, string matching.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 21 / 23
Rhythm
Structural segmentationMain goal: Obtain a musical meaningful segmentation of a songinto large time-scale structures.Again, self similarities play a big role.Also, HMM were applied for labeling states at beat level, and thenfind similarities in the state distributions to obtain segments (Levyand Sandler (2008)).
Figure: Compare structure of a query in form of a score to the structure ofaudio (Martin et al. 2009)
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 22 / 23
References
Gouyon-05 Gouyon, F. A Computational Approach to Rhythm Description. PhD Thesis, Pompeu FabraUniversity, Barcelona, 2005
Dixon-06 Dixon, S. Evaluation of The Audio Beat Tracking System Beatroot. Journal of New MusicResearch 36 (1), pp. 39-50, 2006
Davies-07 Davies, M. and Plumbley, M. Context-Dependent Beat Tracking of Musical Audio. IEEETransactions on Audio, Speech, and Language Processing 15 (3), pp. 1009-1020, 2007.
Ellis-07 Ellis, D. P. W. Beat Tracking by Dynamic Programming. Journal of New Music Research,36(1), pp. 51-60, 2007.
Gainza-09 Gainza M., Automatic musical meter detection, in Proc. ICASSP 2009, pp. 329-332,Taipei, Taiwan
MIREX-06 http://www.music-ir.org/mirex/wiki/2006:Audio_Beat_Tracking
Uhle-03 Christian Uhle, Juergen Herre, Estimation of Tempo, Micro Time and Time Signature fromPercussive Music, in Proc. of 6th Int. Conference on Digital Audio Effects (DAFX-03),London, UK, September 8-11, 2003
Gulati-11 Sankalp Gulati, Vishweshwara Rao and Preeti Rao, Meter Detection from Audio for IndianMusic, CMMR/FRSM 2011, Bhubhaneswar, March 2011
Miron-11 M. Miron, Automatic Detection of Hindustani Talas. Master’s thesis, Universitat PompeuFabra, Barcelona, Spain, 2011.
Srinivasamurthy et al. (UPF) MIR tasks 10 July, 2012 23 / 23