music genre classification using traditional and

Music genre classificationusing traditional and relational approaches

Jorge Valverde-Rebaza, Aurea Soriano, Lilian Berton,Maria Cristina F. Oliveira and Alneu de Andrade Lopes

Laboratory of Computational Intelligence (LABIC)

Laboratory of Visualization, Imaging and Computer Graphics (VICG)

University of Sao Paulo (USP)

Brazil

October 2014

Outline

1 Introduction

2 Method

3 Experimental Evaluation

4 Conclusion

Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 2 / 20

Outline

1 Introduction

2 MethodMusic FeaturesGraph constructionGraph construction

3 Experimental EvaluationDatasetsExperimental setupResults

4 Conclusion

Introduction

Many music collections, typically very large

Manual music classification: a non-expert person can identify the genre of a music with

72% accuracy after hearing 3 seconds of it [Perrot and Gjerdigen, 1999].Classifying a large collection demands time, effort and expertise

Automatic music classification: Solutions achieve high accuracy (ranging from 63 to

84%) [Shao et al., 2004, Pampalk et al., 2005, Scaringella and Mlynek, 2005,

Yaslan and Cataltepe, 2009, Poria et al., 2013]require music files with the same sizeapplied on controlled environments, i.e. without considering class imbalanceemployed traditional classifiers

Introduction

Outline

1 Introduction

4 Conclusion

Music feature extraction

Feature extracted from MIDI description

(12)Histograms

Distance: DTW

(8) Moments(4) Melody and (4) Rhythm: themean, standard deviation, entropy

and uniformityDistance: Euclidean

(248)StructureIdentification of patterns from

chord sequences. Generatesvectors of different sizes.

Distance: DTW

(12)Histograms

Distance: DTW

(12)Histograms

Distance: DTW

(12)Histograms

Distance: DTW

(12)Histograms

Distance: DTW

Graph construction

Graph construction techniques

Connect each vertex onlyto its k nearest neighbors

mutual-kNN

Two vertices areconnected only if theneighborhood pertinencecondition is met by both

regular-kNN

All the vertices have thesame degree

Graph construction

mutual-kNN

regular-kNN

Graph construction

mutual-kNN

regular-kNN

Graph construction

mutual-kNN

regular-kNN

Graph construction

mutual-kNN

regular-kNN

Classifiers

Traditional

Map input data to acategory

Decision trees, naıveBayes, neural networks,support vector machine,etc

Relational

Map relational input data to acategory

Weighted vote relational neighbor,network-only Bayes, probabilisticrelational neighbor, network-onlylink based [Macskassy, 2007]

Classifiers

Traditional

Relational

Classifiers

Traditional

Relational

Classifiers

Traditional

Relational

Classifiers

Traditional

Relational

Outline

1 Introduction

4 Conclusion

Datasets

Tabela: Music genre distribution for the music collection considered

Genre # Tracks

Classical 31Brazilian Backcountry 243Pop/Rock 550Jazz 95

Total 919

Experimental setup

Feature vectors

Histogram Moments Structure

Graph construction (1 ≤ k ≤ 15 )

kNN mutual-kNN regular-kN

Classifiers

Traditional Relational

Decision tree (J48)

Naıve Bayes (NB)

Multilayer perceptron withbackpropagation (MLP)

Support vector machine (SMO)

weighted vote relational neighbor (wvrn)

network-only Bayes (no-Bayes)

probabilistic relational neighbor (prn)

network-only link-based

mode-link (no-lb-mode)count-link (no-lb-count)binary-link (no-lb-binary)class-distribution-link (no-lb-distrib)

Experimental setup

Feature vectors

Classifiers

Decision tree (J48)

Naıve Bayes (NB)

Experimental setup

Feature vectors

Classifiers

Decision tree (J48)

Naıve Bayes (NB)

Traditional classification results

Tabela: Traditional classifiers performance measured by AUC

J48 NB MLP SMO

Histogram 0.619 0.607 0.665 0.506Moments 0.706 0.750 0.771 0.585Structure 0.738 0.920 0.816 0.724

Average rank 2.667 2.000 1.333 4.000

1 2 3 4

MLPNB J48

Figura: Post-hoc test results for traditional classifiers performance

Traditional classification results

Tabela: Traditional classifiers performance measured by AUC

J48 NB MLP SMO

Histogram 0.619 0.607 0.665 0.506Moments 0.706 0.750 0.771 0.585Structure 0.738 0.920 0.816 0.724

Average rank 2.667 2.000 1.333 4.000

1 2 3 4

MLPNB J48

Figura: Post-hoc test results for traditional classifiers performance

Relational classification results: using kNN

Tabela: Relational classifiers performance evaluated by AUC in kNN networks

no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn

Histogram 0.575 (k=11) 0.723 (k=9) 0.622 (k=2) 0.71 (k=8) 0.712 (k=9) 0.515 (k=1) 0.537 (k=1)

Moments 0.547 (k=5) 0.635 (k=13) 0.575 (k=8) 0.644 (k=7) 0.644 (k=9) 0.563 (k=2) 0.571 (k=3)

Structure 0.834 (k=7) 0.939 (k=14) 0.851 (k=4) 0.945 (k=14) 0.931 (k=15) 0.922 (k=15) 0.903 (k=9)

Average rank 6.333 2.000 4.667 1.667 2.333 5.667 5.333

1 2 3 4 5 6 7

no-lb-distribno-lb-count

wvrnno-lb-binary

prnno-Bayesno-lb-mode

Figura: Post-hoc test results for relational classifiers built on kNN networks

Relational classification results: using kNN

Tabela: Relational classifiers performance evaluated by AUC in kNN networks

Moments 0.547 (k=5) 0.635 (k=13) 0.575 (k=8) 0.644 (k=7) 0.644 (k=9) 0.563 (k=2) 0.571 (k=3)

Average rank 6.333 2.000 4.667 1.667 2.333 5.667 5.333

1 2 3 4 5 6 7

no-lb-distribno-lb-count

wvrnno-lb-binary

prnno-Bayesno-lb-mode

Figura: Post-hoc test results for relational classifiers built on kNN networks

Relational classification results: using mutual-kNN

Tabela: Relational classifiers performance evaluated by AUC in mutual-kNNnetworks

Moments 0.570 (k=1) 0.657 (k=14) 0.588 (k=2) 0.633 (k=15) 0.630 (k=14) 0.578 (k=1) 0.574 (k=1)

Average rank 6.000 2.333 5.000 1.667 2.000 5.333 5.667

1 2 3 4 5 6 7

no-lb-distribwvrn

no-lb-countno-lb-binary

no-Bayesprnno-lb-mode

Figura: Post-hoc test results for relational classifiers built on mutual-kNNnetworks

Relational classification results: using mutual-kNN

Tabela: Relational classifiers performance evaluated by AUC in mutual-kNNnetworks

Moments 0.570 (k=1) 0.657 (k=14) 0.588 (k=2) 0.633 (k=15) 0.630 (k=14) 0.578 (k=1) 0.574 (k=1)

Average rank 6.000 2.333 5.000 1.667 2.000 5.333 5.667

1 2 3 4 5 6 7

no-lb-distribwvrn

no-Bayesprnno-lb-mode

Figura: Post-hoc test results for relational classifiers built on mutual-kNNnetworks

Relational classification results: using regular-kNN

Tabela: Relational classifiers performance evaluated by AUC in regular-kNNnetworks

Moments 0.569 (k=1) 0.652 (k=13) 0.571 (k=1) 0.620 (k=8) 0.625 (k=6) 0.560 (k=1) 0.565 (k=1)

Average rank 5.333 2.333 5.000 1.667 2.000 6.000 5.667

1 2 3 4 5 6 7

no-lb-distribwvrn

no-lb-modeprnno-Bayes

Figura: Post-hoc test results for relational classifiers built on regular-kNNnetworks

Relational classification results: using regular-kNN

Tabela: Relational classifiers performance evaluated by AUC in regular-kNNnetworks

Moments 0.569 (k=1) 0.652 (k=13) 0.571 (k=1) 0.620 (k=8) 0.625 (k=6) 0.560 (k=1) 0.565 (k=1)

Average rank 5.333 2.333 5.000 1.667 2.000 6.000 5.667

1 2 3 4 5 6 7

no-lb-distribwvrn

no-lb-modeprnno-Bayes

Figura: Post-hoc test results for relational classifiers built on regular-kNNnetworks

Best performances

regular-kNNmutual-kNN

Figura: Post-hoc test results for identifying the influence of network constructiontechniques in relational classifiers

1 2 3 4

no-lb-distribwvrn MLP

Figura: Post-hoc test results for the comparison between the best traditional andrelational classifiers

Best performances

regular-kNNmutual-kNN

Figura: Post-hoc test results for identifying the influence of network constructiontechniques in relational classifiers

1 2 3 4

no-lb-distribwvrn MLP

Figura: Post-hoc test results for the comparison between the best traditional andrelational classifiers

Outline

1 Introduction

4 Conclusion

Conclusion

We introduce a novel feature vector (called Structural) whichcaptures information encoded in the MIDI files

We evaluated traditional and relational classifiers on a musiccollection with an imbalanced distribution of four music genres

The Structural features resulted in improved performance of bothtraditional and relational classifiers

The regular-kNN networks provided the relational model mostsuitable to improve the performance of relational classifiers

Relational classifiers perform better than traditional classifiers onmusic genre classification

Conclusion

References

Macskassy, S. A. (2007).

Improving learning in networked data by combiningexplicit and mined links.In AAAI, pages 590–595.

Pampalk, E., Flexer, A., Widmer, G., et al. (2005).

Improvements of audio-based music similarity andgenre classificaton.In ISMIR, volume 5, pages 634–637.

Perrot, D. and Gjerdigen, R. (1999).

Scanning the dial: An exploration of factors in theidentification of musical style.In Proceedings of the 1999 Society for MusicPerception and Cognition, page 88.

Poria, S., Gelbukh, A., Hussain, A., Bandyopadhyay,

S., and Howard, N. (2013).Music genre classification: A semi-supervisedapproach.

In Pattern Recognition, pages 254–263.

Scaringella, N. and Mlynek, D. (2005).

A mixture of support vector machines for audioclassification.IEEE MIREX, London.

Shao, X., Xu, C., and Kankanhalli, M. S. (2004).

Unsupervised classification of music genre using hiddenmarkov model.In Multimedia and Expo, 2004. ICME’04. 2004 IEEEInternational Conference on, volume 3, pages2023–2026.

Yaslan, Y. and Cataltepe, Z. (2009).

Audio genre classification with semi-supervised featureensemble learning.In Second International Workshop on MachineLearning and Music.

Thank you!

Jorge Valverde-Rebaza

[email protected]

music genre classification using traditional and

Documents