conditional restricted boltzmann machines for mono/polyphonic … 11 05 bnaic.pdf · 11/19 dataset...

Conditional Restricted Boltzmann Machinesfor Mono/Polyphonic Composer Identification

BNAIC 2015, November 6th 2015Johan Loeckx, Joeri Bultheel

Artificial Intelligence LabVrije Universiteit BrusselBelgium

2/19

Ultimate goal

Given a symbolic representation of a composer's piece

Identify the composer

Based on a set of previous compositions

More specifically... Mozart vs. Haydn string quartets (contemporary composers with a similar style)

3/19

Outline

I. Computational representation of music➔ Existing representations➔ Subtleties in music

II. Methodology➔ Conditional RBMs➔ Topology➔ Dataset

III. Results & discussion➔ Accuracy➔ Training/test set division

IV. Conclusions

4/19

– I –Computational representations of music

5/19

Computational representations

A lot of computational symbolic representations exist

Model type➔ Markov models➔ (Context-free) Grammars➔ Finite State Machines➔ Feature models➔ N-grams➔ ...

Issues ➔ Musicality / interpretation ➔ Cultural aspect vs. Cognitive ➔ Semantics vs. syntax➔ Discourse level

6/19

Musical subtleties

Motion: different resolutions: hierarchy of ascending - descending

Seconds in high & low register

Pedal tones

In which key is this piece?

Long term “dependencies” (often with local corrections / interactions)

What is not there, matters

7/19

Composer identification:State-of-the-art

Hand-crafted feature models (polyphonic) [1]➔ 5-fold cross-validation● 80% accuracy

N-gram models (monophonic) [2]➔ Leave-one-out ➔ 75.4% accuracy

When discriminating between Mozart & Haydn's string quartets...

8/19

– II –Methodology

Do Conditional RBMs make good unsupervised feature extractors for composer identification?

9/19

Conditional Restricted Boltzmann Machine

In our case a 10s “memory”, composed of 80 125ms quantized sections

of 16th notes

10/19

Topology

11/19

Dataset

Complete set of Haydn's & Mozart string quartets➔ Very similar musical style (classical)➔ Each string quartet consists of 3 movements➔ The data set was ordered by time of writing

Three scenario's were considered➔Scenario 0: all movements were shuffled➔Scenario I: test & training movement never belong to same quartet ➔Scenario II: all training data was “written” before test data

12/19

– III –Results & Discussion

13/19

Scenario 0

Monophonic binary prediction accuracy

Polyphonic binary prediction accuracy (voting)A voting scheme based upon the 4 monophonic predictions

above, yielded a 96% accuracy

5-fold cross-validation on movement-level

≈ baseline / similar for both composers (bass line most determined by style and not by the composer?)

14/19

15/19

Scenario 1 (training & test movements never belong to same quartet)

(not all data used as a consequence)

16/19

Scenario 2 (training movements written before the test movements)

17/19

– IV –Conclusions

18/19

Conclusions

Conditional RBMs useful as feature extractors➔Considerable improvement over the state-of-the-art

Style evolves within a composer's work➔ If the model is not trained on contemporary pieces (compared to the test set)

➔ If the model is trained on “older” pieces only

Future work➔ What does it teach us (insight?)

19/19

References

[1] William Herlands, Ricky Der, Yoel Greenberg, and Simon Levin. A machine learning approach to musically meaningful homogeneous style classification. In Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014.

[2] Ruben Hillewaere, Bernard Manderick, and Darrell Conklin. String quartet classification with monophonic models. In ISMIR, pages 537–542, 2010.