Download - Advanced Functional Programming in Industry
-
8/10/2019 Advanced Functional Programming in Industry
1/84
Advanced Functional Programming in Industry
Jose Pedro Magalhaes
November 21, 2014London, United Kingdom
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 1 / 46
-
8/10/2019 Advanced Functional Programming in Industry
2/84
Introduction
Haskell: a statically typed, lazy, purely functional language
Modelling musical harmony using Haskell
Applications of a model of harmony: Musical analysis Finding cover songs Generating chords and melodies Correcting errors in chord extraction from audio sources Chordifya web-based music player with chord recognition
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 2 / 46
-
8/10/2019 Advanced Functional Programming in Industry
3/84
Demo: Chordify
Demo:
http://chordify.net
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 3 / 46
http://chordify.net/http://chordify.net/ -
8/10/2019 Advanced Functional Programming in Industry
4/84
Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 4 / 46
-
8/10/2019 Advanced Functional Programming in Industry
5/84
What is harmony?
IVIVI V/V
C F D G7 7
C
{Ton TonSDom Dom
Harmony arises when at least two notes sound at the same time
Harmony induces tension and release patterns, that can be describedby music theory and music cognition
The internal structure of the chord has a large influence on theconsonance or dissonance of a chord
The surrounding context also has a large influence
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 5 / 46
-
8/10/2019 Advanced Functional Programming in Industry
6/84
What is harmony?
IVIVI V/V
C F D G7 7
C
{Ton TonSDom Dom
Harmony arises when at least two notes sound at the same time
Harmony induces tension and release patterns, that can be describedby music theory and music cognition
The internal structure of the chord has a large influence on theconsonance or dissonance of a chord
The surrounding context also has a large influence
Demo: how harmony affects melody
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 5 / 46
-
8/10/2019 Advanced Functional Programming in Industry
7/84
Simplified harmony theory I
A chordis a group of tones separated by intervals of roughly thesame size.
All music is made out of chords (whether explicitly or not).
There are 12 different notes. Instead of naming them, we numberthem relative to the first and most important one, the tonic. So weget I, II, II . . . VI, VII.
A chord is built on a root note. So I also stands for the chord builton the first degree, V for the chord built on the fifth degree, etc.
So the following is a chord sequence: I IV II
7
V
7
I.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 6 / 46
-
8/10/2019 Advanced Functional Programming in Industry
8/84
Simplified harmony theory II
Models for musical harmony explain the harmonic progression in music:
Everything works around the tonic(I).
The dominant(V) leads to the tonic.
The subdominant(IV) tends to lead to the dominant.
Therefore, the I IV V I progression is very common. There are also secondary dominants, which lead to a relative tonic.
For instance, II7 is the secondary dominant of V, and I7 is thesecondary dominant of IV.
So you can start with I, add one note to get I7, fall into IV, change
two notes to get to II7, fall into V, and then finally back to I.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 7 / 46
-
8/10/2019 Advanced Functional Programming in Industry
9/84
An example harmonic analysis
IVIVI V/V
C F D G7 7
C
{Ton TonSDom Dom
Piece
PT
T
I
C
PD
D
D
V/V
V7
G:7
II7
D:7
S
IV
F
PT
T
I
C
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 8 / 46
-
8/10/2019 Advanced Functional Programming in Industry
10/84
Why are harmony models useful?
Having a model for musical harmony allows us to automaticallydetermine the functional meaning of chords in the tonal context.The model determines which chords fit on a particular moment in asong.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 9 / 46
-
8/10/2019 Advanced Functional Programming in Industry
11/84
Why are harmony models useful?
Having a model for musical harmony allows us to automaticallydetermine the functional meaning of chords in the tonal context.The model determines which chords fit on a particular moment in asong. This is useful for:
Musical information retrieval (find songs similar to a given song) Audio and score recognition (improving recognition by knowing
which chords are more likely to appear)
Music generation (create sequences of chords that conform to themodel)
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 9 / 46
-
8/10/2019 Advanced Functional Programming in Industry
12/84
Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 10 / 46
-
8/10/2019 Advanced Functional Programming in Industry
13/84
Why Haskell?
Haskell is a strongly-typed pure functional programming language:
Strongly-typed All values are classified by their type, and types areknown at compile time (statically). This gives us strongguarantees about our code, avoiding many common
mistakes.
Pure There are no side-effects, so Haskell functions are likemathematical functions.
Functional A Haskell program is an expression, not a sequence ofstatements. Functions are first class citizens, and explicitstate is avoided.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 11 / 46
-
8/10/2019 Advanced Functional Programming in Industry
14/84
Notes
dataRoot =A | B | C | D | E | F | GtypeOctave=Int
dataNote =NoteRoot Octave
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 12 / 46
-
8/10/2019 Advanced Functional Programming in Industry
15/84
Notes
dataRoot =A | B | C | D | E | F | GtypeOctave=Int
dataNote =NoteRoot Octave
a4, b4, c4, d4, e4, f4, g4 ::Note
a4 =Note A 4b4 =Note B 4c4 =Note C 4d4 =Note D 4e4 =Note E 4
f4 =Note F 4
g4 =Note G 4
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 12 / 46
-
8/10/2019 Advanced Functional Programming in Industry
16/84
Melody
typeMelody= [Note]
cMajScale ::MelodycMajScale = [c4, d4, e4, f4, g4, a4, b4]
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 13 / 46
-
8/10/2019 Advanced Functional Programming in Industry
17/84
Melody
typeMelody= [Note]
cMajScale ::MelodycMajScale = [c4, d4, e4, f4, g4, a4, b4]
cMajScaleRev ::Melody
cMajScaleRev = reverse cMajScale
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 13 / 46
-
8/10/2019 Advanced Functional Programming in Industry
18/84
Melody
typeMelody= [Note]
cMajScale ::MelodycMajScale = [c4, d4, e4, f4, g4, a4, b4]
cMajScaleRev ::Melody
cMajScaleRev = reverse cMajScale
reverse :: [ ] [ ]reverse [ ] = [ ]reverse (h : t) = reverse t ++ [ h]
(++) :: [ ] [ ] [ ](++) =. . .
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 13 / 46
T i i
-
8/10/2019 Advanced Functional Programming in Industry
19/84
Transposition
Transposing a melody one octave higher:
octaveUp ::Octave OctaveoctaveUp n = n + 1
noteOctaveUp ::Note NotenoteOctaveUp (Noter o) =Noter (octaveUp o)
melodyOctaveUp ::Melody MelodymelodyOctaveUp m = map noteOctaveUp m
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 14 / 46
G i l i
-
8/10/2019 Advanced Functional Programming in Industry
20/84
Generation, analysis
Building a canon from a melody:
canon ::Melody Melodycanon m = m ++ canon m
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 15 / 46
G ti l i
-
8/10/2019 Advanced Functional Programming in Industry
21/84
Generation, analysis
Building a canon from a melody:
canon ::Melody Melodycanon m = m ++ canon m
Is a given melody in C major?
root ::Note Rootroot (Noter o) = r
isCMaj ::Melody BoolisCMaj = all (cMajScale)map root
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 15 / 46
D t il l ft t
-
8/10/2019 Advanced Functional Programming in Industry
22/84
Details left out
We have seen only a glimpse of music representation in Haskell.
Rhythm
Accidentals
Intervals
Voicing
. . .
A good pedagogical reference on using Haskell to represent music:http://di.uminho.pt/~jno/html/ipm-1011.html
A serious library for music manipulation:http://www.haskell.org/haskellwiki/Haskore
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 16 / 46
Table of Co te ts
http://di.uminho.pt/~jno/html/ipm-1011.htmlhttp://di.uminho.pt/~jno/html/ipm-1011.htmlhttp://di.uminho.pt/~jno/html/ipm-1011.htmlhttp://www.haskell.org/haskellwiki/Haskorehttp://www.haskell.org/haskellwiki/Haskorehttp://di.uminho.pt/~jno/html/ipm-1011.html -
8/10/2019 Advanced Functional Programming in Industry
23/84
Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 17 / 46
Application: harmony analysis
-
8/10/2019 Advanced Functional Programming in Industry
24/84
Application: harmony analysis
Parsing the sequenceGminC7 Gmin C
7 FMaj D7 G7 CMaj:
Piece
PT
T
IC:maj
PD
D
DD
V7
G:7
V/V
II7
D:7
S
IV
F:maj
V/IV
I7
C:7
V/I
Vmin
G:min
SIV
ins
V/IV
I7
C:7
V/I
Vmin
G:min
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 18 / 46
Table of Contents
-
8/10/2019 Advanced Functional Programming in Industry
25/84
Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 19 / 46
Application: harmonic similarity
-
8/10/2019 Advanced Functional Programming in Industry
26/84
Application: harmonic similarity
A practical application of a harmony model is to estimate harmonicsimilarity between songs
The more similar the trees, the more similar the harmony
We dont want to write a diff algorithm for our complicated model;
we get it automatically by using a generic diff The generic diff is a type-safe tree-diff algorithm, part of a students
MSc work at Utrecht University
Generic, thus working for any model, and independent of changes tothe model
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 20 / 46
Table of Contents
-
8/10/2019 Advanced Functional Programming in Industry
27/84
Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 21 / 46
Application: automatic harmonisation of melodies
-
8/10/2019 Advanced Functional Programming in Industry
28/84
Application: automatic harmonisation of melodies
Another practical application of a harmony model is to help selecting
good harmonisations (chord sequences) for a given melody:
IV
III
V
V
I
III
IV
III
II
We generate candidate chord sequences, parse them with the harmonymodel, and select the one with the least errors.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 22 / 46
Visualising harmonic structure
-
8/10/2019 Advanced Functional Programming in Industry
29/84
Visualising harmonic structure
Piece
Phrase
Ton
I: Maj
C: Maj
Dom
Dom
V: Dom7
G: Dom7
II: Dom7
D: Dom7
Sub
IV: Maj
F: Maj
III: Min
E: Min
Ton
I: Maj
C: Maj
You can see this tree as having been produced by taking the chords ingreen as input. . .
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 23 / 46
Generating harmonic structure
-
8/10/2019 Advanced Functional Programming in Industry
30/84
Generating harmonic structure
Piece
Phrase
Ton
I: Maj
C: Maj
Dom
Dom
V: Dom7
G: Dom7
II: Dom7
D: Dom7
Sub
IV: Maj
F: Maj
III: Min
E: Min
Ton
I: Maj
C: Maj
You can see this tree as having been produced by taking the chords ingreen as input. . . or the chords might have been dictated by the structure!
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 23 / 46
A functional model of harmony
-
8/10/2019 Advanced Functional Programming in Industry
31/84
A functional model of harmony
PieceM [PhraseM] (M {Maj, Min})
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 24 / 46
A functional model of harmony
-
8/10/2019 Advanced Functional Programming in Industry
32/84
A functional model of harmony
PieceM [PhraseM] (M {Maj, Min})
PhraseM TonM DomM TonM| DomM TonM
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 24 / 46
A functional model of harmony
-
8/10/2019 Advanced Functional Programming in Industry
33/84
y
PieceM [PhraseM] (M {Maj, Min})
PhraseM TonM DomM TonM| DomM TonM
TonMaj IMajTonMinI
m
Min
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 24 / 46
A functional model of harmony
-
8/10/2019 Advanced Functional Programming in Industry
34/84
y
PieceM [PhraseM] (M {Maj, Min})
PhraseM TonM DomM TonM| DomM TonM
TonMaj IMajTonMinI
m
Min
DomM V7M
| VM
| VII0M| SubM DomM
| II7M V7M
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 24 / 46
A functional model of harmony
-
8/10/2019 Advanced Functional Programming in Industry
35/84
y
PieceM [PhraseM] (M {Maj, Min})
PhraseM TonM DomM TonM| DomM TonM
TonMaj IMajTonMinI
m
Min
DomM V7M
| VM
| VII0M| SubM DomM
| II7M V7M
SubMaj IIm
Maj
| IVMaj| IIImMaj IVMaj
SubMinIVm
Min
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 24 / 46
A functional model of harmony
-
8/10/2019 Advanced Functional Programming in Industry
36/84
y
PieceM [PhraseM] (M {Maj, Min})
PhraseM TonM DomM TonM| DomM TonM
TonMaj IMajTonMinI
m
Min
DomM V7M
| VM
| VII0M| SubM DomM
| II7M V7M
SubMaj IIm
Maj
| IVMaj| IIImMaj IVMaj
SubMinIVm
Min
Simple, but enough for now, and easy to extend.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 24 / 46
Now in HaskellI
-
8/10/2019 Advanced Functional Programming in Industry
37/84
A naive datatype encoding musical harmony:
dataPiece=Piece[ Phrase ]
dataPhrasewherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 25 / 46
Now in HaskellI
-
8/10/2019 Advanced Functional Programming in Industry
38/84
A naive datatype encoding musical harmony:
dataPiece=Piece[ Phrase ]
dataPhrasewherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
dataTonwhereTonMaj::Degree Ton
TonMin::Degree Ton
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 25 / 46
Now in HaskellI
-
8/10/2019 Advanced Functional Programming in Industry
39/84
A naive datatype encoding musical harmony:
dataPiece=Piece[ Phrase ]
dataPhrasewherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
dataTonwhereTonMaj::Degree Ton
TonMin::Degree Ton
dataDom whereDomV7 ::Degree DomDomV ::Degree DomDomVII0 ::Degree Dom
DomIVV::SDom Dom DomDomIIV ::Degree Degree Dom
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 25 / 46
Now in HaskellI
-
8/10/2019 Advanced Functional Programming in Industry
40/84
A naive datatype encoding musical harmony:
dataPiece=Piece[ Phrase ]
dataPhrasewherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
dataTonwhereTonMaj::Degree Ton
TonMin::Degree Ton
dataDom whereDomV7 ::Degree DomDomV ::Degree DomDomVII0 ::Degree Dom
DomIVV::SDom Dom DomDomIIV ::Degree Degree Dom
dataDegree=I | II| III . . .
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 25 / 46
Now in HaskellII
-
8/10/2019 Advanced Functional Programming in Industry
41/84
A GADT encoding musical harmony:
dataMode=MajMode|MinMode
dataPiece(::Mode)where
Piece::[ Phrase ] Piece
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 26 / 46
Now in HaskellIIG
-
8/10/2019 Advanced Functional Programming in Industry
42/84
A GADT encoding musical harmony:
dataMode=MajMode|MinMode
dataPiece(::Mode)where
Piece::[ Phrase ] Piece
dataPhrase(::Mode)wherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 26 / 46
Now in HaskellIIA GADT di i l h
-
8/10/2019 Advanced Functional Programming in Industry
43/84
A GADT encoding musical harmony:
dataMode=MajMode|MinMode
dataPiece(::Mode)where
Piece::[ Phrase ] Piece
dataPhrase(::Mode)wherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
dataTon(::Mode)whereTonMaj::SD I Maj Ton MajModeTonMin::SD I Min Ton MinMode
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 26 / 46
Now in HaskellIIA GADT di i l h
-
8/10/2019 Advanced Functional Programming in Industry
44/84
A GADT encoding musical harmony:
dataMode=MajMode|MinMode
dataPiece(::Mode)where
Piece::[ Phrase ] Piece
dataPhrase(::Mode)wherePhraseIVI::Ton Dom Ton PhrasePhraseVI :: Dom Ton Phrase
dataTon(::Mode)whereTonMaj::SD I Maj Ton MajModeTonMin::SD I Min Ton MinMode
dataDom(::Mode)whereDomV7 ::SD V Dom
7 Dom
DomV ::SD V Maj Dom DomVII0 ::SD VII Dim Dom DomIVV::SDom Dom Dom DomIIV ::SD II Dom
7 SD V Dom7 Dom
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 26 / 46
Now in HaskellIII
-
8/10/2019 Advanced Functional Programming in Industry
45/84
Scale degrees are the leaves of our hierarchical structure:
dataDiatonicDegree=I | II| III | IV |V | VI | VIIdataQuality =Maj | Min | Dom7 |Dim
dataSD(::DiatonicDegree) (::Quality)whereSurfaceChord::ChordDegree SD
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 27 / 46
Now in HaskellIII
-
8/10/2019 Advanced Functional Programming in Industry
46/84
Scale degrees are the leaves of our hierarchical structure:
dataDiatonicDegree=I | II| III | IV |V | VI | VIIdataQuality =Maj | Min | Dom7 |Dim
dataSD(::DiatonicDegree) (::Quality)whereSurfaceChord::ChordDegree SD
Now everything is properly indexed, and our GADT is effectivelyconstrained to allow only harmonically valid sequences!
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 27 / 46
Generating harmony
-
8/10/2019 Advanced Functional Programming in Industry
47/84
Now that we have a datatype representing harmony sequences, how dowe generate a sequence of chords?
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 28 / 46
Generating harmony
-
8/10/2019 Advanced Functional Programming in Industry
48/84
Now that we have a datatype representing harmony sequences, how dowe generate a sequence of chords?
QuickCheck! We giveArbitraryinstances for each of the datatypes in our
model.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 28 / 46
Generating harmony
-
8/10/2019 Advanced Functional Programming in Industry
49/84
Now that we have a datatype representing harmony sequences, how dowe generate a sequence of chords?
QuickCheck! We giveArbitraryinstances for each of the datatypes in our
model.
. . . but we dont want to do this by hand, for every datatype, and to haveto adapt the instances every time we change the model. . . so we usegeneric programming:
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 28 / 46
Generating harmony
-
8/10/2019 Advanced Functional Programming in Industry
50/84
Now that we have a datatype representing harmony sequences, how do
we generate a sequence of chords?
QuickCheck! We giveArbitraryinstances for each of the datatypes in ourmodel.
. . . but we dont want to do this by hand, for every datatype, and to haveto adapt the instances every time we change the model. . . so we usegeneric programming:
gen :: .(Representable, Generate(Rep))Gen
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 28 / 46
Generating harmony
-
8/10/2019 Advanced Functional Programming in Industry
51/84
Now that we have a datatype representing harmony sequences, how do
we generate a sequence of chords?
QuickCheck! We giveArbitraryinstances for each of the datatypes in ourmodel.
. . . but we dont want to do this by hand, for every datatype, and to haveto adapt the instances every time we change the model. . . so we usegeneric programming:
gen :: .(Representable, Generate(Rep))[ (String,Int) ] Gen
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 28 / 46
Examples of harmony generation
-
8/10/2019 Advanced Functional Programming in Industry
52/84
testGen :: Gen(Phrase MajMode)
testGen = gen [("Dom_IV-V", 3), ("Dom_II-V", 4)]example :: IO ()example =letk =Key(Note C)MajMode
insample testGen>>= mapM (printOnKey k)
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 29 / 46
Examples of harmony generation
-
8/10/2019 Advanced Functional Programming in Industry
53/84
testGen :: Gen(Phrase MajMode)
testGen = gen [("Dom_IV-V", 3), ("Dom_II-V", 4)]example :: IO ()example =letk =Key(Note C)MajMode
insample testGen>>= mapM (printOnKey k)
>example[C: Maj, D: Dom7, G: Dom7, C: Maj][C: Maj, G: Dom7, C: Maj][C: Maj, E: Min, F: Maj, G: Maj, C: Maj]
[C: Maj, E: Min, F: Maj, D: Dom7, G: Dom7, C: Maj][C: Maj, D: Min, E: Min, F: Maj, D: Dom7, G: Dom7, C: Maj]
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 29 / 46
Generating a melody for a given harmony
-
8/10/2019 Advanced Functional Programming in Industry
54/84
We then generate a melody in 4 steps:
1. Generate a list of candidate melody notes per chord;
2. Refine the candidates by filtering out obviously bad candidates;
3. Pick one focal candidate melody note per chord;
4. Embellish the candidate notes to produce a final melody.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 30 / 46
Generating a melody for a given harmony
-
8/10/2019 Advanced Functional Programming in Industry
55/84
We then generate a melody in 4 steps:
1. Generate a list of candidate melody notes per chord;
2. Refine the candidates by filtering out obviously bad candidates;
3. Pick one focal candidate melody note per chord;
4. Embellish the candidate notes to produce a final melody.
These four steps combine naturally using plain monadic bind:
melody ::KeyState MyState Songmelody k = genCandidates>>= refine>>= pickOne>>= embellish
>>= returnSongk
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 30 / 46
Example I
-
8/10/2019 Advanced Functional Programming in Industry
56/84
&?
#
n
Phrase
Ton
I: Maj
C: Maj
Dom
Dom
V: Dom7
G: Dom7II: Dom
7
D: Dom7
Sub
IV: Maj
F: Maj
III: Min
E: Min
Ton
I: Maj
C: Maj
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 31 / 46
Example II
-
8/10/2019 Advanced Functional Programming in Industry
57/84
?#
## n#
Phrase
Ton
I: Min
E: Min
Dom
Dom
Dom
V: Dom7
B: Dom7
II: Dom7
F: Dom7
Sub
IV: Min
A: Min
Sub
IV: Min
A: Min
Ton
I: Min
E: Min
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 32 / 46
Table of Contents
-
8/10/2019 Advanced Functional Programming in Industry
58/84
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 33 / 46
Back to Chordify: chord recognition
-
8/10/2019 Advanced Functional Programming in Industry
59/84
Yet another practical application of a harmony model is to improve chordrecognition from audio sources.
0.92 C 0.96 EmChord candidates 0.94 Gm 0.97 C
1.00 C 1.00 G 1.00 EmBeat number 1 2 3
How to pick the right chord from the chord candidate list? Ask theharmony model which one fits best.
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 34 / 46
Chordify: architecture
-
8/10/2019 Advanced Functional Programming in Industry
60/84
Frontend Reads user input, such as YouTube/Soundcloud/Deezer links, or files Extracts audio Calls the backend to obtain the chords for the audio Displays the result to the user Implements a queueing system, and library functionality
Uses PHP, JavaScript, MongoDB
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 35 / 46
Chordify: architecture
http://www.lilypond.org/http://hackage.haskell.org/package/HarmTrace -
8/10/2019 Advanced Functional Programming in Industry
61/84
Frontend Reads user input, such as YouTube/Soundcloud/Deezer links, or files Extracts audio Calls the backend to obtain the chords for the audio Displays the result to the user Implements a queueing system, and library functionality
Uses PHP, JavaScript, MongoDB Backend
Takes an audio file as input, analyses it, extracts the chords The chord extraction code uses GADTs, type families, generic
programming (see theHarmTracepackage on Hackage) Performs PDF and MIDI export (usingLilyPond) Uses Haskell, SoX, sonic annotator, and is mostly open source
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 35 / 46
Chordify: numbers
http://hackage.haskell.org/package/HarmTracehttp://www.lilypond.org/http://www.lilypond.org/http://hackage.haskell.org/package/HarmTrace -
8/10/2019 Advanced Functional Programming in Industry
62/84
Online since January 2013
Top countries: US, UK, Germany, Indonesia, Canada
Views: 3M+ (monthly)
Chordified songs: 1.5M+
Registered users: 200K
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 36 / 46
How do we handle these visitors?
-
8/10/2019 Advanced Functional Programming in Industry
63/84
Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB harddrive
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 37 / 46
How do we handle these visitors?
-
8/10/2019 Advanced Functional Programming in Industry
64/84
Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB harddrive
Single server hosts both the web and database servers
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 37 / 46
How do we handle these visitors?
-
8/10/2019 Advanced Functional Programming in Industry
65/84
Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB harddrive
Single server hosts both the web and database servers
Can easily handle peaks of (at least) 700 visitors at a time
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 37 / 46
How do we handle these visitors?
-
8/10/2019 Advanced Functional Programming in Industry
66/84
Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB harddrive
Single server hosts both the web and database servers
Can easily handle peaks of (at least) 700 visitors at a time
Chordifying new songs takes some computing power, but most songsare in the database already
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 37 / 46
How do we handle these visitors?
-
8/10/2019 Advanced Functional Programming in Industry
67/84
Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB harddrive
Single server hosts both the web and database servers
Can easily handle peaks of (at least) 700 visitors at a time
Chordifying new songs takes some computing power, but most songsare in the database already
Queueing system for busy periods
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 37 / 46
How do we handle these visitors?
-
8/10/2019 Advanced Functional Programming in Industry
68/84
Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB harddrive
Single server hosts both the web and database servers
Can easily handle peaks of (at least) 700 visitors at a time
Chordifying new songs takes some computing power, but most songsare in the database already
Queueing system for busy periods
Infrastructure costs are minimal
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 37 / 46
Frontend (PHP/JS) and backend (Haskell) interaction
-
8/10/2019 Advanced Functional Programming in Industry
69/84
Frontend receives a music file, calls backend with it
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 38 / 46
Frontend (PHP/JS) and backend (Haskell) interaction
-
8/10/2019 Advanced Functional Programming in Industry
70/84
Frontend receives a music file, calls backend with it
Backend computes the chords, writes them to a file:
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 38 / 46
Frontend (PHP/JS) and backend (Haskell) interaction
-
8/10/2019 Advanced Functional Programming in Industry
71/84
Frontend receives a music file, calls backend with it
Backend computes the chords, writes them to a file: 1;D:min;0.232199546;0.615328798
2;D:min;0.615328798;0.998458049
...
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 38 / 46
Frontend (PHP/JS) and backend (Haskell) interaction
-
8/10/2019 Advanced Functional Programming in Industry
72/84
Frontend receives a music file, calls backend with it
Backend computes the chords, writes them to a file: 1;D:min;0.232199546;0.615328798
2;D:min;0.615328798;0.998458049
...
Frontend reads this file, updates the database if necessary, andrenders the result
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 38 / 46
Frontend (PHP/JS) and backend (Haskell) interaction
-
8/10/2019 Advanced Functional Programming in Industry
73/84
Frontend receives a music file, calls backend with it
Backend computes the chords, writes them to a file: 1;D:min;0.232199546;0.615328798
2;D:min;0.615328798;0.998458049
...
Frontend reads this file, updates the database if necessary, andrenders the result
Backend is open-source (and GPL3); only option is to run it as astandalone executable
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 38 / 46
The importance of the UI
-
8/10/2019 Advanced Functional Programming in Industry
74/84
Lets have a look at four different online services giving you the chordsfor a song (Radioheads Karma Police).
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 39 / 46
The importance of the UIChordie
-
8/10/2019 Advanced Functional Programming in Industry
75/84
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 40 / 46
The importance of the UIRiffstation
-
8/10/2019 Advanced Functional Programming in Industry
76/84
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 41 / 46
The importance of the UIYoutab.me
-
8/10/2019 Advanced Functional Programming in Industry
77/84
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 42 / 46
The importance of the UIChordify
-
8/10/2019 Advanced Functional Programming in Industry
78/84
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 43 / 46
Logistics of an internet start-up
-
8/10/2019 Advanced Functional Programming in Industry
79/84
Chordify is created and funded by 5 people
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 44 / 46
Logistics of an internet start-up
-
8/10/2019 Advanced Functional Programming in Industry
80/84
Chordify is created and funded by 5 people
If you can do without venture capital, do it!
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 44 / 46
Logistics of an internet start-up
-
8/10/2019 Advanced Functional Programming in Industry
81/84
Chordify is created and funded by 5 people
If you can do without venture capital, do it!
You might end up doing more than just functional programming,though: Deciding on what features to implement next Recruiting, interviewing, dealing with legal issues related to
employment Taxation (complicated by the fact that we sell worldwide and support
multiple currencies) User support Outreach (pitching events, media, this talk, etc.)
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 44 / 46
Logistics of an internet start-up
-
8/10/2019 Advanced Functional Programming in Industry
82/84
Chordify is created and funded by 5 people
If you can do without venture capital, do it!
You might end up doing more than just functional programming,though: Deciding on what features to implement next Recruiting, interviewing, dealing with legal issues related to
employment Taxation (complicated by the fact that we sell worldwide and support
multiple currencies) User support Outreach (pitching events, media, this talk, etc.)
But its fun, and you learn a lot!
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 44 / 46
Summary
-
8/10/2019 Advanced Functional Programming in Industry
83/84
Musical modelling with Haskell: A model for musical harmony as a Haskell datatype
Makes use of several advanced functional programming techniques,such as generic programming, GADTs, and type families
When chords do not fit the model: error correction
Harmonising melodies
Generating harmonies
Recognising harmony from audio sources
Transporting academic research into industry
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 45 / 46
Play with it!
-
8/10/2019 Advanced Functional Programming in Industry
84/84
http://chordify.net
http://hackage.haskell.org/package/HarmTrace
http://hackage.haskell.org/package/FComp
Jose Pedro Magalhaes Advanced Functional Programming in Industry, FP Days 2014 46 / 46
http://chordify.net/http://hackage.haskell.org/package/HarmTracehttp://hackage.haskell.org/package/FComphttp://hackage.haskell.org/package/FComphttp://hackage.haskell.org/package/HarmTracehttp://chordify.net/