automatic design of sound synthesizers as pure data patches using coevolutionary mixed-typed...
TRANSCRIPT
![Page 1: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/1.jpg)
E N G A G I N G T H E W O R L D
Philippe Pasquier [email protected]
1
Matthieu Macret [email protected]
Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming
Wednesday, July 16, 2014
![Page 2: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/2.jpg)
The Synthesis Calibration Problem
2
Target sound
Sound Synthesizer
i1
i2
...
in
=Synthesized sound
Input parameters
Wednesday, July 16, 2014
![Page 3: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/3.jpg)
Automatic calibration of Sound synthesizers
3
Synthesis technique Op/miza/on technique Reference
Addi)ve HMM Heise (AES Conven)on 2009)
Substra)ve Par)cle Swarm Yoshimura (Speech Communica)on and Tech. 1999)
Substra)ve Neural Network Roth (AES Conven)on 2011)
FM Cellular Automata Serquera (Applica)ons of Evolu)onary Computa)on 2010)
FM GA Horner (Computer Music Journal 1993)
Addi)ve GA Horner (Computer Music Journal 1996)
Granular GA Fujinaga (ICMC 1994)
Wednesday, July 16, 2014
![Page 4: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/4.jpg)
ModFM synthesizer (Macret SMC 2012)
4
Wednesday, July 16, 2014
![Page 5: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/5.jpg)
ModFM synthesizer (Macret SMC 2012)
4
Target
Wednesday, July 16, 2014
![Page 6: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/6.jpg)
ModFM synthesizer (Macret SMC 2012)
4
Target
Wednesday, July 16, 2014
![Page 7: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/7.jpg)
ModFM synthesizer (Macret SMC 2012)
4
Wednesday, July 16, 2014
![Page 8: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/8.jpg)
ModFM synthesizer (Macret SMC 2012)
4
Synthesized
Wednesday, July 16, 2014
![Page 9: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/9.jpg)
ModFM synthesizer (Macret SMC 2012)
4
Synthesized
Wednesday, July 16, 2014
![Page 10: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/10.jpg)
More complex synthesizer• VST Instruments with GA (Yee-King ICMC 2011)
5
Wednesday, July 16, 2014
![Page 11: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/11.jpg)
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Wednesday, July 16, 2014
![Page 12: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/12.jpg)
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Target
Wednesday, July 16, 2014
![Page 13: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/13.jpg)
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Target
Wednesday, July 16, 2014
![Page 14: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/14.jpg)
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Wednesday, July 16, 2014
![Page 15: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/15.jpg)
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Synthesized
Wednesday, July 16, 2014
![Page 16: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/16.jpg)
OP-1 Synthesizer (Macret SMC 2013)
6
Mul)-‐objec)ve Gene)c Algorithm
Synthesized
Wednesday, July 16, 2014
![Page 17: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/17.jpg)
General Sound Synthesis Problem
7
Target sound
Sound Synthesizer
i1
i2
...
in
=Synthesized sound
Input parameters
Wednesday, July 16, 2014
![Page 18: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/18.jpg)
Genetic Programming - Garcia (DAFx 2001)
8
Produc)on rulesin had hoc synthesis system
Sound Synthesizer
i1
i2
...
in
=
Synthesized sound
Target sound
GPGA
Wednesday, July 16, 2014
![Page 19: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/19.jpg)
Coevolutionary Mixed-typed Cartesian GP
• Coevolution of:–Mixed-typed Cartesian Genetic Programming•Evolve synthesizer architecture
–Genetic Algorithm•Optimize Input Parameters
9
Wednesday, July 16, 2014
![Page 20: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/20.jpg)
Pure Data• Graphical programming environment for audio, video, and image processing• http://puredata.info/• Popular among musician, artist and sound designers
10
Wednesday, July 16, 2014
![Page 21: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/21.jpg)
11
Wednesday, July 16, 2014
![Page 22: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/22.jpg)
11
Wednesday, July 16, 2014
![Page 23: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/23.jpg)
11
Wednesday, July 16, 2014
![Page 24: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/24.jpg)
Mixed-typed Cartesian GP
12
• Harding (GECCO 2012)
• Characteristics:–Directed Acyclic Graph–Mixed-typed
Wednesday, July 16, 2014
![Page 25: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/25.jpg)
13
0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1
CGP graph
Pd Patch
Wednesday, July 16, 2014
![Page 26: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/26.jpg)
14
Gene Input 1 Input 2 Extra Param.
Descrip/on
Output Type
3 control control -‐ Division control
4 signal control -‐ Oscillator signal
Wednesday, July 16, 2014
![Page 27: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/27.jpg)
15
0 0 0 5 1 4 5 9 2 10 3 0 2 9 7 3 5 6 0 7 9 3 5 4 3 2 0 1
CGP graph
Pd Patch
Wednesday, July 16, 2014
![Page 28: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/28.jpg)
Fitness function• MFCC (Mel-Frequency Cepstral Coefficients) in windows of 23 ms
16
Euclidean distance between synthesized sound and target sound
Wednesday, July 16, 2014
![Page 29: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/29.jpg)
Fitness function• MFCC (Mel-Frequency Cepstral Coefficients) in windows of 23 ms
16
Euclidean distance between synthesized sound and target sound
Wednesday, July 16, 2014
![Page 30: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/30.jpg)
Evaluation
17
i1
i2
...
in
i1
i2
...
ini1
i2
...
in
i1
i2
...
in
GA popula)on (150 individuals) CGP Popula)on (5 individuals)
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Pd PatchPd PatchPd Patch
Wednesday, July 16, 2014
![Page 31: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/31.jpg)
Genetic Algorithm
18
• Binary encoding of chromosome• Binary tournament selection• 2-points crossover• Bitflip mutation
Wednesday, July 16, 2014
![Page 32: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/32.jpg)
Mixed-typed Cartesian GP• Integer encoding of chromosome• Evolutionary Strategy 1+4• Mutation-only approach
19
Wednesday, July 16, 2014
![Page 33: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/33.jpg)
20
Initialize CGP /GA population
Evaluate CGP / GA population
Variate GA population
Select best CGP individual
Variate CGP population
Termination criteria ?End
No
Yes
Wednesday, July 16, 2014
![Page 34: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/34.jpg)
Clarinet sound
21
Wednesday, July 16, 2014
![Page 35: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/35.jpg)
Clarinet sound
21
Wednesday, July 16, 2014
![Page 36: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/36.jpg)
Clarinet sound
21
Wednesday, July 16, 2014
![Page 37: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/37.jpg)
Clarinet sound
21
Target
Wednesday, July 16, 2014
![Page 38: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/38.jpg)
Clarinet sound
21
Target
Wednesday, July 16, 2014
![Page 39: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/39.jpg)
Clarinet sound
21
Wednesday, July 16, 2014
![Page 40: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/40.jpg)
Clarinet sound
21
Synthesized
Wednesday, July 16, 2014
![Page 41: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/41.jpg)
Clarinet sound
21
Synthesized
Wednesday, July 16, 2014
![Page 42: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/42.jpg)
Clarinet sound
22
Wednesday, July 16, 2014
![Page 43: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/43.jpg)
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
![Page 44: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/44.jpg)
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
![Page 45: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/45.jpg)
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
![Page 46: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/46.jpg)
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
![Page 47: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/47.jpg)
23
Pd patch used to generate the target sound Evolved Pd Patch
Contrived sound
Wednesday, July 16, 2014
![Page 48: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/48.jpg)
Stats
24
Wednesday, July 16, 2014
![Page 49: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/49.jpg)
Conclusions• New approach to automate the design of sound synthesizers using Coevolutionary Mixed-typed Cartesian Programming• Synthesizers represented as reusable Pd patches
25
Wednesday, July 16, 2014
![Page 50: Automatic design of sound synthesizers as Pure Data patches using Coevolutionary Mixed-typed Cartesian Genetic Programming](https://reader030.vdocuments.mx/reader030/viewer/2022020411/5a65e6ab7f8b9ab21e8b49b7/html5/thumbnails/50.jpg)
Future works• Our methods involves many parameters– Parameter sensitivity analysis
• Extend our system to reproduce not only one target sound but a set of target sounds• Improve our fitness function– Move to a multi-objective fitness function
• Quantitative/Qualitative comparison with previous similar systems
26
Wednesday, July 16, 2014