artificial evolution for sound synthesis

THE UNIVERSITY OF NEW SOUTH

WALES

SCHOOL OFELECTRICAL ENGINEERING AND

COMPUTERSCIENCE AND ENGINEERING

Arti�cial Evolution

for Sound Synthesis

Jonathon Crane (2172214)

Bachelor of Engineering (Computer Engineering)

July 5, 1996

Supervisor:Andrew TaylorAssessor:Tim Lambert

Abstract

A technique ofArtificial Evolution is applied toSound Synthesis algorithms withthe goal of producing an effective, intuitive way of searching the space of possiblesounds.

Interactive Evolutionary Algorithms (IEA’s) are optimisation techniques inspiredby the process of biological evolution. In this study, an IEA is applied to a FrequencyModulation (FM) synthesis algorithm. This produces a tool which allows users toexplore, sculpt and evolve sounds without any knowledge or understanding of theunderlying algorithm.

To determine the effectiveness of the system, seven different users compared threedifferent exploration techniques: manually adjusting parameters of the algorithm; ran-domly adjusting parameters; and using the IEA to adjust the parameters.

It was discovered that the random method performed best, but did not offer thedegree of control required by users. The manual method provided this fine grainedcontrol, but was slow and difficult to use - it required the users to understand theunderlying FM algorithm. In contrast, the IEA method provided all the advantagesof the random method, combined with the control of the manual method. It enabledusers to rapidly locate good sounds and then refine them as necessary, even with noknowledge or understanding of the FM synthesis algorithm.

Contents

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Objectives . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Outline . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Previous Work 52.1 Introduction . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Sound Synthesis . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2.1 Describing Sound .. . . . . . . . . . . . . . . . . . . . . . . 52.2.2 Synthesis Algorithms . . . . . .. . . . . . . . . . . . . . . . 72.2.3 Additive Synthesis. . . . . . . . . . . . . . . . . . . . . . . 82.2.4 Subtractive Synthesis . . . . . .. . . . . . . . . . . . . . . . 92.2.5 Frequency Modulation . . . . .. . . . . . . . . . . . . . . . 102.2.6 Sampling .. . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2.7 Other Methods . .. . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Evolutionary Algorithms (EA’s) . . . .. . . . . . . . . . . . . . . . 132.3.1 Genetic Algorithms (GA’s) . . .. . . . . . . . . . . . . . . . 142.3.2 Evolution Strategies (ES’s) . . .. . . . . . . . . . . . . . . . 152.3.3 Evolutionary Programming (EP). . . . . . . . . . . . . . . . 17

2.4 EA’s for Sound Synthesis .. . . . . . . . . . . . . . . . . . . . . . . 172.5 Interactive Evolutionary Algorithms (IEA) . . . . . .. . . . . . . . . 18

2.5.1 Dawkins’ Biomorphs . . . . . .. . . . . . . . . . . . . . . . 192.5.2 Oppenheimer’s Artificial Menagerie . . . . .. . . . . . . . . 212.5.3 Smith’s Bugs . . .. . . . . . . . . . . . . . . . . . . . . . . 212.5.4 Sims’ Artificial Evolution . . .. . . . . . . . . . . . . . . . 232.5.5 Moore’s GAMusic 1.0 . . . . .. . . . . . . . . . . . . . . . 242.5.6 van Goch’s P-Farm. . . . . . . . . . . . . . . . . . . . . . . 25

2.6 Summary . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3 System Design 283.1 Introduction . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 283.2 System Outline . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 28

ii

CONTENTS iii

3.3 Considerations . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 293.3.1 Sound Synthesis .. . . . . . . . . . . . . . . . . . . . . . . 293.3.2 Evolutionary Algorithm . . . .. . . . . . . . . . . . . . . . 313.3.3 User Interface . . .. . . . . . . . . . . . . . . . . . . . . . . 32

3.4 The Design Decision . . .. . . . . . . . . . . . . . . . . . . . . . . 333.5 Evaluation . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4 System Development 364.1 Introduction . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2 EvoS 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.2.2 Implementation . .. . . . . . . . . . . . . . . . . . . . . . . 384.2.3 Results . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 404.2.4 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.3 EvoS 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.3.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.3.2 Implementation . .. . . . . . . . . . . . . . . . . . . . . . . 424.3.3 Results . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 434.3.4 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . 45




4.7 EvoS 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614.7.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614.7.2 Implementation . .. . . . . . . . . . . . . . . . . . . . . . . 614.7.3 Results . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 624.7.4 User evaluation . .. . . . . . . . . . . . . . . . . . . . . . . 664.7.5 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . 68

CONTENTS iv

5 Results 705.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705.2 Implementation . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.2.1 Manual Tool . . .. . . . . . . . . . . . . . . . . . . . . . . 715.2.2 Random Tool . . .. . . . . . . . . . . . . . . . . . . . . . . 735.2.3 Evolution Tool . .. . . . . . . . . . . . . . . . . . . . . . . 74

5.3 Method . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 775.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5.4.1 Raw Data .. . . . . . . . . . . . . . . . . . . . . . . . . . . 795.4.2 Analysis of Data .. . . . . . . . . . . . . . . . . . . . . . . 795.4.3 An Alternative Analysis . . . .. . . . . . . . . . . . . . . . 825.4.4 User Opinions . .. . . . . . . . . . . . . . . . . . . . . . . 84

5.5 Conclusions . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6 Conclusion 896.1 Conclusion . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 896.2 Other lessons learnt . . . . . . . . . . . . . . . . . . . . . . . . . . . 906.3 Further Work . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 906.4 A final note . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 92

A Evolving Samples 93A.1 Introduction . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 93

A.1.1 Implementation . .. . . . . . . . . . . . . . . . . . . . . . . 94A.1.2 Results . .. . . . . . . . . . . . . . . . . . . . . . . . . . . 97A.1.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

List of Figures

1.1 This Moog synthesizer illustrates the complexity of synthesis algo-rithms. Each knob controls a separate parameter that can be adjustedby the user to effect the resulting sound - taken from [Mus98]. . . . . 2

2.1 Patch required for additive synthesis - taken from [Wil88]. . . . . . . 82.2 A basic subtractive synthesis patch - taken from [Wil88]. . . . . . . . 92.3 The most basic setup required for FM synthesis - taken from [DJ85]. . 102.4 A more complicated FM patch - taken from [DJ85]. .. . . . . . . . . 112.5 An illustration of 5 point crossover - taken from [B¨ac95]. . . . . . . . 152.6 Evolution of “biomorphs” with Dawkins’ system - taken from [Daw86]. 202.7 Example of an evolved plant form - taken from [Ope88]. . . . . . . . 212.8 Biomorphs evolved with Smith’s system - taken from [Smi91]. . . . . 222.9 An image generated from an evolved symbolic lisp expression - taken

from [Sim91]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.10 A screen shot of the GAMusic 1.0 user interface. . .. . . . . . . . . 252.11 A screen shot of P-Farm’s user interface.. . . . . . . . . . . . . . . . 26

3.1 The three main components of the proposed system .. . . . . . . . . 29

4.1 A screen shot of the EvoS2 user interface. . . . . . .. . . . . . . . . 394.2 A sample evolution run for EvoS2. . . .. . . . . . . . . . . . . . . . 414.3 Evolution of synthesis parameters in EvoS 3. . . . .. . . . . . . . . 444.4 Evolution of variances in EvoS 3. . . . .. . . . . . . . . . . . . . . . 444.5 An envelope (top) controls the amplitude of a waveform (bottom). . . 464.6 The envelope of figure 4.5 shown with an envelope produced by mu-

tation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.7 The envelope of 4.5 compared with an envelope produced by random-

ization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.8 EvoS 4 evolution with a variance of1

50. . . . . . . . . . . . . . . . . 51

4.9 EvoS 4 evolution with a variance of110

. . . . . . . . . . . . . . . . . 524.10 EvoS 4 evolution with a variance of1

20. . . . . . . . . . . . . . . . . 52

4.11 EvoS 5 evolution with a variance of 1/50. . . . . . .. . . . . . . . . 554.12 EvoS 5 evolution with a variance of 1/10. . . . . . .. . . . . . . . . 564.13 EvoS 5 evolution with a variance of 1/20. . . . . . .. . . . . . . . . 57

v

LIST OF FIGURES vi

4.14 EvoS 7 evolution: Modulation indices (I1 andI2). . . . . . . . . . . . 634.15 EvoS 7 evolution: Frequency ratios (N1 andN2). . . . . . . . . . . . 644.16 EvoS 7 evolution: Amplitude Envelopes.. . . . . . . . . . . . . . . . 654.17 EvoS 7 evolution: Modulation Index Envelopes. . . .. . . . . . . . . 65

5.1 Screen shot of the manual tool user interface. . . . .. . . . . . . . . 725.2 Screen shot of the random tool user interface . . . .. . . . . . . . . 735.3 Screen shot of the evolution tool . . . .. . . . . . . . . . . . . . . . 755.4 Sounds Likedplotted againstTime for all search tools. . . . . . . . . 825.5 Sounds Likedvs. Sounds Auditionedfor all search tools. . . . . . . 835.6 Sounds Likedplotted against search tool . . . . . .. . . . . . . . . 84

A.1 Example of a heterodyne analysis file, the amplitude envelopes . . . . 94A.2 Example of a heterodyne analysis file, the frequency envelopes . . . . 95A.3 Example of a nicely mutated amplitude envelope . .. . . . . . . . . 96A.4 Example of a nicely mutated frequency envelope . .. . . . . . . . . 96A.5 Example of a badly mutated amplitude envelope . . .. . . . . . . . . 97

List of Tables

2.1 Papers published by Andrew Horner et. al. on the subject of applyingGA’s to sound synthesis . .. . . . . . . . . . . . . . . . . . . . . . . 18

4.1 Ranges and variances for the parameters of EvoS 2. .. . . . . . . . . 404.2 Variances and ranges for thevariances of EvoS 3 . . . . . . . . . . . 434.3 Effects of different variances in EvoS 4.. . . . . . . . . . . . . . . . 534.4 Effects of different variances in EvoS 5. . . . . . . . . . . . . . . . 58

5.1 Raw statistics collected for the manual tool . . . . .. . . . . . . . . 795.2 Raw statistics collected for the random tool . . . . .. . . . . . . . . 795.3 Raw statistics collected for the evolution tool . . . .. . . . . . . . . 805.4 Mean statistics for each search tool. . .. . . . . . . . . . . . . . . . 80

vii

Acknowledgements

I would firstly like to thank Emma Hogan, who supported me throughout this thesis -thank you for all the meals you cooked me.

Next, I would like to thank my family: my mother, Colleen, who has supportedme financially these last four years; my brother, Andrew, who got me out of somedodgy situations; and my aunt, Nathalie, who also cooked a lot of meals for me.

In relation to this project I would like to thank all those users who volunteeredto test the system: Emma Hogan (again); Brett Webb; Nick Perkins; Dougal May;Rachael James; Tom Fryer; Chris V.; Nick Mariette; and Brendan Hanna (I’m sorryabout the outlier stuff). I would also like to thank Tom and Will Edwards for lettingme use their scanner.

Academically I would like to thank my supervisor, Andrew Taylor, especially forapproving my project which was a bit out of the ordinary. And also my assessor, TimLambert, for dealing with my frequent anxiety attacks ;). Similarly I would like tothank those people who answered my emails with querys about this project: JasonMoore; Arno van Goch; and Waleed Kadous.

Finally, I would like to thank anyone else that I may have left out - I apologise forthis and hereby give you permission to run a guilt trip on me.

In memory of my father

(“The journey of a thousand miles starts with the first step.”)

ix

Chapter 1

Introduction

In theory at least, any sound can be synthesized electronically...

The synthesizer is a wonderfully versatile musical instrument. It cansound like dozens, hundreds, or even thousands of instruments... Manyother sounds are only possible through sound synthesis.

—Delton T. Horn,Music Synthesizers

Indeed, sound synthesis can produce any sound you desire, but the above quotedoes not mention the difficulty involved in obtaining such sounds.

This thesis is concerned with using a process ofartificial evolution1 to producenovel and interesting sound spectra. More specifically, an Interactive EvolutionaryAlgorithm (IEA) is used here to generate parameter sets for sound synthesis algo-rithms.

The results will show thatartificial evolution provides an efficient and intuitiveway for people to search through the space of possible sounds.

1.1 Motivation

Why should anyone bother with such an exercise? — There are a number of reasons.

Synthesis algorithms typically involve a large number of parameters (see figure1.1). This often makes it difficult to obtain a desired sound quickly. Users of thesynthesis algorithms either have to understand exactly how each parameter will ef-fect the final sound, or they have to spend a lot of time aimlessly “twiddling knobs”

1An optimisation procedure modeled on biological evolution.

1

1. Introduction 2

until a satisfactory sound is obtained. What is needed is an efficient, intuitive andenjoyable way to search through the multi-dimensional space created by the synthesisparameters.

Figure 1.1: This Moog synthesizer illustrates the complexity of synthesis algorithms.Each knob controls a separate parameter that can be adjusted by the user to effect theresulting sound - taken from [Mus98].

Primarily this could have application as a creative tool for musicians, composersand electronic artists. Instead of twiddling knobs all day they could very quicklyobtain novel and interesting sounds for use in their compositions. Indeed, the processof evolving a sound could even form the basis of a composition.

Secondly, it would open the world of sound synthesis to many people who don’thave the time or patience to understand the algorithms. The only expertise you needis your opinion of ‘what sounds good’ and ‘what sounds bad’.

Finally, another application may lie with researchers experimenting with newkinds of synthesis algorithms. The system would provide an easy way to quicklytest the boundaries of any new algorithm they discover.

1. Introduction 3

1.2 Goals

The goals of this project were:

� To investigate the use of an Interactive Evolutionary Algorithm applied to soundsynthesis.

� To determine whether this provides a useful tool for people interested in explor-ing the space of possible sounds.

1.3 Objectives

The objectives accomplished in the process of achieving the above goals are sum-marised below:

� To conduct a review the previous research done in this field. This will serve asa background for an informed investigation.

� To assess the possible avenues of implementation for the idea and decide on themost feasible.

� To execute this most feasible avenue of implementation - resulting in a systemthat embodies the goals of this project.

� To collect data (by conducting appropriate experiments) in order to asses whetherthe system is effective or not.

� To analyse the collected data in a suitable manner and so asses the feasibility ofthis idea for the applications suggested.

� To document the findings of this investigation in a concise and legible manner- accessible to people in related fields of research, or anyone interested in theresults.

1.4 Outline

So far we have defined the goals and objectives of this thesis, the rest of the reportwill be structured as follows:

1. Introduction 4

Chapter 2

Following the standard format of an undergraduate thesis, we begin with a reviewrelevant literature. This will provide a picture of what has already been achieved inthis area of research and illustrate the originality of this project.

Chapter 3

We then consider the possibilities for implementation of the project. The pros andcons of each design issue are discussed and the final implementation decision is de-scribed and justified. The problem of measuring the project’s success is also dis-cussed.

Chapter 4

Next, we describe how the software for this project was developed. This is presentedin the form of a series of experiments, each with its own aim and conclusion. Thefinal experiment in this series combines all the previous discoveries into a completesystem.

Chapter 5

This chapter sees the complete system fashioned into an experiment that comparesthree different ways of searching the same sound space. The experiment is describedand the results collected are reported and analysed.

Chapter 6

Finally, this chapter will summarise the findings of this investigation and suggest av-enues of future research which could expand on the work presented here.

So without further ado, lets get down to business2... I hope you enjoy it!

2A note on grammar: throughout this document there is occasional use of language traditionallyregarded as informal. For example, the use of dashes ‘-’ to add afterthoughts, and the word ‘But’ atthe beginning of a sentence. It is the author’s opinion that in some circumstances the use of informallanguage helps to communicate ideas, and this is the justification for its use in this document. Informallanguage also serves the purpose of adding the occasional ‘personal touch’ which can promote thereaders interest in an otherwise bland report.

Chapter 2

Previous Work

2.1 Introduction

This chapter gives a summary of the background research that was conducted forthis project. First a general introduction to the field of Evolutionary Algorithms isgiven. This is followed by a review of the research done in applying these techniquesto sound synthesis. Next, applications of Interactive Evolutionary Algorithms arereviewed, and finally a short summary is given. This will demonstrate the validity andoriginality of this project in the context of the research described.

2.2 Sound Synthesis

The process of synthesizing sound has a long and detailed history that is beyond thescope of this document. Instead, this section aims to give a very brief introduction,focusing on aspects relevant to this thesis.

2.2.1 Describing Sound

Before methods of synthesizing sound are discussed, it would be good to discuss somecharacteristics of sound itself. The basic properties of a (musical) sound are its pitch,amplitude, duration and timbre:

5

2. Previous Work 6

Pitch

Pitch is our perception of the frequency of a sound. Frequencies are usually measuredin Hertz (Hz) which is a measure of the number of cycles per second in the sound.Most sounds have many frequencies present at the same time. If these frequencies arerelated in aharmonic series then the pitch we hear for the complete sound is that ofthe lowest frequency or thefundamental.

People are very used to the idea of controlling the pitch of a sound - take forexample the piano. Each key on the keyboard generates a note of a different pitch.You can change the pitch of the sound by pressing different keys on the keyboard.

Amplitude

Amplitude (or volume) is our perception of how loud a sound is. Amplitude is com-monly measured in decibels (dB).

Using the example of the piano again, you control the amplitude of the sound bystriking keys with different speeds. The faster (or harder) you hit a key, the louder theresulting sound.

Duration

Duration is the time a sound lasts for. It is usually measured in seconds.

On the piano the duration of the sound is controlled by the amount of time a keyis held for. The longer you hold a key down, the longer the duration of the sound.

Timbre

This is much harder to describe. Timbre is “the characteristic tone quality of a partic-ular class of sounds”1. You describe timbre of a sound when you say things like “thatsounds like a trumpet” or “that sounds very metallic”. The timbre of all brass instru-ments is quite similar - a trumpet sounds like a trombone, yet the timbre of brass andwind instruments are quite different - a trumpet does not sound at all like a flute. Al-though timbre cannot be measured on any scale, two important aspects that contributeto the timbre of a sound are its spectrum and its amplitude envelope.

The amplitude envelope of a sound is the way the volume of a sound varies over itsduration. Sounds whose envelopes have a sharpattack segment will be heard as per-

1From page 48 of [DJ85].

2. Previous Work 7

cussive. Those with very long attack segments will sound like they are being playedbackwards.

The spectrum of a sound is determined by Fourier analysis which describes it asa sum of simple sinusoids. Changing the spectrum of a sound can radically alterits characteristics. A sound with a simple or narrow spectrum will sound “thin” and“pure” - like a sine wave. A sound with a wide spectrum will sound “rich” and “buzzy”- like a square wave.

In our example of the piano, there is no way to control the timbre of the sound2. Itwas not until the advent of electronic musical instruments that very radical changes intimbre could be effected. Today, a synthesizer can provide many controls that effectthe timbre of a sound in different ways.

2.2.2 Synthesis Algorithms

Research in sound synthesis is usually focussed with musical applications in mind.As a result, a lot of work has been done in finding ways to simulate the sounds of var-ious acoustic instruments. Often along the way, many completely new and unnaturalsounds are discovered. These (usually very large) portions of the unnatural timbralspace can only be accessed via sound synthesis algorithms.

Synthesis algorithms can be implemented with analog electronics or digital com-puters. Early analog sound synthesisers were large machines3, laden with knobs4 andstrewn with patch cables which generated all manner of weird and wonderful sounds.To create a sound, one manually “patched” the output of an oscillator, for example, tothe input of a filter or any other sound processing module which was available. Anysuch configuration of modules was called apatch5.

The advantage of synthesis in this manner was that each parameter (e.g. oscilla-tor frequency, filter cut-off frequency, etc.), had an associated knob which could be“tweaked” by the user. This real-time tweaking of knobs, combined with the abilityto patch the output of any module to the input of another afforded an intuitive andenjoyable way to explore the space of possible sounds.

The disadvantages of modular analog synthesis are its high cost and the sheerphysical size of the machines. If these are concerns, you should probably implementyour synthesis algorithm on a digital computer.

2You could make very minor alterations to the timbre by doing things like opening the lid of thepiano - this would cause a slight difference in tone quality.

3Indeed, decent modern analog synthsisers are still very large.4Look back at figure 1.1.5A term still in use today and throughout this document.

2. Previous Work 8

The advantage of synthesising sound on computers is thatany analog configura-tion can be simulated, but the cost and size of the machine is much smaller. Howeveryou lose the intuitive control (and usually the real-time response) of an analog system.

The next few sections describe some popular sound synthesis algorithms as ex-plained in [Pre92], [DJ85] and [Wil88]. Although these are usually implemented ondigital computers, in order to understand them it is sometimes easier to picture themin an analog implementation.

2.2.3 Additive Synthesis

This is the most direct way to form any kind of sound spectra you desire. For eachsinusoidal component (harmonic)6 of the desired spectra you use a separate sinusoidaloscillator to produce it. The outputs of all the oscillators are then simply added to-gether.

In most natural sounds, the volume and pitch of each sinusoidal component varieswith time. To mimic this, two envelopes are used to control each oscillator. Oneenvelope controls the amplitude and the other varies the pitch. This results in thepatch illustrated in figure 2.1. Here, you can see each oscillator being controlled by avolume envelope generator (VEG) and a pitch envelope generator (PEG).

Figure 2.1: Patch required for additive synthesis - taken from [Wil88].

6The terms ‘sinusoidal component’ and ‘harmonic’ are usually interchangeable.

2. Previous Work 9

Using this technique, researchers have been able to very accurately synthesizethe sounds of many instruments. All they have to do is analyse a given instrumenttone (via Fourier decomposition) to determine the shape of the envelopes for eachharmonic. These envelopes are then used to control the volume of oscillators setat the correct frequencies. The resulting sound is almost indistinguishable from theoriginal instrument tone.

There are a few disadvantages of the additive synthesis technique however. Firstly,because a separate oscillator and envelope generator are required for each harmonic(typically there are more than ten harmonics) it is computationally expensive. If afast response is required between specification of the parameters and synthesis ofthe sound, you will need either a very powerful computer or specialised hardware.Secondly, because this technique involves specification of so many parameters, it canbe hard for a musician to achieve a desired sound.

2.2.4 Subtractive Synthesis

In contrast to additive synthesis, subtractive synthesis starts with a dense spectrumand carves away selected portions to produce the desired sound. Instead of sine waveoscillators, square and sawtooth wave oscillators are used and combined with noisegenerators. The dense spectrum that results is passed to various combinations of highpass, low pass or band pass filters. To create sounds that change their character (tim-bre) over time, envelope generators and other oscillators are used to control the cut-offfrequencies of the filters. Figure 2.2 illustrates the concept of subtractive synthesis.Here, the filter is controlled by an envelope generator (EG) and a low frequency os-cillator (LFO).

Figure 2.2: A basic subtractive synthesis patch - taken from [Wil88].

Subtractive synthesis is useful for imitating instruments with harmonic spectra

2. Previous Work 10

such as wind and string instruments.Inharmonic spectra (such as the sounds of bellsand drums) can be produced by combining other oscillators or using other devicessuch asring modulators. The range of sounds producible with this technique dependson the way in which you interconnect modules rather than the numbers you throw atan algorithm.

2.2.5 Frequency Modulation

Frequency Modulation (FM) is a very popular synthesis method that was first defini-tively described by John Chowning [Cho73].

FM synthesis involves using one oscillator (the modulator) to control the fre-quency of another (the carrier). The simplest FM setup is depicted in figure 2.3.

Figure 2.3: The most basic setup required for FM synthesis - taken from [DJ85].

In order to understand the effect of FM, imagine that the frequency of the modu-lating oscillator (fm) is very low (1 to 10Hz). The output of the carrier will be a tonethat wavers up and down in pitch. The pitch will vary between(fc � d) and(fc + d),wherefc is the frequency of the carrier andd is the amplitude of the modulator. Musi-cians call this wavering pitch effect “vibrato”. If the amplitude (d) of the modulatingoscillator is increased, the tone will waver up and down more wildly - like a police

2. Previous Work 11

siren. Thus, increasing (d) increases the depth of the vibrato. If thefrequency of themodulating oscillator (fm) is now increased, the tone will waver up and down faster.As the modulating frequency goes into the audio range (above 20Hz), the output ofthe carrier is no longer heard as a vibrato, but obtains a distinct timbre of its own.Many different timbres can be obtained by varying the frequency and amplitude ofthe modulating oscillator.

Remember this is only the simplest FM patch, a more complex one is shown infigure 2.4. Here, the constantd is replaced by an envelope generator. The envelopedynamically changes the amplitude of the modulating oscillator over time. This re-sults in a time varying spectrum at the output - a very interesting sound that changesits character over time. Notice also that the amplitude of the carrier is shaped by an en-velope. This envelope dynamically controls the volume of the time-varying spectrum.

Figure 2.4: A more complicated FM patch - taken from [DJ85].

Recall that two important contributors to the timbre of a sound were its spectrumand amplitude envelope7. This simple FM algorithm, has achieved control of bothof these! Compare this to the additive synthesis approach which required tens ofoscillators. In the late 60’s and 70’s, the prospects of FM synthesis seemed so goodthat Yamaha purchased the rights to it8.

7See section 2.2.1.8See [Wil88] page 52.

2. Previous Work 12

However, the disadvantage of FM is that it is hard to control. For a given set ofparameters, it is difficult for a human to predict how the result will sound.

2.2.6 Sampling

Sampling synthesis has become a lot more popular in recent years due to the fallingprice of digital electronics.

Sampling is the process of converting an analog audio source into digital form(via an ADC9) and storing this information. The digital audio is stored as a string ofbits and can be played back at any time via a DAC10. It can also be played back atdifferent rates to simulate different pitches. For example to synthesize a piano, youcould sample just one note of a real piano. The rest of the notes can be synthesized byplaying the sampled note back at different rates.

Sampling has proved very successful for synthesising all manner of instrumentsand usually provides a much more realistic imitation than any other synthesis method.There is a price to pay for this however - it is very data intensive. To accuratelysynthesize an instrument you have to sample it a number of times at a very high rate.This can lead to huge amounts of data which has to be stored and processed. Comparethis with a sophisticated FM algorithm with which you only need to store about 20parameters.

2.2.7 Other Methods

Other methods of sound synthesis includewaveshaping, discrete summation synthe-sis, group synthesis and many others. Receiving particular attention in recent timesis granular synthesis which creates sound by combining together tiny sound “grains”.However, we cannot possibly hope to cover all these methods - and besides, they arenot really relevant to this thesis11.

And now for something completely different... we move to a discussion of Evolu-tionary Algorithms. The material that follows is based largely on the excellent frame-work presented by B¨ack [Bac95].

9Analog to Digital Converter, pronounced “aydack”.10Digital to Analog Converter, pronounced “dack”.11However, future projects may exploreartificial evolution applied to these and other, even more

bizarre synthesis methods.

2. Previous Work 13

2.3 Evolutionary Algorithms (EA’s)

Evolutionary Algorithms (EA’s) are a broad class of optimisation techniques thatmimic the process of biological evolution. In nature, populations of organisms adaptto their environment through a process of reproduction and selection. Unfit organismsdo not survive to reproduce, which leaves the fitter organisms to reproduce and dom-inate the population. Occasionally, mutations in thegenes of an individual organismwill lead to a more successful creature. This creature’s genes will come to dominatethe gene pool and so raise the level of fitness of the population.

Likewise, the EA works by optimising artificial genes which represent parametersof the problem you are trying to solve. The following pseudo code shows basicallyhow the EA works12:

1 <Initialise> population’s genes randomly.2 While (population unfit), do3 <Recombine> genes of individuals with others (mating).4 <Mutate> genes of individuals.5 <Evaluate> the fitness of each individual.6 <Select> the individuals which form the new population.7 od.

This description skips a lot of detail. Each of the operations in the above pseudocode needs careful consideration for effective implementation:

<Initialise> This involves setting the genes of each individual in the populationto a random number. What probability distribution should you use?

<Recombine> Recombination (mating) can be sexual13 or panmictic14. Selectingthe individuals that mate with each other is usually random. There are also dis-crete and intermediate forms of mating. Discrete recombination forms a childby copying genes from one parent or the other. Intermediate recombinationallowsinterpolation of gene values from parents to form the child.

<Mutate> Mutation introduces slight variations into the population by randomlymodifying genes of individuals. You have to decide what proportion of theparent population is subject to mutation and also the degree of mutation - howmuch different should the child be from the parent?

<Evaluate> This is a big step. First, you have to translate each individual from agenotype to aphenotype15. You then have to evaluate each phenotype using a

12Adapted from [Bac95] page 66.13Involving just two individuals from the parent population.14Involving three or more individuals.15How you do this will depend on how the genes are encoded.

2. Previous Work 14

fitness function. The fitness function must give an indication of how close anindividual is to the optimum solution.

<Select> Once you have found the relative fitness of each individual, you mustselect which individuals survive to form the next population. This can be assimple as just selecting then most fit individuals16, or more complicated. Usu-ally, an individual is more likely to be selected if it has a high fitness score, butunfit individuals still have a chance of being selected.

As the field of EA’s is relatively new, there is still much debate as to the bestmethod for implementing all these operations17. There are currently three main flavoursof EA, which each handle these problems in different ways. They are Genetic Algo-rithms, Evolution Strategies and Evolutionary Programming.

2.3.1 Genetic Algorithms (GA’s)

The Genetic Algorithm (GA) seems to be the most popular form of EA and no doubt,you have already heard of it. They were developed in America John Holland [Hol75]and also popularised David Goldberg [Gol89]18.

The most distinctive feature of the GA is utilisation of a binary encoding for genes.That is, the mutation and recombination operations operate on raw bitstrings - se-quences of zeros and ones. This leads to very simple operations and a very versatileproblem solver. Since everything on a computer is represented as a bitstring at thelowest level, you can apply a GA to a very wide range of problems. A disadvan-tage with this approach is that implementation is complicated as it is often difficult tooperate on single bits.

Mutation is achieved by very occasionally ‘flipping a bit’19 of an individual. Re-combination works in two steps. First, a number ofcrossover points are chosen alongthe length of the genome. Next, the child genome is formed by taking the first parent’sgenome up until the first crossover point, taking the second parent’s genome until thenext crossover point and so on. This process is illustrated in figure 2.5.

There are two possible children formed from the same set of crossover pointsdepending on which parent is chosen first. In GA literature, the recombination processis often just referred to as “crossover”.

GA philosophy, stresses that mutation is a background operator and crossover doesthe real work of optimisation. This claim is backed up by experimental evidence and

16Literally “survival of the fittest”.17A lot of it is guesswork.18For an entertaining and inspiring story of the discovery of the GA, see [Lev93].19Changing a zero to a one, or vice versa.

2. Previous Work 15

Figure 2.5: An illustration of 5 point crossover - taken from [B¨ac95].

also Holland’s schema theorem [Hol75]. This theory of schemata basically proposesthat crossover raises the fitness of the population by combining “building blocks” ofhigh fitness together in different ways20.

Although GA’s are very versatile, they are often slow to converge on a solution.Back demonstrated that GA’s converged slower than both Evolution Strategies andEvolutionary Programming in a number of test problems [B¨ac95].

2.3.2 Evolution Strategies (ES’s)

The Evolution Strategy21 (ES) was a developed in Germany independent of the GA[Bac95]. ES’s use a real encoding for the genome; each gene is represented by a realnumber as opposed to a single bit in the GA. An individual is formed by a vector ofreal numbers. While this means that ES’s are not as flexible as GA’s, they are oftenmuch easier to implement.

In the most basic ES, an individual�!a is represented by a vector of object variables�!x which are the parameters being optimised:

�!a = (x1; x2; : : : ; xn)

Mutation is performed by adding a small amount of noise to each parameter:

x0i= xi +N(0; �i)

whereN(0; �) denotes a normally distributed random variable with zero mean andstandard deviation�. The standard deviation may be different for each parameterxiand must be specified in advance. The child individual is formed as a vector of all themutated parameters:

�!a 0 = (x01; x0

2; : : : ; x0

n)

20See [Hol75] for alot more detail.21Or more correctlyEvolutionsstrategie.

2. Previous Work 16

This mutation procedure is all we need to implement the simple (1+1)-ES22; an evo-lution strategy where 1 parent is mutated to form 1 child. The best individual out ofthe parent and child is chosen to become the parent of the next generation, the pro-cess is then repeated. Obviously recombination (mating) is impossible with a parentpopulation of one.

Much more complex kinds of ES arise when populations of more than one areconsidered. The(�+ �)-ES works by recombining� parents which are then mutatedto form � children. The best� individuals are chosen from the parents and childrento form the parent population of the next generation. The current state-of-the-artevolution strategy is the(�; �)-ES. Here again the� parents are recombined thenmutated to form� children, but the new parent population is selected only from thechildren.

The current state-of-the-art Evolution Strategies also employ self optimisation.Individuals are not only represented by a vector of object variables�!x as before, butalso by a vector of standard deviations�!� and rotation angles�!� :

�!a = (�!x ;�!� ;�!� )

The standard deviations and rotation angles are subject to mutation as described above.After these have been mutated, they are used to modify the probability distribution formutation of each object variable. Thus each object variable is mutated by a specialnormal distribution that is being optimised as well [Bäc95]:

x0i= xi +N(0; C(�0

i; �0

i))

In the above equationC�1(�; �) is a covariance matrix. If a variablexi needs a largestandard deviation, its�i will eventually be mutated to a larger value. Using thismethod, you do not have to worry about specifying standard distributions for eachvariable, they will be set to optimum values automatically.

In contrast to GA’s, mutation is stressed as the main operator in ES’s. The contri-bution of recombination is not disregarded, but all the self optimisation features de-scribed above are for the benefit of mutation, not recombination. Bäck demonstratesthat the ES converges much faster than both GA’s and Evolutionary Programming ona variety of fitness landscapes [Bäc95].

22This notation is used in [Bäc95] to distinguish the different flavours of ES. The first number is thesize of the parent population. The second number is the size of the child population that is formed fromthe parent. If the numbers are separated by a ‘+’ (plus), the new parent population is chosen from theold parents and the children. If the numbers are separated by a ‘,’ (comma), the new parent populationis chosenonly from the children.

2. Previous Work 17

2.3.3 Evolutionary Programming (EP)

Evolutionary Programming (EP) is basically the American equivalent of the ES23. Ituses a real encoding for genes and has some self optimising features. In EP, variancesare stored in the genome as opposed to ES’s which store standard deviations. Finally,EP has no recombination operator and relies on the power of mutation alone. It stillperforms admirably however, converging faster than GA’s24 on a variety of fitnesslandscapes [B¨ac95].

We now move on to look at how EA’s have been applied to the field of soundsynthesis.

2.4 EA’s for Sound Synthesis

Almost all of the published research on EA’s applied to sound synthesis has beenconducted by Andrew Horner and his colleagues. He first applied a GA to a musicalapplication with David Goldberg in 1991 [HG91]. Given a starting and finishingpattern of notes, they used a GA to evolve patterns of notes that bridged the startpattern to the finishing pattern25. The results were gratifying and the authors suggesteda further application could be found in using a GA to evolve the timbre of a sound26,as opposed to a sequence of notes.

This is exactly what Horner went on to do in subsequent years, publishing a veri-table barrage of papers on the subject. These are shown in table 2.4.

In his first paper on the subject, Horner et. al. outline just why GA’s are useful forsound synthesis [HBH93] :

FM synthesis is a very efficient, though not always easily controlled tech-nique for generating interesting sounds.

The paper reviews the techniques used so far to solve the problem of choosing param-eters for FM synthesis and labels them “ad hoc.” Instead [HBH93]:

The task of finding FM parameters to match musical tones is typical ofproblems that defy traditional optimisation, yet are suited to [a] geneticalgorithm solution.

23However, it seems that EP is not as advanced as ES.24But slower than ES’s25For musicians: this process is called “Thematic Bridging”.26Evolving timbre’s with an EA is exactly what this thesis is concerned with, so on reading this I

was very interested to find out if Horner had followed up his suggestion.

2. Previous Work 18

Year Title Reference1993 GA’s and their Application to FM Matching Synthesis [HBH93]1995 Wavetable Matching Synthesis of Instruments with GA’s [Hor95a]1995 Envelope Matching with Genetic Algorithms [Hor95b]1996 A GA Based method for Synthesis of Low Peak Amp Signals[HB96]1996 Group Synthesis with Genetic Algorithms [CH96a]1996 Common Tone Adaptive Tuning using Genetic Algorithms [HA96]1996 Discrete Summation Synthesis using GA’s [CH96b]1997 Hybrid Sampling-Wavetable Synthesis with GA’s [YH97]

Table 2.1: Papers published by Andrew Horner et. al. on the subject of applying GA’sto sound synthesis

All of the papers in table 2.4 use the same basic method to apply a GA to a soundsynthesis algorithm27: First, a sample of the instrument to be synthesised is taken,for example a trumpet tone. This original sample will form the fitness function byenabling a comparison between it and the synthesized sound. The GA then choosesparameters for the synthesis algorithm. The tone is synthesised and a fitness score isobtained from the relative spectral error between the synthesised tone and the originalsample. As the generations pass, each population of individuals matches more closelywith the sampled tone. In the example of the trumpet tone, as each generation passesyou would hear synthesised tones that get closer and closer to the sampled trumpetsound.

The results of this technique were very successful in all of the papers in table 2.4.Most of the papers conclude that the Genetic Algorithm method of choosing synthesisparameters is much more effective than any of the “ad hoc” methods used to date.

Horner and his colleague’s work may be extensive, but note that his method doesnot involve human interaction. The user of the system gives only a sample of theinstrument they want synthesised, and the GA goes away and does the rest. Thisimplies that the method is limited to producing sounds that already exist. That isgood news for this project: Horner’s work has shown that GA’s can be successfulwhen applied to sound synthesis algorithms, but he hasn’t tried an interactive systemcapable of producing unknown, novel and unheard-of sounds.

2.5 Interactive Evolutionary Algorithms (IEA)

So far we have only considered Evolutionary Algorithms where the fitness functionis objective. For example, in Horner’s work, the fitness of a synthesised instrument

27The synthesis algorithm was different for each paper.

2. Previous Work 19

tone was calculated from the relative spectral error between it and the original sam-pled tone. What if instead, a human steps in and tells the computer how good thesynthesised tone sounds? In this case the fitness function becomessubjective. Therehas actually been a fair amount of investigation into EA’s which use subjective, hu-man supplied fitness functions. This section will present a summary of this work. Itis given in chronological order so you can develop the story as you read.

2.5.1 Dawkins’ Biomorphs

Richard Dawkins [Daw86] was the first to demonstrate the power of a subjective fit-ness function. He aimed to construct a system that would demonstrate the role ofmutation in evolution; his system would enable a human user to “breed” pictures oftrees and plants. He was astounded testing his system for the first time, when afteronly 20 generations he had evolved bug-like forms that he later called “biomorphs”.

Dawkins’ system28 worked with a simple recursive tree drawing algorithm thattook nine parameters. These parameters determined the form of the resulting tree.The user of the system is presented with a screen of “mutant offspring” trees that aregenerated by randomly changing one parameter of the “parent” tree. The user exam-ines the offspring and selects the most aesthetically pleasing29 - the chosen tree goeson to be the parent for the next screenfull of mutant progeny. Figure 2.6 demonstratesevolution using Dawkins system.

The picture shows each individual biomorph in a box. A line between boxes con-nects each biomorph to its parent, forming a sort of “family tree”. You can see howquick the process is - a bug like creature is evolved in just a few generations.

This mutation and selection process produced surprising results. Dawkins quicklyfound that the scope of his simple algorithm was not limited to drawing trees. Hewas able to produce all manner of plants, bugs and sea creatures. He even managedto evolve the letters of his own name! Clearly, this manner of forms did not seemapparent when he first designed the algorithm. By manually plugging numbers intothe algorithm, he may never have found the variety of forms he saw. It was by usingthe EA as a way to explore the parameter space of the algorithm that he was able tosee its real potential.

28The system is described in much more detail in [Daw88].29This is Artificial selection as opposed to Natural Selection.

2. Previous Work 20

Figure 2.6: Evolution of “biomorphs” with Dawkins’ system - taken from [Daw86].

2. Previous Work 21

2.5.2 Oppenheimer’s Artificial Menagerie

Peter Oppenheimer also described a system of interactive artificial evolution [Ope88].However, his images were more complex than Dawkins. Oppenheimer used an al-gorithm with 15 parameters to evolve impressive three dimensional plant forms (seefigure 2.7).

Figure 2.7: Example of an evolved plant form - taken from [Ope88].

As in Dawkins system, mutation is the only genetic operator used and “Fitness, ofcourse, is in the eye of the beholder.”30

2.5.3 Smith’s Bugs

Joshua Smith extended Dawkins’ work with “biomorphs,” as well as formalising someconcepts about GA’s with subjective fitness functions [Smi91].

Smith implemented a system like Dawkins’ but with a larger breeding populationand a genetic recombination operator added. This allowed the user to select multiplebiomorphs and have them mate as opposed to just mutating them. Also, a two di-mensional Fourier series was used to generate the biomorphs as opposed to Dawkins’

30From [Ope88].

2. Previous Work 22

recursive tree drawing algorithm. Here, the genes of an individual are Fourier coef-ficients31. Figure 2.8 shows a sample of biomorphs evolved using Smith’s system.

Figure 2.8: Biomorphs evolved with Smith’s system - taken from [Smi91].

Smith defined the term Interactive Genetic Algorithm (IGA) to refer to systemswhere a human user acts as the fitness function for a GA32. Three criteria are specifiedfor applicability of an IGA to a problem [Smi91]:

� The problem can be formulated as a search through a parameter space.

� Candidate solutions to the problem can be generated in near real time.

� The utility of candidate solutions can be compared by humans, but not (practi-cally) by means of a precisely specified formula.

31Smith uses a real encoding for his genes which implies that he is using an ES as opposed to a GAwhich are characterised by binary encodings. However, Smith considers ES’s to be a subset of GA’sas he states in [Smi94]. This is contrary to B¨ack’s belief that ES’s and GA’s fall under the umbrella ofEA’s [Bac95].

32Likewise I use the term Interactive Evolutionary Algorithm (IEA) to refer to an EvolutionaryAlgorithm (EA) that uses a human user as the fitness function. I believe this to be a more politicallycorrect term as IGA implies the use of a binary encoding.

2. Previous Work 23

Note that the problem of generating novel and interesting sounds with synthesis algo-rithms (artificial evolution for sound synthesis) satisfies these criteria:

� Finding interesting soundsis a search through the algorithm’s parameter space.

� Current computing power allows synthesis of sound from parameters in nearreal time33.

� Deciding whether a sound is “novel” or “interesting” cannot be achieved with aprecisely specified formula, but can be easily accomplished by a human.

2.5.4 Sims’ Artificial Evolution

Karl Sims’ [Sim91] constructed one of the most elaborate demonstrations of thesetechniques to date34. Sims used a powerful super-computer to generate and evolveastounding images (see figure 2.9).

Figure 2.9: An image generated from an evolved symbolic lisp expression - takenfrom [Sim91].

33Not to mention the real time synthesis available with specialised hardware34Indeed, this paper was the main inspiration for this thesis.

2. Previous Work 24

Sims evolved 3D plant structures generated from procedural models as in [Ope88],but he also added four different recombination operators. This allowed the user tomate plant structures in a variety of ways to form all manner of offspring. Sims wenton further and used symbolic lisp expressions as genotypes for the evolution process.In this case, evolution was not limited to number of parameters in a procedural model,but was “open ended.” The symbolic expressions were mutated and mated to generate2D images35, 3D volume textures and even animations. Although the resulting imagesare rather abstract, they are nevertheless beautiful. In a later paper, he reported thathe had extended the system to evolve 3D shapes and even 2D dynamical systems[Sim93].

2.5.5 Moore’s GAMusic 1.0

Jason Moore’s program “GAMusic 1.0” [Moo94] is an interactive melody evolver andup until recently was the closest example of work similar to this thesis.

The program runs on Microsoft Windows and consists of a simple interface toenable the user to evolve melodies that play over the PC speaker. The user auditionseach melody and assigns a fitness value (good, average or poor). Melodies are rep-resented as a 128 bit binary string and a simple GA is used to recombine and mutatethese. There are controls that enable adjustment of the mutation and recombinationfrequency. After the user assigns a fitness value to each melody in the population, theGA mutates and recombines the binary strings that represent them. The newly createdpopulation is again auditioned by the user and the process is repeated until the user issatisfied with the melody.

Figure 2.10 shows a screen shot of the GAMusic user interface. You can see thecontrols for mutation and recombination frequency, the population of 12 melodies(with fitness ratings) and bit string that represents the current melody.

There are a number of problems with GAMusic. Since the melodies play over thePC speaker as simple bleeps, they are not very pleasing to listen to. Also, there is noway (except by ear) to obtain the melody data so you can use it in other applications(e.g. a composition that you are working on).

However, the most frustrating aspect of GAMusic is that it takes a long time toevolve decent melodies. This is partly due to the time it takes to audition a singlepopulation. There are 12 melodies in each population which can last up to 5 secondseach. On average, it takes about 1 minute to go through all the melodies and judgetheir fitness. Many populations must be auditioned to get a good melody - evolutionprogresses slowly and the user quickly becomes bored36. In documentation for the

35Again, see figure 2.9.36Especially after listening to the PC speaker bleeps for so long.

2. Previous Work 25

Figure 2.10: A screen shot of the GAMusic 1.0 user interface.

program, Moore mentions plans to develop a program to evolve more complex soundsusing a PC sound-card37, but these have subsequently been canceled [Moo98].

2.5.6 van Goch’s P-Farm

When I first discovered this program I was shattered. Arno van Goch’s “P-Farm”[Goc96] achieved exactly what I hoped this project would - and it did it well! Theproblem was, I only discovered it halfway through the year when my literature surveyhad been completed and I was thinking about implementation.

P-Farm is an experimental program that works with an external synthesizer. Inits current implementation (version 0.3) it supports the Roland Juno 106 and YamahaV50 synthesisers, and runs under Microsoft Windows.

To use P-Farm, you must have an external synthesiser. The program works byevolving patches, then sending the patch information to the synthesiser via MIDI38.The user can then play the synthesiser to evaluate the fitness of the patch that has justbeen sent. Figure 2.11 shows a screen shot of P-Farm’s user interface. The externalsynthesizer also required to use the system is not shown.

37A program that evolves sounds is the topic of this thesis.38Musical Instrument Digital Interface.

2. Previous Work 26

Figure 2.11: A screen shot of P-Farm’s user interface.

P-Farm uses a GA with a population of 32 patches. The fitness function is binary:the user can choose either to keep a patch or delete it. After you have gone throughthe population keeping only the fit patches, the deleted patches are filled with newindividuals formed by crossing and mutating the fit patches. The program lets youcontrol three parameters of the GA, namely:

crossover ratio What proportion of each parent’s genes appear in the child.

mutation rate The probability that a given gene will mutate.

transposition rate The probability that genes will get transposed to different parts ofthe chromosone.

After you get used to it, you can breed quite good sounds using P-Farm. There arehowever some criticisms. Evaluating and keeping track of all 32 patches is sometimesdifficult. You frequently have to go back to a patch and remind yourself of what itsounded like. Also, switching between the computer keyboard and the synthesiserkeyboard can be frustrating. Although some of the program functions are accessiblevia the synthesiser keyboard, when you have to switch around it can make evolutionslow and painful.

The work of van Goch is very close to the subject of this thesis. So close in fact thatI could base my work on extending or modifying the algorithms used in P-Farm. This

2. Previous Work 27

however is not a possibility. Although the program is still under active development,technical details39 are not available [Goc98].

2.6 Summary

The survey of literature conducted gives a good indication that an Interactive Evo-lutionary Algorithm applied to sound synthesis would yield fruitful results. This ishighlighted by the following facts:

� Techniques such as FM synthesis can produce a wide variety of timbres froma small number of parameters. However, choosing the right parameters is diffi-cult.

� Evolutionary Algorithms are optimization procedures that have proved to beuseful in searching the parameter spaces of a wide variety of problems.

� Horner and his colleagues have shown that EA’s can be successfully applied tosound synthesis algorithms in a non-interactive manner.

� Dawkins, Sims and Oppenheimer demonstrated the power of Interactive Evolu-tionary Algorithms for searching the parameter space of procedural models forcomputer graphics.

� The problem of evolving “interesting” and “novel” sounds fits Smiths criteriafor applicability of an IEA.

� Very few people have tried to apply an IEA to sound synthesis before. Moorecanceled his plans and the only other known work is that of van Goch, which isstill very experimental.

So it seems thatartificial evolution for sound synthesis is a good idea. Using thesurveyed literature as a foundation we can now begin to design a system that willembody the goals of this thesis – this is the topic of the next chapter.

39That is, source code that I could modify.

Chapter 3

System Design

3.1 Introduction

This chapter is concerned with the initial design of a system that can achieve thegoals of this project. First, the problem is divided into a number of subproblems –this gives an outline of the basic system. Then, for each of these subproblems issuesrelated to implementation are discussed. Next, the design decisions actually takenare explained and justified. Then finally, we look at ways of evaluating the system inorder to determine whether the goals have been achieved.

3.2 System Outline

A system that applies an Interactive Evolutionary Algorithm to sound synthesis canbe broken into three main parts as shown in figure 3.1.

Each of part achieves a different function:

User Interface This part allows the user to audition individuals (hear the sounds pro-duced by theSound Synthesissection) and rate their fitness (evaluate them).

Evolutionary Algorithm This part takes the fitness data from theUser Interfaceand accordingly mates and mutates the synthesis parameters.

Sound SynthesisThis part generates the actual audio data from the sound synthesisparameters passed from theEvolutionary Algorithm section.

28

3. System Design 29

Evolutionary Algorithm

of individualsMutation and mating

Sound Synthesis

Generates sound from individual

User Interface

Selection of fit individuals.

Synthesis Parameters

Audio DataSelected Individuals

Figure 3.1: The three main components of the proposed system

3.3 Considerations

For each part of the proposed system (figure 3.1), a number of issues have to beconsidered before you can decide on an implementation. These issues are discussedin the following sections.

3.3.1 Sound Synthesis

There are basically two ways you can implement this part: in hardware or in software.

Hardware

A hardware implementation would use an existing stand-alone synthesizer1. TheEvo-lutionary Algorithm would send parameters to the synthesizer via MIDI2. The userwould then be able to audition the sound by playing the synthesizer as normal. This

1Sometimes the abbreviation ‘synth’ is used in this document - it refers to a stand-alone hardwaresynthesizer.

2Musical Instrument Digital Interface. See [Boo87] for a good introduction to MIDI.

3. System Design 30

is exactly the way that van Goch’s system works3.

The advantage of a hardware approach is speed. The only delay required is thetime taken to transfer the parameter data (typically less than 200 bytes) from a com-puter to the synth. The user can then test out the synthesis algorithm in real time,playing any combination of notes desired.

The disadvantages of a hardware system are cost and inflexibility. Synthesizers aretypically quite expensive and they only implement a very limited number of synthesisalgorithms4. Since each synthesizer manufacturer has their own way of interpretingparameters, you could only hope to support one particular synth from one particularmanufacturer. Users who didn’t own that particular synth wouldn’t be able to use thesystem. Even if they did own it, they would always be restricted to exploring the setparameter space that comes with that particular synth.

Software

Alternatively, a software implementation of theSound Synthesispart would provideinfinite flexibility at a lower cost, since no extra hardware would be required by theuser. If they possessed a basic computer capable of sample playback5, they would beable to use the system.

Freely available packages such as Csound [Cso98] enable the user to experimentwith virtually any known synthesis algorithm. MatLab [Mat98] also allows construc-tion of custom algorithms. MatLab is not as musically oriented as Csound, but itprovides other useful features such as data visualisation and user interface tools.

Another advantage of a softwareSound Synthesispart is the increased integra-tion. Users don’t have to divide their attention between a musical keyboard and acomputer keyboard6. They can audition individuals, then rate their fitness all from thesame place.

However, the disadvantage of a software approach is speed. A software synthesisalgorithm will always be slower than a dedicated hardware synth. This is cause forconcern, as recalling one of Smith’s criteria: candidate solutions must be generated innear real time7. Never fear – today’s computing power is fast enough to satisfy thisrequirement as long as the algorithm is not too complex8.

3Refer back to section 2.5.6 for a description of van Goch’s system.4Most hardware synths only implement 1 synthesis algorithm.5Most computers these days come with sound cards - these provide sample playback functionality.6This was one of the annoying features of van Goch’s system - see section 2.5.6.7Smith’s criteria for application of an IGA were reported in section 2.5.3.8There are now real time versions of Csound that run with Pentium processors.

3. System Design 31

3.3.2 Evolutionary Algorithm

Implementing theEvolutionary Algorithm part of the system also raises issues whichmust be considered. It is assumed that this part of the system will run on a smallcomputer (UNIX or Windows) as this is easiest and cheapest option. Given this, thefollowing issues have to be considered:

Representation

How will the genes of individuals be represented?

The basic choice here is between a binary encoding (as in a GA) or a real encoding(as in an ES). Binary encodings are much more flexible, but they are harder to imple-ment. A real encoding for genes leads to an easier implementation – each gene is justa real number representing a different parameter of the synthesis algorithm. It couldalso lead to faster convergence9 – that is, the user is able to find interesting soundsmore quickly.

Operators

What kind of genetic operators will be useful for a sound synthesis application?

Mutation seems an essential operation, and is also fairly easy to implement. Mat-ing (recombination) requires more thought and also has many alternatives10. Sincehardly any work has been done in this field, it is not known which types of matingwork better for an interactive sound synthesis application.

Random Numbers

Whichever operators for theEvolutionary Algorithm are chosen, good random num-ber generators will be required for their implementation. This fact should be kept inmind when making a decision on the implementation language.

Self Adaption

Should any of self adaptive features of ES’s11 be used in the system?

9Section 2.3 discusses the convergence rates of different EA’s.10These were discussed in section 2.3.11These were discussed in section 2.3.2.

3. System Design 32

Self adaption has the potential to make convergence very fast. However, it hasonly been used in non-interactive ES’s with large populations – it may not work inan interactive system because population sizes are much smaller and are evolved forfewer generations.

3.3.3 User Interface

TheUser Interface component possibly requires the most design thought. The issuehere is basically: How do you make the system easy and intuitive to use?

SomeUser Interface considerations are:

Population size

A user of the system can only hear one sound at a time12. This is a major difference tothe graphical systems of Dawkins, Sims and Smith13 where the user can very quicklyaudition a lot of images. Only being able to audition one sound at a time impliesthat population size should be somewhat smaller. If it weren’t, auditioning the wholepopulation would take a long time, and the user may become bored.

Fitness rating

Would it be better to have a binary rating (e.g. good or bad) or some kind of scale(e.g. 1 to 10)? This decision will also be dependent on the population size and thegenetic operators available.

Implementation Platform

Because the interface to the system is likely to be graphical, the possible implementa-tion platforms need to be considered.

Java provides very good GUI14 facilities and also has the advantage that it could beused over the World Wide Web. However at present, the audio support in Java is lessthan adequate – Java cannot generate or play audio files on most architectures. This

12A possible method to overcome this limitation involves use of the “cocktail party effect” [CE95].This is the phenomenon whereby a person can distinguish multiple sound sources as long as they arespatially separated. For example, at a cocktail party where there are many conversations occurring atonce, you are able to clearly understand the one you are focussed on.

13Described in section 2.5.14Graphical User Interface

3. System Design 33

could possibly be circumvented by using a CGI script to generate audio files. MostWeb browsers support playback of audio files, so you could send the pre-generatedaudio files across the Web to the client. Alas, this idea is also impractical: audio filesare typically quite large, and thus would take a long time to send over the Web. Thiswould make the system slow and frustrating to use.

Microsoft Windows provides good GUI support once you come to grips with itsridiculous conventions. However, its sound playback utilities are very hard to use.

Tcl/Tk for Unix provides very good and easy to use GUI facilities. The powerfulredirection features of Unix would make audio playback a breeze.

MatLab also has easy to use GUI features, but these are limited in some respects.

3.4 The Design Decision

The final decision for the implementation of the three system parts was not reached ina single instant, but evolved15 over the course of the project.

The initial experiments and system development (chapter 4) were conducted inMatLab. MatLab was very easy to use and provided a rapid development environment.Later in the project when the basic system had been worked out, it seemed easiest tostick with MatLab. Also, the extra time saved in not having to re-implement thesystem in another language meant that more could be accomplished. As a result, thefinal system was implemented using the GUI features of MatLab (chapter 5).

Implementing all three system parts in MatLab had a number of impacts. Theseare described as follows:

Sound Synthesis The sound synthesis algorithm ran in software. This cost less thanusing external hardware, but was slower. The slow speed of the software meantthat users were limited to auditioning just one note of each sound. However,the advantage of having everything implemented in MatLab was increased in-tegration - there were no problems switching between computer keyboards andsynthesizer keyboards. A software approach also offers flexibility in the choiceof synthesis algorithm, however this project mainly focussed on FM synthesis16.

Evolutionary Algorithm Being a statistical package, MatLab provides very goodrandom number generators – this was an advantage for this part of the system.The big disadvantage however was the lack of data structures. This made some

15This pun was bound to happen, sooner or later.16An experiment was also conducted using additive synthesis - see Appendix A.

3. System Design 34

tasks quite difficult and meant concepts like ‘Object Oriented’ did not exist.Fortunately, implementing a real encoding for genes was no problem – vectorsof real numbers are supported well by MatLab.

User Interface In preliminary experiments, a simple keyboard interface was usedand this was no problem to implement in MatLab. In the final system, a GUIwas built. MatLab’s GUI features were somewhat limited and at times painfulto use. However, due to MatLab’s interactive nature, development was probablya lot easier and quicker than it would have been on other platforms.

Other design decisions not explained here (e.g. population size) will be illumi-nated as the system is developed in chapter 4.

3.5 Evaluation

It’s all very well having a system that generates “novel” sounds, but how can youmeasure the success of such a system?

A successful system is one that satisfies its goals. The first goal as stated in section1.2 could be satisfied by implementing the system described above. What about thesecond goal?

One way to approach this would be to collect opinions from a number of users.These users could be composers or musicians who require such sounds for their work,or merely novices who are seeking entertainment. Some questions that could be askedof the users to determine the success of the system include:

� Is the system easy to use?

� In your opinion, st the evolution of sounds controllable?

� Is this system useful? Would you be able to use the sounds generated in com-positions?

� Is using this system easier than other methods of obtaining the same kinds ofsound?

Another way to test success would be to collect statistics from users. These statis-tics would be obtained from a number of experiments:

� First the user would be asked to use a system where they manually had to adjustparameters of the sound synthesis algorithm. The number of changes they hadto make until they had a satisfying sound would be recorded.

3. System Design 35

� This process would be repeated on a system where the parameters of the al-gorithm were chosen at random. Finally the users would use the evolutionarysystem.

� After these tests, we would have a measure of how intuitive each system was touse. A comparison could be made about which is the most effective system.

The actual experiment conducted to assess the success of the system forms thesubject of chapter 5.

Now that we have discussed the various issues relating to the implementation andassessment ofartificial evolution for sound synthesis, we move on to chapter 4 whichtells the story of how the actual system was developed.

Chapter 4

System Development

4.1 Introduction

This chapter documents the steps taken in developing the software that forms the basisof this project – software that appliesartificial evolution to sound synthesis1.

The guiding philosophy throughout the development of this system was to imitateDawkins’ system2, only applied to sound instead of graphics. Recall that Dawkins’implemented an IEA based on a very simple tree drawing algorithm with mutationas the only genetic operator. Humble as it was, the fantastic results obtained inspireda whole new generation of research. It seems justified to attempt to repeat this forsound synthesis. The idea then, is to keep the synthesis algorithm simple and focuson mutation.

To this end, FM synthesis was chosen as the sound synthesis algorithm. FM syn-thesis can produce a wide variety of timbres with only a few parameters, yet it is veryhard to ‘control’. It seemed a perfect candidate for artificial evolution.

The software development is presented in a series of experiments. As describedbefore, these experiments were conducted in MatLab as it was easy to use, and pro-vided a rapid development environment. The programs that performed the experi-ments were named EvoS which stands forEvolution Strategy.

An outline of this chapter is as follows:

EvoS 2 aims to implement a most basic form of evolution with the most basic FMalgorithm.

1Yeah, well... it was a spur of the moment thing.2See section 2.5.1.

36

4. System Development 37

EvoS 3 experiments with self-adaption, but concludes it is unsuitable for an interac-tive system.

EvoS 4 tackles the tricky problem of mutating envelopes.

EvoS 5 develops a method of mutating modulation indices.

EvoS 6 describes how the problem of mutating frequency ratios was overcome.

EvoS 7 finally, combines all the previously developed mutation techniques into a sin-gle system. This system is tested by a number of users.

That being said, lets start at the beginning...

4.2 EvoS 2

4.2.1 Aim

The goal of EvoS 23 was to create a most basic interactive Evolution Strategy. To keepthings very simple, a (1+1)-ES was decided on. Here, one parent sound is mutated toform a single child sound. If the fitness function4 deems the child sound is better thanthe parent sound, it is replaced by the child who becomes the new parent. On the otherhand, if the child sound is worse than the parent sound, that child is discarded and anew one is formed by mutation of the same parent. Note that there is no recombinationor mating involved, the only genetic operator is mutation.

Likewise, to keep things simple in the sound synthesis section, a very basic al-gorithm was used: fixed index frequency modulation (FM)5. This requires only threeparameters to produce a sound:

fm = modulating frequencyfc = carrier frequencyI = modulating index, a measure of how much the modulator deviates

the carrier frequency.

Combining the FM synthesis with the Evolution Strategy, EvoS 2 aimed to produce asystem that provided interactive evolution of sounds.

3What happened to EvoS 1? – It was a mutant that got out of control and had to be deleted.4Remember, in an interactive system the human user acts as the fitness function5FM synthesis was described in detail in section 2.2.5. Also, see figure 2.3 for an illustration of the

basic FM algorithm implemented in EvoS 2


4.2.2 Implementation

Implementing the system described above in MatLab required a number of subpro-grams or procedures. These subprograms were drawn together in the main program(EvoS 2) which acted as a user interface.

Since there are three parameters required to generate a sound using FM, an indi-vidual is represented by a vector of three real numbers - these are its genes:

�!a = (fc; fm; I)

There are a number of procedures that operate on the genes of an individual andachieve the vital functions of the ES:

eval This procedure takes the genotype of an individual and evaluates the pheno-type. Although this sounds complicated, all it involves is extracting the synthe-sis parameters (fm, fc andI) from the vector, generating the resulting sound,and playing this sound to the user. In a non-interactive system,eval wouldalso asses the phenotype according to the fitness function, but here we must letthe user decide how good it sounds.

mutate This procedure takes the genes of a parent individual and returns genes ofthe mutant child offspring. The ‘mutation’ is just a random deviation appliedto each gene of the parent. This is achieved by sampling a normal distributionwith a mean of the parent gene. The variance of the normal distribution differsaccording to which gene is being mutated. For example the variance for mu-tating the frequency genesfm andfc is different to the variance for mutatingthe index geneI. Deciding on these variances is tricky and will be discussedlater. So basically,mutate works by applying the normal distribution to eachelement in the vector of numbers passed to it. What results is a mutated childgenome.

init This procedure returns a random set of genes. It is used to initialise the firstparent before any mutation can take place. To generate the random genes, auniform probability distribution is sampled. Here we have to decide on therange of legal values for each gene, this is also a tricky issue that deserves adiscussion of its own.

Wheneval, mutate andinit are combined with a suitable user interface, anEvolution Strategy is formed. The interface randomly initialises the first parent usinginit and then forms the first mutant child withmutate. These two sounds areplayed to the user by callingeval. The user then decides which sound they likemore: If they like the parent more, the old child is discarded andmutate is called


on the parent once again; If they like the child more, the child takes the parent’s placeand a new child is formed viamutate. With this system the user explores the spaceof possible sounds by listening to some randomly chosen directions, deciding whichis the best and then moving forward in that direction.

Figure 4.1 shows a screen shot of EvoS 2, demonstrating the user interface. The

Figure 4.1: A screen shot of the EvoS2 user interface.

user executes commands by entering a character corresponding to a command in themenu. The following commands can be seen in the menu in figure 4.1:

Parent This plays back the current parent sound to the user.

Child This plays back the current child sound to the user.

Mutate This mutates the parent to form a new child sound and then plays this newsound so the user can hear it.

Replace This replaces the parent with the current child. A mutant child is formedfrom the new parent and this sound is played to the user.

Genes This prints out the actual genes (fc, fmandI) of the current parent and child.

Display The user can obtain a graph of all the parents they have chosen to date. Thegraph shows how they have navigated through the space of possible sounds.


History The user can hear all of the parents they have chosen to date, starting withthe first and ending with the most current. This enables the user to hear how thesound they started with has gradually changed to become the sound it is now.

Variances and Ranges

One of the tricky issues encountered in implementingmutate andinit was decid-ing on valid ranges and variances for each of the genes in the genome. Remember,the genes represent parameters of a synthesis algorithm and so certain values will notmake any sense – for example iffm or fc are negative.

Once a range valid range for each parameter was chosen, a variance was needed.Choosing the “right” variance was critical to the effective operation ofmutate. Forexample, suppose valid frequencies forfm andfc range from 0 to 5,000Hz. If thevariance is�1Hz then the user will not notice the difference between a parent soundand its mutant child. They will sound almost exactly the same. As a result, all di-rections in the sound space will sound the same and the user won’t be able to decidewhich way to go. On the other hand, suppose the variance is�1000Hz. The user willnot be able to tell that a parent and its mutant child are related. They will sound vastlydifferent, the user will end up hopping at random around the space of possible sounds.

Clearly a balance must be reached where the variance is small enough that theuser can hear how the child sound relates to its parent, but large enough so there is adiscernible difference.

In EvoS 2 the main focus was to get a system up and running, so deciding oneffective ranges and variances was left to later versions of the system. The somewhatarbitrary ranges and variances selected are shown in table 4.1.

Parameter Minimum Maximum Variancefc 0Hz 5,000Hz �100Hzfm 0Hz 5,000Hz �100HzI 0 30 �2

Table 4.1: Ranges and variances for the parameters of EvoS 2.

4.2.3 Results

Although EvoS 2 was very simple it did provide some gratifying results. As therewere only three parameters, it meant that you could graph these in 3 dimensional


space and obtain a picture of how the user navigated through the parameter values(the ‘Display’ feature). Figure 4.2 shows one such picture.

Figure 4.2: A sample evolution run for EvoS2.

Each cross in figure 4.2 represents a parent that the user has chosen. The evolutionstarts where there is no cross and finishes at the asterix6.

The thing to notice about figure 4.2 is the clustering of points in one area. Thiscluster represents an area of the parameter space that the user found “interesting”and wanted to keep exploring. The user has auditioned points outside this cluster butfound them unpleasant. You can see at the start, the user has cut a straight course awayfrom the initial point and into the cluster. This implies the initial point represented aparameter combination resulting in an unpleasant sound.

In reality, the sounds produced by EvoS 2 werevery simple, ranging from mildbleeps to harsh distorted tones. The only real form of navigation the user could con-duct was to make the discomforting outbursts more sublime. The cluster in figure 4.2actually represents an area of the sound space that was less harsh than its surround-ings. Using the ‘history’ feature you could hear how each parent was related to its

6Just an apology about the graphs: the results for the EvoS experiments were collected at differenttimes. Later on, clearer methods of displaying the data were discovered, but is was too late for theseearly results. As well as being less clear, it means that the display format is inconsistent betweengraphs. Whenever there is a format change, it is explained in detail, but once again I apologise for thisinconvenience


predecessor, but you could also notice the difference. This suggests that the choice ofvariances was adequate.

Another interesting result was the effect of a small initial gene pool. In nature,when the gene pool becomes small, it is disastrous. If a change in the environment isintroduced that kills one individual, all individuals will die because they are so similarto one another. In EvoS 2 the gene pool is very small – the entire population consistsof just one individual. If this individual sounds very displeasing, it is most likely thatall its mutant children will sound displeasing. Subsequently the user cannot pick anappropriate child and the entire population is doomed. This problem may be alleviatedif the initial population consists of a few random parents. Hopefully, at least one ofthese will sound slightly pleasant and allow meaningful evolution.

4.2.4 Conclusion

EvoS 2 demonstrated that a user could navigate through a simple FM sound spaceusing a (1+1) evolution strategy. Further work needs to address the problems of:choosing appropriate ranges and variances for parameters; and generating sounds thatare actually pleasant to listen to.

4.3 EvoS 3

4.3.1 Aim

The goal of EvoS 3 was to trial an automated method of choosing variances for pa-rameters.

In EvoS 2 it was noted that deciding on effective variances for each of the parame-ters was a tricky matter. Why not, instead of guessing the variances by trial and error,include the variances as part of the genome? The variances would then evolve to suit-able values just as the synthesis values do. This is the kind of self-adaption or “meta-evolution” that is characteristic of more advanced Evolution Strategies [B¨ac95].


In order to evolve the variances for each of the parameters, extra genes were addedthe genome from EvoS 2. These were:

�2fm

The variance used for mutation of the modulating frequency (fm).


Parameter Minimum Maximum Variance�2fc

0Hz 200Hz �100Hz�2fm

0Hz 200Hz �100Hz�2I

0 4 �2

Table 4.2: Variances and ranges for thevariances of EvoS 3

�2fc

The variance used for mutation of the carrier frequency (fc).

�2I

The variance used for mutation of the modulation index (I).

These additions resulted in a genome of the form:

�!a = (fc; fm; I; �2

fc; �2

fm; �2

I)

Theeval subroutine remained unchanged from EvoS 2, as it only required thethree synthesis parameters to generate the sound.

Theinit subroutine needed a slight change in order to generate random initialvalues for the new parameters. Once again this raised the issue of a valid range forthe new parameters.

The mutate subroutine was modified so that the variances of each parameterwere mutated first. These mutated variances were used to mutate the synthesis pa-rameters. This again raised the issue of what variance and valid range to choose formutating the variances�2

fm, �2

fc, and�2

I.


In EvoS 3, the variances and valid ranges of�2fm

, �2fc

and�2I

were chosen in the aimof achieving a similar results to EvoS 2, however there was still a lot of guessworkinvolved. The values chosen are shown in table 4.2.

4.3.3 Results

EvoS 3 produced very similar results to EvoS 2. The sounds produced were still quiteunpleasant, but the user did have some control of the evolution.

It seems that evolving the variances did not achieve anything. This is illustratedby the figures 4.3 and 4.4 which show a typical evolution session using EvoS 3. As


Figure 4.3: Evolution of synthesis parameters in EvoS 3.

Figure 4.4: Evolution of variances in EvoS 3.


before, each cross represents a parent that the user has chosen. The evolution starts atwhere there is no cross and ends at the asterisk.

Figure 4.3 shows the evolution of the synthesis parameters with the small clustersthat were illustrated in EvoS 2. It was concluded that these clusters were evidence ofcontrolled evolution, the user had found an area in the sound space that was pleasantand stayed there. Figure 4.4 shows the variance parameters of the genome. It is clearthat there is no convergence on any area in this space, it is a wild mess. Togetherfigures 4.3 and 4.4 demonstrate that while the user can navigate the sound space ade-quately, they wildly flop around the variance space.

One reason for this might be that the fitness function doesn’t stay stable for longenough for the variances to converge on any point. For instance, if a freak mutantwhose large variances’ places it in an interesting area of the sound space, the humanuser will choose it, instead of sticking to the goal they were following before. Clearlya human fitness function is not as stable as an objective assessment of fitness.

Another reason could be the small number of individuals that can be tested bythe human user. In a proper Evolution Strategy with a computer evaluated fitnessfunction, thousands more individuals can be evaluated in a shorter amount of time.Subsequently, convergence in the variances may be observed whereas in EvoS 3, itwas absent.

4.3.4 Conclusion

EvoS 3 demonstrated that evolving the variances of synthesis parameters does notoffer any advantage over fixed variances.

4.4 EvoS 4

4.4.1 Aim

Since automatically choosing variances seemed like a dead end in EvoS 3, EvoS 4aimed to begin addressing the problem of creating more interesting sounds.

Generation of more musically useful sounds using FM synthesis requires the useof ‘envelopes.’ Envelopes allow the values of synthesis parameters to vary over theduration of a sound. For example in EvoS 2 and EvoS 3 the modulation index pa-rameter (I) remained constant throughout the course of the sound. A much moreinteresting result would occur if the value ofI was varied as the sound proceeded.This can be accomplished by using an envelope to control the value ofI.


The simplest kind of envelope is one that controls the amplitude (volume) of asound. This situation is depicted in figure 4.5.

Figure 4.5: An envelope (top) controls the amplitude of a waveform (bottom).

To use envelopes in an Evolution Strategy, you have to be able to mutate them - thisis not as easy as it appears. In order to isolate the problem of mutating envelopes, EvoS4 aimed to implement an instrument thatonly evolved amplitude envelopes. Thisseems like a step backwards in the journey toward interesting sounds, as an amplitudeenvelope modifying a sine wave produces more boring results than the bleeps of EvoS2. However, this intermediate step was necessary in order to effectively advance.

Another goal of EvoS 4 was to start isolating the timbre of sound from othermusical characteristics7. One of the problems of EvoS3 was that as well as timbrebeing subject to evolution, pitch was also being evolved. It would be desirable toseparate pitch from the evolution process. A composer would then be able to workout a melody, and then use the evolution system to find an appropriate timbre for thatmelody.

Finally, EvoS4 addressed another shortcoming of the previous versions. Recallthatinit was called to generate a random parent to start off the evolution process.If the user did not like this initial sound, they had to quit the system and start again.

7Timbre is the characteristic tone quality of a particular instrument. See section 2.2.1 for moreinformation.


This was the problem of the small initial gene pool. EvoS4 would allow the user torandomize the genome from within the system. This would allow the user to quicklyget to other parts of the sound space if they had become disinterested in the currentarea.


Representing Envelopes

Representing an envelope with a genome, requires the following parameters:

n The number of ‘breakpoints’ in the envelope.

xi The point in the sound where each breakpoint occurs.

yi The amplitude value at each breakpoint.

So an individual�!a has the following form:

�!a = (n; x1; : : : ; xn; y1; : : : ; yn) where xi; yi 2 [0::1]

For example, the envelope in figure 4.5 would be represented as:

�!a = (3; 0:166; 0:333; 0:833; 1; 0:75; 0:625)

Notice here that only 3 breakpoints are specified, the endpoints(0; 0) and(1; 0) areadded implicitly.

Mutating Envelopes

Once we have a representation for envelopes, we can work out a strategy that mutatesthem. This might be pretty straight forward, except for a few considerations:

� If we change the number of envelope points (n), we have to decide which pointsto remove or add. This can be done by randomly picking one (or more) of thebreakpoints and deleting or copying it. Also note thatn is an integer and so if itis mutated with a normal distribution, it must be rounded back to an integer.

� If we change thexi’s we must make sure that they remain monotonic, other-wise we might end up with an envelope that runs backwards in time! We canmake sure of this by sorting thexi’s in increasing order after we have made anychanges.


� If we change theyi’s we should scale them so that the largestyi has a value of1. This ensures that synthesized sound is played with maximum volume8.

With these constraints in mind, the mutation procedure can be described in thefollowing steps:

1. Mutaten, the number of envelope points. Any change inn occurs fairly infre-quently so we set the variance quite low. Also if we do changen, we must makesure it is still an integer.

2. For eachxi, mutate with a normal distribution in the same way as in EvoS2.Once thexi’s have been mutated, sort them to make sure they are monotonicallyincreasing.

3. For eachyi, mutate as for thexi’s. Once they have been mutated, scale thevalues so the largestyi is 1.

These steps were implemented in themutate subroutine.


What about the variances and ranges for the parameters in the above mutation proce-dure?

In this experiment, to make things easier, mutation of the number of breakpointsn was disabled. Envelopes always had 3 breakpoints. This means the focus was onfinding a good variance for thexi’s andyi’s. Note that bothxi’s andyi’s are in therange [0..1] so they will have the same variance. A few different values of variancewere trialled, and the results of these are documented in the next section.

Randomizing Envelopes

As mentioned in the aim, EvoS4 let the user randomize envelopes without havingto quit and start again. This was simply done be providing an option that replacedthe current child with one generated by theinit procedure.init also had to bemodified to deal with the envelope genome. It used a very similar strategy tomutateexcept that uniform distributions were used instead of normal distributions.

8If we wanted sounds to play with different volumes, we would have overall-volume as a parameterin the genome. Here, we are only interested in the relative values of envelope breakpoints.


Isolating Timbre

A neweval subroutine was implemented that allowed any sequence of notes to beplayed with the evolved parameters. This allowed a user to enter a melody that wouldbe heard each time with a new mutant timbre.

4.4.3 Results

The envelope mutation procedure seemed quite successful, figure 4.6 shows a parentenvelope and its mutant child.

Figure 4.6: The envelope of figure 4.5 shown with an envelope produced by mutation.

The most interesting result observed was the difference between mutation andrandomization. Randomization produced envelopes that were completely unrelated tothe parent sound (see figure 4.7), whereas mutation produced slightly different ones(compare figures 4.6 and 4.7).

The degree of similarity of mutated envelopes was controlled by the variance usedfor mutation of thexi’s andyi’s. Being able to compare the results of mutation andrandomization provided a useful metric for tuning this variance. If the variance wastoo small, no difference would be heard between the parent and the child and the user


Figure 4.7: The envelope of 4.5 compared with an envelope produced by randomiza-tion.

resorted to using randomization to search the space of possible sounds. If the variancewas too large, there seemed to be no difference between randomizing an individualand mutating it, both produced equally dissimilar sounds. However, if the variancewas just right, mutation would produce a sound that was satisfyingly different yetclearly related to the parent sound. It was here that useful evolution really took place.The user can feel that they are trying different directions in the sound space and thenchoosing the one that sounds best. If this area of the space proves uninteresting theycan jump to an entirely new area by randomizing.

This idea is illustrated in the following figures which show the use of differentvariances for mutation. Each of the figures plots how thexi’s (top graph) andyi’s(bottom graph) changed as the user evolved a sound. Each cross represents a parentthat the user has chosen. A solid line joining two parents means mutation was used tomake this jump, whereas a dotted line means randomization was used. The evolutionrun starts at the asterisk and ends at the circle. Note that the number of breakpoints(n) is always 3 in these graphs, that is,n was not subject to mutation. This restrictionwas necessary in order to effectively visualize the results.

Figure 4.8 shows an evolution where the variance was set at 0.02 (1

50of the param-

eter range). You can see the very tightly bunched clusters of points separated by dottedlines. These represent groups of mutants separated by randomizations. The mutants


Figure 4.8: EvoS 4 evolution with a variance of1

50.

are so similar to each other that you can’t hear (or see) the difference between them.This variance is too small.

Likewise, figure 4.9 shows an evolution where the variance was set at 0.1 (1

10of

the parameter range). Here, the distance between mutations (solid lines) and random-izations (dotted lines) is almost the same. It is difficult to tell the difference betweena mutant sound and a random one. As a result, randomization seems just as effectivefor exploring the space as mutation. This variance is too large.

Finally, figure 4.10 shows an evolution where the variance was set at 0.05 (1

20of

the parameter range). In this figure you can see that mutations are separated by a smallyet still visible distance (as opposed to figure 4.8). Randomizations traverse a muchlarger distance than mutations (as opposed to figure 4.9). This allows other parts ofthe space to be explored that would take too long to reach via mutation alone. Thisvariance is just right.

Table 4.3 summarises these results at a glance.



10.


20.


Variance Normalised Shown in... ObservationsVariancea

0.02 1/50 Figure 4.8 Too small, no audible differencebetween parent and child.

0.1 1/10 Figure 4.9 Too large, mutation no different torandomization.

0.05 1/20 Figure 4.10 Just right! Useful evolution ob-tained.

aNormalised variance is the variance expressed as a fraction of the parameter range

Table 4.3: Effects of different variances in EvoS 4.

Isolating Timbre

The melody playing feature worked well. It isolated the timbre of the sound fromother musical characteristics such as pitch and duration. However, it was too slow tobe of much use. The more notes that the melody contained, the longer MatLab tookto produce the accompanying sound. It was so slow in fact, that the only viable wayof using the system was to just have one note programmed in the melody.

By the way, the sounds produced by EvoS 4 were quite boring. As it was onlya volume envelope that was being evolved, you really had to know what you werelistening for to tell the difference between sounds.

4.4.4 Conclusion

EvoS 4 demonstrated a successful system for evolving simple envelopes. It was foundthat a variance of one twentieth of the parameter range facilitated useful evolution.

4.5 EvoS 5

4.5.1 Aim

EvoS5 planned to advance the complexity the evolution instrument a step further.Instead of applying an envelope to the volume of a tone, an envelope was applied tothe modulation index (I) input of an FM instrument. When this was done, two extraparameters were included to control the range that this envelope varied the modulationindex over. These are calledI1 andI2.


For example ifI1 = 0 andI2 = 5 and we have an envelope as shown in figure4.5, then the modulation index will start out as 0, rise quickly to 5 at the maximumamplitude and then decay slowly back down to 0. IfI1 = 5 andI2 = 0 then the envelopewill be flipped. The modulation index will start at 5, drop quickly to zero and thenrise slowly back to 5.

Because the modulation index is a control of the spectral density of the resultingsound, quite interesting and lively sounds can be produced merely by changing thevalues ofI1 andI2.

In EvoS 4, an effective method for evolving envelopes was developed. EvoS5aimed to develop an effective method for evolving the parametersI1 andI2.


Representation

In order to concentrate on the parametersI1 andI2, all other aspects of the sound werekept constant. The amplitude envelope, the modulation index envelope and the ratioof fm to fc remained unchanged. So the genome for an individual in EvoS5 was verysimple:

�!a = (I1; I2) where I1; I2 2 [0::20]

Mutation

I1 andI2 were very easy parameters to mutate. Their effective range was [0..20] and itdidn’t matter if their values were real or integer. Consequently the mutation procedure(mutate) just involved adding a small amount of normally distributed noise as usual.The only consideration is the variance of the normal distribution. The next sectionshows how an effective variance was obtained.

Changinginit to provide a randomization procedure was equally simple. It justinvolved sampling a uniform distribution with the range [0..20].

4.5.3 Results

In EvoS5, the most interesting result was once again the contrast between the mutationand randomization procedures. This was illustrated particularly well as the space wasonly two dimensional. Different variances were trialled for the mutation procedureand you can really see their effect on the corresponding graphs. The graphs show how


the values ofI1 andI2 changed while a user was searching for interesting sounds. Itwas interesting to note that the results were very similar to EvoS4, that is a varianceof 1/20 (one twentieth) the parameter range led to the most successful evolution. Thenext few paragraphs document the results of the trials conducted.

Figure 4.11 shows a typical user’s course through the parameter space with a vari-ance of 0.4 (1

50of the parameter range).

Figure 4.11: EvoS 5 evolution with a variance of 1/50.

As with the pictures in EvoS 4, each cross represents a parent that the user haschosen. The evolution starts on the left of the figure and ends at the cross with thecircle. Mutations are represented by solid lines joining crosses and randomizationsare a dotted line between crosses.

Notice how tightly the clusters of points are bunched. This is because the smallvariance doesn’t allow mutants to occur very far away from the parent sound. Asa result mutants always soundvery similar to the parent. In fact with a varianceof 1/50 there was no discernible difference between a parent and its mutant child.Consequently, this is not very useful for evolving sounds – the user can never makeany progress as all mutant children sound exactly the same. Also note how theseclusters of points are widely spaced. These large jumps are caused when the userdecides to randomize the parameters.

Figure 4.12 shows a user’s path with a variance of 2 (1

10of the parameter range).


You can immediately see the contrast to figure 4.11. The clusters of points are no


longer tightly bunched and it is difficult to tell a mutation from a randomization. Whatthe user finds when listening to the sounds is that each mutant child sounds quitedifferent from the parent – so different that the user can’t tell how they are related. Asa result, using randomization seems just as effective for getting good sounds as usingmutation.

Figure 4.13 shows a user’s path, this time with a variance of 1/20 (1

20of the pa-

rameter range). Here is a balance between the extremes of figures 4.11 and 4.12.The mutant children are spaced widely enough from the parent that the user can hearthe difference in the sound, yet they are close enough that the user can hear how thesound is related to the parent. Consequently when the user hears something they like,they can follow that direction and obtain a pleasing sound. Instead of getting ran-dom clusters of points you get ‘lines’ of points that correspond to the user followinga particular direction in the sound space. Note also in figure 4.13 that you can see thedifference between mutation steps and randomization steps. It is not as marked as itwas in figure 4.11 but it is still visible.

Table 4.4 sums up at a glance the discussion of the previous paragraphs.

A final point is that the sounds produced were now a bit more interesting. Insteadof just a volume envelope modifying a sine wave (as was the case in EvoS 4), the



sounds had characteristics of trumpets and oboes.

4.5.4 Conclusion

EvoS 5 demonstrated an effective system for evolving the parametersI1 andI2. Itwas found that a variance of 1/20 the parameter range resulted in successful evolution,which was the same result obtained in EvoS 4. Could this hold true as a general rule9,or is this a little early to be hypothesizing?

4.6 EvoS 6

4.6.1 Aim

In EvoS4 and EvoS5 effective methods for evolving envelopes and modulation indiceswere discovered. With these we could build an FM instrument with an amplitudeenvelope and a modulation index envelope which would be able to produce a fair range

9The “1/20 rule”?


Variance Normalised Shown in... ObservationsVariancea

0.2 1/50 Figure 4.11 Too small, no audible dif-ference between parentand child..

2 1/10 Figure 4.12 Too large, mutation nodifferent to randomiza-tion.

1 1/20 Figure 4.13 Just right! Useful evolu-tion obtained.

aNormalised variance is the variance expressed as a fraction of the parameter range

Table 4.4: Effects of different variances in EvoS 5

of sounds. However, Chowning’s FM instrument [Cho73] has one more parameterthat permits synthesis of a very wide range timbres - the ratio of carrier frequency tomodulating frequency:

fc

fm=

N1

N2

EvoS6 aimed to discover an effective method of evolving this ratio.

The ratio offc to fm was actually being evolved in EvoS2 and EvoS3 as wheneverthe values offc or fm were mutated, the ratio was changed. This form of mutation isundesirable because as stated before, we want our system to evolve timbres that areindependent of pitch. The pitch of a note is usually determined by its fundamentalfrequencyf0. In FM synthesis,f0 is given by the following equation:

f0 =fc

N1

=fm

N2

where N1; N2 are integers with no common factors.

Clearly, if fc andfm are mutated arbitrarily as they were in EvoS2 thenf0 willchange with each mutation. To see how this would be undesirable, consider the fol-lowing example:

John is a composer who has just come up with a brilliant melody. It isvery simple, consisting of two notes: middle C followed by the A abovethis. John worked out the melody on his piano, but he now wants to playit with a ‘space-age brassy’ kind of instrument. I suggested he use EvoSto find his perfect sound. Taking my advice, John uses EvoS and finds asound that he really likes. However, when he uses the parameter settingsto play his melody, he finds it is out of tune with the piano part! Angered


by the time he has wasted, John goes to a traditional synthesizer where heknows if he presses the middle C key, that is the note he will hear. Aftermany days of twiddling knobs he doesn’t understand, John finally settlesfor a sound that is not what he wanted - a very second rate ‘space-agebrassy’ timbre. However, he is consoled by the fact that when he playshis melody it will be in tune with whatever other parts he has worked outfor his composition.

Although the above example is fictional, we would like to avoid anything like thishappening in real life. For this to be the case, we need make sure the fundamentalfrequencyf0 stays constant after mutation and even randomization so only the timbreof the sound is changed.

To do this we simply mutate the parametersN1 andN2. When it comes time tosynthesize the sound, we work outfc andfm according to the pitch (f0) that the userwants to hear. That is:

fc = f0N1 and fc = f0N2

The problem is that mutation ofN1 andN2 is not straight forward. IfN1 andN2 are integers, the FM sound produced will be ‘harmonic’. That is, each frequencypresent in the spectrum of the sound is a whole number multiple of the fundamentalfrequency. The human ear is used to hearing harmonic sounds in string and windinstruments, harmonic sounds are ‘nice’. IfN1 andN2 are not integers then the FMsound produced is ‘inharmonic’. Examples of inharmonic sounds are drums and bells– they are usually ‘harsh’.


An EvoS6 individual is simply represented as:

�!a = (N1; N2)

In order to implement an effective procedure for mutatingN1 andN2 we haveto distinguish between the two domains of harmonic and inharmonic sounds. Sincemutation should produce sounds that are similar to the parent sound yet different, thefollowing guidelines should be followed:

� When the parent sound is harmonic, most mutant children should be harmonictoo, since an inharmonic sound is very different from a harmonic sound.


� When the parent sound is harmonic, mutant children that are harmonic shouldonly occasionally possess a different value ofN1 orN2 from their parents. Thisis because within the harmonic domain, different ratios ofN1 andN2 producequite different timbres.

� When the parent sound is harmonic, there is a very small chance that a mutantchild could be a closely related inharmonic sound.

� When the parent sound is harmonic, and the values ofN1 andN2 have been mu-tated, all common factors of these numbers should be eliminated. This ensuresthat you never get the situation of a child timbre being the same as the parenttimbre but just transposed by a certain amount. For example ifN1 = 2 andN2

= 2, the resulting sound has the same timbre as whenN1 = N2 = 1, but it is justtransposed up an octave.

� When a sound is inharmonicN1 andN2 may not be integers. In this case,mutation is straight forward: a small normally distributed random variable isadded to the parent settings. Of course, there is a very small chance that bothN1 andN2 end up as integers after mutation. If this happens, the rules forharmonic mutation apply subsequently.

Themutate subroutine was modified to accommodate the above guidelines.

Theinit subroutine which is used for initialisation of individuals and the ran-domization feature gave the individual an equal chance of being harmonic or inhar-monic.

4.6.3 Results

After a little fine tuning,N1 andN2 were being evolved successfully. If you startedwith a harmonic sound, the mutant children would generally be the same, only occa-sionally changing to another harmonic sound. This effect was achieved by having aharmonic variance10 that gave about 1 chance in 8 of changing the values ofN1 andN2. Starting with an inharmonic sound, each mutant child was different yet clearlyrelated to the parent. This was achieved by having an inharmonic variance11 set to 1

20

the range. The useful range ofN1 andN2 was found to be [1..10].

10The variance whenN1 andN2 are integers.11The variance ofN1 andN2 when they are not integers.


4.6.4 Conclusion

EvoS6 demonstrated an effective system for evolving the ratio of carrier frequency tomodulating frequency. The parametersN1 andN2 are not trivial to mutate as they arediscontinuous in the timbral space. It is interesting to note how the complexity of themutation procedure blossoms in an attempt to provide a continuous timbral space.

4.7 EvoS 7

4.7.1 Aim

EvoS 7 finally put all the pieces together. It implemented an evolution strategy for thecomplete FM instrument described in [Cho73]. To describe a particular timbre withthis instrument, you had to specify a separate envelope for amplitude and modulationindex, the modulation index range and the ratio of carrier to modulation frequency.

Since this instrument was finally capable of producing semi-decent sounds, it wasdecided to measure its success by gathering the opinions of a few users. The usersfirst assessed Arno van Goch’s P-farm [Goc96], then used EvoS 7. After this theywere asked to comment on the merits and faults of each system.


Implementation of EvoS 7 consisted of just tacking together all the previous versionsof EvoS and making them work in harmony.

Representation

As in previous versions, an individual is represented by a vector of real numbers. Thegenome in EvoS 7 was the most complicated so far:

�!a = (nI ; xI:1; : : : ; xI:nI ; yI:1; : : : ; yI:nI ; I1; I2; N1; N2;

nV ; xV :1; : : : ; xV :nV; yV :1; : : : ; yV :nV

)

The genome starts with the modulation index envelope. This is specified in the sameway as the amplitude envelopes were in EvoS4:nI is the number of points in theenvelope and(xI:i; yI:i) are the coordinates of the breakpoints. The modulation indexenvelope is followed byI1 and I2 specifying the range that this envelope operatesover (as in EvoS5).N1 andN2 specify the ratio of carrier to modulator frequency as


in EvoS6. Finally the amplitude envelope is specified, also following the conventionadopted in EvoS4:nV specifies the number of points in the envelope and(xV :i; yV :i)are the coordinates of the breakpoints.

Of course theeval procedure was modified to accept individuals of this form. Ithad to be able to interpret each of the parameters in order to synthesize the sound andplay it to the user.

Mutation and Randomization

Mutation and Randomization functions are accomplished using the techniques de-scribed in EvoS4 to EvoS6.

Themutate subroutine takes an individual and splits it up, applying the mutationschemes of previous versions to each section. The amplitude and modulation indexenvelopes are mutated using the technique described in EvoS4. The modulation indexrange (I1 andI2) is mutated as described in EvoS5.N1 andN2 are mutated using thetechnique of EvoS6.

A similar scheme is used to achieve the randomization feature through theinitsubroutine.

User Interface

The user interface used for EvoS 7 is still the same simple keyboard interface that wasused in EvoS 2 (refer back to figure 4.1).

4.7.3 Results

To illustrate the effectiveness of EvoS 7, figures 4.14 to 4.17 show an example of anevolution run. The figures show how each parameter in the EvoS 7 genome changedover the course of the evolution.

First look at figure 4.14 which shows howI1 and I2 change. Parents that theuser has chosen are represented by a number - the generation. The evolution startsat generation 1 and ends at generation 15. A solid line joining two points meansmutation was used to make this step, whereas a dotted line implies randomization.

Looking at figure 4.14 you can see that mutation was used from generations 1 to5, then a random jump occurred between 5 and 6. Again, mutation is used from 6 to10 followed by another random jump from 10 to 11. Finally, generations 11 to 15 aretraversed via mutation. Figure 4.14 is quite an exquisite example of evolution. The


Figure 4.14: EvoS 7 evolution: Modulation indices (I1 andI2).

sections of mutation exhibit a strong sense of direction, implying that the user couldcontrol the search through the sound space. This is contrasted by the large leaps ofrandomization which rapidly transport the user to parts of the sound space that wouldby unreachable via mutation alone.

Now look at figure 4.15 showing howN1 andN2 changed over the course of theevolution.

In generations 1 to 5 we note that the sound is inharmonic12. A small path isfollowed (via mutation) until the random jump occurs between generations 5 and 6.This jump lands the user on a harmonic sound13 where it stays until generation 11.Recall from EvoS 6 that when the sound is harmonic, you don’t want it to changeoften because the harmonic sounds will be quite different from each other. This iswhy there is no movement from generation 6 to 10. The random jump to generation11 also results in a harmonic sound, but from 11 to 12 and 12 to 13 there are somechanges via mutation. Notice that the sound is still harmonic - this illustrates how theharmonic and inharmonic spaces are disjoint under mutation. Finally the evolutioncomes to rest from generation 13 to 15.

Finally, examine figures 4.16 and 4.17 which show the amplitude and modulation

12Remember, an inharmonic sound results when eitherN1 orN2 is not a integer.13Harmonic sounds result whenN1 andN2 are integers.


Figure 4.15: EvoS 7 evolution: Frequency ratios (N1 andN2).

index envelopes respectively.

Although it is hard to pick out much detail in these figures, you can clearly seethe contrast between mutation and randomization. In both figures, the envelope shapechanges gradually from generation 1 to 5 as it is mutated. At generation 6 the shapeis quite different due to the randomization step. From generation 6 to 10, mutationchanges the shape gradually again until the random jump at generation 11. Note againthe marked difference in shapes between generation 10 and 11. Finally in generations11 to 15, the shape is changed gradually via mutation.

Note that each of these figures leave out a lot of information. For each generation,only the chosen parent is shown - you don’t see the many children that may havebeen auditioned and discarded. Imagine what figure 4.14 would look like if you did:around each parent point you would see a cluster of sounds that user had auditioned,only one of these (the one you can see on the graph) would be joined by a line tothe parent. The point being made is that it is not a fluke that the first five generationsin figure 4.14 form a line going vertically down - the user has tried other directionsbut decided ‘down’ is what they like the sound of best. If you appreciate that at thesame time, around 14 other parameters are being mutated which also exhibit similartrends14, then you can see that EvoS 7 (and artificial evolution in general) is reallyquite an effective tool for searching this large, multi-dimensional space.

14Refer back to figures 4.15 – 4.17


Figure 4.16: EvoS 7 evolution: Amplitude Envelopes.

Figure 4.17: EvoS 7 evolution: Modulation Index Envelopes.


4.7.4 User evaluation

To further asses the utility of EvoS 7 a number of people spent some time using bothEvoS 7 and Arno van Goch’s P-Farm [Goc96]. Since P-Farm uses Roland’s Juno-106synthesizer this allowed extra comparisons. The Juno allows you to manually adjustthe sliders that control its parameters, so this method could be compared with evo-lution using P-Farm. Also, P-Farm can generate a population of completely randompatches. Once again, this random method of finding good sounds could be comparedwith evolution using P-Farm.

The users (Brett, Emma and Nick) have had little musical training, however theyall have a strong interest in electronic music and enjoy listening to ‘off-the-wall’sounds and timbres. Each user spent about an hour experimenting with the systems,the next sections summarise their experiences and opinions.

Brett

Brett found using P-Farm was easier than manually moving the sliders on the Juno.After ten generations he had found a few sounds which he was reasonably happywith. Brett thought it was very hard keeping track of the sounds that he liked. P-Farmpresents a population of 32 sounds to the user who has to audition them all to find theones they like most. Brett felt if there had of been a quick way to note sounds that youliked (for example, a fitness rating of 1 to 10) it would have been much easier to keeptrack of which sounds were good.

Brett thought that using EvoS 7 was a lot easier than P-Farm. In EvoS 7 youare only presented with one sound at a time, and you have to make a decision as towhether it is better or worse than the current parent sound. Brett felt that this providedless distractions than P-Farm where you are constantly thinking about the 31 othersounds and whether they are better or worse than the current one. Brett noticed thatEvoS 7 was slower than P-Farm when generating sounds for auditioning. P-Farm hasa real-time response – as soon as you click on an individual you can play a tune withit on the keyboard. EvoS 7 however, takes a little while to generate the melody usedfor auditioning. The longer this melody is, the longer the delay. Finally, Brett did notlike the fact that you couldn’t save in EvoS 7. If you got a sound you were reasonablyhappy with, you were too afraid to explore anywhere else as you may lose the soundyou worked so hard for. To overcome this, Brett suggested a ‘stud-farm’ idea. As yougo along exploring the sound space, if you find a sound you like, you can add it toyour ‘stud-farm’. Sounds stored in the stud-farm can be retrieved at any time, so youno longer have to worry about exploring further and losing a good sound. Also, after awhile the stud-farm will contain a portfolio of your favourite sounds. You could matethese favourites together (just like in a real stud-farm) to see if anything interesting


happens.

Emma

Emma used P-Farm for 8 generations and found two sounds that she was happy with.She found evolution with P-Farm was much more intuitive than just moving the sliderson the Juno without knowing what they did. A very interesting result however is thatout of 32 completely random patches (one random P-Farm population) she found 4sounds that she liked even more than the 2 she took 8 generations to evolve.

When using EvoS 7 Emma thought that all the sounds it made were a lot harsherthan the ‘nice’ sounds created with P-Farm. She felt that in order to use EvoS 7effectively you have to know what kind of sound you are looking for to begin with.Emma also thought it would be good to be able to keep and go back to any soundsthat you liked, but she also suggested an ‘undo’ feature. Sometimes when you areauditioning sounds very quickly, you make a mistake and accidently destroy a childsound that you wanted to keep. It would be good to be able to backtrack a few stepsand rectify your mistake. Emma thought that to make an evolutionary sound systemreally useful you must be able to mutatesamples. For example if I’m listening to asong and I hear a trumpet sound that I really like – I should be able to sample thissound and then mutate it into something similar yet different. Emma thought shewould also like to take samples of all her favourite sounds, then breed these soundstogether to make new and interesting sounds.

Nick

Nick thought that using P-Farm was much easier than adjusting the Juno’s slidersmanually. He also found P-Farm evolution easier than randomly generating patches.Nick used P-Farm for 10 generations to evolve an “organ” type sound.

Nick found using EvoS 7 much easier than P-Farm. He felt that EvoS 7 gaveyou more control over where the sound was going – you could see trends that wereoccurring. Although sometimes progress was a bit slow, he felt eventually you wouldget closer to the sound you desire. On the downside, Nick thought it was frustratingthat you can sometimes go ‘backwards’, away from the sound you like. When you dothis, there is no going back. Also, sometimes when you are being cautious you canmiss an opportunity to get closer to your desired sound. Both these problems could befixed with an ‘undo’ feature that allows the user to backtrack a number of steps andpick up where they left off. Nick disliked the slow speed of EvoS 7 and would havepreferred more notes to test the sounds on. Finally Nick suggested a feature whereyou specify areas of ‘like’ and ‘dislike’. If you were in a bad spot in the sound space,you could tell the system and it would steer you away from that area in future. This


would stop you going round in circles, constantly encountering the same bad area ofthe sound space.

4.7.5 Conclusion

A number of conclusions can be drawn from analyzing the user’s comments:

� P-Farm demonstrated that evolution is more intuitive than adjusting synthesisparameters manually. This is especially the case when the user does not knowhow each parameter effects the overall sound.

� The user interface of EvoS is more effective than that of P-Farm. It seemspresenting the user with one sound at a time provides less distraction than whenthey are faced with 32 sounds. The user interface of EvoS enables the user tohear how each mutant child is different from its parent.

� The sounds created by P-Farm are generally more pleasing than those of EvoS.This may be a result of the different synthesis models in use. P-Farm utilises aRoland Juno-106 which is a subtractive synthesizer while EvoS uses FM syn-thesis. Also, the Juno is a well designed instrument. The ranges on all theparameters have been set very carefully. As a result, it is very hard to get abad sound out of the Juno. This is why Emma had just as much success withrandomization as she did with evolution using P-farm. It seems that in orderto get better sounds out of EvoS, it might be wise to experiment with differentsynthesis models.

� EvoS was much slower than P-Farm in generating sounds. Unfortunately, thisfault in EvoS is not so easy to rectify. The way to improve speed would beto use a hardware synthesizer (like the Juno) that provides real time response.However a fair amount of driver software is required to interface any computerprogram with the synthesizer. Due to time constraints, this will not be possible.

� EvoS most certainly needs a ‘save’ feature. This would allow any parent orchild sound to be saved so the user could return to it at any time. This featuremight be extended to implement the ‘stud-farm’ that Brett suggested. Here,sounds that you have collected can be bred together to make new ones.

� EvoS also needs an ‘undo’ feature. This would enable the user to backtrackthrough a number of steps and rectify a mistake.

Overall, EvoS 7 performed surprisingly well when put to the test against P-Farm.If the suggested features are implemented EvoS will become an even more effectivetool for interactive evolution of sounds.


So what’s next?

Development of the artificial evolution system for this project was basically complete.What remained was to implement the users suggestions and then extensively test thesystem to measure its effectiveness.

Emma’s suggestion of evolving samples was explored, and this is documented inAppendix A. Alas, this avenue did not prove successful. Instead, the user suggestedimprovements were implemented in the next version of EvoS – it was this system thatwas used as a basis of the experiments in the next chapter.

Chapter 5

Results

5.1 Aim

Chapter 4 described the development of EvoS - a system that could evolve sounds.The aim was now to measure the effectiveness of that system. This chapter describesexactly how the effectiveness of the EvoS system was measured and what results wereachieved.

How can you measure the effectiveness of a system that relies so much on subjec-tive opinion? This was a tricky problem which I thought about for a long time.

In the case of an interactive system, at the very least you must acquire some useropinions. After a few different users had tried your system, you would get an ideaof just how good it is. However, this thesis is not concerned with just how ‘fun’or interesting a program is to use, but more importantly, whether or notartificialevolution is a useful tool forsound synthesis.

In order to determine this, a comparative test was made. Users tested the evolutionsystem, but also tested other (more traditional) methods of searching the sound space.Through the results of these tests, we will see not only how intuitive the evolutionsystem was, but also how it rated as a search tool beside other methods.

It was decided the comparison would occur between three search methods:

Manual A manual search corresponds to specifying parameter values exactly andthen listening to the resulting sound. This is akin to manually turning the dialson an analog synthesizer. To do this effectively usually requires knowledgeof the synthesis algorithm in order to predict the effect of changing variousparameters to achieve the sound you want.

70

5. Results 71

Random In a random search, parameters are assigned a completely random valueand the resulting sound is listened to. This is the same as randomly spinning thedials of an analog synthesizer and hoping that where they land results in a goodsound. It is a lottery - you might win the first time you try it, or you may spendyears trying with no luck.

Evolution This method of searching works via mutation and selection. It was de-scribed in detail in chapter 4

To compare these three search methods, a user interface was required for each ofthem. The next section describes the implementation of these user interfaces. Fol-lowing this, the method of the experiment conducted will be explained and the resultsdiscussed and analysed. Finally, conclusions will be drawn from this analysis.

5.2 Implementation

All of the search methods for comparison were implemented using MatLab. For aneffective comparison, each search method used the same underlying synthesis algo-rithm: the FM synthesis algorithm of EvoS7. Since this was already implementedin MatLab, it seemed an obvious choice. MatLab also provides basic GUI1 facili-ties that are easy, if at times tedious to use. The following subsections describe theimplementation of each search method or tool.

5.2.1 Manual Tool

Figure 5.1 shows a screen shot of the manual search tool user interface.

Look at the left side of figure 5.1 and recall the genome of an EvoS7 individual2:

�!a = (nI ; xI:1; : : : ; xI:nI ; yI:1; : : : ; yI:nI ; I1; I2; N1; N2;

nV ; xV :1; : : : ; xV :nV; yV :1; : : : ; yV :nV

)

The manual tool interface simply allows the user to type in a value for any of theseparameters. Considerable work was involved in keeping parameters consistent. Forinstance, when the user changes the number of points in the amplitude envelopenV ,the corresponding vectorsxV :i andyV :i should shrink or grow to reflect this value.

The user can hear the current sound by pressing thePlay button. This causes thecurrent settings of parameters to be read and then synthesized into a sound. Synthesis

1Graphical User Interface.2Refer back to section 4.7.2 for a more detailed explanation of the genome.

5. Results 72

Figure 5.1: Screen shot of the manual tool user interface.

is achieved easily by using the existingeval subroutine from EvoS7. A small graphof the amplitude and modulation index envelopes is displayed beside the parametersettings. This assists the user in achieving the desired envelope. Also assisting the useris the status bar. This is the line of text at the bottom of the screen. It is frequentlyupdated to notify the user of program’s status. The status bar has been implementedin all of the search tools.

Figure 5.1 illustrates another feature common to all three search tools. Down theright-hand side is thePatch Store. This is an implementation of the “stud farm”idea3. By typing in a name4 and pressingSave As:, the current parameter settingswill be saved in thePatch Store. These can be recalled later by selecting the nameof the patch you want5 and pressingLoad.

When you have a number of parameter settings recorded in yourPatch Store youwill want to save them all to disk. This can be accomplished by using theSave File...button. Conversely, to load a previously createdPatch Store file you can use theOpen File... button.

Finally, when the user is finished using the manual search tool they can hitQuit

3The stud farm idea was suggested by Brett - see section 4.7.4.4For example, “Chowning 1” in figure 5.1.5For example, “Trumpet” in figure 5.1.

5. Results 73

and exit the program.

Searching the sound space using the manual tool requires the user to have knowl-edge of how each parameter effects the sound - or at least have a feel for it. Theuser listens to the current sound and tries to guess which parameter needs adjustingto achieve the desired sound. The adjustment is made and the user listens again. Ifthe resulting sound is further away from the desired sound, the change made to theparameter is reversed. Other changes are auditioned until a step closer is made. If theresulting sound is closer to the desired sound, other parameters may be ‘tweaked’ tobring it even closer. When the user is satisfied, the sound is saved in thePatch Storefor later reference.

5.2.2 Random Tool

The random search tool is much simpler than the manual tool. A screen shot of theuser interface is shown in figure 5.2.

Figure 5.2: Screen shot of the random tool user interface

The two buttons in the center are all that is required. PressingRandomize gener-ates a new patch using totally arbitrary parameter settings. The user no longer has totediously enter all the parameters as they did in the manual tool. The downside is thatthere is absolutely no control over what the parameters will be. This randomization

5. Results 74

is implemented by using the existinginit subroutine from EvoS7. When called,init simply returns an EvoS7 genome with completely random contents. TheRe-play button simply plays the current randomized sound and is implemented using theeval subroutine from EvoS7.

Looking at the right hand side of figure 5.2 you can once again see thePatchStore. This performs exactly the same function as in manual tool. If the user hearsany sounds they like, they can save them in thePatch Store for later reference. Notethat the file formats used in all the search tools are compatible. Thus if a user found anice sound in the random tool but wanted to ‘tweak’ it a little, they could save it in afile, load it up in the manual tool and there adjust the parameters manually to get thedesired effect.

Searching using the random tool is pretty straightforward. The user simply pressesRandomize until an interesting sound is heard. The interesting sound can be saved inthePatch Store. To continue the search, the user presses randomize again. When theuser is finished searching the sound space, they can save all their interesting patchesin a file (usingSave File..) and exit the program by pressingQuit.

5.2.3 Evolution Tool

Last, but not least is the evolution search tool - shown in figure 5.3.

The evolution tool is a GUI that replaces EvoS7’s keyboard interface. Lookingat the left side of figure 5.3 you can see theParent box. This box is divided in twohalves. One half is thePlay button which plays the current parent sound6. The otherhalf is aPatch Store allowing the current parent sound to be saved or previous soundsto be loaded.

Below theParent is theChild box. This is also divided into two halves. The lefthalf allows you to select a particular child and play it, the right half is aPatch Storeallowing you to save the current child sound. As in EvoS7, children are created usingMutate andRandomize . Mutate mutates the current parent sound using EvoS7’smutate subroutine, andRandomize creates a random child usinginit.

TheReplace button replaces the current parent with the current child. So throughthe three buttonsReplace, Mutate andRandomize you have all the functionalityof the EvoS7 system. Users can search through the sound space by mutating andreplacing. When they are satisfied with any sound they can save it in thePatch Store.

But what about thePartner box and thatMate button?6Playback is once again implemented with EvoS7’seval subroutine.

5. Results 75

Figure 5.3: Screen shot of the evolution tool

Mating Sounds

Mating or recombination has previously been ignored in this study - the main focushas been on mutation. However, it would seem foolish to conduct an investigation ofartificial evolution for sound synthesis without even experimenting with a recombina-tion operator. To this end, it was decided to include one in this final incarnation of thesystem. Unfortunately as time was running out, this mating operator had to be keptextremely simple. The following paragraphs will describe its implementation.

One point crossover

The simplest form of recombination isone point crossover: one point along thegenome is chosen at random, a child is formed by taking the genes of the first parentup to this point, and the genes of the second parent from after this point.

For example, consider two parents�!a and�!

b which will form a child�!c . Thegenomes of the parents look like this:

�!a = (a1; a2; :::; an�1; an)

�!

b = (b1; b2; :::; bn�1; bn)

5. Results 76

A random crossover pointi is chosen wherei 2 [1; n � 1]. Thus the child genomewill look like this:

�!c = (a1; a2; :::; ai�1; ai; bi+1; bi+2; :::bn�1; bn)

But notice this assumes parent�!a goes first. If we decided that parent�!

b should gofirst we would get a different child:

�!c = (b1; b2; :::; bi�1; bi; ai+1; ai+2; :::an�1; an)

So for each crossover pointi there are two possible children. Thus for two parents�!a

and�!

b with n genes each, there are2(n � 1) different children that can be formedwith one point crossover.

Implementing one point crossover

When this scheme of one point crossover is applied to the genomes of EvoS7, somemajor problems occur. One of these is caused by the fact that EvoS7 genomes do nothave a fixed length. For example, imagine parent�!a has a modulation index envelopewith two points (nI = 2) and parent

�!

b has one with ten points (nI = 10). If thecrossover point is chosen to be one (i = 1), then the child will inheritnI = 2 fromparent�!a , and all its following genes from parent

�!b . This will be an invalid child -

nI will specify two envelope points, but there will beten envelope points following it,care of parent

�!b !

OK, so this problem could be fixed up with some fancy coding and a bit of pa-tience, but there were other problems as well7. To avoid having to deal with all these,a much simpler scheme was devised: the genome could only be split at points wherethe results were well defined. This resulted in a ‘meta genome’ that looked like this:

�!a = (Index Envelope; I1; I2;Freq Ratios;Amplitude Envelope)

This scheme worked quite well and solved all of the previously mentioned problems.The downside of the scheme was that in terms of mating, the genome’s length wasdrastically reduced - there are only 4 possible crossover points. This means that forany two parents, there are only2 � 4 = 8 possible children!

Although it would have been desirable to have more combinations available, thecrossover system seemed to work adequately. A MatLab procedurematewas writtenthat implemented the scheme. The procedure takes two parent genomes, randomlychooses a crossover point and randomly chooses which parent will ‘go first’. Theresulting child genome is the return value of the procedure.

7Think about what happens if the crossover point is right in the middle of thex’s andy’s of anenvelope.

5. Results 77

Mating in the evolution tool

Now we can finally explain the mystery parts of figure 5.3. ThePartner box allowsyou to load any sound previously saved in thePatch Store. When theMate buttonis pressed, a child is formed by mating8 the currentParent with the currentPartner.This child can then be subjected to more mating or mutation using the previouslydescribed features of the evolution tool.

When the user is finished searching the sound space via mutation, mating andrandomization, they can save all their acquired sounds with theSave File... button.Once again, this is the same file format used in the random and manual tools - the usercan change their sounds with the other programs by using theOpen File... button.Finally, the user can pressQuit which will exit the program.

5.3 Method

After the three search tools were implemented, it was time to actually conduct theexperiment and get some results. The experiment involved getting users to try each ofthe search tools.

The users first tried the manual tool, then the random tool and finally the evolutiontool. Before they used each tool, detailed instruction was given and the user was takenthrough an example of the tool’s use. Three statistics were recorded for each tool:

Time The time that the user used the tool for. This was usually around 10-15 minutes.

Sounds Auditioned The number of different sounds auditioned. Each time the usermade a change and listened to a new sound, it was noted.

Sounds Liked The number of sounds that the user ‘liked’ out of all the sounds theuser auditioned. This statistic was collected at the end, after all the tools hadbeen used. Along the way, the user was encouraged to save any sounds that wereremotely interesting. After all the tools had been used, all the saved soundsfor each tool were reviewed. The user specified in retrospect which soundsthey really did think were interesting, useful or amusing - the number of thesewas noted. It was necessary to collect this statistic at the end because quitefrequently in the early stages of the trial, the user would save a lot of soundsthey thought were great. However, after using the other tools and seeing whatother sounds they could get out of the algorithm, they would change their minds.

8Using the aforementionedmate subroutine.

5. Results 78

The motivation for these statistics was to see how many good sounds you couldfind with each tool in a certain time. It might be found that with one tool, users finda lot of good sounds quickly, whereas with another, it takes a long time to find a few.For this comparison to be valid, it was important that each tool be used for an equalamount of time. However an ‘equal amount of time’ was not so easy to judge. Forinstance, the manual tool took a long time to get used to. Also when using the manualtool, people usually think a while, make a few deliberate changes and then listen tothe result. These factors result in the user spending a lot of time with the manual tooland only listening to a few sounds. The random tool however is very simple and usersget the idea straight away. Many sounds are auditioned in a very short time. To get agood comparison between these two tools one would like the manual tool to be usedfor longer, just to ensure that the user had ‘the hang of it’, and was not spending allthe time just trying to work out how to use the interface.

As a result, each tool was used until the user got ‘the hang of it’. Although thisdoesn’t seem like a very good metric of time, it proved to be the only viable option.

As well as collecting the three statistics mentioned, after using the search tools,users were asked a question along the lines of:

“If you needed some sounds (say, for a composition you were writing)and you were only able to use this algorithm and the three search tools,what strategy would you use to find the sounds you desired?”

It was hoped by asking this question, further insight could be gained into which toolthe users thought was most effective for searching the sound space. Their answer tothe question and any other comments they had about the three search methods wasduly noted.

The whole process of instructing the users, collecting statistics and asking thequestion took about 1 hour. This is quite a large amount of time to ask volunteers todonate9 - because of this, just seven users were surveyed in total.

Of these seven users, only two of the people surveyed were active composers ofelectronic music - people who have a real need for these kind of tools. The rest of theusers, although having an interest in electronic music, would not have a need for thesearch tools. Although this may seem like a deficiency, it would be good to see if anyof the search methods allows ‘normal’ people to explore the sound space effectively- people who have no knowledge of how the synthesis algorithm works, but simplyknow what sounds nice and what doesn’t.

9Especially considering they were not getting paid!

5. Results 79

5.4 Results

5.4.1 Raw Data

The raw data for the statisticsTime, Sounds AuditionedandSounds Liked is pre-sented first. The results for the manual tool are shown in table 5.1; the random tool intable 5.2; and the evolution tool in table 5.3. Each table shows the data collected fromeach user and also the mean and standard deviation of this data.

User Time Sounds Sounds(minutes) Auditioned Liked

Emma 18 20 3Dougal 21 40 2Rachael 9 25 2Tom 18 21 2Brett 14 24 2Chris 14 25 3Nick 15 20 2Mean� � 15.57� 3.867 25.00� 6.976 2.286� 0.488

Table 5.1: Raw statistics collected for the manual tool



Table 5.2: Raw statistics collected for the random tool

5.4.2 Analysis of Data

Table 5.4 presents a comparison of the previous statistics – it compares the averagesof each statistic for each search tool. It will be useful to analyse each line of this table.

5. Results 80



Table 5.3: Raw statistics collected for the evolution tool

Average Search ToolStatistic Manual Random Evolution

1. Time 15.57� 3.867 6.143� 2.116 15.00� 4.5832. Sounds Auditioned 25.00� 6.976 37.71� 6.396 60.00� 15.573. Sounds Liked 2.286� 0.488 4.286� 1.113 5.571� 1.5124. Liked� Time 0.155� 0.049 0.762� 0.351 0.395� 0.1425. Liked� Auditioned 0.097� 0.032 0.120� 0.048 0.095� 0.0226. Auditioned� Time 1.685� 0.574 6.757� 2.511 4.185� 1.230

Table 5.4: Mean statistics for each search tool.

5. Results 81

In the first line we see that the averageTime spent using the Manual and Evolutiontools was about the same. This implies that the tools were of similar complexity andusers took about 15 minutes to get comfortable with them. The averageTime spentusing the Random tool however was much less - only about 6 minutes. This reflectsthe fact that the Random tool was much simpler than the others and users quicklybecame proficient in it.

The second line of table 5.4 shows the average number ofSounds Auditionedwith each tool. Sounds Auditioned for the Manual tool is quite low because themanual tool required users to think about what they were doing before modifyingparameters. The value for the Random Tool is a little higher - it was easy to auditionsounds with this tool, but users didn’t spend muchTime with it. The Evolution toolhad a much higher average than both of the others as auditioning sounds was easy(you didn’t have to think as much compared to the Manual tool), but users spent a lotof Time getting used to the tool (due to its complexity).

Looking at theSounds Likedstatistic in table 5.4 we see a very positive result forthe Evolution tool. On average, with the Manual tool, users found 2 sounds they liked;with the Random tool, they found 4; however with the Evolution tool they found 5.5sounds that they liked. On the surface, we could conclude from this result that theEvolution tool performed the best and thus Artificial Evolution is the most effectiveway of searching the sound space. But doing this would neglect the facts that eachtool was not used for an equal amount of time; and an unequal number of sounds wereauditioned using each tool. If we analyse someratios of these statistics we can gainfurther insight into the actual meaning of the data.

The fourth line of table 5.4 shows the average ofSounds Likeddivided byTimefor each search tool - this gives us a measure of “Sounds Liked per Minute”. Youcan see that the Random tool scores highest here, with Evolution coming second andManual last. This implies that using the Random tool you can find more sounds in ashorter amount of time. Does this mean that the Random tool is the most efficient forsearching the sound space? This is in contradiction to the conclusion reached fromlooking at theSounds Likedstatistic alone.

Furthermore, look at the fifth line of table 5.4 which shows the ratio ofSoundsLiked to Sounds Auditioned. This could be another measure of efficiency - howmany sounds do you have to audition before you get one you like? Again we seethat the Random tool is the best, with the Manual and Evolution tools being aboutequal. This seems to imply that if a user auditioned an equal amount of sounds withthe Random and Evolution tools, they like more of the sounds discovered with theRandom tool. So again, the efficiency of the Evolution tool is questioned.

Finally, line six of table 5.4 shows the ratio ofSounds Auditionedto Time. This,along with theTime statistic provides another measure of the tool’s simplicity - how

5. Results 82

many sounds can you audition per minute. The Random tool ranks best again - it isa very simple tool to use and encourages rapid auditioning of sounds. The Evolutiontool comes second - it also encourages rapid auditioning, but it is not as simple as theRandom tool. Lastly, the Manual tool - it is quite complicated and requires users tothink before auditioning sounds.

5.4.3 An Alternative Analysis

Before making any conclusions on the above analysis, let us consider another possi-bility. What if the number ofSounds Liked is not dependent on the search tool atall, but some other variable likeTime? Instead of the trend being “people found lesssounds with the Manual tool, and more sounds with the Evolution tool”, it could be“the more time people used any system, the more sounds they found.”

We can see if this is the case by looking at a graph ofSounds Liked with anytool vs. theTime spent using that tool. This graph is shown in figure 5.4. Note thateach point is labelled with a letter that identifies the tool used:E for Evolution;R forRandom; andM for Manual.

Figure 5.4:Sounds Likedplotted againstTime for all search tools.

Apart from points of the same search tool being clustered, there is no trend infigure 5.4. This implies thatSounds Liked is definitely not dependent onTime spent.

5. Results 83

For example, if someone used a search tool longer than another person, you could notpredict that they would find more sounds.

On the other hand, look at figure 5.5, which showsSounds Likedplotted againstSounds Auditioned. Again, each point is labelled with a letter to identify the searchtool used.

Figure 5.5:Sounds Likedvs. Sounds Auditionedfor all search tools.

A clear trend is shown in figure 5.5: no matter which tool is used, moreSoundsAuditioned implies moreSounds Liked. Before we were trying to argue that oneparticular search tool would produce moreSounds Liked than another. This graphgoes against this argument, claiming that it doesn’t matter which search tool you use,just how many sounds you audition.

For comparison, figure 5.6 showsSounds Liked plotted against the search toolused.

Here there is also a clear trend: the Evolution tool produces the mostSoundsLiked , the Random tool produces less, and the Manual tool produces even less again.So which trend is more dominant? IsSounds Likeda function ofSounds Auditioned(figure 5.5); or isSounds Likeda dependent on which search tool is used (figure 5.6)?

An attempt was made to answer this question by fitting polynomials to the twographs and checking confidence intervals, however there were too few data points to

5. Results 84

Figure 5.6:Sounds Likedplotted against search tool

get meaningful results. Instead, these numerical results should be taken with a grainof salt. Although conducting a test like this seemed a good idea at the start, drawingconcrete conclusions from the data proved to be quite difficult. It might be moreinstructive to go straight to the horse’s mouth and find out what the users actuallythought about each of the search tools.

5.4.4 User Opinions

Recall that as well as collecting the statistics, each user was asked to answer thisquestion:

“If you needed some sounds (say, for a composition you were writing)and you were only able to use this algorithm and the three search tools,what strategy would you use to find the sounds you desired?”

This section presents a short summary of each users answer to this question andany other comments they may have made.

5. Results 85

Emma

To find sounds, Emma would use all three search tools. She found the manual toolvery confusing at first, but it then became fun as she understood it. The random toolwas the easiest to use and you could get quite good sounds from it. The evolutiontool was also confusing at first - it was much more complicated than the others. Sheliked the mating feature and got quite good results from it, although she thought theevolution tool could be more straight-forward.

Overall, she thought the tool used would depend on the situation. Sometimes youmight be in a rush, and so use the random tool to get sounds quickly. At other timesyou may want to slowly refine sounds by mutation. She thought it would be best tohave a system that combined all three tools. This way you would enjoy the advantagesof each.

Dougal

For finding sounds, Dougal thought the evolution tool was the best. He thought itwas much easier to use than the manual tool. He noted (as did many other users) thatthe evolution tool contains all the features of the random tool and more - it would bestupid to use the random tool when the evolution tool is available. Dougal had a lot offun using the evolution tool, and wanted to continue even after the survey had ended.

Rachael

Rachael would also use the evolution tool to find sounds. She thought it was quiteeasy to use, except sometimes you forgot to save sounds10. Rachael thought that ifyou understood the algorithm in depth, you would be better off using the manual tool.This would give you exact control over the sound you wanted.

Tom

Tom thought he would use the manual tool to find sounds. He liked the manual toolbecause you could see the envelopes that were being used. Also, when using themanual tool he knew the sounds generated werehis and not some random creation thatthe computer cooked up. Tom thought that the evolution tool was good for refiningsounds. He thought a good strategy to find sounds would be to first create something

10This was due to a poor interface design... my fault!

5. Results 86

with the manual tool and then work on it and refine it in the evolution tool11.

Brett

To find sounds, Brett would first use the random tool and get a number of raw soundsthat had desirable characteristics. He would then take these to the evolution tool andstart working on them. He really liked theMate feature and thought it was successfulat combining the traits of two different sounds. He thought thatMutate was very goodfor fine tuning a sound. Brett liked the random tool because its simple interface didnot distract you from listening to the sounds. This is why he would use it first, eventhough all its features are contained in the evolution tool.

Brett thought the only use for the manual tool would be to ‘tweak’ sounds that youobtained via other means.

Chris

Before Chris’ comments are presented, a special note should be made: unlike otherusers, Chris is an active composer of electronic music. His compositions involve theuse of many different synthesizers, both digital and analog. Because of this, he hasan in depth understanding of many synthesis algorithms. Chris has been composingfor over five years - he is the kind of person that would really utilize the search toolsbeing assessed here. As a result, special attention should be paid to his comments.

Chris strongly stated that the evolution tool was the most efficient way to searchfor sounds. Using the evolution tool, his search strategy would involve first gener-ating some random sounds. These would be refined a bit using mutation and anyvaguely interesting sounds would be saved. Next, Chris would mate the saved soundsin different combinations and hopefully obtain his desired sounds.

Chris would have liked to see the envelopes of sounds in the evolution tool. Hebelieved this would provide a visual feedback, indicating to the user what part of thesound had changed. He also would have liked a control that varied the degree ofmutation.

Chris thought it was good having a variety of ways to change the sound - in theevolution tool there were three: randomization, mutation and mating. Ideally though,Chris thought it would be good to enable manual modification of parameters if youneeded it. This would create a ‘super tool’ with all the features of the three existingtools. The ‘super tool’ would provide the composer with the most flexibility.

11Tom thought that all the programs could do with catchier names. He expected something alongthe lines of “SoundGen 2000” rather than “evolution tool”.

5. Results 87

Finally, Chris would have liked a ‘loop’ feature. This would constantly replay thecurrent sound and enable you to sequence drums or other sounds on top of it. Theloop feature would help you audition sounds; you could see if a sound fitted in withthe rest of your composition. It would also make using the system a lot more fun; youwould really be making music as the sounds mutated and evolved.

Nick

Nick is also an active composer of electronic music. Although not as experienced asChris, he would still have a use for the search tools presented here. Special attentionshould also be paid to Nick’s comments.

Nick had two strategies for finding sounds. The first was like Brett’s: the randomtool would be used to get a few interesting sounds and then these would be refinedusing the evolution tool. The second strategy would involve using the manual tool toset the desired envelopes - this is easier here as the envelopes are displayed. Then theevolution tool would again be used to refine the sounds and set the other parameters.Nick saw the evolution tool mainly useful in refining sounds obtained through othermeans.

Nick got some good results using the mutation feature of the evolution tool. Usingit he was able to define a small subspace of sounds that he really liked. He thoughtit would be really cool to be able to interpolate sounds within this subspace. Ideally,this would be done in real time, while performing a composition.

Nick was a bit disappointed in the underlying synthesis algorithm and quickly gota feel for all the sounds it could produce. He thought it would be good to use analgorithm that could produce a wider range of timbres.

5.5 Conclusions

Taking all of the previous discussion and analysis of results into hand, we can formu-late a number of conclusions:

� Drawing conclusions from the statistics collected proved difficult. This was dueto each tool not being used for an equal amount of time and an unequal amountof sounds being auditioned. This couldn’t be helped because some of the toolstook longer to get used to, and some of them encouraged more auditioning ofsounds. Also, with a small sample space it was difficult to tell which variableswere actually effecting the number of sounds liked.

5. Results 88

� We might tentatively say that based on the statistics, the Random tool provedto be most efficient for searching the sound space. The Random tool producedmore interesting sounds in less time, and with less auditioned sounds than boththe others. The Random tool was also the easiest to use, with users getting usedto it much quicker than the other tools. The Evolution tool found more soundsthan the Manual tool in less time, but on a sounds liked vs. sounds auditionedmeasure, they were about equal. The Evolution tool was easier to use than theManual tool.

� Remember that the Random tool is a subset of the Evolution tool - the Evolutiontool provides all the functionality of the Random tool and more. This impliesthe good result for the Random tool mentioned above is also a good result forthe Evolution tool.

� The users comments generally enforce the above two points. The Random toolis good for finding sounds quickly, but it doesn’t have any control - you justwildly jump around the sound space. What is needed is a finer level of controlthat allows sounds to be ‘tweaked’ to get them sounding just right. Both theManual and Evolution tools provide fine grained control or ‘tweakability’, butmost users preferred the Evolution tool as it was easier to use.

� Even with a simple implementation, theMate feature of the Evolution toolproved quite successful. Three of the users obtained good results with this fea-ture.

� A definite bonus is Chris’ strong support for Evolution as his tool of choice.Being a active composer, he may be representative of the people who wouldgain most use out of these tools.

� Some users expressed the utopian philosophy that “all the search tools weregood.” In this conclusion, you can’t say that one tool is better than the others,because they each have their unique advantages and disadvantages. Each toolis the perfect choice in certain situations.

In summary, the success of the Random tool was a bit unexpected - this simplemethod seemed to perform better than the more complex Manual or Evolution tools.However, the Random tool does not have the fine grained control required of users -it is here where the Evolution tool excels. The Evolution tool enables users to tweaksounds but is much easier to use than the Manual tool. What’s more, you don’t needto understand the underlying synthesis algorithm in order to use the Evolution tool.If we note that the Random tool is a subset of it, the Evolution tool can definitely beregarded as the most efficient method for searching the sound space - combining rapidsound location and fine grained control. In short,artificial evolution is a useful toolfor sound synthesis.

Chapter 6

Conclusion

6.1 Conclusion

This investigation has explored the application of artificial evolution to sound syn-thesis. It has been shown that interactive evolution is a useful tool for exploring thesound space of synthesis algorithms.

The above conclusion was reached from building and testing a system that evolvedparameters for a simple FM synthesis algorithm.

A preliminary version of the system was assessed by 3 users. It was also comparedwith the only similar work known to the author - van Goch’s P-Farm. Against P-Farm, EvoSperfumed well. Its main differences were a focus on mutation rather thanmating, and a much smaller population size. The users preferred these features andrated EvoShighly. The only drawback of EvoSwas the quality of sounds it produced -this was a limitation of the FM synthesis algorithm.

In addition, a more objective study was conducted that compared artificial evo-lution to more traditional methods of parameter adjustment. Three tools were con-structed in order to compare manual, random and evolutionary methods of searchingthe sound space of the FM synthesis algorithm. Seven users took part in the survey,two of which were composers of electronic music.

The results of the study suggested that the random method was the most efficientway to search the sound space. However, it did not provide the fine grained controlrequired by users to ‘tweak’ sounds. The evolutionary method proved easier and moreefficient in providing this control than the manual method. This is emphasised againwhen it is realised that the random method is just a subset of the evolutionary method.

Both composers and non-composers found the evolutionary method useful. This

89

6. Conclusion 90

shows that artificial evolution provides a tool, not only for people who understand theunderlying synthesis algorithm, but also non-experts who would still like to experi-ment with sound synthesis.

6.2 Other lessons learnt

Along the way, some other valuable lessons were learnt - usually the hard way:

� Evolving sounds wasn’t as easy as first expected! At first I thought I could justthrow a Genetic Algorithm at a synthesis algorithm and fantastic things wouldhappen. Sadly, this was not the case - the task involved a lot more carefuldesign. Suitable ranges for each parameter had to be determined, usually bytrial and error. One example of these difficulties was the problem of evolvingthe carrier to modulator frequency ratios1. Trying to make the parametersN1

andN2 seem ‘continuous’ washard and not simply a matter of blindly applyingan Evolutionary Algorithm.

� The interactive evolution technique developed did not prove successful whenscaled up by a large amount. In order to surmount the shortcomings of theFM synthesis algorithm used, artificial evolution was applied to an alternativealgorithm2. With around 20 parameters, it worked fine - with around 2000parameters, it failed dismally!

6.3 Further Work

Although much has been learnt during the course of this project, as always, there isstill much that could be gained from research in the future. Some unexplored avenuesinclude:

� More experimentation could be done with self adaption in the EvoSsystem.Looking back at EvoS3 where the trial of self adaption was conducted3, thevariances and ranges chosen for�2

fm, �2

fcand�2

Iseem ridiculous! It would be

good to go back and see if some better results could be obtained by using somesensible values4.

1See section 4.6.2This experiment is described in appendix A.3see section 4.3.4You could apply the 1/20 rule that was developed later (see section 4.5.4).

6. Conclusion 91

� As was suggested by a number of users, it would be good to implement a‘variable-variance mutation’ feature. This would allow theuser to vary theamount by which a mutant child differs from its parent. For example, as theuser approached their desired sound they would decrease the variance so mu-tants would lie closer to the parent sound - thus they would be able to ‘home in’on the desired sound.

� The mutation procedure could be further extended to allow certain parametersto befixed as the user desired. For example, if a user knew they didn’t wantto mutate the amplitude envelope further, they could ‘fix’ it. In subsequentmutant children, the amplitude envelope would remain constant while the otherparameters were being optimised.

� The EvoSsystem would benefit with an increase in speed. Currently, there isa significant delay encountered when auditioning sounds. This delay couldbe eliminated by using a faster implementation language, for example usingCsound instead of MatLab. Better still, real time response could be obtainedby using dedicated hardware synthesizers in an approach similar to van Goch’sP-Farm [Goc96].

� A large problem that needs to be dealt with is the limitations of the currentsynthesis algorithm. FM synthesis only provides a limited range of timbres. Itwould be good to adapt the system to an algorithm that can produce a muchwider range of sounds. Furthermore, it would be good to make the systemmodular enough to be adapted to any synthesis algorithm the user cares to ex-periment with. Be warned however, this is no easy task.

� A preliminary attempt solving the above problem was made (see Appendix A),but it was not very successful. However, using artificial evolution for additivesynthesis should not be disregarded. A mating operator seems quite feasible andthe mutation operator could be made to work with a bit more experimentation.The payoff for creating such a system would be large: evolving samples of realsounds into unreal mutant relatives is a prospect to be relished by musicians,composers and anyone with a taste for the bizarre.

� I believe the work presented here follows in the footsteps of Richard Dawkins.The system combines a simple synthesis algorithm with an EA whose main op-erator is mutation. The next step is to follow in the footsteps of Karl Sims. Thiswould require the evolution of symbolic lisp expressions that would be inter-preted as sounds instead of images. Achieving this would solve the problem oflimited-timbre synthesis algorithms – the evolution would be “open-ended.”

� Another large project would involve integrating the EvoS system into a largercompositional framework. This would better serve musicians and composersby combining artificial evolution with existing musical composition software.

6. Conclusion 92

In the short term, progress toward this goal could be made by implementing a‘looping’ feature which allows drum patterns to be sequenced and played along-side of evolved sounds. It would also be good to provide access to all searchmethods from the same interface - manual and random, as well as evolution.

� Finally, we have seen that artificial evolution for sound synthesis provides auseful tool for composers - who use the system for finding sounds that willbe used in compositions; and also non-composers - who use the system forentertainment and experimentation. It would be to see if the system is alsouseful for researchers who explore new sound synthesis algorithms. This wouldinvolve getting some researchers to trial the system and give their opinions.

And there are, without doubt, many other avenues of future research related to thiswork that I have not mentioned. Don’t let this stop you from pursuing them!

6.4 A final note

Maybe it is not really important that one method of searching the sound space bebetter than another. Each method has its own appeal depending on the circumstancesand the needs of the user – all three methods presented here make up a whole.

Ultimately we are trying to build tools that enhance human creativity – unblockit – set it free. I look forward to the day that a machine can interpret my vision andmake it a reality.

Appendix A

Evolving Samples

A.1 Introduction

After the EvoS eexperiments had been conducted (chapter 4) it was decided to tryevolving some more complicated sounds. The sounds from EvoS were interesting, yetvery few were suitable for practical use - for example, in a composition. A user thatevaluated the EvoS system suggested that a system that could mutate sound sampleswould be very useful to composers1. But how could you mutate samples?

One practical way of mutating any pre-recorded sound is via additive synthesis2.Recall that additive synthesis can produce any possible sound by using a separatesinewave oscillator for each harmonic in the sound. Each oscillator has a separateenvelope controlling its amplitude and pitch deviation. This is a bonus as we havealready had a lot of experience in mutating envelopes – it should be a piece of cake.However, additive synthesis is computationally expensive, with most sounds typicallyrequiring 8 or more harmonics for accurate reproduction. As was found with EvoS,MatLab is already slow implementing a couple of oscillators and envelopes for FMsynthesis. Clearly, MatLab would be unsuitable for experimenting with additive syn-thesis at interactive rates.

All is not lost however, the Csound program has a very quick implementation ofadditive synthesis and also another major advantage: a heterodyne filter analysis tool.This tool can analyse any audio sample and return a data file containing the amplitudeand pitch envelopes of each harmonic. Another Csound command can then be calledwhich reads the analysis file and uses the amplitude and pitch envelopes to synthesizethe sound. Using Csound, an evolution system can be built very easily by followingthese steps:

1Emma suggested this - see section 4.7.4.2Refer to section 2.2.3.

93

A. Evolving Samples 94

1. Sample the sound you wish to evolve.

2. Analyse the sample using Csound’s heterodyne filter analysis tool.

3. Read and modify the analysis data file. This can be done with an external pro-gram like MatLab. The modifications you make represent the mutation of thesound.

4. Synthesize the modified analysis file using Csound. Play the synthesized soundto the user. The user can decide whether they like this sound (discard the orig-inal analysis file), or would rather try another mutation of the original (discardthe mutated analysis file).

5. Repeat steps 3 and 4 until a satisfactory sound is obtained.

A.1.1 Implementation

The first step in implementing the evolution system described above was to read andinterpret the Csound analysis files. Due to the simple structure of the files, this provedquite easy to do with MatLab. And using MatLab it was easy to visualise the results.Figures A.1 and A.2 show the data of a particular analysis file.

Figure A.1: Example of a heterodyne analysis file, the amplitude envelopes


Figure A.2: Example of a heterodyne analysis file, the frequency envelopes

You can see that the amplitude and frequency envelopes are quite detailed – eachhas about 100-200 points.

The next (and hardest) task was developing a procedure that sensibly mutatedthese envelopes. At first an almost identical strategy to that used in EvoS 4 was ap-plied3. Each envelope breakpoint was mutated by a small random amount. A varianceof 1

20of the amplitude range was used to control the amount of mutation. This re-

sulted in some envelopes being mutated nicely, while others were badly distorted. Theproblem was that some envelopes were much more insignificant than others. Look-ing at figure A.1 you can see that the amplitude of the higher harmonic envelopes ismuch less than the lower harmonic envelopes. The lower harmonic envelopes werebeing mutated nicely (because the variance was1

20of their amplitude range), but the

higher harmonic envelopes were badly distorted (because the variance was many timesgreater than1

20of their amplitude range).

To overcome this problem, an adaptive variance was used for mutation. Eachenvelope was taken on its own, the range of amplitude values was calculated and avariance calculated that was1

20of this value. This variance was then used to control

mutation for that envelope alone. Using this scheme, nicely mutated envelopes wereobtained, see figures A.3 and A.4 for example.

3Refer to section 4.4.2.


Figure A.3: Example of a nicely mutated amplitude envelope

Figure A.4: Example of a nicely mutated frequency envelope


However, there were still problems with this scheme. Figure A.5 shows a mutatedamplitude envelope that is very noisy. This noise is undesirable and seemed to occur

Figure A.5: Example of a badly mutated amplitude envelope

when breakpoints in the envelope were clustered close together. Although this prob-lem could be rectified with more complex mutation schemes4, due to time constraintsthis was not possible. As most envelopes were being mutated well, it was decided topress on and experiment with evolution using this mutation scheme.

After the mutation procedure had been implemented in MatLab, the rest was easy.Evolution was performed ‘manually’ by first mutating an analysis file in MatLab, thenusing Csound to play the resulting sound. It would be a simple matter to construct auser interface that automated this task.

A.1.2 Results

For this experiment, evolution was conducted with three main samples: an organ, abass guitar and a voice sample. The results were very disappointing. The problemwas that all mutants sounded the same. For a given parent with two mutant children,it was impossible to tell the two children apart. The obvious solution to this was toincrease the variance from1

20to a larger factor. However this didn’t help matters at

4For example, filtering the noisy envelope.


all. No matter which mutants you chose, the sample just ended up sounding like it wasbeing played underwater - the result was the same for the organ, the bass guitar andthe voice sample. When smaller variances were tried, not only were mutant childrenindistinguishable from each other, but you couldn’t even tell the difference betweentheparent and the mutant child!

Was this due to the badly mutated envelopes described before (figure A.5)? Cer-tainly, the underwater/bubbling sounds produced could have been a result of rapidlyoscillating, noisy envelopes. But even if this weren’t the case, I suspect that you stillwould not be able to tell the mutant children apart from each other. The problem isreally the increased number of dimensions in the sound space.

In EvoS7, there were about 20 to 30 parameters being mutated at a given time.Here, there are 20 envelopes (10 for amplitude and 10 for frequency), each with 100to 200 breakpoints which are all being mutated at once. This is a total of 2000 param-eters - a sound space with 2000 dimensions. The human ear just cannot detect smalldifferences in this space, so a human user cannot possibly hope to choose a directionand follow it.

To test this theory a small experiment was conducted with a very simple set ofamplitude and frequency envelopes. Four envelopes were used (2 amplitude and 2frequency), each with four breakpoints. This give a total of 16 points - a sound spacewith 16 dimensions. These were mutated and evolved using the exactly the same pro-cedure as above. It was found that in this case, youcould tell the difference betweenmutant children. In fact, it was quite easy to control the evolution and steer it in thedirection you desired. It seemed that with 16 dimensions, evolution worked fine butwhen you scaled this up to 2000, it fell apart. Maybe this result can be excepted asevidence that this technique doesn’t work when the number of dimension is increasedabove 20 or 30.

A.1.3 Conclusion

As no useful results were obtained from this experiment, the idea of mutating sampleshad to be scrapped - time was running out. This was a shame because if you couldmake the idea work, it would be great. Possibly all it needs it a more complex mu-tation procedure5, but I suspect not. More likely, you need to find a way of reducingthe dimensional space to about 20 parameters. This might be achieved by swappingenvelopes between harmonics instead of mutating each point in every envelope. Itis also a shame to drop this idea as mating samples would have been fairly easy toimplement. To mate two samples you could just randomly swap some envelopes ofone sound for envelopes of the other sound.

5Maybe one that involves filtering to get rid of the noisy envelopes.

Bibliography

[Bac95] Thomas B¨ack, Evolutionary Algorithms in Theory and Practice, Oxford:Oxford University Press, 1995.

[Boo87] Michael Boom,Music through MIDI, Redmond, Washington: MicrosoftPress, 1987.

[Cho73] John M. Chowning, “The Synthesis of Complex Audio Spectra by means ofFrequency Modulation”,Journal of The Audio Engineering Society, Volume21, Number 7, pp 526–534, 1973.

[Cso98] Csound - sound synthesis software. Csound is free for educationaland research purposes. The lastest version is avaiable via ftp fromftp.math.bath.uk in the /pub/dream directory. Other Csoundinformation and resources can be found at theCsound Front Page,http://www.leeds.ac.uk:80/music/Man/c_front.html

[CE95] Kai Crispien and Tasso Ehrenberg, “Evaluation of the ‘Cocktail Party Effect’for multiple stimuli within a spatial auditory display”,Journal of the AudioEngineering Society, Volume 43, Number 11, pp. 932–941, November 1995.

[CH96a] Ngai-Man Cheung and Andrew Horner, “Group Synthesis with Genetic Al-gorithms”,Journal of the Audio Engineering Society, Volume 44, Number3, pp. 130–147, March 1996.

[CH96b] San-Kuen Chan and Andrew Horner, “Discrete Summation Synthesis ofMusical Instrument Tones using Genetic Algorithmms”,Journal of the Au-dio Engineering Society, Volume 44, Number 7/8, pp. 581–592, July/August1996.

[Daw86] Richard Dawkins,The Blind Watchmaker. Harlow: Langman Scientific &Technical, 1980.

[Daw88] Richard Dawkins, “The Evolution of Evolvability”, inArtificial Life,Christopher Langton, Editor, Addison-Wesley, 1988.

99

BIBLIOGRAPHY 100

[DJ85] Charles Dodge and Thomas A. Jerse,Computer Music: Synthesis, Compo-sition and Performance, New York: Schirmer Books, 1985.

[Goc96] Arno van Goch, “P-Farm”, an experimental version of this program is avail-able fromhttp://www.xs4all.nl/˜avg/pfarm.html

[Goc98] Arno van Goch. Personal Communication. August 1998.

[Gol89] David E. Goldberg,Genetic Algorithms in Search, Optimisation and Ma-chine Learning, Reading, Massachusetts: Addison-Wesly, 1989.

[HA96] Andrew Horner and Lydia Ayers, “Common tone adaptive tuning using ge-netic algorithms”,Journal of the Acoustical Society of America, Volume100, Number 1, pp. 630–640, July 1996.

[HB96] Andrew Horner and James Beauchamp, “A genetic algorithm-based methodfor synthesis of low peak amplitude signals”,Journal of the Acoustical So-ciety of America, Volume 99, Number 1, pp. 433–443, January 1996.

[HBH93] Andrew Horner, James Beauchamp and Lippold Haken, “Machine TonguesXVI: Genetic Algorithms and Their Application to FM Matching Synthe-sis,” Computer Music Journal, Volume 17, Number 4, pp. 17–29, Winter1993.

[HG91] Andrew Horner & David E. Goldberg, “Genetic Algorithms and ComputerAssisted Music Composition”,Proceedings of the Fourth International Con-ference on Genetic Algorithms

[Hol75] John H. Holland,Adaption In Natural and Artificial Systems, Ann Arbor:The University of Michigan Press, 1975.

[Hor95a] Andrew Horner, “Wavetable Matching Synthesis of Dynamic Instrumentswith Genetic Algorithms”,Journal of the Audio Engineering Society, Vol-ume 43, Number 11, pp. 916–931, November 1995.

[Hor95b] Andrew Horner, “Envelope Matching with Genetic Algorithms”,Journalof New Music Research, Volume 24, Number 4, pp. 318–341, December1995.

[Lev93] Steven Levy,Artificial Life: The quest for a new creation, London: Penguin,1993.

, In Richard K. Belew and Lashon B. Booker, editors, San Mateo, California:Morgan Kaufmann Publishers Inc., 1991.

[Mat98] MatLab computing software, The MathWorks, Inc., seehttp://www.mathworks.com/products/matlab/ for moreinformation.

BIBLIOGRAPHY 101

[Moo94] Jason H. Moore, “GAMusic 1.0”, available via ftp fromfly.bio.indiana.edu in the/science/ibmpc/ directory

[Moo98] Jason H. Moore. Personal Communication. May 1998.

[Mus98] Music Machines, website athttp://machines.hyperreal.org

[Ope88] Peter Oppenheimer, “The Artificial Menagerie”, inArtificial Life, Christo-pher Langton, Editor, Addison-Wesley, 1988.

[Pre92] Jeff Pressing,Synthesizer Performance and Real-Time Techniques, Madison,Wisconsin: A-R Editions Inc., 1992.

[Sim91] Karl Sims, “Artificial Evolution for Computer Graphics”,Computer Graph-ics, Volume 25, Number 4, pp. 319–328, July, 1991.

[Sim93] Karl Sims, “Interactive evolution of equations for procedural models”,TheVisual Computer, Volume 9, Number 8, pp. 466–476, 1993.

[Smi91] Joshua R. Smith, “Designing Biomorohs with an Interactive Genetic Al-gorithm”, Proceedings of the Fourth International Conference on GeneticAlgorithms, In Richard K. Belew & Lashon B. Booker, editors, San Mateo,California: Morgan Kaufmann Publishers Inc., 1991.

[Smi94] Joshua R. Smith,Evolving Dynamical Systems with the Genetic Algorithm,Honors Thesis, Williams College, Williamstown, Massachusetts, 1994.

[Wil88] Scott R. Wilkinson,Tuning In: Microtonality in Electronic Music, Milwau-kee: Hal Leonard Books, 1988.

[YH97] Jennifer Yuen and Andrew Horner, “Hybrid Sampling-Wavetable Synthesiswith Genetic Algorithms”,Journal of the Audio Engineering Society, Vol-ume 45, Number 5, pp. 316–330, May 1997.

artificial evolution for sound synthesis

Documents