synthetic harmonies- an approach to musical semiosis by means of cellular automata

8/2/2019 Synthetic Harmonies- An Approach to Musical Semiosis by Means of Cellular Automata

1/8

Leonardo

Synthetic Harmonies: An Approach to Musical Semiosis by Means of Cellular AutomataAuthor(s): Eleonora Bilotta and Pietro PantanoReviewed work(s):Source: Leonardo, Vol. 35, No. 2 (2002), pp. 153-159Published by: The MIT PressStable URL: http://www.jstor.org/stable/1577196 .

Accessed: 08/02/2012 08:48

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

The MIT Press andLeonardo are collaborating with JSTOR to digitize, preserve and extend access to

Leonardo.
http://www.jstor.org/action/showPublisher?publisherCode=mitpresshttp://www.jstor.org/stable/1577196?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/1577196?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=mitpress


2/8

GENERAL ARTICLE

Synthetic Harmonies:A n Approach t o Musica l Semiosisb y M e a n s o f Cel lular Automata

EleonoraBilottaand PietroPantano

Music is the arithmetic fsoul, that usesnumberswithoutreal-ising it.

LeibnizThis article discusses music, mathematics and artificial-life (A-Life) models, linking them together through a semiotic ap-proach. Music and mathematics, as expressions of creativethought, can be studied through their reciprocal relationship(as has been previously noted by both ancient and modernphilosophers). Music and mathematics can be analyzed usingmodels of the auditory perception used by the human mindduring listening. They can also be studied in relation to theemotions they evoke, creating that "arithmetic of the soul"towhich Leibniz referred. They can be analyzed from the for-mal point of view as well, which entails investigating the struc-tures of the models that artists use for musical composition.We have chosen to use cellular automata (CA) (dynamicstruc-tures in which space, time and states of the systemare discrete;mathematical models able to simulate the complex behaviorof some physical and/or biological systems) to study com-plexity and then to translate this complexity into music. Therelative simplicity of CA and their extraordinary capacity tomimic both evolution and growth in biological life seem tohave some basic peculiarities in common with natural humanlanguages (and thus with music) and with semiotics. Like lan-guages, CA exhibit dynamic behavior. An important aspect ofmost complex systemsis that they are massivelyparallel in op-eration, with all their parts working simultaneously. This al-lows their features to change in many ways, exhibitingemergent and self-organizingqualities.In order to understandcomplexity, we need to identify the laws in this behavior, theproperties that remain unchanged (invariant).The link between mathematics and music, combined withcertain models taken from modern science, allows music tobe reproduced, synthesized and made to evolve through manytheoretical-conceptual instruments offered by the A-Life field.As Christopher Langton has said, "ArtificialLife is a field of

Eleonora Bilotta (researcher), Centro Interdipartimentale della Comunicazione, Univer-sita della Calabria, Cosenza, Italy.E-mail:.Pietro Pantano (researcher), Centro Interdipartimentale della Comunicazione, Universitadella Calabria, Cosenza, Italy.E-mail:.An earlier version of this paper waspresented at the Seventh International Conference onArtificial Life (Alife VII), 1-6 August 2000, Portland, OR, U.S.A. Firstpublished in M.A.Bedau,J.S. McCaskill,N.H. Packard and St. Rasmussen, eds., ArtificialLifeVII:Proceedingsofthe Seventh nternationalConferenceCambridge, MA: MIT Press, 2000). Reprinted bypermission.

studydevoted to understanding lifeby attempting to abstract the fun-damental dynamical principles un-derlying biological phenomena,and recreating these dynamics inother physicalmedia-such as com-puters-making them accessible tonew kinds of experimental manip-ulation and testing" [1].In fact, one of the most promis-ing sectors of contemporary art isthe application of A-Life models toart, design and entertainment [2]

ABSTRACT

The authorsxplorehecreationf artificialniversesthat re xpressiblehroughmusic ndnternallyomprehen-sible scomplexystems. hesemioticpproachhispaperpresentsouldlsoallowhedevelopmentf new ools finvestigationntohecomplexityofartificial-lifeystems. hroughcodificationystemssingmusicalanguage,t spossibleto understandhepatternshattheglobalynamicsfcellularautomataroducend o use heresultsn hemusicalomain.ntheauthors'pproach,usicanbeconsideredhe emanticsfcomplexity.he uthorsdentifyanalogiesetweenlementsfcellularutomatand lementsof musicalorm,reatingnarrativeusicalrameworkhathasallowedhemodevelopproductive,omputationalndsemanticethodology.usicfosters n ncreasedapabilityforanalyzingnd econstructingcomplexity,rovidingnexpectedinsightntotsorganization.

as well as music [3]. Some characteristics of such forms of artare related to the complexity of their production: startingfromsimple and repeatedly applied rules, and slightly modifyingcertain elements, the artist has the ability to generate ever-differing artifacts. The perspective provided by A-Life ap-proaches raises the question of the ontological status of theartwork:the concept of the unique and immutable artifact isgiving way to that of creations that can be replicated in ever-differing ways,applying the same fruitful rule. Nevertheless,the issue of what an artwork might be like, how it is possibleto manage its overall organization and its detailed content, ac-cording to its deep structural models of production, is not welldefined; there is no common grammar of creativity n art. Be-sides, the artwork evolves in time: a given structural configu-ration might receive many instantiations, each unique in itslocal details but all changing by means of evolutionaryprocesses.This article reviews theworksthat we have produced to date,using A-Life models for musical production. Some recent re-sults will also be presented. Beginning with the simplest one-dimensional CA, we moved on to improving our musicalcompositions by means of genetic algorithms. We then ad-vanced to analyzing multi-stateCA,realizing new musificationcodes that reproduce the emergence of complexity.Finally,we reporton a preliminary grammarof musical com-positions, linked with certain ideas about complexity and emer-gence and the possibilityof creating a conceptual frameworkfor discussion of complexity and music.

SEMIOTICS,ARTIFICIALLIFE AND MUSICIt is possible to create artificialuniverses hat are comprehen-sible through music. The relationship between music andA-Life models can be realized as a semiotictriangleof significa-

LEONARDO, Vol. 35, No. 2, pp. 153-159, 2002 153MIT ress


3/8

PIan of theexpression. . I -

Phenomena::allthe possibleConfig iationsThat can beCeaularAutomata.Ot.her..:..ysic.Phenomena.

con ent~::

readin.gof th.e::IPSC~E.*


4/8

crete dynamic systemconsistingof a finitelattice of identical cells, whose values areupdated simultaneously according to alocal rule. The value of the site i at timet is indicated by at). This value will de-pend on the values of the neighborhoodof the site at the time before t - 1. So therule of evolution can be expressed as:a(t) = - 1F [a-l) a(-l)a, =Fa- r , _,.+i . . .

at-1) . ] (1)

where r representsthe radiusof the neigh-borhood and 2r+ 1 s the number of sites(around the site i), determining the valueof the site itself. The value of the site is awhole number between 0 and k - 1.Local rules can be listed in a look-uptable, univocally identified by the fol-lowing string:

G = (SI, 5, ... S,, I,.. .) (2)For example, if k = 4 and r = 1, the look-up table can be writtenas shownin Table1,and (2) will be:

G= (2,0,1,3,2,...,0,3,1,2)The string (2) can be considered the"genome" of the CA. Starting from this

genome, one can activate an evolution-ary process through the application ofa genetic algorithm (GA). For generick and r the number of rules will bekk . The number of rules will in-crease enormously as the values of k andrincrease.

Table1. Look-upable for a CAk = 4, r = 1.333 332 331 330 3232 0 1 3 2

Bymeans of these codes, we have trans-lated into music the complex rules of agreat varietyof one-dimensional CApres-ent in literature [9], showing how thepresence of auto-organization andstrange creatures can be translated intomusic [10]. Using as a fitness criterion ameasure of complexity-the input-entropy function-we have constructeda GA to search for complex rules formulti-stateCA [11].We have done this in order to exploitmore fully the great diversityand beautythat complex rules can manifest and toobtain more significant musical compo-sitions.

How, then, does one decide on thequalityof musical compositions? Musicalfitness, like visual fitness [12], isgenerallyassessed intuitively by human evaluators.But this method is not very efficient (weneed many generations to produce ac-ceptable results) and is very costly interms of time and resources [ 13].Instead of using fitness criteria de-rived from human aesthetic judgments,we have realized a GA to detect the bestmusical compositions by means of an au-tomatic mechanism of selection thatuses a fitness criterion based on conso-nance. We have generated a sample setof musical compositions. The CA rules,which give birth to these groups of com-

.. 003 002 001... 0 3 1 0002

positions, have been codified into a ge-netic code, and we have selected thosefamilies most capable of adapting to theenvironment and reproducing them-selves [14].The procedure we have used is the fol-lowing:a. start from CA genomes such as thestring (2);b. generate various individuals (se-quences of sounds) associated withthese genomes;c. select the genome sequences mostsuitable for evaluating the capabili-ties of the individuals (in this casethe most consonant sound se-quences);d. make these genomes evolve fromone generation to the other, casu-ally modifying their features andusing combination rules based onsexual pairing;e. continue the process for many gen-erations.

Wecomputed the consonance and dis-sonance values between two notes in amelody.The succeeding generations, cre-ated by random genetic mutation, wereselected according to their fitness, andthe process was repeated many times.After many generations, we observed astrengthening and improvement in somepopulations' fitness (see Fig. 2) and

GENERATING AND EVOLVINGMUSICAL COMPOSITIONSTo this point we have developed differ-ent types of musification codes, amongwhich the most important seem to be:a. local musification codes, throughwhich it is possible; to read the gridof a CA site by site;b. global codes, through the functions

of input-entropyand the progress ofthe populations by CA;c. mixed codes, reading portions of CAconfigurations.These codes may be indirect,n whichcase algorithms are used that in some-way transform and/or manipulate thenumeric structure of the CA configura-tion, before the CA are translated intomusic [8].

Fig. 2. Fitness growth given various mutation probabilities. The best musical melodies corre-spond to the fitness curve with 1%of mutation percentage.Fitness growth for four experiments35000000

4o41,L,_

30000000250000002000000015000000100000005000000

0W - L r 0C M ' ) -- O - 0oCN In OG M ne r ioe r- - CN Nt 5N

Generations

Bilottaand Pantano,Synthetic Harmonies 155


5/8

emergent properties in improvingmelodic organizations. The results showthat the evolutionary process can be use-ful in defining consonance/dissonancewhen applied automatically to musicalcompositions. Using an algorithm basedon consonance, we searched for the rep-etition and recombination of recogniza-ble melodic motifs and fragments,aswellas the clear interpretation of patterns(generated from different processes ofevolution) that could help to create unityand coherence while also ensuring vari-ety in the musical pieces. We also foundthat in the musical pieces generated byagenetic algorithm based on consonance,consonant harmonies are those thatsound stable;dissonant harmonies soundunstable or seem to clash or to have lowfitness values. They tend to be resolvedinto consonant harmonies [15].Given the broad applicabilityof evolu-tionary concepts and tools to music, thequestion arises of how they may be usedin relation to music theory. We find thatsuch concepts can be useful in under-standing both local issues of music struc-ture and organization and global issuesof configuration and diachronic changein musical style.A second issue (which isstill potential infieri) concerns the real-ization of computer-based programs (ormodified computational tools) that pro-

duce music automatically by means ofother kinds of evolutionary processes,without human intervention.

NARRATIVEMUSICAND COMPILEXITYThe graphical configurations in whichCAevolve can describe the wayin whichcomplexity is manifested or codified.Complexity can be seen as a process ofencoding, and it is possible to read therepresentations complexity producesonly if one possesses the code.In our approach, music can be con-sidered the semanticsof complexity. Wehave translated into musical composi-tions domains and gliders of complexmulti-state CA [16] and have observedthat complexity has some organizationallaws, which could be similar to those ofmusical composition [17].On the one hand, music builds elabo-rate structures of sounds, but, as Gestaltpsychologists have pointed out, its aes-thetic and perceptual comprehensiondoes not rely merely upon the rawprop-erties of the individual sounds. Music alsobuilds upon structural concepts such asscales,modes and keyrelations,and a va-riety of transformations, which includetranspositionand repetition.Experiencedlisteners make use of a large number of

musical concepts or schemes in listeningto music. Traditionally,music is thoughtof as havinga horizontal and a verticaldi-mension. This derivesfrom musicalnota-tion, in which the horizontal axis standsfor the temporal succession of sounds thatforms the melody,and the vertical axisde-picts pitch relations,or the simultaneoussounds that form harmonies.

Translating a sequence of soundsthrough physical parameters does notamount to generating music. That is amore complex human expression, withits own grammar and aesthetic: melody,harmony, consonance, canons, fuguesand rhythm transform a sequence ofsounds into music. Unlike physical factswhose laws areunchangeable, musical ex-pression depends on rules that evolvewith time and are stronglybound to theirhistorical period; this makes the genera-tion of music a difficult and verycomplexprocess.Cellular automata alsodisplayrich andcomplex patterns, whose organization iscompletely unpredictable. Wolfram clas-sified CAqualitatively according to theiraperiodic behavior: class 1 (homogene-ity);class2 (periodicity); class 3 (chaos);and class 4 (complexity) [18].The firstclass consistsof automata thatevolve to a unique, homogeneous state,after a limited transient.The second class

Fig. 3. Analogies between CAand musical composition: many elements of complex systems correspond to elements of musical systems.

156 Bilotta and Pantano, Synthetic Harmonies

CA classesSear.ch.ingor C:A omp.lex esDiscoveryof familyof complexrulesAnalysisof anyrulesFamily analysisCatalogue cr...........tion

*Particlesor gliders, .....*Glidersnteraction,*Com?plexity rules_:':...::..:_"...;'::;_._...

Musicalcomposition(according o

C ..... ...Musical formsConventionsCommonformalpatternMajorl.l^t p.....:.........*~ondF~s:... .'#Repeitions'

' ontr. ......t . ......:'Development,.?:...2:;.::;"ii''i:i':.:. :'... .': .''Rules ofompositi;'':n:'::::::'i;%:'::i''::~'.i. !;.i.i:::.;.::'i:..

_ ____ _._.___

00-

.I----------- ----- --- --------


6/8

r,,.Hij^ _ ".,,;, . T .-_l lT^ *- .;^**. ,.' ~8^_>i.;-. * ;,...,* - . .. . . -~-. , . *'~ ^ ^ i-T ...


7/8

Fig. 5. Musical form: variation using different CApatterns. The three patterns produce different melodies, according to the differences inthe visual configurations.tions are based on contrast as well asrepetition of sections. If we use twoforms, the resulting compositionwill consist of two contrasting sec-tions that function as statement andcounter-statement.The pattern maybe, for example, XY, or it may becomplicated by repetition (XXY)or variation (XXk'YYk',where Xk'means a variation of X). For thistype of composition, we used differ-ent CA rules or domains (see Fig. 6).In order to identify consonant com-positions, we used as an evaluationscale the fitness of the rules com-pared with the consonance scalethat uses very small proportions.4. Development (in which compo-nents of the original section, such asa melodic fragment or arhythm,aretaken apartand recombined in newways to create a new section). Forproducing this category of compo-sitions, we used, as before, differentinitial data from one CA, insertingdata from the domains and chang-ing them in a creative manner. Inthis case, we are developing a sort ofnarrativeof producing musical com-positions in which, as in other mod-els of thought, creativity plays an

important role. We are planning tobuild up a systemfor describing CAcomplexity in which, by means of amusicallanguage, it is possible to re-alise sentences and then compositesof sentences in order to obtainhigher-level compositions. Such amusical narrative also provides anexample of hierarchical levels ofform. The alteration of contrastingsections can be expanded in formsas in the concerto.

CONCLUSIONSThe scenario A-Life science reveals tomusiciansiscompletely unexpected. Newmethods, paradigms and tools for study-ing, producing, managing and creatingmusic are actually available. Unfortu-nately,thisgreat rangeof approachesandpossibilities is not fully known and ex-ploited. New keys of reading the com-plexityof A-Lifesystemscould be realizedusing the semiotic approach this paperpresents. Through various codificationsystems using musical language, it is pos-sible to give meaning to many character-istics of the patterns that CA globaldynamics produce and to use the resultsin the musical domain. In our approach,

music can be considered the semanticsof complexity. We identified analogieswith some elements of CA and elementsof musical form.Musical form goes beyond sectional

patternsand is created bythe composer'sorganization of melody, rhythm andharmony, elements that involve creativethought. Butthe organizational structurewe identify in analogy with elements ofCAcomplexity can exist on severallevels,and it is possible to identify basic formalpatterns having unity, variety and sym-metry. It seems that there is a corre-spondence between the catalogelementsand the musical features, the rules ofcomplexity and the rules of musical com-position. So we can build up a mathe-matical model for musical composition.This model helps us in interpreting com-plexity (or music in general) as well as inproducing musical compositions as nar-rative pieces. This mathematical modelallows us to realize an engineering pro-cess for musical compositions. This nar-rative musical framework helps us todevelop a productive methodology. Infact, we can produce musical pieces withsome pre-defined characteristics. It iscomputational; as we can create a com-puter program to perform the process of

158 Bilottaand Pantano,Synthetic Harmonies


8/8

C d ~ l

.

U.* BMU?

BI m

I: A: . . r , EE I_:E::E ^t :'.t Ei U. .. :Fig. 6. Musical form: contrast. Using different CApatterns, we obtain different melodic lines.

analysis by detecting the CA elements,creating the corresponding musicalforms and then recombining them in aproductive manner (synthesis). It is se-mantic because we produce musical com-positions,which demonstrate complexity,expressing the laws of emergence andauto-organization.

References1. C.G. Langton, ed., Aitificial Life: An Overviezw(Cambridge, MA: MIT Press 1995).2. C.C. Maley and E. Boudreau, eds., Artificial Life VIIWorkshoproceedings Portland, OR: Reed College,2000).3. E. Bilotta, E.R. Miranda, P. Pantano and P.Todd,"ArtificialLife Models for MusicalApplication," Eu-ropean Conference on Artificial Life VI, WorkshopProceedings, Prague, 8-14 September 2001(Cosenza, Italy:Editoriale Bios, 2001).4. E. Bilotta, P. Pantano and V. Talarico, "SyntheticHarmonies: An Approach to Musical Semiosis byMeans of CellularAutomata," n M.A.Bedau,J.S. Mc-Caskill,N.H. Packardand S. Rasmussen,eds., ArtificialLifeVII:Procedings ftheSeventhnternationalConference(Cambridge, MA:MITPress,2000) pp. 537-546.5. Bilotta et al. [4].6. U. Eco, Trattatodi semiotica Milan, Italy: Bom-piani, 1975).7. S. Wolfram, "Universalityand Complexity in Cel-lular Automata,"PhysicaD 10 (1984) pp. 1-35.

8. E. Bilotta and P. Pantano, "Artificial Life MusicTells of Complexity,"in Workshoproceedings3].9. Wolfram [7].10. E. Bilotta, P. Pantano and V. Talarico, "MusicGeneration through Cellular Automata: How to GiveLife to Strange Creatures," Proceedings f GenerativeArt (Milan, Italy:AleaDesign, 2000).11. E. Bilotta, A. Lafusaand P.Pantano, "Searchingfor Cellular Automata Complex Rules by Means ofGenetic Algorithms," submitted for publication.12. K. Sims, "InteractiveEvolution of Equations forProcedural Models," The Visual Computer , No. 8,466-476 (1993).13. Bilotta et al. [4].14. M. Mitchell, An Introduction o GeneticAlgorithms(Cambridge, MA: MIT Press, 1996).15. E. Bilotta, P. Pantano and V. Talarico, "Evolu-tionary Music and Fitness Functions,"in A.M.Anile,V. Capasso and A. Greco, eds., Progressn IndustrialMathematics t ECMI 2000 (Berlin: Springer-Verlag,2002), in press.16. Bilotta and Pantano [8].17. E. Bilotta and P. Pantano, "Observations onComplex Multi-State CAs," in J. Kelemen and P.Sosik, eds., Advances n ArtificialLife (Proceedings fthe SixthEuropeanConferencenArtificialLife,Prague,September2001) (Berlin: Springer-Verlag, 2001)pp. 226-235.18. Wolfram [7].19. J.H. Conway, "WhatIs Life?"in E. Berlekamp,J.H. Conwayand R. Guy, eds., WinningWaysor Your

Mathematical lays,Vol. 2 (New York:Academic Press,1982) Chap. 25.20. A. Wuensche, "ClassifyingCellular Automata Au-tomatically: Finding Gliders, Filtering and RelatingSpace-Time Patterns,Attractors Basins and the Z Pa-rameter," Complexity 4, No. 3, 47-66 (1999).21. W. Hordijk, C.R. Shalizi and J.P. Crutchfield,"Upper Bound on the Products of Particle Interac-tions in Cellular Automata," Physica154D (2001)pp. 240-258.22. These families are derived from the process de-scribed in Bilotta et al. [11].23. Hordijk et al. [20].24. Bilotta and Pantano [8].

Eleonora Bilotta is professor of General Psy-chologyat the Arts and Humanities Faculty,University of Calabria, Italy. Her current re-search interests include intelligent systemsineducation, psychology of programming, psy-chologyof music, and artificial life and music.Pietro Pantano is professor of Classical Me-chanics and Applied Mathematics at theEn-gineering Faculty, University of Calabria,Italy. His current research interests includenon-linearphenomena and wave propagationtheory, complexity,self-organized criticityandartificial life, and generative and evolutivemusic.

Bilottaand Pantano,Synthetic Harmonies 159

ILL L ; I

r E

synthetic harmonies- an approach to musical semiosis by means of cellular automata

Documents