laske: a conversation with marvin minsky

8/12/2019 Laske: A Conversation With Marvin Minsky

1/15

Articles

FALL 1992 31

A CONVERSATION WITHMARVIN MINSKY

Marvin Minsky and Otto Laske

The following excerpts are from an interview with Marvin Minsky which took placeat his home in Brookline, Massachusetts, on January 23rd, 1991. The interview,

which is included in its entirety as a Foreword in the book Understanding Musicwith AI: Perspectives on Music Cognition (edited by Mira Balaban, Kemal

Ebcioglu, and Otto Laske), is a conversation about music, its peculiar features as ahuman activity, the special problems it poses for the scientist, and the suitability of

AI methods for clarifying and/or solving some of these problems. The conversation isopen-ended, and should be read accordingly, as a discourse to be continued atanother time.

AI Magazine Voume 13 Num er 3 1992 AAAI


2/15

Articles

32 AI MAGAZINE

attempts to straighten out things that dontfit the chosen formalism, in order to makethem look more logical.

Problems withFormalization

MM : Well, yes, I like how you put that.Indeed we can use logic to make things lookmore logical but we can also use it to produceillusions. I suppose that the term originallyreferred to matters involving understandingand knowledge. But in current computer sci-ence logical now means mainly to try toexpress everything in terms of two quantifiers,viz., for all X, and for some X and twovalues, viz., true and false. But thosemenus seem just too small to me! We needmuch richer ideas than that.

OL: In the past, weve had other logics, suchas, for instance, Hegels dialectical logic whereone doesnt work with two quantifiers, but with

the notions of thesis, antithesis, and synthe-sis. (Antithesis dates from Platos idea ofOther in the dialog entitled Sophistes.) So,one always knew there is some other way

MM: Well, Im not familiar with any appli-cations of dialectic logic, but I certainlywould favor going beyond binary logics.However, the extensions that Ive seen in theso-called modal logics seem no improvementto me. Yes, they can express some ideas likeit is possible, or it is necessary, and soforth, but extensions like those seem inconse-quential. We much more urgently need waysto express ideas like usually or it is often

useful to, or for all objects that resemble Xin regard to aspect Y.OL: So you think computer scienceor at

least AIneeds other, richer kinds of descrip-tive ideas and terms?

The Need for aVariety of Methods

MM: Yes indeed. I want AI researchers toappreciate that there is no one best way torepresent knowledge. Each kind of problemrequires appropriate types of thinking and

reasoningand appropriate kinds of repre-sentation. For example, logical methods arebased on using rigid rules of inference tomake deductions. This works well insideformal domainsthose artificial worlds thatwe imagine for ourselves. But to cope withthe unknowns and uncertainties of reality, wemust base our actions on experienceandthat requires us to reason by analogy, becauseno two situations are ever quite the same. In

The Notion of FormalizingMusical Knowledge

OL: I wish we could read peoples brainswhen they are engaged in music-making.Since that is, alas, beyond the state of the artof AI, we are forced to deal with such episte-mological issues as how to represent knowl-edge on the basis of what people tell us they

do, which in most cases isnt very close towhat they are actually doing. A further prob-lem is that we cant use what they are tellingus in the form it is reported; rather, we haveto translate their verbalizations into evenmore formal code that is still farther removedfrom actual musical activities than the initialreport. So, of course, two questions immedi-ately come to mind, viz., how should one for-malize musical knowledge, and, can it bedone effectively, i.e., so as to generate somekind of musical action?

MM: And that in turn raises the question ofis formalizing the right idea? There aremany kinds of reasons for writing descrip-tions, and here we ought to have a softerconcept than formalize. When we writedown things we know about other crafts, wedont usually feel any need to imitate mathe-maticians. Why should we always feel com-pelled to do that when describing ourmusical dispositions?

OL: Indeed, this convention is not muchquestioned. One thinks backwards from onesresearch goals, and since one knows how tocompute, one concludes with little hesitationthat one ought to formalize. The term for-malizing is linked to the notion of computa-tion

MM: Quite soever since the earliest daysof computing. Maybe because most of thefirst pioneers were concerned with mathe-matical matters. Consequently, what we callcomputer science quickly became quitetechnical. In some domains of computerdevelopment that was a good thing. But inother areas, particularly in the semantics ofprogramming, it seems to me that the cur-rently popular formalisms are premature andunnecessarily limited in conception, beinginadequate for expressing some of the most

important aspects of computation. For exam-ple, mathematical logic plays a dominant rolein contemporary formalizations, and yet isquite inept at expressing the kinds of heuris-tic knowledge we need for developingArtificial Intelligence.

OL: Formalization typically amounts tomaking a post facto summary of some stateof affairs that is based on specific interpreta-tions. One looks at some description and

0738-4602/92/$4.00 1992 AAAI

isformalizing

the rightidea?


3/15


4/15

Articles

34 AI MAGAZINE

from their mundane bondage to realitytothen be free to create new things.

Music or Musics?

OL: Would you say that holdsfor all kinds of music?

Music seems to be anotion like God orlove, somethingeverybody can identify

with, but which actuallycovers many different, even opposite, phe-nomena

MM: I think calling so many differentthings by the same name, music, certainlymakes it hard to think about those subjects.The modern tendency, to be tolerant and sayanything is music, is bad for the mind,both of the listener and of the critic also. (Idont know whether its bad for the compos-er, if he can make a living.) A portmanteauword like music that is used for so manyactivities cannot be an element of a seriousdiscussion.

OL: So, we would have to speak ofmusics, and would have to define what wemean in any particular instance, whichwould be very cumbersome.

MM: Yes, like having to say we are going totalk about certain German music from theeighteenth century, or Indian music fromsuch a place at a certain time. And then onecan ask, to what extent do these engage simi-lar mental activities?

OL: How should we proceed, music being

really a universal?MM: For serious analysis, I think we simplymust avoid such universal. When I writeabout some mechanism of intelligence orlearning, I try not to use words like intelli-gence and learningexcept in the title orsummary.

Music as a Label forSocietal Acceptance

OL: That is indeed the usual practice instudies in AI and music, where you find, forinstance, systems for doing harmonic analy-

sis, or systems that generate compositionalmaterialthese are all specific kinds of things,and the claim is not that we know what musicis. It seems to me that, viewed in the light ofsuch studies, the term music expresses ratheran acceptance on the side of society that some-thing is o.k. The composer, as composer,doesnt care whether something is music. Heis driven to generate something, and if anaudience finds that what he produced is accept-

able, or is music, he is likely to be happy,and otherwise he is notbut its essentiallynot his doing as much as it is societys.

MM: That raises an exciting and interestingquestionthat many people are reluctant toconsider, which is the question: what is, orought to be, o.k.? One thing I like to do is toconsider the major human activities, and tryto get people to ask: is it ok? All over theworld many people listen to music for hourseach day; in this country many spend sub-stantial portions of their incomes on record-ings, high-fis, personal earphone devices,rock concerts, and tolerate background musicin their workplaces, restaurants, airplanes,and what not. Is that all right? Similarly, weought to wonder whether it is reasonable toengage in sports. I ask people, isnt theresomething funny about grown people gather-ing in a huge stadium to see other grownpeople kicking a ball from one end to theother? Each of those persons is using a mul-titrillion synapse brain. It would be fun to askthe religious ones to consider whether it isnot a sin to waste such wondrous hardwareon watching adults kicking balls around? Myown view is that this is less a sin than asymptomof infection by a parasitic meme(namely, one that carries the idea that suchan activity is o.k.) which has self-propagatedthrough our culture like a software virus, acancer of the intellect so insidious that virtu-ally no one thinks/dares to question it. Now,in the same way we see grown people playingand working in the context of popular musicthat often repeats a single sentence or melod-

ic phrase over and over and over again, andinstills in the mind some harmonic trick thatsets at least part of ones brain in a loop. Is ito.k. that we, with our hard earned brains,should welcome and accept this indignityor should we resent it as an assault on an evi-dent vulnerability?

Music as a Device for DirectingHuman Activity

OL: You have suggested somewhere thatmusic is often used for escaping the painful-ness of thought.

MM: Certainly that is how music seems tobe used at times. It seems very much asthough a person can exploit music as anexternal intervention (in contrast to using aninternal, perhaps chemical, regulatorysystem) to suppress one or another part of thebraine.g., parts that might otherwise beoccupied with sexual or social or other typesof thinking which that person may presentlynot want to entertain. Clearly, people use


5/15

music for directing their mental activities.After all, thats what it means when we speakof music as stimulating, or as soothing, as likean opiate for relieving pain or anxiety. Orencouraging us to march and fight, or tosorrow at a funeral.

OL: The idea that music can be used to con-trol inner states is, of course, a very old idea.There is after all such a thing as music therapy

MM: There are also other ways of listeningto music that people like you and I use a lot.Sometimes when hearing some music, I reactin the ways we just mentioned, but at othertimes, when my attention is drawn to music,I find myself more concerned with how tomake music that sounds that way. Whatsthat sequence? How would you finger it? Howdid the composer get that idea? What would Ihave to learn to be able to do that myself?Heavens, how appropriate to double the hornand the oboe here. Did the composer figurethat out anew or get the idea from some previ-ous piece? Thus music can be the most stress-ful of all activitiesfrom the perspective ofthe potential composer, because each intrigu-ing new idea portends some unmasteredaspect of ability, or newly recognizeddeficiency in ones own musical machinery.From this viewpoint, it can be very stressfulto find oneself forced (from somewhere elsewithin oneself) to like a piece of music with-out understanding the psycho-musical trickthat makes it so effective.

OL: So, in that case music is not somethingthat gives pleasure.

MM: Right. And even when it does give

pleasure, there is no reason to take that plea-sure at face value. What pleases you canalso control you, by causing you to do some-thing other than you would have done other-wise. The way an adolescent can be enslavedby, and infatuated with, a pretty face borneby a person with no other evident merit. Ithink Paul Goodman once suggested thatPleasure is not a goal; it is a feeling thataccompanies an important seeming activity.

Music as a Tool for EmotionalExploitation

OL: Apparently, then, music can be used inpart as an exploitative device?

MM: Yes. To be a popular composer, what isit you must do? Perhaps you must learnenough tricks that cause people to have pleas-ant sensations without too much stress. Whatare the tricks for making something thatcatches on in a listeners mind, and keepsrepeating long after the performance? There isa superb novel, The Demolished Man, by Alfred

Bester that depicts how the tunesmiths of thefuture develop jingle-composing to such adegree that it can be used, in effect, formental assassination. A victim infested bysuch a tune is helpless to do any useful work,and must hire a specialized therapist to helpremove the tune from memory.

OL: However, wouldnt such a use of musicentail knowing the person one is imprisoningvery well?

MM: That is a profound question: how uni-versal are our musical techniques? And evenif there are powerful universals, there mustalso be powerful non-universals and, so, per-haps in the future when we have better brainreading instruments, people may commissioncomposers to produce works designed not foraudiences, but for particular clients? Surelyyou can write a much better piece of musicfor a single listener (better, of course, inthat persons view). And then perhaps it willbe considered most masterful to write a piecethat only that single client will adore, andeveryone else will abhor.

OL: Now, if such a work were possible,wouldnt we have almost no reason at all to

understand music as something that has uni-versality, and addresses itself to some charac-teristic people share?

MM: Good question. One could argue thateven if we find compositional techniques thatare widely universal, in the sense that theyevoke strong and similar reactions in most lis-teners, those techniques would by that fact bein some sense rather shallowbecause, inbeing so nearly universal, then it must simplybe because of filling some niche, or exploitingsome mechanism, or taking advantage ofsome bug that all human brains have. Con-sider how many people tend to tap the foot

to the rhythm of a piece of music withoutknowing that theyre doing it. An alien beingmight regard that as some sort of mechanicalbug, even if humans regard it as natural.

The Present AI and Music Scene

OL: If we look at what people in AI and musicdo these days, well find that problems likethose we have discussed are on nobodys

Articles

FALL 1992 35

the enterprise of making an artificialintelligence involves not only some tools butalso a larger-scale outlook.


6/15


7/15

straints that motivate an author or composer.I suspect that when we learn how to do that,many regularities that today are regarded asgrammatical will be seen as results of how thecomposers motivations interact with theknowledge-representation mechanismsshared by the composer and the listener. Inany case it seems to me that, both in musicand language, one must understand thesemantics of tension-producing elementsatleast in the forms that resemble narrative.Each initial discord, be it melodic, rhythmic,harmonic, or whatever, can be seen as a prob-lem to be later resolved. A lot of what a com-poser does is setting up expectations, andthen figuring out how to frustrate them. Thatgives the composer some problems to solve.The problems and their solutions are thenlike elements of a plot, and compositionbecomes a kind of story telling.

OL: If we could understand composing as avariety of story-telling, and performed music

as a pseudo-story, wouldnt that help us toarrive at a theory of musical discourse?

The Heart of AI

MM: Yes, indeed. And if we hope to apply AIideas to that, well want to exploit the besttools that AI can offerdifferent sorts of pro-cessing, different ways to represent knowledge,and so forth. But the enterprise of making anartificial intelligence involves not only sometools but also a larger-scale outlook. And AIwill fail to illuminate music, just as linguisticsdid (because of trying to isolate meaning apart

from syntax) if we use only the parts but rejectthe heart. For just as linguistics has a hearttofind how we communicate, so AI too does havea heartto find out how machines can bemade to solve significant problems. Now, inpast years we have seen a certain amount ofapplying AI tools to traditional music analysis.But I would like to see more from the heart ofAI, the study of problem solving, applied toissues of how you solve musical problems.

Making a Composer

OL: So, then, for you to apply AI to music, if

one can say applyMM: would be making composers, or at

least listenersOL: By making, do you mean to produce

a robot-like creature that does certain thingslike, observably composing?

MM: Yes, indeed. And in the case of listen-ing it would have to know when to say oh,this is exciting, or how very tender, andthe like. I havent seen much of that.

OL: In Japan, one has built a robot that iscapable of reading music and then playing iton the piano.

MM: Yes, that fellow at Mazda.OL: Is that something you have in mind

here?MM: Not at all. Because Im more con-

cerned about what happens at larger scales ofphrase and plot. Our listening machinewould have to understand the music wellenough to recognize from each moment tothe next which problems have been solved,and which remain open.

OL: How would such understanding haveto become manifest?

MM: Well, for example, an understandinglistener can hear a piano concerto and appro-priately say things like that was a good ideahere in the cadenza, but he didnt carry itthrough. Id want the robot to make similaranalyses.

OL: To do that, the robot must be able to

recognize solutions, good or bad. How wouldit communicate such solutions to others?MM: One way might be to have it write the

sorts of sentences that critics write. Or to havethem work more in the musical realm by per-forming as a teacher does, explainingdifferences by demonstrationLookhow much better it would be to delaya little these notes here, and makethose near the end more staccato,like this, and this. And ofcourse if our machine turnedout to able to produce interest-ing enough interpretations,

then we might be satisfied bythat aloneif many listenerswere to agree that really, thatperformer has a lot of goodideas about this music, andbrings out stuff that I didntrealize was there.

Does Music Need a Body?

OL: For me, this brings to mind the people,especially at MIT, who have begun to buildrobotic insects. This nouvelle AI seems tobe an approach that is in some contrast to the

top-down symbolic approach AI has used inthe past. I dont know whether you see thematter quite my way. Regarding a musicalrobot, for instance, I would want to know howit is programmed. Is it programmed on thebasis of symbolic representations (i.e., highlevel constructs), or rather on the basis of amultitude of simple circuits, or of neural net-worksor do all of these have a role to play?

MM: Well, surely understanding music

Articles

FALL 1992 37


8/15

for some of its effects on being able to arouseemotions; therefore, in order to build a com-petent composing or listening machine, wemight need to understand such activities wellenough to simulate them. Some of this couldsurely be done with the sorts of decentralizedagents and agencies being used in thoseinsect-like AIsbut I dont suppose theywould work well for this unless we managedto program them with roughly the requiredemotions and cognitions. I have never heardof any non-artificial insect that could learn toprefer Schubert to Mendelsohn.

OL: Right. These artificial insects wouldhave to climb the tree of knowledge first.

The Need for anEmotional Culture

MM: Yes, indeed. And to use them inside ourmusic machine, they might have to evolve tobe quite similar to the ones inside our humanbrains. Our reactions to the sound of a violinmay exploit our reactions to the pitch centersof female voices; the sound of a cello mayseem more male. A machine that was reallycompetent to listen to nineteenth centuryclassical chamber music might well needsome knowledge-understanding of humansocial affairsabout aggression and concilia-tion, sorrow and joy, and family, friendshipand strangership. And of course there areother constraints on how we use those instru-ments. Generally the cello will have to playlonger and slower notes that emphasize foun-dational notes of the chord and theres noth-

ing sexual about thatbut nevertheless, ourhuman perceptual experience will tend toassociate the higher notes with childrensvoices, the middle octave with female, andthe lower one with male voices. It might evenbe natural to link yet lower tones with thoseof large fierce animals.

OL: There is sufficient historical evidencethat suggests what you are saying is entirelyto the point. The violin was developed inRenaissance Italy, by people who were veryeager to simulate the human voice. For them,the human voice was the telos which theystrove to re-embody in the instruments of the

violin family.MM: I have to admit that the first time Iencountered a modern synthesizer, the mostexciting stop for me was the choral stop, thehigh female voices. Having been a pianist allmy life, it was so astounding to be able totouch a key, and have those soprano voicescome out, and basses, tenors, contraltos.Completely entrancing. For an hour or two,like a fairyland. But eventually it seemed

requires many levels, and a good deal of cul-tural knowledge.

OL: Dont you also need a body level tobuild an artificial composer or a listener, torender actions?

MM: I dont think that this will be impor-tant.

OL: So you dont consider this to be a criti-cal problemthat emotion, as distinguishedfrom cognition, in all known cases manifestsitself through an organic body?

MM: Not really, because I dont expect thatemotional behavior will turn out to be singu-larly hard to simulate. Im inclined to agreewith Niko Tinbergen that the basic emotionscome from the activities of various almostseparate processes, brain-systems that eachhave different, and rather clear-cut goals. Forexample, in the brain stem at least a dozensuch systems have evolved and, to a largeextent, they behave like different animals. InThe Society of Mind I called them Protospe-cialists. When youre low on sugar, the pro-tospecialist for hunger gets activatedandcauses you to apply your available knowledgeto find a way to get food. If youre too cold,then another specialist gets activated, sup-pressing the others; it uses the availableknowledge to get out of the chill and into thesun, or to cover your body with insulators, orto huddle together with your friends. I thinkmost people assume that emotions are verydeep and complicated, because they seem sopowerful and hard to understand. But myview is just the opposite: it may be largelybecause they are basically simplebut wired

up to be powerfulthat they are hard tounderstand. That is, they seem mysterioussimply because theyre separate and opaqueto your other cognitive processes. However,there is also another thing that makes theseemotions hard to understand, at least in theirolder, more adult forms. It is that althoughan emotion may be simple by itselfforexample, a brain center that is geneticallywired to learn actions that keep onewarmthe end result of what it may learncan become arbitrarily complicated. Theproblem is that each infantile emotion, suchas hunger, defense, nutrition, or sex, can

eventually acquire a huge cognitive systemfor achieving its own goal, perhaps byexploiting the other emotions. And presum-ably virtually every adult cognitive activitydevelops in some complex way from infantileactivities. So I see as false the commonsensedistinction between intellect and emo-tion. It would be better to emphasize thedevelopment over time from simple to com-plex. Nevertheless, music appears to depend

Articles

38 AI MAGAZINE


9/15

wrong that I couldnt make these voices talk,they were always making the same bleatingsound. But what a marvelous experience.

Requirements of Being aMusic Scientist

OL: We were wondering why none of theseideas about cognition and emotion seem tobe of relevance in present-day AI and musicresearch

MM: I think thats because the AI people

have suffered from the same misconceptionthat most cognitive psychologists have suf-fered from, viz., the idea that well, well dothe easy things first, like understandingmemory and simple reasoning and so forth;but emotions are surely too difficult, so letsput off researching them for now. I oncecame across a statement by Freud in which hecomplains along lines like this: people thinkI work on emotions because those are the pro-found, important things. Not so, What Idreally like to understand is common sensethinking. And it is only because thats sodifficult, so incredibly complicated, that I

work on emotions insteadbecause emotionsare so much more simple.

This is why I like Douglas Lenats project tobuild a commonsense data base. Im sure thatFreud would have liked it too. This is a bigproject; the cYC data base will involve mil-lions of different items, fragments of our cog-nitive machinery. But until our machineshave access to commonsense knowledge,theyll never be able to understand storiesorstory-like music. One reason that AI has notgone very far in such domains is becauseresearchers have been afraid to say: I thinkemotions are simple enough that we canmake useful models of them. Im not sayingthat emotions are trivial; they surely involvesome complicated machinery. But you cantmake progress unless youre willing to beginwith simple theories, to serve as first approxi-mations to the science well eventually needto understand musical activities.

OL: Could you explain that a little further?MM: Sure. I mean that one should not be

daunted by the apparent complexity of emo-tions, because it may be feasible to get a goodstart by making what may seem to be over-simplifications. You could begin by sayingmaybe there are only three basic emo-tionshappy, sad, and whatever you like.The sentics model of Manfred Clynes stipu-lates seven. It doesnt matter, so long as youstart somewhere. I want to avoid the disasterthat has befallen syntactic linguistics which,for all its apparent accomplishments, is deadin the water because of failing to include even

the simplest caricature of a theory of whatwords mean. In my view, syntax has led tosome useful discoveries, but to fewer thanwould have come from building more com-prehensive models of language productionand understanding. So I would like to seesome music-theorists start with models basedon simulating a few postulated emotions,each linked to a few procedural rules abouthow various sorts of rhythmic, harmonic, andmelodic elements might work to arouse, sup-press and otherwise engage a listeners variousfeelings and attitudes. Unless you start some-where youll go nowhere.

OL: Are you saying: the best way toapproach music is to go at it from somehypothesized basic emotions, rather thanfrom ideas about problem solving, or the ideathat music is a sort of intellectual game likechess, or is something explainable by psy-choacoustics?

Music as a Pseudo-Story

MM: Well, emotion might not be the rightword, but we must somehow formulate andengage what we believe are the importantmusical problems. And clearly, an importantaspect of understanding music experience isthe listeners experience of apprehensions,gratifications, suspense, tensions, anxieties,and reliefsfeelings very suggestive of painsand joys, insecurities and reassurances, dreadsand reverences, and so forth. So, just as agood story confronts us with conflicts andresolutions, so do good musics, which takeover the listeners disposition with feelings

Articles

FALL 1992 39

until our machines have access to commonsense knowledge,theyll never understand stories


10/15

compact theory that solves many more prob-lems. A theory that takes account of bothtransmission and reception, of composingand listening.

Chapters 22 and 26 of my book The Societyof Mind propose such a theory of how lan-guage works. The idea is that when a speakerexplains something to a listener, the goal is toproduce in the listener a structure that resem-bles a certain semantic network in the speak-ers brain. Many linguistic tricks are used forcontrolling the growth of that network. Myconjecture is that mechanisms of this sortcould lead to good descriptions of what utter-ances are understandableand could do this,I suspect, with simpler and fewer rules than

the usual kinds for describing senten-tial grammarbecause many

grammatical rules that seemseparate and indepen-dent might result fromthese deeper mecha-

nisms. It is OK to beginby looking at surfaceregularities.

Musicological Taboos

OL: Its certainly a step forward to have begunto describe musical products in terms of therules that may underlie their generation(although in most cases, such as Lehrdahland Jackendoff, such rules are rarely if evertested empirically, and therefore have littlecognitive reality). Rules codify actions, or ele-ments making up an action. For the longest

time, a taboo has existed in music theory andmusicology regarding any attempt of under-standing music as an activity. Researchershave typically concentrated on music as anobject, an artifact, a product, rather thanstudying the process by which music is gener-ated, both in the composer and the listener.How do you explain this taboo? Are youaware of analogous taboos in other sciences?

MM: I think the taboo made sense in thepast simply because before the advent ofmodern theories of complex informationprocessing, there really was no useful way tothink about how minds might do such things.

In my view, the theories of philosophersbefore the era of Freud, Piaget, and Tinbergenwere too primitive to provide much insight.Freud recognized that higher forms of humanthought involved the pursuit of goals byacquiring and exploiting knowledge. But itwas not until the 1960s that workers likeAllen Newell and Herbert Simon formulatedadequate theories of systems with goals. To besure, there were earlier attempts to base psy-

like: Im worried that something bad willhappen unless this conflict is resolved. Prob-lems and solutions. Once I was trying tofinish a two-part invention but there wassomething wrong with it. I asked a musicalfriend for help and was surprised to hear asimple answer. Look here, you went up intothis octave but then you never made any-thing of it. You cant do that. When you let avoice get into a new range, like breaking newground, you must have a good reason for it,and you must also think of a way to get thatvoice out of there afterwards. But here youjust left those notes hanging up there, and Iended up wondering what will happen tothem. Now I wonder if this doesnt reflectsome sense of territoriality instinct, in whichthe listener is disturbed by notknowing who controls which area.

OL: Well, it also relates to rules ofmusical discourse, doesnt it? If youput forth an argument, you have to

follow it up MM: Rightand that reminds meagain of the constraint-like opera-tions of Lehnerts Plot Units. Thosedangling high notes evidentlyrequire follow-up, some resolu-tion, and unless the composerbrings them down, the listener is left withsome unresolved conflict, concern, or prob-lem. As though one of your emotional proto-specialists were left in an active state. Afterall, what are musical problems if not prob-lems having to do with the resolution ofconflict?

OL: Any book on counterpoint will instructyou that having made a particular step,there are only so many things you can do, aswell as not following up this step is a mis-take in terms of the rule system we are adher-ing to. Whereas you were saying, followingup some compositional decision is requiredfor being emotionally or story-wise consistentin ones music.

Where All Those Rules Go

MM: Yes, and that raises questions about thenumbers of and characters of those rules

and those two dimensions are far from inde-pendent. Indeed, this is what I was thinkingabout in that analogy with syntax. For if youtried to analyze contrapuntal music, youcould write rule after rule to constrain its sur-face structure, but youd never get quiteenough rules to tell composers what to do.However, I suspect that if we managed tobuild the right sort of model for musical plotsand conflicts, we might end up with a more

Articles

40 AI MAGAZINE


11/15


12/15


13/15

Music as a Bug betweenBrain Areas

MM: Yesand perhaps in some years fromnow well be able to see those processes evenmore directly by using high-resolution brainactivity imaging instruments. I would certain-ly like to be able to see whats happening inmy brain when I improvise, because I have

virtually no direct insight into that; Imthinking about other things when I doitsometimes even about some completelydifferent piece of music. I suspect that mymusic production machinery is mostly in mymotor brain regionsthat is, in the front halfof the brainmore than in the back or sensoryhalf of the brain. (I really hate to hear somany people repeating superstitions aboutthe left and the right sides of the brain; thebrain has hundreds of different parts, so youcan divide it in two in many ways.) The resultof this is really annoying to me; I have greatdifficulty writing down music on paper untilmy hands play it on the keyboard so that Ican hear it. Now, I dont actually mean hear-ing, literally, because I can play it on an imag-inary keyboard in the air. But I cant hear themusic nearly so well without moving thefingers. This probably means that in order toclose the loop, I have to use some bundles ofnerve fibers that tunnel under the centralsulcus from the motor regions back to thesensory regions of the cortex. It would be niceto understand thisand theres no reasonwhy not, with new instruments.

OL: If you are right with your hunch thatto understand how humans make music weneed to know how they use their brains,then, of course, we are right at the questionof how music developed in the history of thehuman brain.

MM: Yes, and that should be extremelyinteresting because music seems to have noevolutionary origin. So I suspect that in eachmusical person there has been some earlyincident in which some musical process-knowledge comes to occupy some piece ofbrain that isnt dedicated to something else,and it probably happens somewhat different-ly in each person. I dont think anybody has

mapped this very much.OL: You just said something for me quiteastonishing, viz., Music seems to have noevolutionary origin.

MM: Well yes, in the sense that one cantsee anything much related to it in animalsdescended from our ancestors.

OL: There are no traces MM: of musical concerns or abilities in

our ancestors or relativesso far as I know,

no sign of musical interest inany other primates. Nor, so faras I know, in any otheranimals. I know of no ani-mals that even tap theirfeet to rhythms or any-thing like that. I dontconsider bird-songs to besongs; yes, they havecommunicative functionsbut no reason to relatethat to music. Or whalesongs. We call themsongs but we still haveno idea about their func-tions and again, I see noreason to call them musical. (I suspect thatthey contain information about major oceancurrents or coastline features or lists of wherevarious individuals have been seen, or otherfunctional things like that. But no oneknows.) Of course we all meet romanticpeople who maintain that their plants enjoyand thrive in musical environments. But theonly careful experiment Ive seen onlydemonstrates that vibrations tend to retardplant growth by injuring rootlets. My bestguess is that music became possible becauseof some anatomical innovation that just hap-pens to facilitate interactions between other,older functionsfor example between someof the brain that does planning for paths inspace and some of the parts involved withlanguage, or story-like memory systems. Ifthat were the case, it might explain why hear-ing certain kinds of sounds might come to

give you the feeling that you understoodsomething, or give you the experience ofbeing in some other place. If so, we couldeven regard music as being related to somesort of hardware bug perhaps involvingbrain areas concerned with visualization,kinetic imagery, language, or whateversomecombination of structures that could lead toalmost autonomous kinds of activity that welabel musical.

OL: Looking back at ancient Greek culture,youll find that the term mousike denotessomething very specific, viz., the linkagebetween poetry and sound. This culture really

has no term for instrumental music as weknow it. Sound-making that was not linked tothe human voice was considered as being akind of techne, and thus much more lowlythan mousike. It seems, then, that the splitbetween poetry and music, and the cominginto being of music as we know it today (i.e.,instrumental and electroacoustic musics) is arather recent event, even in terms of the shorthistory of humanity.

Articles

FALL 1992 43


14/15

tures of storiesand of musical composi-tions. (In Roger Schanks recent theories oflearning and reasoning, story-structures playcritical roles.) Or, perhaps, most important ofall, we need to know more about commonsense reasoning, and the data structures thatunderlie it. Because no computer in the worldtoday yet knows the meaning of enoughwords to understand a story.

OL: And this will remain true for sometime to come

MM: That is the gap that Doug Lenat istrying to fill, viz., by working on a commonsense data base. And similarly we need somesort of musical common sense data base.

OL: So it would seem.MM: In college, I attended one of Walter

Pistons courses. He used to complain that noone knew enough about what makes goodtunes. He said that there were only rules ofthumb, like the rules for good manners or goodbehavior; a good melody must show somesense of direction and return; it shouldnt betoo jerkyand it shouldnt be too smooth

but there were no useful formal theoriesabout such things. And I remember thinkingthenbefore I ever thought about AImaybethere actually cant be such rules. Becauseperhaps it works a different way, by analogy.Perhaps I match each tune I hear with a hun-dred tunes that I learned when I was a baby.A machine to do this might have to know allthose nursery rhymes and folk songs and lul-labies and Christmas carols. Perhaps thereason why we like certain tunes is largelybecause of already liking other similar ones?

OL: Most likely.MM: And if that were so, then the impor-

tant thing about a tune would be how well itresembles some of those repertoried tunesand what are their interesting differences. Inany case, those music courses didnt seem toknow how to tell me how to compose. Theywere infuriating

OL: They still are MM: Now, today, things ought to be better.

We have so many new good ideas. Case-basedideas, grammatical ideas, problem-solving

MM: Right.OL: Of course, one would have to look at

other cultures.MM: Surely. And in any case, however the

brain is involved, music clearly interacts withmany memory mechanisms. Not only inmusic but in other realms, it seems mucheasier to remember things that are grouped inregular temporal structures that resemblerhythm and meter. The other day, I com-plained to Carl Sagan that we have no goodtheory of why people are so attracted to regu-lar rhythms. He pointed out that themothers heartbeat is a prominent context ofevery babys development. Im sure that thereis something to thatexcept that dogs andcats hear heartbeats too, but do not seem totap their feet or otherwise show much sign ofbeing affected by music.

A Musical Common SenseData Base

OL: Indeed, music is a human privilege, a

reflection of the human condition, and itprobably holds the key for much that psy-chology is attempting to unearth abouthuman nature. Alas, our time is almost up.Therefore, to conclude our conversation,could I ask you what you think has beenachieved in the field of AI and music so far,and beyond that, what you think should beworked on, or might be worked on, in thenext decade or so? Is that a fair question toput to you?

MM: A vital question. And to guess at ananswer, we need to understand our positionin history. AI is only 30 or 40 years old, but

people are always asking: What has it donelately? or What are its important achieve-ments? The trouble is, we cant really say,because what seems important at onemoment may not be what turns out to havebeen important 10 or 20 years later. We cantbe certain which are the good ideas yet. Per-haps we need more research on case-basedreasoning. Or on structures like Lehnerts plotunits, to understand more about the struc-

Articles

44 AI MAGAZINE

perhaps most important of all, we need to know moreabout common sense reasoning


15/15

ideas. Expert-system ideas. So much has beenaccomplished that we may be ready for therenaissance of many an old inquiry. Perhapsthe research community as a whole alreadyhas a critical mass of the needed insightsand is only waiting for someone to see a wayto pull them together. Anyway, in my viewthe most critical thing, in both musicresearch and general AI research, is to learnhow to build a common music data base. Butnobody in music research works on that yet,do they?

OL: Not that I know of There exist, to besure, small data bases of scores of the so-called Western tradition, but that is not whatyou have in mind here

MM: Its mostly the same in AI. Most hopeto understand language by using compacttheory-tricks, like formal grammars. But thatsimply cant do enough by itself. You needsome sort of data base, of experience. A fewlittle stories about each word. And the samefor music. Surely you cant react properlyunless you possess some stories about eachchord sequence, or melodic turn, or rhythmicvariation.

OL: So, thats what you call a commonsense musical data base: a collection of storiesabout important compositional or auditoryconstructs?

MM: Precisely. And consequently, despiteall those popular, fancy, formal theories, weremissing all the substance beneath. We need amusical Lenat to start a musical CYC project.

BibliographyAlker, H. R.; Lehnert, W. G.; and Schneider, D. K.

1985. Two Reinterpretations of Toynbees Jesus:Explorations in Computational Hermeneutics. In

Intell igence and Text-Understanding: Plot Units and

Summarization of Procedures, ed. G. Tonfoni, 5194.Parma, Italy: Edizioni Zara.

Laske, O. 1986. Toward a Computational Theory of

Music Listening. InReason, Emotion, and Music, eds.L. Apostel et al., 363392. Ghent, Belgium: Commu-

nication and Cognition.

Laske, O. 1984. KEITH: A Rule System for Making

Music-Analytical Discoveries. In Musical Grammarsand Computer Analysis, eds. M. Baroni et al.,

165199. Florence, Italy: Leo S. Olschki.

Laske, O., and Drummond, S. 1980. Toward an

Explicit Theory of Musical Listening. ComputerMusic Journal 4(2): 7383.

Lehnert, W. 1982. Plot Units: A Narrative Summa-rization Strategy. In Strategies for Natural Language

Processing, eds. W. Lehnert and M. Ringle. Hillsdale,

N.J.: Lawrence Erlbaum.

Lehnert, W., and Loiselle, C. 1985. Plot Unit Recog-

nition for Narratives. In Intelligence and Text-Under-

standing: Plot Units and Summarization of Procedures,ed. G. Tonfoni, 947. Parma, Italy: Edizioni Zara.

Minsky, M. 1989. Music, Mind, and Meaning. In

The Music Machine, 639656. Cambridge, Mass.: The

MIT Press.

Minsky, M. 1986. The Society of Mind. New York:Simon and Schuster.

Minsky, M. 1980. K-Lines: A Theory of Memory.

Cognitive Science 4:117133.

Tonfoni, G., ed. 1985. Intelligence and Text-Under-

standing: Plot Units and Summarization of Procedures.Quaderni di Ricerca Linguistica 6. Parma, Italy: Edi-

zioni Zara.

Marvin Minsky has contributed to the domains ofsymbolic description, knowledge representation,symbolic applied mathematics, computationalsemantics, and machine perception. His main con-cern over the past decade has been to discover howto make machines do commonsense reasoning. Thefoundations of his conception of human psycholo-

gy are described in his book The Society of Mind.Minsky is a 1990 recipient of the Japan Prize thatrecognizes original and outstanding achievementsin science and technology.

Otto Laske is a composer, and the President ofNEWCOMP, Inc., The New England Computer ArtsAssociation, Boston, Massachusetts.

Articles

FALL 1992 45

the most critical thing, in both music reserch andgeneral AI research, is to learn how to build a commonmusic data base.

laske: a conversation with marvin minsky

Documents