Musical Time in Visual Space

Brian Evans

Department of Art, University of Alabamabrian.evans@ua.edu

The laws of harmony are the same for painting and music.Maurice Ravel

AbstractThere is a renewed interest in composing visual music. Atime-based, visual art, using abstract material, can look tofundamentals from traditional Western music practice as aplace to start. These fundamentals map into the visual artfoundations of design and color. Visual equivalents of themusical ideas of consonance, dissonance, tension, andrelease, can be easily understood and applied to thecomposition of abstract animation. With the current state ofdigital technology the medium of time-based sonic/visual artis now available to any composer interested in expandingtheir artistic expression into the visual domain.

1 IntroductionMusic composers create coherent temporal structures

using the materials of soundpitch, timbre, rhythm, etc.They organize sonic events. The hope is that the unfoldingof these events has a quality that would be described asmusical. There is no broad agreement on what musicalmeans, but there is a wide array of sonic art that most callmusic and so consider musical, from plainsong toMississippi blues, North Indian ragas to the extended dronesof the Aboriginal digeridoo.

Musical motion (at least in most Western music) iscreated through tension and resolution of tension throughcontrolled dissonance resolving to consonance. This tensionand release is expressed through harmonic pitchrelationships and rhythmic patterns that together develop aschords, motives, phrases and cadences. These are the basisof a musical syntax and provide a framework upon whichexpressive, musical ideas are articulated.

Like music, the fundamental dimension of abstractanimation is time. Many animators of non-representationalimages seek a visual unfolding of events that they describeas musical. Examples are many, from the work of VikingEggling (1923) and his visual counterpoint to the digitalharmony of John Whitney (1980), to recent time-basedvisual works by formally trained music composers. (Evans,2003, Miller 2002) Hence many experimental animators callwhat they do visual music. Can a truly musical expression

be made using non-narrative visual space? Are there visualcounterparts for the traditional building blocks of musicaltime?

Focusing on motion and the materials of the traditionalart foundations, design, pictorial composition, and colorharmony, a time-based grammar of visual music can bedeveloped. The construction of this grammar starts bydefining visual consonance and dissonance. As in musicalharmony, controlling movement from dissonance toconsonance supplies a means of moving through timemusically. With a grammar established a visual musictheory can be developed. Visual music can be composedand abstract animation, in truth any time-based visualexpression, can be seen as musical.

2 Music FoundationsStravinskys statement, Music means nothing outside

itself, illustrates a common mindset, held by musiccomposers for centuries. (Stravinsky, 1956) Musiccomposition was a formalist activity. This was also amodernist view. Modernist ideas have been somewhattattered over the past few decades, but they can still presenta viable basis for discussing musical time.

We can start with a simple definition of music as thestructuring of time with the materials of sound patterns.(Lets exclude literary forms such as song, opera andtheatre.) The development of musical instruments has ahistory measured in millennia. These instruments weredesigned as generators of abstract sounds, without a referentin the real world beyond the instruments themselves. Theseabstract sounds were used to create temporal structures suchas sonatas, fugues and symphoniesabsolute music.

In general Stravinsky is talking about art for artssake. From this we can define musical as an aestheticresponse to the perception of sonic pattern t h eappreciation of significant form, a primary focus ofmodernism and its formalist leanings. (Bell 1914)

In music these patterns are built on foundations ofrepetition, contrast and variation. There is no structure, nopattern, without repetition. Repetition by itself can of coursebecome boring, so contrast is useful in keeping the listenerengaged. As any utterance is multi-dimensional, it ispossible to repeat in one dimension while contrasting

Proceedings ICMC 2004

another. For example playing the same notes on a differentinstrument repeats the pitches while contrasting the timbre.This is variation, a more subtle but effective technique forcreating and developing musical pattern.

The fundamental dimension of music is time. To makemusic is to move coherently through time. This is evident inWestern tonal music where the music moves the listenerthrough time by first establishing a tonal center (balance andharmonic stability). From this stability the listener is movedto a sonic dissonance that builds tension. Tension isresolved through resolution back to tonal consonance andstability. The music returns to the comfort of familiarity.Any activity that moves us through time, including dance,poetry, theatre, etc., uses the idea of tension/release. Boygets girl Boy loses girl. Boy gets girl.

From here we can construct a syntax, a consensus on thecomponentspatterns that repeat, vary and contrast, and thecombining and ordering of these patterns to create tensionand resolution. Combinations of sound materials becomemusical statements. The motif, the minimal material neededto expresses a musical idea, becomes the building block ofthe phrase, a musical line. Phrases are combined andarticulated in time by cadences or punctuation points.

3 A Visual MusicGauguin in describing his painting Manao Tupapau said,

The musical part: undulating horizontal lines; harmonies oforange and blue, united by yellows and purples (theirderivatives) lit by greenish sparks. The literary part; thespirit of a living person linked to the spirit of the dead.Night and Day. (Chipp, 1968)

Here we get an insight into a visual artists separation ofform and content. Gauguin used the word musical todescribe the formal aspects of the work. When visual artiststalk about composition they are focused on the design, theorganization of materials in 2D or 3D space. Composition istraditionally only one aspect of a visual artwork. Theformalist explorations of the 20th century illustrate the desireby visual artists to achieve a purely musical expression, tocreate a visual music by bringing form and content together.

Kandinsky, considered by many to be the father ofmodernist abstraction, wrote, A painter, who finds nosatisfaction in mere representation, however artistic, in hislonging to express his inner life, cannot but envy the easewith which music, the most non-material of arts todayachieves this end. He naturally seeks to apply the methodsof music to his own art. And from this results that moderndesire for rhythm in painting, for mathematical, abstractconstruction, for repeated notes of color, for setting color inmotion. (Kandinsky, 1914)

This became truly possible with film. To follow the pathestablished by traditional music practice, visual music wasformal, abstract and temporal. In the early years of film,abstract animation was developed by visual artists whodesired to compose visual music. (Russet and Starr, 1976)

The impulse was quickly lost (but never abandoned) in theHollywood deluge of character animation and big screenstorytelling. With developments in technology over the pastdecade there is a renewed and growing interest in visualmusic composition.

Art theorist Rudolph Arnheim says, One of the basicvisual experiences is that of right and wrong. This idea isthe basis of design foundations as taught in most art schools.(Arnheim, 1966)

Visual rightness is visual consonance. Anyintroductory design book talks about how to achieve visualbalance and harmony and good visual composition. (fromDow, 1899, see Figure 1, to Lauer and Pentak, 1999) Fromthis premise composing visual music is a simple process. Ifrightness is codified and understood, wrongness is easilycreated by not being right. Movement from visually wrongto visually right is a construction of tension/release.

Film and video artists have developed their owngrammars over the past century, and movement fromtension to release is a part of those grammars. The practiceof storyboarding is an example. (Begleiter, 2001) A directorplans a scene as a series of moments, composed as a seriesof drawings that guide the eventual motion, framing, andcomposition of the picture plane as it changes in time. Ascene is built on phrases that are delineated and cadencedwith moments of visual rightness.

The documentary films of Ken Burns are good examplesof this. (Burns 1990) The camera, under strict motioncontrol and compositional planning, pans over stillphotographs. Motion and picture composition, structured inphrases, punctuate time with cadences of visual rightness.The viewer moves through time musically.

Color is another visual dimension available to the visualmusic composer. Ideas of color harmony have developedover the past centuries based on the premise that there is an

Figure 1. Illustration of visual rightness from the bookComposition by Arthur Wesley Dow, 1899. Dow defines

the image on the left as a motif, and the other four asvariations.

Proceedings ICMC 2004

experience of balance when color combinations reduce to anexperience of neutral. (Cheuvral, 1854 and Albers, 1963)For example combining a primary and its opposingsecondary color is considered harmonious. When visuallymixed these colors reduce to grey. Grey is the most neutral,balanced color experience. Moving from weighted tobalanced color spaces can also be effective in moving usthrough time in visual space. (Evans 1990)

Consider the 1939 film of The Wizard of Oz. WhenDorothy is safe in the stability of home the film is in blackand white. When she is off on her adventures in Oz the filmis in color. The film returns to neutral black and white at theend, when Dorothy returns to the comfort and safety ofhome. (Baum, 1939)

Codified art foundations in design and color theory areuseful as entry points for those interested in composingvisual music. These foundations were developed empiricallyover centuries by practicing visual artists. Over the pastdecade there has been significant research in theneurobiology of vision. This research validates and expandson these art fundamentals. Our knowledge of how the eyesand the brain process and understand visual information isgrowing rapidly. This research offers a fertile field uponwhich to continue exploring time-based visual art, andindicates promising directions for future work. (Zeki, 2000,and Livingstone, 2002)

4 After FormalismJohn Cage had a different view of musical time. In the

book Silence he wrote, Music means nothing as a thing.(Cage, 1958) With his music composition 4' 33" he removeshimself from the modernist viewpoint and loosens up theidea of musical. He simply frames time by specifyingduration, claiming that all sounds heard in the specifiedduration. whatever they may be, are the musical materials ofthe piece.

Again, the fundamental dimension of music (sonic orvisual) is time. In truth, the most fundamental dimension ofexperience is time. (It is odd that some art schools areintroducing time into their foundation courses, calling it 4D,as an afterthought to two and three dimensional work. Ofcourse for human experience, time is the first dimension asnone of the other dimensions exist for us without it.)

We understand the world through signals receivedthrough the senses. We are able to make sense of the worldbecause the signals are structured. Perceptions of sound andlight are built from the reception of waves (patterns) ofchanging air pressure and electro-magnetic energy. Hencepattern is axiomatic for experience, for life. For us, timepasses as the experience of patterns. Experienced time is bydefinition musical. The set of possibilities is infinitely largefor time-based art.

Formalist ideas of music composition give us a basis forentering visual space with musical intent. We can learnmuch from the traditions of all art forms, as in the end they

are all time-based. Western music tradition is a well-codified and understood practice and can provide guidancein the creation of visual music work. Whether formalismcan succeed as an end in itself continues to be debated. Itdoes continue to be of value as a technique, a device ofconstruction, and a basis for exploring visual musiccomposition.

Technology has opened visual space to composersinterested in expanding their musical ideas into new media.Fundamentals of music composition can be easily mappedinto time-based visual design and new avenues of musicalexpression are possible. As Morton Feldman oncecommented to Cage, on realizing the new possibilities thathad opened up for composers in the early post WWII years,Now that things are so simple, theres so much to do.

ReferencesArnheim, R. (1966). A review of proportion, Module,Albers, J. (1963) Interaction of Color. New Haven, CT: YaleBaum, L. F. (1939). The Wizard of Oz. Warner Studios. (Motion

Picture)Begleiter, M. (2001). From Word to Image, Storyboarding and the

Filmmaking Process. Studio City, California: Michael WieseProductions.

Bell, C. (1914). Art. London.Burns, K. (1990). A Civil WarA Film by Ken Burns. PBS Home

Video.Cage, J. (1980) Silence. Wesleyan, CT: Wesleyan University

Press.Cheuvral, M. E. (1854). The Principles of Harmony and Contrast

of Colors and their Applications to the Artst. West Chester,Pennsylvania: Schiffer Publishing Ltd.

Chipp Herschel B. (1968). Theories of Modern Art. Berkely andLos Angeles, California: University of California Press.

Dow, A. W. (1899) Composition. Garden City, NY: Doubleday.Eggling, V. (1923) Diagonal Symphony: Rhythm 21. Germany.

(Motion Picture).Evans, B. (1990). Temporal Coherence with Digital Color.

Digital ImageDigital Cinema, SIGGRAPH 90, Art ShowCatalog, LEONARDO, Supplemental Issue. London: PergamonPress

Evans, B. (2002). limosa. Tuscaloosa, AL. (Music Animation).Kandinsky, W. (1914). The Art of Spiritual Harmony. London:Constable and Company, Ltd.Livingstone, M. (2002). Vision and Art: The Biology of Seeing.

New York: Henry N. Abrams.Lauer, D. and Pentak S. (1999). Design Basics. Orlando: Harcourt

Brace and Company.Miller, D. (2002). Vis a Vis. Boston, MA. (Music Animation).Russett, R. and Starr C. (1976). Experimental Animation. New

York: Van Nostrand Reinhold Company.Stravinsky, I. (1956). The Poetics of Music in the Form of Six

Lessons. New York: Vintage Books.Whitney, J. (1980) Digital Harmony. Peterborough, NH: McGraw-

Hill.Zeki, S. (2000) Inner Vision, An Exploration of Art and the Brain,

Oxford: Oxford Press.

Proceedings ICMC 2004

IndexICMC 2004 Home PageConference InfoWelcome MessageChairman Invitation

SessionsMonday, 1 November 2004MonAmOR1-Perception/PsychoacousticsMonAmOR2-Computer Music Software and PracticeMonAmOR3-Algorithmic CompositionMonPmSS1-Demos 1MonPmPO1-Posters 1MonPmOR1-Studio Reports 1MonPmOR2-Music AnalysisMonPmOR3-Computer Music Software and Practice IIMonPmOR4-Sonification

Tuesday, 2 November 2004TueAmOR1-Interactive SystemsTueAmOR2-Panel: Abstract Mathematics in Computer MusicTuePmSS1-Demos 2TuePmPO1-Posters 2TuePmOR1-Computer Music Languages and EnvironmentsTuePmOR2-Studio Reports 2TuePmOR3-Computer Music Software and Practice IIITuePmOR4-Analysis III

Wednesday, 3 November 2004WedAmOR1-DiffusionWedAmOR2-History-AestheticsWedPmSS1-Demos 3WedPmPO1-Posters 3WedPmOR1-SynthesisWedPmOR2-Panel: Crossing the divide

Thursday, 4 November 2004ThuAmOR1-LatencyThuAmOR2-Alignment, Segmentation and DecompositionThuPmSS1-Demos 4ThuPmPO1-Posters 4ThuPmOR1-Statistical Models and Parameter Spaces

Friday, 5 November 2004FriAmOR1-Compositional SystemsFriAmOR2-Gestural and Haptic InterfacesFriPmOR1-Music Information RetrievalFriPmOR2-Composition Practice

Saturday, 6 November 2004SatAmOR1-EducationSatAmOR2-AnalysisSatPmOR1-Symbolic ProcessingSatPmOR2-Interactive SystemsSatPmOR3-Panel: Standards in Computer MusicSatPmOR4-Analysis IISatPmOR5-Computer Music Software and Practice IV

AuthorsAll AuthorsABCDEFGHIJKLMNOPQRSTUVWXYZ

PapersAll PapersPapers by SessionPapers by Topics

TopicsAlgorithmic CompositionArtificial Intelligence and MusicAesthetics, Philosophy, Analysis, Appreciation, Critici ...Acoustics of Musical Instruments and the VoiceAudio Analysis/ResynthesisAudio HardwareAudio Signal ProcessingComposition Systems and TechniquesComputer Music and Digital ArtComputer Music Performance PracticeDigital Audio Signal ProcessingEducation on Music and TechnologyGestural and Haptic Interfaces for MusicHistory of Electroacoustic and Computer MusicInteractive Performance SystemsMachine Recognition of AudioMachine Recognition of MusicMIDI ApplicationsMusic AnalysisMusic Data Structures and RepresentationsMusic EducationMusic GrammarsMusic Information RetrievalMusic LanguagesMusic Notation and PrintingMusic WorkstationsMusical InformaticsPerformance InterfacesPsychoacoustics, Perception, and CognitionReal-Time SystemsRoom AcousticsSound in MultimediaSound Diffusion and SpatializationSound Synthesis LanguagesSound Synthesis MethodsStudio ReportsStudio DesignsTuning and IntonationWireless Audio Systems

SearchHelpBrowsing the Conference ContentThe Search FunctionalityAcrobat Query LanguageUsing Acrobat ReaderConfigurations and Limitations

AboutCurrent paperPresentation sessionAbstractAuthorsBrian Evans