Artificial, Natural, Historical: Ambiguities of SyntheticSound in Documentary Film

Julian Rohrhuber

ABSTRACTThis paper discusses the role of synthetic sound in the con-struction of authenticity and historicity of documentary filmin the light of the documentary “All that we have” [9] and itsmaking. It argues that the delimitation between the “nat-ural” and the “artificial” constitutes a key element in thereflection on documentary film sound. After a general in-quiry of cinematic sound (part 1 and 2) it is shown whatrole algorithmic composition can play within a process ofinvestigating notions of “authentic source” and “historicalidentity” (part 3 and 4).

1. THE AUTHENTIC SOURCE

“[Sound film] has conquered the world of voices, but it haslost the world of dreams. I have observed people leaving thecinema after seeing a talking film. They might have beenleaving a music hall, for they showed no sign of the delightfulnumbness which used to overcome us after a passage throughthe silent land of pure images. They talked and laughed, andhummed the tunes they had just heard. They had not losttheir sense of reality.” Rene Clair [3]

Pleasant or unpleasant, many commentators who have expe-rienced the transition from silent film to sound film describea similar experience: It seems as if film had woken up, hadlost its dream-like character. In the new genre of ‘talkies’,the human voice had entered the scene and brought withit a whole field of narrative possibilities, which were usedexcessively, and still today dialogue forms one of the maindimensions along which most films are organized. Dialogueis to a great extent tied to the correlation of voices to theiraccording persons: It is the person who is the cause of thevoice, and, from the perspective of the narrative, it is theperson’s network of intentions, obligations and contingenciesthat causes what is said.

The dream-like character of the silent movie period might bea result of a distortion of this causal relation: It seems as if aperson tried to say something, but no voice leaves her mouth— then, a little later, we read in the following intertitle whathas been said. The sensorimotor cycle is inhibited by thetemporal interval that separates visual from the narrative,and opens a space for the psychic and for the ambiguitybetween inside and outside.1

Closing this gap, though, does not dispel the doubt aboutwho is the true originator of an utterance: Was somethingsaid because the author wanted us to hear it? Was it saidby a character in order to cause an action of another char-acter? Is it an intrigue altogether?2 One can never be sure.At least the truthful source of the voice itself will be even-tually reaffirmed, in a “cartesian strategy”, by the consis-tency of physical causality in filmic space: The consistencyof “natural sound” provides the necessary background forthe voice’s corporality. To this end, the sound is composedso that coincides with the picture in an expectable manner,forming a causal texture for the events. Thus, the physicalimpression of sound is the seal on the authenticity of theirmaterial source: the spoon rattling next to the cup, a quietharrumph, or the famous footsteps in the gravel.

In fiction film, the origins of actions usually is either at-tributed diegetically to the film’s inner logic and its char-acters, or simply to the motivation of the authors. Con-versely, a documentary film is expected to be faithful toactions that either happened while the picture was filmed,or to those that happened possibly long before, where doc-umentary takes on the role of a witness to the past. Thisfidelity can be passive, in the sense that the ‘profilmic’ ac-tions are untouched by production and simply reflected, itmay also be active in the sense that the film investigates

1In Triebe und Triebschicksale [7, p. 212] (1915), Freuddescribes the first differentiation between inner and outerworld as the result of the experience that some stimuli canbe avoided by muscular activity (e.g. escape), while others,necessities caused by drives, withstand such attempts.For Bergson, the indetermination between action and re-action is a key function that creates two systems: one inwhich the images change dependent on the movements of asingle image (the body) and one for which the images varyby themselves (“Matiere et Memoire”[1, p. 9]). This investi-gation of indetermination is taken up by Deleuze and servesas a basis for the distinction between image types [6].2The films by Ernst Lubitsch, such as ‘To Be or Not ToBe’, are a good example for the extensive use of intrigue indialogue that was common in the early talkie era.

Figure 1: A scene from All That We Have, situatedon a location of local historical relevance

in order to clarify the delimitation between fact and fiction.This expectation is rooted in the promise, that the agencyof protagonists, be they human or non-human, inscribes itstrace as directly as possible into the film’s diegetic space.This explains the critical role of ‘natural sound’ in docu-mentary film: It is witness to the ontic status of its materialcause and stabilizes the realist perspective of the documen-tary dispositive.

2. NATURAL AND ARTIFICIALInterestingly, in the early sound films, the film music is veryoften diegetic film music3 — some of the films circle magi-cally around dancehalls, attractive musicians, or find otherexcuses to place a musical sound source somewhere. Keep-ing in mind the conventional role of natural sound in film,this is no surprise. Contrary to silent film, by placing musi-cal instruments on screen instead of letting them play in the“cinematic orchestra pit”, they are naturalized as materialsound sources, thus closing the sensimotor cycle and, to adegree, freeing film from its psychic ambiguities.

Just like fiction film, many conventional documentary filmsuse nondiegetic film music as an emotional background, oras a compositional method, e.g. to smooth out picture se-quences. This sound texture also provides an honorable ac-companiment for the ‘voice of god’, who tells the audiencewhat the picture means. At first, this seems to contradictthe fidelity of a documentation — but it is clear, that insteadof supporting the topos of direct transmission, it enhancesthe audience’s faith in the sincerity of the witness, repre-

3The definition of diegesis is debated: This term was coined(by Etienne Souriau) in opposition to the term ‘narrative’,to refer to all the film’s denotations, to the totality of thefictional world in which the narrative is only a part. Thus,nondiegetic sound conventionally denotes sound that is notcaused and cannot be perceived by anyone who is part of thefilm’s denotative space. Background music and off-screennarration are therefore nondiegetic elements. But not unlikethe distinction between form/content or medium/message,it is an external decisionabout where the delimitation is tobe drawn. For a controversial discussion of these issues seee.g. [2]

sented by the narrator’s voice. As long the level of explana-tion, which can be ‘artificial’ (i.e. fabricated by the film),is distinguishable from the level of the object, which needsto be ‘natural’ (i.e. factual), an ‘authentic’ relation betweenthe two can be established. In conventional documentary,this relation is expected to be a clear separation between theobserver and the observed, where the diegetic sound sourcesare what is observed, and the nondiegetic sound forms asedan on which the observer is brought safely to the end ofthe film.

One might argue that a film can hardly be in control overits interpretation and the attribution of authenticity is acomplicated process.4 Since its early stages, there have beennumerous film makers that have (implicitly or explicitly)made the documentary genre itself a subject of their work.Investigating the frontier between the attribution of artificialand natural provides a frame that allows us to reason abouthow sound may work in films that attempt to reflect on howauthenticity is caused or fabricated. This study is likely toget involved in the relation between physical acoustics andwhatever falls outside of this category.

3. THE SOUND OF HOME

“[ . . . ] this feeling of unity and the history and all that. . . that’s what we want to preserve, that’s our heritage, ourroots, that’s where we came from, we didn’t just fall fromthe sky, that’s our real history [ . . . ] without the past we’renothing, that’s my opinion anyway. You can’t just go aroundclaiming that we just suddenly happened, and here we are,now, and it’s all good and everything that was, was bad.That’s . . . history has taught us . . . oh well, that should’ve. . . I don’t want to stray from the point, . . . and the logicalconsequence of all this would be to build a Local History Mu-seum.”5

Quite similarly to the natural and authentic, the past is anembattled place. Both objectivity and identity depend on aconsent on what has happened so far, and any dispute aboutwhat is real and what is allowed seems to unavoidably re-vert to history. Since individual memory as well as the past‘itself’ are inaccessible, external, timeless representations ofpast events serve as proxies and educational means to es-tablish a stable notion of historical fact — on the level ofthe universal (like the state) or the local (like a family).History as a science maintains a distinct skepticism towardthe idea of consistent history and holds that the historicalis characterized by contingency regarding the past events,their mediation and their interpretation today.6

The documentary ‘All That We Have’ [9] investigates a typ-ical representation of local historical identity, a local historymuseum in northern Germany. Being a documentary film

4It can be claimed, e.g. that the differentiation betweenfictionality and nonfictionality is independent of the film it-self, but is the consequence of an “implicit contract” betweenviewers and producers [11].5Helmut Wattenberg, town archivist, transcript from an in-terview 20026Regarding individual memory, the same skepticism hasbeen maintained by psychoanalysis of course.

— a classical medium of historical representation — it si-multaneously makes it a subject of discussion what role filmplays in the production of authenticity.

Following the list of important localities that is published inthe museum’s book “Lebendige Heimat”7 the film is com-posed of a series of panoramic 360◦ panning shots (e.g. Fig-ure 1). The voice off-camera explains the history of theplace, why it is characterized by a series of fires over manycenturies, and that, as there are hardly any historical build-ings left, a place of historical identity is indispensable. Ina fluent transition, the male narrator’s voice blends into afemale voice8, who explains the more recent history of thelocal history museum itself, which has been set fire to reg-ularly since the 1970s. The pictures show places that beartraces of a certain particularity, but obviously fail to presentany authentic ‘past-as-past’. While the moving gaze of thecamera sometimes gets hold of a car or a person, we havea particular, but fairly generic outlook until the cumulationof the final scene.

Due to the peculiarities of ‘natural sound’ in documentaryfilm that embed the observer in an envelope of authenticity,the film sound in ‘All That We Have’ investigates the borderbetween natural and artificial sound. The terms “naturalsound” and “original sound”9 often refer to the same thing.This indicates that the idea of the ‘origin’ is typically tiedto the idea of the ‘natural’, which is another clue suggest-ing a certain inconsistency may be caused by the desire forauthenticity. In order to be able to abstract from referencesto origin, the film restricts itself entirely to sound synthesisalgorithms and does not use location sound recording.10 Ina conversational process from memory, without listening oranalysis of any recordings, the authors11 tried to find thesound for a certain situation. Negotiating a possible recon-struction of a sound memory, each of the acoustic situationsis an acoustic portrait, or in some cases a sound icon of whatcould be called the phantasmatic sound object. Partly, thisresults in a kind of hyperrealism, where each sound is overlyclear and separated, an impression that is magnified by thefact that in most cases one can only hear the sound whosesource is visible.12 On the other hand, the unity of spaceis dubious — on many occasions the physical origin of thesound clearly becomes suspicious (Figure 2). Lacking clearlocality, the attribution of the origin becomes perceivable

7“Living Homeland”, see: [8]8The speakers are: Sarina Tappe, Helmut Wattenberg andVolko Kamensky9German “O-Ton”

10The algorithms have been implemented in the program-ming language SuperCollider ([10]). Apart from the off-screen narration, no recorded sound has been used as mate-rial.

11The soundtrack was programmed during the year 2003 byJulian Rohrhuber, in conversation with Volko Kamensky, atthe Hamburg Art Academy. It was mixed down by Albertode Campo, Rohrhuber and Kamensky at the Academy ofMedia Art, Cologne.

12In this way it could be claimed (in contradiction to Metz)that the impossibility of off-screen sound is only conven-tional: Metz had stated a fundamental difference betweenimage and sound due to the fact that, according to him,there is no off-camera in sound, as it always surrounds theaudience.

Figure 2: An agricultural vehicle is passing a church,a pipe organ is playing inside.

as a process, sometimes even as a deliberate choice of theobserver. Instead of attempting to simulate what is suffi-cient to cause a smooth and realistic impression, ‘All ThatWe Have’ plays with the borderline of failure of what maystill count as authentic. Since eventually, what specifies thesound is a program text, we may infer that the natural itselfis a kind of text that is inscribed by the intervention of aninvestigation of truth.

4. SOUND AND FORMALISMWhat is the origin of an algorithmic sound? Usually, realsound synthesis tries to construct a simplified, but precisemodel of a physical situation13 according to the scientificformalisms that were developed in order to reason aboutnatural causality. Once such a model is implemented, typ-ically, the parameters of the physical model are controlledfrom a graphical user interface, setting cue points along atimeline. Experimentation is done with different parametersof the model, adjusting them to their appropriate values.Both the formalism that was originally used to implementthe application, as well as the algorithmic process itself arenot interesting at that point, because on the level of sim-ulation, the object already exists. While such a procedurecauses similar abstraction effects,14 by itself, it presumesthe existence of the natural entity on the one hand, and for-mally reconstructs the unity of space on the other. In otherwords, the origin of the algorithmic sound is the simulatedobject that is mediated through the timeless mechanism ofa physical law.

Since “All That We Have” attempts to make authenticity asubject of discussion, the assumption of an existing originsimulated by an algorithm would have been dissatisfactory.It seemed more appropriate to make the programming pro-cedure itself part of the conversation, and to discuss changesin the code and the sound synchronously; thus, to establisha negotiation between sound impression, program text, indi-vidual acoustic memory and expectation of the algorithmic

13See e.g. [5, 12]14See e.g. [13]

possibilities. This just in time programming technique15 iscentered around the abstract relation between a textual rep-resentation of a sound algorithm and the acoustic perceptionit causes. To achieve this, parts of the program are rewrittenwhile the whole process is still running, allowing us to hearthe result of changes in the text directly as changes in thesound. The absence of a graphical user interface causes pro-gramming to be a kind of formal experimentation, as wellas a conversational literary genre. Due to the interactionon a textual level, the delimitations of algorithmic causeand effect become part of the discussion and the paradigmof manipulating a simulation of a given entity is avoided.Negotiating the relation between “sound cause” and “soundeffect”, between function and argument, the notion of an ob-ject at the source of the sound is suspended and the causalsource is displaced to the algorithmic process and its textualdescription.

Whilst this algorithm (both as text and as process) is anactive part of what caused the acoustic situation, the for-malism itself is the result of other situations though. Theorigin of the algorithmic sound cannot be ascertained any-more. Depending on the individual case, it might be a halffaded memory of a sound, it might be the particularity ofthe formal system, it might be a coding mistake that rang abell. Of course these considerations apply to the process offilmmaking itself. Watching the film, eventually, the ques-tion is open again.

5. REFERENCES[1] H. Bergson. Materie und Gedachtnis. Ullstein

Materialien, 1982 (1896).

[2] A. Casebier. Film and Phenomenology. CambridgeUniversity Press, New York, 1991.

[3] R. Clair. The art of sound, 1929, http://lavender.fortunecity.com/hawkslane/575/art-of-sound.htm.

[4] N. Collins, A. McLean, J. Rohrhuber, and A. Ward.Live coding in laptop performance. Organized Sound,2004.

[5] P. R. Cook. Real Sound Synthesis for InteractiveApplications. AK Peters, Ltd., 2002.

[6] G. Deleuze. Das Bewegungs-Bild (Kino I), DasZeit-Bild (Kino II). Suhrkamp, Frankfurt am Main,1989 (1983), 1991 (1985).

[7] S. Freud. Triebe und Triebschicksale. in: SiegmundFreud, Gesammelte Werke. Fischer TaschenbuchVerlag, 1999 (1915).

[8] H.-H. Gatjen and K. Buse. Lebendige Heimatbeiderseits der Wumme. Number 31 in RotenburgerSchriften. Heimatbund Rotenburg (Wumme),Rotenburg (Wumme), 1988.

15For more details about the theoretical background of In-teractive Programming / Live Coding in art and science, see[14]. For an overview of different methods and systems, see[4] and [15]

[9] V. Kamensky. Alles was wir haben (all that we have),2004, http://swiki.hfbk-hamburg.de:8888/MusicTechnology/491.

[10] J. McCartney. Supercollider programming language,http://supercollider.sourceforge.net/.

[11] Plantinga. Defining documentary: Fiction, nonfiction,and projected worlds. Persistence of Vision, 5:44–54,1987.

[12] M. Rath, F. Avanzini, N. Bernardini, G. Borin,F. Fontana, L. Ottaviani, and D. Rocchesso. Anintroductory catalog of computer-synthesized contactsounds, in real-time. Proceedings of the XIVColloquium on Musical Informatics (XIV CIM 2003),2003.

[13] D. Rocchesso and F. Fontana. The Sounding Object.Phasar Srl, Firenze, 2003.http://www.soundobject.org/.

[14] J. Rohrhuber, A. de Campo, and R. Wieser.Algorithms today - notes on language design for justin time programming. In Proceedings of InternationalComputer Music Conference. ICMC, 2005.

[15] toplap. Organization home page, 2004,http://toplap.org/.