1

Materialism  and  Interactive  Documentary:  Sketch  Notes  

Adrian  Miles  RMIT  University  

This article approaches interactive documentary from the

point of view of a new media scholar, trying to think about

what new media scholarship might offer documentary, rather

than the point of view of documentary and wondering what 'new'

media offers it. More specifically it wants to use recent work

in materialist media studies and digital media to sketch a

method for how to think about the material specificity of what

interactive documentary is. In this regard this article

continues the recent invitation of Nash, Hight, and

Summerhayes who argued:

While documentary scholarship has frequently

considered the contexts in which documentary is

produced (and, to a much lesser text, consumed), an

ecological framework calls for an extension of this

to foreground the interdependent relationships

between media. (Nash, Hight, and Summerhayes 2014:

3.)

Between, in this essay, is about the programmatic and

computational conditions of an individual software system, and

the way these conditions addresses and proposes a particular

way to consider the relations between the video clips that are

the basis of the interactive documentaries that this system

creates.

2

essay  and  style  The essay, including the essay film has, as Rascaroli

demonstrates, a transgressive and protean form that is most

significantly marked by the two fundamental characteristics of

'reflectivity and subjectivity' (Rascaroli 2008: 25) These two

terms allow for a discursive model and template for this (my)

article as it wants to play around, or at least skirmish with,

the teleological security of the canonical academic paper. I

want to take at his word Latour when he suggests that good

academic writing:

is a narrative or a description or a proposition

where all the actors do something and don't just

sit there. Instead of simply transporting effects

without transforming them, each of the points in

the text may become a bifurcation, an event, or the

origin of a new translation. (Latour 2005: 128)

Such a writing for Latour entails 'a risky account,

meaning that it can easily fail – it does fail most of the

time – since it can put aside neither the complete

artificiality of the enterprise nor its claim to accuracy and

truthfulness' (Latour 2005: 133, italics in original).

Disjunction, exploration, asides, rambles, excursus, and

even digression have their place alongside the clarity of

evidence based argument and claim. This article, borrowing

then the spirit of the essay film, wants to take seriously the

implications for sense making for makers and audiences that

emerging interactive documentaries pose and to do so by

thinking in possibly absurd detail about one interactive

3

documentary system, Korsakow. It also proposes that it is time

for our academic writing to follow the example of the essay

film to be able to, as Gibson argues, find 'our way through

today's complex and changeful world' (Gibson 2010: 10).

ontographic  thick  description  In Alien Phenomenology: Or What It's Like to be a Thing

Ian Bogost proposes ontography as the:

name for a general inscriptive strategy, one that

uncovers the repleteness of units and their inter

objectivity. From the perspective of metaphysics,

ontography involves the revelation of object

relationships without necessarily offering

clarification or description of any kind. (Bogost

2012: 38.)

As a method you make ontographs by 'cataloging things,

but also drawing attention to the couplings of and chasms

between them' (Bogost 2012: 50). (I should point out that for

Bogost a 'unit' is a privileged term that has some of the

characteristics of Latour's 'network' but more particularly

units are 'made up of a set of other units (again human or

nonhuman), irrespective of scale' (Bogost 2012: 19).) This

intention to list and catalogue has particular affinities to

the sorts of poesis that the Korsakow interactive video

software system enables and constructs for interactive

documentary which is what this article will explore.

Ontography wants to make explicit the density of things, their

affinities and distances from each other and us. It is a

method and claim that participates in the recent wave of

4

theoretical work that can be informally collected under the

banner of 'new materialism' or 'object orientated ontology',

and, for our purposes here, 'materialist media studies' (for

example Fuller and Parikka).

Similarly, the ethnographic method of thick description,

which matters because to understand something 'you should at

look at what the practitioners of it do' (Geertz 1977: 5)

rather than its theories or findings, as our world 'is a

multiplicity of complex conceptual structures, many of them

superimposed upon or knotted into one another, which are at

once strange, irregular, and inexplicit' (Geertz 1977: 10)

provides a simple format by which to note, well, things.

In this article I appropriate Geertz's thick description

(his characterisation of thick description in ethnography is

remarkably similar to Bogost's account of ontography in

relation to things and unit relations) and combine it with

ontography into what can be described as ontographic thick

description.

I do this to test the premise that the specific qualities

of a program's computational architecture, in combination with

its media, network, authors, users, screens, operating

systems, servers, and protocols, forms a unit operation (or

assemblage, or Latourian actor network – while there are

important differences between each it is the general sense of

a complex, operative techno–human ecology that matters) that

is fundamental to the practices of making and viewing the

particular interactive documentaries that Korsakow films are.

Code, software, and hardware have agency, certainly as much as

film speed, frame rate, focal length, aperture, light, and

5

format have in traditional film making, and these, in concert

with media, maker, and audience, constitutes the milieu of

Korsakow. Korsakow, through the specific conditions of its

code, invites an iterative form of making and viewing that is

a direct result of this computational architecture, yet recent

theoretical work in interactive documentary has not engaged

with the materiality of these tools and systems in interactive

documentary, relying instead on an approach to media and film

analysis that is overly reliant on a disavowal of the

materiality of our machines and their entanglements. Instead

theory looks to the semiotic relativism of audiences, texts,

and institutions, which can be characterised as 'the cultural

studies Anglo–American style of focusing on content, users and

representations' (Parikka 2011: 54). Ontographic thick

description is my first step in undertaking an 'intensive look

inside the machines' (Parikka 2012 3), at what Parikka has

characterised as the 'weird materialities' (Parikka 2012: 2)

of technological culture.

three  traditions  The most common way of understanding what K–films are has

been to rely on one of the traditional tripartite 'prisms'

used in media and cinema studies. The three approaches Parikka

describes can be characterised as relying on either textual,

institutional or audience specific approaches. In the textual

mode we might consider Korsakow in relation to the sorts of

films it creates, and undertake a critical reading of the

works themselves. Such a method may be hermeneutic, and would

be inflected by a particular theoretical framework, and its

6

aim is to elucidate and unveil what the work means through the

application of this framework.

An equally viable approach, and one often grounded in

political and sociological approaches, would be to look at

Korsakow (either as software or the works it creates) by

emphasising the institutional nature of its media practices

and consumption. Where does Korsakow sit in relation to other

platforms and systems? How are the films it creates similar to

or different to others? Why? What sort of labour does it

entail? To what extent might it participate in other economies

or models of documentary production, distribution, and

reception?

Finally, analysis can be approached in relation to what

audiences do with Korsakow. How are the film's understood? How

do they define, construct, enable, or disrupt what a viewer is

and does? How does an audience create meaning within such

works? What is common, and different, to other documentaries

and, for that matter, other interactive, online works?

In this article I am not directly using any of these

frameworks. One reason is that while we still want to affix

the appellation 'new' to media these traditional

methodological points of view run the risk of looking past or

through what is peculiar and weird about systems like Korsakow

because they tend to begin from questions of 'what does it

mean?', and 'how?', whereas I want to begin from the facticity

of describing what it is as a generative system and unit

operation consisting of code, an authoring application,

software, interface, and network – the very stuff that lets a

Korsakow film be a Korsakow film rather than any other

7

particular interactive documentary platform or work. Where

meaning is put first objects or things become problems of

representation and we 'miss the point that the question of the

object is not an epistemological question, not a question of

how we know the object, but a question of what objects are'

(Bryant 2011: 18, italics in original). As a result 'object

here is not an object, not an autonomous substance that exists

in its own right, but rather a representation' (Bryant 2011:

22). An implication of Bryant's position is that understanding

what things do is a different proposition to what they have

been made to do, and understanding what things can do might

lead to different understandings of individual films and

interactive documentaries more broadly. I believe this is a

necessary precursor to semiotic claims. I also believe that

Korsakow, as an object, has an autonomous agency as a

generative, combinatory engine – an aleatory machine – and

this autonomy is integral to the poetics of Korsakow.

things  that  do  In lieu of meaning I am left wondering about what sort of

thing Korsakow is and what it can do. I am wanting to address

the specificity of the parts that constitute Korsakow because

individual platforms and systems are idiosyncratic in their

material specificities and this matters to how they come to be

used and the interactive documentaries they create.

My point is not to critique work undertaken through other

theoretical frameworks, but it is to argue that these methods

consistently generalise digital and networked materiality,

sort of missing the individual differences of platforms and

systems by moving immediately to concerns of semiotic

8

connotation. Code, bandwidth, codec, network, browser,

protocol, gamma, luminescence, video, interface, screen, and

electricity, this is the stuff that interactive documentary is

literally premised upon but is occluded by our epistemological

fascination with meaning. (While this might be overstating

things it is the weird specificity of Korsakow as a particular

software system, and what this requires of makers and users in

the works it wants to create and makes possible, that becomes

lost when meaning is our only currency.) An approach that is

materialist does not shirk the question of critique and

analysis, and is premised on observing what Korsakow does in

itself as a software system and using that as a basis to

understand what it is.

Hence, I intend to catalogue what things Korsakow does

and what these allow (its affordances, to borrow a term from

Donald Norman's design vocabulary), because having an

understanding of this I hope will let us recognise what K–

films are capable of and therefore what individual K–films are

actually doing, and then we can understand what they mean.

This is a call for the sort of media specific analysis that

Kate Hayles advocates, and while her claim is in the context

of electronic literature, when she writes that 'MSA [media

specific analysis] aims to electrify the neocortex of literary

criticism into recognising that strands traditionally

emphasizing materiality...are not exceptions but paradigmatic

of the ways in which literary effects emerge from and are

entwined with the materiality of texts' (Hayles, 2004: 72), it

seems a short step to what interactive documentary scholarship

needs to also do. What Korsakow 'can do' is subject to the

9

constraints that software and hardware requires, and the set

of actions able to be performed upon media objects by Korsakow

is what constitutes Korsakow as a particular unit. The films

that emerge from Korsakow are specific to this software

system, and it is only then that they participate within

textual, institutional, and audience actions which are

codependent on what the operationalised software and hardware

does.

software  Korsakow is a software application written in Java that

you install on a personal computer. It is a program that is

intended to make interactive films (what are usually known as

'K–films') that can be viewed within any contemporary web

browser. These films can be online, or viewed locally via your

hard drive or from portable media, for instance a DVD or USB

thumbnail drive. It is software that is simple to use, yet

allows for reasonably complex, procedural and generative works

to be made by non programmers. It is an authoring application

where media files – usually video, graphics, and audio – are

prepared outside of Korsakow and then bought into a project as

assets. The software allows the export of the project into a

web ready HTML package with a Flash runtime engine. When

viewed via a web browser the Flash engine loads the K–film and

'calls' the media as required. Korsakow allows, and is

premised upon, the creation of multiple, simultaneous links,

that is relations, between the individual video clips that

make up a K–film, and it is the simultaneous multiplicity of

these relations that is Korsakow's most significant quality in

the context of interactive documentary more broadly.

10

A Korsakow project can be made up of any number of such

video clips, depending on the scale and complexity of the

work, and K–films containing hundreds of clips are common.

These clips, as well as additional sound files, images, and

custom interface designs, become a library of assets to be

used within a project, and relations between all of these

assets is authored with the Korsakow software. Interactivity,

in Korsakow, is a consequence, not the cause, of the density

of relations that it affords as in a Korsakow film a mechanism

is required to find your way amongst these relations. This is

different to many multilinear and interactive systems where

links are often manually made between the parts and so

interactivity becomes synonymous with multilinear navigation.

The density of interconnection that emerges in Korsakow, quite

apart from decisions of navigation, poses particular questions

about how to make sense of Korsakow as software and a media

form.

My elevator pitch would be that Korsakow is software for

authoring generative, associative, and processual films. These

films are complex, possibly autopoetic systems that rely on

patterns of relation to emerge for author and users.

functions  As a programmatic environment Korsakow provides

particular functions that are able to be used in the creation

of a K–film. These functions are the affordances and

attributes that facilitate the development of dense relations

that Korsakow enables, and are what makes Korsakow Korsakow.

They are the program's particular milieu of authoring and

building interactive films (and all programs propose

11

particular milieu's of making). These functions consist

primarily of an editor to build presentation interfaces for K–

films, a clip editor (what the software describes as the SNU –

'smallest narrative unit' – editor) that allows particular

operations to be performed upon individual video clips, and a

basic set of tools to add, rename, and delete media in a

Korsakow project.

A published Korsakow film currently consists of a HTML

and Flash based interface that usually has a single main video

window which, when loaded in a web browser, auto plays.

Generally there are other images that appear within the

interface that operate as thumbnail buttons, where, if

selected by the user, will load the video clip that they are

attached to. This new video clip automatically begins to play.

There are no constraints on the number of interfaces that a K–

film might use. These interfaces of video and thumbnails is

largely what a Korsakow film looks like – though hardly

describes what and how it does.

interfaces  The Korsakow interface editor allows for particular

interfaces to be attached to specific video clips. Interfaces

are limited to presenting thumbnails (the number of location

of which is defined by the film's maker), a video window, and

optionally a background graphic. You can include widgets that

operate as buttons to enable volume control or full screen

viewing, embed time code to indicate the duration of the

current clip, a playhead that shows playback progress within

the current video clip, and the visibility, placement and

typography of text associated with thumbnails and video clips.

12

sound  A Korsakow film can have a continuous audio track that

plays independently of individual video clips that a user

selects. This audio track is loaded and runs while a Korsakow

film is viewed and can by of any duration. It can also be set

to loop. Video clips within a K-film can also have their own

synchronous audio track so a K–film could include and combine

the 'external', independent and continuous soundtrack and the

audio of each individual clip. In addition, while rarely used,

a sound file can also be added to a project's media library to

be used as a default selection sound for a film. This sound

file would be played (and so heard) whenever a thumbnail was

selected while viewing the film.

media  editor  The attributes that are made prorgrammaically visible via

the SNU editor become the functions that can be attached to

individual video clips within Korsakow. These attributes are

the qualities that matter programmatically for video in

Korsakow and extend video's capacity or ability to do.

Programmatically, video becomes a computational object in

Korsakow that, as an object, can then be addressed via these

programmatically defined attributes. These attributes have

more agency than being functions, if only because 'function'

too easily risks misunderstanding as an instrumentalised or

quantified attribute that awaits authorial enaction. When

considered as qualities these attributes become possible

events upon each video object's horizon and through this

13

Korsakow quietly expands what video can do, and in turn what

it then means.

How the software addresses these assets, the facets and

attributes of the video clips that are revealed to

programmatic attention, defines the aleatory logic of

Korsakow. What the software can do with each of these small

pieces of video within these programmatic constraints is what

allows for different varieties or styles of connection and

relation to be spatially and temporally sculpted when making

and viewing a K–film.

These qualities, which are instrumentalised as functions,

are keywords, the number of links (search results) that can be

returned, a nominated preview image for that video, preview

text, whether the clip is to be a 'start SNU', whether the

clip is to be an 'end SNU', text that could be displayed when

that clip plays, the number of times it is possible for this

clip to appear within the work ('lives'), which interface the

clip should use when displayed, whether any background

soundtrack should be muted during this clip's playback,

whether the clip should loop, and, through a 'SNU rating' how

more or less likely this clip should appear as a result of

keyword searches from other clips.

Video clips therefore form the centre of Korsakow, though

not in the obvious sense of just being content nodes but from

the point of view of the agency those nodes now gain through

the programmatic and processual operations that can occur upon

and around them. This is the palette that constitutes

Korsakow. All of the attributes listed can be varied, so

decisions about each attribute, for every clip, become

14

significant for what a K–film will do. This makes a K–film

much more complicated than the simple linking of video clips

together via thumbnail based menus, particularly since such

literal linking in itself has little possibility for the

emergent structures, iterative design, and complexity that we

see in K–films.

words  are  key  Korsakow relies on keywords to create loose connections

between video clips in a film. Keywords are text strings that

are attached to all video clips using the program's SNU

editor, and it is through keywords that a relational structure

is generated. The system's runtime engine utilises this

keyword structure to perform searches to find matching content

as a K–film is viewed. It is the results of these searches

that appear as thumbnail choices in a K–film's interface.

From the point of view of an individual clip within a K–

film the relationality of keywords has two facets. There is

that set of videos that it can find, and there is that set of

videos that are able to find it. Within Korsakow these two

facets are separate, addressable attributes for each video

clip and are known as 'out' and 'in' keywords. In practice,

when viewing a K–film the current video clip's 'out' keywords

are used by the runtime engine to find matching text strings

that have been applied to the 'in' keyword of clips – in other

words out keywords are matched to in keywords. Where the

engine finds a match the results are displayed as clickable

thumbnail images within an interface, and if one of these

thumbnails is then selected by a viewer then, in turn, the

video associated with that thumbnail is loaded, played, and

15

that video's 'out' words are used as search terms to match

against the 'in' words for other clips in the project library.

search  as  algorithm    Though in and out keywords are text strings they are

programmatically distinct and there is, pragmatically, no

reason why out and in keywords cannot be identical. For Hayles

'print is flat, code is deep' (Hayles 2004: 75) and it is

through keywords in Korsakow that video becomes similarly

'deep'. It is this deep search that enables associative

connections to emerge in a K–film. The identification of these

associations as thematic, visual, formal, or poetic is

fundamental for sense making in Korsakow. This need for sense

to arise from these patterns applies to makers and users.

not  a  database  The catalogue of media files that is built when authoring

in Korsakow can be understood as a database for the storage,

indexing, searching, and retrieval of these assets during

authoring and runtime presentation. The attribution of

informal metadata to the video in this database, through the

use of in and out keywords, is stored for later use. When a

clip is selected and the engine performs the text search for

keyword matches, displaying the results as thumbnails, it is,

in this respect, no different to any other search based query

system that uses similar informal keywords, such as Flickr's

tag folksonomy.

However, this is where comparisons to search and database

media ends. In database systems search is for information

discovery or disclosure. For example, I might want to find

16

photographs tagged with 'Melbourne' in Flickr, or books in the

library written by Lev Manovich. In these contexts search,

regardless of the formal or informal queries made, or the

metadata scheme available, is instrumental, pragmatic, and

largely teleological. Sense making here is banal and literal.

We expect of these databases a computational and procedural

domesticity where they ought to tell us how many items there

are that match our search, where they are, and any ambiguity

of terms or results, incompleteness of listing, disclosure, or

discovery, is, in general, anathema to the principles of

access and use. These systems are the regime of computer

engineers, usability gurus, efficiency experts and other

branches of our techno–rationalist professional classes.

hacks  In Korsakow search is not for information discovery or

transparent access. Korsakow's keywords are not visible to its

users and they are merely a pragmatic means for the system to

programatically generate associative, poetic, and so loose

relations between clips. It's a hack. Keywords are internal

text strings that are simply a lightweight solution to the

problem of fluid connections within a programmatic environment

where code can have little brook with ambiguity and fluidity.

The relations that Korsakow discovers through the use of

keywords between its clips are only partially visible to a

Korsakow author, as while the maker knows the keywords that

have been used this is not the same as having an understanding

of the multiple combinations possible while viewing the work.

17

loose  From the point of view of the software itself links

between in and out keyword synonyms are implicit to the

system. These create loose connections between parts as all

they describe is contiguity, not literal connection. Keywords

are then not so much metadata (though they are explicitly

declarative text strings applied to video clips), as an

authorial idiolect than can be as literal or poetic as one

wishes. Therefore it is not the particular terms used as

keywords that encourages Korsakow's emergent poetics – a

highly poetic K–film is as easily made using literal keywords

as abstract ones – but the program's facilitation of aleatory

patterns of relation through the matches it enables. This is

why being able to see or discover keywords does not then

necessarily add to our understanding of a K–film, for sense

making here does not assume or suggest any necessary

denotative or connotative connection between keywords, clips,

patterns, and meaning. A simple linguistic enumeration of

keywords is not the hermeneutic key to a K–film. Making

explicit the keyword terms that generate the deep architecture

of a K–film then risks confusing the pragmatic role of a text

string in a program with the content, intent, and form of the

patterns that result, a bit like thinking the plastic wheels

of a Spiragraph set are the same thing as the procedural

drawings they allow. Certainly one is needed for the other,

but to reduce them to the same is to misjudge the qualitative

difference that exists between the rules of a procedural

system and what the performance of those rules creates.

18

yet  dense  and  fuzzy  Keywords allow loose and fuzzy possible connections

between clips. They are loose and fuzzy because, through the

presence of shared keywords, a K–film may have many

connections between an individual clip's out keywords and the

matching in keywords from other clips. This is a one to many

relationship, where rather than the single anchor and link

model required of the HTTP (web) protocol, clips in Korsakow

can have multiple, simultaneous destinations. This creates a

looseness between clips that is resolved by the relationships

that emerge between keywords, clips and interfaces. Where a

large number of keyword matches may exist what appears in a K–

film while viewing is a loosely choreographed dance between

the number of thumbnails available in the interface, keyword

matches, SNU rating, the 'max links' attribute, and available

lives.

For example, imagine as a thought experiment a K–film

that contains twelve video clips, and an interface that has a

video window and three thumbnail panes. Each video has an out

keyword that matches the in keyword for all twelve clips. The

film is launched. The simplest procedure that now happens is

that the software performs the keyword search when the first

clip loads and plays, it finds all matches (in this case all

twelve clips), and then examines any constraints in relation

to clip lives, removing those that are no longer eligible.

Next the system algorithmically applies any SNU rating to each

of the remaining candidate clips that are the product of the

search, varying their relative ranking as necessary. Finally,

of the set of clips remaining as candidates, a random

19

selection of three to populate the three available thumbnail

fields is made. These three, and only these three of the

possible twelve, are what the user will see, and this is the

only 'action' that the user is witness to.

In this instant of programmatic and procedural operating

we can see that Korsakow's default behaviour largely excludes

strict linear sequencing and instead replies upon

probabilities of connection between what clips might become

connected with other clips, subject to the programmatic

constraints and affordances described.

And there's the rub, nub, and hub of it. Korsakow is not

an engine for building informational, didactic, instrumental

interactive documentaries, but kludges together, within the

constraints that programmatic media requires, a system to

enable generative, associative patterns to emerge amongst its

parts while the work is being authored and played.

promiscuity  The dance of possible connections available between clips

as a K–film is played, versus those that become present in the

interface, versus what thumbnail is actually selected, is the

programmatic choreography that is Korsakow. Where many clips

lie closely together – in Korsakow because they have an out

keyword–in keyword symmetry – they form contiguous clouds as a

loose federation of similarly connected things. The relations

that arise between these clips, that is the possible

combinations of sequences that may or may not come to be

formed within this cloud, are a potentially enormous set which

in turn expresses a milieu, mood, or even constellation of

views upon an idea, topic, or event. The pleasure and

20

sophistication of making and viewing here arises from the

density of these multiple threads that emerge amongst these

clips. This is more or less Bogost's description (via Bryant)

of flat ontology and unit relations where 'something is always

something else, too: a gear in another mechanism, a relation

in another assembly, a part in another whole' (Bogost 2012:

26). As we make a K–film individual clips remain near to hand,

readily available through their promiscuous connectivity,

reverberating between what they are individually and what they

always become when inserted into, and becoming part of, new

wholes that each viewing necessarily creates. A K–film is

about contour and constellation, rather than arc and

resolution, (passage and peak).

aleatory  non–narrative  A consequence of the multiple possibilities that

circulate around each video clip in a K–film is that trying to

visualise or map them outside of any K–film's individuated

viewing is to tilt at windmills. Such a desire misreads the

generative autonomy of a system that operates through

associative, aggregative and aleatory ways as a narratological

map. These multiple possibilities in themselves have their own

varying densities.

variable  density  Dense nodes are clips with many connections to other

nodes. This density happens in three distinct ways in

Korsakow. The first is through the use of keywords, where a

dense node is one that would share one or more out keywords

with the in keywords of many other clips. This can be thought

21

of as a promiscuous form of connection, and is, topologically,

the simplest to understand as it is a node that, by virtue of

its liberal listing of keywords, has many links out.

Connection can also become dense in a Korsakow film

through the provision of lives, so an individual film may

utilise clips that have limited lives (meaning they will

disappear from the film as their number of lives, which is the

number of times they have been viewed, is reached), and a clip

that then does not die, or has a much higher number of lives,

will statistically appear more often. This density is not

necessarily achieved through their number of links but by

sharing keywords to allow that clip to traverse from one

thematic cloud to another.

Finally, a similar outcome is achieved through exploiting

the SNU rating of individual nodes, where increasing or

decreasing this rating makes the clip more or less likely to

appear in the system's search results. Where a higher SNU

rating is combined with more lives it is possible to create a

clip that is particularly dense within a work, and so becomes

a pivot, or fulcrum, to provide passage between otherwise

intertwined and inwardly orientated clouds and series of

clips.

This varying density creates a topology that through

keyword relations, SNU rating, and lives has the

characteristics of a small world network. A small world

network consists of nodes and connections between these nodes

and this node and link structure can be used to map

friendships between people, the power grid, or even protein

pathways, and in a K–film it is the clips and their relations.

22

Within small world networks there are always 'nodes with an

anomalously large number of links' (Barabási 2002: 56) which

are described as connectors. Clips that have many links

(literally in the case of those with saturated keywords,

temporally in the case of those that rely on lives and/or SNU

rating) are such connectors and these 'dominate the structure

of all networks in which they are present...with links to an

unusually large number of nodes, hubs create short paths

between any two nodes in the system' (Barabási 2002: 64).

Dense clips in a K–film therefore allow users to traverse from

one loosely interconnected cloud in a Korsakow film to

another. The role of these different sorts of dense nodes in a

K–film is similar – there is a constellation of related

material that is loosely interconnected, and within these

clouds there is a clip, or clips, that allows connection

outward, generally to other similarly loosely interconnected

clouds and constellations. However, that these dense nodes can

occur spatially through shared keywords, or temporally via SNU

rating and lives, is a significant difference and does create

different topologies and different types of K–film structures.

For instance, Matt Soar's Korsakow diary film Ceci N'est

Pas Embres (Soar 2012) documents his family's three month

sojourn in a French village (for a detailed discussion see

Soar 2014). It consists of three major clouds, corresponding

to the three seasons of their stay. Within each of these there

is a simple to and fro rhythm between clips where an upbeat or

positive sequence is followed by a downbeat or negative

sequence. Such a pattern is easily achieved in Korsakow by

using, for example, an in keyword on the positive clips such

23

as 'springHappy', and an out keyword of 'springUnhappy', and

reversing this on the 'negative' clips so that they have an in

keyword of 'springUnhappy' and and out keyword of

'springHappy'. This in no way determines what specific clips

will be returned as the system performs its searches but it

does mean that when there is a positive clip playing that

matched clips will all be negative and vice versa. This

establishes two series within each season's cloud of clips,

allowing the user some agency to explore the work, and the

system its own agency to procedurally generate its varying

search results around this affective rhythm. In addition,

within the first two seasons there is a connector that

provides a link from one season to the next, what Soar

describes as a 'pinch point' (Soar 2014: 165). The work always

begins with winter, their season of arrival, and in each

viewing, if you stay long enough and make sufficient choices,

the film traverses from winter through spring to end with

summer and the end of their stay. There is then a large scale

directional arc to the work, moving through the seasons in

calendrical sequence, and within each season specific

syntagmatic series are formed by the regular repetition of

good and bad vignettes.

What comes to matter for the film is the continuing

multiplicity of variations between the available clips. As you

view the work new sequences form between the clips that the

film contains, and so what comes to be the film is not any

particular series you experience as you watch it the first, or

even the second or third time, but understanding that the work

is the set of these possible connections between its parts,

24

and that any viewing is always a provisional and

idiosyncratically specific finding of one path amongst others.

reading  performs  the  work  In Korsakow every viewing is an individual performing of

the work. Here, to return to an individual K–film offers not

so much the reward of an increasing hermeneutic density

(though this is certainly available) but the ongoing unveiling

and discovery of a deep structure which is realised as nuanced

pattern, of an immanent autopoetic complexity. As a K–film is

returned to, viewed again and again, different senses, moods

and relations between parts emerge because different clips now

come to lie together, offering new facets by which to view and

understand the work as a whole. Understanding the role of this

readerly poesis for sense making in Korsakow – in concert with

what its author has enabled through their more or less

deliberate use of the affordances available – lets us see that

sense making has dramatically shifted not because it is

digital, or even by the activity of users choosing which links

to follow, but specifically because emergent and generative

qualities are realised through the programmatic affordances

that Korsakow allows. This dance between the programmatic,

video, sound, author, user, and the processual logic of the

computational, is Korsakow's dispositif. To make sense with

Korsakow as an author and user we can no longer rely on the

common and naturalised practice of watching a film from a

beginning to an end, nor even the more recent equivalent of

following all its links to each of their destinations. Our

historical hermeneutics of reading that has been grounded on

the linear and temporal fixity of media as a consequence of

25

its technical substrate is dissolved. What you and I watch are

no longer the same things. This, in relation to sense making,

is radical, and shifts Korsakow in dramatically different

directions to many other interactive documentaries because

many of these other works, for all their linked complexity and

visual splendour (what might be called, with a nod to Gunning

more than Eisenstein, their 'interactivity of attractions')

are resolutely and indifferently fixed in predetermined

relations. In these other interactive documentaries, even

where their shape and scale may change with additions from

makers and audience, links are to single constant

destinations. Even where there are databases that utilise

search, the same results are returned from my first, second,

and third queries. In these interactive documentaries what

changes is what I choose to select next, but what I might

choose from rarely alters. Any deeper architecture of the work

is rendered as surface through menus and single links to

single destinations, interaction design reduced to the

pragmatics of information architecture. Such works do not

change architectonically as a consequence of their viewer's

actions. Exceptions to this instrumentalised linking – an

informatics of connection rather than an aesthetics of

relation – abound in online media, yet, interestingly, in the

milieu of interactive documentary the majority of works remain

resolutely fixed by the imprimatur of an auteurism that

remains largely untroubled (and unproblematically celebrated).

poesis  Structure in a Korsakow film emerges as a consequence of

its making, and its making involves the crafting of relations

26

between clips through keywords in concert with the system's

programmatic affordances. It is not a system for designing a

structure of specific or predetermined inter–connections in

advance using a narrative or schematic map that is then

translated into Korsakow. A K-film is built from within,

utilising its keywords, lives, interfaces and SNU ratings to

iteratively develop a patterned structure of variable dense

relations in the time of its authoring. Building a K–film

relies upon an iterative, sketch like practice of incremental

changes and additions, exporting and evaluating what has

happened. This is a very particular sort of sense making

through praxis that, anecdotally, is closer to design or even

performance rehearsal rather than the more common industrial

modes of production that film–makers are acculturated to and

expect.

a  post  industrial  aside  Industrial modes of media production, historically due to

the capital costs involved, need to 'map' requirements before

something is made, and so making is to a significant extent

the creating and assembling of what has already been defined.

For example, a film script leads to a storyboard which

provides a shot list, which works as a blueprint for location

and studio shooting, and finally editing. Similarly, a great

deal of new media work follows similar procedures where wire

frames, mock ups, prototyping, user testing, and information

architectures are defined, and then translated at different

resolutions or scales into testable services and systems.

However, to work in this way in Korsakow is to domesticate its

deep invitation to poesis and misread its procedural

27

complexity for instrumental intention. Korsakow's invitation

and method on the other hand is more analogous to

programming's 'agile development' than what traditional film

making practice has (or can) afford. Korsakow's technological

and aesthetic footprint is an alternative site for interactive

documentary practice as authoring is a series of intuitive and

more or less directed improvisations which have the tenor of

'what if?', or 'what happens when?' and 'I think I'll try this

instead'.

Reading a K–film requires similar procedures of small

scale repetition, wondering and returning to the already

viewed because deeply multilinear structures require a

hermeneutics of repetition. As a result of this a K-film needs

to be returned to, that is viewed and read more than once, for

its patterns to emerge, for it is when a clip reappears,

recognised as already having been viewed, and now, through the

thumbnails available, offering new and different possible

connections, that patterns of relation become manifest. (If

clips only appeared once, to forever disappear, then a work

might as well be a strictly connected branching tree, and so

repetition, of clips and thumbnails, a looping returning to

the already visited to find new possibilities and relations

this is what lets makers and users realise that their choices

alter the form of the work in itself.) It is not possible to

view a K–film in all its possible variations, yet some sense

of this variability is necessary to understand any particular

work. Generative, interconnected works, architectonically, are

premised on the return to the already visited as their key

trope to indicate audience agency – you return to a clip, and

28

it now has different next possibilities (and therefore

different possible sequences). This also helps us recognise

the Heraclitean principle contained within Korsakow films

where, indeed, you do not 'return' to the same clip as its

repetition means it is and cannot be identical to its first

appearance. The structure, therefore, of this return is

musical more than narrativist, it is verse and chorus rather

than the literary's normative embrace of time's arrow, and

what clips come to mean change because of, not in spite of,

their reoccurrence.

documentary  knowledge  That material specificities are significant actants in

what they do and require of makers and users in interactive

documentary needs to be foregrounded in our scholarship. In

documentary studies more broadly as there is common

understanding of the mechanics of making a film, including the

ability to recognise and engage with the role that

technologies play within this. For instance, it is a truism of

documentary cinema studies that the rise of smaller, portable

and cheaper 16mm cameras, combined with the Nagra and faster

film stock, allowed for radical changes to documentary form

and practice. Here the relation of technology to form and

meaning is almost taken for granted. Critical writing in

interactive documentary, because of the variety of platforms

and systems available, requires a similar subtlety as being

digital, using code and networked does not erase the

significant differences between platforms and systems, and

these differences exceed any consideration of their maker's

intent or interpretive taxonomic schemes.

29

a  milieu  The specificity of Korsakow as a particular software

system matters because as software it proposes a milieu of

making that is grounded by the programmatic terms of the

software much more than in the traditions or histories of

documentary and film making. To not know these terms, what is

in effect the program's world view in relation to interactive

documentary, is to struggle to critically engage with the work

that it produces. Sense, in this context, needs to be

recognised as a consequence not only of the consumption of

distributed works, but also of the programmatic and networked

dispositif that these works define.

It can be difficult to explain why the affordances and

constraints of software matter to those who are not engaged in

digital making, as from 'outside' it can too easily appear

that the digital is infinitely malleable to our will.

Insisting on its agency then seems to be a variety of

technological determinism, however, a Korsakow film is a

Latourean actor network where the members of this network are

not only browser, video, narrative, text, maker, and audience,

but includes code, program, programming, language,

instruction, protocols, as well as screen, interface, mouse,

keyboard, bandwidth, resolution, server, gamma, compression,

and electricity. These properties have a materiality that is

evident as much by what they constrain as by what they enable.

They are real actants in relation to making, the maker, and

audience and the programmatic decisions that Korsakow enacts

around video become the expressible and performable of a K–

film.

30

splendour  Let's face it, there is a splendour in the poesis that

Korsakow performs. This splendour is found in the risk to

order, coherence and sense that generative procedural systems

invite, and this is manifested in Korsakow through emergent,

variable dense relations. Korsakow foregrounds a computational

rather than narrative logic, which is perhaps why it has been

so misunderstood critically (Soar 2014), and why to work

successfully with Korsakow requires some surrendering of your

agency to the procedural demands of the unit or system.

Korsakow's desire to engender multiple, loose, possible

connections is a computational response that makes material

the force that is already immanent in cinema to connect and

establish relations between otherwise separate parts, what we

usually call editing. Korsakow becomes, then, an engine and

instrument for allowing any clip to maintain and sustain all

those connections it might wish to have with other clips. A

desire that demands author and audience accept the risk of the

maybe, perhaps, and meanwhile, surrendering my always

imaginary control as author, and my regal authority as reader,

about what might, is, and will happen. To make sense in and of

Korsakow, for making and viewing, it needs to be recognised,

deeply and seriously as a unit, system, and actor network. We

are actants within this system, but never its centre or

origin. For documentary this proposes a reading and making of

the world that is not pre–determined nor fully controllable,

for maker, reader, narrator, or the work.

In a gesture to the beginning, and the over striving for

closure that is the hallmark of good academic writing (and a

31

reflexive moment signalling the penumbra of modernism's

formalism), we might claim, returning to Latour, that a

Korsakow film for documentary offers 'a risky account, meaning

that it can easily fail – it does fail most of the time –

since it can put aside neither the complete artificiality of

the enterprise nor its claim to accuracy and truthfulness'

(Latour 2005: 133, italics in original.) Or, on the other

hand, that the 'unfathomable density, the black hole outside

of which all distinctions collapse into indistinction' (Bogost

2012: 22) provides an account of the nature of the world,

Korsakow, and a passage to investigate further the affinities

between this density and interactive documentary's capacity to

cinematically engage with this world.

Acknowledgements  I would like to acknowledge the patience of the editors

of this issue of Studies in Documentary Film, Craig Hight and

Ramaswami Harindranath. I would also like to thank the

nonfictionLab, RMIT University, for providing support during

the writing of this article.

biography  Dr. Adrian Miles is a Senior Lecturer and currently the

Program Director of the consilience Honours lab at RMIT, in

Melbourne, Australia. He does research on hypertext media and

networked interactive video, and undertakes theoretically

inflected digital projects. Adrian's research interests

include interactive nonfiction, pedagogies for new media, and

digital video poetics - with a Deleuzean cinematic inflection.

32

References  Barabási A-L (2002) Linked: The New Science of Networks.

Cambridge (MA): Perseus.

Bogost I (2012) Alien Phenomenology, or What It's Like to Be

a Thing. Minneapolis: University Press of Minnesota.

Corrigan T (2011) The Essay Film:From Montaigne, After

Marker. New York: Oxford University Press.

Geertz C (1977) Thick Description: Toward an Interpretive

Theory of Culture. In: The Interpretation Of Cultures. New

York: Basic Books, pp. 3–30.

Gibson, Ross (2010) The Known World. Text, 8(October).

Hayles NK (2004) Print Is Flat, Code Is Deep: The Importance

of Media-Specific Analysis. Poetics Today, 25(1), 67–90.

Latour B (2005) Reassembling the social: an introduction to

actor-network-theory. New York: Oxford University Press.

Norman DA (1988) The Design of Everyday Things. London: MIT

Press.

Parikka J (2012) New Materialism as Media Theory:

Medianatures and Dirty Matter. Communication and

Critical/Cultural Studies, 9(1), 95–100.

Rascaroli L (2008) The Essay Film: Problems, Definitions,

Textual Commitments. Framework: The Journal of Cinema and

Media, 49(2), 24–47.

Rombes N (2005) The Rebirth of the Author. Ctheory: Theory,

Technology and Culture, td016,

http://www.ctheory.net/articles.aspx?id=480. Accessed 6

March 2010.

33

Soar Matt (2012), Ceci N'est Pas Embres.

http://www.embres.ca/exhibit/watchthefilm/. Accessed May

27, 2014.

Soar M (2014) Making (with) the Korsakow System: Database

Documentaries as Articulation and Assemblage. In: Nash K,

Hight C, and Summerhayes C (eds), New Documentary

Ecologies Emerging Platforms, Practices and Discourses.

London: Palgrave Macmillan, pp. 154–73.