enacting the point of being_pov

7/21/2019 Enacting the Point of Being_pov

http://slidepdf.com/reader/full/enacting-the-point-of-beingpov 1/273

iii

Table of Contents

List of Figures and Tables......................................................................................v

1 Introduction .................................................................................................1

1.1 SOURCES AND REFERENCES. SOURCE CRITIC REMARKS..............................................................................71.2 CHAPTER OVERVIEW ..................................................................................................................................10

2 Cognitive theory in the study of audiovisual media ......................... 13

2.1 DEFINTIONS OF COGNITION ........................................................................................................................132.2 IN THE BEGINNING WAS HUGO MÜNSTERBERG .........................................................................................202.3 POST PSYCHOANALYSIS: THE COGNITIVE APPROACH IN SHORT ...............................................................22

3 Point of view and subjectivity................................................................ 29

3.1 WHICH POINT OF VIEW? .............................................................................................................................293.2 THE OPTICAL POINT OF VIEW......................................................................................................................333.3 THE METAPHORICAL POINT OF VIEW..........................................................................................................38

3.4 SUMMARY...................................................................................................................................................48

4 Affordances and constraints: Some notes and elaborations onGibson’s ecological approach to visual perception ........................... 53

4.1 THE ECOLOGICAL ENVIRONMENT ACCORDING TO GIBSON.......................................................................534.2 THE AFFORDANCES OF THE ENVIRONMENT ...............................................................................................564.3 TECHNICAL AFFORDANCES AND CONSTRAINTS OF MEDIA ........................................................................584.4 THE MISSING DON’T FIRE BUTTON: JOHNNY’S PROBLEM ENCOUNTERED AGAIN.....................................674.5 SUMMARY...................................................................................................................................................68

5 Experientialist cognitive theory and the embodiment of mind......71

5.1 OUTLINES....................................................................................................................................................71

5.2 BASIC-LEVEL CATEGORIZATION AND BASIC LEVEL PRIMACY ...................................................................745.3 METAPHOR .................................................................................................................................................775.4 IMAGE SCHEMAS ........................................................................................................................................845.5 COGNITIVE / PERCEPTUAL SETS .................................................................................................................1025.6 SUMMARY.................................................................................................................................................109

6 What is in the sound?..............................................................................111

6.1 BLIPS AND BLOPS ARE NOT ONLY BLIPS AND BLOPS ................................................................................1126.2 HEAR-SEEING: ANAPHONES AND BASIC SOUND TERMINOLOGY .............................................................1146.3 SUMMARY.................................................................................................................................................123

7 Interaction and Interactivity: computer games asaudiovisuo(perceptuo)-motor experiences........................................ 125

7.1 THE INTERACTIVITY HYPE........................................................................................................................1257.2 JENS F JENSEN: AN OBJECTIVE APPROACH TO INTERACTIVITY ..............................................................1307.3 BRENDA LAUREL. THEATRE AS AN INTERFACE METAPHOR...................................................................1337.4 JOHN ALEXANDER: AUDIOVISUAL NARRATIVE AND VIEWER INTERACTION ..........................................1377.5 MARK JOHNSON AND GEORGE LAKOFF: A SUBJECTIVE AND EXPERIENTIAL APPROACH TO

INTERACTIVITY AND INTERACTION .........................................................................................................1407.6 SUMMARY.................................................................................................................................................143

8 The Game Ego ..........................................................................................145

8.1 I DREAMT I WAS BRIGITTE BARDOT AND THAT I KISSED ME: THE SUBJECT AND THE SELF .................1458.2 THE SELF AND THE IDENTITY: SUBJECTIVE EXPERIENCE .......................................................................1488.3 CONTROLLABILITY I: FILM INTERFACE VERSUS COMPUTER GAME INTERFACE......................................153

8.4 CONTROLLABILITY II: CONTROLLING THE GAME EGO...........................................................................1608.5 GAME EGO VERSUS AVATAR...................................................................................................................1668.6 SUMMARY.................................................................................................................................................168



iv

9 Analyses and examples...........................................................................171

9.1 ZORK ........................................................................................................................................................ 1719.2 TENNIS FOR TWO AND PONG ................................................................................................................... 1749.3 SPACE INVADERS ..................................................................................................................................... 1799.4 PAC-MAN ................................................................................................................................................. 1849.5 BUBBLE TROUBLE.................................................................................................................................... 1949.6 DEFENDER................................................................................................................................................ 1959.7 DUKE NUKEM 3D..................................................................................................................................... 1979.8 NHL 2000 ................................................................................................................................................2009.9 DAY OF THE TENTACLE ........................................................................................................................... 2059.10 MYST........................................................................................................................................................ 220

10Results and conclusions: Towards a theoretical framework forcomputer games, interaction and film theory: ................................. 247

10.1 WHAT THIS DISSERTATION WAS ALL ABOUT ........................................................................................... 24710.2 THE TACTILE MOTOR / KINESTHETIC LINK................................................................................................ 25110.3 THE GAME EGO........................................................................................................................................ 25110.4 THE POINT OF VIEW ................................................................................................................................. 253

10.5 THE POINT OF BEING ............................................................................................................................... 25311References.................................................................................................255

11.1 LITERATURE MENTIONED ........................................................................................................................ 25511.2 BOOKS AND ARTICLES OF IMPORTANCE NOT MENTIONED ...................................................................... 26111.3 COMPUTER GAMES................................................................................................................................... 26211.4 MOVIES .................................................................................................................................................... 26411.5 OTHER SOURCES (INTERNET ETC.)........................................................................................................... 265



v

List of Figures and TablesFIGURE 1: FILM- AND COMPUTER GAME INTERFACE....................................................................................................155FIGURE 2: RUSSEL AND NORVIG’S AI SCHEMA............................................................................................................156FIGURE 3: SCREEN SHOT OF PONG ................................................................................................................................176

FIGURE 4: PONG CONTROLS ..........................................................................................................................................178FIGURE 5: COMPUTER SPACE CONTROLS......................................................................................................................178FIGURE 6: ODYSSEY GAMING SYSTEM..........................................................................................................................181FIGURE 7: SCREEN SHOT OF SPACE INVADERS.............................................................................................................182FIGURE 8: SCREEN SHOT OF PAC-MAN GAME ENVIRONMENT. ....................................................................................187FIGURE 9:SCREEN FROM SOUNDEDIT 16......................................................................................................................189FIGURE 10: PAC-MAN CABINET. ...................................................................................................................................192FIGURE 11: PAC-MAN CONTROLS.................................................................................................................................193FIGURE 12: SOUNDWAVE AND SPECTRUM OF BRØDERBOUND LOGOTYPE MOVIE.......................................................225FIGURE 13: THE DOCK IN MYST. NOTE THE USE OF LINEAR PERSPECTIVE .................................................................245FIGURE 14: FROM THE FORECHAMBER BESIDE THE DOCK. NOTE THE LOCATION OF THE BUTTON.............................245TABLE 1: GRODAL’S GAME PROTOTYPES EXTENDED WITH TYPE OF CONTROL ...........................................................163TABLE 2: MANIFESTATION OF THE GAME EGO ............................................................................................................168

TABLE 3: VOCAL INTERACTION EVENTS / INTERACTIONAL SPEECH ACTS ...................................................................211TABLE 4: MOTOR INTERACTION EVENTS ......................................................................................................................212



1

1 Introduction

What is going on in the embodied human mind while playing computergames?

In my work on this dissertation, I have found that computer games make

possible the study of salient features of our daily environment. In order to

make a computer game the designers have to have some basic knowledge

about how human beings behave in a natural environment. That is, how we

move around, how we plan our movement, what distracts us from carrying

out these plans and so on and so forth. By designers, I do not necessarily

mean single persons but rather the team making the games i.e. thegraphical artist, the sound designer, the programmer, the scriptwriter, and

others. Much work on computer games is done intuitively. The designers

know that this or that way of making things works. Intuition is nothing

more than knowledge we are unaware of having. We all have a basic

understanding of our real world environment. The earth is experienced as

mainly flat, things will fall to the earth if dropped, if we stack things in a

pile, it will be higher the more we add, we move in the direction of our eyes

etc. The experience of being within the world with a human body makes up

the basis for our conceptual system according to George Lakoff and Mark

Johnson. I argue that the field of computer games will benefit from being

studied not only from a theoretical approach stemming from film- and media

theory at large but one should also take into account the basis for the human

conceptual system. I have found the experientialist cognition of Lakoff and

Johnson to be a fruitful way. In part, it explores and explains the intuitive

way of making computer games. Computer games are also striking examples

of how human perception works. The ecological approach to visual

perception as explained by James J. Gibson is an important part of theframework suggested. Gibson’s theory is partly incorporated within

experientialist cognition. My dissertation will show that Gibson’s ideas of

affordances and constraints apply well to computer games. In fact, very

much of computer game environment construction relies on how the human

perceptual system is able to find the affordances of the objects contained

within the game environment. Some games allow the game player more or

less unlimited time to find out these affordances, e.g. Myst (Brøderbund

1993), Zork (Infocom 1981), while others do not e.g. Pac-Man (Namco 1980)

and Unreal Tournament (Epic Mega Games/GT Interactive 1999).



ENACTING THE POINT OF BEING

2

The objective of this dissertation is to put forth a theory on how the

embodied human mind plays a role in shaping our understanding of

computer games, the worlds, the environments and characters within them

and the experience of computer game playing. Central to this work are

concepts such as point of view, subjectivity, identity, identification,interaction, and interactivity, which I will examine, question, challenge, and

even in some cases replace with what I consider more well suited terms. In

order to set forth the objectives of this dissertation, I will make a number of

claims that I will approach and discuss thoroughly in the following chapters.

I have already made one claim when saying that the human mind is

embodied. That is my foremost claim. Playing a computer game involves the

human body as well as the human mind. It is an interaction process of the

body/mind. We use our human motor system to manipulate a control deviceof some sort while playing. The manipulation of the control device will have

effects on the objects within the game environment. These objects might be

taking on a visual or auditory appearance. That is, they might be categorized

by us perceivers as sounds or images. We use our mind to bring into being an

understanding of the game’s objectives, the game environment, the

characters and objects within the game and plan a strategy concerned with

how to play the game. That is, we make us an initial hypothesis about the

game situation that later we might be forced to alter. There is an interaction

of events going on between the body/mind of the game player and thecomputer game. The human mind and the human body are dependent upon

each other. That which goes on in the mind is to large extent a result of the

constitution of the human body and the society/culture in which this human

body resides. This is the essence of a theory central the present work,

namely what is known as experientialist cognition as explained by George

Lakoff and Mark Johnson in numerous works.

Humans have a conception (and an understanding) of the world in which

they exist. Most of us have some awareness of the regularities of nature. If one holds a stone in one’s hand and lets go, it will most probably fall to the

ground or at least fall down to a lower place than from which it was dropped.

It is one of those phenomena that we, as human beings, have a shared

knowledge of. It is a regular thing to occur in nature. Stones dropped will

fall to the ground. The explanation why it falls is part of a culture. To clarify

what I mean by this consider the following: Aristotle, Newton, and Einstein

would explain why the stones falls to the ground in very different ways.

Aristotle held the belief that a falling body was seeking its natural position.

Newton would say that gravity made the stone fall while Einstein would say



INTRODUCTION

3

that no force made the stone fall. All these three try to explain in different

ways observations of a natural regularity equal to them all. They belong to

different ages and different cultures. Scientific traditions, paradigms, are

part of and even affects culture. Cultures, though, are not stable. They

develop over time and mix with each other. By this I just want to make clearthat we must be aware that some parts of our conceptions and explanations

of how the world works are the result of the cultures in which we live and

some parts are equal to most of us from the fact that we are humans.

Humans constantly construct the world out of sensory data by being within

the world. The world and the environment we occupy have a known way of

functioning. The sun rises at dawn and sets in the evening for instance. This

is something we know. However, this is not correct if we apply a scientific

explanation. The sun only appears to rise due to the rotation of the earth.

However, the experience of this is that sun itself rises. Hence, we have astrong basis in our human experience of this kind of relation. The sun is an

object in the sky, and that object moves according to a specific pattern.

What I am aiming at is to make clear that the relation between the world

and our human body is of importance for this study. We are apt to make us

explanations of why the world is the way it is, and why we perceive it this or

that way. We have a need to know things about the world in order to survive

in it. Knowledge about falling objects and the regularities of the rising and

setting sun for instance are important factors in human life. Furthermore, ineveryday life we perceive our body within world, within the environment;

e.g. we know where our arms are; we know what our feet do etc. mostly

without thinking about it. This is due to proprioception, which is the sense

that keeps track of the parts of the body, for instance, the positions of our

limbs and our head. As we will find, certain computer games allow the game

player to establish a proprioception based on vision, audition, and tactile

motor action. These add up to a tactile motor/kinesthetic link and

kinesthesia, which is a sensory awareness of the positions of the limbs and

body in a game environment. In addition, when so doing we have a strong

performative experience of interaction and of being within a world (or rather

an environment).

Computers are multi-semantic, multi-modal, multi-tasking, and multi-

purpose machines. To claim that computers are multi-semantic and multi-

modal is to say that computers are machines that handle signs and sign

systems in many different modes. It may be through visuals, sound and

sensory motor interaction. It may perform all these modes at the same time

and for several different purposes. A computer can survey and control the




4

production in a factory. It can be a web-server. It can be used to record

music, edit video, serve as an advanced typewriter, and perform extremely

complicated calculations of sorts, control the flow of gasoline in a car, and it

may also be used to play games. This dissertation is primarily about

computer games and how what is known as experientialist theory of cognition can be used as a theoretical framework in the analysis of such

games. Computer games are multi sensory experiences. One can describe

them as consisting of at least three predominant senses in claiming that

they are audiovisuo-motor experiences. In other words, they involve hearing,

vision and concrete motor action. Motor action is part of the proprioceptive

system giving a sensory awareness of the body’s position in the environment,

i.e. kinesthesia.

Not only the experientialist theory of cognition is important for myanalytical framework however. I will as mentioned earlier, use some terms

and ideas from James J. Gibson’s work on visual perception such as

affordances and constraints etc. Gibson’s work shares some factors with

Lakoff’s and Johnson’s experiential approach to cognition, which we will get

back to in chapters to come.

Some further basic questions to guide the reader into the objectives of this

study might be in place. Why are we able to know how to operate a game like

Pac-Man or find our way through the world of Myst? Computer games aremanipulable and interactable systems that somehow invite us as players to

take part in them.1 Computer games are about manipulating the

environment and the objects making up the environment within the game. A

game does most often suggest the presence of a location that we interpret as

a kind of rule governed environment.

Computer games are commonly incorporating images, sound and direct

manipulation. Object manipulation is what differentiates computer games

from film as will be shown in the following chapters.

Let us study a shortened passage from the book Only You Can Save

Mankind by Terry Pratchett, for the purpose of getting into the idea of what

might lie behind how and why we engage in computer game playing. In this

passage we will meet eleven-year-old Johnny, an experienced computer

game player, who plays a new computer game and who suddenly encounters

something, for him at least and probably for most, highly unusual while

playing. My analysis of this quote goes a little deeper than to commentary on

1 The term interactable is a neologism that I introduce in this dissertation. The full meaning of it and itsreason for being will be revealed and discussed in chapters 4 and 7.



INTRODUCTION

5

the quote itself since I also acknowledge the context of the quote within the

book it is taken from:

[…]We wish to talk.

Johnny blinked at the message on the screen.[…]

We wish to talk.

His finger hovered on the Fire button. Then, withoutreally looking, he moved it over to the keyboard andpressed Pause.

Then he read the manual. […]

There was nothing in the manual about messages.Johnny riffled through the pages. It must be one of the New Features the game was Packed With.

He put down the book, put his hands on the keysand cautiously tapped out: Die, alein [sic] scum/

No! We do not wish to die! We wish to talk!

It wasn’t supposed to be like this, was it?

Wobbler Johnson who’d given him the disc andphotocopied the manual on his dad’s copier, had saidthat once you’d completed level 10 you got given anextra 10.000 points and the Scroll of Valour andmoved on to the Arcturus Sector, where there weredifferent ships and more of them.

Johnny wanted the Scroll of Valour.

Johnny fired the laser one more time. Swsssh. Hedidn’t really know why. It was just because you hadthe joystick and there was the Fire button and thatwas what it was for.

After all, there wasn’t a Don’t Fire button.(Pratchett 1993, 10-11. Pratchett's italics).

To some extent, the quote crystallizes why we engage in computer games.

They provide something desirable to us. They provide experiences that are

schematic in their nature and that are well suited for rapid understanding of

the game environment and sensory immersion. They provide structuredevents and possibilities for interaction. If we study the quote closely, we will




6

find some interesting things. “We wish to talk”, says the game. This is

communicated by written language on the screen as the following sentence of

the quote reveals. Johnny’s reaction in turn reveals that he is surprised by

this output from the game. He has a general understanding of what

computer game playing is all about being a well experienced player. It is notwithin the usual schematic structure for the game to behave like this. It is a

problem so unusual even that Johnny has to read the manual before

proceeding with his session. Although he reads the manual, he can not find

any explanation for the game’s unexpected behavior. In order to solve this

problem and to ease his confusion, he decides to make up a hypothesis of his

own on why the game behaves this way. He decides that this must be some

result of the new features the game is supposed to be packed with. In

addition, such features do match the general schematic understanding of

computer games. That is to say, it matches well the presumptions to bemade about such games. When Johnny has tried to adjust to what is

happening and found a plausible explanation, he starts over and begins to

interact with the game. He writes a response to the whish the game system,

or rather the inhabitants of the game, puts forth. In this response, he makes

a very personal attack on them when offending them: “Die alein scum!” (sic.)

as he writes. In other words, he is personalizing the game system and he is

animating it. Again, the game does something unexpected when repeating

its request of wishing to talk (as opposed to dying) and Johnny once again

has to readjust his hypothesis of what is going on. He has to think back on

what a friend has said and not said about the game and he finds that there

is no explanation to be found in this. He only knows that once past level ten

he will be rewarded. This is one of is motivations for playing the game. He

wants to reach new levels and get rewards for his achievements. This is

reinforcement of his actions. So… Johnny now gets himself back to an initial

state of game playing and fires the laser although not really sure why he

does so. He finds again a motivation for his action: there is a Fire button and

this urges him to press it and to fire at the opponent. He fires his weaponseven if this particular opponent behaves unexpectedly and does things that

have never been heard of before. That the opponent in the game wishes to

talk things over instead of blasting Johnny to pieces is something that

Johnny has never encountered earlier (which is suggested by Johnny

reading the manual to find what is going on).

We will now leave Johnny and his trouble of coping with this, for him, very

unusual situation. However, we will get back to him later, more precisely in

chapter 4, 67, when we have some additional terminology to employ on thisquote to deepen the analysis.



INTRODUCTION

7

Having now introduced the objectives of this dissertation, it is time for some

formal remarks about sources, references, criteria for choosing certain games

and movies for analysis etc.

1.1

Sources and references. Source critic remarks

Within the present work, I have used an array of sources of different kinds.

Some of them, the ones that are the most theoretical, are traditional

academic sources such as books and articles published by well known

publishing houses or journals. Other sources are movies, television program

and computer applications. The latter are mainly computer games including

computer games played on arcade cabinets, home computer game systems

like Nintendo Entertainment System (NES), Atari, Odyssey, Sony

PlayStation etc. or emulated ROMs from such games and of course gameswritten for home PCs of some kind be it AMIGA, Macintosh or IBM

compatibles. I have also used several Internet sources such as news groups

and web sites. What follows is a presentation of the most important sources

and how I have dealt with them.

1.1.1 Books

The most important source books for this dissertation provide its main

theoretical framework of experientialist cognitive theory. This is the work by

George Lakoff and Mark Johnson. The source material includes their co-

written volumes Metaphors We Live By (1980) and Philosophy in the Flesh

(1999) and their individual work The Body in the Mind (Johnson 1990),

Women, Fire, and Dangerous Things (Lakoff 1987a). There are also some

articles of importance from the pen of George Lakoff. ’Cognitive Semantics’

(1987b), ’The Contemporary Theory of Metaphor’ (1993) and, ’ Sorry I’m Not

Myself Today’ (1996).

James J. Gibson’s The Ecological Approach to Visual Perception (1976) is

also a book worth to mention in this overview since it in part contains ideas

that Lakoff and Johnson adopts in their work. Furthermore, as this

dissertation will show Gibson’s ecological approach to visual perception and

his theory of affordances are in great parts directly applicable on computer

games.

Edward Branigan’s Point of View in the Cinema (1984) and David Bordwell’s

Narration in the Fiction Film (1997) are discussed in chapter 3 on point of

view as a concept. I contrast two ways of understanding this concept by

examples from Branigan and Bordwell respectively.




8

1.1.2 Internet sources

Internet sources of significance are http://www.classicgaming.com, which as

the name reveals is a site for information on classic computer games. It is

also a source for emulators and such material.

http://videogames.gamespot.com/ is another site well worth a visit. Some of

the material on Pac-Man comes from this site for instance.

At University of California at Berkeley is the web-site

http://cogsci.berkeley.edu/metaphors / a useful source for those of you who are

interested in further studies on metaphor.

Internet sources, e-mail, news groups, the World Wide Web or other net

sources, are troublesome since they:

1) Might contain more rumors and myths than verified material (news

groups and such). Rumors are maybe to be considered to be kind of a

backbone of the internet (think about how many e-mail you have got saying

that they originate from a guy called Bill Gates at Microsoft giving away this

or that application or from Ericson giving away free WAP cell phones and so

on). The World Wide Web is no better. Game sites handling demos of games

are constantly giving a lot of space to rumors about forthcoming games, new

game systems technical specs. etc.

2) They might be gone when you try to visit them. This is a constant

problem. Some sites contain copyrighted material that is not owned by the

people running the site and hence they must relocate every now and then.

There is also the problem of economics. It is not always free to keep a site

running.

Internet sources in the case of this present work are good in many ways:

1) They provide a dynamic source of information (news groups and such). It

is possible to find information otherwise unpublished or that is soon to be

published through other media channels. The fact that almost anyone

connected to the Internet as a possible content provider is an amazing thing

never before heard of in the history of man.

2) Internet sources provide insight to a culture of computer games and is a

large part of this culture. (There are numerous sites and home pages

containing material of various kinds e.g. specialized PONG sites

(http://www.emuclassics.com/slydc/pong.htm), Pac-Man sites

(http://www.videogames.com) and the like. Almost any of the “big hits” of the



INTRODUCTION

9

arcade of the 1980s has one or more sites on the Internet. Several of these

sites provide game players’ comments about and arguments for the benefits

or shortcomings of this or that game. These have been a valuable source of

information since this kind of material is possible to analyze on the same

premises as the computer games. In other words, this material containsanalyzable judgments about games that reveal how game players have

conceived the games when they where new and how they remember them.

Through these kinds of sources, it is possible to get an understanding about

their interaction experience with these kinds of games.

3) Internet sources contain material otherwise more or less impossible to get

hold of within reasonable time and cost constraints. Game flyers, sound

samples, ROM images, photographs of long since obsolete arcade consoles

and home computer game systems are out there for anyone to find. There aresites whose only content is just game flyers, for instance. Such material is

important since it provides a cultural – and in some cases social context – for

the games. It also reveals the physical setup of game cabinets considering

design, size, control panels, etc. This is important since, as I argue and show

in my analyses we do get a proprioception of the game environment while

playing. In part, this is due to such factors as placement of the monitor and

controls. This in turn results in kinesthesia, i.e. the subjective sensory

awareness of the position of limbs in the environment. Nowhere today, with

the exception maybe of museums such as the American Musem of theMoving Image (AMMI) in New York, some other museum or private

collectors, will one have the possibility to get such a good overview of the

range of games that once were and in some cases still are out there.2 If

interested, one may dwell deep into technical descriptions of software and

hardware, learn how to hack ROM images etc. at such internet sites.

1.1.2.1 Criteria for the use of internet sources

To use Internet sources demands sorting and filtering (as always) andcertain precautions must be made: who is the publisher? On which kind of

server does this information reside? Is it a “dot com” (.com) or “dot.gov” (.gov)

or “dot edu” (.edu)? Which kind of information might I find in a .com or .gov

and is the information “correct” and trustworthy? How shall Internet sources

be judged? Since there are numerous hoaxes, commercial interests, rumors

etc. flooding the infrastructure of the Internet what value may one assign to

information obtained through this channel? In a digital world like the

2 You can visit AMMI's Web-site to get an idea of what a 1998 exhibit incorporated.http://www.ammi.org/exhibitions/cs98 /




10

Internet, everything is possible to falsify to large extents. 3 We don’t hesitate

to trust a book published by a well known publishing house as having some

academic credibility do we? An Internet source like http://www.eb.com

(Encyclopedia Britannica Online) is trustworthy to large degree. However,

all sites can be hacked and altered. Therefore, one has to be cautious and tryto cross check material obtained from the Internet if possible. The problem is

as mentioned that some material is otherwise impossible to obtain. In

addition, as I have argued, some material is part of the culture of computer

games.

1.2 Chapter overview

What follows is a chapter overview. I will provide the reader with a “map” of

my work by making an outline of the chapters for the ease of reading.

Chapter 2: Cognitive theory as a framework for the study of film is

introduced in this chapter. The work of Hugo Münsterberg is briefly

introduced and discussed, as are some important scholars that have taken

on a cognitive approach to the study of film: David Bordwell, Noël Carroll,

Joseph D. Anderson, Edward Branigan and Torben Grodal.

Chapter 3: This chapter contains an elaborate discussion on the point of view

concept as the basis for subjectivity, identity, and identification.

Chapter 4: James J. Gibson’s ecological approach to visual perception is

introduced in this chapter, as is his understanding of affordances and

constraints.

Chapter 5: The focus is put on the work of George Lakoff and Mark Johnson

and what they call experientialist cognition. This chapter is quite long due to

the importance their theory has for my suggested approach. Also, some other

ideas are presented such as cognitive and perceptual sets etc. When

discussing metaphorical structures of the human mind I try to connect themto the present work and the analyses in chapter 9.

Chapter 6: This chapter tries to show that a cognitive approach, as well as

Gibson’s ecological approach to visual perception and the environment

3 A striking example hereof is what happened to the Swedish Web-site Levande Historia (Living History)which was a state funded project that had as its goal to provide information of the holocaust. This site wascopied by antisemitic interests. Its structure, its layout, the persons interviewed etc look very similar to the

original site and were actually in some cases identical with the exceptions of a few words here and therethat severely changed the meaning of what was said on the original site. The falsified site was called SannHistoria (True History). For a more detailed discussion see Mattus 1999.



INTRODUCTION

11

within which we are contained are a well-suited ground for the study of

sound as such and in the combination with visuals.

Chapter 7: Interaction and interactivity are two important terms that are

thoroughly discussed and elaborated on in this chapter. Jens F. Jensen’sobjective understanding of the terms is questioned and an alternative

understanding is suggested: a subjective experiential approach to the terms

with its ground in the experientialist theory of cognition.

Chapter 8: My concept of the Game Ego is explained in this chapter. The

Game Ego is a bodily based function that we use to control a game

environment.

Chapter 9: This chapter contains a number of analyses of computer games.

The games analyzed are as follows: Zork, Tennis for Two (Higinbotham

1958), Pong (Atari 1972), Space Invaders (Taito 1979) Pac-Man, Bubble

Trouble (Metcalf, Wareing and Ambrosia Software 1995-97), Defender

(Williams 1982), NHL 2000 (EA Sports 1999), Day of the Tentacle (Lucas

Arts 1993), and Myst. The analyses in chapter 9 are in some cases referred

to in earlier chapters and in these cases I refer to those chapters in the text.

Chapter 10: This is the summary of the whole dissertation where I try to tie

everything together and put forth the benefits of the theoretical framework

suggested.




12



COGNITIVE THEORY IN THE STUDY OF AUDIOVISUAL MEDIA

13

2 Cognitive theory in the study of audiovisual media

The following chapter introduces cognitive theory and the cognitive approach

for the analysis of audiovisual media. Throughout this dissertation, I will

argue for the need of a theoretical framework incorporating cognitive theory

for such analysis. In this chapter, I will briefly present some of the theorists

that have introduced a cognitive approach to the study of film. There will be

a more elaborate discussion in chapter 5 on the experientialist cognitive

theory as presented by George Lakoff and Mark Johnson. For the purposes

of providing a broader basis for understanding their critique against

objectivist traditions of cognitive theory presented in this following chapter

and the outlines of experientialist cognition presented in chapter 5, I will say

just a few words about them. At the time of writing, Lakoff is Professor of Linguistics at the University of California at Berkeley. He is found in the

field of cognitive semantics and has been an active scholar within linguistics

since the mid-1960s. Mark Johnson is Professor of Philosophy at the

University of Oregon. He has a special interest in cognitive theory,

philosophy of language and philosophy of art. He is also interested in Kant

studies, which this chapter will show.

2.1 Defintions of cognition

First, we must notice that “cognitive theory” is a vast field of theories and

not a unified approach to a field of research. Lakoff and Johnson suggest

that cognitive science was founded as a scientific discipline in the 1970s

(Lakoff, Johnson 1999, 10). However, it has it origins further back in time

with theorists such as Jean Piaget for instance. My aim is to show how a

particular direction of cognitive theory can be applied within the study of

audiovisual computer games: the experientialist cognitive theory of George

Lakoff and Mark Johnson.

However, before taking on experientialism head on, let me first provide a

lexical definition of what cognition is considered to be. This I do in the

purpose of outlining what will be meant by cognition in the present work,

since cognition is a term that has undergone definitional changes. I will

comment upon this lexical definition provided and point to some of the

problems that it evokes for an experientialist approach to the field of

cognitive studies:




14

[Cognition :] the process involved in knowing, or theact of knowing, which in its completeness includes

perception and judgment. Cognition includes everymental process that can be described as anexperience of knowing as distinguished from anexperience of feeling or of willing. It includes, inshort, all processes of consciousness by whichknowledge is built up, including perceiving,recognizing, conceiving, and reasoning. The essenceof cognition is judgment, in which a certain object isdistinguished from other objects and ischaracterized by some concept or concepts.

(” cognition” Encyclopædia Britannica Online.<http://www.eb.com:180/bol/topic?idxref=82659>2000-04-12).

Under this type of description cognition is something concerned with

knowledge and not with feelings and emotions since this lexical definition

implicitly assumes and rests upon a philosophical tradition stating that

feelings and emotions are impossible to describe precisely. However, this is

certainly not a complete understanding of the field of cognition as related to

feelings and emotions.

This lexical definition excludes any unconscious activities and/or processes of

the mind. Cognition is consciousness building knowledge. It is concerned

with perception, recognition, conception, and reason as conscious processes.

One could of course understand the above definition as presupposing an

unconscious state of mind that is at a pre-cognitive phase or level. However,

this again makes a division between what is and what is not cognition.

Maybe I am a little narrow in my inference of the definition here, stressing

this division between conscious and unconscious but I am doing so for good

reasons. Lakoff and Johnson offer a definition of what cognition is and also

provide an explanation to some of the confusion that the term cognitive has

caused because of its different meanings within cognitive science and

philosophy respectively. Cognitive science, in their understanding, is the

scientific discipline that studies conceptual systems. (Lakoff, Johnson 1999,

10). Cognitive within cognitive science denotes mental operations or

structures of any kind that can be studied in precise terms. Visual

processing as well as auditory processing is then cognitive. However, visual

and auditory processing is not conscious. We are not aware of each neuralprocess and we can not be so either.




15

In addition, memory and attention is cognitive. That is to say that all

aspects of thought and language whether they are conscious or unconscious

are cognitive (as long as they can be described in precise terms). (Lakoff,

Johnson 1999, 11). Lakoff and Johnson, and not only they, but also a

majority of modern neuropsychologists note that most cognitive processing isunconscious. Not unconscious in the way of Freudian psychology though, as

repressed by consciousness, but as they put it “in a sense that it operates

beneath the level of cognitive awareness, inaccessible to consciousness and

operating too quickly to be focussed on”. (Lakoff, Johnson 1999, 10). Thus, it

is important not to confuse unconscious in the Freudian psychoanalytic

tradition with unconscious in this sense and within the context of

experientialist theory of cognition. This is noted by Torben Grodal, who

strives to merge emotions and cognition within film theory. He advocates an

ecological and evolutionary approach to film studies. The way Freud and hisfollowers in psychoanalysis is concerned with cognitive process as secondary

processes are downright converse to how an evolutionary approach treats

them, Grodal remarks. In an evolutionary approach the cognitive processes

are:

[…]the primary processes, [which Freud and otherscall ’secondary’ processes] which we share with therest of the animal kingdom because we want to

perceive and represent the world in such a way thatby actions we can implement our body-brain-preferences in an optimal way. (Grodal 1994, 7-8.Grodal’s italics).

Within certain philosophical traditions however, the term cognitive denote

only conceptual or propositional structure and the application of rule-

governed operations on such structures. Cognitive meaning within such

traditions is truth-conditional and does not admit internal meaning of the

body and mind. Meaning is instead a relation between a concept and a thing

in the external world. This is objectivism and neither Lakoff nor Johnson

adapts to this tradition. On the contrary, they have both been on a “crusade”

against objectivism and objectivist cognition in favor of a theory based in

experientialism. Let us now therefore turn our attention to their critique of

this tradition. They find that objectivist cognition is a tradition that does not

sufficiently explain some salient features of the human conceptual system.

As Johnson puts it:

[…]”Objectivism” [is the] offending cluster of

assumptions that has led to this blindness towardimagination. (Johnson 1990, ix-x).




16

We must note however that Johnson does not want to give up an objective

understanding of the world but this objectiveness must take on a new

dimension: the embodied mind.

Objectivity does not require taking up God’sperspective, which is impossible; rather, it requirestaking up appropriately shared human perspectivesthat are tied to reality through our embodiedimaginative understanding. (Johnson 1990, 212).

Imagination is an important factor in how the human conceptual system

works according to Johnson. Imagination is essential for human beings.

Without imagination, nothing in the world could bemeaningful. Without imagination, we could nevermake sense of our experience. Without imagination,we could never reason toward knowledge of reality.(Johnson 1990, ix).

As we see from this quote, imagination is that what makes the world

meaningful.4 The role of imagination has been neglected by theories on

meaning and rational reasoning Johnson claims. However, to understand

this standpoint one need to get a deeper understanding of what is actually

meant with imagination within this context. Johnson remarks that many

conceive of imagination as something that has to do with artistic expression,fantasy etc. but that this is a 19th century romantic conception of what

imagination is. (Ibid. 139). Imagination in Johnson’s understanding of it is:

[…] our capacity to organize mental representations(especially percepts, images, and image schemata)into meaningful, coherent unities. (Johnson 1990,140).

Johnson builds his understanding of imagination on Kant’s use of the term

“Einbildungskraft” i.e. how knowledge involves judgments in which sense

percepts, images, or concepts are unified and ordered under representations

that are more general. Imagination is the act of making this unification.

(Ibid. 148). Imagination is non-propsitional and does consequently not fit

into an objectivist tradition where such judgments as true and false are

predominant. As we will se later during a discussion of image schemas, Kant

had an understanding of “Schemata” as non-propositional structures of

4

There is, to me, a resemblance here to the difference between the physical world and the ecologicalenvironment discussed in chapter 4. The physical world is meaningless to the animal while the ecologicalenvironment is meaningful according to James J. Gibson.




17

imagination that connect concepts with percepts and as procedures for

constructing images. (Ibid. 19 and 21).

Johnson has been criticized for his inability to overcome Kant’s formalism.

“Coleridge’s account of imagination”, writes David S. Miall, “provides abetter foundation for examining the bodily basis of meaning, while

remaining compatible with Johnson’s intentions and his more valuable

insights.” Miall continues: “Unlike in the aesthetics of Kant to which

Johnson appeals, there is no dichotomy of body and mind. The body can

prompt thought, or can be its instrument.” (Miall 1997, 191-210). Miall

points out that Johnson puts Coleridge into the romantic tradition of

imagination. In doing so, Miall suggests, Johnson belittles Coleridge’s effort

and does not take into account Coleridge’s recognition of imagination as

being a force that “dissolves, diffuses, dissipates, in order to recreate”(Originally from Coleridge, 1817, chapter 13, quoted both in Miall’s paper as

well as in Johnson’s book). 5 Miall is right in saying that Johnson does not

really go deep into the imagination concept of Coleridge’s. However, Johnson

does provide a reason why:

Putting aside the interpretative difficulties of finding a coherent and intelligible theory in Biographia Literaria, we can at least say thatColeridge captured the Romantic, antireductivist view of meaning. […] What Coleridge neversupplied, however, was an account of the specificnature of this creative, unifying activity of metaphorical imagination. (Johnson 1990, 69).

Miall finds also “a neglect of other bodily influences on thought, especially

kinaesthetic and affective aspects” in Johnson’s work. (Miall 1997, 191-210).

It is true that Johnson is mainly concerned with spatial relations and the

human being moving around in the world as the bodily basis for meaningful

metaphorical concepts. When studying Johnson a little closer though, onemight find ” a path less traveled by” to paraphrase Robert Frost.6 Johnson

does suggest that manipulation of objects and perceptual interaction is part

of this meaningful structuring of concepts. (Johnson 1990, 29). So, there is

not really a neglect of other bodily influences but rather a focusing on what

Johnson finds to be salient aspects of the bodily basis of human cognition.

5

Miall however does not list this book in his references.6 "I took the one less traveled by, And that has made all the difference" is the last phrase of Robert Frost'spoem The road not taken (Frost 1920).




18

There have also been other critical voices raised against Johnson’s approach

and what has been found as a vagueness and ambiguity in his thesis. One

reviewer of The Body in the Mind, R J. Wallace, found that it is unclear

whether Johnson wishes to claim that all meaning remains within the

context of bodily experience, or whether meaning emerges from bodilyexperience by projection and transformation. (Wallace 1988, 225-227).

Let us now turn our attention to Lakoff’s critique of the obejctivist tradition

within cognitive theory. Lakoff notes that there are two aspects of the

tradition of objectivist cognition:

1) The algorithmic theory of mental processes: allmental processes are algorithmic in themathematical sense, that is, they are formal

manipulations of arbitrary symbols without regardto the internal structure of the symbols or to theirmeaning.

2) The symbolic theory of meaning: arbitrarysymbols can be made meaningful in one and onlyone way: by being associated with things in theworld (where ”the world” is taken to as having astructure independent of the mental processes of any beings). (Lakoff 1987b, 119).

Objectivist cognition separates the symbols from their meaning. The symbols

function as internal representations of external reality. Lakoff points out the

problem of such a view when asking how the symbols used in human

thought are made meaningful? (Lakoff 1987b, 119f). The objectivist theory

claims that the meaning of symbols is made through the algorithmic

manipulation of arbitrary abstract symbols that are meaningless in

themselves, but get their meaning by being associated with things in the

external world. The external world has its own structure independent of

mental processes of any being. However, this in turn would mean that:

1) Meaning is based on reference and truth.

2) Truth consists of the correspondence between symbols and states of affairs

in the world.

3) There is an objectively correct way to associate symbols with things in the

world.




19

Two important things are left out of the objectivist theory of cognition:

1) The role of the body in characterizing meaningful concepts, and

2) The human imaginative capacity for creating concepts and modes of

irrationality that goes well beyond any mind-free external reality.

To clarify the consequences of this let us study the following example that

Lakoff provide:

What is the meaning of ”Tuesday”? If, as objectivistcognition suggests, symbols get their meaning onlyby being associated with things in the world, thenweeks must be things in the world. But weeks do notexist in nature. Different cultures have differentlengths of weeks. In Bali, there are many kinds of weeks of various lengths, all of which existsimultaneously. Weeks are an imaginative creationof the human mind. In order to know what”Tuesday” means, we need to know what weeks areand how they are structured.

The kind of imaginative structures required for thedefinition of concepts such as ”Tuesday” have beencalled ”frames” or ”schemas”. The central claim forcontemporary cognitive anthropology is that most of our cultural reality resides not in the artifacts of society, but in the culture specific schemas imposedby human beings […] ”Tuesday” is meaningful onlyrelative to a weeks-schema. […] Culturally definedschemas are a product of human imaginativecapacities and, as such do not have a place withinobjectivist cognition. (Lakoff 1987b, 135-136).

Culture does play a great role in shaping our concepts as the result of social

experience as do our bodies play a central role in shaping the conceptual

system. Concepts are neural structures writes Lakoff and Johnson (Lakoff,

Johnson 1999, 20). Moreover, neural structures are bodily structures.

[…] the structure inherent in our experience makesconceptual understanding possible andconstrains–tightly in many cases–the range of possible conceptual and rational structures. (Lakoff 1987b, 120).




20

Having now provided an initial definition of cognition and pointed to some

problems with this field of research let us turn our attention to the study of

film in the following paragraph. We will get back to an elaborate discussion

on Lakoff’s and Johnson’s approach to cognition in chapter 5.

2.2 In the beginning was Hugo Münsterberg

Film theory has undergone quite some changes during its relatively short

life span. Philosopher and psychologist Hugo Münsterberg’s two premier

works on film, Why We Go to the Movies, an article published in

Cosmopolitan December 15, 1915 and his later book The Photoplay: A

Psychological Study, from 1916 are considered to introduce the earliest

theory of film and being the first studies of film with a scientific approach.

They also constitute the body of the most direct theory of film. As J. Dudley Andrew points out Münsterberg had no foregoers and he had not been

interested in going to the movies prior to his study. (See Andrew 1976,

chapter 1). In fact, he had only been taking part in movies for about ten

months while preparing his study. While doing so, he approached them not

as crude and silly things but with the intention to make a scientific and

philosophical study of how film works.7 Münsterberg had a good knowledge

about the empirical studies carried out in psychology when he set out to

write his book on film.8 On the invitation of the psychologist William James,

Münsterberg became a visiting professor at Harvard University between1892-95. In 1897 he returned to Harvard to be the director of the

psychological laboratory. His work there is often considered to be the

foundation to what has become applied psychology. His 1914 publication

Psychology: General and Applied of course hints at this direction of his

studies. Münsterberg’s theory on film was never elaborated by himself in

more works. He died 1916, the same year it was published. (” Münsterberg,

Hugo” Encyclopædia Britannica Online.

<http://www.eb.com:180/bol/topic?eu=55666&sctn=1> [2001-01-10]).

7 Cf. J. Dudley Andrew's The Major Film Theories in which the first chapter is an elaborate exploration of Hugo Münsterberg's work. Other that has acknowledged Münsterberg's work are Joseph D. Anderson.(The Reality of Illusion), John Alexander, (ScreenPlay), Torben Grodal ( Moving Pictures), and David

Bordwell ( Narration in the Fiction Film), to mention just a few.8 Furthermore, Münsterberg had also a background in philosophy. For a period he was even chair of thephilosophy department of Harvard.




21

Münsterberg’s primary interest in film lies more in the audience and the

way we take part in film than in how filmmakers actually produce film. That

is to say, ” the spectator” side of film was of greater interest to him than the

production side. What goes on in the human mind while taking part in a

photoplay (as he called the narrative film) is what he is concerned with. Thematerial of movies is the human mind and its narrative capabilities.

Münsterberg divided the development of cinema into three stages: playing

with visual gadgetry, being a tool for information and education and finally

being, by virtue of its possibility to be narrative, a thing of the human mind.

As Joseph D. Anderson notes, Münsterberg recognized that ” the motion

picture is structured in a way that is analogous to the human mind” .

(Anderson 1996, 4). To Münsterberg, the human mind’s narrative

capabilities played a great role. The human mind in his understanding has

several levels of organization where the higher ones depend on the lower

ones. Film has its basis not in technology but in the human mind. (Cf.

Andrew 1976, 18-20).

Andrew notes that Münsterberg, like the Gestalt Psychologists, which he

preceded, held the opinion that all experiences are PART–WHOLE

structures and that attention is a factor in organizing the perceptual field.

Münsterberg used, what is now ” classical examples” of figure and ground

reversal to show how we willingly can shift our attention towards a stimulus

and make the figure become the ground. Movement of an object, a cloud in

front of the moon, can be reversed to become movement of the moon behind

the clouds, for instance. (Andrew 1976, 16-17). As chapter 5 will show,

Lakoff’s and Johnson’s experientialist approach to cognition acknowledge

that:

Rational thought is the application of very generalcognitive processes - focusing, scanning, superimpo-sitioning, figure-ground reversal, schema etc. [to

certain well structured aspects of bodily andinteractional experience to abstract conceptualstructures]. (Lakoff 1987, 121).

This goes to show that Münsterberg’s work does connect to the

experientialist theory of cognition in that it makes clear that such primary

levels of the human mind incorporate very important functions.

Münsterberg’s description of the Phi-phenomenon as something that works

and organizes our minds at a very basic level shows this, for instance. The

Phi-phenomenon is an active organizing function of the mind for him. We donot need to decide whether the description of the Phi-phenomenon is actually




22

a correct description of how the visual sense works. Gibson would not adhere

to it for instance. It is enough, in this case anyway, to recognize that the Phi-

phenomenon is something that works at a basic level of the human mind.

Also the fact that both Münsterberg and Mark Johnson are influenced by thework of Immanuel Kant (Münsterberg’s work does in its second part use

Neo-Kantian ideas in describing the aesthetics of movies) indicates a

relation between Münsterberg and Lakoff/Johnson.

To summarize, Münsterberg’s study brought film into science and science

into film so to speak. His serious treatment of what was largely considered a

vehicle of simple entertainment, in time has given film a higher status as

both a tool for psychological study and a field worth research in its own

right. He also brought to film an aesthetic approach from a morephilosophical point of departure, the Neo-Kantian school. Psychology made it

into the analysis of movies as did philosophy.

2.3 Post psychoanalysis: The cognitive approach in short

Psychology and psychoanalysis have always been important for the field of

film theory as the previous paragraph shows.9 Psychology was, as we have

found out in the previous paragraph, at the basis for the first theory of film:

the theory put forth by Hugo Münsterberg. Freudian, Lacanian, and Jungian

psychology later found its way into and periodically later dominated the

study of film. These theoretical frameworks were also accompanied by

Sausurrian semiology, Peirceian semiotics, Marxist theories, feminist

theories, narrative and structuralist theories, general post modern theories,

social-psychological theories etc. This list can be made even longer. As we

can conclude from this, there are many ways to approach film or other

audiovisual media. It is not the case that such methods as those mentioned

do not make possible a deeper understanding of film. In fact, one might

wonder if there really is a need for yet another attempt to re-frame film andaudiovisual media theoretically. However, the theories and frameworks

mentioned do not cover everything of course. Nor do they always provide the

best tools for specific rather basic questions about the human being and the

conceptual system of human beings. I am not suggesting that cognitive

theory would be some kind of super theory or grand theory that explains

everything we will ever want to know about audio-visual media and the

human being. Since cognitive theory is not even a coherent scientific field of

9

I would say that the influence of psychoanalysis really is for good and for bad. This is however not theplace for an elaborate discussion about pros and cons with a psychoanalytical approach to film studies. Wewill come to such a discussion in chapter three to some extent though.




23

research, we can not expect it to be this super theory. Nevertheless, if one is

interested in alternative answers to questions of identification with

characters, the me-and-other-relation, the foundation for human concepts,

how computer games work and why they are engaging one would maybe be

wise to explore this field of research. There have been some important effortswithin film theory to bring to it theories on cognition.10 These efforts have

stirred things up and brought attention to problems in analyzing movies in

the tradition of psychoanalysts like Freud, Lacan and Jung and the tradition

of semiotics and semiology.

2.3.1 Bordwell

The impact and importance of David Bordwell’s work on film theory can not

easily be disregarded. In numerous works, he has put forth substantialanalyses of the film medium and also of the theories concerned with film. He

provoked theorists and scholars rooted within such theories by naming them

” SLAB Theory” where ” SLAB” of course stands for Saussure, Lacan,

Althusser, Barthes). In Making Meaning (1991) he puts forth an elaborate

critique on how film studies has been carried out especially pointing to the

problem of the dominant traditions of psychoanalysis and semiotics. For the

present work however, some other works by Bordwell are maybe even more

important than Making Meaning. The next chapter for instance will include

a close study of how Bordwell uses the term point of view in his Narration inthe Fiction Film. Bordwell has also written Film Art: An Introduction (1993)

together with Kristin Thompson. I have used part of their ideas on sound

presented in that book in the present work. I will therefore not go into a

detailed discussion on Bordwell in this paragraph since it is incorporated in

other parts of the dissertation.

2.3.2 Carroll

Noël Carroll’s Mystifying Movies (1988) is another attack on psychoanalysisas a framework for filmstudies. As the title of Carroll’s work suggests, he

finds that psychoanalysis can be a way of mystifying movies rather than

explaining them by locking movies into psychological systems of explanation.

To Carroll, castration anxiety, Oedipus complex etc. serves not to explain

10 The maybe most well known scholars within film theory that have explored and applied cognitivetheories in their work are Noël Carroll (Theorizing the Moving Image, Mystifying Movies), DavidBordwell (Making Meaning and Post-Theory co-written with Carroll), Edward Branigan (Point of View in

the Cinema, Narrative Comprehension and Film), Joseph D. Anderson (The Reality of Illusion: An

Ecological Approach to Cognitive Film Theory), Torben Grodal ( Moving Pictures: A New Theory of Film

Genres, Feelings, and Cognition) Carl Plantinga (Passionate Views: Thinking about Film and Emotion co-written with Greg M. Smith), and Ed Tan ( Emotions and the Structure of Narrative Film: Film as An

Emotion Machine).




24

film but to mystify what really goes on in the human mind while taking part

in them. I will however not go into detail on Carroll in this present work.

2.3.3 Anderson

Joseph D. Anderson’s The Reality of Illusion: An Ecological Approach to

Cognitive Film Theory, is an attempt to merge an ecological approach to

visual perception, i.e. the theory of J.J. Gibson with cognitive theory. In that

intention his work is close to what I am intending here. Anderson takes

another path though in that he does not as I do explore the experientialist

cognitive theory of Lakoff and Johnson. However, he does make some

remarks that we will have reason to study further since he (for instance)

points out some common factors to the two approaches concerning

categorization. Categorization is at the very core of Lakoff and Johnson’stheory. Unfortunately, Anderson does not explicitly make the connection

between Lakoff/Johnson and Gibson but to Eleanor Rosch and Ulric Nessier.

There are in fact good reasons for exploring Lakoff’s work on categorization

which chapters to follow reveals.

2.3.4 Branigan

Edward Branigan is yet another scholar that has taken a cognitive approach

to the study of film. His possibly best-known work is Narrative

Comprehension and Film. Since his work (in some respects) is important for

my dissertation and my general approach to the study of film and other

audiovisual media, I will go into some detail on his theory. To simplify,

schema theory and narrative structure are the topics for Branigan’s study:

Narrative is a perceptual activity that organizesdata into a special pattern which represents andexplains experience. More specifically, narrative is away of organizing spatial and temporal data into a

cause-effect chain of events with a beginning, middleand end that embodies a judgement about thenature of the events as well as demonstrates how itis possible to know, and hence to narrate, theevents. (Branigan 1992, 3).

” Narrative” may refer either to the product of storytelling or to the process of

storytelling as such. Branigan uses the word ” perception” in the meaning of:

a ” percept” derived from reality

preconscious assumptions about reality




25

an intuition (color seems to be intrinsic and permanent to an object, to be

created and contingent)

propositional conclusion about sensory perception from reasoning

an attitude we adopt when confronted by something that is a representation

of something else.

This is the context for understanding the use of ” perception” in the following

quote:

[…]narrative can be seen as an organization of experience which draws together many aspects of our spatial, temporal and casual perception.

(Branigan 1992, 4).In a narrative, there is something and there is something going on. That is,

there are objects (existents, or even better: beings) and processes within a

narrative. There is also a pattern that gives structure to the processes and

the existents. Within a computer game, the structure may be based on

participation based on interactivity of some kind. As will be discussed in

chapter 7 of the present work, Danish media scholar Jens F Jensen defines

interaction as a continuum. (Jensen 1998). However, for reasons explained

in chapter 7, this definition of interactivity is perhaps not an altogether

fruitful approach.

Branigan notes that theories about narrative based on the linguistics of

Ferdinand de Saussure and Noam Chomsky is now more and more modified

and superseded by other theories for example the use of ” fuzzy” concepts and

” fuzzy sets” , metaphor and frame arrays etc. has been introduced as

theoretical frameworks for the study of film.




26

Identifying an event as a ” story event” is a matter of deciding where actions begin, how they break off,and which actions belong together. We must judgenot only the temporal status of special cross-cut

actions but the time implied by the juxtaposition of any two shots including ensuring that time has notstopped or otherwise shifted within a shot. (Note,again, that the physical material of film – such asthe break between two shots – does not guarantee a priori a specific temporal relation nor guarantee achange in temporal relations.) We use hypothesesabout time to search for causation and, reciprocally,we use hypotheses about causality to establishtemporal order. Identifying what counts as an event

involves searching for an ” equilibrium” amongpossible ” values” for space, time and causation.(Branigan 1992, 48).

Here we find some differences between film and computer games. The action

in computer games is sometimes continuos and there might be fewer shifts of

location and so on. Day of the Tentacle is one source for such an analysis

because of its resemblance to film. There are narrative parts of the game

that lie beyond the game player’s control – i.e. the game player can not

control the movement of the characters, their utterances or the evolvement

of the story. Narratives are not necessarily something that has to do with

language as verbal activity even if they may be represented as and through

such activity. Narratives are representations of audio-visuo-emotional-

cognitive-motor schemata, whose central form is the experiential action

sequence.

2.3.5 Grodal

Let us now get back to the lexical definition that introduced this chapter. In

that definition, we had a dichotomy between cognition and emotions. TorbenGrodal offers a different understanding of this dichotomy of knowledge, i.e.

cognition, versus emotions and feelings, bringing together emotions and

cognitive processes instead of separating them. Grodal remarks that there is

really no reason to believe that it should be easier to predict a cognitive

reaction in a film viewer than an emotional one. They are both in the

body/mind system interior of the viewer. (Grodal 1994, 7). In addition,

Grodal provides an overview of two main trends within emotion research.

One school states, following the work of James and Lange, that emotions are

identical with the bodily reactions in a given situation.




27

Another school follows Cannon’s work and his claim that emotions rely on

subcortical structures. Grodal’s position is a middle course. “[…] the

emotions cued by visual fictions rely heavily on cognitive evaluations but the

strength of the experience also relies on the bodily activation.” (Ibid. 16).

Commenting the work of Orthony, Clore and Collins, Grodal notes that theirunderstanding of emotions is a purely cognitive one defining emotions in

cognitive terms as “valenced reactions to events, agents, or objects, with

their particular nature being determined by the way in which the eliciting

situation is constructed.” (The quote is originally from Orthony, Glore, and

Collins 1988, 13. Here quoted from Grodal1994, 16). In making this remark,

Grodal also points out the problem inherent in giving a definition that

separates emotions from feelings. Feelings and emotions may be discerned

by their intensity with emotions being the stronger of the two. In addition,

feelings are often non-object directed whereas emotions are object directed.(Grodal 1994, 16-17).

2.3.6 My position

My position is close to Grodal’s. As we will find later in this chapter, I

emphasize, in accordance with Lakoff and Johnson, that our human bodies

play a quintessential role in shaping our conceptual system. I do hold the

position that the human mind is embodied. Feelings and emotions are states

of the human body/mind system. This is not contrary to Grodal’sunderstanding rather an emphasis on the bodily basis of cognition, the

conceptual system, feelings and emotions and the role bodily activation have

when taking part in audiovisual media. Cognition and bodily activation are

looped in my understanding of the body/mind system. Grodal’s simulation

theory aims in the same direction. The bodily activation stemming from a

film experience may create simulations of real events in the mind of the

perceiver. Computer games on the other hand afford the game player to act

the events out and to be a part of an interactive system.

The title of this dissertation, Enacting the Point of Being: Computer games,

Interaction and Film Theory, indicates that it will include film theory. Film

theory is a vast field of research incorporating many different approaches to

the medium of film. Point of view and the questions of identity and

identification with diegetic characters are two main topics within film theory

that will be addressed and discussed within the present work. Subjectivity,

identity and identification are not clear-cut or coherent terms as they are

used in film theory, why there is a need to discern how I will use these terms

and how they are used by others.




28

Point of view is another very troublesome term. It is a loosely defined term

within film theory (and theories on media at large) and has many

interpretations associated with it. The next chapter will discuss the point of

view as an important concept within film theory and hopefully bring the

attention to some of the problems I recognize with this concept and someways to resolve these problems.



POINT OF VIEW AND SUBJECTIVITY

29

3 Point of view and subjectivity

The following chapter sets out to explore what I think of as a specific

problem of today’s film theory tradition: the term point of view. At the core of

my discussion in this paragraph is a question on subjectivity, identity and

identification and how these relates to the point of view concept. To

reformulate and reframe: what constitutes subjectivity, identity, and

identification within audiovisual media? In addition, to take the discussions

even a little further: what constitutes being within audiovisual media? I will

begin the discussion recognizing the problems of mixing different definitions

of point of view, which is a very common thing to do. I will then go on to

explore in more detail two different understandings of the concept: a literal

and a metaphorical. Doing so I will discuss how David Bordwell and EdwardBranigan respectively use point of view. In addition, James Gibson’s

ecological approach to visual perception will be discussed since it opens for

an alternative way of understanding the experience of subjectivity, identity,

and identification.

3.1 Which point of view?

What is really a point of view in a film? Is it the ideas put forth by the

filmmaker(s)? Alternatively, is it maybe the use of camera angle to suggest a

character’s gaze, a character’s viewpoint in film? To some film scholars it

may be an idea, an opinion, put forth in a movie. To some, it might be the

way a ” viewer” makes sense out of the narrative. Point of view is a widely

used term that designates all of these.

To clarify my claim that there are many uses of point of view I will now go

into a short discussion on feminist film theory, which I consider a good

example of this. I do not doubt that the feminist cluster of film theories, and

feminist theory at large, have come to some valuable and interesting results

concerning the roles of men and women in movies and in society. Without

feminist theory and feminist critique, I do not believe we would have had

progress in the number of women directors, for instance. Laura Mulvey’s

article Visual Pleasure and Narrative Cinema stirred things up and blew life

into the film theory discussion of the 1970’s (in Screen 1975 and in Mulvey

1989). This article has been frequently used in many contexts and is to be

considered a landmark within feminist film theories.11 As a theoretical

11

See for instance Astrid Söderbergh Widdings article Kvinnoroll och åskådar perspektiv (The role of women and audience perspective my translation of the title) in Häften för Kritiska Studeier 3-4 pp. 9-19.(Stockholm 1994).




30

approach feminist film theory roots within the psychoanalysis of Sigmund

Freud. Freud’s theory on scopophilia has been and still is a very important

influence for feminist film studies.12 However I feel this tradition, among

others, does create some problems when it comes to the questions of point of

view, subjectivity, identity, and identification. I will now make a short, verygeneral description of what feminist film theory is all about and then briefly

go into how point of view is used within this tradition.

Feminist film theory, or rather feminist theory at large, is concerned with

gender issues. Questions on how gender restrict a character’s possibility to

act and behave within a narrative structure are discussed in this tradition,

this cluster of theories that is commonly lumped together and called feminist

film theory. For instance: what do we as parts of an audience assume about

women in particular genres like the femme fatale in film noir? Also on theagenda is the question on how film expresses what is prototypically male

and prototypically female. There are feminist theories on how “the male

gaze” establishes the visual space in film and these connects to questions of

gender identity.13 In feminist theory the gaze is gender-bound and

ideological. To make a very general description the camera is typically

considered to be representing a white heterosexual man’s gaze. This man is

an ideological observer of the events and the characters within the movie.

The male gaze is in these contexts said to be normative of, or even

determining, to use Laura Mulvey’s words, how the film will be visuallypresented. (Mulvey 1989, 19). Women are construed as being objects for this

male gaze and often they are also considered to be willing to be such objects.

Mulvey claims that women in film connote a ” to-be-looked-at-ness” in that

they are displayed within a traditional role of exhibitionism. Although she is

not making a direct reference, ” to-be-looked-at-ness” echoes the affordance

concept of J.J. Gibson, which the next chapter will elaborate on. Women in

film contain the affordance of being looked at. It is the primary reason for

their being within the movie.

12 Freud introduced his ideas on scopophilia in Three Essays on Sexuality (1905). Standard Edition, 7,136-243. Freud claims that, scopophilia turns into a perversion under the following conditions: 1) it isrestricted exclusively to the genitals; 2) it is connected to an overriding disgust; or, 3) it supplants thenormal sexual aim. (p. 157). By perversion Freud means any diversion from a standard mature

heterosexuality.

13 See Kaplan 1983 and Mulvey 1989.




31

Mulvey exemplifies this with voyeurism and the controlling Peeping Toms

” whose only sexual satisfaction can come from watching, in an active

controlling sense, an objectified other” . (Mulvey 1989, 17). 14

To summarize what I recognize as a problem with feminist film theory: Thegaze is established with camera positioning and editing. The gaze is also

established through a long tradition of viewing15. What we have here is in

other words a fusion of two ways of using the concept of point of view.

Point of view is understood as a pro-filmic camera used to record a specific

visual event. The camera represents a point in the diegetic world from which

someone sees. This is a literal understanding of the concept.

Point of view is also understood as a way of interpreting this recorded event

in a narrative. This is a metaphorical way of understanding the concept. The

point of view is not the point of view of any character but is mainly the gaze

of a white heterosexual man.

The optical point of view of the camera is combined/mixed with a subjective

point of view of a diegetic character or of a male invisible observer through

which the visuals are intended to be understood and made meaningful.

Mulvey exemplifies this fusion by referring to the opening sequences of ONLY

A NGELS H AVE WINGS (Hawks 1939) and TO H AVE AND H AVE NOT (Hawks

1944). These movies use a combination of the male gaze of the spectator andall the male protagonists in the film.

Mulvey divides the male gaze into three instances:

the camera (most commonly operated by a man)

the male characters

the audience

14 This also is something that somewhat connects to Gibson. Visually controlled locomotion is a centralissue in his ecological approach to visual perception.

15 This is also something that is noted by David Bordwell in his Narration in

the Fiction Film, 34. I.e. that we can be more or less tuned to perceive

something in a specific way. This in turn goes well along with the idea of

perceptual and cognitive sets as explained by Bugelski and Alampay

disucussed in chapter 2 of the present work. For a detailed discussion on this

see Bugelski, Alampay 1961, 201-11.




32

Using point of view as a quality of the camera is animating the camera,

giving it life like qualities, already from the event of recording. However,

this makes in part sense in this context. The camera is often confused with

the photographer and his or her position in the room to be recorded i.e. there

is presence of a living intelligent agent that is the origin of the imagesdisplayed. Consequently, what we end up with, in Mulvey’s theory, is a

collapsed concept of point of view. The point of view establishes and

expresses not only a sort of being within the diegetic world and puts it under

an ideological designation but it also genderizes it. It is always a man that

the visual originates from. The visual is what a man sees. Not so only in

movies but also in society.

The above discussion is of course a generalization of the cluster of feminist

film theory. However, I do believe this generalization contains some salientfeatures of this cluster and that it illustrates the problem of point of view,

subjectivity, identity and identification somewhat and hence it serves its

purpose. I will use the terms subjectivity, identity, and identification

somewhat differently why we leave this discussion on feminist film theory

now.

In the following, I will be especially concerned with two well-known film

scholars: David Bordwell and Edward Branigan. They have both greatly

contributed to the field of film theory through their work. Bordwell’s Narration in the Fiction Film and Making Meaning and Branigan’s

Narrative Comprehension and Film are, as mentioned in chapter 2 of the

present work, books that have had quite an impact in introducing cognitive

theory to film studies.

I will now explore their studies of point of view. Bordwell and Branigan

stand, to a large extent, for two different traditions or traits within film

theory. Branigan uses a metaphorical interpretation based on the logical

reading of a text. Bordwell opposes this and favors what he calls an ” opticalpoint of view” . That is, Bordwell has a more literal understanding of the

concept.16 I will also continue to make connections to Gibson and the

ecological approach to visual perception since I find that this is a possible

solution to some of the problems that arises. At least, the ecological approach

could be used as the beginning to an alternative model of subjectivity,

identity, and identification. In addition, Gibson’s theory is in part one of the

building blocks that Lakoff and Johnson use in experientialist cognition.

16

I am a bit ambivalent about using the terms metaphorical and literal here. I am so since George Lakoff and Mark Johnson use the term metaphor in a different way and I will use this term in their conception of it in the present work.




33

Gibson stresses the constant interaction animals have with their

environment and the interdependency of the animal and the environment. In

fact, Gibson claims that environment and animal presuppose one another. In

order for there to be an environment there has to be an animal. If not, there

might be a world but not an environment. (Cf. Gibson 1986, 8 and 33).George Lakoff acknowledges this part of Gibson’s theory –interaction with

the environment– to be essential for experientialist cognition. (Lakoff 1987a,

216).

3.2 The optical point of view

Let us now turn our attention to the work of David Bordwell. In Narration in

the Fiction Film, Bordwell makes a good attempt to diversify what he calls

the optical point of view of a character, from the point of view concept of literature and the mimetic tradition within film theory. That is, Bordwell

wants to diversify a literal conception of the term from a metaphorical one.

In cinema, the concept of “Point of View” has usuallybeen loosely employed, especially within the mimetictradition. When critics speak of a character’s Pointof View, they are usually referring to the rangeand/or depth of knowledge, which the narrationsupplies. (Bordwell 1985, 60).

Bordwell is right about the difficulties that arise because of the confusion of

different loosely held definitions. Using point of view in the way he describes

in the above quote makes it a broad concept used to explain what is

knowledgeable for whom at a certain point in a narrative. If a story is told,

narrated, from a certain point of view and point of view is understood to be

the basis for who might know what at a certain time in a story it could make

sense. It is useful and explanatory and really sorts things out. However, it

also implicitly stresses that seeing is knowing by using a visual designation

of knowledge. Bordwell tries to avoid the use of point of view in thismetaphorical way in favor of a literal one based in the fact that something is

seen from a character’s ” optical point of view” :

To avoid blurring these distinctions, I will use theterm “Point of View”, only to refer to the optical orauditory vantage point of a character; thus “Point of View shot” is synonymous with “optically subjectiveshot” (Ibid. 60).

Let us study what is actually the content of what he writes and what theconsequences would be if one employed Bordwell’s concept of Point of View




34

fully to a movie. Unfortunately, this passage of Bordwell’s text provides us

with yet more problems. Point of View according to Bordwell is equal to the

optical or auditory vantage point of a character (Bordwell writes, “only refer

to” which I understand as “equal to”. Being equal to is not the same as

actually being the other but just means that they have a common value).Bordwell’s intention is, I guess, to make a clarifying explanation of how he

prefers to use the concept of point of view but the explanation he provides

complicate things even more. It does so because in an average movie very

few shots are to be taken literally as a character’s point of view (i.e. what

Bordwell calls optical point of view). Very few shots in an average movie are

photographed from the exact point in space from which a character might

see something.

I do understand Bordwell’s positioning against the invisible observer to someextent. “In sum, the invisible observer-model lacks coherence, breadth, and

discrimination” Bordwell writes. (Ibid. 12). Well, maybe. But how shall one

think about all shots in a movie that are not a character’s point of view?

Actually, most shots in movie are not a character’s point of view. This is used

for the more part of a film only in rare cases. I would guess the most well

known attempt is L ADY IN THE L AKE (Montgomery 1947). It was not

especially successful.

Point of view in a movie is a character’s point of view, according to Bordwell.The human being experiencing a movie as an audience, as a spectator etc.

(none of these terms is especially good since they are sense focused) has no

first hand representation in the movie when it comes to Point of View. The

experiencer has not anyone representing only him or her within the movie.

Point of View in Bordwell’s conception, is always a specific character’s optical

Point of View. This is only true if the human being taking part in the movie

is considered as being outside the movie as an external percipient of

something as through a window. On the other hand, if the movie is

considered to be an internal process of the experiencer (commonly called a viewer) and the experiencer has an agent in the movie actively taking part in

the construction of the movie and all non-character shots are thought of as

being the experiencer’s point of view it is not true. In order to solve this one

can instead of point of view use J.J. Gibson’s term point of observation.

Gibson does not limit observation to only designate vision. Hearing and

smell are also important to Gibson since this is what the medium of air

affords humans. To have in his own words:

Any point in the medium is a possible point of observation for any observer who can look, listen or




35

sniff. And these points of observation arecontinuously connected to one another by paths of possible locomotion. Instead of geometrical pointsand lines, then, we have points of observation andlines of locomotion. As the observer moves frompoint to point, the optical information, the acousticinformation, and the chemical information changeaccordingly. Each potential point of observation inthe medium is unique in this respect. The notion of amedium, therefore, is not the same as the concept of space inasmuch as the points in space are notunique but equivalent to one another. (Gibson. 1986,17).

A point of observation is a possibility rather than a factual singular point.

Gibson explicitly defines a point as a position in ecological space and not as

an abstract point in a geometrical space. A point is a place or location where

someone might be and from which an observation could be made. The

ecological space consists of places, locations or positions. (Gibson 1986, 66).17

There is an obvious link in this to the experientialist theory of cognition. As

will be shown in chapter 5 of the present work, much of our daily concepts

are based on spatial relations, on paths, points of departure, destinations

etc. That is to say: daily bodily activity of moving about in the environment

has a lot to do with our conceptual system.

Furthermore:

[…] a point of observation is never stationary, exceptas a limiting case. Observers move about in theenvironment, and observation is typically made froma moving position. (Gibson 1986, 66).

According to Bordwell, point of view can not consist of both sound and image

that are subjective. Bordwell writes: “I will use the term ’Point of View’, only

to refer to the optical or auditory vantage point of a character; thus ’Point of

View shot’ is synonymous with ’optically subjective shot’.” (Bordwell 1985,

60). This would mean, if taken literally, that it is either the optical OR the

auditory vantage point that makes the shot a Point of View shot. Not both at

the same time. In other similar contexts he do write and/or as in the

previous quote: “When critics speak of a characters Point of View, they are

usually referring to the range and/or depth of knowledge, which the

narration supplies.” (Ibid. 60). Is there something at all that can be thought

17 This is contrary to what a point means in Euclidean geometry where a point is that which has no part.




36

of as an auditory vantage point? The vantage point is that point in space

from which something is supremely perceived. It is a subjective spatial

position with a value attached to it by subjective judgement. The concept of

vantage point has been debated and defined over and over during the

centuries, as Bordwell notes making a lengthy and elaborate study of this inhis first chapter on the mimetic theories of narration.

Perspective, in its various guises, is the central andmost fully elaborated concept within the mimetictradition of narration. (Ibid. 7).

To make a brief comment on what Bordwell calls the auditory vantage point:

Film has often been accompanied by sound in one way or an other since its

very beginning. Live music, narrators and sound effects were common in the

early days of film. Hence, sound has been considered as externally added tothe film and not directly generated by the film itself. There have been both

conceptual and physical divisions between the origin of the sound and the

origin of the visual. E.g. the orchestra had its place in the pit, which led to

the placement of loudspeakers to be also in the pit when cinemas were wired

for sound. This is a physical division within the room containing the

cinematic experience. The sound source (the orchestra in the pit) is not in

the same part of the room as the perceived source of the image (the screen).

The physical division turned into a conceptual division in that the engineers

(and the patrons of the cinema) mapped the experience of hearing the

orchestra sound to be valid also for a loud speaker installment. Sound should

conceptually have a place in the room and this place was the orchestra pit.

Rick Altman’s study of sound in the silent era ”The Silence of the silence”

(Altman 1997, 648-718) shows that movies actually were shown without any

sound in the early days of the 20th century. This is contrary to the usual

belief that movies and sound goes together way back to the vaudeville

tradition.

Cinema is of course a kind of visual system. A movie without sound is still

considered to be a movie but a sequence of sound without moving images is

not so as Michel Chion notes. (Chion 1994, 143).18

18 However, Chion mentions one possible exception: Walter Ruttmann's Weekend (1930) which istechnically a film since this sound piece is stored on an optical track and hence must be played through aprojector. (Chion 1992, 143). This is a technical definition of what film is. It is not the experience of it.

Lately, since the mid 1990's we do have a new technical definition of what a film is with the coming of Apple Computers QuickTime technology. In this technology, sound without images as well as imageswith or without sound or even still images can be called a film.




37

Chion also states that the image in itself can not provide all the information

that will provide material for meaning in a sound film. Sound can expand

the meaning of the visual sequences through what he calls ” added value” :

By added value I mean the expressive andinformative value with which a sound enriches agiven image so as to create the definite impression,in the immediate or remembered experience one hasof it, that this, information or expression ”naturally”comes from what is seen, and is already contained inthe image itself. Added value is what gives the(eminently incorrect) impression that sound isunnecessary, that sound merely duplicates ameaning which in reality it brings about, either on

its own or by discrepancies between it and theimage. (Ibid. 5).

Added value is a useful idea but it generates the same problem as most other

concepts of meaning used in cinema studies: it takes the visual as the

standard and primary being. It is based on an assumption that the point of

view and the positioning of a pro-filmic camera is the predominant idea of

the cinema and the other senses being not of the same importance. One may

apply Gibson’s terms of affordances and constraints on the added value

thinking. In Gibson’s ecological approach to the visual perception,

affordances and constraints are thought of as being inherited within objects

and materials.

However: why use a visual terminology to designate auditory phenomena?

Why use point of view, which is a term, referring to the visual system when

dealing with subjective auditory perception? It makes little sense if we are

not to understand point of view in a metaphorical sense, and that is exactly

what Bordwell opposes, as I understand his argument. He is locked inside a

language from which he can not escape.19 Even more questionable is the

construction ” optically subjective” in the former quote. This construction

blurs two very different domain of scientific studies namely those of what is

objective (in this case optics) and what is subjective (in this case vision).

Optics is the objective study of light and has really nothing to do with the

subjective experience of vision. If ” optically” refers to the camera we are

defining the view through the camera which presupposes that someone sees

19 I am aware of the difficulty of escaping from visual metaphor. A large part of the human brain is linkedto the processing of visual information and the rest of the modalities are often represented via visual

metaphors. The roots of a percept need not be visual although it is visually represented. Indeed, visualmetaphors are studied within the experientilalist theory of cognition. (C.f. Lakoff, Johnson 1980, chapter6).




38

something. I.e. it presupposes a subjective experience. Ocular point of view

would maybe be a more accurate term to use if one wants to stick with a

definition that has its reference in the human eye. However, as Gibson

points out, we do not see with our eyes:

The eye is considered to be an instrument of themind, or an organ of the brain. But the truth is thateach eye is positioned in a head that is in turnpositioned on a trunk that is positioned on legs thatmaintain the posture of the trunk, head and eyesrelative to the surface support. Vision is a wholeperceptual system, not a channel of sense. One seesthe environment not with the eyes but with theeyes-in-the-head-on-the-body-resting-on-the-ground.

Vision does not have a seat in the body in the waythat the mind has been thought to be seated in thebrain. The perceptual capacities of the organism donot lie in discrete anatomical parts of the body butlie in systems with nested functions. (Gibson 1986,205. Gibson’s italics). 20

The title to this paragraph is ’The optical point of view’ , which is also an

impossible construction. A view is a subjective experience of vision. I named

the paragraph ’The optical Point of view’ in reference to Bordwell’s use of

this term. Let us now compare Bordwell and his more literal understandingof point of view with the metaphorical interpretation of it, suggested by

Edward Branigan.

3.3 The metaphorical point of view

The aim of Branigan’s theory put forth in his doctoral dissertation Point of

View in the Cinema, is to give an account of the logic and procedures of how

we read a narrative. His perhaps better known Narrative Comprehension

and Film continues these efforts bringing in more cognitive theory to thestudy of film and emphasizing the role the human conceptual system plays

in construing the narrative and recognizing the narrative discourse.

Subjectivity for Branigan is not necessarily the subjective experience of

seeing something from someone’s point (position) in space. It is rather a

matter of how a film presents or portrays its story or character. Branigan

distinguishes the telling of a story from what is told by a story. Subject

matter, as he writes, becomes a result of a process of telling. (Branigan 1984,

20 A similar quote from Gibson is to be found in Joseph D. Anderson's The Reality of Illusion, 44.Anderson's application of Gibson's theory on film is substantial and thorough.




39

1). He finds that such a distinction is a basic requirement for blocking out

any paradoxes that would arise if a statement would “refer to itself

independently of the situation of its utterance and discourse”. (Ibid. 19). A

good insight of Branigan’s is that much of the terminology used in film

studies refers to Man. As he writes “our perceptual abilities dictate whatmay be constructed for us.” (Ibid. 6). This goes well along with Lakoff and

Johnson’s theory of experientialist cognitive theory, as we will se in chapter

5. Branigan notes that the concept of point of view originates from a

conception of man as the center of the world.

Both man and his Point of View must be relocatedas a function of the discourses of man rather than asan essence which surpasses society and is capable of all possible views, and points of views. (Ibid. 20).

According to Branigan, point of view is a feature of a text accessible to a logic

of reading. He tries to demystify the concept by breaking with the idea and

conception of it as being an entity or a feeling possessed by authors or

characters. Point of view is a symbolic process: not a site for consciousness

and speculations of the human psyche. Point of view is a generative capacity

of a text. It is also a part of a reader’s general competence.

However, if it is so, why not replace the point of view concept? It would be

better to replace it rather than redefine it. Of course, one need to have anunderstanding of how the concept has been used but to come to new results

it would be better abounded. In Point of View in the Cinema Branigan

provides the reader with a very good and substantial exploration of the

concept point of view. As an analysis of how the concept has come to be and

how it has been used within film theory, it is excellent. However, it may be

the case that point of view as a concept restricts thinking since it is a sense-

focused term. A quote to exemplify what I mean:

The barrier creates a disparity of awarenessbetween Lisa and Stefan that is exploited by thenarration to merge Lisa’s aural point-of-view withStefan’s hands. (Branigan 1992, 186).

At first, this seems reasonable. However after a while a peculiarity

crystallizes. Aural point-of-view?! The sense of vision is mixed and confused

with audition. It is striking that Branigan chooses to use the rather odd

construction ” aural point-of-view” . I find that this use of language is similar

to Bordwell’s construction ” optically subjective” . I could understand the useof ” auditory point” , ” point of audition” a Gibsonian ” point of observation” or




40

any such a construction but ”aural point-of-view” I can neither understand

nor support the use of. A concept like ” aural point-of-view” relies on the

visual to be the standard being. To be is to have knowledge of the world

through sight. Hearing and listening are subordinate, if at all, an order of

being. Furthermore, since aural point of view relies on vision to be theprimary, standard, being, Branigan deviates from his idea that subjectivity

has little or nothing to do with a pro-filmic camera and an optical point of

view, which is one of the basic ideas of Point of View in the Cinema. An aural

point of view is just another version of the pro-filmic recording device. In this

case, not a camera but some kind of audio recorder is regarded as the sense-

representation of a character within diegesis. In relation to this, consider the

audiovisuality of Stanley Kubrick’s 2001: A SPACE ODYSSEY (1968) which is a

good example of how the profilmic camera position and its point of recording

merges with the point of audition. In the “Jupiter Mission” section of 2001,there is for instance, a sequence that shows how one of the crew members of

the spaceship, Frank Poole, receives a message from his family back on earth

congratulating him on his birthday. The computer onboard the ship, HAL

9000, is also present in this sequence.21 HAL’s voice has a completely

different room ambience than has Frank’s voice. HAL’s voice is omnipresent,

not belonging to the room visually shown while Frank’s voice is

unmistakably a part of the visual room space. (See chapter 6 of the present

work for an elaboration on sound qualities). His voice stems from his visual

position. It has the salient acoustic traces of being where he is located within

the visual field. Frank Poole is clearly located and anchored in the field of

view as well as the auditory field at a distance from the perceiver, whilst

HAL is in the perceptual field much closer to the perceiver. To somewhat

forego chapter 8 (paragraph 8.4.3.1) of the present work, note that Closeness

Is Strength of Effect. This gives an intimacy between HAL and the perceiver

that may be offending and threatening. (See also Lakoff, Johnson 1980, 128

for more on Closeness Is Strength of Effect). HAL is in a location without

significant room acoustic phenomena and this lack may be interpretable asan internal sound of the perceiver. We are located in a position that makes

the sound have (at least almost) internal qualities. There is however,

nothing but the camera angle that suggests that we are looking at Frank

Poole from the point of observation of HAL. The point of observation we are

located at is clearly a coherent construction when it comes to how Poole’s

voice relates to the visual perspective. Other sequences in 2001 however, are

clearly marked as HAL’s point of being (both a visual and auditive point of

observation that is). This is manifest in the use of fish-eye lens and red color

21 HAL is always present on the ship since he is a part of it. He is built into the ship's hardware andsoftware so to speak. However, in some sequences he is silent and not making his presence explicit.




41

that suggests that we se what HAL see. HAL has a red ” eye” that we are

introduced to quite early in the ” Jupiter Mission” sequence of the movie why

it is very easy to perform this mapping. It is also obvious that it is HAL’s

point of being (established through a point of view and point of audition) in

some sequences, especially when HAL takes a look at some drawings andwhen he reads the lips of Frank and Dave in a pod. Nevertheless, in this

sequence, where Frank Poole gets his birthday message, there is no use of

fish-eye lens and hence it is not suggested that it really is HAL’s point of

view we are occupying. The construction is impossible in a real environment

maybe but is very typical for what is possible in a mediated situation. The

sequence is a bit uncanny since we are placed at a location in the ship where

we are looking down at a crewmember and we hear his voice as being tiny

and weak from this location. We also have closeness to HAL suggested by

the lack of room reverberation that makes the location somewhat strange.We hear from one point that is subjective and that matches the view point.

HAL’s voice suggest that we are in close proximity to him and the lack of

room ambience even suggest that we are HAL. However, the visuals does not

suggest this, in this case as it does in other parts of the movie. There is in

other words a tension in the construction of the environment.

Most sequences where HAL is present the sound of his voice has this lack of

room acoustics i.e. a lack of room ambience. There are a few exceptions. In

one sequence, HAL’s voice is transmitted via radio to the pod in which Davetries to rescue his colleague Frank. Hall’s voice then takes on the qualities of

the radio and the room acoustics/ambience of the pod. The image then shows

a distance shot of the mother ship and the pod. This gives that the visuals

have a point of being, external to the ship and pod and the audio is placed

within the pod. There is no trouble for the perceiver to superimpose those

perceptual inputs and create an understanding of the whole audiovisual

configuration. What we have got here, in other words, are mediated multiple

points of observation in both examples. In the second example, Branigan’s

” aural point of view” places us inside the pod and Bordwell’s point of view

places us outside the ship and pod. We are both inside and outside the pod at

the same time. We have what we can call ” character point of being” (HAL is

in the ship and Dave is in the pod, as well as a ’viewer-constructed’ point of

being (we are outside both the ship and the pod) observing the event. We

map our bodily being in front of the movie screen onto the being within the

environment that the screen displays, and at the same time, we have a point

of audition from another location.




42

Maybe one could say that Branigan’s ”aural point-of-view” concept makes

the same mistake as the concept of ”oral literature.” Walter Ong states that

this concept i.e. ”oral literature” defines oral culture in a way that

presupposes that written language is the known and the first to be known

instead of the opposite. (Ong 1982, 24-25). It defines oral culture or oraltradition of ”rapsodien” from a secondary phenomenon namely literacy. He

exemplifies this by giving a definition of a horse to someone who has never

had any knowledge of a horse from the known concept of a car. The horse

then is defined from what it is not and not from what it is. One can not, Ong

writes, do this without a considerable distortion occurring. Not only

Branigan, but many if not most film-, media- and art scholars, fall into the

same tradition of defining being as a result of/from viewing (when using the

concept of Point of View). That is to say a tradition of defining a concept of

great importance from a secondary phenomenon instead of a primary one. Inthis tradition, seeing is considered to be primary and being secondary just

like the car defines a horse. I would suggest that being is the primary

condition to perceiving anything at all. Sight, vision, and view follow from

being, not the other way around. In cinema and other media, it is true that

being might be established through a Point of View but it is not the whole

truth. It is just one way to do it and a heavily conventionalized way as well.

The difficulty of equating optical (perceptual) point of view with the

experience of being that character (feeling the character’s feelings) leadscritics toward attitude, identification, or language as additional conditions

on subjectivity, or as an entirely new attempt to define subjectivity.

Branigan mentions Nick Browne and Seymour Chatman and their writings

on this. Browne argues that point of view is a complex interaction between

the way we view and what we view. Chatman says that there might be two

different points of view at the same time. It could be that we see a character

from another character’s point of view but our sympathy is for the character

we see and not for the one representing the seeing. (Branigan 1984, 7). Using

point of view this way, once again shows how the term splits into two

different fields of understanding. First, we have an optical explanation of

what we see: a character’s optical point of view, that is. And, as we have

already concluded, this is a logical impossibility bringing together optics and

view, i.e. an objective and a subjective approach at the same time. However,

we have also a more general use of the term when Chatman claims that we

may feel sympathetic to the one being seen rather than the one who sees.

These are different levels of “reading the text”. The first is a direct sensory

registration much like: I now see through the eyes of someone in the storyworld. The second is a more complex way of interpreting the data from the




43

visual: I now see someone that I sympathize with through the eyes of some

one else in the story world. The latter approach demands an understanding

of the story and the characters within this story.

As Branigan notes, Andre Bazin links a formal trait (a camera movement) toits effects on the viewer and Jean Mitry finds five kinds of subjectivity in

film:

Point of View shot

Memory images

Purely mental or dream images

Subjectivizing the objective (the camera participates with and mirrors acharacter’s mental state)

and imaginary or fantasy narrative.

Meaning depends upon the relation between present and non-present units

in a formal system. This gives that meaning is a play of differences. If this is

the case, there are no inherent meanings. Branigan gives the example of a

dissolve that could mean a short time lapse in one film but not in another. It

is not the dissolve as unit that gives it meaning, but its relation to other

units within the formal system. No one film is the system but just an

instant, and transformation of a system. (Branigan 1984, 29-30).

Branigan defines narration as the textual activity of telling and receiving

that realizes the narrative. A narrative is a result of a process of logic of

reading. Subjectivity he considers to be a specific level of narration, a specific

level of that process, where the telling is attributed to a character in the

narrative and that we as readers receive the telling as if we were in the

situation of that character. (Branigan 1984, 73).

Telling and/or representing is primarily a creation of space. Telling,

Branigan states, is an act of vision. It is a display of the visual through acts

of vision. (Ibid. 73). Space is visual in Branigan’s way of thinking. How

should one understand this? Is narrative space always visible or made

visible through narration? However, if there is only a description or

representation of what happens, the realization of the space in which the

action literally takes place must by necessity be very vague. One could only

shape an understanding of the action object(s) relation to its own parts or to

other action objects i.e. only the agents and the relations between the agents




44

would be rendered. Passive objects would not be considered, since they would

only display and make clear the field of action i.e. the space of actions that

occur.

The relation between ’frame’ and ’origin’ is of particular importance. Theframing is linked to a specific character and that character’s vision is the

origin of the shot. Branigan notes that this link might be direct or indirect

(as a tactile motor/kinesthetic link may be, as we shall see later!). A point of

view structure gives a first-hand representation of what a character see.

First the character position in space, as Branigan uses the term, is made

clear and then the characters visual experience is represented. The spatial

field of vision is an approximation of what a character may see from this

position within the diegetic world.

In computer games one can utilize the same process. It might also be

realized through the effect a game player’s motor action will have on a

displayed image. If the game environment starts to move when a game

player press a button or otherwise manipulates the control device, it is not

far fetched to draw the conclusion that what is seen is a representation of

what a Game Ego (a character) within the game environment see as well.

However, the image must have some properties of a first person point of view

if we are to identify the environment seen as our (visual) position in the

game. That is, our ocular point of observation equals the ocular point of observation of the Game Ego. Such visual properties that indicates that it is

a first person point of view one is perceiving might be body parts displayed

so as to simulate the way one actually see oneself outside computer games.

There is not only vision and a visual representation but also a body within

whose boundaries the perceiving is contained. Gibson notes that our limbs

protrude into the field of view and this is a perceptual basis for our body

being at a certain location relative to the environment. (Gibson 1986, 208).

To somewhat forego the discussion on experientialist cognition, note that

what we have here is a bodily basis for containment. Containment is animportant concept in this specific theory of cognition.22

As mentioned the link between frame and origin might also be indirect. An

indirect link is a kind of character projection where ways other than just

first person point of view are used to join space and character. When

22 See the chapter 5 on experientialist cognition in the present work. See also George Lakoff',Women, Fire

and Dangerous Things, 267, 271-73, 282-84, 286-288, 290, 300, 354, 362-63, 383-84, 387, 434 and 450.Mark Johnson, The Body in the Mind , 21-23, 34-35 and 39-40. Lakoff, Johnson, Metaphors We Live By

Chapter 12 "How Is Our Conceptual System Grounded", 56-60. Lakoff, Johnson,Philosophy in the Flesh,20, 31-33, 36, 40, 117, 153, 156, 176, 275, 338, 341, 376, 380-82, 544-55 and 574-75.




45

determining subjectivity it is therefore important, writes Branigan, to

examine the logic that links the framing of a space to a certain character.

This logic is conventionalized. (Branigan 1984, 73). These conventions does

not come naturally but must be learned he claims.

Important to note when reading Branigan is that he equates ideology with a

reading convention. It is also important to note how difficult it is to use his

concept of camera. He is does not use his own definition consistently.

I will define the camera not as a real, profilmicobject (which leads to misunderstanding about the viewer’s access to reality) but as a construct of thereader– a reading hypothesis which seeks to makeintelligible the spaces of a film. (Branigan 1984, 53).

But this definition is, as I understand it, abandoned at least for a moment on

page 74 when he claims:

In real space the camera and a character could notoccupy the same point at the same time;nevertheless, in the Point of View shot they doexactly this and without interfering one another.

Branigan might have written they way he does as an example of the

problems that arise when thinking of the camera as a profilmic object. Hedoes mention later on that he defines the camera in a special way as a way of

making an hypothesis. Still, the reason is not explicitly connected to the

quote above why I at least feel a little lost here. It is clear that his definition

of the camera would solve this problem of real space position and that the

point a character is located at could be occupied by the camera and hence

give a first person point of view shot from this location. It is a question of

thinking about the production of the medium content or thinking about

perception of medium content, I would say. To think of a camera one must

know something about film production. To accept a shot as a point of viewone need not to know something about film production but of real world

visual perception and the way it is commonly put forth in film story telling.

The concept of a camera is not necessary for someone taking part in the story

telling. It is only necessary to know how the elements of visual narration are

usually organized. This is I believe what Branigan calls a logic of reading.

Therefore, he could let go of the concept of camera completely and only say

that certain visual elements in a film could be thought of as being a

representation of a character’s vision and by other means be linked to a

character within the story world.




46

“Total Point of View” is another of Branigan’s concepts. If one shows both a

character and what that character see at the same time (i.e. a first person

point of view), through superimposing or split screen for instance, one would

need a hypothesis about space at the same time. One would need to rethink

how space and time are related in the narrative and think of a possibleposition outside the diegetic space, argues Branigan. In the classical film (a

term which is not defined by Branigan by the way) there are few examples of

this. He mentions Abel Gance’s Napoleon but that is also the only film he

has found this in. (Branigan 1984, 74). In some computer games however

split screens are frequently used.

Some computer games, like Duke Nukem 3D, do use split screen and

variable point of view. In spite of that, one can, as a game player, seldom

focus on more than one at the time, this is not really a problem and does notreally force one to think of the shifting environment. This is due to the

tactile motor/kinesthetic link and the act of controlling the Game Ego. The

act of controlling is dominant in this case and is part of the overall game

structure. Is this a total Point of View in Branigan’s sense or is it more like

multiple Point of Views? Or is multiple Point of View the same thing as a

total Point of View? I would say: No it is not. A total Point of View literally

incorporates a character’s view point with the point at which he looks. A

multiple Point of View situation on the other hand would show many

different points of view held by a character at the same time. It collapsesspace and time into a multiple perception of many spaces and not necessarily

the bilateral space of what Branigan calls total Point of View. In Branigan’s

concept lies the idea that a character sees something and that there is a view

from the opposite direction aimed at the character that is the seeing subject,

as I understand it. Dziga Vertov obtains something like this in that famous

shot from Man with a Movie Camera. It would seem to me that a mirror shot

would do the same thing and mirror shot is not uncommon at all. Indeed,

Branigan mentions this when he discusses six elements of narration: origin,

vision, time, frame, object, and mind:

Character narration requires that all the sixelements of narration be referred to character. Forexample, in the subjective flashback, the origin isspecified as a character; vision is character vision;time is the mental time of character; frame is whatis placed before us by the character’s memory; andmind is the character’s state of memory, which is thenominal logic, the coherence, the unity of thatrepresentation. When all six units of representation




47

are referred to character, the unity of thatrepresentation is, exactly, the character as subject;and the telling or representation is called subjective.(Branigan 1984,75).

Time, frame and mind are the variable bases for types of subjectivityaccording to Branigan. (Ibid. 76). Origin, vision, and object are invariant in

all types of subjectivity in classical film. Origin and character must be equal

to give a subjective narration.

A point in space must be established and related tothe space occupied by a character; that is, characteror the perspective position of character will bedesignated as the origin of a production of space for

the viewer. Vision, too, is always related tocharacter–space exists as seen by, or generated by, acharacter, though the activity of seeing may becomemetaphorical, as in memory or dream. Finally, theobject of vision–what a character sees–is irrelevantto the form or type of that seeing. A character maysee the same thing in many ways: in reality, in adream, in a flashback, etc. What we are interestedin is how, under what conditions, a character maysee, not specifically what he sees. (Ibid. 76.

Branigan’s emphasis).

The literal placement of the camera as a pro-filmic object does not decide the

hypothesis about space that we may make in reading the sequence. (Ibid.

124). Point of view is a reading structure and editing could enhance the point

of view and make it ambiguous. However, it is still a structure that is in

play. It is a kind of textual logic that supports Point of View. Branigan

touches on the idea that observers do make hypotheses about the world in a

Gibsionian way. Sadly, he only mentions Gibson in two footnotes in Point of

View in the Cinema. The literal placement of a camera would benefit fromthe idea of point of observation that Gibson puts forth.

A set of inferences […] may substitute for a literalcamera framing and free us from the optical vantagepoint of character while maintaining the subjectivityof space. We see that a space is subjective but do not(directly) see its subjectivity. […] It would be moreaccurate to say that we understand (know, expect,see) that a space is subjective though we do not,

literally, see its subjectiveness (from the place of thecamera). (Branigan 1984,124).




48

Note a connection to what Bordwell has to say about point of view here.

Bordwell uses a sharp definition of point of view as the optical vantage point.

From this Branigan goes on to examine metaphor.

A theory of metaphor is a theory of how, and underwhat conditions, transference occurs. The nature of the transference may be narrative and/ornarrational. […] we may generalize and say thateven when where the framing is not strictly from thecharacter’s point in space, there can be ametaphorical transfer from what is looked at to thecharacter who looks (to the looking). (Ibid. 125).

3.4 Summary

To summarize the discussion of point of view, subjectivity, identity, and

identification and reach an initial conclusion for further elaboration:

Bordwell’s approach using point of view as a character’s visual point of

observation is a camera analogy that is body based. We are the camera and

the camera is a character. We posses and occupy a location equal or close to

the point of location of a character in the film or game. Branigan’s approach

with a metaphorical use of point of view is considerably wider and to tend to

confuse point of view in a more literal conception (i.e. Bordwell’s approach)

with the logic of reading and the overall organizing structures of the

narrative and the human mind’s capacities for understanding and making

sense of the narrative. The human mind is for Branigan, and also for

Bordwell, what actually matters in the construction of the narrative, much

like it was for Münsterberg as shown in chapter 2. The human mind is the

material for narratives, so to speak.

The concept of Point of View is problematic. It is not a coherent concept

within film theory and it gets even more complicated when it comes to the

analysis of computer media and computer games. Point of view in film

theory often narrows the study of film to consist of mainly studies of the

visual. Sometimes point of view is used to designate both the visual and

auditory vantage point of a character within a movie without a clear-cut

distinction between them.

Furthermore, not only two senses are mixed within the point of view concept.

There is a confusion concerning whether point of view is the so called optical

point of view of a character or if the term shall be defined as a way of

reading the text. However even this effort of making a distinction is




49

problematic. To be able to understand whether a shot is the point of view of a

character one needs to have an understanding of the context of the shot

within the film. In turn, this of course involves a way of reading the story, to

keep within the terminology of Branigan.

Point of view is understood in a number of ways. It may be the optical point

from where something is seen as Bordwell suggests and it might be a

narrative device to put forth a story in a certain way as Branigan claims. To

use Edward Branigan’s and David Bordwell’s terms, respectively, it may be

the use of the camera associated with a hypothetical- (Branigan) or an

invisible- (Bordwell) observer who takes part of the events in a story but who

does not intervene or take any active part in those events. That is, there is

an observer, whom is only present through the witnessing of the ongoing

events. Note that the invisible observer is something that Bordwell isopposed to and not something that he advocates. Primarily the observation is

considered to be of the visual order in this tradition and context.

The point of view concept is often assumed to designate a point in space from

which something might be observed. Observed in this context means that

” something is seen”. There seems to be a widespread custom in cinema

studies, film theory, art theory, and theories of what is usually called new

media often to use point of view this way. That is, seeing equals observation

and observation equals being. This in turn means that seeing equals beingand that being is established by seeing. To put in a more logical format:

Seeing = Observation = Being

gives

Seeing = Being

Alternatively as Branigan might set it up in logical format since he stresses

knowledge as an important factor for construing a point of view:

Seeing = Knowledge = Being

gives

Seeing = Being

Being is primarily established and made concrete through the visual

appearance of something. (Cf. Bordwell 1997, 161). To see something

demands someone who sees i.e. to see is a subjective experience and thisexperience anchors the being in itself. This has resulted in a cinematic model




50

in which point of view often is made equal to and the only proper designator

of subjectivity. This model rest to a high degree on tradition within film

theory as well as on some ecological factors. One such ecological factor, is

that vision is a very dominant sense of the human being. However, Gibson

acknowledge other senses than sight to be part of the observation process.Gibson elaborates on what he calls ” points of observation” in his theory on

ecological psychology. He does not limit the observer to be using only vision

while observing something. Hearing and smell are parts of the observation

as well as vision. (Cf. the discussion on Medium in chapter 2 of Gibson 1986,

16-19). The use of point of observation will make the logical formula

different:

Perception = Observation = Being

gives

Perception = Being

To use a concept like Point of Being would take into account not only vision

but all sense modalities available to human beings. This suggested formula

will be subject to revision in later chapters since it does not contain the

whole truth. As the following chapters will show, our motor capacities play a

great role in the manifestation of being. Lakoff’s and Johnson’s theory shows

that our conceptual system has its foundation in the use of the human body(i.e. motor action) and the fact that we live within societies.

The conceptual system does have a lot to do with motor performance. This is

also something that goes well together with Gibson’s theory. He points out

that motor performance in the form of visual kinesthesis is a very important

factor for visual perception. Visual kinesthesis combines body action and

perception, stressing the locomotion of the body as a factor in perceptual

processes.

Consequently, what I suggest is a way of thinking about the issue of

subjectivity, identity, and identification on more of a bodily basis. In

chapters to follow, I will ask questions like how we, as film experiencers or

as game players, perceive and experience the environment the character is a

part of; and how do we incorporate the body of this character? That is: How

are we within film and computer games? How is our being established?




51



A FFORDANCES AND CONSTRAINTS

53

4 Affordances and constraints: Some notes and

elaborations on Gibson’s ecological approach to visual

perception

Remember the eleven-year-old Johnny in the first chapter of this

dissertation, who struggled to adjust his hypothesis about what a specific

computer game was all about? (Chapter 1, 5). He had trouble to make an

adjustment of his mind and to change mode from Fire to Don’t Fire. To

further outline the objectives of this dissertation, the following chapter is

concerned with how technology and human presumptions about computer

gaming poses some affordances and some constraints in relation to sound

and image. There are always things you can do (affordances) or can not do

(constraints) and things you are likely or unlikely to do while manipulatingyour environment and the objects contained within it. Technology is not that

very different. There are always some actions and manipulations that are

more or/and less likely to be performed in the interaction with objects and

technical equipment such as cameras, computers etc. A book, for instance,

has by its design and the context in which the book is accessed, some

affordances and some constraints. It “normally” invites one to open the book,

turn the pages, and read them. If the content of the book is in some way

offending one might rip out the pages, throw the book away or maybe most

commonly just close it and put it aside. The first two paragraphs of the

following chapter will be a closer study of Gibson’s theory on visual

perception and the affordances of the environment. I will then go into a

discussion on how affordances and constraints might be used to understand

the technical aspects of media. In the last part of this chapter, we will get

back to Johnny who found himself in that troublesome situation when we

left him. By then, we will have the basis for analyzing further what was

going on by employing the concepts of affordances and constraints.

4.1 The Ecological environment according to Gibson

In the foregoing chapter on point of view and subjectivity, I touched on

numerous occasions, upon Gibson’s theory. Now is the time to elaborate this

and to clarify a couple of traits within it. First of all it is important to note

that Gibson’s theory is concerned with perception and not cognition. As

Lakoff remarks, parts of Gibson’s theory are essential to experientialist

cognition, especially the fact that Gibson stresses the interaction processes

between animals and their environment. (Cf. Lakoff 1987a, 215-216).




54

In addition, as we will see in the following chapters, this issue is of great

importance for both experientialist cognition and the present work

(especially so for the chapters 7 and 8).

Gibson wanted an alternative to the traditions of mentalism andbehaviorism and developed a theory of ecological psychology. Mentalism was

discarded by Gibson because of its subjectivity, and behaviorism because of

its over-reliance on habituation. To understand Gibson’s term affordance,

one must first have some basic notion of how he defines environment more

specific than that it is something that surrounds an animal. In order to

clarify, I will now provide an outline of an ecological environment according

to Gibson. The main source for the following is part one of Gibson’s The

Ecological Approach to Visual Perception (i.e. 7-44).

Gibson emphasizes the interactional relation between and interdependency

of animals and the environment they inhabit. There can be no environment

without living beings and no living beings without an environment, he

claims. Animals and environment constitutes an inseparable pair. Without

animals, we have what we might call a world or a space. In his theory,

Gibson wants to leave the physical world description, i.e. physical reality

and the descriptions thereof aside and instead be concerned with the

ecological reality. The animal is writes Gibson ” a perceiver of and a behaver

in the environment. But this is not to say that it perceives the world of physics and behaves in space and time of physics.” (Gibson 1986, 8. Gibson’s

italics). To further illustrate the differences between the physical and

ecological approach: the physical world and the ecological environment have

different scales of measure. In the ecological environment, the scale is based

on the size of the animals that inhabit it. That is to say that the scale is a

meaningful scale for the animal. As Gibson writes: ” […] the sizes and masses

of things in the environment are comparable with those of the animals.”

(Gibson 1986, 9).

An environment has three categories of elements: Media, Substances and

Surfaces. Gibson contrasts them to the tradition of classical physics in which

the universe consists of detached bodies in space and that, at least implicitly,

poses that what we perceive are just objects in space. This is very different to

objects contained in an environment. Objects in an environment can be both

attached or detached and for them to afford behavior, they need to be

comparable in size to the animal within the environment. The elements of an

environment have the following properties:




55

1) The Media affords respiration; permits locomotion; can be filled with

illumination to permit vision; allows detection of vibration and detection of

odor; it is homogenous; and has an absolute axis of reference of up and down.

2) The Substances do not freely transmit light or odor and that does notfreely permit the motion of bodies and locomotion of animals. They need not

be homogenous.

3) The Surfaces separate the media from the substances. The surface is the

visual part of a substance. It is on the surface between media and substances

that most actions are carried out. The surfaces of substances give away a lot

of information about that substance’s properties i.e. that substance’s

affordances. The surface has a layout that is perceivable by the animal

observer. The layout in turn has an intrinsic meaning for behavior. This is very unlike abstract, formal, intellectual concepts to be found in

mathematical descriptions of space that reign within physics.

For the present study, these elements are of great interest since computer

games are based on interaction between the game player and the game

environment and the fact that we map our everyday experience onto such

game environments. The medium, in the Gibsonian sense, simulated in a

computer game, often allows the substance’s surfaces to be visible and reveal

the affordances of objects. The surface texture of the objects in the gameenvironments has become important for the game play due to the reliance on

photo-realistic imagery. One can say that the level of abstraction has gone

down at the same time as the level of detail has gone up. To put it in a more

formal way: High resolution = Low abstraction. Computer games like Myst

and its sequel Riven (Brøderbund/Cyan 1997) are based on high and even, in

the case of Riven, very high resolution digitally produced images. The

images look like photographs. Why make them look as photographs when

they could perhaps take a step towards human visual perception instead? Is

the camera analogy so powerful and useful that we use it even when we donot need to? Why must we think about photorealism or photoreality when

handling and interacting with computers? Is it perhaps the case, that a

camera, through its optical way of function, and the photograph, through its

photochemical way of function, emphasize how we perceive the visual field

and the visual world, to use the Gibsonian terminology? May we think about

the camera as a reinforcement of the basic level of perception? Hopefully, the

following paragraphs contain the answers to at least some of these

questions.




56

4.2 The Affordances of the environment

”What is, then, an affordance?” one might ask. Gibson suggests that ” the

affordance of anything is a specific combination of the properties of its

substance and its surfaces taken with reference to an animal” . (Gibson 1977,

67. In essence, this is the same text as chapter 8 in The Ecological Approach

to Visual Perception, 127-143). Furthermore:

The affordances of the environment are what itoffers the animals, what it provides and furnishes,for good or ill. (Gibson 1977, 68).

This basic definition of what affordances are is useful in the analysis of

audiovisual media and narrative studies. One need not adopt and accept all

of Gibson’s ideas and thoughts concerning the term affordance. It is still auseful idea that the environment has some salient features that affords

something to the animals, human or not, residing and living within an

environment. To go a little further into this we may consider the following:

If a substance is fairly rigid instead of fluid; if itssurface is nearly horizontal instead of slanted; if thelatter is relatively flat instead of convex or concave;and if it is particularly extended, that is, largeenough, it affords support. (Ibid. 68).

This kind of substance/surface relation makes the surface stand-on-able and

even walk-on-able. There is a persistence in the substance that the surface

reveals so to speak. The surface can also be sit-on-able: Sit-on-ability

demands the surface to be raised up to the knee of a human. (Ibid. 68).

Rigidity, levelness, flatness, and extension are parts of and form a complex

invariant affordance of walk-on-ability. Vision is the predominant sense that

perceives this information. Furthermore, it is easier to perceive the complex

affordance than the properties in isolation Gibson states. This is because the

complex invariant affordance, for instance walk-on-able, is meaningful whilethe single isolated property is not. (Ibid. 67-68). In this discussion on the

properties making up complex affordances Gibson touches on categorization.

Categorization is essential for all neural beings. (Cf. Lakoff, Johnson 1999,

chapter 3. See also Lakoff, 1987a and 1987b, which are concerned with the

cognitive basis of categorization). An important factor in Gibson’s theory is

his conception of the ambient optical array. The ambient optical array is

what makes it possible to perceive an affordance of an object. There is

always some light, even at night that makes the surface of a persisting

substance potentially visible and tangible. The surface is very important for




57

Gibson. The surface is where most of the action takes place Gibson rightfully

claims. (Gibson 1986, 23). In addition:

A potentially visible surface is one that could belooked at from some place in the medium where ananimal might be. (Gibson 1986, 23. Gibson’s italics).

The use of potentially visible surfaces is extensive in many computer games.

Consider a game like Myst, for instance. It consists of something like 2500

still images and some animated sequences. Not all these images are

absolutely necessary for the game play but they contribute to the immersion

of the player in the game environment. (See also the discussion on space in

computer games and film in chapter 8 of the present work).

The visual system is good at sensing motion and movement. The motorsystem of the eyes (that is, one of the organs making up the visual system)

connects to other movement sensing organs as for example the inner ear.

The eyes might move even when they are shut due to sensing movement in

those inner ear areas. This is due to a reflex called optokinetic nystagmus,

which ensures that when we see a moving scene it keeps stationary on the

retina. When seeing some movement we can also get the sense of moving

ourselves although we are not moving until we see the movement. As the

next paragraph will show, Gibson noted that vision is kinesthetic. (Gibson

1986, 183). We do not see only with our eyes but the with a more complexbody system. (Gibson 1986, 205). We also encountered this idea in the

foregoing chapter in the discussion on the optical point of view. To further

exemplify: if we see a roller coaster ride we might get the impression of a

rapid down slope and because of this impression we move our head slightly

which is then recognized by regions in our inner ear telling us that we are

moving. We move because we have seen movement. Visual movement may

make motion of the body likely to occur. We have here an interesting link to

Lakoff and Johnson’s theory:

Moving objects generally receive a FRONT–BACK orientation so that the front is in the direction of motion (or in the canonical direction of motion, sothat a car backing up retains its front). (Lakoff,Johnson 1980, 42).

As human beings, we are predisposed for movement or more precisely for

locomotion. Locomotion of the human body is primarily a motion in the

direction of our eyes since evolution has shaped our bodies in a way thatencourage such movement. The human body has a very clear and utilized




58

FRONT–BACK orientation. We are primarily, body wise, set up for frontal

action. The main part of the perceptual field is conceptually in FRONT of us

since it is the field of successful interaction. To forego the discussion in

chapter 8, it is easier to move what I call the Game Ego straight a head since

we map our front onto the Game Ego. The game environment if it consists of linear perspective makes us want to move into-forward the direction in

which we look. Strifing is conceptually harder to arrive at. That is moving

the Game Ego from side to side. The environmental layout primary affords

the ” head on” motion. Visually controlled locomotion tends to enhance the

moving straight ahead within computer game environments. Backing up is

also harder than moving in the direction of vision. That is, to move the Game

Ego backwards and not being able to see where one is going is troublesome

but often necessary in games like Unreal, Quake and the like.

Having now sorted out some of Gibson’s terminology and provided some

exemplification of its applicability on audiovisual media experiences, it is

time to make some remarks on how the technical devices as such might have

some affordances and some constraints.

4.3 Technical affordances and constraints of media

My aim in this dissertation is not to provide the reader with a full-fledged

history of photography, film, television, and computer games. There issubstantial work already done covering the history of these media.23

However, I do whish to make some relevant remarks on technical

development, since technology has had important consequences for what has

been possible to put forth in these media and the way media content has

been structured.

From the discussion in chapter 3, we can conclude that a great deal of film

theory has its base in some notion of the camera i.e. a technical device that

affords certain things to be done. That is to say that the camera has certainaffordances in its hardware set up and the conceptualization of it as a

recording device. As chapter 3 showed, it may be how the camera simulate a

23 The history of film has been covered in numerous books. Here is not the place to go into detail aboutthis. I will just mention a few works. David A Cook's History of the Narrative Film is a standard work infilm education as is Kristin Thompson and David Bordwell's Film History: An introduction. The history of photography is also well covered. Geofrey Batchen's work on photography, Burning With Desire is arecent work that questions much of traditional photo history. For a nice introduction to the history of computer games I can recommend the website http://www.emuunlim.com/doteaters/index.htm byWilliam Hunter. Another website of interest is of coursehttp://www.classicgaming.com which has a lot on

the history of computer games and also many links to other sites that might be well worth a visit. J.CHerz's book Joystick Nation has some interesting passages on the history of video games. Also Steven L.Kent's book The First Quarter explore this field.




59

character’s view point, how the camera represents a third person present but

invisible in the diegetic space. It could also be studies of camera movement

or mise en scene. (This last was not discussed in chapter 3 though). What we

have is, in other words, camera thinking, photographic thinking and

thinking influenced from a tool for recording and making representation. Toelaborate the discussion begun in chapter 3, on point of view and the camera

analogy, I will now go into a brief discussion on the camera and how the

camera analogy constitutes a constraint when analyzing computer media.

This of course is to widen the ideas of affordances and constraints somewhat

and go beyond what Gibson meant with these terms. However, I find them

suitable to use also in this broader sense. This in turn connects to the

following chapters 5, and the discussion on conceptual and cognitive sets, 6,

and the discussion on how sound has affordances and constraints, 7, that

elaborates on interaction and interactivity, and 8, where we get a moreelaborate discussion on the Game Ego. Chapter 9 contains a number of

analyses that should be read with the idea of affordances and constraints in

mind.

The camera is the central tool for the recording of visual communication,

claims Torben Grodal. (Grodal 1999a). I agree in part on that. The camera

and its predecessors have been the major and central tools for this recording.

With the coming of digital computers and their use for graphical purposes,

this might change. The image needs not to be photographed even if it maylook, seem and appear like photography. The computer is capable of being a

tool for digital manipulation of photographs. It can also be a tool for

providing what looks like photographed material. An image may have its

material origin not only in a world outside the camera (it maybe the case

that this never has been the issue) but could be produced through the

manipulation of mathematical algorithms within a computer. These

algorithms in turn may be modeled on how photographed objects look. The

mathematical algorithms need not simulate reality but be simulating some

kind of photo-reality. Photo-realism is, I believe, somewhat overrated as the

goal for visual appearance in computers. Yes it may have a certain attraction

value and yes, we may get impressed by the system’s capabilities to display

images that look like photographs. However, I would find it more compelling

to model actual human perception of the environment than make a

representation of another medium so to speak.

Gibson did actually write about movies and photography in the last chapter

of The Ecological Approach to Visual Perception (292-302). He contrasts

photography, what he calls the ” arrested picture” to what he calls the




60

” progressive picture” i.e. what is commonly known as motion pictures. He

does not advocate the use of the term motion pictures. It implies that motion

has been added to the picture, which is not the case, he argues. What we

have got is a system that allows the display of a changing optic array and

not a series of still images he argues. It is the changes in the optical arraythat makes us perceive motion. There are several techniques employed to

achieve this, the stroboscopic technique of the movie camera and projector is

only one of many, he claims. (Gibson 1986, 292). The eye has developed to

register change and transformation. The result of this recognition is that the

progressive picture is the basic form of depiction and stilled images, arrested

images are special forms of the visual world, something, which Gibson

rightfully notes, is contrary to traditional optics. (Ibid. 293). In a computer,

there is no real camera. Computers do not work that way. Images all come

down to models of representation. Since human beings have been using thephotographic camera for more than 150 years the camera has become quite

familiar to a lot of people. And the result of using cameras i.e. photographs of

one kind or another have also become very familiar. We know how

photographs look. We know certain things about and expect certain things

from a photograph. We are aware that a camera has some similarities with

the human eye, we know that time and light are essential for a photograph

for instance. However, as Gibson claims, vision is kinesthetic ” in that it

registers movements of the body just as much as does the muscle-joint-skin

system and the inner ear system” . (Ibid. 183). Furthermore:

Vision picks up both movements of the whole bodyrelative to the ground and movement of a member of the body relative to the whole. Visual kinesthesisgoes along with muscular kinesthesis. […]. Visionobtains information about both the environment andthe self. In fact, all the senses do so when they areconsidered as perceptual systems. (Ibid. 183.Gibson’s italics).

This of course connects to the forgoing paragraph and the optokinetic

nystagmus reflex. It also points to the way we perceive ourselves in the

environment and this is an important factor in how we construct a Game

Ego while playing computer games. The Game Ego will, as mentioned, be

discussed in chapter 8. For now we can conclude, that visual kinesthesis is

one factor in the process of construing a Game Ego.

The graphical computer was not an obvious thing, as we know it today. As

with very many technical innovations, the main purpose with computerswas, in the beginning, military use. To use a computer as we do today, as an




61

increasingly narrative unit, was not something that was in common thought

when the first computers were realized. The computer, as a medium, was not

in common thought at all. Albeit this, Vannevar Bush wrote about a

research device he called ” memex” . (Bush 1945/95). In the article named ’ As

We May Think’, he in many ways drew the outline for an interactablecomputer. (The term interactable is a consequence of taking on Gibson’s

terminolgy and a subjective definition of interaction as will be shown in

chapter 7 of the present work). Bush himself visualized/imagined the memex

to be a mechanical device, but his basic ideas has been a great source of

inspiration for people in the computer developing business such as Nicholas

Negroponte (at MIT) and Steve Jobs (Co-founder of Apple Computer and at

present CEO at this company). “Consider a future device,” says Bush “for

individual use, which is a sort of mechanized private file and library. It

needs a name, and to coin one at random, ” memex’’ will do. A memex is adevice in which an individual stores all his books, records, and

communications, and which is mechanized so that it may be consulted with

exceeding speed and flexibility. It is an enlarged intimate supplement to his

memory.“ (Bush 1945/95)

The above quote, pretty much summons what a modern personal computer

can do for you and what it affords you to do with it. In addition to this, the

modern version of the memex – the personal computer, that is – can store

and play interactive films, games etc. It also shows a connection to Gibson’sidea of affordances and constraints. To think about such a device in this way,

one needs to have an idea of how to manipulate a machine and organize the

material. The idea of affordances and constraints is well suited to apply on

this manipulation and organization process. The machine affords interaction

between itself and the user. It affords the display of images and sound, it

affords storage and retrieval of information etc. Furthermore, the idea of the

memex takes into account the way we think, how we associate and reason. I

would say that it takes mind to be the material of the ” narrative” (the

processes of organization i.e.) to connect it to Münsterberg’s work on film

presented in chapter 2 of the present work.

To take the discussion further let us now turn our attention to computer

game systems. Consider the following as an attempt to introduce what will

be further developed in chapter 7 on interaction, and in chapter 8 on

controllability. The first commercially successful computer games were very

” simple” things like Pong. Pong was developed and introduced by Atari in




62

1972.24 It was not primarily a storytelling game but an interaction based one.

It gave the player an opportunity to actually extend his or her motor actions

into actions on the computer screen. Film has the ability to make the

audience willing to act but at the same time it shuts the audience out from

any interaction result i.e. from any product of interaction. When taking partin a film (or of television) we often find ourselves in a situation in which we

would like to intervene. Children are often both yelling and jumping around

when taking part in film or television, as are grown ups sometimes. This is

true especially concerning sport broadcasts like football, ice hockey, and the

like. Computer games on the other hand, allows the player, not only to

watch, hear and intellectually take part in something, but also affords the

player to play a part in the game. That is, it affords a player to take part in

the affordances and constraints of the game environment in a process of

interaction. The computer has become more and more of a narrativemedium. The computer as a medium affords narration and the acting out of

narratives. The narratives narrated with the help of computers make use of

some of the conventions used in film and television but can not use them all.

Film in itself, as a strip of film, affords splicing segments of film together to

make an edit. A computer can in turn be used to simulate such an act. It can

also be used to simulate the film medium experience on the computer screen.

Since the screen is not as big or has the same proportions as a movie screen,

you must have that in mind while composing the images. The computer

screen as such is often more like a television screen in its proportions but

game makers often use a movie-like area on the screen for the actions to

take place. The range between the screen and the player must also be

considered. How far from the screen is the player? How big is this screen and

how many colors can that screen display? That is to say: What does the

system afford the game player when it comes to the layout of the

environment. This is a basic question that Gibson’s theory helps us to ask.

Every technical aspect has its consequences for the narrative one way or

another. The technical shortcomings, i.e. the technical constraints, of

computer game systems have actually been an advantage. They have

brought computer games into being games of the basic level of categorization

of the objects contained within the game environment.

Basic level categorization is further discussed in chapter 5, and is an

essential part of the experientialist cognitive theory of Lakoff and Johnson.

24

Maybe Pong is in one way of bigger importance than we realize at first. Pong could be thought of as thestart of a new field in the use of computers: Interactive visual game playing. Pong started something backin 1972 that is still in progress and which is today a billion dollar world industry.




63

Low resolution of both images (and sound) has been a basic problem for

computer game designers. Storage capacity for data is one reason for this

and it all really comes down to the system speed and the capabilities of the

system. The first digital computer game, Spacewar (Russel 1961), took 9

kilobytes and was designed to run on a computer system called PDP-1. Thiscomputer was a monster of power back in its day i.e. the early 1960s. You

will have to put this into context though. You have probably close to or more

computing power in your WAP cell-phone, microwave and calculator. To put

this in a computer game context: One can not use certain levels of narration

if the technical device is not set up for it, i.e. if it does not afford it. The game

designers have to make certain decisions based upon the technical

preconditions. Another aspect of the narration is that the computer as a

narrative medium is still in its infancy compared to film and television. We

often tend to forget that someone actually had to come up with the idea of classical editing, montage, fade in, fade out, split editing etc. which we more

or less take for granted. We really do not know how to tell a story with the

help of a computer – yet. This is, I would say comparable to the exploration

of sound in the movies. Sound affords other ways of storytelling, other ways

of presenting the layout of the environment since objects and their

affordances can be made to have an existence with a sound, as chapter 6 will

show.

Cinema can be thought of as a time-based emotional affection andmanipulation with a low degree of audience participation in the sense of

motor interaction and ability to affect what is going on in the film. (See also

chapter 8 of the present work and the discussion on film as interface). The

border between the audience and the media is a rather sharp one. Maybe one

could say that cinema fills some kind of need/urge for being told a story

without the ability to intervene. This is not to be confused with our wish to

intervene in the actions as most of us want to do when we take part in a

movie. What I suggest is that we, in one way or another, find pleasure in the

cinematic situation just because we are kept out of the action within the

diegesis and that we are not to blame for what happens on the screen. Some

one else is in charge and all we can do is sit back and perceive the actions

taking place.

In interactable computer games the question of the diegesis is even more

troublesome since the player can actually make things happen inside the

constructed environment. Film theorist Christian Metz argued that theatre

did not have a diegesis since it was not a firm and fixed construction. The

audience can, and always does, affect what is going on on stage. They might




64

laugh in the wrong places, give long applause and so on. In a computer game

the whole idea is to make things happen one way or another.25 The player

shall learn to take control over the situation at hand. The player is allowed

what is denied the audience in a film situation, namely immediate

interaction. For the time being, a player is allowed to use his fingers, hands,and sometimes feet in this interaction process. Voice command is not well

developed in most games.

A filmmaker and a game designer must ask the questions “What is in the

visual?“ and “What is heard?“ as well as ” What do I want the perceiver to

feel?” We have already had one big change in the cinematic room

construction with the coming of synchronized speech and sound. The inter-

titles more or less disappeared. In silent movies, they had not only the task

of representing speech. They also told the audience the time and place forcertain actions, suggested non-visible rooms with intervening comments and

they could also tell the emotional status of a character. However, the titles

told not everything. Some things were visually shown with a cut-in sequence

to clarify its status. I will now provide two examples on how the use of

sound, a sound affordance, can affect the visual flow of a sequence. First

from the large production of Alfred Hitchcock and then from some LucasArts

computer games like Day of the Tentacle (LucasArts 1993).

Hitchcock made England’s first full-length all-talkie movie, BLACKMAIL, in1929. He was supposed to do only the last reel with sound but decided, of

which the film company BIP was unaware, to do the whole movie as a sound

film. 26 This resulted in two versions of the film; a silent version and a sound

version.27 Hitchcock staged some situations differently in the two due to the

fact that sound could be used as an important part of the environmental

layout. One very good example of this is the following sequence. The main

character, a young woman named Alice White (visually played by Anny

Ondra and dubbed live on the set by English actress Joan Barry due to

Ondra’s Czech accent)28 has earlier in the movie stabbed a man with a knifetrying to avoid being raped by him. She is seated by a table to have a meal. A

female neighbor is talking about the stabbing and Alice is mesmerized by a

knife, which is lying on the table.

25 Cf. the discussion on Christian Metz's use of Diegesis in Stam, Burgoyne and Flitterman-Lewis 1992,38.26

BIP stands for British International Pictures.27The silent version runs 75 minutes and the sound version runs 86.28 http://www.eusa.ed.ac.uk/societies/filmsoc/films/blackmail.html accessed 2001-02-27 11.37




65

The sound version is construed by three shots over a period of twenty-eight

seconds. The sequence starts with a half shot of the female neighbor. Then

the camera pans to the right and stops at a half-shot from a high angle of

Alice seated at a table. Then there is a cut to a medium close up of her which

is held for approximately thirteen seconds before the camera tilts down to aclose up of a loaf of bread and a bread knife. Alice slowly grabs the knife with

a trembling hand. The sequence ends with an overview of the room. During

the pan, the medium close up and the finishing close up of the bread and the

knife, the neighbor’s voice changes. From being perfectly clear, it turns in to

a mumbling with only one word coming through loud and clear. That word is

“knife“. This word is repeated several times during the sequence. A non-

visible person from outside the frame asks Alice to cut some bread. This

question triggers the tilt down to the close up of the bread. Just as she grabs

the knife the word ” knife” is shouted in a high pitched voice and Alice throwsthe knife away. A metallic sound (coming from the knife) is also heard.

The silent version starts with a similar half shot of the female neighbor.

Then there is a cut to a title saying “And as I was saying and always will say

– knives is not right“ (sic). After the title, there is a cut to Alice in a medium

close up from a high angle. Alice sits in the left corner of the screen with her

right side towards the camera looking down at the table. There is a panning

from Alice to the loaf of bread and the knife in which Alice slowly moves her

hand towards the knife. Her hand throws its shadow over the knife and thebread.29 Then a cut to a very short half shot of Alice facing the camera

looking down at the knife. Another cut to a tight close up of the bread and

the knife with Alice’s hand moving towards the knife. Now comes the big

difference. In the silent version Alice is frightened by a doorbell and not by

the word knife. The doorbell is cut in with a very short visual sequence just

before she throws the knife away. The last shot is an overview of the room

similar to the one in the sound version. The silent version is, as the example

shows, built, up by a montage of seven separate shots, including the title,

over a period of approximately twenty-three seconds.

This means that the visual focus can change when going from silent to sound

as Hitchcock did. A new set of visuals combined with a soundtrack makes

the knife present by other means in the sound version. If Hitchcock had used

the soundtrack to the same visual flow as in the silent version it might have

been redundancy in the total information flow and not merely as effective

29

This is very similar to the “Ship of Death“ sequence in F.W Mürnau's “Nosferatu“ (1922) in whichCount Orlac's shadow is cast at the sail. It even more resembles a sequence near the end of the same moviewere the shadow of Orlac's hands are reaching for the heroines body.




66

and affective. With the use of sound, he could visually focus our attention on

Alice’s facial expression instead of the knife. The knife is there anyway by

the voice putting it forth. The rapid repetition of the word “knife“ makes the

visual montage redundant. It shows her emotions and feelings towards the

knife and the present situation. In other, more Gibsonian, words: The layoutof the environment is different in the two versions. Sound affords the

conceptual presence of an object not visible and makes it possible to visually

focus on a character’s reaction to this presence rather than on the visible

presence of the object.

Let us put this into the context of computer games. In a computer game such

as Day of the Tentacle (to be further analyzed in chapter 9), you can choose

whether you like to play the game with spoken language, written language

or with both. You can also choose the loudness of the speech and theloudness of the background music. This is not a unique feature for this game.

It is very common to have these choices. However, how then is the game

organized? Do the game makers change the visual flow or the auditive flow

according to your choice? I have not found one single example of this though

it would be technically possible to do it. Of course, there are economical

problems involved in the process. It is still expensive to produce a computer

game. However, would it not be interesting to have a dynamic game of this

kind? It is not just a stylistic question but also question of how the game

interaction with the player is put into being. How is the story told? Whatpieces of information are delivered in what way? Michel Chion states that

“sound in film is, above all, voco- and verbocentric because human beings in

their habitual behavior are as well.“ (Chion 1990, 6). I will not argue against

this and we will find reason to get back to this specific quote later in the

analysis of Myst in chapter 9. Nevertheless, what about the combination of

written and spoken language? When being stuck in a game like Day of the

Tentacle, Full Throttle (LucasArts 1994), The Secret on Monkey Island

(LucasArts 1990) or the like, it could help to turn the written language on

and not just listen to what is said but read it as well. Written language could

help you solve a problem. This also works the other way around. The way

things are said, i.e. the prosody of language can produce a clue to what is

expected of you as a game player. Yet another aspect of written language is

the graphical form it could give an utterance.

The written language makes the audible visible. Typographical style is

important in all audiovisual media as well as it was in the silent movies.

Computer games often use inter-titles to explain the location of a certain

action. It could also use longer texts to give a background story, e.g. the




67

background story for Descent, or to give information of other sorts, e.g. the

computer in Marathon I-III, which communicates via a computer screen.

To take this yet another step let us imagine a computer game made with 3D

animation software. We could keep the story to one room only in order tokeep the amount of objects to model and render down. One room, one story,

and a limited time make our drama/narrative. Also, imagine that we render

the same actions from different positions. We do not allow free locomotion

but keep the control of how the visuals will display the environment. This is

of course a little time consuming but fully possible. Let us also render some

unique scenes for different visual flows in this game. We could now construct

a game in which the choices made concerning sound and written language

could make up completely new sets of visuals. Of course, we could make new

sounds to the different versions as well. All sounds stemming from anyobject in the room could be linked to the object so that if the object moves,

the sound would move too in the three dimensional space.

4.4 The missing Don’t fire button: Johnny’s problem encountered again

Now it is finally time to get back to the confused Johnny. We now have a set

of terms that might help us understand what is happening in the situation

he is in. The game system exposes Johnny to both affordances andconstraints in its interface towards him. It has a Fire button but not a Don’t

fire button. This is an affordance and a constraint. That is, the system as

such affords Johnny one thing, a Fire button and it does not afford him a

Don’t fire button. Consequently, the design of the game system encourages

Johnny to press the Fire button since the opposite of a Fire button does not

exist in the environment. Johnny has also been pressing this afforded button

for quite a while when we meet him in the quote I presented. He is into it, so

to speak. The idea of not pressing the Fire button seems hard for him to

arrive at. The game system does not suggest that this is a possibility by anyother means than the message on the screen: “We wish to talk”. This

however demands a thorough rethinking of the game’s objectives on

Johnny’s part and he is not able to do so more than write “Die alein scum!”.

The absence of a Don’t fire button demands a shift in the schematic

performance structure Johnny is into. He has to shift from one mode of

operating the game to another and he can not at this moment figure out how

to cope with this new situation. To do that Johnny needs to adopt a new

cognitive set (which are discussed in paragraph 5.5 of the present work).

This short analysis shows some basic things about playing computer games




68

and the situation of computer game playing. It stresses also that every

computer game system has a number of affordances and constraints in its

interface towards the game player. These are manifest in the layout of the

computer game environment and in the system as a whole. If the system can

not show color for instance, this can be used to make a game environmentbased in the affordances of contrast, size, velocity of moving objects etc. and

stress the available qualities instead. In the specific case analyzed, the

computer system did not have something that the was wanted by some

character within the game environment: a Don’t fire button why Johnny

used what was literally at hand i.e. the Fire Button just because it was there

and he had the control of it.

This reveals that there is not only on the sender side of the media that there

are affordances and constraints. The game player might also feel that thereare such when taking part in the game environment. In the case of the

fictional Johnny he became confused and tried to solve the problem by

reading the manual. When it goes that far, that one needs to read the

manual to the game, it has some severe problems in its communication with

the game player. It is a problem not only to solve the puzzles in the game,

but the game also becomes puzzling in itself.

4.5 Summary

The diegetic world in a movie or in a television serial is not for the audience

to manipulate. The game environment of a computer game is the other way

around. It is there for the audience/the player to manipulate. Yet this is only

so within game specific premises. A game like Phantasmagoria (Sierra

1995), has less manipulable objects in each situation than for instance

Mumin (Bullhead 1995). In every media, there are limitations of what can be

done. A computer game is pre-programmed, as well are movies and

television series. Narration is still controlled in many ways even if the

medium is supposedly interactable. All media has technical limitations(constraints) as well as narratological ones. The narratological constraints

are often caused by the technical limitations of the medium. There are of

course also dramaturgical problems involved in the process. How to use a

new, unexplored media is not an obvious thing. We have to invent ways of

storytelling in the media. Today, computer game makers do not make use of

all the possibilities that the medium contains. As mentioned above the choice

between text and sound or text or sound does not change the visual flow as it

could do. Why not construct a game in which such choices make a difference?

My point is that it is the mind that is the material for audiovisual media and




69

audiovisual narration – like Hugo Münsterberg suggested in his work The

Photoplay. This makes it natural to discuss the cognitive competence of a

game player or film perceiver. A game player, as well as a film perceiver

tries to find out what the game or film environment affords the characters

within it. In the case of playing interactable computer games, this is veryimportant. The game player is the manipulator of the environment. The

interaction processes (discussed in chapter 7) are integral parts of developing

the syuzhet (discussed in chapters 5, 8 and 9). The theoretical framework I

suggest in the present work takes the above into account when incorporating

Gibson’s ecological approach to visual perception, as well as the

experientialist cognitive theory of Lakoff and Johnson which is the topic for

the following chapter.




70



E XPERIENTIALSIST THEORY AND THE EMBODIMENT OF MIND

71

5 Experientialist cognitive theory and the embodiment of

mind

This chapter will show why and how experientialist cognitive theory is arelevant and fruitful approach to the study of audiovisual computer games.

It is an elaboration of chapter 2 of the present work, in which cognitive

theory was briefly introduced.

I will first briefly introduce the basic outline of the experientialist theory of

cognition as its key advocates George Lakoff and Mark Johnson explain it.

We have already seen their critique on the objectivist tradition within

cognitive theory in chapter 2. I will continue with an overview of some basic

terms within Lakoff and Johnson’s theory and put them in relation to myapproach.

5.1 Outlines

Experientialist cognition is truly a theory about the human being. It is a

theory concerned with the human conceptual system and its foundations. It

is concerned with the MIND-BODY problem and stresses the role of

imagination as an important factor for how humans make meaning out of

the world. Their claim is that the human conceptual system is embodied.

Hence, the dualism between Mind and Body that has dominated western

philosophy and western thinking for more than two thousand years is

questioned. There is no separation between Mind and Body, Lakoff and

Johnson claim.

Experientialism should definitely not be understood in the empiricist sense

as mere sense impressions that give form to a passive tabula rasa. On the

contrary, we must do nothing of the sort:

We take experience as active functioning as part of anatural and social environment. We take commonhuman experience – given our bodies and innatecapacities and our way of functioning as part of areal world – as motivating what is meaningful.”Motivating” does not mean ”determining”. We arenot claiming that experience strictly determineshuman concepts or modes of reasoning; rather thestructure inherent in our experience makesconceptual understanding possible and constrains –

tightly in many cases – the range of possible andrational structures […]. (Lakoff 1987b, 120).




72

Lakoff and Johnson propose that the concepts we use in understanding the

world are embodied. This is a radical break with the major part of how

philosophy in the western culture has dealt with the MIND-BODY relation.

This is the core of experientialist theory of cognition. Lakoff and Johnson

have gone from more modest suggestions that concepts do show traces of embodiment ( Metaphors We Live By) to the presentation of an elaborated

theory on this bringing in research from neuroscience in order to support

their claim ( Philosophy in the Flesh):

An embodied concept is a neural structure that isactually part of, or makes use of, the sensorimotorsystem of our brains. Much of conceptual inferenceis, therefore, sensorimotor inference.

If concepts are, as we believe, embodied in thisstrong sense, the philosophical consequences areenormous. The locus of reason (conceptual inference)would be the same as the locus of perception andmotor control, which are bodily functions. (Lakoff,Johnson 1999, 20. Their italics).

Embodied concepts are central to experientialist theory of cognition.

Humans make meaning from/of the world by metaphor and metonymy.

Our ordinary conceptual system, in terms of whichwe both think and act, is fundamentallymetaphorical in nature. (Lakoff, Johnson 1980, 3).

Experientialism posits:

1) Basic level concepts. This is a result of findings in theories on

categorization. Categories are not firm and rigid things in which all category

members share the same features. There are prototypical examples of

category members, for instance. This indicates that there is a basic level of

categorization. This function of the human mind will be discussed later in

this chapter.

2) Image schemas (i.e. schematic structures of: containment and containers,

paths, links, PART–WHOLE schemas, force dynamics etc.). These schemas

have a nonfinitary internal structure. Some of these image schematic

structures will be discussed and elaborated upon later in this chapter. For

now we can note that such image schematic structures play a great role in

structuring our concepts. The container schema for instance has its basis in

our bodily containment. To put it in a very simplified way: the surface of our




73

skin is separating us from the rest of the world.30 As Lakoff and Johnson

write:

We are physical beings, bounded by and set off fromthe rest of the world by the surface of our skins, andwe experience the rest of the world as outside us.Each of us is a container with a bounding surfaceand an in-out orientation. (Lakoff, Johnson 1980,29).

We will encounter this quote again in this chapter when elaborating on the

CONTAINER Schema in paragraph 5.4.1.

The central claims for the experientialist theory of cognition are:

– Meaningful conceptual structures arise from twosources:

1) from the structured nature of bodily and socialexperience and

2) from our innate capacity to imaginatively projectfrom certain well structured aspects of bodily andinteractional experience to abstract conceptualstructures.

Rational thought is the application of very generalcognitive processes - focusing, scanning,superimpositioning, figure-ground reversal, schemaetc. - to such structures. (Lakoff 1987b, 121).

According to Lakoff, cognition consists in imaginative projecting rather than

correspondence between an arbitrary symbol and an external object, as the

objectivist theory of cognition posits. Abstract cognitive models are derived

from social and bodily experiences and not by correspondence between

external objects and arbitrary symbols. Thus, human thought is notdisembodied, as the objectivist theory of cognition states, but tightly

connected to the human body. This is the reason I will use this theory for my

study. We have human thoughts because we have human bodies. The relation

between the human body and the external world is the basis, the foundation

for human thought. The symbols used for abstract cognitive models are not

30

There is an obvious link to Gibson's theory on the elements of an environment to be made here. Seechapters 3 and 4 of the present work on Gibson, substance, medium and the surface that separate these twofrom each other.




74

arbitrary but a consequence of humans having human bodies, which have a

certain disposition and a way of functioning.

5.2 Basic-level categorization and basic level primacy

Categorization is essential to all neural beings. One of the findings of

cognitive science, is that we categorize the world constantly. Moreover,

cognitive science gives empirical data on how we do this. Many philosophers

and scholars over more than two millennia have employed the term

"category" in varied contexts. The classical understanding of categories poses

that categories exist as natural kinds. That is to say: categories exist

independently of any mind conceiving them and in a reality external to the

human mind. A category in this sense has clear boundaries and is defined by

its common properties. (Lakoff 1987a, 16).

The above is the way the term "category" was used by Aristotle to denote

predicate types such as substances, quantities, relations, and states to

mention just a few. This postulates that a shift of relation is not a shift in

substance, as shift in quantity is not a shift in state since they fall into

different classes. Substance, claimed Aristotle, is the one thing that has an

existence of its own independent of anything else. All other categories have a

relation to substance. (Nordin 1994, 98). As Lakoff and Johnson note, (and

not only they but also other scholars of different fields of research,) theimpact of Aristotle’s philosophy and way of thinking on western culture is

hard to overestimate. (Lakoff, Johnson 1999, 373).

This dissertation is concerned with computer games. To define the concept of

“game” is not an easy task. A game could take on many forms and ways of

being. Cricket, for instance, is an outdoor field game with what is usually

considered very complex rules (the rules even regulate breaks for lunch, tea

and drinks!).31 Go Fish is a card game with very simple rules. But both are

nevertheless considered to be games. Computer games pose yet newproblems when it comes to definitions. After a long struggle Ludwig

Wittgenstein found that games are best described as something that one

might categorize by family resemblance.32 A family consists of a wide set of

properties. Not all members in a family may have all the common properties

of the family but only some of them. This states that some members may be

more central (they will show more common properties than others).

31

Breaks in Cricket are dealt with in law 16 in the Rules and Regulations for Cricket. These are, at thetime of writing, obtainable at http://laban.vr9.com/ie.html32 Wittgenstein 1953, 66-71.




75

Eleanor Rosch later developed “the theory of prototypes and basic-level

categories”. Lakoff and Johnson note that this theory was a break with

classical theory in which all members of a category must have all the

common properties. (Lakoff, Johnson 1980, 39-56). Games do share common

properties as family resemblance.

Important for the experientialist theory of cognition, my analyses of

computer games and a backbone of the theory that I propose in the present

work are basic level primacy and basic level categorization. Basic level

primacy is, in Lakoff’s words:

The idea that basic-level categories are functionallyand epistemologically primary with respect to thefollowing factors: gestalt perception, image

formation, motor movement, knowledgeorganization ease of cognitive processing (learning,recognition, memory, etc.) and ease of linguisticexpression. (Lakoff 1987a, 13).

It is, I believe, necessary to go into some detail on this basic level

categorization to reach a deeper understanding of basic level primacy and its

implications for the theory I propose. Lakoff and Johnson constantly return

to this term in their work. It is one of the cornerstones of their theory.

Drawing on the work of Brown, Berlin and Rosch, Lakoff and Johnson pointout four conditions also found in the above quote that distinguish the basic

level from superordinate and subordinate categories. (Lakoff, Johnson 1999,

27-28).33

Condition 1: The basic level is the highest level at which a single mental

image can represent the entire category. Their example is that we are able to

form one mental image of a chair but not of furniture (which is then more

general).

Condition 2: The basic level is the highest level at which category members

have similarly perceived overall shapes. Furniture comes in a number of

shapes: the chair does not. The chair has a basic outline that makes it a

chair rather than something else.

33

Chapter 2 of Lakoff's Women, Fire and Dangerous Things contains an elaborate discussion on prototypetheory from Wittgenstein to Rosch. He provides substantial examples on how the basic levelcategorization has been explored and become manifest within experientialist cognition..




76

Condition 3: The basic level is the highest level at which a person uses

similar motor actions for interacting with category members. This is of

course important for the interaction experience of computer games.

Condition 4: It is the highest level at which most of our knowledge isorganized.

That is part of the reason why the games analyzed below ”work” and are

playable. For instance, human beings have the ability to superimpose

different visual perspectives making a whole understandable perceptual

image from these parts. Moreover, we are doing so without any particular

effort just because of this basic-level primacy. However, basic level

categorization is not only a matter of vision. Sound has also basic level

qualities, as we will discuss in chapter 6.

As I mentioned in chapter 2, when commenting on the work of Joseph D.

Anderson, there are connections to be made between Lakoff/Johnson and

Gibson. The experientialist theory of cognition both embraces and

acknowledges some of Gibson’s findings. Indeed, some of them are essential

to their theory, as it meshes with Gibson’s approach in some respects. In

chapter 4, I described how Gibson stresses the importance of the constant

interaction between human beings (or animals at large) with their

environment, i.e. the human being (or animal), is an inseparable part of itsenvironment. There can be no environment whatsoever if there is not an

animal contained within it and which the environment surrounds. The term

"environment" and the term "animal" are interdependent in Gibson’s theory.

This goes well together with the embodiment of mind and the interactional

properties of the world that Lakoff and Johnson’s theory points to and that

the present chapter explores. Gibson’s affordances of the world are close to

the ” world-as-experienced” as suggested by Lakoff and Johnson. (Cf. Lakoff

1987a). However, there is a major difference, Lakoff writes: the Gibsonian

environment is very similar to an objectivist universe in that the subjectiveexperience of the observer is neglected. Lakoff’s argument is that Gibson’s

environment ” is not the kind of world-as-experienced needed to account for

the facts of categorization” . (Lakoff 1987a, 216). There is a problem here,

then, to merge Gibson’s theory with the theory proposed by Lakoff and

Johnson. There is also a problem with how Lakoff quotes Gibson and how he

chooses to understand the essence of what Gibson might have meant.




77

To complicate things more, consider the following quote from Gibson:

The doctrine that says that we must distinguishamong the variables of things before we can learn

their meanings is questionable. Affordances areproperties taken with reference to the observer.They are neither physical nor phenomenal. (Gibson1986, 143).

The implied subjectivity in the reference goes well alongside the

experientialist cognitive theory, although Lakoff chooses to disregard this

aspect. The questioning of the need to distinguish among the variables of

things, does not, however, go along with experientialist cognition.

Categorization is essential, as already stated. However, an affordance is, in

my understanding of the term, very close to basic level categorization. AsGibson writes: ” Ludwig Wittgenstein knew that you cannot specify the

necessary and sufficient features of the class of things to which a name is

given.” (Gibson 1986, 134). I find, as does Joseph D. Anderson, that there

really is a close relation between basic level categorization and that what

Gibson calls affordances. The concept of affordances, apply in fact, as

Anderson writes, directly to the process of basic level categorization.

(Anderson 1996, 50-51). The basic level is, as we have seen above, the

highest level at which a single mental image can represent the whole

category. Note that Lakoff and Johnson are concerned with mental images

though and not visual perception in this first condition. However, in the

second condition they write about ” similarly perceived overall shapes” and

hence couples the categorization process to perception. In the third

condition, the motor actions are taken into account in a way that is very

similar to how Gibson describes the affordances for manipulation and how to

use objects.

5.3

Metaphor

Do you get lost within my argument? Do you grasp what I am saying? Am I

talking over your head?

The three questions above all contain metaphors where a sensorimotor

experience (getting lost, grasping something, something passes over your

head) is used to understand a subjective experience. That is to say that there

is mapping of the sensorimotor domain of mind onto the domain of subjective

experience in those questions. The following paragraph goes into some detail

on metaphor and metaphorical concepts. Metaphor is an immensely




78

important factor in the structuring of concepts and it is a central term used

in Lakoff and Johnson’s theory. In Metaphors We Live By (1980), Lakoff and

Johnson introduce the metaphorical structuring of concepts and the

observation of such structures in language. Metaphor is, however, not only a

question of language, they claim, but rather something that structure ourlife and our thinking. Language reveals the traces of this metaphorical

structuring of mind. Metaphorical structuring of concepts means that

concepts are (partly) understood in terms of other concepts as in the example

above. We are living our lives within metaphorical structures. We act

metaphors out. In their more recent work, Philosophy in the Flesh (1999),

they go into detail on how the mechanisms of metaphorical structuring of

concepts work and the consequences the embodied metaphorical mind theory

has on western philosophy. The Integrated Theory of Primary Metaphor that

they suggest is a composite of four theories that have taken Lakoff’s andJohnson’s claim about metaphor seriously and that have provided

substantial work on this issue. The theories are Christopher Johnson’s

theory of conflation, Joe Grady’s theory of primary metaphor, Srini

Narayanan’s neural theory of metaphor and Mark Turner and Gilles

Fauconnier’s theory of conceptual blending. (Lakoff, Johnson 1999, 46-47).

Let us now examine the main points in these respective theories.

5.3.1.1 Conflation

Christopher Johnson’s theory of conflation provides an insight in how small

children learn. There are two domains of experience: sensorimotor

experiences and subjective (non-sensorimotor) experiences. The young child

does not differentiate between subjective experiences and sensorimotor

experiences when they occur together. This is called conflation. When an

infant is held there are both the subjective experience of affection and a

correlating sensorimotor experience of warmth, Lakoff and Johnson

exemplifies. Conflation is the formation of cross-domain associations. The

period of conflation is later followed by a period of differentiation. However,the associations between the conflated domains will remain. This shows with

language and utterances such as “a warm smile” and a “close friend”. The

two experiential domains have still their correlation intact. (Ibid. 46). The

sensorimotor domain is the source domain and the non-sensorimotor,

subjective experience is the target domain in this model.

5.3.1.2 Primary metaphor

The second part of the Integrated Theory of Primary Metaphor is JoeGrady’s theory of primary metaphor. The conflation theory proposed by




79

Christopher Johnson is the basis for Grady’s work. Metaphorical structuring

of concepts is a two step process. There are primary “atomic” metaphors and

there are complex “molecular” metaphors made up from combinations of the

“atomic” primary metaphors. (Lakoff, Johnson 1999, 45-59). Complex

metaphors are the result of blending primary metaphors.

In the process, long term connections are learnedthat coactivate a number of primary metaphoricalmappings. Each such coactive structure of primarymetaphors constitute a complex metaphoricalmapping. (Ibid. 49).

Primary metaphors are acquired at large and automatically when

functioning within the world.

5.3.1.3 Neural metaphor

The third part is Srini Narayanan’s Neural theory of metaphor. According to

this theory, permanent neural connections are being established across

domains during conflation i.e. metaphor interpretation is grounded in

embodied primitives. In other words, this is a realization of the associations

made during conflation. If a sequence of neural activation, A, has a result in

further neural activation, B and B is connected to a neuronal cluster in the

network that characterizes another conceptual domain, C, then A, throughthe activation of B can also activate cluster C. This gives in Narayanan’s

theory a relation between A and C that is metaphorical since A and C are in

different domains. (Lakoff, Johnson 1999, 47). That is; metaphor is the

neural connection between the source domain and the target domain. (See

also Lakoff, Johnson 1999, 289 and the discussion of The Self).

5.3.1.4 Conceptual blending

The fourth and last part is Fauconnier and Turner’s theory of conceptual

blending that poses that distinct conceptual domains can be coactivated and

connections across the domains can be formed. The resulting conceptual

blend might be either conventional or wholly original. (Lakoff, Johnson 1999,

47).

The result of merging these four theories, Lakoff and Johnson claims, is that

we get an elaborate understanding of the embodiment of metaphor. Primary

metaphors are unconscious, i.e. they reside within the cognitive unconscious

and we are not consciously aware of them. They are established as neural

structures through ordinary functioning in the world. Since many activities




80

and experiences of the human being within the world are (more or less)

universal, a vast amount of the primary conceptual metaphors are as well.

However, it must be stressed that we are talking about a neural learning

process. Metaphorical concepts are not innate to humans. Rather we have an

innate capacity to mentally project from such metaphors. Primarymetaphors are immediate conceptual mapping via neural connection and not

a result of a conscious process of multistage interpretation. (Lakoff, Johnson

1999, 57).

Culture, folk theories, widespread well accepted assumptions and primary

(atomic) metaphors are the foundation for complex (molecular) metaphors.

(Lakoff, Johnson 1999, 60).

In sum: Metaphors are embodied. They have their foundation in the socialand bodily experience of living in the world with a human body as shown in

the foregoing paragraph. To present it in the words of Lakoff and Johnson:

“Metaphor allows for conventional mental imagery from the sensorimotor

domains to be used for domains of subjective experience.” (Lakoff, Johnson

1999, 45). This means metaphor engages two domains of the mind: a source

domain in the sensorimotor activity and a target domain within subjective

experience. The subjective experience is understood and made meaningful

through the mapping of the sensorimotor experience upon the subjective

(non-sensorimotor experience). Being lost in the woods is a kind of sensorimotor experience. That is, you fail to literally find your way to a

specific location. That kind of sensorimotor experience, walking around in

the woods without finding one’s way, is a source domain for the target

domain of the subjective experience of being lost in a metaphorical sense

such as being lost within an argument. We can take it a step further.

Exploring a game environment, Myst for instance, to forego the more

elaborate analysis in paragraph 9.10, is a sensorimotor experience as well as

a subjective experience. We are controlling the field of view when we decide

where to look and where to move. The understanding of the gameenvironment relates to how we may move around within it i.e. a

sensorimotor activity. Feeling lost in Myst is not only feeling lost within the

environment. It is also a question feeling lost in solving the game. In Myst

there are objects that afford manipulation, and even the field of view is a

subject for manipulation. A basic question such explorative games ask the

game player is ” What are you going to do?” . Such a question is as we will see

when discussing the ACTION–LOCATION metaphor also a question of

where one is going to do something.




81

The metaphorical structuring of mind also allows a sensorimotor action,

pressing a button on the joystick or computer keyboard means something

rather different than just pressing a button. In the case of Johnny and the

alien space fleet Johnny could not find the Don’t Fire button simply because

there was not any. There was however a Fire button and he did manipulatethis just because it was there, at hand, and afforded shooting the alien ship

to pieces. In this case pressing a button did mean Fire and not anything else.

Other button presses, for instance left and right arrow keys might mean

turn around in this or that direction, pressing ” W” might mean ” Walk” etc.

5.3.2 Orientational Metaphors/UP-DOWN

Orientational metaphors are the mapping of spatial experience, i.e. the

experience of moving around in the environment, onto more abstractconcepts such as MORE and LESS, HAPPY and SAD. More is UP. Less is

DOWN. Happy is UP. Sad is DOWN. The orientational metaphorical

structures organizes whole systems of concepts with respect to one another.

(Lakoff, Johnson 1980, 14). The basis for the orientational metaphors is our

bodily constitution. The ground for HAPPY is UP is not an arbitrary one as

are not other orientational metaphors. HAPPY is UP and SAD is DOWN are

maybe rather obvious. A person feeling good is probably more upright than a

sad person. This might seem banal. However, the consequences of this

structuring are not banal. We are able to elaborate from these structuresand get to GOOD is UP, BAD is DOWN, ENERGY is UP, LOSS OF

ENERGY is DOWN. Such structures are manifest within spoken natural

language, written language and audiovisual narrative. All these are

common, more or less universal concepts. Let me provide just a few

illuminating examples here. First, think about how characters are placed

within the visual container in a movie, for instance. Strong characters are

often placed in the upper part of the frame. To make a character seem big a

low angle of the camera is often used. This means that the character is

powerful. It connects to ENERGY is UP. UP might also be used to show thata person is GOOD. In Disney’s version of Cinderella (Geronimi, Jackson and

Luske 1950), she does not live down by the stove as she does in the folktale.

Instead, Cinderella lives high in a tower since she is a GOOD character that

we are supposed to feel sympathy and empathy for. The same goes for the

girl Carrie in the movie by the same name (de Palma 1976). She does not live

in the basement. Her room is on the second floor. She is basically GOOD but

circumstances make her take a gruesome revenge. The following paragraph

sets out to explore this ” a little deeper” .




82

5.3.2.1 The ME-FIRST Orientation

Since people typically function in an uprightposition, see and move frontward, spend most of their time performing actions, and view themselves

as being basically good, we have a basis in ourexperience for viewing ourselves as more UP thanDOWN, more FRONT than BACK, more ACTIVEthan PASSIVE, more GOOD than BAD. (Lakoff,Johnson 1980, 132).

This quote from Lakoff and Johnson focuses on the ME-FIRST orientation as

observed by William Cooper and John Robert Ross. The word whose meaning

is NEAREST the prototypical person comes FIRST. A consequence of this is

that certain orders of words (and metaphorically based concepts) are

conceived as being more normal than others. For example, up and down is

more normal than down and up. (We will find reason to get back to this

quote during the analyses later! See especially the analysis of Pac-Man).

Let us study another example on this UP-DOWN, FRONT–BACK

disposition focusing on our bodily experience. This is from the pen of Danish

musician, music scholar, Peter Bastian:

If we would lift a stone 2 meters we will have to use

force. We shall lift it, i.e. do something actively. Toget the stone down again we will just have to let goof it, the rest will gravity take care of and here ourengagement is passive.

Precisely the same is the case when it comes tomusic. We internally conceive of rising intervals asmovement upwards. If we sing one octave upwardswe have to strain and put more pressure on our vocal cords so physiologically speaking high tones

demands more effort than low ones. In addition,even if we just mentally imagine a rising octave, itfeels like that we have to “lift it upwards” It takeseffort to achieve this. Inversely: when the interval isgoing down, we move towards a state of rest.

We immediately associate rising melodic movementwith activity and falling with passivity and weexperience a parallel to gravity in our inner room of timbre. (Bastian 1987, 56. My translation).




83

Let us compare this to the quote from Lakoff and Johnson above. It is a

direct physical bodily experience we have here: the body at work and the

body at rest. This in turn means that we can connect the spatial concepts

“high” and “low” to this activity of bodily effort. We understand a ”high” note

as ”high” because of the fact that to sing a ”high” note we need to put in moreenergy to our vocal cords than to produce a ”low” note. The abstract cognitive

model ”high” is related to the body at work and the body at rest as Bastian’s

example shows. Concerning music there is also a homology between the

abstract cognitive models of high and low and the way we make notations as

images when we write music on paper as sheet music. In our western

notation system, we find the high notes on the upper part of the music-sheet.

What we have is a direct coupling between bodily experience/activity and a

cognitive model, in this case the model of high and low. I admit that there

are other ways of making notation of music such as tabulators and such butthey also are mapped in the human experience of producing sound though in

this case the sound producing is an instrumental activity involving more

than just the human body. In ancient Greece one used a system that was the

inverse of our modern western system i.e. what we call high pitched notes

were found at the bottom and vice versa. This is due to that the Greeks

mapped the notation system not on the human effort of producing a sound

with the human voice but on the handling of the instrument, that generates

the tone. Hence, they used the length of a string and the placement of the

hand and fingers to produce the tone as the contextualizing element of their

notation system. Then it makes perfect sense to have the high notes at the

bottom part and the low ones at the top since a long string produces a tone

that correspond with the notation system. The homology then is not in the

bodily experience of singing but in the bodily interactional process of

instrumental manipulation.

5.3.3 ACTION–LOCATION metaphor

Certain actions can only be performed within particular locations. The ACTION–LOCATION metaphor is a primary metaphor (i.e. an unconscious

metaphor). (See Lakoff, Johnson 1999, 45-59 and the present work,

paragraph 5.3.3). The ACTION–LOCATION metaphor is based on the

connection between Being at a Location and performing certain Actions at

this specific Location. For instance consider the sentence: I start the car. You

may only start a car when you are in the car. That is; the action of starting

the car is conditional and may only be performed when being in the car. This

provides that starting the car means not only what action is performed but

also where this action is performed. Starting the car means being in the car.




84

The action and the location for it are intimately connected in accordance

with the integrated theory of primary metaphor. “An Action Is Being In A

Location”. (Lakoff, Johnson 1999, 204). For the study of audio visual media

and especially the study of computer games this primary metaphorical

concept is highly interesting. As I write in paragraphs 5.4.4.2, 5.4.4.3 and 7.5on Euclidean geometry, we do think in spatial terms. The

ACTION–LOCATION metaphor is an example of this. When there is a

purpose with the action performed, such as in a computer game, the Location

is understood as Destination. To be able to perform certain actions in

computer games one has to move the Game Ego to specific locations within

the game environment. As we will see, the LINK schema and its spatial

relations SOURCE–PATH–GOAL/destinations is a more complex structure

with common features. The ACTION–LOCATION metaphor of course has a

close relation to the Gibsonian Point of Observation and further also to visually controlled locomotion. The concept of location is important to

Gibson. A location is a place at which to be within an environment. It is a

specific point and not a geometrical abstraction as noted earlier. A game like

Day of the Tentacle, that uses both spoken and written language, body

language of characters (e.g. facial expression, gestures etc.) and visual

objects that are possible to manipulate, construct a complex game

environment to handle for the game player. As my analysis of this particular

game shows the ACTION–LOCATION metaphor is of significance and

strongly relied upon through out the game. (See paragraph 9.9) The speech-

acts of the characters contain information that will lead the game player to

perform certain actions with specific objects meant and predisposed for game

player manipulation.

5.4 Image Schemas

According to Lakoff and Johnson, our experience is preconceptually

structured through what they call Image Schemas. Mark Johnson explains

the reason for using this terminology: image schema primarily function asabstract structures of images. Furthermore:

They are gestalt structures, consisting of partsstanding in relations and organized into unifiedwholes, by means of which our experience manifestsdiscernible order. When we seek to comprehend thisorder and to reason about it, such bodily basedschemata play a central role. (Johnson 1990, xix).

A schema is a recurrent pattern, shape, andregularity in, or of, these ongoing ordering activities.




85

These patterns emerge as meaningful structures forus chiefly at the level of our bodily movementsthrough space, our manipulation of objects, and ourperceptual interactions. (Johnson 1990, 29.Johnson’s italics).

In sum, image schemata operate at a level of mentalorganization that falls between abstractpropositional structures, on the one side, andparticular concrete images, on the other. (Johnson1990, 29).

Cognition and conceptualizing are based on image schematic pre-conceptual

structures that organize perceptions into meaningful concepts to the mind.

That is to say, image schemas precede the abstract concepts of mind. They

have a structuring function that brings concepts into being. However, image

schemas are not to be understood as rich mental images but rather as:

[…] structures that organize our mentalrepresentations at a level more general and abstractthan that at which we form particular mentalimages. (Johnson 1990, 23-24).

It is, however, all too easy, because of the use of the word image, to believe

that image schemas have only to do with the visual sense. This is clearly notthe case. As Johnson points out:

It would seem that image schemata transcend anyspecific sense modality, though they involveoperations that are analogous to spatialmanipulation, orientation, and movement. (Johnson1990, 25. My emphasis).

The use of image is a bit distracting and lets us think in terms of visual

qualities. Johnson credits Immanuel Kant with realizing that schemata cannot be identical with an image “since the image or mental picture will

always be of some particular thing, which may not share all the same

features with another thing of the same kind”. (Ibid. 24). Before going into

detail on how such schemas organize our concepts and providing examples of

image schemas, it is important to note that ” schemata” and/or ” schema” in

the sense Lakoff and Johnson use the terms differs in significant respects to

the standard meaning those terms denote within today’s cognitive science.

” Schemata” within cognitive science” , writes Johnson, ” is mainly thought of

as general knowledge structures such as conceptual networks, scripted




86

activities, narrative structures and theoretical frameworks” . (Johnson 1990,

19). Such structures are propositional to their nature.

However, in Lakoff and Johnson’s work, the use of the term schemata is

much closer to the way Immanuel Kant used the term. As mentioned, Kanthad an understanding of schemata as non-propositional structures of

imagination that connects concepts with percepts and as procedures for

constructing images. (Ibid. 19 and 21). Johnson makes clear that Kant’s use

of the term schemata is more constrained than his own because of Kant’s, as

Johnson puts it, “peculiar view of concepts”. (Ibid. 21). Concepts for Kant are

opposed to intuitions as representational types. Receptivity is the source for

intuitions, whereas concepts have their source in the understanding i.e. in

spontaneity, as rules for synthesis. (Kant. A Critique of Pure Reason1st ed. 19

and 2nd

ed. 33). Johnson and Lakoff have, as we have seen above, anotherunderstanding of what a concept is. A concept for them is a neural structure,

which is metaphorical in its nature. Hence, concepts are embodied and

conceptual inference is bodily inference. To put it in their own words:

[metaphors] are a consequence of the nature of ourbrains, our bodies, and the world we inhabit.(Lakoff, Johnson 1999, 59).

Primary metaphors, from a neural perspective, are

neural connections learned by coactivation. (Ibid.57).

This means that image schemata for Lakoff and Johnson is ” a recurrent

pattern, shape, and regularity in, or of, these ongoing ordering activities” .

(Johnson 1990, 29). Moreover and most important in the context of the

present work and the framework I am suggesting, Johnson claims that

image schemata have a kinesthetic character since they are not tied to any

one specific perceptual modality. Referring to a study by Brooks (1968),

Johnson notes that visual images can derive from tactile experience(Johnson 1990, 25). Image schemas are non-propositional and transcend any

specific sense modality.

To make a connection to film theory we may consider that Edward Branigan

refers to schematic processes when processing a narrative as a narrative

schema:

The notion of a schema is basic to much of cognitivepsychology. A schema is an arrangement of

knowledge already possessed by a perceiver that isused to predict and classify new sensory data. The




87

assumption underlying this concept is simply thatpeople’s knowledge is organized. (Branigan 1992,13).

This understanding of schemata is closer to the more general definition of

the term, it seems. It also has a striking resemblance to what is known ascognitive and perceptual sets as we will see later in paragraph 5.5. While

discussing cognitive schemas and other ways of associating data, Branigan

comments on the human memory and the way we make use of different

kinds of associating tools for the manipulation of perceived data. He notes

that transient memory registers sensory information and that short-term

memory sorts and classifies recent information. The latter is limited to

manipulate about five to nine ” chunks” of data at the time. A chunk is

anything stored in memory as a unitary whole (Loftus, Loftus1983, 79). The

fewer the chunks you have to access to accomplish a task the more efficiently

the task can be done (Ibid. 79). Loftus and Loftus exemplify this (as do

others) with a chess experiment (one of the classics!). Random board

configurations and board configurations derived from actual games where

shown to both novices and chess experts. Neither chess experts nor the

novices could reproduce the random configurations. The experts, however,

could reproduce the configurations from actual games. The experts could

perceive the somewhat twenty pieces as a smaller chunk in a configuration

but novice saw twenty pieces. (Ibid. 79. See also Grodal 1994, 62-63 where heexemplifies this with the grouping of dots). The PART–WHOLE and

Configuration structure of image schemas are in play here.

Below follows a brief overview of some of the image schemas that I find

important for an understanding of experientialist cognitive theory as well as

relevant for the present work. Many of these schemas relates to space and

the human conception of spatial relations. It seems like the most salient

schemas really are schemas of spatial relations. There is nothing strange

about this. The human body and its relation to its placement within theenvironment (or world to have it Lakoff and Johnson’s terminology) is a

basic and necessary prerequisite for the survival of the organism.

5.4.1 The CONTAINER Schema

The first schema of interest for the present work and a schema that really do

belong to the first we develop is constituted by the CONTAINER schema.

This schema is a structure for how we understand the world as relations

between our body as a container and objects outside this container. These

objects can also be thought of as containers. (Lakoff 1987,141). That is: the




88

body is a container that could be inside or outside other containers. Consider

this quote from Lakoff and Johnson once again:

We are physical beings, bounded by and set off fromthe rest of the world by the surface of our skins, andwe experience the rest of the world as outside us.Each of us is a container with a bounding surfaceand an in-out orientation. (Lakoff, Johnson 1980,29).

The structural elements of this schema are INTERIOR, BOUNDARY, and

EXTERIOR.

When studying computer games we could have a CONTAINER Schema of

the following properties:

Bodily experience: The game player has a character to control. The game

player is outside (exterior) the computer controlling actions inside (interior)

the computer. The game player’s movements extend into (interior) the

computer via the control device (the boundary). These outside actions then

loops back to the outside via the screen (boundary). The game player could

also be thought of as being inside the game environment since he or she is

able to control and manipulate characters or objects within this world. The

game player is inside represented by a Game Ego i.e. an agent within thegame.

Johnson also exemplifies this container schematic structure:

Our encounter with containment and boundednessis one of the most pervasive features of our bodilyexperience. We are intimately aware of our bodies asthree-dimensional containers into which we putcertain things (food, water, air) and out of which

other things emerge (food and water wastes, air,blood, etc.) From the beginning, we experienceconstant physical containment in our surroundings(those things that envelop us). We move out and inof rooms, clothes, vehicles, and numerous kinds of bounden places. We manipulate objects, placingthem in containers (cups, boxes, cans, bags, etc.) Ineach of these cases there are repeatable spatial andtemporal organizations. In other words, there aretypical schemata for physical containment. (Johnson

1990, 21).




89

It is interesting to note how Johnson’s conception of containment and

awareness of the body and the body as a contained container differs in part

from that of social behaviorist George Herbert Mead. Mead put forth that

the self and body where distinguishable.

The self has a characteristic that it is an object toitself, and that characteristic distinguishes it fromother objects and from the body. The body can bethere and can operate in a very intelligent fashionwithout there being a self involved in theexperience. The self has the characteristic that it isan object to itself, and that characteristicdistinguishes it from other objects and from thebody. It is perfectly true that the eye can see the

foot, but it does not see the body as a whole. Wecannot see our backs; we can feel certain portions of them, if we are agile, but we cannot get anexperience of our whole body. (Mead 1934, 136).34

Like Mead, Lakoff and Johnson are concerned with how we conceive of a

Self. The fact that we, as human beings, commonly and more or less

universally seem to conceive of a person as being split in two, a self and a

body in Mead’s terminology or as we shall soon see, a Subject and a Self in

the terminology of Lakoff and Johnson, is intriguing. At first, it seem to

contradict what experientialist cognitive theory has found out about how the

mind works. The conception of the embodied mind does not fit with the idea

of a person being bifurcated/split into two entities. However, Lakoff and

Johnson find that the metaphorical system for conceptualizing the self does

fit their theory although the metaphors within the system are to some extent

contradictory.

The Self is understood as a container for the Subject. (Lakoff 1996, 103 and

Lakoff, Johnson 1999, 275). Moreover, we have a very clear connection to

Gibson here as well. He also stresses the bodily containment. As Gibson

notes, and that we have already touched upon in chapter 3, and the

discussion on the metaphorical point of view, parts of our bodily container

extend into our field of view making us aware of our bodily container. We

34 I want to make a speculative remark here on the transcript of Mead. Since Mead did not write the abovepassage himself and the quote being a transcript of a lecture it could be that he did not express himself theway the transcript lets us believe. Mead might have meant “It is perfectly true that the I can see the foot,but it does not see the body as a whole.” I and ey e are pronounced the same way and is separated bycontext in most cases. However, in this particular case it could make sense to understand the transcribed

“eye” as “I” instead. The “I” and the self could be understood as equal. This is just a speculation of courseand it is by no means meant to be held for being true to what Mead meant. In other texts actually writtenby Mead himself, this is not suggested. The idea is compelling though, I believe.




90

have a visual anchoring so to speak of our body within the environment. The

field of view is in itself also a container for vision i.e. a visual container.

5.4.2 The LINK Schema

Also the LINK Schema (Lakoff 1987b, 143) is of importance in

understanding computer games. The structural elements of a LINK Schema

are two elements, A and B and a LINK that connects them. Social relations

and interpersonal relationships are as Lakoff says, often understood as

links. Making connections and breaking social ties, are examples of this

(Ibid. 143).

In a computer game, we can have some object or objects that we are in

control of as game players. There is a LINK between game player and the

controllable object. This means that there is a ” social” link to this object.

Often this link is so strong that the object in control ceases to be understood

as an external objects but rather is understood as an integral part of the

player. This is what I call the tactile motor/kinesthetic link that makes up

the basis for my model of a Game Ego. This is elaborated in chapter 8.

5.4.3 The PART–WHOLE Schema

The PART–WHOLE Schema (Lakoff 1987b, 143) has its foundation in the

experience of the human body as a WHOLE with PARTS in aCONFIGURATION. We conceive also other objects as PART–WHOLE

CONFIGURATIONS. The basic level perception (discussed above in

paragraph 5.2) distinguishes the PART–WHOLE structure needed for

functioning in every day physical environment. As Lakoff writes:

[…]the overall perceived PART–WHOLE structureof an object correlates with our motor interactionwith that object and with the functions of the parts(and our knowledge of those functions). (Lakoff 1987a, 50).

The PART–WHOLE Schema is structured as A WHOLE with PARTS in a

CONFIGURATION. This Schema is asymmetric and irreflexive. The

CONFIGURATION is what makes up THE WHOLE. As an example, Lakoff

gives the caste structure of India as being the CONFIGURATION of the

society. If then the PARTS i.e. the castes did not exist, there would be no

society since THE WHOLE can not exist without its PARTS.

The PART–WHOLE Schema has an interesting connection to movement.Think about a steering wheel of a car. You may place your hands on top of




91

the steering wheel and turn left by moving the wheel left. However, if you

place your hands at the bottom of the wheel and want to turn left you need

to move the hands to the right. Actually you move your hands down and left

or up and right in this example. There is a natural mapping of the movement

of your hands, the direction of the car and a PART–WHOLE structure thatpresupposes the complete movement. You move your hand a part of the full

circle. You know how to act just a part of the whole movement.

When playing computer games, we are integral parts of a computer game

system. Without our active participation there would be no game process.

Furthermore, computer game environments are also PART–WHOLE

structures consisting of adjacent surfaces of objects. Some objects might be

manipulable, that is they afford manipulation and some may not. The point I

want to make is that many computer games have as their objective to changethe CONFIGURATION of a PART–WHOLE structure. Such changes are

changes of the environmental layout, of course, to put it in Gibson’s

terminology. This is obvious for games like Tetris (Pazhitnov 1985) where

the objective is to manipulate objects in the environment so as to hinder

them from reaching the top of the game board, so to speak. The configuration

is meant to be fluid and to avoid a static state. Other games also have this

structure. Pac-Man, for instance affords the eating of Ghosts. When all

ghosts are eaten the level is finished and a new level begins. The

configuration has changed and the game must go to the next level. Alternatively, Pac-Man, our Game Ego dies, and the game ends. This is also

a change of the configuration that marks an end point.

5.4.4 The SOURCE–PATH–GOAL Schema

The SOURCE–PATH–GOAL Schema is related to the body in motion and

structures the concepts of movement. (Lakoff 1987b, 144). We have a

starting point, i.e. a SOURCE, a DESTINATION, that is the spatial end

point, and a DIRECTION between the SOURCE and the DESTINATION.There is a PATH connecting the SOURCE with the DESTINATION. The

PATH covers intermediate points on the way. Note that Lakoff chooses to

use two terms for the end point of the SOURCE–PATH–GOAL Schema. The

reason for this is that GOAL is not specifically referring to a concrete spatial

location while DESTINATION is. In a computer game there can be many

SOURCE-PATH-DESTINATION sequences. This is obvious in platform

games where the GOAL is to reach a DESTINATION of a spatial kind for

completing the level. In such a structure GOAL and DESTINATION has a

very close relation to each other. In the study of narration and narrativessuch structures are flourishing, both in the language used for the study and




92

in the material analyzed. Michail Bachtin, for instance, has elaborated on

the time-space relation of novel and drama in his work on the Chronotope

(Bachtin 1991). His analyses of the ancient Greek novel shows that there are

vast areas of space covered and that this is needed for the syuzhet to

organize the fabula. The GOAL of a character in this kind of storytelling, theancient Greek novel, is often achieved by getting some place, that is, by

getting to a DESTINATION.

My analyses in chapter 9 show this in detail. See for instance the analysis of

Pac-Man (9.4), Day of the Tentacle (9.9), and Myst (9.10).

Interesting to note is that any arcade game of the 1970s had a complete

overview of the game environment, that is, a game board that did not change

between levels or while being on a certain level. Examples of this are Pac-Man, Space Invaders and Robotron 2084 (Williams 1980). The frame of the

game environment stayed the same. The objective in such games, is to empty

the game environment of opponents, enemies. The next level will have the

same game environment but the opponents, (the enemies) will have new

motion patterns and speed, as in Pac-Man, as well as being more of them, as

in Robotron 2084.35 This is a change of the CONFIGURATION as discussed

in the foregoing paragraph.

To have a full view of the environment or to have only a selected view aretwo very different things that in itself have certain affordances and

constraints. A full access to what is in the environment will generate other

strategies of game play than a limited access. As Gibson notes, the

environment is seldom open but often cluttered with obstacles that limit

vision. These objects in turn have affordances for god and bad. (Cf. Gibson

1986, 36).

Another aspect of this is that arcade games are coin operated, pay to play,

games and it is part of their design that they lack a narrative end point.

Entrepreneurs engaging in this business want to maximize their profits and

the life span of the game. Therefore, these games are not classical

narratives. They are made to make money. Their overall function is to make

the game player feel that he or she has a possibility to play the game

perfectly even if he or she did not succeed to do this in the last round.

However, another try at it will maybe be the perfect game. To achieve this

the game must not be a narrative in the classical sense with a clear end

point. There must always be the possibility to perform better, to get a higher

35 The first scrolling game and game world is to my knowledge Defender (Williams 1980)




93

score or reach yet another level which is harder to master. (Loftus and

Loftus comments on how games of the late 1970s and early 1980s were

designed to make game players spend more and more money. Loftus, Loftus

1983, 14). Conversely, however for the same reason of maximizing profits but

shorten the life span of a game, adventure games for home PCs do have anend of the narrative often with some possibilities for a sequel. When played

to an end there is little point in playing the game repeatedly if there is no

score kept and there are none or few possibilities to perform better and

better. Games for the home market must have a relatively short life span so

the companies can sell new games. Consequently, they often take on a more

narrative structure with a bigger world to explore. Of course, there are

hardware and software reasons for why the 1970s arcade games had small

worlds as well as the maximizing profit paradigm. Those games where built

on platforms that were hard up on memory for storing the world. Processorspeed was not that high so low resolution and vector graphics were used.

5.4.4.1 The relation between fabula, syuzhet and style in computer games.

The main purpose of this paragraph is to study how the terms of fabula,

syuzhet and style can be used when analyzing computer games and how an

experientialist cognitive approach will provide a wider understanding of

these terms than is common within film theory. (In studies on computer

games available at the time of writing (February 2001) I have not found anystudies of this. John Alexander for instance does not cover this issue in his

Screen Play – Audiovisual Narrative and Viewer Interaction, 1999).

The most compelling definition of fabula, syuzhet and style for film theory,

in recent years, stems from David Bordwell’s book Narration in the Fiction

Film. Bordwell draws on the Russian Formalists (e.g. Jacobson, Shlovsky,

Eikenbaum, Tynianov) when he, Bordwell, defines these terms as follows:

[…]the fabula embodies the action as chronologicalcause-and-effect chain of events occurring within agiven duration and a spatial field. […] Putting thefabula together requires us to construct the story of the ongoing inquiry while at the same time framingand testing hypotheses of about past events.36

(Bordwell 1997, 49).

The syuzhet (usually translated as “plot”) is theactual arrangement and presentation of the fabula

36 The NOW-PAST-FUTURE relations within the construction of the Fabula are interesting and are alsodiscussed in later in this chapter.




94

in the film. […] The syuzhet is a system because itarranges components–the story events and states of affairs–according to specific principles. (Ibid. 50).

Style also constitutes a system in that it too

mobilizes components–particular instantiations of film techniques–according to principles of organization. There are other uses of the term“style” (e.g. to designate recurrent features of structure or texture in a body of films, such as“neorealist style”), but in this context “style” simplynames the film’s systematic use of devices. Style isthus wholly ingredient to the medium. (Ibid. 50).

In short, this is the essence of Bordwell’s definition of the three terms:

Fabula is the story content.

Syuzhet is the organization of the story content.

Style is the utilization of specific techniques available.

Let us now put the two first in relation to experientialist theory of cognition.

Fabula is a container containing the chronological events that are the basis

for the narrative. The fabula container may be filled with a complex causalchain taking place in large or small spaces, locations, and over longer or

shorter duration of time.

The syuzhet is the structuring schema of those events. Syuzhet uses the

spatial metaphorical concepts of mind in its realization as a narrative. The

ACTION–LOCATION metaphor for instance, is a salient major feature in

what is categorized as the ” action genre” .

Let us now go even a step further and put fabula, syuzhet and style into thecontext of analyzing computer games in the framework of experientialist

cognitive theory.

There is some kind of fabula in most computer games. That is, there is some

kind of story in a broad sense that is undertaken and told (constructed by

the game player) when the game is played; a chain of cause-and-effect

events. However, there is one thing that might/will differ from Bordwell’s

definition and that is “occurring within a given duration” (see quote above).

Bordwell defines duration and describes three variables (Ibid. 80).




95

Fabula duration. This is the time a viewer (Bordwell uses the term viewer to

designate the human being taking part in a movie) assumes that the story

action takes.

Syuzhet duration. The dramatized stretches of time that build up the movie.

Screen duration. The time it takes for the film to be screened.

However, one can also define duration, as the time it takes for something to

occur which is not quite the same. We are then talking about the duration of

an event that gives us an amount of time. A clarification might be in order

here. In a film the duration of any event is fixed (more or less) because of the

media’s hardware constitution (one frame follows the other frame on some

reels of film and the frame rate is approximately 24 frames per second). The

duration is in this case defined from the medium per se and has its base in

the medium set up. It is a definition of the medium duration and not the

duration of the content of the media. Bordwell call this screen duration. I

will call this “medium duration”. This broadens the concept to cover more

than the film medium, and when thinking about computers one will also use

a screen. However, the screen in the computer situation has another relation

to the media than the silver screen of the cinema. Computer games are more

flexible than cinema (in terms of how the narrative flow is possible to

constantly rearrange) and are able to use a definition of duration as beingthe time an event takes no matter how long this time is. The latter is a

definition based on an event that takes any amount of time. I will call this

“event duration”. Event duration does to some extent incorporate fabula and

syuzhet duration, and even merges those terms with one another, since

event duration designates the actions that are the basis for the fabula and

the syuzhet. Without action of some kind there is no fabula and hence

nothing to organize as syuzhet. Bordwell also uses the term equivalence,

which is when two durations in Bordwell’s theory are equal. I will think of

equivalence as being a possibility to merge medium duration and eventduration.

What we have here is very similar to the problems that arise when Bordwell

defines point of view (as described above in chapter 3, 33). One needs to

discern the technical concept relating to the medium from the experiential

concept for which the medium provides the raw data.

Computer games may also utilize a time driven approach in which certain

tasks must be performed in a certain time. It is not a medium duration (the

medium is not fixed with respect to time in the sense that severely will affect




96

the presentation of the fabula37) but a question of syuzhet and style more

than fabula since it comes down to how the actions in a fabula are presented.

5.4.4.2 Narrative and linearity.

One way to model a narrative is as an Euclidean space of lines and points.

Euclidean geometry as such is problematic. The language used by Euclid is

opaque and not all terms are defined properly. What follows is a comparison

of some basic elements of Euclidean geometry and some of the image

schemas of the experientialist theory of cognition.

This first postulate of Eucild says that given any two points such as A and B,

there is a line AB which has them as endpoints. Euclid defines a a straight

line as a line which lies evenly with the points on itself. When thinking

about narratives as analyzable “objects” I tend to think of them as a timeline

with the fabula placed on this line through an organizing syuzhet. The

syuzhet is the organization of the fabula over time. However, this is when we

consider a narrative as something that has already occurred and than we are

to analyze. It is a post factum way of thinking about a narrative and not an

experiential one.

A narrative may have to use large spaces for the action to take place as

Bahktin has remarked in his work on the chronotope. The Greek adventure

novel for instance had to take place in huge areas states Bahktin. A

narrative is imaginative, i.e. “vorstellungsbar” as lines and points in space.

There is a starting point, an end and intertwined points of action that have

bearing on the fabula. This mapping of narrative may be transduced to how

we think. If, let us say, we are trying to get to know something specific, we

have a starting point: a point of departure. From this point, we try to find a

specific piece of information that will be the end of the line of search. Along

the way to the goal, we might stumble upon more or less relevant

information that may focus or distract us from our present purpose. (In fact

this text is in itself a kind of distraction of my own thought. I did not set out

to write this text: but I was struck by it and had to write it down which I am

doing right now as I sit here). (This mapping of space on information search

can be found also in Vannevar Bush’s text As We May Think).

The line from departure to the goal is seldom (never?) the pure Euclidean

straight line in a “good” narrative. If it were, the narrative would be

extremely boring. The Syuzhet would be straightforward. (Note that

37 Media duration might in some cases affect computer games. The working speed of the computer and OSmight affect the duration of events. Event duration might be fast or slow due to these factors.




97

straightforward is a concept, which clearly shows the adoption of Euclidean

space conception!).

Narrative is concerned with space and time, to put it in a classical physical

description, and with events and locations to put it a more Gibsonianfashion. Narratives do have Source–Path–Goal structures. Lakoff and

Johnson comment on the linearity and spatial understanding of language

(Lakoff, Johnson 1980, 126).

Since speaking is correlated with time and time ismetaphorically conceptualized in terms of space, it isnatural for us to conceptualize language in terms of space. (Lakoff, Johnson 1980,126).

The Fabula consists of the total sum of the actions that make up the causalevents that are the basis for narration. The syuzhet is the way we are to take

through those events. Syuzhet is the way taken through the narrative, one

might say, to again get back to the conception of space in narrative. It is a

line with detours. Alternatively, a folded line, a zigzag line, or a curved line.

There is not just the point A, the point B and a point C that make up the

narrative. There are lots of points that might be displayed in non-

chronological order. The points are connected with lines of action. Points in

the time-action flow may divide a singular flow into multiple flows.

Now, this is a rather philosophical approach to the problem of linearity.

There are simpler ways of discussing this. When a movie is experienced the

first, and maybe the only time, the linearity of the movie is not always

obvious. Branigan notes that the transformations described by Tzvetan

Todorov are not apparent to the perceiver until it has been interpreted.




98

Todorov’s transformations are as follows:

A state of equilibrium at the outset (A)

A disruption of the equilibrium by some action (B)

A recognition that there has been a disruption (-A)

An attempt to repair the disruption (-B)

A reinstatement of the initial equilibrium (A)

There is a logical form to discover which is covered by the narrative

representation. What is not told (narrated) is the important part that makes

the perceiver smile (the humor resides in the hidden and the realization thatsomething has happened. For the limerick to work, the perceiver must have

some knowledge with him/her from outside of the narrative). A quote from

Edward Branigan could help us get a little closer to the core of the narrative

structuring:

Narrative is a way of experiencing a group of sentences or pictures (or gestures or dancemovements, etc.) which together attribute abeginning, middle and end to something. The

beginning, middle and end are not contained in thediscrete elements, say, the individual sentences of anovel but signified in the overall relationships,established among the totality of the elements orsentences. For example, the first sentence of a novelis not itself ” the beginning” . It acquires that statusin relationship to certain other sentences. ( NarrativeComprehension and Film. 4).

This shows that we must have some kind of over all experience of the

PART–WHOLE structure of the narrative from the actual experience of it or

through our speculation about the events to come. The unique experience of

film as a one-time view, should perhaps be considered as a standard

experience of film, as Torben Grodal suggests. (Grodal 2000). Computer

games consist of environments that afford numerous different ’fabulas’,

although the problems linked with ’optimization’ of path in relation to

affordances and constraints will make some ’fabulas’ more attractive than

others. This includes the way in which players play with a limited

knowledge of the total options. When we start playing a game like Myst, forinstance, we do not know all the options, all the affordances of the game




99

environment and the objects therein. As the player acts with limited

knowledge about the game environment, a ’straight line’ from beginning to

end is not meaningful. Thus a game environment affords a series of ’linear’

narratives, given the level of information that controls the player. If the path

is fuzzy and unclear, this is only visible from an ’Olympian point of view’ i.e.from a point of total observation.

5.4.4.3 Computers and narration

Computers are tools and vehicles for narration. They literally incorporate

narration. Computers have a built in structure that allows a narrative to be

reprogrammed at any instant of the narrative development. A single linear

structure or multilinear structure can merge with other single linear or

multilinear structures at any time. All data stored in a computer is possibleto expand and compress, to put forth or surpress, to show or to hide at any

moment.

When perceiving the world around us the world consists of accessible

possibilities. We make assumptions and speculations on what the world will

be at any moment on the duration of time. We are passing time and time is

passing us.38

To create a coherent world in a computer game one has some choices when

designing the interface. Both what concerns the visuals and the audio. The

sound objects and visual objects need to be in the world as possibilities that

are, at some point in time, accessible to the perceiver.

In a real time 3D game, that is, a game that renders the visual objects and

sound objects in three dimensions (I’d better define three dimensional sound:

location of sound source, shape of the room and the perceiveing agent’s

location etc.) when they are needed for display, the world has a basic

existence in a sequence of binary code. It is programmed and possible to

manipulate to some extent. It is a stored world. A world described in codeddata that takes place (i.e. has spatial existence) on a storage medium of some

kind be it a magnetic hard drive, an optical CD-ROM, a random access

memory. This stored digital world description may take on other, analog,

existential qualities such as displayed images on the computer screen,

sound, tactile and motor sensory qualities.

Narrative has at least two developmental lines going on at the same time.

First, it has some kind of material cause (i.e. the events of a fabula put forth

38 The time metaphor is discussed, explained and exemplified by Lakoff in Lakoff 1993, 14-16.




100

in a medium of some sort). This material cause is somehow brought to mind.

Secondly, we have the perceiver’s understanding of what the media tries to

communicate to the mind. Between medium and mind we have preconscious

image schemas that categorize the perceptions and then go on to handle

them as cognition. Memory is involved in this process of cognition. (Shortterm, intermediate, long term etc).

NOW is related to PAST and FUTURE in something that could be visualized

as a triadic relation schema:

The NOW is always either the passed time (PAST) or the time to be

(FUTURE). We tend to use the concept NOW for duration of time that is

within short term memory and in some fields of academic research and study

one argues for the existence of many contemporary NOW as for instancemusicology and narratology. However, the essence of NOW is that it is a

moving point. A point in the Euclidean sense does not have any spatial

existence such as width, length, or breadth. It is an indivisible location. In

my conception and understanding of NOW and NOWNESS I have to take in

also duration and movement along a line or a surface to fully conceptualize

the NOW and NOWNESS. This is comparable to Gibson’s Point of

Observation. The end of a line is a point, a location.

The NOW–PAST–FUTURE relation is the foundation of environmentalconstruction in visual media. Let me exemplify this with a sequence of film

of the following order that some of my students did as an assignment:

A man sits at a table pouring himself a glass of whisky. Cut to close up of

glass and bottle. Cut to the man hanging with his head looking down at the

table. Audio cut to the sound of a train. Visual cut to the legs of a walking

woman. Sound of a train still on the sound track. The woman walks out of

frame. Cut to a long shot of the man sitting at the table still looking down.

The woman enters the frame from the right seen from behind. She stops.

Sound of the train still on the sound track. Close up from opposite view of

the woman’s face. Cut to the previous angle and the woman starts to move

into the room.

What we have here is a duration of NOW-PAST-FUTURE relations. The

introduction of the train sound might imply that there is a train in such

proximity within the diegetic space that it would be audible from the visual

field. It might also work as a meta-diegitec sound suggesting a relation to

the man who takes place within the visual field. Yet, another possible way of

understanding it that it suggest movement of some sort. All these are




101

speculations on the sounds meaning within the present context. And I would

suggest that the present could be understood as more toward past or more

toward the future. If we like, we can imagine the present as a flux between

past and present – like a ball on a string that shifts position back and forth

between time polarities.

The duration events that take place in this described sequence clearly show

how duration is introduced (the sound of the train precedes the walking

woman and even the cut to the close up of the man. It is also constantly a

part of the audiovisual duration. This gives a perceptual basis for the event

duration. It is reducible to one perceptive chunk of data. When the woman

stops, her stopping and standing still is introduced in one medium duration,

it is confirmed in a second and terminated in a third as she starts to walk

into the room again. So there is also possible to relate the NOW-PAST-FUTURE to INTRODUCTION-CONFIRMATION-TERMINATION. This

suggests that the sequence of duration is essential for the understanding of

them. This is not a new insight. This kind of phenomenon has been known as

the Kuleshov effect. (But the Kuleshov effect has to my knowledge not

earlier been contextualized in this way, i.e. in relation to experientialist

cognitive theory). It also goes to show that INTRODUCTION-

CONFIRMATION-TERMINATION is possible to understand as a

SOURCE–PATH–GOAL structure.

The break of continuity of space in film are due to this process of speculation

and strategic thinking (on strategic thinking see Alexander 1999, chapter 4,

68 f.). There are of course numerous examples of broken continuity of space

in film and I will provide just one. In Edwin S. Porter’s film THE GREAT

TRAIN ROBBERY (1903) there is a classical example.

The train robbers stop the train and disconnect the engine from the rest of

the train. The train moves on the tracks in its frontward direction from the

bottom right corner to the upper left corner of the image. That is, the tracksare diagonal from the lower right corner to the upper left corner. Then

follows a cut to a sequence that features the train and the track in an

opposed diagonal, from the lower left corner to the upper right. One robber

jumps out of the train and is followed by a crowd of passengers. The

passengers are lined up (or are lining up) in the same diagonal as the train

now occupies. There is not much space between the robber and the

passengers. The field of view now contains a lot of swaying movement from

right to left as the passengers line up (and in addition to the swaying

camera, the print is not the best of visual quality anymore. The printanalyzed is stained, blurred and fuzzy). The other two robbers leave the




102

train. One leaves it near the end of the stream of passengers and the last as

the last person to leave the train. Then two things happen in rapid

succession. As the other robbers reenter the field of view from the right, one

of the passengers that is trying to escape is being killed. Since there is a

contemporary action and the field of view is stuffed with informative chunksof data it is easy to miss this entrance of the other robbers. They appear and

are suddenly there. They collect goods and money from the passengers and

leave the field of view by running out to the right. Then a cut and a sequence

follow showing the tracks and engine as in the first shot analyzed. The

robbers enter the field of view from the right, jump on the engine and this

then move to the left. This suggests that our first understanding of the

visual space as coherent is not a valid belief. There is no possibility for the

robbers to enter the field of view from this direction if they have not at some

point crossed the tracks. And this point is not shown. The train is in themiddle sequence shown from an angle 180 degrees opposite from the first

sequence. This shows how we make an assumption on the visual space at

hand from the visual space established earlier. We have to think back in

time and compare what is present to what was in the past.

Let us now leave this discussion and follow it up with some remarks on

cognitive and perceptual sets. The NOW-PAST-FUTURE relations described

above connect to such structuring functions of the human mind.

5.5 Cognitive/perceptual sets

The following paragraph is concerned with what is known as cognitive or

perceptual sets. They are not the same functions as image schemas but do

share some features. Such sets structure clusters of perceptions and focuses

the mind’s attention towards specific salient features of the environment.

The human mind tends to expect more of the same at all time. That is: the

human mind seems to be self-organizing concepts into structures where

more of the same is an important factor. (See for instance de Bono 1997. 77ff and Lakoff, Johnson 1999, 43). A perceptual set is a predisposition to

perceive something in relation to a prior experience of perception. Culture

can play a great role in how an individual interprets data and so can also

situational factors. The former, i.e. culture are long-term conditions and the

latter, the situation, are short-term conditions. If we expose someone to a

succession of sensory input, like a series of sounds or a series of images,

moving or still, with a common theme, the theme as such is likely to be

constructed by the perceiver even if the input at some point in the chain is

ambiguous. For a more detailed study, and a classic example, I suggest




103

Bugelski and Alampay’s The role of frequency in developing perceptual sets

(1961 201-11) although they make the reservation that a more proper term

for their findings might be “cognitive set”. They want to reserve the term

perceptual set for the kind of predisposition ”where a subject is looking for or

expecting to see a specific stimulus, a friends face in a crowd, for example, ora particular colour of book cover on a shelf” (Ibid. 206). As human beings we

are able to give ourselves self-instructions. Edward de Bono exemplifies this

in his book I am right You are wrong (1997) in the chapter on context. One of

his examples is that if we decide to look for people wearing red we will

suddenly be aware of many more people dressed in red. If we change to

yellow, we will notice more yellow etc.

The idea of cognitive and perceptual sets are relevant for the present work

since it do explain something about how the human mind works whenhandling data: how we make concepts from data, how language is a factor in

structuring association and how we relate the present to the past and to the

future. It is a factor in how we receive sound and images in a game. What we

suppose to find in the game environment is very much dependent upon the

cognitive/perceptual sets the game evokes. Furthermore, such sets are not

easy to break once established. If one lets some time pass between

establishing the set and the introduction of another set it seems easier to

create a new understanding. The set is not so much broken as it seems to be

replaced by another set according to Bugelski and Alampay. (Bugelski, Alampay 1961, 209).

Let me at this point, briefly exemplify how an understanding of this quality

of the human mind is of importance in the use of sound and images. If we are

using a car door slamming sound in a movie sequence simultaneously

showing car doors being slammed, we are likely to superimpose the image on

the sound or vice versa. We think we hear what we hear because of what we

see. We have a general basic experience of car door slamming which is

mapped upon the sequence. We also have the experience of the nature beingin synch when objects that produce sound are not to far away. As filmmaker

Peter Kubelka has noticed, cinema need not be in synch. “CINEMA IS

NEVER IN SYNCH!” he claims. “It looks like the natural synch world but it

is not the picture making the sound. We only believe it is reality. Sound and

image are completely different, connected by our body.” (Kubelka 1998). Let

us elaborate this made-up example. Say that we expose someone to a series

of car door closing sounds and then end the chain with a sound that shares

some of the properties that are found in the car door closing sound but is

actually something else. In this case, it would be likely that the person




104

exposed to the sounds would judge all sounds to be within the same thematic

structure. That is, that all sounds are in the same conceptual domain and

having the same causal source.

In addition to this example, we can make a comparison to Lakoff andJohnson. “Brains”, they write, “tend to optimize on what they already have,

to add only what is necessary”. (Lakoff, Johnson 1999, 43). We compare the

known to mind, our experience, with the perception of the audiovisual

sequence, not necessarily on a conscious level but on the level of the

cognitive unconscious. (Ibid. 10-11). Michel Chion also makes a similar

remark about the conscious and unconscious perception of the audio-visual.

The ear and the mind are inseparable when it comes to listening as are the

eye’s looking and the mind’s seeing he claims. The conscious and active

perception is only a part of a much wider perceptual field. Or to have it in hiswords:

The consequence for film is that sound, much morethan the image, can become an insidious means of affective and semantic manipulation. On one hand,sound works on us directly, physiologically(breathing noises in a film can directly affect yourown respiration). On the other, sound has aninfluence on perception: through the phenomenon of

added value, it interprets the meaning of the image,and makes us see differently. And so we see thatsound is not at all invested and localized in the sameway as the image. (Chion 1990, 34).

Chion even coins the term "synchresis" (from synchronicity and synthesis) to

denote this and refers also to what he calls ” added value”. We will find

reason to get back to Chion and his terminology later.

Cognitive/perceptual sets are structuring factors of the human mind. If there

is amiguity in a chain of percepts the foregoing perceptions will play a role inhow we interpret new sensory data. This implies that we maybe should take

another path than the one of making objects in computer games look or

sound as realistic as possible, as realistic objects might be hard to identify.

They might be too complex in their shape and/or texture for instance. There

are studies on how human visual perception works considering what we are

aware of seeing, the cultural dependency of seeing (i.e. the cultural

dependency of the interpretation of what we are seeing) and how fast we can

process visual data and recognize objects and their properties. These studies

tend to show that some features of objects are more rapidly recognized than




105

others. E.g. the color of an object might be more rapidly recognized than its

shape if it moves fast.

What is clear form such studies is that an arousal effect of a large image

happens irrespective of the content. (See Detenber, Reeves 1996). An imageseems first to be categorized from its size. It is not the content of the image

that creates arousal when viewed quickly but the space it covers. The image

is categorized as a large object. Size is predominant. An image is conceived

and made meaningful in accordance with its size. A large image makes the

perceiver smaller. The image is first and foremost an object and is put in

relation to the perceiving container, the Self. It is only when this size-

relation (and also the distance to the object one might speculate) has been

judged (is this object a threat of not?) that the Subject of the Self is able to

make a judgement on the content of the image.

If we reduce the amount of data chunks, that is, if we reduce the resolution

of the image for instance and thereby reduce the level of visual detail we

may reach the basic level of perception and categorization and have a

stronger sensory immersion experience. This is what is going on in games

like Unreal Tournament and Quake III Arena. The edges of the field of view

get blurred in order to suggest proximity of objects and also to suggest

velocity of the movement performed. It is has very much in common with

how rapid movement is treated within traditional cartoon animationtechniques. The animator adds motion blur to the animated object to obtain

a more rapid movement and a better simulation process of actual moving

objects. Motion blur is a natural regularity, which the human being

perceives and has an experiential basis of. We have learned that things that

move fast result in a blurred, unsharp visual experience. In addition, we

have learned that by moving oneself at high velocity the world gets blurred.

It is nothing strange about this. The visual system and the proprioceptive

systems in the body have a basic experience of rapid movement that is

triggered by the motion blur of objects in the field of view. And we are alsoable to make approximations about the relative velocity by which objects

move or by which we are moving. To exemplify the use of motion blur in

animated movies one could consider The Lost World: Jurassic Park II

(Spielberg 1997).

The visual representation of rapid motion in cartoons is most often made

from simplification of objects and textures and the so-called stretch and

squeeze phenomenon. However, one can not simplify the objects too much

because there are some evidence for gender differences (but those might alsobe culturally bound) concerning how we interpret simple images. (Coren,




106

Porac and Ward 1978, 413). What a man might interpret as a centipede a

woman might interpret as a comb. This means that over simplification of

images could be/is a problem. We ought to adapt ideas from abstract

painting, like impressionism in which one emphasizes the sensory response

to input and not the exact recreating of nature as such. If we are to make asimulation of rapid locomotion we must not make objects too detailed but go

the other way around and make them appear as the would if we moved at a

certain velocity. Through perceptual sets of representation, we are able to

simulate impressions and thereby generate sensory immersion. The human

mind is adaptable to successive information creating ” the supposed next step

in the chain” . We expect “more of the same” when taking part in the world

and of course also when we take part in a narrative and/or when performing

audio-visual interaction experiences. We associate the ongoing events with

similar events known to mind in a state of strategic thinking. This is whyshock is so effective. When the causal logic of the next step in the chain is

broken and not fulfilled we might experience shock. The initial and manifest

hypothesis is necessary to reconstruct. Association is the act of merging the

previously known with the perceptual input. It is not necessarily a conscious

act but a function of the cognitive unconscious. Bordwell has written a

passage that fits well into this context:

Generally, the spectator comes to the film already

tuned, prepared to focus energies toward storyconstruction and to apply sets of schemata derivedfrom context and earlier experience. This efforttoward meaning involves an effort toward unity.(Bordwell 1997, 34).

Cognitive/perceptual sets are also related to affordances of the environment.

The affordances of the environment can, so to speak, tune us into a specific

way of understanding the environment. Walk-able surfaces encourage

walking for instance. In addition, if mind is self-organizing and expects more

of the same we will continue to execute walking until something makes us

change our mind – literally, change the organizing structure of our mind at

work that is. We will otherwise risk being locked into a specific pattern of

visually controlled locomotion, for instance, in this case walking. This

happens in numerous games and is a way for game designers to prolong the

game duration at large or a specific event duration. Spyro the Dragon

(Insomniac Games 1998) is one example of a game in which this is put into

use at several levels of the game.




107

Myst is another game with a similar salient structure of the field of view as

is parts of Day of the Tentacle. Even Pac-Man show traces of this. Let us

study two examples from Spyro the Dragon.

Spyro the Dragon is a game in which we have direct motor control of a GameEgo, Spyro the Dragon. We do not have a visual representation of the game

environment from the eyes of the Game Ego (what could be called a first

person point of view) but we see the Game Ego’s body from the outside of it.

Our field of view is not that of the Game Ego but relative to it.

In what is called Magic Crafters (a ” world” within the game) and the specific

level of Wizard Peaks we find our Game Ego, Spyro the Dragon, in a

situation in which a spatial locomotive structure is established through

visually controlled locomotion and the affordances and constraints of theenvironmental layout. We enter a cave through an opening and this cave

turns out to be a system of caves with connecting openings. As Gibson

rightly points out, doors are openings in the surface that are walk-through-

able and that affords the animal to go through. To present it in Gibson’s own

words:

An open environment affords locomotion in anydirection over the ground, whereas a cluttered

environment affords locomotion only at openings.[…] The general capacity to go through an openingwithout colliding with the edges […] is acharacteristic of all visually controlled locomotion. A path affords pedestrian locomotion from one place toanother, between the terrain features that preventlocomotion. The preventers of locomotion consist of obstacles, barriers, water margins and brinks (theedges of cliffs). (Gibson 1986, 36. Gibson’s italics).

So, we are now in a system of caves with connecting openings that affordslocomotion and walls that prevent it. There are also steps, that force us to

jump rather than walk. In the caves, there are creatures that will attack us,

our Game Ego Spyro, as soon as they get a chance. And those creatures are

very quick, they do not die when we use one of the weapons we have, fire,

which forces us to run rather than walk. Sooner or later we will most likely

reach an opening in the system of caves that leads out into a brink with a

barrier that prevents us from accidentally falling of the cliff. At this location

there is a fairy that kisses the Game Ego and gives us ” Superflame” , which

we have learned in the game previously kills this kind of attacking creatureswhich otherwise are invincible. And now it gets really interesting. We lose,




108

for just a few seconds, control of the Game Ego and that means a link

between us and the game system comes to a momentary halt. There is a

Loss-of-Self structure here. (See chapter 8 of the present work for an

elaborate discussion of the Subject-Self metaphors as related to the term

Game Ego). We will regain control of the Game Ego and when we do, we willbe in a hurry to kill of the creatures since the Superflame only lasts a limited

time. What happens then when we lose control of the Game Ego and why is

this so important? When getting close to the fairy our control is broken. The

Game Ego will kiss the fairy and turn around, landing with its head towards

the opening into the cave. And since we map our being and frontal direction

onto the Game Ego, and our primary locomotive direction is in the direction

of vision and there is an opening affording locomotion and this is the way

from which we came, we will start running in this by the environment layout

and bodily set up suggested direction. An indeed we will kill of some of thecreatures and reach the last cave in the system just in time for the

superflame to be inactivated. We will probably repeat this strategy several

times, trying to optimize the path of locomotion but we will never succeed to

eliminate the last creature. What we have to do is instead breaking our

hypothesis of what to do and change the path of locomotion, which also

means that we have to break a cognitive structure of SOURCE-PATH-

DESTINATION. Standing at the brink, we see that there is a barrier that

prevents the Game Ego form falling down. There is, in other words, a

constraint in the environmental layout. However, the barrier not only

prevents the Game Ego from falling down. It also affords jumping up on and

thereby getting a new field of view, a new content in the visual container of

the screen. The barrier is not only an end point of a path but a source for a

new one. Jumping up on the barrier makes us see that there is another path

to be taken, that one can very well take an outside shortcut to reach the last

creature to kill it. And this is of course the only way that one can do this.

This is used on more levels and in other locations in Spyro the Dragon.

Another example is in the Beast Makers world and the level Haunted

Towers. We have to literally disregard, that is try not to see, the obvious

affordance but a hidden one. To reach a desired location far off we must rush

the Game Ego down a pathway to get it ” supercharged” i.e. we can run faster

than otherwise. We are filled up with energy, which is indicated by the

sound, and can also fly longer distances when supercharged. In this

sequence, we are put to the test to change a structure of visually controlled

locomotion. Rushing down the pathway and out into the open through an

opening we reach an open area that affords locomotion in any direction. Tothe right is a curved raised slope that affords us to run up on it and fly.




109

The slope is much like a catapulting runway when we are supercharged.

However that is very difficult since we will most probably fall of the slope

and die. And if we succeed we will not get anywhere anyhow. So what to do?

The answer lies in changing structure of course. Instead of rushing up this

slope we will have to rush the open area, fly to a nearby area that we havealready been at and run up a slope still supercharged and just when leaving

this slope begin to fly, turn left sharp and go to the desired area. Once again

the layout of the environment has made us do things in a specific way and

then change our mind.

These two examples show that visually controlled locomotion is a strategy

used by game designers, them knowingly or not, to make the game a little

harder to play and solve. It also shows that visually controlled locomotion is

a forceful way of manipulating us, in the sense of making us do things in aspecific way due to the environmental properties surrounding us. That the

human mind expects more of the same structure is also indicated. That is,

cognitive and perceptual sets play roles in how we understand the

environment.

5.6 Summary

As hopefully shown above, there are many reasons for using experientialist

theory of cognition for the analysis of computer games. What else is acomputer game if not a structured bodily and social experience?

Understanding a computer game involves the act of playing the game. The

game player is interacting on several levels with the computer game while

playing such as the level of sensorimotor interaction, the level of intellectual

interaction, the level of social interaction and so on. The experientialist

theory of cognition claims that the abstract level is founded on the bodily and

social level due to our innate capacities to mentally project from them.

Furthermore, there is no separation of mind and matter i.e. between mind

and body.




110



WHAT IS IN THE SOUND?

111

6 What is in the sound?

Can sound have a “texture”? Which qualities in a sound give a texture in

that case? Here one will need to use what Chion would call reduced listening

and listen for the qualities of a sound. Or do we not? Do we create texture in

sound from the source (the cause) of the sound? Sound is always a process.

Do we render the texture of a sound from what we know about the source?

The answer is yes! The answers to these questions are to be found in the

following chapter. See also the analysis of Myst in the present work for an

elaborate discussion on this (paragraph 9.10. A sound always has a texture

insofar that it will always render some kind of visual response when

perceived by a human. By this I mean that when we hear a sound we render

a visual representation of that sound. If any sound will produce a visualresponse then it is very obvious why language is so full of visual metaphor.

The opposite will also occur. We know what a room should sound like when

we see it. We map our previous experience from “the real world” (a term

about which I am dubious since I really can’t come up with any good

definition about what this real world would be) and from mediated worlds

that might be fictious. There are processes of audio-visualization and visu-

audiozation.

Sound as well as images may work on a basic level and do so within manycomputer games. Sounds also have affordances. They reveal something

about the objects producing the sound. To exemplify, we might perceive

sounds as ” dense” or ” light” suggesting that there are qualities that

constrains or permits locomotion. A ” dense” sound might be conceptualized

and understood as ” a wall of sound” . We can not walk through walls if they

do not have openings, and openings afford locomotion as Gibson shows. What

I mean is that a sound that we understand as ” dense” does not afford

locomotion in the same way as a ” light” sound. When creating sound-scapes

either as stand-alones or to merge with images, this is important to notice.One might create a ” big” and ” massive” sound that have some ” openings” in

it. It also the case that a ” dense” sound might be so powerful that it is

unpleasant to get any closer to it. That is to say, it does not afford locomotion

in the direction of the sound source. In the following chapter I will first

briefly make some remarks concerned with the relation between sound and

images in computer games and then turn the attention to sound and its

specific qualities.




112

6.1 Blips and blops are not only blips and blops

One often hears people talk about computer games and criticize the sound as

being just blips and blops and furious explosions. At the first occasion of The

School of Sound in London, April 1998, a discussion on computer game

sounds arose where precisely this kind of critique was flourishing, and also

added the critique that these blips and blops really had nothing to do with

the content of the games played. Nothing in my opinion could be more wrong

than such general claims. Most computer games that has such limited

sound-scapes as blips and blops make use of their restricted sound world

quite well. Or rather, the sounds do really have a strong link to the content

of the game and the act of playing it. The sounds are seldom arbitrary but

have a clear and meaningful relation to the game play and the graphics.

They denote what is actually going on the game and may, as I tried to putforth in the foregoing paragraph, tell us something about the environment

that they are a part of. The sound might be an obstacle for or afford

locomotion, for instance. In my recollection of the discussion at the School of

Sound I remember that one of the games discussed and that was given harsh

critique was Tomb Raider (EIDOS 1997). Tomb Raider is not a game with a

sound-scape built only on blips and blops. Furthermore, the sounds in this

game are content-dependent to large degree. There are of course games that

are maybe not that particularly “good” in the sense of how it works in game

play when it comes to sound, such as the: Atari 2600/Intellivision version of Pac-Man System:

The Atari 2600 version of Pac-Man was, simply put,awful. The music was bad, the graphics werehorrible, and the levels were beyond repetitive.Little charms like the Pac-Man death melody andthe cute cherries were replaced with garishcounterparts. In a nutshell, the brilliance that wasarcade Pac-Man was lost and never to be found.

Some game historians have compared this bomb of atitle to the monstrosity that was E.T: The Game.Though there are gamers out there who lovenostalgia and would love to play old 2600 games justfor the fun of it, they would probably not play thistitle. (Doug Trueman

http://videogames.gamespot.com/features/universal/ hist_pacman/p4_01.html).

There are games that fail to meet the requests of the gamers, as this quote

shows. In the case of the Atari Pac-Man version, it was obviously a complete




113

failure. Neither sound, the graphics or the game play was anything like the

original Pac-Man. However, to claim that all computer games are horrible in

their use of sound and graphics is not a constructive approach. Most games

that survive, like the original Pac-Man and Tomb Raider, do use sound and

graphics in a coherent way, much as Brenda Laurel writes about the use of multiple sense modalities in Computers as Theatre (1993):

The primary criterion for deciding which sensorymodalities should be part of a representation isappropriateness to the action, in both mimetic andoperational terms. (Laurel 1993, 159. Laurel’sitalics).

Multiple modalities are desirable only insofar as

they are appropriate to the action being represented(Ibid. 160).

I find this approach to be a notable characteristic of successful computer

games like the original Pac-Man and Tomb Raider. The multiple sense

modalities of sound and images are used in an appropriate manner.

Let us now turn our attention to sound. Sound has some salient features in

its physical being and its conception in the human body/mind system. Sound

is a mind-dependent phenomenon. Fluctuations, vibrations i.e. movement in

a medium of some sort, be it air, wood, metal or water, for instance, may be

conceived as sound in a body/mind system in some living being. The human

body/mind system for sound perception has a range that conceives vibrations

approximately between 20 periods per second to, in its extreme, 20.000

periods per second as sound. However; that is the human body/mind system.

As is the case with visual perception, the frequency range that is perceivable

differs from life form to life form. We can not see all wave lengths of light.

We have not evolved to see in the infra red spectrum for instance. The

human body/mind range of sound is dependent on its necessity for thesurvival of the organism. The sound a human can perceive, due to evolution,

have or have had some importance for the perceiving organism, otherwise we

would not have the ability to perceive it in the first place. The human being

uses sound to act upon. It is part of the survival system of the human being.

Sound is the body/mind system result of vibrations in a medium and it is the

result of movement of objects in the surrounding environment. As

movement, it will have a relation to duration. Duration is possible to divide

into phases. Like a narrative, a sound has a beginning, a middle, and an end.Unlike a narrative, it must have it in a chronological order when it is a




114

natural experience. A sound does not occur out of nothing. The phases of a

sound is commonly described by an ADSR curve. This kind of curve describes

the way a sound develops over time by providing parameters for the

description of the Attack, Delay, Sustain and Release of the sound. Not

surprisingly, those very parameters are to be categorized as being on thebasic level of the sound.

I argue that also sound has affordances. Gibson was concerned with vision

and his work resulted in an elaborate theory on the human visual

perception. He does mention other senses in his work though such as smell

and hearing in his discussion on the medium the human animal is mainly

contained within, i.e. air. Air has the affordance of light refraction. In the

medium of air the ambient light array makes the affordances of objects

perceivable through the sense of vision. Air also allows sound waves tospread and be reflected by the surfaces of substances.

6.2 Hear-seeing: Anaphones and basic sound terminology

Sound is understood as something that can be seen and that has a visual

relation to the subject of hearing a sound. Visualization of something is not

the putting onto paper something that can been seen. Visualization is a

human internal act of cognition. Sounds are shapes in the environment of

man and other animals. Sound is categorized the same way as other senseswhen it comes to meaning. There is a tight coupling between the basic level

categorization of sound and its affordances. At the basic level sound has

general affordances. There are UP-DOWN, COMING CLOSER, SOFTNESS,

HARDNESS structures in the sound for instance. Do not be confused by this

claim. Things will be clarified soon. First let us note some considerations

about how we listen to the environment we are within. The French film-

theorist and composer Michel Chion suggests three different listening modes

that we might use when listening to film (and the environment at large I

suspect though this is not really clarified, in his Audiovisuon–Sound onscreen. Those three listening modes are:

Causal listening i.e. what is the source of this sound? This is an important

mode in listening since it might reveal an upcoming danger as well as it

might a source of joy. It might be the sound of an approaching car or the

voice of one’s beloved. Causal listening is the most common mode Chion

claims and this seams reasonable. Furthermore he notes that ” causal

listening is not only the most common but also the most easily influenced

and depictive mode of listening” . (Chion 1990, 26). Chion also notes that if the source of the sound is visible the causal listening mode supplies




115

additional information concerning the source. Tapping an enclosed container

might reveal how full it is for instance. When the source is not visible, we

make an initial hypothesis, of what causes the sound from prior knowledge

of what objects causes specific sounds. (Ibid. 26). Chion further claims, and I

do not fully agree with this, that there is a big difference in taking notice of aperson’s vocal timbre and identifying him or her, as he writes, ” having a

visual image of her and commiting it to memory and assigning her a name.”

I do believe that we do make us an image of how someone looks through such

things as the vocal timbre for instance. We might even come up with a name

to make it easier to remember. That it is not a real match between what we

here and the actual cause is not something that matters in this respect. We

need to anchor the perception of a voice and hence we make us an image of

the source. There is always the option of later revision. At another level we

are able to discern sounds at a more basic level as being human voices,mechanical equipment or such Chion suggests. This is, of course, an

important thing to be able to do. We are also able to follow the causal history

of a sound even if we can not exactly tell the sources. We can hear that ” this

is the sound of something that scrapes against something” even if we can not

tell exactly what is scraping against what, Chion claims. (Ibid. 27). However,

I find that we do formulate an initial hypothesis anyway since we speculate

on what produces the sound. Even if we are not right we do try to create for

ourselves an understanding of how the texture of the objects are etc. We

speculate.

Semantic listening i.e. what is this sound regarded as language or code

telling us? This is the most studied listening mode since it has been the topic

for linguistic research. The two modes of causal and semantic listening are

often blended. It is fully possible as Chion notes to hear that something is a

voice and what the voice is telling us in the use of a language. (Ibid. 28).

Reduced listening i.e. what qualities has this sound in

it self disconnected from its source? ” Reduced listening” , Chion writes, ” takesthe sound–verbal, played on an instrument, noises, or whatever–as itself the

object to be observed instead of as a vehicle for something else” . (Ibid. 29).

Furthermore, Chion discards the use of spectrographs for analysis of sound.

It does not say anything about the sound he argues. This is at least my

understanding of what he writes. It is not all that clear though that it is

what Chion means, or if he really refers to other people’s arguments here.

This is what he writes:

Others might avoid description by claiming toobjectify sound via the aids of spectral analysis or




116

stopwatches, but of course these machines onlyapprehend physical data. (Ibid. 29).

However, as will be thoroughly discussed in my analysis of Pac-Man and

Myst, such spectral data is not an arbitrary display of what is in a sound.

The display of data this way has its basis in very general metaphorical

structures of mind as described in chapter 5 of the present work. UP–DOWN

and SOURCE-PATH-DESTINATION structures are protruding in this kind

of visual manifestation of sound. Otherwise the data displayed would be

meaningless and impossible to relate to the sound in the first place.

Reduced listening is actually not a term of Chion’s but a term he has

borrowed from Pierre Schaeffer.39 In addition, it is also a little troublesome

to use. What is a quality of a sound really? How can we think about a quality

in a pure form disconnected from everything else. That is; a sound

disconnected from its material origin. Schaeffer gives a hint on this in an

interview from 1986.

Take a sound from whatever source, a note on a violin, a scream, a moan, a creaking door, and thereis always this symmetry between the sound basis,which is complex and has numerous characteristics,which emerge through a process of comparison

within our perception. If you hear a door creak and acat mew, you can start to compare them -- perhapsby duration, or by pitch, or by timbre. Thus, whilstwe are used to hearing sounds by reference to theirinstrumental causes, the sound-producing bodies,we are used to hearing musical sounds for theirmusical value. We give the same value to soundsemanating from quite different sources. So theprocess of comparing a cat’s meuw to a door creak isdifferent from the process of comparing a violin note

to a trumpet note, where you might say they havethe same pitch and duration but different timbre.This is the symmetry between the world of soundand the world o f musica l va lues .(http://www.cicv.fr/association/ shaeffer_interview.html).

Well… Why should there be a difference between the processes of sound

comparison? Do I misunderstand Schaeffer in what he says? Does he mean

39http://www.cicv.fr/association/shaeffer_interview.html




117

that it is more common to use a certain way of comparing sounds outside

music and another inside music? The problem with the idea of reduced

listening is that Chion and Shaeffer tries to describe the experiental aspect

of context as a perceptual phenomenon. All sounds however be described

from how they are perceived, as opposed to a description from their semantic value. The ” reduction” in ” reduced listening” consists in letting the semantic

aspects out it seems. However, I find this virtually impossible. As an idea

stemming from electro-acoustic music, or ” musique concrete” to stick with

Schaeffer’s terminology, of the 1950s, reduced listening made possible new

ways of thinking about sound. It was important then to put it on the agenda.

Today, in the analysis of film, I find the term more problematic than helpful

though.

Let us now expand our understanding of sound going from these listeningmodes described by Chion, to the work of David Bordwell and Kristin

Thompson. According to Bordwell and Thompson, a sound has three basic

qualities: loudness, pitch and timbre. (Bordwell, Thompson 1993, 295-296).

This is a crude model of what sound is but still useful for the purposes here.

The two first terms are classical physical descriptions of what a sound is and

the last one is perhaps more ecological in its essence since it is not

measurable in the same way or at least does not have as close a relation to a

physical description as the two other terms.

Loudness has to do with perceived sound level on a scale from the inaudible

to the painfully loud. Loudness also has conceptually to do with the size of

the object that produces the sound. The sound and the sound source have a

volume. Puny things do not generally produce big sounds. The result we

arrive at is that loud sounds generally equal big objects.

APPROACHING–LEAVING is yet another complex schematic structure

found in sound and that relates to the perceived size of an object. We judge

an object’s distance from us not only from its visually perceived size but alsofrom the loudness of the sounds it may produce. We can judge if a sound-

producing object is APPROACHING us or LEAVING – that is, if it is

COMING CLOSER or if it is GOING AWAY. This has to do both with the

loudness of a sound as well as with the pitch of sound, as will soon be

discussed. For now let us settle with that the closer a sound-producing

objects is to the perceiver, the louder the sound will be perceived and vice-

versa. The effect of APPROACHING–LEAVING is that we perceive a change

in the sound that denotes this quality of the object.




118

Pitch has to do with the periodicity of the vibration in the substance

transduced via the medium to our auditory system i.e. the frequency of the

vibration discussed above. Note that pitch is not equal to frequency when we

are speaking of musical terminology though there is a rather firm relation.

As shown earlier in chapter 5, UP and DOWN are orientationalmetaphorical concepts structured by image schemas. Sound also have an

UP–DOWN orientation in the way we categorize them.

We often map an understanding of high and low on sound as the term level

suggests in itself. We conceptualize and increased frequency rate as going up

in pitch and a decreased frequency rate as going down in pitch.

Furthermore, to do this mapping we conceptualize a motion of going up and

of going down, as we perceive such changes. This is to say that there is also aSOURCE–PATH–GOAL/DESTINATION structure in this conceptualization.

To have a motion UP we must first have some location source from which

that sound (and its source) move to another position. Remember what Peter

Bastian wrote about the human voice and its production of sounds as

compared to lifting a stone from the ground up in the air? We have to strain

our vocal cords to produce a rapid oscillation. That is to say, we have

produced a rapid vibration also termed a high frequency. The conceptual

metaphorical structure of More-is-up is structuring this kind of experience

as Bastian’s example of lifting the stone shows. It takes effort, energy, to liftthe stone and it takes effort to produce the high note. Just as gathering more

material, substances, in a heap, will make the level of it raise gives more

equals up. Highness and moreness are within the same conceptual domain.

They denote the use of energy and/or that an object has an amount of energy.

It also goes the other way around of course. Falling pitch may denote the loss

of energy.

Noteworthy is that a raise in pitch is commonly conceptualized as ” an object

is approaching me” (or ” I am approaching an object” ) as mentioned earlier.This is not only an effect of shifts in loudness but also due to what is called

the Doppler effect. When an object that produces a sound is approaching a

perceiver the sound waves are compressed why there is a raise in pitch and

vice versa.40 That of course means that a changing pitch is possible to

understand not only in an UP–DOWN configuration but as a LEAVING-

APPROACHING configuration as well. This in turn is not a contradiction. I

40 The velocity of sound, about 340 meters per second, is invariant. Due to the velocity of a moving sound

producing object, the sound waves in front of the objects will be compressed and behind the object, theywill be expanded. The result of this an increased pitch in front of the object and a decreased pitch behindit. This is known as the Doppler effect.




119

do not mean that UP = LEAVING or that DOWN = APPROACHING. I only

mean to point out that UP–DOWN is one structuring aspect and that

APPROACHING–LEAVING is another. Very much like how we conceive of

time as either moving (in constructions like ” Christmas is closing in on us” )

or that we move in time (in constructions like ” We are heading forChristmas). The orientational and spatial metaphor structures of mind are

salient. The spatiality of the sound-image is protruding.

APPROACHING–LEAVING is also tightly related to the human

FRONT–BACK orientation, which in turn is an important object for

computer game play. The tactile motor/kinesthetic link between the Game

Ego and the game player relates also to the Game Egos FRONT–BACK

orientation. The preferred direction of movement is often FRONTWARD

movement since we map our physical being upon the Game Ego. In the caseof Pac-Man (see paragraph 9.4) the FRONT–BACK orientation is suggested

by Pac-Man’s mouth or rather our mapping of our mouth onto the yellow

circular shape that we are in control of. The direction of movement per se,

will also be understood as the direction one starts to move something on the

screen will be in the direction of its FRONT if the object itself does not have

a clear mapable FRONT–BACK orientation.

The human auditive system is not equally sensitive to all frequencies, which

in turn means that some sounds are perceived rather well even if they arenot what we would objectively call loud. However, the subjective experience

of loudness is not the same as measuring sound pressure with a machine.

We are most sensitive for sound in the frequency span between 1kHz to

5kHz.41 That is, we can hear sounds that do not have a high sound pressure

(which also is called high amplitude). There are reasons for this of course.

Human beings are ecological systems (and live in ecological environments in

constant interaction with this environment. The latter, on interaction, is

pointed out by Gibson as well as Lakoff/Johnson). If we were to hear sounds

of low amplitudes we would possibly be spending our days with listening toour own heart beat, digestive system, muscular activities and so on. If so

maybe we would be extinct as a species since we would not hear things of

greater importance to us. It is a matter of evolution.

Why now take up time with this, you might ask. It is very simple: In this

diversified frequency-response lies an answer to Michel Chion’s claim that

we are voco- and verbocentric above all. Human speech lies within the most

41 Cf. Zetterberg 1995. This is of course more or less common knowledge among sound editors and soundengineers.




120

sensitive frequency span of human beings. Sounds do not need to be of high

amplitude to be conceived within this range. Speech is an important way for

humans to communicate. Which came first, speech or the frequency range of

it, I do not know and I do not think it really matters. We do have a basic

capability of recognizing sounds of low amplitude in a certain frequencyrange.

Timbre is that quality of a sound which makes it sound as the sound of

something specific. As Bordwell and Thompson write, timbre is

indispensable to describe a sound texture or the feel of a sound. (Bordwell,

Thompson 1993, 296). That is, in other words, a tactile description of the

sound. The sound is conceptualized as a surface of something. If we again

couple this to Gibson’s theory on visual perception we have a relation

between sound transduced within the medium and sound conceived of as asurface of a substance. It all connects and makes up a coherent conception of

the term "timbre". One could describe timbre as that which gives a surface to

a sound. This is very close to what Philip Tagg calls "tactile anaphone"

presented in his article Towards a Sign Topology of Music. (1992).

The term "anaphone" is a neologism that means ” the use of existing models

in the formation of (musical) sounds” . (Tagg 1992, 3). Anaphones can be

divided into three categories:

Sonic anaphone. There is a perceived similarity to paramusical sound.

Kinethic anaphone. There is a perceived similarity to paramusical

movement.

Tactile anaphone. There is a perceived similarity to paramuscial sense of

touch.

By hear-seeing, I mean the simultaneous experience of hearing and seeing

and making sense of it. The merging of the heard with the seen makes upthe pattern of understanding the sound image combination. One result of

this is that sound is understood as something that can been seen and that

has a visual relation to the subject of hearing a sound. Visualization of

something is not the putting onto paper or screen something that can been

seen. Visualization is a human internal act of cognition that gives meaning

to a sound. It is of course not limited to audition but any sense modality may

result in such an internal visualization. Another result of this is the effects a

combination of written language and vocal language might have. We bring

together these two sense modalities and often impose the written language




121

on the vocal/oral language to make sense out of it. This is striking in a recent

trend on at least Swedish Internet sites. At http://come.to/hatten for

instance, Swedish written language has been added to a song from what to

me sounds like an Arabic or at least eastern culture. The effect is stunning.

The Swedish written text that accompanies the song makes one hearSwedish instead of the original language due to synesthetic effects of hearing

and seeing/reading at the same time. It is impossible not to hear what is

written if one can read and understand Swedish. This effect of synesthesia is

important for games that afford the use of both written and spoken

language, as my analysis of Day of the Tentacle indicates (paragraph 9.9).

The combination of written and spoken language is powerful as is the

possibility to shift from one mode to another i.e. to shift from either using

only spoken language to use both written and spoken or only written

language.

The effects of synesthesia are also important for how we conceive and make

sense of room acoustics. When we see a room we use our prior knowledge of

room acoustics (i.e. we use our experience of being within the environment of

a room) and map it on the room we see. We make assumptions of how the

sounds produced in the room would sound to us from the observation point of

the visual representation, that point i.e. which is often called the point of

view in this context. We judge materials and their textures, distances,

possible sound sources and possible “off-screen” spaces. We know some of theacoustic features of several prototypical rooms. If the audiovisual

representation is not coherent/consistent (i.e. if the visual inference and the

auditive inference of the whole configuration do not match) we are able to

feel this even if we can not put it in words or recognize what it is that is

abnormal about the audiovisual situation. So far, it is pretty simple.

However, there is also a factor of mediation engaged here. We do not only

judge our audiovisual perception of a room from unmediated situations but

we adapt to mediated conventions. That is to say: we refer not only to the

real world but also to representations of the world that we take part in from

film, television and computer games, for instance.




123

When working with low resolution images and low resolution sound in a

computer game one might be wise to adopt Tati’s way of merging sound and

images. The basic level is what gives meaning to the event and denotes what

is going on. When describing sounds as heavy, light, happy or sad, image

schemas are structuring the sound i.e. the structuring of the soundexperience is a cognitive process, in which image schemata provide the basic

meaning comprising process of mind. We make assumptions about the sound

source and its salient features and then make our judgement thereafter.

6.3 Summary

The embodied mind is important for the construction of a sound and its

affordances. Sounds have spatial qualities. Sounds have also implied

textures, which are a kind of a spatial quality. Through the simultaneousappearance of sound and images, we will have effects of synesthesia.

The embodied mind is important for the construction of a Game Ego, which

is the objective of chapter 8. However, to make it easier to follow my

arguments put forth in chapter 8, we first need to have an understanding of

two other terms: "Interaction" and "Interactivity". These are consequently

the objectives of the following chapter, chapter 7.




124



INTERACTION AND INTERACTIVITY

125

7 Interaction and Interactivity: computer games as

audiovisuo(perceptuo)-motor experiences

The following chapter sets out to explore the terms interaction andinteractivity. Interaction is crucial for games. Without interaction we would

not have games at all. Interaction is the basis for performative experiences.

It may be experiences of motor performance and changes of mental states

related to the motor performance (motor interaction), it may be vocal and

verbal interaction with another human agent (social interaction) etc. Note

that performance and the enactment of being are important in this context.

We have already noted the importance of interaction in Gibson’s theory and

his explanation of the interdependency of the animal and the environment in

chapter 4 of the present work. ” What is interactivity then?” you might ask.Well, this term is complicated, as the following paragraphs will show. For

reasons to be explained, "interactivity" is a term that I think would be better

to be abandoned in some cases. As terms, both "interaction" and

"interactivity" are overused and under-defined, so there is a need to

elaborate what one means when using them. This has been noted for

instance by Jens F. Jensen and Brenda Laurel as we shall see in the

following paragraphs. Jensen adopts an objective definition of interactivity,

as will be the topic for discussion in paragraph 7.2. Laurel adopts what she

calls a theatre metaphor for computers. She is not as concerned withproviding definitions of interactivity as is Jensen. This is discussed in

paragraph 7.3. This chapter will also contain some remarks on John

Alexander’s use of the term, in paragraph 7.4, as well as Mark Johnson’s and

George Lakoff’s adoption of it, in paragraph 7.5.

Before rushing into the discussion on Jensen, Laurel, Alexander, and Lakoff

and Johnson, however, I will make some remarks on the concepts of

interactivity and interaction myself and also go into discussion on how we

tend to personify a computer game system while playing a game.

7.1 The interactivity hype

There has been, and still is, a bit of hype, concerning the need for

interactivity and interaction in media. There has been an assumption that

the ”media user” has a specific need for his or her manipulation of and

contribution to the “story told” by the media. That is, that a media user has a

need to be an agent in the media and not only a passive beholder of ready

made content. This idea may be correct and may have some value.




126

However, we are never passive in a media situation. We have to form an

understanding of the media content. This is of course an active cognitive

process involving the cognitive unconscious as well as the conscious parts of

human cognition. It is an active body/mind process. The concepts used by

human cognition stems from our bodily being and enactment withininteractional contexts in the world. We may have more or less opportunity to

actively affect the objective media content but subjectively we are

experiencing the media and understand it through bodily based concepts.

7.1.1 Patterns for interaction: categorization of interaction

We have little chance of affecting the written text in a newspaper in a way

that would be meaningful to others to take part in while we are reading it for

the first time at the breakfast table. We could write the editors of thenewspaper a letter in response to what they wrote but there is maybe little

chance that this will be published and even if it would it might take some

time for it to be so. It will be a slow-paced interaction. The general pattern

for interaction is more rapid than this. It is more of a

social/mechanical/instrumental one. Human beings tend to be impatient. Let

me provide an example from a more mechanical/instrumental understanding

of how the world works. Think about how you would react if you were to

hammer a nail into a piece of wood and it would take two minutes from that

moment you hit the nail until it started to move. It would be very unnaturaland you would feel that you had not performed any action that really

mattered. You would probably prefer a more rapid succession of events than

this. The same might be valid for media use. We could of course adapt to the

media pace to some extent but still a more mechanical understanding of how

the world actually works would intrude on this adaptation. Most people who

have ever used a computer consciously know that one may be wise not to

expect an immediate response when pressing a key on the keyboard. Still we

do so, at least within the cognitive unconscious. This conflict may result in

stress. Motor action tends to give an expectation of immediate response.Games that provide this immediate response or the feeling/sense of such

match a general pattern of motor-mechanical interaction. It is even so that

motor interaction is innate to humans. Consider the reflexes infants have

such as sucking, swallowing, gripping etc. Reflexes are unconscious actions.

We also have a pattern for interaction with other humans, which is not to be

neglected in this context. Social interaction is an equally important aspect of

what interaction means to human beings as the mechanical pattern of

interaction. In addition, it does not differ that much from the mechanicalinteraction as one might first believe. They seem to be two sides of the same




127

basic structure. By this, I suggest that there are several features in common

to mechanical and social interaction. They are within the same conceptual

domain. We want our presence to be recognized by our counterpart in a

social situation. If we feel neglected in some way, we do not get the feeling of

being part of the other person’s attention and hence no interaction is takingplace. There is no mutually interdependent relationship established if our

counterpart does not react to our presence in a way that lets us believe that

he or she is aware of us and that we make a difference to him or her.

7.1.2 Personification of the computer system

The computer and the human user, a game player for instance, make up a

system of interaction and interactivity. When being engaged in a computer

situation and especially when playing computer games of some kind, it iscommon for the user to adopt a way of thinking about the computer and the

game system as another person rather than an instrument or a dynamic

system for handling some task. One might think that the reason for this is

that computer games often do have animated (in the word’s meaning of

objects being given life that is) characters that are in the likeness of man (or

another animate being) or that there are human actors acting within the

game. The fact that many games might include the human voice as a central

part of their narration system can also be thought of as part of the reason for

this. In such cases, it is a natural mapping occurring. We experience ahuman voice and map this voice and its characteristics as human voice onto

the computer system in it self. However, even without the animated

characters and the human voice, users might conceptualize the computer as

another person rather than an instrument or a tool for performing a task.

The computer game system may also be addressed with pronouns suggesting

that it is conceived and understood as other person during game play. This is

easily observed when someone is playing a computer game or using a

computer for some other task. An observation made in real life does not have

high status as being scientific though, so to further ground this statement,that is, that computers are sometimes addressed and conceived as animate

beings rather than instruments (tools) I will refer to a more scientific study

of the phenomenon; ’The Computer as Alter’ (Scheibe, Erwin, 1979. 103-109.

See also Loftus, Loftus 1983, 87- 89 where they refer to this study). The

main purpose with the experiment reported was to record the spontaneous

verbalizations the subjects made during a 20 minutes session of computer

game play and from that data try to find out the answer to two hypotheses:

1) Subjects might show more evidence of personification of the computer inrelatively isolated settings.




128

2) More personification occurs when the computer acts in a relatively

intelligent way than when it acts less intelligent.42

The study by was performed with 40 university undergraduates (20 males

and 20 females) as study subjects who where randomly set up to play one of five different versions of a computer game called GUESS in two different

locations. The versions of the game differed in the conditions on which the

computer game made its choice. The locations differed in that one location

was a large room with twelve terminals and other people in the room and the

other location was a smaller room with only one terminal and usually

unoccupied by other than the subject and the experimenter. The computer

game GUESS was a simple binary game that allowed the game player to

choose between two numbers, 1 and 0. The goal was to act in a way so that

the computer did not match the game players input. Each trial gave onepoint and the game was played until the subject or the computer had fifty

points. Each session of game play was recorded with a small tape recorder

placed in full view on top of the computer screen but no explicit information

about this was provided to the subjects. If asking about the recorder subjects

were told that they were being recorded since it was interesting to record

that what might be said during the experiment but not that this was the

main purpose for them being there. The experimenter was sitting by the

subject during the experiment but did not speak during the game session if

not directly addressed by the subject. The information of how the gameworked was provided by written language on the computer screen.

The result of the study showed that 39 of the 40 subjects did spontaneously

talk to and about the computer (or as I would rather say, the system of the

computer game and the computer understood as a PART–WHOLE structure)

as something animate and non-instrumental. They refereed to it as it (244

times), he (57 times), you (51) and they (6 times). That is in all 358 pronoun

references to the computer. Note that none of the subjects used the pronoun

she.43 The subjects showed affective responses and some of them asked if itwas not really a human providing the output from the computer system.

Scheibe and Erwin find in the verbalizations four basic categories: –

42 “When media conform to social and natural rules […] no instruction is necessary. People willautomatically become experts in how computers, television, interfaces, and new media work. Becausepeople have a strong bias toward social relationships and predictable environments, the more a mediatechnology is consistent with social and physical rules, the more enjoyable the technology will be to use.”(Reeves, Nass 1996, 8).43

On the reasons for this exclusion of the pronoun she I might only speculate. Maybe the computer at thetime of the study, 1979, was more associated with at the time prototypical male environments and maletasks. But this is nothing but a speculation.




129

1) direct remarks to the computer (such as apologies for responding too

rapidly)

2) exclamations (such as “Wow” and “C’mon”)

3) commentary (such as “It seems to know what I’m going to do”)

4) questions to the experimenter (such as “Is there any system to it?”)

(Scheibe, Erwin 1979, 107).

This structure is very close to how the dialogue in Day of the Tentacle is

organized. See analysis of this game in paragraph 9.9.

The computer is easily conceptualized as another person. At least

momentarily and during an engaging task performing activity. It is not aneutral agent in socialization. The harder the game was, the more personal

pronouns the subjects used. That is; the more personification there was as

the hypothesis claimed. (Scheibe, Erwin 1979, 108). What we have here is a

sociological study and interaction with the computer system defined as social

interaction.

The above is also an analysis of the use of language in a computer game

situation. Language is an expressive and extroverted way of personificating

something outside one self. The study shows, as the categories tell, that thesubjects addressed the computer, and sometimes it was comments to the

experimenter.

I argue that this also works the other way around: there is a part of ones

Self/Selves within the computer game that I will call a Game Ego (see

chapter 8, for an elaborate discussion on this term). A computer game is a

structured bodily and social experience. Understanding a computer game

involves the act of playing the game. When playing certain computer games

one tends to identify with the game character(s) referring to it as ” I ” and” Me”. One might say, ” I just died” ”The enemy shot at me” and so on. This is

evident in ”simple” games like Space Invaders, Pac-Man, as well as in more

”complex” games like Duke Nukem 3D. Scheibe and Erwin also show this.

One of the subjects, a female that lost seven times in a row, did say “Oh,

shoot. It’s killing me” and later also “Come on. I’m going to beat you”

(Scheibe, Erwin 1979, 107-108. My emphasis). Not that there is a particular

Game Ego in the game GUESS but this shows that there is a tendency to

identify a relationship between the computer game system and the self as aninterpersonal relationship. Scheibe and Erwin (as do Loftus and Loftus




130

without giving credit to the former) relate this phenomenon of

personification to a passage of the writings of the behaviorist/social

psychologist George Herbert Mead.

It is possible for inanimate objects, no less forhuman organisms, to form parts of the generalized…other for any given human individual, in so far as heresponds to such objects socially or in a socialmanner… Anything–any objects or set of objects,whether animate or inanimate, human or animal, ormerely physical–towards which he acts… socially isan element in what for him is the generalized other.(in Scheibe, Erwin, 108-109. Originally in Strauss1956, 213).

This indicates that we are (at least in our imagination) interacting with an

intelligent agent. So, we do tend to give the computer system some qualities

that make us interact on a social level with the system. With this in mind it

is now time to study Jens F. Jensen’s definitions of interactivity.

7.2 Jens F Jensen: An objective approach to interactivity

Jensen’s writings are concerned with (and favors) giving interactivity an

objective definition as continuum i.e. “as a quality which can be present to a

greater or lesser degree.” (Jensen 1998, 191). As Jensen remarks,interactivity and interactive media became buzzwords around 1993.44 They

have mainly so far had positive connotations in the press (Cf. Jensen 1998,

185, the example from News week). They are high tech, involve freedom,

choice, grassroots democracy etc. (Jensen 1998,185). Jensen does not

elaborate on the experience of interaction and/or interactivity. To exclude the

experience of interaction from the agenda is to give too much attention to

only a part of the process of interaction. It will, for instance, be hard to find

out what motivates subjects to interact in specific ways. If we, for instance,

take on an ecological approach to explore interaction, we will come to

different results. As Torben Grodal suggests:

An ecological theory of interaction must take theperspective of the agent and his behavior andexperience, not in some absolute ideas of ’trueinteraction’. (Grodal. His notes from a lecture inRouvaniemi 1999).

44

1993 was the year when Internet and The World Wide Web encountered a new broad “audience”.Households got wired. A huge amount of newspaper articles focused on the “new media” as did televisionand radio.




131

Interaction in Jensen’s conception of it would typically mean exchange,

interplay, and mutual influence. He defines the term from its conception

within sociology, media and communication studies and how its is used

within informatics. Jensen finds that within the field of sociology the word

interaction always refer to the mutual relationship between two or morehumans, while in the case of informatics it always refer to the relationship

between a human and a machine, but not to the mutual relationship

between two or more humans mediated via a machine. In the case of media

and communication studies, the word refers to the relationship between a

reader and a text but also to the mediated social interaction. (Jensen 1998,

188-190). However do none of these conceptions and understandings of the

term interaction reveal anything about the experience of the interaction

since they do not take in account interacting with something or someone.

Although a computer game is preprogrammed to some extent, the gameplayer will often find that actions toward the game system will be gratifying

in one way or another. That is, interaction with the game will provide an

experience of interaction with and even within an environment that is

possible to manipulate and to be engaged in. There is a subjective difference

in how one conceives of interaction and interactivity in different situations.

Even if the game forces the game player to perform certain actions he or she

might still feel that he or she has control of the situation and that it is his or

her actions that control the game and not the other way around. As long as

the submission to the game is not too obvious, the experience will be one of

mutual influence and not the experience of a forced set of actions. It will be

an interactive situation and the subjective experience will be one of real

interaction. (See also chapter 8 of the present work and the discussion on

The Game Ego).

Difference making is perhaps one key here. Not the only key but one key

important enough to include in this context. Loftus and Loftus provide an

explanation to why we engage in games in the first place and why we keep

on playing them. In their explanation, they focus on the reinforcements one

gets while playing.

Central to all [theories of reinforcements], is theidea that any behavior that is followed byreinforcement will increase in frequency. In short,computer games that do something to make theplayer feel good will be played again and again.(Loftus, Loftus 1983, 14).




132

We engage in game activities since we assume that they will make us feel

good. And if this assumption is recognized, i.e. if the game does make us feel

good, we will continue to play the game over and over again. As Loftus and

Loftus point out, the reinforcements are controlled in the game play to

maximize the time a game player will feel good and therefore continue to beengaged in the game. Devices that game designers use to achieve this are,

for instance, the rate of points given for the successful execution of actions

made, the possibility to beat a high score, the possibility to get a free game

once in a while, etc. Since their study is mainly, in this case, focused on

arcade games, the time a game player will be engaged in the game is

essential and so is of course his or her willingness to put in another coin or

two in the machine and play yet another game and yet another and so on.

Why is then this interaction with anthropomorphic agencies so gratifying?

The experience of the self as difference making and the experience of feeling

good are merged in the interaction with the world.45 It does not matter if the

world is a virtual game environment: the interaction within the virtual

world still makes a difference. This is so even if it is preprogrammed and the

final outcome of the interaction(s) in the long run will be possible to have a

notion of. In part, this notion of possible outcome might be the reason for

taking part in the interaction in first place. It is the excitement of

presumptions and expectations that make us engage in the game activityand the game environment. Either we are freshmen and engaging in the

game for the first time or we are experienced players. As freshmen we may

have heard friends talk about this or that game and want to try it out or as

experienced players we know very well what we are headed for and engage

in the game for that reason. The expert player might show off a little, reach

a high score, and hence get the admiration of his or her friends. In on-line

games such as Unreal or Quake, the expert player will have positive

feedback as reinforcements in the successful killing of his or her opponents.

There will be comments from his or her team mates and the name of theplayer will be displayed at certain situations in the game for everybody to

see. If I kill a lot of enemies while playing Unreal and not get killed myself

the system will grant me my name displayed and a comment like ” (o)Wolf

(SWE) is dominating” . This is a strong reinforcement and motivation for me

to carry on killing my opponents.

45 With feeling good I specifically mean feeling good as in goodness and feeling well as in healthy. Butgoodness is relative and related to the survival of the Game Ego or the game players own self. It is a

utmost subjective feeling and state of mind. One would probably not say playing Dungeon Keeper(Bullfrog 1997) successfully is an act of objective goodness (if something like that exists). The aim andmain goal of this particular game is to crush the good guys and bring the evil forces to victory.




133

7.3 Brenda Laurel. Theatre as an Interface Metaphor

Brenda Laurel has as chosen a theatre metaphor for the human interaction

with computers. (Laurel 1993). Commenting on the developers of one of the

earliest computer games, Space War she touches upon two things that are

essential for the present work, namely action and participation.

They regarded the computer as a machine naturallysuited for representing things that you could see,control and play with. Its interesting potential laynot in its ability to perform calculations but in itscapacity to represent action in which humans could participate. (Laurel 1993, 1).

Well, this is good of course. Participation and action are the basis for

computer games. However, ” real life” is also a question of participation,

action, and interaction both in social contexts and more object related ones.

Something that is really not that different from one another as Scheibe and

Erwin show and that was the objective in paragraph 7.1.2. One does not

really need to link this to theatre. In Laurel’s work, much of what she claims

does not only go for theatre but also goes for real life. The theatre metaphor

is perhaps not that needed to describe some of the processes of interest, i.e.

processes of interaction within an environment and the social roles of

participants. The representation of action is perhaps what could be keptwithin a theatre metaphor. However, for reasons I will soon get back to,

many computer games would benefit from being considered not being

representations since they actually do incorporate things found also in real

life and that are possible to act out. Not to act as in theatre but to act as in

taking action that is.

One good thing in Laurel’s work is that she notes that there has been a

debate concerning the meaning of the concept of interactivity for quite some

time. The INtertainment conference in 1988 collapsed under the argumentsover this concept. There was obviously a great confusion about what

interactivity really means and most delegates seemed to have their own

ideas about it. (Laurel 1993, 20). What was probably going on was what

Jensen so systematically has pointed out: the confusion of terminology

blending several different understandings of the term interactivity and its

use within different fields of study. There were people from some of the areas

that Jensen covers i.e. from the business of personal computers (informatics

conception of interactivity and interaction) and computer games (media

conception of interactivity and interaction). However, this is not explicit inLaurel’s work, I suppose that there was also a case of blending and confusing




134

of the terms interactivity and interaction. 1988 might not seem a long time

ago but at the time of writing (2001) it is actually like a geologic time span.

Laurel has earlier (in her 1986 doctoral dissertation) defined three variables

of interactivity as continuum that she refers to in Computers as Theatre (20)and that would go along with Jensen’s way of defining the term.

Interactivity for Laurel at this stage was possible to divide into three

measurable entities:

1) frequency (how often you could interact)

2) range (how many choices were available)

3) significance (how much the choices really affected matters)

Jens F. Jensen remarks that Laurel’s definition is within what he calls a

consultation pattern ” since choice is the reappearing term” (Jensen 1998,

196). Jensen does not mention anything about the fact that she had later

changed her opinion and revised this definition. Nor does John Alexander

who also somewhat misleading put the words "user" and "optimum

interactivity" in quotation marks, suggesting that Laurel has used those

terms in the context, which she has not (Alexander 1999, 186). What she

actually does write is that:

Optimizing frequency and range and significance inhuman choice-making will remain inadequate aslong as we conceive of the human as sitting on theother side of some barrier, poking at therepresentation with a joystick or a mouse or a virtual hand. (Laurel 1993, 21).

That is to say that these levels are not everything. Sensory immersion and

the tight coupling of kinesthetic input and visual response also matters in

giving the representation life.

You either feel yourself to be participating in theongoing action or you don’t. (Ibid. 20-21).

This is close to what Torben Grodal propsed in the notes for a lecture in

Rovaniemi 1999 and that has later been elaborated and published in his

article Video games and the pleasures of control (2000). His argument though

is concerned with the phase of automation of performing actions that in turn

probably will lead to a sense of not being engaged and participating in the




135

ongoing action. Returning to Laurel, she continues to discuss the

participation of an agent:

The experience of interactivity is a thresholdyphenomenon, and it is also highly context-dependent. The search for a definition of interactivity diverts our attention from the realissue: How can people participate as agents withinrepresentational contexts? (Laurel 1993, 21).

The answer to this last question is, Laurel suggests, that we do know a lot

about how people participate in such representational contexts through the

study of theatre. Actors and small children do participate in a representation

while playing make-believe stories. There are two good reasons for

considering theatre as the fundamental structure for thinking about anddesigning human-computer experiences Laurel states (Ibid. 21):

Theater is representing actions with multiple agents.

Theatre gives a basis for a model of human-computer activity that is

familiar, comprehensible, and evocative.

Laurel notes that several voices have been raised against the theatre

metaphor on the following grounds: It is entertainment and can

entertainment be serious? Will the theatre metaphor draw attention fromthe work to be done? Might the graphical approach be too decorative Laurel

asks? The graphics is an indispensable part of the representation itself.

Theatre gives good representation and the representation is all there is she

claims. This is not a quote but the essence of what Laurel writes on page 22.

In contrast to this, I would say that there is not representation of something

in some (maybe even most) computer games when it comes to the core of

subject matter of a game and its manifestations. Consider Pac-Man for

instance. The maze in Pac-Man making up the game board and the game

environment really is a maze. The hunt is a hunt. The actions performed are

performed. It is a real hunt, not a fake one. The way it is performed with a

Game Ego is in some respects different from running around being hunted

by ghosts in real life (if now there are ghosts in real life that is).

Nevertheless the experience is quite similar and uses to a large extent the

same motor based cognitive schemas. We do not need to use theatre to frame

this. Real life is enough since it provides the basis for understanding of our

being. The theatre metaphor in itself might divert us from the real issue, to

paraphrase Laurel: The issue is how do we get a feeling of being within anenvironment. What is the basis for our sensory awareness of being? John




136

Alexander, whom we soon will turn our attention to for a more elaborate

discussion on his ideas and what he calls ” viewer interaction” , would

probably reject the idea of Pac-Man not being a representation, a simulation

of something else. In his understanding, a game is make-believe, as are

fiction films, novels etc. A game of chess is like a battle but is not a battle, heclaims. (Alexander 1999, 29). Well, of course this depends on the meaning

one assigns the word battle. It has its conceptual basis in the same thing:

two concurring parties trying to defeat each other on a battle ground, and we

have this in chess. A way to solve this problem of real contra simulation

would be to use Lakoff and Johnson’s metaphor theory that makes up the

basis for their theory on cognition. The metaphorical structuring of mind

uses the ” real world” and maps this onto ” fiction” . I would not suggest that

we are not capable of discerning that what Alexander calls reality from that

what he calls fiction. Indeed, he himself uses the word metaphor to clarifythe relation between these two experiences. However, he does not seem use

metaphor in the way Lakoff and Johnson suggest i.e. in an embodied sense

as a neural structure.

To get back to Laurel, she finds that human-computer activities can be

divided into two categories:

1) productive (e.g. word processing. This is aimed at an outcome of some sort

in the real world).

2) experiential (e.g. computer games. This is aimed at the process of playing

the game).

Seriousness in both drama and human-computeractivities is a function of the subject and itstreatment in both formal and stylistic terms. Dramaprovides means for representing the whole spectrumof activity from the ridiculous to the sublime.

(Laurel 1993, 23).

I would suggest that there is often a level of social interaction in the

experience of playing a computer game. We do as Scheibe and Erwin have

shown, animate the computer (or the computer game system) when using it

and playing games on/in/with it. Laurel, in turn, stresses that multi-modal

activities do make us more prone to be sensorily immersed in a game or a

computer activity:

The linkage between visual, kinesthetic, andauditory modalities is the key to the sense of immersion that is created by many computer games,




137

simulations, and virtual-reality systems. (Laurel1993, 161).

Again, is theatre really needed to explain this? If we are to take

Münsterberg, Bordwell, and Branigan seriously, the human mind is in itself

the material that makes narration possible. The meaningfulness of life

comes not from theatre but from the structures, that handles our daily life.

Daily life consist of routines of different kinds. Taking part in drama and

theatre might be one such routine. However to make meaning out of drama

one must understand life. Drama and theatre do mirror life but its is life

itself, living, that makes drama possible in the first place. Drama and

theater may condense aspects of life and put forth salient schemas of how we

behave, that is true. But maybe theatre as a metaphor is a distraction when

trying to find the basis for being.

Having already made some comments on John Alexander’s work and what

he calls ” Viewer Interaction” it is now time for a closer study of his work.

7.4 John Alexander: audiovisual narrative and viewer interaction

First let me point out that Alexander’s doctoral dissertation Screen Play:

Audiovisual Narrative and Viewer Interaction makes some good points on

how what he calls viewers constructs narratives from audio visual material.

He does incorporate game theory in his work and shows the usefulness of such an approach in the study of audio visual media at large. Game theory,

film as play and viewer interaction are the keywords for John Alexander. To

my disappointment however, Alexander constantly use the word viewer in

his introduction to define the human recipient of the film event. It is used

extensively as to really make sure that the reader will not miss the fact that

is the experience of viewing that is of interest. Alexander even provides an

explanation for why he has chosen the word viewer rather than spectator,

observer, user, player and interactor:

The choice of ’viewer’ may seem at odds with theproposal that ’speculation’ is part of the viewer’splaying process – a word that shares the sameetymological root with spectator, yet, this is an issuecentral to the thesis – screen play consists of bothspeculation, but also the identification of/withdilemma and assimilation of insights – levels of participation better articulated through ’points of view’ than ’spectatorship’ or ’observation’.(Alexander 1999, 10. Alexander’s italics).




138

User and interactor are excluded on the grounds that are overstating the

role of psychological processes involved. Player is omitted since it ” may be

misconstrued by virtue of the breadth of the word’s applications” . (Alexander

1999, 10). Well, viewer is not narrow in that particular meaning since it

might be applied to a person both seeing and hearing stuff obviously. Hedoes not consider to use participant or ” constructor” or any such term that

could be used instead and besides this he completely misses the fact that he

deals with audiovisual narratives and hence in a way is himself overstating

one sense on behalf of all other. Which in turn is something that he

obviously is anxious not to do. Furthermore, he uses the construction

Audiovisual Narrative and Viewer Interaction… well think about it. What

does that imply on the term viewer if not a broad application of the term

viewer? Maybe this is not entirely his own misconception. He is in good

company when using such terminology, as I have put forth in chapter 3, andthe discussion on point of view.

Let us now leave this and instead be concerned with how Alexander uses the

term interaction. He recognizes that interaction is:

[…]the subjective process between game and player,between stimulation and story; interface is thecommon boundary between narrativecomprehension – the screen, the page, the gameboard. (Alexander 1999, 212).




139

According to Alexander the film viewer is a game player performing a screen

play since, as he writes:

[…] screen play consists of both speculation, but also

the identification of/with dilemma and assimilationof insights – levels of participation, betterarticulated through ” points of view” than” spectatorship” or ” observation” . (Ibid. 10).

That is to say: to have ’point of view’ is to be participating within the screen

play through the mere recognizing of situations within the story through

visual means. Speculation on the other hand is coupled to strategic thinking

and theories of game play. It is the active process of thinking ahead of the

story and trying to find out of things may work out.

Computers do not generate narratives, Alexander claims. Rather they

generate the pretext for the construction of narratives. (Ibid. 7). However, if

we are to follow the theses of Bordwell and Branigan so does any media. This

is, in other words, nothing that is a unique feature of the computer. It is also

somewhat misleading to say just that computers do not generate narratives.

A computer is nothing else than hardware. It is not until there is an

interaction process between a computer system consisting of

hardware/software and human that we will have something like a narrative

at some instance.

Alexander uses the term “optimum interactivity” to designate how a "user" is

allowed to do what he or she wants when he or she wants to do it. He is

questioning the interactivity of computer games since they are as he writes,

“after all, limited to the options provided by the software programme” .

(Alexander 1999, 7). Computer game interactivity can position a player with

a limitation of options, rather than an extension of options”. (Ibid. 8).

Optimum interactivity must as I understand it means that one can do

anything one wants at any time all the time without being endangered. All

environments will have constraints of one kind or another. This is maybe not

so fruitful after all. This means the unlimited and unconstrained exertion of

force of some kind or maybe even the realization of every possible thought

that can appear within the human mind. Optimum interactivity is

impossible for all beings (with the possible exception of God who is

almighty). To question the interactivity obtainable in computer games is of

no more value than to question our being within the “real world”. It would

probably be more fruitful to examine the limitations and constraints of interaction within computer games constructively. Be it in “real life” or in a




140

computer game you can only act location-wise anyhow. The place where you

are constrains your actions and your interactions with the environment as

J.J. Gibson’s ecological approach to visual perception shows. (See especially

Gibson 1986, chapter 8 which is an elaborate version of the article The

Theory of Affordances in Shaw and Bransford. Perceiving, Acting and Knowing). Moreover, as Lakoff and Johnson explain with the

ACTION–LOCATION metaphor, certain actions designate not only the

action but also the location where the action is performed.

Alexander furthermore describes the difference between computer games

and narrative film as the absence of character in computer games, an

impedant narrative trajectory, a random narrative trajectory and the

indeterminate or self-determined duration of play. This can be compared to

Narayanan’s paper Embodiment in Language Understanding (1996). According to Narayanan, comprehending a story corresponds to finding the

set of trajectories that satisfy the constraints of the story and that are

consistent with the domain knowledge. This may involve filling in missing

values for the abstract domain features, as well as inferring values for

unmentioned target features implied by the story. The most probable

trajectory can then be retrieved as the most likely explanation of the story.

Features with highly selective posterior distributions are likely to be present

in the recall of the story.

To conclude this discussion on Alexander’s work let me clarify that one good

and important side of it is his definition of narrative as game. This definition

diversifies the abstract narrative from form - and emphasizes the experience

of narrative and its subjective interpretation. Narrative is the subject for

narratology i.e. the study of text. Narrative experience on the other hand is

possible to study within the field of cognitive psychology, phenomenology,

reception studies and so on. (Cf. Alexander 1999, 35).

Let us now go on to an alternative approach to interactivity and interactionwith its basis in the experientialist theory of cognition.

7.5 Mark Johnson and George Lakoff: a subjective and experiential approach to interactivity and interaction

In the work of Mark Johnson and George Lakoff interaction is ever present

both implicit and explicit. The agency of the human being, i.e. our ability to

act within the world is a foundation for our concepts of interaction and

agency.




141

Let us first study how Mark Johnson uses the term interaction. Johnson

does not provide an explicit definition of interaction. However, one can

extract what interaction means in his understanding of the term by looking

closer at his text. He goes from an argument on how complex structures of

meaning is generated by image schematic structures of experience (discussedin chapter 5 of the present work) and arrives at “a second ever present

dimension of our experience, that of forceful interaction.” (Johnson 1990, 41).

Forceful interaction supplements the analyses of boundedness and the

container schemas with considerations of motion, directedness of action,

degree of intensity and structure of causal interaction. Johnson uses the

term interaction in explaining the pre-conceptual gestalt structures

characteristics for force (Ibid. 41-43). I.e. in his words ” their nature as

coherent, meaningful, unified wholes within our experience and cognition.

(Ibid. 41). To further ground how Johnson thinks about interaction considerthe following quotes:

In order to survive as organisms, we must interactwith our environment. All such causal interactionrequires the exertion of force, either as we act uponthe other objects, or as we are acted upon by them.Therefore, in our efforts at comprehending ourexperience, structures of force come to play a centralrole. Since our experience is held together byforceful activity, our web of meanings is connectedby the structures of such activity. (Ibid. 42).

This means that bodily motor action is of importance for our conceptual

structures and this we have taken part of in chapter 5. Furthermore, it

stresses that the experiences of interaction are experiences of force. To

continue this argument:

We easily forget that our bodies are clusters of forces

and that every event of which we are a part consists,minimally of forces in interaction. However, amoment’s reflection reveals that our daily reality isone massive series of forceful causal sequences. Wedo notice such forces when they are extraordinarystrong, or when they are not balanced off by otherforces. (Ibid. 42).

[…] force is always experienced through interaction.We become aware of force as it affects us or someobject in our perceptual field. (Ibid 43).




142

Note the use of the term "perceptual field" here. This concept is traceable to

Gibson and is a concept I use in the analyses below as describing the total

field of possible perception within game environment.

There is no schema for force that does not involveinteraction or potential interaction. (Ibid. 43).

[…] because we experience force via interaction,there is always a structure or sequence of causalityinvolved. (Ibid. 44).

To summarize this: Interaction for Johnson is the exertion of force up on us

by the environment or vice versa. Interaction is causal. It has a reason for its

exertion. Interaction is what holds our conceptual system together. Force

and interaction are tightly coupled since the experience of force is alwaysrelated to interaction or potential interaction. This is to say that interaction

must not always be acted out but may be conceived as a possibility. This

means that interactivity is the possibility for interaction embedded within

the causal situation.

In Women, Fire and Dangerous Things, Lakoff shows a similar

understanding of interaction. In relation to Eleanor Rosh’s work on basic-

level structures (see also paragraph 5.2 of the present work) he writes:

What determines basic level structure is a matter of correlations: the overall perceived PART–WHOLEstructure of an object correlates with our motorinteraction with that object and with the functionsof the parts (and our knowledge of those functions).It is important to realize that these are not purelyobjective and ” in the world” ; rather they have to dowith the world as we interact with it: as we perceiveit, imagine it, affect it with our bodies, and gainknowledge of it. (Lakoff 1987a, 50).

[…] the relevant notion of a property is notsomething objectively in the world of any being; it israther what we will refer to as an interactionalproperty–the result of our bodies and cognitiveapparatus. Such interactional properties formclusters in our experience, and prototype and basic-level structure can reflect such clusterings. (Lakoff 1987a, 51).




143

In Lakoff and Johnson’s co-written Metaphors We Live By we also find

interaction used in this way:

[…] the structure of our spatial concepts emergesfrom our constant spatial experience, that is, ourinteraction with the physical environment. (Lakoff,Johnson 1980, 56-57).

Newton’s theories on physics are close to the structures of how the mind

works in this model and way of understanding. How may we relate Newton’s

conception and understanding of the world to Johnson? Is it that the

language is common to them or is it that the underlying structures are very

similar, i.e. that Newtonian mechanics and even Euclidean geometry do

match the structures that organizes our minds? We do conceive of the world

as Newtonian mechanics rather than Einstein’s relative universe or asquantum mechanics. This has been noted also by Edward Branigan.

(Branigan 1997, 103). An alternative approach to how we understand the

surrounding world, or environment, is as has been shown, J. J Gibson’s

theory on ecological visual perception. However, it does not differ that much

in this specific respect to how we conceive of the world.

The work of Lakoff Johnson shows the basis on which we make our concepts

of interaction and agency. Interaction is agency i,e. the exertion of force.

Interaction is what an agent performs within an environment. In addition, itis what the environment performs on the agent. It is the mutual interchange

that is in focus. The interaction may be constrained or encouraged by the

surrounding environment in which the agent is due to the affordances and

constraints the layout reveals.

7.6 Summary

Having now come to terms with what a subjective experience understanding

of interaction would mean we can conclude this chapter on interaction withsome basic outline of this idea before taking on what I will call the Game

Ego.

Interaction is agency. That is, interaction is what an agent performs within

an environment. The environment and the agent – the animal performing

actions – are interdependent. The environment affords the agent to perform

certain actions and put constraints on other actions.

Interactivity is the quality that describes how the interaction is performed.Interaction always involves agency of some sort and interactivity is the term




144

to designate that interaction is possible. Jens F Jensen’s idea to use

interactivity to designate the process of mediated communication is perhaps

not the best way possible. To get more consistency in the use of language I

would suggest that we forget about the term interactive in constructions like

” interactive games” and instead use a term in a more Gibsonian fashion:interactable. That would stress that there is an affordance within the game,

that is interactable and that interacts with the game player. That is to say,

that there is something that is the essence of interaction, i.e. a basic

property that is immediately perceivable and that ends up in agent action

and environmental interaction.



THE GAME EGO

145

8 The Game Ego

In the following chapter I will argue and show that, when playing computer

games, we construct an abstract cognitive model of a Game Ego. The Game

Ego has its basis in the tactile motor/kinesthetic link between us as game

players and the controllable object within the game we are playing. The

Game Ego can be thought of as a container in accordance with the

experientialist theory of cognition. Our bodily container (see chapters 2 and

5) extends into the computer and can perform actions in the game via this

tactile motor/kinesthetic link. Tactility as such as part of the game interface,

is not yet as developed as it might be. What the game player can physically

touch differs from game to game and interface to interface. It might be a

keyboard, a mouse, a joystick, a steering wheel etc. but nevertheless: thegame player is physically connected to the action in the game environment

through this link. The player is able to activate, through cues from the

interface of the computer game, a simulation of a fictious world inside the

computer. The game player can also experience him- or herself outside the

game, sitting in front of the computer playing the game in a non-simulated

environment. A computer game player is conceptually a part of the game

system.

8.1

I dreamt I was Brigitte Bardot and that I kissed me: The Subject and the Self

Since a large part of this dissertation is concerned with Being, Point of Being

and the concept of a Game Ego, it is necessary to discuss the problem of The

self and the Subject, that do have its place within Lakoff’s and Johnson’s

thinking, as well as in my theoretical framework.46 The following paragraph

draws mainly on three sources: Lakoff’s Multiple Selves (1993) and Sorry I’m

Not Myself Today (1996). Also chapter 13, The Self, of Lakoff and Johnson’s

Philosophy in the Flesh (1999) is referred to. The latter text elaborates onwhat Lakoff noted in the two former ones. I consider this chapter of Lakoff

and Johnson’s book as a very important part of the framework suggested in

this dissertation.

46 The Self is also a central part of the theories put forth within social behaviorism and the lectures of George Herbert Mead (Cf. Mead 1934).




146

The Self is a troublesome concept of mind. As Lakoff and Johnson show in

chapter 13 of Philosophy in the Flesh, there is a conceptualization of a split

or bifurcated person in play here not fully consistent with the idea of an

embodied mind.

Lakoff suggest that to understand a sentence like “I dreamt I was Brigitte

Bardot and that I kissed me” (an example first put forth by Jim MacCawley)

one has to conceptualize a person as consisting of two parts. (Lakoff 1996,

93). There is a formal structure of such a sentence that is a bit confusing.

What is the reference of I, Me, and Myself? Lakoff notes that there has been

and still is a general assumption that these are first person pronouns that

all refer to the same person; the speaker of the sentence. However, this can

not be the case Lakoff claims. For the analysis of the sentence “I dreamt I

was Brigitte Bardot and that I kissed me” Lakoff uses the terminology of Andrew Lakoff and Miles Becker (Lakoff, Becker 1991). The Subject, George

Lakoff understands as being the locus of subjective experience i.e.

consciousness, perception, judgement, will and capacity to feel. The Self is in

Lakoff’s discussion outlined as being the body. However, this is just in the

first part of the discussion and is later elaborated. A sentence like the one

cited is violating the condition that first-person pronouns always refer to the

speaker of the sentence.47 The Subject (consciousness etc.) of the speaker is

within the other person’s Self that is, within the other person’s body. As

George Lakoff explains it:

“ I is conceptualized as split into two parts: call them Subject-of-I and Self-of-I . Similarly you areconceptualized as split into Subject-of-you and Self-of-you. (Lakoff 1996, 93).

The Subject-of-I occupies the locus of Self-of-you. To make it a little simpler:

The body is a container for the Subject. The container is the Self. (Ibid. 103).

The container has the physical characteristics of a person, it social roles etc.Fauconnier’s theory of mental spaces is needed to express the meaning of

sentences that go beyond formal logic, such as the sentence exemplified, that

suggest a Subject/Self split. (Ibid. 98). 48

It is now time for a comment that relates this to the present work. While

playing computer games a game player is in a state of play and if the game

contains a Game Ego of some sort we will find a similar Subject/Self split.

47

This is what logic and formal linguistics assume. Lakoff comments on this and admits that he is one of the scholars who advocated this assumption during the 1960s. (Lakoff 1996)48 See also the discussion in chapter 5 of the present work on the integrated theory of metaphor.



THE GAME EGO

147

There are two (conceptual) metaphors that can describe this. The Divided

Person metaphor and the Projected-Subject metaphor described below:

A person is an ensemble (containing one person, the Subject, and at least one

other entity, a Self).

The experiencing consciousness is the Subject.

The bodily and functional aspects of a person constitutes a Self.

The relationship between Subject and Self is spatial: the Subject is normally

either inside, in possession of, or above, the Self. (Ibid. 102).

The Subject cannot act within the environment without the Self. It can

reason but not act. The Self can act directly in the environment. The Self isacting and the Subject is reasoning.

This is the core of a Game Ego. There is one part acting and one part making

sense of the actions taken. To get deeper into this let us now turn our

attention to what Lakoff and Johnson calls The Objective-Subject metaphor.

In this structure, the Self is a Container for the Subject. Being subjective is

staying inside the Self. Being objective is going outside the Self.

Source Domain Knowledge: When one is inside a container one cannot see

the outside of the container, the part that others see. Only when one is

outside the container can one see it. One is normally inside, and going

outside takes more effort and more control than being inside.

Target Domain Knowledge: When one is being subjective, one can not know

the Self that others know. Only when one is being objective, can one know

oneself as others do. One is normally subjective, and being objective takes

more effort and more control than being subjective. (Lakoff, Johnson 1999,

103 and 275).

One may also loose one’s Self. The Self is a possession of the Subject. Control

of Self by Subject is possession. Loss of control is loss of possession.

The source domain knowledge for this is: If a possession of yours is taken,

then you no longer have it.

The target domain knowledge for this is: If something takes control of you,

you no longer have control.




148

Since this is study of playing computer games and how to apply a theoretical

framework of cognitive theory to this game playing, (or to describe game

playing with the help of insights into experientialist cognitive theory

concerned with the human conceptual system) we have at least two main

areas of research: the game play as such, i.e. the body/mind state of gameplaying and secondly the game player’s description of game playing. What I

am suggesting is that while playing computer games one does project ones

Subject onto a Game Ego that is another Self and/or an extension of one’s

own Self. As Lakoff and Johnson suggest, we can have one or more selves.

The strength of the other Self, i.e. its integrity is interesting. How “strong” is

the Game Ego and why is it that we do conceive of the Game Ego as a part of

one’s Self? Playing computer games is an act of subjective performative

interaction. It is a subjective experience. As Lakoff puts it: ” The Self acts in

the world.” (Lakoff 1996, 110). Furthermore: ” The Self is the object otherpeople see while looking at us from our outside. We do not have access to our

outside, to our self easily” . (Ibid. 102). I suggest that a Game Ego provides

such an access to a Self.

We also show traces of a conception of a split bifurcated person giving a

reflexive statement about a game session. One really is the Game Ego to

some respect since ones motor action is highly involved in the game play, i.e.

in controlling the Game Ego and trying to master the game/game

environment. The mental space for this projection, the possible space of thegame environment, is where the Game Ego resides and acts.

8.2 The Self and the identity: Subjective Experience

The Subject/Self distinction is also discussed and elaborated in Lakoff and

Johnson’s Philosophy in the Flesh (267-289). There is not one such

Subject/Self distinction but many. There is no possibility to reduce them to

one consistent conception of Subject and Self. The many different

distinctions are not even consistent. However, they do express universalexperiences and do form universal metaphors that in turn seem unavoidable,

which are grounded in other universal experiences. (Lakoff, Johnson 1999,

268). This means that there is a cross-domain mapping going on. Lakoff and

Johnson points out the obvious: there is a contradiction between how we

conceive of the Subject and Self and the findings of cognitive science.

However, the conception of the Subject is not embodied within the Divided-

Person metaphor. The Subject within the Self has an existence in our

conceptualization of it as independent of the Self (of the body). This is

inconsistent with what we know about the human mind. Moreover, we use



THE GAME EGO

149

this distinction every time we make an utterance like “I was not myself!” and

such.

What is then a Subjective Experience? In the integrated theory of metaphor

Lakoff and Johnson claim that, drawing on the work of C. Johnson, Grady,Narayanan, Fauconnier and Turner, the answer is quite simple:

sensorimotor experiences are what structure subjective experiences. This

means that sensorimotor experiences are the source domain and the

subjective experience is the target domain. This is only partly coherent with

the Subject/Self distinction that they describe (Lakoff, Johnson 1999, chapter

13). The Subject is normally residing within the Self. The Self is that what

acts in and upon the world. However, the Subject is conceived of as being

independent of the Self (the body). The conception of the Subject-Self is a

part of the cognitive unconscious.

There is, writes Lakoff and Johnson, a single, general metaphor schema that

all subject-Self metaphors relate to and of which they are special cases.

(Lakoff, Johnson 1999, 269). There is a basic structure to the Subject-Self

system of metaphor. It is grounded in four types of everyday experiences:

manipulating objects,

being located in space,

entering into social relations, and

empathic projection-conceptually projecting one’s Self onto someone else, as

when a child imitates a parent.

Lakoff and Johnson notice that there is also the notion founded in the folk

theory of essence concerning the real Self. The real Self is the Self among all

other selves a whole person may have that really is the essence of that

person. There is only one Self compatible with the essence and that is thetrue or real Self. (Lakoff, Johnson 1999, 269). This gives to show that the

Subject-Self metaphorical system in its general outline is a description of

what computer game playing is about. When playing computer games, we

are manipulating objects and through this we also get a feeling of being

located in space (that also has ACTION–LOCATION metaphorical

structures. See the discussion in paragraph 5.3.3 of the present work and

Lakoff, Johnson 1999, 204-205). We are entering social relations and project

the Self onto someone else through the tactile motor/kinesthetic link to the

Game Ego.




150

8.2.1 The Subject and The Self/Selves

The Subject in Lakoff and Johnson’s theory is “that aspect of a person that is

the experiencing consciousness and the locus of reason, will and judgement,

which by its nature, exists only in the present.” (Lakoff, Johnson 1999, 269).

The Self is that part of a person that is not pickedout by the Subject. This includes the body, socialroles, past states, and actions in the world. Therecan be more than one Self. And each Self isconceptualized metaphorically as either a person, anobject, or a location. (Lakoff, Johnson 1999, 269).

In this context, it is important to note that a Game Ego is primarily a

function. It need not necessarily be manifest within the visual container of

the game or be within the sound-scape of the game. Rather, it is a way of controlling the visual container and its content as well as the sound-scape.

That is to say that the Game Ego is not necessarily a fully visible character

in the game. It is that part of a game that allows and exerts action in the

game environment. In games like Tetris, which are often for some strange

reason called ” abstract games” although they are very concrete in their

objectives and their manifestations of manipulable objects, this function of

manipulation is important. There is an agency in the game which is not

visible but still the result of the actions taken are definitely visible. There is

a definite and clear presence of an agent in the game.

The Game Ego is the agency that exerts force upon the game environment

and that the game environment exerts its force upon. As also Joseph D.

Anderson notes, the proprioceptive system of our human body/mind is

always at work.

Knowing where we are is so basic to our survival, sofundamental to our perception, that we are

uncomfortable with disorientation, and we panicwhen we are lost. The senses, especially vision andaudition, work together with proprioceptive systemsto provide veridical perception of where we are at alltimes. (Anderson 1996,112).

We can make a link here to the work of John Alexander. If we accept that

Anderson is right in claiming the importance of our proprioceptive systems,

this will have consequences for how our being within an audiviusal

environment such as a movie or computer game is established. To look and

not blink is the essence of Hitchcock’s movie ROPE (1948) Alexander



THE GAME EGO

151

suggests. (Alexander 1999, 131). He compares this movie and its unusually

long takes and masked edit points to the average take of a commercial film

that is about ten seconds (Ibid. 130). Well maybe this is so statistically.

However, a statistical take does not necessarily exist in the real world. It is

nothing more than an average. It only means that if you count all takes in amovie and divide the number of takes with the runtime of the film you end

up with a value designating nothing more than a mathematical process. It

has really little to do with the film material analyzed. This is of course a

” point of view” based assumption of movies. The camera in ROPE is used as a

hypothetical- (Branigan) or an invisible- (Bordwell) observer i.e. an un-

noticed character in the movie. Hitchcock did this movie as a stunt. He did

however maintain his usual rhythm and pacing, he claims. (Truffaut 1968,

135-136. Also quoted in Alexander 1999,130). However, in a way, computer

games like Unreal perform the same stunt then. Not so that the observingand participating character goes unnoticed (even if it is an option since such

games often allows a guest mode in which one just observes the game play

without affecting it). We do not conceive of this as being the least strange or

troublesome. It is part of the game environment construction and the point

of being within the game in interaction with this environment. Our visual

experience is more like a game of Unreal Tournament than Hitchcock’s ROPE

or Tarkovsky’s STALKER (1979) for that matter, in that manner that we are

able to shift our field of view very rapidly, from left to right for instance, by

turning our heads or moving our eyes. Since the Game Ego is really a part of

the game player in a PART–WHOLE configuration, the blinking of the game

player's eyes are enough. There is no need for editing here. One could argue

that the use of warp zones in the game environment is like an edit and so it

is. It is a question of relocation often combined with a sound that

corresponds to and enhances the feeling of relocation.49 We change places

when using warp zones. However, this is a known feature of this kind of

games. To further ground this we can consider Gibson and his remarks on

the field of view. (Gibson 1986, 111-112). Our noses stick into this field andare an integral part of it. We do not need to blink with the camera since we

blink ourselves. We do not need to put a nose into the field of view of the

movie screen (or computer screen for that matter) since we already have one

in our field of view. When playing computer games, we try to find our

position within the game environment. We can put the above in relation how

49 This is obvious in Tarkovsky's Stalker. When entering the heavily guarded Zone there is a sound

composition that clearly marks this event as a shift of location. The sound of a railway trolley is slowlymixed with echoing sounds of musical instruments. The rhythms of the sounds merge and manifests thefeeling of change of location. See also my analysis of Myst in chapter 9.




152

Mark Johnson understands interaction and that we have touched upon in

the previous chapter:

We have bodies that are acted upon by ’external’ and’internal’ forces such as gravity, light, heat, wind,bodily processes, and the obtrusion of other physicalobjects. Such interactions constitute our firstencounters with forces, and they reveal patternedrecurring relations between ourselves and ourenvironment. Such patterns develop as measuringstructures through which our world begins to exhibita measure of coherence, regularity, andintelligibility.

Soon we begin to realize that we, too, can be sources

of force on our bodies and other objects outside us.We learn to move our bodies and to manipulateobjects such that we are the centers of force. Aboveall, we develop patterns for interacting forcefullywith our environment–we grab toys, raise the cup toour lips, pull our bodies through space. Weencounter obstacles that exert force on us, and wefind that we can exert force in going around, over, orthrough those objects that resist us. Sometimes weare frustrated, defeated, and impotent in ourforceful action. Other times we are powerful andsuccessful. Slowly we expand the meaning of ’force’.In each of these motor activities there are repeatablepatterns that come to identify that particularforceful action. These patterns are embodied andgive coherent, meaningful structure to our physicalexperience at a preconceptual level, though we areeventually taught names for at least some of thesepatterns, and can discuss them in the abstract. Of course, we formulate a concept of ’force’, which wecan explicate in propositional terms. But itsmeaning–the meaning it identifies–goes deeper thanour conceptual and propositional understanding.(Johnson 1990, 13. Johnson’s italics).

From this quote, we get a further elaboration on how metaphorical

structures map our interaction with an environment onto more abstract

concepts. As noted in chapter 5 (on Conceptual Blending, 79), pressing a

specific button while playing a computer game means more than just

pressing a button. It may designate a command like ” Walk” , ” Run” , ” LookLeft” , ” Fire Weapon” , ” Jump” etc. This, the exertion of force, leads us into the



THE GAME EGO

153

core question on how this is actually performed in computer games i.e. into

the question of controllability, which is the topic of the next paragraph.

8.3 Controllability I: Film interface versus computer game interface

What follows is a discussion on how film and computer games

“communicate” with their users regarding the production and presentation of

space and the identification processes.

8.3.1 Film interface

That a computer environment has an interface between itself and the human

mind is a well-accepted and working model of the relation between human

computer users and computer systems. That a film has an interface is

perhaps not so obvious and the concept of interface in not so commonly usedwhen analyzing film. Joseph D. Anderson uses the term interface in his The

Reality of Illusion (161). However, it is not elaborated as a useful term. Also

John Alexander uses the term interface to describe the boundary between a

film narrative and a viewer. In Alexander’s theory these are the game and

the player respectively, although game can also be “narration itself”

(Alexander 1999, 35 and 212). Alexander in turn draws on the work of John

Barry who notes that an interface in the most general sense is a “link” or a

“connection”. (Alexander 1999, 212).

However, I do believe that if we used the term interface more when

discussing film and film theory we would gain some insights to film as a

system. We would gain a deeper understanding of how a film considered as

medium, creates the raw material for spatial understanding of the diegetic

world, how characters within this world are construed by the perceiver etc.

A film has an interface between itself and the human mind. It consists of

sounds and images organized linearly in time. The events and objects seen

and heard do not need to be linear or causal within the narrative, story, orcontinuum of the film but they are presented in a linear mode i.e. nothing

the film perceiver does can affect the structuring of events within the

medium. (Cf. Grodal’s Rovaniemi lecture notes where he remarks that the

linear structure is not obvious in the unique one-time viewing situation).

There are established conventions of how time is represented in a film e.g. a

sequence in black and white with out-of-focus boundaries and a voice-over

could be a representation of a recollection of a diegetic character. Film works

mainly towards our perception of representative sounds and representative

images. It does so in a mode that makes the film perceiver passive in




154

meeting time i.e. the time metaphor of the film interface is time (like a

moving object) passing a non-moving perceiver while many computer games

have an interface where time is moving as well as the Game Ego is passing

through time. 50 Some things will not happen if the game player does not

execute certain actions. That is: actions turn into events of the game. Eventshave duration over time. Time can be the main organizing factor in both film

and computer games. (Cf. chapter 5, paragraphs 5.4.4.1to 5.4.4.3).

The film interface is a transmittative one. It communicates one-way and

shuts the film perceiver out of possible actions that could affect the way in

which the linear presentation of sounds and images are made.

Important to the theory on cognition, emotion and visual fiction put forth by

Torben Grodal is that film can be thought of as a linear emotionalmanipulation. (Grodal 1994 and 1999b). We take part in a film because we

want to be emotionally affected in one way or another. When we play a game

we have the goal to win the game i.e. to ”be number one” and conquer the

other players, who in the case of computer games can be humans or a

computer application of some algorithms i.e some kind of agency acting upon

us. This, the goal to win, is also a kind of emotional state. I am definitely not

suggesting that we are passive receivers when taking part in a movie. We

must perform acts of decoding, understanding and making meaning of the

system of information that constitutes the movie. We are involved in themovie and are able to relate things we hear and see to things of which we

have bodily and social experience, and build our understanding of the movie

on these relationships. This is part of the cognitive unconscious. We are not

aware of all these processes but our mind is performing them nevertheless.

As Grodal points out, we can, when we see coffee drinking on the screen,

relate this to our own experience of coffee drinking and simulate a feeling of

a coffee cup in our hands, the taste of coffee in our mouth, the smell of coffee

in our nose etc. However, this is not simply a choice we make. Coffee

drinking on the screen may activate a neural cluster of the experience of coffee drinking. However, not necessarily. And not voluntarily. (Grodal

1999b, 22).

In addition to this, we may even want to interact motorically with the movie

but whatever we do will have no effect in the pre-organized story line, the

syuzhet, within the movie. It could and will, however, affect the way we

interpret the story line.

50 The time metaphor is discussed, explained and exemplified in Lakoff 1993, 14-16. It is also discussed atlength in Lakoff, Johnson 1999, chapter 10 i.e. 137-169).



THE GAME EGO

155

8.3.2 Computer game interface

Let us now broaden the idea of an interface and understand it as part of an

environment. Gibson’s three elements of an environment (substance,

medium and surface) again prove to be useful. We then have a model of

understanding consisting of a primary surface for actions taken. It will get a

little troublesome, though, to apply this to a computer system. However, the

surface is the interface that not separates but connects the user and the

computer so as to make the user and computer a system in a PART–WHOLE

configuration.

When the user is able to control the motor schema of a character or any

object from some kind of input device, keyboard, joystick, glove etc., we are

dealing with what could be referred to as a game just because of this motor

aspect. The level and kind of motor control is what matters the most, not theaudiovisual presentation as such concerning color depth, technical sound

quality etc. If the game player is given motor control of a character’s or an

object’s motion, he or she is more likely to identify with that character or

object. The player’s own motor actions are extended into the game and cause

effects that in turn are looped back to the player and could be a new cause

for action. See the figure below.

Film

Stimuli

Emotional/motor response

without effects on filmic

events

Computergame

PlayerPerceiver

Stimuli

Emotional/motorresponse with

possible effect onthe game

Tactile motor/

kinesthetic

link

Figure 1: Film- and computer game interface

This figure is a simplified illustration of a stimuli/response system. I do not

however advocate a stimuli/response way of thinking. This kind of stimuli/response thinking is dualistic. As an illustration of the basic




156

difference between film and computer games it works well however. It is also

very similar to the way Artificial Intelligence is often studied. See the

illustration below which is taken from Russel and Norvig’s Artificial

Intelligence: A Modern Approach (1995, 32).

Figure 2: Russel and Norvig’s AI schema

A computer game can use not only the film interface way of identification

based on point of view thinking but also a very different one through tactile

motor interaction. Whereas film is bound to use camera angles, framing,

point of view, dialogue, point of audition etc. for the identification, a

computer game builds the identification from the point of being. Being

requires a location to be at which leads us to the next part of this study,namely space in film and computer games.

8.3.2.1 Space in computer games and film

Gibson makes it clear that the ecological environment differs from a physical

description of a world. A surface has a geometrical counterpart in what is

know as a plane. A plane, however, has no color, but a surface has. This is an

example of Gibson’s claim that an ecological environment is meaningful to a

living animal. Space as a physical term is not meaningful in his

understanding of the concept. However, I will in the following discussion use

the term space to designate the locations in which actions are performed.

Film presents a static, fixed diegetic environment, in the sense that the

person taking part in the film can not act upon it and make changes in it;

the motor actions of the film perceiver do not make any difference to the film

environment. Computer game environments, compared to film, can be (and

often are) very different. It may look like a diegetic film environment but it

is of another order. There is a big difference in the presentation of space in

film and three dimensional game environments such as those in Marathon



THE GAME EGO

157

(Bungie 1995), Quake (Id Software 1996) and Duke Nukem 3D. 51

Technically, there is a difference in that the game player can relatively, or

even very freely, explore the space at hand. A game like Myst with its fixed

static frames has more in common with a film understood from its technique

of presentation of the field of view. In a film the choices of what is shown isalready made. In a computer game, the game environment is only a

possibility waiting to be explored. It consists of possible points of observation

to put it in Gibson’s terminology. Of course this game environment is

organized and made but what is seen and heard of is very much up to the

player to discover. Some games, like Myst or Riven have spaces that are not

absolutely necessary to find but still are there as possible places to explore

and places to make certain (for the game not absolutely necessary) actions.

In a game, this is natural but in a (mainstream Hollywood) film, it would

probably be conceived as strange. It is not that common within film to havesounds and images that are not necessary for the story or emotional

manipulation one way or another. Even so-called ”transport scenes” are

placed in the narrative for some reason. I find that the extent of

”unnecessary” places of being are used more in games because that doing so

strengthens the game. If just the necessary places were in the game

environment, the game would probably be boring. The ”unnecessary” places

make the world bigger and could also function as a way of telling the game

player how this game environment works. Hence, they are not truly

redundant but very much a part of the game. Older games, like those

adventure games for the Nintendo Entertainment System, e.g. Zelda II: The

Adventure of Link (Nintendo 1987) have few spaces of this kind. They are

more like film in this respect. And maybe for good reasons; neither the

graphics nor the sound is of high technical quality and it would be disturbing

to be forced to watch and hear too much that is not found indispensable. (Cf.

Gibson’s discussion on potentially visible surfaces in The Ecological

Approach to Visual Perception, 23. See also Paul Mayer’s doctoral

dissertation A Social Semiotic approach to the analysis of computer media,the chapter on Myst and his study of the frames).

8.3.2.2 Visual and sonic fields

Let me first point out that the use of sound might in some respects, be

different to the use of sound in film. Since the objective in a computer game

is to act it out, there is often a very close relationship between the status of

the Game Ego and the sounds of the game as shown in chapter 6 of the

present work.

51 Duke Nukem is not a fully 3D environment. The game characters are actually 2 dimensional.




158

Visual and sonic space could be divided into fields. Lakoff and Johnson point

out:

We conceptualize our visual field as a container andconceptualize what we see as being inside it. (Lakoff,Johnson 1980, 30).

However, we are not concerned solely with visual fields or field of view to use

Gibson’s term, in this paper. The fields of interest are audiovisual. I have

consequently found it suitable to add to the set of fields some definitions for

practical reasons.

1) The screen field. The screen field is what is actually seen on the screen

and sounds caused by objects on the screen at any given moment. Some

games only have this kind of field. Since the whole visible game environmentis complete within the field, it is not necessary to think about or imagine any

other space. The screen field is mainly a visual field since the objects to be

manipulated or fought off need to be seen in order to take action. However,

the sounds produced when doing so are likely to have a tight coupling to the

visual events and sometimes even take on a determinative function in that

they are the only clue to the state of game play. In other words, the sound

will be telling the game player how to plan the game play and when it is

time to change the strategy of game play. For instance, to some extent Pac-

Man uses sound in this way as does Elk Attack (M. Hahn 1987). This gamefor the Atari 2600 is basically a version of Pac-Man and sound is used in a

somewhat ” smarter” way as my analysis in chapter 9 will show. This tight

coupling of sound and visuals is different from mainstream film since the

game player must plan his or her strategy and be more aware of the

audiovisual world to be able to continue game play. The movie will go on and

on, no matter where the perceiver’s attention is directed.

Screen field or screen space is somewhat problematic to deal with when it

comes to sound. Michel Chion writes about the audiovisual scene in chapter4 of Audio-Vision (66-94).

[…]there is no auditory container for film sounds,nothing analogous too this visual container of theimages that is the frame. (Chion 1990, 68).

To elaborate on this, on-screen or off-screen sound is troublesome if we

consider a game like Robotron 2084. There is a limited field of view that is

framed. There are borders that one can not transcend or go beyond making

up a rectangle. This rectangle is the game board, the visible part of the game



THE GAME EGO

159

environment. This is the only space in which a game player may control the

Game Ego. There are no other such spaces in this game. It is not even

suggested that there are such spaces possibly apart from the use of pit-music

to use the terminology of Michel Chion (Chion 1994, 80) i.e. non-diegetic

music or non-source music stemming from a place not visible on the screenand being outside the time and space of the action).52

2) The surrounding field. The surrounding field above, below and to the left

and right of the screen. In some games these fields are never shown but still

exist. E.g. shoot–'em–up games in which objects come into the visual field

from an invisible field. One could imagine a possible space beyond the visual

field shown. The surrounding field may encourage motor action and

stimulate moving in space to make new parts of the game environment

visible. A classic example of such a game is Defender (Williams 1980). Thesurrounding field is a dynamic and relative field constantly under flux. It is

often a possible field to enter into. One can make a comparison to the

relative and absolute concealed or hidden spaces that is described by Gilles

Deleuze (Deleuze 1992a and Deleuze 1992b) and that is commented upon by

Astrid Söderbergh Widding (Söderbergh Widding 1992).

3) The field behind the Game Ego. This field is the opposite of the visible

screen in games with a first person point of view and is to be considered a

special case. This kind of field is commonly used e.g. in three dimensionalaction games like Doom (Id Software 1989). Duke Nukem 3D (3d Realms

1995), and Quake (Id Software 1996). This field is present via audio when

playing the game. It is often connected to panning surround sound i.e. the

sound is tied to its (visual) origin within any field. It simulates reality. The

field behind the Game Ego is often an active field with sounds telling the

game player that there is something behind that might be dangerous.

Objects coming into the screen field from a point behind the Game Ego can

also introduce it. If the Game Ego focuses on actions in front of it, this makes

it a ”passive” target for others. The field behind the Game Ego also triggersmotor action and stimulates moving in space to make new parts of the game

environment visible.

The next paragraph will be a study of different types of object control within

computer games since the type of control is essential for how a Game Ego is

constituted. The visual and sonic fields will be dealt with further in

connection to this.

52 Of course, one may speculate on the existence of another location since the shift between levels of thegame uses a "warp zone" effect. Where are we when not being within the game environment so to speak.




160

8.4 Controllability II: Controlling the Game Ego

The following part of this chapter will be concerned with the issue of

controllability of the controllable objects in computer games. Controllability

is, as we have seen, essential to the theory proposed. It is so because of my

conviction that motor activity and cognition add up to a Point of Being – a

way of perceiving oneself as Being within the game environment.

The concepts used by mind in playing computer games are results of our

bodily constitution and our culture. Furthermore, they manifest computer

games as audiovisuo-motor experiences.

Loftus and Loftus argue that cognition and motor activity are independent of

one another (Loftus, Loftus 1983, 66-69). The essence of their argument goes

like this: If you have learned a motor activity (a motor skill in theirterminology) and you then are unable to explain how you perform this task

then it is beyond cognition. Thus, cognition in this case is tightly connected

to verbal activity and the ability to communicate something with words in

their way of thinking. I can not agree with them on this. A lot has changed

since they wrote their book in 1983. Lakoff and Johnson for instance, put

forth the argument for the cognitive unconscious in Philosophy in the Flesh.

Cognition is not only what can be verbally expressed and/or explained. It is

much more than the communicable aspects of the human mind.

8.4.1 Types of control: Direct or indirect control

A computer game is at the level of hardware/software possible to describe as

a physical system consisting of various parts needed to play the game. Most

computer games have a von Neuman architecture i.e. a typical set up for a

computer game system would be a CPU, a monitor and speakers, a control

device, storage device and the software. The software in turn may be

distributed in various forms, on CD-ROM, DVD, network or other media.

Control devices may take on many different forms. What a game player

desires (if we are considering rapid action games) is immediate responses to

his or her actions, much like the mechanical pinball machines provide. These

responses create a sense of direct physical contact. (See also chapter 7 of the

present work and my discussion on the mechanical patterns for interaction

with the environment in paragraph 7.1.1). Atari did try to introduce a very

unusual accessory game control device for the Atari 2600 in 1983. It was

called the Atari Mindlink controller and consisted of a headband, which had

infrared sensors. The Mindlink was able of detecting the muscle impulseswhen a game player moved his or hers eyebrows and using these movements



THE GAME EGO

161

to control a game. Hence, you had to put in a lot of effort to obtain this which

mainly resulted in headaches in those who tried it. It was not a great success

and was never released as a commercial product.

(http://www.classicgaming.com/museum/2600.shtml). It could however have

been used in a different way. Instead of controlling the controllable object(the Game Ego) it could have been a way of giving feedback to the game and

translating the facial expression of the game player to some degree as a way

of communicating the emotions of the game player to the game. But this was

probably completely beyond the scope of the games of 1983. However, today:

why not try it or some other way to achieve this. Maybe by a camera and a

kinesthetic-recognizing application? Alternatively, why not by vocal input to

the game revealing the game player’s attitude and emotional status? Or

simply by using sound amplitude as a way of feedback? It is easy to image a

hypothetical game that uses this. Let us say we have this hypothetical gamecalled Ornithologist. The main goal with this application as a game is to

collect a huge amount of rare shy birds on film. You have to choose a camera,

a hiding place etc. You must also be very quiet during your bird watching. If

you speak the birds will fly and you will not get you desired photograph. One

could even use a little tiny web camera to detect your movements in front of

the computer. One could also imagine a game that allows for loud noises,

body movements, facial expressions and even encourages these to

successfully play the game. We could have a game in which the computerized

opponents would get scared if we were to shout at them. We can call it

something like Scare the Bear. The more you holler the more frightened the

bear will be – maybe. The bear might also eat you. It is just an illustrative

example in which I attempt to show what is really possible to do.

8.4.2 Games with indirect control

Indirect control of a character means that the game player’s motor action is

not immediately transferred into an action by the controllable character

within the game environment (i.e. the Game Ego). Mostly, when it comes toindirect control, there is some kind of point and click interface that is used to

control the character. This type of interface is the most common in narrative

adventure and mystery games and is found in games like Secret on Monkey

Island, Sam & Max Hit the Road (LucasArts 1993), Day of the Tentacle (see

analysis paragraph 9.9), Full Throttle, Phantasmagoria, The Daedalus

Encounter (Mechadeus 1995) etc. One can use keyboard control as well in

some of these games but only in a limited way.53 Not surprisingly, this group

53 In Full Throttle there are some passages of the game where you will have to use more direct keyboardcontrol (The Old Mine Road sequence and the Demolition Derby).




162

is the one, which shows the closest similarities with narrative film. Game

events i.e. manipulation actions, motion control, dialogue control etc. are

intersected with film events that lie beyond the game player’s control. They

are literally out of reach for the game player.

This type of interface delays the action and generates a weak motor link

between the player and the Game Ego. The game player is given time to

think between action of his or her own and the reaction (the resulting

output) from the game. This type of game interface is not focused on a tactile

motor/kinesthetic link but more on conventional film narration with the

game player triggering game events by his or her actions. It is comparable to

a big puzzle or mystery, which the game player have to solve, step by step, in

order to reach a final solution54 that might be fixed i.e. the game always end

the same way; or non-fixed i.e. there is more than one ending for the game.55

The definite order of events is not fixed: only suggested.

8.4.3 Games with direct control

Action, shoot–'em–up games, and games related to the simulation of vehicle

control are all most commonly built on the direct control of a controllable

object. This object can be a character, a weapon, a spaceship, a car etc. It is

not necessary that the object is a representation of some kind of organic

living being. It becomes animated, that is gets its liveliness, through itscontrollable mobility and the tactile motor/kinesthetic link. The link between

game player and controllable object is a LINK Schema in a PART–WHOLE

CONFIGURATION. Action in itself is the stuff that interfaces both enables

and represent.’

Brenda Laurel notes that the concept of Direct manipulation interface was

coined by Ben Shneiderman at the University of Maryland. It has three

criteria:

- Continuos representation of the object of interest.

- Physical actions or labeled button presses instead of complex syntax.

- Rapid incremental reversible operations whose impact on the object of

interest is immediately visible. (Laurel 1993, 8).

54 That is what may be called a ”correct solution” not including the untimely death of the Game Ego55 A game with different preprogrammed endings is for example Myst.



THE GAME EGO

163

Direct engagement is a feeling that occurs when a user experiences direct

interaction with the objects in a domain. This is the functionality of the

Game Ego. The qualities of the action and the quality of the subjective

response provide the feeling of direct engagement. What is then required for

to produce a feeling of taking action in a representational world? BrendaLaurel states: Continuous representation, ” physical” action and apparent

instantaneity of response is the very essence of directness. (Laurel 1993, 9).

Scnhedierman and Hutchin’s analysis stresses the graphical representation,

Laurel notes. Sutherland’s Sketchpad application from 1963 is named the

granddaddy of direct manipulation.

Graphical (and, by extension multisensory)representation are fundamental to both the physicaland emotional aspects of directness in interaction.(Laurel 1993, 9).

Laurel compares the graphical designer with a scene designer. They both

make the place for actions to take place. They both use metaphors. But it

takes characters as well. The set does not make the story.

The concept of interface is criticized by Laurel. Her argument is that the

simplistic definition of an interface is how humans and computers interact

but that it is avoiding the central issue of what this means in terms of realityand representation. The drama, as in the computer application, is meant to

be acted out. It gives a performance. When a play works the technique

behind is not of interest for the audience. It is the same with computer

applications that work. Theatre audiences also affect the actors and interact

with them and the play performed. They are not passive. In fact they are not

even an audience if they enter the stage. They are actors.

Games of direct manipulation are, for instance, Pong, Space Invaders, Duke

Nukem 3D, Quake, Doom, Marathon, F18 Hornet, Pac-Man, amonghundreds of others. There is a great scope of genres and types but they all

have this directness in common. As shown in the figure below, direct control

is to be found in many prototypes of games. (This table is my elaboration of a

table found in Torben Grodal’s Video games and emotional control, paper,

presented at the Mediapsychological Conference ’Only Entertainment’, in

Hannover 8-9 May 1998).

Table 1: Grodal’s game prototypes extended with type of control




164

Prototype Game example Type of control

1) Computer simulatedpuzzles and games

Crystal Calliburn (Pin Ballsimulator)

Chess

Direct

Indirect

2) World scenariosimulations

Civilization Combination

3a) 2-dimensionalgames related tocharacter/vehiclecontrol

Pong, Space Invaders

Pac-Man

Direct

3b) 3-dimensionalgames related to vehiclecontrol

F18 Hornet Direct

4a) Narrativesimulations sub group1. 2-dimensional.

Day of the Tentacle Indirect

4b) Narrativesimulations sub group2. 3-dimensional.

Duke Nukem 3D Direct

8.4.3.1

Consequences of direct and indirect control

How actions are performed is important when concerned with the question of

how we identify with a Game Ego. Can an ”Ego” or an ”I” be considered as an

abstract cognitive model derived from a bodily experience? I find the answer

to be yes. An ”I” or an ”Ego” is a construction of the human mind. An Ego

does not exist outside the mind in the external world. Objectivist cognitive

theory would have problems explaining the Ego.

Control of objects and movement connects to the perception of space. Theability to move around in an environment is a SOURCE–PATH–GOAL

Schema (Lakoff 1987b, 144). There is a SOURCE from which the controllable

object moves via a PATH to reach a GOAL. This goal could be a new

SOURCE for a new movement. When a game is founded on direct control the

space perceived is a personal, subjective, continuous space belonging to the

Game Ego, even if the visual perspective is not that of a first person subject.

The SOURCE–PATH–GOAL Schema of a game with indirect control gives a

perception of space that is discontinuous and broken up into segments. It

weakens the link to the controllable object. The game player points with thecursor and clicks the mouse and the object moves to this point. A new click



THE GAME EGO

165

on another point generates a new movement and so on. This is an act of

vision and motor control. The interface is a three-step interface where the

player has to perform the following:

First, find out where to let the Game Ego move, and then

point at that location on the screen, i.e. within game environment and then,

click the mouse to make the Game Ego perform the desired action.

A direct control is also an act of sensorimotor action and vision but it has

fewer steps:

first find out where to let the Game Ego move, and then

perform the action desired.

This does not mean however that a controllable object based on direct control

is not allowed to stop along the path. It simply means that from the time

when a game player points and clicks, until the movement of the controllable

object has come to a halt, the game player lacks control of it. This is evident

in games like Phantasmagoria and Day of the Tentacle. These games have

similarities with the construction of space seen in movies i.e. sequences

edited together. See also the discussion about loss of self control, in

paragraph 5.5. There are similarities with the spatial aspects of (the

English) language syntax. Lakoff and Johnson note that:

The closer the form indicating CAUSATION is to theform indicating the EFFECT, the stronger thecausal link is. (Lakoff, Johnson 1980, 131).

As an example they give, the two sentences ”Sam killed Harry” and ”Sam

caused Harry to die” The syntax indicates how direct the causal link is

between what happened to Harry and what Sam did to cause his death. Asthey put it:

Closeness Is Strength of Effect. (Ibid. 128).56

As stated above several visual and sonic fields build up the game

environment. The terms put forth above can be considered as rather media

bound terms. One could also experiment with the term perceptual field,

which would cover every possible perception at any given point. The

perceptual field is under flux when moving around in the game environment

56 See also chapter 3, 40 on this issue.




166

(and even when not moving since things can come into and go out of this

field).

There is more than one way of experiencing the environment. There is ”pure

perception” which is a disembodied and object-centered perception. There isalso ”combined perception” which is ego-centered and where there is a link

between emotions and actions in the perception of the environment. The

Game Ego is based on the latter one. That is. the field of view of a game like

Duke Nukem 3D is a first person simulation i.e. it is an embodied

perception. The combined perception is of importance for the spatial

orientation within the game environment. Space that is off the visual field

could be brought into the visual field by moving in space. You can not move

in space without motor activity. The visual field of Duke Nukem 3D is a

product of moving in space, making the non-visible space visible, possible tomanipulate, and to take action within. Consequently, it is necessary to move

in space so that the Game Ego is relocated and is able to perform its actions

in the right direction i.e. in the visible direction. As Lakoff and Johnson

point out:

Moving objects generally receive a FRONT–BACK orientation so that the front is in the direction of motion or in the canonical direction of motion. (Ibid.42).

The affordance of the field behind the Game Ego is that it is possible to

become visible through motor action on behalf of the game player. It affords

a possible point of observation, a possible point of being with the game. The

perception of the environment is, as the analyses in chapter 9 will show,

connected to how it is possible to move within it.

8.5 Game Ego versus Avatar.

To close this discussion just some words that explains why I prefer to use theterm Game Ego instead of the maybe more well-known and wide spread

term Avatar for denoting the agency a computer user might have in a virtual

environment such as on the internet in a MUD or MOO. I do so because

Game Ego as a term emphasizes its connection to the game player as being a

part of him or her. I hold the position that conceptual sets within language

do tend to make us think in certain structures. That is, language is a kind of

self-organizing neural system that is “lazy” by nature and prone to make

short cuts. Consequently, a term like Game Ego is considerably more focused

on the relation between it and the game player than Avatar. Moreover,



THE GAME EGO

167

Avatar has also religious connotations. The word is a Sanskrit word for the

incarnation of a Hindu deity (such as Vishnu), or the incarnation in human

form of a concept, and thirdly a variant phase or version of a continuing

basic entity. (http//www.eb.com:180/cgi-bin/dictionary?va =Avatar accessed

2000-5-18). Maybe the third definition is closest to my term Game Ego, sincea Game Ego does not necessarily come in the shape of a human. Therefore,

the third definition can be thought of as more or less equivalent to Game

Ego. However, since the two foregoing definitions are considerably far away

from what I mean by Game Ego I do not intend to use the term Avatar. Why

should one depart from the conceptual domain of game play when it is not

necessary? I fully realize that Avatar is a well-known and widespread term

and that I by coining yet another term am contributing to an already

massive plethora of terms. Nevertheless, I am convinced of the benefits of

the term Game Ego compared to Avatar.

It is possible to raise critique of the Game Ego term, of course. The use of

Ego can be troublesome for instance. Ego is a term within the psychoanalytic

tradition of Sigmund Freud. A person’s division in Id, Ego and Super Ego

may seem similar to Game Ego and game player, Self and Subject and so on.

The ego in Freud’s theory is concerned with tasks of reality i.e. perception,

cognition, and executive actions. So the Game Ego is not that far from these

aspects of a Freudian Ego in that the Game Ego is the agent for actions. I do

admit that there is a “psychological” link between the game player and theGame Ego. However, it is a neural link, psychology being neural, and it is a

link founded on the motor performance of the game player as a part of the

interface loop created by the PART–WHOLE configuration of game player

and Computer Game System. Psychoanalytic theories may provide models

for the analysis of audiovisual computer games no doubt. However, as

Torben Grodal points out, psychoanalytic theories:

[…] describe desire and emotions in relation to

cognitive functions within a romantic dualist modelincompatible with a theory which describescognition and emotions as aspects of a functionallyunified psychosomatic whole. (Grodal 1994, 7).

The key element here is dualism. There is little room for dualism within a

theory that poses and claims an embodied mind. Them mind is part of and

functionally dependent upon our bodies. However, as shown in the

paragraphs above in this chapter and the discussion of The Self and The

Subject, human beings have the conception that there is some kind of

separation between different parts of our beings. It is possible for a human




168

to understand a situation in life as “some part of me died” etc. This is due to

the conception of The Self and The Subject. There is a big difference between

a theory founded in dualism (psychoanalysis) and a theory stating that

human beings conceive of them selves as being split in parts (experiential

cognition).

8.6 Summary

The tactile motor/kinesthetic link between a game player and the Game Ego

is a function that shows the metaphorical Subject-Self relation as described

by Lakoff and Johnson in several of their works. We may have possession of

a Game Ego’s motor capacities in a game. This means in turn that our bodily

container, our Self, is mapped on a control function within the game. This

function in turn may be manifest through the visual or sonic appearance of acontrollable character within the game environment, or through or

possibility to control the field of view and/or the field of audition by means of

motor action.

The Game Ego may be manifest in different ways in different games. It may

have some kind of visual manifestation as a fully visible character. It may

also be manifest with only parts of a body, as in Myst where the visual Game

Ego is only a hand used like a navigation tool and to manipulate to

environment. In some games, it is only the result of manipulation thatmanifests that actions have been performed. This is obvious in Tetris and

such games in which the manipulation of objects is performed from keyboard

commands without any visible manipulating agent in the game

environment. The objects move because of the game player’s actions in a

very direct mode but there is no agent visible that performs the

manipulation. The important thing in computer games is the sophistication

of the player’s internal relations between perception, cognition, emotion and

action, and not necessarily the degree of visibility of a Game Ego.

To simplify the functions and manifestations of a Game Ego consider the

following table:

Table 2: Manifestation of the Game Ego

Visibility Directcontrol

Indirectcontrol

Fully visible Pac-Man Day of the Tentacle

Partly visible Unreal Myst



THE GAME EGO

169

Non visible Tetris

The physiognomy of a Game Ego has its importance for the game play. This

has of course consequences for what actions could be performed and how

they will be performed. Our perceptual capabilities as well as our cognitivebasis in motor performance, transcend into how the Game Ego can be

manifest. A Game Ego with limbs of a human kind may afford manipulation

that is close to how we actually perform certain kinds of manipulation in our

real environment. That is, a fully visible Game Ego that is humanoid will

probably mimic how we do things. If we take this a step further, this could

affect the setup of the game interface. And it does. There are games that use

our body to greater extent than pressing keys or manipulating joysticks,

steering wheels etc. This is the case of course in simulators of different kinds

where one may have foot control and such devices. Also those dancing games

that have been introduced in gaming arcades lately are examples of this. The

motion of our legs and feet is used to control the Game Ego.

As I suggest in chapters 4 and 5, affordances of the game environment also

stress that the game player has an important role in the development of the

game, i.e. the development of the syuzhet. A game is organized in some way.

It has rules for how things can and must be preformed in order to solve

certain puzzles, to initiate specific game events, for the manipulation of

objects etc. Also the orientation capacity, (i.e. where am I? Where is this or

that object located in reference to my own location?) makes a difference since

most games do have salient ACTION–LOCATION structures (see also

chapter 5 and 9 on this). To perform certain acts one must be in a specific

location which one must find on ones own. The perceptual capacities, the

cognitive capacities and the motor skills are important and the successful

performance of them is gratifying for the game player. The game would

otherwise come to an early halt and stop before the syuzhet has been fully

developed. This is obvious in games that have an overall narrative structurelike adventure games. Yes, I know that there are games that are not

considered at all narrative, such as Tetris, MineSweeper (Finley 1992) etc.

However, in such games we also have these kinds of structures. There are

SOURCE–PATH–GOAL/DESTINATION schemas just as in more narrative

games like Day of the Tentacle for instance. These schemas are the very

basis for the progressive narrative structures and this is important to note.

A film has a pre-organized structure of SOURCE–PATH–

GOAL/DESTINATION that we are to find out not for the continuance of the

presentation but for the understanding of what is going on in the narrative.




170

We will map our being onto the film environment and the characters within

it. To make sense out of the narrative we have to obtain a basic spatial

orientation and find out the social relations between characters and so on.

However, the film (as a hardware) will continue to roll through the projector

no matter if we are able to successfully carry out these operations. Thenarrative, on the other hand, as opposed to the film hardware will possibly

not develop although the film is running. We may get lost on the way, so to

speak.

I believe we tend to overrate the significance of narratives being told, and

not enacted. I cannot make such a distinction. Narratives, in my

understanding, can both be told and enacted. In fact, when we take part in a

narrative that is told to us, we understand it by means of concepts derived

from our daily life using our human body and living in human society. Thehuman mind organizes the told material in a simulation mode that makes it

meaningful.

The next chapter will contain some analyses that further develop and

exemplify my points put forth in the dissertation. Some analyses are more

elaborate than others. This is the case with the analyses of Pac-Man, Day of

the Tentacle, and Myst.




172

>open the mailbox (this is what I type in as input to the game system).

Opening the small mailbox reveals a leaflet.

>take leaflet (My input again).

Taken.

>read leaflet (My input).

’WELCOME TO ZORK!

ZORK is a game of adventure, danger, and low cunning. In it you will

explore some of the most amazing territory ever seen by mortals. No

computer should be without one!’ ”

As this example shows the game player is invited by the game system to

take action on the environment. The medium permits locomotion and vision.

The field is open, i.e. its affords locomotion, there is a house with boarded

front door that constrains locomotion. At least the field implies an affordance

of locomtion since it is open. We do get any information about the surface

texture of the field. Once we perform a command like ” walk west” we might

endanger ourselves. There is mail box (a classical example from Gibson by

the way, to be found in The Ecological Approach to Visual Perception 139)

that affords mailing of letters in a letter writing society and the deliverance

of such letters in a mail box.

We will all have a different mental images projected from this information

but the basis of these mental images will be the same. Your white house is

not similar to my white house in all respects but they do share the same

basic level qualities.

The game structure is obvious. You read and you type your commands. The

written language makes you form an understanding of the environmentthrough the words used. They have affordances in their description of the

objects that constitute the environment. The game works on a basic level

categorization. The objects you encounter have general motor schemas in

most cases. However, once in a while you will have to break the bounds of

the cognitive and perceptual sets that the game exposes you to (and that we

encountered in chapter 5 of the present work). You will have to use objects in

a way that might not that obvious until you find the logical structure for the

use of them. A rope for instance might be climb-able as well as tie-able.



ANALYSES AND EXAMPLES

173

The contexts you find yourself in help you to find the proper contextual use

for the objects.

Futhermore, Zork is sometimes referred to as a Second person game. This is

so because of the use of ” you” . However, this is clearly a misconception of thegame. It is not a ” you” that performs the action. The way the game

communicates with you is just the same as I do now. It tells you certain

things about the environment just as I tell you certain things about this

game. However, when you think about the game and the game environment

you will refer it to your Self and you Subject as suggested in chapter 8. This

means that we have a Game Ego presence in this game. The referent of ” you”

is the performative agent ” I” when you execute the commands you do. To call

Zork a game of a second person is therefore out of the question. Who would

otherwise perform the actions if not the Game Ego? In addition, the use of present time locates me and you as game players in the environment at the

present time. Just as in games like Pac-Man, Pong, Spyro the Dragon etc.

I will not go any deeper than this into the environment of Zork. This does not

mean, however, that the game is not interesting. On the contrary, it is very

interesting. However, to undertake a full-fledged analysis of this game would

need yet another dissertation. The narrative development of the game for

instance is worth further investigation as is the use of written language, the

pros and cons of this kind of text based interface etc. The important points inthe context of my approach I have already made. However, let us now go on

to audiovisual games, namely two of the archetypes of computer games: Pong

and its predecessor Tennis for Two.




174

9.2 Tennis for Two and Pong

Pong is the game that started a billion-dollar industry back in 1972.

However, it has a traceable predecessor as far back as 1958. William

Higinbotham is sometimes given the credit of being the inventor of the firstknown computer game. He, who during World War II was a member of the

team that constituted the classified Manhattan Project (i.e. the atomic bomb)

and in 1958 was the head of Brookhaven National Laboratory’s (BNL’s)

Instrumentation Division is the grandfather of computer gaming as we know

it today. (http://kratos.osti.gov:85/videogame.html, http://www.fas.

org/cp/pong_fas.htm et.al.). One can of course also argue that Alan Turing is

the grandfather of computer gaming. His work on first simulating a

computer as such and than also his work on making them intelligent (the

Turing test etc.) is the basis for the computers of our time. However, this

discussion is out of topic here. The reason that Higinbotham constructed this

game, called Tennis for Two, was very simple and down to earth: the BNL

had visitors’ days during the fall each year and many people came to visit

the facilities. To illustrate what the laboratory could design Higinbotham

and Robert V. Dvorak started to work on Tennis for Two.

” I knew from past visitors days that people were notmuch interested in static exhibits,” said

Higinbotham, ” so for that year. I came up with anidea for a hands-on display – a video tennis game.”

It was wildly successful, and Higinbotham could tellfrom the crowd reaction that he had designedsomething very special. ” But if I had realized justhow significant it was, I would have taken out apatent and the U.S. government would own it!” hesaid. (http://www.osti.gov/accomplishments/videogame.html 2001-02-22 10.25).

Luckily for devoted computer gamers all over the world, Higinbotham did

not take out a patent. If he had, we would have stopped here and now

perhaps? His utterance shows an insight to what I call Playing is Being. It is

not enough to only walk around and look at things: we are human beings

and as such, we have an urge to manipulate or surrounding environment.

What we got here is what I have earlier described as Difference Making and

Performative Interaction. (See chapter 7, 131, and chapter 8, 148). Playing

Tennis for Two was obviously a situation where the players were in close

proximity to each other sharing not only the game environment but also a” social space” in the “real world”. They could see and talk to one another




175

while playing the game. They shared a common experience, so to speak, in

real time both in the real environment and in a game environment. The

paddle used for hitting the ball is an extension of the game player’s motor

action and the paddles are used intentionally and with an explicit purpose.

Their usability for other tasks are constrained. You can not do much else (if anything) than hitting or missing the ball with the paddle. This is unlike

real world racquets that can be used for other, unorthodox purposes (as

shown in the movie ROXANNE (Schepisi 1987) for instance where a racquet is

used to hit a person instead of a ball. It used in the same conceptual domain

though so difference is not that salient. To use a racquet to dig would be in a

slightly other domain. The shape of the racquet is similar to that of a tool

used for digging: a spade).

There is a video clip available (at the time of writing athttp://www.osti.gov/accomplishments/videogame.html) showing how Tennis

for Two works. The court is a long horizontal base line with a vertical line at

the middle representing the net. The court is in other words seen at a low

visual angle from one of its longer sides. The visual display is an ordinary

analog oscilloscope and the computer handling the game is an analogue one.

There are two control units, quite large hand held metal boxes that have two

controls each. There is a knob to adjust the angle of the ball and a press

down button for hitting the ball to the other side. The ball moves in ballistic

curves and not straight lines. The analog computer supplies the game withmechanical sound from the relays when hitting the ball. That is to say, even

if there are no programmed sound effects as part of the game there are the

sounds of hitting the ball rendered with the visual so as to create a coherent

game environment configuration of sound and images. There is in other

words a consistency in the environmental layout. The medium in the Tennis

for Two environment permits motion that is very similar to our daily

environment. No wonder! Higinboatham was involved in ballistic

simulations with the help of computers during the Second World War. It is

therefore only natural that his game implements this and manifests it

visually.

Let us now turn our attention to its later incarnation: Pong. Pong is very

similar too Tennis for Two. The graphic of Pong is two bit. That is, there are

only two colors: black and white. The game environment consists of a very

simplistic two-dimensional top view of a table tennis board. The image is low

resolution and has high contrast. (See screen shot below). There is no change

in location during game play. There is only one game environment accessible

for interaction.




176

Figure 3: Screen shot of Pong

The original Pong has only one sound and that is the sound generated by

hitting the ball. It sounds like “Pong” and this is of course one of the reasons

the game got its name. Another reason is that it is table tennis being

simulated and that game is also sometimes called ping pong. In turn, this

has the same reason: it is the sound of hitting the ball and the bounce of the

ball in the table that has given the game the name of ping pong.

The Pong sound works in a similar way as sound in Jacques Tati’s movies

(see also chapter 6, 122). The Pong sound incorporates the basic levelcategory of hitting something. It is a sound that has a fast attack, which

provides the hitting.

9.2.1 Controllability

There are three controllable objects; two paddles and a ball to hit with the

paddles. The paddles are directly controlled by the game players and the ball

indirectly by hitting it. Or rather, you do not actually hit the ball in this

game. The ball is moving by itself and the object is to put the paddle so thatball hits it. Hence, we have rather the avoidance of missing it and letting it

past your paddle really. The response to game player motor action is

immediate. There are no glitches, which is important for the sensory

immersion of the game player into the game environment. Rapid and

” mechanical” motor action is the very foundation for this game. (See also

chapter 8 on controllability and chapter 7 on patterns for interaction).

The experience of playing Pong is one of immediate interaction: action taken

on action that is. In addition, the action is carried out in a very limited andflat environment that permit motion of the racquets UP and DOWN and the




177

motion of the ball in accordance with the ” laws of physics” in some respects.

The incoming angle equals the outgoing angle. Just like in pool and pinball

that is. And, pinball in turn was quite popular at the time when Pong was

released. The paddles, or racquets, do have a general motor schema in this

possibility of moving them UP and DOWN. There are version of the gamewith a screen mounted in a horizontal position, that is you have to look down

onto the screen instead of looking at it from the side. You then move the

paddles SIDEWAYS instead of UP and DOWN. The interface is construed so

that the manipulation of it is consistent, however. You use a wheel to control

the paddle that is yours. In addition, a wheel, as we have seen in chapter 5,

is possible to handle in a PART–WHOLE structure. Left and right, that is

SIDEWAYS, might also be UP and DOWN.

The game instructions to Pong says it all:

Deposit Quarter

Ball will serve automatically

Avoid missing ball for highscore

It really can not get any simpler if we are talking about the conceptual

meaning of the game. However, it takes a lot of motor skill to actually play

the game successfully. In addition, such motor activities make up the basisfor the human conceptual system as shown in chapter 5 of the present work.

We can compare the success of Pong to failure of Computer Space (Nutting

1971). Computer Space was not as simple as Pong that it preceded. The

game instruction was, at the time of the game’s release, too complicated to

sort out and the possibility for controlling the space ships in the game was

simply too good. See the photos below of the Pong the Computer Space

control, respectively. It is ironic that because of the great controllability of

the game it failed to be a smash hit.




178

Figure 4: Pong controls

Figure 5: Computer Space controls




179

9.3 Space Invaders

The following quote provides some interesting recollections about the

experience of playing Space Invaders. It also provides judgement about the

game and some noteworthy remarks about the games appearance, which willbe part of the analysis.

One day, back in 1979, my brother Matt went toPizza Hut with his friends […]. While waiting forthe pan pizza to get cooked, Matt beheld a new bluegame in the corner. It was the original SpaceInvaders... It took quarters... and it had the heaviestbass sound that could be recalled since LedZeppelin’s Lemon Song.

My own personal memory is one of walking with mybrother and friends to a dark, crummy two-lanebowling/beer joint on the east coast that had beenaround since at least the ’50s. The place was filledwith smoke, dank smells, the sound of bowling pinsbeing knocked over, and bassy analog eruptionsfrom games like Asteroids, Donkey Kong, and of course, Space Invaders. (Greg and Matt Rossiterhttp://www.rossiters.com/greg/spaceinvaders.html

2000-05-24 11.30)

I find it interesting to note how the sound is of importance in this quote.

Computer/computer games are part of a culture in which sound is an

indiscernible part. The reference to Led Zeppelin’s Lemon Song gives a clue

to the attraction value of the sound. The sound in this case, due to the

personal values of the perceiver, made it attractive to go over and take a look

at the machine that produced the sound. Although the sound as such might

suggest a non-moveable direction due to its density, some people obviously

find such sounds attractive rather than constraining. Now to somecomments concerned with the layout of the game cabinet and the experience

of being nine years old and playing this game:

Its eerie, glowing cabinet, painted with menacingblack aliens towered over my nine year-old body. Itsat there quietly as the attract mode displayed anarmy of marching alien figures (the front-projectedimages suspended ghost-like in the air on mirroredglass), and a lone fire-base frantically firing and

weaving back and forth to protect its cratery moon-planet. I put a quarter in and the silent machine




180

abruptly came to life. Its deep thumping sound wasintimidating; the shrill laser blasts added to thesense of urgency; and the sudden approach of a UFOcaught me off guard. I fired off a few desperateshots, some managing to wipe out an occasionalinvader. But it wasn’t long before my base wasrepeatedly left a crumbling smolder and my fewquarters were gone.

We left this dark cave of strange new entertainmentwith empty pockets and lasting impressions of coolness in our minds. […] (Greg and Matt Rossiterhttp://www.rossiters.com/greg/spaceinvaders.html2000-05-24 11.30).

I find these quotes very interesting in that they reveal an interpretation of the game environment as well as a general description of the environment in

which the game was placed. Of course, this is a recollection and as such is

not reliable as a true statement. However as probably being dramatized and

somewhat glorified through the passing of time, it has maybe even greater

value for this study. It is really an idealized version of the experience of

being nine years old and as such it enhances some salient features and

qualities of the game. The game was played in the glory days of this person

so to speak and hence we find some pretty strong words putting forth the

feeling of being there playing the game. There are several emotional

expressions used in the quote such as ” towered over my nine year old body”

” the silent machine abruptly came to life” , ” the shrill laser blasts added to a

sense of urgency” , ” I fired off a few desperate shots” , and ” lasting impression

of coolness in our minds” etc. This game is definitely sensory immersive and

also evokes strong emotional states of mind i.e. it is a game that produces

arousal.

9.3.1 Short background history and description of the game

Space Invaders was designed and programmed by Toshihiro Nishikado, in

1978, and released 1979 by Taito. The rumor on the Internet has it that

Nishikado had a strange dream that inspired him to design the game: It was

on Christmas Eve and while Japanese school kids were waiting for Santa

Claus to appear in the sky, they suddenly saw row upon row of aliens

advancing slowly from Venus. The kids, clever Japanese ones of course,

realized that this was a great threat to Earth so they quickly put together a

laser blaster from the hubcap, spark-plugs and battery of a parked car.

Moving left and right, left and right, they were then blasting aliens out of




181

the sky. After maybe four waves of aliens attacking them, the aliens gave up

and the Earth was saved. The next morning i.e. Xmas Day the kids were

rewarded with extra presents and figgy pudding. So they did not do it for

nothing. (http://spaceinvaders.retrogames.com/html/history.htm 2000-04-28

14.09).

The object of Space Invaders is similar to the dream described above. The

game player must hinder an alien fleet of space ships to reach the ground

and from destroying buildings. The alien fleet is moving back and forth

across the screen descending towards the Game Ego and for each level they

are getting faster and faster. Occasionally and randomly, a big flying saucer

is crossing the upper part of the screen. Shooting this to pieces of course

yields high scores. The game is mainly about coordination and shooting

things down from the sky.

9.3.2 Visual style and features

Space Invaders is displayed on a black and white monitor. However a

transparent plastic sheet, much like the ones fitting Odyssey games, is

placed on top of the screen and hence there is color added to the game.

Odyssey is home computer game system from the 1970s, which used such

plastic sheets as an overlay to add color and also made up the game board.

See the photo below showing an Odyssey system.

This particular Odyssey system was part of an exhibit at the American

Museum of the Moving Image October 2000.

Figure 6: Odyssey gaming system

The game environment of Space Invaders is very limited. The visual

container is static, i.e. there is no movement of the visual point of being and

hence the visual display that part of a game environment where action isgoing on. We are playing in a two dimensional flat surface of a screen in




182

which the action takes place. The surface is the place for action, just as

Gibson states and we encountered that in chapter 4 of the present work. We

also have the ACTION–LOCATION metaphor of Lakoff and Johnson here

(see chapter 5). Shooting down an alien space fleet is Being at this specific

location in this specific game environment. You can do nothing else if you donot want to get your Game Ego killed off by the Invaders. After all: there is

no Don’t fire button – just as in the example of poor eleven year old Johnny

that we encountered in chapters 1 and 4 of this dissertation.

Let us now take a closer look at the visual field, which as mentioned makes

up the visual game environment. It consists of a horizontal base line on

which a rectangular object with a vertical line on top can be moved sideways.

It affords horizontal locomotion. This is the Game Ego. Above the base line,

that is the ground, there are four objects providing shelter for the Game Ego.They are Get-underneath-able to use Gibsonian language. That is, the

objects afford the Game Ego shelter from the alien attack. The Game Ego

must defend these objects from the Space Invaders attack. If the Invaders

reach them, they will win the game. The objects might represent buildings.

The Game Ego may hide under them but the Invaders can shoot the shelter

to pieces little by little (as can also the Game Ego if necessary. The

characters within the game are not humanoids but rather what we conceive

of as aliens.

Figure 7: Screen shot of Space Invaders. Space Invaders © Taito 1979




183

9.3.3 Audio style and features

The sound of Space Invaders has a low technical quality compared to the

sound track of a contemporary movie. However, it does contain basic level

qualities that have a tight coupling to the visuals. It has a sturdy and

dominant rhythm that matches the visual. The Space Invaders are

descending towards the Game Ego. The sound consists of several parts

making up the sound space. The basic sound is rhythm. The tonal interval

consists of Gb-B-Bb-Eb. It describes a downward movement. That is to say, a

falling scale and a falling interval that goes well together with the visuals

(see also chapter 6).

The sound of using the weapon, something that shoots at the invaders, a gun

on a tank (as suggested by the shape of the Game Ego), is a falling pitch. In

this context, however, the falling pitch refers to LEAVING and AWAY. Theonly direction one can shoot is UP. This means that we have a special case of

a change in pitch. The sound goes from high frequencies to low frequencies,

which could suggest a motion from a high location to lower. However, due to

the Doppler effect (discussed in chapter 6) this is not the way we understand

the sound in this context. We are firing a gun and the missile that goes UP is

also LEAVING the gun. The change in pitch hence suggests that we are

firing a weapon. The point of audition is the same as the Game Ego’s location

on the base line. We do not hear the sound from our point of being in front of

the screen but from a point of observation within the game environment.

This is similar to the sequences described from Kubrick’s 2001 (in chapter3).

The sound accompanying the flying saucer that randomly passes by is one of

rapid oscillation between two distinct sound points. I prefer to use

” accompanying” rather than the sound ” of ” . It is not obvious that the sound

stems from the flying saucer as such. It might be an alarm that tells us that

there is a flying saucer present. The difference is notable but the sound is

referring to the presence of a new object entering the field of view, in both

cases.

We map our being within the environment on the game environment. We

have the possibility to control a Game Ego through tactile motor

manipulation of the game system. The link between the game player and the

Game Ego is strong. We have immediate response in the system and our

bodily container extends into the game environment. We are within the

game environment due to our possibility to act within it. It is not so that we

just see and hear, but we may also exert force and take action. The system in

turn also exerts force upon our Game Ego.




184

9.4 Pac-Man

I will begin the following analysis with a short historical background to thegame Pac-Man since it really is one of the most successful games in the

history of computer gaming.

The main source for the historical background is

http://videogames.gamespot.com/features/universal/hist_pacman/index.html.

Pac-Man has a place of its his own in the history of computer games. Pac-

Man is rumored to be the best-selling coin-operated game in the history of

computer games. When one first encounters Pac-Man it perhaps may seem

like a reasonable simple application maybe. However, it took eight people

around fifteen months to complete it. In the first year of production, more

than 100,000 Pac-Man machines were sold world wide. The design of the

Pac-Man character is held to be the result of eating. The Namco designer

Tohru Iwatani, who was one of the team behind the game, went out for the

evening with some friends and had a pizza. When a slice was cut from the

round pizza Iwatani realized that it looked like a face with a huge mouth.

The basic-level image schema structured what he saw as a facial structure.

The basic design of Pac-Man was conceived. In 1982 Pac-Man becametelevised. ABC launched the animated The Pac-Man Show. It was on for two

years. Pac-Man also found its way into popular music when Jerry Buckner

and Gary Garcia made Ted Nugent’s song Cat Scratch Fever into Pac-Man

Fever.

A recent estimation claimed that Pac-Man had been played at least ten

billion times by 1999. However, until July 1st of that year no one had ever

played a perfect game. On that day, Billy Mitchell played a six-hour session

and completed a perfect game. That is to say, he played all 256 boards, ateall the four ghosts with each of the “power pellets” and all the bonus fruits

on all boards reaching a high score of 3333360 points. Pac-Man is a game

designed to keep on going and going and encourage the game players to

spend yet another coin and yet another coin until they are out of change.

Pac-Man is a time killer. (See also chapter 5, 92 and chapter 7, 132). No

wonder it has been said “Pac-Man hooks only those people who confuse

victory with slow defeat.” (Goodman 1982. Cited in Loftus, Loftus 1983, 11).

Well it is right when we still are talking about Pac-Man (the computer game

system as such and its programmer(s)) versus the game player. Pac-Man willalways win in the end, maybe perhaps with the exception of this time to




185

date, unless the game player does not smash the machine up in frustration.

Which in turn only goes to show that the programmers have done their job

and made the game player really engaged. A game that is not smacked is not

considered a particularly good game as Eugene Jarvis, member of the

programming teams at Williams Entertainment in the early 1980s has said.Jarvis was involved in programming games like Defender, Stargate

(Williams 1981), Robotron 2084, Blaster (Williams 1983) and Narc (Williams

1988) e.g. He remarks:

“You spend months and months and months justplaying the game and percentaging it and seeinghow you could create maximum frustration in theplayer. If a player smashes a game, that’s actuallythe best thing you want. You want a strong reaction.

If a player does not care enough about a game tokick in the coin door or break the glass, then youknow it’s not a good game” (Herz 1997, 79).

However, if we consider Pac-Man as a social activity we have the possibility

of two or more game players competing to get the best score of the day.

Alternatively, reaching the top of the high-score list, get the reinforcement of

writing their name there for others to see and getting the admiration of

others. When played in arcades we have also the economical game of getting

game players to play, that is, to spend their money on these games. Thus, wemay have another winner in the arcade owner.

Pac-Man gave computer games a face and was the first computer game

character to become a star. J.C. Herz remarks that Pac-Man provided the

first character of computer games that a game player could really identify

with. A few months after its release Pac-Man was to be found on pillowcases,

cereal boxes, T-shirts etc. (Herz 1997, 131-132). Further more, in 1999,

nineteen years after its original release Pac-Man was officially proclaimed

Game of the Century by Twin Galaxies Intergalactic Scoreboard57

at theCGE ’99 in Las Vegas. At the Tokyo Game Show, Walter Day (a Twin

Galaxies referee) presented the President of Namco, Mr. Masaya Nakamura,

a certificate that attested this.

Now it is time for the analysis. On the enclosed CD-ROM you will find a

number of example files that you may use in purpose of easier follow the

57 Twin Galaxies Intergalactic Scoreboard is the video game trade trackers of high scores in vidoe gamesand pin ball machines since February 9, 1982 That status was assigned to Twin Galaxies by the two coin-

op trade magazines: RePlay and Playmeter, and by seven of the premiere manufacturers of that era i.e. theearly 1980s: Williams, Atari, Bally, Nintendo, Exidy, Stern and Universal. (http://www.twingalaxies.com/ ourstory.html)




186

discussion. They contain sound excerpts from the game that I will refer to

(.WAV files) as well as a recorded game session (PAC-MAN.MOV).


As mentioned, Pac-Man has been around since 1980. It belongs to prototype

3 a, i.e. 2-dimensional games related to character/vehicle control. The Pac-

Man name is derived from the Japanese word for chewing (puc). An early

version of the game accordingly was named Puc-Man. And this also reveals

was the game is all about: chasing food (or dangerous enemies that are

eatable) or being eaten. The use of the mouth is important in the game.

However, and now I speculate, writing ”Pac-Man” looks better in print since

it is more balanced and symmetric so this might have a been a reason for

renaming the game. It gives a more compelling structure so to speak.Furthermore, Pac-Man sounds harder than Puc-Man. Pac is ” sharper” than

Puc. One can also make a connection to the English word ” impact” . So, for

what ever reason, Puc-Man became Pac-Man.58

Pac-Man and Bubble Trouble (see separate analysis) are versions of the

same game idea. Pac-Man was first designed as an arcade console game.

There are some differences between playing the game on an arcade console

and playing it on a computer. My analysis is based on the ROM from an

arcade game but played on an emulator on a computer. I will however makea few remarks on the hardware set up of the arcade game since it is relevant

for my study and my theoretical approach.

58 Puc-Man also sounds a little too much like Puke-Man.




187

Figure 8: Screen shot of Pac-Man game environment. ”PAC-MAN” Characters &Musical Works © 1980 NAMCO LTD. ALL RIGHTS RESERVED

The Pac-Man game has a number of screens. The one we are mainly

interested in here is the active game board (see screen shot above). The

Game Ego moves around within a limited visual field, a limited visual

container. In this case the Game Ego is a yellow circular shape (Pac-Man)

equipped with a ”chewing mouth”. The ”chewing mouth” gives the Game Ego

a FRONT- BACK orientation, the mouth being the FRONT since we map our

physical being as humans on the Game Ego and the fact that the Game Ego

does move in the direction of its “chewing mouth”. The visual field has

certain boundaries making up a maze. In the maze, there are dots of two

kinds; a Pac-Man (Game Ego) and four ghosts.59 The goal is to eat all the

dots and all the ghosts. As said there are two kinds of dots: small constantly

shown dots that will give the game player ten points when eaten and bigger

blinking dots that will give him or her fifty points. These dots are

functioning as turning points of the event structure, much like dramatic

turning points of a narrative, like the ones described by Bachtin (TheChronotope), Branigan ( Point of View in the Cinema and Narrative

Comprehension and Film), Bordwell and Thompson (in Film Art) an by

numerous other theorists engaged in narrativity. I will get back to this issue

59 As mentioned above (see chapter 5, 87 and 102) it is necessary to minimize the amount of data chunksto somewhere in between five to nine at the time if we are to perform well when playing a computer game.This is also found in successful Pac-Man playing. There are four ghosts, one Pac-Man, one maze, and fourenergizers plus an occasional bonus fruit. This makes eleven objects that are important for the game play.

However, a player that is good and reaches a high score might perform a reduction of chunks. A smarttactic is to mentally reduce the four energizers to one square. It is then handled as one object instead of four separate ones. We have then not eleven objects but seven.




188

soon, commenting on Bob Foss’s Narrative Technique and Dramaturgy in

Film and Television.

In contrast to the Game Ego residing in Bubble Trouble, Pac-Man, the Game

Ego, moves all the time and can not be stopped except by hitting a boundaryi.e. one of the walls making up the maze. In Bubble Trouble there are also

boundaries but these can be pushed around and even destroyed by the Game

Ego. So, what we have is the affordance of locomotion according to a certain

environmental layout that makes up the maze. The maze is fully visible and

possible to have a complete overview of. The only suggested location outside

the visible game environment is that place where the Game Ego shortly

disappears when going sideways in the middle of the game board. This

functions like a warp zone. One can have the Game Ego go to the left and

then reappear at the right side of the game board. Furthermore, theenvironment consists not only of firm and static objects that constrain

locomotion. The four ghosts also constitute a variable constraint. They are

there for to chase us around and eat us while playing. They constrain the

Game Ego’s locomotion. However, this situation is reversible and in flux. The

ghosts may also be eaten and the ones being hunted.

I suggest that there is a basic narrative structure to be found in this game.

We have already touched upon the SOURCE–PATH–GOAL schemas that

are so important for games and narratives in the foregoing analyses as wellas in previous chapters (especially chapter 5). I promised to get back to Bob

Foss’ ideas on narratives and now the time has come to fulfill this promise.

He puts forth a schema over narrative development. It consists of: Opening -

presentation - point of commitment - intensification of conflict -

confrontation - climax - resolution (Foss 1992, 152). To some extent, these

elements are also to be found in a game like Pac-Man. The game has an

introduction sequence that presents the objectives of the game. It is a short

animation where the Game Ego is hunted by the ghosts, eats a big dot and

the chases the ghosts. The ghosts are also presented by names and nicknames construing them as characters and not merely objects. This is, in

other words, an opening and a presentation. The point of commitment is

obvious: you have to find your way through the maze and find blinking dots

in order to survive, so it is a game about your own survival within the

environment. The intensification of conflict is the level of difficulty. It gets

harder and harder to survive the levels the more levels you gain access to by

conquering the ghosts. The crisis, confrontation, and climax are more or less

constant and not really very much like their counterparts in film. We are

acting within a continuos crisis/confrontation/climax state in the game.




189

There are turning points made up by the blinking bigger dots that change

the status of whom hunts whom. There are short pauses with cut scenes that

make it possible for us as game players to release some of the tension the

game evokes. The resolution can be the death of the Game Ego, the success

of playing through all levels or simply a decision like ” Ok that’s enough forthis session I’ll now do something else” .60


The sound of Pac-Man is interesting. It functions on a basic-level. The game

is introduced by a short melody that ends with a raising scale that suggests

that we are in for an energetic session (listen to example file

GAMEBEG.WAV on the CD-ROM). The rise in pitch correlates with a state

of the body/mind system. An up-tempo musical cliché used here clearlymarks a starting point for something. Motion in this game is quiet in itself.

Eating the small dots results in an eating sound. See the screen shot below

and listen to the example file PACCHOMP.WAV on the CD-ROM.

Figure 9:Screen from SoundEdit 16

This screen shot shows a spectrum of the chewing and oscillating sounds. The continuoussound curve from 500 Herz to 1000 Herz is the siren like sound. The sharp discontinuessounds are the chewing sound that is the result of eating small dots. As the screen shotshows the chewing has an emphasis on a motion that goes down. ” PAC-MAN” Characters &Musical Works © 1980 NAMCO LTD. ALL RIGHTS RESERVED

This can be written like ” Puca Puca” or ” Paca Paca” or something similar. It

has the basic properties of chewing. You will of course write and spell out the

sound differently in different languages since the whole language structure

will affect how such onomatopoetic words are written and realized. They will

inherit much of the rest of the language so to speak. The chewing sound

blends with the surrounding language.

60

One of my students, Eva Gustavfsson, has successfully applied this schema of Foss onto Super MarioBros. (Nintendo 1986) with a similar result in an assignment for one of my courses at the Department of humanities, University of Skövde, Sweden.




190

There is also an oscillating sound, which could be described as a slow paced

not-too highly pitched police siren. See the screen shot above and listen to

the example file PACCHOMP.WAV on the CD-ROM. The endpoints are the

most significant in a glissando between two basic frequencies (from 500 Herz

to 1000 Herz as the spectrum above shows). There is a very clearSOURCE–PATH–GOAL structure in the sound. This sound is getting faster

and more high-pitched as the game progress and the Game Ego (Pac-Man)

chews up the dots and the ghosts. When chewing bigger blinking dots the

ghosts as mentioned, all turn blue and get a different shape (BIGEAT.WAV).

They are now eatable. The sound changes as well. A rapid oscillation

between two high-pitched tones is replacing the previous sound. However,

this sound does not change over time as in Elk Attack on the Atari 2600

platform. In Elk Attack, the pitch is lowered when the Game Ego loses

energy and this makes it easier to plan game play. Eating a ghost results ina sound that is a rising scale (UP = GOOD) and this sound could be likened

with a swallowing sound (GHOSTEA.WAV). An eaten ghost is reduced to a

pair of eyes that swirl to the “ghost-home” at the center of the maze. The

sound for this is a sharp high-pitched two-tone siren like sound. (The Game

Ego can not enter the “ghost-home”). The death of Pac-Man is a falling scale

(DOWN = BAD) that ends with two rapid staccato tones that goes up

suggesting a possible continuation of the game (KILLED.WAV). Even if the

game states Game Over, you may always, as long as you have one, put in

another coin and restart the game.

However, how do we conceive of a tone scale rising or falling? It is quite

simple, as chapters 5 and 6 show. To produce a high pitched (high frequency-

rapid vibration) tone with the human voice one has to put in more energy to

the vocal cords than to produce a lower pitched tone. It is a question of the

body at work or the body at rest. The ultimate rest is death. Death is total

relaxation of the body.

Sounds also have physical responses. The increased speed and pitch of thesound provide a stress factor. This gives a sensory immersion. When we are

stressed we tend to get more focused on what we are doing. We focus on

what is important at the time being and for fulfilling certain purposes. The

stress caused by sound is providing focus on the game process and gives the

perceptual system a chance to suppress irrelevant sensory input.

Furthermore, when playing the game of Pac-Man (or any similar audiovisual

game) one is composing an audiovisual environment. One is a composer of

sound and images. The sounds produced by the enactment of being withinthe game environment do have a musical value. They are possible to play




191

like a musical instrument. The Game Ego has a sonic function within the

game environment as the agency of composing. What happens when one

plays a game, like Pac-Man, is that a kind of sonic/musical piece is created.

Video gaming is not only about manipulation of images but also about the

manipulation and arrangement of sonic entities. Pac-Man and any otheraudiovisual game can be played much like a musical instrument. They are

rhythmic and dynamic sound experiences.

With this short and summary analysis of the sound I hope to have shown

that sound does have the same properties as visuals in some respect. They

are able to function at a basic level.

9.4.3 The Ghosts’ UP–DOWN relation

I would say that the most interesting characters in The Pac-Man game are

the ghosts (Shadow, Speedy, Bashful and Pokey). They are seen from their

front with their eyes indicating their direction of motion. When moving

SIDEWAYS, their eyes are aimed at the corresponding side, when moving

DOWN their eyes are aimed DOWNWARD etc. However, there is one funny

thing about this. When moving UP their eyes move UP on their heads

suggesting a change in visual perspective making the ghosts seen as from

above. They perform the same act as the characters in Bubble Trouble: they

turn the back towards the game player but here the back is seen as fromabove. There is no other change than this relocation of the eyes in the visual

representation, suggesting a shift of perspective, but its is enough to

establish the same schema of movement. UP = AWAY and DOWN =

COMING CLOSER.

This UP = AWAY and DOWN = COMING CLOSER can be related to the

physical setup of the computer hardware. The control device used for playing

the games are most commonly placed somewhere BELOW the screen on/in

which the action takes place. The relation between what is close and far

away is a relation between the game player’s hands and what he may control

with them. The inference of the images is in other words related to the

physical medium and the game player’s body. Note that the screen in the

image below showing an arcade console of Pac-Man is not vertically

positioned but tilted down. Note also the location of the control device below

the screen.

This will give another visual inference of the images shown on the screen.

This is due to the connection between the inner ear and the eye muscles that

feed our perceptual system with information concerning our bodily position




194

9.5 Bubble Trouble

This is a very short analysis related to Pac-Man. The game idea is very

similar to Pac-Man but a bit more elaborate. The objective is to have a

yellow fish eat other animals (mainly other fish and such) within a mazestructure. The maze is not static, however, since the borders of it (to some

extent anyway), are possible to both move around and destroy. This in turn

is similar to an older game called Pengo (Sega 1982).


This game has two basic different visual perspectives. The Game Ego, in this

case a simulation of a yellow fish, is manifest by visuals showing it either

from the front, the back or from one of both its sides i.e. the Game Ego isshown from canonical points of view.

The immediate and first understanding of the game, however, is that it

represents a game environment as seen from above. In a little (non

statistically bulletproof) survey I found that 10 out of 10 persons understood

the game environment as seen from above. However, this is only a first and

immediate impression. There are ”air bubbles” coming from the bottom of

the screen ascending towards a non-visible water surface above. Things get

even more complicated when one starts to play the game. Then the conceptsof UP–DOWN and SIDEWAYS are launched, so to speak. Those concepts are

related and refer to the computer screen, which have an UP/TOP-

DOWN/BOTTOM and SIDEWAYS (LEFT/RIGHT) orientation, especially is

this the case when one moves the Game Ego SIDEWAYS. The fish is then

moving in its frontal direction (its head and eyes in the direction of its

movement) but the game player can only see it move from one side of the

screen to the other.

When moving the Game Ego UP it turns its back on the game player. Whenmoving DOWN it turns its front towards the game player. When moving

DOWN the eyes of the fish are looking DOWN giving emphasis to this

movement. This suggests UP = AWAY and DOWN = COMING CLOSER. It

is a natural mapping of space in play here. (A game player acts out and is

acted upon by the game).




195

9.6 Defender

What I like about computer games is that they playwith your survival instinct. That’s the big difference

between computer games and pinball.. (EugeneJarvis quoted by Lowe, Jr. Playboy March 1982).

Defender (Williams 1980) is mainly a creation by Eugene Jarvis of whom the

netizens into computer games often say “he made the best games of the early

1980s”. Of course, this is a subjective statement but since we are concerned

with subjective experiences of interactions such judgments are of interest.

Such judgments bear witness of how game players of the era experienced the

games. (See also chapter 1 on source critique for a more elaborate discussion

on this topic).

The objective of this game is to save fellow humans on the ground by picking

them up in your little space ship, which is your Game Ego.


As with many of the games Jarvis has been involved in, control is an

important issue of Defender. To manipulate and control the Game Ego and

create a strong link between the game player and the Game Ego seems

essential for Jarvis approach to computer games.

Eugene Jarvis games are the absolute best, thepinnacle of video achievement. They reek of innovation--this is the man who INVENTED thesmart bomb and the twin joystick control. They areplayability. They define the term. Games likeRobotron and Defender throw seemingly impossibleamounts of problems at the gamer. But, at the sametime, they gave the gamer total control over what

happened. (Sometimes they gave a little too muchcontrol - how many buttons did Stargate have - 32?)

These games not only enable, but demand that thegamer ’Get Into The Zone’. Once in the Zone, thegame loses its level of abstraction. The gamer stopsthinking about moving the joystick here andshooting that, in fact, the gamer stops thinking atall. The feeling is exhilarating - almost mystical. You simply see and react, but you are coping and

winning, and you don’t really know why. You startplanning several seconds ahead of what you are




196

currently doing, but this is still not happening on astrictly conscious level. Planning isn’t the rightword. You simply know what’s going to happenseveral seconds down the road -- and you’re dealingwith an insane amount of problems, and you’rehandling it, and you’re happy. Then you get thewarm, fuzzy feeling of, Essential Coolness. (BrookBaklay http://www.geocities.com/SiliconValley/Haven/5521/index.html 2000-05-21 15.43).

The Game Ego of Defender is a little space ship with a distinct

FRONT–BACK orientation. Just one glance at the Game Ego reveals this.

How come one might ask. It is very simple. The Game Ego, i.e. the space

ship, is shaped in the likeness of an arrow and a sports car. The shape of an

arrow has also a very specific FRONT–BACK orientation as has thesportscar. As a game player one has tremendous control of the Game Ego. Or

rather, one can have it. It takes a lot of motor skill to control the Game Ego

and fight of the opponents in this game. That is what makes it fun. It takes

long time to master Defender, and that is one of the points of the game.

The environment in Defender is not possible to have in full and complete

overview. It is a side-scrolling environment. To my knowledge, this game

was the first to have such a presentation of the game environment. It means

that UP–DOWN and SIDEWAYS are strongly emphasized. The fact that youmay scroll the environment both left and right, stresses that the

SOURCE–PATH–GOAL schema is dynamic and fluid. You will have to

search the environment to solve the game, i.e. successfully carry out your

mission to save all your fellow humans. The suggested direction of motion

the Game Ego has when the game starts might show less successful than its

counterpart. You might have to turn around and search the ground and sky

and move in the opposite direction than the suggested one. This is similar to

what is going on in Spyro the Dragon (see chapter 5 on perceptual and

cognitive sets, 108).




197

9.7 Duke Nukem 3D

Duke Nukem 3D is an action game in first person and three dimensions. The

episode I have chosen for my analysis is LA. Meltdown played at ”Let’s rock”

skill level.61

The visual perspective in the game Duke Nukem 3D (and that of many other

action/shoot–'em–up games) is a simulation of a first person when the game

starts. The visual perspective is a representation of what you would see if

you were there and as such actually put you into this situation, that is, into

this world.

The SOURCE–PATH–GOAL Schema is continuous within each level of

Duke Nukem 3D. It is only between levels that the environment is broken up(or when one uses ”teleporters” to move in within a level and of course, by

death of the Game Ego before completing a level). The main part of the game

has a continuous visual space. This space is often chosen by the game player

to be that of a first person. This space is not possible to overview completely

by the game player since the whole game environment is much larger that

what the visual field can contain.

1) Initially we see the hands, or rather the hands loading a gun. This action

lies beyond our control. We do not actually see much of the hands later on inthe game: we see the different weapons and the hands are visible when

loading or triggering weapons.62

2) We can also see the feet when we perform a certain command on the

keyboard.63

3) The whole character can be seen on surveillance screens, in mirrors and

from above or behind if we want to change the visual perspective of the game

from first person to third person.

The point of audition in the game Duke Nukem 3D (and in many other

action/shoot-'em-up games) is a simulation of a first person. The point of

audition is the same as the point of view, which makes up a coherent point of

being. The point of audition and the point of view are connected to the motor

action of the controllable character. When there is a change of being there is

61 Duke Nukem consists of four episodes and four different skill levels.62 What you can see of the hands depends on which weapon you choose and how you set your screen. If

you choose to leave out the information about your status and weapons in the screen’s lower part you willsee more of the hands.63 This command is up to you to decide but there is, as in most games, a default setting available.




198

a change of point of view and point of audition. This gives emphasis to the

embodiment of perception i.e. a subjective rather than objective experience.

It is subject centered.

We can hear the voice, of which we have no control. That is, we can notdecide what the Game Ego says.64 The Game Ego comments on actions in the

game and even addresses the game player directly as the game player with

lines like ”What are you waiting for?” This comment creates a certain

distance between the game player and the Game Ego, but this distance is

soon outweighed by the motor actions the game player can perform.

We can hear grunts of opponents, which is a subgroup of voices.

There is also noise created when moving around e.g. thumps of feet on boxes,

sound of feet in ventilation drums etc. And of course we can hear the sound

of weapons used.


Which actions can be performed? Or, even more interesting: How can actions

be performed? Let us first study some actions that can be performed in Duke

Nukem 3D:

1) Moving in all directions

2) Control weapons (change weapon, holster/upholster weapon and firing,

throw pipe bombs, aim weapon etc)

3) Kicking with one foot

All input is done by direct keyboard manipulation, or if the game player

prefers, combinations of mouse and keyboard control and input control motor

action.65 The Game Ego in Duke Nukem 3D is due to these factors motoric

rather than verbal. The identification process at work is of a bodily kind,involving direct motor interaction with the game environment. The actions

which the game player is allowed to perform constitutes the Game Ego. The

directness is of great importance. The Game Ego is not only constituted by

64 Other games could let the game player have certain control over what a character utters. Day of theTentacle (LucasArts 1993) is an example. In this game, the game player is allowed to choose from alimited set of lines. This means that the game player knows what the controllable character is about to saybefore he, she or it says it. This does, however, not mean that the game player knowshow the controllablecharacter will say the line. Qualities of the voice, its timbre, loudness etc. are not known before hand

through typography.65 The game player is allowed to decide which keys should control what. You may also use a joystick,however I have found it hard to successfully play this game that way. On the other hand, you might not.




199

its visual and aural appearance. It does not matter whether a game is

visually presented on a flat two-dimensional screen where a complete

overview is possible or a complex three-dimensional world with landmarks

both aural and visual; the tactile motor/kinesthetic link is what matters the

most. The game is a bodily experience made up of more than just sight andhearing, since it also includes motor action. In the case of Duke Nukem 3D,

we can not see the complete environment at one instant. One has to move

around and find ones way through it. In this respect, the environment is

much like a maze.




200

9.8 NHL 2000

As noted earlier, Lakoff and Johnson propose that we as human beings arefrontward oriented. Our primary concern is aimed in the direction of our

eyes. Our bodily movement possibilities are primarily aimed frontward. We

have encountered this in the above analyses of Pac-Man, Bubble Trouble,

Defender and Duke Nukem 3D. Zork also shows traces of this, as do Spyro

the Dragon analyzed in chapter 5, 108. Let us return to a quote that we

already have studied.

Since people typically function in an upright

position, see and move frontward, spend most of their time performing actions, and view themselvesas being basically good, we have a basis in ourexperience for viewing ourselves as more UP thanDOWN, more FRONT than BACK, more ACTIVEthan PASSIVE, more GOOD than BAD. (Lakoff,Johnson 1980, 132. See also the present workchapter 5, 82).

When playing computer games one uses this frontward orientation. A study

of the game NHL 2000 will show how this can be utilized for theimprovement of game play, i.e. the game’s playability.

Before doing the analysis I will introduce the game company Electronic Arts

and its brand EA Sports since it provides a context to read the analysis

within.

NHL 2000 is an ice hockey simulator from EA Sports. There are previous

versions of the product like NHL 98 and NHL 99. One of the basic things

about this series is that it provides an extension of the real National Hockey

League. The teams in the game are the same as in the NHL as are the

players. The faces of the real NHL players are rendered on the virtual

players of the game. That gives that one can easily detect players like Peter

Forsberg, Pavel Buré et. al. The game player can also render his or her face

onto a player and of course give a player his or her name.




201


The game player is allowed to decide the visual perspective from which the

game will be viewed from. There are several choices that can be made, such

as overhead cameras from different angles, cameras from the long side andfrom the perspectives of an individual player, to mention just a few.

The overall visual style of the game can be best described as a mix of

contemporary American television broadcasting of NHL and collectible

hockey cards. It is not primarily founded on earlier hockey games such as

those available on the Nintendo Entertainment System of the 1980s where

the game player could see the whole rink, or a scrolling rink, as from above

(like Ice Hockey from Nintendo 1988). The visual field in NHL 2000 is open

for blending many graphical elements of different kinds. The entire rink isnot kept in sight when playing the game. The spatial construction is

obtained by either the simulation of moving the “ virtual camera”, or by

changing the visual angle by a cut, fade or wipe of some sort. The simulation

of camera movements is of complex nature. The camera movements simulate

the camera movement of a broadcast NHL game. It would be perfectly

possible to keep the puck in the center of the visual field with out any delay

of the “camera”. The game engine must “know” where the puck is going and

hence would be able to monitor the puck constantly. But it does not do so.

One reason for this could be that the game player likely has the experienceof taking part in hockey through television broadcasts. In a broadcast

situation, the camera operator can only make a more or less intelligent guess

of where the puck goes and hence there will be a delay in the camera

movement when the operator tracks the puck. NHL 2000 does the same

giving the impression that there really are some camera operators (and a

producer) providing the images. The conceptual ground of the visual style is

the televised hockey game and not the game of ice hockey in itself.




202


The overall audio style is that of contemporary American broadcasting of

NHL featuring excited on location expert commentators, lots of cheering or

booing crowds, music accompaniment, players judgmental comments of thereferee, and the sounds of the game, is heavily emphasized. When players

hit the side walls of the rink it sounds just as it should – it confines itself to

the audio style of television by rendering the sound with lots of low

frequency attack. The game uses 3D-surround sound as do many broadcasts

of NHL games. This provides sensory immersion. The game player is

immersed in the game environment through this world’s sound as well as

through its game play.

The game player can choose which sounds there shall be in the game playingprocess and the relation between the sounds. This is a very common practice

within computer games. (Compare this with Day of the Tentacle, for

instance).


The game has a complex set of parameters. Some of them relate to the

computer game as such (keyboard set-up, visual perspective, detail level etc.)

and others to the set up of the hockey team (choice of which players will playwhat position in the team etc.).

Direct control or direct object manipulation are significant to this game.

Which control keys that control what is tightly connected to which viewing

mode is chosen i.e. which virtual camera set up the game player chooses to

have. Furthermore, the game player may choose how the game shall

represent the Game Environment. I will only comment on some of the

choices:

Overhead 1: Displays the rink from above at a reasonably straight angle.

The whole width of the rink is always in display. This of course distances the

game player from the game. The visual field has a top-bottom configuration

with one goal at the top of the screen/rink and one at the bottom of the

screen/rink that matches the keyboard control. Arrow key up is towards the

top of the screen and vice versa.

Overhead 2: Displays the rink from above with a slightly greater angle and

closer to the rink. The game is thus acted out closer to the game player. The

configuration of the rink is the same as Overhead camera 1.




204

FRONTWARD is either left of right since the Game Ego is displayed as a

side view.

To make some short connections to Gibson, we can note that the medium in

NHL 2000 is a simulation of our daily medium, air. It refracts light, (if welike it to and if our system is set up for it that is) it permits visually

controlled locomotion, which is naturally constrained by the hockey rink. We

can also hear the sound of cheering crowds so the medium also allows

audition. We cannot smell popcorn or hamburgers though. The medium does

not afford olfactory information. This might seem ridiculous even to mention.

However, I do feel it necessary at to least think in such terms. Which

affordances has the computer as medium and which affordances has the

environments presented by means of the computer? Since we are able to

simulate coffee drinking, the taste and the smell of coffee it is hence not astupid thought but a relevant question. (See Grodal on this in chapter 8,

paragraph 8.3.1). How will we make the game player sensory immersed in

the game? By which means will we do this? By tactile motor/kinesthetic

action, visually controlled locomotion, auditive information or visual

imagery? Or all of this? With this, I would just like to point out that there

are important questions to be asked that we probably do not ask ourselves

while interaction with or creating computer game environments – not on a

conscious level any way.




205

9.9 Day of the Tentacle

George Lucas founded LucasArts Entertainment Company in 1982.

Considering Lucas being a filmmaker with some major box office successes

on his record, it is not surprising to find games developed from the stories orcharacters from films like Star Wars and Indiana Jones among the titles

from LucasArts.66 There are also original stories developed such as Full

Throttle, The Secret of Monkey Island etc. Lucas as a filmmaker is well

known for his interest in special effects, some of which have been

incorporated one way or another into the games. Many of the games also

show traces of filmmaking. There are often quite long animated sequences

that are nothing less than movies found in the games from LucasArts. It is

also not very surprising to notice that sound is important for this company

and the Games coming from it. George Lucas is one of the persons behind

THX and he has shown a great interest for sound use, sound production, and

sound reproduction in his movies.

9.9.1 Description of the game

Day of the Tentacle (LucasArts 1993) is an adventure game from Lucas Arts.

It is sort of a sequel to another game, namely Maniac Mansion (LucasFilm

1987).67 Day of the Tentacle has a clear fabula and a somewhat dynamic

syuzhet structure.68 The fabula could be described as follows:

At his mansion the (mad-) scientist Dr. Fred has in the likeness of Dr.

Frankenstein created a new life form: the tentacles. One of them, Purple

Tentacle, drinks water poisoned by Dr. Fred’s Sludge–O–Matic machine and

from that point on he becomes evil. Evil, in this case, means that he wants to

enslave the human species and take over the world. A visual sign of his

evilness is that he mutates and gets arms which Tentacles do not have in

66

Lucas is a successful director and executive producer. He has not directed nearly as many movies as hehas produced, though. Among the movies he has produced we find the movies in the Star Wars series(1977, 1980, 1983, 1999), and the Indiana Jones series (1981, 1984, 1989) to mention just a few.Characters from these movies have been used within games from LucasArts.67 With this particular game, LucasArts changed the user interface style of adventure games from beingparser-driven to the point-and-click interface. They developed a story engine, SCUMM (Script CreationUtility for Maniac Mansion) which is still at the core of their adventure games. In Maniac Mansion,student heroes are off to rescue a cheerleader from a mad scientist in a Victorian mansion. ManiacMansion was later on (1990) licensed to Jaleco for the Nintendo Entertainment System.(http://www.lucasarts.com/static/pr/mileston.htm 2000-05-22 10.24)68 I say somewhat dynamic since there are situations in the game that demand that an earlier situation orseveral situations be successfully solved before the present situation is possible to execute. An example of this is: you have to first obtain a "Help Wanted" sign as the character Bernard, then put it in a time

machine and send it two hundred years back in time to Hugie and then hand it over to a forefather of DrFred’s, Red Edison, to get access to a workshop and something you'll need later on.




206

their original form. The outgrowth of arms makes him in a sense more

human i.e. he fits better the image schema of a human being. Three

youngsters, Bernard, Hugie and Laverne, are then called in by a human-

friendly tentacle (Green Tentacle) to stop him. To do this they have to travel

back in time to the day before and turn of the Sludge–O–Matic and hencehinder Purple Tentacle from drinking the poisoned water and by doing so

stop the situation of a hostile Tentacle takeover of the world from ever

occurring. When performing this time travel with the help of Dr. Fred, the

time machine and its modules break down and send the youngsters to three

different ages. Hugie is sent back two hundred years (which makes it

possible for him to encounter George Washington, Benjamin Franklin,

Thomas Jefferson and John Hancock). Laverne goes two hundred years in to

the future and is then caught in a world completely overtaken by the

tentacles and where humans are held as pets and slaves. Bernard is stuck inthe present and has to deal with the problems of our own age such as the tax

authorities. By cooperating over the centuries with the help of the, to some

degree functional, time machine modules, the three youngsters are able to be

brought back to present time. After some more problem solving, Purple

Tentacle’s attempt to take over the world is put to a successful end.

That is the basic outline of the fabula. The syuzhet however is a bit more

complicated since it is somewhat dynamic and dependent on the game

player’s actions. Of course, there is a fastest possible solution to the game,i.e. one could perform all necessary “moves”, so to speak, in a best possible

order. However, this is not the obvious purpose of game playing in this type

of game. Why spend a lot of money on a game that is too easy to solve and

play through? The syuzhet is suggested by the setup of the locations. The

contents of the different rooms in the mansion and the surroundings, in all

three ages, all provide material, manipulable objects necessary for the player

to find and use.69 These objects might in some cases have an obvious

affordance that is distracting for the contextual use. It is perhaps not that

obvious to first blow George Washington’s teeth to pieces and then replace

them with a set of mechanical chattering teeth, is it? You have do a bit of

thinking before coming up with such an idea. It helps to know that

Washington had wooden teeth. In Day of the Tentacle, one is likely to try to

use the chattering teeth on a mummy that is supposed to take part in a

beauty contest but this will not work. The mummy shall instead have set of

teeth from a horse. Yes I know this all seems absurd but within the game,

absurd logic is used extensively. It is what makes such games fun and

69 To my knowledge there are just few things, that does not have any specific purpose other than beingconfusing.




207

entertaining in the first place. It also shows that some games have an overall

narrative structure that is meant to be enacted. They shift from a traditional

narrative mode with things being told to you as game player and situations

that you must enact and realize yourself. In Day of the Tentacle the border

between these modes are not very sharp. However, when there a cut-scenesthese are displayed in a more wide-screen like format than the actual game

sequences.

Now let us get back to the different time periods in which the characters end

up due to the cheapness of Dr Fred. There are reasons for the characters’

placement within their respective situations and periods.

Hugie is a roadie. He is used to work with heavy musical equipment, solve

technical problems and so on. He also plays some instrument in a band.Hence, he is suited to assist both a forefather of Dr. Fred and Benjamin

Franklin’s exploration of electricity.

Laverene is a ” sociopath” (which is maybe best shown in her vocal statement

early in the game that a hamster sent out by Green Tentacle as a messenger

could be used in her dissection lab the following day. She is forced to deal

with social interaction, such as conversation with other characters, within

her struggle to get back to her own time. There is for instance a sequence

where Laverene has been put in prison together with some ancestors of DrFred. She must then talk herself out of the predicament rather than using

violence and physical force. She is also later to perform some rather horrid

acts concerning the mentioned hamster, such as putting it in a microwave

oven.

Bernard is a ” smart dumb kid” and he constitutes the link between Hugie

and Laverne placed somewhere in the middle of their respective

characteristics and hence fits nicely within the present if we conceive of the

time as a linear scale. All these characteristics are established in the first

encounter we as game players have with them and they are made explicit

from written and spoken language and is implicitly hinted at in their visual

appearance.

Day of the Tentacle has a point and click interface. The characters are

indirectly controlled by the game player by first choosing which action the

Game Ego is to perform and then by pointing with the cursor assign the

Game Ego to perform the action. This is a much weaker link than a direct

control. It is time consuming, i.e. it takes a noticeable period of time, a




208

duration of time, between the choice of action and the realization of the

choice.

The visual style of Day of the Tentacle is perhaps best described as two-

dimensional animated color movie, a cartoon. The colors are mainly clearand bright and the different objects within the field of view and in the game

environment are clearly distinguishable. Not all objects are manipulable

though, which at least in the beginning of the game is a bit frustrating.

The over all sound style of Day of the Tentacle is within the tradition of

classical Hollywood cinema and especially that of adventure/detective and

monster movies. There are thematic background music and audible dialogue

through out the game.

Day of the Tentacle allows the game player to choose whether he or she

wants to use spoken language or written language as the means for

characters to communicate. It also allows for having both spoken and

written language at the same time and adjusting the speed of displaying

written language. The game player may choose the relative loudness

between sounds such as background music and dialogue. It is hence possible

to play the game as a whole or parts of it without any background music.70

Day of the Tentacle is not a game that relies only on the images and their

relations as narrative points. It is also a game, which emphasizes explorativeinteraction of the written and spoken (English) language that make up the

dialogue in the game. It is a game of wordplay in which the meaning of

speech-acts are highly contextual and coherent with the game environment.

The preprogrammed multiple choice speech-acts of characters, (what John

Alexander would call “opti-linear” speech acts) are therefore of great

importance in order to successfully play the game through. Playing the game

successfully means obtaining the goal and hindering the game story from

ever happening in the first place. The dialogue structure builds around the

speech-acts branch points. The act of choosing a specific utterance opens up

the dialogue and creates a path to follow, as my analysis below will show. It

also temporarily cuts off other branches from the path. This creates a focus

in the dialogue that is quite easy to follow when once noticed. There is no

mixing of topics within the branches or paths, but an exploration of its initial

features even if the initial features are perhaps not so obvious when first

making a choice to explore it. The branch reveals information in-depth, so to

speak. From the initial choice of a branch, the speech-acts give away clues

70 To play the game without music I personally would humbly suggest not to do since a lot of the joy of this games stems from the music soundtrack and its use of clichés from classic Hollywood cinema.




209

for further exploration of the game environment. The speech-acts have the

function of providing parts of the configuration that make up the game

environment – i.e. the whole. The speech-acts are to large extent concerned

with providing ACTION–LOCATION metaphors to the game player (see

chapter 5). You cannot do certain things if you are not in the right location.This is not a feature unique to this particular game though. It is rather

standard procedure in game design to do so. In a game like Zelda II, for

instance, you are given clues from wise men and others that lead you to

specific places that often is the hiding place for needed objects such as

gloves, boots etc. Utterances such as “with boots I could walk on the water”

is not only an utterance that reveals what possible action there is to perform

but also where this action can be carried out. Remember a sentence like “I

start the car” that reveals not only which action is performed but also where

the action is carried out. (on the ACTION–LOCATION metaphor in chapter5). The ACTION–LOCATION metaphor is answering two basic questions:

what happens and where does it happen. It also provides constraints on

actions and location as well as affordances. If you are at a specific location,

like when you are starting your car, you are obviously not performing certain

other actions such as taking a bath and so on. In Day of the Tentacle, the

dialogue structure is constraining certain actions as well as permitting and

encouraging other actions. In a way, one is as game player somewhat caught

in a word environment, an environment of speech-acts suggesting what to

do. It is neither pure action on action or pure word on word interaction but

rather a combination. However, there is always a motor act involved since

the interface is construed as a point and click system. And even if you were

to talk or type commands such acts are motor acts.

The purpose of the following example is to clarify how the kind of multiple

choice branching dialogue described above works. The character Bernard

enters a room in which he finds the character Nurse Edna. She is sitting in

front of some surveillance equipment consisting of among other things a

VCR and some video screens. I will describe both the input one has to

perform as a game player, that is one’s actions, and the output in which

these actions result. It is possible and suitable to conceive of this process as a

cause and effect chain with the game player as the producer of something

and the game’s output as the product of the producer’s actions. As mentioned

the ACTION–LOCATION metaphor is important for the understanding of

the game environment in Day of the Tenatcle. In the analysis I will comment

on how the speech-acts constrain and suggest (by actually trying to put a

constraint on the use of things) how to explore that game environmentfurther and how to manipulate which objects by making comments on them.




211

parts are still the same when moving to a new space? If the controllable

character Bernard is ordered to ”Use Door” and he walks towards the door

and suddenly appears in a different room, he is the thing that is still the

same. Sound could be another link between spaces as it often is in film. The

SOURCE–PATH–GOAL Schema of this game has more similarities with afilm than with the early computer games.

9.9.3 How is a Game Ego constituted?

In this kind of game we have a LINK Schema but it is not based on tactile

motor/kinesthetic action as much as visual appearance and spoken language

information. The game is not as much action as it is analyzing information

given by spoken language and visual information. Since there are three

controllable characters it is also hard to focus on one of them, even if Bernard has something of a superior position in the game, being the first

character that is controllable and being the initiator of the action in the

beginning of the game.72 He is also the one who is cast in present time;

Hoagie is placed two hundred years back in time and Laverne two hundred

years into the future. As mentioned in chapter 7, interaction is a word so

filled up with content that it has become more or less meaningless. Or

rather, very personal. Depending on whom you ask what interaction is you

will have different answers. I will use two terms in the following analysis

that might need a definition. They are:

Vocal interaction/interactional speech-acts, by which I mean spoken

language in several levels, in this case in a game, (dialogue, monologue etc.).

Motor interaction, by which I mean both the direct and indirect tactile

motor/kinesthetic link established between the game player and the game.

Both of the terms above could be divided into several levels depending on

type, address, and context. They also relate to how we tend to personify the

computer game system as Scheibe’s and Erwin’s study shows and that weencountered in chapter 7, 127. See tables 3 and 4.

Table 3: Vocal interaction events/ interactional speech acts

1) Question answer

72

And therefore the closest. Closeness is strength of relation.




213

is a conditional change in the visual flow as well as in the sound flow, i.e. it

depends which modes (written or spoken language or both) are chosen by the

game player. Bernard says: ”Excuse me...”. Nurse Edna replies to this in one

of several ways of which I just cite a few:

”Hi there stud. Eeeheeeheeeeheee.”

”Hi there my little squinky poo. Eeeheeeheeeeheee.”

”Hi there schooky doodle. Eeeheeeheeeeheee.”

”Hi there googly woogly. Eeeheeeheeeeheee.”

”Hi there you manly hunk, you”.

Edna’s voice is graphically presented by orange text i.e. a color that is

different to the one which by which Bernad’s voice is presented. The color is

personal and is part of the graphical prosody, so to speak. Note that all of

Edna’s answers reveal something of her personality: She is quite found of

men and likes to flirt. However, this conclusion is perhaps only possible to

draw if one repeatedly performs this action of talking to Nurse Edna.

When Nurse Edna has answered the game interface will change, which is a

change of the visual flow, and from this also follows a change of what is

possible to manipulate. The interface now shows a set of dialogue choices

which when chosen trigger what Bernard will say. This change in visual flow

introduces a turn-taking procedure. From the starting point of the dialogue,

of the conversation, where talking, communication, was initiated it is now

time for turn-taking and this is ruled by the set up of the interface.

We have three alternatives to choose from. I will call this part of the

dialogue the branching point when referred to later in the text. Let us first

study each alternative one by one.

”How’s Dr. Fred doing?”: This alternative aims at Nurse Edna in her role as

a nurse i.e. a medically trained person able to give judgements on peoples’

health. It is a question concerning Dr. Fred’s health and it will put the

conversation along a branch that puts some light on how Dr. Fred behaves

and the reasons for his behavior. This in turn provide crucial information

about were to find an important document that is necessary for the game

play and solution of the game. The ACTION–LOCATION metaphor evolves

around Dr Fred’s characteristics and his past actions.




214

”This is quite an array of gadgetry you have here!”: This speech act focuses

the attention toward the technical equipment in the room. It has not to do

with Nurse Edna in her role as a nurse. Rather it is concerned with her role

as an authority and surveilour. It is a statement of astonishment. This opens

a branch, which provides information about the surveillance system’s visualabilities, which of course also is an important clue for the solution of the

game. The ACTION–LOCATION metaphor suggest that this room is made

for high-tech purposes and that certain actions of game play related to such

issues must be carried out here.

Both of these speech-acts are quite natural and are both related to what is in

the visual part of the game environment i.e. they are both related to the

visual field. The question about Dr Fred’s health is natural to put to a nurse

and the comment on the contents of the room is an obvious comment thatanyone with a technical interest might perform within such a location. This

gives to say that the location and the content of the location motivate these

initial speech acts.74

“I’ll let you get back to what you were doing”: This is the end point of the

dialogue. It is a statement. It does explicitly invite Nurse Edna to do or say

nothing more than possibly good bye and getting back to what she was

doing. This speech act closes the dialogue event

Let us take a look at how this multiple choice dialogue evolves.

Input to game (cause): Point and click the text: ”How’s Dr. Fred doing?”

Output from game (effect): Edna replies vocally and/or with written

language ”Well he’s still upset about the family financial situation, seeing it

as his fault and all, but he seems a lot better now since he stopped

sleepwalking”. This sentence reveals a lot of information about Dr Fred. He

is upset and has been so for a while. He and his family do have problems

with their finances. Dr Fred blames himself for this economical situation. Hehas a history of sleepwalking but now he has recovered from that. He does

seem to feel better since he quit sleepwalking. All this information

encourages the game player to further explore each possible branch. We also

do have a change in the visual flow that gives us five new alternatives for

Bernard to utter:

74 Of course, one might be retrospective and make this comment on the room at hand later to someone but

since there is a character present in the room and this person also is engaged in some kind of interactionwith the machines in it is only natural to perform the act of astonished comment in this location.




215

”How’d he wreck the family finances?”

”What’s wrong with sleepwalking?”

”How did he manage to stop sleepwalking?”

”Well enough about Dr. Fred...”

”I´ll let you get back to what you where doing”.

As we see the information given in Nurse Edna’s utterance is put into

consideration and is up for a deeper study. I will call this the first choice of

branch one when referred to later in the text.

Input to game (cause):

”How’d he wreck the family finances?”


language ”Well, We should have made millions on that computer game they

made about our family, but the resident genius locked the contract in the

safe in his office and forgot the combination”.

What is made explicit in this utterance is that millions of dollars are

withheld because of Dr Fred’s action. There is a safe somewhere, probably inthe house, in which there is a fortune locked. Since one of the first rooms one

as game player has access to prior to the analyzed sequence does contain a

safe it is not to far fetched to hold that that safe is the safe in question.

Implicitly it is said that Dr Fred has a bad memory for numbers and it is

also hinted at a tense relation between Nurse Edna and Dr Fred since she

calls him “the resident genius” in quite an ironical tone. This gets somewhat

lost in then written language context though.

This leaves us with four alternatives for Bernard’s next utterance:

”What’s wrong with sleepwalking?”







216

As shown we do not have any new possible input but just the omission of the

question just asked. The branching is closing and getting narrower.

Input to game (cause): ”What’s wrong with sleepwalking?”


language ”Ordinarily nothing, but when Fred sleepwalks he remember the

combination to the safe. I find him in the office, opening it, screaming like a

cat in the oven, slamming it again, something about what’s in there really

scares him. Unfortunately I was never able to catch the combination since he

works it so fast.”

This leaves us with three alternatives. Nothing is added but just as in the

previous case, the question asked is omitted.




Input to game (cause):



language, ”He stopped sleeping. Fred drinks a LOT of coffee. Me, I only

drink decaf.”

This causes also a change in the alternatives left. Instead of just omitting

the last question asked we are given these two alternatives:

”This is quite an array of gadgetry you have here!”


As we can see the passage ”Well enough about Dr. Fred...” is omitted and the

second alternative i.e. ”This is quite an array of gadgetry you have here!”

from the start of the dialogue is back again leading us to the next branch of

the dialogue. Up until this point, we have played the dialogue in a causal

way from top to bottom. However, this is not necessary. One might choose

any order.

Input to game (cause): ”This is quite an array of gadgetry you have here!”




219

processing written language and spoken language we will encounter

problems further on if we omit the possibility to use written language as well

as spoken language in a game like this.




221

initial hypothesis is based on our everyday experience and on other game

experiences. It is not the case that there really is some kind of dichotomy

between these two. Game experiences might be a part of the daily routine for

some people, for instance. They are to a large degree interdependent, and it

is not possible to make an absolute distinction between them. To clarify:game playing involves other everyday activities since it would be virtually

impossible to play a game without any experiential knowledge of the world,

as this dissertation shows. Our basic cognitive schemas, which we need for

game playing, are established already in our infancy. They are even innate

capabilities of the human mind. In addition, game playing also makes

possible new neural connections between primary metaphors, i.e. new

complex metaphorical concepts. Primary metaphors are as we have learned

the cross-domain mapping of sensorimotor experience upon subjective

experience. There is in other words a tight connection between acting withinthe environment and conceptualizing it. This goes for the environment that

usually surrounds us and the game environment as well. (For a more

detailed discussion on this see chapter 5 of the present work. See also Lakoff,

Johnson 1999, chapter 4 and Lakoff, Johnson 1980). Playing an interactable

computer game is cognitively not very different from other daily activities. It

is still the same conceptual system at work why there really is no need to

construct a dichotomy of real world, contra game play (or any media

experience for that matter) in the first place.

To get back to the quote from Brandom, it perhaps shows obvious

similarities with Gibson’s ecological approach to visual perception and his

ideas concerning the ecological environment versus a physical world.

Affordances and constraints are nothing less than any object’s

” appropriateness for various practical roles and its inappropriateness for

others” revealed by its appearance. Gibson’s division of the environment into

the three elements, medium, surface and substance, of which we mainly

encounter the two first with our senses, points in the same direction as

Heidegger’s idea of Dasein, as put forth by Brandom: we encounter a world

and make it an environment by our being within this world. The

environment has its affordances and its constraints in its layout. We have to

learn how to handle and manipulate the environment that surrounds us.

Before continuing into a more elaborate analysis, I will now go into a

description of the game Myst.




222

9.10.1 Description of the game

Brothers Robyn and Rand Miller did set a new standard of computer game

play, graphics, and audio style with the release of Myst. The game consists of

more than 2500 ray-traced still images, a number of animations and

monologues.76 The images are displayed as two dimensional though they are

produced with software for making three dimensional objects. The technical

image quality is 256 colors in 72 dpi and the sound quality is 8 bit sound at

11127 Herz. With today’s standards (i.e. January 2001) this does not

especially impress, though in its time, it really was something very different.

This very technical description does not do justice to the experience of

playing the game. It is in a way an audiovisual version of earlier text based

games like Zork. It is an explorative puzzle adventure. There are bits and

pieces of information to be found and put together in order to form a deeperunderstanding of what is really at stake in the game. In other words, Myst

is, in a way, a giant puzzle that one may solve in different ways. There are a

couple of solutions to the game of which one is perhaps to be considered

better than the others.

Myst is also a narrative. There is a fabula in the game as well as an

organizing syuzhet. The latter is not as firm as in a movie, in that the

organization of events, to some extent, is left to the game player to actually

execute and put into being. As a narrative, it has one of the basics of

Aristotelian drama: a family conflict, where two sons oppose the authority of

their father and ruin his work. As a syuzhet, it is explorative. The game

player is allowed some control of the order of events taking place. The

environment is there to figure out, explore, and manipulate to solve the

puzzles and thereby get a deeper understanding of the fabula. The style of

the game has often been described by reviewers as ” surrealistic” and ” calm” .

(Cf. Nilsson, Hedberg 1995, Thente 1997 among numerous other articles on

this game).

Myst has been the subject for analysis also for others interested in computer

games. See for instance the chapter on Myst in Paul Mayer’s dissertation a

76 Ray-tracing is a technique that allows the image to take on a photorealistic appearance since it rendersand displays light refraction in the medium. There is a lot to be said about the issue of photorealism sincethis is a very problematic term. However, this is perhaps not the time and place for an elaborate discussionon this topic. However, as this dissertation shows, it may be wise to take on a broader understanding of audiovisual systems and not only use a camera analogy when coining terms. Photorealism means that theimage has a photo-like appearance. If we consider that we are accustomed taking part in film and otheraudiovisual media that is good, since we already have a basis in our experience to understand the images

we see and we are able to shape a perceptual set from the appearance of the images. However, it would bepossible to get a little further and display the images more like human visual perception than applying acamera analogy. (See also chapter 4 of the present work, 55 on this issue.




224

property in the sound that I will get back to later in this analysis. It ends

with an emphasis of the sound possiblly best described as something that

first goes down and then rapidly up.

This is an animation. That means that the sound and the image, from atechnical definition, are two separate entities that have been brought

together within a common medium duration. There is no common cause for

the sound and the image in a recording-of-an-event sense. However, we

conceive of this sequence as one event with one basic property: that of

revolving up and down. This is duration of one event. We do think ahead and

speculate (that is we try to look for a possible conclusion) about where this

sound will end. Where it will end is also h o w it will end. This is basic

narrative speculation of an event. Or rather, the same structuring of mind,

that thinks ahead and speculates of how this sound will emerge, is the onethat allows narrative speculation. The same type of schema that Edward

Branigan so well applies to narratives and narrational processes in his

Narrative Comprehension and Film, is at work even in this short sequence of

only 13 seconds’ duration.

To get back to the logotype and sound construction: as noted, it is only one

event in our mind. This is so, since there is a strong similarity between the

images and the sound in that they describe the same event. It is a kinetic

anaphone in play here: that of revolving. (See also chapter 6 of the presentwork on this). There is a perceived similarity between the sound and the

image. It is not then a question of ” added value” , to connect this to Chion’s

terminology. Certainly, there is a definite impression of revolving motion.

However, there is motion in both of the manifest expressions, i.e. both the

sound and the image contain the same information therefor we can not

really talk about any added value in Chion’s sense. Revolving is already

contained within the image itself and in the sound itself. Together sound and

image form a sound-image of revolving upward and downward, and although

the image can not be the real cause, it is the conceived cause for the sound.We know that objects may produce sound so that is not a problem. Even

without the image to the sound, we would perceive a raising and probably

revolving object that produces the sound that we perceive. What we have

here is not, then, added value in Chion’s definition of the term, since this

explicitly means that sound and images do not contain the same information.

This case instead shows a boost effect, since we have the same information

from two sense modalities. I would not hesitate to suggest that our haptic

system is at work here, shaping yet a deeper understanding of what is going




225

on in the sequence. The image and the sound works together to create a very

strong conception of revolving.

It is interesting to note that there is such a stronger relation here. We can

study the sound and image relation with the help of a piece of computersoftware that makes this even more obvious. I have taken the Brøderbund

logotype movie into an application called SoundEdit 16 (Version 2.0) and got

some very interesting and clarifying results that show the cross modality of

our sense perception in this sequence. I imported the movie as a QuickTime

movie and let the software display the sound as both waveform and

spectrum.

Figure 12: Soundwave and spectrum of Brøderbound logotype movie. All Myst, imagesand text © Cyan, Inc. All rights reserved. Myst ®

The waveform shows the amplitude of the sound and an overall structure of

this amplitude over time, i.e. how the sound evolves as sound waves in its

duration. This is useful since it shows how the sound-curve relates to the

image in ” size” so to speak. Loud is Big or Intense events going on or about

to happen. As salient object within the visual field might change its size and

have a corresponding sound making the event one event rather than two.

The display of sound as spectrum reveals which frequencies there are in thesound, and their development over time. These two ways of displaying a

sound, as waveform and spectrum, are much like taking freeze-frames of the

sound for study. It is not the ” sound” but the relations within the sound that

is displayed. One can form an understanding of the sound from these

displays. In addition, if we also display the moving images or key-frames of a

sequence we will find the relation between sound and image.

Of course, we also come closer to the images as such when displaying them

as key-frames, since this approach will emphasize their development overtime as well. Michel Chion, as we might recall and that we have already




226

touched upon, does not seem to be positive about the use of spectral analysis

of a sound since it apprehends only physical data. (Chion 1994, 29. See also

the present work and the discussion on the three listening modes, 114). I

would say that my analysis above shows that this is not the case. There

really is a lot to take part in concerning the spectral analysis of a sound.Especially, if it is an audioviusal sequence that we are analyzing. It will

reveal the relations between the sound and the motions of the images as well

as the internal development of the sound.

Taking freeze frames of a sound has been said to be impossible since it

occurs over time. (Cf. Bordwell, Thompson 1993, 292). They also note the

temporal relation between sound and image (Bordwell, Thompson 1993, 313-

316). Metha Hallgren also notes this problem in her M.A. essay on

Marguerite Duras’ India Song. As she points out, the images are also time-dependent in order to be fully understood as what they are in the projection

situation as moving images. (Hallgren 2000, 22). To be moving, the images

need to be displayed in rapid succession. To somewhat widen the discussion

we may consider Detenber and Reeves’ work on arousal effects from visual

stimuli. Their work not only show that the arousal effect is a matter of the

image size rather than image content but also shows that we are keen to

interpret motion into stilled images from movies.

As moments frozen in time, the still images beg forspeculation on what has immediately preceded andwhat is likely to follow the event depicted.(Detenber, Reeves 1996, 79).

So, we do speculate and get the feeling of motion even if the image does not

display the actual motion as moving images. There might be an implied

motion in the image that makes us do such speculation. We are prone to

follow the begun structure of thought as the example of cognitive and

perceptual sets clearly shows (see chapter 5, 99, 102).

There is also a question about the size of the image in their study. Big

images are conceived as big objects, thus pictures seen as large images elicit

stronger feelings of arousal than the same pictures seen as small images.

This indicates that big objects elicit stronger feeling than small objects.

Sound is the result of vibrations in surfaces of substances transduced via a

medium to an animal that perceives the vibrations and makes it to a sound.

Big (massive) objects produce loud sounds when making vibrations since

they have more energy due to their mass.




228

and the sound as separate, although my point is that they provide the same

thing in this specific case. It is also for the sake of argument that I choose to

have it this way even though this might seem to contradict my point. For my

present purposes, it will work to have this separation of sound and image

and treat them as if they were separate.

The images as mentioned, are displayed as key-frames. That is to say, not all

images are shown, and the number of images displayed depends on how one

chooses to display the sound. If one chooses to have a short timeline there

will be fewer key-frames and vice versa.

What my study reveals, is that the images of the animation describes the

same motion as the sound. The key-frames and sound spectrum are showing

almost identical curves! That is, there really is similarity in how the soundand images are displayed. In turn, I claim, this means that there really is a

similarity in how they are perceived. There is strong relation between the

sound, the image that makes the impression of being the formal cause of the

sound, and the way the spectrogram is displayed. By making the

spectrogram display the sound in this specific way, one makes it to an image,

which shows a topological structure. It is a curve that starts at a low point,

rises to a higher point and then goes back down. It ends with a sharp rise

and a down slope. The sharp rise corresponds with the visuals in a shift of

the letter’s making up the word Brøderbund. The slope is corresponding tothe fade out of the sound i.e. both sound and the salient object in the visual

(the word Brøderbund) disappears. The sharp rise in pitch is also a rise in

loudness. The sound gets louder when it first goes down and then up. A

sound that gets louder is conceptualized as something that is coming closer.

And indeed, this is an effect that is distinguishable in the example movie.

The letters take on a new brighter monochrome texture and hence look

clearer. They also change in size! They get bigger and bigger, which means

that they appear closer to the observer due to a natural mapping. The sound

gets louder and the letters get slightly bigger. The sound as well as the visuals occupy more of the perceptual field when their internal relations

changes at the same time and with the same basic structure. They again

prove to have similar conceptual structures. Consider the following

structures for instance.

UP–DOWN metaphors: high and low frequencies that correspond to

positions of the visual object in an UP–DOWN configuration within the

visual field.




231

bodies) that we perceive as a totality. They go together and underscore each

other. The sound works as in previous examples as a kinetic anaphone. We

conceive of the sound and image motion as one event with the motion

causing the sound.

There is some perceived similar structuring of the sense perceptions going on

that is the cause for this coherence. There is a motion and spinning/turning

of the letters. They come into to the visual field gradually rather than the

whole letter or the whole word at once. We are used to nature being in synch.

That is, we know that a sound, its source, and the event that causes the

sound are one event that is concurrent in time. And yes there are some

exceptions to this. Lightning and thunder are separated in time, and, to

some extent as events. (And if we do see lightning, and hear the sound of

thunder at the same time, we will be busy taking shelter. This is often not apleasant experience but an experience of clear and immediate danger that

makes us seek for shelter by ducking and hiding). Even if we know that

lightning means that we soon will hear thunder, it is easy to separate these

events I presume. The sight of an airplane in mach 1, is also rathter

separate from the sound it produces, since the plane is moving faster than

sound and that light has a higher velocity in the medium of air than what

causes us to hear a sound, i.e. vibrations transduced in the medium. The

distance to the object, the airplane, of course also matters for this. Peter

Kubelka claims that cinema is never in synch and nature is always in synch.Film sound works mainly between sequences of images, sequences of sound,

and sequences of sound an images. This is noteworthy but perhaps not fully

correct. However, the point is, as I understand it, that we very easily map a

” real world” experience onto cinema. The latter is of course correct. There is

not really a need to cognitively our perceptually separate these two things

since they really are not two absolutely separate experiences but rather very

much the same since the cognitive processes are very much the same.78

9.10.6

The monologue

This part of the analysis refers to example file INTRO.MOV on the CD-

ROM. I will explore the introductory monologue in a number of possible

ways. It has several levels of content, as we will discover. I will begin with a

78 For a further discussion on this see Grodal, 1994, 26-27 and his comments on Carroll. Carroll holds theopinion that there is no difference between the way we read fiction and non-fiction which Grodal opposes.

I would not go as far as Carroll and claim that there is no difference at all. I rather say that the basiccognitive schemas used are the same. And since most of cognition is unconscious the difference lies thenin the conscious parts of mind.




235

there but we do not focus our attention to it. This is, of course, nothing that

is unique to this specific example. It is rather a common and necessary

practice to let the sound of music or other sound fade to give a location for

the voice in the perceptual field. I believe that you will find this in almost

any handbook of editing audiovisual material. What you do not find in thosehandbooks though is the explanation of what is actually going on in the

human mind in this fade process as I have just described it. We also use the

term fade in the conceptual domain of things that does not stands as clear to

us as they used to and that has lost its importance. Memories fade, for

instance. Love fades away. Life itself might be described as fading when

growing old.

Let us get back to the concrete example from Myst: what we have are two

parties that are not focused on the music but on the voice: the producer of itis concerned with the production of words and we are focused on listening to

the voice, trying to hear and understand what the voice is saying. That is, we

are in a semantic listening mode to use Chion’s terminology. We will soon

encounter the semantic meaning of the words produced. However, before

that, some words on the timbre of the voice is necessary, since timbre is an

important part of this semantic meaning.

The voice has a male pitch and sober timbre. Timbre is considered a bit

problematic since it encourages subjective judgement. However, in thecontext of this dissertation this is I believe not a problem but rather a

positive, i.e. good, thing. (Which again is a subjective judgement of good and

bad!). In my opinion, the voice is sober to its timbre. It is a smooth

(subjective judgement), warm (subjective judgement), and pleasant timbre

(subjective judgement). What we have here in my description of the timbre,

is (as you by now might suspect) the product of metaphorical concepts of the

mind. It is inescapable to use such metaphorical concepts when it comes to

timbre. Timbre as a concept is a manifestation of metaphorical cross-domain

mapping. In German, timbre would translate into ” klangenfarbe” i.e. thesound color, which is also a strong indication of this. The language for sound

incorporates vision into itself. In turn this also indicates something else:

synesthesia. The cross modal synthesis of different sense perceptions.

The voice has an echo of 0.3 seconds roughly. If you listen to the last word

” written” it comes through very clear that this word is repeated. It stands

out so clear because the music almost pauses before it delivers a final hitting

sound (which is synchronized with the book falling down and bouncing but

we will get back to this aspect later). This echo of the voice makes usspeculate about the environment that contains the human agent speaking.




236

What kind of environment will produce such effects on a voice? The

container of the human agent must be big to have this effect. There is an

easy formula for knowing when there will be an echo. When there is one

tenth (0.1) of a second delay from that moment one speaks a word and it gets

back as a sound we call this an echo. That means that there must be 17meters to a wall so that the sound travels 34 meters in total. (Sound waves

have a velocity of 340 meter per second in air. 17 + 17 = 34. 34 is 0.1 of 340.

Voila! There is an echo. Of course, this formula is nothing that we think

about. It is rather something that we know from being human beings within

environment where there are surfaces that have this effect. In this case of

Myst there is a 0.3-second delay before we hear the voice again. This would

by the same formula mean, that there is a reflecting surface at 25.5 meters.

In our conceptualizing of the voice and this effect it simply means that the

room is large.

In addition, this practice is a commonly used cliché for designating power

and making the producer of words larger than his bodily container. This is

known as ” The Voice of God” . When Moses meets God in the visual shape of

a burning bush in Cecil B. DeMille’s THE TEN COMMANDMENTS (1956) the

voice of God is large due to a similar effect of reverberation. It is a low

pitched prototypical male voice. An other example of this is when the fake

Wizard of Oz is introduced, his voice is larger than the real human agent

suggesting that the wizard is very powerful. (THE WIZARD OF OZ. Fleming1939).

I will now go into the ” semantic” part of the analysis. I have deliberately

waited on material with the semantic analysis of spoken or written

language. This enterprise is maybe not the best possible way since it does

not take into account what has preceded the language in the sequence. It is a

matter of understanding the true context of the utterance that I miss when

going directly to a semantical analysis of language. It is not so that there has

not been any semantic value in what precedes the use of human language.That is not what I am suggesting. To clarify: What the previous analyses

have shown is that there are cognitive semantic structures providing a

context for understanding the language part of the pieces analyzed. Yes,

there are grammar, syntax, prosody etc. in the sequence to follow. These are

understood in the context of their presence within the game though as well

as speech-acts in the case to follow.

What follows is a transcript of the monologue:




237

I realized the moment I fell into the fissure that thebook would not be destroyed as I had planned.

It continued falling into that starry expanse of which I had only a fleeting glimpse. I (ha)ve tried to

speculate where it might have landed but I mustadmit however such conjecture’s futile. Stillquestions about whose hands would one day hold myMyst book were unsettling to me. I know myapprehensions might never be allayed.

And so I close realizing that perhaps the ending hasnot yet been written.

After this short monologue, delivered to us by a man, for us yet unknown, we

have obtained a lot of information on the game’s objectives even if we areunaware of this fact. Let us now study the monologue step by step.

I realized the moment I fell into the fissure that thebook would not be destroyed as I had planned.

The man speaking planned to destroy a book for some reason which is not

revealed to us. There is both the putting-forth and seclusion of information

in the first utterance. The man has been falling (I fell into the fissure). In the

visual present of the sequence he is in fact still falling. Or rather, someone

that we speculate being the speaking agent is falling during the first line of

the monologue. We map the prototypic male voice to an equally prototypic

shape of a human. The falling of the man begins at ” the moment” and

continues roughly to ” that starry expanse” .

It continued falling into that starry expanse of which I had only a fleeting glimpse.

At this point in the utterance, ” that starry expanse” i.e. the shape of the man

fades out and is replaced by what has the general shape of a book. Wecategorize the object at the basic level both because it has a specific look and

that the word ” book” is previously mentioned therefore the concept as such is

already in mind. Sound and image work in fusion. This also goes well along

with the phrase, ” It continued falling” referring to the book just mentioned

and to the book that we see falling. The man has lost control of himself (loss

of self control) and of the book (loss of object control). They are both falling

although they do not fall to the same place. The man falls into the fissure as

does the book but the book somehow continues falling into another location

of which the man has only ” a fleeting glimpse” . They are separated bylocation. Where the man falls and lands we do not get information about.




238

I (ha)ve tried to speculate where it might havelanded but I must admit however such conjecture’sfutile.

This sentence reveals that quite some time has probably passed since what

see and what the man describes happened. The fact that he has tried tospeculate suggests that time has passed. It is also pointless and meaningless

to do this since this knowledge is unobtainable. The book did fall to a

location that is not to be found, i.e. not to be observed by him. This in turn

implies that we as game players are the ones to find this out since we see the

book continuously falling downwards within the visual container. There is

also an implication that it is not all that easy for the producer of the voice to

take this step since he says ” I must admit however” . This marks

dissatisfaction on his behalf with the given situation.

Still questions about whose hands would one dayhold my Myst book were unsettling to me.

The sense of past time is enhanced by this utterance. ” Still questions” and

” would one day” intensifies this passed time. It also indicates and enhances

the idea that the book’s planned destruction was really important. This is

manifest in the use of ” unsettling to me” which also indicates that it is a

personal involvement of some kind here. The book belonged to the man

speaking (” my Myst book” ) and the loss of it is somehow and for some reasonnot good for him. The destruction of the book would have been a controlled

loss. The loss of it and its disappearance is not a controlled loss. The

possibility of someone else getting hold of the book is a danger. There is an

implied question from the speaker concerning the intentions of such a

discoverer of the book. What would that person do with the book? In

addition, since we see the book and follow its trajectory we are able to

speculate about the possibility that we are the ones to find it. This in turn

puts forth a question to us: what will you do with the Myst book? This

question is enhanced by the following sentence:

I know my apprehensions might never be allayed.

The book’s disappearance is so disturbing to the speaker that he can not put

it to rest. It will always trouble him.

And so I close realizing that perhaps the ending hasnot yet been written.

This last utterance points out yet another affordance of a book in general: itis write-able. The end of events may be left up to us to write into the Myst




239

book. It is suggested that an agent might take action if he or she finds the

book and do something that was not meant to be done. The chain of events is

not cut off but rather paused by the loss of the book and the failure of its

destruction. The voice also manifests a synchronous falling scale in

conjunction with the musical track and the content of the visual container inthat the book stops falling, the music goes down in pitch. The effect of this is

that we have reached an end point of one chain of events. The spoken words,

however, point to the fact that this is also a new starting point, a source, for

new events to begin.

The style of the monologue is sophisticated and also indicates that the

person speaking does not belong fully in our own time. The choice of words is

not the most common in daily contemporary English. In addition, the

grammatical structure, at least to me, makes an impression of older use of language. It indicates that the person speaking is an educated man

belonging somewhere else in time than we do. Although the sound of the

voice makes him large and also powerful (large objects contain more energy

than small ones in our conceptualization of them), he is restricted in power

by the uncontrolled loss of the Myst book.

What the monologue has described is a SOURCE-PATH-DESTINATION

structure. There once was a location at which a certain action was meant to

take place. This action was not performed because of some unknown factorthat created a fissure in the environmental surface opening a gap to another

location. The book and the man fall via a PATH in a medium to reach two

separate DESTINATIONS. This kind of structure is a basic narrative

structure. Change the starting point, the medium of the path, the surface

layout and the destination and you will have another realization of the

fabula.

Up to this point we have been exposed for the basis for construing a cognitive

set. As my analysis has shown, we do perceive structures of DIRECTIONSsuch as, DOWNWARDS, SIDE-TO SIDE, and INTO. This is, I believe, not

coincidental but a result of the game makers intuitive or conscious use of

sound and image structures as coherent entities to introduce the game’s

objectives. And as suggested mind seeks to use what it has and add only the

necessary this makes sense. Literally make sense that is. We are getting

tuned into the structure of directionality and moving and that is much of

what Myst is about: visually controlled locomotion and the constructive use

of environmental affordances that are based in the efforts of solving logical

problems by object/environment manipulation.




240

9.10.7 Entering Myst.

This part of the analysis refers to example files INTRO2.MOV and

MYST.MOV on the CD-ROM. The latter is a recording of an actual game

session. We are now left with a lot of information from the monologue and

the book lying in front of us. A book has some general affordances. It is open-

up-able and hopefully also read-able in some way. The open-up-ability is

however the first affordance of a book that we meet when encountering it.

The read-ability is an implied affordance of a book that we encounter after

opening it up. As the above analysis of the monologue showed there is yet

another affordance of a book in that it is write-able.

We have also the sound of wind blowing and within the visual container a

new object appears. It is a hand pointing with its index finger. A hand

affords manipulation as the word in itself suggests. Since this is a computergame and hence is conceptualized as an interactble environment, we may

establish a conceptual link between the hand in the visual container and

ourselves. This is manifest if we move the input device to the game, i.e. the

mouse with one of our hands. The hand within the game will move

accordingly. This is our first chance, from that moment we started the game

to perform manipulation of the game environment. We may move the hand

and click the mouse to explore what happens. Nothing will happen though

until we put the hand on the book and click on it. It then gets bigger and

reveals that it really is the Myst book. Or at least a Myst. There might be

others but we have only been told something about one Myst. The word Myst

is written in capitals on it. It opens up, and reveals that it is not an ordinary

book. Instead of text, it contains a visual container with moving images. The

moving image sequence shows a flight over water ending up at a dock. A

similar structure of the sound as in the monologue sequence ends the display

of moving images synchronized with the images. They both drop and stop

hard which marks a new end point, a destination that is a possible new point

of departure. A still image is displayed showing a part of a dock. If we nowmove the hand into that part of the field that contains a still image it will

change shape. Instead of pointing it grabs.81 This suggests that clicking

would be grabbing the image getting it closer to us. However, if we do click

the mouse to do this, we will find that this generates a repeated phasing

sound and a black screen that turns into a larger version of the still image

just displayed. We have not picked up the image though to get a closer look

at it. This is suggested by the sound and its phasing quality. As in the

81

Some might say that the hand is open showing the back of it however I do believe that to be amisinterpretation. The index finger gets shorter and the length of the other fingers are to short for the handto be fully open.




241

previous examples, this sound structure indicates motion and in this case

even relocation. Phasing effects such as this are used in many movies

designating relocation and even entering the unknown. The same basic

quality of a sound is used for instance in Tarkovsky's STALKER when three

men are entering a guarded and closed off zone. They ride a motorizedtrolley on a railroad and the sound of the wheels going over the gap between

rails are slowly mixed with a phasing bass sound that also has an echo effect

added to it. In combination with close ups of the men this is a very

suggestive audiovisual sequence in which sound and images turn into one

coherent event: the event of transformation and relocation.

It is also a widely used effect in other computer games. Robotron 2084 for

instance has similar basic structure in the sound that accompanies the

Game Ego’s entrance into the game board and change between levels.

Within the environment of Myst, the visual seems to reign. The game is

structured, at least in the beginning of it after entering the Myst book,

through the use of linear perspective and manipulation of the Game Ego’s

localization. The succession of images, when beginning to explore Myst does

play a significant role in how a game player is made to move in the

environment. At certain points, one is lured away from one’s initial planned

path, so to speak.

There is a play going on between us as game players and the way the

environment presents its layout to us. It is close to how Mark Johnson

defines interaction as being forces of the world acting upon us as humans

and our action up on the world. The world could in this context easily be

replaced by the word environment. We conceive of ourselves as live animate

objects contained within the world. Other humans are other objects outside

us, i.e. they are parts of an outside world. In other words, the environment

and our human agency are parts of a PART–WHOLE structure. There is no

need to discern human, social interaction from interaction with machines if we stress this a little. We are always interacting with the outside world. The

environment of Myst is acting upon us while we are within it due to the

presentation of its layout. Much in the way suggested by Lakoff in Women,

Fire and Dangerous Things (Lakoff 1987a, 50) and that is discussed in

chapter 6 of the present work.

Gibson’s idea of visually controlled locomotion is applicable on the

environment of Myst. Moving around in Myst is performed by pointing (and

clicking) the hand of which we are allowed control. This hand is themanifestation of the tactile/kinesthetic motor link between game player and




242

game environment. Depending on where in the visual container it is placed

by the game player it points in suggested directions of locomotion. If we

choose to click the mouse while the hand is pointing straight ahead, we will

see a new image from a different point of visual observation. We will also at

many such instances hear things slightly differently, since our point of auditive observation has also changed. (They are not really separate things.

Gibson does not discern them this way but rather claims that observation is

not only made through vision). We have moved in our frontal direction

straight ahead. This is obvious due to these changes in point of observation.

If, however, we choose to click the mouse when the hand points to either side

it may not be as obvious in the first inference of what has happened. In the

later case, we often only have a change in the visual point of observation

that is discernible.

We can not move freely around. There are constraints in the environment

that hinder us from doing this. There is a tight bond between the direction of

vision and the direction of locomotion.

We can plan to go a certain way but the next image might not be what is

expected even if it is a logical image to present. If we take the walk from the

dock we are first placed at when entering the world of Myst the visual

perspective suggests us to move in a specific direction. It does not suggest us

to turn around and by doing so get an overview of the place one is standingat. Instead, most players start to go in the suggested direction. In addition,

when doing so one can not help but to see a thing to the right that has

something switch-like on it and most players do try to switch this switch-like

thing. It will result in an animation of the switch as well as a sound that is

also switch like. Then, many players see another switch to the left. To reach

that switch one has to move up some stairs. Performing this action one gets

to an intermediate point on one’s way that might lead one astray. Instead of

immediately getting to the desired switch, one will find oneself in the middle

of the stairs. It is easy then to loose one‚s way and start walking in anotherdirection than the planned one. It is, in other, words easy to forget what one

initially planned to do and do something else that the environment affords

and encourages. This is a direct result of the game structure and the use of

still (or ” arrested images” to use Gibson’s terminology in a slightly different

meaning than he does himself). When playing Myst, one is in most cases

forced into specific points of observation that are maybe not the most obvious

locations to place oneself at if one where to carry out the intended

SOURCE–PATH–GOAL/DESTINATION schema. There is a constant loss of

self control in play that literally manipulates our Game Ego presence. To get




243

back to the example, we now stand at the stairs that should lead us to the

next switch. However, to the left of us we see another path and many game

players take the left at that point for the next move. A new stair then shows

and in the next picture one will see a piece of paper on the ground that is

pick-up-able and contains a message meant for the eyes of someone namedCatherine. Of course, one starts to read it. The message will probably make

us re-plan our locomotion and go back to the dock.

I will stop here. The point is already made. The layout of the environment is

very important. The file MYST.MOV contains a long stroll which illustrates

the above and even continues further on into the game environment.

As Torben Grodal notes (Grodal 1999a) the film camera may simulate

human visual perception and have some objects in focus and some out of focus. He also notes that even when that occurs we may as viewers focus on

the out of focus parts of the image displayed on a movie or computer screen.

(Grodal 1999a, 6). Most, if not all, early computer games up until about the

time of Myst have all its visual components in sharp focus. Or rather one

might say, it has them in the same visual focus all over and does not discern

objects far away from the Game Ego’s point of visual observation from

objects close to this point in terms of focus. Their size and sometimes even

their movement and the fact that they where overlapped by other visual

objects was the main (only) cue for showing them as distant. The depth of the visual world is not often strongly emphasized in early computer games.

Such computer games are acted out on a horizontal and vertical plane rather

than in depth. They are acted out on the closest surface within reach from

the point of being established by the point of observation. They are

concerned with motion (visual action from motor action) that emphasizes the

“up-and-downess” and the “side-to-sideness” of the media’s visual system.

One could also say that they are up-and-downable and side-to-sideable

games.

In Gibson’s theory of visual perception motion is the key for perceiving

depth. The fact that we are moving within the environment and that the

point of observation is changing creates the sense of depth. This is due to the

visual kinesthesis. We are aware that we are moving within the

environment when walking around, driving a car etc. As he notes, cinema

does not allow full access to the visual controlled locomotion since he is

” helpless to intervene” . (Gibson 1986, 295).

The human eye does have a narrow focus, a narrow visual field (to useGibson’s term), which is of importance when comparing a large film screen to




244

a smaller computer monitor (computer screen). The center of attention is

sharper than the periphery. A large screen in cinema makes available a

range of visual effects with shifting attention focus. And we do get aroused

by images primarily due to their size when viewed rapidly as we have seen

in the work of Detenber and Reeves. (Detenber, Reeves 1996, as referred toearlier in this chapter).

How does this work?

There are several reasons for this.

UP–DOWN and SIDEWAYS are strong schemas that are easily transduced

to a game environment.

Distance is not only scale and size but also the richness in texture of objects.In games like Quake and Duke Nukem 3D one will experience a kind of

reversed perception of texture. At a distance objects will be rendered as

texture-rich objects but in close up, i.e. if one moves the Game Ego close to,

say, a wall, the wall will be seen as construed of large blocks of a mosaic.

Texture will be reduced the closer the Game Ego gets. This will result in a

visual experience, which is perhaps not often found outside the game

environment. Outside the game environment we may have access to more

information about any object’s texture the closer we get, but in game

environment (up to date) it will often be less information accessible to us.This is, of course, due to how a computer image is rendered and is the result

of a finite number of bits representing the texture of an object in a digital

environment and the ability of the hardware to generate an image. A digital

image is built up from pixels that may be either bit mapped or relate to

vectors. A computer generated image that would mimic a brick wall perfectly

would have to hold a number of informational bits that would be astronomic

if it would be possible to get an extreme close up of it without pixilation. And

to perform this in real-time is nothing that a personal computer of today will

cope with. You may render an image with extremely high-resolution and use

it in a sequence with other images but to perform this task in real-time…

that’s another story, entirely. Imagine the possibility to simulate just one

brick, get a close up and then even chop it up in pieces and examine them in

close up and even in a microscope. No wonder that the visual world in game

environments sometimes might feel strange.




245

Figure 13: The Dock in Myst. Note the use of linear perspective. All Myst, images andtext © Cyan, Inc. All rights reserved Myst ®

Figure 14: From the forechamber beside the dock. Note the location of the button. AllMyst, images and text © Cyan, Inc. All rights reserved. Myst ®




246




248

capability to afford an interaction experience. This makes it an interaction

experience of the human body/mind rather than some external objectifiable

quality that is possible to measure in terms of frequency (how often you

could interact), range (how many choices were available) and significance

(how much the choices really affected matters). This is the formulasuggested by Brenda Laurel, which she abandoned for good reasons (as

chapter 7 showed). They will divert us from the real issue of the interaction

as an ongoing process. Jens F. Jensen on the other hand seems to cohere to

this way of thinking with his considerably more objective approach to the

issue of interactivity. Jensen seems to embrace what Laurel has left behind.

This is something that I do not. I propose a more subjective approach to this

issue as suggested in chapter 7 of the present work.

Furthermore it is important to note that a disembodiedness is most stronglyfelt if the interaction process is blocked in some way.

A theory of the audiovisual seems to be bound for ending up in a theory of

the process of visual conceptualizing. What film-studies do in using the point

of view concept, as a concept that has its basis in the visual phenomenon of

the movie, is misleading for the thoughts about this medium. The

understanding of the movie will be a visual kind of understanding since the

concepts used for understanding are based on image schemata (which are

not images in them selves but schemata for structuring perceptions).However, this does not mean that the basis for the cognitive processing of

image schemata is necessary an image. It could, as Mark Johnson writes be

any sense modality, but the visual system is predominant. (See Johnson

1990, 25-26 and the present work, 85). Hence, it is not the best of ideas to

think about point of view as something literally visual. Rather we must

rethink the concept to allow for more than one sense or to scrap it in favor of

other concepts that discover the sense modalities employed in film or

computer games.

Sound and images, hearing and visual perceptions are merged, as are other

sense modalities, in the processing of image schemata. Visualization is not

the putting on paper of some image. No, it is rather the process by which we

understand and conceive the environment and the phenomenon in it. It is

the process of making things concrete and tangible, and it does not rely

solely on the visual perception. The visualization may have its origin in an

audible perception but the result is not an auditive understanding but a



RESULT AND CONCLUSIONS

249

visual understanding. 82 We form a shape of the sound and try to connect it

to another phenomenon that is distinguishable as a visual phenomenon.

Likewise, image schemata structures also our motor performances. A

SOURCE-PATH_GOAL schema is a motor schema. From the structuring

image schemata, we are able to project rich mental images. 83

Some questions might be in order here. Is the language cue ” image” enough

to overtake the essence of the process? (See Mark Johnson, The Body in the

Mind, 26). Are we performing the same mistake when using the term image

schema/schemata as are film scholars when using the concept of point of

view? Anything described is added to some other medium’s features and is

no longer in its original phenomenological state of being. It is represented by

a language description, which might hide or highlight its actual

phenomenological state. Language is a medium of hiding or highlighting thephenomenon in the world to put it Johnson’s terminology. What seems

reasonable to conclude is what Lakoff and Johnson find out: language

reveals a lot of our functioning in the world and the functioning is based in

the conception of ourselves as visual beings and the world as a visible map of

relations between our bodily container and the rest of the world. To put it in

more Gibsonian terms, the language is not the environment but a

description of it.84 Language is the human artifact of environment-grasping

and is based on cognitive processes beyond language itself. The environment,

is understood as spatial and temporal relations. The spatial relations are themost dominant. (See Lakoff and Jonson. Philosphy in the Flesh. chapter 10

and the present work The SOURCE–PATH–GOAL Schema in chapter 5

where I discuss fabula and syuzhet in relation to this). It is also the case,

that the image schematic structures perform a mediation of the world into

structured conceptions. The world is never conceived as it is but as its is

mediated by such structures. The influence of Immanuel Kant is easily

noticeable here. As Johnson makes clear, his, (Johnson’s), use of image

schemata is very close to Kant’s use of image structures. As Kant writes and

as Johnson points out: ” The schema of the triangle can exist nowhere but in

thought.” (Johnson 1990, 24). 85 Note also Johnson’s reference to Lakoff’s

explanation on this issue:

82 Hence the joy of cartoon animation maybe? Sound and images already merge in the perceptual phase. Itis also maybe a salient feature of Kubrick´s 2001, which is easy to experience but hard to describe bylanguage. It is conceivable as an audiovisual experience with some narrative parts but is often found hardto make a language description of.83 Another reason to stress this point is to avoid a psychoanalytic conception of the body.84 This is not the essence of Gibson's theory but my adjustment of Lakoff and Johnson's terminology to

make it fit Gibson's terminology.85 Originally from Kant's Critique of Pure Reason. Eng. translation by Norman Kemp Smith New York1965 A 141/B180)




250

The names that we have given to image-schemas,and to image-schema transformations, are verymuch in keeping with the kind of symbolization thatmight be used in studies of computer vision. But thenames are not the things named. (Johnson. The Body in the Mind. 27).

Important no notice in this context is that Lakoff explicitly writes that the

term ” image” is not intended to be limited to visual images. We have also

auditory images, olfactory images and images of how forces act upon us.

(Lakoff 1987a, 444.)

So, what I suggest is that we have to think about which kind of visual

understanding we should employ with movies or audiovisual media at large.

The language-based point of view conceptualized visual understanding orthe image-schemata-multi-sense-modal-based view? Alternatively, a merging

of them? It is all too easy to fall into the common binary trap of either/or.

What can the two contribute to the analysis of movies or computer games or

any audiovisual medium? And what will be the result of a merging of them?

First lets get back to Johnson’s writing. He claims:

The very structure of orientation is perspectival. And grasping the relevant perspective is not usuallya matter of entertaining a proposition, such as I’m viewing the container from outside” ; rather it issimply a point of view we take up , because it is partof the structural relations of the relevant schema.(Johnson 1990, 36).

Which perspective is relevant will depend upon thecontext. (Ibid. 36).

We may divide the process of human conceptualization into perceptual phase

and cognitive phase. In so doing we can assume that this will affect the userinterface design. How does perception relay into cognition?86

When a game player is given direct, immediate, control of a character in a

game, he or she tends to identify with the game character referring to it as

”I” no matter how the visual representation is constructed regarding first or

third person perspective. The use of combined embodied perception is

important in founding a Game Ego as discussed in chapter 8. As chapter 9

shows, this is evident in ”simple” games like Space Invaders, PacMan, etc. as



RESULT AND CONCLUSIONS

251

well as in more ”complex” games like Duke Nukem 3D and NHL 2000. The

direct control offers another kind of identification with the controllable object

within the game than an indirect control does. The motor action of a player

extends without noticeable delay into another action within the game

causing a direct tactile motor/kinesthetic link between the game player andthe game. This, of course, is valid only if the processing speed of the

computer allows for this directness. When the link is distorted by slow speed

not intended to be in the game etc. the link becomes weaker (it breaks) and

the identification with controllable object is hindered.

10.2 The Tactile motor/kinesthetic link

Computer games require action. Action is using the body. The body and the

embodiment of perception are crucial for the constitution of a Game Ego andthe manifestation of a game environment. 87 Visually controlled locomotion

through direct manipulation of an interactable audiovisual environment is

what computer/computer games are all about. As a result of the application

of experientialist theory of cognition on computer games and film a central

concept is achieved: the tactile motor/kinesthetic link. Computer games are

able to use other ways of creating an environment and emotions than do film

and television. This is due to the embodiment of cognition and by the act of

playing.

The link makes it possible to actively manipulate what is displayed on the

screen. It allows concrete motor action to extend into and affect the visual

and auditive world. It does not matter much if the visual container allows a

three dimensional representation to be manipulated in real time as in

Unreal, Quake etc. or if it is a series of still images that one control the

display of as in Myst. The sense of motion of one’s Game Ego is still

protruding through this manipulation. The change of the visual and auditive

container is what manifests the visually controlled locomotion. The tactile

motor/kinesthetic link also allows more direct object manipulation with thegame environment. The tactile motor/kinesthetic link between a game player

and a Game Ego is the primary basis for identification with the Game Ego.

10.3 The Game Ego

I have in this dissertation introduced the concept of a Game Ego. It is not a

question only of point of view as being the central means by which

identification with a computer game character is established. My work

87 ”Metaphor is primarily a matter of thought and action and only derivatively a matter of language”.(Lakoff, Johnson 1980, 153).




252

provides a study of the interaction between output from game > perception >

cognition > interaction > input to game > new output from game. As noted in

chapter 8, the sophistication of the player’s internal relations between

perception, cognition, emotion and action, is more important than the degree

to which the Game Ego is visible. In a movie, such as T HE L ADY IN THE L AKElong takes without a visible body of a protagonist, to use film theory

terminology, might be conceived as disturbing while in an interactable

computer game the identification process is different. In the movie situation

we seek for a visual anchoring of the perceiving subject that we are meant to

identify with. In games like Duke Nukem 3D, or Doom we have something

that looks similar to ” subjective camera use” . In games like Spyro the

Dragon, we have what looks like a tracking camera.

However, due to the tactile motor/kinesthetic link, which is the relationbetween perception, cognition and action, we will not conceive of the

identification in the last case, as being stronger than in Doom. In a movie, it

is the lack of motor control, motor access to the diegetic environment that

makes us wish for a visual presence of a protagonist. Computer games, on

the other hand, will afford motor control. A lack of bodily presence and a lack

of motor control is the very basis for a ” subjective shot” . It is what makes the

shot subjective.

The body of a Game Ego is anchored, it is made manifest, through the tactilemotor/kinesthetic link between the game player and the game environment.

We are really there. The concrete actions taken by the game player are the

foundation for the identification with the abstract cognitive model of a Game

Ego. It does not matter whether the movement is performed in 2-dimensions

or 3-dimensions, as long as it is direct. The game player imagines him- or

herself as moving in the environment. The color depth, i.e. the amount of

colors, is also not important. Two colors are enough. As soon as something is

possible to be perceived, to be something separated from its environment, it

gets to be an object. It gets to be a PART in a PART–WHOLE configuration.

Computer games are audiovisuo-motor experiences. Whereas a film in the

traditional meaning has a plethora of sound and image configurations from

which the perceiver builds the narration and the narrative, a computer game

may use all this, a few more dynamic ways of sound and image

configurations, and at least a tactile motor/kinesthetic link based on concrete

motor action. The relation between sound and image in a computer game

considered as hardware/software (i.e. at the level of medium definition) is

not necessarily fixed. Its is always possible to have a dynamic flow instead of fixed relations between sound and image in the computer game situation.



REFERENCES

255

11 References

11.1 Literature mentioned

Alexander, John (1999): Screen Play – Audiovisual Narrative and Viewer

Interaction. Stockholm University.

Altman, Rick (1997): ’The Silence of the silence’. In Musically Quarterly # 80

volume 4, 648-718

Anderson, Joseph D. (1996): The Reality of Illusion: An Ecological Approach

to Cognitive Film Theory. Carbondale, Southern Illinois University Press

Andrew J. Dudley (1976): The Major Film Theories: An Introduction. New

York, London, Oxford: Oxford University Press,

Bachtin, Michail (1991): ’ Kronotopen’ (’The Chronotope’. My translation) in

Det Dialogiska Ordet, Swedish translation, Gråbo,Anthropos,

Bastian, Peter (1987): In i musiken ( In to the music. My translation of the

title). Swedish translation, Stockholm:

Batchen, Geoffrey (1999) Burning With Desire. Boston: MIT Press

Bordwell David and Thompson, Kristin (1994): Film History: An

introduction. New York: MacGraw-Hill

Bordwell, David (1991): Making Meaning: Inference and Rhetoric in the

Interpretation of Cinema. Cambridge: Harvard University Press

Bordwell, David (1997): Narration in the Fiction Film. London: Routledge

Bordwell, David and Thompson, Kristin (1993): Film Art: An Introduction.

New York: McGraw-Hill

Bordwell, David, and Carroll Noël (1999): Post-Theory. Madison Wisconsin:

University of Wisconsin Press

Brandom, Robert (1992): ’Categories in Being in Time’. In Dreyfuss, Hubert

and Hall, Harrison, eds. Heidegger A Critical Reader. Cambridge




256

Branigan, Edward (1984): Point of View in the Cinema: A Theory of

Narration and Subjectivity in Classical Film. New York: Mouton Publishers

Branigan, Edward (1997): ’ Sound, Epistemology and Film’ in Allen and

Smith eds. Film Theory and Philosophy. Oxford: Clarendon Press,

Branigan, Edward. (1992): Narrative Comprehension and Film. London/New

York: Routledge

Brooks, L. (1968): ’ Spatial and Verbal Components of the Act of Recall’. In

Canadian Journal of Psychology, 22, 349-368

Bugelski, B R and Alampay Delia A. (1961): ’The role of frequency in

developing perceptual sets’. In Canadian Journal of Psychology 15 (4) 201-

211

Bush, Vannevar. (1945): ’ As We May Think’. In The Atlantic Monthly July

1945. Downloaded version of the text prepared by Denys Duchier, at the

University of Ottawa, Canada in April 1994 and later updated 1995.

http://www.isg.sfu.ca/~duchier/misc/vbush/

Carroll, Noël (1988): Mystifying Movies. New York: Columbia University

Press

Carroll, Noël (1996): Theorizing the Moving Image. Cambridge: Cambridge

University Press

Chion, Michel (1994): Audio-Vision: Sound on Screen Eng. translation, New

York: Colombia University Press,

Coleridge, Samuel Taylor (1817) Biographia Literaria chapter 13. London:

Rest Fenner

Cook, David A. (1990): History of the Narrative Film. New York: Norton

Coren, Stanley, Clare Porac and Ward Lawrence M. (1978): Sensation and

Perception. International Edn. New York

Currie, Gregory (1995): Image and Mind: Film, Philosophy, and Cognitive

Science. New York and Cambridge: Cambridge University Press

de Bono, Edward (1990): Jag har rätt- Du har fel ( I am right, you are wrong)

Swedish translation. Stockholm: Svenska Dagbladets Förlag



REFERENCES

257

Deleuze, Gilles (1992a): Cinema 1: The Movement Image. Eng. translation,

London: Athlone Press

Deleuze, Gilles (1992b): Cinema 2: The Time Image. Eng. translation,

London: Athlone Press

Detenber, Benjamin, and Byron Reeves (1996): ’A bio-informational theory of

emotion: motion and image size effects on viewers’. In Journal of

Communication no:46

Foss, Bob (1989): Narrative Technique in Film and television. Stockholm,

SVT Training

Freud, S. (1905). (1953, pr. 1956): ’Three essays on sexuality’ . In The

standard edition of the complete psychological works of Sigmund Freud. Vol.7, 1901-1905. Eng. translation under the general editorship of James

Strachey ; in collab. with Anna Freud ... London: Hogarth

Frost, Robert (1920): ’The road not taken’. In Mountain Interval, New York:

Henry Holt and Company,

Gibson. James J. (1977): ’The Theory of Affordances’. In Shaw and Bransford.

Perceiving, Acting and Knowing. Hillsdale, New Jersey: LEA

Gibson. James J. (1986): The Ecological Approach to Visual Perception. New

Jersey, London: LEA

Goodman, Ellen (1982): ’ I Should Have Known He Was… Temptation’. In

Seattle Times, May 18, 1982. Cited in Loftus, Loftus 1983

Grodal, Torben (1994): Cognition, Emotion and Visual Fiction. Copenhagen:

University of Copenhagen

Grodal, Torben (1998): ’Computer games and emotional control’. Paperpresented to the Mediapsychological Conference ’Only Entertainment’, in

Hannover May 8-9

Grodal, Torben (1999a): Filmoplevelse: En indföring i filmteori ( Film

experience: An introduction to film theory. My translation of the title).

Copenhagen: University of Copenhagen

Grodal, Torben (1999b): Moving Pictures: A New Theory of Film Genres,

Feelings, and Cognition. Oxford: Clarendon Press,




258

Grodal, Torben (2000): ’Video games and the pleasures of control’ . In Zillman,

Vorderer ed. Media Entertainment: The Psychology of its Appeal.

Gustavfsson, Eva (2001): ’Hur Tv- och datorspel alltmer liknar film i dess

använding av tid, rum och ljud.’ (’The way TV- and computer games take ona film approach to time, space and sound.’ My translation of the title).

Student assignment.

Hallgren, Metha (2000): India Song: ett exempel på filmberättandets

möjligheter. (India Song: an example of film-narrational possibilities. My

translation of the title). M.A. Thesis. Stockholm: University of Stockholm

Herz. J.C. (1997): Joystick Nation. London: Abacus

Jensen, Jens F. (1998): ’ Interactivity’. In Nordicom Review Volume 19, no:11998 Göteborg: Nordicom/Göteborgs Universitet

Johnson, Mark (1990): The Body in the Mind. Chicago: University of Chicago

Press

Kant, Immanuel (1965): Critique of Pure Reason. Eng. translation by

Norman Kemp Smith. New York

Kaplan. E. Ann (1983): Women and Film: Both Sides of the Camera, New

York/London: Routledge

Kent. Steven L. (2000): The First Quarter: A 25-year history of video games.

Bothell WA: BWD Press

Lakoff, Andrew, Becker, Miles (1991): ’ Me, Myself and I ’. Unpublished

manuscript. University of California at Berkeley

Lakoff, George (1987b) ’Cognitive semantics. Two views on cognition’. In Eco,

U, Santambrogio, M. and Violi, eds. Meaning and Mental Representations

Lakoff, George (1996): ’ Sorry I’m Not Myself Today’. In Fauconnier and

Sweetser eds. Spaces, Worlds and Grammar. Chicago: The University of

Chicago Press

Lakoff, George, Johnson, Mark (1980): Metaphors We Live By. Chicago:

University of Chicago Press

Lakoff, George, Johnson, Mark (1999): Philosophy in the Flesh. New York:

Basic Books



REFERENCES

259

Lakoff, George. (1987a): Women, Fire and Dangerous Things. Chicago:

University of Chicago Press

Lakoff, George: (1993) ’The Contemporary Theory of Metaphor’. In Ortony, A.

Ed. Metaphor and thought. Cambridge: Cambridge University Press

Langemark, Gunnar (1996): ’Computerspillet som Tegn’. (’Computer games

as sign’. My translation of the title). In Drotner ed. Øjenåbnere - Unge,

medier, modernitet. Dansklærer-foreningen, Danmark

Laurel, Brenda (1986): Toward the Design of a Computer-Based Interactive

Fantasy System. The Ohio State University

Laurel, Brenda (1993): Computers as Theatre. Reading Massachusetts:

Addison Wesley

Loftus, Geoffrey R. and. Loftus, Elizabeth F. (1983): Mind at Play. New

York: Basic Books

Manovich, Lev. (1998): ’ Navigable space’ (downloaded from

http://www.manovich.net/docs/navigable_space.doc)

Mattus, Maria (1999): ’The Hypertextual Dialogue between Living History

and True History’.Paper presented at the 14th Nordic conference for media

and communication studies in Kungälv

Mayer, Paul (1999): A Social Semiotic approach to the analysis of computer

media. Copenhagen: University of Copenhagen

Mead, G. H (1934): Mind, Self, and Society. Chicago: University of Chicago

Press

Miall David S. (1997): ’The Body in Literature’ . In Journal of Literary

Semantics, 26 (3), 191-210

Mulvey. Laura (1989): ’Visual Pleasure and Narrative Cinema’. In Visual

and Other Pleasures. London: MacMillan (The article ’Visual Pleasure and

Narrative Cinema’ was originally published in Screen 1975)

Münsterberg, Hugo (1915): ’Why We Go to the Movies’. In Cosmopolitan

December 15

Münsterberg, Hugo (1916): The Photoplay: A Psychological Study. New York:

D. Appleton




260

Narayanan. Srini (1996): ’ Embodiment in Language Understanding:

Modeling the semantics of causal narratives’. AAAI Fall-Symposium on

Embodied Cognition and Action, MIT November. Downloaded from

http://www.icsi.berkeley.edu/~snarayan/

Nilsson, Mats-Eric, Hedberg Mats (1995): ’Toppbetyg till Myst igen’. (’Myst

still at the top’. My translation of the title). Stockholm: Svenska Dagbladet

95 -12-11

Nordin, Svante. (1994): Filosofins historia. Lund: Nordin and

Studentlitteratur

Ong, Walter (1982): Orality and Literacy: The Technologizing of the Word.

Bristol: Ong

Orthony, Glore, and Collins (1988): The Cognitive Structure of Emotions.

Cambridge: Cambridge University Press

Plantinga , Carl, and Smith, Greg M. (1999): Passionate Views: Thinking

about Film and Emotion. Baltimore: Johns Hopkins

Pratchett, Terry (1993): Only You Can Save Mankind. Corgi

Reeves, Byron and Nass, Clifford, (1996): The Media Equation. Cambridge:

Cambridge University Press

Russel, Stuart and Norvig, Peter (1995): Artificial Intelligence: A Modern

Approach. London: Prentice Hall

Stam, Robert, Burgoyne, Robert and Flitterman-Lewis, Sandy (1992): New

Vocabularies in Film Semiotics: Structuralism Poststructuralism and

beyond. London: Roultledge

Strauss. A. (1956): The Social Psychology of George Herbert Mead. Chicago:University of Chicago Press

Söderbergh Widding, Astrid (1992): Gränsbilder; det dolda rummet hos

Tarkovsky. Stockholm: University of Stockholm

Söderbergh Widding. Astrid (1994): ’Kvinnoroll och åskådar perspektiv’.

(’The role of women and audience perspective’. My translation of the title). In

Häften för Kritiska Studier 3-4. Stockholm

Tagg, Philip (1992): ’Towards a Sign Topology of Music’. Originally in Secondo convegno europeo di analisi musicale, 369-378, ed. Dalmonte, R.



REFERENCES

261

and Baroni, M. Trento, Università degli studi de Trento. Here quoted from

http://www.theblackbook.net/acad/tagg/articles/

Tan, Ed S. (1996): Emotions and the Structure of Narrative Film: Film as An

Emotion Machine. Mahwah, New Jersey: LEA

Thente, Jonas (1997): ’ Modet att Öppna Nya Världar’. (’The Courage to Open

New Worlds’. My translation of the title). Göteborgs Posten 1997-12-18

Thorstensson, Niklas (1999 ): Male/female–identifying of voices in

SpeechDat. B.A. Thesis. Stockholm: KTH

Truffaut. François (1968): Hitchcock om Hitchcock. ( Le Cinéma selon

Hitchcock). Swedish translation. Stockholm: Pan/Norstedts

Wallace, R. J. (1988): Review article of Johnson’s The Body in the Mind. In

Philosophical Books, 29

Wittgenstein. Ludwig (1953): Philosophical Investigations. Oxford: Blackwell

Zetterberg, Lennart (1995): Ljudinspelningens ABC. (The ABC of Sound

recording. My translation of the title). Halmstad: Spektra

11.2 Books and articles of importance not mentioned

This is a list of other works not mentioned in the text but that have at some

point been a source of inspiration and quotation but they did for various

reasons not make it into this version of the present work.

Carrol, Jon (1994): ’Guerillas in the Myst’. In Wired, August

Carroll, Noël (1990): The Philosophy of Horror. London: Routledge

Chatman, Seymour (1990): ’The Cinematic Narrator’. In Coming to terms:

The Rethoric of Narrative in Fiction and Film. Ithaca: Cornell University

Press

Finneman, Niels Ole (1993): Tanke, sprog og maskine - en teoretisk analyse

af computerns symbolske egenskaper. (Thought, Language and Machine – a

theoretical study of the symbolic capabilities of the computer. My translation

of the title). Aarhus: University of Aarhus

Fiske, John (1990): Introduction to communication studies. London:

Routledge




262

Fiske, John (1987): Television Culture. London: Routledge

Gianetti, Louis (1990): Understanding Movies. Engelwood Cliffs, New

Jersey: Prentice Hall

Gorbman, Claudia (1987): Unheard Melodies: Narrative Film Music.

Bloomington: Indiana University Press

Jensen, Jens F. (1990): Computer-Kultur, Computer-Medier, Computer-

Semiotik, Aalborg: Nordisk Sommeruniversitet

Jensen, Jens F. (1997a): ’ Interaktivitet’. In Mediekultur no.26

Jensen, Jens F, (1997b): ’Vejkort til Informationsmotorvejen’ . (’Roadmap to

the Information highway’ . My translation of the title). In Mediekultur no.27

Johnson, Vida T. and Petrie, Graham (1994): The Films of Andrei Tarkovsky:

A visual fugue. Bloomington: Indiana University Press

Prinz, W, Sanders, A.F. eds, (1984): Cognition and Motor Processes. Berlin,

New York: Springer-Verlag

Sacks, Oliver (1988): Mannen som förväxlade sin hustru med en hatt, (The

Man Who Mistook His Wife for a Hat: And Other Clinical Tales) Swedish

translation. Stockholm: Prisma

Thompson, Roy (1993): Grammar of the Edit. Oxford: Focal Press

Weis, Elisabeth and Belton, John. Eds. (1985): Film Sound: Theory and

Practice. New York: Colombia University Press

Ödeen, Mats (1988): Dramatiskt Berättande. ( Dramatic Narration. My

translation of the title). Stockholm: Carlssons Bokförlag

11.3

Computer games

Blaster (Williams 1983)

Bubble Trouble (Metcalf, Wareing and Ambrosia Software 1995-97)

Civilization (Micro Prose 1992)

Crystal Calliburn (Little Wing 1993)

Day of the Tentacle (LucasArts 1993)



REFERENCES

263

Defender (Williams 1982)

Doom (Id Software 1989)

Duke Nukem 3D (3d Realms 1995)

Dungeon Keeper (Bullfrog 1997)

Elk Attack (M. Hahn 1987)

F18 Hornet (Graphic Simulation Corporation 1993)

Full Throttle (LucasArts 1994)

Grim Fandango (LucasArts 1998)

GUESS

Ice Hockey (Nintendo 1988)

MacBrickout 4.1 (Leapfrog Software 2001)

Maniac Mansion (LucasFilm 1987)

Marathon (Bungie 1994)

Marathon II (Bungie 1996)

Marathon III (Bungie 1998)

MineSweeper (Finley 1992)

Myst (Brøderbund 1993)

Mystery of Monkey Island (LucasArts 1989),

Narc (Williams 1988)

NHL 2000 (EA Sports 1999)

Ornithologist (suggested game idea. Wilhemlsson 2000)

PacMan (Nanco/Midway MFG 1980)

Pengo (Sega 1982)

Phantasmagoria (Sierra 1995)




264

Pong (Atari 1972)

Quake (Id Software 1996)

Riven (Brøderbund/Cyan 1997)

Robotron 2084 (a.k.a. Robotron)(Williams 1980)

Sam & Max Hit the Road (LucasArts 1993)

Scare the Bear (suggested game idea. Wilhelmsson 2000)

Secrets of the Luxor (Mojave 1996)

Sim City 2000 (Maxis 1993)

Space Invaders (Taito 1979)

Spacewar (Russel 1961)

Stargate (Williams 1981)

Super Mario Bros. (Nintendo 1986)

Super Wing Commander (Origin 1996)

Tennis for Two (Higinbotham/Brookhaven National Laboratory 1958)

Tetris (Pazhitnov 1985)

The Daedalus Encounter (Mechadeus 1995)

The Secret on Monkey Island (LucasArts 1990)

Tomb Raider (EIDOS 1997).

Unreal Tournament (Epic Mega Games/GT Interactive 1999)

Zelda II: The Adventure of Link (Nintendo 1987)

Zork (Infocom 1981)

11.4 Movies

2001 (Kubrick 1968)

Blackmail (Hitchcock 1929)



REFERENCES

265

Cape Fear (Thompson 1962, Scorseese 1991)

Carrie (de Palma 1976)

Cinderella (Geronimi, Jackson and Luske 1950)

Indiana Jones series (Spielberg 1981, 1984, 1989)

Lady in the Lake (Montgomery 1947)

Les Vacanses de Monsiour Hulot (Tati1953)

Nosferatu (Murnau 1922)

Only Angels Have Wings (Hawks 1939)

Rope (Hitchcock 1948)

Roxanne (Schepisi 1987)

Se7en (a.k.a. Seven) (Fincher 1995)

Stalker (Tarkovsky 1979)

Star Wars series (Lucas 1977, Kershner 1980, Marquand 1983, Lucas 1999)

The Great Train Robbery (Porter 1903)

The Lost World: Jurassic Park II (Spielberg 1997)

The Ten Commandments (DeMille 1956)

The Wizard of Oz (Fleming 1939)

To Have and Have Not (Hawks 1944)

Weekend (Ruttmann 1930)

11.5 Other sources (internet etc.)

” cognition” Encyclopædia Britannica Online.

http://www.eb.com:180/bol/topic?idxref=82659 (2000-04-14)

http://spaceinvaders.retrogames.com/html/history.htm(2000-04-28 14.09)




266

” Münsterberg, Hugo” Encyclopædia Britannica Online.

http://www.eb.com:180/bol/topic?eu=55666&sctn=1

(2001-01-10)

Brook Bakay. Eugene Jarvis Homepage found at:http://www.geocities.com/SiliconValley/Haven/5521/index.html

(2001-03-05-11-05 14.33)

http://come.to/hatten

http://laban.vr9.com/ie.html

http://videogames.gamespot.com/features/universal/hist_pacman/index.html

(2001-03-05 12.27)

http://www.ac.wwu.edu/~market/semiotic/lkof_msl.html

http://www.ammi.org/exhibitions/cs98/

http://www.britannica.com/bcom/eb/article/5/0,5716,115435+8+108681,00.ht

ml?query=dasein

http://www.cicv.fr/association/shaeffer_interview.html

http://www.classicgaming.com (2001-03-05 11.55)

http://www.classicgaming.com/museum/2600.shtml

http://www.dtstech.com/cinema/index.html (2001-03-05 12.10)

http://www.dtstech.com/cinema/index.html (2000-02-21 10.10)

http://www.eb.com:180/cgi-bin/dictionary?va =Avatar (2000-05-18 10.20)

http://www.emuclassics.com/slydc/pong.htm (2001-03-05 12.25)

http://www.emuunlim.com/doteaters/index.htm by William Hunter. (2001-

03-05 9.35)

http://www.eusa.ed.ac.uk/societies/filmsoc/films/blackmail.html

(2001-02-27 11.37)

http://www.fas.org/cp/pong_fas.htm

(this page was accessed in December 1999 but has since then been closed

down and is at the time of writing not accessible!).



REFERENCES

http://www.geocities.com/SiliconValley/Haven/5521/index.html

(2000-05-21 15.43).

http://www hfac uh edu/cogsci/keytopics html

enacting the point of being_pov

Documents