Perceptual CartoonsPerceptual Cartoons

Towards a spatial ontology in Towards a spatial ontology in artificial sound environmentsartificial sound environments

Peter Lennox, University of DerbyPeter Lennox, University of DerbyTony Myatt, University of YorkTony Myatt, University of York

ONTOLOGYONTOLOGY

What we perceive, and

what really exists

Is the ontology of a given artificial Is the ontology of a given artificial environment:environment:

• Arbitrary?• Subjective?

• Convenient-but-not-real?

What could be realistic (i.e. perceptually plausible) in an artificial sound

environment?

Plato might have a thing or two to Plato might have a thing or two to say about itsay about it

• What can possibly be “real” in an artificial environment?

(if the choice of metaphorical representation seem arbitrary, surely it’s all just fiction?)

Ontological Hierarchy = levels of description

Leucippus and DemocritusLeucippus and Democritus

…There are only atoms and the void All else is just surface appearance

• So Plato and Democritus actually agreed on something!

Plato’s CavePlato’s Cave

What makes real things What makes real things real?real?

• Are “objects” simply the convenient names for clumps of atoms at our sort of scale?

• Are “events” (featuring causes, processes and effects) simply convenient shorthand for a chunk of space-time in which change is apparent?

• And what are “places”?

Jaron LanierJaron Lanier

Why cats and apples are real…Boundaries (physical, but most importantly,

causal)That is, it is the constraints inherent in

causal boundaries that determine relational properties

Boundaries are why “things” can have this set of relational properties, and not that set

Hence Ontology=Hence Ontology=

• Hierarchical description of causal boundaries (giving rise to relational properties) conveniently labelled:

“entities” (objects or organisms) “places”, “events”, ”behaviours”, “intentions”

So, the ontology of an artificial So, the ontology of an artificial environment environment isis what makes it what makes it

‘informative’‘informative’

• An artificial environment’s ontology is not arbitrary (though a composer’s use of it might be)

• Focussing on elements in a given ontological hierarchy is what defines ‘plausibility’ or ‘realism’

Artificial audio examples:Artificial audio examples:

• Image properties such as apparent source width (ASW), focus, blur, stability, localisability

• Place characteristics such as depth of field / ensemble depth, spaciousness , size

• Behaviour characteristics such as trajectory, smoothness

More?More?

• What about:• The integrity of events,• The distinction between entities and

objects• Between behaviours and intentions• Item interactions

In what way are metaphors In what way are metaphors realistic?realistic?

Causal coupling, not ‘like’ (in the sensory sense)

i.e this is metaphorically representative of that if the representational properties of one co-varies with those of the other (along some dimension) in a [potentially] intuitive way, such that a perceiver cognitively engaging with the representation is in some way engaging with the represented.

examplesexamples• Steering wheel, accelerator, level meter,

Geiger counter, boat tiller

• Just a thought…are metaphors the actual currency of perception? -Plato and the cognitive representationalists are right…?

• Hence, “perceptual cartoons” - what we know of environments is constrained by what we can know

Can we Can we stretchstretch perception perception• Inventing new perceptual cartoons, featuring

novel combinations of ‘off the shelf’ items?

• We suggest that that is what perception in fictional, artificial environments is

(I.e - music is not Pinker’s “auditory cheesecake”)

Composers constrained by legacy Composers constrained by legacy metaphors…metaphors…

• The informative components of the ontological hierarchy of an artificial spatial sound environment are limited more by conception than by technology

Composers don’t want to be too Composers don’t want to be too literalliteral

• Strong traditions of being metaphorically ‘like’:

• Flight of the Bumble Bee, Dance of the Knights, L’apres mid d’une faun

Strong chance composers wouldn’t Strong chance composers wouldn’t want to be too want to be too spatiallyspatially literal literal

• So, what could they have to work with?“entities” (objects or organisms) “places”,

“events”, ”behaviours”, “intentions”• But, they could also access an ontological

hierarchy at a different level, playing with the relational properties that define entities that behave in locations in places

Entities: objectsEntities: objects

• Behaviours such as trajectories, bouncing, scraping, smashing, coalescing

Entities: organisms (the hard Entities: organisms (the hard problem)problem)

• aware of each other, interacting: Swarming, flocking, chasing, cautious, bold - intentions

Places: away from sound fields, Places: away from sound fields, sensory attributes and passive sensory attributes and passive

recipientsrecipients• Passive recipients do not need cognitive

causal maps, exploring perceivers do• Spatio-temporal causal cognitive maps are

central to perception; they are how we understand places

• Can we make artificial audio places that are more comprehensible?

Examples (1)Examples (1)

• A long thin corridor, with balls being struck so that they bounce along, changing size, consistency and mass, as they travel.

• Can this be done without the result becoming formless, random, nonsensical?

Examples (2)Examples (2)

• Can a place be explorable, spatially ‘textured’ and extended?

• Can one place be inside another? (a building in a town, a tent inside a cathedral)

• Can adjacent places, experienced successively, yield information amounting to a narrative?

The Morning LineThe Morning Line• Currently in Vienna• “The design of The Morning Line's audio system

encourages experimentation in approaches to spatialization. Its highly irregular speaker array frees composers from the impetus to utilize space in the traditional way—as movement around a sweet spot for universal reproduction—and instead encourages exploration of the potential for spatialization within a very specific system.”

• (Malham et al, 2010, AES 128th convention London)

The Morning LineThe Morning Line

… is a large outdoor sculpture, conceived by Mattew Ritchie and commissioned by Thyssen-Bornamitza Art Contemporary (TBA-21) consisting of a series of truncated tetrahedral modules constructed of aluminium plate that are linked together to form an open, fractal based structure… 24m long x 12m wide x 8m high.

The Morning Line:- The Morning Line:- http://www.tba21.org/presshttp://www.tba21.org/press

Conclusions (and further work)Conclusions (and further work)

• We find that spatial cognition in a fictional environment can be engaged in various ways, in extended artificial places populated by fictional entities displaying unreal-yet-perceptually plausible behaviours.

• It is (fortunately!) not necessary to be sensorially literal – hence “perceptual cartoons”

• We are developing compositional environments on these principles

• We have not yet tackled ‘intelligent agents’• We are still arguing with Plato