mapping inside out

Mapping Inside Out

Stephen Boyd Davis, Magnus Moar, Rachel Jacobs, Matt Watkins, Mauricio Capra

2 Mapping Inside Out Mapping Inside Out 3

1 Introduction

Commuting is dull, especially if it involves a long walk which is the same every day: the boring shops, the dreary pavement, which even listening to an MP3 player cannot make enjoyable. How much more tempting to go by car or other transport. Could this preference be changed by experiencing a pervasive other reality, one which is responsive both to where I am and to something within myself? This question has been explored in ’Ere be Dragons, a pervasive game and digital artwork created by a small multidisciplinary team of artists, technologists and health scientists.

The project was funded by the Wellcome Trust under the SciArt programme, with a view to stimulating thought and debate about issues surrounding biomedical science, exploring art and science crossovers, and initiating new areas of research. The focus of ’Ere be Dragons is the relationship between art, technology and health. It playfully recasts players’ relationship both with the space they inhabit and with the unseen space within their bodies. The player’s own body becomes an engaging new companion, a soft machine whose physical response to their exertions they can sense and understand as they walk along. In discipline terms, the project combines the playful, exploratory tradition of art and design with the rigour of health science.

Here we set out the health science agenda, some issues in designing with live heart rate data, and our approach to mapping the exploration in the game. We begin by describing the player’s experience and the technical implementation.

Social Games

’Ere be Dragons has been trialled in Nottingham, London, Singapore and Berlin and presented in Dublin, Glasgow, Bristol, Sao Paolo and Taipei. It is a developing project which has passed so far through two iterations, one single-player and one multi-player.

In its simplest form ’Ere be Dragons is experienced as follows. The player wears a heart-rate monitor, and inputs his or her age into a pocket PC with attached GPS module. Carrying the pocket PC, all the other equipment is placed in a small shoulder bag. On the basis of the player’s age, an optimal heart rate is calculated

and the player proceeds to walk wherever and however they wish. During the walk, an on-screen landscape is built up which maps the player’s real location (Figures 1 and 2). This landscape corresponds spatially to the real one around the player. By using GPS, the game environment is tied to the player’s real space, so features in the virtual world are encountered in a fixed relation to the real one: a certain location will be seen on screen each time the player passes a particular place in the real world. If the player turns right, they see the trace of their journey make a rightward turn. If they take a walk which returns to a place already visited, the new path intersects the old. However, the virtual landscape does not already exist when the game begins: it is created by the player through the gameplay itself.

If the player does well, adequately exercising their heart, the landscape flourishes. The heart rates are split into five ranges on the basis of the optimal heart rate originally calculated from their age. Territory is presented as isometric tiles with distinctive colours and landscape features. Insufficient exertion causes the current landscape tile to become impoverished, while over-exertion leads to the growth

Fig. 1. Two scenes from the landscape of ’Ere be Dragons, showing the different characters of terrain resulting from the player’s heart-rate performance


Fig. 2. A player explores Singapore, and the Dragons world, at the ACM Multimedia Art Exhibition in November 2005.

Fig. 3. The player’s opening encounter with ’Ere be Dragons, designed as part of the game experience. Players lie still to enable their heart rate to be captured while they watch an introductory video. In the foreground, part

of the personal heart rate monitoring equipment.

of dark, forbidding forest. If the heart rate remains too low or too high for more than three minutes then the player begins to lose territory. If they lose more than ten tiles they are eliminated from the game and a message appears on the screen telling them to return to base. In addition to changing the visualisation of the landscape on screen, the five bands are used to affect the soundtrack, so that looking at the screen is not always essential. There is also a simple status bar, principally to make the response more obviously ‘live’.

A simple point-scoring system is used to log their overall performance. The player’s motivation is to keep the game world alive and flourishing, by behaving in the healthiest way, creating a new world which, though objectively mapped, is theirs alone.

2.1 Multiplayer Experience

For Version Two of the game, premiered at the Radiator Festival in Nottingham, staff of the Mixed Reality Laboratory at Nottingham University created a global interface for an audience to view the players’ movement during their journeys, and created a client-server system to send and receive data between the global interface and the

players. This altered the game significantly by allowing social, multiplayer aspects which had been absent from the earlier versions. The experience was also designed with a performance element which complemented the gameplay itself.The players’ experience began in a room where they lay down while their hearts slowed to resting heart rate (Figure 3). This choice of method is discussed below (Designing with Heart Rate). Examination couches and medical screens were used to rhetorically reinforce the health agenda of the game. While players relaxed, they watched a video showing what the game was about. After putting on the heart rate sensor chest strap and shoulder bag, they waited outside for the GPS fix before embarking on the game.

As before, players control the visualisation of the virtual world on the screen of the pocket PC through their own live heart rate data, with the goal of maintaining optimum heart rate. But now players can also track another player’s journey. If they follow the same track as another player they can score more points, and can see other players’ journeys if they cross paths (Figure 4). These are indicated as white isometric squares within the player’s virtual landscape. The game computes a slightly higher heart rate than the real one whenever the player is following the track of another player, as a reward for entering their territory.


Players are not expected to interact with the pocket PC directly but rather to influence its response through their physical behaviour: it responds transparently to everyday actions. They need perform no explicit interaction with the screen of the device, though they do receive messages to inform them of activity within the game play every five minutes (and alerts to technical issues). After twenty minutes the game displays a message asking the player to return to where the game started. Along the way, the player has a last surprise. An artist wearing a dragon costume tracks their return to the base and hands them a leaflet as a reward for finishing the game (Figure 5). This acts as an information tool about the game and a certificate of taking part.

2.3 Player Reactions

Players’ reaction to ’Ere be Dragons was gathered in a number of ways: direct observation, field video recording, questionnaires, and informally.

When Version One was played in Singapore, heat and humidity militated against prolonged use. Nevertheless, visitors gave a very positive response to the integration of health issues and the player’s awareness of heart rate activity within an engaging game experience. It might seem that the use of an ECG belt and a package of equipment joined by cables would be an alienating experience,

but players very much enjoyed this aspect. The ‘liveness’ of the heart rate data changing the virtual landscape created fascination and provoked some very interesting discussions with players about health, psychology and engagement.Some players became immersed in the game to the extent that it was dangerous – this is clearly an issue for all pervasive media experiences of this kind. Despite the presence of the responsive soundtrack, players tended to attend to the screen even when this was unwise and in some cases had to be warned of dangers such as traffic by passers-by. One player was robbed, losing the GPS module of the equipment to a street thief. An obvious partial solution is to transfer more of the communicative burden to the aural channel. Schminky [36], a project also built on the Mediascape infrastructure, has prioritised sound, as has Riot! 1831 [12]. However auditory experiences bring their own problems of navigation and user understanding.

In accordance with our intentions, discussed later in the chapter, players chose their own mode of play. Some chose to play competitively, talking for example in terms of ‘stealing territory’ while others, though in a multiplayer game, elected to explore on their own, simply enjoying the developing landscape and its response to their own physiology. But the multi-player version of the game was well liked, with players comparing game strategy after playing, wanting to play again, beat their own score and explore more of the territory. Players liked the fact that they could play the game as they pleased.

Fig. 4. Dragons played as a multi-player game. Some players chose to play competitively, others to play alone, on the streets of Nottingham in December 2005.

Fig. 5. The player’s final encounter with ’Ere be Dragons. The player is greeted by an artist in a dragon costume, introducing an additional element of performance, and surprise, to the experience.


Most players liked the isometric landscape. The small size of the display was a problem for some. Players enjoyed the messages and soundtrack but felt the sounds could be more responsive to the heart rate. In terms of usability, in the shared version of the game, not everyone understood the direction they were going in and how to find other players, and some refining of on-screen messages was required to make the experience as direct and engaging as possible. Most players understood the metaphors and symbols within the game. People enjoyed feeling that they were engaging in scientific exploration of biomedical issues through technology without having to learn how to use it.

3 Implementation

For ’Ere be Dragons we needed to develop a playable game within the timeframe of a short research project. Others have integrated bio-sensing with location data in a game: for example Headon and Curwen [20] developed such a system to drive the game Quake, but this was tied to an indoor location while we needed to integrate these data in a device as portable and free-ranging as possible.

The Mediascape architecture, developed by Hewlett Packard’s UK research laboratories and Bristol University for the Mobile Bristol project, is a flexible tool for authorising spatialised interactive media. It allows multiple devices, including location-sensing technologies, to contribute data to a single software interface

Fig. 6. The architecture of the mobile component of ’Ere be Dragons.Fig. 7. Heart rate data was captured using an inexpensive chest-band

transducer (left), connected to a sensor-bus unit (right).

[24]. The framework has three parts: an XML-based specification language for laying out the virtual world of the application and defining how client devices respond to events; a PC-based graphical authoring tool for creating application specifications in this language; and portable client hardware and software that can download and interpret such specifications. The authoring tool is typically used to attach software events to zones in a predefined territory [2][12][46]. Our use was to enable access both to GPS data and heart-rate data from within our chosen tool for rapid-prototyping (Figure 6). Our preliminary research had revealed that most of the proprietary systems for interfacing pocket computers, PDAs or smart phones to external sensors were (at that time) closed systems which we could not easily program ourselves. The Mediascape architecture by contrast is designed to facilitate just the kind of work we wanted to make. Since location-awareness is fundamental to Mediascape, the interfacing of a standard GPS device was simple, but no previous project involved live heart rate monitoring.

The heart-rate input came from chest-band transducer heart-rate monitors communicating wirelessly with a sensor unit developed by ScienceScope of Bath, UK. This device then converts the data received, sending it to the pocket PC through a cable.

The data streams were interpreted, and the graphics and sound delivered, using Macromedia Flash. This was not ideal in terms of speed and capability, but allowed affordable rapid development. At one stage we experimented with three-


dimensional imagery in the visual interface, but this proved prohibitively slow. The solution, using a quasi-three-dimensional isometric view, added a nostalgic aspect to the graphics, evocative of earlier digital games.

For Version Two, the Mixed Reality Laboratory, our project partners at Nottingham University, built the global interface for an audience to view players live on their journeys, and created a client-server system to send and receive data between the global interface and the players. Score and territory explored were transmitted to the other players using GPRS. Different combinations of hardware were used. Some devices had Bluetooth GPS and a Tri-band GPRS card in the Compact Flash slot (an Audiovox RTM 800). Others had a Compact Flash GPS card and used a Bluetooth smartphone for GPRS data-transfers. Network calls to the server were made from within the Flash program using URL-encoded strings. All devices had a single ear-piece to enable the player to hear the soundtrack which was also driven by the Flash interface.

The Web server was a standard Linux, Apache, MySQL, PHP (LAMP) installation. All communication was done via HTTP through PHP scripts and the data was persistently stored in a MySQL database. The need for mobility required interfaces between PDA, GPS receiver, mobile phone and heart rate monitor in order to keep the data streamed. GPS and GPRS were not consistently available and this

sometimes spoilt the player’s experience. Notoriously, GPS can be problematic in city centres through shadowing by high buildings and multipath reflection. This was not a concern when the receiver lost contact for a few seconds, but there were some regions in the city where the players walked or ran for several minutes without signal which made their virtual world disappear. We resorted to a message informing the player of the lack of GPS signal and a reminder to keep away from tall buildings!

GPRS contention was another point to be considered. Trialling the game in a busy commercial centre at the height of the Christmas shopping season, mobile network demand seemed at times to exceed the maximum number of users, resulting in disconnection of some of the devices. As with most portable electronic devices, battery-life was also a problem.

4 Health and Play

As indicated, the project was a collaboration involving both health scientists and artists. Health scientists despair of solving the current obesity crisis using conventional public health messages [17[[34], and some are interested in new ways of engaging people with the health consequences of behaviour. Our team included health scientist Professor Chris Riddoch, who is not only an authority on obesity and fitness but also advocates alternative, novel approaches to health education. Together we hoped we might begin to make a difference to people’s perception of healthy living.

4.1 The Health Problem

There is a growing tendency to obesity in the populations of developed countries. Figure 8 indicates the percentage of those in England classed as obese from the 1980s to the present: the figure has roughly tripled in that period. It is common to blame diet, especially the consumption of fatty foods, for this increase. It is however the balance between energy consumption (from fats and carbohydrates) and energy expenditure (physical work, exercise and sport) which is crucial to the trend. Figures 9a to 9c indicate some components of the picture. In Figure 9a, overall energy consumption (and, in particular, fat consumption) can be seen not to have

Fig. 8. Trends in Obesity in England, 1980-2001 (Percentage of the population classed as obese). Office of National Statistics, Health and Safety Executive, various years.


increased since the 1970s – in fact to have declined. But when television viewing hours and car ownership are plotted as indices of inactivity (Figure 9b), there is an obvious similarity to the growth in obesity over the same period (Figure 9c).

Though obesity is a direct cause of some health problems, it is almost more important as an indicator of a lack of physical activity. Someone may be slim, but still unhealthy, if the cause of their slimness is low energy consumption rather than high energy expenditure. It is known that there is a strong relationship between physical activity and several aspects of health. For some health problems, activity is far more beneficial as a prevention than a cure. Examples are coronary heart disease, type-2 diabetes and osteoporosis [17]. Public health scientists consider it urgent that the public become more aware of the issues and how these relate to personal behaviours. A variety of strategies has been tried, but simple exhortations to live more healthily have had little effect. The alternative approach that we wanted to try was to engage people with the interior world of their own bodies – specifically the performance of their heart – in a playful context and in so doing perhaps incite them to behave differently.

The players we aimed to engage were not those already interested in sport or health issues, who are increasingly well catered for by mobile technologies. There are already fitness programmes available for portable devices such as watches, phones and music players, some of which interface with heart-rate monitors – for example the combination of Nokia mobile phone, Polar heart-rate monitor and Fitness Coach software [29] – but these are aimed at those already committed to fitness. Our audience on the other hand is the rest of the population – the majority – who do not have this prior commitment. It includes avid gamers, who are interested in videogames rather than fitness, but who might be incited to engage with their own health if it were presented in a playful context.

Though great fun, videogame playing is of course systemically unhealthy. Limiting interaction to mouse or game-pad presses does little to promote a healthy lifestyle, as Mueller and Agamanolis note in their argument for a mixed reality football game [32]. Lankoski et al [27] point out that a requirement for physical movement may actually repel hard-core gamers.

Our project is at the opposite extreme from health applications such as Shih et al’s iPaq-based remote monitoring of subject’s health [42], a form of (benevolent)

surveillance. We want ’Ere be Dragons to be experienced as a pleasurable, creative challenge, not one that induces guilt or tells players what they must do for their own good. Feedback from the various trials of ’Ere be Dragons suggests that players do indeed enjoy discovering more about their own physical responses [8][9]. However, it would require far more extensive trials to determine any longterm effects on behaviour.

There is increasing interest in the relationship between health and digital games. Three Games for Health conferences (at Madison, Wisconsin in 2004; Baltimore in 2005 and 2006) have brought together developers, researchers, and health professionals. The remit is broad: only some of the projects are concerned with actually integrating digital game activity with exercise. Many games which combine interactive digital experience with physical body action are derived from familiar competitive sporting games which may already be repellent to those whom we would like to attract. Recent projects which have aimed at those already committed to fitness include Virku, which connects an exercise bike to a virtual environment, ‘a fitness computer game which aims at making the exercise session more motivating and rich in experiences’ [30] (emphasis added). Shadow Boxer [22] – which includes heart rate monitoring but does not use it as part of the game – is also aimed at those explicitly interested in increasing their fitness.

Figs. 9a-c. Obesity in Britain: gluttony or sloth? 9a: Energy consumption, percentage in relation to mean. 9b: Inactivity, percentage in relation to mean. 9c: Obesity, Body Mass Index data. Based on Andrew M Prentice &

Susan A Jebb. British Medical Journal. 1995. Vol 311. 437-9


The interest in health related games has converged with activities in interfacing novel devices, including bio-sensors, to interactive digital systems. Raposa [37] created a game which aimed to tackle asthma symptomatically by combining a modified games console, heart rate and respiration monitoring. Since 2003 there has been at least one commercial quasi-game which uses heart rate sensing: The Journey to Wild Divine [45] uses three sensors worn on the finger-ends and connected to a PC. By controlling their heart-rate players achieve a range of experiences within a virtual world displayed on the monitor.

4.2 A Different Kind of Play

’Ere be Dragons has an unusual relationship to play which sets it apart from most digital games. Huizinga [23: 195-197] traced the status of the body in relation to play from its deprecation in the Middle Ages to the English systematisation of sport in the nineteenth century. Even now, many of the games which combine

interactive digital experience with physical body action are derived from familiar competitive game genres ([14] has a useful summary). These games are both systematised and agonistic: struggles against other players or computer-mediated enemies, within strongly defined narrative structures or with clearly prescribed end-goals. Our work explores instead what Caillois [10] characterised as mimicry (make-believe) and ilinx (the pleasures of movement).

Since so many digital games involve agonistic confrontations either with other players or with a variety of virtual enemies or monsters, we wanted to develop a game which, while it would borrow from the imagery of the conventional game, would pit the player as much against their own self as others. Of the two broad categories proposed by Callois – ludus, a goal-oriented, structured game and paidia, freeform, exuberant play – our project is nearer to the latter. Typical ludus games amongst other projects are Nautilus [43], and Touch-Space [14], both mixed reality games played in a fixed space with a strong game narrative and clear objectives. While our game has an imperative (for the player to maintain their heart-rate close to the optimum level), it is up to the player to decide how to play. In fact, they could increase their heart rate to dangerous extremes if they wished, or attempt to make the rate fall as low as possible. They can follow the same route on every occasion, or explore new territory every time. Sessions can be resumed, or a new one can be initiated at any time; in the multi-player version, players can play together or on their own.

As the creators of this game we are open to players doing what they will with it. This approach has been characterised by Gaver as play-like in two senses: it is the design of pleasurable experiences, activities motivated by curiosity, exploration, and reflection rather than externally-defined tasks, but is itself also an open, playful design process which is open to unforeseen possibilities [19]. We enjoy the wide range of subjective interpretations of the feedback which is offered by Dragons. Much of the ascribing of meaning to the visual and aural representations we create is deliberately left open to the player. Our study of the gameplay in action shows that this is clearly a source of pleasure.

5 Designing with Heart Rate

The integration of live heart rate data into a pervasive game raises some issues which are of interest from both a technical and game-design point of view.

Fig. 10. The Opening Screens of ’Ere be Dragons Version 1. Players choose their age from a pop-up menu. Optimal heart rate and related values are calculated using a simple formula.


Heart rate responds to the demands of exertion. For any individual, it is possible to define an optimal heart rate: that rate at which players are exerting themselves but are not putting their heart under undue strain. Not all players’ heart rates will respond in the same way to the same demands: those who are fittest will exhibit the smallest increase in rate. Importantly, the normal heart rate of the player when not exerting is also very varied, being dependent principally on age and fitness. It is therefore impossible to specify a single optimal heart rate applicable to all players. The standard method of compensating for the differences outlined is to establish the individual’s resting heart rate, the approach used in Version Two of ’Ere be Dragons. This is ideally done early in the day before the player has risen, but a reasonable figure can be ascertained by obliging the player to rest for a while before a reading is taken. Since either of these methods presents some obvious practical problems, an alternative is to ignore the individual fitness component and derive the optimal heart rate solely from a simple agerelated formula as used in Version One. The insensitivity of this method could be compensated for by allowing personal history factors (for example, concerning illness or heart condition) to be included in the calculation through manual input.

Assuming an adequate method has been found for establishing the optimal heart rate, it is then necessary to anticipate the kinds of changes in heart rate which can be usefully monitored. Increase in heart rate from physical exertion is not instantaneous: there is typically a one-minute delay between the exertion and the peak. Alterations in heart rate arising from psychological factors rather than physical factors, however, are almost immediate. Finally, as with any other sensor technology, there are breaks in the signal, noise and errors. This has been noted as a primary issue in the design of many pervasive games [1].

In considering all these factors, we were not devising a system for medical use: there was no need for high levels of accuracy or very fast response. From the game design point of view, even a simple response to the heart rate data would be likely to facilitate an interesting game. However, a number of criteria had to be satisfied for the heart-rate data to be used successfully.

Setting an Optimal Heart Rate for the Game. Given the wide variation in players’ ages it was essential to establish optimal heart rates rather than use a single rate. If such variation was ignored, there could be consequences for the player’s safety. From the game design point of view, it would also be counter-productive to set

a target heart rate that, for a given player, was either too easy or too difficult to achieve. This could easily happen using a single, standardised target rate.

In the first version of Dragons, trialled at the ACM Multimedia Art exhibition in Singapore in November 2005 [7], players started the game by inputting their age (Figure 10). The software calculated their optimal heart rate using standard formulae. For the second version, used at the Radiator Festival in Nottingham in December 2005, an area was set aside where players could rest prior to play and their resting heart rate was the basis of the calculation. This threw up wide variations between individuals even of the same age, and so suggested the value of using real resting heart rate data rather than the age-based formula.

Representing Heart Rate Data for Gameplay. In order for any representation of the heart rate data to be rapidly apprehended, it was necessary to avoid excessive subtlety. There would be no point, for example in creating a large number of different visual or aural mappings of various heart-rate levels, if these could not successfully be exploited in the design of the game. Successful exploitation would of course include the ability of the player to understand what was going on. It is easy to establish two thresholds based on the player’s optimal heart rate: a lower

Fig. 11. Example heart-rate data from the ScienceScope electrocardiogram. Spikes require elimination to produce data suitable for mapping.


value at which they would be exerting themselves insufficiently, and a higher value which represents the threshold of excessive exertion. Between the optimum and these extremes, lie the values at the borders of the optimal band. At any one moment therefore, one of five mappings is derived from the incoming heart rare stream centred on these values. At either extreme, the graphics disappear after three minutes and the game ends. In the low optimal range, desert is seen; around the central value grass, trees and flowers; in the high optimal range, forest. To use as few as three visual outcomes might seem unreasonably crude, but two factors compensate. Firstly, the incoming heart rate data is time-based and, if the player is moving (which is the main point of the game), successive readings are mapped consecutively on screen along the route. What the player therefore sees is not a single momentary reading, but a landscape of which the character at any location is a response to that moment, but whose overall character – pre-dominantly dark, luxuriant or bare – suggests the character of the player’s overall recent performance. Secondly, the mapped, created landscape is not the only representation of the player’s performance. If the player wishes to know their current heart rate, they can see it represented by an indicator at the top of the screen, where both a number and the position across the screen give a finer level of representation of the incoming data. The animation of this indicator also serves to reassure the player that the system is live and working. The aural design also responds in different ways to the five bands: 1. quiet, slow; 2. with more melody; 3. complex melodies; 4. industrial-sounding; 5. fast. We originally used a heart-beat sound, but quickly discovered that this imitated too closely the feeling of a panic attack and created an adverse effect for the player! The final sound design was created by composer James Flower.

A Representation to Cope with Imperfect Data. The design of the game should not offer a misleading impression of its accuracy or responsiveness. The mapping of the data from the heart rate sensor must not become misleading, or unsatisfactory from a game design point of view, as a result of signal interruptions and false readings. ECG monitors, such as the inexpensive bipolar sensor we are using, work by measuring the natural electrical activity of the heart. The physical mechanism consists of two electrodes embedded in a semi-rigid chest strap which is elastic at the back. The sensor we selected (range 0 to 240 beats/minute; resolution 1 beat/minute) communicates wirelessly to a logging device. It is not intended for clinical use but was fully satisfactory for our purpose. For schools laboratory work this device

is connected to a PC through a fairly substantial interface box, but in our case we used the prototype Sensor Bus (developed by ScienceScope in collaboration with Mobile Bristol) which is a smaller, more portable version. To assist in maintaining good contact with the player’s body, a gel is applied to the electrodes. Nevertheless, during use the electrodes may lose contact, leading to missing or nonsense data. For instance, when the player needed to look both ways before crossing a street the sensor could lose the contact with the skin due to the body movement.

The firmware of the device gives some inbuilt smoothing and ensures that the data is in a readily usable form, pre-processed as beats per minute. While even to the untutored eye it is fairly clear that spikes such as those in Figure 11 are errors rather than real data, it was important not to allow such spikes to be represented in the game (for example, if they were, the sliding indicator would suddenly leap to one end of its scale or the other), and so some additional smoothing was undertaken. For the purposes of the current game, we were

Fig. 12. The Bellman’s blank map from Lewis Carroll’s Hunting of the Snark (1876, redrawn)


Finally, we discuss the artistic and philosophical underpinning of the project, particularly in relation to mapping the two complementary data-streams of GPS and heart-rate data. In Lewis Carroll’s Hunting of the Snark (1876), the Bellman proudly presents the ship’s crew with a map which is “a perfect and absolute blank”. The poem tells us that the crew rejoice, principally it seems at the elegance of the design, without ‘conventional signs’ and distracting shapes of islands and capes. In ’Ere be Dragons we also begin with a blank map, but for different reasons. Dragons is an artwork as much as a game. Two members of the project team, Jacobs and Watkins, practice as Active Ingredient. They have created many projects offering interaction with digital artworks, some of them with a science component (for example [13]). The roots of their work lie in Dadaist events, Fluxus happenings and more recent live art and installations with strong influences of the non-western and the indigenous. They use interactivity to enable participation in a creative process: in ’Ere be Dragons players are not experiencing a readymade virtual landscape, but constructing one through their own activity. Players are creating an alternative world which, though contained within the limits of the pocket PC screen, is experienced as overlaid on the mundane actual world of their surroundings. Visually inspired by the aesthetics of game worlds, Dragons turns walking into a creative experience.

Closely related to the open form of play that we espoused was our approach to exploration. The Situationist project to discern the ‘precise laws and specific effects’ of the geographical environment on the individual [16] is implemented in ’Ere be Dragons both in its simple and its inverse form: the city is mediated through the subjective states of the body and the body is mediated through perceptions of the city. The central metaphor of the project is based on the travellers’ tales told by early explorers and on their maps such as the Hereford Mappa Mundi. Early mapmakers were reliant on anecdotal information often further embellished through being second-hand [6: 47-100]. The unknown was a source of fear and therefore populated with dangerous beings; the fantasy imagery of the MUD and

primarily interested in the grosser and slower changes in the heart rate arising from changes in physical exertion, so smoothing did not risk losing any true data of value. However for a project where the more transient changes caused by psychological factors may be of greater interest, it is important to set the smoothing threshold so as not to eliminate potentially interesting rate changes.

We found that the wireless ECG sensors can interfere with one another if used by two players too close together. We have also been told that proximity to powerful motors can create spurious readings, though we did not experience this.

Other Issues. It is important to note some steps that we did not attempt. The interface we created is a combination of two independent data streams which are related to one another in two ways. The source of both is the activity of the player; and the audiovisual representation reunites them in a single view: location on the screen – and in the extended virtual terrain on which the screen is a portal – responds to real location, while the character of the landscape at that point represents the player’s heart rate performance. However, the two streams are not correlated in any other way. For example, one could imagine a system in which the GPS data is used to discover whether the player is climbing a hill, or walking down one, and this could affect in a temporary way the target which the player must meet. Our system does not currently attempt any such correlation and several issues militate against it. These include the obvious question of whether it would improve the gameplay or the informational benefit to the player. The resolution and reliability of altitude data in real world GPS use is a problem; and the one-minute delay already noted in the heart’s response to exertion may present problems of resolution in mapping performance accurately to landscape features.

The player can choose at any stage to continue the current session or to start a new one. In our present version, new sessions begin with a clean slate. There is no attempt to build up a longer-term representation of the player’s performance over more than one session. This is a development we are keen to attempt. It raises interesting design issues concerning the effective audiovisual representation of both intra-session and inter-session patterns, especially within a limited screen space.

“He had bought a large map representing the sea, Without the least vestige of land.”Lewis Carroll. The Hunting of the Snark. 1876

6 Mapping Inside and Out


the videogame can be seen as descendants of this genre. Such maps were not of course made within our own cultural framework: symbolic and metaphorical motives dominated what we might now regard as the truly geographic. ’Ere Be Dragons revisits such mapmaking. The parts of the territory which are mapped, and the visual form that the mapping takes, are subjective in the double sense that mapping only occurs for the immediate locations through which the interactor passes – it is a record of a personal journey, like the periplus, historically a sequential list of features encountered as one navigated a coastline [28: 254]. It is also subjective in the sense that the physical – and implicitly the psychological – states of the interactor are reflected in the visual forms used for each point along the way. In this respect it echoes the metaphorical journeys of literature since Pilgrim’s Progress (Bunyan 1678) in which geography becomes a surrogate for events in the traveller’s personal development [31: 48-50].

Historically there is a long tradition of inscribing images and other ‘foreign’ representations into topographic maps. Holmes [21] presents many examples with naïve enthusiasm, some of which effectively corrupt the data by excessive pictorialisation. However even the soberest topographic maps generally incorporate some pictorial elements, such as the marshland and forest symbols of the UK Ordnance Survey. A powerful example has often been presented by Tufte [44]: Minard’s 1869 map of the route of Napoleon’s army in the Russian campaign of 1812, inscribes a line whose width diminishes as the death-toll from cold and starvation mounts.

Following a period when geographic theory largely turned its back on maps [36: 193], it is now recognised that the role of audience and objectives is pivotal in determining the messages they convey, and geographers question the former ‘obsession with landscape’ [25: 207]. Relativistic thinking informs feminist and other challenges to traditional absolute spatial geographies [39 passim], so that Cartesian distances are replaced by models representing, for example, accessibility by particular users. Recent pervasive digital projects exploiting such relative spatiality include the game Feeding Yoshi [4] where players reveal the normally invisible technological infrastructure of the city, re-seeing the city in unfamiliar ways, and Blast Theory’s projects investigating behavioural changes brought about by ubiquitous location-aware mobile devices [15][3]. The various Urban Tapestries projects of Proboscis [26], with a socio-political as well as an artistic agenda, offer mechanisms for individuals and groups to construct their own subjective overlays on the city.

6.1 Outside and In

Though the subjective relation to the city is at the heart of the Dragons project, this subjectivity is not of an obvious kind, concerned with impressions and preferences, since it is itself formed by data streams, GPS and heart-rate, each objectively measured, though inherently personal to the individual. Players see themselves from outside, as though they were distant elements of the cityscape; they also see a process from their own interior space combined with landscape data. As they walk they are sharing the city with others, aggressively or collaboratively in the multi-player version; but it is also a unique personal city, uniting the physical and symbolic in truly ‘embodied interaction’ [18: 205-7]. These issues have been touched on by other artists using sensor technologies, for example Raposa’s project mentioned earlier [37]. Nold [35] has tracked emotional states in a landscape using galvanic skin response data which is uploaded to a mapping interface at the conclusion of each journey. This merges the data to produce a personalised ‘emotion map’, tracing the peaks of the individual’s physical responses onto Google Earth images, so that a saw-tooth vertical trace is inscribed on the landscape.

A distinction made by Schön highlights an important difference between Nold’s project and our own. Both are special cases of external cognition [40], but, rather than representing processes of the mind, they represent those of the body. As Nakakoji and Yamamoto [33] point out, much of the work in this tradition is based on the external representation as an object, not on the externalizing process as such. Schön [41], they note, is unusual in arguing the value of reflection-in-action, as against reflection-on-action, where the constructive, creative experience is enhanced by constant feedback, such as when a designer draws, and observes the drawn marks, in a continuous lived experience. In Nold’s projects, the interactor uploads bio-data to the system after walking and reflects then on the experiences that gave rise to them. By contrast, instant feedback is the essence of our project and of MIT’s Relax To Win [5], an interactive game (static, not pervasive) that uses bio-data to encourage players to change their behaviour, specifically to relax. Coincidentally based on the image of a dragon, in Relax to Win players must lower their galvanic skin response in order to make their on-screen creature fly. In ’Ere be Dragons too, action and reflection are simultaneous, so that players are constantly incited to adjust their behaviour in light of the representation of their current state.


Carter [11: 207] said of mapping the mind that ‘There is no ghost in this place, no monsters in the depths, no lands ruled by dragons. What today’s voyagers are discovering is instead a biological system of awe-inspiring complexity. There is no need for us to satisfy our sense of wonder by conjuring phantoms’. ’Ere be Dragons has remained true to the principles and motives of health science, but has also creatively ‘conjured phantoms’ in order to reveal to players both the city, and the body, they inhabit.

7 Acknowledgements

We gratefully acknowledge the support of the Wellcome Trust. We are indebted to Richard Hull and Tom Melamed of Hewlett-Packard’s Bristol (UK) research laboratories, to David Crellin of ScienceScope, Bath, UK for access to prototype technologies and expertise, to Robin Shackford for additional programming, and to the Mixed Reality Laboratory at the University of Nottingham, in particular to Leif Oppermann. The project would have been impossible without the health science expertise of Prof. Chris Riddoch and Karl Cooke.

References

1. Benford, S., Magerkurth, C., Ljungstrand, P.: Bridging the Physical and Digital in Pervasive Gaming. In: Communications of the ACM, Vol. 48, 3 (2005) 54-57

2. Benford, S., Rowland, D., Flintham, M., Drozd, A., Hull, R., Reid, J., Morrison, J., Facer, K.: Life on the Edge: Supporting Collaboration in Location-Based Experiences. In: Proceedings of CHI2005, (2005) 721-730

3. Benford, S., Seagar, W., Flintham, M., Anastasi, R., Rowland, D., Humble, J., Stanton, D., Bowers, J., Tandavanitj, N. Adams, M., Farr, J.R., Oldroyd, A., Sutton, J.: The Error of our Ways: The Experience of Self-Reported Position in a Location-Based Game. In: Proceedings UbiComp 2004, 6th International Conference on Ubiquitous Computing, Nottingham, UK (2004)

4. Bell, M., Chalmers, M., Barkhuus, L., Hall, M., Sherwood, S., Tennent, P., Brown, B., Rowland, D., Benford, S. Hampshire, A., Capra, M.: Interweaving Mobile Games With Everyday Life. In: Proceedings of CHI’06, Montreal, Quebec, Canada (2006)

5. Bersak, D., McDarby, G., Augenblick, N., McDarby, P., McDonnell, D., McDonald, B., Karkun, R.: Intelligent Biofeedback Using an Immersive Competitive Environment. Online Proceedings for the Designing Ubiquitous Computing Games Workshop, Atlanta GA (2001) http://medialabeurope.org/mindgames/publications/publicationsAtlanta2001rev3.pdf

6. Berthon, S., Robinson, A.: The Shape of the World, George Philip, London (1991)

7. Boyd Davis, S., Moar, M., Cox, J., Riddoch, C., Cooke, K., Jacobs, R., Watkins, M., Hull, R., Melamed, T.: Ere be Dragons: An Interactive Artwork. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, Multimedia’05. ACM Press, New York, NY (2005) 1059-1060

8. Boyd Davis, S., Moar, M., Jacobs, R., Watkins, M., Riddoch C., Cooke., K.: Ere be Dragons: Heart and Health. In: Strang, T., Cahill, V., Quigley, A. (eds.): 3rd International Workshop on Pervasive Gaming Applications, Dublin, Ireland (2006) 157-164

9, Boyd Davis, S., Moar, M., Jacobs, R., Watkins, M., Riddoch C., Cooke., K.: Ere be Dragons: Heartfelt Gaming. In: Digital Creativity, Vol. 17, 3 (2006) 157-162

10. Caillois, R.: Man, Play and Games. Translated by Meyer Barash from Les Jeux et les Hommes, University of Illinois Press (1958)

11. Carter, R.: Mapping the Mind. Weidenfeld and Nicholson, London (1999)

12. Cater, K., Clayton, B., Geelhoed, E., Hull, R., Reid, J.: Parallel Worlds: Immersion in Location-based Experiences. In: Proceedings of CHI’05 (2005) 1733-1736

13. Chemical Garden. Website. Retrieved 25 October 2006 from http://www.i-am-ai.net/chemicalgarden/ (2005)

14. Cheok, A., Yang, X., Ying, Z., Billinghurst, M., Kato, H.: Touch-Space: Mixed Reality Game Space Based on Ubiquitous, Tangible, and Social Computing. In: Personal Ubiquitous Computing, Vol. 6, 5-6 (2002) 430-442

15. Crabtree, A., Tandavanitj, N., Steed, A., Benford, S., Rodden, T., Greenhalgh, C., Flintham, M., Anastasi, R., Drozd, A., Adams, M., Row-Farr, J.: Orchestrating a Mixed Reality Game “on the ground”. In: Proceedings of CHI’04, ACM Conference on Human factors in Computing Systems. ACM, New York (2004)


16. Debord, G.: An Introduction to a Critique of Urban Geography, originally published in Belgian surrealist journal Les Levres Nues, no. 6, September 1955, translation by Knabb, [online], http://www.bopsecrets.org/SI/urbgeog.htm, accessed 6 January 2006 (1955)

17. Department of Health: 5 a Week: Physical Activity and Health Outcomes: A Review of the Chief Medical Officer. Department of Health, London (2004)

18. Dourish, P.: Where the Action Is: The Foundations of Embodied Interaction. MIT Press, Cambridge, MA (2001)

19. Gaver, W., Boucher, A., Pennington, S., Walker, B.: Cultural Probes and the Value of Uncertainty. In: Interactions, Vol. 11, 5 (2004) 53-56

20. Headon, R., Curwen, R. (2002) Movement Awareness for Ubiquitous Game Control. In: Personal Ubiquitous Computing, Vol. 6, 5-6 (2002) 407-415

21. Holmes, N.: Pictorial Maps. Herbert Press, London (1992)

22. Höysniemi, J., Aula, A., Auvinen, P., Hännikäinen, J., Hämäläinen, P.: Shadow Boxer: A Physically Interactive Fitness Game. In: Proceedings of the Third Nordic Conference on Human-Computer Interaction, Vol. 82, ACM Press, New York, NY (2004) 389-392

23. Huizinga, J.: Homo Ludens: A Study of the Play-Element in Culture, first published in German in Switzerland in 1944. Routledge & Kegan Paul, London (1949)

24. Hull, R., Clayton, B., Melamed, T.: Rapid Authoring of Mediascapes. In: Proceedings of UbiComp ‘04 (2004) 125-142

25. Jackson, P.: Towards a Cultural Politics of Consumption. In: Bird, J., Curtis, B., Putnam, T., Robertson, G., Tickner, L. (eds.): Mapping the Futures: Local Cultures, Global Change, Routledge, London (1993) 207-228

26. Lane, G., Thelwall, S., Angus, A., Peckett, V., West, N.: Urban Tapestries: Public Authoring, Place and Mobility. Retrieved from http://socialtapestries.net/outcomes/reports/UT_Report_2006.pdf. 28 October 2006.

27. Lankoski, P., Heliö, S., Nummela, J., Lahti, J., Mäyrä, F., Ermi, L.: A Case Study in Pervasive Game Design: The Songs of North. In: Proceedings of the Third Nordic Conference on Human-Computer Interaction , NordiCHI ‘04, Vol. 82. ACM Press, New York, NY (2004) 413-416

28. Lindberg, D.C.: The Beginnings of Western Science: The Western Scientific Tradition in Philosophical, Religious, and Institutional Context, 600BC to AD1450. University of Chicago (1992)

29. Mobiletech News. 26 April 2005. Web Market, Inc. http://www.mobiletechnews.com/info/ 2005/04/26/113110.html (accessed 15 February 2006)

30. Mokka, S., Väätänen, A., Heinilä, J., Välkkynen, P.: Fitness Computer Game With a Bodily User Interface. In: Proceedings of the Second International Conference on Entertainment Computing, ACM International Conference Proceeding Series, Vol. 38. Carnegie Mellon University, Pittsburgh, PA (2003) 1-3

31. Moretti, F.: Atlas of the European Novel, 1800–1900. Verso, London (1998)

32. Muelller, F., Agamanolis, S.: Sports over a Distance. In: Computers in Entertainment. Vol. 3, 3 (2005) 1-11

33. Nakakoji, Y., Yamamoto, Y.: Toward A Taxonomy of Interaction Design Techniques for Externalizing in Creative Work. In: Proceedings of HCII’03, 10th International Conference on Human-Computer Interaction (2003) 1258-1262

34. National Audit Office. Tackling Child Obesity – First Steps. Report by the Comptroller and Auditor General, Her Majesty’s Stationery Office, London (2006)

35. Nold, C.: Bio Mapping, Retrieved 28 October 2006 from http://www.biomapping.net/index.htm (no date)

36. Pickles, J.: Texts, Hermeneutics and Propaganda Maps. In: Barnes, T.J., Duncan, J.S. (eds.): Writing Worlds: Discourse, Text and Metaphor in the Representation of Landscape, Routledge, London (1992) 193-230

37. Raposa, J.: Biofeedback in Educational Entertainment. MA Dissertation. Interaction Design Institute, IVREA. http://people.interaction-ivrea.it/j.raposa/thesis/jan%20raposa%20 -%20masters%20thesis%20(low-res).pdf, accessed 15 February 2006 (2003)

38. Reid, J., Lipson, M., Hyams, J., Shaw, K.: Fancy a Schmink? A Novel Networked Game in a Café. In: Proceedings of the 2004 ACM SIGCHI international Conference on Advances in Computer Entertainment Technology, ACE ‘04, Vol. 74. ACM Press, New York, NY (2004) 18-23

39. Rose, G., Gregson, N., Foord, J., Bowlby, S., Dwyer, C., Holloway, S., Laurie, N., Maddrell, A., Skelton, T.: Introduction. In: Women and Geography Study Group (eds.) Feminist Geographies: Explorations in Diversity and Difference, Addison Wesley Longman, Harlow, UK (1997)

40. Scaife, M., Rogers, Y.: External Cognition: How do Graphical Representations Work? In: International Journal of Human Computer Studies, Vol. 45 (1996) 185-213

41. Schön, D.: The Reflective Practitioner. Jossey-Bass, San Francisco (1983)

42. Shih, E., Bychkovsky, V., Curtis, D., Guttag, J.: Continuous Medical Monitoring Using Wireless Microsensors. In: Proceedings of the 2nd international Conference on Embedded Networked Sensor Systems, SenSys ‘04. ACM Press, New York, NY (2004) 310-310

43. Strömberg, H., Väätänen, A., Räty, V.: A Group Game Played in Interactive Virtual Space: Design and Evaluation. In: Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS ‘02. ACM Press, New York, NY (2002) 56-63

44. Tufte, E.: Beautiful Evidence. Graphics Press, Cheshire, CT (2006)

45. Wild Divine. http://www.wilddivine.com/ (accessed 15 February 2006).

46. Williams, M., Jones, O., Fleuriot, C., Wood, L.: Children and Emerging Wireless Technologies: Investigating the Potential for Spatial Practice. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ‘05. ACM Press, New York, NY (2005) 819-828

mapping inside out

Documents