mindtouch: embodied mobile media ephemeral transference

8/13/2019 MINDtouch: Embodied Mobile Media Ephemeral Transference

http://slidepdf.com/reader/full/mindtouch-embodied-mobile-media-ephemeral-transference 1/5

MIND TOUCH : MOBILE MEDIA R ESEARCH

With the MINDtouch project I sought to explore embodied,non-linguistic [1] interaction through the use of wearable,biosensing devices and mobile phones. These served as the“interfaces” to remotely connected media experiences withinperformative social events. My aim was to investigate howpeople “connect to each other” through technology, in un-conventional (non-verbal/non-textual) ways, in a simulationof synesthetic dream exchange. With this project I sought to

understand how bodily sensations, perceptions and responsesmight be meaningfully utilized to find unique ways to visualizebody/mind activity and to experience that activity in a col-laborative performance environment. I propose the mobile videophone as a means to communicate non-verbally, visuallyand sensually across distance. In this project I explored no-tions of ephemeral transference, distance collaboration andparticipant-as-performer as means to study the experience of“presence” and “liveness” through mobile technologies withinreal-time performance contexts.

In this performative research project I developed a series oflive, iterative and in-person participatory social events usingmobile technologies. The five performative social events tookplace from July 2009 to spring 2010. These “scratch” or per-

formance experiments involved improvisation while generat-ing live, collaborative visualizations. Participants were guidedthrough specific activities developed to intensify their embod-ied interaction and engagement. Mobile video clips (Figs 1and 2) were triggered and mixed by biosensors receiving datafrom participants’ bodies. The participation by in-person andremote interactors was combined by a daisy chain of technolo-gies through the network space. The various aspects were woven together by computer and smartphone code that enabledthe phones and sensors to communicate with one another inas close to real time as possible. Using live body data (breath,muscle activity, temperature and galvanic skin response), con-

current streams of mobile video were mixed into a multi-threaded,non-linguistic, collaborative, visualdialogue that enabled an embod-ied, meaningful and personalizedexchange between remote partici-pant groups.

THEORY IN PRACTICE

Conceptually MINDtouch was aboutconnection : connecting ideas, expe-riences, body and mind, technologyand art with contrasting philosophi-cal and theoretical concepts. It was about connecting databases, using digital networks andstreaming video to create participatory performance withmobile phones and body data. Most of all, it was about con-necting people with each other in intriguing ways. Notions ofpresence, liveness and embodiment were connected and inter- woven concepts, as each were about sensing others—whetherin the room or across vast distance—in near-to-real-time [2].My primary concern was with sensing at a distance and how itcould be done with our mobile technologies to exchange morefamiliar experiences, such as vivid dreams, preconsciousnessand emotive or visceral sensations, which are hard to expressor share other than in their true experiential form. Three do-mains were manifested in the project: the digital, the body andephemeral consciousness. Philosophical threads of extending

© 2013 ISAST doi:10.1162/LEON_a_00560 LEONARDO, Vol. 46, No. 3, pp. 221–224, 2013 221

A R T I S T ’ S N O T E

MINDtouch: Embodied MobileMedia Ephemeral Transference

Camille C. Baker

Camille C. Baker (lecturer), Brunel University, Uxbridge, Middlesex, UB8 3PH, U.K.E-mail: <[email protected]>. URL: <www.swampgirl67.net>, <www.swampgirl67.net/mindtouch_perf.html>.

This article is based on Camille Baker’s Ph.D. thesis “MINDtouch —Ephemeral Trans-ference: ‘Liveness’ in Networked Performance with Mobile Devices,” University ofEast London, SMARTlab Digital Media Institute, December 2010. The abstract for thethesis received the highest ranking in 2011 by the Leonardo Abstracts Service (LABS).Those wishing to submit abstracts to LABS can find thesis abstract submission forms at<leonardolabs.pomona.edu> (for English-language abstracts) and <www.uoc.edu/artnodes/leonardolabs> (for Spanish).

See <www.mitpressjournals.org/toc/leon/46/3> for supplemental files associated withthis issue.

A B S T R A C T

This article reviews discover-

ies that emerged from the

author’s MINDtouch media

research project, in which a

mobile device was repurposed

for visual and non-verbal com-

munication through gesturaland visual mobile expressivity.

The work revealed new insights

from emerging mobile media

and participatory performance

practices. The author contex-

tualizes her media research on

mobile video and networked

performance alongside relevant

discourse on presence and the

embodiment of technology.

From the research, an intimate,

phenomenological and visual

form of mobile expression

has emerged. This form has

reconfigured the communication

device from voice and text/SMS

only to a visual and synestheticmode for deeper expression.

Fig. 1. Still from par-ticipant video madeduring the July 2009

first performance.(© Camille Baker)



222 Baker, MINDtouch

consciousness and embodying technol-ogy through the network connectedthese notions. They were demonstratedthrough practical exploration and play-ful activities throughout the duration ofthe project. I felt it was critical to find ways to simulate, emulate and facilitateconnections between participants, aid-ing them to sense presence, co-presence,collaboration and/or liveness of othersduring the mobile social events. My aim was to observe and understand how or whether people could connect to eachother by sending the other’s presencethrough mobile video over the networkas a creative visual simulation of an ab-stract dream exchange.

Over the last 10 years or more, peoplehave augmented or maintained emo-

tional connections over distance through virtual presence (i.e. Skype, SMS, textmessaging, etc.). We experience thisconnection inside ourselves, as the restof the world is experienced—throughour bodies and sensations. Thus, weexperience sensations when we are con-nected to others emotionally, whetherthey are in the room, on the phone oracross the world—we “feel” people in ourbodies since we feel these intense sensa-tions through regular contact with them.For MINDtouch participants, wearing thebiofeedback sensors during live events

was meant to induce and capture sensa-tions of connection and “feltness.” Thesebiosensing devices were to connect par-ticipants in a more intriguing embodiedmodality, making the live connectioninto a more physicalized connection.They were to sense each other by send-ing their body data through the networkto “touch” one another and, throughthe video expressions of their body sen-sations, “breathe” life into the mobile video. This created a feedback loop of

body-to-body connection, with a digitaltranslation of personal experience. Bodydata and bodily activity were processedthrough custom software and then usedto trigger the video mixing. The wirelesssensors were the “interface” that trans-lated the embodied data into the non-linguistic visual expressivity of the re-mixed video. Mobile networks acted asthe invisible threads connecting peopleusing this new visual exchange. Thus,sensation was transformed, transducedor translated into a digital video to allowparticipants to “touch” and “play” withothers, to locally and remotely interpretconscious and synesthetic experiences ofanother’s internal world using the exter-nal environment and mobile video. Yetinstead of a verbal or gestural language,

the immediate surroundings of thebody, the architecture and the landscape within the frame became personal expe-rience enfolded into the visual languageof a mobile video collage.

The notion of liveness here includesthe sense of someone or something be-ing “in-person,” physically present, in

the here-and-now, being or taking placein the moment, in front of an audience(as in a conventional performance) orremotely (such as a live TV broadcast oronline). Liveness is intertwined with timeand/or space and physicality simultane-ously, usually with someone witnessingit. Yet for MINDtouch it was not a prior-ity that it be experienced in the samephysical locale as the event itself, or “inthe flesh.” However, the event had to beexperienced as taking place now and wit-

nessed now , in real time or within secondsof the event. Real-time streaming video was the primary mode of expression, en-gagement and embodiment, used to en-able liveness and presence to be sensedduring the mobile social events, as well asto externalize the internal sensations ofparticipants within live, social and medi-ated environments.

The interpretation of embodimentI used was that the mind has an elasticability to move beyond the boundaries ofthe skin, which enables someone to sendtheir presence across a distance. Thiselasticity is both inside and out, crossingthe boundaries of the senses [3], so thatthe senses function separately from eachother, entirely outside of consciousness(and outside of one’s awareness), butnot detached entirely from the body.The influential argument by philoso-pher Don Ihde [4] holds that technol-ogy and the body meet and interface toextend beyond the skin and enhance ourengagement with technology in the pro-cess. Paul Dourish [5] has argued thatpeople can embody the technology theyuse within certain contexts and in variousmanifestations. He focuses on the ideathat ephemeral “beingness” or (tele)presence of being, without a physicalskin encasement, manifests when usersengage in certain activities such as usinga flight simulator for pilot training or in videogaming [6]. Richardson [7] posited

Fig. 2. Still from par-ticipant video madeduring the July 2009first performance.(© Camille Baker)

Fig. 3. Screenshot from the custom biofeedback visualization software made in QuartzComposer for mixing mobile video, 2008. (© Camille Baker)



Baker, MINDtouch 223

that a form of non-corporeality and pres-ence exists within virtual contexts. Shesuggested that this form arises during invisible exchanges of sentiment, such asduring online chat and text messaging[8], through a digital mode of projec-tion, or by sending consciousness usingtechnology as the conduit. From thesetheories, it could be said that embodi-ment is (1) being in the body and themind/body integration and (2) being inthe body, but also sensing and extendingthe mind beyond the skin, during dreamsor in altered states, virtual technologicalspace or other modes of consciousness[9]. These ideas all informed my re-search process as well as the explorationof technological embodiment during themobile media events.

I used participatory performance inMINDtouch as a way to enable people toexplore embodying technology and send-ing their own sensations over distance.It also served as a way to (re-)engagepeople with the physical world using themedia tool. Boal’s Theatre of the Oppressed [10] became my essential guide to usingtheater games, facilitating engagementand clarifying ways to motivate partici-pants. His techniques provided a criti-cal framework that guided me as I askedparticipants to connect with their bodies,to tune into themselves and to interprettheir own perceptions and sensations forcreative visual expression.

My interest in non-verbal, visual com-munication was piqued when I wasfacilitating activities during the videocollection workshops and observed

participants repurposing the phones.Surprising methods of communicatingthrough gestural imagery emerged whenparticipants were asked to express theirperceptions using their videophones.They used movement and physicality when exploring ways to represent theirinternal experiences, exploiting the

external space and its visual elements.The size and portability of the mobilephone easily enabled this behavior. Us-ing the videophones and imagery tospeak for them, participants drew uponthe visual material in their environmentto create their video expressions. Those with prior knowledge or experience of visual or cinematic language found waysto create a visual meaning from the rawenvironmental materials, constructingcommunicable video “sentences” or “ut-

terances” [11]. These activities resultedin a novel mode of non-verbal expressionand emotive interchange—more thanany professional, staged performance or video production could have realized.

Mobile images are often experiencedas personal, intimate [12] and private.Once they are sent, their immediacy feelslike blowing a kiss to another or sharingpieces of yourself through the network.Thus, if a text message is like thoughttransfer, then an image or video is likesending your sight, visual experiences,thoughts and feelings, as well as yourunique expression, perception and per-spective. It is a more intimate, phenom-enological and everyday-life document, which the ordinary person can relate to.Hence, it feels more “real” (authentic), asit is imbued with memory and emotion.Owing to its lower-quality image, mobile video encourages a more personal, non-expert, “every-person,” frank expression,not possible with professional-quality HDor 3D video image. The advanced imageand video capabilities of the latest mo-bile phones inspired participants to be

more creative and playful, while enablinga level of engagement and mobility notpossible with a desktop computer. I val-ued the mobile phone in this project forits innate encouragement of spontaneity,its close relationship to the body and itscompelling ability to go anywhere withpeople. It seemed a natural vehicle toengage people in performative and cre-ative activities—to use their most closelyheld and fetishized technological objectsto play, collaborate and sense each otherover distance.

Mobile media phones for MINDtouch

acted (1) as a conduit for non-literalor abstracted, non-verbal expression ofexperience and as an extension of thebody/mind for participants and (2) asa vehicle to express inner sensations be-tween participants. It was not about themapping dimension of mobile devicesbut instead about the fascination with theembodied possibilities of mobile mediaor the “situatedness” [13] of a personand their experience, as well as their at-tentiveness to themselves and the space

or locativeness within their own bodies.This sometimes involved the participants“mapping” their body parts or creatingabstracted “mappings” of their internalthoughts, emotions or physical sensa-tions (Fig. 3).

The final mobile media “perfor-mances” manifested as events that uti-lized the database of archived video clipsfrom the previous workshops. As this pri-mary series of mobile media social events were developed and iterated, changes were made for the next event based uponparticipants’ feedback from previousevents. Some participants, wearing bio-sensors embedded in custom garments,“performed” or “VJed” the visuals, creat-ing unique “mixes” with their body data.During the live events, the customizedmobile software went through a series ofinteractions, triggering the video clipsfrom the server (see Fig. 3); then these

modified visuals were streamed backto the remote and local audiences and viewed as a collaborative, nonlinear mon-tage or “remix” installation. In the livesetting, video mixes were displayed on anLCD monitor or projection for the localgroup and streamed for remote partici-pants to view online or on their phones.The final live collage (Figs 4–6) could beseen as converging distinct technologiesand practices for a collaborative visualperformance and global mobile-cast, ora conversation between the participants’bodies, the mobile video makers and the

remote audience.

CONCLUSIONS

In the MINDtouch project I was able tobring together diverging areas of newmedia research and media art/perfor-mance practices. MINDtouch providednew methods for working with wear-able devices and mobile phones, usingparticipatory activities and video in per-formance. New processes emerged that

Fig. 4. Still image from the live mixedcollage, output for the custom mixing

software, 2009. (© Camille Baker)

Fig. 5. Still image from the live mixed collage,

output for the custom mixing software, 2009.(© Camille Baker)



224 Baker, MINDtouch

helped people to tune into their bodiesand to translate their sensations andperceptions visually and playfully usingtheir mobiles devices. During the processa new mobile media screen aesthetic wasrevealed. The project changed people’srelationship to mobile devices to enablea new non-verbal interaction.

The research also revealed methods to

embody technology and transmit pres-ence and emotion remotely to loved onesand friends, through a new mobile visualmodality. I discovered that this extensionof presence can be done consciously andthat directed emotional, interpersonalconnections can be transferred throughmobile devices when facilitated by cer-tain activities. With intent and desire toconnect, participants demonstrated thatthey could send and transform their pres-ence through the device, as one doesthrough Internet engagement. Thus,it became clear that we can embody or

send our presence over distance throughour mobile technologies, beyond the typ-ical voice and text modalities.

The project was a success because itachieved the goal of weaving together di-

verse technologies, practices and ideas toconnect remote groups and enable themto find new ways to engage with eachother creatively, affectively, expressivelyand visually. The project also revealednew ways to think about these concepts within the digital media context.

References and Notes

Unedited references as provided by the author.

1. This means there was no sound or speech to be

heard in the videos, but more importantly, partici-pants were guided to create the mobile videos with-out speaking at all.

2. Defining “live” and “real time” is complex, but forthis article “real-time” can be defined as the actualtime that an event is unfolding—as it is happening,or as close as possible, given network complexities.

3. Johansson explains Serres’s theory of “The FiveSenses: Philosophy of Mingled Bodies,” which saysthat tactility is “the most fundamental sense of thehuman soul and its experience of itself and its environment” (Johansson, T.D. “The Tactile Image: A Veronican Approach to Pictoriality in Michel Serres’s Aesthetics of the Senses,” Sense and Senses in Aesthet- ics, Backstrom, P. and Johansson, T.D. [eds.] NSUPress: Helsinki, Finland, 2003, p. 143, [paraphrasingSerres]), and that this can be extended to how we ex-perience images. He states that tactility encompassesall senses and that photography and cinematographyare physical impressions of the referential world or a“true image of reality” presented on a “membrane”or skin (p. 144). Thus, “a tactile image being a me-dium of (physical) impressions and emotions, of im-pressions of the physical world and their implicationsfor sensation” (pp. 144-–145).

4. Ihde, D. Bodies in Technology, Electronic Mediationsseries, Vol. 5, University of Minnesota Press: Minne-apolis, Minnesota, 2002.

5. Dourish, P. Where the Action Is: The Foundations of Embodied Interaction, The MIT Press: Cambridge, Mas-sachusetts, 2001.

6. Dourish [5].

7. Richardson, I. “Mobile Technosoma: Some Phe-nomenological Reflections on Iterant Media De-

vices,” Fibreculture Journal 2005, Issue 6: Mobility, NewSocial Intensities and the Coordinates of DigitalNetworks, 2005 [online] Available at <http://six.fibreculturejournal.org/fcj-032-mobile-technosoma-some-phenomenological-reflections-on-itinerant-media-devices/> (accessed 10 June 2007).

8. Richardson [7].

9. Here “virtual” is in the digital and online context.

10. Boal, A. Theatre of the Oppressed, Routledge: Lon-don and New York, 1992.

11. Johansson [3].

12. The term “intimacy” is used here in a genericsense, meaning closeness, private, personal, familiar,in relationship to or on the body, or between twobodies/people in relationship.

13. In Don Ihde’s sense that “to be situated entailsthat the knower is always embodied , located, is a body ”;Ihde [4] p. 68.

General Bibliography

Auslander, P. Liveness: Performance in a Mediatized Cul- ture . Routledge: London and New York: 1999.

Baars, B.J. “In the Theatre of Consciousness: Global Workspace Theory, A Rigorous Scientific Theory ofConsciousness,” Journal of Consciousness Studies, Im-print Academic: Exeter, U.K.; 1997, Volume 4(4),pp. 292–309.

Christensen, M.S. As We May Feel: Interpreting the Cul- ture of Emerging Personal Affective Mobile Media , Ph.D.Dissertation, Denmark: Department of InnovativeCommunication at the IT University of Copenha-gen, 2006.

Manovich, L. The Language of New Media , Cambridge,Massachusetts: MIT Press, 2001.

Massumi, B. Parables for the Virtual: Movement, Affect,Sensation , Duke University Press: Durham and Lon-don, 2002.

Sheldrake, R. (2003), The Sense of Being Stared At:And Other Aspects of the Extended Mind, New York, NY:Crown Publishers.

Manuscript received 12 September 2011.

Camille Baker is a media artist-performer/ researcher/curator working within variousart forms: mobile media, participatory perfor-

mance, electronic and e-fashion, interactiveinstallation, live cinema, responsive environments, telematics, video art, and music performance. She has presented at workshops, festivals and conferences around the world.

Fig. 6. Still image from the live mixedcollage, output for the custom mixingsoftware, 2009. (© Camille Baker)



Publish in Leonardo!

(© Chris Ewels)

(© Elisabeth Weissensteiner)

(© Danqing Shi)

Environment 2.0: Through Cracks in the Pavement

Guest Editor: Drew Hemment

In urban environments we are separated from the consequences of our ac-tions as surely as the tarmac of the road cuts us off from the earth beneath.But between the cracks in the pavement, another world flourishes—local ac-tivism, recycling, environmental collectives, permaculture, urban gardening.Leonardo solicits texts that document the works of artists, researchers, andscholars involved in the exploration of sustainability in urban environments.

Full call for papers: <http://leonardo.info/isast/journal/calls/environment-2.0_call.html>Author guidelines: <http://leonardo.info/isast/journal/editorial/edguides.html>Submissions: <[email protected]>

Art and Atoms

Guest Editor: Tami I. Spector

The modern world of chemistry is vast and its connection to art strong.From nanocars and extraterrestrial materials to DNA origami and biofuels,chemistry—like art—expresses its transformative, material essence. Chem-istry’s unique connection to art is the focus of this special section.

Full call for papers: <http://leonardo.info/isast/journal/calls/artandatoms.html>Author guidelines: <http://leonardo.info/isast/journal/editorial/edguides.html>Submissions: <[email protected]>

Re-Imagining the Moon

Guest Editor: Sundar Sarukkai The moon has profoundly influenced the human imagination over the cen-turies, in the domains of myths, religion, art and science. This special sectionaims to publish articles from a variety of disciplines exploring various socialand cultural aspects related to the moon as well as those that engage with therelation between the moon and the artistic and scientific imaginations.

Full call for papers: <http://leonardo.info/isast/journal/calls/moon-call_2008.html>Author guidelines: <http://leonardo.info/isast/journal/editorial/edguides.html>Submissions: <[email protected]>

(© Betsy Damon)

ArtScience: The Essential ConnectionGuest Editor: Robert Root-Bernstein

What is the value of artistic practices, techniques, inventions, aesthetics andknowledge for the working scientist? What is the value of scientific practices,techniques, inventions, aesthetics and knowledge for the artist? When doesart become science and science, art? Or are these categories useless at theirboundaries and intersections? Artists, scientists, artist-scientists and research-ers of all sorts are invited to explore such questions in the pages of Leonardo.

Full call for papers: <http://leonardo.info/isast/journal/calls/artsciencecall.html>Author guidelines: <http://leonardo.info/isast/journal/editorial/edguides.html>Submissions: <[email protected]>

Leonardo seeks articles in the following areas of special interest

mindtouch: embodied mobile media ephemeral transference

Documents