Paint With Me: Stimulating

Creativity and Empathy in

Perspective-Taking Virtual RealityLynda Joy Gerry

MA, Cognitive Science and Phenomenology

University of Copenhagen

Paint With Me is a virtual environment

where users see and hear from an

artist’s point of view, following her

movements as she guides them

through her creative imagination.

INTRODUCTIO

N

Project Aims

- Applied Phenomenology: We can use VR to

simulate the modes of sensory, bodily, perceptual,

and cognitive experiences as lived by another

person. Transmitting experiences can be a tool to

cultivate a new form of sharing and

communication.

- Creativity: Evoke wonder for creative imagination

and encourage self-expression in task-

independent creativity.

- Empathy: Test dominant theories of empathy in

direct application.

- Education: Use perspective-sharing for education,

expert-novice skills transmission

Sharing First-Person

Experience?

“Think about it:

There is no experience that you have had

that you were not the absolute center of. The

world as you experience it is there in front of

YOU or behind YOU, on YOUR TV or YOUR

monitor. And so on.

Other people’s thoughts and feelings have to

be communicated to you somehow, but your

own are so immediate, urgent, real.” – David Foster Wallace, “This is Water”

Commencement Speech at Kenyon College (2005)

What is Phenomenology?

Phenomenology is the scientific philosophy

of first-person experience and the structures

of lived experience, crucially focusing on

embodiment, selfhood, agency, and

empathy. Virtual and augmented

environments are the first research tools that

phenomenologists have had to explore and

manipulate the structures of experience.

Let’s use these tools wisely.

Why Painting?

- The ability to transmit something from your mind into a

reality seems to be really central to creativity. Practical

aspects: 20 minutes, hands (fine motor), and the

process needs to be in field of view from first-person.

- 12 interviews with professional abstract painters:

Painting involves letting go of judgments and watching

something emerge and develop. Many are guided by an

initial idea or inspiration, but the rest is a process of

discovery.

THEORETICAL

FRAMEWORK

Embodied Sense of Self:

Body Ownership

Body ownership is

the feeling that the

body I inhabit is my

own, an integral part

of ‘me’, in ways that

other objects and

other people are not

(Tsakaris, 2010).Kilteni, K., Maselli, A., Kording, K. P., & Slater, M. (2015). Over my fake body: Body ownership illusions

for studying the multisensory basis of own-body perception. Frontiers in Human Neuroscience, 9.

Embodied Sense of Self:

Sense of Agency

Sense of Agency is the

feeling of authorship for self-

generated movements and

the external events they

cause (de Vignemont & Fourneret, 2004)

Body Ownership Illusions

and Agency

- Body Ownership Illusions: Paired visual-tactile

stimulation (Rubber Hand Illusion) or paired visual-

motor stimulation

- Tracked and rendered full-body motion in fully-

immersive virtual environments creates sense of

Agency

- Bottom Line: Under specific multisensory

conditions, we can experience artificial body parts,

fake bodies, or virtual bodies as our own body parts

or whole body.

Avatar Effects

Peck, T. C., Seinfeld, S., Aglioti, S. M., & Slater, M. (2013). Putting yourself in the skin of a black

avatar reduces implicit racial bias. Consciousness and Cognition, 22(3), 779-787.

doi:10.1016/j.concog.2013.04.016

The Machine to Be Another:

An Empathy Machine?

“The machine to be another is

designed to stimulate empathy

through embodied interaction

between individuals.”

(Bertrand et al., 2014)

Theories of Empathy

- Theory of Mind (ToM) –empathy involves perspective-taking (Gopnik and

Wellman)

- Simulation Theory –empathy involves feeling what another person feels

(Vignemont & Singer)

- Kinesthetic Empathy –

empathy involves motor resonance (Sheets-Johnstone)

- Narrative Practice (Hutto and Gallagher) – stories

are the ways we come to situate and understand our own

and one another’s experiences

19

Gallagher et al. (2015)

Neurophenomenology

of Awe and Wonder

20

Promoting Helping

Behavior in VR

Ahn, S. J., Le, A. M. T., & Bailenson, J. (2013). The effect of embodied experiences on self-other

merging, attitude, and helping behavior. Media Psychology, 16(1), 7-38.

The Painter Project:

Stereo Footage with Binaural Audio

Go-Pro Hero 4 Bicycle

Helmet Camera Mount

(Laser printed at FabLab

RUC)

Ebba Chambert

(Painter) wearing

camera mount

The Painter Project:

Cognitive Perspective Taking

Participant/User

Painting

What the user sees

inside the HMD

METHODS

Research Questions

RQ 1: What is the relationship

between agency and body ownership?

- Ownership: “I felt like the painter’s hand was my own hand”

measured ownership on 7-Point Likert Scale.

- Agency: “I felt like my movements were indistinguishable

from the movements of the painter” measured agency on 7-

Point Likert Scale.

Results:

Phase 1: Users did not feel an Ownership illusion, but they did

experience an Agency illusion.

Phase 2: Low ownership and high agency in EE condition, low

ownership and agency in Screen condition

Research Questions

RQ2: Can an agency disruption lead

to a greater sense of self-other

merging and empathic accuracy?

RQ3: Do users feel more empathic

towards the painter?

RQ 4: Do users feel more creative?

Dependent Measures:

Interpersonal Reactivity Index

4 Subscales: Empathic Concern, Perspective

Taking, Fantasy, and Personal Distress

Aron, A., Aron, E.N., & Smollan, D. (1992). Inclusion of Other in the Self:

Scale and the structure of interpersonal closeness. Journal of Personality

and Social Psychology, 64(4). 596-612.

Dependent Measures:

Self-Other Merging

Dependent Measure: Empathic

Accuracy (Ickes, 1995)

H1: Agency, Self-Other (S-O)

Merging, and Empathic Concern (EC)

will be highly correlated with Empathic

Accuracy (EA).

H2: Agency, S-O Merging, EC and EA

will be higher in Embodied Experience

(EE) vs. Screen Condition.

29

30

31

32

In The Republic, Plato asks, "Will we

say, of a painter, that he makes

something?" and answers, "Certainly

not, he merely imitates."

33

Imitation, Not Creation

34

“Imitated” Paintings by

Subjects

RESULTS

Dependent Measure: Empathic

Accuracy (Ickes, 1995)

Findings Summary

- Higher Perspective Taking and Empathic Concern subscales of

the IRI were correlated with higher Empathic Accuracy scores in

both conditions. There was a statistically significant correlation

with Empathic Accuracy and Self-Other Merging (r = .571, p <

.05) scores in the Embodied Experience (EE) condition but not

the Screen condition.

- The Personal Distress (PD) subscale of the IRI was significantly

negatively correlated with Empathic Accuracy (r = -.563, p =

.036).

- Participants in the Embodied Experience (EE) condition had

statistically significantly (p < .05) higher scores on Self-Other

Merging, Empathic Accuracy, Presence, and Agency than the

Screen condition, indicating that the main effect of the

experimental condition is significant.

Subject Quote

““It almost feels like collectively

discovering something that was in her

head all along. The experience allowed

this completely different way of

connecting to someone that doesn’t rely

on facial expressions or anything.” (S4)

Subject Quotes

- “I think I got to experience the process of her painting.

Whereas before if you look at the painting it gives off a

feeling, but this journey lets you understand why she

painted it a certain way because she was explaining the

process…you get the artist’s own meaning and feelings.

It’s interesting too because since I am not a professional

painter, it helps to understand what a painter thinks or

goes through while he or she paints.” (S1)

- “I didn’t notice the passage of time at all. It was quite

trancelike.” (S2)

- “I was like making my own version of her thoughts in her

painting, I think.” (S3)

DISCUSSION

In comparison to watching the same

video on a 2-dimensional screen, a

between-subjects design indicates that

the VR perspective taking experience

allowed users to feel closer to the artist

(greater self-other merging), users had

higher empathic accuracy scores, and

that they reported having an easier

time following her movements.

Preliminary Results: Phase 2

• Not Just Replicating

Moving Together

• Beyond Technique

“Soft Skills” of Creativity

Learning an artistic skill can be a

stressful process; Instead, this is

a meditative experience that

involves an intimate connection to

an artist.

FUTURE

DIRECTIONS

Eye Tracking and

Foveated Imaging

Measuring Creativity

- Task-Independent: Analogy Finding

Task (Green, Cohen, et al. 2012; Green et al. 2015)

- Task-Dependent: How subject paints

after the “instruction”

47

The Machine to Be Another:Live Stream Perspective Sharing

Performer

(Wearing Camera)

User (Wearing Head-

Mounted Display)

48

The Machine to Be Another:

Bodyswap

References

Aron, A., Aron, E.N., & Smollan, D. (1992). Inclusion of Other in the Self: Scale and the structure of

interpersonal closeness. Journal of Personality and Social Psychology, 64(4). 596-612.

Banakou, D., Groten, R., & Slater, M. (2013). Illusory ownership of a virtual child body causes overestimation of

object sizes and implicit attitude changes. Proceedings of the National Academy of Sciences, 110(31), 12846-

12851. doi:10.1073/pnas.1306779110

Peck, T. C., Seinfeld, S., Aglioti, S. M., & Slater, M. (2013). Putting yourself in the skin of a black avatar

reduces implicit racial bias. Consciousness and Cognition, 22(3), 779-787. doi:10.1016/j.concog.2013.04.016

Bertrand, P., Gonzalez-Franco, D., Cherene, C., & Pointeau, A. (2015). 'The Machine to Be Another':

Embodiment performance to promote empathy among individuals. Creative Commons Attribution-

NonCommercial-ShareAlike CC BY-NC-SA.

Groom, V., Bailenson, J. N., & Nass, C. (2009). The influence of racial embodiment on racial bias in immersive

virtual environments. Social Influence, 4(3), 231-248. doi:10.1080/15534510802643750

Ickes, W. (1993). Empathic Accuracy. Journal of Personality, 61(4), 587-610. doi:10.1111/j.1467-

6494.1993.tb00783

Kilteni, K., Maselli, A., Kording, K. P., & Slater, M. (2015). Over my fake body: Body ownership illusions for

studying the multisensory basis of own-body perception. Frontiers in Human Neuroscience, 9.

Petkova, V. I., & Ehrsson, H. H. (2008). If I Were You: Perceptual Illusion of Body Swapping. PLoS ONE, 3(12).

doi:10.1371/journal.pone.0003832

Thanks for your attention.

Project Collaborators

Questions?

lynda@whatisexperience.com

www.paintwithmevr.com

Download the Mobile App:

lyndajoygerry.com/painter.apk