emotional interaction design (giles colborne)

@gilescolborne

Imagine what it must feel like to grow up in a slum like this.

This kind of building was thrown up all over Britain in the 1950s.

To create this, you’d need to have no sense of the emotions of the people living here.

Welcome to emotional interaction design.

Emotional interaction designGiles Colborne

2http://www.flickr.com/photos/jamesclay/2264414513/

Before we talk about interaction design, we should talk about emotion in design in general.

@gilescolborne

When you look at this picture your stress levels will drop and your ability to concentrate rises (see for example, At Home with Nature: Effects of ‘Greenness’ on Children’s Cognitive Functioning” by Nancy Wells, Environment and Behavior, 2000).

But emotional design is about more than sticking pictures of trees on your website. We respond to far deeper patterns in nature.

@gilescolborne

Like the beautiful arrangement of seeds in this flower.

@gilescolborne

And the same arrangement of leaves in this succulent. A spiral based on the golden ratio.

@gilescolborne

a b

a+ba

ab

=

The golden ratio is the ratio of two lines that fir this equation.

@gilescolborne

a b

You can use these lines to draw a series of squares...

@gilescolborne

Which, in turn, define a spiral. Which is what we saw in those plants. But the Golden Ratio crops up throughout nature.

@gilescolborne

Like... the bones in your hand. The Golden Ratio defines our sense of perfect proportion and beauty.

@gilescolborne

Which is why people have been using it to create beautiful things for centuries. People judge beautiful things to be good, true, honest, simple. It would be a cool idea if someone used this in web design.

@gilescolborne

@gilescolborne

Many of our notions of beauty are hard wired into our brains. Like the baby face effect - people associate child-like features with innocence, honesty, friendliness.

@gilescolborne

And Aaron Walter points out in Emotional Design that he’s used this to good effect throughout his design. Most obviously in MailChimp’s mascot. making boring email newsletter admin seem fun and friendly.

@gilescolborne

Of course we see faces everywhere. But this doesn’t mean ‘design everything to the Golden Ratio’ or ‘put faces on everything’. Rather, it means you must ‘understand the importance of line and form in design’.

Okay, that was 2,500 years of art and mathematics and industrial design in a dozen slides.

@gilescolborne

Hey, good looking –what about interaction?

But this is design as object what about design as interaction?

@gilescolborne

This guy’s interaction with his computer is certainly emotional.

http://www.youtube.com/watch?v=HtTUsOKjWyQ

@gilescolborne

Something about computers brings out the devil in us.

@gilescolbornehttp://www.flickr.com/photos/thomashawk/54164255/

In response, some interaction designers act like Nurse Ratched in One Flew Over the Cuckoo’s Nest. They see emotion as the enemy.

But I think that misunderstands the importance of emotions.

@gilescolborne

AI pioneer Marvin Minsky sees emotions as ‘ways to think’. Emotions tune the machinery of the brain.

Even a ‘negative’ emotion like anger is useful. It gives us the energy to address a threat and sends out social signals that warn others we’d better get our way.

@gilescolborne

People without emotions are less effective thinkers. Antonio Damasio describes one patient, Elliot, with above average IQ but frontal lobe damage that means his emotions are impaired making him a kind of Mr Spock.

When faced with a ‘rational’ task like scheduling an appointment, he endlessly weighs and compares choices. He doesn’t feel boredom, frustration or embarrassment. He takes for ever. Emotions are vital to decision making.

@gilescolborne

This design from HP is overly rational. The focus on numbers and specifications flips us into rational evaluation mode. Can I find a computer that weighs 10 grammes less? Can I find a computer that’s 3 mm thinner? We find ourselves in a rational spiral.

A better approach would be to connect on a gut level - as light as a hardback book would tell most people what they need to know.

@gilescolborne

This classic ad shows a chick being put in a heatproof box... and boiled for what seems like ages before it’s revealed alive and well.

Why make your audience so anxious?

Because we remember stressful events. They’re playing with our emotions to fix the idea of the heatproof box in our memory.

@gilescolborne

I’ve found that when I ask people about delightful experiences, they remember ones associated with stressful events.

Like dropping your iPod –and discovering that it paused automatically when the headphones popped out.

How delightfully clever. How memorable.

So I look for stress points when I design because I know users will remember what my products do next.

@gilescolborne

Why do we like to watch horror movies? Because they help us extend our emotional range and understand our emotions.

Emotions are important to people. They’re linked to our strongest memories and the most meaningful events in our lives.

So we’re right to reject the Nurse Ratched view that we should eliminate emotion.

What if we accept there will be ups and downs in the relationship between humans and computers?

What if we try to give our designs the emotional intelligence to ride those waves?

@gilescolborne

Marvin Minsky wants to create artificial emotional intelligence by building an artificial brain. Awesome!

But that’s a few decades away. Meanwhile, I have a website to design.

So are there ways I can cheat?

@gilescolborne

Back to this guy. The word that describes him is ‘postal’.

So maybe we can learn how to deal with this situation by talking to some experts.

@gilescolborne

FBIThese folks train negotiators to deal with crisis situations. Maybe they can help us.

@gilescolborne

active listening

empathy

rapport

influence

behaviour change

Behavioural Change Stairway Model

active listening

rapport

behaviour change

They use this model to get through hostage negotiations. It recognises that if you want to get to ‘behaviour change’ you need to start with ‘listening’ and ‘empathy’.

@gilescolborne

Clearly his computer isn’t listening to him.

When interfaces don’t listen we get angry.

@gilescolborne

It looks like you’re giving a presentation

Get help with giving the presentation

Just give the presentation without help.

Don’t show this tip again

Clippy has all the elements that we’re supposed to include in emotional design. He’s informal yet direct. He’s cute looking.

I’ve always thought there was a lot of good thinking behind him. But people hated him. Why? Because he’s so bad at listening.

@gilescolborne

Clifford Nass

My hero Clifford Nass redesigned Clippy very simply to listen and empathise.

When Clippy offered advice, he would ask ‘was that useful?’. If people said ‘no’ Clippy would say ‘that really ticks me off. Let’s tell the folks at Microsoft I need to be reprogrammed.’ and Clippy would encourage them to write in and vent their dissatisfaction.

And it worked: users liked Clippy.

@gilescolborne

When you watch this video, you notice that there are lots of signs that things are going wrong. Our guy begins by giving the screen a hard stare. Then he seems to swear. He slaps the keyboard. And then he really starts to lose it.

A more sophisticated computer might have picked up on those warning signs and saved itself a beating.

@gilescolborne

In Affective Computing, Rosalind Pickard suggests that computers could use input from many sensors (facial recognition, audio input and so on) and pattern matching to detect users’ emotions.

She recognises this is complex (even people occasionally misread emotions). We’re a few years away from this.

But Clifford Nass shows us that listening can be as simple as saying ‘how am I doing?’.

@gilescolborne

Wrong:‘I know how you feel’

Right:‘I’ve never been in your situation before, but I imagine you must be feeling very depressed and lonely’

The FBI knows there’s a right way to empathise. Don’t say ‘I know how you feel’. It’s too easy for other person to say ‘Oh no you don’t’.

Instead, show you care and create opportunities for dialogue. You don’t need sophisticated technology for this. In fact, it’s been around since the 1960s.

@gilescolborne

ELIZA

Eliza is a computer ‘therapist’ that asks users how they feel and uses pattern matching to respond and draw them out.

It’s crude, but good enough to pass a basic Turing test - some people think Eliza’s answers come from a real person.

@gilescolborne

Be positive, upbeatReassure hostage-taker that things will work out well

Be credibleShow you understand their reasons but don’t be too eager to please

FBI negotiators know they need to project the right personality.

Now imagine if you hooked up Eliza’s pattern matching to an online database and gave it an upbeat personality with a bit of edge to it.

That sounds familiar...

@gilescolborne

Siri is an evolution of Eliza’s pattern matching approach but with better jokes.

That creates a personality and a basis for empathy.

@gilescolborne

Clifford Nass ran an experiment where he gave participants blue wrist bands and asked them to complete tasks a computer.

For half the participants he put ablue border on the computer screen and said ‘you and the computer arethe blue team’. For the other half, he gave the computer a green borderand said ‘you’re the blue guy working on the green computer’.

When the colours matched, people tried harder and thought the computer was smarter.

Building rapport doesn’t require complex technology. Just good psychology.

@gilescolborne

Listen and empathise

Excellent. We’re half way up the FBI’s behavioural change model and we’ve not had to build an artificial brain.

@gilescolborne

The FBI has a lot to tell us about how to handle emotions that arise from situation.

But sometimes conflict arises from personality differences.

@gilescolborne

For managing relationships, this book was recommended to me.

I love it because it centres on a simple model. (Which we can use when we’re designing interactions.)

@gilescolborne

Task focus

Relationship focus

AggressivePassive

Tank

Think they know it allYes person

It can’t be done

You have to understand people’s disposition (passive - aggressive) and motivation (task - relationship).

@gilescolborne

Task focus

Relationship focus

AggressivePassive

Tank

Think they know it allYes person

Whiner

In the centre is the ‘normal zone’. At the edges of the graph are extreme types who can be difficult to get along with.

@gilescolborne

Task focus

Relationship focus

AggressivePassive

Tank

Think they know it allYes person

Whiner

What I find normal and acceptable will be different from what you find normal and acceptable.

Everyone is someone’s difficult person.

We need to tune our behaviour and responses to get the best out of those conflicts.

@gilescolborne

Task focus

Relationship focus

AggressivePassive

Get it done

Get appreciatedGet along

Get it right

And the secret to that is understanding the intention that drives those personality types.

@gilescolborne

Maja Mataric

Maja Mataric has built socially assistive robots that coach stroke victims through their physiotherapy.

She tried tuning the coaching to each patient’s personality. The robots would tell extroverts ‘Come on, try harder’. But introverts would be told ‘I know it’s hard, but it’s for your own good’.

And... it worked! Patients preferred the robots that were tuned to their personality and tried harder for them.

@gilescolborne

You don’t need to give your users a personality test before they start. You could learn their personality in the same way that Pandora learns your taste in music.

@gilescolborne

And you might even be able to pick up information about users’ personality from specialist services.

@gilescolborne

Task focus

Relationship focus

AggressivePassive

Get it done

Get appreciatedGet along

Get it right

@gilescolborne

If you need to be appreciated, you’ll like the fact that TripAdvisor tells you when people have read your reviews.

If you’re a ‘get it done’ kind of person, you might feel this was unnecessary and spammy.

@gilescolborne

And if you’re the kind of person who needs to get along, you’d appreciate this error message that says ‘It’s my fault’. (Personally, I find it rather craven.)

@gilescolborne

Task focus

Relationship focus

AggressivePassive

Get it done

Get appreciatedGet along

Get it right

So instead of designing fixed patterns of behaviour, maybe we should design flexible patterns that adjust to users’ disposition.

@gilescolborne

active listening

empathy

rapport

influence

behaviour change

Behavioural Change Stairway Model

And we’ve seen how a model like this can help us think about listening, empathising and building rapport, rather than rushing towards outcomes.

@gilescolbornehttp://en.wikipedia.org/wiki/File:Coca_Cola_ad_ca._1943_IMG_3744.JPG

A lot of the discussion of emotional design today centres around ideas of ‘brand’. And on old-fashioned, static, monolithic brands.

But I hope I’ve shown you that we can do much more.

We can create a flow of emotions and more dynamic, adaptable personalities for our designs.

We’re designing responsive web layouts, why not responsive interaction rules?

@gilescolborne

If we put some emotional intelligence into the interactions we design, we can disarm conflict, create richer, more memorable experiences, and improve performance for our users.

The future may bring us better ways of reading emotions.

But the tools we need to get started are already in our hands.

Let’s use them.

@gilescolborne

By the way, Samsung just patented this. So soon your smartphone will be able to use its camera to detect your emotion - one of six ‘universal’ facial expressions that correspond to common emotions.

@gilescolborne

Researchers have identified six or seven universal emotions. But not everyone who sees these photos has the same experience.

@gilescolborne

When you see the fear in others, your amygdala lights up. You experience stress. That empathy stops you from wanting to dominate others by manipulating them.

Psychopaths don’t have this response. They become charming manipulators.

This is the final part of emotional intelligence: empathy. Without it you’re just a manipulator.

@gilescolborne

Which brings us back to this. Tom Cordell tracked down the architects of the 1950s and asked them what they thought they were doing.

@gilescolborne

We were trying to build heaven on earth

Their dream of building tower blocks surrounded by parks was subverted by the money men who used the space to cram in more roads, shops and tower blocks.

We should learn from their lesson.

@gilescolborne

We need to make sure that we, the companies we deal with, and the systems we create feel for the users we’re serving.

If we don’t we’ll end up using these powerful tools to manipulate them.

And we’ll be no better than the architects of 1950s Britain who, like us, thought they were creating a better world.

@gilescolborne

What are emotions and how could we create them in a machine?

Still the go-to text on using emotions in computing.

Applying the triune brain model to the problem of creating emotionally resonant designs.

How law enforcement agencies negotiate in highly charged, criminal situations.

A guide to recognising emotionally charged relationships and to getting the best from them.

A refutation of the idea that emotions have no place in higher thought.

How your mood - and even your facial expressions can affect your experience of a situation.

Applying theories of story to interaction design