nux liverpool ux utopia

Sam Barton - Head of User Experience

NUX Liverpool - March 2016

@sam_barton @shopdirectux

@sam_barton | @shopdirectux

@sam_barton | @shopdirectux

Survival- led Business-led Customer-led

Dec ‘13

84%

TRANSFORM A

CATALOGUE BUSINESS

INTO A WORLD CLASS

DIGITAL RETAILER

@sam_barton | @shopdirectux

Digital Transformation

2003-07

Integration and

consolidation

2007-12

Catalogue to

etail capabilities

2012-today

Vision for world

class digital

@sam_barton | @shopdirectux

UX in Shop DirectWays of working

Problem solving Qualitative testing Quantitative testing

@sam_barton | @shopdirectux

Evidence led approach

Through data In our UX lab On site (100/month)

@sam_barton | @shopdirectux

Focus on learning

ExperimentsResearch

suggest hypotheses

evidence and learning

Not a hypothesis

“I think we should add this feature to

the page and measure it.”

@sam_barton | @shopdirectux

Hypothesis

“I predict change will effect metric.”

@sam_barton | @shopdirectux

@sam_barton | @shopdirectux

Rapid experiments

Reduce barriers to experiments to

push volume and innovation.

@sam_barton | @shopdirectux

Fail fast to learn better

Focus on learning through uncertainty.

90% Fail

Nearly All Fail

67% Fail

90% Fail

66% Fail

@sam_barton | @shopdirectux

“Trying new things constantly

and then abandoning them

without further study or work is

not iterating. That’s flailing.”

Laura Klein

@sam_barton | @shopdirectux

Iterate towards redesign

Incrementalism RedesignIteration

@sam_barton | @shopdirectux

Prioritise

based on

evidence

@sam_barton | @shopdirectux

Tools and technologyMix quantitative and qualitative research to focus on learning.

@sam_barton | @shopdirectux

Full UX tool kit

Experimentation

UX Lab/Guerrilla

Remote surveys (Userzoom)

Eye tracking

Session Replay (Use it Better)

Ease Measure

Quant Observation Huge sample

Qual Observation Small sample

Qual/quant Subjective Medium sample

Quant Observation Low sample

Quant Observation Big sample

Quant Observation Big Sample

@sam_barton | @shopdirectux

UX LabBring the customer into the business

@sam_barton | @shopdirectux

Eye tracking

@sam_barton | @shopdirectux

Quant UX research tools

Session Replay Ease Measure Remote testing

@sam_barton | @shopdirectux

AB Testing / Experimentation

AB Testing Analytics

@sam_barton | @shopdirectux

People and skills

UX Research

@sam_barton | @shopdirectux

Broad skills and specialisms

UX QA

UX Design

UX Testing

UX Dev (iDev)

Psychology, human behaviors

Cross device testing, automated testing

UI design, deep understanding of the sites

ABn testing and data analytics

HTML, CSS, advanced JavaScript

UX Research

@sam_barton | @shopdirectux

Team evolution2010

UX QA

UX Design

UX Testing

UX Dev (iDev)

UX Research

@sam_barton | @shopdirectux

Team evolution2010 2016

UX QA

UX Design

UX Testing

UX Dev (iDev)

@sam_barton | @shopdirectux

Broad UX vision

Building a team with broad skill sets,

with specialisms that can hone in on

the details.

@sam_barton | @shopdirectux

Notable experiments

@sam_barton | @shopdirectux

Product page:

Error messaging

£3.3m

2.9% increase in demand,

2.5% in AOV

0.4% in conversion.

@sam_barton | @shopdirectux

Gallery page:

Layout

less people viewing product page

increase in conversion from products viewed.

1.1%

@sam_barton | @shopdirectux

Multi-device test:

Sticky headers

1.4% lift in conversion on mobile

3.1% drop on desktop

@sam_barton | @shopdirectux

Mobile gallery

(0.2)%

1

2

3

@sam_barton | @shopdirectux

Persuasive messaging

2.7% significant lift in conversion.

2.7%

@sam_barton | @shopdirectux

Persuasive messaging - iteration

@sam_barton | @shopdirectux

Persuasive messaging - iteration

@sam_barton | @shopdirectux

Persuasive messaging - iteration

@sam_barton | @shopdirectux

Persuasive messaging - iteration

@sam_barton | @shopdirectux

Persuasive messaging - iteration

@sam_barton | @shopdirectux

What’s next?

@sam_barton | @shopdirectux

What’s next?Experimentation 2.0 Scaled efficient & intelligent experimentation

Connected data sources and visualisation of data

Machine learning technologies

Personalised testing

@sam_barton | @shopdirectux

In summary

• Investment in infrastructure and skills

• Environment to innovate

• Fail fast, learn faster

• The HiPPO is almost extinct!

Which

Test

Won ?

Control Challenger

@sam_barton | @shopdirectux

Gallery page layout Hypothesis accepted

1.1% increase to

conversion

Control Challenger

@sam_barton | @shopdirectux

Managing abandonment Hypothesis rejected

(2.8)% negative to

conversion

Control Challenger

@sam_barton | @shopdirectux

Product add to basket Hypothesis rejected

(1.2)% negative to

conversion

Control Challenger

@sam_barton | @shopdirectux

Collect+ messaging Hypothesis accepted

1.8% increase to

conversion

Control Challenger

@sam_barton | @shopdirectux

Default sort order Hypothesis accepted

0.5% increase to

conversion

Control Challenger

@sam_barton | @shopdirectux

Empty review stars Hypothesis rejected

(0.8)% negative to

conversion

Control Challenger

@sam_barton | @shopdirectux

Cookie messaging Hypothesis accepted

0.5% increase to

conversion

Control Challenger

@sam_barton | @shopdirectux

Enable filtering (latest offers) Hypothesis accepted

1.2% increase to

demand

Control Challenger

@sam_barton | @shopdirectux

Increased font size Hypothesis accepted

0.5% increase to

conversion

@sam_barton | @shopdirectux

Tie break question

Question: How many experiments did Shop Direct run last year?

693

Thank you

@shopdirectux | @sam_barton

shopdirect.com/careers