A/B & multivariate testing in big companies

@laurencevealewww.iqcontent.com

A/B & Multivariate Testing...

Easy, right?

Wrong!

What’s so hard?

• Complex CMS & code

• Politics

• Risk aversion

• Red tape: Policies & procedures

“Oh Fuck”

New dog, old tricks

What to test

Ad copySite contentEmail Radical design change (ab)Page elements (mvt)

"The headline is the most important element. It is the telegram which decides the reader whether to read the copy"

Buy now

Buy now

Buy now

Buy now

Add to cart

Add to cart

Add to cart

Add to cart

Call to action

What’s around it (context & contrast)

Predominant brand colours

Size, shape

Labels

Buy now

Where to test firstTop entry pages

with high bounce rates. Pages with good traffic and clear

purpose & conversions

Buy now

Biggest loser, most dramatic results

Where to test

Checkout flow

1 23 €

What to test

Expectation - does the page people land on match the promise made?

What to test

Clarity of page purpose visual hierarchy, clarity and purpose of content

What to test

Urgency: are we influencing this?

Buy now only 3 left

What to test: trust

What to test

Distractions

results are only as good as the

hypothesis being tested

(not opinions)

Forming an hypothesis?2

I’m on TwitterFollow me on twitterYou should follow me on twitterYou should follow me on twitter here.

UX

Getting all this donein a big company

Getting all this done

Find allies - motivated & enthusiastic(they’ll help you get stuff done)

Getting all this done

Include everyone who needs to be,

don’t do sneaky AB(this gets buy-in)

Getting all this done

Start small (remember the Biggest Loser)

Getting all this done

Take time on designing the test

(results are only as good as the hypothesis)

Tell a good story

Use the results to start a good story...

(not just about this test, but about testing-driven optimisation)

Give credit

To everyone involved