selling performance - bristol webperf meetup 2017-07-20

Selling Performance @AndyDavies · NCC Group · May 2017

https://www.flickr.com/photos/chrisjohnbeckett/10446312055

Performance often isn’t at the top of people’s concerns• ~~~~~~~~~~~~~~~~~~~~~ • ~~~~~~~~~~~~~~~~~~~~~ • ~~~~~~~~~~~~~~~~~~~~~ • ~~~~~~~~~~~~~~~~~~~~~ • ~~~~~~~~~~~~~~~~~~~~~ • Security• Performance • Accessibility ☹

So what do they care about?

Conversions?

https://www.flickr.com/photos/z0/19814021505

But it’s not just about getting visitors to buy things!

- sign ups / subscriptions- completing tasks- interaction with adverts- sharing things- finding a store / restaurant- reducing costs - hosting / acquisition

Science explains visitor behaviour

Response Time in Man-computer Conversational TransactionsRobert B. Miller, 1968

Instant

Seamless

Yawn!

100ms 1s 10s

How we perceive response times

Response Time in Man-computer Conversational TransactionsRobert B. Miller, 1968

Instant

Seamless

Yawn!

100ms 1s 10s

Median start render time for 600+ UK retailers = 1.68s

How we perceive response times

So yes, our attention span is shorter than a goldfish

https://www.flickr.com/photos/watts_photos/23124797946

https://www.flickr.com/photos/ericparker/5416356195

“Badly performing web sites make us concentrate harder”Foviance

https://www.flickr.com/photos/dhollister/2596483147

“Stress levels comparable to watching a horror movie”Ericsson Mobility Report 2016

Challenge assumptions

Most people don’t have an iPhone

% sales by OS, three months to April ’17 - http://uk.kantar.com/tech/mobile/

0

10

20

30

40

50

60

70

Android iOS Windows Other

Everyone has fast broadband… right?

Average UK connection speed is 16.9Mbps!

Akamai State of the Internet Q1 2017

92% above 4Mbps60% above 10Mbps42% above 15Mbps

Which will load faster?

www.bbc.co.uk/news

10Mbps1Mbps

Which will load faster?

www.bbc.co.uk/news

10Mbps1Mbps

Which will load faster?

www.bbc.co.uk/news

10Mbps1Mbps

Which will load faster?

www.bbc.co.uk/news

10Mbps1Mbps

https://www.flickr.com/photos/joebenjamin/5009411920

Which will load faster?

www.bbc.co.uk/news

10Mbps1Mbps

Case studies demonstrate what’s possible

Wallmart demonstrated speed affects conversion in 2012

Walmart 2012

Cook reduced load times by 1 sec saw 7% increase in conversions

https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/march/how-we-predicted-an-increase-in-conversions-from-a-decrease-in-load-times

http://engineroom.ft.com/2016/04/04/a-faster-ft-com/

FT saw lower engagement when they made the site slower

Case studies are a great place to start

https://wpostats.com

“Case studies don’t represent our demographic, our audience, our

products, our proposition…”

Has some truth

Boun

ce R

ate

0%

25%

50%

75%

100%

Page Load Time (s)0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Site 1Site 2Site 3

Bounce rates across three retailers

Conv

ersio

n Ra

te

0%

2.5%

5%

7.5%

10%

Page Load Time (s)0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Site 1Site 3Site 4

Conversion rates vary between sites too

Noisy as few data points at end of range

So what about real visitor experiences?

We’re used to the idea that conversion rate varies by deviceiPhone 6/7

iPhone 5/5s

iPad

iPhone 6/7 Plus

Galaxy S6

Galaxy S7 Edge

Galaxy S7

Galaxy S6 Edge

Galaxy S5

Galaxy A3

Conversion / Bounce Rate0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Conversion RateBounce Rate

But existing analytics tools are weak when it comes to speed data

But existing analytics tools are weak when it comes to speed data

Sample contains < 4% of

impressions

They’re no substitute for measuring your own visitor’s experience

Performance Analytics aka Real User Monitoring

To understand how performance affects visitor behaviour

Conv

ersio

n Ra

te

0%

1%

2%

3%

4%

5%

6%

Boun

ce R

ate

0%

20%

40%

60%

80%

100%

Page Load Time (s)0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Bounce RateConversion Rate

The impact on key business metrics

Conversion Rate drops 1% over 3

seconds

Conv

ersio

n Ra

te

0%

1%

2%

3%

4%

5%

6%

Boun

ce R

ate

0%

20%

40%

60%

80%

100%

Page Load Time (s)0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Bounce RateConversion Rate

The impact on key business metrics

And modelling the benefits improved performance delivers

Only NCC Group, Soasta and Blue Triangle’s Real User Monitoring products do this

Real User Monitoring is a strong driver of performance improvement

Other complementary approaches too

Filmstrips (and videos) are hugely powerful

https://webpagetest.org

http://heatmap.webperf.tools

Can be sumarised into heat maps

Waterfalls require explanation

(I tend to avoid them with non-technical audiences)

http://requestmap.webperf.tools

But enable fascinating insights

Block third-parties understand how they affect performance

Produce comparison videos / filmstrips to show others

Can also compare against competitors

Can also compare against competitors

Videos prolong the painful moment

Or for fun…

…compare against an F1 pit crew

https://www.youtube.com/watch?v=aHSUp7msCIE

Vs

iPhone 6 on WiFi

https://www.youtube.com/watch?v=aHSUp7msCIE

Vs

iPhone 6 on WiFi

https://www.youtube.com/watch?v=aHSUp7msCIE

Vs

iPhone 6 on WiFi

https://www.youtube.com/watch?v=aHSUp7msCIE

Vs

iPhone 6 on WiFi

Wait… they changed all four wheels?https://www.flickr.com/photos/cellardoorfilms/7620375702

So where next?

Establish which metrics people care about

Experiment- Change the performance of a page- Measure how metrics change

(Same principles as AB / MV testing)

Can build negative experiments too e.g make pages bigger

Measure Visitors Experience- Use the data to predict how improvements will

change the metrics- Improve performance- Verify the result

Even ‘small’ performance improvements can have a significant impact

How much was 150ms improvement worth?(was 300ms on Android)

Share your wins!

https://www.flickr.com/photos/borkazoid/221185817

http://www.flickr.com/photos/auntiep/5024494612

@AndyDavies

andy.davies@nccgroup.trust

http://slideshare.net/andydavies