sponsored by: get the complete 23-chart report free at how to use these charts: this research
TRANSCRIPT
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
How to Use These Charts: This research is copyright protected. Our goal is to encourage the global growth of testing. Therefore, you may use the individual charts included in this zip file to: • Create presentations for public or internal viewing• Post online via blogs, social media or on your site• Distribute the charts via email, print-outs, or other media to your contacts and
colleagues.
You may not: • Excerpt or alter the charts in any manner (aside from overall size) including but not
limited to removing logos and copyright information. • Sell the data contained herein – or derivative works based in part or whole on it.
For further permission and copyright information, [email protected]
Background:• State of Online Testing research conducted August 28-September 5, 2014. • 841 total respondents.• Published by WhichTestWon, the unbiased resource for people who love testing.
http://WhichTestWon.com• Sponsored by Marketo, a leader in online testing. http://Marketo.com
For a free copy of our 20-page analysis report on this data, go to: StateofOnlineTesting.com
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Percent Of Organizations Testing Their Sites And/Or Landing Pages
Landing Pages Email Mobile Sites Mobile Apps0%
10%
20%
30%
40%
50%
60%
70%
59% 57%
30%
7%
66%60%
39%
11%
What Are Marketers Testing?2013 2014
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo.© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Big Results49%
Moderate Results42%
Neutral Results9%
No0%
Is Web Testing Worth It?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=418© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Online Sales Marketers Lead Gen Marketers Engagement & Brand Awareness Marketers
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
74%
58%
27%
78%
59%
45%
Which Types Of Marketers Are Most Likely To Be Testing?
2013 2014
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo.© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
B2C B2B0%
10%
20%
30%
40%
50%
60%
70%
80%
69%
46%
75%
52%
Who Is Conducting Online Tests?Consumer vs. B2B Marketers
2013 2014
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. © 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Testing
Marketing
Web Analytics
Usability
Web Desig
n & Development
Customer E
xperie
nce IT
Corporate
0%
10%
20%
30%
40%
50%
60%
70%
1%
59%
1%6%
14%3% 1%
14%
4%
66%
11% 8%3% 3% 1% 3%
12%
42%
20%
9% 10%3% 3% 1%
Small Organizations Mid-Sized Organizations Large Organizations
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=381© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Which Department Is In Charge of Managing Testing?
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
USA / Canada Europe Australia / New Zealand Rest of World0%
10%
20%
30%
40%
50%
60%
70%
80%
66% 67%
56%
64%
In What Part of the World Are Marketers Most Likely To Be Testing?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=841© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
More Than 1000 Employees
100-1000 Employees 20-100 Employees Fewer Than 20 Employees
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
64% 66% 65%
46%
78%69% 66%
53%
Which Size Organizations Are Most Likely To Be Testing?
2013 2014
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo.© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Online Sales Marketers Lead Generation Marketers Engagement & Brand Awareness Marketers
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
48%42%
51%
77%
48%43%
Which Types Of Marketers Get The Best Results From Testing?
Percent of Marketers Who Indicated Testing Was "Significantly Worth It"
2013 2014
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo.© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Security Icons
Cart Design Add-To-Cart Elements
Product Page
Template
Site Search Results
Template
Category Tests
Landing Page Tests
Homepage Tests
0%
10%
20%
30%
40%
50%
60%
70%
15%
44%37% 36%
31%28%
40%32%
56%
46%
55% 57%51%
58%51% 54%
29%
11% 8% 7%
18% 14%8%
14%
High Impact Moderate Impact No Impact
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=255© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Which Basic Online Sales Tests Have The Highest Impact?
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
0%
10%
20%
30%
40%
50%
60%
70%
28%21%
48%
32%26%
23%
53%
36%
56%60%
47%
58% 55%62%
42%
56%
16% 19%
6%10%
18% 15%
5%9%
High Impact Moderate Impact No Impact
Which Basic Lead Generation Tests Have The Highest Impact?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=166© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
0%
10%
20%
30%
40%
50%
60%
70%
9%
40%
19%14%
10% 8% 10%
59%
40%
58%
44%
66% 63%59%
32%
20% 23%
42%
24%29% 31%
Which Basic Engagement Marketers’ Tests Have The Highest Impact?
No Impact Moderate Impact High Impact
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=166© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Quality Of Conversions31%
Conversions49%
Immediate Clicks20%
How Far Down The Conversion Funnel Do Marketers Measure Test Results?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=542© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Persona-based Research/Design
Eyetracking-style Heatmaps
Visitor Surveys
Usability Studies
Conversion Quality
Our "Gut"
Best Practices in Design/Copy
Ideas from Other Sites or Case Studies
Web Analytics
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
30%
34%
38%
47%
50%
61%
68%
68%
94%
Which Are the Most-Used Research Tools to Construct Hypotheses?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo N=545© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
A/A Tests
Dynamic Content
No. of Pages in Path, Form or Cart
Multivariate Tests
Concurrent Tests
Segmentation
0% 10% 20% 30% 40% 50% 60% 70%
25%
32%
34%
52%
54%
59%
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo N=471© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Which Sophisticated Testing Practices Are Most Used?
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Only run sequential tests8%
Run both sequential & split tests
46%
Only run split tests46%
Sequential vs. Split Testing
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=540© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
For How Many Weeks Do Marketers Run Tests to Very Low-Traffic Pages
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. © 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
2013 2014
Average 5.4 weeks 4.8 weeks
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
0%
5%
10%
15%
20%
25%
30%
35%
5%8%
29%26% 25%
7%
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=267© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
How Long Do Marketers Run Tests to Very High Traffic Pages?
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Less Than 80% If Test Ran A Long Time
80% Is OK 85% Is OK 90% Is OK 95% Is OK 99% or better0%
10%
20%
30%
40%
50%
60%
70%
3%8%
4%
16%
64%
5%10% 8% 6%
24%
46%
6%
At What Rate of Conclusiveness Do Marketers Consider It Safe To Declare A Winner?
Testing Vendors Staff Marketers
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=537© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
0%
5%
10%
15%
20%
25%
30%
35% 33%
23%
17%
13%
10%
5%
Top 6 Reasons Marketers Aren’t Testing Websites Or Landing Pages … Yet
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=264© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Resource
sTim
e
Culture/P
olitics
Knowledge
Budget
Traffic
Technology
Other0%
5%
10%
15%
20%
25%
30%27%
23%
11% 11%9% 8%
6% 6%
What's The Biggest Hurdle Stopping Your Company Or Division From Doing More Or Better Testing?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=519© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
No One Curre
ntly
Part of a
Staffer's...
1 Full-Tim
e Staffer
2 or More Full-T
i...
7 or More Full-T
i...0%
10%
20%
30%
40%
50%
60%
70%
80%
18%
73%
5% 4% 1%3%
71%
10%14%
3%6%
51%
13%18%
12%
Typical Number Of Testing StaffersSmall Organizations Mid-Sized Organizations Large Organizations
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=413© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
Business Is Booming We're Seeing Moderate Growth
About the Same As Last Year
Business Is Slacking Off0%
10%
20%
30%
40%
50%
60%
24%
49%
24%
3%
33%
54%
12%
1%
Did Testing Agencies And Technology Businesses Grow In The Past 12 Months?
2013 2014
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo.© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.
Sponsored by:
Get the complete 23-chart Report free at www.StateofOnlineTesting.com
0%
5%
10%
15%
20%
25%
30%
35%
40%
9%11%
31%
14%
34%
0%
Will Organizations Hire More Testing Staff Over The Next 12 Months?
Source: State of Online Testing Survey 2014, WhichTestWon & Marketo. N=409© 2014, WhichTestWon, a division of Anne Holland Ventures Inc. All rights reserved.