Testing for Success:

Top tips for unlocking revenue

Presenters:

Mike Norland,

Sr. Marketing Strategist, AdRoll

Hudson Arnold,

Strategy Consultant, Optimizely

Speakers

Mike Norland

Sr. Marketing Strategist

AdRoll

Hudson Arnold

Strategy Consultant

Optimizely

Speakers

▶ Data driven to data informed

▶ Building a testing roadmap

▶ 6 tips for creating a testing culture

▶ Q&A

Agenda

Engaging Cold SDR Leads with Email Nurturing CampaignsThe evolution from “Data Driven” to “Data Informed”

Data Driven Data Informed

Data never lies.

Do what the data says.

This is a messy problem.

Don’t let data oversimplify it.

Engaging Cold SDR Leads with Email Nurturing CampaignsInterpreting Results

Opens, Impressions, Traffic

Clicks, sales qualified leads,

form fills

Account signups, conversions

Spend, product adoption, churn

Surface level metrics

Downstream results

Engaging Cold SDR Leads with Email Nurturing CampaignsThe right metrics to inform your decision

Subject Lines:

A: “Testing for success: Top tips for unlocking revenue”

B: “Soooo…do you like testing things?”

C: “Subject Line: Test B”

Engaging Cold SDR Leads with Email Nurturing CampaignsQualitative feedback is also important

Engaging Cold SDR Leads with Email Nurturing CampaignsCustomer proof vs. simplification

Original

Test

Engaging Cold SDR Leads with Email Nurturing Campaigns

OriginalResults (19k Visitors)

- 2% “Try Us”

+22% Request a demo

(84% CL)

+8% All Conversions

✓

Customer proof vs. simplification

“Customer proof” has a strong impact on our customers and can be applied to other tactics

Customer proof vs. simplification

Engaging Cold SDR Leads with Email Nurturing CampaignsIterate to drive even better results

B. From: Matt Saydah

A. From: AdRoll

+12% Open rate

TEST 1:

B. From: Matt Saydah, AdRoll

A. From: Matt Saydah

+13% Open rate

TEST 2:

Engaging Cold SDR Leads with Email Nurturing CampaignsPoll: Which one performed better?

Original

Test

Engaging Cold SDR Leads with Email Nurturing CampaignsNot every test has a winner...

Original

Test

✓

✓

Results (56k Visitors)

No difference!

(1542 v. 1557 click try

us for free)

Insight:

Sometimes you learn

that a certain element or

positioning doesn’t have

a big impact on

conversion.

OPTIMIZELY SLIDES

Engaging Cold SDR Leads with Email Nurturing CampaignsGetting Started: Orienting On Goals

Engaging Cold SDR Leads with Email Nurturing CampaignsPutting it into practice: Goal Orientation

Revenue:$xxx

Revenue Per Visitor:

$xxx

Visitors:#m

Engaging Cold SDR Leads with Email Nurturing CampaignsPutting it into practice: Goal Orientation

Revenue:$xxx

Revenue Per Visitor:

$xxx

Average Order Value:

$xxx

Conversion Rate:xxx%

Visitors:#m

User acquisition:

#m

User retention

x%

Engaging Cold SDR Leads with Email Nurturing CampaignsGoal Orientation

Revenue:$xxx

Revenue Per Visitor:

$xxx

Average Order Value:

$xxx

Average quantity:

xxx

Average per unit price:

$xxx

Conversion Rate:xxx%

Number of Steps

Visitors:#m

User acquisition:

#m

User retention

x%

Engaging Cold SDR Leads with Email Nurturing CampaignsGoal Driven Testing Themes

AOV

Discounts

Amount of discount

Discount timing

% vs. Dollar off

Campaign coordination

Look & Feel

Banners

Framing

Related products

Search results

Cross Sells

Related Products

Recommendations

Product and Add to Cart

Conversion Rate

Number of Steps

Linking strategy

Search vs. Navigation

Cart optimization

Bounce Rate

Symmetric Messaging

Homepage Balance

Sell vs. Context

Product Searches

Search Algorithm

Filters

Default Sorts

Navigation

Add to Carts

PDP layout

Validation

Imagery

CTA optimization

Reviews

Checkouts

Cart review

Simplicity

Data Entry

Top vs. Side

Filter Titles

Order of Filters

Drop-Down vs. Check vs.

Modal

Mobile Presentation

Engaging Cold SDR Leads with Email Nurturing CampaignsBrainstorming/Hypothesis Generation

“If we move filters to a more prominent location above

search results, users will

a. Be able to review search results with less distraction

b. More easily use filters to find the right product,

leading to more product page views and purchases.”

Engaging Cold SDR Leads with Email Nurturing CampaignsStrategic brainstorming needs strategic prioritization

Effort

Impact

Engaging Cold SDR Leads with Email Nurturing CampaignsRoadmaps: Basic

A place for all of your experiment ideas.

Descriptions & hypotheses

Pages and links to screenshots

Critical test logistics

Impact & effort – essential prioritization criteria

Engaging Cold SDR Leads with Email Nurturing CampaignsRoadmaps: Advanced

Advanced resourcing

Metric and sample size estimation

Advanced prioritization methodology

Engaging Cold SDR Leads with Email Nurturing CampaignsRoadmaps: Advanced Prioritization

Does this test directly

affect KPIs?

Does this test experiment

with brand and company

messaging?

Is this test located on one

of your key flows?

Does this test add

completely new

information, or remove

existing elements?

Does this test remove

steps in a process?

=> What’s the ratio

between estimated effort

and value?

Engaging Cold SDR Leads with Email Nurturing CampaignsRoadmaps: Advanced Reporting

Reporting:

Project timelines and scoping

Load-balancing by experiment theme

Engaging Cold SDR Leads with Email Nurturing CampaignsAll this work…

Top 6 tips to build a testing culture

Engaging Cold SDR Leads with Email Nurturing Campaigns

Source: 2014 Pulse of the Profession® In-Depth Report: Executive Sponsor Engagement — Top Driver of 2014 Pulse of the Profession® In-Depth Report: Executive Sponsor Engagement —

Top Driver of Project & Program Success, http://www.pmi.org/learning/pulse.aspx

Engaging Cold SDR Leads with Email Nurturing CampaignsGet executive sponsorship

Engaging Cold SDR Leads with Email Nurturing CampaignsEngaging Cold SDR Leads with Email Nurturing CampaignsEngaging Cold SDR Leads with Email Nurturing CampaignsBest test ideas can come from anywhere

Tactic to test:

Stakeholders:

Hypothesis:

Business Impact:

N-Size:

Engaging Cold SDR Leads with Email Nurturing CampaignsEngaging Cold SDR Leads with Email Nurturing CampaignsEngaging Cold SDR Leads with Email Nurturing CampaignsAlways be testing, but don’t test everything

Engaging Cold SDR Leads with Email Nurturing CampaignsShare customer insights cross-functionally

Gamify testing

Engaging Cold SDR Leads with Email Nurturing CampaignsStay customer centric

Engaging Cold SDR Leads with Email Nurturing CampaignsThink outside the local maximum

Better Design

Local Maximum

Engaging Cold SDR Leads with Email Nurturing CampaignsNot everything is an optimization problem

● Strategy

● Behaviors

● Markets

● Products

Q&AMike Norland, Sr. Marketing Strategist,

AdRoll

Hudson Arnold, Strategy Consultant,

Optimizely