marketo's secrets to campaign optimization

Marketo’s Secrets to Campaign Optimization

Mike Madden

Sr. Marketing Manager

Marketo

Proprietary & Confidential

Housekeeping

© Marketo, Inc. 10/10/2017 Proprietary & Confidential

• This webinar is being recorded! Slides and recording will be sent to you after the webinar concludes.

• Have a question? Use the chat box and I’ll get to your questions after the webinar.

• Posting to social? Use our hashtag - #mktgnation

• There is a brief survey after the webinar


What is A/B Testing?

1

Testing Methodology

2

The 5 Tests

3

Conclusion

4

Agenda



What is A/B Testing?

A/B testing, also known as split testing, compares two versions of the same campaign on a certain channel. These channels include (but are not limited to):

Website

Social Media

Emails

Mobile Apps

Landing Pages

Digital Ads


Why is A/B Testing Important?

A/B testing increases engagement with your buyers

A/B testing enhances campaign effectiveness and optimizes programs for a company’s

target audience

A/B testing enhances a marketer’s

awareness and expertise about

audience preferences


The A/B Testing WorkflowChoose one element to

test

Ask a question /

write a hypothesis

Decide on the sample

group

Define what

success looks like

Set up the test

Look at the test results

Determine the winner

Implement the new control


Be Sure to Use Statistical Significance Calculators!


1 EMAIL SEND TIME


What is the best time of day to send an email?


The Time Test Set Up

Total Audience

All marketable names, last

engagement date in past 1 year,

marketing job titles

5am (Control)

10am

1pm

Details:

• Split total audience 3 ways

• Backed into numbers to see at least

1,000 opens per test (for sig.)

• Sent each email from a different

campaign to ensure exact timing on

the send

• Tested the time testing over 7

different email campaigns

• Over 200,000 emails sent

• Tested in early, mid, and late buying

stages


The Results

Results by significance:

• 1pm has a 14% higher open rate than 5am (100% statistical significance)

• 1pm has a 5% higher click to open rate than 5am (statistically insignificant)

• 1pm has a 20% higher click through rate than 5am (100% statistical significance)

• 1pm has a 7% higher unsubscribe rate than 5am (statistically insignificant)

1pm gives us 20% higher email engagement

Sent Delivered % Delivered Opened % Opened Clicks CTR CTO Unsub. Unsub %

5am (Control) 70128 68791 98.1% 5983 8.7% 306 0.4% 5.1% 98 0.14%

10am 68034 66763 98.1% 6227 9.3% 342 0.5% 5.5% 112 0.17%

1pm 68067 66754 98.1% 6642 9.9% 356 0.5% 5.4% 102 0.15%


‒ It may take a while to find significance, but keep testing

‒ Make the times different enough to see some results



2 GENERIC VS PERSONALIZED FROM NAME


What the Industry Says About Personalization

Personalized emails deliver 6x higher transaction rates. - Experian

Personalized email messages improve click-through rates by an average of 14% and conversions by 10%. - Aberdeen

When asked to prioritize one capability that will be most important to marketing in the future, 33% of marketers answered: “personalization.” - AdAge


What Was Our Normal Personalization Process

These are tokens in Marketo, which personalize values based on fields in the database.

They give us two options:

1)1) Personalized

• From Name: Lead Owner’s Name

• From Address: Lead Owner’s Email Address

• Reply-to: Lead Owner’s Email Address

2) Default

• From Name: Team Marketo

• From Address: [email protected]

• Reply-to: [email protected]


Is a Personalized Email Really Better?


The Simple Personalization Test

Best regards,

Mike Madden

Sr. Marketing Manager

1.650.376.2300

Best regards,

Team Marketo

Marketo, Inc.

1.650.376.2300

Control

Test

v

s


Testing Details

16 total campaigns

107,840 emails sent

Tests in early, mid, and late stage buying

stages


Where to Introduce Personalization?

Winner: Generic

Results:

• 10% higher open rate

• 26% higher click to open rate

• 38% higher click through rate

100% statistical significance

Early Stage



Mid StageEarly Stage

Winner: Generic

Results:





Winner: Generic

Results:

• 3% lower open rate






Late StageMid StageEarly Stage

Winner: Generic From Name

Results:






Results:

• 3% lower open rate





Results:





Since these numbers aren’t significant and these are late stage leads, we stick with the Personalized From Name.


‒ Introducing personalization in the beginning of a buyer’s journey might be worse for the buyer.

‒ Make the right business decision even if a test says otherwise.

‒ Use this test to find the right handoff point for personalization



3 HTML VS. TEXT-BASED EMAILS


HTML or Text-Based Emails: Which is Better?


Emails Are Like Landing Pages

www.marketo.com Social Links

3 Main Offer Links


Testing Details

5 total campaigns

99,702 emails sent

Tested with different

offers, but held

templates constant


The HTML vs. Text Email A/B Test

vs


Data at First Glance

Over the course of 5 different email drops, we have found text emails to do the following:

• Roughly the same open rate as HTML emails

• 11% higher click to open rates (95% statistical significance)

• 8% higher click through rates (86% statistical significance)


Deeper Dive into Unique Link Performance

Email Name Link Clicks % of Clicks Buyers % of Buyers

HTML Email Twitter Link 16 12.8% 8 9.1%

HTML Email Facebook Link 13 10.4% 5 5.7%

HTML Email Marketo Logo Link 1 0.8% 1 1.1%

HTML Email Offer Link 95 76.0% 74 84.1%

Total 125 100% 88 100%


Final Data

Looking solely at unique offer clicks, the data strongly favors text-based emails…

• Roughly the same open rate as HTML emails

• 21% higher click to open rates (100% statistical significance)

• 17% higher click through rates (99% statistical significance)


‒ Sometimes the pretty emails aren’t the winners

‒ On the bright-side, text-based emails require less creative resources

‒ Test these out across asset types. You may find that HTML emails work better for some programs



4 WEBINAR OPTIMIZATION


We Have 3 Types of Buyers

Behavior Score 0 - 5

Early Stage

Behavior Score 6 - 19

Mid Stage

Behavior Score > 19

Late stage


Those Leads Plotted by Behavior Score

0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000

Early

Mid

Late

Leads Broken Down by Stage


Who registers for our webinars? Who attends them?


An Early Stage Webinar

Abstract:

As Seth Godin put it, why waste a sentence saying nothing? If you want content opened, read, and clicked, it all starts with the subject line. Listen to Mike Madden, Sr. Demand Generation Programs Manager at Marketo, for this on-demand webinar, 3 Hacks to Boost Email Open Rates, to learn three simple and surprising tips to launch your open rates sky high!


After the 1st Email Invite, Here’s What Happened…

5

185

1037

0 200 400 600 800 1000 1200

Early

Mid

Late

Registered Leads Broken Down by Stage


Another Early Stage Webinar

Abstract:

Whether you’re new to social media marketing or a seasoned professional, there are a few big mistakes that you’ve probably made in the past, might be making now, and can avoid making in the future. Learn from Lisa Marcyes, Sr. Social Media Marketing Manager at Marketo, as she discusses the top eight social media marketing mistakes—from strategy, to content, to metrics—and how you can avoid them.


And Yet We See It Again!

155

687

2716

0 500 1000 1500 2000 2500 3000

Early

Mid

Late

Registered Leads Broken Down by Stage


8 Biggest Mistakes Social Attendee List

23

184

889

0 100 200 300 400 500 600 700 800 900 1000

Early

Mid

Late

Attended Leads Broken Down by Stage


8 Biggest Mistakes Social – Attendance Rates

15%

27%

33%

0% 5% 10% 15% 20% 25% 30% 35%

Early

Mid

Late

Attendance Rates Broken Down by Stage


Let’s go back to the 3 Hacks to Boost Open Rates Webinar…


Email Performance Broken Down by Stage

Stage Sent Delivered % Delivered Opens Open Rate Clicks CTR CTO Unsub. % Unsub. Registered LP Conversion

Early 96955 94954 97.9% 11160 11.8% 147 0.2% 1.3% 158 0.2% 1 0.68%

Mid/Late 51969 50546 97.3% 13720 27.1% 1959 3.6% 14.3% 49 0.1% 1208 61.66%

Index Index Index Index Index

100 100 100 100 100

231 2420 1084 53 9065


3 Hacks Webinar – Final Attendee Breakdown

207

726

1745

53

253

544

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Early

Mid

Late

Final Attendee/Registration List by Stage

Attended

Registered

31% attendance

35% attendance

26% attendance


‒ It’s all about investment

‒ Give people what they want, when they want it

‒ Put more focus on early stage



5 UPDATING CTA TREATMENT


Ahh, Email Design Challenges…


Does CTA button treatment affect click to open rates?


Which CTA Button Would You Click?

Control:

• Squared corners

• All uppercase text

Test:

• Rounded corners

• Proper case italicized text

vs


The Test Button Has Spacers for Alignment

Button Attributes:

• Table expands to 100%, not a fixed width

• 5 pixels of padding on top of the button

• 10 pixels of padding within the button to keep text centered


The Test Button Renders More Consistently

And it just looks pretty!


Testing Details

7 total campaigns

154,073 emails sent

Tested with different

offers, but held CTA Buttons constant


The Results

Results by significance:

• New button has a 4% higher click to open (statistically insignificant)

• New button has a 25% lower unsubscribe rate than the control (100% statistical significance)

The rounded button with italicized text and proper spacing reduces our unsubscribe rate by 25%!

Sent Delivered % Delivered Opens % Opened Clicks CTR CTO Unsub. Unsub. %

Control 77227 75171 97.3% 11786 15.7% 1036 1.4% 8.8% 183 0.24%

New Button 76846 74797 97.3% 11744 15.7% 1073 1.4% 9.1% 136 0.18%


‒ Sometimes you test for one thing and solve for another

‒ Use well-built HTML buttons (you can find them online)


Questions?

https://www.linkedin.com/in/michaelmadden824

marketo's secrets to campaign optimization

Technology