dmaw digital day - testing mayhem
DESCRIPTION
Sean Powell from The Engage Group and Heather Marsh from ABD Direct spoke about testing at DMAW's Digital DayTRANSCRIPT
![Page 1: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/1.jpg)
Testing MayhemSustainer Pop Ups + New Responsive Design + Hitting
your Budget = WTF?!?!
![Page 2: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/2.jpg)
Intros
Heather Marsh, Director of Digital Marketing, ABD Direct
Sean Powell, Online Fundraising, The Engage Group
![Page 3: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/3.jpg)
Website Testing
The Evolution of Websites◦It’s not just for younger people anymore.◦85% of all Americans are online.◦51% of adults bank online, up 5% since 2010.◦35% of mobile users bank online, up from 18%
in 2011.◦39% of adults took part in a Political
engagement activity during the 2012 election.
*Pew Internet Research
![Page 4: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/4.jpg)
Setting the stage
Setting the stage◦Super fresh tests of designing and testing
responsive rollouts◦Likewise with sustainer pop ups◦These are some of our preliminary results
![Page 5: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/5.jpg)
Website Testing… WTF?!
Three sections today…◦Pop-ups/Lightboxes◦Intercepts◦Responsive Design
![Page 6: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/6.jpg)
Pop-Ups – The Wilderness Society
![Page 7: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/7.jpg)
Popup – The National Campaign
![Page 8: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/8.jpg)
Lightboxes - TWS
![Page 9: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/9.jpg)
Website Testing… WTF?!
Tests we’ve run (that have improved conversions)…◦Test timed delays for opening◦Test targeting messaging by section◦Test both graphic only CTAs versus inline forms
![Page 10: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/10.jpg)
Website Testing… WTF?!
Pop-ups/LightboxesInterceptsResponsive Design
![Page 11: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/11.jpg)
Intercepts – American Cancer Society
![Page 12: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/12.jpg)
Intercepts - TWS
![Page 13: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/13.jpg)
Some things to think about when doing an intercept (sustainer or otherwise)◦Low dollar and high dollar threshold◦For sustaining gifts, what the resulting gift will
be? ◦Where and how will the upgrade be displayed?◦What happens when the user tries to close the
popup?◦Focus on reducing form abandonment (donor
freak out)
![Page 14: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/14.jpg)
Sustainer results
Results◦4% of donors who see it converted◦Saw slight decline in conversion rate
Not statistically significant though
![Page 15: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/15.jpg)
Website Testing… WTF?!
Pop-ups/LightboxesInterceptsResponsive Design
![Page 16: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/16.jpg)
Responsive Design… Why?
28% of Internet usage comes from a mobile phone
86% of adults own a mobile phone47.5 million tablets in the US1 in 3 visits to digital retail websites is
exclusively through mobile devices.
*Pew Research Data; Forbes; comScore
![Page 17: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/17.jpg)
Responsive Design… Why?
47% of email is now opened on a mobile device
44% of email recipients made at least one purchase last year based on a promotional email
*The Art and Science of Email for Nonprofits, Blackbaud; Jay Baer “15 Email statistics that are shaping the future.”; comScore
![Page 18: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/18.jpg)
Responsive Design… Why?
Mobile and tablet usage doubled in a year.
*Monetate Ecommerce Quarterly
![Page 19: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/19.jpg)
Responsive Design… Why?
28% of adults 55-64 own a tablet18% of adults 64+ own a tabletUsage evenly split between gender and
ethnicity.
Why Responsive vs. Mobile?
*Pew Internet Research
![Page 20: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/20.jpg)
Non-Responsive vs. Responsive - TWS
![Page 21: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/21.jpg)
Responsive Design – Defenders of Wildlife
![Page 22: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/22.jpg)
Responsive Design – Defenders
![Page 23: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/23.jpg)
Responsive Design - Defenders
![Page 24: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/24.jpg)
Responsive Rollouts
Some of our results…◦Security seals matter
Without decreased conversions by at least 12%◦Tactile-ish buttons help ( 44px x 44px )
In all designs, users focused on them, they were easy to find
◦Mixed results website vs. email◦ Increased conversions on tablets and mobile
significantly Conversion increased 30+%
◦ Increased sustainer rate
![Page 25: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/25.jpg)
Other Types of Tests
Tests we always try (and are always good for institutional knowledge)◦ Ask strings
Descending vs. Ascending (ascending has been winning lately) Dynamic versus static Open text box – some interesting results for conversions and
average gift (prospects?). Dare you to try it . ◦ Copy
Short form vs. long form ◦ Imagery
Square/horizontal None at all Very little copy/large image cta
◦ Buttons vs. other CTAs Text vs. graphic
![Page 26: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/26.jpg)
Next up…
What’s next◦Inline upgrades◦Popups: Images versus inline forms◦Sustainer popup tweaking
Ask amount Design
◦More client rollouts = more data◦Tweaking mobile version of forms
Less fields Quick donate?
![Page 27: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/27.jpg)
How to Test
Testing Platforms used◦Google Experiments◦Optimizely
52 other options: http://whichtestwon.com/techguide
![Page 28: DMAW Digital Day - Testing Mayhem](https://reader034.vdocuments.mx/reader034/viewer/2022052522/54787276b4af9fa6478b4603/html5/thumbnails/28.jpg)
Questions?