a/b testing on the obama campaign winning with data
TRANSCRIPT
![Page 1: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/1.jpg)
A/B Testing on the Obama Campaign
Winning With Data
![Page 2: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/2.jpg)
Winning with A/B Testing
![Page 3: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/3.jpg)
What impact can testing have?
![Page 4: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/4.jpg)
Testing = constant improvement Little improvements add up
Improving 1% here and 2% there isn’t a lot at first, but over time it adds up
![Page 5: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/5.jpg)
Tests upon tests upon tests
Subject & draft tests Full-list tests Background personalization tests
Review: Every piece of communication was an opportunity to test
A single email often had many tests attached
![Page 6: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/6.jpg)
Lessons
Click icon to add picture
![Page 7: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/7.jpg)
Don’t Trust Your Gut
Lesson #1
![Page 8: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/8.jpg)
Don’t trust your gut
We don’t have all the answers Conventional wisdom is often wrong Long-held best practices are often wrong You are not your audience
There was this thing called the Email Derby… If even the experts are bad at predicting a
winning message, it shows just how important testing is.
![Page 9: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/9.jpg)
Foster a culture of testing
Lesson #2
![Page 10: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/10.jpg)
The culture of testing
Check your ego at the door
Use every opportunity to test something
Compare against yourself, not against your competitors or “the industry” Are you doing better this month than last
month? Are you doing better than you would have
otherwise?
![Page 11: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/11.jpg)
Use data to make the user experience more personal
Lesson #3
![Page 12: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/12.jpg)
Big data ≠ big brother
Testing allows you to listen to your user base Let them tell you what they like Whether through A/B testing or behavioral
segmentation, optimization gives them a better experience
Usually, the interactions that are the most human are the ones that win
![Page 13: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/13.jpg)
Be human!
In general, we founds shorter, less formal emails and subject lines did best. Classic example: “Hey”
When we dropped a mild curse word into a subject line, it usually won “Hell yes, I like Obamacare” “Let’s win the damn election” “Pretty damn cool”
![Page 14: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/14.jpg)
Good segmentation: behavioral Behavioral segmentation was much more
effective than demographic segmentation Donor vs. non-donor High-dollar vs. low-dollar Volunteer status What issues do people say they care about?
After using A/B tests to create a winning message, we could tweak it slightly for various behavioral groups and get better results
![Page 15: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/15.jpg)
Conclusions
![Page 16: A/B Testing on the Obama Campaign Winning With Data](https://reader034.vdocuments.mx/reader034/viewer/2022051401/56649cff5503460f949d0664/html5/thumbnails/16.jpg)
Conclusions
Test everything, especially your gut instinct
Foster a culture of testing
Use data to make it personal