![Page 1: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/1.jpg)
Prepared for :Coca-Cola Mobile Innovation WorkshopSeptember 24-26, 2013Shanghai, China A/B Testing for Apps
A/B Testing for AppsHow I Learned to Stop Worrying and Love
Data-Driven Decisions
![Page 2: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/2.jpg)
User Base
...and see which one is driving desirable user behavior.
Publish two different versions of your app...
50% sees version B50% sees version A
![Page 3: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/3.jpg)
Which Version Won?
Version A Version B
Source: WhichTestWon
![Page 4: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/4.jpg)
Version B: 114% Improvement
Version A Version B ✔ ✗
Marketer’s Surprise: ‘FREE’ Loses
![Page 5: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/5.jpg)
Multivariate Testing
• Every screen has X components (ex: Marilyn’s hair)
• For each, we can test Y variations (ex.: Green)
• In total, we have [Y1 x Y2 x Y3] combinations
![Page 6: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/6.jpg)
How Obama Raised $60 Million
Source: Optimizely
![Page 7: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/7.jpg)
Four Button Variations
![Page 8: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/8.jpg)
Six Media Variations
![Page 9: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/9.jpg)
24 Combinations!
![Page 10: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/10.jpg)
And the Winner is...
+40%increase in conversion
rate
2.9 millionadditional donators
$60 millionvalue of additional
donations
![Page 11: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/11.jpg)
Obamalytics
• Original Conversion Rate: 8.3%
• New Conversion Rate: 11.6%
• 10 million signups from New Version would have been 7.12 million signups with the Original Version
• +2.88 million additional signups
• $21 average donation per signup
• Approximately $60 million in additional donations
![Page 12: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/12.jpg)
Testing to Increase Virality
Headline
Call to Action
Workflow
![Page 13: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/13.jpg)
Data-Driven App Marketing
• App marketing and analytics is in early stages
• App stores are opaque, approval processes slow down iteration cycles
• Few options for making decisions• HiPPO: Highest Paid Person’s Opinion• IHOP: In-House Optimization
• Scalable, reliable app measurement tools are emerging
![Page 14: A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)](https://reader034.vdocuments.mx/reader034/viewer/2022051612/54c655ac4a795977328b459b/html5/thumbnails/14.jpg)
Thank You!
For more information:
Zac Aghion, CEO & [email protected]
China: (+86)1592-1631-924USA: (+1)617-750-6684
www.splitforce.com