taking your site performance to the next level with optimization testing
DESCRIPTION
For more ecommerce webinars visit us at http://www.elasticpath.com/webinars/archiveTRANSCRIPT
![Page 1: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/1.jpg)
Elastic Path™
Presented by Linda Bustos, Director of Ecommerce Research, Elastic Path Software June 25, 2010
Taking Your Site Performance to the Next Level with Optimization Testing
Listen to audio using your computer’s microphone & speakers (VoIP) or dial in: United States: + 1 323 417 4600 United Kingdom: + 44 (0) 121 368 0265 Access Code: 646-797-773
![Page 2: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/2.jpg)
Elastic Path™
Introducing Elastic Path Software
• Enterprise ecommerce solutions for software, media, telco, online services, and digital reseller markets
• Trusted ecommerce experts
• #1 ecommerce blog www.getelastic.com
• For more on-demand ecommerce resources: www.elasticpath.com/resources
![Page 3: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/3.jpg)
Elastic Path™
Agenda
• What is optimization testing?
– A/B split testing vs. MVT (multivariate testing)
• What to optimize
• Designing your test
• Testing “gotchas”
• Understanding tools (free vs. paid)
![Page 4: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/4.jpg)
Elastic Path™
What is optimization testing?
![Page 5: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/5.jpg)
Elastic Path™
What is optimization testing?
• Not user testing
– Qualitative data
– Goal is to identify usability issues
![Page 6: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/6.jpg)
Elastic Path™
What is optimization testing?
• A/B split and multivariate testing (MVT)
– Quantitative data
– Goal is to improve the performance of a KPI (key performance indicator, metric of success)
– Hypothesis testing (user testing may help form your hypothesis)
![Page 7: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/7.jpg)
Elastic Path™
What is optimization testing?
• Why test?
– The only way to truly know what works and what doesn’t for your industry, customers and products
• No “gut feel” or opinions
• No “best practices”
• No “blind redesign”
• No missed opportunities
![Page 8: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/8.jpg)
Elastic Path™
A/B
A B
![Page 9: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/9.jpg)
Elastic Path™
A/B
A B C D
![Page 10: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/10.jpg)
Elastic Path™
A/B
A
B
![Page 11: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/11.jpg)
Elastic Path™
MVT
A B C D
V1a V1b V2a V2b
![Page 12: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/12.jpg)
Elastic Path™
A/B or MVT?
• A/B
– Broad decisions
– Just getting feet wet
– Limited traffic/conversions
– Tool restrictions
• MVT
– Tweaking a “winner”
– Large traffic/conversions
– Need to test >10 recipes at one time • More accurate
• Faster
![Page 13: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/13.jpg)
Elastic Path™
What to optimize
![Page 14: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/14.jpg)
Elastic Path™
What to optimize
• Processes/workflows
![Page 15: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/15.jpg)
Elastic Path™
What to optimize
• Processes/workflows
• “Landing pages”
– Any individual page, even if it’s not a landing page
![Page 16: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/16.jpg)
Elastic Path™
What to optimize
• Processes/workflows
• “Landing pages”
– Any individual page, even if it’s not a landing page
• Campaigns
– Affiliate
– Pay Per Click
![Page 17: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/17.jpg)
Elastic Path™
What to optimize
• Processes/workflows
– Checkout/sales funnels
– Account registration
– Product finder
– Product configurator
![Page 18: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/18.jpg)
Elastic Path™
Control checkout
![Page 19: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/19.jpg)
Elastic Path™
Treatment checkout
+ 257.26% Checkout Completion
+ 8.54% Average Order Value
+ 0.54% Overall Site Conversion
![Page 20: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/20.jpg)
Elastic Path™
What to optimize
• Landing pages
– Top content
– Top landing pages (may not be home page!)
– Highest bounce rates
• Segment!
– Top exit pages
– Flagship products
– Underperforming products/categories
![Page 21: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/21.jpg)
Elastic Path™
Design
• Colors
• Navigation
• Tabbed information
• Search
• Placement (what’s above fold, etc)
• Imagery
• Banners
• Content spaces
• Calls to action
![Page 22: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/22.jpg)
Elastic Path™
Content
• Images (zoom, 360 rotate,
color swatch, multiple views)
• Headlines
• Descriptions / Copy
• Call to action text
(button labels, links)
• Review feature
• Video
• Interactive tools
• Interstitials
![Page 23: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/23.jpg)
Elastic Path™
Merchandising /Promotions
• Banners
• Cross-sell, up-sell
• Bundles
• Pricing
• Offers (% off, $ off, free gift, free shipping etc)
• Products on promotion
![Page 24: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/24.jpg)
Elastic Path™
Designing your test
![Page 25: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/25.jpg)
Elastic Path™
Start with web analytics
• Look for areas to improve • Funnel abandonment
• Top landing pages, bounce rate
• Top content, call to action
• Identify your conversion goal/KPIs
• Make sure you’ve segmented your data
![Page 26: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/26.jpg)
Elastic Path™
![Page 27: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/27.jpg)
Elastic Path™
![Page 28: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/28.jpg)
Elastic Path™
Analyze your [page]
![Page 29: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/29.jpg)
Elastic Path™
![Page 30: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/30.jpg)
Elastic Path™
Select variables that influence each other
![Page 31: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/31.jpg)
Elastic Path™
When to stop
http://tinyurl.com/gwo-calculator
![Page 32: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/32.jpg)
Elastic Path™
Statistical significance
![Page 33: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/33.jpg)
Elastic Path™
Testing “gotchas”
![Page 34: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/34.jpg)
Elastic Path™
Testing “gotchas”
• Can’t measure the “why,” only the “what” – (Supplement with user testing)
• “Half-life” – Results may move back to baseline over time (traffic
mix changes, economy (offers), etc.)
• Can’t tell you if you should build a feature – User testing
• Your application • A competitor • A site with a similar feature
• Problem might be product related – Inventory, assortment
![Page 35: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/35.jpg)
Elastic Path™
Comparing the tools
![Page 37: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/37.jpg)
Elastic Path™
Major players
• Google – Free
– 95% CI only
– “Full factorial” only
– Can’t handle segmentation or personalization
– Limits on variables/branching factors, recipes
– Can’t use on email campaigns
– Can’t use offline data
– Can’t self-host (but less reliance on IT)
– No customer support (unless you have Google rep)
– Requires a web developer to help set up
![Page 38: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/38.jpg)
Elastic Path™
Major players
• Omniture Test&Target
– Plays nicely with other products in Omniture suite
– “Predictive learning” accepts offline data (e.g. product holdings, credit score, time on file)
– Design tests right on the site
– Usually a four-figure monthly fee
![Page 39: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/39.jpg)
Elastic Path™
Should you buy a tool?
• If you bought your tool, don’t change – Omniture Test&Target plays nicely with other products
in Omniture suite
• If you haven’t bought, use Google (“Get good at free”)
• Who owns testing? Is IT-free important? • Do you need to test email? (most email platforms
have at least A/B testing) • Who owns the data? Could budget constraints
force you to cancel the tool? • Some tools have API to port into other tools
(analytics, spreadsheets)
![Page 40: Taking Your Site Performance to the Next Level with Optimization Testing](https://reader034.vdocuments.mx/reader034/viewer/2022051513/5479952d5806b59e048b46cc/html5/thumbnails/40.jpg)
Elastic Path™
Thank you!
http://www.getelastic.com/
For more on-demand ecommerce resources: www.elasticpath.com/resources
Questions?