A/B optimizely technical details

Download A/B optimizely technical details

Post on 02-Jul-2015

637 views

Category:

Technology

2 download

Embed Size (px)

DESCRIPTION

Technical overview about how optimizely works.

TRANSCRIPT

  • 1. A/B Testing whats that and how it worksKonrad Synoradzki Allegro Group 2013

2. What is A/B testing? A/B testing is comparing two versions of a web page to see which one performs better.Konrad Synoradzki Allegro Group 2013 3. web-page visitorsKonrad Synoradzki Allegro Group 2013 4. slice visitorsKonrad Synoradzki Allegro Group 2013 5. Then compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. Konrad Synoradzki Allegro Group 2013 6. exampleKonrad Synoradzki Allegro Group 2013 7. exampleKonrad Synoradzki Allegro Group 2013 8. exampleKonrad Synoradzki Allegro Group 2013 9. What can you test? Headlines Sub headlines Paragraph Text Testimonials Call to Action text Call to Action Button Links Images Content above or below the fold Social proof Konrad Synoradzki Allegro Group 2013 Media mentions Awards and badges Banners efficiency Rotation speed New version of page Removing ads 10. Types of testsKonrad Synoradzki Allegro Group 2013 11. A/B testing Best used to measure the impact of a two to four variables on interactions with the page.Examples: new layout Labels on buttonKonrad Synoradzki Allegro Group 2013 12. Multivariate Testing compares a higher number of variables, reveals more information about how these variables interact with one another. traffic to a page is split between different versions of the design. Examples Test headline, button color, button text and image on landing pageKonrad Synoradzki Allegro Group 2013 13. Multi-Page Testing A/B test on multi-page Example: Colors on primary action in process Badges on promo itemsKonrad Synoradzki Allegro Group 2013 14. Getting Started What is and how to use optimizely?Konrad Synoradzki Allegro Group 2013 15. Getting Started Step 0: Implement the snippetKonrad Synoradzki Allegro Group 2013 16. Getting Started Step 1: Enter your URL on Optimizely.comKonrad Synoradzki Allegro Group 2013 17. Getting Started Step 2: Add page variationsKonrad Synoradzki Allegro Group 2013 18. Getting Started Step 3: Set up goals for trackingKonrad Synoradzki Allegro Group 2013 19. Getting Started Step 4: Target and allocateKonrad Synoradzki Allegro Group 2013 20. Report pageKonrad Synoradzki Allegro Group 2013 21. Short movie https://fast.wistia.net/embed/iframe/gxttknv8aa?auto Play=true&controlsVisibleOnLoad=true&version=v1&vo lumeControl=trueKonrad Synoradzki Allegro Group 2013 22. Technical details but how??Konrad Synoradzki Allegro Group 2013 23. Organizing ExperimentsKonrad Synoradzki Allegro Group 2013 24. User RolesKonrad Synoradzki Allegro Group 2013 25. Each Project Has Its Own Project Code (Snippet)or asynchronous versionKonrad Synoradzki Allegro Group 2013 26. Optimizely Snippet Contents Active Experiments Draft Experiments Project Goals Optimizely execution and tracking logic Optimizely jQuery Settings (optional version 1.6.4 included by default)Konrad Synoradzki Allegro Group 2013 27. Two modesKonrad Synoradzki Allegro Group 2013 28. Optimizely translates every change made in the visual editor into a line of jQuery code, which we call Variation Code:Konrad Synoradzki Allegro Group 2013 29. Optimizely Testing ProcessKonrad Synoradzki Allegro Group 2013 30. Order of ExecutionKonrad Synoradzki Allegro Group 2013 31. Snippet Timing LogicKonrad Synoradzki Allegro Group 2013 32. Optimizely Architecture OverviewKonrad Synoradzki Allegro Group 2013 33. Optimizely Architecture Overview Google App Engine is used to store and manage your account and experiment configuration settings, including your account login and password information. Passwords are hashed and stored securely using 160 bit cryptographic hashes. Amazon Elastic Compute Cloud and Simple Storage Service is used to store your conversion/traffic logging data, which Optimizely uses to compute experiment results. Akamai is used to store and serve your Optimizely snippet JavaScript file(s). This file is publicly accessible by necessity.Konrad Synoradzki Allegro Group 2013 34. Collecting data The visitors browser makes an asynchronous request to https://log.optimizely.com, which logs the following information: The Optimizelyassigned unique user ID IDs of the experiments and variations this visitor has seen Your Optimizely account number Visitors browser/OS version Visitors IP address, with last octet removed A string identifying this conversion goal, such as the page URL Date and timeKonrad Synoradzki Allegro Group 2013 35. thats allKonrad Synoradzki Allegro Group 2013 36. question?Konrad Synoradzki Allegro Group 2013 37. Konrad Synoradzki Allegro Group 2013