A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson

Download A/B Testing – How to Break Things and Fail Fast (Without Breaking Things) - Mark Halvorson

Post on 21-May-2015

424 views

Category:

Software

2 download

DESCRIPTION

Everyone wants to optimize their web property, especially if that web property is their only sales channel. In this talk, we will explore how Atlassian has used their CMS along with JIRA, Confluence, and other tools to operationalize A/B testing to continually optimize Atlassian's website. Using this process we can balance failing fast with keeping things running smoothly.

TRANSCRIPT

1. Yntlassian i viA/ B TestingHow to break things and fail fast. .. without breaking things_ IHANK MAI! -"JR. 'N MFAB OF INYFRQCYIVF ATI AS5|AN }uVT''A5IF G79. -C115 About Atlassian. Yvv>". VlV'i -1-~. .r -~. _ . lAf li .. l i. .. l . ..l in- Products include:JIRA.Confluence.Bitbucket.Hipchat. .. - Founded in 2002 - Ovet 37.000 Customers- Ovet 700 employees globally - No sales people ,_| _ + + Atlassian 2. How to break things and fail fast. .. without breaking thingsLn-UAIIK NAUlOH$0N READ OF INTERACTIVE ATLA-SEIAN SOVTWANE GPIJ4-0112 About Atlassian Products include:JIRA.Confluence.Bitbucket.Hipchat. .. Founded in 2002 Over 37.000 Customers Over 700 employees globally No sales peopleXlttlassian 4 _ . | |:__._n| . _ _o , --I. - - 5 , __o -, __|I- -2. 3. YAthSSi3 Pi-. --- . ~&-nag-n ..... ... ...- . - - .. .4 . u . . I-4 ... ... ... . . ... - . . . .._-. . . ... ... ... ..... _.. ..- . ... - . . . =.. ._. ... . TNT . - .._ _ . .. .._ _ . -._-. - ... ._. .. . .._. .- . ... . _. ..... ... . . . . _ . - . - . - . .._. -. . ... ... ... .... . .._. ... ... ... . T1 jbhcccc wlh 13-. + JIRA - Ticket * I . ... ._... --YnyFrIo1IUOaIri 4-i. | l)All 5SCHEDULE TO AVOID CONFLICTSRUN EXPERIMENTSANAIYII Ill Slll I3 Optimizely - Results 38. IIIIACI IIASID ON lDllNTll IV5A(. l| |IJ| lD DUI L000 LANOIDAYI 5SCHEDULE TO AVOID CONFLICTS5. UN EXPERIMENTSEE. +PEE I "". "i1;- What are you are trying to improve? 39. I: S_. ... ..n. -.. ..- - . .-. ._. - . ... ... ... _.. ... .What are you are trying to improve? Blimp Dashboard . ... - c. .. Q . .. ._ ': 'f"""". -__ ___ - II:.o- yum xx vhnnafry xx xk ______ :1 - --o M-. n--S--Non I. .- : - - ' x. xx% . .. '1';v u -uA vwu :2 -j: _ E_"""' x. xxk._; "" P x. xxk._. __ ' -. -A-nu.. -. -A-nos."' - xx-xx% . . ' xx. xx% . . h -to '1 t I I--d0-CAINL. .. -. r -.H.l. , . ._. - Ouunnuy . .-. .~. ... . . ..... ._ __.TM! -__ 39:- __- re __ ; Iv """ _ . ... ... ... .._. .-. . . 5-D 3-I-no-tints-I """ _ _. ... ... ..... ... ... _.. ... .. 40. _ T xxxk '3 T" xx. xk. x. xx%. Ev " .M. - x. xx. -< x. xxk.xx. xx35. - xx. xx%. ...Optimizely Results . ... . TT 1 :4; W ! ".__. "_"Y 7. f; ."I - 1:2: Q Conuence - Corresponding Test Page Confluence - Results Archive 41. Conuence - Corresponding Test PageT Confluence - Results ArchiveOur Test Philosophy- Everyth"1g aectnng COITVSTSI-Sn is tested - Be aggessive - Regular cadence.no tranc wasted- Never run a test that you wouldnt want to win[1] 42. + .+ Confluence - Results Archive +- . . . . nun I-av j : - : ~a o-an-u--o-co -s- ---. o.. ~~. -. ... n Q1? :1 - nu -unnuc-1 :2 2'. ': .. .. ''122-:-* : : :2 '_-. ::: ::: :..;: :-- "rm--~- A --- on-n -o~-an-run--In'*' .o v-no-a-n-pjylco1- - - Ijg;_ Our Test Philosophy - Every1hing aecting conversion is tested - Be aggessive - Regular cadence.no traic wasted - Never nn a test that you wouldn't want to Wit , Key takeaways: | '_leu3 we e-very'w'Ie'e CUHF-"' as 'na'ry a5 3 L-'l