web analytics workshop
TRANSCRIPT
Making Your UX Process Effective and Persuasive
with Web AnalyticsO’Neil | Rahul
Somesh Rahul@SomeshRahul
Daniel O’Neil@phoenix1189
Namaste! We’re Information Architects at The Understanding Group (TUG)
How Somesh Got Here
How Daniel Got Here
Intended Takeaways
1. Be able to apply a framework for balancing quantitative and qualitative research methods.
2. Have a grasp of several key Google Analytics tools that are most relevant to UX practices.
3. Learn through labs, workshops and case studies how web analytics is applied to actual UX projects.
Why Web Analytics
What Web Analytics Can and Can’t do
Analytics track actions, not intent!
Web Analytics and User Experience
Behaviors can infer intent Quantitative guides Qualitative
Search Analytics for Your Site
by - Lou Rosenfeld
The Web Analytics Framework
Business Goals and User Needs
Market, Audience, Seasonality
Testable On-Site Behavior
Website Analytics Works Best When They are Measuring the Distillation of True Value
Business Goals / User Needs
Website GoalsDescribed by: Hypotheses
MarketAudience
Seasonality
Constrained and Organized by:Filters, Segments, Time
Testable On-Site Behavior
Tested by:
Descriptive Analytics
Flow, Page Navigation, Nonstatistical narratives
Statistical Analytics
A/B testing of Page Variations and Dimension Segments
Goals and Hypotheses
Business Goals / User Needs
Website GoalsDescribed by: Hypotheses
Websites Run on GoalsA “Goal” is a measurable outcome resulting in a user completing some desired activity on your website. Typical goals are:
– Confirmation page at the end of a sales transaction.
– Thank-you page after filling out a contact or quote request form.
– Application or content downloads.
– Playing a game or watching a video on a site.
The Best Goals are Existentially Critical
– Is it a “Holiday Bonus” question?
– If this goal stopped happening, would your organization (or your department) still exist?
– Most companies should have a few goals filtered through many segments.
BUT...Goals Can’t Say Why or How
– Goals describe behavior badly.
– Goals can’t describe intent at all.
Hypotheses Link Goals, Behavior, and Theories about Intent.
Hypotheses
A hypothesis suggests a functional change based on a theory of action that has a measurable outcome.
Goals and Hypotheses
– Goals link clear up or down numbers to the outcome of a specific site behavior.
– Hypotheses provide the testable narrative about how the user’s experience on the site affects those goals.
– The testable narrative does not have to BE a goal, but should specifically be IN SERVICE OF a goal.
Filters and Segments
MarketAudience
Seasonality
Constrained and Organized by:Filters, Segments, Time
User Segments
shops like consumer
designer involved
in specifying VIP atcurrentcontract
customer
VIP atcontractprospect
?investor
Problems User Segments Addresses
Problems User Segments Addresses
Filters
- Todo: something about filters here
5minutes
Descriptive Analytics
Testable On-Site Behavior
Tested by:
Descriptive Analytics
Flow, Page Navigation, Nonstatistical narratives
Statistical Analytics
A/B testing of Page Variations and Dimension Segments
User Flows
- Structure User Interviews- Create User Journey- Find Path of Least Resistance
Stand-alone
Integrated
External / Social
Statistical Analytics
Testable On-Site Behavior
Tested by:
Descriptive Analytics
Flow, Page Navigation, Nonstatistical narratives
Statistical Analytics
A/B testing of Page Variations and Dimension Segments
What is Statistically Significant?
Determining whether the differences seen in data is more than random chance.
Why Use It?
- Addresses the HiPPO problem.- Saves time by getting to outcomes faster.- Uncovers subtle effects.- Confronts our own biases about aesthetic and design.
Quantifying a “measurable outcome”
If goals have been set up properly, outcomes can be measured using simple statistics. And simple is all we need!
The recommended statistical method for UX professionals is the A/B test.
Appropriate A/B Tests Should:
- Be immediately apparent to anyone looking comparing the pages.
- Be defined in a functional UX way.- Represent a set of coherent conceptual changes against
a single hypothesis.
Typical A/B Test Candidates
Question Testing For Best Testing Tool
Is the navigation layout affecting conversion rate?
Conversion rate by template
Google Analytics Experiements (Not out of the box but you can hack it)
Which of two landing pages performs better?
Conversion rate by page version
Google Analytics Experiments
Which User segment converts better
Conversion rate compared by User segment
Advanced segments, Confidence Interval test
What Statistics Don’t Tell You
- Why a test failed. This can be just as critical as a success.
- Why it succeeded. - How to thoughtfully create testable hypotheses.
Marrying User Experience & Web Analytics
Your UX Process
Abstracted UX Process
Discovery Research and Analysis
Design and Testing
Discovery
Discovery
● Establish clearly the “Why” and “Who” for the site.
● Organizational goals are articulated and prioritized.
● The audience is clearly identified.● The ultimate measures of success
are agreed upon.
Research and Analysis
Research and Analysis
● Research how your users approach your current site.
● Evaluate the website’s design and information architecture.
● Synthesize the details into high-level models that represent both user needs and a high-level information architecture.
Design and Testing
Design and Testing
● Specify the site structure.● Determine the ways in which the
goals will be achieved through site structure.
● Test.
Qualitative and Quantitative Research
Discovery Research and Analysis
Design and Testing
Qualitative Research (UX)
Quantitative Research (WA)
Stakeholder InterviewsIntention Modeling
User InterviewsPersonasUser Journeys
PrototypesLive Testing
Hypothesis GenerationGoal Context
User SegmentsUser Flows
A/B Tests
Thank You!