3 web measurement problems, solved
DESCRIPTION
TRANSCRIPT
© 2011 Critical Mass, Inc. All Rights Reserved
3 Web Measurement Problems Solved!How Critical Mass Marketing Science is leveraging innovative thinking, modeling and technology to overcome web measurement’s biggest challenges.
2© 2011 Critical Mass, Inc. All Rights Reserved
We are Critical Mass
Amsterdam Calgary Chicago
London New York Costa Rica
Toronto
Nashville
166508
YearsEmployeesOffices
3© 2011 Critical Mass, Inc. All Rights Reserved
We are Critical Mass
Amsterdam Calgary Chicago
London New York Costa Rica
Toronto
Nashville
Dedicated MS SpecialistsYears combined measurement expertise
166508
YearsEmployeesOffices
25+150+
4© 2011 Critical Mass, Inc. All Rights Reserved
A selection of our clients
5© 2011 Critical Mass, Inc. All Rights Reserved
What is Marketing Science?
A brief overview...
6© 2011 Critical Mass, Inc. All Rights Reserved
Definition
Marketing Science is the application of measurement methods to ascertain, within a quantifiable degree of accuracy, the effectiveness of a marketing initiative.
Or put simply it’s proof!
7© 2011 Critical Mass, Inc. All Rights Reserved
Use data
• Measure performance• Find opportunities for
optimization• Improve marketing
campaigns
• Deliver customized content• Improve the customer
experience• Reduce costs• Improve sales
To
© 2011 Critical Mass, Inc. All Rights Reserved
Problem: Unreliable data
10© 2011 Critical Mass, Inc. All Rights Reserved
Unreliable Data
What causes web analytics data to be unreliable?• Flawed set up
· Developers didn’t understand the tagging· Tracking of flash or AJAX not included· Applications or cross domain tracking not tagged· Specific metrics aren’t being captured
• Code “broke”· Page or section released without code· Code rollback broke tagging· Wrong file released to production
11© 2011 Critical Mass, Inc. All Rights Reserved
Quick Poll
How often in the last year has your site suffered incorrect or broken web analytics coding?
12© 2011 Critical Mass, Inc. All Rights Reserved
Unreliable Data - Solutions
Flawed set upNew Technology: Tag Management Systems / Universal Tag
WEBSITES
IT / AGENCY / DEVELOPERS
WebAnalyticsVendors
Voice ofConsumer
(VOC)
AdServers
TestingSolutions
TargetingTools
Pay PerClick
(PPC) Tools
AffiliateTracking
13© 2011 Critical Mass, Inc. All Rights Reserved
WEBSITES
Tag Management System
WebAnalyticsVendors
Voice ofConsumer
(VOC)
AdServers
TestingSolutions
TargetingTools
Pay PerClick
(PPC) Tools
AffiliateTracking
MarketingScience
Flawed set upNew Technology: Tag Management Systems / Universal Tag
Unreliable Data - Solutions
14© 2011 Critical Mass, Inc. All Rights Reserved
Why Tag Management?
1. Single line of code2. Any vendor tag can be added3. Analytics resources make updates4. Works with HTML, AJAX Flash, Flex, Air5. Remove/pause/replace tags at any time
Tagging changes and updates go
15© 2011 Critical Mass, Inc. All Rights Reserved
Tag Rules
Page Request
Client Web Infrastructure
Third PartyWeb Analytics Or other system
DataBrowser
Tag Management System
Tag Management Systems – How it works
16© 2011 Critical Mass, Inc. All Rights Reserved
Unreliable Data - Solutions
Web analytics code was installed but then “broke” New Technology: Analytics Audit and Monitoring Tools
17© 2011 Critical Mass, Inc. All Rights Reserved
Why Analytics Audit and Monitoring Tools?
1. Manual verification too time intensive2. Provides an overall view specific
attributes3. Automated tag discovery - every page
checked4. Scheduled reports, alerts and audits5. No calls to web analytics systems6. Works on Flash and secure sites
18© 2011 Critical Mass, Inc. All Rights Reserved
Audit and Monitoring – How it works
© 2011 Critical Mass, Inc. All Rights Reserved
Problem: Complex reporting needs
20© 2011 Critical Mass, Inc. All Rights Reserved
Complex Reporting Needs
• Multiple reports with rapid delivery requirements· Deliver dozens of dashboards in hours, not weeks· Different stakeholders, different data , different times
• Dashboards and data are not easily accessible· Stakeholders lack system training· Emails with dashboards get deleted or misfiled· Executives on-the-go have no immediate access to data
21© 2011 Critical Mass, Inc. All Rights Reserved
Quick Poll
How complex is your data reporting structure?
22© 2011 Critical Mass, Inc. All Rights Reserved
Complex Reporting - Solutions
Multiple reports with rapid delivery requirementsInnovative delivery: Web-based reporting and data warehousing
23© 2011 Critical Mass, Inc. All Rights Reserved
Why web-based reporting and data warehousing?
1. Merges data from web analytics and other sources
2. Doesn’t require a BI department3. Systems connect directly via APIs4. Automatically generates highly
customized reports5. Stakeholders can access from
anywhere at any time
24© 2011 Critical Mass, Inc. All Rights Reserved
Web-based reporting – How it works
Display
Rich Media
Search
Affiliate
MediaOnline and Offline
ETLNormalizationDe-duplication
Customized Portal
25© 2011 Critical Mass, Inc. All Rights Reserved
Complex Reporting - Solutions
Dashboards and data are not easily accessible Innovative delivery: Mobile dashboard applications
26© 2011 Critical Mass, Inc. All Rights Reserved
Why mobile dashboard delivery?
1. Data in a mobile format = increased engagement
2. Connects to a variety of data sources
3. View, analyze and interact with data visualizations
4. Secure, hosted and scalable 5. Available from data warehousing
and web analytics vendors
27© 2011 Critical Mass, Inc. All Rights Reserved
Mobile Dashboards – How it works
Data to data warehouse to phone
© 2011 Critical Mass, Inc. All Rights Reserved
Problem: All reports, no action!
29© 2011 Critical Mass, Inc. All Rights Reserved
Organization Culture and Measurement Spectrum
High CostHigh Consideration
Low CostHigh Consideration
Low CostLow Consideration
Culture of Research
Culture of Advertising
Culture ofAccountability
30© 2011 Critical Mass, Inc. All Rights Reserved
All data, no action! - Solutions
• No clear objectives, roadmaps, or KPIs · What is the website is supposed to “do”· Website decisions made by gut feel or HiPPO
• No or inadequate analytics resources · Tech manages web analytics installation · A BI department took on web analytics · No one watching the data
All data, no action!
31© 2011 Critical Mass, Inc. All Rights Reserved
Quick Poll
What’s your company culture now?
32© 2011 Critical Mass, Inc. All Rights Reserved
All data, no action! - Solutions
Getting from data to actionHolistic analytics approachWeb analytics maturity modelling and roadmapThe Critical Mass Marketing Science site and project process modelAnalytics collaboration and integration
33© 2011 Critical Mass, Inc. All Rights Reserved
Holistic Analytics Approach
Project
Online Ecosystem
Enterprise
• Inform and strategize• Implement and measure• Optimize
• Evaluate the business web analytics maturity level in several key areas
• Define a high level roadmap to a more data-driven eBusiness culture
• Ongoing measurement and optimization program
34© 2011 Critical Mass, Inc. All Rights Reserved
0
1
2
3
4
5
6Management, Adoption
Objectives
Scope
Team and Expertise
Continuous Improvement Process & Methodology
Tools, Technology & Data Integration
Current State
Target State
Ideal State
Enterprise Web Analytics Maturity Model Example
6 Analytically Addicted
5 Competitor
4 Integrated
3 Operational
2 Initiated
1 Analytically Unengaged
* Stéphane Hamel - http://immeria.net/oamm See Appendix
35© 2011 Critical Mass, Inc. All Rights Reserved
Site and Project Marketing Science Process
Implementation
OptimizationGoals, KPIs, Strategy
Data Collection System, Tagging
Software
Reporting & Analysis System
Design
Training, Education
1. Confirm Project /
Opportunity Objectives
2. Pre-Analysis
3. Set Targets / Goals
4. Launch and Measure
5. Record Learnings,
Test, Optimize, Socialize
36© 2011 Critical Mass, Inc. All Rights Reserved
Resources and Analytics Collaboration
We never work in isolation.
Executive
TechMarketing
Analytics
© 2011 Critical Mass, Inc. All Rights Reserved
Thank you. Future Topics?
Dan Linton Marketing Science Director
e [email protected] 416.673.5308criticalmass.com
38© 2011 Critical Mass, Inc. All Rights Reserved
Appendix
39© 2011 Critical Mass, Inc. All Rights Reserved
Links and Resources
Ensighten Tag Management Systemhttp://www.ensighten.com
ObservePoint Audit and Monitoringhttp://www.observepoint.com
SiteCatalyst for iPhonehttp://itunes.apple.com/us/app/sitecatalyst/id289760802?mt=8
40© 2011 Critical Mass, Inc. All Rights Reserved
Experience DesignBrand experience auditsBrand definitionUser experience and interaction designDesign, copy and content servicesContent development and classificationRich media production
Technology ServicesTechnology architectures Applications, web and mobile developmenteCRM database and profile developmenteCommerce personalization and web servicesContent and digital asset managementRich Internet application development
Program ManagementProject and program managementStaff planning and resource managementStakeholder relationship managementBudget and requirements facilitation
Insight & PlanningStrategic roadmaps and frameworksBrand planningResearch and usabilityPersona and scenario modeling & designInformation ArchitectureConsumer insight development
Experience DistributionSocial strategy and governanceVideo syndication and aggregationCommunity development and social activationDisplay and behavioral planningSearch engine marketing and optimizationMobile and location-based serviceseMail and digital campaign management
Marketing ScienceBusiness analysis and ROI modelingDigital program KPI developmentMulti-channel analyticsProgram analysis and optimization planning
Full-Service Digital Capabilities
41© 2011 Critical Mass, Inc. All Rights Reserved
The WAMMTHE WEB ANALYTICS MATURITY MODELA strategic approach based on business maturity and critical success factors - Stéphane Hamel - http://immeria.net/oamm
1. Impaired:Characterized by the use of out of the box tools & reports, limited resources lacking formal training (hands on skills) and education (knowledge). Web Analytics is used on an ad-hoc basis and is of limited value and scope. Some tactical objectives are defined but results are not well communicated and there are multiple versions of the truth.
2. Initiated:Working with metrics to optimize specific areas of the business (such as marketing or the ecommerce catalogue). Resources are still limited but the process is getting streamlined. Results are communicated to various business stakeholders (often director level). However, web analytic might be supporting obsolete business processes and thus, be limited in its ability to push for optimization beyond the online channel. Success is mostly anecdotal.
3. Operational:Key Performance Indicators and dashboards are defined and aligned with strategic business objectives. A multidisciplinary team is in place and use various sources of information such as competitive data, voice of customer and social media or mobile analysis. Metrics are exploited and explored through segmentation and multivariate testing. The Internet channel is being optimized; personas are being defined. Results start to appear and be considered at theexecutive level. Results are centrally driven, but broadly distributed.
4. Integrated:Analysts can now correlate online and offline data from various sources to provide a near 360° view of the whole, value chain (see “Limits of web analytics” note). Optimization encompasses complete processes, including back-end and front-end. Online activities are defined from the user perspective and persuasionscenarios are defined. A continuous improvement process and problem solving methodologies are prevalent. Insight and recommendations reachthe CXO level.
5. Competitor:This level is characterised by several attributes of companies with a strong analytical culture (Davenport and Harris 2007):a) One or more senior executives strongly advocate fact based decision making and analytics b) Widespread use of not just descriptive statistics, but predictive modeling and complex optimization techniques c) Substantial use of analytics across multiple business functions or processed) Movement toward an enterprise level approach to managing analytical tools, data, and organizational skills and capabilities.
6. Addicted: This level matches Davenport’s “Analytical Competitor” characteristics: deep strategic insight, continuous improvement, integrated, skilled resources, top management commitment, fact-based culture, continuous testing & learning and most important: far beyond the boundaries of the online channel.
42© 2011 Critical Mass, Inc. All Rights Reserved
Infiniti Global ChallengeAs the global presence of Infiniti grew, a strong need for consistent and actionable reporting became obvious. In addition to the challenge of delivering a variety of customized reports to many individual country regions as well as the central brand showcase in a timely manner, the Global team also faced language and cultural differences that required a unique analysis approach.
SolutionCritical Mass initiated and developed a unique, automated reporting dashboard with a 3rd party data warehouse supplier. This reporting solution integrated with a challenging variety of Omniture SiteCatalyst report suites, each requiring regional customization. This automated solution reduced the “reporting” hours required for the multitude of country sites and allowed the CM team to focus on analysis and recommendations. It also allows each country to access their reports at any time via the simplified data warehouse interface.
Results & InsightOne of the key challenges faced was delivering dozens of reports in a timely fashion. This one-of-a-kind reporting interface has allowed the CM Marketing Science team to focus on analysis and actionable recommendations. By freeing up this extra time, CM has developed a deeper understanding not only of the metrics, but also of various regional language and cultural differences. The result is more actionable recommendations and more detailed analysis.
42
43© 2011 Critical Mass, Inc. All Rights Reserved
P&G ChallengeP&G’s channel agencies reported results to the client individually, yet none had a comprehensive view of their initiatives (that drove to the website), or understood how they were part of the digital ecosystem overall. Critical Mass responded to this situation by recommending, creating and implementing a measurement program focusing heavily on Marketing Mix Analysis where, on a monthly basis, engagement and acquisition via paid and unpaid traffic sources by origination were analyzed and optimized.
SolutionWe self-initiated and led the collaboration with six other agencies (Carat, Arc, iProspect, Merkle, Atos, CRMMetrix) to bring together data from banner (Doubleclick), paid search (DART), organic search (Google), survey (CRMMetrix), email (Epsilon) and consumer database data (Merkle). Together the group analyzed performance results and collectively presented to the client on a monthly basis with recommendations for improvement.
Results & InsightOne of several key findings we found through the monthly process, was the need for deeper analysis related to acquisition and engagement. Subsequently, we found some high volume acquisition campaigns didn’t necessarily result in long-term relationships with consumers. The strategy needed to be refined first for quality over quantity.
44© 2011 Critical Mass, Inc. All Rights Reserved
Visit Las Vegas Mobile SiteAs part of mobile project for Visit Las Vegas, Critical Mass Marketing Science was tasked with designing and implementing the Omniture analytics tagging.
A complexity revolved around the fact that leads were measured in referrals to other sites, in addition to all the regular metrics that the client wished to capture.
SolutionCritical Mass Marketing Science developed the measurement strategy and created the necessary tagging.
Working with developers, and utilizing Omniture’s HTTP based tracking API and DeviceAnywhere for testing and QA, a full analytics package was deployed on the site that was capable of tracking lead referrals to dozens of other sites.
In addition, the mobile site had to provide both segmented (mobile only) data, as well as roll-up into the overall Vegas reporting.
Results & InsightCritical Mass Marketing Science can now provide the client with not only how many people visited the site, but can also provide:
-How many visits generated phone calls- Which properties were called most frequently- How many times property details were emailed- And of course, how many referral leads were generated to the extensive list of accommodation providers