critical mass 3 measurement problems solved webinar

© 2011 Critical Mass, Inc. All Rights Reserved

3 Web Measurement Problems Solved!How Critical Mass Marketing Science is leveraging innovative thinking, modeling and technology to overcome web measurement’s biggest challenges.

2© 2011 Critical Mass, Inc. All Rights Reserved

We are Critical Mass

Amsterdam Calgary Chicago

London New York Costa Rica

Toronto

Nashville

166508

YearsEmployeesOffices

3© 2011 Critical Mass, Inc. All Rights Reserved

We are Critical Mass

Amsterdam Calgary Chicago

London New York Costa Rica

Toronto

Nashville

Dedicated MS SpecialistsYears combined measurement expertise

166508

YearsEmployeesOffices

25+150+

5© 2011 Critical Mass, Inc. All Rights Reserved

What is Marketing Science?

A brief overview...

6© 2011 Critical Mass, Inc. All Rights Reserved

Definition

Marketing Science is the application of measurement methods to ascertain, within a quantifiable degree of accuracy, the effectiveness of a marketing initiative.

Or put simply… it’s proof!

7© 2011 Critical Mass, Inc. All Rights Reserved

Use data

• Measure performance• Find opportunities for

optimization• Improve marketing

campaigns

• Deliver customized content• Improve the customer

experience• Reduce costs• Improve sales

To

© 2011 Critical Mass, Inc. All Rights Reserved

Problem: Unreliable data

10© 2011 Critical Mass, Inc. All Rights Reserved

Unreliable Data

What causes web analytics data to be unreliable?• Flawed set up

· Developers didn’t understand the tagging· Tracking of flash or AJAX not included· Applications or cross domain tracking not tagged· Specific metrics aren’t being captured

• Code “broke”· Page or section released without code· Code rollback broke tagging· Wrong file released to production

11© 2011 Critical Mass, Inc. All Rights Reserved

Quick Poll

How often in the last year has your site suffered incorrect or broken web analytics coding?

12© 2011 Critical Mass, Inc. All Rights Reserved

Unreliable Data - Solutions

Flawed set upNew Technology: Tag Management Systems / Universal Tag

WEBSITES

IT / AGENCY / DEVELOPERS

WebAnalyticsVendors

Voice ofConsumer

(VOC)

AdServers

TestingSolutions

TargetingTools

Pay PerClick

(PPC) Tools

AffiliateTracking

13© 2011 Critical Mass, Inc. All Rights Reserved

WEBSITES

Tag Management System

WebAnalyticsVendors

Voice ofConsumer

(VOC)

AdServers

TestingSolutions

TargetingTools

Pay PerClick

(PPC) Tools

AffiliateTracking

MarketingScience

Flawed set upNew Technology: Tag Management Systems / Universal Tag

Unreliable Data - Solutions

14© 2011 Critical Mass, Inc. All Rights Reserved

Why Tag Management?

1. Single line of code2. Any vendor tag can be added3. Analytics resources make updates4. Works with HTML, AJAX Flash, Flex, Air5. Remove/pause/replace tags at any time

Tagging changes and updates go

15© 2011 Critical Mass, Inc. All Rights Reserved

Tag Rules

Page Request

Client Web Infrastructure

Third PartyWeb Analytics Or other system

DataBrowser

Tag Management System

Tag Management Systems – How it works

16© 2011 Critical Mass, Inc. All Rights Reserved

Unreliable Data - Solutions

Web analytics code was installed but then “broke” New Technology: Analytics Audit and Monitoring Tools

17© 2011 Critical Mass, Inc. All Rights Reserved

Why Analytics Audit and Monitoring Tools?

1. Manual verification too time intensive2. Provides an overall view specific

attributes3. Automated tag discovery - every page

checked4. Scheduled reports, alerts and audits5. No calls to web analytics systems6. Works on Flash and secure sites

18© 2011 Critical Mass, Inc. All Rights Reserved

Audit and Monitoring – How it works

© 2011 Critical Mass, Inc. All Rights Reserved

Problem: Complex reporting needs

20© 2011 Critical Mass, Inc. All Rights Reserved

Complex Reporting Needs

• Multiple reports with rapid delivery requirements· Deliver dozens of dashboards in hours, not weeks· Different stakeholders, different data , different times

• Dashboards and data are not easily accessible· Stakeholders lack system training· Emails with dashboards get deleted or misfiled· Executives on-the-go have no immediate access to data

21© 2011 Critical Mass, Inc. All Rights Reserved

Quick Poll

How complex is your data reporting structure?

22© 2011 Critical Mass, Inc. All Rights Reserved

Complex Reporting - Solutions

Multiple reports with rapid delivery requirementsInnovative delivery: Web-based reporting and data warehousing

23© 2011 Critical Mass, Inc. All Rights Reserved

Why web-based reporting and data warehousing?

1. Merges data from web analytics and other sources

2. Doesn’t require a BI department3. Systems connect directly via APIs4. Automatically generates highly

customized reports5. Stakeholders can access from

anywhere at any time

24© 2011 Critical Mass, Inc. All Rights Reserved

Web-based reporting – How it works

Display

Rich Media

Search

Email

Affiliate

MediaOnline and Offline

ETLNormalizationDe-duplication

Customized Portal

25© 2011 Critical Mass, Inc. All Rights Reserved

Complex Reporting - Solutions

Dashboards and data are not easily accessible Innovative delivery: Mobile dashboard applications

26© 2011 Critical Mass, Inc. All Rights Reserved

Why mobile dashboard delivery?

1. Data in a mobile format = increased engagement

2. Connects to a variety of data sources

3. View, analyze and interact with data visualizations

4. Secure, hosted and scalable 5. Available from data warehousing

and web analytics vendors

27© 2011 Critical Mass, Inc. All Rights Reserved

Mobile Dashboards – How it works

Data to data warehouse to phone

© 2011 Critical Mass, Inc. All Rights Reserved

Problem: All reports, no action!

29© 2011 Critical Mass, Inc. All Rights Reserved

Organization Culture and Measurement Spectrum

High CostHigh Consideration

Low CostHigh Consideration

Low CostLow Consideration

Culture of Research

Culture of Advertising

Culture ofAccountability

30© 2011 Critical Mass, Inc. All Rights Reserved

All data, no action! - Solutions

• No clear objectives, roadmaps, or KPIs · What is the website is supposed to “do”· Website decisions made by gut feel or HiPPO

• No or inadequate analytics resources · Tech manages web analytics installation · A BI department took on web analytics · No one watching the data

All data, no action!

31© 2011 Critical Mass, Inc. All Rights Reserved

Quick Poll

What’s your company culture now?

32© 2011 Critical Mass, Inc. All Rights Reserved

All data, no action! - Solutions

Getting from data to actionHolistic analytics approachWeb analytics maturity modelling and roadmapThe Critical Mass Marketing Science site and project process modelAnalytics collaboration and integration

33© 2011 Critical Mass, Inc. All Rights Reserved

Holistic Analytics Approach

Project

Online Ecosystem

Enterprise

• Inform and strategize• Implement and measure• Optimize

• Evaluate the business web analytics maturity level in several key areas

• Define a high level roadmap to a more data-driven eBusiness culture

• Ongoing measurement and optimization program

34© 2011 Critical Mass, Inc. All Rights Reserved

0

1

2

3

4

5

6Management, Adoption

Objectives

Scope

Team and Expertise

Continuous Improvement Process & Methodology

Tools, Technology & Data Integration

Current State

Target State

Ideal State

Enterprise Web Analytics Maturity Model Example

6 Analytically Addicted5 Competitor4 Integrated3 Operational2 Initiated1 Analytically Unengaged

* Stéphane Hamel - http://immeria.net/oamm See Appendix

35© 2011 Critical Mass, Inc. All Rights Reserved

Site and Project Marketing Science Process

Implementation

OptimizationGoals, KPIs, Strategy

Data Collection System, Tagging

Software

Reporting & Analysis System

Design

Training, Education

1. Confirm Project /

Opportunity Objectives

2. Pre-Analysis

3. Set Targets / Goals

4. Launch and Measure

5. Record Learnings,

Test, Optimize, Socialize

36© 2011 Critical Mass, Inc. All Rights Reserved

Resources and Analytics Collaboration

We never work in isolation.

Executive

TechMarketing

Analytics

© 2011 Critical Mass, Inc. All Rights Reserved

Thank you. Future Topics?

Dan Linton Marketing Science Director

e danl@criticalmass.comp 416.673.5308criticalmass.com

38© 2011 Critical Mass, Inc. All Rights Reserved

Appendix

39© 2011 Critical Mass, Inc. All Rights Reserved

Links and Resources

Ensighten Tag Management Systemhttp://www.ensighten.com

ObservePoint Audit and Monitoringhttp://www.observepoint.com

SiteCatalyst for iPhonehttp://itunes.apple.com/us/app/sitecatalyst/id289760802?mt=8

40© 2011 Critical Mass, Inc. All Rights Reserved

Experience DesignBrand experience auditsBrand definitionUser experience and interaction designDesign, copy and content servicesContent development and classificationRich media production

Technology ServicesTechnology architectures Applications, web and mobile developmenteCRM database and profile developmenteCommerce personalization and web servicesContent and digital asset managementRich Internet application development

Program ManagementProject and program managementStaff planning and resource managementStakeholder relationship managementBudget and requirements facilitation

Insight & PlanningStrategic roadmaps and frameworksBrand planningResearch and usabilityPersona and scenario modeling & designInformation ArchitectureConsumer insight development

Experience DistributionSocial strategy and governanceVideo syndication and aggregationCommunity development and social activationDisplay and behavioral planningSearch engine marketing and optimizationMobile and location-based serviceseMail and digital campaign management

Marketing ScienceBusiness analysis and ROI modelingDigital program KPI developmentMulti-channel analyticsProgram analysis and optimization planning

Full-Service Digital Capabilities

41© 2011 Critical Mass, Inc. All Rights Reserved

The WAMMTHE WEB ANALYTICS MATURITY MODELA strategic approach based on business maturity and critical success factors - Stéphane Hamel - http://immeria.net/oamm

1. Impaired:Characterized by the use of out of the box tools & reports, limited resources lacking formal training (hands on skills) and education (knowledge). Web Analytics is used on an ad-hoc basis and is of limited value and scope. Some tactical objectives are defined but results are not well communicated and there are multiple versions of the truth.

2. Initiated:Working with metrics to optimize specific areas of the business (such as marketing or the ecommerce catalogue). Resources are still limited but the process is getting streamlined. Results are communicated to various business stakeholders (often director level). However, web analytic might be supporting obsolete business processes and thus, be limited in its ability to push for optimization beyond the online channel. Success is mostly anecdotal.

3. Operational:Key Performance Indicators and dashboards are defined and aligned with strategic business objectives. A multidisciplinary team is in place and use various sources of information such as competitive data, voice of customer and social media or mobile analysis. Metrics are exploited and explored through segmentation and multivariate testing. The Internet channel is being optimized; personas are being defined. Results start to appear and be considered at theexecutive level. Results are centrally driven, but broadly distributed.

4. Integrated:Analysts can now correlate online and offline data from various sources to provide a near 360° view of the whole, value chain (see “Limits of web analytics” note). Optimization encompasses complete processes, including back-end and front-end. Online activities are defined from the user perspective and persuasionscenarios are defined. A continuous improvement process and problem solving methodologies are prevalent. Insight and recommendations reachthe CXO level.

5. Competitor:This level is characterised by several attributes of companies with a strong analytical culture (Davenport and Harris 2007):a) One or more senior executives strongly advocate fact based decision making and analytics b) Widespread use of not just descriptive statistics, but predictive modeling and complex optimization techniques c) Substantial use of analytics across multiple business functions or processed) Movement toward an enterprise level approach to managing analytical tools, data, and organizational skills and capabilities.

6. Addicted: This level matches Davenport’s “Analytical Competitor” characteristics: deep strategic insight, continuous improvement, integrated, skilled resources, top management commitment, fact-based culture, continuous testing & learning and most important: far beyond the boundaries of the online channel.