online audience measurement: from panel measurement to website

Online Audience Measurement: From Panel

Measurement to Website Analytics

Saigon Digital Marketing Conference

23rd May 2009

Joe Nguyen Vice President, Southeast Asia | comScore, Inc. ( NASDAQ:SCOR)

hp+65 9478 2100 | [email protected]

2© comScore, Inc. Proprietary and Confidential.

The web is the most measurable

medium in the history of marketing.

Now all that’s left to do is figure out

how to measure it.

The McKinsey Quarterly

“How poor metrics undermine digital marketing”


Before we get to the HOW, let’s ask WHY?

Typical needs…

� Track visitors to my website: Who, when, where, and what

� Track effectiveness of my campaigns: SEO, SEM, email, Banner ad,

Affiliate, branding campaigns, even TV or print

� Track success events, registrations, DOLLARS made

3

What about…

� Market & competitive knowledge

� Planning your media campaign

� Key demographic knowledge (age, sex, income)


Website Analytics – Site Centric, Server side

Measurement


Site Centric Data Collection

5


Key Common Metrics

� Page Views

� Visits

� Visitors (Daily and Monthly)

� Average Visits / Visitor

� Average Page Views/Visit and Average Page Views/Visitor

� Visit number / New vs. Repeat visitors

� Return Frequency

� Registrations (of various types)

6


Advance Reports: Countries, Region, Cities

7


Advance Reports: Search Engine Performance

8


Advance Reports: Search Engine Performance

9


Advance Reports: Channel Performance

10


Advance Reports: Pathing Analysis

11


Advance Reports: Campaign Performance

12


Advance Reports: Revenue - “Show me the money!”

13


Advance Reports: Conversion Funnel Analysis

14


Dashboards!!!!

15


Panel Based Audience Measurement


How Does Panel Measurement Work?

Collect Traffic

Representative set

of individuals

Collect Traffic

Representative set

of individuals

Final reports of market data

based on projections of

individuals

Final reports of market data

based on projections of

individuals

Apply

comScore

Projection

&URL

Dictionary

Apply

comScore

Projection

&URL

Dictionary

Data ServersData ServersProjected

Traffic

Projected

Traffic


What We Measure

18

AUDIENCEDEMOGRAPHICS

LIFE STAGE

PC ATTRIBUTES

SEGMENTATION

LINKED DATASETS

Acxiom; Claritas/Prizm; internal CRM database, etc.

Browser: IE, AOL, Netscape, …

CONTENT VISITATION

VIDEO / RADIO STREAMING: SEARCH

COMMUNICATION

Email, IM, Mobile

Transactions

Purchase / Subscriptions;

Configurations; Price

Request

Software (Microsoft. Live, Google, Yahoo)

ACTIONS

PC LOCATION:

Home; Work (Corporation, Laptop); University

INTERNATIONAL:

US/Canada

Europe

Asia

Latin America

Language Regions

WORLDWIDE

LOCATIONADVERTISING:

Display, Streaming, search

Reach/Frequecy

Tracking & Exposure

Effectiveness

Sales Results/clickthrough/referrals

ATTITUDINAL:

Awareness, preference, customer sat

IMPACT


19

37 Countries Processed and Reported Separately…


20

…but data collected from more than 171 countries


comScore Media Metrix Methodology: the Building Blocks

How do we establish the universe?

How do we acquire respondents?

How do we collect the raw data?

How do we know who is using the

machine?

How do we correct for bias?

(weighting and projection)


comScore Media Metrix Methodology: the Building Blocks

How do we establish the universe?

How do we acquire respondents?

How do we collect the raw data?

How do we know who is using the

machine?

How do we minimize bias?

(weighting and projection)

•Enumeration •Online recruitment

•The cproxy •Biometrics

•The Calibration Sample

•The Client Focused Dictionary


Enumeration

� Accurate estimates of the size and composition of the target populations are

critical to audience measurement. This field of research is called enumeration

research.

� comScore devotes significant effort to researching and developing these

estimates.

� Enumeration research enables comScore to accurately estimate the target

population sizes for the countries we track.

� comScore enumeration sources vary by market.

– comScore surveys: In certain countries we conduct our own surveys to estimate the

size of our target population.

– Other sources: Otherwise we use publicly available, third-party studies for our

enumeration process.

comScore Universe (Outside US and CA): Persons aged 15+ who accessed the

Internet from either a home or a work computer in the past 30 days.


24

Enumeration Sources for Asia Pacific Markets

� Following are the enumeration sources for countries in Asia Pacific

market, reported by comScore:

Country Enumeration Source

Australia Australian Bureau of Statistics

China China Internet Network Information Center, Intermedia

Hong Kong Annual insertion in Synovate omnibus

India Internet and Mobile Association of India

Japan Bi-annual insertion in Ipsos omnibus, n=1000

MalaysiaComputer Industry Almanac, Malaysian Commumications and Multimedia Commission

survey

New Zealand Stats New Zealand, Computer Industry Almanac

Singapore Singapore Infocomm Develoment Authority

South Korea Insertions in Ipsos omnibus through 2007, National Internet Development Agency of Korea

Taiwan Insertion in TNS omnibus 2007, Taiwan Network Information Center

The Enumeration sources might change as comScore continuously monitors various sources

available in a market to identify the most reliable source for universe estimates.


25

Recruitment

� The goal of comScore recruitment

activities is to recruit a representative

sample from the target universe

� Broad set of value propositions to

appear to the diverse online

population

� comScore recruits panelists

exclusively online, in two ways:

– The Affiliate Program (“banners-plus-

Permission Research”)

– Third Party Application Providers (TAP)


26

The Affiliate Program

� Affiliate program

– Affiliate network comprised of web

entities which meet comScore's

quality criteria

– On these sites, panelists are recruited

via banner ads

– Appeals are targeted through a broad

array of smaller web entities

– Respondents are directed to our

online intake entity

www.permissionresearch.com


Value Propositions

A sampling of the many value propositions offered….


28

Privacy and Disclosure

Potential members

are informed that

they would be

joining an online

market research

panel that tracks

their online

browsing and

purchasing

behavior.

Potential members

are informed of the

company’s

commitment to

protect the privacy

of members.


29

Collect User Demographics; Double Opt-in

The Privacy

Statement & User

License

Agreement

informs users that

they would be

participating in

market research

program involving

passive tracking

of their online

browsing &

purchasing

behavior AND

requires their

affirmative

consent.

Registrant

provides own

demographic

information --

including birth

date


30

Collect Demographics of Other HH Members

After installation

of software, user

asked to provide

household

information


31

comScore Proprietary Collection Software

Browser

CProxy

Collection

Server

Collection

Server

HTTP

Website

HTTP

Website

HTTPS

Website

HTTPS

Website

Panelist’s PC

Streaming

Site

Streaming

Site

■ comScore captures:

– url

– Engagement (active versus passive)

– Keystroke and mouse activity and intensity

– Information passing between the user and the

Internet entity

– Application installation and usage (Excel, Word

etc.)

■ Data capture tiers

– HTTP: comScore can capture just about

anything over the HTTP protocol (including

HTTP and HTTPS)

– Proprietary Technology for real time

measurement of advanced protocols like

streaming

– Measurement of AOL proprietary and IM

engagement

comScore’s cproxy software agent allows us to “see” user activity at the machine-

or screen-side (user experience as opposed to site-centric)


32

Who’s Using the Machine?

� In the digital age, tracking the behavior of machines is

easier than ever before

� Moving from machines (who don’t buy products or view

advertisements) to people still poses unique challenges

� In panel measurement, the ideal is passive observation

over an extended time period

� One approach: use “who are you?” pop-ups

– Requiring respondents to self-identify each user session

(similar to traditional media construct used in TV People

Meter)

– Intrusive

– Fatiguing

– Biasing

� Intrusiveness drives turnover to unmanageable levels


33

Passive Biometric Observation

� comScore develops a unique biometric signature for each panelist

– Respondents passively self-identify through their online behavior

Fill out forms, email/account IDs, etc.

– User Demographic Reporting (UDR): a proprietary, patented technology

that creates biometric “fingerprints” for each person using the machine

Based on keyboard and (with UDR v2) mouse behaviors

� UDR is used to passively identify users

– For a given session, UDR matches the user’s biometric signature to stored

signatures for each machine user

� Machines do not make Media Metrix In-tab until behavior can be attributed

from the beginning of the reporting month

– Media Metrix qualification rules require that at least 90% of a machine’s

usage be attributed to a person for that machine to make monthly in-tab

Activity not attributed to a user will cause the machine to fall out of in-tab for the reporting

month


Persons Under Measurement (PUM)

� comScore PUM is persons with at least one computer being actively

measured by comScore within the previous 30 day period.

34

Country People Under Measurement

(PUM)

Australia 30,000

China 50,000

Hong Kong 5,000

India 100,000

Japan 10,000

Malaysia 30,000

New Zealand 5,000

Singapore 10,000

South Korea 5,000

Taiwan 5,000

Above are the PUM numbers for Asia Pacific Market


35

Stratification and Projection

Consistent set of stratification variables:

■ Age

■ Gender

■ Duration Categories

■ Anglophone Cap*

*In certain markets where English is not the primary language, Anglophone Caps are applied to minimize the bias in

weights induced by panelists with a Heavy English bias in their browsing behavior.


Client Focused

Dictionary (CFD)

Client Focused

Dictionary (CFD)

Proprietary and Confidential Do not distribute without written permission from comScore

comScore’s Client Focused Dictionary

110+ Categories & Sub-

Categories

110+ Categories & Sub-

Categories6-tier reporting structure6-tier reporting structure

PROPERTYPROPERTY

MEDIA TITLEMEDIA TITLE

CHANNELCHANNEL

SUB-CHANNELSUB-CHANNEL

GROUPGROUP

SUB-GROUPSUB-GROUP

TelecomsTelecoms

TravelTravel

FinanceFinance

AutomotiveAutomotive

Ad-NetworksAd-Networks

etcetc

Time WarnerTime Warner

AOLAOL

AOL Ent.AOL Ent.

AOL MusicAOL Music

AOL RadioAOL Radio

AOL Radio

Streaming URL’s

AOL Radio

Streaming URL’s


Core Reports

• Key Measures

• Demographic Profile

• Digital Calculator

• Source/Loss

• Media Trend

• % Change Media Trend

• Target Trend

• % Change Target Trend

• Cross Visiting

• Instant Messenger Report

comScore Media Metrix Core Reports

AUDIENCE MEASUREMENT

& MEDIA PLANNING

Summary Reports

• Monthly Flash

• Media Dashboard

• Dictionary Definition

Other Tools

• Reach & Frequency

• Brand Metrix

• qSearch

• Segment Metrix

• Mobile Metrix (1H ‘09)


Regional Overview

38

Audience: All Persons 15+ at Home/Work

Locations

Total Internet

Total

Unique

Visitors

(000)

Total

Minutes

(MM)

Total

Pages

Viewed

(MM)

Average

Usage

Days per

Visitor

Average

Minutes

per

Visitor

Average

Pages per

Visitor

Total Visits

(000)

Average

Minutes

per Visit

Average

Visits

per

Visitor

Asia Pacific 446,496 446,961 751,681 12.0 1,001.0 1,684 16,360,948 27.3 36.6

Europe 312,493 441,968 781,528 15.7 1,414.3 2,501 14,899,425 29.7 47.7

North America 187,679 386,548 551,209 22.7 2,059.6 2,937 13,468,197 28.7 71.8

Latin America 83,912 129,119 163,261 13.4 1,538.7 1,946 3,513,594 36.7 41.9

Middle East - Africa 77,451 127,325 145,687 13.3 1,643.9 1,881 3,485,231 36.5 45.0


Asia Pacific – Top Properties Report

39

Audience: All Persons 15+ at Asia-Pac Home/Work

Locations

Total

Unique

Visitors

(000)

% Reach

Total

Minutes

(MM)

Total

Pages

Viewed

(MM)

Average

Minutes

per

Visitor

Average

Pages

per

Visitor

Total

Visits

(000)

Average

Minutes

per Visit

Average

Visits

per

Visitor

Total Internet : Total Audience 446,496 100.0 446,961 751,681 1,001.0 1,684 16,360,948 27.3 36.6

Top 100 Properties N/A N/A ... ... ... ... ... ... ...

Google Sites 275,085 61.6 30,383 49,714 110.5 181 4,557,566 6.7 16.6

Yahoo! Sites 227,250 50.9 47,350 49,241 208.4 217 4,081,676 11.6 18.0

Microsoft Sites 205,488 46.0 36,923 14,085 179.7 69 2,189,382 16.9 10.7

TENCENT Inc. 153,458 34.4 26,820 33,966 174.8 221 3,618,394 7.4 23.6

Baidu.com Inc. 146,483 32.8 8,294 19,366 56.6 132 1,711,533 4.8 11.7

SINA Corporation 109,398 24.5 5,061 5,938 46.3 54 855,398 5.9 7.8

Sohu.com Inc. 79,484 17.8 3,594 4,893 45.2 62 463,551 7.8 5.8

Wikimedia Foundation Sites 76,024 17.0 901 905 11.9 12 272,434 3.3 3.6

Alibaba.com Corporation 75,471 16.9 4,506 10,187 59.7 135 469,810 9.6 6.2

NetEase.com Inc. 74,981 16.8 3,375 5,242 45.0 70 587,640 5.7 7.8

Xunlei Networking 68,206 15.3 3,730 3,911 54.7 57 404,005 9.2 5.9

CBS Interactive 67,055 15.0 610 999 9.1 15 146,185 4.2 2.2

FACEBOOK.COM 55,960 12.5 8,717 11,381 155.8 203 856,153 10.2 15.3

Tudou Sites 55,069 12.3 1,501 1,371 27.3 25 244,442 6.1 4.4

Youku 53,085 11.9 2,463 1,965 46.4 37 254,507 9.7 4.8

Amazon Sites 52,431 11.7 475 927 9.1 18 151,141 3.1 2.9

Apple Inc. 44,836 10.0 241 315 5.4 7 77,470 3.1 1.7

eBay 42,742 9.6 1,394 2,586 32.6 60 130,348 10.7 3.1

AOL LLC 40,609 9.1 832 1,393 20.5 34 129,218 6.4 3.2

Rakuten Inc 37,636 8.4 1,488 3,204 39.5 85 269,904 5.5 7.2


Top Vietnamese Site for Asia Pac Users

40


Key Takeaways


Differences

Panel Server Side

� Person demographics

� 360-degree view of customer

� Better duration and usage

measurement

� UVs not affected by bots or cookie

deletion

� Cross-site usage

� Audience overlap

� Difficult to game

� However:

� Universe limited to H&W

� Niche audiences difficult to

measure

� Census: Visibility into total universe

tonnage (outside panel universe)

� More timely reporting

� However:

• Requires industry participation

• No person demographics

• Affected by bot traffic

• UV measurement inaccurate due

to cookie blocking or deletion

• No standardization around page

counting


Different needs

Panel Audience Measurement

� Methodology: Recruits

representative users to be on panel

and continually tracks & reports

online behavior

� Primary Uses: Understanding the

MARKET, media planning,

demographic profile

� Providers: comScore, Nielsen

Site Centric Analytics

� Methodology: Tools/codes on

publishers’ site that tracks visitors’

behavior as they navigate through

the site.

� Primary Uses: Understanding YOUR

visitors behavior, Site optimization,

campaign tracking and reporting

� Providers: Omniture, Google

Analytics & many, many others

43


Thank You



Understanding Online Measurement: From Panel

Audience Measure to Website Analytics

Saigon Digital Marketing Conference

23rd May 2009



online audience measurement: from panel measurement to website

Documents