Unparalleled Insight into Consumer Behavior

The “Professional Respondent” Problem

In Online Survey Panels Today

Gian M. Fulgoni

MRA Annual Conference

June, 2005

comScore Networks, Inc.

Founded in 1999

Corporate headquarters: Reston, VA

Offices in Chicago, NYC, San Francisco, Seattle, Toronto and London

300 full-time employees

80 employed in sales/servicing survey-based research

Experienced senior leadership team with an outstanding record of innovation in the market research industry

Created the only continuous survey panel that also passively captures all web-wide behavioral and transaction data for a sample of 3.5 million people

Global access to survey 10 Million people, with 5 Million resident in the US.

More than 500 clients

Recognized For Our Accomplishments

America’s Fastest Growing Private Companies

comScore ranked 97th among the nation’s fastest growing privately held firms. November 2004

Top 100 Innovative Companies

Named one of the 100 companies that will shape the next year in technology. December 2004

Fifth Fastest Growing Research Firm

Currently ranked the 30th largest U.S. research firm – up ten spots from previous year.June 2004

World’s Largest Database

Grand prize winner for developing the largest NT decision support warehouse.December 2003

Honomichl Top 50

Marketing NewsMarketing News

Rising Star

Named one of only three “Rising Stars” among Virginia-based technology companies.October 2004

High Tech Award

Recognized as a technology leader for successfully developing innovative products.November 2004

Online Surveys:Many Positives … But Also Some Emerging Concerns

• Custom survey research is increasingly moving online because of the benefits of faster turnaround and lower cost

• This is creating an apparently insatiable demand for online samples

• But, problems of over-fishing and declining response rates are raising concerns about respondent quality

A Possible Cause of the Problem:

Multiple Suppliers Using the Same Recruiting Site

Other Suppliers Are Using the Same Recruiting Sites Favored By their Competitors

The comScore Panel

• comScore has recruited an opt-in consumer panel that has been validated to be representative of the online population and projectable to the total U.S. population.

• The comScore panel consists of millions of people who have given comScore explicit permission to passively measure all of their online activities:– Validated in comparison to Dept. of Commerce sales data– Validated in comparison to individual site server log data– Validated in comparison to offline surveys

• Panelists are recruited across thousands of sites not used by other panel suppliers and they do not have to be willing to answer surveys to be accepted to the panel

How The comScore Panel Benefits You

A higher quality panel that delivers more value to your end clients:comScore is the ONLY panel that can:1. Screen out professional respondents.2. Truly validate the identity of the respondent.3. Contextually pop surveys anywhere on the internet based on actual behavior.4. Append surveys with online behavioral data.5. Append surveys with other profile and offline information.

This enables your company to:1. Deliver better quality, accuracy, validation, and insights. 2. Have the assurance and confidence that your results and advice are based on sound and projectable information.

comScore’s Customer Knowledge Platform:

A 360° View of Millions of People:Observed Behavior and Survey Measurement

Designed to be projectable to the total U.S. population Ernst & Young certified for information privacy & security

DEMOGRAPHICS• Self-reported• Appended (e.g. Axiom) • Individual & Household Level • Technology Ownership (e.g. PDA, cell)

WEB VISITING & VIEWING• Web Site/Page Click Streams • HTML & Proprietary Content• Search Engine Queries• Product & Service Research

ATTITUDES• General• Behavior-specific • Awareness & Intent • New Product Concepts, etc.

ONLINE TRANSACTIONS• Purchases/Subscriptions • Applications/Configurations• Value/Price Paid • Shipping & Handling • Promotion Incentives

OFFLINE PURCHASING• Third Party Sources: Cars, Houses, etc • Purchase Surveys• Supermarket Shopping Data • Credit Card Purchases

MARKETING STIMULI• Banner Ads• Referrals

Measurement Capabilities Of the comScore Panel

• comScore’s technology captures all panelists’ online activity, including the details of all surveys taken.

• All demographics (including name and address) are verified

• Using the comScore panel, it’s possible to measure the number of surveys taken by the members of any survey panel, encompassing surveys from all panels to which they belong.

The “Professional Survey Respondent”:Alive and Well Online

• comScore research shows that:

– Only 0.25% of all Internet households account for 30% of all surveys taken

– These heavy survey responders each took an average of 80 surveys per quarter in 2004 – that’s about 1 a day!

– Some, but not all, of the leading panels contain too many of these “professional respondents”

– On average, a member of each of the leading panels belongs to at least 8 other panels

Some, But Not All, Leading Panels Are Biased Towards “Heavy Survey Respondents”

(Data Shows % of Each Panel Completing Specified Number of Surveys From All Suppliers in a Three Month Period)

A Leading panel Other Leading Panel4.66%

3.12%

3.28%

3.32%

3.24%

2.99%

2.86%

3.05%

3.20%

2.81%

3.22%

2.92%

2.89%

2.50%

2.85%

2.70%

2.49%

2.60%

2.56%

1.63%

2.47%

1.75%

2.18%

1.69%

2.14%

1.98%

1.59%

0.90%

0.88%

25.54%

Surveys comScore

1 25.62%

2 15.17%

3 10.97%

4 8.50%

5 7.46%

6 5.50%

7 4.43%

8 4.16%

9 4.12%

10 3.24%

11 2.48%

12 1.83%

13 1.70%

14 1.22%

15 0.82%

16 0.87%

17 0.68%

18 0.39%

19 0.28%

20 0.15%

21 0.08%

22 0.16%

23 0.08%

24 0.00%

25 0.00%

26 0.00%

27 0.00%

28 0.00%

29 0.11%

30+ 0.00%

8.03%

6.15%

5.23%

5.33%

5.24%

5.50%

5.31%

5.58%

5.17%

5.40%

5.43%

4.10%

4.48%

3.74%

3.53%

3.95%

2.78%

2.38%

2.40%

2.36%

1.45%

1.39%

1.06%

0.69%

1.15%

0.30%

0.39%

0.24%

0.08%

1.15%

The “Professional Survey Respondent”:Some important skews to consider when doing CPG research

comScore research shows that while heavy survey respondents are “normal” in many ways, there are important differences that can create problems in certain surveys. Compared to the average person, professional survey respondents…– Are significantly less likely to be employed full time and are less well educated– Are significantly more likely to be influenced by advertising – Shop much more frequently at grocery stores but are less likely to be warehouse/club

store shoppers – Are not as likely to consider themselves to be impulse shoppers – Are significantly more likely to use coupons when they shop– Are significantly more likely to think that buying food and/or household products with

a brand name that you can trust is important– Underestimate how many survey panels they belong to, as well as grossly

underestimate how many surveys they actually complete in an average month

But, professional respondents are also similar to the general population in some ways

• Spend similar amounts of time grocery shopping’

• Spend similar amounts of money on groceries

• As likely to use a list for groceries

• As likely to have a budget for groceries and stick to it

• Were as likely to vote in 2004 election – and who they voted for was similar

• Are similar in age

• Similar household size and composition

• Similar residence profiles

• As likely to think of themselves as:– Politically active– Spiritual– Creative– Organized– Comfortable w/change– Action-oriented– Positive/hopeful about US Economy– Concerned about environment

• Participate in many activities w/similar frequency:– Exercise– Go to movies– Read a book– Eat out at a restaurant– Go to religious services– Go out with friends

The Screen-out Challenge

• comScore research shows that professional respondents:– Underestimate how many survey panels they belong to

– Grossly underestimate how many surveys they actually complete in an average month (or they don’t want you to know!)

• This makes it very difficult to screen out professional respondents simply by asking them how many surveys they’ve taken.

Some Ways to Minimize the Inclusion of Professional Survey Respondents

in an Online Survey Panel

• Avoid recruiting panelists at sites favored by other panel suppliers

• Don’t recruit only those panelists who want to participate in continuous survey programs:– Use large numbers of “occasional” survey responders also

• Limit the aggregate financial incentive you offer panelists by limiting the number of surveys you invite them to complete:– And be sure to take into consideration that your panelists are

likely to also be compensated by their membership in other survey panels

Advantages of comScore’s Survey Panel

Unsurpassed accuracy, quality and panel hygiene- ARF-endorsed sampling methodology- Random online recruiting integrated with “RDD calibration sample” avoids the

bias of recruiting at only a few large sites- Patent-pending electronic monitoring of online behavior allows confirmation of

respondent demographics- comScore’s ability to measure households’ participation in all survey programs

permits the identification and exclusion (as required) of “professional respondents”

“Contextual pop” survey based on actual behavior- Increases accuracy of screening by reducing reliance on respondent memory

and minimizing recall errors- Permits real time surveying of consumer behavior - Address the latency issue

Gian M. FulgoniChairman

comScore Networks, Inc.

Gfulgoni@comscore.com

312 775 6481