Concept Testing for 2011 …and Beyond

Twitter: @PeanutLabsMRFollow us at #PLConcept

Today’s Agenda:• reveal how prediction markets work;

• reveal why prediction markets are more cost effective than traditional methods;

• help you to determine when to utilize a prediction market for your research; and

• demonstrate why prediction markets deliver accurate concept tests more quickly than traditional methods.

John Barrett• Joined Infosurv in 2010 and brings over 20 years of experience in

marketing research, consultative sales, and business development.• Has extensive experience in the Marketing Research Industry in providing

strategic insights and unique solutions to business challenges for his clients.

• John’s experience spans multiple verticals and industries including, CPG, pharmaceuticals, tel-com, retail, hospitality, financial services, consumer electronics, technology, and apparel.

• John holds a B.S. from Northeastern University in Business Administration and has done Master’s work at Rutgers University.

• He lives in the Atlanta area with his wife and three children and is active in the community as a Little League Coach and a volunteer with Junior Achievement.

Tapping the Wisdom of Crowds

A Comparison Case Study on Prediction Markets for Concept Testing

Infosurv Overview• Full-service online research firm focused on

innovation• Core service is online survey design, administration, &

analysis

• Diverse and cutting-edge research services • Employee, customer, & B2B/B2C market research

surveys• Infosurv Concept Exchange (iCE) prediction market

system

• Strong industry reputation• Founded in 1998 • 300+ clients worldwide including many Fortune 500's & major

governmental agencies

What if you could predict the future?

6

• New Product Concepts

• Advertising Concepts

• Packaging & Design Concepts

• SKU Optimization

7

large groups of people are smarter than an elite few, no matter how brilliant—better at solving problems, fostering innovation, coming to wise decisions, even predicting the future.

The Wisdom of Crowds

What is a Prediction Market?

8

• Market for predicting future outcomes– Traders invest in outcomes– Weight their decision with currency allocation– Rewarded for accurate predictions

• Think

Real World Prediction Markets

9

How Do They Compare?

10

Survey

•Targeted•What would you do?•Dozens of metrics•Flat incentive

•3-4 weeks

Prediction Market

•General Population•What would they do?•Most important number•Variable incentive based on performance•3-4 days

Client Screening Program

• Methodology– Mall Intercept

– Screen for target population•Consider sufferer or user

– Screen for 2 – 4 categories per quarter•Each respondent evaluates a MAXIMUM of 8 concepts across 2 categories

– Sample size is approximately 250 per category

11

Client Screening Program

• Question Areas– Purchase Intent– Uniqueness– Value– For Top 3 Ideas

• Competitive Strength• Immediacy of purchase• Feature/Benefit most interested in• Uniqueness of feature/benefit above

12

Prediction Markets for Concept Testing: How They Work

13

• Traders Get Virtual Dollars

• Each Concept Has a Starting Price

• Prices Fluctuate as Trading Occurs

• Price Movements are Analyzed

14

• Methodology– Tested 9 concepts in two categories; 4 in health, 5 in beauty– Separate prediction market for each category– One control concept used per category to gauge absolute concept strength– General population sample; self-selection for market participation– Total of ~200 traders per market

iCE Methodology

15

Results for Health Category

Volume Weighted Average Price

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176 183 190$15.00

$20.00

$25.00

$30.00

$35.00

$40.00

Concept A

Concept B

Concept C

Control Concept

VW

AP

# of traders

CPG Prediction Market Results Compared to Survey Data - Health*

16

Concepts VWAP**iCE

Index SIiCE

RankStrength

Index Rank

Concept A $36.41 146 102 1 1

Control Concept $22.94 92 100 2 2

Concept B $20.72 83 75 3 4

Concept C $19.04 76 85 4 3

**Volume-Weighted Average Price

Open-Ended Comments:Winning Concepts - Health

Concept A (Winning Concept)

# of Responses

% of Traders*

Long lasting 68 53%

Would sell better/appeal to consumers 16 13%

Works quickly 9 7%

Good concept/product/idea 9 7%

I would use it 9 7%

Price/Value 7 5%

Good/Popular/Well-known brand 6 5%

Flavor/taste 6 5%

Packaging 6 5%

17

Concept CONTROL (Second Place Concept)

# of Responses

% of Traders*

Effectiveness 60 58%

Good concept/product/idea 7 7%

Works quickly 7 7%

Would sell better/appeal to consumers 7 7%

Long lasting 6 6%

Price/Value 6 6%

Not specified 5 5%

* Note: Traders could provide more than one answer. Therefore, percentages may add up to more than 100%.

Demographics: Health Market

18

• The iCE Health Market skews younger than the survey—40% of survey respondents were under age 45, while 56% of iCE participants were under 40.

• The iCE market also relied more heavily on female respondents (62%), while the survey was half male/half female.

CPG Prediction Market Results Compared to Survey Data - Beauty*

19

Concepts VWAP**iCE

Index SIiCE

RankStrength

Index Rank

Concept D $33.48 167 158 1 1

Concept A $23.87 119 74 2 3

Control Concept $17.49 87 100 3 2

Concept B $12.66 63 58 4 5

Concept C $11.97 60 68 5 4

*Relative Study Investment: Prediction Market = 50% of traditional survey

**Volume-Weighted Average Price

Demographics: Beauty Market

20

• The iCE market included female respondents (46%), while the survey was exclusively males (100%).

21

“Excellent way to judge a product. Makes one think of the best logic and back it up since money is on the line”

“I loved it. You should have more of these to do. They’re fun, and helps you to learn more about stock marketing trading.”

“I loved the idea! It was a super fun way to preview new products and get feedback on what other people think of the idea of those products”

Trader Quotes

22

• In both categories, the iCE prediction market successfully predicted the most preferred concept and the same directional rank order of the other concepts, with minor exceptions.

• The study was completed in less than two business days and utilized a general population sample. By comparison, the traditional survey data was obtained and analyzed over a period of weeks and relied upon a targeted sample with incidence rates as low as 7%.

• Over 70% of traders described the process using words such as “fun, interesting, cool, love it”.

• The client estimates that using prediction markets as a substitute for traditional methods would cut concept screening costs. .

Conclusions & Project Notes

What does this mean for researchers?

• Prediction markets are powerful alternatives to surveys for concept screening

• Incentivizing thoughtful, measured responses is possible

• Your customers are not always the best predictor of their own behavior

• Other applications for prediction markets and crowdsourcing in MR?

23

Questions?

Contact:

John Barrett1-404-846-6878John.Barrett@infosurv.com

Q & A Session

We welcome any questions you may have regarding the content of today’s Webinar.

Special thank you to our presenter!

Thank you for joining us!

The slide deck along with a recording of today’s presentation will be available for download via our

website. We will be sending all attendees a link to theslide deck as soon as it is available.