customized jmp analytics · customized report generator → t2 interactive data builder report...

Customized JMP Analytics: Mondelēz International’ Consumer

Test Evaluation Package

Jeff Stagg, Manager MEU Statistics

David Rose, SAS Professional Services

Tuesday, Sept. 16, 4:00 – 4:45 pm

JMP Discovery Summit 2014 SAS Headquarters, Cary, NC

2

Workflow for presentation

Order of ‘play’

• MCTE Package Overview

• Benefits

• Functionality

• Brief recap of 2012 Summit presentation

• JMP Summit 2014 – Regional consumer test

• Case Study Description

• Case Study Analysis

• Variant selection using DoE platform

• Consumer Drivers of Liking

• Customized Report Generator

• Q&A

This presentation will demonstrate ‘live’ specific features of Mondelēz International’s Multiproduct Consumer Test Evaluation (MCTE) package on a real case study

The application is too extensive to demonstrate fully, we hope that showing specific features with some interpretation will make it more meaningful & impactful for you.

Benefits of MCTE Package

• Double Efficiency

2-5 rather than 5-10 man days for analysis, interpretation and report generation.

Graphics can be customized within agreed report formats and exported into business presentation.

• Simpler, less error prone

Point & Click on JMP replaces use of several statistical software packages and self-customisation of macros for case study specifics.

• Extended user capability

Several active users per RD&Q site rather than reliance on a few company experts/ external agencies.

Local knowledge of project/process applied better to the interpretation.

• Use of best practice analysis

Customized scripts use Mondelēz International’s best statistical practices for global deployment – experts focusing more on this.

• Harmonization of Systems

Replacement of other more expensive analysis options.

Training efficiencies.

3

4

Most of the analyses described in this presentation were performed in JMP using a bespoke analysis toolkit (“MCTE”) that was written in a collaboration between Mondelēz International and SAS UK.

This application has featured twice in previous Summits (2010 & 2012). Since then its functionality has been continually improved to take full advantage of JMP v11 upgrade and broaden its capabilities to cover Mondelēz International’s best analysis practices.

MCTE application is now deployed in the form of a JMP Add-In to Mondelēz International RD&Q sites around the world, together with its own customized help file.

The MCTE Package

5

Basic data manipulation Raw Data Analysis Summarised Data Analysis

Customized report generator

Main menu

Sub menus:

The MCTE Package - Functionality

Package functionality is comprehensive; scripts are customised to Mondelēz International’s requirements for data manipulation, best practice analysis & reporting of consumer test studies

6

‘Multiple Comparisons by Subgroup’ control panel

Sub menus:

The MCTE Package – Graphics management Graphics are edited via control panel; changes to colours, text, scale, reported statistics & labels within graphics and size & layout of multiple graphics are all possible

Outcome

• Current product [237] dominates cluster 1 but is rated worst and significant lower than all other products in cluster 2

• Prototype 815 (Fluffy_F_3x5_DC) was launched to complement current product in market; having all four points of differentiation.

7

JMP Summit 2012 – Presentation Overview Business Objective

Launch a line extension chocolate tablet product

Technical approach • Use new/different process

technology to give points of difference

• Design of Experiments

Consumer Test Design

• Central location test (CLT)

• 153 consumers

• 12 products (DoE)

• Each consumer evaluates all the products

Cluster 1 (59%) Cluster 2 (41%)

All Test persons - Overall Liking

Firmness as main overall point of difference

Iterative clustering – 50 runs

Consensus clustering solution

8

JMP Summit 2014 – Case Study Description

Business Objective

Identify a product design that satisfies both regional and within country consumer acceptance; product selection weighted by country volume & margin.

Technical approach • Use new/different process technology to give points of difference from current

product

• Select 6 products that have required functionality and have point of difference

from current product and from other.

• Execute a consumer test to acquire consumer acceptability of products and use JMP MCTE package to analyze o Identify consumer drivers of liking from JAR ratings o Recommend new product design for harmonization strategy across region.

9

Consumer Test Design

• Central location test (CLT)

• 3 countries

• 100 consumers per country; pre-recruited

• 6 products; same set tested in each country

• Consumers evaluate all 6 products in one session

• Consumers rate products for

o Overall & attribute liking questions (scale 1-9)

[1=Dislike Extremely, 9=Like Extremely]

o Just About Right (JAR) attribute intensity ratings

[1= Much too Little; 3= Just Right; 5= Much too Much]

• Managed by external market research agency


10

JMP Summit 2014 – Live Demonstration

What we will show today Three new analysis procedures will be covered.

1. Candidate Selection – a novel way to select a product set for consumer test; one that has optimal coverage of target flavor space and includes required reference products. A technical, repeatable solution to an otherwise judgmental process.

2. Consumer Attribute Drivers of Liking – a novel way to summarize traditional penalty analysis results. Two new metrics of JAR ratings are introduced Facilitates simple product change discussions/ decisions with the project

stakeholders.

3. Customized Report Generator – this new MCTE capability enables user customization of statistical report contents & layout. Its introduction replaced another reporting system on a key Mondelēz International RD&Q site.

MCTE application now used on all main RD&Q sites (global deployment).

a) Penalty Impact: Drop in liking when a product is not scored ‘Just About Right’ .

b) Penalty Balance: An estimate of the opportunity to adjust a product attribute to improve its liking.

11

JMP Summit 2014 – Product Selection using DoE Platform

Product Selection Objective

Select 6 products from the 19 possible that lie within the target profile space for consumer testing – these should have optimal coverage of this space and include current product.

Technical approach

• Conduct descriptive sensory analysis (QDA) on all products using an in-house expert sensory panel (trained consumers).

• Perform principal component analysis (PCA) to generate sensory factors from 42 sensory attributes.

• Use main (3) sensory factors as covariates in DoE Custom Design to identify six products including current that will ‘best’ estimate sensory factor main effects

12

JMP Summit 2014 – Product Selection using DoE Platform

Product selection results

Free choice selection doesn’t include current product. Current product is ‘forced’ into selection by entering ‘References’ as a 4th covariate This principle can be used to force a set of required products & get an optimal selection to cover target space from the rest – ‘Forced entries’.

Naming convention: P?_Flavor:Firmer:Coarser 5 levels = Low (L): Low-Med (l): Med (M): Med-High (h): High (H)

13

The Data

The consumer test data is read into JMP from an Excel File and stored for analysis by JMP & MCTE package


MCTE

14

P5 & P4 variants perform better than Current product for all counties; and in Regional assessment, softer is rated significantly higher.

Results shown are un-weighted for country volumes & margins

Regional RGT Results - Overall liking

Naming convention: P?_Flavor:Firmer:Coarser

Note : Products joined by bars are rated statistical parity for range test applied [here Fishers LSD, α = 0.05]

Raw Data Analysis → R2 Multiple Comparisons by Subgroup JMP Summit 2014 - Case Study Analysis

Country = AAA Country = BBB Country = CCC

RGT Results - Overall liking

When 2017 projected weightings are applied P4 variant is now most liked

When 2014 current weightings are applied to centered country liking scores, P5 & P4 variants still perform better in regional assessment than Current product; P5 most liked

=> P5 variant to validation test then launch as regionally more liked and business efficiencies to harmonize.

=> Investigate softer plus coarser variant design options in further testing

Note: These results are not generated by MCTE tool as weightings are applied to country centered averages, derived from different consumer bases

15

JMP Summit 2014 - Case Study Analysis


Most

important

Least

important

JAR frequency is the % of consumer scoring attribute ‘Just About Right’ Penalty Impact is how much on average overall liking is penalized for not being ‘Just About Right’

Raw Data Analysis → RA JAR Scale tabulations

Penalty Analysis of all samples for all countries combined generates regional rank order of importance of JAR attributes

RGT Results – Drivers of Liking (DoL)

Consumer Attribute Drivers

of Liking

Penalty Impact numbers are weighted for 2014 current country volume & margin

16


Most

important

Least

important

• Penalty Balance is a measure that indicates the potential opportunity to adjust an attribute in order to improve performance [less opportunity as PB get closer to zero]

• All variant adjustments to improve performance are to correct over-delivery on attributes. Softer variant has little insight on how to improve it, whereas all other variant require texture adjustment [make softer]; higher flavor variants over- deliver on taste & sourness.

Raw Data Analysis → RA JAR Scale tabulations

Penalty Analysis across all countries combined generates regional insights for product improvements

RGT Results – Drivers of Liking

DoL

Penalty Balance numbers are weighted for 2014 current country volume & margin

17



Customized Report Generator → T1 Interactive Data Labeller

Consumer attribute ratings are assigned roles according to how they can be analysed.

Optionally full questions from consumer test questionnaire can be imported and assigned to JMP variables (column headers) . Reporting can then either use shorter or long form.

RGT Results – Report Generator setup

18


Customized Report Generator → T2 Interactive Data Builder

RGT Results – Report Generator output – default format

19


Here, default statistics are reported for a set of liking attributes then for a set of

JAR scales



Report Generator output – flexible format

20


• Products & attributes can be

selected as required.

• The order of products & attributes

entered is the order in which they

are reported.

• Grouping variables can be

specified to report data by group

level

• Default statistics are set for each

variable role; these can be

changed for customized reporting

• Switch between labels and full

questions

• Reports can be in horizontal or

vertical layout


Report Generator output – a flexible format

21


Here, report has specified order of products & attributes; showing selected

statistics for country AAA only


Q&A

Jeff Stagg, Manager MEU Statistics

David Rose, SAS Professional Services

JMP Discovery Summit 2014

SAS Headquarters, Cary, NC

22

Customized JMP Analytics: Mondelēz International’ Consumer Test

Evaluation Package

23

Consumer Test Evaluation Using a Combination of JMP® Functionality Plus Customized JSL Scripts Ticks All Efficiency and Effectiveness Boxes for Mondelēz International RD&Q

Jeff Stagg, MEU Statistics Manager, Mondelēz International David Rose, PhD, Analytics Consultant, SAS Marlow UK

Topic: JSL Application Development – Level: 3 Mondelēz International empowers its staff by developing user-friendly tools for customized data handling, calculation and reporting. JMP software houses a multiproduct consumer test evaluation (MCTE) tool used by the global Consumer Science function. Previously used for single-consumer test evaluation, the tool’s recent enhancements in functionality enable easier regional (multi-country) assessment and flexible format report generation. Approaches to product selection using standard JMP functionality and regional analysis and reporting using Mondelēz International’s MCTE tool will be demonstrated. More specifically, customized JMP scripts evaluate Consumer JAR scales that summarize attribute importance and product change opportunity. JAR scales measure levels of a product's attribute relative to a respondent's theoretical ideal level. These scales have an anchored midpoint of "just-about-right” and endpoints anchored to represent intensity levels of the attributes that are higher and lower than ideal. A user-friendly report generator is another recent development. While innovative in design to meet diverse user requirements, its success was more dependent on its capability to produce structured Microsoft Word reports. The MCTE tool has significantly reduced training and analysis time and ensured use of good statistical practices. Its success results from a close collaboration between Mondelēz International UK and SAS Professional Services in Marlow.

customized jmp analytics · customized report generator → t2 interactive data builder report...

Documents