market intelligence session 12 contadina, regression, database marketing
TRANSCRIPT
Market Intelligence Session 12
Contadina, Regression, Database marketing
2
Agenda
• Contadina Case• Regression
– Multicollinearity– Database Marketing
3
Agenda
• Contadina Case• Regression
– Multicollinearity– Database Marketing
Contadina STM
4
• Trial and Repeat – Users and Non-Users– Pizza and Toppings
Contadina Epilogue
What happened?
• Cunliffe decided to launch Contadina Pizza without further testing.
• Sales (volume, trial, & repeat), share, and profit performance were below expectations (& $45 million cut-off):
1991(Q3&Q4) 1992 1993 $7,097 $21,919 $17,187
7
What went wrong
• Inaccurate inputs– Penetration 10%, not 24%– Market support $11M instead of $18M
• Bad luck: price & promotion war b/w Dominos and Pizza Hut, delivery now same price as Contadina
• Change in product– Topping spoilage 60%, changed product to cheese and
cheese & pepperoni only – not what was rated in BASES!
8
Agenda
• Contadina Case• Regression
– Multicollinearity– Database Marketing
Regression - Basics
• Terminology– In simple regression with a single variable, we get
a zero-order effect (full effect) because we do not control for anything else.
– In multiple regression we technically speak of the coefficients as partial effects because it is the change in Y from a change in X holding everything else in the regression constant
10
Regression – Homework assignment
Regression• In marketing, approach to regression depends
on what we are trying to do– PROMOTION ANALYSIS, PROFITABILITY ANALYSIS
• Uncover the marginal effects of a marketing action on customer response – e.g., coupons on sales
– DATABASE MARKETING • Forecast customer response – not particularly interested in
the marginal effects of any action but very interested in having an overall model that can predict the dependent variable (Y) very well
11
12
Part 1: Regression for promotion analysis
• Run multiple regression with outcome variable (often sales) as Y and marketing actions (e.g., 4 P’s) as predictors
Multicollinearity in Multiple Regression
• Multicollinearity: when 2 or more of your predictor variables are highly correlated with each other
13
14
Detecting MC
• Correlation matrix among explanatory variables: values (off-diagonal elements) of > .5 are often interpreted as indicating MC.
• Large changes in the estimated regression coefficients when a predictor variable is added or deleted
• Insignificant regression coefficients for the affected variables, but a significant F-test for overall regression
• Multivariable regression finds an insignificant coefficient of a particular predictor, yet a simple linear regression of the same predictor yields a significant coefficient
• Variance inflation factor (VIF) > 5
Coefficientsa
40.680 1.154 35.266 .000
7.278 1.923 .167 3.786 .000 1.000 1.000
25.288 1.841 13.734 .000
-4.875 2.119 -.112 -2.301 .022 .683 1.465
5.455 .536 .496 10.179 .000 .683 1.465
(Constant)
H=Any Children Under 6?
(Constant)
H=Any Children Under 6?
C=Household Size
Model1
2
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig. Tolerance VIF
Collinearity Statistics
Dependent Variable: B=Weekly Food Expenditure Dollarsa.
Diagnostics
• Tolerance=
• VIF =Variance Inflation Factor
= 1/Tolerance
2| 12
1 XXR
Rule of thumb: MC problem if VIF > 5
16
Remedies for MC• Leave model as is
– Significance of coefficients may be reduced– Can still apply model to new data if predictors follow same pattern of
MC • Drop predictor
– Remaining predictors will be more significant but will also be biased high• Obtain more data
– More data can produce more precise parameter estimates (with lower standard errors)
• Aggregate predictors– Factor analysis or PCA to reduce factors – Create index
17
Remedies for MC• Leave model as is
– Significance of coefficients may be reduced– Can still apply model to new data if predictors follow same pattern of
MC • Drop predictor
– Remaining predictors will be more significant but will also be biased high• Obtain more data
– More data can produce more precise parameter estimates (with lower standard errors)
• Aggregate predictors– Factor analysis or PCA to reduce factors – Create index
Doritos Example
• Identification of Promotion Effects– Effect of Price Promotions on Sales of XL size
• IRI Dataset (Market Level, Weekly Data)• Sales Models for XL Size
– Effects of own price (own price effect) & price of other sizes (cross price effects) on sales of XL size
– Multicollinearity exists: we’ll look at options
18
19
Promotion Example: Symphony-IRI data of Dorito Weekly Sales
SM XL 2XL 3XL
Sizes:
20
Promotion Example: Symphony-IRI data of Dorito Weekly Sales
SM XL 2XL 3XL
Sizes:
21
Correlations
1.000 .018 .449** .502** .085
. .854 .000 .000 .391
104 104 104 104 104
.018 1.000 .091 .120 -.801**
.854 . .356 .224 .000
104 104 104 104 104
.449** .091 1.000 .950** .107
.000 .356 . .000 .279
104 104 104 104 104
.502** .120 .950** 1.000 .067
.000 .224 .000 . .500
104 104 104 104 104
.085 -.801** .107 .067 1.000
.391 .000 .279 .500 .
104 104 104 104 104
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Lbs Extra LargeSize 9 Oz $2.19
AveragePrice Per
PoundSmall Size
AveragePrice PerPound XL
Size
AveragePrice Per
Pound2XL Size
AveragePrice Per
Pound3XL Size
Lbs ExtraLarge Size9 Oz $2.19
Correlation is s ignificant at the 0.01 level (2-tailed).**.
22
Correlations
1.000 .018 .449** .502** .085
. .854 .000 .000 .391
104 104 104 104 104
.018 1.000 .091 .120 -.801**
.854 . .356 .224 .000
104 104 104 104 104
.449** .091 1.000 .950** .107
.000 .356 . .000 .279
104 104 104 104 104
.502** .120 .950** 1.000 .067
.000 .224 .000 . .500
104 104 104 104 104
.085 -.801** .107 .067 1.000
.391 .000 .279 .500 .
104 104 104 104 104
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Lbs Extra LargeSize 9 Oz $2.19
AveragePrice Per
PoundSmall Size
AveragePrice PerPound XL
Size
AveragePrice Per
Pound2XL Size
AveragePrice Per
Pound3XL Size
Lbs ExtraLarge Size9 Oz $2.19
Correlation is s ignificant at the 0.01 level (2-tailed).**.
Doritos Regression Equation
• Dependent Variable = Sales (lbs.) of XL size• Model 1: Overloaded
– Independent Variables: Price XL, Price SM, Price 2XL, Price 3XL
• Model 2: Omitted variable
23
SalesXLt a b1PSM _ t b2PXL _ t b3P2XL _ t b4P3XL _ t
24
Which variable to drop?
• Which is more correlated with DV?• Which is more correlated with remaining
predictors?• Which is less important variable to you?
25
Correlations
1.000 .018 .449** .502** .085
. .854 .000 .000 .391
104 104 104 104 104
.018 1.000 .091 .120 -.801**
.854 . .356 .224 .000
104 104 104 104 104
.449** .091 1.000 .950** .107
.000 .356 . .000 .279
104 104 104 104 104
.502** .120 .950** 1.000 .067
.000 .224 .000 . .500
104 104 104 104 104
.085 -.801** .107 .067 1.000
.391 .000 .279 .500 .
104 104 104 104 104
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Lbs Extra LargeSize 9 Oz $2.19
AveragePrice Per
PoundSmall Size
AveragePrice PerPound XL
Size
AveragePrice Per
Pound2XL Size
AveragePrice Per
Pound3XL Size
Lbs ExtraLarge Size9 Oz $2.19
Correlation is s ignificant at the 0.01 level (2-tailed).**.
Doritos Regression Equation
• Dependent Variable = Sales (lbs.) of XL size• Model 1: Overloaded
– Independent Variables: Price XL, Price SM, Price 2XL, Price 3XL
• Model 2: Omitted variable– Independent variables: Price XL, Price SM, Price
2XL (note: NO 3XL) 26
SalesXLt a b1PSM _ t b2PXL _ t b3P2XL _ t b4P3XL _ t
SalesXLt a b1PSM _ t b2PXL _ t b3P2XL _ t
Doritos Regression Equation
• Dependent Variable = Sales (lbs.) of XL size• Model 1: Overloaded
– Independent Variables: Price XL, Price SM, Price 2XL, Price 3XL
27
SalesXLt a b1PSM _ t b2PXL _ t b3P2XL _ t b4P3XL _ t
28
Coefficientsa
316.844 2445.178 .130 .897
253.361 515.280 .033 .492 .624
-1915.729 136.477 -.813 -14.037 .000
3590.806 2495.219 .267 1.439 .153
-1413.885 2574.564 -.106 -.549 .584
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
MODEL 1: XL SALESDORITOS - PREDICTING XL SALES FROM PRICE OF SM, XL, 2XL, 3XLDoritos - Only Own XL Price Coefficient is Significant
Doritos Regression Equation
• Dependent Variable = Sales (lbs.) of XL size• Model 1: Overloaded
– Independent Variables: Price XL, Price SM, Price 2XL, Price 3XL
• Model 2: Omitted variable– Independent variables: Price XL, Price SM, Price
2XL (note: NO 3XL)
29
SalesXLt a b1PSM _ t b2PXL _ t b3P2XL _ t
30
Coefficientsa
625.958 2371.187 .264 .792
175.976 493.905 .023 .356 .722
-1924.665 135.029 -.817 -14.254 .000
2305.377 861.524 .172 2.676 .009
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
MODEL 2: XL SALESDORITOS - PREDICTING XL SALES FROM PRICE OF SM, XL, 2XL (DROP 3XL)Doritos - Own XL and 2XL Price Coefficients Significant
31
Coefficientsa
625.958 2371.187 .264 .792
175.976 493.905 .023 .356 .722
-1924.665 135.029 -.817 -14.254 .000
2305.377 861.524 .172 2.676 .009
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Coefficientsa
316.844 2445.178 .130 .897
253.361 515.280 .033 .492 .624
-1915.729 136.477 -.813 -14.037 .000
3590.806 2495.219 .267 1.439 .153
-1413.885 2574.564 -.106 -.549 .584
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Model 1 (Over-loaded)
Model 2 (omitted variable)
vs.
32
Coefficientsa
625.958 2371.187 .264 .792
175.976 493.905 .023 .356 .722
-1924.665 135.029 -.817 -14.254 .000
2305.377 861.524 .172 2.676 .009
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Coefficientsa
316.844 2445.178 .130 .897
253.361 515.280 .033 .492 .624
-1915.729 136.477 -.813 -14.037 .000
3590.806 2495.219 .267 1.439 .153
-1413.885 2574.564 -.106 -.549 .584
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Model 1 (Over-loaded)R2 = .67
Model 2 (omitted variable)R2 = .56
vs.
33
Coefficientsa
625.958 2371.187 .264 .792
175.976 493.905 .023 .356 .722
-1924.665 135.029 -.817 -14.254 .000
2305.377 861.524 .172 2.676 .009
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Coefficientsa
316.844 2445.178 .130 .897
253.361 515.280 .033 .492 .624
-1915.729 136.477 -.813 -14.037 .000
3590.806 2495.219 .267 1.439 .153
-1413.885 2574.564 -.106 -.549 .584
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Model 1 (Over-loaded)
Model 2 (omitted variable)
vs.
34
Coefficientsa
625.958 2371.187 .264 .792
175.976 493.905 .023 .356 .722
-1924.665 135.029 -.817 -14.254 .000
2305.377 861.524 .172 2.676 .009
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Coefficientsa
316.844 2445.178 .130 .897
253.361 515.280 .033 .492 .624
-1915.729 136.477 -.813 -14.037 .000
3590.806 2495.219 .267 1.439 .153
-1413.885 2574.564 -.106 -.549 .584
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Model 1 (Over-loaded)
Model 2 (omitted variable)
vs.
35
Coefficientsa
316.844 2445.178 .130 .897
253.361 515.280 .033 .492 .624
-1915.729 136.477 -.813 -14.037 .000
3590.806 2495.219 .267 1.439 .153
-1413.885 2574.564 -.106 -.549 .584
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Average Price PerPound 3XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Model 1 (Over-loaded)
Overloaded model: will provide more accurate R2, but individual predictors won’t be as significant due to inflated standard errors. Result: you’ll know how much total variance is accounted for by model but won’t know which predictors have significant effect
36
Coefficientsa
625.958 2371.187 .264 .792
175.976 493.905 .023 .356 .722
-1924.665 135.029 -.817 -14.254 .000
2305.377 861.524 .172 2.676 .009
(Constant)
Average Price PerPound Small Size
Average Price PerPound XL Size
Average Price PerPound 2XL Size
Model1
B Std. Error
UnstandardizedCoefficients
Beta
Standardized
Coefficients
t Sig.
Dependent Variable: Lbs Extra Large Size 9 Oz $2.19a.
Model 2 (omitted variable)
Omitted variable bias: R2 will be reduced so you won’t get as accurate info on total variance accounted for. Individual influential predictors will be significant but they will be biased (inflated) because they also include effect of omitted variable
Summary
• Interpretation of Regression Coefficients• Diagnose potential Multicollinearity and Omitted Variables
issues– Cost of these issues is decreased precision or bias in coefficient
estimates• Know your options. Every situation will be different.
– Are there other, less correlated, variables available as substitutes for the correlated variables?
– For strategy purposes is it acceptable to drop or combine a variable and measure a compound effect such as coupons and promotions?
– Can you create an index or aggregate correlated predictors? 37
Part 2: Regression for Database Marketing
• Form of direct marketing that uses databases of (potential) customers to generate personalized communications in order to promote a product/service• Common in financial services, telecommunications, and retail
• Database: usually name, address, transaction history (internal sales/delivery systems), or compiled list from 3rd party – Sources: charity donation forms, product warranty cards, applications
for free products, subscription forms, credit card applications, etc.• Uses Multiple Regression to predict likelihood of database of
customers purchasing each product/service– scores then used to select customers for communication
38
39
Database Marketing: Steps
• Step 1: Calibrate model• Step 2: Customer “scoring”• Step 3: Target new customers
Step 1: Calibrate model• First send an offer to a small subset of the database.
Collect Data and code response to predict response by others in database– Logistic regression model where Dependent Variable is
(0/1) binary (e.g., took offer or not)– Objective is to predict Response well (Y-hat), measured by a
high R-squared, so correlated predictors OK
40
)...( 2211 ikikiii xbxbxbafy
)...(ˆ 2211 kikiii xbxbxbafy
Step 2: Customer Scoring
• For remaining customers in database: Use the regression results to predict the probability of response to an offer– Predicted Y-hat = predicted probability of taking
offer
41
42
Step 3: Target new customers
• Make offer to all customers for whom:– (probability of response * margin) > (cost of
contact)– Anything over 0, although would like to recover
fixed costs as well as variable costs• In the long run, each individual receives
customized treatment (different set of offers)
Database Marketing Example: Rodale International
43
Database Marketing Example: Rodale International
Step 1: Calibrate model – 1 million customers in database; send offer for
handweights to a subset (5,000)– Response probability for the subset = 1.9%
• Logistic Regression Model:– Likelihood of taking offer =
Function (Age, Previous purchases, Years with Rodale, Runner’s world subscriber, household income, Payment method, Subscription offers sent last year, Sweepstake entrant, Years at current residence, Frequency of purchase, Returns, Net $ amount purchased, Recency of last purchase, etc.).
44
Database Marketing
45
1) Use sample (5,000) to build response model using regression 2) Y-hat Prob = f (a + b1(Age) + b2(#PrevBuys) + b3(Years) + b4(Runner) + …
+ bn(HHinc)) within this sample of 5,000
Prospect ID Buy Age #PrevBuys
Years with us
Runner’s World
… HH Inc
1 1 28 2 2 0 … 28k
2 0 51 5 6 1 … 49k
3 0 38 0 0 1 … 89k . . . . . .
5,000 1 71 0 4 0 … 61k
Database Marketing Example: Rodale International
Step 2: Customer scoring• 5000 customers initially contacted are small
subset of total potential customers in database
• We now want to use regression equation to create a “score” for other 995,000 customers: how likely is each to buy handweights?
46
Database Marketing3) Now predict probability each prospect 5,001 – 1,000,000 in
your database would buy, if sent the offer
47
Prospect ID Prob Buy
Age #PrevBuys
Years with us
Runner’s World
… HH Inc
5001 .40 38 3 9 1 … 85k
5002 .26 25 0 3 1 … 40k
5003 .13 43 0 5 0 … 25k . . . . . .
1,000,000 .32 25 1 1 0 … 12k
Database Marketing3) Now predict probability each prospect 5,001 – 1,000,000 in
your database would buy, if sent the offer
48
Prospect ID Prob Buy
Age #PrevBuys
Years with us
Runner’s World
… HH Inc
5001 .40 38 3 9 1 … 85k
5002 .26 25 0 3 1 … 40k
5003 .13 43 0 5 0 … 25k . . . . . .
1,000,000 .32 25 1 1 0 … 12k
These are customer “scores”
Database Marketing Example: Rodale International
Step 3: Target new customers• Determine break-even and send offer to
those whose probabilities (scores) are above that
49
Database Marketing ExampleAssume:• The margin from a prospect who accepts the offer is $2.00• It costs $0.40 to send an offer to one prospect
BEprob =
50
Prospect ID Prob Buy
Age #PrevBuys
Years with us
Runner’s World
… HH Inc
5001 .40 38 3 9 1 … 85k
5002 .26 25 0 3 1 … 40k
5003 .13 43 0 5 0 … 25k . . .
1,000,000 .32 25 1 1 0 … 12k
Database Marketing ExampleAssume:• The margin from a prospect who accepts the offer is $2.00• It costs $0.40 to send an offer to one prospect To determine the break even probability: $0.40 = Prob*$2.00
BEprob = 0.20 Send offer to prospects with scores X higher than .20
51
Prospect ID Prob Buy
Age #PrevBuys
Years with us
Runner’s World
… HH Inc
5001 .40 38 3 9 1 … 85k
5002 .26 25 0 3 1 … 40k
5003 .13 43 0 5 0 … 25k . . .
1,000,000 .32 25 1 1 0 … 12k
52
Internet and database marketing
• Internet is powerful database marketing vehicle
• 2 examples of changes internet has brought to database marketing– Customized or personalized websites for each
customer – Send offers via web (e.g., email)
53
Customized websites
• Customize and/or personalize pages to show items of interest
• Easy ROI calculation using experiment with test and control group
• 100,000 customers– 80,000 get personalized web page– 20,000 (control group) gets generic one– Can determine value in about a week
54
Example of internet marketing experiment
Number Sales Sales/Customer
Test 80,000 $2,272,000 $28.40
Control 20,000 $469,000 $23.45
Total 100,000 $2,741,000 $27.41
Cost of Personalization
($45,000) ($0.45)
Gain from Personalization
$4.95
55
Example of internet marketing experiment
Number Sales Sales/Customer
Test 80,000 $2,272,000 $28.40
Control 20,000 $469,000 $23.45
Total 100,000 $2,741,000 $27.41
Cost of Personalization
($45,000) ($0.45)
Gain from Personalization
$4.95
compare
56
Sending offers via web• Customer communications: relationship building • Low cost to send offers
– Always include web response option:
Phone
Web
$0.00 $0.50 $1.00 $1.50 $2.00 $2.50 $3.00 $3.50 $4.00 $4.50
57
Course Wrap Up
Goals of course
• Tools to reduce demand uncertainty
• Learn how to evaluate different types of research – before and after it’s done
• Act as a research provider to become a more sophisticated user
58
Tools for common marketing decisions
• Competition: Quadrant analysis, disaggregate choice modeling, perceptual mapping, conjoint, cannibalization analyses
• Positioning: disaggregate choice modeling, Quadrant analysis, perceptual maps
• Segmentation and targeting: Index #s, Interactions, a priori v. clustering methods, choosing a basis, 3 criteria, Segments of 1, (index #s, interactions, clustering, 1-to-1
• New Product Development: Focus groups, conjoint analysis, BASES, Disaggregate choice modeling, perceptual mapping
• Pricing: Conjoint analysis & Regression, experiments• Promotion Effects: Multiple regression, experiments• Product Design: Focus Groups, Conjoint• Database Marketing: Regression for customer scoring
(forecasting), customer data (e.g., loyalty) programs 59
Finally, …
• End up regularly using material from this class in your job? Please consider being a future guest speaker.
• Thank you for a great term!
60