search marketing cannibalization - merkle inc. analytics...• ~11,000 visits lost per week when...
TRANSCRIPT
Search Marketing Cannibalization
Analytical Techniques to measure PPC and
Organic interaction
2
Search Overview
How People Use Search Engines
• Navigational
• Research
• Health/Medical
• Directions
3
• News
• Shopping
• Advice
Leveraging Search Demand Data
• General Trends
• Estimating effects of other media campaigns
• Measuring Brand Equity
• Aiding in creating Advertising Value Propositions
• Competitive Advantages/Disadvantages
• Shifts in Consumer Preferences
• Attitudinal Measurement
– http://www.people.fas.harvard.edu/~sstephen/papers/RacialAnimusAndVotingSethStephensDavidowitz.pdf
4
Search Demand Data Examples
5
Source: Google Insights for Search
Media Effect Example: godaddy
Consumer Preferences/Advertising Propositions
Search Demand: Competitive Measurement
6
macys dillards
7
Paid vs. Organic Search
Cannibalization
Cannibalization History
8
• Dates back to the on-going struggle between Finance &
Marketing
Why should we pay for something we already
get for free?!
We need to be where our market is,
regardless of the cost!
Have no fear! Digital Analytics
are here!
• As you may have guessed, PPC does take some clicks that otherwise would have gone to SEO, a process known as cannibalization
• However, PPC typically provides incremental traffic and conversions above and beyond the cannibalized traffic
Measurement Methods
• Econometric modeling techniques
– Measuring the expected vs. actual traffic
– Incorporating pulse effects into model
– Leveraged when no control group was implemented (Paid search entirely turned off)
– Typical scenario occurring in the real world
• “In Market” Testing
– Randomly assign markets to treatment (PPC) and control (no PPC)
– T-tests/ANOVA vs. ANCOVA
• Google’s study leveraging Bayesian techniques: http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/37161.pdf
9
Important Data Elements for Analysis
• Organic ranking a very important component in the analysis:
• If Organic position is “below the fold” then analysis will commonly prove that PPC is highly incremental
– Google’s study did not segment based on organic position
– Historical organic ranking information is commonly unavailable
• Therefore, analysis typically focuses on “Brand” keywords as opposed to “Non-Brand” because organizations typically rank in 1st position for their brand terms
10
11
ARIMA Forecasting
ARIMA Forecasting Process
• Familiarize yourself with the Time Series
– Plot the data!
• Identifying Stationarity
– Augmented Dickey Fuller Tests
– Differencing
• Leveraging Autocorrelation and Partial Autocorrelation Correlelograms to identify the appropriate process
– Common plots to identify Autoregressive vs. Moving Average components
• Evaluating Model Performance
– Holdout time periods
– Common diagnostic measures (AIC, SBC, MAPE)
12
Common SAS Syntax
PROC ARIMA DATA=inputDSN;
IDENTIFY VAR=depvar(differencing options) crosscorr=(exogenous variables) stationarity=(adf) esacf;
ESTIMATE p= q= input=(exogenous variables) method=ml;
FORECAST back=n lead=n id=datevar out=outputDSN;
RUN;
QUIT;
13
Plotting the Time Series
14
• Plotting the time series reveals that there appears to be an upward trend in the data
• Additionally, there appears to be external shocks and/or seasonal trends
220,000
230,000
240,000
250,000
260,000
270,000
280,000
290,000
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Total Visits by Week
2008 2009 2010 2011
Shock/Seasonal effect
Shock/Seasonal effect
Plotting the Time Series
15
• On average, Paid traffic appears to contribute ~70K visits per week (~30%) of the overall traffic
• When Paid was paused, there was an immediate decline – However, visits did increase in the 3rd /4th week
– But then decline again the 5th/6th week
0
50,000
100,000
150,000
200,000
250,000
300,000
1/1
2/2
00
82
/12
/20
08
3/1
2/2
00
84
/12
/20
08
5/1
2/2
00
86
/12
/20
08
7/1
2/2
00
88
/12
/20
08
9/1
2/2
00
81
0/1
2/2
008
11
/12
/20
081
2/1
2/2
008
1/1
2/2
00
92
/12
/20
09
3/1
2/2
00
94
/12
/20
09
5/1
2/2
00
96
/12
/20
09
7/1
2/2
00
98
/12
/20
09
9/1
2/2
00
91
0/1
2/2
009
11
/12
/20
091
2/1
2/2
009
1/1
2/2
01
02
/12
/20
10
3/1
2/2
01
04
/12
/20
10
5/1
2/2
01
06
/12
/20
10
7/1
2/2
01
08
/12
/20
10
9/1
2/2
01
01
0/1
2/2
010
11
/12
/20
101
2/1
2/2
010
1/1
2/2
01
12
/12
/20
11
3/1
2/2
01
14
/12
/20
11
5/1
2/2
01
16
/12
/20
11
Trend of Paid and Organic
Visits_Paid Visits_Organic
Plotting the Time Series
16
• Leverage an econometrics model to forecast the expected sales while controlling for trend and media influences
0
20
40
60
80
100
120
140
160
200,000
210,000
220,000
230,000
240,000
250,000
260,000
270,000
280,000
290,000
1/1
2/2
00
82
/12
/20
08
3/1
2/2
00
84
/12
/20
08
5/1
2/2
00
86
/12
/20
08
7/1
2/2
00
88
/12
/20
08
9/1
2/2
00
81
0/1
2/2
008
11
/12
/20
081
2/1
2/2
008
1/1
2/2
00
92
/12
/20
09
3/1
2/2
00
94
/12
/20
09
5/1
2/2
00
96
/12
/20
09
7/1
2/2
00
98
/12
/20
09
9/1
2/2
00
91
0/1
2/2
009
11
/12
/20
091
2/1
2/2
009
1/1
2/2
01
02
/12
/20
10
3/1
2/2
01
04
/12
/20
10
5/1
2/2
01
06
/12
/20
10
7/1
2/2
01
08
/12
/20
10
9/1
2/2
01
01
0/1
2/2
010
11
/12
/20
101
2/1
2/2
010
1/1
2/2
01
12
/12
/20
11
3/1
2/2
01
14
/12
/20
11
5/1
2/2
01
16
/12
/20
11
Trend of Visits w/Media
Media Visits_Actual
Identification
17
• Augmented Dickey Fuller Test suggests that no differencing is required
• ACF/PACF suggests both Autoregressive and Moving Average elements
• Incorporate transfer functions to control for exogenous variables of media/trend – Forecast the expected sales prior to the test
– Additionally, rebuild model after test w/ the intervention effect identifying the period in which Paid Search is paused
Model Parameters Pre-test/Forecast
18
• Built ARIMA(X) model pre test to establish the expected overall traffic from both Paid and Organic search prior to the test – All parameter estimates significant
– Trend (Num1) and Media (Num2) are both significant
Model Parameters Pre-test/Forecast
19
• Forecast accuracy very high as the Forecast vs. Actual prior to the forecast (4/9)
• ~11,000 visits lost per week when comparing the Actual vs. Forecast during the “dark” period – Or roughly 16% of the Average Paid Traffic
0
20
40
60
80
100
200000
210000
220000
230000
240000
250000
260000
270000
1/1/2011 2/1/2011 3/1/2011 4/1/2011 5/1/2011
Forecast vs. ActualPre-Test Model
media Visits_Actual Forecast for Visits_Actual
Actual Results w/Intervention Effect
20
• Subsequent to the “dark” period, the model was rebuilt with an intervention effect for the period of paused paid activity
– Parameter for the intervention effect (0/1) confirms the comparison of the Forecast vs. Actual
– Significant with a parameter estimate of -11,363
• Note additional models were built and evaluated with log transformations on the visits and media
– Models results were similar (intervention effect interpretation is slightly different)
21
Applying Learnings
Implementing Learnings
22
• Is the incremental paid traffic worth the cost? – CPCs can be adjusted upwards by taking into consideration the
cannibalization rate, for example:
– Adj CPC=Actual CPC/Incremental Rate or
– Adj CPC=Actual CPC/(1-Cannibalization Rate)
Bottom Line: Implement learnings into your bidding algorithms!
23
Appendix
Common ACF/PACF Correlogram Plots
24
-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
1 2 3 4 5 6 7 8 9 10
ACF of AR(1) phi>0
ACF of AR(1) phi>0
-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
1 2 3 4 5 6 7 8 9 10
ACF of AR(1) phi<0
ACF of AR(1) phi<0
Common ACF/PACF Correlogram Plots
25
-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
1 2 3 4 5 6 7 8 9 10
ACF of MA(1) theta>0
ACF of MA(1) theta>0
-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
1 2 3 4 5 6 7 8 9 10
PACF of MA(1) theta>0
PACF of MA(1) theta>0
-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
1 2 3 4 5 6 7 8 9 10
ACF of MA(2) theta1, theta2<0
ACF of MA(2) theta1, theta2<0
-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0
1 2 3 4 5 6 7 8 9 10
PACF of MA(2) theta1,theta2<0
PACF of MA(2) theta1,theta2<0
ACF/PACF Cheat Sheet
26
Process ACF PACF
ARIMA(0,0,0) no significant spikes no significant spikes
ARIMA(0,1,0) d=1 slow attenuation 1 spike at order of differencing
ARIMA(1,0,0) phi>0 exponential decay, positive spikes 1 positive spike at lag 1
ARIMA(1,0,0) phi<0 oscillating decay, begins with negative spike 1 negative spike at lag 1
ARIMA(2,0,0) phi>0 exponential decay, positive spikes 2 positive spikes at lags 1 and 2
ARIMA(2,0,0) phi1<0 phi2>0 oscillating exponential decay 1 negative spike at lag 1, 1 positive spike at lag 2
ARIMA(0,0,1) theta>0 1 negative spike at lag 1 exponential decay of negative spikes
ARIMA(0,0,1) theta<0 1 positive spike at lag 1 oscillating decay of positive and negative spikes
ARIMA(0,0,2) theta1,theta2>0 2 negative spikes at lags 1 and 2 exponential decay of negative spikes
ARIMA(0,0,2) theta1,theta2<0 2 positive spikes at lags 1 and 2 oscillating decay of positive and negative spikes
ARIMA(1,0,1) phi>0 theta>0 exponential decay of positive spikes exponential decay of positive spikes
ARIMA(1,0,1) phi>0 theta<0 exponential decay of positive spikes oscillating decay of positive and negative spikes
ARIMA(1,0,1) phi<0 theta>0 oscillating decay exponential decay of negative spikes
ARIMA(1,0,1) phi<0 theta<0 oscillating decay of negative and positive spikes oscillating decay of positive and negative spikes
