experiences on incremental response modellingsas … · different approaches for incremental...

EXPERIENCES ON INCREMENTAL RESPONSEMODELLINGSAS User Forum Finland

Helsinki May 24th 2017

Jaakko Riihimäki

Senior Data Analyst, Customer Insights & Analysis

Telia Finland Oyj

OUTLINE

• Telia Company

• Incremental Response Modelling

in Marketing

• Optimising Outbound Marketing

Campaign

• References

TELIA COMPANY

TELIA COMPANY PROVIDES

COMMUNICATION SERVICES

HELPING MILLIONS OF PEOPLE

TO BE CONNECTED AND

COMMUNICATE, DO BUSINESS

AND BE ENTERTAINED. BY

DOING THAT WE FULFILL OUR

PURPOSE TO BRING THE

WORLD CLOSER – ON THE

CUSTOMER’S TERMS

TELIA COMPANY IS THE LEADING NEW GENERATION OPERATOR IN THE NORDICS AND BALTICS…

SEK BILLION

84.2 NET SALES

25.8 EBITDA

15 CAPEX

21,000 EMPLOYEES

FOCUS ON NORDICS &

BALTICS

December 31 2016 figures refer to continuing operations, i.e. the group excluding the former segment region Eurasia

INCREMENTAL RESPONSEMODELLINGIN MARKETING

EXAMPLE: MARKETING ACTION FOR PRODUCTUPDATE

• Given observations from a marketing action, build a conditional probability model for an

update. Response variable: update / no update

P(update|𝐗, 𝐲) 𝐗 = explanatory variables, 𝐲 = response variable

• For example, in binary classification problem: logistic regression, a neural network model with

a logistic output layer and a prior that favours smooth solutions, a Gaussian process with a

probit likelihood function…

Marketing

action

Probability

model:

P(update|𝐗, 𝐲)

Targeted

marketing

action with

the model

EXAMPLE: MARKETING ACTION FOR PRODUCTUPDATE

• Given observations from a marketing action, build a conditional probability model for an

update. Response variable: update / no update

P(update|𝐗, 𝐲) 𝐗 = explanatory variables, 𝐲 = response variable

• For example, in binary classification problem: logistic regression, a neural network model with

a logistic output layer and a prior that favours smooth solutions, a Gaussian process with a

probit likelihood function…

• Instead of modelling the probability for an update, we should model the probability of an

update because of the marketing action!

Marketing

action

Probability

model:

P(update|𝐗, 𝐲)

Targeted

marketing

action with

the model

DESIGN OF EXPERIMENT FOR INCREMENTALIMPACT

• Given observations from a marketing action and from a control group, build a conditional

probability model for an update given the marketing action. Response variable: update / no

update conditioned to the marketing action.

P(update|𝐗, 𝐲, 𝐭) 𝐗 = explanatory variables

𝐲 = response variable

𝐭 = indicator whether the marketing action was received or not

Marketing

actionIncremental

response

model:

P(update|𝐗, 𝐲, 𝐭)

Targeted

marketing

action with

the modelControl

group

INCREMENTAL RESPONSE MODELLING

Different approaches for incremental response modelling (or uplift modelling)



• Single probability model P update 𝐗, 𝐲, 𝐭

• Simulate the incremental impact of a marketing action using the model P update 𝐗, 𝐲, 𝐭





• Two separate probability models: one for the target group and one for the control group

• Subtract the predicted probabilities obtained using the control group from the predicted probabilities obtained

using the target group: P𝑡 update 𝐗, 𝐲 − P𝑐 update 𝐗, 𝐲








• Algorithms:

• For example in binary classification where 𝑦 ∈ {0,1} instead of a response variable 𝐲 use a transformed variable 𝐳that consists of responses 𝐲 from the target group unchanged and responses from the control group inverted 1 −𝐲. With certain assumptions: P𝑡 update 𝐗, 𝐲 − P𝑐 update 𝐗, 𝐲 = 2P "update due to a marketing action" 𝐗, 𝒛 − 1








• Algorithms:

• For example in binary classification where 𝑦 ∈ {0,1} instead of a response variable 𝐲 use a transformed variable 𝐳that consists of responses 𝐲 from the target group unchanged and responses from the control group inverted 1 −𝐲. With certain assumptions: P𝑡 update 𝐗, 𝐲 − P𝑐 update 𝐗, 𝐲 = 2P "update due to a marketing action" 𝐗, 𝒛 − 1

• Semi-supervised style solutions where in addition to a single uplift model, two additional models for the target

and control groups are built. An algorithm defines the training observations to each model and information flows

via training samples between the models.

EVALUATING INCREMENTAL IMPACT

• To evaluate incremental impact (or uplift), we need to measure the number of updates both

from the target and control groups but for a single observation only one of them is known




• One solution is to measure updates at different times but the measurement times can affect

the number of updates

• Need to build a model for ”adjusting” the effect of different measurement times




• One solution is to measure updates at different times but the measurement times can affect

the number of updates

• Need to build a model for ”adjusting” the effect of different measurement times

• Alternative solution: evaluate incremental impact for a group of observations

• The assumption: Similarly modelled obervations behave similarly

• Example: Uplift% for the highest decile = Update% for the observations ranked at the highest decile in the target

group – Update% for the observations ranked at the highest decile in the control group

• Cumulative uplift% can be computed at each decile

EXAMPLE: AREA UNDER THE UPLIFT CURVE (AUUC)

• A point at 100% gives the total

uplift% in success probability if

the whole target group is

contacted

• A diagonal line connecting

points corresponding to 0% and

100% describes the random

selection for the marketing

action

• One measure to summarise the

model performance: the Area

Under the Uplift Curve (AUUC)

MODEL SELECTION USING AUUC

• Model performance can be

summarised using the Area Under

the Uplift Curve (AUUC)

• Models can be compared, for

example, by computing the

differences in AUUC

• To assess the model performance

with respect to different data

partitions, use cross-validation

OPTIMISINGOUTBOUNDMARKETINGCAMPAIGN

EXAMPLE: OUTBOUND MARKETING

• Outbound marketing campaigns

can be optimised with incremental

response modelling

• Relevant communication to

customers

• Reduce unnecessary marketing

costs

EXAMPLE: OPTIMISING MARKETING CAMPAIGN

• In addition to incremental response

modelling, the marketing profits and

costs can be included into the model

• Return on Investment (ROI) can be

used as a measure to summarise a

marketing campaign

• ROI can be optimised, for example,

with respect to the number of

customers contacted given a follow-up

time period (as illustrated)

REFERENCES

THANK YOU!

experiences on incremental response modellingsas … · different approaches for incremental...

Documents