blair hudson, innovation portfolio manager for data science, pepper money

Driving Innovationwith DataCDAO Sydney6-8 March 2017

Blair Hudson (@blairhudson)Innovation Portfolio Manager – Data SciencePepper Group Limited

Blair Hudson (@blairhudson) Copyright 2017 Pepper. Confidential.

2

Pepper Group – diverse global financial services

AustraliaMortgage lendingAuto and equipment financePersonal loansThird party servicingProperty advisory

South KoreaMortgage lendingAuto and equipment financePersonal loansDeposits

United KingdomMortgage lendingThird party servicingProperty advisory

IrelandMortgage lendingThird party servicingProperty advisory

SpainPoint of sale financeThird party servicing


3

OpportunityOpportunity InitiativeInitiative ConceptConcept PrototypePrototype PilotPilot LaunchProduction

Innovation is part of our DNA

In 2016 we defined our process to support company-wide innovation,and structured key strategic portfolios including Data Science

4

Copyright 2017 Pepper. Confidential.

How would you like to use machine learning today?


5

Reality is more like

OpportunityOpportunity Launch

Like design, the art of data science can be begin uncertain,but unravels with focus on a single outcome

Production

Reality is more like Damien Newman’s squiggle


6

So how do we turn this…

OpportunityOpportunity LaunchProduction

(Damien Newman’s squiggle)


7

…into this?

OpportunityOpportunity InitiativeInitiative ConceptConcept PrototypePrototype PilotPilot LaunchProduction

(Awesome value-driven process for Data Science innovation)

8


1. Opportunity


9

Tools for discovery and understanding

Opportunity Canvas Value Sequence


10

Value Sequence

Tools for discovery and understanding

Opportunity Canvas


11

Opportunity Canvas

Work Activities

Risks

RiskMitigators

Returns

ReturnAccelerators

BusinessFunction

Core / Support

Adjacent Functions


12

Opportunity Canvas

Work Activities

Risks

RiskMitigators

Returns

ReturnAccelerators

BusinessFunction

Core / Support

Adjacent Functions


13

Value SequenceSequence Sequence

Track

Track

Track

Track


14

Value SequenceSequence Sequence

Track

Track

Track

Track

15


2. Initiatives


16

Initiative SorterInitiative Scorecard

Tools for planning your roadmap


17

Initiative SorterInitiative Scorecard

Tools for planning your roadmap


18

DVF criteria

Feasibility

ViabilityDesirability

Is this a valid data science opportunity?Does supporting data exist already?

How long could it take?What are the expected benefits?

Is this a strategic priority?Is the use case easily understood?


19

Opportunity

Initiative Scorecard

Sequence

Function

TrackAccelerator / Mitigator

Total Score

FeasibilityDesirability Viability

Revenue / Risk Measure

Work Activity


20

Opportunity

Initiative Scorecard

Sequence

Function

TrackAccelerator / Mitigator

Total Score

FeasibilityDesirability Viability

Revenue / Risk Measure

Work Activity


21

Initiative Sorter15 14 13 12 11 10 9 8 7 6 5 4 3

Horizon 1

Horizon 2

Horizon 3

Ready Now

Needs DataF < 4

Needs TechnologyV < 4

Needs OwnerD < 4


22

Initiative Sorter15 14 13 12 11 10 9 8 7 6 5 4 3

Horizon 1

Horizon 2

Horizon 3

Ready Now

Needs DataF < 4


Needs OwnerD < 4


23

15 14 13 12 11 10 9 8 7 6 5 4 3

Horizon 1

Horizon 2

Horizon 3

Initiative Sorter

Ready Now

Needs DataF < 4


Needs OwnerD < 4

24


3. Concept


25

Thinking ahead…

Prototype ProductionPilot

Validation(Offline)

AB Test(Online)


26

Experiment Plan

Tools for conceptual (experiment) design


27

Experiment Plan

Tools for conceptual (experiment) design


28

Experiment PlanObservation

Sequence

Target

Hypothesis Test

Control

AcceptanceModel


29

Experiment PlanObservation

Sequence

Target

Hypothesis Test

Control

AcceptanceModel

30


4. Prototype


31

Tools for rapid prototyping

(This isn’t so easily summarised into simple graphical tools…)


32

Our ethos drives sustainability

Avoid technical debt at all costs

Be relentlesslyvalue driven

Walk first, then run!

33


5. Pilot


34

Pre-conditions for pilot

Before pilot (AB test), the prototype must be validated on unseen historical data,otherwise continue prototyping (or reformulate concept hypotheses)

Modelling (80%) Validation (20%)

Historical data


35

AB testing

Determine

minimum sample

for A and B

Apply model on B group

and action

Observe results in A and B

until minimums reached

Analyse results

for significa

nce

36


6. Production


37

Pre-conditions for production

Before launch, the AB test must reach statistical significance, otherwise keep piloting (or reformulate concept hypotheses)

https://xkcd.com/1478/


38

Three types of consumption

Model summaries

Batchscoring

Real-time scoring


39

If you’ve made it this far…

Domain Expertise

Computer

Science

Mathematics and Statistics

Change and

Strategy

You’ve joined the dots betweenfour key fields!


40

Thank you

[email protected]

@blairhudson

linkedin.com/in/blairhudson

Blair Hudson

mailto:[email protected]

http://www.twitter.com/blairhudson

http://www.twitter.com/blairhudson

http://www.linkedin.com/in/blairhudson

http://www.linkedin.com/in/blairhudson

Pepper Group Limited ABN 55 094 317 665. Australian Finance Services Licence 286655; Australian Credit Licence 286655 Pepper Asset Finance Pty Ltd CAN 165 183 317; Australian Credit Licence 458899 © Copyright 2016 Pepper Group Limited. All

rights reserved.


blair hudson, innovation portfolio manager for data science, pepper money

Data & Analytics