1340 keynote minkowski_using our laptop
TRANSCRIPT
PA
GE
1
October 30, 2017
Julia Minkowski Principal, Fraud Analytics, Signifyd
Real-Time Fraud Detection:
Strategies for Speed and Actionability
PA
GE
2 Agenda
Use Case:
Fraud Prevention in E-Commerce What problem should your team be solving?
Best Practices:
Turning Data Mining Strategy into Action
PA
GE
3
PA
GE
4 What is special about Fraud Prevention?
1 Fraud is performed by
organized criminal groups
using sophisticated
technologies and logistics
PA
GE
5 What is special about Fraud Prevention?
2 Hard to detect: target has low
frequency (2 in 10,000)
PA
GE
6 What is special about Fraud Prevention?
3 The cost of mistakes is very
high
60% increase
in customer
attrition if you
misclassify (False Positives)
$ Losses if
you fail to
detect fraud (False
Negatives)
PA
GE
7 What is special about Fraud Prevention?
4 The environment changes
fast, so you need to adapt
quickly
PA
GE
8 What is special about Fraud Prevention?
5 Fraud prevention is a great
field for the application of
predictive analytics
PA
GE
9 Analytics for Fraud Prevention
Explore
& Understand
Anticipate
&
Control
Monitor
1 2 3
PA
GE
11
Risk Management: Goals and Constraints
Goals
Help the merchants to expand to
more profitable markets
(international, cross-selling), while
keeping loss rates constant, and
their customers happy
Constraints
• Build a flexible system that adapts to new
fraud patterns
• Service the existing client base
• On-board new merchants
• Minimize the time that the production systems
will be off-line or reset
• Build the next-generation of strategies with
very limited resources
PA
GE
12
Data Miner Survey by Rexer Analytics
While 6 out 10 data miners report the data is available for analysis within days of capture, the time to deploy the models takes substantially longer. For 60% of the respondents, the deployment time will range between 3 weeks and 1 year.
Everyone
might forget
about
deployment –
but it is a most
important
component!
PA
GE
13
In Fraud Mitigation – Speed is the Key
How long can you wait to deploy a solution?
PA
GE
14
Evolution of Model Deployment
14
3 months to collect data, build and deploy a model
2 weeks to estimate model
1 week to install rules 4 hours to estimate a model
1-2 days to install rules
4 hours to build a model
Few hours to implement
Same day analysis and rules deployment
2014 2016 2017 2018
PA
GE
15
Agenda
Use Case:
Fraud Prevention in E-Commerce What problem should your team be solving?
Best Practices:
Turning Data Mining Strategy into Action
PA
GE
16
Best Practices in Analytics
16
Select Best Option(s)
Success Factors and Constraints
• ROI /Cost
• Profitability
• Operations
1. Identify Benefits & Constraints
Install into Production
• Run A/B testing
• Start Small and Increase Gradually
Data
Scientist
3.Turn Strategy into Action
IT
Manager
Select the Appropriate Infrastructure
• DB Architecture
• Modeling techniques
2. Develop the Strategy
Provide Actionable Insights
Estimate Impact for the Business Track Benefits and KPI
• Test Predictive Models
• Simulate scenarios (Monte Carlo) Score
models on KPI
Collect & Process Data
• Run Descriptive Analytics
• Identify patterns
Business
Manager
• Align your Team’s Incentives
Involve Key Stakeholders
PA
GE
17
Involve the Right Stakeholders
17
Business Manager
Analyst / Data Scientist IT Manager
• Preserve Service Level
Agreement
• Reduce Operational Risk
• Preserve Budget
PA
GE
18
Conflict of Interests?
18
Cannot agree on success factors?
Wonder why?
PA
GE
19
IT Manager’s Strategy
19
• Preserve Service Level Agreements (SLA)
• Stable systems
• Ease of roll-back
• Minimize Operational risk
• Control Costs
19
PA
GE
20
Analyst / Data Scientist’s Mind
20
• Estimate the Best Model Possible
• Improve Detection Rates
• Better Algorithms, Faster Hardware
• Big(ger) Data!
• Explore New Algorithms
• Put some power behind it !!
PA
GE
21
Business Manager’s Mind
21
• Maximize Productivity: Build for specific
needs
– What is the cost?
– What is the impact on customer
experience?
– Why does it take so long?
– And: Don’t talk to me in Tech-Speak !
“First we ran a chi- square test, and then we converted the categorical data
to ordinal, next we ran a logistic regression, and then we lagged the
economic data by a year…”
Source: Davenport, Tom (2013), “Keep up with your quants", Harvard Business Review, Issue Jul-Aug 2013
PA
GE
22
Communication issue
Presenting a solution What the analyst sees What the audience sees
What the audience
remembers
What the presenter
remembers Feedback on the
solution
Source: Eric Hixson, PhD, Cleveland Clinic, 2014
PA
GE
23
Managing the Quants (Tip for Managers)
23
• Define clearly the objective and constraints
• Implement SMART* goal setting
• Establish a timeline for delivery then multiply x 2
• Get familiar with basic analytics concepts
• Coursera, EDX, Lynda, TDWI
• Make sure you understand enough to explain to other
executives: you will champion this initiative and negotiate the
budgets
* SMART Goal setting involves establishing Specific, Measurable, Achievable, Realistic and Time-
targeted goals. Wikipedia, 2016
PA
GE
24
Taking Care of Business (Tip for Analysts)
24
• Communicate clearly business level information
• When and what is the expected result
• Present the key concept in 2 phrases
• Avoid technical language for communication
• If asked for more details, then present the “How”
• Provide a Business Dashboard
• Provide the $$ metrics profit/loss reduction
• Show the impact of algorithms deployed / provided
• Current vs. Historical
• Pick the right model - the model that maximizes the
ROI
Source: Davenport, Tom (2013), “Keep up with your quants", Harvard Business Review, Issue Jul-Aug 2013
PA
GE
25
Tracking Performance: Dashboard
25
Our dashboards tracked the key performance metrics:
• Historical Trends for Fraud Rates and $ Losses (Business KPI)
• Percentage of Transfers affected by Risk Mitigation (Business KPI)
• % of population affected by policy and % of fraud prevented (KPI for Analytics)
• Fraud detection rates for models and rules installed (KPI for Analytics)
PA
GE
26
Super-Leader characteristics
26
Source: Alexander Linden, Key Trends and Emerging Technologies in Advanced Analytics, Gartner 2014
PA
GE
27
Key Takeaways On Fraud Analysis and Modeling
When dealing with fraud, the speed to implement a new model is the most important factor
Improvements in accuracy may be lost due to delays in deployment; systems with fast turnaround have better ROI than complex algorithms with long implementation times
Turning Strategy into Action
Involving the key stakeholders early in the process maximizes your chance for success. Once you have aligned the incentives for the team, selecting the appropriate techniques and infrastructure becomes much simpler
It is crucial for business managers to correctly define the problems and objectives, asking the right questions and learning the basic analytical concepts
For data scientists it is important to select their models and projects based on the expected business impact and to translate their findings into the relevant metrics
27
1
2
1
2
3
PA
GE
28
THANK YOU!
28