machine learning and fairness in commercial …...machine learning and fairness in commercial...
TRANSCRIPT
![Page 1: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/1.jpg)
Machine learning and fairness in commercial insuranceInsurance Data Science ConferenceLondon, UK
July 2018
![Page 2: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/2.jpg)
p2
Cytora uses artificial intelligence to improve risk targeting, selection, and pricing for commercial insurance
About Cytora
![Page 3: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/3.jpg)
p3
- Credit scoring- Insurance- Airport security- Crowd monitoring- Reoffending rates- Advertising
Decisions that personally affect us are increasingly data-driven
Automation
Intuitive decision
Risk scores and metrics
Automated system
![Page 4: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/4.jpg)
Defining fairness
![Page 5: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/5.jpg)
p5
Fairness means treating individuals from different groups equally
Protected attributes
![Page 6: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/6.jpg)
p6
Fairness through unawareness is not sufficient to guarantee equal treatment for individuals in protected groups
![Page 7: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/7.jpg)
p7
Data
- Inherent data biases- Reasoned vetting of variables- True measures of latent risk- Measure the protected attribute
Careful consideration is necessary when designing decision systems
Modelling
- Quantify feature contributions- Tune for fairness- Bias in, bias out
![Page 8: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/8.jpg)
p8
Protected attributes encoded in “harmless” rating factors
Kusner, Loftus, Russell, Silva (2018)
"Counterfactual Fairness"
Sex Driving aggression
AccidentCar colour
Protected attr
Rating factor
Latent
Target
![Page 9: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/9.jpg)
Strategies for fairer pricing in insurance
![Page 10: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/10.jpg)
p10
1. Observe relevant rating factors
Sex Driving aggression
AccidentTelemetrics
Protected attr
Rating factor
Latent
Target
![Page 11: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/11.jpg)
p11
- Profit (accuracy)- False positive rate, equal opportunity (FP/N)- False negative rate (FN/P)- Equalised odds (FPR & FNR)- Equality of opportunity (FPR)- Calibration (true probabilities)- Demographic parity
2. Adjust premiums to optimise metrics of fairness
Berk, Heidari, Jabbari, Kearns, Roth (2017) "Fairness in Criminal Justice Risk Assessments: The State of the Art"
Kleinberg, Mullainathan, Raghavan (2016) "Inherent Trade-Offs in the Fair Determination of Risk Scores
TN FP
FN TP
Predicted
Act
ual
LossNo loss
No
loss
Loss
Confusion matrix per protected group
![Page 12: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/12.jpg)
p12
- Structural models - Kilbertes, et. al. (2018) "Avoiding discrimination through causal reasoning" - Kusner, Loftus, Russell, Silva (2018) "Counterfactual Fairness"
- Penalised / constrained loss functions - Zafar, et. al. (2017) "Fairness Beyond Disparate Treatment & Disparate Impact" - Zhao, et. al. (2017) "Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints"
- Model inspection - Tan, Caruana, Hooker, Lou (2018) "Detecting Bias in Black-Box Models Using Transparent Model Distillation"
3. Design and train algorithms with fairness baked-in
![Page 13: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/13.jpg)
p13
Rating factor: Cuisine type = Krusty Burgers
Protected attribute: Shelbyville or Springfield resident
Solutions:1. Observe management quality, menu, online reviews...2. Geographic analysis of offered premium (adjust?)3. Use fairness-calibrated algos
Example: Restaurant shutdown
![Page 14: Machine learning and fairness in commercial …...Machine learning and fairness in commercial insurance Insurance Data Science Conference London, UK July 2018 p2 Cytora uses artificial](https://reader034.vdocuments.mx/reader034/viewer/2022043019/5f3ba119c34a673881491030/html5/thumbnails/14.jpg)
p14
Data-driven modelling and machine learning can improve fairness by
1) Find better approximations of latent risk2) Quantify effects on domain-specific fairness metrics3) Calibrate decision making process to optimise fairness
Summary