Slide 1

Government Actuary's Department18 November 2014

Norman FentonQueen Mary University of London

and Agena Ltd

Bayesian Networks forRisk Assessment

Slide 2

Outline

Overview of Bayes and Bayesian networks

Why Bayesian networks are needed for risk assessment

Examples and real applications in financial risk

Challenges and the future

Slide 3

Our book

www.BayesianRisk.com

Slide 4

Overview of Bayes and Bayesian Networks

Slide 5

A classic risk assessment problem

A particular disease has a 1 in 1000 rate of occurrence

A screening test for the disease is 100% accurate for those with the disease; 95% accurate for those without

What is the probability a person has the disease if they test positive?

Slide 6

Bayes Theorem

E(evidence)

Now get some evidence E (“test result positive”)

P(H|E) = P(E|H)*P(H) P(E)

P(E|H)*P(H)P(E|H)*P(H) + P(E|not H)*P(not H)

=

1*0.001

1*0.001 + 0.05*0.999P(H|E) = =

0.001

0.50050.02

But we want the posterior P(H|E)

H (hypothesis)

Have a prior P(H) (“person has disease”)

We know P(E|H)

Slide 7

A Classic BN

Slide 8

Bayesian Propagation

Applying Bayes theorem to update all probabilities when new evidence is entered

Intractable even for small BNs

Breakthrough in late 1980s - fast algorithms

Tools implement efficient propagation

Slide 9

A Classic BN: Marginals

Slide 10

Dyspnoea observed

Slide 11

Also non-smoker

Slide 12

Positive x-ray

Slide 13

..but recent visit to Asia

Slide 14

The power of BNs

Explicitly model causal factors

Reason from effect to cause and vice versa

‘Explaining away’

Overturn previous beliefs

Make predictions with incomplete data

Combine diverse types of evidence

Visible auditable reasoning

Can deal with high impact low probability events (we do not require massive datasets)

Slide 15

Why causal Bayesian networks are needed for risk assessment

Slide 16

2 . 1 4 4 2 4 3 . 5 5N T

Irrational for risk assessment Rational for risk assessment

Problems with regression driven ‘risk assessment’

Slide 17Slide 17

‘Standard’ definition of risk

“An event that can have negative consequences”

Measured (or even defined by):

Slide 18

..but this does not tell us tell us what we need to know

Armageddon risk: Large meteor strikes the Earth

The ‘standard approach’ makes no sense at all

Slide 19

Risk using causal analysis

A risk is an event that can be characterised by a causal chain involving (at least):

The event itself

At least one consequence event that characterises the impact

One or more trigger (i.e. initiating) events

One or more control events which may stop the trigger event from causing the risk event

One or more mitigating events which help avoid the consequence event (for risk)

Slide 20

Bayesian Net with causal view of risk

Meteor strikesEarth

Risk event

Meteor on collision course

with Earth

Trigger Blow up Meteor

Control

Build Underground

citiesMitigant

Loss of Life

Consequence

Slide 21

Examples and real applications in financial risk

Slide 22

Note that ‘common causes’ are easily modelled

Causal Risk Register

Slide 23

Assumes capital sum $100m and a 10-month loan

Expected value of resulting payment is $12m with 95% percentile at $26m

Regulator stress test: “at least 4% interest rate”

Simple stress test interest payment example

Slide 24

Expected value of resulting payment in stress testing scenario is $59m with

95% percentile at $83m

Simple stress test interest payment example

This model can be built in a couple of minutes with AgenaRisk

Slide 25

Stress testing with causal dependency

Slide 26

Stress testing with causal dependency

Slide 27

Op Risk Loss Event Model

Slide 28

Operational Risk VAR Models

Scenario dynamics

Contributing outcomes

Aggregate scenario outcome

Slide 29

Stress and Scenario ModellingPandemic

Civil Unrest

Travel Disruption

Reverse Stress

Slide 30

Business Performance

Holistic map of business enhances understanding of interrelationships between risks and provides candidate model structure

Risk Register entries help explain uncertainty associated with business processes

KPIs inform the current state of the

system

Business Performance Indicators serve as ex-post indicators, we can then use the model to explain the drivers underlying business outcomes

Slide 31

Policyholder Behaviour

Slide 32

The challenges

Slide 33

Challenge 1: Resistance to Bayes’ subjective

probabilities

“.. even if I accept the calculations are ‘correct’ I don’t accept subjective priors”

There is no such thing as a truly objective frequentist approach

Slide 34

Challenge 2: Building realistic models

Common method:

Structure and probability tables all learnt from data only (‘machine learning’)

DOES NOT WORK EVEN WHEN WE HAVE LOTS OF ‘RELEVANT’ DATA!!!!!!!!!!!!!!!

Slide 35

A typical data-driven study

Age Delay in arrival

Injurytype

Brain scanresult

Arterialpressure

Pupildilation

Outcome (death y/n)

17 25 A N L Y N

39 20 B N M Y N

23 65 A N L N Y

21 80 C Y H Y N

68 20 B Y M Y N

22 30 A N M N Y

… … … .. … …

Slide 36

Delay in arrival

Injurytype

Brain scanresult Arterial

pressure

Pupildilation

Age

Outcome

Purely data driven machine learning algorithms will be inaccurate and produce counterintuitive results e.g. outcome more likely to be OK in the worst scenarios

A typical data-driven study

Slide 37

Delay in arrival

Injurytype

Brain scanresult Arterial

pressure

Pupildilation

Age

Causal model with intervention

Dangerlevel

Outcome

TREATMENT

..crucial variables missing from the data

Slide 38

Challenge 2: Building realistic models

Need to incorporate experts judgment:

Structure informed by experts, probability tables learnt from data

Structure and tables built by experts

Fenton NE, Neil M, and Caballero JG, "Using Ranked nodes to model qualitative judgements in Bayesian Networks“, IEEE TKDE 19(10), 1420-1432, Oct 2007

Slide 39

Challenge 3: Handling continuous nodes

Static discretisation: inefficient and devastatingly inaccurate

Our developments in dynamic discretisation starting to have a revolutionary effect

Neil, M., Tailor, M., & Marquez, D. (2007). “Inference in hybrid Bayesian networks using dynamic discretization”. Statistics and Computing, 17(3), 219–233. Neil, M., Tailor, M., Marquez, D., Fenton, N. E., & Hearty, P. (2008). “Modelling dependable systems using hybrid Bayesian networks”. Reliability Engineering and System Safety, 93(7), 933–939

Slide 40

Challenge 4: Risk Aggregation

Estimate sum of a collection of financial assets or events, where each asset or event is modelled as a random variableMethods not designed to cope with the presence of Discrete Causally Connected Random VariablesSolution: Bayesian Factorization and Elimination (BFE) algorithm - exploits advances in BNs and is as accurate on conventional problems as competing methods.

Peng Lin, Martin Neil and Norman Fenton (2014). “Risk aggregation in the presence of discrete causally connected random variables”. Annals of Actuarial Science, 8, pp 298-319

Slide 41

Conclusions

Genuine risk assessment requires causal Bayesian networks

Bayesian networks now used effectively in a range of real world problems

Must involve experts and not rely only on data

No major remaining technical barrier to widespread

Slide 42

Follow up

Try the free BN software and all the models

www.AgenaRisk.com

Get the bookwww.BayesianRisk.com

Propose case study for ERC Project BAYES-KNOWLEDGEwww.eecs.qmul.ac.uk/~norman/projects/B_Knowledge.html