department of surgery and cancer imperial college london 20 may 2014 norman fenton queen mary...
TRANSCRIPT
![Page 1: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/1.jpg)
Department of Surgery and CancerImperial College London
20 May 2014
Norman Fenton
Queen Mary University of London and
Agena Ltd
Improved Medical Risk Assessment and Decision-making
with Bayesian Networks
![Page 2: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/2.jpg)
Overview
• Why Bayes?• Why Bayesian networks?• Why NOT learn the models from data only?• Case study• Challenges and conclusions
![Page 3: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/3.jpg)
1. WHY BAYES?
![Page 4: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/4.jpg)
The Harvard ProblemOne in a thousand people has a prevalence for a
particular heart disease. A test to detect this disease has:• 100% sensitivity• 95% specificity If a randomly selected person tests positive what is the probability that the person actually has the disease?
![Page 5: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/5.jpg)
Bayes Theorem
E(evidence)
We now get some evidence E.
H (hypothesis)
We have a hypothesis H with prior probability P(H)
We know P(E|H) but we want the posterior P(H|E)
P(H|E) = P(E|H)*P(H) P(E)
P(E|H)*P(H)P(E|H)*P(H) + P(E|not H)*P(not H)
=
1*1/1000
1*1/1000+ 5/100*999/1000P(H|E) = =
0.001
0.001 + 0.049950.0196
Waste of time showing this to most people!!!
![Page 6: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/6.jpg)
Slide 6
Imagine 100,000people
![Page 7: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/7.jpg)
Slide 7
Out of whom100 has thedisease
![Page 8: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/8.jpg)
Slide 8
But about 5% of theremaining99900 peoplewithout thedisease testpositive.That is 4995 people
![Page 9: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/9.jpg)
Slide 9
So 100 out of 5095 who testpositiveactually havethe disease
That’s justunder 2%
That’s very different fromthe 95% assumed by most medics
![Page 10: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/10.jpg)
Total people100,000
1/1000
999/1000
Have the disease100
Don’t have the disease
99,900
So 100 out of 5,095who test positive actuallyhave the disease, i.e. under 2%
Test positive100
Test negative0
Test positive4,995
Test negative94,905
100%
0%
5%
95%
![Page 11: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/11.jpg)
2. WHY BAYESIAN NETWORKS?
![Page 12: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/12.jpg)
A Simple Bayesian Network
![Page 13: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/13.jpg)
..but here is a typical
causal model
Calculations from first principles are
infeasible and incomprehensible
![Page 14: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/14.jpg)
Actual model in medical negligence case
This model already reaches limit of comprehensibility for
manual calculations and event trees
• MRA• CA
• Ischaemic• Small aneurysm• Large aneurysm• CSP
![Page 15: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/15.jpg)
Detected by Test9,900
Undetected by Test100
Detected by Test90
Undetected by Test10
Detected by Test0
Undetected by Test10,000
Die from burst/bleeding
Die from CSP
99%
1%
90%
10%
50%
0%
100%
2%
2%
CA Test PathwayCause of Palsy Test Result Outcome Deaths
2
0
5000
5002TOTAL
= 1.495%
1
14,952 out of 1,000,000 give risk
Stroke
Strokes
Don’t die
99
Stroke
Stroke
Die from burst/bleeding
Don’t die Stroke
Don’t die Stroke
1%
1%
1%
1%
1%
50%
98%
98%
99
2
981
1
1
0
10 0 0
5000
500050
1%Stroke
50
97999799
9950
Total people1,000,000
Large9,900
Small100
CSP10,000
Others (ischaemic)980,000
1%
1%
1%
98%
Aneurysm10,000
99%
![Page 16: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/16.jpg)
Total people1,000,000
Large9,900
Small100
CSP10,000
Others (ischaemic)980,000
Detected by Test9,405
Undetected by Test495
Detected by Test50
Undetected by Test50
Detected by Test9,000
Undetected by Test1000
Die from burst/bleeding
Die from burst/bleeding
Die from CSP
Die from CSP
1%
1%
1%
98%
95%
5%
50%
50%
50%
90%
10%
2%
2%
20%
MRA Test PathwayCause of Palsy Test Result Outcome Deaths
10
1
1800
500
2311TOTAL
= 0.2311%
0
0
0
2311 out of 1,000,000 give risk
Aneurysm10,000
99%
![Page 17: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/17.jpg)
Much better solution…use a Bayesian Network tool
![Page 18: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/18.jpg)
Computation for Catheter Angiogram
Mean:9950
Mean:5002
![Page 19: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/19.jpg)
Computation for MRA Scan
Mean:0
Mean:2311
![Page 20: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/20.jpg)
The Calculator Analogy
![Page 21: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/21.jpg)
No need for p-tests or classical confidence intervals
• Drug “Precision” weight loss: Everyone in trial lost between 4.5 and 5.5 pounds
• Drug “Oomph” weight loss: Everyone in trial lost between 10 and 30 pounds
• Which drug can we ‘accept’, i.e. reject null hypothesis of ‘no weight loss’?
• Classical stats provides nonsensical answers
![Page 22: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/22.jpg)
No need for p-tests or classical confidence intervals
![Page 23: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/23.jpg)
3. WHY NOT LEARN THE MODELS FROM DATA ONLY?
![Page 24: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/24.jpg)
A typical data-driven study
Age Delay in arrival
Injurytype
Brain scanresult
Arterialpressure
Pupildilation
Outcome (death y/n)
17 25 A N L Y N
39 20 B N M Y N
23 65 A N L N Y
21 80 C Y H Y N
68 20 B Y M Y N
22 30 A N M N Y
… … … .. … …
![Page 25: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/25.jpg)
Delay in arrival
Injurytype
Brain scanresult Arterial
pressure
Pupildilation
Age
Outcome
A typical data-driven study
Purely data driven machine learning algorithms will be inaccurate and produce counterintuitive results e.g. outcome more likely to be OK in the worst scenarios
![Page 26: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/26.jpg)
Delay in arrival
Injurytype
Brain scanresult Arterial
pressure
Pupildilation
Age
Causal model with intervention
Dangerlevel
Outcome
TREATMENT
..crucial variables missing from the data
![Page 27: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/27.jpg)
Determining drug effectiveness
![Page 28: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/28.jpg)
Basic results for drug effectiveness
Drug AThe mean financial benefit is $4156
Drug BThe mean financial benefit is $2777
Ban drug B?
![Page 29: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/29.jpg)
Model with latent variable (same data)
Note that most patients in the sample had minor
case of the condition
…and most patients were given drug A
![Page 30: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/30.jpg)
Results with 'Patient condition' major
Drug B30% positive outcome.The mean financial benefit is $1000
Drug AOnly 10% positive outcome.The mean financial benefit is $400
![Page 31: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/31.jpg)
OK, so we might need expert judgment when we have missing data, but with good experimental design and lots of good quality data we can surely remove dependency on experts ……
![Page 32: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/32.jpg)
A machine learning fableA and B are two medical conditions very well known to doctors Bill and Ludmila. These conditions are pretty rare (both have an incidence of about one in 1,000 people). There is a third medical condition C (whose name is “FiroziliRalitNoNeOba”) that Bill has heard the name of, but knows nothing about. But Bill has heard that patients with either A or B usually also have C. Bill has a massive database of 600,000 people with the details of which conditions they have.
![Page 33: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/33.jpg)
Bill’s dataPatient number A B C 1 No No No 2 No No No 3 Yes No Yes 4 No No No 5 No No No 6 No No No 7 Yes No Yes 8 No Yes Yes 9 No No No 10 No No No 11 No No No 12 No Yes Yes 13 No No No 14 No No No …. … … …. … … 600,000 No No No
![Page 34: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/34.jpg)
Bill’s machine learning mate FredCan use this database to ‘discover’ the underlying causal model (Bayesian Network) relating A, B, and C. But Ludmila says she knows the correct model without data:
Fred warns against this
She also “knows” the probability tables
![Page 35: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/35.jpg)
Fred’s learnt model
• Ludmilla disagrees with the last column of table C • Fred: “Not enough data for that”• Bill: “…why can’t we simply conclude that C must be true when
both A and B are?”
600 out of 600,000 have condition A
600 out of 600,000 have condition B
Every single person with condition A also has C and every single person with B also has C.
![Page 36: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/36.jpg)
Ludmilla’s knowledge
• The name of Condition C - FiroziliRalitNoNeOba - is actually a Russian word.
• Its literal translation is:– ‘A person suffering from either Firoz or Ralit but
not both’. – ‘Firoz’ is the Russian word for condition A and
‘Ralit’ is the Russian word for condition B.”
![Page 37: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/37.jpg)
Moral of the story
• Sometimes you have to trust experts to provide more informed quantitative judgement than you will get from data alone.
• Even really big datasets will be insufficient for some very small problems.
• Trusting the expert can save you a whole load of unnecessary data-collection and machine learning effort.
![Page 38: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/38.jpg)
4. CASE STUDY
![Page 39: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/39.jpg)
Trauma Care Case Study• QM RIM Group
– William Marsh– Barbaros Yet
• The Royal London Hospital– Mr Zane Perkins– Mr Nigel Tai– ACIT Data
• US Army Institute of Surgical Research– Lower Extremity Injury
DataYet, B., Perkins Z., Fenton, N.E., Tai, N., Marsh, W., "Not Just Data: A Method for Improving Prediction with Knowledge", Journal of Biomedical Informatics, 2014 Apr;48:28-37
![Page 40: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/40.jpg)
BN v MESS Score
• Prediction: coagulopathy, death (c.f. GCS, TRISS)• Flexible inputs• Patient’s physiological state
– Causal modelling: informed by knowledge
How the BN Model Differs
![Page 41: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/41.jpg)
Life Saving: Prediction of Physiological Disorders
![Page 42: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/42.jpg)
Limb Saving: Prediction of Limb Viability
![Page 43: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/43.jpg)
www.traumamodels.com
![Page 44: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/44.jpg)
![Page 45: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/45.jpg)
![Page 46: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/46.jpg)
5. CHALLENGES AND CONCLUSIONS
![Page 47: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/47.jpg)
Challenges• Apparent paradox on using experts• Expert systems have a bad reputation• Resistance to subjective priors• Building new large-scale BN models, especially
with minimal data• Interacting with large-scale BN models• Explaining the results of BN models
BAYES-KNOWLEDGE (Effective Bayesian Modelling with Knowledge Before Data)www.eecs.qmul.ac.uk/~norman/projects/B_Knowledge.html
![Page 48: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/48.jpg)
Conclusions (1)
• Purely data driven approaches using Machine learning and statistics DO NOT WORK
• At best captures what did happen Vs what would have happened
• Need to move to data + knowledge approach• BNs provide the key
![Page 49: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/49.jpg)
Conclusions (2): BN Benefits
• Data + knowledge• Models uncertainty and causality• Predictions and diagnosis• Avoid medical statistics fixation on p-values
and confidence intervals• Incorporate qualitative and quantitative
variables• Identify causal effects without RCTs• New generation expert systems
![Page 50: Department of Surgery and Cancer Imperial College London 20 May 2014 Norman Fenton Queen Mary University of London and Agena Ltd Improved Medical Risk](https://reader038.vdocuments.mx/reader038/viewer/2022103101/5697c0021a28abf838cc3018/html5/thumbnails/50.jpg)
Blatant Plug for Book
CRC Press, ISBN: 9781439809105 , ISBN 10: 1439809100