decision modelling for n00bs
TRANSCRIPT
Clinical decision modeling for n00bs: The absolute
minimum you NEED to know
Suranga Nath Kasthurirathne
What is a Decision Model?
An intellectual template for perceiving, organizing, and managing the business logic
behind a business decision [1].
When applied to Computer science, and explained in plain English,
A computer based system that predicts an outcome or makes a decision
[1] Gottinger, H. W., & Weimann, P. (1992). Intelligent decision support systems. Decision Support Systems, 8(4), 317-332.
Why do we care?
Decision models:
• Allow us to leverage existing silos of information for valuable purposes
• Saves time
• Lets the computer (and not a human) do extremely complicated thinking
• Is very cool and science-y.
Decision models
Classification
J48
Logistic Regression
Etc. etc.
Clustering
EM
Simple Kmeans
Classification Vs. Clustering
Decision models
Classification
J48
Logistic Regression
Etc. etc.
Clustering
EM
Simple Kmeans
Classification Vs. Clustering
What we wont cover
• Clustering / unsupervised approaches
• How algorithms work
• How modeling tools work
What happens inside the reviewers brain?
Criteria 1 Criteria 2 Criteria 3 Outcome
Mark TRUE FALSE FALSE Reject
Alex TRUE TRUE FALSE Accept
Sarah TRUE TRUE TRUE Accept
The reviewers brain is trained to watch out for specific criteria.So basically….
What can Dr. Jones do wrong?
• Selects wrong criteria
• Places emphasis on the wrong criteria
• Forgets
• Makes mistakes
• And what if Dr. Jones is doing this for the first time?
How does a computer replicate this?
The computer;
• Obtains a data vector, just like our brain does
• ‘Trains’ itself on this vector (just like our brains do)
• Delivers decisions based on this knowledge (just like we do!)
Computers also share our weakness
• Over fitting
– An applicant has won 10 Nobel prizes
• Imbalance
– Applications for a postdoc position are mistakenly sent to Dr. Jones
• Missing data
– A blank doesn’t mean zero. It means, “I don’t know”
Computers have their advantages
• Feature selection
• Boosting unbalanced data
• Being very very impartial and unbiased
Since real life is always more complicated…
• No binary outcome
– Accept, reject, waitlist, no scholarship, small scholarship
• No binary criteria
– Excellent track record
– Meh track record
– Slightly impressed track record
– Incomplete
Performing decision modeling: guidelines
• Know thyself, know thy data
• Have an ‘unquestionable’ gold standard (outcome variable)
• Use an existing tool
• Get a statistician involved, EARLY