markov models: overview gerald f. kominski, ph.d. professor, department of health services
Post on 15-Dec-2015
217 Views
Preview:
TRANSCRIPT
Markov Models: OverviewMarkov Models: Overview
Gerald F. Kominski, Ph.D.Gerald F. Kominski, Ph.D.Professor, Department of Health ServicesProfessor, Department of Health Services
Markov Models: Why Are They Necessary?Markov Models: Why Are They Necessary?
Conventional decision analysis models Conventional decision analysis models assume:assume:- Chance eventsChance events- Limited time horizonLimited time horizon- Events that do not recurEvents that do not recur
What happens if we have a problem with:What happens if we have a problem with:- An extended time horizon, say, over a lifetimeAn extended time horizon, say, over a lifetime- Events can reoccur throughout a lifetimeEvents can reoccur throughout a lifetime
State-Transition Diagram for Atrial FibrillationState-Transition Diagram for Atrial Fibrillation
WellWell
Post-Post-StrokeStroke
DeadDead
pp1212=0.2=0.2
pp2222=0.9=0.9
pp3333=1.0=1.0
pp1111=0.7=0.7
pp2323=0.1=0.1
pp1313=0.1=0.1
The probabilities for all paths out of a state must sum to 1.0.The probabilities for all paths out of a state must sum to 1.0.
Death is known as an Death is known as an absorbing state,absorbing state, because individuals who enter that because individuals who enter that state cannot transition out of it.state cannot transition out of it.
Transition ProbabilitiesTransition Probabilities
WellWell Post-Post-StrokeStroke
DeadDead
WellWell 0.70.7 0.20.2 0.10.1
Post-Post-StrokeStroke
0.00.0 0.90.9 0.10.1
DeadDead 0.00.0 0.00.0 1.01.0
State of State of Current CycleCurrent Cycle
State of Next CycleState of Next Cycle
Transition probabilities that remain constant over time Transition probabilities that remain constant over time are characteristic of are characteristic of stationary Markov models, aka stationary Markov models, aka Markov chainsMarkov chains
Markov Model DefinitionsMarkov Model Definitions
Any process evolving over time with uncertainty is a Any process evolving over time with uncertainty is a stochastic process, stochastic process, and models based on such and models based on such processes are stochastic or probabilistic modelsprocesses are stochastic or probabilistic models
If the process is both stochastic and the behavior of the If the process is both stochastic and the behavior of the model in one time period (i.e., cycle) does not depend model in one time period (i.e., cycle) does not depend on the previous time period, the process is on the previous time period, the process is MarkovianMarkovian- The process has “lack of memory”The process has “lack of memory”- Even processes where the previous state does matter can be Even processes where the previous state does matter can be
made Markovian through definition of temporary states know made Markovian through definition of temporary states know as as tunnel statestunnel states
Tunnel StatesTunnel States
WellWellPost-Post-StokeStoke 11
Post-Post-StrokeStroke 22
Post-Post-StrokeStroke 33
Post-Post-StrokeStroke
DeadDead
Defining a Markov ModelDefining a Markov Model
Define the initial statesDefine the initial states
Determine the cycle lengthDetermine the cycle length
Consider possible transitions among statesConsider possible transitions among states
Determine transition probabilitiesDetermine transition probabilities
Determine utilities, and costs (if cost-effectiveness Determine utilities, and costs (if cost-effectiveness analysis), for each stateanalysis), for each state
Evaluating Markov Models:Evaluating Markov Models:Cohort SimulationCohort Simulation
StateState
CycleCycle WellWell Post-Post-StrokeStroke
DeadDead Sum of Sum of Years LivedYears Lived
SurvivalSurvival
00 10,00010,000 00 00
11 7,0007,000 2,0002,000 1,0001,000 9,0009,000 0.90000.9000
22 4,9004,900 3,2003,200 1,9001,900 8,1008,100 0.81000.8100
33 3,4303,430 3,8603,860 2,7102,710 7,2907,290 0.72900.7290
44 2,4012,401 4,1604,160 3,4393,439 6,5616,561 0.65610.6561
55 1,6811,681 4,2244,224 4,0954,095 5,9055,905 0.59050.5905
66 1,1761,176 4,1384,138 4,6864,686 5,3145,314 0.53140.5314
77 824824 3,9593,959 5,2175,217 4,7834,783 0.47830.4783
9393 00 11 9,9999,999 11 0.00010.0001
9494 00 00 10,00010,000 00 0.00000.0000
The data in the last column is used to produce a survival curve, aka a Markov trace.The data in the last column is used to produce a survival curve, aka a Markov trace.
Estimating Markov Models:Estimating Markov Models:Monte Carlo SimulationMonte Carlo Simulation
Instead of processing an entire cohort and applying Instead of processing an entire cohort and applying probabilities to the cohort, simulate a large number probabilities to the cohort, simulate a large number (e.g., 10,000) cases proceeding through the transition (e.g., 10,000) cases proceeding through the transition matrixmatrix- Monte Carlo simulationMonte Carlo simulation- TreeAge will do this for you quickly, without programmingTreeAge will do this for you quickly, without programming
The advantage of this approach is that it provides The advantage of this approach is that it provides estimates of variation around the meanestimates of variation around the mean
Monte Carlo simulation is most valuable because it Monte Carlo simulation is most valuable because it permits efficient modeling of complex prior historypermits efficient modeling of complex prior history- Such variables are known as Such variables are known as tracker variablestracker variables
Example of a 5-State MarkovExample of a 5-State Markov
Source: Kominski GF, Varon SF, Morisky DE, Malotte CK, Ebin VJ, Coly A, Chiao C. Costs and cost-effectiveness of adolescent compliance with treatment for latent tuberculosis infection: results from a randomized trial. Journal of Adolescent Health 2007;40(1):61-68.
Key Assumptions of the Markov ModelKey Assumptions of the Markov Model
Variable Value (Range) ReferenceEfficacy of IPT 0.85 (0.75-0.98) 19Cost of treating active TB $22,500 ($17,000-$30,000) 17 Cost of IPT Varies by study group and whether 6-
month IPT is completedCurrent study
TB cases per 100,000 250 (120-560) 20TB case fatality rate 0.0045-0.16 (varies with age) 17All-cause mortality rate per 100,000
19-15,476 (varies with age) National Center for Health Statistics, 1999 mortality tables
Hepatotoxicity of IPT 0.0008 (age<35, started IPT)0.0012 (age<35, completed IPT)
21
Hepatitis fatality rate 0.002 21 Cost of treating IPT-induced hepatitis
$11,250 ($8,500-$15,000) Authors’ assumption
QALY – Healthy 1.00 (0.95-1.00) Authors’ assumptionQALY – Positive Skin Test, but Incomplete IPT 0.90 (0.80-0.95) Authors’ assumption
QALY – Active TB 0.50 (0.20-0.90) Harvard Center for Risk Analysis
QALY – IPT-induced hepatitis 0.75 (75-0.90) Harvard Center for Risk Analysis
Discount rate 0.03 (0.00-0.07) Panel on Cost-Effectiveness
top related