managerial epidemiology part iii: measuring health outcomes
DESCRIPTION
Managerial Epidemiology Part III: Measuring Health Outcomes. Ty Borders, Ph.D. Assistant Professor Department of Health Services Research & Management School of Medicine. Learning objectives. Describe and interpret validity Describe and interpret reliability - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/1.jpg)
Managerial EpidemiologyPart III:
Measuring Health Outcomes
Ty Borders, Ph.D.
Assistant Professor
Department of Health Services Research & Management
School of Medicine
![Page 2: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/2.jpg)
Learning objectives
• Describe and interpret validity
• Describe and interpret reliability
• Calculate sensitivity and specificity
• Identify potential health status and satisfaction measures
• Interpret a disease management guideline
![Page 3: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/3.jpg)
Validity and Reliability
• Validity– the degree to which you are measuring what you
intend to measure
• Reliability– Consistency of a measure– Often expressed as a correlation coefficient
![Page 4: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/4.jpg)
Types of validity
• True validity is based on some kind of gold standard, but usually there is not a gold standard
• Content validity (face validity) – the degree to which a measure represents a concept– determined by a panel of experts
• Construct validity
– compare results of measure to other phenomenon
![Page 5: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/5.jpg)
Types of validity
• Criterion validity
– Compare a measure to other measures– Sensitivity
• Proportion of those who test positive for a trait and have a trait
– Specificity• Proportion of those who test negative for a trait and do
not have a trait
![Page 6: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/6.jpg)
Sensitivity and specificity
Disease
Measure results Present Absent Total
Positive True + False + a + b
Negative False - True - c + d
Totals a + c b + d a+b+c+d
sensitivity = a / (a+c)specificity = b / (b+d)
![Page 7: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/7.jpg)
Example: Cage questionnaire
Alcoholism
Score Present Absent Total
3-4 positives 130 11 141
0-2 positives 164 516 680
Totals 294 527 821
sensitivity = 130/294 = 44%specificity = 516/527 = 98%
![Page 8: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/8.jpg)
Assessing Health Status - What is health?
• A simple definition– The presence or absence of disease
• WHO definition – “complete physical, mental, and social
well-being, and not merely the absence of disease or injury”
![Page 9: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/9.jpg)
What are disease and illness?
• Disease– Professionally defined
– Precise
– Used for treatment
• Illness– Lay definition
– Individual’s reaction to biological state
– Influenced by culture
![Page 10: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/10.jpg)
Assessing health status
• 5 outcomes of disease (5 Ds)
– Death (if unexpected or untimely)
– Disease (a set of symptoms, signs, and laboratory abnormalities)
– Discomfort (pain, nausea, itching, etc.)
– Disability (impaired ability to go about usual activities)
– Dissatisfaction (emotional reaction to disease)
![Page 11: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/11.jpg)
A broader definition of health:Health-Related Quality of Life
• Health-related quality of life (HRQL) encompasses those aspects of life that are dominated or significantly influenced by personal health or activities performed to maintain health.
![Page 12: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/12.jpg)
Core concepts and domains of HRQL
• Health perceptions – General health perceptions – Satisfaction with health
• Functional status– Social function– Psychological function– Physical function
![Page 13: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/13.jpg)
Core concepts and domains of HRQL (cont.)
• Impairment– Symptoms/subjective complaints– Signs (observable evidence of abnormality)– Diagnoses (clinical judgments)
• Death and duration of life
• Opportunity– Resilience; social or cultural disadvantage
![Page 14: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/14.jpg)
HRQL measures
• Disease-specific– Arthritis Impact Measurement Scale
• Generic– Sickness Impact Profile– Quality of Well-Being Index– SF - 36– SF-12
• Shorter version of the SF-36
• Measures physical and mental/emotional health
![Page 15: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/15.jpg)
SF - 12
• Validity– Comparison of scores among people with
different diseases– Comparisons with other measures
• Test-retest reliability– Physical component score 0.864 to 0.890– Mental component score 0.760 to 0.774
![Page 16: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/16.jpg)
Health-Related Quality of Life (SF-12)Adjusted for Predisposing, Enabling, and Need Factors
43.0552.7
41.27
54.9
0
20
40
60
80
100
Mean PhysicalScore
Mean MentalScore
Hispanics Non-Hispanic Whites
![Page 17: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/17.jpg)
Measuring quality
• Technical care (Donabedian)
– Application of science, technology of medicine (and other health sciences) to the management of personal health
• Interpersonal care – Management of the social and
psychological interaction between client and practitioner
![Page 18: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/18.jpg)
Dimensions of satisfaction
• Overall satisfaction with health care
• Overall satisfaction with provider
• Satisfaction with accessibility– Ability to see personal doctor– Ability to get appointment with specialist– Appointment waiting time– Costs of care– How often get help when phone doctor’s office
![Page 19: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/19.jpg)
Dimensions of satisfaction (cont.)
• Satisfaction with quality– Interpersonal manner of physician– Communication– Office waiting times– Facility characteristics– Time spent with doctor– Technical competence
![Page 20: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/20.jpg)
Level of measurement
• Clinic or provider level– Medical Outcomes Study (MOS) survey– Measures satisfaction with a particular encounter
• Health plan or population level– Consumer Assessment of Health Plans Survey– Developed by Agency for Healthcare Quality and
Research (formerly AHCPR)– Frequently used to assess health plan performance
![Page 21: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/21.jpg)
Phys. function
Psych. function
Social function
Plan, institution, systemOrganized team
Several practitioners
Individual practitioner
Patient PersonIndividual Case load Individual Population
![Page 22: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/22.jpg)
Comparison of satisfaction measuresInstrument Dimensions Data collection
PSQ-III Patient satisfaction 50 items, mailed
with medical care
CSQ-8 General satisfaction 8 items, self-admin.
SERVQUAL Reliability, empathy, 22 items, self-admin.
responsiveness,
assurance
VSQ Outpatient satis. 7 items, self-admin.
following visit
CAHPS Access, coms, choice 47 items, multiple
continuity, overall modes of collection
satisfaction, plan satis.
![Page 23: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/23.jpg)
CAHPS• Single items, 0 (worst) to 10 (best)
– Global rating of health plan– Global rating of quality of care– Global rating of personal doctor– Global rating of specialist
• Composites– Getting care you need (4 items, never to always)– Getting care without long wait– Doctors communicate well with patients– Doctors spend enough time with patients– Reasonable paperwork, handling of payments
![Page 24: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/24.jpg)
On a scale from 0 to 10 with 10 being the best…
how would you rate the health care you received
0 20 40
10
8
6
4
2
0
Rat
ing
Percentage
Global Satisfaction
![Page 25: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/25.jpg)
Threats to the reliability and validity of surveys
• Validity– Choose non-validated measures– Choose wrong measures
• Reliability– Unrepresentative sample– Inadequate sample size
– Poor response rate
![Page 26: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/26.jpg)
Administration
• In-person
• Telephone
• Mail survey
• Social desirability– Tendency to offer answers that are consistent
with values the respondent believes to be held by interviewer/person conducting survey
– Confidentiality issue
![Page 27: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/27.jpg)
Response rates
• # completed surveys / # distributed
• Non-reponse error can occur if the response rate is not 100%
• As a rule of thumb, do not use data if the response rate is below 60%
• Increasing the sample size does not reduce the non-response error
![Page 28: Managerial Epidemiology Part III: Measuring Health Outcomes](https://reader035.vdocuments.mx/reader035/viewer/2022062323/568157ad550346895dc539f9/html5/thumbnails/28.jpg)
Sampling and sample size
• When conducting surveys, more efficient to sample a subset of the population
• A sample approximates the population
• Sample size depends on several factors
• For more information, refer to Aday’s, Designing and Conducting Health Surveys