kevin p. weinfurt, phd - ippcr.nihtraining.com p. weinfurt, phd ... osteoarthritis trial cat heart...
TRANSCRIPT
Patient‐Reported Outcomes in
Clinical ResearchKevin P. Weinfurt, PhDIntroduction to the Principles and
Practice of Clinical Research National Institutes of Health
November 30, 2015
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Overview
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
1. Why measure patient‐reported health status?
2. Different types of PROs3. Development and evaluation of PROs4. “New” methods using Item Response
Theory5. Interpreting scores on PRO measures
Why measure patient-reported health status?
1
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
How do we assess benefit?
Copies of virus in blood
Tumor size
Blood pressure
PeakVO2
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
For endpoints to inform decisions, they must matter to patients, clinicians, and payers.
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
PeakV02
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
r = .53Flynn et al, 2015
Overactive Bladder Syndrome
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Pad weight to measure leakage
volume
What is a meaningful
reduction in volume?
Treatment Benefit
Feel Function SurviveFDA
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Patient-Reported Outcome (PRO)
“A measurement based on a report that comes directly from the patient (i.e., study subject) about the status of a patient’s health condition
without amendment or interpretation of the
patient’s response by a clinician or anyone else.”
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
PROMIS® Fatigue Measure
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Different Types of PROs2
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Biological and Physiological
Variables
Symptom Status
Functional Status
General Health
Perceptions
OverallQualityof Life
Nonmedical Factors
Adapted from Wilson & Cleary, (JAMA1995)
Characteristics of the Individual
Characteristics of the Environment
1. Dilution of effects of biological interventions
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
2. Correlation between successive boxes
decreases
Development and Evaluation of PRO
Measures
3
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
“Fatigue”
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
1. Determine what PRO concept we want to measure and why
Fatigue
2. Collect qualitative data to understand meaning of the PRO concept
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Fatigue
3. Write items you think will measure the concept
Item 1Item 2Item 3Item 4Item 5Item 6Item 7Item 8
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
4. Test items for understanding (cognitive interviews)
Item 1Item 2Item 3Item 4Item 5Item 6Item 7Item 8Item 9Item 10
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
5. Administer items to a large sample of people
Item 1Item 2Item 3Item 4Item 5Item 6Item 7Item 8
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
6. Use psychometric (statistical) analyses to see how well items are working and develop
FatigueItem 3Item 4Item 5Item 6Item 7Item 8
scoring method
How well do items fit this model?
Item 1Item 2
Fatigue Score
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
7. Evaluate the reliability and validity of the measure
ValidityMeasures what it’s
supposed to measure
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
ReliabilityMeasures with little error
(a.k.a.precision)
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Types of ValidityContent
Face
ConstructConvergent/discriminantKnown groups Predictive
Responsiveness
Convergent Validity:PROMIS Depression Domain
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
If I have not changed, I should get the same score . . .
using different sets ofitems from the samemeasure
over time
regardless of who scores it
Internal Consistency(Cronbach’s alpha)
Test‐Retest
Interrater
“New” Methods Using Item ResponseTheory
4
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Everyone must complete the same items.
All items are necessary to obtain ascore.
Score might not be on the same metric as other measures of the same thing.
Traditional Off-the-Shelf PRO Measure
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
An item bank is a large collection of items measuring a single domain.
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Any and all items can be used to provide a score for that domain.
Dynamic, not fixed.
Physical Functioning Item Bank
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 8
Item 9
Itemn
50 1000
Are you able to get in and out of bed?Are you able to stand without losing your balance for 1 minute? Are you able to walk from one room to another?Are you able to walk a block on flat ground?Are you able to run or jog for two miles? Are you able to run five miles?
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Using a Traditional Off-the-Shelf PRO Measure
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Using a PRO Item Bank
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Fixed-Length Measures
Computerized Adaptive
Tests (CATs)Ready-made Make-your-own
Item Bank
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Fixed-Length Measures
Item Bank
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Computerized Adaptive
Tests (CATs)Next item administered depends on answer to
previous item
Fatigue Item Bank
Same metric, same meaning
Chemotherapy trial
Items 1‐10
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Osteoarthritis trialCAT
Heart failure trial(NYHA Class III)
Items 6‐12
Heart failure trial(NYHA Class I)
Items 1‐5
Diabetes trialItem 7
Item 1Item 2Item 3Item 4Item 5
Item 1Item 2Item 3Item 4
Different Sites, Different Measures
Item 1Item 2Item 3Item 4Item 5Item 6
Measure 1(in HCS 1)
Measure 2(in HCS 2)
Measure 3(in HCS 3)
Metric 1
Metric 2
Metric 3
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Item 1Item 2Item 3Item 4Item 5
Item 1Item 2Item 3Item 4
Item Bank
Measure 1(in HCS 1)
Measure 2(in HCS 2)
Measure 3(in HCS 3)
Item 1Item 2Item 3Item 4Item 5Item 6
Item 1Item 2Item 3Item 4
One Metric
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Other items
Traditional PRO Measure PRO Item Bank
All items are required to compute a score
Any and all subsets of items can generate a score
Everyone must take same items
Different people can get different items
Use it “off the shelf ”Use items in bank to create
measure for specific use
Scores not easily comparable to scores from another
measure of the same domain
Cross-walk between scores from different measures in
the same item bank
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Examples of PRO Resources Based on Item Banks
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Adult and pediatric item banks measuring domains relevant across chronic diseases
Freely available
www.nihpromis.org
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Multidimensional set of brief measures assessing cognitive, emotional, motor and sensory function from ages 3-85Freely available
www.nihtoolbox.org
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Core set of instruments for use in chronic neurological conditions (supplemental set for specific diseases, patient subgroups)
Freely available
www.neuroqol.org
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Differential Item Functioning
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
In the past 7 days, did you cry?
Yes No
(Depression item)
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Differential Item Functioning
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Item behavesdifferently for 2 or more groups.
In the past 7 days, did you cry?
Pro
babi
lity
of“Y
ES
”
0.00
0.25
0.50
0.75
1.00
Depression
-3 -2 -1 0 1 2 3
Item ResponseTheory
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
In the past 7 days, did you cry?
Pro
babi
lity
of“Y
ES
”
0.00
0.25
0.50
0.75
1.00
Depression
-3 -2 -1 0 1 2 3
Item ResponseTheory
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
In the past 7 days, did you cry?
Pro
babi
lity
of“Y
ES
”
0.00
0.25
0.50
Depression
-3 -2 -1 0 1 2 3
Item ResponseTheory
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
1.00
0.90
0.75
Differential Item Functioning
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Item behavesdifferently for 2 or more groups.
The “map”between depression and item is different for 2 or more
groups.
In the past 7 days, did you cry?P
roba
bilit
y of
“YE
S”
0.00
0.50
0.75
1.00
Depression
-3 -2 -1 0 1 2 3
FemalesMales
0.90
0.300.25
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Interpreting Scores on PRO Measures
5
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Exercise vs Usual care
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Changes in Kansas City
CardiomyopathyQuestionnaire
(Score Range: 0 -100)
1.93
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
(95% C.I., 0.84,3.01)
Exercise arm has statistically greater rate of change between baseline and 3 months.
Spertus et al., 2005IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Mean Diff = 1.93
Exercise = 54%Control = 29%
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
Review
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015
1. Why measure patient‐reported health status?
2. Different types of PROs3. Development and evaluation of PROs4. “New” methods using IRT5. Interpreting scores on PRO measures