1 class 2 measurement terminology and locating measures september 28, 2006 anita l. stewart...
TRANSCRIPT
1
Class 2Measurement Terminology and
Locating Measures
September 28, 2006
Anita L. StewartInstitute for Health & Aging
University of California, San Francisco
2
Overview of Class 2
Measurement terminology– Confusion, variation, and overlap
Types of response scales– Contribution to concept definition
Measurement models– Depicting structure of measures
Locating measures
3
Measurement
Assignment of numbers to aspects of objects or events according to a rule
Assignment of numbers to an unobservable characteristic of individuals (construct)
4
Construct, Concept, Variable, Latent Trait, Latent Variable
Terms defined differently by different investigators/authors
Meaning is very similar– considerable overlap among them,
Often used interchangeably
5
Construct/Concept
A variable that is relatively abstract as opposed to concrete and is defined or operationalized in terms of observed indicators
An idea developed or constructed through informed scientific theory. Concepts that are directly or indirectly observable.
An abstraction based on observations of certain behaviors or characteristics (e.g. pain, stress)
6
Latent Trait, Factor
Latent - present but not visible, unobservable Latent trait - unobservable set of characteristics
that can be empirically inferred and estimated through answers to a set of questions
(Medical Care glossary) Factor - a latent variable or theoretical construct
operationalized in terms of associations among indicators in a factor analysis
(Stewart and Ware, 1992 glossary)
7
Latent Variable, Manifest Variable
Latent variable - unobservable latent trait given a label
Manifest variable is the “observed” information, i.e., the answers to items used to infer the latent trait
8
Domain, Dimension, Sub-domain, Component
Domain: a state, attitude, perception, behavior, or other sphere of action or thought related to health …. All entities in a single domain have some property in common ….
(Patrick & Erickson, 1993) Dimension: a distinct component of a
multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health)
(Stewart & Ware, 1992, glossary)
9
Example: Dimension, Component, Sub-domain
Mental health (dimension)– Psychological distress (component)
» Depression (sub-domain)» Anxiety
– Psychological well-being» Positive affect» Sense of belonging
10
Components of Domains of MOS Long-form Measures
Role Role limitations due to:Functioning -- physical health
-- emotional problems
Social Social limitations due to healthFunctioning Sexual problems
Family functioningMarital functioning
Pain Pain severityEffects of painDays with pain
11
Multidimensional and Unidimensional
A multidimensional “measure” or “instrument” provides several scores– A “profile” of scores
Unidimensional measure - all items measure only one concept– Only one scale score is obtained
Dimensionality must be empirically tested– e.g., factor analysis identifies one factor,
or more than one factor
12
Example of Multidimensional “Instrument” -
Patient Satisfaction Questionnaire (PSQ) 55 items, 18 subscales, e.g.
– Access to care– Technical quality– Interpersonal manner– Explanations– Continuity of care
Marshall, GN Psychological Assessment, 5:477-483, 1993
13
Example of a Multidimensional “Instrument” – SF-36Measures 8 health concepts (36 items): Physical functioning Role limitations due to physical health Role limitations due to emotional problems Social functioning Pain Vitality (energy/fatigue) Mental health Health perceptions
14
Example of Unidimensional “Measure”
Perceived Stress Scale (PSS) 14 items, subjective experiences of stress
– felt confident could handle life’s problems
– able to control irritations in your life
– difficulties piling up so high, could not overcome them
Single score from all itemsCohen, S, J Health Soc Behav 24:385-396, 1983
15
Instrument, Profile, Battery, Measure, Questionnaire, Survey, Tool, Inventory Instrument - typically refers to a published,
named measure or set of measures Profile - same, but all scores on the same
metric and can be compared (0-100 scales) Battery (of independent measures)
– collection of measures from diverse sources Measure - single- or multi-item scale or
index (umbrella term)
16
Generic vs. Disease-Specific Approaches
Generic– Applicable across many conditions and
diseases (e.g., sleep, health perceptions, fatigue, pain)
Disease-specific– Specific to particular disease or condition
(e.g., arthritis, cancer, depression)» Generic approach (e.g., general dexterity scale for arthritis
study)» Specific (nausea due to chemotherapy, back pain)
17
Generic or Disease Specific bySingle Domain, Profile, or Index
Single domain
Profile Summary index
Generic Pain
Fatigue
SF-36
SIP
PCS
MCS
QWB
Disease specific
Nausea
Back pain
Diabetes impact
18
Preference Based or Utility Measures
Utilities are numeric measurements that reflect the desirability people associate with a health state or condition– Value of a particular health state
– Preference for a health state compared to a different one
19
Questionnaire, Survey, Tool, Inventory
Questionnaire - any of the above formatted into a “questionnaire” or set of measures
Survey - same as questionnaire Tool = Questionnaire = Survey? Inventory?
20
Examples of Names of “Instruments”
SF-36 Short Form Health Survey Beck Depression Inventory Health Assessment Questionnaire Sickness Impact Profile Center for Epidemiological Studies
Depression Scale
21
Example of a Battery
A survey for a particular study containing the following measures/instruments:– SF-36 (8 domains)– Perceived stress scale (1 domain)– Social support survey (4 domains)– Pain measure
Typical in clinical trials, epidemiological studies– Each measure serves particular purpose (outcome,
predictor, mediator, covariate)
22
Measure, Scale, Index, Item
Measure - single- or multi-item scale or index (umbrella term)
Scale - aggregation of items from one concept or domain, scored according to an accepted scaling method
Index - aggregation of 2 or more scales into a summary score
Item - a single question or statement including its response scale
23
Overview of Class 2
Measurement terminology– Confusion, variation, and overlap
Types of response scales– Contribution to concept definition
Measurement models– Depicting structure of measures
Locating measures
24
Item
Consists of an item stem and a response scale or response choices
Item stem – – The question or statement
Response scale or choices –– Answers presented to a respondent with which to
respond to a (closed-ended) question
25
Example of a Response Scale
How often have you felt tired inthe past week? 1 Never 2 Once or twice 3 A few times 4 Fairly often 5 Very often
Responsescale
Itemstem
26
Response Choice Formats
Verbal descriptors of all levels Verbal descriptors of endpoints only Verbal descriptors of endpoints plus
midpoint Visual analogue scales:
10cm line with endpoints labeled
27
Format: Verbal Descriptors of All Levels
1 - All of the time2 - Most of the time3 - Some of the time4 - A little of the time5 - None of the time
28
Vague, Imprecise Quantifiers
How often?– Very often, pretty often, not too often
– Sometimes, often, never How much?
– Too little, about right, too much
– Below average, average, above average
Bradburn NM, Vague quantifiers. Public Opinion Quart 1979, 92-101.
29
Intensity Scales – “How distressed have you been?”
1 – Not at all
2 - Slightly
3 - Moderately
4 - Quite a bit
5 - Extremely
1 – Not at all
2 - A little
3 - A fair amount
4 - Much
5 - Very much
30
Intensity Scales – “How distressed have you been?” (even more scales)
1 – Not at all
2 - A little
3 - Fairly
4 - Quite
5 - Extremely
1 – Not at all
2 - A little
3 - Somewhat
4 - Very much
31
“How Often” Response Scales
1 - Never
2 - Once or twice
3 - A few times
4 - Often
1 – Never
2 - Hardly ever
3 - Some days
4 - Most days
5 - Almost every day
6 - Always, every day
32
“How Often” Response Scales - more
1 - Never
2 - Almost never
3 - Sometimes
4 - Fairly often
5 - Very often
6 - Always
1 - Once or twice
2 - A few times
3 - Fairly often
4 - Very often
5 - Almost every day
6 - Every day
33
Format: Verbal Descriptors of Endpoints Only
0 1 2 3 4 5 6 7 8 9 10 No Pain pain as bad
as you can imagine
34
Format: Verbal Descriptors of Endpoints with Midpoint Labeled
Very Very
much 1 2 3 4 5 6 7 muchworse No better
change
35
Format: Verbal Descriptors of Endpoints with Midpoint Labeled
-3 -2 -1 0 1 2 3Very No Very much change muchworse better
37
Optimal Number of Response Choices
Are 100 levels better than 5 or 10? 5-points is probably optimal for respondents
– Especially if items will be combined into multi-item scales
For single-item measures– 7 points may be preferred– Allows distribution over 5 points since people
seldom use endpoints Hard to come up with labels for 7-point scales
38
Format: Verbal Descriptors of Endpoints Only
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
No Pain pain as bad
as you can imagine
39
Magic Number 7 + 2
George Miller determined that 7 “chunks” of information is the most that short-term memory can retain– Plus or minus two (5-9 possible)
Seven (+ 2) is also the most points that people can discriminate along a continuum
Principle has held up well in measurement
Miller, GM Psychological Bulletin, 63:81-97, 1956
40
Can You Make a 7-point Intensity Scale? (How physically active were you this week?)
1 - Not at all active
2 - A little bit active
3 - Moderately active
4 - Quite active
5 - Extremely active
1 - Not at all active
2 - A little active
3 - Fairly active
4 - Quite active
5 - Very active
6 - Extremely active
41
Overview of Class 2
Measurement terminology– Confusion, variation, and overlap
Types of response scales– Contribution to concept definition
Measurement models– Depicting structure of measures
Locating measures
42
Measurement Models
The dimensional structure of a measure in terms of how the items related to the constructs (latent traits, factors, concepts)
Can be depicted as a list or visually Standard visual format for structural
equation modeling (SEM)
44
Measurement Model (List format)
Physical Functioning defined in terms of:– Walking
– Climbing stairs
– Bending
– Reaching
46
Measurement Model (List format)
Psychological distress– Depression
» Sad» Lost interest» Can’t get going
– Anxiety» Restless» Nervous
47
Measurement Model (Visual format)
Psychological Distress
Depression Anxiety
SadLost
interestCan’t
get goingRestless Nervous
48
Measurement Model: Family Factors (List format)
Family environment– Conflict– Cohesion
Parenting behaviors– Nurturing– Discipline– Expectations
49
Measurement Model: Family Factors (Visual format)
Family Factors
Family Environment
Parenting Behaviors
Conflict Cohesion
Fight a lot
Throw things
Help and support
Together-ness
50
Measurement Model: Family Factors (Visual format)
Family Factors
Parenting Behaviors
Nurturing Discipline Expectations
PlayTake to
park
Spank Once
per week
Yell at for
whining
Wash &dry hands
Put Awaytoys
Family Environment
52
Content Areas: Four Depression Measures
CES-D HAM-D Beck GDS
Sleep problems X X X X
Concentration X X
Fatigue, energy X X
Appetite loss X X
Depressed mood X X X
Anxiety, restlessness X X
Irritability X X
Sense of failure X X
Social withdrawal X X
Guilt X
Self-criticism X X
53
Content Areas: Four Standard Measures of Physical Functioning
AIMS MOS HAQ SIP
Walking X X X X
Climbing stairs or steps X X X X
Bending, kneeling X X X X
Lifting, carrying X
Getting out of bed X
Bathing X X X
Running errands X
Opening jars X
Vigorous activities X X X
54
MOS Physical Functioning: SF-36 versus SF-12
SF-36 SF-12 Vigorous activities X Moderate activities X X Lift/carry X Climb several flights stairs X X Climb one flight stairs X Bend/kneel/stoop X Walk > 1 mile X Walk several blocks X Walk 1 block X Bathe/dress X
55
MOS Mental Health Items:SF-36 versus SF-12
SF-36 SF-12 Nervous X Down in the dumps X Calm and peaceful X X Downhearted and blue X X Happy person X
56
Overview of Class 2
Measurement terminology– Confusion, variation, and overlap
Types of response scales– Contribution to concept definition
Measurement models– Depicting structure of measures
Locating measures
57
Locating Specific Measures
Compendia Web Organizations, foundations National and state surveys Large research studies Many other sources
58
Locating Specific Measures: Compendia
Compendia of measures– Many books review and critique various
measures Web based review of such books:
www.measurementexperts.org/instrument/book_compendium.asp
59
Locating Specific Measures: Compendia by Web
Health and Psychosocial Instruments (HaPI)– Over 120,000 instruments
– http:/gateway.ovid.com/
– Need ovid account – UCSF has one if use computer at work (or vpn) – no need to enter password
60
Locating Specific Measures: Compendia by Web
Measurement Excellence and Training Resource Information Center (METRIC)
– http://www.measurementexperts.org
– Click on “find an instrument”
61
Locating Specific Measures: Organizations and Foundations
RAND publishes measures, scoring manuals, and lists citations– Measures of quality of care, patient satisfaction– Measures of health-related quality of life
» Generic and disease specific
– All Medical Outcomes Study measures http://www.rand.org/health/
– Go to “surveys and tools”
62
Inter-University Consortium for Political and Social Research
http://www.icpsr.umich.edu/
Maintains archive of social science data Membership-based organization – over 500
member colleges/universities– UCSF is a member
Can search website using keywords to locate studies, data, and questionnaires
63
Commonwealth Fund
The Fund tracks trends in health coverage, access and quality, and general policy/practice issues in the U.S. and internationally. All information is available free of charge. Raw data are available to researchers one year post release.
Surveys and data: – http://www.cmwf.org/surveys/surveys.htm
64
Commonwealth Fund: Example
2001 Health Care Quality Survey This survey assessed racial and ethnic
differences in patient experiences with the health care system, focusing on such quality measures as patient–physician communication and access to care.
http://www.cmwf.org/surveys/surveys_show.htm?doc_id=228171
65
Locating Specific Measures: National Surveys
National Center for Health Statistics– Surveys and data collection systems
Can download any survey or portion of survey
Summary of Surveys (see handout)– http://
www.cdc.gov/nchs/data/NCHS_Survey_Matrix.pdf#search=%22Summary%20of%20Surveys%20CDC%22
66
National Center for Health Statistics: Examples of Surveys
National Health Interview Survey National Health and Nutrition Examination Survey National Health Care Survey
– Ambulatory health care data (NAMCS)– National Home and Hospice Care Survey
National Survey of Family Growth National Maternal and Infant Health Survey Longitudinal Studies of Aging (LSOA)
www.cdc.gov/nchs/express.htm
67
Locating Measures: National Agencies
Agency for Healthcare Research and Quality (AHRQ)– National Quality Measures Clearinghouse
– Consumer Assessment of Health Plans Survey (CAHPS)
– National Healthcare Quality Report
– National Healthcare Disparities Report
68
Locating Measures: National Agencies
Centers for Disease Control Behavioral Risk Factor Surveillance
System– Surveys of health behaviors
http://www.cdc.gov/brfss/questionnaires/index.htm
69
Locating Specific Measures: NCI
National Institutes of Health: special initiatives on measures– National Cancer Institute
Health Information National Trends Survey (HINTS)– Working group compiled cancer screening
questions, identified best ones, conducted extensive pretesting, cognitive interviewing
– Measures are on the NCI web site
70
Locating Measures: State Surveys
http://www.chis.ucla.edu/ California Health Interview Survey
(CHIS) “Methodology – review questionnaires”
71
Locating Measures: Projects and Centers
Projects/Centers– Toolkit to measure end of life care (TIME)– Stanford Patient Education Research Center– Michigan Diabetes Research and Training Center– Resource Centers for Minority Aging Research
Proprietary measures– SF-36, SF-12 (have to pay for instruments, scoring
manuals)
72
Tool Kit of Instruments to Measure End of Life Care (TIME)
www.chcr.brown.edu/pcoc/toolkit.htm Instruments to measure “patient-focused,
family centered” care Annotated bibliographies of instruments
that measure outcomes of care at the end of life
73
TIME Annotated Bibliographies Available:
Quality of life Pain and other
symptoms Emotional and
cognitive symptoms Advance care
planning
Spirituality Continuity of care Caregiver well-being Grief and
bereavement and more
74
Michigan Diabetes Research and Training Center: Survey Instruments
www.med.umich.edu/mdrtc/survey/
•Diabetes Care Profile (DCP) •Diabetes History (DMH) •Diabetes Knowledge Test (DKT) •Diabetes Attitude Scale (DAS-3) •Diabetes Empowerment Scale (DES) •Michigan Neuropathy Screening Instrument (MNSI)
75
Locating Specific Measures
Large research studies on topic similar to yours– Health ABC, CARDIA
– Women’s Health Initiative
– EPESE – Established Populations for Epidemiological Studies of the Elderly
76
Locating Measures: Large Studies
Study of Women’s Health Across the Nation (SWAN)– physical, biological, psychological, and
social changes of women during their middle years
www.edc.gsph.pitt.edu/swan/public Questionnaire can be downloaded
77
UCSF Data Coordinating Center
https://rds.epi-ucsf.org/ go to eForms Warehouse link (lower left)
– Specific study? ALL
– Annotated? NO
– Specific content search?» pulldown menu with types of measures
78
Proprietary and Non-Proprietary Measures: An Example
SF 36– www.sf-36.org
RAND SF-36: non proprietary– www.rand.org/health/survesy/sf36item/
79
Locating Specific Measures: Bibliographic Searches
Published measurement articles– Medline Searches
– MESH headings or keywords:» health status indicators
» outcome assessment (health care)
» psychometrics (methods)
» questionnaire
» measurement
80
Locating Specific Measures: Finding Authors of Measures
Published research using measure you are interested in– Unpublished measures often described in
methods– Authors may provide measures
81
Summary
Don’t let the terminology get you down– Be aware of alternative uses of words/labels
Measurement models – SEM depiction is useful
Locating measures– Maintain a file of measures of concepts you
are interested in