social desirability and social approval biases in dietary self-report: examples of epidemiologic...
TRANSCRIPT
Social Desirability and Social Approval Biases in Dietary Self-Report: Examples of Epidemiologic Effect Modification
James R. Hebert, ScD
Professor and ChairDepartment of Epidemiology and BiostatisticsNorman J. Arnold School of Public Health University of South Carolina
Other Collaborators:
Lynn Clemow, Ph.D.
Ira S. Ockene, M.D.
Cara B. Ebbeling, Ph.D.
Yunsheng Ma, M.D., M.P.H.
Thomas G. Hurley, M.Sc.
Milagros C. Rosal, Ph.D.
Charles E. Matthews, Ph.D.
Judith K. Ockene, Ph.D.
cognitive difficulties
emotional aspects issues of prying
misleading responses
You’ve got to be kidding!
Emotional Aspects of Eating -
same part of brain processes sensations of taste, smell, and
basicemotional and instinctual
behavior.
social messages get overlaid on psychological and emotional ones.
Self-report of dietary intake could be biased bysocial desirability or social approval thus affectingrisk estimates in epidemiologic studies. These constructs produce response set biases, which are evident when testing in domains characterized by easily recognizable correct or desirable responses. Given the social and psychological value ascribed to diet, assessment methodologies used most commonly in epidemiologic studies are particularly vulnerable to these biases.
Background
Social Desirability is a Response Set reflecting the defensive tendency to respond in such a way as to avoid criticism in a situation perceived to be a test.
Social Approval is a Response Set reflecting the tendency to actively seek approval in a situation perceived to be a test.
Social desirability bias in dietary self-report may compromise the validity of dietaryintake measures.
International Journal of Epidemiology 1995; 24: 389-398.
WORCESTER
AREA
TRIAL for
COUNSELING in
HYPERLIPIDEMIA
WATCH
We Developed Seven-Day Diet Recall (7DDR)
In WATCH
The 7DDR looks very much like a FFQ, but it asks participants to recall specific meals and snacks over the past week. It includes 118 foods, 13 beverages, and a worksheet. Thus it combines elements of the FFQ and produces estimates of dietary intake more like trait measures than does 24HR.
WATCH External Validation Study
3 Weeks
Pre7DDR
Post7DDR(7 randomly selected 24-Hrs)
Exactly two years latersocial desirability and approval assessed
STUDY DESIGN
Hypothesis: Response Set Biases are expressedon structured questionnaires such as the Seven Day Dietary Recall (7DDR) or the FFQ.
it is closed-ended and has obvious correct responses
grid-like obvious response categories
requires report of diet as a trait rather than a state
Social desirability and social approval biases were tested by comparing nutrient scores derived from multiple twenty-four hour diet recalls (24HR) on randomly assigned days with those from two seven-day diet recalls (7DDR) (similar in some respects to commonly used food frequency questionnaires), one administered at the beginning of the test period (pre) and one at the end (post). Statistical analysis included correlation and multiple linear regression. Except for b describing relation between methods (where b should equal 1.0) , H0:=0.
Methods:
Social Desirability Approval Assessment
33-item (true/false)
Marlowe-CrowneSocial
DesirabilityScale
20-item (5-point Likert)
Martin-LarsenApproval
MotivationScale
Summary Statistics (from 24HR)
Women (n=27) Men (n=14)
Energy (kcal/d) 1491 (62) 1971 (600)
Fat (g/d) 51.9 (17.3) 65.1 (23.8)
% Fat 31.3 (5.5) 29.4 (4.6)
Calcium (mg/d) 668 (277) 785 (218)
Cholesterol (mg/d) 185 (65) 261 (195)* Tabulated values are mean (standard deviation)
WATCH – External Validation Study – Summary Statistics
Variable % or Mean/SD
% Married 74.1% 64.3% % White 81.5% 85.7% Age (years) 51.3/16.1 47.2/13.9Social Desirability Score 20.5/6.4 18.5/5.9Social Approval Score 48.4/9.6 44.9/9.1Total energy (k/cal) 1490/461 1970/599Total fat (g) 51.9/17.3 65.1/23.8
Female Male (N=27) (N=14)
Statistical Model
Nutrient test method = Nutrient comparison method +
+ BMI + Covariates
Social Approval + Social Desirability
WATCH – External Validation Study – Regression Results (Pre)
24-HRScore
SDScore
Total fat (g) 1.41 -2.34 0.39 1.17 n.s. (0.06)
Saturated fatty acids (g) 1.35 -0.68 0.29 0.32 n.s. (0.04)
Total Energy (kcal) b = 0.96 -50.23 SEb = 0.27 22.58P-value = n.s. (0.06)
24-HRScore
SDScore
Total fat (g) 0.90 -1.18 0.20 0.59
n.s. (0.06)
Saturated fatty acids (g) 0.87 -0.30 0.17 0.20 n.s. n.s.
WATCH – External Validation Study – Regression Results (Post)
Total Energy (kcal) b = 0.97 -25.50 SEb = 0.17 14.49P-value = n.s. (0.09)
Saturated fatty acids -0.85 (0.45) -0.29 (0.67) -0.62 (0.21) 0.22 (0.49)
Total energy (kcal) -68.05 (30.31) -38.90 (32.00) -47.33 (14.35) -17.31 (32.97)Fat (% energy) -0.33 (0.23) -0.13 (0.20) -0.01 (0.22)
-0.09 (0.21)Cholesterol -7.17 (2.55) -11.25(7.56) -3.68 (2.13) 1.48 (6.00)
Results of General Linear Models to Assess Gender Differences In Social Desirability Bias
Pre Measurements Post Measurements Women Men Women Men b SEb b SEb b SEb b SEbDependent Variable
Total fat(g) -3.36 (1.57) -1.11 (1.79) -2.06 (0.64) -0.35 (1.43)
Pre Measures Post MeasuresDependent Variable b Seb b SEb Fat
Quartile 1 -1.11 0.85 -1.15 0.64
Quartile 2 0.33 0.65 -0.29 0.32
Quartile 3 -1.42 1.12 -1.48 0.24
Quartile 4 -3.62 2.95 -1.65 1.08Total Energy
Quartile 1 -18.88 19.56 -22.17 5.76
Quartile 2 6.39 3.42 10.27 4.90
Quartile 3 -22.79 11.22 13.34 24.01
Quartile 4 -72.66 84.96 -13.02 45.27
Results of nutrient quartile stratification to assess variation in social desirability bias
A downward bias in the 7DDR due to social desirability was observed in women.
It amounted to about 700 kcal/day across the interquartile range of social desirability scores for women
WATCH – External Validation Study Conclusions:
The overall purpose of the Worcester Area Trial for Counseling in Hyperlipidemia was:
To evaluate the effectiveness of a physician-delivered nutrition intervention counseling program in reducing dietary fat and LDL
WATCH StudyHebert JR, Ma Y, Clemow L, Ockene IS, Saperia G, Stanek EJ, Merriam PA,Ockene JK. Gender differences in social desirability and social approval bias in dietary self report. Am J Epidemiol 1997; 146:1046-1055.
Over 8,000 patients screened to determine eligibility
Age 20-65 years
No prior drug treatment and RD referral
First fingerstick for testing cholesterol
Upper 25% of cholesterol distribution
Second fingerstick Consent
Data Collection in WATCH Study:
1,278 patients
One yearSingle 24HR7DDR2 lipid profileBMI(kg/m2 )SD andSA measuresMarch 1995
Baseline Single 24HR 7DDR 2 lipid profiles BMI(kg/m2 )
Variable Responders Non-responder P-value*
MEAN (SD) MEAN (SD)
Age (Years) 49.44 (10.63) 48.29 (10.45) 0.06Baseline BMI 28.80 (5.29) 29.51 (5.90) 0.05
Baseline BloodCholesterol (mg/dl) 233.60 (46.85) 223.48 (56.59)0.0008
Baseline LDLCholesterol (mg/dl) 152.82 (39.39) 148.89 (43.19) 0.01
# of extra lipidmeasures fromFallon 2.15 (1.26) 1.92 (1.22) 0.01
*P value based on Chi-square for categorical variables and two sample t-test the two group means
Responders Vs. Non-Responders
Male (n=325) Female (n=434)
MEAN SD# MEAN SD# P- value*
7DDR Nutrients at Baseline
Total Energy (kcal/d) 2085.63 927.83 1786.32 752.11 0.0001
Total Fat (g/d) 88.22 50.94 77.49 41.76 0.004
% Calories from Fat 37.11 8.96 38.17 8.73 0.12
Saturated Fat (g/d) 29.39 17.66 25.25 13.50 0.0009
% Calories from Saturated Fat 12.38 3.78 12.55 3.44 0.52
*P value based on two sample t-test the difference of two group means
Male and Female Comparison
Results of GLM for Males, WATCH Study
Independent Variables:
Dependent Variables: 24HR SD SCORE AP SCORE BMI
Total Fat (g/d) b 0.29 0.70 1.21 1.04 (p) (0.0003) (0.24) (0.01) (0.04)
Total Saturated Fat (g/d) b 0.26 0.25 0.45 0.50 (p) (0.0001) (0.23) (0.001) (0.05)
Total Energy (kcal/d) b 0.36 9.18 21.50 27.24 (p) (0.0001) (0.38) (0.01) (0.01)
Results of GLM for Females, WATCH Study
Independent Variables:
Dependent Variables: 24HR SD SCORE AP SCORE BMI
Total Fat (g/d) b 0.28 -0.78 -0.01 0.52 (p) (0.0002) (0.07) (0.98) (0.18)
Total Saturated Fat (g/d) b 0.26 -0.26 0.02 0.15 (p) (0.0001) (0.05) (0.78) (0.24)
Total Energy (kcal/d) b 0.27 -19.16 0.02 8.55 (p) (0.0002) (0.02) (0.99) (0.22)
In females: SDSCORE 75th percentile=23, 25th percentile=15, 8 points x 19.16=153.3 kcal energy underreport, or a 6.2g fat underreport
In males: APSCORE 75th percentile=41, 25th percentile=32, 9 points x 21.50=193.5 kcal overreport, or a 10.9g fat overreport
Some Perspectives
WATCH Social Desirability Study Conclusion
Social desirability was associated with a downward bias in dietary fat and energy intake in females
Social approval was found to be related to over-reporting energy and fat intake in males
Further studies are needed to establish models to adjust for the bias
The Treatwell 5-a-Day Study
Multi-ethnic sample of community health center workers
Representing three control sites
Multiple 24HR as “relative criterion”
Uses three methods for comparison, including Harvard/Channing FFQ
Predominantly (~85% women)
Hebert JR, Peterson KE, Hurley TG, Stoddard AM, Cohen N, Field AE, Sorensen G. The effect of social desirability trait on self-reported dietary measures among multi-ethnic female health center employees. Ann Epidemiol 2001; 11:417-427.
Black(n=23)
Hispanic(n=31)
Variable: b SEb b SEb
Total Energy Intake (kcal/d) 15.1 31.1 18.9 20.0
Total Fat Intake (g/d) 0.64 0.94 1.03 1.07
The Treatwell 5-a-Day Study – Social Desirability Results by Ethnicity, Women Only
White(n=30)b SEb
-4.5 15.9
-0.23 0.62
Non-Professional
(n=52)
Professional
(n=39)
Variable: b SEb b SEb
Total Energy Intake (kcal/d) 31.8 18.5 -20.6 14.5Total Fat Intake (g/d) 1.12 0.67 -0.19 0.57Fruit (servings/d) - FFQ 0.011 0.042 -0.004 0.037Fruit (servings/1000kcal/d) -0.008 0.031 0.002 0.029
The Treatwell 5-a-Day Study – Results by Occupational Category, Women Only
p-value for
Ho: <coll>coll
<0.005<0.05
nsns
Less Than College(n=52)
College Degree or More
(n=39)
p-value forH o: <coll>coll
Total Energy Intake (kcal/d) 36.1 20.0 -23.6 12.8Total Fat Intake (g/d) 1.23 0.78 -0.50 0.41Fruit (servings/d) - FFQ -0.003 0.046 -0.027 0.033Fruit (servings/1000kcal/d) -0.005 0.032 -0.002 0.026
b SEb b SEbVariable:
<0.001<0.001nsns
The Treatwell 5-a-Day Study – Results by Education, Women Only
The Treatwell 5-a-Day Study – Conclusions
The FFQ also appears to be biased by social desirability in women, but …..
the critical factor determining the bias is education which is …..
more important than occupational category or ethnicity/race.
As in the WATCH study, bias is oriented toward fat/energy intake
The Energy Study, Worcester, MA - 1997
First such study to focus on the most widely used FFQ (NCI/WHI)
First study to focus on these biases employing stable isotope methods for comparison (TEE from DLW)
Hebert JR, Ebbeling CB, Matthews CE, Ma Y, Clemow L, Hurley TG, Druker S. Systematic errors in middle-aged women's estimates of energy intake: Comparing three self-report measures to total energy expenditure from doubly labeled water. Ann Epidemiol 2001; (In Press):00-000.
Overview of Study
days
0 71 14
Doubly-Labeled Water Metabolic Period
Baseline questionnaires
Demographic data (education)
Social desirability (Marlowe-Crowne Scale, 33-item, true/false)
Food frequency questionnaire (WHI)
Married 47 64.4White 72 98.6Pre-menopausal 41 56.2Bachelors Degree or more 33 45.2Employed Full Time 44 60.3Professional, Managerial Work 33 55.0Current Smoker 7 9.6Sedentary 38 52.1
Description of the Study Population, The Energy Study (N=73)
n %
InterquartileRangeMean Standard
Deviation Minimum 25% 75% Maximum
Age (years) 49.0 6.8 40 44 53 65
Body Mass (kg) 70.0 10.4 43.9 62.1 76.9 90.5BMI (kg/m2) 27.1 4.1 18.7 24.5 29.8 38.2Fat-Free Mass (kg) 42.4 5.1 32.3 38.1 46.3 53.7
Social Desirability Score 17.4 5.9 4.0 15.0 22.0 29.0
Further description of the Study Population, The Energy Study (N=73)
TEE from DLW (kcal/d) 380 1378 1830 2318 3337
24-Hour Recall-DerivedData (7-day average)Energy Intake (kcal/d) 1820 464 1147 1494 2002 3566Food Quotient 0.90 0.03 0.82 0.88 0.92 0.99
Day-0 Administration 1735 764 429 1229 2089 4986Day-14 Administration 1622 594 639 1186 2028 3703
FFQ Energy (kcal/d)
2102
InterquartileRangeMean Standard
Deviation Minimum 25% 75% Maximum
Further description of the Study Population, The Energy Study (N=73)
All Education Levels: Whole Sample (n=73) -36.6 (-65.7, -7.5)
Excluding “Outliers ” (n=69) -12.2 (-34.7, 13.1)
High Education (college +) Whole Sample (n=33) -73.3 (-113., -32.9)
-31.9 (-63.6, -0.2) Excluding “Outliers ” (n=31)
Social Desirability Bias (kcal/day/point) by EducationLevel (FFQ-Derived Energy Intake Versus TEE from
DLW, Beginning of Metabolic Period), The Energy Study (N=73).
All Education Levels: Whole Sample (n=73) -10.8 (-34.7, 13.1)
Excluding “Outliers ” (n=72) -13.7 (-35.8, 8.4)
High Education (college +) Whole Sample (n=33) -21.8 (-53.5, 9.9)
Social Desirability Bias (kcal/day/point) by Education Level (FFQ-Derived Energy Intake Versus TEE from DLW, End of Metabolic Period), The Energy Study
(N=73).
Social Desirability Bias
-150
-125
-100
-75
-50
-25
0
25
50
WholeSample
(n=75)High(n=33)
Low(n=42)
Education
Beginning
EndBia
s (k
cal/d
ay/p
oint
)
Revisiting WATCH --- Why?
Is there an effect of education when cut at college+?
What happens with these biases after an intervention?
Hebert JR, Ma Y, Ebbeling CB, Matthews CE, Ockene IS. Self-report data. Compliance in Healthcare and Research. Armonk, NY: Futura, 2001:163-179.
Social Approval Bias in Males, by Education, WATCH Study,
Worcester, Massachusetts, 1991-1995.
< College (n=150)
Social ApprovalScore
BMI
BaselineTotal Energy (kcal/day) 29.8 (0.003) 29.1 (0.07)Total Fat (g/day) 1.63 (0.004) 1.60 (0.07)Total SFA (g/day) 0.59 (0.003) 0.53 (0.09)One-year < College (n=112)
Total Energy (kcal/day) 36.7 (0.0003) 53.4 (0.001)Total Fat (g/day) 1.50 (0.004) 1.72 (0.04)Total SFA (g/day) 0.41 (0.02) 0.57 (0.05)
Social Approval Bias in Males, by Education, WATCH Study,
Worcester, Massachusetts, 1991-1995.
College (n=70)
Social ApprovalScore
BMI
BaselineTotal Energy (kcal/day) 8.6 (0.49) 48.6 (0.05)Total Fat (g/day) 0.58 (0.39) 3.87 (0.05)Total SFA (g/day) 0.26 (0.26) 1.34 (0.07)One-year College (n=56)
Total Energy (kcal/day) 19.9 (0.14) 33.7 (0.14)Total Fat (g/day) 1.05 (0.11) 0.70 (0.52)Total SFA (g/day) 0.25 (0.18) 0.15 (0.63)
College (n=220)
Social ApprovalScore
BMI
BaselineTotal Energy (kcal/day)Total Fat (g/day)Total SFA (g/day)One-year College (n=172)
Total Energy (kcal/day)Total Fat (g/day)Total SFA (g/day)
Social DesirabilityScore
-14.8 (0.14)-0.53 (0.34)-0.14 (0.45)
-3.6 (0.77)-0.57 (0.43)
-0.2 (0.97)-0.02 (0.95)0.03 (0.76)
11.0 (0.07)0.36 (0.32)0.14 (0.21)
6.9 (0.43)0.25 (0.61)0.05 (0.75)
11.1 (0.32)0.36 (0.58)0.21 (0.32)-0.15 (0.52)
Social Approval and Social Desirability Bias in Females, by Education, WATCH Study,
Worcester, Massachusetts, 1991-1995.
College (n=64)
Social ApprovalScore
BMI
BaselineTotal Energy (kcal/day)Total Fat (g/day)Total SFA (g/day)One-year College (n=53)
Total Energy (kcal/day)Total Fat (g/day)Total SFA (g/day)
Social DesirabilityScore
-24.3 (0.04)-1.28 (0.07)-0.53 (0.01)
-9.5 (0.54)-0.21 (0.80)-0.05 (0.86)
-2.9 (0.72)-0.34 (0.49)-0.10 (0.52)
-5.1 (0.61)-0.23 (0.67)-0.06 (0.74)
19.7 (0.11)1.42 (0.05)0.41 (0.07)
35.5 (0.04)1.98 (0.02)0.75 (0.01)
Social Approval and Social Desirability Bias in Females, by Education, WATCH Study,
Worcester, Massachusetts, 1991-1995.
WATCH Study Conclusions:
Education modifies the effect of the social desirability and social approval
The effects differ by gender
There appears to be a differential effect of the intervention on the bias according to gender and education
The Role of Social Desirability in Epidemiologic Confounding
SD Score PsychologicPredispositions
PhysiologicResponses(e.g., ImmuneFunction)
Disease
TrueDiet
ReportedDiet
Total Fat and Saturated Fat
-12
-10
-8
-6
-4
-2
0
Ch
ange
in f
at in
take
(% t
otal
ene
rgy)
-0.71 -0.26 -4.95 -2.13 -8.24 -2.73
±0.38
±0.14
±1.36
±0.49
±1.39
±0.50
Total Fat
Saturated Fat (n=645)
Never Referred < 3 Sessions 3 Sessions
WATCH Nutritionist Intervention:
Total Cholesterol and LDL
-0.75
-0.6
-0.45
-0.3
-0.15
0
0.15
Ch
ange
s in
ser
um
ch
oles
tero
l (m
mol
/L)
0.01
-0.02 -0.15 -0.13 -0.43 -0.48±0.03
±0.03
±0.12±0.11
±0.13±0.11
TC
LDL-C (n=555)
Never Referred < 3 Sessions 3 Sessions
WATCH Nutritionist Intervention:
Actual Changes in Total Cholesterol vs. 7DDR - Predicted Values
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
-9.9 -8.4 -9.1
Cha
nge
in T
otal
Ser
um C
hole
ster
ol
(mg/
dL)
ActualKeys Prediction
Hegsted Prediction
WATCH Nutritionist Intervention:
Variable P *
Self-reported data Fat intake (% energy) -0.22 0.002
Body weight (kg) -0.02 0.59
Measured data
Serum LDL-C (mmol/L) 0.004 0.48
Body weight (kg) 0.02 0.59
Table 4. Effects of social desirability on self-reported and measured change scores, WATCH Study, Worcester,
Massachusetts, 1991-1995.
* P-value for the test of H0:=0