presentation to randomised controlled trials in the social sciences: methods and synthesis york 2008...
Post on 20-Dec-2015
215 views
TRANSCRIPT
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Sarah Cotterill, Peter John and Hanhua Liu
Institute for Political and Economic Governance
http://www.ipeg.org.uk
How to get those recycling boxes out: a randomised
controlled trial of a door to door recycling campaign
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Overview
• Context and background
• Research design
• Canvassing and Participation Monitoring
• Preliminary results
• Costs
• Next steps
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Background and context
• Government target: 50% of household waste to be recycled or composted by 2020 (31% 2006/7)
• Key actions include “Culture Change”(Defra: Waste Strategy for England 2007)
• Citizen behaviour change essential for achieving better environmental outcomes
• Many environmental acts are low effort, but need reminders, habit or brief contacts to activate
• Kerbside recycling popular ….. but not everyone does it
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Encouraging recycling
• A variety of means to encourage recycling: incentives, campaigns and leaflets
• Face to face door knocking with a professional team can impact on behaviours like voting (John and Brannan, 2008)
• Canvassing has been found in other studies to raise recycling rates (Bryce et al 1997)
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
EMERGE Recycling Service
• Wide range of materials
• Boxes and bags
• Weekly collection
• Visibly different vehicles, sorted on street
• Promotion of the scheme
Quality of recycling service affects recycling rates(Harder et al 2006; Woodward et al 2005)
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Old Trafford and Gorse Hill
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
The research site
Old Trafford– 53% white– Relatively deprived (some parts in lowest 6%
of English neighbourhoods)– Mix of terraced and semi-detached housing
Gorse Hill– 85% white– Less deprived (in the lowest 30% nationally)– Mix of terraced and semi-detached housing
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Sample and randomisation
• 6580 households in 194 streets
• Street based design: we expect street effect
• Streets randomly assigned to– Canvass group (97 streets, 3468 houses)– Control group (97 streets, 3112 houses)
• Stratified by district (Old Trafford/Gorse Hill) and street length
• Thanks to the York Trials Unit for doing the random assignment
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Canvassing
• May/June 2008 6 weeks
• 4 canvassers recruited and trained
• 3pm-7pm Mon-Fri 11am-3pm Sat
• Whole area canvassed twice
• Spoke to 2129 of the 3468 households (61% of households)
• Awareness, Attitudes, Barriers (Recyclers and Non-Recyclers) (Shaw et al 2007)
• Leaflets
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Measurement
• Participation Monitoring (WRAP 2006)
– Same day as recycling collection– Independent monitor– 3 weeks– Participation = household recycles at least once
• March/April 2008
• July 2008
• October 2008
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
CONSORT Flow Diagram
List of streets N = 209 streets
(N = 6616 houses)
Randomised streets N = 194 streets
N = 6580 houses
Dataset amendments: Streets without houses (N = 11)
Duplicate streets (N = 4) Non residential properties
Canvass Group N = 97 streets
N = 3468 houses
Control Group N = 97 streets
N = 3112 houses
Canvassed N = 97 streets
N = 2129 houses (61%)
Participation Monitoring 97 streets monitored pre- and
post- intervention
Participation Monitoring 97 streets monitored pre- and
post- intervention
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Preliminary Results: frequencies
Households Recycled before
Recycled after Change
Canvass Group
3468
1619 46.7%
1852 53.4%
233 6.7%
Control Group
3112
1682 54.0%
1651 53.1%
-31 -1.0%
Canvassing raised recycling by 7.7%
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Preliminary Results: neighbourhood effects (canvass group only)
Households Recycled before
Recycled after Change
Gorse Hill 1225 (35%)
677 55.3%
685 55.9%
8 0.6%
Old Trafford
2243 (65%)
942 42.0%
1167 52.0%
225 10%
Canvassing was more effective in raising recycling rate in Old Trafford than in Gorse Hill.
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Variables involved in the analysis
• Outcome variable: – change in the means of participation rate
between the pre- and post-intervention periods
• Explanatory variables:– Group (coded 0=Control Group, 1=Canvass
Group) – District (coded 0=Old Trafford, 1=Gorse Hill)– Street size (total number of households per
street)
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Analysis
• Weighted test: tests whether there is a significant
difference between the intervention and control groups
• Regression analysis using the robust standard error or
Huber-White standard errors method: examines how the
outcome variable is predicted by the three explanatory
variables Group, District and Street size.
• Analyses take into account both the variation in street
size and the clustering data structure.
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
0.70.60.50.40.30.20.10.0-0.1-0.2-0.3-0.4-0.5-0.6-0.7
Observed Value
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
-2.0
-2.5
-3.0
-3.5
-4.0
-4.5
-5.0
-5.5
Exp
ecte
d N
orm
al
Figure 2. Normal Q-Q Plot of change in the means of recycling participation rate
Distribution of change in means of participation rate
Normal Q-Q Plot of change in the means of recycling participation rate
Kolmogorov-Smirnov D statistic 0.099, p=0.000
Shapiro-Wilk W statistic 0.954, p=0.000
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Preliminary Results 1: Weighted test of the difference in recycling rates before and after the intervention
95% CI (SE ) t p
Lower Upper
Group (canvass/control)
0.077 (0.015)
5.220 0.000 0.048 0.106
Intercept (cons) -0.010 (0.011)
-0.930 0.354 -0.031 0.011
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Preliminary Results 2: Regressions with Group, District and Street Size
95% CI (SE ) t p
Lower Upper
Group (canvass/control)
0.072 (0.013)
5.420 0.000 0.046 0.098
District (Old Trafford/Gorse Hill)
-0.049 (0.014)
-3.610 0.000 -0.076 -0.022
Street Size
0.000 (0.000)
1.780 0.077 0.000 0.001
Intercept
(cons)
-0.007 (0.017)
-0.450 0.657 -0.040 0.025
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Preliminary Results: the intraclass correlation coefficient (ICC)
95% CI Source SS df MS F p ICC
Lower Upper
Between street
78.582 193 0.407 1.580 0.000 0.017 0.007 0.026
Within street
1649.217 6386 0.258
SS: square sum
df: degree of freedom
MS: mean square
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Preliminary Conclusions
• Canvassing successfully raised recycling
participation
• Canvassing was most successful in Old Trafford
• Street effect is not as strong as we predicted
• Street size did not affect the change in
participation
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Costs
• Canvassing costs = £5605.59– Wages of 4 canvassers– Additional hours for Emerge supervision– Expenses (phone, training)– (not bags, boxes, jackets, data entry)
• Additional recyclers = 233 households
• Cost for each new household that started recycling = £24.06
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Next steps
• Participation Monitoring October 2008– Recycling decay?
• Further Analysis– Ethnicity, Poverty, Street effects
• Dissemination of findings
Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008
Sarah Cotterill, Peter John and Hanhua Liu
Institute for Political and Economic Governance
http://www.ipeg.org.uk
How to get those recycling boxes out: a randomised
controlled trial of a door to door recycling campaign