lessons from reviewing and synthesising evidence on crime reduction
TRANSCRIPT
Lessons from reviewing and synthesising
evidence on crime reduction
What Works in Crime Reduction Conference
January 23rd 2017, British Library
Dr Lisa Tompson
UCL DEPARTMENT OF SECURITY AND CRIME SCIENCE
Lessons from reviewing and synthesising
evidence on crime reduction[ ramblings in a post-EMMIE, post-truth world ]
Overview
• The evidence landscape ground zero
• Assembling the ‘gold standard’ of the evidence base
• The ‘EMMIE-ability’ of the current crime reduction evidence
base
– At systematic review level
– At primary study level
A philosophical minefield
(So-called) evidence hierarchies
RCTs
Quasi-experimental
design
Controlled observational
studies
Observational studies (no control group)
Expert opinion
Systematic reviews
and meta-analysis
Systematic reviews
• “a review of research literature using
systematic and explicit, accountable
methods” (Gough, Oliver and Thomas, 2012:2)
• Find, sort, sift and synthesise
• Supposedly overcome the limitations of
individual studies and assess the evidence
base as a whole
Searching for SRs - inclusion criteria
• Systematic review or meta-analysis
• Measured outcome of crime prevention/
crime reduction
• Focused on a single intervention
~17,000 studies
838 studies
337
studies
82 studies
Sample (82 SRs of single interventions)
EMMIE
• Effect
– Effect direction and size
• Mechanism
– Mechanism/s activated (why it worked)
• Moderator
– Moderators/contexts that activate the mechanism/s
• Implementation
– Implementation conditions that support or obstruct delivery
• Economic
– Economic assessment of the cost effectiveness
or cost-benefit ratio of what is delivered
Effect Size (EMMIE)
Mechanisms (EMMIE)
Moderators (EMMIE)
Implementation (EMMIE)
Economics (EMMIE)
What have we learnt?
• Appraising quality is subjective automated Q-scoring
helps to standardise
• Most reviews don’t use the language of EMMIE
– Different fields have very different reporting conventions
• The evidence is generally weak on effect, and often on
other dimensions, BUT:
– Reviews can be strong on (say) moderators or implementation,
even if effect is weak
– We need to remember that reviews rely on primary study evidence
The misnomer of the ‘single intervention’
• Any intervention applied to people in the real world is
fraught with complexity and variation
After-school
programmes
to prevent
delinquency
Effect
The misnomer of the ‘single intervention’
• Any intervention applied to people in the real world is
fraught with complexity and variation
After-school
programmes
to prevent
delinquency
Effect
What’s the problem?
The misnomer of the ‘single intervention’
After-school
programmes
to prevent
delinquency
Effect
After-school
programmes
to prevent
delinquency
After-school
programmes
to prevent
delinquency
What’s in primary studies for EMMIE?
• To date we have done systematic reviews on:
Effect Size (EMMIE)
• Randomised control trials are scarce in crime prevention
– None in anti-theft tag literature
• Threats to bias are common
StudySelection
bias
Measurement
bias
Regression
to the
mean
Contamination
effects
Farrington et al. 1993 Medium Low Medium Medium
Bamfield, 1994 High Medium Unclear Medium
DiLonardo and Clarke, 1996 Medium Medium Unclear Medium
Hayes and Blackwood, 2006 Medium Low Low Medium
Beck and Palmer, 2011 Medium Medium Medium Medium
Retailer A 2015 Medium High High Unclear
Retailer B 2015 High High High Unclear
Mechanism-related information in the studies
19% Yes,
alluded to
48% Not
mentioned
• 25 studies mentioned tag-
related mechanisms
• We spoke to retailers and
loss prevention managers
to corroborate the
literature
Mechanism > Effect
Increase the
effort
Increase the risks Reduce the
rewards
Reduce
provocation
Remove excuses
1. Target harden 6. Extend
guardianship
11. Conceal
targets
16. Reduce
frustrations and
stress
21. Set rules
2. Control access
to facilities
7. Assist natural
surveillance
12. Remove
targets
17. Avoid
disputes
22. Post
instructions
3. Screen exits 8. Reduce
anonymity
13. Identify
property
18. Reduce
emotional
arousal
23. Alert
conscience
4. Deflect
offenders
9. Utilize place
managers
14. Disrupt
markets
19. Neutralise
peer pressure
24. Assist
compliance
5. Control
tools/weapons
10. Strengthen
formal surveillance
15. Deny
benefits
20. Discourage
imitation
25. Control drugs
and alcohol
Mechanism > Effect
Increase the
effort
Increase the risks Reduce the
rewards
Reduce
provocation
Remove excuses
1. Target harden 6. Extend
guardianship
11. Conceal
targets
16. Reduce
frustrations and
stress
21. Set rules
2. Control access
to facilities
7. Assist natural
surveillance
12. Remove
targets
17. Avoid
disputes
22. Post
instructions
3. Screen exits 8. Reduce
anonymity
13. Identify
property
18. Reduce
emotional
arousal
23. Alert
conscience
4. Deflect
offenders
9. Utilize place
managers
14. Disrupt
markets
19. Neutralise
peer pressure
24. Assist
compliance
5. Control
tools/weapons
10. Strengthen
formal surveillance
15. Deny
benefits
20. Discourage
imitation
25. Control drugs
and alcohol
Contextual variations that Moderate the effect
• Type of tag
– Hard vs. soft vs. ink tags
– Overt (with or without warning message) vs. discreet
• Type of shop
– Layout
– Staff availability and willingness to intervene
– Other security: signs, guards, CCTV
• Tagging strategy
– Product lines
– Dosage
– Source-tagged vs. tagged in store
Implementation > Mechanism > Effect
Increase the
effort
Increase the risks Reduce the
rewards
Reduce
provocation
Remove excuses
1. Target harden 6. Extend
guardianship
11. Conceal
targets
16. Reduce
frustrations and
stress
21. Set rules
2. Control access
to facilities
7. Assist natural
surveillance
12. Remove
targets
17. Avoid
disputes
22. Post
instructions
3. Screen exits 8. Reduce
anonymity
13. Identify
property
18. Reduce
emotional
arousal
23. Alert
conscience
4. Deflect
offenders
9. Utilize place
managers
14. Disrupt
markets
19. Neutralise
peer pressure
24. Assist
compliance
5. Control
tools/weapons
10. Strengthen
formal surveillance
15. Deny
benefits
20. Discourage
imitation
25. Control drugs
and alcohol
Conclusions
• Extracting information that speaks to EMMIE is a murky
business
• Synthesising EMMIE data requires in-depth understanding
– Of crime prevention theory
– Of evaluation theory
• Systematic review evidence depends on primary studies,
so it is crucial that we raise awareness of EMMIE
• Qualitative research is crucial for understanding the unique
qualities of interventions – we ignore their value at our peril
Thank you
Aiden Sidebottom, Amy Thornton, Lisa Tompson, Jyoti Belur,
Nick Tilley, Kate Bowers and Shane D. Johnson
University College London
UCL DEPARTMENT OF SECURITY AND CRIME SCIENCE