Formal experiments: randomisation + study size
Concept of randomisation• Biology, 1926: Sir Ronald Fisher
• Medicine, 1947: Sir Austin Bradford Hill Randomised Controlled Trial
• Criminal justice ?
Randomisation in medicine
• Toss of coin determines experimental or control treatment RCT assignment unpredictable
• Fair [=> ethical] allocation of scarce resource
• Balance treatment numbers overall, in each hospital, and for major prognostic factors
RCT Telephone Randomisation
Licensing of pharmaceuticals: requires efficacy in RCTs
• Patients’ informed consent [ethics]
• Sufficiently-large [precise answer]
• Randomised [unbiassed: like with like]
“What works” in UK criminal justice?
Large RCTsessentially untried . . .
(bar restorative justice)
Judges prescribe sentence on lesser evidence than doctors
prescribe medicines
Is publicaware?
Drug Treatment &Testing Orders (DTTOs)
• England & Wales: 210 clients
• Scotland: 96 clients
• Targets for DTTO clients in E&W: 6,000+ per annum
• DTTO clients: 21,000+ by end 2003
RSS Court DTTO-eligible offenders: do DTTOs work ?
• Off 1 DTTO• Off 2 DTTO• Off 3 alternative =• Off 4 DTTO• Off 5 alternative =• Off 6 alternative =
Count offenders’ deaths, re-incarcerations etc . . .
UK courts’ DTTO-eligible offenders: ? guess
• Off 7 DTTO [ ? ]• Off 8 DTTO [ ? ]• Off 9 DTTO [ ? ]• Off10 DTTO [ ? ]• Off11 DTTO [ ? ]• Off12 DTTO [ ? ]• Off13 DTTO [ ? ]• Off14 DTTO [ ? ]
(before/after) Interviews versus . . . [ ? ]
Evaluations-charade• Failure to randomise
• Failure to find out about major harms
• Failure even to elicit alternative sentence funded guesswork on relative cost-effectiveness
• Volunteer-bias in follow-up interviews
• Inadequate study size re major outcomes . . .
Power (study size) matters!
Back-of-envelope sum for 80% power
e. g. Percentages
If MPs/social scientists don’t know,
UK plc keeps hurting
For 80% POWER, 5% significance: compare failure (re-conviction) ratesRandomise per treatment group,
8 times STEP 1 answer
STEP 1: Success * fail rate + Success * fail rate for new disposal for control ------------------------------------------------------------(new success rate – control success rate)2
DTTOs: target 60% versus control 70% re-conviction rate?Randomise per ‘CJ disposal’ group, 8 times STEP 1 answer = 8 times 45 = 360
STEP 1 answer:
40 * 60 + 30 * 70 2400 + 2100 DTTOs control --------------------------------- = --------------- (40 – 30)2 100
Four PQs for every CJ initiative• PQ1: Minister, why no randomised controls?
• PQ2: Minister, why have judges not even been asked to document offender’s alternative sentence that this CJ initiative supplants?
{cf electronic tagging}
• PQ3: What statistical power does Ministerial pilot have re well-reasoned targets?
{or, just kite flying . . .}
• PQ4: Minister, cost-effectiveness is driven by longer-term health & CJ harms, how are these ascertained? { database linkage}
Randomised controlled trials
to
police Policy
by
Home OfficePrisons
&
Criminal Justice
HMP Peterborough PILOT: Kalyx prison, Social Finance run, & payment by results.What happens?
HMP Peterborough Eligible Pre-release Inmate (serving less than 12 months)
Taken on by SF
Yes No
Transfer’dOut . . .
No Yes No Yes
Deselected
by SF
No Yes N/A orYes
AlreadyDone!
N/A
HMP Peterborough release
YES Yes, but . . .
NO Yes, but . . .
NO
Matchedcontrols?
? ?
HMP Peterborough PILOT: Kalyx prison, Social Finance run, & payment by results.
Per SF-release, comparators are? 10 same-sex ‘matched’ prisoners who also served
less than 12 months & were released on same day but from other prisons
{All Kalyx-run? Where? Functionality? Locality?De-selections & transfers?}
Reduce convictions in 1st year post-release by 7.5% . . . Conviction costed how . . . ???
{eg 60% to 55.5% convicted within 1 year of release ~ or reduce conviction-count by 7.5%}
Guardian Society: 17 Nov. 2004
“Some statisticians are so severe that they would stop social policy
making in its tracks.
For example, Bird would forbid the government to introduce any policy that had not been assessed through
controlled trials. . . ”
SIMPLE RANDOMISATIONSTEP 1: Correspondence between random
number (see tables) & CJ disposal: EVEN random number (0) DTTO ODD random number alternative
STEP 2: Document starting point in tables & direction of reading: SMB = down
03 (row) 07 (column)
RANDOM NUMBER tables72137 73850 3273348083 50731 5058416602 26772 81250 row 3, column 7:55480 29910 89693 read down77708 83761 47184
12601 5443265664 73669
SIMPLE RANDOMISATION: down
Offender Random # Disposal Outcome Court & JCJ01 6 DTTO CB2CJ02 9 alternative CB2CJ03 3 alternative CB2CJ04 4 DTTO CB2CJ05 3 alternative CB2CJ06 etc CB2CJ07 CB2CJ08 CB2Correspondence: random # & disposal
EVEN = DTTO
ODD = alternative
Row 03Col 07
Randomisation by minimisation: next client = Cambridge, male,
18-24 years, >5 prison termsClient RCT assignmentscharacteristic so far to DTTO alternative
Cambridge 20 15 *male 205 190 *18-24 years 100 * 108>5 in-prison 180 * 185
SUM points: 505 498 **
RANDOMISE this client preferentially (eg 80:20) to ** because lower on points
Critical reading: BMJ, Lancet etcStatistical guide-lines for contributors to medical
journals (20+ years): beware “bars”
Structured ABSTRACT: essential design & primary outcome (s)
CONSORT flowchart for reporting RCTs: beware i) post-randomization exclusions from analysis;
ii) post-hoc subgroups.
STROBE guide-lines for reporting observational studies: beware bias in many guises
(especially: how explanatory variable is coded at analysis - eg binary!! & retrospective classification: deaths on transplant waiting list ~ count
survival contribution on the waiting list of transplantees beofre operation)