Researching the Arts in

Criminal JusticeAndrew Miles

ESRC Centre for Research on Socio-cultural Change

University of Manchester

Issues for the Arts in Criminal Justice

• Demonstration of impact key to recognition from policymakers and funders

• Large volume of anecdotal evidence suggests the arts have a positive impact in the criminal justice arena

• But formal case-making is weak and unpersuasive

REACTTResearch into Arts & Criminal Justice Think Tank

• Unit for the Arts and Offenders • DCMS, ACE, DfES, (Home Office),

academics (e.g. CATR) , consultants• Strengthen the evidence base for the

arts as an effective medium for offender rehabilitation

• 3 point plan– Literature Review – 2004– Feasibility Study – 2005/6– Longitudinal Research Project – 2007ff

Arts in Criminal Justice Feasibility Study

• What is a ‘robust’ evidence base?• Is it achievable?• Design:

– 6 projects mixing setting, gender, artform

– ‘good order’ & ‘progression routes’– Logistical, practical and methodological

issues in carrying out effective research

The Evidence-based Practice Agenda

• ‘What works?’• ‘Outcome’ research – 2 year

reconviction rates• Hierarchy of research methods by

‘quality’ scale• ‘Gold Standard’ = Experimental

design• Randomised Control Trials

A tough context for the Arts

• Disparate• Inconsistent• Small scale• Short lived• Unembedded• Opportunistic • Resistant to standardisation

The RCT – Key Limitations

• Practical issues– Problem of controlling for and

randomising variables in complex social situations

• Epistemological issues– Statistical association between

variables is not an explanation of causality

Critical realism

• We need to know why and in what circumstances programmes affect subjects before they can be said to work – ‘what works for whom in what circumstances’

• This requires a generative model of causation - based on hypothesising and testing the relationship between the context, mechanisms and outcomes of an intervention

Critical realism as an alternative for the Arts in

Criminal Justice?• Rules context in• Process central to the

understanding of outcomes• Large numbers not vital• Takes individuals seriously• Methodologically inclusive

Researching the Arts in Criminal Justice

Contextual Issues• Arts projects difficult to recruit• No common structures or working

models• Shifting timetables and content• Unaccredited so lacking status within the

criminal justice system• Operation defined by institutional

requirements and convenience • Conflict of ethos between arts and

criminal justice organisations• Shared ambivalence towards research

Researching the Arts in Criminal Justice

Practical Issues• Arts organisations often lack or are unable

to articulate a clear methodology• Recruitment of participants is ad hoc,

unspecified or institutionally manipulated• Participant group often small and unstable• Control over access to participants and

resistance to ‘inappropriate’ methods• Accessing information about participants • Time and space for research

Design and Implementation of Research Methods

• Research design varied according to the circumstances and dynamics of each project

• 6 data gathering methods in various combinations trialed: participant profiling; psychometric testing; interviews, observation, diaries, tracking

• Design shaped by theory where aims and objectives specified, otherwise mechanisms derived inductively

Theoretical Framing

• Pro-social development– Resiliency theory– Performance theory

Profiling information

• Some – limited - profiling information obtained locally but access to centrally held (e.g. OASys) records not achieved

• Participant numbers too small to be utilised for sub-sampling

Psychometrics

• Rosenberg Self Esteem (1965); Locus of Control (1984); Eysenck Impulsivity Scale (1994)

• Quality of responses affected by suspicion, apathy or over-familiarity

Interview

• Insight into personal experience, motivation, justification

• Over time - developed trust and the articulation of personal narratives

• Used to contextualise and cross-reference other types of evidence

• Filming helped to draw out some and captured the dynamics of group interactions

Observation

• Observation frame and scoring matrix

• Structured, consistent insight into the dynamic characteristics of interventions and their impacts

• Practitioners found onerous and difficult to engage with

Diaries

• Unprompted and deeper level impressions of the intervention process

• Low completion rates• Female/male responses differ

markedly

Tracking

• Regular intervals post-intervention probing sustainability and transfer

• Profiling records, interview, questionnaire• Complex, time consuming, low success rate• Lines of communication and access difficult

to maintain• Problem of transfer and release information• Transience of addresses and mobile phone

numbers

Summary of Findings (1)• Positive shifts in engagement, confidence,

self-control, co-operation and reflection• Conditioned by offence-type, length of

sentence, experience of other interventions, educational ability, age, gender

• Younger and more vulnerable participants most affected

• Females benefited from the collective focus of interventions

• Sustained in some but mostly ‘for the duration’ and some negative effects

Summary of Findings (2)

• Culture and physical context of the intervention important – ‘non-judgmental space’

• Artform specific impacts - dance/trust; reading and story-telling/reflection and self-expression; dance & drama/alternative learning styles & liminal space

Conclusions (1)

• Home Office Model not feasible and not necessarily desirable – not so much scientific power as simple science

• Multi-method realist approach more sympathetic, workable and compelling

• Current limitations of structure, culture and context in the arts in criminal justice hinder systematic research of any kind

Conclusions (2)

• More scope for development of a critical realist model in community-based resettlement and youth-justice settings

• Logistical platform for effective research requires institutional championing and embeddedness

Conclusions (3)

• Particular methodological attention to:– the delineation of indicators of change– the development of arts-specific theoretical

insight– structured and consistent approaches to the

generation and analysis of qualitative evidence

– the quantification of qualitative outcomes– creating an explanatory synthesis from

different combinations of methods

Conclusions (4)

• Arts engagement with and clarification of programme aims and objectives

• The integration of evaluation as core element in programme design

• Support from specialist research teams or trained practitioner-researchers

• Long-term projects, longitudinal research, a tracking methodology with linked multi-agency support