‘evidence-informed policing: an introduction to emmie and ... · 4 wilcoxon signed-rank test is a...

1

What Works Centre for Crime Reduction

Work Package 7 Final Report

‘Evidence-informed Policing: An Introduction to EMMIE and

the Crime Reduction Toolkit’

A Pilot Training Evaluation

Fleming, J., Fyfe, N., & Wingrove, J.C.

2

This document provides:

- An introduction to the requirement for the pilot training,

- The methodology used to evaluate the pilot training

- The findings in the context of police reaction to the training programme and evidence-

informed policing

- Recommendations for the College, individual forces, police trainers and academia

- The pilot evaluation materials as appendices

COPYRIGHT

This is an open-access article distributed under the terms of the Creative Commons Attribution

4.0 International License, which permits unrestricted use, distribution and reproduction in any

medium, provided the original author(s) and source are credited. In addition there should be an

indication if any changes were made to the original text and there should be no suggestion that

the authors or any of their collaborators endorses you or your use.

This research was co-funded by the College of Policing and the Economic and Social Research

Council (ESRC); Grant title: 'University Consortium for Evidence-Based Crime Reduction'.

Grant Ref: ES/L007223/1.

Fleming, J., Fyfe, N., & Wingrove, J. (2016a). Evidence-informed Policing: A Pilot Training Evaluation

Executive Summary

Introduction

In March 2013 the Cabinet Office launched the ‘What Works Network’, a nationally co-ordinated initiative

aimed at positioning the research evidence on ‘what works’ at the centre of public policy decision-

making, developed in a political environment increasingly amenable to the idea of evidence-based

decision-making, particularly in the context of ‘austerity’ and cost effectiveness. Currently there are seven

research centres1 focusing on six key areas of public policy, intended to build on existing models of

delivering evidence-based policy - such as the well-established and well-funded National Institute for

Health and Clinical Excellence (NICE).

For some years now, the College of Policing (the College) and its predecessor the National Policing

Improvement Agency (NPIA) have been promoting the importance of research evidence to inform

practice in policing and crime reduction. The College Five Year Strategy (2014a) outlines its intentions to

promote understanding of ‘what works’ in policing and use this understanding to set standards and

enable members to share knowledge and evidence around ‘what works’ (College of Policing, 2014a).

Within a context of austerity and changing policing demands (e.g. Brain & Owens, 2015;

http://www.college.police.uk/Documents/COP_infographic.pdf) the College is promoting

‘professionalisation’ of police.

As part of achieving many of the aforementioned aims, the ‘What Works Centre for Crime Reduction’

(WWCCR) was established in 2013 to develop a strong evidence base for decision-making around crime

reduction. It is led by the College and supported by a Commissioned Partnership Programme2 (CPP). A key

component of the WWCCR programme is the development and piloting of a Police Development

Programme (Work Package 6/7) to enable police officers to appraise and use evidence to inform their

decision-making.

The purpose of the current research was, therefore, to design, pilot and evaluate the implementation of a

police development training programme to develop officers’ theoretical and practical understanding of

evidence-informed approaches, equip them with the skills required to use evidence to inform their

decision-making and support them to appraise evidence and commission research (Hereafter referred to

as ‘Evidence-based Policing’: EBP3). See Fleming, Fyfe & Wingrove (2016a) for an overview of the design

methodology and the training programme outline.

1 National Institute for Health and Care Excellence (NICE), Sutton Trust/Educational Endowment Foundation, College of Policing What Works Centre for Crime Reduction, Early Intervention Foundation, What Works for Local Economic Growth, the Centre for Ageing Better, and the What Works Centre for Wellbeing. 2 University College London, Institute of Education University of London, London School of Hygiene & Tropical Medicine, University of Southampton, Birkbeck, University of Surrey, Cardiff University and University of Dundee. 3 The term Evidence-Informed Policing is preferred by the authors of this report (see Fleming, Fyfe & Wingrove, 2016a for further information). However, to avoid confusion when discussing the wider literature in this area, the term Evidence-based Policing is used throughout this report.

http://www.college.police.uk/Documents/COP_infographic.pdf


4

This report focuses on the evaluation of the pilot training programme. The main objective of the

evaluation of the pilot training was to assess and understand trainee reaction and to note aspects of

learning following training.

The following research questions were addressed:

To what extent do police officers/staff respond positively to EBP?

To what extent do police officers/staff believe EBP training will enhance their role?

To what extent does the training enable police officers/staff to use the Crime Reduction Toolkit?

Methodology

Four groups of police officers/staff across four forces (N=90) took part in the pilot training. Evaluation

involved pre and post-training repeated measures assessing:

- Reaction and immediate (and potentially short-term) attitudinal outcomes,

- Observational speculation on participant learning, and

- Self-reported planned behaviour.

Quantitative and qualitative data was collected through pre and post-training questionnaires completed

by the participants as well as through participant observation by the researchers. Wilcoxon signed-rank4

tests were conducted to examine whether any statistically significant differences were present in

participant beliefs before and after the pilot training and Fisher’s exact5 tests were undertaken to explore:

Differences in awareness of and participation in College activities and use of research/evaluation

across demographical and individual characteristics (e.g. force, rank, years in service, level of

education).

Differences in pre and post-pilot training beliefs across demographical and individual

characteristics (e.g. force, rank, years in service, level of education)

Qualitative data was analysed thematically using a grounded theory approach (Glaser, 1978, 1992;

Glaser & Strauss, 1967; Strauss, 1987). Grounded theory refers to a qualitative analysis approach that

develops theory based on the data, in which findings emerge, as far as possible, without the researcher

imposing a pre-existing theory onto the data (Strauss & Corbin, 1998).

Findings

This report provides the reader with an overview of the current organisational context and demographics

of each pilot force in order to aid interpretation of the overall findings. This will also enable the easy

provision of an edited report for each pilot force, which includes a brief overview of any findings specific

to their force. However, the sample sizes of each individual force do not warrant individual analysis and

any significant findings with regard to variance between the forces have been identified through

consideration of the sample as a whole.

4 Wilcoxon signed-rank test is “a non-parametric test’ that looks for differences between 2 related samples” (Field, 2013, p.886). It is used here to compare repeated measures of ordinal variables, related to the same participants. 5 Fishers exact test is “a way of computing the exact probability of a statistic” (Field, 2013, p.875) and is used when there are small samples within the grouping categories which are being explored. Here it is used to understand the likelihood of the variance in categorical variables.


5

Quantitative findings

Findings suggest that those keen to participate in EBP related training/activities are likely to have a higher

level of education and that, of the ‘rank and file’ ranks, Constables may be more open to EBP. It is worth

considering the characteristics which have impacted on who has proactively chosen to be involved in the

pilot training, as these may reveal which police officers/staff could be most easily reached by

communications and activities intended to advance EBP practice within the profession.

A high percentage of participants had positive beliefs about the relevance and value of the training,

research/evaluation and their own research abilities prior to the pilot training. This level of positivity prior

to the pilot training appeared to reduce in a small but statistically significant way following the training. It

is suggested that once participants realised the time and resources required to implement what they had

learned, they were less sure they would have the support of their senior management to achieve this.

Across all of the statements regarding perceptions of the training itself (e.g. “the training was

interesting”; “I am more able to assess the effectiveness of a crime reduction programme”; “The Case

Study task was effective”), there was an average of 83.66% positive feedback.

Qualitative findings

Many of the themes identified during the pilot training were similar to those identified in initial focus

groups undertaken by the research team (Fleming & Fyfe, 2015).

However, the pilot training revealed a stronger level of positivity around the idea of using research to

inform practice, having a toolkit to use and interactions with others in this way.

‘Experience’ is a very key component of officer responses to EBP and will need to be considered when

trying to implement any changes in practice. The role of the trainer is to pre-empt, and have an

understanding of, the staff/officer beliefs.

Officers are open to evaluation of their practice, but there will need to be significant changes to

organisational practice and performance management to allow this to become part of everyday practice

or understanding. Police will need to be confident in their ability to progress their career whilst practicing

EBP approaches, including when evaluating interventions which do not lead to successful outcomes.

Teaching will need to be imparted by ‘credible voices’. Participants were very keen to have real

operational examples of EBP, and appear to rely heavily on their peers for credible experience.

Individual forces and the attitude of senior management will be key to the ‘uptake’ of EBP, as resourcing

and support mechanisms will need to be in place for officers to act upon the understanding of EBP they

developed through the pilot training, in order to apply it in practice.

Recommendations

More clarity is required in the wider professionalisation agenda and how it will impact individual

forces and roles, to ensure police feel they have the organisational support required to

implement an EBP approach.


6

Emphasis should be placed on the role evidence can play in informing practice. It is vital that the

wider organisational change and professionalisation agenda communicate that the intention is

for evidence to be integrated into practice, to be drawn on alongside other aspects of evidence

including the officer’s experience.

The training should remain an interactive session, taking place over a two day period in order to

allow trainees to take in the information.

The delivery of the training should be decentralised/devolved to local forces

The trainer should ensure facilities for the training day are functional and well-designed.

Real, operational examples of EBP should be included in the training and ideally be delivered by

operational officers.

Credible trainers should be aware of, and pre-empt, common trainee concerns and perceived

benefits of EBP.

Given their commitment and participation, it would be beneficial for the individual pilot forces to

be given the full evaluation report.

Various recommendations are made in the report with regard to academia and potential future

research:

o Developing or making recommendations with regard to training police officers and staff

in applied evaluation skills, particularly exploring what is an appropriate, viable, timeline

in which individuals without a background in scientific research could build up such

knowledge

o Evaluate longitudinal outcomes of any implementation of EBP training

o Expand the literature in areas most relevant to operational policing , using EMMIE in both

primary research and systematic reviews


7

Table of Contents 1 Introduction ........................................................................................................................................ 10

1.1 UK Government Context ............................................................................................................. 10

1.2 UK Policing Context ..................................................................................................................... 10

1.3 What Works Centre for Crime Reduction ................................................................................... 11

1.4 The current Research .................................................................................................................. 12

2 Methodology ....................................................................................................................................... 13

2.1.1 Evaluation Measures ........................................................................................................... 13

2.1.2 Participant sample ............................................................................................................... 14

2.1.3 Data collection procedure ................................................................................................... 14

2.1.4 Evaluation data analysis ...................................................................................................... 14

2.1.5 Evaluation Limitations and Future Recommendations ........................................................ 15

3 Individual Force Overviews and Findings ............................................................................................ 17

3.1 Police Scotland ............................................................................................................................ 18

3.1.1 Organisational context and demographics ......................................................................... 18

3.1.2 Pilot training programme .................................................................................................... 19

3.1.3 Summary ............................................................................................................................. 22

3.2 Hampshire Constabulary ............................................................................................................. 23



3.2.3 Summary ............................................................................................................................. 27

3.3 Sussex Police ............................................................................................................................... 29


3.3.2 Pilot training programme findings ...................................................................................... 29

3.3.3 Summary ............................................................................................................................. 33

3.4 Police Service of Northern Ireland .............................................................................................. 34



3.4.3 Summary ............................................................................................................................. 38

4 Overall Results & Discussion ............................................................................................................... 39

4.1 Demographic Information ........................................................................................................... 39

4.2 Quantitative Findings .................................................................................................................. 41


8

4.2.1 Descriptive findings ............................................................................................................. 41

4.2.2 Pilot findings........................................................................................................................ 47

4.2.3 Quantitative Findings Summary .......................................................................................... 54

4.3 Qualitative Findings .................................................................................................................... 55

4.3.1 Perceived benefits of training ............................................................................................. 55

4.3.2 Perceived limitations of training ......................................................................................... 59

4.3.3 Suggested improvements to training .................................................................................. 62

4.3.4 Feedback on the Crime Reduction Toolkit .......................................................................... 65

4.3.5 Reflections on EBP .............................................................................................................. 67

4.3.6 Qualitative Findings Summary ............................................................................................ 73

5 Recommendations .............................................................................................................................. 74

5.1 The College .................................................................................................................................. 74

5.2 Individual Forces ......................................................................................................................... 75

5.3 Police Trainers ............................................................................................................................. 75

5.4 Wider Academia .......................................................................................................................... 76

6 References .......................................................................................................................................... 78

7 Appendices .......................................................................................................................................... 82

7.1 Appendix A: Pre-pilot Training Questionnaire ............................................................................ 82

7.2 Appendix B: Post-pilot Training Questionnaire .......................................................................... 86

7.3 Appendix C: Observation Topic List ............................................................................................ 89

7.4 Appendix D: Force Information Sheet ........................................................................................ 91

7.5 Appendix E: Participant Information Sheet & Consent Form ..................................................... 94

7.6 Appendix F: Training Slides ......................................................................................................... 96


9

Figure 1: Pie chart illustrating percentage of participants per force .......................................................... 39

Figure 2: Pie chart illustrating percentage of participants per rank/role ................................................... 39

Figure 3:Stacked bar chart illustrating rank/role by force .......................................................................... 40

Figure 4: Clustered bar chart illustrating gender by rank/role ................................................................... 40

Figure 5: Stacked bar chart showing level of education by force ............................................................... 41

Figure 6: Stacked bar chart illustrating method of enrolment by rank ...................................................... 42

Figure 7: Stacked bar chart illustrating method of enrolment by force ..................................................... 42

Figure 8: Stacked bar chart illustrating method of enrolment by level of education................................. 43

Figure 9: Stacked bar chart illustrating awareness of EBP by rank/role ..................................................... 44

Figure 10: Pie chart illustrating percentage of participants who had heard of POLKA per rank/role ........ 45

Figure 11: Stacked bar chart illustrating percentage of use of POLKA by level of education ..................... 46

Figure 12: Stacked Bar chart Illustrating Percentage of Responses to “The skills I learnt during this course

will be valued by my force” per Force ........................................................................................................ 49

Figure 13: Clustered bar chart illustrating frequency of responses to "research can be useful to my role"

before and after training............................................................................................................................. 50

Figure 14: Clustered bar chart illustrating frequency of responses to "research is important to my role"

before and after training............................................................................................................................. 51

Figure 15: Clustered bar chart illustrating frequency of responses to "research can be used to inform my

day-today work" before and after training ................................................................................................. 51


10

1 INTRODUCTION

1.1 UK GOVERNMENT CONTEXT In March 2013 the Cabinet Office launched the ‘What Works Network’, a nationally co-ordinated initiative

aimed at positioning the research evidence on ‘what works’ at the centre of public policy decision-

making. Currently there are seven research centres6 focusing on six key areas of public policy. These

‘research hubs’ are intended to build on existing models of delivering evidence-based policy - such as the

well-established and well-funded National Institute for Health and Clinical Excellence (NICE). This provides

independent evidence-based guidance to the NHS and health professionals about the targeting of funding

and the most effective ways to prevent, diagnose and treat disease and ill health.

The What Works centres are being developed in a political environment increasingly amenable to the

idea of evidence-based decision-making, particularly in the context of ‘austerity’ and cost effectiveness.

Government White Papers (e.g. Cabinet Office, 2011; HM Government, 2012) assert a government

commitment to scrutiny and transparency across departments, and initiatives are in place which aim to

facilitate access to government administrative data for the purposes of research and evaluation (Mulgan

and Puttick, 2013; UK Administrative Data Research Network, 2012). However, a recent report by the

National Audit Office (NAO) (2013) suggests that these ambitions are not yet embedded in practice. The

NAO’s assessment of the frequency and quality of impact and cost-effectiveness evaluation across key

government departments, and the use of such evidence to support resource allocation and policy

development, highlighted a number of issues. These included a lack of robust impact evaluations, a lack

of clarity in government decisions about what to evaluate and a failure to effectively apply learning from

evaluative research.

1.2 UK POLICING CONTEXT For some years now, the College of Policing (the College) and its predecessor the National Policing

Improvement Agency (NPIA) have been promoting the importance of research evidence to inform

practice in policing and crime reduction. The College Five Year Strategy (2014a) outlines its intentions to

promote understanding of ‘what works’ in policing and use this understanding to set standards and

enable members to share knowledge and evidence around ‘what works’ (College of Policing, 2014a).

Within a context of austerity and changing policing demands (e.g. Brain & Owens, 2015;

http://www.college.police.uk/Documents/COP_infographic.pdf) the College is promoting

‘professionalisation’ of police through various methods, including the development of a Code of Ethics

(College of Policing, 2014b), undertaking national demand analysis which attempts to understand police

workload (College of Policing, 2015a) and a leadership review (College of Policing, 2015b), as well as

delivering training packages and introducing new paths into policing (e.g. direct entry). These approaches

embed evidence-based practice into police standards and policies.

6 National Institute for Health and Care Excellence (NICE), Sutton Trust/Educational Endowment Foundation, College of Policing What Works Centre for Crime Reduction, Early Intervention Foundation, What Works for Local Economic Growth, the Centre for Ageing Better, and the What Works Centre for Wellbeing.

http://www.college.police.uk/Documents/COP_infographic.pdf


11

A recent survey conducted by the College reported that many police officers and staff have concerns

about the consistent implementation of standards across forces. These concerns included: training and

role responsibilities, quality assurance of training, variance in the value of training across forces, and

variance in the order of training (Pearce, 2015). Pearce suggests that the following initiatives should be

organisational priorities:

Creation of minimum levels of skills and knowledge for specialist roles

Raising the value of education within police

Development of an education and qualification framework as well as a structure for continued

professional development

Accreditation of trainers and training providers

Focus on learning goals and outputs of training

This sits appropriately alongside the College’s “responsibility for ensuring that standards set in policing

are based on the best available evidence” (College of Policing, 2014a, p.13).

1.3 WHAT WORKS CENTRE FOR CRIME REDUCTION As part of achieving many of the aforementioned aims, the ‘What Works Centre for Crime Reduction’

(WWCCR) was established in 2013 to develop a strong evidence base for decision-making around crime

reduction. It is led by the College and supported by a Commissioned Partnership Programme7 (CPP).

As part of its remit the CPP is developing a series of systematic evidence reviews on crime reduction

topics (both the collation of existing reviews and the writing of new reviews on topics identified by

stakeholders) and has created a standard system to rate and rank interventions in terms of their

effectiveness and cost-savings (EMMIE - a coding scheme for evaluating systematic reviews of the

evidence for crime reduction interventions, Johnson, Tilley & Bowers, 2015). Key outcomes of the

WWCCR include a Crime Reduction Toolkit (CRT) providing access to the crime reduction evidence with

regard to different interventions, which is available online

(http://whatworks.college.police.uk/toolkit/Pages/Toolkit.aspx). The CRT utilises the EMMIE scale by

rating each crime reduction intervention according to its impact on crime, strength of the evidence and

cost. It also summarises the evidence on how and in which circumstances each intervention works, with

the aim of helping practitioners to understand what makes a particular intervention work in a given

operational context.

A key component of the WWCCR programme is the development and piloting of a Police Development Programme (Work Package 6/7) to enable police officers to appraise evidence and use evidence to inform their decision-making. This contributes to the College aims around critical evaluation and the use of evidence, and to incorporate tests of such understanding into selection and promotion processes (College of Policing, 2014a). Following the development of the key WWCCR outcomes described above, the programme design for Work packages 6 and 7 focussed on enabling police officers and staff to use the Crime Reduction Toolkit and EMMIE. Through this, the programme aimed to develop officers’ theoretical (EMMIE) and practical

7 University College London, Institute of Education University of London, London School of Hygiene & Tropical Medicine, University of Southampton, Birkbeck, University of Surrey, Cardiff University and University of Dundee.

http://whatworks.college.police.uk/toolkit/Pages/Toolkit.aspx


12

(the Crime Reduction Toolkit) understanding of evidence-based approaches, equip them with the skills required to use evidence to inform their decision-making and support them to appraise evidence and commission research (Hereafter referred to as ‘Evidence-based Policing’: EBP8). The programme was designed with the intention of being added to the National Policing Curriculum (NPC), which “comprises the national standards for learning, development and assessment within the police service” (Clare, 2015, p.7).

1.4 THE CURRENT RESEARCH The purpose of the current research is to design, pilot and evaluate the implementation of a police

development training programme (see Fleming, Fyfe & Wingrove, 2016a, for an overview of the design

methodology and the programme outline) with four groups of police officers/staff across four forces.

The main objective of the evaluation was to assess and understand trainee reaction and to note aspects

of learning following training. The following research questions were addressed:

To what extent do police officers/staff respond positively to EBP?

To what extent do police officers/staff believe EBP training will enhance their role?

To what extent does the training enable police officers/staff to use the Crime Reduction Toolkit?

The remainder of this report provides:

The methodology used to evaluate the pilot training (including the evaluation materials as

appendices)

The findings with regard to police reaction to the training programme and evidence-based

policing

8 The term Evidence-Informed Policing is preferred by the authors of this report (see Fleming, Fyfe & Wingrove, 2016a for further information). However, to avoid confusion when discussing the wider literature in this area, the term Evidence-based Policing is used throughout this report.


13

2 METHODOLOGY

Evaluation of any pilot training programme aims to assess the effectiveness of the training, whether or

not the training contributes to organisational aims, and identify ways in which the training can be

improved. There are various levels on which training can be evaluated, for example, accounting for

cognitive, attitudinal and behaviour changes in trainees following training (Kirkpatrick & Kirkpatrick,

2006).

The following levels of evaluation are widely accepted in the training literature, and are a combination of

work by scholars such as Hamblin (1974), Philips and Stone (2002), Kirkpatrick and Kirkpatrick (2006), and

Buckley and Caple (2009). These levels formed the basis of the design of the evaluation methodology for

this research.

Reaction refers to customer satisfaction, and considers participant feelings about the learning

structure, content, training methods (including pace and visual aids), trainers and training

environment

Learning refers to the learning of principles, facts and techniques following training, and may be

measured generally through attitude change, improvement in knowledge or increases in skills

Job behaviour and performance considers changes in participants’ organisational behaviour and

performance resulting from training

Results/return on investment relates to the costs of designing and implementing training when

compared to financial outcomes and/or results related to the original training needs/objectives

2.1.1 Evaluation Measures

The following evaluation measures were developed to assess participant reaction, learning, self-reported

planned behaviour, and attitudinal outcomes as a result of the pilot training:

Pre-pilot training questionnaire (Appendix A) to identify participant demographics and baseline

attitudes. The themes evaluated in both the pre and post-pilot training questionnaires were based

largely on Kirkpatrick and Kirkpatrick’s (2006) ‘reaction’ level for effective evaluation of training

programmes. The structure of the questions and the questionnaire itself was informed by various

psychological and social science literature including that cited in Bradburn, Sudman & Wansink

(2004).

Post-pilot training questionnaire (Appendix B) to identify:

- changes in attitudes

- perceptions around the value of the training

- perceptions around the value of EBP to policing generally

- preferences around the content of the training

- perceptions of the relevance of the training to their job

- Other aspects set out in Kirkpatrick & Kirkpatrick’s (2006) guidelines for evaluating trainee reaction

(e.g. How did the content meet their needs? Was the training instructor best qualified to teach?

Were the methods to maintain interest and teach desired attitudes, knowledge and skills the most

effective?).


14

Observation Topic List (Appendix C)9 used to structure notes, when observing each pilot, and a

reflective session between all the researchers involved was held immediately following the training

day to identify researcher perceptions of trainee reaction/knowledge.

2.1.2 Participant sample

Participants were both a selective (in that the forces identified who they felt would benefit most from the

pilot training) and a convenience sample (in that the forces selected who was available on the dates of

the pilot training), identified by the four participating police forces (Sussex, Hampshire, PSNI, Police

Scotland. Please see Appendix D for a copy of the invitation for participants sent to forces. It is important

to note that it is not known how the forces then recruited the individual participants.

2.1.3 Data collection procedure

The research procedures and materials were submitted to, and approved by, the University of Southampton Ethics Committee (Reference number 17128).

On arrival at the scheduled training participants were asked to read the information sheet (Appendix E), ask any questions they may have and then sign the consent part of the sheet. They were then asked to fill out the anonymous pre-training questionnaire (Appendix A). Participants then took part in a six hour (including morning/lunch/afternoon break) pilot training session (for training slides see Appendix F). See Fleming, Fyfe and Wingrove (2016a) for a full overview of the training which included an introduction to EBP and used some of the tools developed by the WWCCR – in particular the evaluation scale entitled ‘EMMIE’ and an online Crime Reduction Toolkit (CRT).

Following the training session, the participants were asked to complete the post-training questionnaire

(Appendix B). As the questionnaires were filled out anonymously, participants fastened their two

questionnaires together to enable pre and post comparison per respondent.

Throughout the day, a member of the research team observed the training with consideration to the

observation topic list and at the end of each training session, the researchers undertook a reflective

session using the observation topic list as reference (Appendix C).

2.1.4 Evaluation data analysis

2.1.4.1 Quantitative Analysis

In terms of evaluating the participant Reaction and immediate (and potentially short-term) attitudinal

outcomes following the pilot training, Wilcoxon signed-rank10 tests were conducted to examine whether

any statistically significant differences were present in participant beliefs before and after the pilot

training. As much of the data collected from the questionnaires can be understood quantitatively, this

also enabled Fisher’s exact11 tests to be undertaken to explore:

Differences in awareness of and participation in College activities across demographical and

individual characteristics (e.g. force, rank, years in service, level of education)

Differences in use of research/evaluation across demographical and individual characteristics

(e.g. force, rank, years in service, level of education)

9 The observation topic list and the actual participant observation was informed by Spradley (1980) 10 See Footnote 4, page 3. 11 See Footnote 5, page 3.


15

Differences in pre and post-pilot training beliefs across demographical and individual

characteristics (e.g. force, rank, years in service, level of education)

2.1.4.2 Qualitative Analysis

Qualitative responses to the post-training questionnaire, observation notes and notes completed as part

of a reflective session following each pilot training day were transcribed and entered into NVivo12 where

they were coded and analysed thematically using a grounded theory approach (Glaser, 1978, 1992;

Glaser & Strauss, 1967; Strauss, 1987). Grounded theory refers to a qualitative analysis approach that

develops theory based on the data, in which findings emerge, as far as is possible, without the researcher

imposing a pre-existing theory onto the data (Strauss & Corbin, 1998). For further explanation of a

grounded theory approach to qualitative analysis, see Corbin and Strauss (2015).

2.1.5 Evaluation Limitations and Future Recommendations

Whilst incorporating all of the aforementioned training evaluation levels would provide the most accurate

and comprehensive review of any pilot training programme, time and resource limits related to the

current research prohibited the measurement of all levels. The evaluation carried out within the time

scale of this project was limited to that for which data could be compiled on each day of the pilot training.

This means that evaluation was limited to pre and post-training repeated measures (rather than a

controlled comparison group) assessing:

- Reaction and immediate (and potentially short-term) attitudinal outcomes,

- Observational speculation on participant learning, and

- Self-reported planned behaviour.

The time period in which this pilot training was undertaken did not allow for a long-term (1 year) follow-up of participants13. However, even if a long term follow up of the pilot training is completed, there should be a consideration of the difficulties inherent in assessing changes in police performance and competency (Patterson, 2011) in a profession which requires the individual to have an ability to make judgements and use discretion (Jarvis, 2010). This is due to the multiple and varied factors which impact on police perception of and response to issues (Werth, 2009). Therefore, it cannot be assumed that any behavioural or decisional outcomes are causally linked directly or solely to the pilot training. Further, deliberation should be given to whether such a direct link is plausible. Weiss (1980) proposes that knowledge originating from systematic research does not tend to influence problem solving in a direct manner, but rather ‘creeps’ into policy decisions around problems. As will become clearer as the findings of the current research are presented, empirical evidence does not tend to lead to a clear cut solution to problems. Therefore, achieving a direct causal link between training in EBP approaches and implementing EBP approaches may not be a reasonable expectation. Additionally, in order to see behaviour change in individuals, the person must not only have a desire to change a behaviour and have the knowledge they need to change it (what to do and how to do it), but also the right climate for change (a supervisor who encourages or requires EBP rather than one that prevents or discourages them from implementing EBP) and a reward for changing (e.g. feelings of satisfaction/ achievement, or praise/recognition) (Kirkpatrick & Kirkpatrick, 2006). Therefore it is clear

12 NVivo is a Computer Assisted Qualitative Data Analysis package (Silver & Lewins, 2014). 13 In the original research proposal for this project, long-term follow up was considered, however this did not prove possible in the time frames of the project.


16

that the pilot training alone will not be able to achieve behaviour change in participants. For further reading on behaviour change literature, see Dolan, Hallsworth, Halpern, King and Vlaev (2010). During the design of the pilot training programme (see Fleming, Fyfe & Wingrove, 2016a), structured

focus groups were undertaken in order to assess, amongst other things, what might be perceived as a

useful programme in order to instruct officers in the value/use of evidence-based research (Fleming &

Fyfe, 2015). The focus group findings suggest that the successful introduction of evidence-based practice

and policy in UK police organisations will require significant organisational behavioural change and

executive level support within individual police forces (Fleming & Fyfe, 2015). It should be noted that the

impact of any training intervention to transform organisational practice within police is often dependent

on other significant institutional changes within, for example, routine police practices (Stanislas, 2014).

The following sections of this report describe the quantitative and qualitative findings of the pilot training

evaluation by providing an overview of each individual force, the findings relevant to each individual

force, the overall results and discussion, followed by recommendations for individual stakeholders in the

area of EBP and EBP training


17

3 INDIVIDUAL FORCE OVERVIEWS AND FINDINGS

The following section is intended to:

- Provide the reader with an overview of the current organisational context and demographics of each

pilot force in order to aid interpretation of the overall findings

- Provide the reader with an overview of the variance between the pilot training delivery within each

force in order to aid interpretation of the overall findings

The sample sizes of each individual force do not warrant individual analysis. Any significant findings have

been identified through consideration of the sample as a whole.


18

3.1 POLICE SCOTLAND

3.1.1 Organisational context and demographics

Established in 2013, Police Scotland is the second largest police force in the UK and covers 28,168 square

miles and 14 divisions (About Us, n.d). Its Annual Police Plan 2015/16 reports that a public consultation

survey finds violent crime, drugs and speeding motorists to be the top priorities for this force.

Since 1977, Police Scotland had comprised of eight police forces. The Police Reform: Outline Business

Case (Scottish Government, 2011) proposed that a single force in Scotland would enable crime reduction

performance levels to remain stable alongside a reduced budget and would more readily enable the

police to address issues around governance and accountability raised by Her Majesty’s Inspectorate of

Constabulary (HMIC).

Following the transition to a single service, the most recent officer and staff opinion survey (Axiom, 2015)

revealed that whilst the majority of respondents felt they had a strong positive connection to their job,

their team and a positive relationship with their manager, there were various concerns around

organisational procedures. Communication was highlighted as a specific issue, with a reliance on non-

verbal communication and retrieval of information (including media, email and the internet). 47% of the

respondents reported feeling overloaded with information. The majority of respondents felt they did not

have access to appropriate information in respect to Police Scotland and Scottish Police Authority (SPA)

aims. 70% of respondents felt they did not receive recognition of good work and “only 18% thought that

they would be supported if they tried a new idea even if it did not work” (p.3). When considering training,

of the 54% of respondents who had received any in the previous 12 months, only 40% thought it was

relevant to their role. For most participants, there was a lack of understanding of how decisions were

made around promotions. Only 8 % of respondents felt their organisation was interested in their

wellbeing. 33% of respondents had intentions to leave, citing this lack of interest in their well-being.

Other reasons included not feeling valued and a lack of resources to do their job.

The pilot training in Scotland took place a week after the early resignation of Police Scotland Chief

Constable Sir Stephen House. It followed media criticism of the force’s policies around ‘stop and searches’

and armed officers, as well as a three day delay in a response to a road traffic collision in which two

individuals died (BBC News, 2015).

Police Scotland had a total of 17261 full time equivalent officers at the time of the pilot training (Police

Scotland, 2015a), with approximately 75% of these working within local policing, 17% acting as regional

officers (for the North, East and West) and 8% belonging to specialist national units. In addition, the SPA

employ approximately 5946 support staff (e.g. analysts).

Police Scotland’s most recent diversity and equality report (Police Scotland 2015b) revealed that 29% of

Police Scotland officers are female, the majority of officers are aged 35-44, and 49% had 0-10 years of

service. 63% of SPA staff are female, with the majority aged 45-54, 59% had 0-10 years of service. There

does not appear to be any accessible data regarding numbers of each officer/staff per rank/role.


19

3.1.2 Pilot training programme

3.1.2.1 Overview

Scotland was the first pilot training day, taking place at the Police College at Tulliallan Castle. The training

was completed faster than expected, proceeding from 09:00 – 15:00 over the course of one day. The

pilot training room was long and narrow, with the slides projected at the front and computers in rows

down both sides and the middle of the room. Some participants were unable to hear or see some of the

trainer instructions as they were sat near the back of the room. One participant mentioned that, “the

long room was a mistake”. Participants also mentioned that the printer situated at the back of the room

was loud, and observers noted that traffic outside the room made it difficult to hear at times. There were

no tea or coffee facilities within the room or provided during the day.

Many participants had travelled up to 3 hours to attend. The initial enthusiasm of many of the

participants was not throughout, potentially due to various factors including an early start, an overly

warm room, and an increasingly anticipated long journey to get home. Significantly, there were cramped

conditions when participants were completing the computer activities earlier in the day due to a lack of

computers with internet function (up to 6 were using one computer at a time). The lack of functioning

computer facilities led to many participants sitting on the edge of the group during group work/case

study activities, less able to participate. However, participants did not seem surprised by the lack of

internet and appeared familiar with the situation of reduced resources in this context.

23 individuals participated, making up 25.6% of our total UK sample. This included 2 Constables, 10

Sergeants, 9 Inspectors and 2 Chief Inspectors. Scotland was the only force that provided no analysts or

other police staff to participate in the training. All of the participating officers had served for at least 5

years, with 19 having served for 15+ years (82.6%). This suggests that the Scotland sample was made up

of officers with more years of service than the overall Police Scotland population. 5 of the participating

officers were female (22.7%), falling only slightly under the Police Scotland population proportions. 14 of

the officers were aged over 41 (66.7%), 6 were aged 26-40 years (28.6%) with only one participant aged

under 25 (4.8%). 56% of the sample were educated to at least BA/BSc level.

3.1.2.2 Findings and comments

3.1.2.2.1 Comparison to other pilot forces

In terms of significant findings when comparing across forces, the officers in Scotland were the least likely

to have:

volunteered to participate in the pilot training,

been aware of EBP prior to the pilot training, or

heard of POLKA prior to the pilot training.

3.1.2.2.2 General reflections on EBP

When participants were reflecting on EBP throughout the training day, their comments revealed a

recognition that a change in approach to research is required within the force.


20

“We are police officers, we are ‘can do’ people”14

“This is a mind-set change”

It is important to note that there were also perceptions that the force had been exposed to continuous

changing organisational practices which may lead to resistance to further change.

“We have tried so many different approaches now”

Despite such concerns, there was an openness to the benefits of an EBP approach.

“We are changing our perspective on academic research, more and more of us have degrees and

thus trust research”

“This is helpful for community work, partnership working”

“Even just having this, accessible like this will impact us changing”

3.1.2.2.3 Reflections on the CRT

Once the trainees started to engage with the case study tasks and the CRT, comments reflected positivity

about the existence of such a toolkit.

“Very good tool, ticks lots of boxes”

“It is good, it makes you think about all different things”

However, there were concerns about the limited amount of information available on the CRT and the

level of interpretation of the information required to enable application operationally.

“It [the CRT] doesn't tell you the answers”

“The evidence can be confusing because it is conflicting”

“Sometimes there seems to be insufficient evidence”

“You wouldn't base it purely on this [the CRT], you would use it to add to an understanding

alongside discussions with your partners and statistical analysis of a specific problem” [Lots of

nodding around the room at this point]

3.1.2.2.4 Challenges related to implementing an EBP approach

When considering the challenges to identifying an evidence-based intervention, the participants made

comments which reflected their concerns about the challenges of implementing an EBP approach.

A primary concern was related to resourcing.

“The first question will be, where is the evidence? We have always done it this way. How much

will it cost? And the information [on the CRT] is lacking here”

14 The number of quotes provided for each theme is not related to the number of participants who reflected a particular theme, but rather were those chosen to illustrate most clearly the themes and interpretation provided.


21

“Time. Nobody has time to sit at the desk and give it some thought”

Another concern was whether frontline practitioners would be subjected to information overload.

“Information overload. They [officers on the ground] are expected to know so much”

“Cops delivering on frontline getting so much information from so many angles”

Additionally, there was apprehension around whether the information sharing required to achieve EBP

would be possible.

“A lack of communication – we are not good at sharing best practice”

“Communication – from management down to the frontline, and across agencies/partnerships”

A final worry participants held about implementing EBP was whether they would be able to acknowledge

that an intervention did not have a positive effect.

“Everyone is always successful, no one will stand up and say this is rubbish, don’t do this”

“We don’t admit that we do things wrong”

3.1.2.2.5 Feedback on the pilot training

Some general feedback on the pilot training day from the Scotland participants includes:

“Overall I found the course very informative and a step forward in modern day policing” (male,

constable)

“Interesting, but overly academic bias” (Male, Sergeant)

“Overall the training was good/very good. The product can easily become a white elephant and

another waste of money. The content needs to be wider as I was underwhelmed by the limited

content” (Male, Constable)

“Learnt a lot about the toolkit which will help me on a day-to-day basis – thank you” (Male,

Sergeant)

“Opened my mind to be looking for additional support in future planning etc.” (Male, Chief

Inspector)

“Excellent to see people’s research feeding into operational policing. I would like to see research

of live operational policing at local levels too” (Male, Sergeant)

The majority of Police Scotland participants identified an increased understanding of the CRT as the

component of the training which was most useful/that they would take back to their workforce and use.

Other themes which the Police Scotland trainees identified as components of the training most useful in

meeting their learning goals/ that they would take back to their workforce and use included:


22

- A new approach to problem solving/decision-making

- An increased understanding of evaluation research

Themes the Police Scotland trainees identified as components of the training least useful in meeting their

learning goals/ that they would not take back to their workforce and use included:

- A lack of appreciation of resource limits when implementing an EBP approach

- A lack of understanding of how to apply the training operationally and the viability within the force

- None! (The majority of trainees did not identify any components they would not take back to their

workplace and use)

Themes the Police Scotland trainees identified as components of the training which could be improved

included:

- A re-consideration of the appropriate target audience for the pilot training (e.g. Chief

Inspector/Superintendent)

- Some changes to the case studies (largely around making them more specific to allow one crime

problem to be addressed at a time)

- Real operational examples

- Improved facilities (the room and internet access)

3.1.3 Summary

Scotland was the first pilot training day. 23 individuals participated and Scotland was the only force that

provided only operational police officers to take part, rather than including police staff/analysts. Scotland

participants were the least likely to have volunteered to participate in the pilot training, been aware of

EBP or heard of POLKA prior to the pilot training. Participants recognised that a change in approach to

research may be needed within the force and were open to the benefits of an EBP approach. However,

they did have concerns that they were fatigued by organisational change. Participants were positive

about the CRT, although they had concerns the information was limited and required much interpretation

to enable implementation of the research findings. In implementing an EBP approach, participants’

primary concerns were resourcing, communication and whether individuals would be able to

acknowledge if an intervention did not have a positive effect. The majority of Police Scotland participants

identified an increased understanding of the CRT as the component of the training which was most

useful/that they would take back to their workforce and use. They felt that more consideration of

resource limits and how to apply the training within their force was needed in the training, and that real

operational examples, improved IT facilities and the inclusion of higher ranks could improve the training.


23

3.2 HAMPSHIRE CONSTABULARY


Hampshire Constabulary covers the counties of Hampshire and the Isle of Wight, and the cities of

Southampton and Portsmouth, totalling 1602 square miles (Hampshire Police & Crime Commissioner,

2013). The Constabulary covers large rural areas as well as the cities of Southampton and Portsmouth,

major roads, Southampton airport and military facilities. Portsmouth itself is the most densely populated

area of the UK outside of London. In April 2015, the force moved to a new policing model centred on

borderless policing, with response and patrol officers being directed by a central control room rather than

by clearly categorised districts (John, 2015).

The Hampshire Police and Crime Plan (2013-2017) reports that the Hampshire population is

predominantly white (93.2%), with significant minority ethnic groups in both Southampton and

Portsmouth (14.2% and 11.7% respectively), with higher proportions of those aged 16-24 in the cities

when compared to the rest of the area, and higher proportions of over 65 year olds outside of the cities.

The majority of the population (85%) live in 15% of the total area.

The Police and Crime Plan (2013-2017) identifies improving frontline policing to deter criminals, placing

victims and witnesses at the heart of policing, working together with communities to reduce crime and

anti-social behaviour and reducing re-offending as its strategic priorities.

As with other forces, in the last few years Hampshire Constabulary has been affected by government

spending cuts, with requirements to deliver £55 million cuts by April 2014 and a further £25 million by

April 2015 (BBC News, 2014). The Chairman of Hampshire Police Federation reported that this has

affected morale (BBC News, 2014).

Although the government Comprehensive Spending Review announced that there would be no further

police budget cuts in November 2015, the pilot training took place at Hampshire several months prior to

this. During the months following the pilots, several media reports suggested that a national survey

completed by the Police Federation revealed that morale within Hampshire police was the worst amongst

all of the other forces in England and Wales15.

The Hampshire Constabulary website states it currently has 3,479 police officers and 1896 police staff.

The most recent government police workforce figures were released in March (Home Office, 2015) and

report that 30.4% of Hampshire’s police officers are female. Of the male officers (N=2130), 0.14% are

Chief Officers, 0.38% are Chief Superintendents, 0.52% are Superintendents, 1.5% are Chief Inspectors,

5.12% are Inspectors, 17.84% are Sergeants and 74.51% are Constables. Of the female officers, officers

(N=934), 0.11% are Chief Officers, 0.11% are Chief Superintendents, 0.96% are Superintendents, 0.96%

are Chief Inspectors, 4.07% are Inspectors, 13.92% are Sergeants and 79.87% are Constables. Of the

police staff, 63.10% are female.

15 E.g. http://www.portsmouth.co.uk/news/crime/hants-police-morale-is-fifth-worst-in-country-1-7016206;

http://www.bbc.co.uk/search?q=hampshire+constabulary#page=2; http://www.bbc.co.uk/news/uk-england-

hampshire-26154049

http://www.portsmouth.co.uk/news/crime/hants-police-morale-is-fifth-worst-in-country-1-7016206

http://www.bbc.co.uk/search?q=hampshire+constabulary#page=2

http://www.bbc.co.uk/news/uk-england-hampshire-26154049



24


3.2.2.1 Overview

Hampshire provided the smallest number of participants of the four pilot forces, with 16 individuals

participating over two days of pilot training, although only 14 provided the research team with data

through the questionnaires, making up 15.6% of the total sample. This included 3 Constables, 1 Sergeant,

1 Inspector, 1 Chief Inspector and 5 analysts. Hampshire provided the largest percentage of police staff

within their sample (57.14%). This means the small sample was not representative of Hampshire’s officer

and staff population. 85.71% (12) of the Hampshire participants were educated to at least degree level.

Both days of the pilot training in Hampshire took place at the Headquarters, in a small computer room.

This made the slides easy to see, and voices could be heard clearly.

On the first day there were 6 participants. On first entering, one such participant seemed uninterested

and had to be asked several times to complete the questionnaire/consent prior to us starting – he then

lost this later in the day and as such could not be included in the questionnaire data. The group were

vocal during the morning session, with comments and questions.

On the second day there were 10 participants, including analysts, officers from neighbourhood policing

and staff in training roles. There was a noticeably negative reaction to the CRT, particularly from a couple

of strong personalities, and thus much of the conversation during the day centred on concerns about the

CRT. At times the day functioned less as a training session and more as a focus group, as the two

dominant personalities continued to comment and react to the CRT. Several participants held beliefs that

they already had sufficient levels of knowledge and understanding of research and evaluation, and

commented that Hampshire already have a ‘knowledge bank’ at a local level which they felt was

sufficient.



The participants in Hampshire, when compared to the others forces, were statistically significantly most

likely to have:

- volunteered to participate in the pilot training (perhaps due to the number of analysts who

participated and the level of education of the participants; both of these factors were significantly

associated with volunteering to participate in the overall sample)

- been aware of EBP prior to the pilot training

- heard of evidence-based master classes prior to the pilot training

- heard of POLKA prior to the pilot training

- previously undertaken a piece of research with a university


When participants were reflecting on EBP during the two days of training, their comments revealed

concerns about implementing it practically.

“Don’t get me wrong, I think this is the right way to go, I’m a fan of EBP, but it’s just putting it into

practice?”


25

There were particular concerns around resourcing.

“The biggest challenge is potential and opportunity. We have a treadmill existence. There is no

time to step aside and work out more efficient ways to do it. We can barely stay up-to-date with

the constant changes in policy and law, let alone research”

“How do we ride our demand, make resources available at the right time in the right place?”

Participants also had concerns about their ability to undertake evaluation.

“I think we get hung up on the uniqueness of the situation, rather than learning what could be

applied more widely”

“As analysts we get tasked with results analysis, but the problem stops and we get tasked to

move on. It’s increasing that going back and stopping and assessing”

“Evaluation needs to be proportionate – I’m not going to do a proper evaluation”

Finally, participants had concerns around how a lack of communication within the organisation may

impact on their ability to implement EBP.

“Even if we know what works, we forget it quickly, we don’t learn from it”

“Even when a debrief happens, the recommendation doesn't get passed forward, it’s based on an

individual, without the structure to help others get the benefits”


Once the trainees started to engage with the training tasks and the CRT, participants’ comments reflected

concerns surrounding the relevance and limitations of the research available on the CRT.

“The literature was not chosen in consultation with police – we don’t care about tax, we can’t do

anything about that”

“The frustration is, it has the possibility to be useful. But I can’t see that it is!”

“It’s borne out of the research that is there, not what we actually care about”

In particular there seemed to be concerns about the focus on crime reduction in the research on the CRT.

“It should consider the whole business of running a public sector organisation, reducing crime is

significant but it’s not the only business we do”

There were also concerns about the lack of recent research on the CRT.

“Research can become obsolete quite quickly for us”

“If it’s not up-to-date, then what’s the point?”

A key issue which is reflected in the participants’ comments reveals concerns as to how the CRT had been

‘sold’ to them as practitioners.


26

“This isn’t a golden bullet but it will inform your decision. But there’s a perception it’s been

marketed that it will solve all your problems…So I disengaged because it doesn't. But you [are]

now saying that it will inform you. That message needs to be spread!”

“The branding needs to be right – senior managers are selling it as a golden bullet”


When considering the challenges to identifying an evidence-based intervention, participants’ comments

reflected concerns around organisational culture and how this will impact on their ability to choose an

intervention this way.

“We are culturally not ready to critically approach our interventions and accept that things went

wrong”

“Doomed to succeed”

“The forums around what does/doesn't work are formal where people don't want to admit to

failure or having made mistakes, or how they could improve”

Gaining buy-in from frontline officers, stakeholders, partners and management was also a key concern

when participants considered using an EBP approach to implement an intervention.

“There will be a preconception that it won’t work – ‘we’ve tried this before’ ‘teaching me to suck

eggs’”

“With the body worn cameras we’d had small scale iterations but there was still resistance, and

they didn't turn it on – because they forgot, or didn't have time, didn't believe it would affect

anything. They won’t believe the hype unless they have personal experience of it”

“It is very difficult to sell to external stakeholders, they think ‘where is the win for our school?’

What is the benefit for them giving us resources?”

“There are certain levels that have to be challenged higher up – they don't want to listen. Some

people don't want to listen”

Finally, participants again felt that the resource limitations would constrain their ability to implement an

approach in this way.

“We don’t have the resources, time or the right skills that happen to be in the right area and on

duty at the right time. Having the resources, time and skills is more by luck”


Some general feedback, received via the questionnaire, on the pilot training days from the Hampshire

participants included:

“Whilst I found the case study interesting and group discussions thought provoking, I am not

convinced attitudes within the force will change. This would need to be rolled out to EVERYONE

to get everyone’s opinions on research/pilots/initiatives changed. Without this nothing will ever

change” (female, staff)


27

“I was very sceptical about what I may learn from the day; but changed to an interested

participant and can see the value of EBP on a day-to-day and strategic level” (male, staff)

“The toolkit is a great concept but currently not fit for purpose. I think it would benefit from

practitioner input” (female, chief inspector)

“Really good training, however subject (toolkit) is still in its infancy and is somewhat limited

making it difficult to implement effectively in the workplace” (female, staff)

“Very enjoyable and thought provoking” (male, constable)

“This was a useful and interesting day but vast improvements need to be made on the toolkit to

make it effective. Local sharing needs to be nailed first. Thank you.” (female, constable)

“The session has helped me to look at a new way of evaluation. This will help me in my role. I

enjoyed the input and the frank discussions with my peers” (male, sergeant)

The majority of Hampshire participants identified an increased understanding of evaluation research as the

component of the training which was most useful/that they would take back to their workforce and use.

Other themes identified by the Hampshire pilot trainees identified as components of the training which

were most useful in meeting their learning goals/ that they would take back to their workforce and use

included:

- An increased understanding of the CRT

- An increased understanding of EMMIE

- The ability to use EMMIE to plan and structure evaluation

Themes the Hampshire trainees identified as components of the training least useful in meeting their

learning goals/ that they would not take back to their workforce and use included:

- The CRT and its perceived limitations

- The CRT in its current state as ‘not fit for purpose’

- The limited relevance of the evidence on the CRT (US based/out of date/not force priorities)

Themes the Hampshire trainees identified as components of the training which could be improved

included:

- Improvements to the CRT

- A broader focus/ examples other than Crime Prevention


3.2.3 Summary

Hampshire provided the smallest number of participants and the largest percentage of police staff when

compared to the other pilot forces. The participants were also significantly most likely to have

volunteered to participate in the pilot training, been aware of EBP, evidence-based master classes and

POLKA prior to the pilot training and to have previously undertaken a piece of research with a university.

Participants had concerns about how they would implement EBP practically, particularly with regard to

resourcing, undertaking evaluation and a lack of communication within the force. With regard to the CRT,

participants had significant concerns about the relevance of the research available, in terms of a focus on


28

crime reduction and the lack of recent research. For Hampshire participants these concerns about the

CRT appeared to be related to how they felt it had been ‘sold’ to them as a ‘golden bullet’. When

considering implementing an EBP approach, participant concerns related to force culture, gaining buy-in

from relevant stakeholders and resources. The majority of Hampshire participants identified an increased

understanding of evaluation research as the component of the training which was most useful/that they

would take back to their workforce and use. However they felt that the CRT was the least useful

component of the training. Improvements to the CRT were suggested to improve the training, alongside a

broader focus than crime reduction and real operational examples.


29

3.3 SUSSEX POLICE


Sussex police covers East Sussex, West Sussex, Brighton and Hove, with a total resident population of 1.63

million (Who we are, n.d). This covers rural and urban areas and includes Gatwick Airport, and 13

districts. Within each district there are individual neighbourhood teams. The Sussex Police and Crime Plan

2013/2017 (Sussex Police & Crime Commissioner, 2013) identifies priority targets as crime and

community safety (measured by reducing recorded crimes), victim focus (measured through victim

satisfaction), public confidence (measured through increases in specific crime types including domestic

abuse and violence) and value for money. The plan also recognises that Sussex is a low crime area, with

crime reductions every year for the past 8 years.

Sussex Police recognise that it has “been on a journey of considerable change for some time” (Sussex

Police, 2015, p.4), with savings of £50 million taking place since 2010. Not least of this is the joint

collaborative vision with Surrey police, agreed in 2013, which commits the two forces “working as one,

operationally and organisationally” (Police and Crime Plan 2014/2017, p.13). Sussex also has an

‘Academic and Research Partnership Policy’ (2015) which states that Sussex Police recognise the benefit

of such partnerships in improving services and supporting the ongoing professionalisation of the police

service.

Although the government Comprehensive Spending Review announced that there would be no further

police budget cuts in November 2015, the pilot training took place at Sussex prior to this. As with other

forces across the UK, Sussex was predicting they would have to make further savings over the next four

years, potentially to a total of £57 million (Sussex, 2015). As such their new local policing model

emphasised reducing demand and developing a flexible workforce (Sussex, 2015).

The Sussex Police website states it currently has nearly 2700 police officers and 2100 police staff. The

most recent government police workforce figures were released in March (Home Office, 2015) and report

that 30.6% of Sussex’s police officers are female. Of the male officers, 0.15% are Chief Officers, 0.41% are

Chief Superintendents, 0.67% are Superintendents, 1.5% are Chief Inspectors, 6.0% are Inspectors,

17.33% are Sergeants and 73.95% are Constables. Of the female officers, officers, 0.11% are Chief

Officers, there are no female Chief Superintendents, 0.70% are Superintendents, 1.28% are Chief

Inspectors, 3.26% are Inspectors, 11.98% are Sergeants and 82.67% are Constables. Of the police staff,

56.34% are female.

3.3.2 Pilot training programme findings

3.3.2.1 Overview

The Sussex pilots took place at Sussex Headquarters in a large computer room with a projector at the

front of the room. The acoustics/size of the room made it difficult to hear participants’ comments, and

the noise of building works impacted on this too. As with other forces, some of the computer facilities

were not working which delayed some parts of the training day. There was tea and coffee available in the

room throughout both days. On the second day of training, there was a fourth observer from the

WWCCR, as well as a senior Sussex officer involved in research within the force, impacting on the

potential for observer effects. The ‘observer’ or ‘Hawthorne’ effect refers to participants modifying their

behaviour due to awareness that they are being observed. Whilst wider research suggests impacts of this


30

kind do exist, it is less clear in which conditions, how or how much such effects occur (McCambridge,

Kypri & Elbourne, 2014). Therefore, this context is provided only for the reader to consider during

interpretation of findings, specifically noting that every other pilot training session involved the same

observers and trainers to minimise the potential for the effects described across overall findings.

29 individuals participated over two days of pilot training in Sussex, making up 32.2% of our total sample.

This included 1 Constable, 11 Sergeants, 11 Inspectors, 1 Chief Inspector and 3 analysts. When compared

to the other participating forces, Sussex provided the largest percentage of the total sample and the

largest percentage of Inspectors (37.93%). This means Inspectors were overrepresented when considered

alongside Sussex Police staff population. 75% of the Sussex participants were educated to at least degree

level and 62% had served for 15+ years.



The participants in Sussex were statistically significantly the least likely to have heard of Evidence-based

Master Classes when compared to the one other force who had heard of them (In 2 of the other pilot

forces no participants had heard of the Master Classes).


When reflecting on EBP, comments across both days of training reflected a level of uncertainty about EBP

as another changing organisational practice.

“The term [EBP] is creeping in, it’s a term of convenience to persuade you to do or not do

something. To stop you going down a particular path, but what does that mean and how do we

translate it into speedy decisions?”

“The research isn’t shared for research sharing – it’s shared to show what they’ve achieved”

“It’s like super-cocooning. The research said it wouldn't work in our area, but the Chief wanted it

regardless”

Participants also had concerns about what role police experience plays in informing an EBP approach.

“This approach jumps ahead, there is strong operational experience which there is no good forum

for sharing. This jumps to a more limited approach”

“This focuses on a very narrow area of research at this time when there’s a wealth of

understanding we can draw on”

Finally, there were clear concerns with regard to resources.

At the moment we don't do research [evaluation] as we just haven’t got the time or resources”

“We have a commitment from our Chiefs about EBP, but it requires time and so we can’t do it

without impacting on everyday business, for instance by taking people out for training. There

must be a threshold – ‘Ok we’re too much into research now!’”


31


Once the trainees started to engage with the training tasks and the CRT, comments reflected their

concerns with regard to having the resources and discretion to use the CRT.

“We don’t get the discretion to check [on the CRT] what to do”

“I doubt response officers would have the time or inclination to do this [use the CRT].”

“I had a similar reaction [to the CRT] and thought what are the resource implications? At a time

when resources are limited”

Participant comments also reflected their concerns with regard to whether the evidence available on the

CRT was credible and relevant to them as practitioners in the UK.

“Is the information on there, can I trust it? That it does work? The stuff that comes up is the fluffy

stuff, the hard stuff like enforcement doesn't come up? So basically, I don't believe it”

“There’s very little UK based information on there, how can I know it will transfer to the UK? We

don’t have the powers they do either. Does the cost transfer?”

“I looked at it once and it looked good, academic stuff, but I just couldn't find anything useful”

Further, participants were apprehensive more generally about the limited amount of information

available on the CRT.

“There’s not a lot of information, it’s quite strategic and vague”

“A limit of one page doesn't do justice to more complex bits”

“Hyperlinks to further information would be good”

“It needs more fine detail!”


When considering the challenges to identifying an evidence-based intervention, the participants’

comments reflected concerns about transferring and applying the research to their own problems.

“Contextual issues – it might not work in the UK, you have to make that decision”

This related to their concerns about organisational risk adversity and whether they would be able to

implement a different approach.

“We’re quite lucky with our boss, we’re innovative, but some organisations might be more risk

adverse to spending money if it doesn't work”

“We’re risk adverse and comfortable with the same interventions”

A key concern the participants had about implementing an EBP approach was gaining buy-in from the

frontline officers.

“What if the guys aren’t immersed, will it work properly? How will they combine it with ‘business

as usual’?”


32

“The people who are doing the intervention need to get committed to it which is really difficult.

It’s hard to sell at a grass roots level if they don't believe in it”

“It’s hard to get constables to do extra stuff. Policy changes all the time, new recommendations

come in all the time, blanket emails are sent out, they delete it. You find no one is doing it”

Many of their concerns were related to perceived resource limitations.

“A lot of people don’t have capacity. Or they have other priorities and pressures”

“Unintended large events may impact implementation, pull you away from it. Priorities change on

a daily basis, resources may have been taken at short notice”

“It is almost impossible to plan anything”

“When we put one intervention in place and it reduces crime, then the problem is gone so the

resources go elsewhere and then the problem comes back”


Some general feedback on the pilot training days from the Sussex participants included:

“Useful and thought provoking –encourages informed/evidence based policing with partners”

(female, inspector)

“Very informative” (male, inspector)

“Interesting, thanks for your time” (male, sergeant)

“Interesting training. A good reminder of the importance of objectivity and rigorous testing”

(male, staff)

“Police already use a simple and more realistic problem-solving toolkit – police cannot change the

world we can only change a few limited aspects of crime and disorder – the EMMIE model seeks

to enable police to try and do more than that and its outside of police capability” (female,

sergeant)

“The group discussions helped me understand the importance of research prior to

implementation of the intervention” (male, sergeant)

“Valuable and interesting in reinforcing the need to carry out structured setting of what is

required and needed to evaluate success and document best practice for knowledge bank”

(intelligence analyst)

Sussex participants most commonly (approximately 50%) identified an increased understanding of

evaluation research as the component of the training which was most useful/that they would take back to

their workforce and use.

Other themes the Sussex pilot trainees identified as components of the training most useful in meeting

their learning goals/ that they would take back to their workforce and use included:

- An increased understanding of the CRT



33


- Contact with academia and research colleagues

Themes the Sussex trainees identified as components of the training least useful in meeting their learning

goals/ that they would not take back to their workforce and use included:

- The CRT and its limitations

- The restrictive and limiting nature of the CRT

- The limited relevance of the evidence on the CRT

- The relevance to role was unclear

Themes the Sussex trainees identified as components of the training which could be improved included:

- The CRT


- Improved IT facilities

- Re-consideration of appropriate target audience (e.g. District commanders/Command team to

attend first)

3.3.3 Summary

29 individuals participated over two days of pilot training in Sussex and the participants were the least

likely to have heard of Evidence-based Master Classes when compared to the one other force who had

heard of them (In 2 of the other pilot forces no participants had heard of the Master Classes). Comments

across both days of training reflected a level of uncertainty about EBP as another changing organisational

practice, the role that police experience will play in an EBP approach and resourcing issues. With regard

to the CRT, participants had concerns about having the resources and discretion to use it and whether the

evidence available was credible and relevant to them as practitioners in the UK. This led to them having

concerns about transferring and applying the research to their own problems. Gaining buy in from

frontline officers, resource limitations and combating risk adversity within the organisation were reflected

as barriers to implementing an EBP approach. Sussex participants most commonly identified an increased

understanding of evaluation research as the component of the training which was most useful/that they

would take back to their workforce and use. They felt the CRT limitations and a lack of clarity in the

relevance of the training to their specific roles were the least useful components of the training.

Suggested improvements to the training included real operational examples, improvements to the CRT,

better IT facilities and the inclusion of higher ranks.


34

3.4 POLICE SERVICE OF NORTHERN IRELAND


The Police Service of Northern Ireland (PSNI) was formed in 2001 and serves the entirety of Northern

Ireland, covering 11 Policing Districts, approximately 13,843 km² and 1.8 million people. The districts vary

in terms of the populations they serve, with Fernmanagh and Omagh having a population of 115,000 and

Belfast having 336,800 (Northern Ireland Statistics & Research Agency, 2015).

The Chief Constable’s most recent Annual Report (PSNI, 2014) states that public confidence currently is at

67.3%, the highest recorded since 2010. This is despite the assertion that 2013/14 was the most

demanding year since PSNI was formed, due to the delivery of the G8 summit, the World Police and Fire

Games and ongoing public order issues. The key development themes noted by the Chief Constable are

partnership and co-operation with both communities and other agencies.

Policing in Northern Ireland has typically been politically sensitive and PSNI taking over from the armed

Royal Ulster Constabulary was key in developing public confidence following policing of the Troubles

(Mulcahy & Orde, 2015). The PSNI Policing Plan 2014-2017 (Northern Ireland Policing Board & PSNI,

2014) highlights the on-going demand on policing from periods of disorder and protest in Northern

Ireland. Such demands lead to various outcomes including a lack of resources for routine policing

activities as well as injuries to officers.

As with other UK police forces, PSNI has also faced reduced resources and financial challenges in recent

years, responding to this with the introduction of the ServiceFirst Change Programme in 2015 (Northern

Ireland Policing Board & PSNI, 2015). This programme led to changes including a move to 11 districts and

seeks to optimise the workforce, embed an organisational culture of Policing with the community along

with various re-structuring of departments.

The PSNI workforce composition figures (PSNI, 2016) state that they currently has 6827 police officers

and 2359 police staff. Of these, 67.28% of police officers and 78.29% of police staff are perceived

Protestant whilst 31.16% of police officers and 19.63% of police staff identify as Roman Catholic. This

compares to the census data which finds that 45.14% of the Northern Ireland population identify their

religion as, or were brought up as, Catholic and 48.36% as Protestant and Other Christians (Northern

Ireland Statistics & Research Agency, 2012).


3.4.2.1 Overview

Northern Ireland was the last of the pilot training sessions and took place over two days at the Police

College in Belfast. The room was large and square, with the slides projected at the front and computers in

horizontal rows throughout the whole room, facing the front. This meant that most participants were

able to check their emails and undertake day-today desk work throughout both days whilst participating

in the training, sometimes to the detriment of their attentiveness to the trainers. There was some noise

from maintenance work on the windows of the room. There was tea/coffee facilities within the room, in

the morning and the afternoon. The training took place both days from 09:00 – 15:00. Some of the

computers would not work, however there were other working computers and it was not a problem for

the research team or participants.


35

24 individuals participated over two days, making up 26.7% of the total sample. This included 2

Constables, 13 Sergeants, 3 Inspectors and 3 analysts. Whilst a Chief Inspector did attend the training,

they were unable to stay for the entire session and thus could not be incorporated into the pilot

evaluation.

59.09% of the participants were educated to at least degree level, 45.83% had served for 15+ years and

50.00% had served for between 5 and 15 years.



There were no specifically statistically significant quantitative findings with regard to PSNI.


In terms of reflections with regard to EBP, comments across both days of training highlighted concerns

around whether they would have management support to implement EBP approaches.

“It’s very relevant, whether [or not] your managers will see the importance”

Another concern which participants highlighted, which may be related to having management support,

was the need for prioritisation of such an approach in order to be able to implement it.

“I think it’s relevant, but not in each individual discipline/section of the force, just

neighbourhood/strategic. But there’s no reason why we can’t do this, it’s just getting it

prioristised”

Participant comments appeared to suggest that such prioritisation might overcome the issue of

resources, a concern for all of the forces we visited. Comments reflected the notion that having the

appropriate resources to implement an EBP approach would rely on it being prioritised.

“We need to say we’re going to use this for a specific problem, over a specific time and give it

resources”


Once the trainees started to engage with the training tasks and the CRT, comments largely reflected

positivity about the clarity of the information available on the CRT.

“Very easy to read, easy to work through. See quickly what you need.”

“This is clear and concise”

“This is great, it’s all in one place”

However, participants did have concerns about the limited amount of evidence available on the toolkit.

“Not much on it”

“All of the stuff on here, we do anyway”

“There’s lots of holes in the toolkit”


36

Additionally, there were concerns as to whether the information which is available on the CRT was

relevant to them specifically, in terms of the topics of the research, where the research originated in the

UK and whether it was recent research.

“Not enough in there – is it on the right stuff?”

“A lot of the evidence seemed to be based in the USA/Canada – why haven’t studies in the UK

been bought into the toolkit?”

“Could there be a way of seeing how old the research is? Some of it is really old. I wouldn't use

the lighting research because things change, for instance the types of crime etc. I would go to the

police library and search, use my judgement”


When considering the challenges to identifying an evidence-based intervention, comments reflected

concerns about gaining buy-in within the organisation, particularly gaining buy in from management.

“Getting a high rank to buy in”

“We wouldn’t have any problem identifying an intervention, we would just have a problem

convincing someone in our organisation to understand”

“If you can’t convince one person, the decision-maker, the strongest and loudest voice, it isn’t

going to happen. And sometimes they rely on their experience instead”

There were also concerns about how a lack of communication would impact on the implementation of

EBP approaches, in terms of the buy-in discussed above and also in terms of gaining knowledge about

what approaches are being used.

“It’s like they don’t understand us, us on the ground know the things that they don’t, but they

won’t listen to us. We’re just sergeants, we don’t matter”

“Communication – finding out what others are doing”

A key issue for participants, as with other pilot forces, was the availability of resources to implement an

EBP approach.

“Competing demands for resources within organisation – if it is not a strategic priority then there

won’t be any resources”

“The nuts and bolts of making it work; shift patterns, resources, overtime implications”

“Competing tasks/priorities – officers on the ground have so many things to do”

Finally, participants reflected on their concerns about the risk involved in identifying and implementing

and evidence-based intervention, and how that will impact on individuals feeling able to make the

decisions required to implement such an approach.

“Personal risk if the organisation is conservative/risk adverse”

“No one wants to make a decision/be an advocate – I will try and pass it off to someone else!”


37


Some general feedback on the pilot training days from the PSNI participants includes:

“Worthwhile and valuable” (male, sergeant)

“Enjoyable and informative” (female, sergeant)

“Very informative – has given me a lot to think about” (female, analyst)

“Very interesting and will be practically useful, would be beneficial to all in frontline policing”

(female, analyst)

“It was relevant to my work and has informed me; made me think objectively about what I do and

what I can achieve” (male, sergeant)

“Well delivered and relevant course for ‘forward thinking’ police” (male, constable)

“I think evidence-based/informed policing is the future!” (male, inspector)

“Very useful course to help me focus on using evidence as a basis for any proposed interventions”

(female, inspector)

“I enjoyed it, particularly the discussion points and it is good to be reminded of strategic level

thinking…It is easy to get bogged down in day-to-day issues, good to look at problem holistically”

(female, sergeant)

“It was useful to get a realistic and practical understanding of the Crime Reduction Toolkit. The

concept of evidence-informed policing as opposed to evidence-based policing is very useful”

(female, staff)

PSNI participants most commonly (approximately 50%) identified an increased understanding of the CRT

as the component of the training which was most useful in meeting their learning goals/that they would

take back to their workforce and use.

Other themes the PSNI pilot trainees identified as components of the training most useful in meeting their

learning goals/ that they would take back to their workforce and use included:


- An increased understanding of evaluation research


- The ability to use EMMIE to plan and structure evaluation

Themes the PSNI trainees identified as components of the training least useful in meeting their learning

goals/ that they would not take back to their workforce and use included:

- None! (A large proportion of trainees did not identify any ‘least useful’ components)

“It was all useful” (female, analyst)

- The CRT and its limitations

- The limited relevance of the evidence on the CRT

- The relevance to individual roles was unclear

- A lack of appreciation of resource limits


38

- How to apply the training operationally and the viability within the force

Themes the PSNI trainees identified as components of the training which could be improved included:


- Re-consideration of appropriate target audience (e.g. Senior management team) “Useful information

but I feel input from higher ranks with PSNI specific details on actual implementation of these would

be very beneficial” (female, sergeant)

3.4.3 Summary

24 individuals participated in the pilot training in Northern Ireland, over two days. Comments across both

days highlighted concerns around whether participants would have the management support and

resource prioritisation required to implement EBP approaches. Participants were positive about the

clarity of the information available on the CRT, however were concerned about the limited amount of

evidence on it and whether the evidence was relevant. Buy-in from senior management, a lack of

communication within the force, resources and the personal risk involved were perceived as barriers to

implementing an EBP approach. PSNI participants most commonly identified an increased understanding

of the CRT as the component of the training which was most useful in meeting their learning goals/that

they would take back to their workforce and use. However they felt that the CRT and its limitations, a lack

of appreciation of the resource limits the force faces and a lack of clarity in how EBP applied to specific

roles were the least useful aspects of the training. Participants suggested that real operational examples

and the inclusion of higher ranks could improve the training.


39

4 OVERALL RESULTS & DISCUSSION

4.1 DEMOGRAPHIC INFORMATION

Figure 1: Pie chart illustrating percentage of participants per force

With regards to the overall sample (n=90), the largest proportion of attendees were from Sussex,

followed by PSNI, Scotland, and then Hampshire (see Figure 1).The largest proportion of attendees were

Sergeants, followed by Inspectors, Analysts, Constables, other police staff (e.g. Deputy Principle,

Executive Officer) and Chief Inspectors (see Figure 2). See Figure 3 for the distribution of ranks/roles by

force, with Scotland standing out as having provided no analysts or police staff and Hampshire providing

the smallest proportions of officers.

Figure 2: Pie chart illustrating percentage of participants per rank/role

9%

39%

27%

4%

13%

8%

Percentage of Participants per Rank/Role

Constable

Sergeant

Inspector

Chief Inspector

Analyst

Other police staff

25%

16%

32%

27%

Percentage of Participants per Force

Scotland

Hampshire

Sussex

PSNI


40

Figure 3:Stacked bar chart illustrating rank/role by force

The majority of attendees had served in the police for more than 15 years (58.9%), followed by 6-15 years

(34.4%) and then less than 6 years (6.7%). The majority of attendees who provided their age (n=87), were

aged 41 and over (56.7%), followed by 26-40 (37.8%) and then aged 20-25 (2.2%).

62.2% of participants who provided gender information (n=89) were male, making up the majority of

participants within each rank/role, with the exception of analysts and police staff (see Figure 4).

Figure 4: Clustered bar chart illustrating gender by rank/role

0

5

10

15

20

25

30

35

Scotland Hampshire Sussex PSNI

Number of Rank/Role by Force (n=90)

Constable Sergeant Inspector Chief Inpector Analyst Other Police Staff

0

5

10

15

20

25

30

Constable Sergeant Inspector Chief Inspector Analyst Other Police Staff

Number of Males/Females per Rank/Role

Female Male


41

The majority of participants who provided educational information (n=87) held education to a BA/BSc

level or above (67.8%). See Figure 5 for the number of each educational level, with all forces providing a

number of participants which each level of education. The majority of Scotland, Hampshire and Sussex

participants were educated to BA/BSc level, whilst most of PSNI participants were educated to A Level.

Figure 5: Stacked bar chart showing level of education by force

4.2 QUANTITATIVE FINDINGS

4.2.1 Descriptive findings

4.2.1.1 Who participated and who volunteered to participate?

65.6% of participants had heard of EBP prior to the pilot training and 54.4% had volunteered to attend,

rather than been instructed to attend. It is important to consider the demographic information when

interpreting the findings of the evaluation. For instance, the small numbers of Chief Inspectors and a lack

of any ranks above this will mean that the findings cannot be applied to those officers. Additionally, if

such data/information is available, it is worth considering whether these levels are comparative to UK

police in general, when considering how representative the sample is.

Fisher’s Exact test16 revealed a significant association between how a participant enrolled on the pilot

training programme and their rank (p < .05). Whilst all of the Chief Inspectors had volunteered to attend

(n = 4), Constables were 10.49 times more likely to have volunteered than Sergeants, 9.80 times more

likely than Inspectors, 2.8 times more likely than Other Police Staff and 2.33 times more likely than

Analysts. See Figure 6 for an illustration of the percentages of each rank who volunteered versus were

16 See Footnote 5, p.3.

0

5

10

15

20

25

30

35

40

A Level or Equivalent HNC/HND BA/BSc PostgraduateCertificate/Diploma

MA/MSc

Number of Participants per Level of Education



42

instructed to attend the pilot training.

Figure 6: Stacked bar chart illustrating method of enrolment by rank

There was also an association between enrolment method and force (p < .05) with Hampshire 5.70 times

more likely to have volunteered than Scotland, 5.19 times more likely than Sussex and 1.51 times more

likely than PSNI. See Figure 7 for an illustration of the percentages of each force who volunteered versus

were instructed to attend the pilot training.

Figure 7: Stacked bar chart illustrating method of enrolment by force

0

20

40

60

80

100

120

Constable Sergeant Inspector Chief Inspector Analyst Other police staff

Percentages of Method of Enrolment by Rank

Volunteered Were instructed

0

20

40

60

80

100

120


Percentages of Method of Enrolment by Force



43

Those participants with a Masters were 27.5 times more likely to have volunteered to participate than

those with HNC/HND (p < .005), 17.89 times more likely than those with A Levels, 9.84 times more likely

than those with BA/BSc and 5.5 times more likely than those with a Postgraduate Certificate/Diploma.

See Figure 8 for an illustration of this generally increasing likelihood of volunteering as level of education

increases.

Figure 8: Stacked bar chart illustrating method of enrolment by level of education

Level of education was also associated with whether a participant had participated in any other EBP

activities prior to the pilot training (p < .05) with those with a Masters being 9.15 times more likely to

have participated in other EBP activities than those with a BA/BSc.

This suggests that those keen to participate in EBP related training/activities are likely to have a higher

level of education and that, of the ‘rank and file’ ranks, Constables may be more open to EBP. 25% of the

Constables (n=8) were educated to masters level, whilst 37.5% were educated to the Postgraduate

certificate/diploma. These levels of education are higher than amongst the other ranks, although the

Constables are such a small sample it is difficult to compare with certainty. However, it can be suggested

that for Constables, the level of education may make them more open to EBP as an approach, although it

is also worth considering whether their more recent recruitment into the field of policing may also have

an impact.

It is worth considering the characteristics which have impacted on who has proactively chosen to be

involved in the pilot training, as they may indicate the characteristics of those police officers/staff who

could be most easily reached by communications and activities intended to advance EBP practice within

the profession.

0

20

40

60

80

100

120

A level or equivalent HNC/HND BA/BSc Postgraduatecertificate/diploma

Masters

Percentages of Method of Enrolment by Level of Education



44

4.2.1.2 Awareness of EBP & College Activities

65.6% of participants had heard of EBP prior to the pilot training. Fisher’s Exact test revealed a significant

association between awareness of EBP prior to the pilot training and force (p < .05). This seems to be

because those in Hampshire were 14.18 times more likely to be aware of EBP prior to the pilot training

than those in Scotland, 10.998 times more likely than those in PSNI, and 4.14 times more likely than those

in Sussex. 35.71% of Hampshire participants were analysts, compared with none of the participants from

Scotland and so these findings may be due to significant associations between awareness of EBP and

rank/role (p < .005); analysts were 14.67 times more likely to have heard of EBP than sergeants, 6.59

times more likely than Inspectors, and 1.57 times more likely than Constables. All of the Chief Inspectors

and other police staff had heard of EBP prior to the training. See Figure 9 for an illustration of awareness

of EBP.

Figure 9: Stacked bar chart illustrating awareness of EBP by rank/role

Hampshire were also five times more likely to have heard of Evidence-based Master Classes than Sussex

(p < .05 – no participants in Scotland or PSNI had heard of the Master Classes).

Other police staff (not analysts) were 14.64 times more likely to have heard of EBP components of

College training programmes than Inspectors (p < .05), 7.96 times more likely than Sergeants, 6.65 times

more likely than Analysts, 2.22 times more likely than Constables and 1.33 times more likely than Chief

Inspectors.

There were no significant differences between the forces with regards to any of the other College

activities asked about in the pre-pilot training questionnaire:

- 4.4% of the sample about Evidence Base Camp (1 individual had participated)

- 6.7% of the sample had heard of Evidence-based Master Classes (1 individual had participated)

- 7.8% had heard of Research Surgeries

- 20% had heard of EBP components of College training programmes

0

20

40

60

80

100

120

Constable Sergeant Inspector Chief Inspector Analyst Other police staff

Percentages of awareness of EBP pior to the pilot training by rank/role

Have heard of EBP before Have not heard of EBP before


45

63.3% of participants had heard of POLKA. Fisher’s Exact test revealed a significant association between

force and awareness of POLKA (p < .001) with Hampshire participants being 39.77 times more likely to

have heard of POLKA than participants from Scotland, 11.85 times more likely than participants from PSNI

and 2.92 times more likely than participants from Sussex.

Analysts were 16 times more likely than Sergeants to have heard of POLKA (P < .005), 8.57 times more

likely than Inspectors, 4 times more likely than both Constables and Chief Inspectors and 1.71 times more

likely than Other Police Staff. See Figure 10 for an illustration of the total 63.3% of the sample who had

heard of POLKA, split by rank/role. When compared with Figure 2 which illustrated the percentages of the

total sample by rank/role, you can see how analysts and other police staff are overrepresented

Figure 10: Pie chart illustrating percentage of participants who had heard of POLKA per rank/role

Level of education was also associated with who had heard of POLKA (p < .05), with the odds increasing

with level of education and those with a Masters being 7.31 times more likely to have heard of POLKA

than those with A Levels.

There were no significant differences between the forces with regards to awareness of any of the other

College resources asked about:

32.2% of participants had heard of the WWCCR

71% had heard of the Crime Reduction Toolkit

16.7% had heard of the College Research Map

10% were aware of whether their force had a Frontline Champion (these were more likely to be

Chief Inspectors, p < .05)

Chief Inspectors were 17.96 times more likely to have heard of the WWCCR than Sergeants (p < .005),

9.01 times more likely than Inspectors, 3 times more likely than either Constables or Analysts and 1.2

times more likely than other police staff. Level of education was also associated with who had heard of

WWCCR (p < 0.005), with the odds increasing with level of education and those with a Masters being 35

times more likely to have heard of the WWCCR than those with A levels.

11%

26%

25%

5%

21%

12%

Who had heard of POLKA?

Constable

Sergeant

Inspector

Chief Inspector

Analyst

Other police staff


46

4.2.1.3 Use of Research and College resources

Prior to the pilot training:

42.2% had used research (journals, books, reference material)

50% had used the College website

10% had used the College research map

31.1% had used the Crime Reduction Toolkit

55.6% had used POLKA

16.7% had used the National Police Library

There was a significant association between Level of Education and use of POLKA (p < .005), with those

with a BA/BSc being 12.264 times more likely than those with A Levels to have used POLKA, 4.33 times

more likely than those with HNC/HND, 1.63 times more likely than those with a Postgraduate

Certificate/Diploma and 1.22 times more likely than those with a Masters. See Figure 11 below for an

illustration of the distribution.

Figure 11: Stacked bar chart illustrating percentage of use of POLKA by level of education

A similar pattern emerged with a significant association between Level of Education and use of the

College website (p < .05) – those with a BA/BSc were 16.63 times more likely to have used it than those

with a HND/HNC.

4.2.1.4 Previous research/evaluation experience

21.1% of the sample had undertaken a piece of research with a university prior to the pilot training.

Fishers Exact test revealed an association between force and previously having undertaken a piece of

work with a university (p < .05) with participants from Hampshire being 14.03 times more likely than

Scotland to have undertaken a piece of research with a university, 9.32 times more likely than those from

PSNI and 5.11 times more likely than Sussex.

0

20

40

60

80

100

120

A level/equivalent HNC/HND BA/BSc Postgraduatecertificate/diploma

Masters

Percentage of use of POLKA by Level of Education

Had used POLKA Had not used POLKA Not sure


47

Other Police Staff were 14.18 times more likely to have undertaken a piece of research with a university

than Sergeants (p < .005), 9.32 times more likely than Inspectors, 4 times more likely than Analysts, and

1.33 times more likely than both Chief Inspectors and Constables.

18.9% reported they had previously formally assessed a crime intervention in terms of its effectiveness.

4.2.2 Pilot findings

4.2.2.1 Pre-pilot training beliefs17

This section provides an overview of participant beliefs about the relevance of the training, the relevance

and value to research and evaluation and their own levels of research skills prior to the pilot training.

Whilst this sample may not be representative of all police officers and staff across the UK, the statements

may provide a level of understanding of the sample baseline beliefs and attitudes with which to compare

post-training beliefs.

4.2.2.1.1 Relevance and value of the training

In their pre-pilot training questionnaires:

82.9% of participants agreed or strongly agreed with the statement “I believe the pilot

programme will be relevant to my job” whilst 17% responded ‘Don’t know’.

85.3% of participants agreed or strongly agreed with the statement “I believe the pilot

programme will be relevant to my force” whilst 13.6% responded ‘Don’t know’ and 1.1%

disagreed with the statement.

75% of participants agreed or strongly agreed with the statement “I believe the skills learnt during

this course will be valued by my force” whilst 21.6% responded ‘Don’t know’, 2.3% disagreed and

1.1% strongly disagreed with the statement.

4.2.2.1.2 Relevance and value of research/evaluation


97.7% of participants agreed or strongly agreed with the statement “Research can be useful for

my role” whilst 1.1% disagreed and 1.1% strongly disagreed with the statement.

93.2% of participants agreed or strongly agreed with the statement “Research is important for my

role” whilst 2.3% responded ‘Don’t know’, 3.4% disagreed and 1.1% strongly disagreed with the

statement.

90.9% of participants agreed or strongly agreed with the statement “Research can be used to

inform my day-to-day work” whilst 2.3% responded ‘Don’t know’, 5.7% disagreed and 1.1%

strongly disagreed with the statement.

71.6% of participants agreed or strongly agreed with the statement “Working with universities

can be useful for my job” whilst 26.6% responded ‘Don’t know’, and 2.3% disagreed with the

statement.

90.9% of participants agreed or strongly agreed with the statement “Formally assessing the

effectiveness of crime reduction interventions is important for my role” whilst 3.4% responded

‘Don’t know’, 4.5% disagreed and 1.1% strongly disagreed with the statement.

17 Where values do not add up to 100%, this indicates missing data where participants failed to complete a question in the questionnaires. Here, valid percentages have been reported.


48

4.2.2.1.3 Research Skills


78.4% of participants agreed or strongly agreed with the statement “I am able to access research

materials” whilst 14.9% responded ‘Don’t know’ and 13.8% disagreed with the statement.

72.7% of participants agreed or strongly agreed with the statement “I am able to identify

appropriate research materials” whilst 12.5% responded ‘Don’t know’, 13.6% disagreed and 1.1%


80.7% of participants agreed or strongly agreed with the statement “I am able to interpret

research” whilst 10.2% responded ‘Don’t know’ and 9.1% disagreed with the statement.

71.3% of participants agreed or strongly agreed with the statement “I am able to assist others in

using research to make decisions” whilst 14.9% responded ‘Don’t know’ and 13.8% disagreed

with the statement.

61.3% of participants agreed or strongly agreed with the statement “I believe I have the

knowledge and skills needed to assess the effectiveness of a crime reduction intervention” whilst

20.5% responded ‘Don’t know’, 15.9% disagreed and 2.3% strongly disagreed with the statement.

48.9% of participants agreed or strongly agreed with the statement “I know where I can get

assistance to assess the effectiveness of crime reduction interventions” whilst 23.9% responded

‘Don’t know’, 25% disagreed and 2.3% strongly disagreed with the statement.

4.2.2.2 Post training beliefs

This section provides an overview of participant beliefs about the relevance and value of the training and

the relevance and value to research and evaluation following the pilot training. Where a belief was

measured in both the pre and post training questionnaires, and a significant difference was found, this is

reported.

4.2.2.2.1 Relevance and value of the training

In their post-pilot training questionnaires:

87.8% of participants agreed or strongly agreed with the statement “The training was relevant to

my role” whilst 2.2% responded ‘Don’t know’, 5.6% disagreed and 4.4% strongly disagreed with

the statement. There was no significant difference between these beliefs and the related pre-

training belief.

95.5% of participants agreed or strongly agreed with the statement “Evidence-informed policing is

relevant to my role” whilst 4.4% disagreed with the statement. There was no significant

differences between this and the spread of beliefs with regard to the pre-training statement “I

believe the pilot programme will be relevant to my job”.

90% of participants agreed or strongly agreed with the statement “I would like to apply the

knowledge/techniques from the training in my role” whilst 1.1% responded ‘Don’t know’ and 8.9%

disagreed with the statement.


49

60% of participants agreed or strongly agreed with the statement “The skills I learnt during this

course will be valued by my force” whilst 18.9% responded ‘Don’t know’, 20% disagreed and 1.1%


Wilcoxon signed rank test18 revealed a small but significant difference between participant beliefs

pre and post training with regard to whether the skills learnt will be valued by their force, with

more individuals appearing to disagree with the statement following the training (T = 261, p

= .002, r = -.02).

When the data was split by force, only Scotland (T = 18, p = .047, r = -.04) and Hampshire (T = 1, p

= .040, r = - .07) remained statistically significant outcomes. Once again, these are very small

effect sizes. However, Figure 12 illustrates that Scotland and Hampshire had higher proportions

of doubt as to whether the force would value the skills learnt during the training (26% and 28.6%

respectively). There were no significant differences when split by rank.

Figure 12: Stacked Bar chart Illustrating Percentage of Responses to “The skills I learnt during this course will be valued by my force” per Force

4.2.2.2.2 Relevance and value of research/evaluation

In their post-pilot training questionnaires:

98.9% of participants agreed or strongly agreed with the statement “Research can be useful for

my role” whilst 1.1% disagreed with the statement. Whilst this is just a slightly higher proportion

of positive responses than prior to the training, a Wilcoxon Signed Rank Test revealed a small but

significant negative difference between participant beliefs pre and post training with regard to

research being useful to their role, with slightly more positive beliefs prior to the training (T = 83,

18 See Footnote 4, p.3.

0

20

40

60

80

100

120


Percentage of each response to “The skills I learnt during this course will be valued by my force” per Force

Strongly Disgaree Disagree Don't know Agree Strongly Agree


50

p = .020, r = -.01) and individuals actually being more likely to have reduced their score following

the training.

Given the figure, it would suggest that participants reduced their scores from ‘strongly agree’ to

‘agree’ following the training, as can be seen in Figure 13. When the data was split by force, this

only remained significant for Hampshire (T = .000, p = .046, r = -.07) although, again, this was a

very small effect. When the data was split by rank the result only remained significant for

Inspectors (T = .000, P = .014, r = -.05).

Figure 13: Clustered bar chart illustrating frequency of responses to "research can be useful to my role" before and after training

91.1% of participants agreed or strongly agreed with the statement “Research is important for my

role” whilst 1.1% responded ‘Don’t know’ and 7.8% disagreed with the statement. Once again,

there was a small but significant negative difference between participant beliefs pre and post

training, with slightly more positive beliefs prior to the training (T = 170, p = .036, r = -.01) and

individuals actually being more likely to have reduced their score following the training.

It again appears that participants reduced their scores from ‘strongly agree’ to ‘agree’ following

the training, as can be seen in Figure 14.

0

10

20

30

40

50

60

Strongly Disagree Disagree Agree Strongly Agree

Frequency of Responses to "Research can be useful to my role" Before and After Training

Before training After training


51

Figure 14: Clustered bar chart illustrating frequency of responses to "research is important to my role" before and after training

88.9% of participants agreed or strongly agreed with the statement “Research can be used to

inform my day-to-day work” whilst 10% disagreed and 1.1% strongly disagreed with the

statement. Once again, there was a small but significant negative difference between participant

beliefs pre and post training, with slightly more positive beliefs prior to the training (T = 146, p

= .006, r = -.01) and individuals actually being more likely to have reduced their score following

the training. When the data was split by force, this only remained significant for Scotland (T

= .000, p = .005, r = -.06) although, again, this was a very small effect. Once again, when the data

was split by rank the result only remained significant for Inspectors (T = .000, P = .020, r = -.05).

Figure 15: Clustered bar chart illustrating frequency of responses to "research can be used to inform my day-today work" before and after training

0

10

20

30

40

50

60

Strongly Disgaree Disagree Don't know Agree Strongly Agree

Frequency of Responses to "Research is important to my role" Before and After Training


0

10

20

30

40

50

60

Strongly Disagree Disagree Don't know Agree Strongly Agree

Frequency of Responses to "Research can be used to inform my day-to-day work" Before and After Training



52

85.5% of participants agreed or strongly agreed with the statement “Working with universities

can be useful for my job” whilst 10% responded ‘Don’t know’, and 4.4% disagreed with the

statement. Whilst the proportion of individuals who agreed or strongly agreed with the

statement increased following the training, this was not significant.

90% of participants agreed or strongly agreed with the statement “Formally assessing the

effectiveness of crime reduction interventions is important for my role” whilst 2.2% responded

‘Don’t know’, and 7.8% disagreed with the statement. There were no significant differences in

belief about this statement before and after the pilot training.


53

4.2.2.3 Perceptions of the training

Table 1 provides the percentage of participant responses to post-pilot training questionnaire questions

regarding their perceptions of the training. Across all of the statements regarding perceptions of the

training, there was an average of 83.66% positive feedback.

Statement on post-pilot training question

% participants who agreed or strongly

agreed

% participants

who responded

‘don’t know’

% participants

who disagreed

% participants

who strongly

disagreed

The training content was interesting to me

91.1 0 7.8 1.1

As a result of this training, I am more able to access research materials

70 6.7 21.1 2.2

As a result of this training, I am more able to identify appropriate research materials

74.5 4.4 20 1.1

As a result of this training, I am more able to interpret research

68.9 7.8 20 3.3

As a result of this training, I am more able to assist others in using research to make decisions

73.5 7.8 15.6 2.2

As a result of this training, I am more able to assess the effectiveness of a crime reduction intervention

85.4 4.4 11.1 0

As a result of this training, I am more likely to get assistance to assess the effectiveness of crime reduction interventions

73.4 7.8 16.7 2.2

The structure/order of the training day was appropriate

95.5 1.1 3.3 0

The pace of the training day was appropriate

95.6 0 4.4 0

The visual aids (e.g. PowerPoint, handouts) were effective in helping me to learn

87.7 1.1 11.1 0

The Case Study task was effective in helping me to learn

91.1 2.2 5.6 1.1

The group discussions were effective in helping me to learn

94.4 1.1 4.4 0

The instructor was effective in helping me to learn

97.8 1.1 0 1.1

The facilities (e.g. the room, the computer access) were satisfactory

72.7 1.1 23.3 3.3

Table 1: Percentages of responses to statements regarding perceptions of the training


54

4.2.3 Quantitative Findings Summary

Findings suggest that those keen to participate in EBP related training/activities are likely to have a higher

level of education and that, of the ‘rank and file’ ranks, Constables may be more open to EBP. It is worth

considering the characteristics which have impacted on who has proactively chosen to be involved in the

pilot training, as they may indicate the characteristics of those police officers/staff who could be most

easily reached by communications and activities intended to advance EBP practice within the profession.

A high percentage of participants had positive beliefs about the relevance and value of the training,

research/evaluation and their own research abilities prior to the pilot training, although only 48% of

participants felt they knew where to get assistance to assess the effectiveness of a crime reduction

intervention prior to the pilot training. This level of positivity prior to the pilot training appeared to reduce

in a small but significant way following the training. It is suggested that once participants realised the time

and resources required to implement what they had learned, they were less sure they would have the

support of their senior management to achieve this.

Across all of the statements regarding perceptions of the training itself (e.g. “the training was

interesting”; “I am more able to assess the effectiveness of a crime reduction programme”; “The Case

Study task was effective”), there was an average of 83.66% positive feedback.


55

4.3 QUALITATIVE FINDINGS The following section reports the themes identified in the qualitative responses to the post-training

questionnaire, observation notes and the notes completed as part of a reflective session following each

pilot training day. Whilst other themes were identified, the themes reported here were identified most

consistently across all of the forces and are collected under the following headings:

- Perceived benefits of the training

- Perceived limitations of the training

- Suggested improvements to the training

- Feedback on the CRT

- Reflections on EBP

The number of quotes provided for each theme is not related to the number of participants who

reflected a particular theme, but rather were those chosen to illustrate most clearly the themes and

interpretation provided. As some of these core findings have already been reported in the overview of

the individual forces in section 3, some of these quotes may be repeated within the following sections

where they are explored further.

4.3.1 Perceived benefits of training

When asked what component of the training they felt was most useful in meeting their learning goals and

what aspects they would take back to their workplace and use, participants across the sample identified

that developing an understanding of the CRT, EMMIE and evaluation research and how it can be useful to

their role enabled them to identify a new approach to problem solving and decision-making within their

role. Participants reflected that the group discussion and the responsiveness and knowledge of the

trainers facilitated an open forum which they felt was beneficial to meeting their learning needs. These

themes will be discussed in the following sections.

“The existence of the ‘what works’ crime reduction activities and the value of it as another tool to

use in conjunction with existing problem solving models” (Male, Constable)

4.3.1.1 Increased understanding of the CRT, EMMIE and evaluation research

The participants most frequently mentioned the CRT, EMMIE and evaluation research as components of

the training which they felt were most useful in meeting their learning needs and that they would take

back to their workplace and use.

Participants discussed an increased awareness of the CRT, how to use it and when it might be helpful as

beneficial for them. The case study tasks were identified as useful in improving the participants’ ability to

apply the toolkit to their day-to-day role, by introducing them to the CRT and providing them with an

opportunity to use it in a practical way.

“Practical ability to see, use and experience the toolkit” (Male, Inspector)

“The use of the toolbox, this gave me an opportunity to see how it is used, how it can be used

and the ways I could potentially apply it in my role” (Female, Staff)

“Learnt a lot about the toolkit which will help me on a day-to-day basis – thank you” (Male,

Sergeant)


56

“I looked prior to the training and found it confusing so training helped” (Female, Staff)

The participants identified various situations in which they would use the CRT within their role, including

when they are writing reports and recommendations and when they are working within partnerships,

primarily focused on providing them with evidence to support a new way of working.

“I will look at the Crime Reduction Toolkit when writing reports (useful for the recommendation

sections)” (Female, Staff)

“Knowing about access to specific research to help me evidence in support of good practice/new

ways of working” (Male, Sergeant)

“Introduction to the Crime Reduction Toolkit and ability to make more informed decisions” (Male,

Sergeant)

“Consulting the tool during any intervention design process” (Male, Constable)

For some, the increased awareness and understanding of the way in which CRT could inform their

practice was particularly important, as they felt that this had not been communicated to them previously.

By introducing the CRT as something which could inform their practice, alongside their professional

experience, participants felt they had a more appropriate understanding of the CRT, which accepted its

limits and did not dismiss the experience and expertise they could bring to problem solving.

“To learn that the toolkit was to help inform interventions my perception beforehand was that it

should be used as the ‘solution’ to the problem” (Female, Staff)

“It was useful to get a realistic and practical understanding of the Crime Reduction Toolkit”

(Female, Staff)

In this way, the participants felt that by simplifying the CRT and its uses, they had been provided with a

new approach to problems they come across in their roles.

“Use of toolkit as a facility to identify other means to address ongoing problems/issues faced on a

daily basis in my area of work” (Female, Sergeant)

“Understanding the method in selecting any particular intervention and how the tool can be

used in this process” (Male, Constable)

This new approach to problem solving was evident in their discussion of how being introduced to EMMIE

had provided them with a way to structure the new approach. In particular, it providing them with a

framework to assess research against and also to structure their design of intervention evaluation.

“EMMIE acronym was a useful tool to incorporate into my and my team’s work” (Female, Analyst)

“EMMIE provides a structure for planning and implementing an evaluation of a crime reduction

intervention” (Female, Chief Inspector)

“Very useful to have a framework to assess research against” (Female, Analyst)

When discussing EMMIE as a perceived benefit of the training, participants were able to identify specific

current and future tasks which they would like to apply EMMIE to.


57

“EMMIE is a useful evaluation tool and one that will be of benefit to me in evaluating training

programmes and product delivery” (Male, Sergeant)

“I have several research projects in the pipeline so I would be keen to apply EMMIE to them”

(Male, Sergeant)

“Framework ‘EMMIE’ and implications for future interventions planned” (Male, Constable)

The participants were highlighting how their understanding of the CRT and EMMIE fed into an increased

understanding of the need for evaluation research. This increased awareness of the need for evaluation

research, and an increased understanding of how it is carried out was highlighted as a key benefit of the

training for many of the participants.

“The need for more and competent evaluation of policing practice” (Male, Constable)

“The general debate which shows how we ‘knee-jerk’ decision make and rarely evaluate.

Reflection has shown that we should not continue with this approach but there are many

challenges ahead to alter this culture” (Female, Inspector)

In particular, the participants emphasised the way in which understanding basic research and evaluation

methods such as identifying before and after measures prior to implementing an intervention can be

used in their roles, and the benefits it can have in changing police culture.

“The session has helped me to look at a new way of evaluation. This will help me in my role”

(Male, Sergeant)

“The importance of having a baseline informed as to what we are looking to change via

appropriate interventions” (Male, Sergeant)

“Setting aims and goals at onset of a project, as well as taking the time at the end to assess

impacts and learn from experience” (Female, Analyst)

“Making it clear that evaluation starts before intervention, rather than post intervention.

Stopping the ‘doomed to succeed’ culture” (Male, Staff)

Participants felt they had been provided with a structure which allowed them to develop definitions,

through identifying a measure for each level of EMMIE, of what ‘success’ of an intervention really looks

like.

“The importance of identifying what the problem is you are trying to effect” (Male, Sergeant)

“Identify and objectify what is success in terms of an intervention” (Female, Sergeant)

“Looking at how to tell if an implemented intervention was working if it was worth the resources

and how to assess this” (Female, Sergeant)

4.3.1.2 Identification of a new approach to problem solving and decision-making

The increased understanding of the CRT, EMMIE and evaluation research described above feeds into the

way in which participants felt they had been provided with a new set of tools and approach to policing.

“Looking at possible intervention methods from a different perspective” (Female, Sergeant)


58

“Very useful course to help me focus on using evidence as a basis for any proposed

interventions” (Female, Inspector)

“An introduction to a system I had no knowledge of beforehand” (Male, Inspector)

For many of the participants there was a level of positivity and enthusiasm for EBP/EIP as an approach,

evident when they were discussing the benefits of the training. For some of the participants this was a

marked change from how they had predicted they would feel about the training.

“It makes you think differently – the rest of the analysts would benefit, are you coming back?”

(No gender/rank recorded)

“I think evidence-based/informed policing is the future! It ties in with P.E.E.L and provides

rationale for decision making” (Male, Inspector)

“I was very sceptical about what I may learn from the day; but changed to an interested

participant and can see the value of EBP on a day-to-day and strategic level” (Male, Staff)

4.3.1.3 Opportunities for group discussion

Many of the benefits of the training which the participants identified were linked to having the space to

discuss the EBP approach and its issues within a group context with their peers.

“The group discussions and hearing other people’s views were particularly useful” (Female, Staff)

“The group discussions helped me understand the importance of research” (Male, Sergeant)

In particular, discussing the approaches and issues which the training addressed were a way of enabling

the participants to share their experiences with their peers in an open forum, providing participants with

a wider understanding of how other people might approach or deal with problem solving.

“The opportunity to share experiences and knowledge with like-minded colleagues” (Female,

Staff)

“I enjoyed the input and the frank discussions with my peers” (Male, Sergeant)

4.3.1.4 Responsiveness and knowledge of the trainers

Alongside the opportunity for discussion with their peers, participants also placed value on the

interaction they experienced with the trainers.

“The opportunity to share experiences and knowledge with like-minded colleagues. Similarly the

valued input from Nick and Jenny. Thank you.” (Female, Chief Inspector)

“Team delivering responded to audience and adapted content relevant to need of group to

facilitate benefits to all in the audience” (Female, Sergeant)

The responsiveness of the trainers was highlighted in particular when the participants reflected on the

challenges related to putting an EBP approach into practice as well as some of the limitations of the CRT

and the research base.

“Trainers were accommodating and knowledgeable – sadly what they were working with let them

down!” (Male, Inspector)


59

“I thought you were very tolerant, accommodating to us. Our frustration is the topic, not the

training” (No gender/rank recorded)

“The trainers were very tolerant of our challenging questions and views, for which thank you”

(Female, Chief Inspector)

“Good presentation with clear messaging, participants showed naivety or lack of knowledge of

organisational initiatives/changes and opportunities” (Male, Staff)

By recognising and responding to the participants’ concerns around a new approach to their roles, the

trainers were able to build a rapport which enabled the participants to appreciate the content of the

training and recognise the value it could have in informing their roles.

4.3.1.5 Enjoyable

Finally, the other most commonly perceived benefit of the training was that it was enjoyable for the

participants themselves, which may have enabled them to consider a challenging and potentially

controversial topic.

“Very enjoyable and thought provoking” (Male, Constable)

4.3.2 Perceived limitations of training

“This was a useful and interesting day but vast improvements need to be made on the toolkit to

make it effective. Local sharing needs to be nailed first. Thank you.” (Female, Constable)

When asked what component of the training they felt was least useful in meeting their learning goals and

what aspects they would not take back to their workplace and use, participants across the sample focussed

on what they perceived as the limitations of the CRT. While another large proportion of the participants

considered all aspects of the training to be useful in some way, other participants had concerns about the

viability of EBP within their force, the relevance to their individual role and a lack of appreciation of

resource limits. This led to a concern that “nothing will change”.

4.3.2.1 Limitations of the CRT

When asked about the limitations of the training, many participants referred to the CRT itself.

“The content of the toolkit is not fit for purpose i.e. it was out of date/not current focused too

heavily on American studies” (Female, Staff)

“The crime reduction toolkit database – this training just highlighted how inadequate as a

resource it is at this time, too few interventions available” (Female, Inspector)

As the feedback on the CRT is covered more widely in section 4.3.4 this will not be covered in depth here.

However, it is worth considering whether the inclusion of other resources to access research in any future

training has the potential to be useful. Many of the concerns participants raised with regards to the CRT

and its impact on the training relate specifically to the systematic approach used to select the evidence

available on the toolkit. If future training were to include access to a wider selection of research

resources, there would need to be a consideration of the threshold which such research and resources

would need to meet to be included.


60

4.3.2.2 None!

“All of it will have some impact/application” (Female, Staff)

When asked to identify limitations of the training and components they would not take back to their

workplace and use, a large proportion of the participants specified that they did not feel there were any.

4.3.2.3 The viability of EBP within the force

Participants noted concerns about how viable the EBP approach is within their organisation. Whilst many

of their concerns in this area are outlined in more detail through their reflections on EBP in section 4.3.5,

some of these are particularly important when considering delivering future training.

“The proposed model is out of sync with the current force direction to a more focused model on

core areas of policing and a move away from crime reduction and crime prevention” (Female,

Sergeant)

“How will it actually work in the organisation?” (No gender/rank recorded)

“I think all aspects were relevant, it would mainly come down to having the time to do it”

(Female, Analyst)

“All relevant, worthwhile finding out about the feasibility of university research as unsure about

this” (Female, Analyst)

The pilot training was unable to provide the participants with a clear understanding of how EBP would be

implemented within their force, particularly in terms of organisational structure and division of resources.

“Appears to be very time consuming and I don’t have the time” (Male, Sergeant)

“Working with universities will/may be difficult at a local level” (Female, Inspector)

“Having the time to carry out research, and carry out detailed reviews, is not practical in my daily

duties” (Male, Sergeant)

“I suppose that nothing changes the limitation of resources at my disposal” (Unknown gender,

Inspector)

Such information is integral to the participants being able to perceive themselves as using the skills they

acquire in the training in their day-to-day role.

“Being able to use the information contained within the training in an operational environment”

(Male, Sergeant)

This clearly leads to a need for future training to consider the role that force clarity in how they intend to

uptake and use EBP and good communication of these intentions can have in improving the training and

desired outcomes. This may be through the input of senior management in the actual training itself, or

through organisational communications which precede or follow any training.

“This will need follow up from the knowledge hub and other areas to have maximum impact”

(Male, Inspector)


61

4.3.2.4 Relevance of EBP to individual roles

Linked to participant concerns about the viability of EBP within their force, were their concerns about the

viability of EBP within their role. Some participants felt that the skills they learnt during the training were

unlikely to be relevant to their role in the organisation.

“Ultimately it comes down to time. I don’t have time to research studies and create fully

evaluated research/interventions. I will focus on the day job and keeping command happy!”

(Male, Sergeant)

“Implementing interventions is not applicable to my role so therefore less useful” (Female, Staff)

“My day-to-day role is reactive policing and not problem solving – not a way I can utilise this

myself” (Male, Inspector)

“Having the time to carry out research, and carry out detailed reviews, is not practical in my daily

duties” (Male, Sergeant)

“Evaluation/review felt less relevant to role due in part to time to undertake this work – time is

very limited and would not be prioritised in this way” (Male, Chief Inspector)

Therefore, greater clarity in how each individual force intends to implement EBP on a role by role basis

will greatly improve the individualisation and focus of the training, whilst also identifying those roles for

whom the training is most suitable and relevant.

4.3.2.5 A lack of appreciation of resource limits

As is evident in many of the previously discussed qualitative findings, including apprehension about the

viability of EBP within individual forces and roles, participants had particular concerns that an EBP

approach, and thus the training itself, did not take into account resource limits.

“Perhaps a greater appreciation of policing budget context” (Female, Chief Inspector)

“The ‘review’ section is not always practical in my role due to staff shortages, workload etc.”

(Female, Staff)

“Appears to be very time consuming and I don’t have the time” (Male, Sergeant)

Once again, if any future training were able to provide clarity around how EBP is expected to fit into

individual organisations and roles, what the responsibility of individuals would be and whether time and

resources would be allocated for the purpose of achieving components of EBP, participants would be

better able to use the training to inform their day-to-day practice.

4.3.2.6 “Nothing will change”

For some participants, there was no clarity on how EBP would be implemented in either their force or the

wider policing profession.

“I will attempt to feed back the positives and not the negative opinion that nothing will change”

(Female, Staff)

“I am not convinced attitudes within the force will change” (Female, Staff)


62

It is suggested that a wider organisational focus on EBP and awareness raising of EBP and its components

across all ranks and roles would enable those who are expected to have a specific role to perceive such

changes occurring.

4.3.3 Suggested improvements to training

Some suggested improvements to the training have been easily incorporated into the Trainer Guide

(Fleming, Fyfe & Wingrove, 2016b) and the training slides, for example relating to the size of the text on

slides and the structure of the training:

“There may be benefit in being clearer about the objectives of the training i.e. to introduce

EMMIE, to introduce evidence-informed policing, to introduce the toolkit” (Female, Chief

Inspector)

“Better structure – i.e. at the beginning, what are we going to learn today, what are the learning

outcomes, what is the purpose etc.” (Female, Analyst)

Therefore some of the slides have been edited to increase the visibility of the text, learning goals have

been inserted at the beginning of the training, alongside an outline of the training. This is in addition to

the trainee take-home materials which already incorporated the learning goals.

Alongside these suggested adjustments to the training materials were wider proposed improvements to

the pilot training. The most commonly recommended improvement was the inclusion of real operational

examples of the implementation of EBP. Many of the participants suggested this. Additionally, linked to

those concerns noted in the perceived limitations of the training, participants felt improvements to the

CRT and a closer consideration of how EBP would work for specific ranks/roles would improve the

training. Better facilities and more time were also common suggestions.

4.3.3.1 Real operational examples

The most recurrent suggested improvement to the training was to incorporate real operational examples, with participant comments seeming to suggest this would provide them with more clarity and confidence in their ability to implement an EBP approach in practice.

“More detailed analysis of real life case studies and concentration on the intervention process

from start to finish” (Male, Sergeant)

“It would probably help the understanding if more examples were available” (Male, Sergeant)

Additionally, it is suggested that participants also sought such examples to provide them with a level of

confidence that such an approach is both possible and effective.

“Examples of problems resolved through the use of the crime reduction toolkit” (Male, Sergeant)

“An example of an initiative that worked (case study)” (Male, Sergeant)

Some of the reflections specifically relate to the participants’ need for an exemplar of EBP approaches

which have been successful in a context which they can more easily relate to their own. This can be

understood in relation to participant concerns about the lack of UK evidence available on the CRT, but

also in relation to their need for an example to come from a local context and individual.

“I would have liked to be shown an example (UK) where it has been used” (Male, Inspector)


63

“Examples of local police services implementing new ideas” (Male, Sergeant) “Useful information but I feel input from higher ranks with [force] specific details on actual

implementation of these would be very beneficial” (Female, Sergeant)

Therefore, there are several interpretations of, and reasons for, participants’ desire for real operational

examples. Some of these may relate to providing them with confidence that such an approach would be

‘successful’ in the simplest sense, reducing crime. Others may relate more closely to a need to have the

training situated more closely within the context of their own force, with an understanding of the

practical and organisational processes which would need to be in place to enable the participants to use

the skills they were learning during the training in their day-to-day job. This may relate back to the

concerns participants held around the viability of EBP within their force.

It is also important to note the value that the participants place on hearing the examples of real

operational practice from actual practitioners. In this way they are seeking credible voices19.

“More discussion and examples from practitioners of good practice” (Male, Inspector)

“More interaction around the case studies, perhaps using speakers to champion how they had used

interventions successfully and how implementation and evaluation had gone” (Female, Inspector)

4.3.3.2 Further consideration of the appropriate target audience and participants

The requirement for credible voices promoting EBP as an approach was also reflected in suggestions that

the pilot training may be more appropriately aimed at senior management, pitched at higher ranks, or

that senior management were at least involved in the delivery.

“Ensure district commanders attend this training first and buy into it before delivering to

subordinates” (Male, Inspector)

“Possible inclusion of Senior Management Team member to address specific local

issues/questions from attendees” (Male, Constable)

“Most suitable for input would have been Chief inspector/super” (Male, Sergeant)

These suggestions relate to the previously discussed concerns participants held around the viability of

EBP within their force and how it applies to their specific roles. The general view was that when and if

senior management attended the training and endorsed it, participants would be confident that they

would have the organisational support, and thus the resources, required to implement EBP.

Despite these concerns, there was also recognition that there is a requirement for the training to reach all

ranks in order to enable effective implementation of EBP. One participant in particular asked who the

session was aimed at:

“PCs? They’ve got enough to do, there are lots of systems to use, too many. They can’t take it all

in, it takes too long” (No gender/rank recorded)

19 Petty & Cacioppo’s (1986) Elaboration Likelihood Model of persuasion posits that when a receiver of a message lacks the ability or motivation to consider several arguments within a persuasive message, they are likely to resort to ‘peripheral’ cues such as whether the source of the message is a credible expert.


64

A response to this comment from another participant identified the way in which the presence of

differing ranks and roles at the training could contribute to improving understandings of how the various

roles are vital to achieving an EBP approach:

“There’s lots of power in the PCs’ hands. I, as an inspector, don’t know everything, my PC can tell

me no this won’t work” (Male, Inspector)

4.3.3.3 Improvements to the CRT

“Improving the toolkit so it is credible and applicable to practitioners”

Once again, as mentioned when discussing what the participants perceived as the limitations of the

training, participants articulated that they felt improvements to the CRT would lead to improvements in

the training itself. The types of improvements suggested were varied and are discussed in more detail in

section 4.3.4, however, they relate generally to a wider selection of evidence being available on the CRT

“More content within the What Works Crime Reduction Toolkit” (Female, Staff)

“A better knowledge base in the toolkit and more real life UK based examples” (Male, Inspector)

“Changing the tool as it lacks value in operational policing. At the very least it should offer a

POLKA style link which allows practical usage of interventions to be considered” (Male, Sergeant)

4.3.3.4 Better facilities

“Only the facilities – computers not working therefore unable to fully participate” (Male,

Inspector)

For some of the participants, their main priority for improving the training would be to ensure the

facilities were appropriate and effective. In particular, the vast majority of the facilities used for the pilot

training had IT issues, either due to computers not working, not being connected to the internet or the

internet running slowly.

“Better IT access – slow/failing IT prevented/hindered our research on the toolkit” (Female,

Constable)

In addition, several participants reflected on the suitability of the training room itself, commenting about

being unable to hear or see the PowerPoint presentations on screen at the back of the room, or that

noise either within or outside of the room affected their attention. For those participants in forces in

which tea and coffee facilities were not provided within the training room, comments were made that

such refreshments would have been appreciated.

4.3.3.5 More time

The final most frequently suggested improvement to the training was that more time, particularly

focussed on the ‘Review EMMIE’ section of the training, focused on evaluation, would have been merited.

“More time. This is not a criticism just that the topic is interesting and of value to the force, so

would merit more time spending on it” (Male, Staff)

“Two days instead of one? More time to spend on case studies” (Male, Sergeant)


65

“More on how to evaluate” (Female, Sergeant)

“More focused training around writing review criteria/plans and how to evaluate” (Female,

Analyst)

However, in incorporating any changes to the length of the training programme, it is also important to recognise the comments of some participants who felt that such a long period of training in a classroom environment would require more frequent breaks.

“For practical officers, sitting for prolonged periods in classroom is almost alien – few short

breaks would have helped concentration levels” (Male, Chief Inspector)

4.3.4 Feedback on the Crime Reduction Toolkit

Much of the feedback both during the training and also in the post-training questionnaires focussed on

the CRT itself, with improvements to the training being related to or reliant on “improving the toolkit so it

is credible and applicable to practitioners “. As several learners commented along the lines of “I would

hope the comments made are used to improve the toolkit”, it was felt appropriate to report the themes

of such feedback.

4.3.4.1 It is restrictive and limited in content

The most frequent comments regarding the CRT related to concerns that the content of the tool, and in

some ways the evidence base, was limited and therefore restricted the learners’ ability to develop an

approach to the case study tasks.

“We were limited by the interventions in the toolkit rather than what we would do practically”

“We are limited by the body of research”

“Some of the research you just know through trial and error, it’s the gaps that we need filled -

where there’s a lack of evidence”

“Sometimes there seems to be insufficient evidence”

“Very good tool, ticks lots of boxes, but the information is a bit lacking”

“There’s a danger because there’s a limited amount on the CRT, the only one which will affect the

case study is to increase tax which is outside our power. But there’s a danger a senior officer will

just pick that because other stuff isn’t on the toolkit”

“Until the toolkit is expanded I do not feel I will utilise it” (Male, Sergeant)

Some of the comments related more to specific parts of knowledge which were missing:

“We do lots of these interventions, it would be nice to know costs etc. Is there something we

should not be doing? That’s not effective?”

“There’s lots of holes in the toolkit – for instance there’s no testing for mechanisms”

Implementing an EBP approach will require officers and staff to have access to research resources, and it

will be important to decide how to respond to concerns about the evidence base. One approach would

be to anticipate concerns about the size of the evidence base and thus recognise and accept them when


66

communicating EBP, emphasising the role evidence can play in informing their practice whilst

appreciating that in some areas there is a lack of systematic research. This approach highlights the role

that experience will still play in informing police practice. Another approach would be to consider

whether providing access to a wider field of research, through lowering the threshold required for

research included on the toolkit, again whilst emphasising the importance of experience and careful

consideration of the specific applied context.

4.3.4.2 Is the evidence relevant?

Some of the concerns about the restricted, limited nature of the toolkit were primarily related to whether

the evidence was relevant to policing practitioners – spatially, temporally and in terms of policing

priorities and roles.

“I think the College have a long way to go to make this product (CRT) relevant. I would want to

see more relevant studies” (Male, Sergeant)

4.3.4.2.1 US based

Participants had particular concerns about that fact that most of the systematic research presented on

the CRT was based in the US, rather than the UK.

“It should mention UK studies even if they weren’t comprehensive enough”

“Should US research be ancillary to UK research?”

The application and transferability of US research in a UK context was particularly worrying for many

participants.

“There’s very little UK based information on there, how can I know it will transfer to the UK?”

“Contextual issues – it might not work in the UK, you have to make that decision”

“There’s not enough information to tell if the implementation would work in the UK”

4.3.4.2.2 Out of date

Similarly, related the requirement for research on the CRT to be systematic, participants felt that the

research was out-of-date and may not be relevant to current operational policing. The fact that the CRT

did not report more recent research seemed to be a larger concern for the practitioners than whether

the research had been systematically reviewed in order to ensure reliability.

“If it’s not up-to-date, then what’s the point?”

“The issue I can see is what if the research is not current? That will always be a problem, how to

keep it up-to-date?” (No gender/rank recorded)

This appeared to relate primarily to participant perspectives that policing represented a constantly

evolving practice which needed to respond quickly to new and varied threats, for which a systematic

approach to research may be inadequate.

“Yes, research can become obsolete quite quickly for us”


67

4.3.4.2.3 Policing priorities

The final point with regard to the relevance of the evidence on the CRT relates to participant concerns

about whether it aligns with policing priorities. In particular, participants disclosed a level of

disappointment around the lack of evidence on topics which they perceived were directly related to their

everyday roles.

“The content of research tool often not relevant to frontline policing” (Male, Constable)

“What’s the point? Why don’t you do it on topics that we care about?”

These concerns were particularly significant in understanding whether participants could ‘buy in’ to an

EBP approach to policing, with comments revealing that perceptions that the research has not been

produced with them as an end user in mind will lead to a disengagement with the literature.

“If you think it’s useful you will buy into it, but if you see music20 interventions you’re going to

think…”

4.3.4.3 Concerns about a lack of practitioner consultation

Given their concerns with regard to the relevance of the evidence on the toolkit, many practitioners

believed that police practitioners were not consulted during its development.

“How do the consortium decide what goes on there? Do they ask police? Because a lot of topics

are not prioristised that we would like [to be prioritised] – people in universities probably have

different views to us”

“It’s borne out of the research that is there, not what we actually care about”

“You should consult practitioners more”

“The literature was not chosen in consultation with police – we don’t care about tax, we can’t do

anything about that”

“The toolkit is a great concept but currently not fit for purpose. I think it would benefit from

practitioner input” (Female, Chief Inspector)

4.3.5 Reflections on EBP

As discussed throughout the qualitative findings in this report, participants regularly identified the

viability of EBP within their force and within their specific role as a concern of implementing what they

had learned throughout the training. Additionally the findings discussed in the following section have

been replicated elsewhere (Fleming & Fyfe, 2015). The reflections in this section should be considered as

key factors when identifying and developing the wider organisational changes and support mechanisms

which will be required in order to enable participants to implement EBP.

Given the widely recognised difficulties in implementing change within policing organisations, it is

important to consider the issues which the officers and staff involved in the pilots identified when

considering the role and implementation of EBP more widely than the pilot training itself.

20 Music making interventions - http://whatworks.college.police.uk/toolkit/Pages/Intervention.aspx?InterventionID=5

http://whatworks.college.police.uk/toolkit/Pages/Intervention.aspx?InterventionID=5


68

“If the last quarter of a century has taught me anything, it has taught me the difficulty of

sustaining progress in policing ... Policing has tended, in the face of multiple challenges, to have a

short attention span and bulimic tendency towards feast (a new initiative or priority) and famine

(usually a return to a focus on crime control), which I have described elsewhere as the ‘vicious

cycle’ (Neyroud, 2007, p.128).

4.3.5.1 Experience as leading decisions

Whilst participants identified that they feel they know what works, but weren’t sure what worked the

best, a large proportion of participants suggested that police often use their experience to lead their

decisions and problem-solving. Participants revealed that they felt this would limit police willingness to

use evidence to inform their decisions. This tendency to use experience as a basis for problem-solving

was not limited to only an individual’s experience, but also to that of their colleagues.

“We need to change the culture in the police of ‘we know this works, it does work’”

“The most applicable thing to frontline policing is following locally what everyone else has

done” ”

Participants felt that the knowledge practitioners brings to any decision, through their experience, could

not be replicated or replaced by the evidence available on the CRT. Specifically, the CRT did not provide

them with the details required to implement interventions in a practical sense – e.g. costs, locations,

partners involved.

“The shared knowledge of practitioners is of more value than the evidence. We would go to

licensees and engage with them looking at opening times etc. Where does it tell me practically

what to do? It’s not helpful” ”

“The first question will be, where is the evidence? We have always done it this way. How much

will it cost? And the information is lacking here” ”

In this way, participant comments suggest an understanding of EBP as an approach which would reduce

the use of their knowledge and thus discretion in problem-solving circumstances. Therefore, there may

be benefits in ensuring communications with practitioners highlight an ‘Evidence-informed’ approach,

where the language used really emphasises the way in which research can be used to inform decisions,

alongside the experience practitioners have to build on.

“As police officers we do this all the time, I was sure they were effective, but the toolkit says so

too”)

“We chose them because we know from experience it would be easy to implement them

alongside our partners” ”

“For implementation, we want something that is familiar, won’t require too many extra bodies,

will fit in with what you’re already doing” ”

An approach which emphasises the value that experience has to offer to an EBP approach may more

easily gain police ‘buy in’.

“If you can’t convince one person, the decision-maker, the strongest and loudest voice, it isn’t

going to happen. And sometimes they rely on their experience instead”


69

However, it is important to recognise there may still be difficulties in gaining practitioner confidence in

some of the research. This will be particularly significant where the evidence contradicts what they know

through their experience. Explicit instruction for officers on how they should consolidate both evidence

and experience will need to be considered.

“We already do this, the toolkit is quite negative about it, but we’ve chosen it. If we were using

the tool, we wouldn’t be able to get funding to do it, but we know it works” ” (No gender/rank

recorded)

4.3.5.2 The difficulties related to evaluation in a policing context

One way to ensure that officer and staff experience is explicitly accounted for within an EBP approach is

to advocate the evaluation of current police practices and interventions for which there is no evidence

base. However, such an approach will bring with it its own difficulties. The participants identified various

concerns related to how evaluation may or may not be executed within a policing organisation, many of

which were related to resource limitations and, in some cases, a perceived inability.

“Who is going to do it? We have no capacity. We can use the daily data we work with. We don’t

have any pot of trained statisticians and researchers to use”

“We know that recording data is good, but we don’t have the systems to do it – we can’t afford to

buy or maintain them”

“It will be hard to identify the individual, rather than cumulative effect of combined

interventions”

Linked to concerns around having the resources to undertake evaluation of police practice, were issues

around the way evaluation programmes are currently used within police organisations (some for personal

promotional purposes for example), and the often short-term nature of such programmes.

“The reason culture will stop EBP is because they do it for promotion. No evaluation is long-term.

Make them accountable for some time after their promotion to check it doesn't collapse”

“The research isn’t shared for research sharing – it’s shared to show what they’ve achieved”

“We need to allow for failure and mistakes”

Other concerns were related more closely to what are considered as successful outcomes, and therefore what objective measurements evaluation would put in place. Again, it can be suggested that this is linked to the performance culture inherent in policing organisations and the way in which programmes acquire funding.

“Value is subjective in terms of outcomes – the money or the lives saved?”

A final, and key, point related to understanding the difficulties of implementing evaluation within policing

organisations is that participants felt there were no current organisational practices which placed value

on such processes. In fact, participants felt that there was a distinct lack of value placed on using an

understanding of outcomes from one police practice to apply it to another area, either spatially or of a

different area of policing practice.


70

“We’re satisfied if a problem has gone away, whether it was the intervention is ‘so what’ we don’t

care”

“Even if we know what works, we forget it quickly, we don’t learn from it”

“We don’t debrief, we don't record, and we don’t share. Even in our force, the north and west

don’t talk to each other”

“I think we get hung up on the uniqueness of the situation, rather than learning what could be

applied more widely”

4.3.5.3 A lack of resources

“It is not lost on us, the benefit of research, its feeling you have the capacity to resource it and

work towards long term goals”

Another commonly identified theme which has been discussed within the previous sections of this report

relates to participants feeling they would not have the resources required to implement an EBP approach.

“It is very difficult to say no to the evidence, so it provides a strong argument, but it’s time and

resources”)

“How do we ride our demand, make resources available at the right time in the right place?”

“The biggest challenge is potential and opportunity. We have a treadmill existence. There is no

time to step aside and work out more efficient ways to do it. We can barely stay up to date with

the constant changes in policy and law, let alone research”

As mentioned in section 4.3.4, individual force clarity in how they intend to uptake and use EBP and good

communication of these may improve individual confidence, commitment and perceived ability to

implement such an approach. Although not withstanding such an approach, there would need to be an

explicit allocation of resources to use for enabling EBP to really empower confidence within the work

force.

“We have a commitment from our Chiefs about EBP, but it requires time and so we can’t do it

without impacting on everyday business (for instance by taking people out for training). There

must be a threshold – ‘Ok we’re too much into research now!’” (No gender/rank recorded)

4.3.5.4 Performance and management issues

The clarity at the organisational level about commitment to an EBP approach really was a key factor in

participants’ concerns about whether they would have the discretion to use EBP approaches. Whilst a

recognition that an organisation was taking such an approach might assist in reducing these concerns,

participants still felt that a move to EBP would require some adjustment of the hierarchical nature of

policing practice.

“It’s very relevant, whether your managers will see the importance”

“There are certain levels that have to be challenged higher up – they don't want to listen. Some

people don't want to listen”


71

“It’s like they don’t understand us, us on the ground know the things that they don’t, but they

won’t listen to us. We’re just sergeants, we don’t matter”

Some of these concerns related to whether management would ‘buy in’ to EBP approaches to police

practice.

“Getting a high rank to buy in”

“We would need someone higher up the chain to sell it/get it implemented again”

Participants seemed to relate such management ‘buy in’ specifically to whether there would be timely

outcomes as a result of implementing such approaches and whether these outcomes could be translated

and understood as performance measures.

“You need to account for the political context, the pressure people are under to achieve things in

a short time”

“Some of the interventions sort the surface, not the root cause. But management want to

address the surface because it looks good. But the root will ensure it never happens again”

“The short-term political cycle means there is always a different focus, so there are no long-term

goals. For instance if we spent money now, we could save in 2 years’ time”

“Even if you make a good case, if you can’t link it to performance then you can’t get funding”

The solutions to these problems are not simple, and potentially cannot be accomplished at the

organisational level. The way in which roles and ranks are rotated within forces and performance is

measured may be able to alleviate some of the concerns raised. However political processes and

functions will have a lasting impact on the ability of individual forces to alter the way their performance is

measured.

4.3.5.5 Considerations about partnership working

It is perhaps apt to move from performance and management difficulties to participant concerns about

how partnership working will impact on their ability to implement an EBP approach. Partnership working

will also mean that some of the solutions to issues which might hinder such an approach cannot be solved

at the force organisational level, and an evidence-based approach and consolidation of performance

objectives may be needed across a wider variety of government services in order to provide police with

the autonomy required to implement EBP most effectively.

“But this is crime reduction, how does it fit with our partners?”

“When we start thinking blue-light partnerships, will the outcomes we want change?”

“Relying on stakeholders/partners – how to get the buy-in/appetite. We need to change their

mindset, they don’t like change/don't have the money/ have different priorities. They expect that

it’s a police problem or issue and not their responsibility”


72

“Responsibilities with other departments – managing expectations of partners about what is our

job and what is their job. We seem to be averse to standing our ground and saying that it is not a

police issue, it is another agency’s responsibility”

Some of the participants’ concerns related to the ability of partners to provide an evidence-based

solution to a crime problem which is outside of police responsibility and whether partners will have the

resources to implement such approaches when they have differing performance measures on which their

funding is based.

“What are the partners’ priorities/objectives/KPIs and do they match up with ours?”

“We don't ‘share’ budgets, it is who contributes what”

“Also it’s about priority, whether it is a police matter or a partner – housing, highways, social

services. We don't always have the leverage”

Other concerns were related to whether the information sharing amongst agencies which would be

required to implement some evidence-based approaches could be achieved. Therefore, participants’

concerns do suggest that a much wider evidence-based agenda across various government departments

and agencies would be required to put in place procedures and processes which would allow for joined-

up delivery of some specific evidence-based solutions to policing problems.

4.3.5.6 Organisational culture

Many of the previously addressed concerns which participants had about effectively implementing EBP,

relate to what they perceive as widely accepted police culture. To a certain extent there were worries

about a level of “organisational cynicism” as a result of the rapidly changing organisational context of

policing. It was suggested that a perceived excess of continuous transformation of the policing landscape

has led the workforce to be pessimistic about any new approaches to the practice.

“There will be a preconception that it won’t work – ‘we’ve tried this before’ ‘teaching me to suck

eggs’”

Other aspects which participants linked to broader policing culture included organisational norms around

projects and proposals being “doomed to succeed” which participants felt would contradict any attempts

at evaluation. Participants also had concerns that a lack of routine ‘institutional memory in the public

sector” might negate the possibility of using evidence and research to inform practice.

“We need to change the culture in the police of ‘we know this works, it does work’”

“We are culturally not ready to critically approach our interventions and accept that things went

wrong”

This can translate back into an inability to perceive changes in practice as a positive process, perhaps due

to the previous saturation of change within policing (Neyroud, 2009). It might be suggested that police

have perceived much change as having been thrust upon them from outside agents, with a feeling of a

lack of practitioner consultation and explicit communication of the intentions of those agents. This may

have led to practitioners feeling a sense of obligation to protect their practices where they perceived

their experience justifies it.


73

“Sometimes you just don’t agree with other people’s ideas so you just don't do it. There’s a

culture of ‘who do you think you are? Just stick to the game plan’” (No gender/rank recorded)

“Police officers are very critical of everything” (No gender/rank recorded)

This does also link to the way in which the police profession is required to be adaptive and responsive in

limited time frames. As a participant noted, “Police are ‘let’s act right away and consider it later’”. The

levels of discretion which are vital to the police profession achieving their aims also led to high levels of

personal responsibility and a requirement for an officer to be able to justify their actions in any given

situation (Stenning, 2009) in order to ensure they will have the support of their organisation. Such

extreme levels of personal responsibility and risk are not often replicated across other professions and

the organisational culture of policing may be something which is not easy to define in a way that would

allow outside agents (such as policy makers) to implement new policies and procedures more effectively.

4.3.6 Qualitative Findings Summary

Many of the themes identified during the pilot training were very similar to those identified in initial focus

groups undertaken by the research team (Fleming & Fyfe, 2015).

However, the pilot training revealed a stronger level of positivity around the idea of using research to

inform practice, having a toolkit to use and interactions with others in this way.

‘Experience’ is still a very key component of officer responses to EBP and will need to be accounted for

when trying to implement any changes in practice. The role of the trainer is to pre-empt and have an

understanding of the staff/officer beliefs.

Officers are open to evaluation of their practice, but there will need to be significant changes to

organisational practice and performance management to allow this. Police will need to be confident in

their ability to progress their career whilst practicing EBP approaches, including when evaluating

interventions which do not lead to successful outcomes.

EBP approaches will need to be communicated through ‘credible voices’. Participants were very keen to

have real operational examples of EBP, and rely heavily on their peers for credible experience.

Individual forces and the attitude of senior management will be key to the ‘uptake’ of EBP, as resourcing

and support mechanisms will need to be in place for officers to act upon the understanding of EBP they

developed through the pilot training, in order to apply it in practice.


74

5 RECOMMENDATIONS

The following sections outline, in no particular order of importance, what the findings of the pilot

training might mean for individual stakeholders in the area of EBP and EBP training. It is worth noting

that some of the recommendations might overlap across sections. Overall, findings reported here and

elsewhere (Fleming & Fyfe, 2015) suggest that the introduction of evidence-based practice and policy

in UK police organisations will require significant organisational behavioural change and Executive

level support.

The recommendations suggested here are those considered most important and substantial with

regards to the pilot training. Although no direct recommendations have been made with regard to

implications for the toolkit or for organisational resourcing, readers will also benefit from considering

the implications of the participants’ wide-ranging concerns around the CRT, resources and time.

5.1 THE COLLEGE - More clarity in the wider agenda. The findings suggest police officers’/staff concern with

regard to implementing the knowledge and skills obtained through the training relate to a

lack of understanding of how EBP will impact their role specifically. Additionally, much of the

resistance towards the CRT (used in the training), related to miscommunications about what

the tool was intended to be used for (problem-focused, rather than initiative focused, and

requiring interpretation, rather than providing the answers).

- Deliver the training over a two day period. A two day training programme would be preferable

in order to allow trainees to take in the information over a longer period. It would allow

particularly for a separate attentiveness to the topic of evaluation (Review Emmie). A focus on

evaluation is considered vital to enabling EBP due to current organisational culture (e.g.

immediacy, resources, ‘doomed to succeed’) within police forces which impacts negatively on

police officers/staffs’ ability to carry out evaluation. Such evaluation training should be basic

and concentrate initially on communicating the requirement for measures to be identified

and put in place prior to the implementation of any intervention. There should also be a focus

on encouraging officers to share results of such evaluation, regardless of the outcome

(positive or negative) through various vehicles such as POLKA/research map.

- Decentralised/devolved delivery of the training. With regard to the above points on clarity of

the wider agenda and organisational change, delivery which is facilitated by each individual

force would allow for force individualisation of information relevant to role and resources.

Importantly, it is suggested that delivery by the force itself may overcome concerns trainees

have about ‘buy-in’ from senior management, which in turn, feeds into their concerns around

risk adversity. Organisational leadership will enable clarification of how EBP be implemented

within each force, what it means for differing roles and how the organisation intends to

support such an innovative approach.

- Emphasis should be placed on the role evidence can play in informing practice. It is suggested it

is vital that the wider organisational change and professionalisation agenda communicate

that the intention is for evidence to be integrated into practice, to be drawn on alongside

other aspects of evidence including the officer’s experience. Such an approach accounts not

only for police perceptions on the importance of experiential knowledge, but also for the gaps

in the research base which mean that practice can only be evidence-informed at some points.


75

- Share this report with the participating forces. It is suggested that, given their commitment

and participation, it would be beneficial for the individual pilot forces to be given the full

evaluation report.

5.2 INDIVIDUAL FORCES - Provide your force with clarity around the wider organisational agenda. The findings reflect

that police officers/staff have concerns about whether they will have support to implement

the knowledge and skills obtained through the training in their roles. Having a clear vision for

what role EBP will play within your organisation, how it will be implemented within differing

roles and what time and resources will be available for such implementation is crucial to

encouraging uptake of the knowledge and skills shared in the training. Such clarity will be

provided through force-wide communication and visible actions from senior management. It

may also be useful to ensure that individual officers/staff are briefed by their immediate

manager about the objectives of the training prior to attending the training (Buckley & Caple,

2009).

- Deliver the training from within your force. With regard to the above points on clarity of the

wider agenda and organisational change, delivery which is facilitated by each individual force

would allow for force individualisation of information relevant to role and resources.

Importantly, it is suggested that delivery by the force itself may overcome concerns trainees

held about ‘buy-in’ from senior management, which feeds into their concerns around risk

adversity (Fleming & Fyfe, 2015). Organisational leadership will enable clarification of how

EBP be implemented within each force, what this means for differing roles and how the

organisation intends to support such innovative approaches.

- Communicate and emphasise the role evidence can play in informing practice. It is suggested it

is vital that the intention for evidence to be integrated into practice, to be drawn on alongside

other aspects of evidence including the officer’s experience is communicated to officers and

staff. Not only might this assuage resistance related to concerns officers have about the use

of discretion and judgement, but it will also place value on the complex decisions that they

make in their day-to-day work. Additionally, much of the resistance towards the CRT (used in

the pilot training), related to miscommunications about what the tool was intended to be

used for a solution, to provide the answers.

5.3 POLICE TRAINERS Whilst a trainer guide has been developed (Fleming, Fyfe & Wingrove, 2016b), the identification of

appropriate trainers will be important. Findings suggest that participants appreciated the academic

expertise of the trainers in this area but also sought the operational expertise of a police officer. The

credibility of the trainer (e.g. Petty & Cacioppo, 1986), will be important to trainee acceptance of the

subject matter and intentions to implement it in practice.

- The training should be delivered in an interactive manner. Many of the perceived benefits of

the pilot training identified by the participants were related to their ability to interact with the

toolkit, case study tasks and their colleagues. This aligns with the findings of Fleming & Fyfe

(2015) that police were cynical about learning being achieved through other modalities such

as e-learning.

- Ensure the facilities for the training day are functional and well-designed. Whilst this might

appear to be a given, this was a key and re-occurring theme throughout seven days of training

across four different forces. The room impacts on the success of the training tasks and on the


76

trainees’ abilities to engage with the training. The room must have working computers with

internet access, which should be checked in advance. A lay out which enables the trainees to

engage with the trainer and any slides being projected at the front is important – therefore it

is suggested a square room, with rows of computers pointing towards the front may be most

effective.

- Deliver the training on evaluation (‘Review Emmie’) on a separate day. With regard to the

recommendation to the College above that the training be split over two days, it is suggested

that the topic of evaluation be the focus of the second day. This would allow such a

demanding topic to be considered from the most basic of starting points, communicating the

requirement for measures to be identified and put in place prior to the implementation of

any intervention. There should also be a focus on encouraging officers to share results of such

evaluation, regardless of the outcome (positive or negative) through various vehicles such as

POLKA/research map. A focus on evaluation is considered vital to enabling EBP due to current

organisational culture (e.g. immediacy, resources, ‘doomed to succeed’) within police forces

which impact negatively on police officers/staffs’ ability to carry out evaluation.

- Include real, operational examples of EBP. Preferably such examples should be local and

should be delivered or co-delivered by a credible source – for example a police officer who

implemented an EBP approach.

- Be aware of common trainee concerns. The trainer should familiarise themselves with the

likely concerns and worries trainees may have about implementing such an approach (e.g.

with regards to the lack of UK research). Being able to pre-empt and address such concerns if

they are bought up will improve trainer credibility and therefore may increase the success of

the training.

- Be aware of and promote common trainee-identified benefits of EBP. Whilst being aware of

the common concerns and criticisms of EBP will impact on trainer credibility, so will being

aware of what trainees perceive as the benefits. Most police organisations are engaged in

public sector partnerships, and several participants in the pilot training noted that being able

to use the CRT and reference evidence during meetings with such partners would be valuable

for them. The trainer may wish to consider promoting such positive uses and purposes,

translating the theory of EBP to a more applied situation.

5.4 WIDER ACADEMIA - It is suggested that academia may have a very specific local role to play in further building on

the current training, developing or making recommendations with regard to training police

officers and staff in applied evaluation skills. Whilst there are certain levels of research

knowledge which are appropriate for the scientific graduate or doctoral student, the pilot

training reveals the need for a lower level of knowledge which must be communicated to

officers and staff prior to building up to such understanding. What is an appropriate, viable,

timeline within which to expect individuals without a background in scientific research to

build up such knowledge must be considered.

- The pilot training evaluation has identified a number of potential areas for further research.

The Police Knowledge Fund21 and similar funding opportunities might provide a vehicle for

academia to work with the College on such research:

21 http://www.college.police.uk/News/College-news/Pages/Police-Knowledge-Fund.aspx

http://www.college.police.uk/News/College-news/Pages/Police-Knowledge-Fund.aspx


77

o Further research is needed to evaluate longitudinal outcomes of any implementation

of EBP training. Opportunities to work with the College and individual police forces

should be considered if the training is rolled out nationally.

o In addition to the above point, it will be important to understand how the training is

used and implemented in a particular force or context. This understanding will

provide a more nuanced insight into police organisational change and police training,

adding to the limited literature in this area.

o The impact of any wider roll out of the training on progress towards

professionalisation of policing should be considered a substantively important

consideration of future research. To contribute to police understanding and

literature, but also to the much wider government agendas in this area across various

professions.

- In order to address some of the participants’ suggested improvements to the training,

academia may have a role in expanding the literature in areas most relevant to operational

policing. Specifically, as discussed by Johnson, Tilley and Bowers (2015), the EMMIE

framework can inform future primary studies as well as continued systematic reviews, and

may improve police abilities to use such research in practice.


78

6 REFERENCES

About Us. (n.d.). In Police Scotland. Retrieved February 17, 2016, from

http://www.scotland.police.uk/about-us/

Axiom (2015). SPA/Police Scotland Opinion Survey 2015. Retrieved from

http://www.scotland.police.uk/assets/pdf/138327/307421/spa-police-scotland-opinion-survey-

2015?view=Standard

Barbazette, J. (2006). Training Needs Assessment: Methods, Tools, and Techniques. San Francisco:

Pfeiffer

BBC News. (2014). Hampshire Constabulary to cut 535 Posts. Retrieved from


BBC News. (2015). Sir Stephen House to quit as Police Scotland Chief Constable. Retrieved from

http://www.bbc.co.uk/news/uk-scotland-34072703

Bradburn, N., Sudman, S., & Wansink, B. (2004). Asking Questions: The Definitive Guide to

Questionnaire Design. San Francisco: Wiley & Sons.

Brain, T., & Owens, L. (2015). Leading in austerity. In J.Fleming (Ed.), Police Leadership: Rising to the

top (pp.17-41). Oxford: Oxford University Press.

Buckley, R., & Caple, J. (2009). The Theory and Practice of Training. London, UK: Kogan Page.

Cabinet Office. (2011). Open Public Services White Paper. Retrieved from

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/255288/OpenPublic

Services-WhitePaper.pdf

Clare, R. (2015). National Policing Curriculum: Curriculum Development Guidance Pack. College of

Policing Limited.

College of Policing. (2014a). Five-Year Strategy. Retrieved from

http://www.college.police.uk/About/Documents/Five-Year_Strategy.pdf

College of Policing. (2014b) Code of Ethics: A code of practice for the principles and standards of

professional behaviour for the policing profession of England and Wales. Retrieved from

http://www.college.police.uk/What-we-do/Ethics/Documents/Code_of_Ethics.pdf

College of Policing. (2015a). College of Policing Analysis: Estimating demand on the police service.

Retrieved from http://www.college.police.uk/Documents/Demand_Report_21_1_15.pdf

College of Policing. (2015b). Leadership Review: Recommendations for delivering leadership at all

levels. Retrieved from http://www.college.police.uk/What-we-do/Development/Promotion/the-

leadership-review/Documents/Leadership_Review_Final_June-2015.pdf

Corbin, J.M, & Strauss, A.L. (2015). Basics of Qualitative Research: Techniques and Procedures for

Developing Grounded Theory (4th ed.). Thousand Oaks: Sage Publications.

Dolan, P., Hallsworth, M., Halpern, D., King, D., & Vlaev, I. (2010). MINDSPACE: Influencing behaviour

through public policy. Retrieved from

http://www.instituteforgovernment.org.uk/sites/default/files/publications/MINDSPACE.pdf

http://www.scotland.police.uk/about-us/

http://www.scotland.police.uk/assets/pdf/138327/307421/spa-police-scotland-opinion-survey-2015?view=Standard

http://www.scotland.police.uk/assets/pdf/138327/307421/spa-police-scotland-opinion-survey-2015?view=Standard


http://www.bbc.co.uk/news/uk-scotland-34072703

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/255288/OpenPublicServices-WhitePaper.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/255288/OpenPublicServices-WhitePaper.pdf

http://www.college.police.uk/About/Documents/Five-Year_Strategy.pdf

http://www.college.police.uk/What-we-do/Ethics/Documents/Code_of_Ethics.pdf

http://www.college.police.uk/Documents/Demand_Report_21_1_15.pdf

http://www.college.police.uk/What-we-do/Development/Promotion/the-leadership-review/Documents/Leadership_Review_Final_June-2015.pdf

http://www.college.police.uk/What-we-do/Development/Promotion/the-leadership-review/Documents/Leadership_Review_Final_June-2015.pdf

http://www.instituteforgovernment.org.uk/sites/default/files/publications/MINDSPACE.pdf


79

Field, A. (2013). Discovering Statistics using SPSS. London: Sage.

Fleming, J., & Fyfe, N. (2015). ‘We can read and write you know!’ Selling the idea of research. Focus

Groups for the What Works Centre for Crime Reduction (WP 6/7) - Year 1 (2013).

Fleming, J., Fyfe, N., & Wingrove, J. (2016a). Evidence-informed Policing: The Design of a Pilot Training

Programme - available from the College of Policing.

Fleming, J., Fyfe, N., & Wingrove, J. (2016b). Evidence-informed Policing: Trainer Guide - available

from the College of Policing.

Fyfe, N., Fleming, J. & Wingrove, J. (2015). Module Outline, Evidence-informed Decision Making in

Policing: An introduction to EMMIE and the Crime Reduction Toolkit - available from the College of

Policing.

Glaser, B. (1978). Theoretical Sensitivity. Mill Valley, CA: Sociology Press.

Glaser, B. (1992). Basics of Grounded Theory Analysis: Emergence versus Forcing. Mill Valley, CA:

Sociology Press.

Glaser, B., & Strauss, A. (1967). Discovery of Grounded Theory. Chicago: Aldine.

Hamblin, A.C. (1974). Evaluation and control of training. Industrial Training International, 9, 154-156.

Hampshire Police & Crime Commissioner. (2013). Police and Crime Plan 2013-2017. Retrieved from

http://www.hampshire-pcc.gov.uk/Document-Library/Police-and-Crime-Plan---full.pdf

HM Government. (2012). The Civil Service Reform Plan. Retrieved from

https://www.gov.uk/government/publications/civil-service-reform-plan

Home Office. (2015). Police Workforce, England and Wales: 31 March 2015. Retrieved from

https://www.gov.uk/government/publications/police-workforce-england-and-wales-31-march-

2015/police-workforce-england-and-wales-31-march-2015

Hunter, G., Wigzell, A., May, T., & McSweeny, T. (2010). An Evaluation of the ‘What Works Centre for

Crime Reduction’ Year 1: Baseline.

Jarvis, P. (2010). Adult Education and Lifelong Learning, Theory and Practice. London: Routledge.

John, R. (2015). Right Here, Right Now: The New-look Force goes Live, Frontline,187. Retrieved from

https://issuu.com/hampshireconstabulary/docs/frontline_187_-

_final_web_issuu?e=14091403/12261718

Johnson, S.D, Tilley, N., & Bowers, K.J. (2015). Introducing EMMIE: an evidence rating scale to

encourage mixed-method crime prevention synthesis reviews. Journal of Experimental Criminology,

11, 459-473.

Kirkpatrick, D.L., & and Kirkpatrick, J.L. (2006). Evaluating Training Programs (3rd Ed.). San Francisco,

CA: Berrett-Koehler Publishers.

McCambridge, J., Kypri, K., & Elbourne, D. (2014). Research participation effects: a skeleton in the

methodological cupboard. Journal of Clinical Epidemiology, 67, 845-849.

Mulcahy, A., & Orde, H. (2015). Police leadership in fractured societies. . In J.Fleming (Ed.), Police

Leadership: Rising to the top (pp.169-189). Oxford: Oxford University Press.

http://www.hampshire-pcc.gov.uk/Document-Library/Police-and-Crime-Plan---full.pdf

https://www.gov.uk/government/publications/civil-service-reform-plan

https://www.gov.uk/government/publications/police-workforce-england-and-wales-31-march-2015/police-workforce-england-and-wales-31-march-2015

https://www.gov.uk/government/publications/police-workforce-england-and-wales-31-march-2015/police-workforce-england-and-wales-31-march-2015

https://issuu.com/hampshireconstabulary/docs/frontline_187_-_final_web_issuu?e=14091403/12261718

https://issuu.com/hampshireconstabulary/docs/frontline_187_-_final_web_issuu?e=14091403/12261718


80

Mulgan, G., &, Puttick, R. (2013) Making evidence useful: the case for new institutions. London: Nesta.

Retrieved from http://www.nesta.org.uk/sites/default/files/making_evidence_useful.pdf

National Audit Office. (2013). Evaluation in government. London: National Audit Office. Retrieved

from https://www.nao.org.uk/wp-content/uploads/2013/12/10331-001-Evaluation-in-

government_NEW.pdf

Neyroud, P. (2007). Community policing. Policing, 1, 127-129.

Neyroud, P. (2009). Squaring the circles: research, evidence, policy-making, and police improvement

in England and Wales. Police Practice and Research, 10, 437-449.

The Northern Ireland Policing Board and The Police Service of Northern Ireland. (2014). Policing Plan

2014-2017. Retrieved from http://www.psni.police.uk/final_pdf_-_policing_plan_2014_-_2017.pdf

Northern Ireland Statistics & Research Agency. (2015). Population and Migration Estimates Northern

Ireland (2014). Retrieved from

http://www.nisra.gov.uk/archive/demography/population/midyear/MYE14_Bulletin.pdf

Patterson, C. (2011). Adding value? A review of the international literature on the role of higher

education in police training and education. Police Practice and Research, 12, 286-297.

Pearce, D. (2015). Better Standards: 'Are inconsistencies undermining police professionalism?’.

Presented at College of Policing Annual Conference November 2015: For better Policing.

Petty, R.E., & Cacioppo, J.T. (1986). The Elaboration Likelihood Model of Persuasion. In L. Berkowitz

(Ed.), Advances in experimental social psychology (vol. 19, pp. 123-205). San Diego, CA: Academic

Press.

Philips, J., & Stone, R. (2002). How to Measure Training Results: A Practical Guide to Tracking the Six

Key Indicators. USA: McGraw-Hill Education.

Police Scotland (2015a). Police Scotland Officer Numbers Quarterly Fact Sheets – Quarter 2 30th

September 2015. Retrieved from http://www.scotland.police.uk/assets/pdf/138327/212520/police-

scotland-officer-numbers-quarter-2-30th-september-2015?view=Standard

Police Scotland (2015b). Police Scotland Equality and Diversity Mainstreaming Progress Report.

Retrieved from http://www.scotland.police.uk/assets/pdf/138327/243045/equality-and-diversity-

mainstreaming-progress-report-april-2015?view=Standard

Police Service of Northern Ireland. (2014). Annual Report and Accounts: For the year ended 31 March

2014. Retrieved from http://www.psni.police.uk/main_account_2014.pdf

Scottish Government. (2011). Police Reform Programme: Outline Business Case. Retrieved from

http://www.gov.scot/Resource/Doc/357534/0120783.pdf

Silver, C., & Lewins, A. (2014). Using Software in Qualitative Research: A step-by-step guide. London:

Sage.

Spradley, J.P. (1980). Participant Observation. New York: Holt, Rinehart & Winston.

Stanislas, P. International Perspectives on Police Education and Training. New York: Routledge.

Stenning, P.C. (2009). Discretion. . In A.Wakefield & J.Fleming (Eds.), The Sage Dictionary of Policing

(pp. 85-87). London: Sage Publications.

http://www.nesta.org.uk/sites/default/files/making_evidence_useful.pdf

https://www.nao.org.uk/wp-content/uploads/2013/12/10331-001-Evaluation-in-government_NEW.pdf

https://www.nao.org.uk/wp-content/uploads/2013/12/10331-001-Evaluation-in-government_NEW.pdf

http://www.psni.police.uk/final_pdf_-_policing_plan_2014_-_2017.pdf

http://www.nisra.gov.uk/archive/demography/population/midyear/MYE14_Bulletin.pdf

http://www.scotland.police.uk/assets/pdf/138327/212520/police-scotland-officer-numbers-quarter-2-30th-september-2015?view=Standard

http://www.scotland.police.uk/assets/pdf/138327/212520/police-scotland-officer-numbers-quarter-2-30th-september-2015?view=Standard

http://www.scotland.police.uk/assets/pdf/138327/243045/equality-and-diversity-mainstreaming-progress-report-april-2015?view=Standard

http://www.scotland.police.uk/assets/pdf/138327/243045/equality-and-diversity-mainstreaming-progress-report-april-2015?view=Standard

http://www.psni.police.uk/main_account_2014.pdf

http://www.gov.scot/Resource/Doc/357534/0120783.pdf


81

Strauss, A. (1987). Qualitative Analysis for Social Scientists. Cambridge, UK: University of Cambridge P

ress.

Strauss, A.L, & Corbin, J.M. (1998). Basics of Qualitative Research: Techniques and Procedures for

Developing Grounded Theory (2nd ed.). Thousand Oaks: Sage Publications.

Sussex Police. (2015). Sussex Police in 2020: Working for a safer Sussex. Retrieved from

http://www.sussex.police.uk/media/912959/tom_pdf_complete_final_online.pdf

Sussex Police & Crime Commissioner. (2013). Safer in Sussex: Police & Crime Plan 2013/2017.

Retrieved from http://www.sussex-pcc.gov.uk/wp-content/uploads/2015/03/SPCC-Police-and-Crime-

Plan-2015.pdf

UK Administrative Data Research Network. (2012). Improving Access for Research and Policy Report

from the Administrative Data Taskforce – December 2012. London: UK Statistics Authority. Retreived

from https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-

administrativedatataskforcereportdecember201_tcm97-43887.pdf

Weiss, C.H. (1980). Knowledge creep and decision accretion. Science Communication, 1, 381-404.

Werth, E.P. (2009). Student perception of learning through a problem-based learning exercise: An

exploratory study. Policing: An International Journal of Police Strategies & Management, 32, 21-37.

Who we are. (n.d). In Sussex Police. Retrieved February 15, 2016, from

http://www.sussex.police.uk/policing-in-sussex/the-organisation/who-we-are

http://www.sussex.police.uk/media/912959/tom_pdf_complete_final_online.pdf

http://www.sussex-pcc.gov.uk/wp-content/uploads/2015/03/SPCC-Police-and-Crime-Plan-2015.pdf

http://www.sussex-pcc.gov.uk/wp-content/uploads/2015/03/SPCC-Police-and-Crime-Plan-2015.pdf

https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-administrativedatataskforcereportdecember201_tcm97-43887.pdf

https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-administrativedatataskforcereportdecember201_tcm97-43887.pdf

http://www.sussex.police.uk/policing-in-sussex/the-organisation/who-we-are


82

7 APPENDICES

7.1 APPENDIX A: PRE-PILOT TRAINING QUESTIONNAIRE

‘Evidence-informed Policing’ Pilot Training Programme Pre-

course Questionnaire

The ‘What Works Centre for Crime Reduction’ was established to develop a strong

research evidence base for decision-making around crime reduction.

A key goal of the Centre is to develop a training programme for police officers and staff

which helps you to consider research and use it to inform your decision-making when

addressing local problems of crime and disorder.

We are conducting research on the value and effectiveness of that training programme. It is

important that we understand your knowledge, attitudes and perceptions both before and

after today’s pilot training to see how effective the programme is. This will inform further

development of the programme prior to it being incorporated in to the National Policing

Curriculum.

Your responses will be anonymous. Please tick the response most relevant for you. The

following demographic information will enable us to understand how effective the training

was for different ranks, roles and levels of experience. It is not intended to allow us to

identify you.

DEMOGRAPHICS

Rank:

Constable Sergeant Inspector

Chief Inspector Superintendent Chief

Superintendent

Other, please state:

……………………………………………………………………………………

Please provide a brief description of your role/specialism:

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

Years in Service:

0 – 5 6 - 15 15+

Gender:

Female Male

Age:

20 - 25 26 - 40 41+


83

Level of Education (Tick all that apply):

A levels or equivalent HNC/HND Bachelor of

Science/Art

Postgraduate Certificate/Diploma Masters PhD

Other, please state:

……………………………………………………………………………………

BACKGROUND

1) Have you heard about Evidence-based Policing before now?

Yes | No

2) Did you participate in the previous focus groups held by Professors Fleming and Fyfe?

Yes | No | Not sure

3) Have you heard of any of the following College of Policing activities? Please tick all those which

you have heard of.

Evidence Base Camp

Evidence-based Master Classes

Research Surgeries

Evidence-based Policing components of College training programmes

4) Have you participated in any of the following College of Policing activities? Please tick all those

which you have participated in.

Evidence Base Camp

Evidence-based Master Classes

Research Surgeries

Evidence-based Policing component of the Senior Leadership Programme

Evidence-based component of the Strategic Command Course

Not sure

5) Have you participated in any other Evidence-based Policing activities before now (e.g. research

conference/workshop)?

Yes | No

If yes, please state what activity this was:

……………………………………………………………….................................................................

………………………………………………………………………………………………………………

6) Have you heard of any of the following before now? Please tick all those which you have heard

of.

What Works Centre for Crime Reduction

The Crime Reduction Toolkit


84

The College of Policing Research Map

POLKA

The National Police Library

7) Have you used any of the following before now? Please tick all those which you have used.

Research (journals, books, reference material)

The College of Policing website

The College of Policing Research Map

The Crime Reduction Toolkit

POLKA

The National Police Library

Not sure

8) Are you aware of whether your force has a Frontline Champion?

Yes | No

9) Have you ever (either by yourself or with a team) undertaken a piece of research with a

university before now?

Yes | No

10) Have you ever formally assessed a crime intervention in terms of its effectiveness?

Yes | No

11) How did you become enrolled on this pilot training programme?

I volunteered to attend | I was instructed to attend

In the table below, please tick the box which demonstrates how much you agree with the

statements on the left. Please tick ‘don’t know’ if you are unsure.

Q Statement

Strongly Disagree

Disagree Agree Strongly Agree

Don’t Know

The Pilot Programme

12 I believe the pilot programme will be relevant to my job.

13 I believe the pilot programme will be relevant to my force.

14 I believe the skills learnt during this course will be valued by my force.


15 Research can be useful for my role.

16 Research is important for my role.

17 Research can be used to inform my day-to-day work.


85

Thank you very much for taking the time to complete this questionnaire.

Jenny Fleming and Nick Fyfe

18 I am able to access research materials.

19 I am able to identify appropriate research materials.

20 I am able to interpret research.

21 I am able to assist others in using research to make decisions.

Universities

22 Working with universities can be useful for my job.

Review

23 Assessing the effectiveness of crime reduction interventions is important for my role.

24 I believe I have the knowledge and skills needed to assess the effectiveness of a crime reduction intervention.

25 I know where I can get assistance to assess the effectiveness of crime reduction interventions.


86

7.2 APPENDIX B: POST-PILOT TRAINING QUESTIONNAIRE

‘Evidence-informed Policing’ Pilot Training Programme Post-

course Questionnaire

This training was designed to help you consider research and potentially use it to inform your

decision-making when addressing local problems of crime and disorder.

It is important that we understand your perceptions of the content and value of the training in order

to see how effective the training materials were. This will inform further development of the

programme prior to it being incorporated into the National Policing Curriculum.

Your responses will be anonymous. In the table below, please tick the box which demonstrates how

much you agree with the statements on the left. Please tick ‘don’t know’ if you are unsure.

Once you have completed the questionnaire please attach it to your pre-course questionnaire and

hand both to a member of the research team.

Q Statement

Strongly Disagree

Disagree Agree Strongly Agree

Don’t know

Training Structure

1 The structure/order of the training day was appropriate.

2 The pace of the training day was appropriate.

Training Methods

3 The visual aids (e.g. PowerPoint, handouts) were effective in helping me to learn.

4 The Case Study task was effective in helping me to learn.

5 The group discussions were effective in helping me to learn.

6 The instructor was effective in helping me to learn.

Facilities

7 The facilities (e.g. the room, the computer access) were satisfactory.

Training Content

8 The training was relevant to my role.

9 Evidence-informed policing is relevant to my role.

10 The training content was interesting to me.


87

Please identify one component of the training which you felt was most useful in meeting your

learning goals today:

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………..

Please identify one component of the training which you think was less useful in meeting

your learning goals today:

11 I would like to apply the knowledge/techniques from the training in my role.

12 The skills I learnt during this course will be valued by my force.


13 Research can be useful for my role.

14 Research is important for my role.

15 Research can be used to inform my day-to-day work.

16 As a result of this training, I am more able to access research materials.

17 As a result of this training, I am more able to identify appropriate research materials.

18 As a result of this training, I am more able to interpret research.

19 As a result of this training, I am more able to assist others in using research to make decisions.

Universities

20 Working with universities can be useful for my role.

Evaluation/Review

21 Formally assessing the effectiveness of crime reduction interventions is important for my role.

22 As a result of this training, I am more able to assess the effectiveness of a crime reduction intervention.

23 As a result of this training, I am more likely to get assistance to assess the effectiveness of crime reduction interventions.


88

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………..

Please identify something you think would improve the training:

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

…………………………….

Please identify one component of the training which you feel you will take back to your

workplace and use:

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………..

Please identify one component of the training which you do not feel you will take back to

your workplace and use:

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

…………………………….

Anything else you would like to say about your experience today?

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

……………………………………………………………………………………………………………

…………………………..

Thank you very much for taking the time to complete this questionnaire.

Jenny Fleming and Nick Fyfe


89

7.3 APPENDIX C: OBSERVATION TOPIC LIST

Observation Topic List

This is to be used by a researcher who is not acting as a trainer to observe each session. The list

provides prompts of aspects which might be valuable to note, in order to evaluate trainee reaction

and learning. It should be noted that comments during coffee breaks, as well as questions asked by

participants etc. may provide an indication of reaction and learning. A reflective session between all

researchers involved immediately following the training day would be beneficial to identify

researcher perceptions of trainee reaction/knowledge. Such a reflective session could also use the

following prompt list.

Reaction

- How do the participants appear to feel about attending the training? Do they seem keen to

be there or not?

- Do the participants appear distracted, for instance using their mobile phones?

- How often do participants leave the training to use their mobile phone?

- Do the participants seem to be happy with the structure and pace of the training?

- Do the participants make any remarks or particular responses to the training methods – e.g.

visual aids, presentations, and case study tasks?

- How do the participants appear to respond to the trainers?

- How do the participants appear to feel about the facilities and physical environment?

- Do the participants know each other?

- How do the participants appear to respond to the social environment? For instance is there

any tendency to defer to a superior colleague?

- Do the participants seem to be interested in the content?

- How do participants seem to feel about the general value of EBP?

- How do participants seem to feel about the value of the training session?

- F How do participants seem to feel about the relevance of EBP/ training to them as individuals

or as a group?

- What aspects of the training do they appear to enjoy or be immersed in?

- What aspects of the training do they appear to dislike or be distracted from?

- How do participants seem to feel about EMMIE evaluation scale?

- How do participants seem to feel about the Crime Reduction Toolkit?

- Do they make remarks which indicate they feel the case study task is a useful learning aid?

- What aspects of the case study task do they think are realistic?

- What aspects of the case study task do they think are unrealistic?

Knowledge

- What do they already seem to have good knowledge about?

- What do they seem to have less knowledge about?

- Are there any concepts they seem to find easier or more difficult to understand?

- Do there appear to be any increases in knowledge throughout the training?


90

- How do they respond to trainer knowledge questions throughout the course (e.g., “now can

someone tell me what definition of evidence-informed policing we are using?”?)

- Do there appear to be any attitude changes during the training (e.g. people who were openly

critical of the value of EBP, who remark that they can see how it might be useful now or visa

versa)

- How do they get on in the case study task?

- What parts of the case study task do they seem to find more difficult?

- What aspects of the case study task do they seem to find easy?


91

7.4 APPENDIX D: FORCE INFORMATION SHEET

The ‘What Works Centre for Crime Reduction’, funded by the Economic and Social Research Council

and the College of Policing, was established to develop a strong research evidence base for decision-

making around crime reduction. A key component of the Centre is the development and piloting of a

training programme which will enable police officers to appraise evidence and use evidence to inform

their decision-making when addressing local problems of crime and disorder. This document provides

information to forces being asked to take part in the pilots.

The Centre has developed a Crime Reduction Toolkit available online which rates and ranks crime

reduction interventions in terms of impact, cost-savings, how they work, where they work and how

best to implement them locally (http://whatworks.college.police.uk/toolkit/Pages/Toolkit.aspx). The

pilot training programme will incorporate the Crime Reduction Toolkit which provides access to

research findings in various areas such as, for example, knife crime, domestic violence and drink

driving.

The College is supporting this work and in due course the training programme will be added to the

National Policing Curriculum.

What are the benefits for your force?

In 2014 we held focus groups and interviews with your officers to gain insight into:

General understandings of what constitutes research

Attitudes towards using research to inform policing practice

How research is currently used to inform police decision-making

The perceived challenges and barriers to using research

What might be useful in a training programme

The findings of these focus groups, and the views expressed by those who participated, have

informed the design of the pilot training programme. As your force agreed to host focus groups for

the project last year, it was agreed that you would have the opportunity to be one of the first

recipients of the pilot training programme. It is anticipated that the training programme will benefit

and be of value to your organisation generally and your officers specifically.

At the end of each pilot session, we will be conducting an evaluation to assess officer attitudes and

perceptions of the content and utility of the session. The findings will be communicated in a final

report which you will have access to, and will inform further development of the programme prior to

it being incorporated in to the National Policing Curriculum.

http://whatworks.college.police.uk/toolkit/Pages/Toolkit.aspx


92

Learning Outcomes

The overall aim of this programme to improve the ability of officers to tackle local problems of crime and

disorder through an introduction to Evidence-informed Policing, providing them with the skills required to

make decisions based on all of the available evidence. This highlights the role research can play in

informing police decision-making.

On completion of this programme, participants will be able to:

Explain the nature and value of evidence-informed approaches to policing

Understand the origins and development of the Crime Reduction Toolkit

Use the Crime Reduction Toolkit to help inform decision-making in relation to tackling a local

crime problem

Assess the use of the Crime Reduction Toolkit after employing it locally

Use their understanding to appraise evidence and commission research

When will the pilot be held?

We intend to complete the pilots in September and October 2015. The course will take 6 hours (e.g. 10:00 – 16:00) which allows for breaks, networking and discussion. Specific timings and dates can be arranged to suit your force requirements.

Where will the pilot be held?

We would like to hold the pilot within your force area. We will require a room and those participating will need access to a computer (one computer per two participants will be sufficient). This will facilitate applied learning through a scenario/case example using the Crime Reduction Toolkit online.

Who would we like to take part?

We suggest inspectors, sergeants and analysts with a responsibility for problem solving within your force.

You may identify other roles and ranks which would benefit from attending, and we welcome this. Those

who attended the previous focus groups are also welcome to attend the pilot.

How many officers/staff would we like to take part?

We are hoping to have 25 – 30 officers and staff from each force participate in the pilot training course.

Whilst it would be preferable to complete the pilot in your force within one day, we can hold it over two

days if resources and availability require this flexibility.


93

Who can you contact for more information?

Professor Jenny Fleming Professor Nicholas Fyfe 077 9226 7426 078 6698 0088


94

7.5 APPENDIX E: PARTICIPANT INFORMATION SHEET & CONSENT FORM The ‘What Works Centre for Crime Reduction’ was established to develop a strong research evidence

base for decision-making around crime reduction. It is funded by the Economic and Social Research

Council and the College of Policing.

A key component of the Centre is the development and piloting of this training, to enable police officers

and staff to consider evidence and use evidence to inform their decision-making when addressing local

problems of crime and disorder. The University of Southampton and The University of Dundee are

addressing this component through the training you are participating in today.

We are conducting research on the value and effectiveness of the training. It is important that we

understand your knowledge, attitudes and perceptions both before and after today’s trial training

programme to see how effective we were in achieving your learning outcomes. This will inform further

development of the programme prior to it being incorporated into the National Policing Curriculum.

What are the benefits for your force?

In 2014 we held focus groups and interviews with officers in your force to gain insight into:

General understandings of what research is

Attitudes towards using research to inform policing practice

How research is currently used to inform police decision-making

The perceived challenges and barriers to using research

What might be useful in a training programme

The findings of these focus groups, and the views expressed by those who participated have informed the

design of the pilot training programme. It is anticipated that the training programme will benefit and be

of value to you, your colleagues and your organization.

What are we asking you to do?

We are asking you to take part in the training today and, in order for us to evaluate the training in terms

of value and effectiveness we shall be asking you to complete a questionnaire, prior to the training and

after the completion of the course. Your name will not be recorded - your responses to these

questionnaires will be anonymous. We will also be observing the training to understand your reaction to

the information provided, the level of existing knowledge and general understanding – this will also be

anonymized when it is recorded. The data we collect will be coded and stored on a password protected

computer in compliance with the Data Protection Act and University Policy.

What is the training about?

The overall aim of the training is to improve the ability of police officers and staff to tackle local problems

of crime and disorder through an introduction to evidence-informed policing. This will provide you with

the skills required to make decisions based on all of the available evidence, highlighting the role research

can play in informing police decision-making.

On completion of this programme, it is hoped participants will be able to:


95

Explain the nature and value of evidence-informed approaches to policing

Understand the origins and development of the Crime Reduction Toolkit

Use the Crime Reduction Toolkit to help inform decision-making in relation to tackling a local

crime problem

Assess the use of the Crime Reduction Toolkit after employing it locally

Use their understanding to appraise research

Who can you contact for more information?

Please speak to a member of the research team if you have any questions, prior to providing us with your

consent to participate below. If you are interested in finding out any more about this research, or

evidence-informed policing please contact:

Professor Jenny Fleming Professor Nicholas Fyfe

[email protected] [email protected]

If you would like to speak to an independent person not involved in the research, about the training or

the trainers, or any other issue, please contact the Head of Research, Governance Office at the University

of Southampton on 02380 595058 or Email: [email protected].

Consent

Please read the following statement and, if you agree, sign below.

I confirm that I have read and understand the information sheet for this research.

I have had the opportunity to consider the information, ask questions and have had these

answered satisfactorily.

I understand that my data will be treated confidentially and any publication resulting from this

work will comprise aggregated data and will not identify me.

I freely agree to participate in this study.

Signature: ………………………………………. Date:…………………………………………..

mailto:[email protected]

mailto:[email protected]


96

7.6 APPENDIX F: TRAINING SLIDES


97


98


99


100


101