using data mining techniques to improve efficiency in police intelligence
DESCRIPTION
This is presentation that was given by Dr Rick Adderley at the UK KDD Symposium on 29th September 2011.TRANSCRIPT
Using Data Mining Techniques to Improve Efficiency in Police
Intelligence
Dr Rick Adderley
Recent Riots
• London
Recent Riots
• Birmingham
Recent Riots
• Manchester
Recent Riots – Policing Cuts
The chair of South Wales Police Federation has warned if "savage" cuts go ahead police will not be able to respond effectively to future riots.
The Mayor of London Boris Johnson has warned the government against cutting police numbers. Mr Johnson said the case for cuts had been "substantially weakened" by the riots and that he opposed the Home Secretary's plans to reduce forces' budgets.
Policing Cuts
Views wanted on £134m police cuts in Greater Manchester
Home secretary defends cuts to policing budget
Crime concerns over Surrey Police cutsClaims that crime will rise in Surrey because of cuts to police funding have been rebutted by the force.
Welsh police voice fears over budget cuts
South Yorkshire Police chief warns of crime rise
• How can the provision of intelligence be strengthened and improved in the light of severe cuts?
Providing Operational Intelligence
• Retired Police Inspector– FLINTS
• Business Intelligence Company– 2003– Policing and Security– EU Research Projects
• PhD – Offender Profiling and Crime Trend Analysis
Introduction
• Validated Examples1. Interview Lists2. Early Detection of Crime Series3. Modelling Forensic Recovery4. Automatic Identification of Priority & Prolific
Offenders
Introduction
• IBM SPSS Modeller (Clementine)• SAS Enterprise Miner• Insightful Miner…or…
Data Mining Tools in PolicingAll of the Validated Examples can be accomplished by using:
Data Mining Tools in Policing
• To automatically interrogate one or more data sets with a view to providing information that will save time, reduce crime, deter offending and enhance dynamic business processes.
Data Mining
• Offenders Learn from Offenders
Additional Complication
• Problem Outline– Analysts Time Constraints– Examine Index Crime & Compare with:• Keyword Type Search• Personal Memory
– Produce a List “Matching” Index Crime• 1 to 2 Hours• 10% to 15% Accurate
– Interviewing Officers Refine List
1. Interview Lists
• MLP– Training Set– Testing Set
1. Interview Lists
1. Interview Lists
Billy Smith
Crime
BCU
Beat
PostCode
• Results of Modelling– MLP– Takes Into Account Whole Range of Criminality– Improved Accuracy• 75% to 85%
– Independent Validation• Intelligence Unit Sergeants• Intelligence Unit Analyst
1. Interview Lists
• Spate of Burglaries/Robberies in an Area– Are They Linked– Who May Be Responsible
2. Early Detection of Crime Series
• Self Organising Map
2. Early Detection of Crime Series
2. Early Detection of Crime Series
2. Early Detection of Crime Series
2. Early Detection of Crime Series
Each Cluster willContain Crimes That Are Similar
• Model Current Offenders• Overlay Onto Crime Map
2. Early Detection of Crime Series
2. Early Detection of Crime Series SOMKey ENTRY FEATURE FEATURE_TYPE METHOD ROOMS_SEARCHED SEARCH_TYPE CR_NO LASTNAME FIRSTNAME DOBK6-3 Rear Window Fixed Climbed All Tidy ZZ/1686/05 K6-3 Side Window Fixed Smash All Tidy ZZ/2139/05 K6-3 Rear Window Fixed Smash All Tidy ZZ/2493/05 K6-3 Rear Window Fixed Forced All Tidy ZZ/4371/05 K6-3 Rear Window Fixed Forced All Tidy ZZ/1681/05 KHAN VIJAY 21/04/1977K7-9 Front Door Fixed Insecure Down Tidy ZZ/1862/05 K7-9 Front Door Roller Insecure One Tidy ZZ/1056/05 K7-9 Front Door Fixed Insecure Down Tidy ZZ/1520/05 K7-9 Front Door Fixed Insecure Down Tidy ZZ/1578/05 K7-9 Front Door Fixed Insecure One Tidy ZZ/2056/05 K7-9 Front Door Fixed Insecure Down Tidy ZZ/2256/05 K7-9 Front Door Fixed Insecure One Tidy ZZ/3937/05 K7-9 Front Door Fixed Insecure Many Tidy ZZ/694/05 BULLEN IAN 11/11/1982K7-9 Front Door Fixed Insecure One Tidy ZZ/3332/05 SINGH JARNAIL 29/08/1979K9-2 Rear Window Casement Smash All UnTidy ZZ/305/05 K9-2 Rear Window Transom Forced All UnTidy ZZ/457/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/788/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1587/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1865/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1979/05 K9-2 Rear Window Transom Smash All UnTidy ZZ/1980/05 K9-2 Rear Window Transom Forced All UnTidy ZZ/3309/05 K9-2 Rear Window Casement Smash All UnTidy ZZ/1668/05 BURTON ALAN 09/06/1979
• Northamptonshire Forensic Science Department– Dr John Bond
• Motivation:-
3. Modelling Forensic Recovery
3. Modelling Forensic Recovery
• Northamptonshire Forensic Science Department– Dr John Bond
• Motivation:-– Which Crimes Should be Attended First?– Which Crime Give Best Opportunity of Forensic
Recovery?
3. Modelling Forensic Recovery
• CRISP-DM– www.the-modeling-agency.com/crisp-dm.pdf
– Naïve Bayes Algorithm– Q-Prop Neural Network Algorithm
3. Modelling Forensic Recovery
3. Modelling Forensic Recovery
• Algorithm Results– 10 Fold Cross Validation on 28,490 Volume Crime
Records
Basic Variables Basic + Temporal Basic + MO vars
Bayes 68.59 69.2 74.5
Q-Prop 63.59 64.01 69.5
3. Modelling Forensic Recovery
• Results Using Live Data
Crime Ref SubDiv Offence Set Month Forensic Collected
Probability Predict Class
NN/24163/03
NN BDwell December 1 0.82 1
NC/2835/05 NC BDwell May 1 0.68 1
NN/7587/02 NN BOther June 0 0.67 1
NN/16787/02
NN TOMV October 1 0.62 1
NN/12133/04
NN TOMV July 1 0.62 1
NC/1757/01 NC TOMV May 0 0.49 0
NC/14454/02
NC TOMV November 0 0.37 0
NN/31877/03
NN BOther March 0 0.20 0
NW/14184/05
NW TFMV February 0 0.16 0
ND/6657/00 ND TFMV January 0 0.16 0
3. Modelling Forensic Recovery
• Gwent Police Trial– 11,800 Volume Crime Records– 10 Fold Cross Validation• Q-Prop Accuracy 81.79%• Naïve Bayes Accuracy 88.84%
3. Modelling Forensic Recovery
• How Good are the Models?
3. Modelling Forensic Recovery
• Every Northamptonshire CSI– 50 Random Crimes– Assess whether a Forensic Sample would be
collected
3. Modelling Forensic Recovery
• Every Northamptonshire CSI– 50 Random Crimes– Assess whether a Forensic Sample would be
collected• 41% Accuracy
• Which Offenders are Causing most Harm• Including Cross Border Offenders – Force Priorities– Harm Matrix
4. Priority & Prolific Offenders
• Which Offenders are Causing most Harm – Current Process:• Offender is “Nominated”• Scored Against Matrix• Placed on List
– Infrequently Reviewed• Insufficient Time• 20 Minutes to 2 Hours to Complete Scoring
4. Priority & Prolific Offenders
• Automated Process:
4. Priority & Prolific Offenders
4. Priority & Prolific OffendersOffender L2Offender
Offend Priority Nab Live Priority Nab Community Safety
Control Of Offenders Reduce Crime
Total Prism Score BEAT AREA CRIME
Eric Smith 20 10 6 0 0 62 98 Z2
Paul Jones 20 0 6 0 0 71 97 Z2
Mary Hands 20 10 6 4 0 54 94 Z1
John Fresh 20 0 0 20 20 30 90 Z2
Ali Khan 20 10 6 0 0 54 90 Z1
Ming Hu 20 10 6 4 0 50 90 Z1
Graham Zhu 20 10 6 0 0 50 86 Z1
Fred Brown 20 0 0 0 0 60 80 Z1
Sally Johns 20 12 6 0 0 40 78 Z1
David Green 20 0 0 16 10 30 76 Z2
Alison Blue 20 10 6 0 0 40 76 Z1
Tom Black 20 0 0 4 0 50 74 Z2
Vinny Smith 20 0 0 24 0 30 74 Z1
Saad Wang 20 12 6 0 0 36 74 Z1
Mendip Kaur 20 12 0 0 0 40 72 Z1
Brian Ling 20 0 0 0 0 52 72 Z1
Billy Smith 20 10 0 0 20 22 72 Z1
Ho Tu 20 10 6 0 0 36 72 Z1
Paul Wells 20 0 0 0 30 20 70 Z2
4. Priority & Prolific OffendersOffender
A1
A2
A3
A4
A5
A6
A7
A8
A9
B1
B2
B3
B4
B5
B6
B7
B8
C1
C2
C3
C4
TotalCrimes
NumBCUs
OffenderLatest BCU
Eric Smith 0 0 0 0 0 3 0 0 0 5 1 3 5 0 0 0 0 0 0 0 0 17 5 X1
Paul Jones 0 0 1 0 0 0 4 0 0 0 0 1 0 0 0 0 0 0 0 0 1 7 4 V3
Mary Hands 0 0 0 0 0 0 0 0 1 4 0 0 0 1 0 0 0 1 0 0 0 7 4 V2
John Fresh 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 4 4 V2
Ali Khan 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 1 0 0 4 4 V2
Ming Hu 0 4 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 5 0 0 10 3 U2
Jill King 0 0 0 0 0 0 0 4 0 0 0 0 1 0 0 0 3 0 0 0 0 8 3 W1
Fred Brown 0 0 0 2 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 7 3 Z1
Sally Johns 0 0 2 4 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 7 3 Z1
Lin Ho Pu 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 4 0 0 0 6 3 V2
Brian Ling 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 6 3 T1
Billy Smith 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 1 0 0 0 0 5 3 U2
Ho Tu 0 2 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 5 3 M3
Paul Wells 0 0 0 0 0 0 0 0 3 0 1 0 0 0 0 0 1 0 0 0 0 5 3 T2
• Motivation:– Sample Offender Test• Same Offender• 4 BSU’s Where Offender Not Known• 20 Minutes to 2 Hours• Scores From Low 100’s to High 400’s
– Place / Not Place on List
• Scores Not Related To Time Taken
4. Priority & Prolific Offenders
• Benefits:– Every Offender is Scored– Objective Scoring• Defendable• Repeatable
– Process Can Be Frequently Run• Offenders’ Scores Updated• Current
4. Priority & Prolific Offenders
• Offender Networks:– The Data WILL Contain Networks– Which Networks Cause the Most Harm• Prioritisation• Scoring
– Degrees of Freedom• Dependant Upon Priorities
4. Priority & Prolific Offenders
• Offender Networks:
4. Priority & Prolific Offenders
Questions?
Using Data Mining Techniques to Improve Efficiency in Police
Intelligence
Dr Rick Adderleywww.a-esolutions.com