Risk Assessment

Possibilities and Impossibilities

Stephen D. HartSimon Fraser University

Violence Risk Assessment

Evaluations of people to: Characterize the risk they will commit

violence in the future Develop interventions to manage risk

The clinical task is to: Understand how and why people chose to act

violently in the past Determine whether these or other factors

may lead them to make similar choices again

Goals of Risk Assessment

Prevent violence

More specifically... Guide intervention Improve consistency of decisions Improve transparency of decisions

Protect clients’ rightsLiability management

Which Risk Factors Should We Assess?

Identifying Risk Factors

Three primary criteria Empirical (predictive accuracy) Professional (practical utility) Legal (fairness and reasonableness)

Problems: Empirical Criterion

Not everything that is important has been proven or validated scientifically Can lead to exclusion of “good” but rare

or difficult-to-assess risk factorsPrediction cause, explanation, or

intervention Can lead to inclusion of “bad” but

common or easy-to-assess factors

Example: The SIEVE

Age Young is badSex Male is badFacial hair Dense is badFoot sizeBig is bad

Problems: Professional Criterion

Focus on dynamic factors may bias risk assessments Can lead to exclusion of “good” but

static or easy-to-ignore factorsConventional wisdom of

professionals may be plain wrong Can lead to inclusion of “bad” but vivid

or dramatic factors

Example: Clinical Intuition

Depression Present is goodAnxiety Present is goodIntelligence High is goodRorschach Seeing viscera is bad

Problems: Legal Criterion

Useful for excluding risk factors, but not for including them

It can be argued that almost any risk factor is unfair or unreasonable in some respect

How Should We ConductRisk Assessments?

Methods of Risk Assessment

Discretionary Unstructured professional judgement Anamnestic assessment Structure professional judgement

Non-discretionary Psychological tests Actuarial tests

Unstructured: Features

No constraints on evaluation Any information can be considered Information can be gathered in any

mannerNo constraints on decisions

Information can be weighted and combined in any manner

Results can be communicated in any manner

Unstructured: Limitations

No systematic empirical support Low agreement (unreliable) Low accuracy (unvalidated) Foundation is unclear (unimpeachable)

Relies on charismatic authorityDecisions are too generalFocus is not on action

Anamnestic: Features

Imposes some structure on evaluation Must consider, at a minimum, nature and

context of past violenceAction-oriented

Logically related to development of risk management strategies

Consistent with “relapse prevention” or “harm reduction” approaches

Anamnestic: Limitations

Unknown reliabilityUnknown validityAssumes that history will repeat

itself Violent careers are static Violent people are specialists

Structured Judgement

Designed to prevent an outcome Action-oriented

Imposes major structure on evaluation Must consider, at a minimum, a fixed and

explicit set of risk factors Specifies process for information-gathering

Imposes minor structure on decision Specifies language for communicating

findings

HCR-20

HCR-20, version 2 Webster, Douglas, Eaves, & Hart (1997) Designed to assess risk for violence in

those with mental or personality disorders

10 Historical, 5 Clinical, and 5 Risk Management factors

Historical

Previous violenceYoung age at first

violenceRelationship

instabilityEmployment

problemsSubstance use

problems

Major mental illness

PsychopathyEarly

maladjustmentPersonality

disorderPrior supervision

failure

Clinical/Risk Management

Lack of insightNegative attitudesActive symptoms

of major mental illness

ImpulsivityUnresponsive to

treatment

Plans lack feasibilityExposure to

destabilizersLack of personal

supportNoncompliance with

remediation attempts

Stress

Structured: Limitations

Requires “retooling” of evaluation process Systematized information-gathering New training and technology

Justification for use requires induction What works elsewhere and with other people

will here and with this personAssumes professionals can exercise

discretion appropriately

Psychological Tests: Features

Measure some disposition that predicts violence, according to past research

Reliability and validity of test-based decisions has been evaluated

Imposes major structure On some part of the evaluation process On some part of the decision-making

process

Psychological Tests: Limitations

Require professional judgment Which tests to use How to interpret scores

Justification of use requires induction What works elsewhere and with other

people will here and with this person

PCL:SV

Symptom construct rating scale Requires clinical/expert judgment Based on “all data”

Data obtained from two primary sources: Review of case history (required) Interview/observation (recommended)

Items

Part 1SuperficialGrandioseDeceitfulLacks remorseLacks empathyDoesn’t accept

responsibility

Part 2 ImpulsivePoor behavioral controlsLacks goals IrresponsibleAdolescent antisocial

behaviorAdult antisocial

behavior

Risk Scales: Features

Designed to predict an outcomeHigh-fidelity

Optimized for specific outcome, time period, population, and context

Impose rigid structure On all of the evaluation process On all of the decision-making process

Risk Scales: Limitations

Still require professional judgment Which scales to use How to interpret scores

Justification of use requires induction What works elsewhere and with other

people will here and with this personResults may be easily misinterpreted

Pseudo-objective, pseudo-precise

VRAG

Violence Risk Appraisal Guide Quinsey et al. (1998) Constructed in adult male patients

assessed or treated at a maximum security hospital

12 items weighted according to ability to postdict violence over 7 year follow-up

Total scores divided into 9 bins, with estimated p(violence) from 0% to 100%

VRAG Items

PCL-R scoreElem. school

problemsPersonality disorderAge (—)Separated from

parents under age 16Failure on prior

conditional release

Nonviolent offense history

Never marriedSchizophrenia (—)Victim injury (—)Alcohol abuseFemale victim (—)

How Accurately Do We Assess Risk?

A Complex Phenomenon

Studying the accuracy of risk characterizations is difficult due to the complexity of… The characterizations The violence The follow-up Indexes of accuracy

The Violence

Nature: Victims (stranger vs. acquaintances); motivations (instrumental vs. reactive), context (co-factors)

Severity: Physical or psychological harm (threats vs. battery vs. homicide)

Imminence: Timing (sooner vs. later)Frequency: Number of events (single vs.

multiple)

The Characterizations

Evaluator: Professional vs. researcher; novice versus expert

Process: Clinical vs. actuarial, contextual vs. context-free

Timing: Admission vs. discharge, static vs. dynamic

Metric: Uni- vs. multi-dimensional, categorical vs. continuous

The Follow-Up

Data source: Patient vs. collaterals vs. records

Time at risk: Weeks vs. months vs. years

Interventions: Dynamic factors, life events

Monitoring: Continuous vs. endpoint

Indexes of Accuracy

Comparison group: Other patients vs. normals

Statistic: Uni- vs. multi-variate, timeWeighting of errors: Equal vs.

differential Interpretation: Chance vs. status quo

vs. perfection

OutcomePrediction Not Violent Violent

Low Risk High Risk

Science Responds to Complexity

Example: Psychopathy

OutcomePrediction Not Violent Violent

Low Risk(Non-psychopath)

90 24

High Risk(Psychopath)

12 40

Harris, Rice, & Cormier (1991)

Example (cont.)

In this study... Accuracy of positive predictions is 77% Accuracy of negative predictions is 79% Overall accuracy is 78% Chance-corrected agreement is 53% Correlation is .53 Odds ratio is 12.5

Example (cont.)

So, how did we do? Relative to chance: Great! Relative to perfection: Awful! Relative to the status quo: ???

What is the status quo?

Predictions of violence using the PCL-R typically have an effect size (r) of about .25-.35; the average effect size for psychosis is about .20-.30

An effect size of .40 may be the “forensic sound barrier”

But what is the status quo in other human endeavors?

Meta-Meta-Analysis

Lipsey & Wilson (1993) reviewed 302 meta-analyses

Determined typical effect sizes for psychological, educational, and medical interventions

Psychological Interventions

CBT — depression .44Psychotherapy — any .39Correctional programs (youths) .23

— any Diversion (youths) — recidivism.20Correctional treatment (adults) .12

— any

Educational Interventions

Small classes — class climate .26Tutoring — grades .20Small classes — grades .10Media campaigns — seatbelt use

.06

Medical Interventions

Speech therapy — stuttering .54Bypass surgery — angina pain .37Cyclosporine — organ rejection.15Bypass surgery — mortality .07ASA — heart attack .04

Violence Predictions in Context

Speech therapy — stuttering .54CBT — depression .44Bypass surgery — angina pain .37Psychopathy — violence .25-.35Psychosis — violence .20-.30Small classes — class climate .26

Possibilities

It is possible to assess violence risk in a reliable and valid manner Risk factors assessed should reflect

scientific, professional, and legal considerations

Risk can — and should — be assessed in different ways

Violence risk assessments are, on average, as good as most other prognostications

Impossibilities

It is impossible to make specific predictions of future violence for a given individual with a high degree of scientific precision or certainty We never know risk; we merely

estimate it assuming certain contexts God doesn’t play dice

Contact Information

Stephen D. Hart, Ph.D.Department of PsychologySimon Fraser UniversityBurnaby, British ColumbiaCanada V5A 1S6Tel: 604.291.5485 / Fax: 604.291.3427E-mail: shart@arts.sfu.caURL: www.sfu.ca/psychology/groups/faculty/hart