Promoting quality in probation supervision: Evaluating the SEED programme in Romania

Presentation to STREAM final conference Malta, 22-24 October 2014

Joanna Shapland and Angela Sorsby

1

SEED

SEED – Skills for Effective Engagement and Development

•Focuses on what practitioners do with convicted persons/offenders in supervision

•Enhancing the effectiveness of one-to-one work and promoting quality

•With the aim of promoting desistance (stopping committing offences)

•It’s a ‘training plus’ package, building on practitioners’ existing skills and training

•Developed by NOMS (National Offender Management Service) in England and Wales – where we evaluated it

•And now, in STREAM, seeing how it might transfer to Romania2

The SEED model

3

The SEED ‘package’

So, SEED training builds on practitioners’ previous training – but it’s not just a ‘refresher course’, nor adding tools:

•Teams are trained together

•In an initial long session, plus four subsequent shorter sessions at three month intervals (Continuous Professional Development)

•With their manager

•Reflecting back each time on what was useful, what difficult

•With, in between the sessions:

– Team practitioner meetings to talk about a particular interesting ‘live’ case together

– Observation of supervision by managers with feedback

4

Some differences between England and Romania

• Initial training 3 days England – 4 days in Romania– then, at 3 month intervals, 3 subsequent sessions of 1 day England – 1½

days Romania, plus fourth review session• because of translation time, travel time• used one English, one Romanian trainer (avoiding translation problems)

• Average length of supervision is about 12 months in England – just over 5 years in Romania

• Evaluated 3 Probation Trusts in England (parts of London, Thames Valley, Merseyside) – Bucharest and Dolj/Brasov in Romania

• Separate control areas in the same Trust in England – Though they weren’t always all that similar to the SEED-trained area

– Non-SEED-trained probation officers in the same office in Romania

• Different histories and supervised populations:– Romania is much younger service (since 2000), but England has had much recent

change in culture and organisation

– Community sentences and licence in England, only community sentences in Romania, including higher proportion of first time convicted persons

5

Evaluating SEED in Romania

We intended just to look at the experience of putting on the training plus in Romania

-but ended up doing an actual evaluation in Romania, as far as possible

Practitioners’ views: on the training, on applicability to their practice– through questionnaires at the end of every training session; interviews

Convicted persons’ views on their supervision – and did they notice?

– requires comparison of trained and control groups

Did it have any effect on desistance?– measure through compliance (but rates depend on supervisory practice)

– in the future, reconviction (in England)

6

The results – practitioner views

Practitioners were very positive – in both countries. They were already familiar with the content, but appreciated the refreshing and having it in a more structured framework:

‘It was like we had a cupboard full of clothes and now we put some order to the clothes’ (Romania)

How useful did you find the SEED training?

1 = very useful 5 = not at all useful7

Looking back over the whole SEED training and your practice, what has been the overall

impact on you and your practice?(1 = very positive; 5 = not at all positive)

8

 

RomaniaMean

EnglandMean

On your confidence in doing one-to-one supervision 1.6 1.7On your ability to deal with different offenders 1.3 1.8On your knowledge and skills 1.6 1.7On your ability to plan the course of supervision 1.6 1.9On your ability to deal with unexpected crises 1.6 2.2On the extent to which you talk with the offender about the purpose of supervision

1.3 2.0

On the extent to which you talk with colleagues about one-to-one supervision

1.4 2.1

On the extent to which you talk with your line manager about particular cases

1.3 2.1

So, what was it about the training?

Almost all the parts of the SEED model were found helpful in both Romania and England:

– Motivational interviewing

– Relationship building

– Collaborative goal setting (RNR)

– Pro-social modelling

– Cognitive behavioural techniques (CBT)

But it was the training on putting them together that was most appreciated – it was a framework, not a rigid programme:

– Structuring sessions

– Endings (only given in Romania)

‘Structuring the activity and buying some time’ (Romania)

‘Using SEED techniques it is easier to adapt to individual needs. I don’t feel surprised by situations, by crises. My work is more structured, more focused.’ (Romania)

9

Any problems?

Probation practitioners want to produce supervision of high quality

But, in England and Romania, perceive there to be some practical obstacles:

– which focus around time and caseload

– (in Romania) space – shared rooms to see convicted persons

– (and previously, in England), overly prescribed targets and standards

SEED puts the emphasis on planning – over the session, over the course of the order

And if practitioners have not previously planned much, it takes time to do this, and reflect afterwards (as well as write up the notes)

But practitioners found after a while it could work:

‘SEED comes somehow to put some order in all these methods, to show how some different methods, that have nothing in common, can be implemented in practice’

10

Working together

SEED is not just the training days, but also teams discussing live cases together between the training, and observations by managers

One-to-one probation supervision can be ‘hidden work’

Both aspects were found helpful – probation counsellors in Romania were more used to discussing cases together than in England:

11

  RomaniaMean

EnglandMean

1= very positive … 5= not at all positive    

Training with your team members 1.4 1.4

Training with your manager 1.4 2.0

1= very helpful … 4= not at all helpful    

Helpfulness of sessions where discussed live cases 1.5 1.6

Helpfulness of observations by managers 1.5 1.7

Observations by managers

Again, more likely to have occurred previously in Romania (17% of Romanian counsellors said it was new to them, 90% in England)

Positively rated – a bit stressful in England (not in Romania)

Romanian comments:

‘It offers the probation officer the possibility to develop in a good way by considering the feedback received.’

‘It is important for the quality of work and for the team relationship.’

 ’It is very useful because you can improve your working techniques with the client.’

But counsellors were worried about increasing time pressures and caseload pressures preventing this occurring (and some teams seemed to have stopped)

12

What did convicted persons think about their supervision?

A questionnaire to convicted persons looked at their counsellor’s use of the various skills in SEED: relationship building, structuring, pro-social modelling, motivational interviewing, risk-need responsivity and cognitive behavioural techniques.

In England, there was one main factor – convicted persons tended to see all the elements of supervision as linking

In Romania, five factors:

1.Relationship with counsellor – structuring sessions – pro-social modelling (relationship)

2.CBT – risk-need responsivity – motivational interviewing – pro-social modelling (effecting change and approaching problems)

3.Challenging behaviour and attitudes (and irritating)

4.CBT (relationships with those around the convicted person)

5.Reviewing progress and working towards goals13

What do we know about convicted persons’ views on quality across Europe (from previous research)?

14

•Developing a relationship with their supervisor, through having sufficient time and consistency to do this, and through their supervisor listening and taking on board where they are and their problems, including knowing about their home and their families

• Having a supervisor who listens, but who keeps on trying to steer them in a desisting direction, through motivating them, encouraging them to solve problems, talking about problems

• Provision of practical help and support in relation to the problems users themselves identify, together with seeking out what referrals to make to external agencies

Probation counsellors are less likely to identify practical help and support as important

Were there differences in convicted persons’ supervision experience between England and

Romania?

Note convicted persons on supervision in Romania more likely to be first time convicted – and to have one probation counsellor

In Romania more likely to be seen monthly (England weekly)

Romanian counsellors were more likely to ‘signpost’ to other agencies, rather than ‘refer’ (make an appointment) or assist the convicted person to refer

Convicted persons in Romania were more likely to say it was agreed at the end of the session what should be done next – and more likely to understand there was an overall plan

15

Did SEED training make any difference?

Overall, convicted persons with SEED trained counsellors and those without (control group) were equally very positive about their counsellors

And slightly more positive than those in England

Was this positivity real or a worry that they might be able to be identified (cultural or linguistic difference?)

‘I am very pleased with the actual supervision and I admire my probation counsellor who wants to reintegrate me’

‘It gave me the strength to move on, to find a job and especially to think more positively than I used to do’

‘My probation counsellor is a special person! She knows what she is doing and she is a real professional’

16

Did SEED training make any difference?

Some slight differences between SEED and non-SEED trained counsellors – as convicted persons saw it:

In England, convicted persons saw SEED trained counsellors as using more SEED skills overall – but no difference in Romania

In Romania, SEED trained counsellors were seen as:

More likely to be focusing on particular aspects in particular sessions rather than talking about almost everything every session

Somewhat more likely to challenge convicted persons (though this might also annoy)

17

What happened in transferring SEED to Romania?

It happened! – and it was very much appreciated

Strong support from senior staff in the Ministry of Justice

We think, at a time of change in both countries

(increase in probation responsibilities in criminal justice in Romania, increase in discretion for staff in England)

•It was seen as investing in staff training and expertise

•It concentrated upon the ‘core job’ – one-to-one supervision

•It picked up from initial and previous training which had concentrated upon particular tools/skills – to provide how to put things together

•It brought teams together, though it was more effort initially

18

Are there any lessons for probation policy transfer?

We had some (minor) difficulties:

•Linguistically (training took longer with translation; commercial translators’ translation of questionnaires needing checking with a specialist)

•Learning the systemic differences on justice statistics (one needs someone familiar with the statistics codes) and on probation practice (but that’s good for us!)

•Visiting was important at the beginning and end of project

•A senior and very helpful coordinator liaison was essential

•Staff respond in terms of their current view on their work and what is happening politically to it – need to be aware

But overwhelmingly, we were impressed by everyone’s commitment to providing high quality supervision practice

19

Some references:

Shapland, J., Bottoms, A., Farrall, S., McNeill, F., Priede, C. and Robinson, G. (2012) The quality of probation supervision – A literature review: summary of key messages . Ministry of Justice Research Summary 2/12. London: Ministry of Justice, at http://www.justice.gov.uk/downloads/publications/research-and-analysis/moj-research/quality-of-probation-supervision.pdf

Robinson, G., Priede, C., Farrall, S., Shapland, J. and McNeill, F. (2013) ‘Understanding “quality” in probation practice: frontline perspectives in England & Wales’, Criminology and Criminal Justice (online)

Sorsby, A., Shapland, J., Farrall, S., McNeill, F., Priede, C. and Robinson, G. (2013) Probation staff views of the Skills for Effective Engagement Development (SEED) project. National Offender Management Service Analytical Summary, at https://www.gov.uk/government/publications/analytical-summary-probation-staff-views-of-the-skills-for-effective-engagement-development-seed-pilot

Shapland, J., Sorsby, A., Robinson, G., Priede, C., Farrall, S. and McNeill, F. (2013) 'What quality means to probation staff in England in relation to one-to-one supervision', in I. Durnescu and F. McNeill (eds) Understanding penal practice. London: Routledge, pp. 139-152.