yosec - assessing how much of 'what works' programmes do - kaye mclaren
TRANSCRIPT
![Page 1: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/1.jpg)
Overview of YOSEC
Development and use of the Youth Offending Services Effectiveness Checklist
All intellectual property rights and copyright for this presentation held by Kaye McLaren. To be used only with her permission.
![Page 2: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/2.jpg)
YOSEC – what is it?! Youth Offending Services Effectiveness
Checklist
Checks how much of ‘what works’ to reduce youth offending a programme is actually doing
![Page 3: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/3.jpg)
Main YOSEC documents
Y o u t h O f f e n d i n g S e r v i c e s E f f e c t i v e n e s s C h e c k l i s t
Y O S E C c h e c k l i s t3 7 i t e m v e r s i o n ' l o n g '
2 0 i t e m v e r s i o n ' s h o r t 'T h i s i s w h a t y o u f i l l i n
S c o r i n g G u i d e l i n eu s e d t o s c o r e c h e c k l i s tI n c l u d e s s c o r i n g p r o f i l eG i v e s p i c t u r e o f s c o r e s
P r o g r a m m e D e v e l o p m e n t G u i d e l i n eE x p l a i n s e a c h i t e m s c o r e
G i v e s r e s e a r c h b a c k g r o u n dG i v e s i d e a s f o r p r a c t i c a l a c t i o n s
Y O S E C d o c u m e n t s
![Page 4: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/4.jpg)
History of YOSEC Long-held dream of mine
Funded by multiple youth justice agencies in NZ who could see the potential
No changes without my agreement – protects YOSEC quality
![Page 5: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/5.jpg)
YOSEC – aims
2. Help funders makes decisions on who to fund, taking into account how much of ‘what works’ a provider is doing
4. Help providers identify their strengths to protect, and where it’s worth putting time and money into development
![Page 6: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/6.jpg)
Informing funding
Funding systems lack way of assessing fit with evidence-base
Funders can quickly and easily assess this using YOSEC
![Page 7: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/7.jpg)
Two halves of the apple
Programme capability AND likely impact on crime
Capability only can mean noless crime or even more crime
‘What works’ only can meaneffective programmes fall over
![Page 8: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/8.jpg)
The impact of doing ‘what works’1. 1/Provide more hours of services to high risk - 10%
less crime
3. 2/ Target factors known to be causally linked to crime - 19% less crime
5. 3/ Use behavioural techniques - 23% less crime
Do all three = 32% reduction in reoffending
Do none = 7% increase in reoffending
![Page 9: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/9.jpg)
Aiding provider development
Works like a mirror – hold up to providerto see areas that are looking good
and those that are not so flash!
![Page 10: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/10.jpg)
Aiding provider development 2 Means you know what
to work on and what to spend money on
Helps funders identify what development to fund most urgently
![Page 11: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/11.jpg)
Development process I systematically reviewed international
research on ‘what works’
Converted findings into a draft checklist
Wrote programme development guide
![Page 12: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/12.jpg)
Development process Held 6 national consultation meetings
with youth justice and cultural experts
Revised checklist and did scoring guide – marathon task!
![Page 13: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/13.jpg)
Cultural consultation process Included Maori and Pacific cultural and
programme experts in initial six consultation meetings nationwide
Reviewed cultural research
My research reviewed by a cultural expert
![Page 14: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/14.jpg)
Cultural consultation process
3 days of hui with Maori cultural experts
3 days fono with Pacific cultural experts
Largely happy but had some suggestions for improving it
![Page 15: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/15.jpg)
Cultural sections Only one section is specifically about
‘culture’
Cultural issues built into most questions
Reflects idea that cultural practice is normal good practice
![Page 16: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/16.jpg)
Flow chart of using YOSEC
![Page 17: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/17.jpg)
Who fills it out Designed to be self-administered by
provider – ‘fudge factor’
Independent assessment is more reliable and builds partnership
Surprised by funder willingness to train and resource assessment staff
![Page 18: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/18.jpg)
How long it takes to do
Both long and short versions take 1.5 to 3 hours to fill in
Depends on how much people talk!
Then more time for looking at documentation or talk to stakeholders – worth it for extra accuracy
![Page 19: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/19.jpg)
YOSEC meetings are not like this!
![Page 20: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/20.jpg)
How long it takes to do Doing development plan
takes a few more hours
Less than half the checklist is questions - less pages wouldn’t mean less time
To get information any other way takes months or even years
Involves process or outcome evaluation
![Page 21: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/21.jpg)
Eg of detailed action plan
Date: 1 February 2007
Item no YOSEC score
Action plan to build or protect strengths Do by date
Level 1 items
B1 Engaging with clients
1rain and encourage staff to be supportive in practical ways, e.g. helping with benefit hassles, food or transport
In staff meeting identify main areas of practical help we can provide by brain storming then choosing the most viable ones (Staff responsible: NAME)
Develop brief training session of information and skills needed to provide this help (Staff responsible: NAME)
Develop information kit on relevant services e.g. benefits, policy on transporting clients etc (Staff responsible: NAME)
1 Feb 08 1 April 07 1 June 07 1 June 07
![Page 22: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/22.jpg)
Using YOSEC over time Can do YOSEC more than once – cheap
and quick (relatively speaking)
Shows you where programme got closer to ‘what works’ over time
Obvious where strengths drifted
Can track what actions still need doing
![Page 23: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/23.jpg)
Considerations re funding
Suggest don’t cut or stop programme funding after first YOSEC assessment
Not fair to providers – historically been little development and training in ‘what works’ in New Zealand
‘Throw money at it and hope for the best’
![Page 24: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/24.jpg)
Considerations re funding Recommend doing development for at least
one year before re-assessing
YOSEC score not only reason for cutting funding – how well staff relate to clients, how willing they are to change
Recommend funding for development
![Page 25: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/25.jpg)
Main provider concernsVery busy – YOSEC another thing to fit
into a hectic schedule
Already have a lot of compliancepaperwork whichthey are not paid to do
![Page 26: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/26.jpg)
Main provider concerns
Anxious about impact on funding, scared of losing it
May not interested in changing programme – think it’s good enough already
![Page 27: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/27.jpg)
Addressing provider concerns Important to find out what concerns are
and reassure providers
Cover how to do this in training
Using a powerpoint presentation like this at the start can help the provider
![Page 28: YOSEC - Assessing how much of 'what works' programmes do - Kaye McLaren](https://reader033.vdocuments.mx/reader033/viewer/2022060200/5598ab4d1a28abb93f8b467a/html5/thumbnails/28.jpg)
Incentives to do YOSEC Pay provider for time doing YOSEC
Guarantee not to stop funding for specific development period
Tag YOSEC reports to funding applications
Provide a significant development fund