Collaboration on Impact Learning from a Shared Measurement Approach

within post 16 Employability Programmes in Northern Ireland

Gladys Swanton - CENI

Lorraine Boyd - NEET Forum

www.inspiringimpact.org

Context for Inspiring Impact

Inspiring Impact UK (2012): 10 year strategy – changing the way that Voluntary/Community sector and its funders think about impact: “putting impact practice at the heart of their work”.

Building Change Trust - Big Lottery £10 million investment focusing on the VCS in Northern Ireland. Funded Inspiring Impact NI Programme for an initial 2-years (2013) (with additional funding from the DSDNI)

IINI built upon existing work in the UK including: an Impact Practice Code; a Measuring Up assessment tool for organisations. IINI has been working across 3 main strands: Support to the VCSE on Impact Practice Engagement with funders to develop impact practice Inspiring Impact Exchange

CENI (Community Evaluation NI) is the strategic partner for the delivery of IINI

THE IDEA FOR JET

3

Shared impact measurement approach•More joined up between funder and funded on impact•Better understanding of the desired impacts and what needs to be measured•Projects supported around a common framework•Potentially more comparable data to get assess achievements & learning

…Journey towards EmploymenT

VOLUNTARY AND COMMUNITY NEET STRATEGY FORUM

We are a group of organisations working with young people and have a shared vision that

by: ‘working together so every young person in Northern Ireland is in

meaningful education, training and employment’

AIM OF THE VOLUNTARY AND COMMUNITY NEET STRATEGY FORUM

The NEET Strategy Forum works collaboratively to inform and showcase proactive work that respects the voice of, and focuses on the right of young people to access, maintain and progress to employment of their choice in Northern Ireland and beyond.

NEET STRATEGY FORUM

T.R.E.E

Oasis Youth Project

Sally Gardens Community Centre

DEVELOPMENT OF THE V&C NEET STRATEGY FORUM

Loose affiliation of organisations came together to :

• to enable those working in the V&C Sector to share information and concerns experienced by young people who experienced difficulties access Education, Training, and Employment

• to influence the development and implementation of a Strategy for young people who in need of education, employment and training

Committee for Employment and LearningInquiry into Young People not in Education, Employment or Training

Volume One

Together with the Minutes of Proceedings, Minutes of Evidence andWritten Submissions Relating to the Report

Ordered by The Committee for Employment and Learning to be printed 15 December 2010

Report: NIA 32/10/11R The Committee for Employment and Learning

NORTHERN IRELAND (NI) EXECUTIVE NEET STRATEGY ‘PATHWAYS TO SUCCESS’ IN JUNE 2012Launch of Pathways to Success  Our aspiration is that, by 2020, every young person will not only have an opportunity to access education, training or other preparation for employment but, to the extent that they are able, also avail of that opportunity.

NEETS ADVISORY GROUP (NAG) INTERDEPARTMENTAL GROUP – CHAIRED BY DEPUTY PERMANENT SECRETARY

Chaired by Deputy Permanent Secretary (DEL)

12 Departments

5 Voluntary and Community Sector NEET Strategy Forum

NILGA – Local Government

Employer representative

NEET YOUTH FORUM

• Core Young people who are in need of

education, training and employment

• Young people from across NI, Age

ranging from 16 – 24

• Share experience and knowledge

• Inform their choice of preferred best practice.

• Have genuine constructive input into Northern Ireland NEET reduction agenda

KEY COMPONENTS OF WORK INCLUDE:

Collaboration

 

Communication

 

Best Practice

 

Mapping provision and production of directory

 

Youth Participation.

• Initial presentation of JET to NEETs Forum

• Building Change Trust support to pilot

• Consultation to develop a draft NI framework

• Testing of JET Framework & Process in NI

• Expression of Interest from 14 C/V/SE groups

• 6 pilots were identified for support

JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – Pilot in NI ………

JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – LAUNCHED MAY 2013

A shared approach to measuring young people’s employability

+ +

+

++

+Intrinsic factors

Extrinsic factors

External factor: The labour market

Personal circumstances

Emotional capabilities

Attitudes

Employability skills

Qualifications education +

training

Experience + involvement

Career management

Journey to EmploymenT (JET) framework

JET OUTCOMES (NI draft)

PERSONAL CIRCUMSTANCES•Access to transport•Personal Safety •Access to the internet•Access to childcare•Access to support for young carers•Supported by family and friends•Access to support for disabled people•Reduced substance abuse•Resisting negative peer influences •Reduced offending/anti-social behaviourEMOTIONAL CAPABILITIES•Self-esteem•Autonomy and control•Grit and determination•EmpathyATTITUDES•Aspirations for education•Attitudes to work•Aspirations for work•Aspirations for the futureEMPLOYABILITY SKILLS•Teamwork•Communication•Problem solving •Self-management•Leadership

QUALIFICATIONS, EDUCATION + TRAINING•Basic skills•Achieving qualifications•Attendance and behaviourEXPERIENCES + INVOLVEMENT•Work experience•Perception of value of work experience•Networks•Volunteering •Inter-community engagement CAREER MANAGEMENT•Career direction•Job search skills•Presentation to employers•Confidence in finding employment •Entrepreneurship

EMPLOYMENT•Entry into employment•Sustaining employment•Quality of employment•Satisfaction with employment

Planning

• Complete theory of change with each pilot group

• Prioritising & matching to JET Framework

Measuring

• Select measures & designing

• Implementing/ sample/timeframe

Analysis & Use

• Pre and post analysis

• Using the learning

Implementing JET in NI

Six Participating Groups

Support delivered May-Oct14

Organisation Pilot Project Area

Barnardo’s Bup4Work

Digital Media Choices

Digital Media Training Programme at Community Centre Belfast

Princes Trust ‘Get Into Social Care’

Sandy Row YES Project - Event Management & Community Engagement

Start 360 Switch On to Employment

Tides Training Tides Training – Shared History Group

JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – Pilot in NI

Step1: THEORY OF CHANGE COMPONENTS

Enabling factors: things that have to exist for the theory of change to happen

Activities: what the charities’ services are

Outputs: what services are provided

Outcomes: Short – Medium term changes that the charity wants to happen

Evidence or assumptions: why you think one outcome will lead to another and cases where it will not

Impact: Ultimate goal - what the charity is trying to achieve

Digital Media Example

Digital media training programme over 30 weeks/one day a week

Enablers:Access to up-to-date quality training, equipment, resources, expertise and practical work experienceLearner friendly approach and peer support

YP have increased knowledge/skills in use of a range of digital media

Y/P have increased confidence/ self-esteem/ achievement

Project activitiesProject Outcomes – Short - Medium

Y/P have improvedemployability skills: •Improved team-working•Improved communication•Improved problem solving•Improved self-management•Showing leadership

Y/P secure jobs in digital media

YP have greater choices whether to pursue further education/ training/ employment /self-employment ie entrepreneurship

YP achieve qualifications and experience required by employers in digital media

YP have positive role models

End Goal/Impact:

Y/P gain entrepreneurial skills / know what is involved in developing a business in the digital media field

Y/P are up-skilled and inspired

Work experience to contribute to digital media projects

Y/P are willing to engage with work/ learning outside of their local area

Priority Outcomes and JET Measures Table Example

Your Theory of Change Outcome

JET Outcome Area

JET Outcome Measure

JET Outcome Tool

Y/P have increased confidence

Emotional Capabilities

Self-esteem Rosenberg’s Self-Esteem (p41)

Y/P have improvedemployability skills

Employability Skills

Employability Skills Confidence/ 

Employability / Personal Development Scale (pg 52)

Y/P gain entre-preneurial skills / how to developing a business in digital media

Career Management

Entrepreneurship

Entrepreneurship / Starting a business (pg 74)

STEP 4: USE THE JET FRAMEWORK TO SELECT MEASURES

22

Example: Rosenberg Self-esteem scale

Step 5: CHOOSE A RESEARCH DESIGN

23

Before the course:Pre-questionnaire

Intervention

After the coursePost-questionnaire

Long term follow up

Outcomes Prioritised by NI Pilot Projects

JET Outcomes

0 1 2 3 4 5 6

Self- Esteem

Employability Skills

Inter -community engagement

Value of Work Experience

Aspiration for the Future

Attitudes to Work

Communication

Confidence in Finding Employment

Empathy

Entrepreneurship

Personal Safety

FINAL STEPS

STEP 6

IMPLEMENTING MEASUREMENT TOOL

25

STEP 7

ANALYSE THE DATA

STEP 8

LEARN AND IMPROVE

Feedback on the Outcomes and measures…

The JET framework enabled a broader range of outcomes to be captured at a project level; outcomes which are not often articulated or measured

The JET pilot was part of the wider strategic level discussions on impact measurement which added

value to the whole process

“ The JET process supported the development of good impact practice within the organisation including: planning impact, using new tools to measure, building in analysis, learning and reporting

The use of externally validated measures strengthens the evaluation process and provides greater credibility when reporting about achievements

Views on use and implementation of JET resources

The visual presentation on the JET process and the Framework was seen as user friendly and eye catching

Looked at ‘Impact Practice’ in more systematic way. Breaking the process down into 8 distinctive inter-linking steps helped to make it manageable – all the steps were important

It was useful to have a menu of measures to draw from and be able to pick what suited – but there are some challenges with “off-the-shelf” measures

“ Theory of Change was a useful process to think through – making the journey explicit “how to enable young people to move from hopelessness to aspiration”

The pre and post questionnaires were straightforward to use and advice in the JET pack on the practicalities of implementation useful

The pre-coded Excel analysis sheets produced a range of data useful for reporting – however user-friendly formats for presentation of data would be useful

Taking it Forward

• All the pilots groups indicated interest in using JET in their work

• Pilot groups valued the CENI support in navigating their way through the 8 steps - training and support is needed

• Feedback from the pilot support informed the final developmental phase – plans for user-friendly web-based version

• Access to Resources

– JET Framework NI on-line

– Case studies made available on-line

– Process Report on Piloting JET in NI

Thank You

• Lorraine Boyd Regional Project Manager NEET Strategy Forum NI

• E: lboyd@brysongroup.org• T: +44(0)2890325 835 • WWW.Neetforumni.org