evaluation of the facilitator development programme-workforce...evaluation of the facilitator...

Evaluation of the facilitator development programmeJanuary 2013

we help develop the skills of social workers

Table of Contents

Executive Summary 2

1. Introduction 5

2. Approach 5

2.1. Aim 5

2.2. Methods 5

2.3. Response 5

2.4. Analysis of responses 6

3. Key findings 7

3.1. Expectations of the course 7

3.2. Impact on skills 8

3.3. Impact on personal practice 12

3.4. Impact on confidence to support NQSWs 17

3.5. Impact on confidence to support colleagues 21

3.6. Range of techniques 23

3.7. Organisational culture 26

3.8. Overall impression of the course 27

3.9. Improvements to the programme 28

4. Conclusions 29

Evaluation of the Facilitator Development Programme, 2013

Published by Skills for Care, West Gate, 6 Grace Street, Leeds LS1 2RP www.skillsforcare.org.uk © Skills for Care 2013 Reference no. R050 Copies of this work may be made for non-commercial distribution to aid social care workforce development. Any other copying requires the permission of Skills for Care. Skills for Care is the employer-led strategic body for workforce development in social care for adults in England. It is part of the sector skills council, Skills for Care and Development. This work was researched and compiled by Liz Burtney of Skills for Care Bibliographic reference data for Harvard-style author/date referencing system: Short reference: Skills for Care [or SfC] 2013 Long reference: Skills for Care, Evaluation of the Facilitator Development Programme, 2013, www.skillsforcare.org.uk

http://www.skillsforcare.org.uk/

2

Executive Summary

Introduction

Following the successful delivery of the Facilitator Development Programme to support NQSWs

in 2010/11, Inter-Logics was asked to repeat the course in 2011/12. During 2011/12 six courses

were delivered in total, each lasting four days. The courses were offered in London (2 courses),

Birmingham, Leeds, Manchester and Truro between October 2011 and March 2012. While 90

participants started the programme, 75 completed the workshops and assessment were

involved in the Facilitator Development Programme (FDP), which offered a combination of

related theory with practical facilitation skills and methods. This summary outlines the key

findings from the evaluation.

Methods

There were two main elements to the research: qualitative telephone interviews and three

waves of questionnaires; pre FDP, immediately post FDP and 4-6 months after completion of

the course. In total 12 participants were interviewed (hereafter referred to as interviewees) and

147 responses to the three waves of the survey were gathered (Wave 1: 51 responses; Wave 2:

51 responses; Wave 3: 47 responses). The responses to the three waves of the survey were

not matched.

Findings

Interviewees pointed out that they were already a highly skilled workforce as reflected in their

current job role, predominantly that of learning and development. Indeed some described

themselves as ‘specialists’ and therefore making it difficult for learning and development of any

kind to have a significant impact. The same is true for respondents to the questionnaires

(hereafter referred to as respondents) who where a highly skilled workforce.

Nonetheless, many interviewees felt that the training had reinforced existing skills and

knowledge and provided an opportunity to reflect and consolidate. For example, there was a

reported increase in skills perceived by respondents pre and post FDP, a finding substantiated

by the qualitative work.

“It’s made me think more about listening to people, valuing their ability to sort

problems out … It has had a real impact – it was good to focus on listening and

questioning."

Impact of the FDP

When asked directly, the majority of respondents to wave 2 and 3 of the survey indicated that

the programme had a direct impact on their understanding of the methods used in action

learning, their range of tools used to support others and facilitation skills with small groups.

Around a third felt the programme had impacted on their skills to deliver action learning

processes, ability to help others identify learning needs, facilitation skills with larger groups and

critical thinking skills. This reflected the expectations set out by respondents at the outset of the

course.

3

Impact on skills

Findings from the survey indicated the course had a significant impact on the following skills:

Critical thinking

Facilitation of both smaller and larger groups

Critical evaluation of own development.

The qualitative study reflected these findings and in addition found that the FDP had also made

an impact on skills associated with broader learning and development. Interviewees felt they

now had more techniques on which to draw depending on the situation, and felt they had the

skills to change from one technique to another to reflect both the context and the audience.

Impact on individual practice

From the qualitative study, interviewees described how the FDP had enabled them to use the

skills and techniques from the course and integrate them into everyday practice beyond action

learning groups per se. Examples given included the change of approach to facilitating and

chairing meetings as skills in devolving responsibility for the direction of discussions and

meetings to those present rather than feeling the need to direct. Others described how the FDP

had enabled them to improve relationships with supervisees through more constructive

feedback based on improved listening skills and the use of specific techniques to provide

support. A number of comments around the use of techniques to improve reflective practice

were made and interviewees were keen to point out the value of taking time to think about

workload planning, approaches to challenges and time pressures.

Supporting NQSWs

Respondents to the quantitative questionnaire were asked about past and current experience in

supervising NQSWs. The FDP had a significant impact on levels of confidence across a number

of areas pre and post course including:

Support the development of a community that shares practice

Help NQSWs develop skills for reflecting on their practice

Help NQSWs to understand and develop emotional resilience

Develop NQSWs critical thinking skills

Help NQSWs identify their own leaning needs

Help NQSWs develop better listening and questioning skills.

The FDP appeared to have a significant impact on the confidence of respondents across all of

the areas listed above. There was no significant impact on confidence levels associated with

facilitating groups of NQSWs. However, it is worth noting that for some there was a slight dip

between wave 2 (immediately following the survey) and wave 3 (4-6 month follow up).

Supporting colleagues

All respondents to the quantitative surveys were asked to reflect on their confidence levels

before and after the FDP across a the same areas and there was a significant improvement in:

Supporting the development of a community that shares practice

Helping others develop skills for reflecting on their practice

Helping others identify their own leaning needs

4

Helping others develop better listening and questioning skills.

Again the sample is small for this study so findings should be treated with caution but will

provide indicative results and trends for the direction of confidence levels among participants.

Findings from the qualitative study would support this improvement in confidence across

motivation and willingness to use action learning and other techniques, to be flexible and

respond to different situations, to help work with groups more effectively and to allow groups to

come to their own solution.

Use of techniques

Interviewees were starting to try out new techniques with supervisees and colleagues. The

majority of interviewees also felt that the programme had a significant impact on their

confidence levels associated with both understanding of the action learning approach, and the

range of techniques explored though the programme.

Conclusions

Findings from the survey and the small-scale qualitative study indicate the FDP has made an

impact on a number of key areas including perceived skills in facilitation, critical thinking and

evaluation. In addition, respondents report significant increases in confidence levels associate

with supporting NQSWs and colleagues. The course also prompted changes in both individual

practice, and in some instances, organisational practices.

5

1. Introduction

Following the successful delivery of the Facilitator Development Programme (FDP) to support

NQSWs in 2010/11, Inter-Logics was asked to repeat the course in 2011/12 but on a reduced

scale reflecting available budgets. Each course was conducted over four days between October

2011 and March 2012 in six separate venues1 across England with a total of 75 participants

completing the workshops and assessment2. The course offered a combination of related theory

with practical facilitation skills and methods. More information on the detail of the course can be

found in Boydell and Abbott (2012)3. This report forms the basis of the evaluation of the work.

2. Approach

2.1. Aim

The aim of the evaluation was to determine the impact of the facilitator development programme

on participants’ knowledge, skills and subsequent practice.

2.2. Methods

Participants were approached at the outset of the learning and introduced to the evaluation of

the work. There were two methods used in the evalaution: online survey and depth telephone

interviews.

Quantitative surveys

Participants were asked to complete a questionnaire at three separate points:

1. Prior to the programme commencing

2. Immediately upon finishing the programme

3. 4-6 months after programme completion.

Respondents had the option to complete the survey either online via surveymonkey, submit an

emailed copy or request a paper-based version of the questionnaire which was subsequently

transferred into survey monkey. In the Phase 3 questionnaire, participants were asked to

indicate their willingness to take part in a qualitative interview to complement the quantitative

work and hear more about the impact of the course back in the workplace.

Qualitative interviews

As indicated above, respondents were asked if they would be willing to take part in subsequent

qualitative work. From the positive responses, a sample was drawn and approached initially by

email to confirm interest and establish a suitable time for interview. Interviews lasted on average

30 mins. The interviews were recorded, typed up and sent back to respondents for further

reflection and confirmation of accuracy.

2.3. Response


In total, 147 (out of a possible 225) responses were received across the three cohorts:

1 Two in London, one in Birmingham, Leeds, Manchester and Truro.

2 In total, 90 started the programme, 79 completed the workshops, of which 75 completed the assessment.

3 Boydell T, Abbott C (2012) Training and development programme for action learning facilitators. Sheffield: Inter-

logics.

6

Wave 1: 51 respondents

Wave 2: 51 respondent

Wave 3: 45 respondents.

A profile of the sample can be found in appendix 1 but Table 1 below gives an indication of the

spread of responses across the six venues.

Table 1: Responses by cohort

Wave 1 Wave 2 Wave 3

London (1st cohort) 19% 15% 14%

Birmingham 25% 21% 25%

Leeds 6% 10% 20%

Manchester 28% 6% 14%

London (2nd cohort) 8% 21% 14%

Truro 13% 27% 14%

Base (n=) 47 48 44

Qualitative research

In total 12 participants agreed to take part in the qualitative telephone interviews. These

responses were spread across the six programmes and were conducted between 6-8 months

after the full and final completion of the programme.

2.4. Analysis of responses


The responses from the survey were initially analysed in survey monkey and further testing

conducted in SPSS. Both parametric and non-parametric tests were used to identify areas of

significance between the three waves, depending the distribution of the responses. Where

findings are reported as significant, we are 95% confident that the reported difference between

the waves of the survey responses are not linked to chance.

Qualitative interviews

Using thematic analysis, the key points from the interviews were drawn out and are presented

below according to the main themes identified.

7

3. Key findings

3.1. Expectations of the course

Wave 1: Prior to commencing the programme, participants were asked through an open-ended

question about their expectations of the programme and what they would hope to be better at

following the FDP. The most popular responses included:

A working knowledge of the methods employed in action learning

A deeper understanding of action learning

An improvement in facilitation skills with both small and larger groups

Better skills to help support other people.

Table 2 below indicates the reflection of the responses from waves 2 and 3 in terms of what the

programme actually delivered that translated into greater skills and understanding.

Respondents were asked to choose up to three areas where they perceived the programme

had made a difference.

Table 2: Impact of FDP on perceived skills

Wave 2 Wave 3

Greater understanding of the methods used in action learning 79% 81%

Broader range of tools to use when supporting others 66% 86%

Improved facilitation skills with small groups 60% 62%

Improved planning skills to deliver active learning processes 40% 40%

Greater skills to help others identify learning needs 38% 29%

Improved facilitation skills with large groups 34% 36%

Improved critical thinking skills 34% 31%

Enhanced skills to provide constructive feedback to others 30% 29%

Other learning outcomes 8% 2%

It would appear that the course was indeed successful in improving understanding of action

learning and associated techniques, facilitation skills, albeit it to a greater extent skills

associated with facilitating smaller groups, and tools to help support others. Table 3 sets out the

intention and reality of changes to practice when back at the workplace indicated by open-

ended responses to the survey.

Table 3: Intended and actual changes to workplace practice following FDP

Wave 2 (intended) Wave 3 (actual)

Planned/actual changes in questioning and

listening skills

13% 9%

Planned/actual change in support and

supervision

6% 20%

Planned/actual change in facilitation approach 1% 14%

Planned/actual set up of action learning set 24% 18%

Planned/use of action learning techniques 41% 28%

Planned/promotion of action learning among

colleagues

16% 10%

8

3.2. Impact on skills

Respondents to the surveys and interviewees involved in qualitative study (hereafter referred to

as respondents and interviewees respectively) were asked to reflect on their skills in a number

of areas that were covered as part of the Facilitator Development Programme (FDP). It is worth

noting that respondents were already highly skilled professionals, with a background in learning

and development (e.g. practice educator, learning and development manager). Again the

majority of interviewees came from a learning and development background, involved in either

developing learning in the organisation or practice education. As a result, this was a difficult

cohort to develop in terms of skills given their background in this work they could be described

as ‘specialist’ in this area. This was in part reflected in the comments about the programme.

Indeed one interviewee did feel that the course didn’t have a great deal of impact on current

skills but reflected that this was an area of their specialism so would be difficult to have a big

impact. Nonetheless, many interviewees felt that the programme had reinforced existing skills

and knowledge and provided an opportunity to reflect and consolidate.

“There was a sense that the skills are already there…but sometimes the

confidence to use them or to work slightly differently isn’t there so the course was

good for that.”

“It did give the opportunity to reflect on skills and build confidence in using the

skills I already have.”

“…not sure it’s made a difference to how I facilitate but it reinforces learning that I

have already.”

One interviewee spoke of how she had previously been involved in action learning but this

course gave her a good opportunity to update her skills in this area.

“Having been on a course a few years back, this course seemed to introduce a lot

more techniques, flexibility in approach and a more relaxed introduction to setting

up action learning.”

Questioning and listening skills

Respondents were asked to reflect on their questioning and listening skills, before the course,

immediately after and again 4-6 months after the course finished. From Chart 1 below it is clear

that self-perception of skills around asking questions appeared to dip upon completion of the

programme although it did appear to increase at the follow up stage. While the shift was not

statistically significant, the pattern is interesting to note. It would appear that respondents felt

less confident after the course, than before, indicating some critical appraisal of their skills in

this area as clarified through the qualitative study.

For listening skills, there was a slight increase in self-perception of skills following the FDP,

although this was not statistically significant.

9

Chart 1: Questioning skills (where 1=no skills in questioning up to 10=very high level

questioning skills4)

The findings from the qualitative study would indicate that questioning and listening skills were

indeed areas where interviewees felt they had existing skills but the FDP had a marked impact

on their perception and subsequent ability. In particular, interviewees reflected on how the

programme had given a structure around which to frame questions and reminded them of the

importance of listening to others’ perspectives.

“It’s made me think more about listening to people, valuing their ability to sort

problems out … It has had a real impact – it was good to focus on listening and

questioning."

“I’ve done coaching and mentoring questions but I think what I find most

challenging is to think about how to ask questions in the right way rather than

give advice or answers to problems. Looking at ’Constellations’ and ‘Story

Telling’ methods helps me not to present my solution.”

“It really hones your questioning and listening skills because you really have to

listen to people and the problem. You really have to think long and hard about

the kinds of questions you think need to be asked …that was the best bit of the

training.”

Facilitation skills

When asked about facilitating groups, respondents reported a significant improvement in skills

for facilitating both small and larger groups. Chart 2 and chart 3 below show the shift between

Wave 1 of the questionnaire before the FDP and again after the course.

4 For this and following questions respondents were given a scale of 1-10 to indicate their relative position on the

scale

10

Chart 2 Skills in facilitating small groups (where 1=no skills up to 10=very high level

skills)

Chart 3 Skills in facilitating larger groups (where 1=no skills up to 10=very high level

skills)

The impact of the course on facilitation skills was also noted in the qualitative research. A

number of interviewees commented on the change in approach to facilitation and supervision

with NQSWs.

“Definitely skills around facilitating groups and approaches to supervision have

been improved since attendance at the training.”

“The trainers did work that made me think of how to take a more relaxed

approach to working with a group.”

11

Critical thinking and reflective practice

Respondents were asked to think about their skills in both critical thinking and reflective

practice. Findings indicate that there was a significant shift in critical thinking skills that were

perceived to be improved following the programme as indicated in chart 4 below.

Chart 4 Skills in critical thinking (where 1=no skills up to 10=very high level skills)

While not significant, there was a shift in respondents perceived skills associated with reflective

practice with fewer rating their skills around 5 or 6 and more likely to rate as 9 or 10.

These findings were supported by the qualitative study. While only mentioned by a small

number of interviewees, there was discussion of the impact the course had on reflective

practice and critical thinking.

I have the skills now to work with them [NQSWs] to reflect on practice and think

about critical analysis

Critical evaluation of own development

There was a significant shift in skills associated with critical evaluation of one’s own

development noted by the respondents to the quantitative survey. Respondents were more

likely to rate their skills in this area as higher following the course. Chart 5 below sets out

responses across the three waves of the survey.

12

Chart 5: Skills in critical evaluation of own development (where 1=no skills up to 10=very

high level skills)

Summary

In summary, the FDP course would appear to have had a lasting impact on self perceived skills,

particularly in consolidating existing learning, improving listening and questioning skills,

reflective practice, facilitation and critical thinking skills.

3.3. Impact on personal practice

When asked about the impact of the course on their practice, interviewees were able to

describe changes in the use of techniques and approaches in the organisation. One interviewee

in the qualitative study did feel that the learning had made no impact on practice; this was linked

to the broad experience already possessed and the collaborative approach already taken in

existing practice. All of the other interviewees were able to describe in some detail the impact

the learning had made on both personal and organisational practice.

Use of approach with others

In addition to employing action learning, interviewees in the qualitative study shared how they

had also introduced some of the approaches and skills developed as a result of the programme

into existing learning and development courses and meetings. For example, some described

how they had taken a new approach to chairing with a definite move to devolve responsibility of

the meeting onto all participants.

“This has come as a direct result of the training….about empowering a group

and helping them find their own path through the learning but being there for

them if they need support…I’m not there to facilitate the group but will support

them.”

One interviewee felt the improved listening skills had led to a better relationship on which to

base constructive feedback and critical analysis with supervisees. Others described the use of

specific techniques, e.g. “Learning from Success” into existing packages.

13

“Have been able to apply the techniques to team meetings, individual

supervision and mentoring and personal reflection.”

Another interviewee described her experience of the “Constellation” approach.

“It really worked because they were a visual group of people and it was a really

good basis for the discussion that we had afterwards.”

Others described how they felt able to take a step back and make changes to their

practice.

“So if something doesn't work or the skills mix or management mix isn’t right for

the approach I had first thought of, then I will change and reflect this back to the

group.”

Approach to work

Interviewees in the qualitative study described a number of ways in which the learning had

impacted on approaches to personal aspects of work. These included attention to workload

planning and prioritisation. One interviewee felt the learning had ‘given permission’ to think

about workloads and how best to organise on a day-to-day basis.

“I’m more effective about how my work is set out and planned and how people

can be more involved.”

“The course served as a really good reminder of the importance of planning and

reflection. Sometimes this is really difficult when you’re under pressure at work,

but taking the time out to think about how you are going to approach something

or what worked well on reflection is really important.”

In particular interviewees were able to set this within the context of reflective practice. The

course seemed to give a number of interviewees to take a step back, resist fire fighting and

think about the bigger picture and how to approach things in a different way.

“I think it means that you have to take a step back as sometimes we can get

bogged down with day to day things and by taking a step back and doing it in a

different way can be really helpful.”

Another described how the course had made them think about how they structure meetings and

supervision sessions. Finally, the impact on facilitating meetings was noted and the subsequent

positive impact on outcomes from meetings.

Implementing action learning groups

When presented with a question to reflect on their skills when setting clear ground rules for an

action learning group there was a clear and significant improvement in perceived ability to do

this noted in wave 2 and 3 compared to wave 1 (pre FDP). Immediately after the course

14

respondents were more confident in their own ability to set clear ground rules but this did dip off

at wave 3.

Chart 6: “I find it difficult to set clear ground rules for action learning groups” (where

1=strongly disagree through to 10=strongly agree)

Findings from the qualitiatve work perhaps throw some light on this finding as a number of

interviewees did comment that while they believed in the potential of the approach, they had

some difficulty identifying an opportunity to use action learning in their workplace. Indeed this

was a frustration expressed by a few of those interviewed. Having completed the learning, there

was a lack of opportunity to put the learning into practice. However, the majority of interviewees

had either introduced or planned to introduce action learning into their approach to work. Those

who hadn’t managed to do so at the time of interview had plans to as soon as possible.

“I am convinced of the potential of the approach both in terms of building

internal capacity but improving how people work together and learn. I hope to

get going with the approach once things have settled down internally.”

Others had been successful in identifying an opportunity to implement action learning in their

organisation with different groups within the organisation, for example, trainers, NQSWs.

“I have used the learning in train the trainers….we used the action learning set

technique and got a group to form questions for the person who had the

problem….that was useful.”

One participant reported back that they had positive responses from colleagues.

“It’s early days but people seem to value the approach and seem to be getting

something they haven’t had from any other route.”

15

Some participants had formed a post programme action learning set of their own to enable them

to keep involved in action learning, refresh knowledge of techniques and to support others in the

group use the learning.

“The aim is to keep us going with the [action learning] approach and we can try

out different methods and techniques in order to see how they work and how we

can improve.”

The value of this follow on support was echoed by other interviewees as a way of keeping in

touch with colleagues, and ensuring the programme remains “live and fresh”.

Cascading knowledge

From the qualitative study it was clear that a small number of interviewees had also begun to

cascade their knowledge of action learning to colleagues and supervisees for example, through

team meetings. This included discussions about action learning itself as well as some of the

techniques that can be used.

“I have been able to share the learning…our next team meeting is all about

leading a discussion in an action learning set way … so yes, actively sharing

the learning with others.”

Other changes in behaviour

The quantitative study specifically asked about other behaviours as reflected in the statements

below:

“I find it hard not to give direct advice”

“I am able to support participants to give each other feedback on issues”

“I am good at modelling what is expected as an action learning participant” (significant

improvement)

“I often forget to celebrate supervisees’ positive work”

“I enable participants to reflect on work issue and act on outcomes”

“Empathising with NQSWs is not my strong point”

“I am good a facilitating a supportive but challenging environment for development”

While no further significant changes were noted between the three waves of the survey for the

statements above, there was a trend for improvement for two of the statements. Respondents

indicated that the FDP had improved their ability to empathise with newly qualified social

workers (NQSWs) and their ability to support colleagues give each other positive feedback on

issues (see charts 7 & 8).

16

Chart 7: “Empathising with NQSWs is not my strong point” (where 1=strongly disagree

through to 10=strongly agree)

Chart 8: “I am able to support participants to give each other feedback on issues”

(where 1=strongly disagree through to 10=strongly agree)

Summary







feedback based on improved listening skills and the use of specific techniques to support. A

number of comments around the use of techniques to improve reflective practice were made

and interviewees were keen to point out the value of taking time to think about workload

planning, approaches to challenges and time pressures. Finally, a number of interviewees had

been active in cascading the knowledge gained through the FDP.

17

Other behaviour changes were queried through the surveys and a significant improvement

noted between wave 1 and 2 in particular for respondents ability to set clear ground rules for

action learning groups. There was also a trend (although not significant) for improvements in

empathising with NQSWs and supporting colleagues give each other feedback on issues. No

significant differences were noted for the other five statements which focused on giving direct

advice, modelling expectations of an action learning participant, celebrating positive work,

encouragement of self reflection and facilitating a challenging environment.

3.4. Impact on confidence to support NQSWs

Across the waves, respondents were filtered according to whether or not they currently

supervise or have in the recent past supervised NQSWs. Those who had no experience were

filtered to another part of the questionnaires. The numbers are small so findings should be

treated with caution but there were some significant differences between the three waves

(Wave 1: 39 respondents; Wave 2: 38 respondents; Wave 3: 29 respondents).

Those with current or recent experience were asked to reflect on their confidence levels when

supporting NQSWs across a number of aspects including confidence to:

Support the development of a community that shares practice

Help NQSWs develop skills for reflecting on their practice

Help NQSWs to understand and develop emotional resilience

Develop NQSWs critical thinking skills

Help NQSWs identify their own leaning needs

Help NQSWs develop better listening and questioning skills

The FDP appeared to have a significant impact on the confidence of respondents across all of

the areas listed above and illustrated in the charts below. It did not appear to have a significant

impact on the confidence of facilitating groups of NQSWs.

Chart 9: Confidence to support a community that shares practice (where 1 = not at all

confident up to 10 = very confident)

18

It is worth pointing out that while confidence has improved since wave 1, there appears to be a

slight dip after the initial post-programme period. This might be linked to the boost that learning

and development of this nature might bring, or the lack of subsequent experience of putting the

skills and techniques into practice.

Chart 10: Confidence to help NQSWs develop reflective practice skills (where 1 = not at

all confident up to 10 = very confident)

Chart 11: Confidence to help NQSWs understand and develop emotional resilience

(where 1 = not at all confident up to 10 = very confident)

As noted above, there does appear to be a dip in confidence with wave 3 respondents for both

these areas although some minor increases in those stating ‘very confident’ noted.

19

Chart 12: Confidence to help NQSWs develop critical thinking skills (where 1 = not at all

up to 10 = very confident)

Chart 13: Confidence to help NQSWs identify their own learning needs (where 1 = not at

all confident up to 10 = very confident)

As above, there appears to be some shift in confidence levels between wave 2 and 3. However,

with these two areas, there appears to be a move to being ‘very confident’. The same applies to

chart 14 below.

20

Chart 14: Confidence to help NQSWs develop better listening and questioning skills


This confidence and willingness to support NQSWs was reflected in the qualitative study. There

was a sense from a number of interviewees that attendance on the course had encouraged

them to work with supervisees and colleagues to encourage reflective practice. Others felt that

students they worked with would have a new understanding of working together and how you

can share the power in working relationships, challenge assumptions and discuss things more

openly.

“The students themselves have gone away recognising that they can reflect on

what they've done as they can discuss things more openly.”

In one instance impact on organisational practices was noted.

“…the head of service got to hear what NQSWs felt about the support they get

using World Café approach and there has been organisational change to reflect

this. Just the drawings were so powerful and the message was really taken up.”

Interviewees recognised the importance of building skills and capacity in others particularly in

the current economic climate.

“I have plans to work with this group to encourage reflective practice, working

together and importantly to build skills with social workers rather than to just

solve problems for them. This is much more sustainable and better for the

individual and organisation in the long term.”

In summary, respondents involved in NQSW supervision report significant changes in

confidence levels particularly around developing communities to share practice, reflection,

development of emotional resilience, questioning, listening and critical thinking skills in NQSWs,

helping NQSWs to identify learning needs. This increase in confidence was supported through

the qualitative study.

21

3.5. Impact on confidence to support colleagues

All respondents to the quantitative surveys were asked to reflect on their confidence levels

before and after the FDP across a the same areas and there was significant improvement in

confidence levels in:

Supporting the development of a community that shares practice

Helping others develop skills for reflecting on their practice

Helping others identify their own leaning needs

Helping others develop better listening and questioning skills.

However, there were no significant differences noted for the following:

Helping others to understand and develop emotional resilience

Developing others critical thinking skills.

Again the sample is small for this study so findings should be treated with caution but will

provide indicative results and trends for the direction of confidence levels among participants.

Chart 15: Confidence to support the development of a community that shares practice


As with NQSWs there appears to be a dip after reported in wave 3 with regard to confidence

levels in this area.

22

Chart 16: Confidence in helping others develop skills for reflecting on their practice


Chart 17: Confidence to help others identify their own learning needs


23

Chart 18: Confidence to help others develop better listening and questioning skills


There would appear to be some indication that more support is needed at least for some

respondents to boost confidence again between wave 2 and 3.

In summary, respondents reported improvements in confidence to support work colleagues,

significantly around supporting community to share practice, helping others develop reflection

skills, listening and questioning skills and to help others identify their own learning needs.

3.6. Range of techniques

From the quantitative study, it would appear that knowledge of techniques used in action

learning sets improved across the board following attendance at the FDP. For example, the

table below sets out the changes in knowledge associated with complexity mapping. Full details

of all techniques included can be found in appendix 2.

Table 3: Change in knowledge for the “Complexity Mapping” technique (where 1 is no

knowledge through to 10 which is a lot of knowledge of the technique)

1 or 2 3 or 4 5 or 6 7 or 8 9 or 10

Wave 1 55% 16% 16% 12 0%

Wave 2 2% 6% 21% 55% 13%

Wave 3 5% 2% 31% 36% 16%

24

However, while knowledge had improved, respondents did not appear to have been using the

techniques to any great extent. The most popular techniques were “Complexity Mapping” and

“Learning from Success” and 10% and 9% of respondents respectively report having used the

technique at the follow up questionnaire (4-6 months after completion of programme). This

appears on the surface to contradict findings above which indicates the use of AL techniques by

respondents. However, findings from the qualitative study would indicate that respondents are

using the techniques but it is more the AL approach that has proven useful, for example,

questioning and listening skills, facilitation skills, general approach to supervision rather than the

specific techniques themselves which has had the biggest impact.

Through the qualitative interviews it was clear that course participants had been able to work

with the techniques explored on the course to varying degrees of success. There was

recognition that one of the skills taken away from the course was the ability to think about a

situation and choose the right method for that situation to maximise the outcomes.

“It’s good to know that a few techniques can be used in different places and the

ones I’ve used are easy to adapt for different situations.”

There had been some use of the “World Café” technique, “Complexity Mapping”, “Five Steps”,

“Gossip” method, “Constellation” method, “Storytelling” and “Learning from Success”.

Five Steps: Experience of this technique was described by one participant as useful in setting

out that there wasn't a right or wrong answer to problems. However, one of those interviewed

did express reservations about the method. In particular, the difficulty in managing the

technique so that it didn’t descend into a more aggressive questioning or advice giving session.

Complexity mapping: There was limited use of this approach and mixed experiences. One

respondent described how it was a useful and familiar approach and the one interviewee who

described using it felt it had limited applicability in their particular situation.

“This is really good as people are more familiar with this kind of approach so it’s

a good introduction and a good way of sorting out the themes that we need to

address”

Learning from Success: This technique had been used by a number of those interviewed and

the view was that this was a really positive approach to use which worked well for both the

individual telling the story but also in terms of extending learning to colleagues.

“There was a lot of learning from the session and there was learning for the

candidate but also the others who were taking part in the session.”

Constellation method: There was limited experience of this approach described however, where

it had been used, respondents were positive.

25

“I find this method useful and have used it the most. I’m most comfortable with

this method”

Story Telling: Again this had limited use and one respondent did describe some

difficulties with this approach.

“I struggle with this…sitting someone in the corner and hearing their story. I

struggle to make sure that the story teller is OK just by looking at their body

language”

Gossip method: Only one participant described using this approach and was

challenged initially.

“I was quite anxious for the first five minutes … this technique takes quite a bit

of management and through training learnt how to develop good and supportive

environment, how to let people out of the technique if they need to stop.”

Confidence to use new techniques

The qualitative study was able to probe interviewees on their views on confidence levels

associated with the techniques explored through the FDP. The majority of interviewees felt that

the programme had impacted on their levels of confidence, and in particular this was linked to

greater knowledge and understanding of the action learning approach associated theory.

“I have more confidence and motivation to go ahead and use action learning.”

In addition, there was an increased confidence and willingness to use the tools that had been

described and used on the course. In addition, interviewees reported increased confidence in

thinking about using different methods to different situations and learners.

“I have the confidence to now think about what to use and when to use different

methods.”

“….[I have] confidence to think about how you might use different techniques

and confidence to introduce into new situations.”

Regardless of previous knowledge, some felt that the programme had given them confidence to

apply techniques and facilitation skills where previously they would have struggled.

“I am now confident that I have got the skills in my toolkit to be able to facilitate a

conversation more effectively rather than brushing over things.”

“I have gained confidence in doing small group work and that has had an impact on the

people I work with.”

26

This in turn has had an impact on practice (discussed in more detail below) but some described

the confidence in themselves now to devolve responsibility of group meetings over to

participants rather than carry the burden.

“Have the confidence to devolve the power from one person and spreads the

responsibility of the meeting and outcomes among all participants.”

“I struggle to hold back and not share my personal experience and opinion but I

need them to come up with own solutions and thinking for themselves. This is

about asking the right questions.”

However, a minority did point out that while confidence had improved, the opportunities to

introduce and use new found skills and techniques were sometimes lacking and this was

problematic.

“One problem has been the lack of opportunity to use the techniques learned at

the training.”

In summary, interviewees were starting to try out new techniques with supervisees and

colleagues. The majority of interviewees also felt that the course had a significant impact on

their confidence levels associated with either understanding of the action learning approach,

and the range of techniques explored though the course.

3.7. Organisational culture

Following attendance at the course interviewees were able to articulate some of the benefits to

the approach in terms of having potential to change the culture within organisations to be more

reflective and supportive of staff. Others felt that the approach was a positive move as it allowed

organisations to learn from within rather than rely on outside ‘experts’ coming into the

organisation. Benefits were viewed both in terms of cost and relevance of learning but also

empowering the organisation and individuals. Finally, the flexibility of the approach was viewed

as a real strength as it could reflect different situations and individual needs.

“The approach helps with changing team culture and encourages more

discussion back at the workplace and greater support from within the team”

However, those interviewed recognised there had to be organisational support for implementing

the approach in order for it to make a real difference to the culture and in order to maximise the

benefits of the learning. There also needed to be buy in from colleagues.

“People need to have commitment to the process otherwise they won’t stay or

contribute if the they there isn’t any value.”

In addition, some struggled to identify opportunities to use the learning from the course and this

was seen as a real barrier to moving forward.

27

3.8. Overall impression of the course

All of those interviewed and open-ended comments taken from the wave 3 questionnaires were

positive about the approach and the programme undertaken describing it as ‘inspiring’,

‘motivational’ and ‘building confidence’.

“It was very good – very informative. I was a bit sceptical to start with but the

course was worth it’s weight in gold and that’s why I decided to take it further.”

It was a cost effective, flexible and good approach to learning and

development.”

“I found the course really stimulating and felt I learned a lot”

“I felt the training was very useful and it was clear to see how I could implement

it within my work with NQSWs and the social care workforce more widely”

Many described the new ideas and approaches to working as the learning while others felt it

had given some insight into the different ways in which people can learn and how to respond to

this. Indeed learning from each other.

“The course is really focused on people learning from each other and to really

understand how powerful that can be is a new thing for me”

Many felt their learning took place in a supported environment and they reported having good

fun, building relationships with others in their cohort and developed understanding of other ways

of working.

“Personally I felt it was really useful to meet people who have had similar

experiences. Not necessarily the same role or same difficulty but similar enough

to make a really good connection and to learn from each other.”

Interviewees were appreciative of the tension between time available and the scope of material

presented but most felt the pace of the course was about right and the supporting materials

useful back in the workplace.

Only one interviewee expressed some difficulties with the programme. They felt that while the

course had been a positive experience, it was more of an introduction to the approach and more

exposure and training was needed before the approach could be fully utilised within the

organisation5. In particular, there was a need for more experience in using different techniques,

more guidance needed in setting up a group, how to explain ground-rules, manage difficult

situations etc. While intrigued by the approach, there was not enough depth or practice.

“You can’t have a go once [using the technique] which is effectively what we

did, and be expected to be experts. That’s not enough depth or practice.”

5 Subsequently the programme has been altered to reflect this perspective

28

Overall there was praise for the course and for the trainers who delivered the

sessions.

“Really enjoyed meeting the trainers and felt their approach was refreshing and

enjoyable.”

“I found them [the facilitators] very supportive outside of the course as well so

they delivered above and beyond the actual course.”

“It felt that we were getting people who really knew their stuff and they were

really interesting characters. It felt like we were getting top drawer input and

that’s good for credibility of the training.”

3.9. Improvements to the programme

From the qualitative study one common theme to arise in response to this question was the

need for some sort of follow up. Whether this is a follow up training day, ongoing contact with

trainers or others within the cohort, the course definitely left participants wanting more6.

“It would be really useful to have a follow up day to keep you in touch with the

learning especially if you’d had less opportunity to use this back at work.”

Indeed a small number of those interviewed were keen to proceed onto the Level 5 course to

deepen their understanding and experience of the approach.

One interviewee felt that the guidance could give more detail on how to go about setting up an

action learning set or how to use the techniques. More of a ‘how to’ document to take you

through the process would have been useful especially for techniques that perhaps were more

complex or hadn’t been used as much on the course.

Finally a small number of participants felt the course could have been longer.

“Maybe longer days would have been beneficial so starting earlier and making

more use of the time the participants had together.”

Although one person felt it could have been condensed, perhaps reflecting previous experience

and styles of learning, a view that the trainers are considering.

“If I was being really critical I think we could have covered the material in 3 days

rather than 4. However we used different approaches and ways to get to know

each other as part of that and it was useful and I’ve used it again.”

6 Self supported managed action learning sets were encouraged in all locations except Truro however only in London

and Birmingham were these taken up.

29

4. Conclusions

The FDP course would appear to have had a lasting impact on self perceived skills, particularly

in consolidating existing learning, improving listening and questioning skills, reflective practice,

facilitation and critical thinking skills.







feedback based on improved listening skills and the use of specific techniques to support. A

number of comments around the use of techniques to improve reflective practice were made

and interviewees were keen to point out the value of taking time to think about workload

planning, approaches to challenges and time pressures. Finally, a number of interviewees had

been active in cascading the knowledge gained through the FDP.

Other behaviour changes were queried through the surveys and a significant improvement

noted between wave 1 and 2 in particular for respondents ability to set clear ground rules for

action learning groups. There was also a trend (although not significant) for improvements in

empathising with NQSWs and supporting colleagues give each other feedback on issues. No

significant differences were noted for the other five statements which focused on giving direct

advice, modelling expectations of an action learning participant, celebrating positive work,

encouragement of self reflection and facilitating a challenging environment.

Respondents involved in NQSW supervision report significant changes in confidence levels

particularly around developing communities to share practice, reflection, development of

emotional resilience, questioning, listening and critical thinking skills in NQSWs, helping

NQSWs to identify learning needs. This increase in confidence was supported through the

qualitative study.

Additionally, respondents reported improvements in confidence to support work colleagues,

significantly around supporting community to share practice, helping others develop reflection

skills, listening and questioning skills and to help others identify their own learning needs.

Findings from the qualitative study would indicate that participants had been able to work with

techniques explored through the programme. Some techniques appeared to be more popular,

e.g. “Complexity Mapping” but all agreed that the range of techniques was useful and offered a

flexible approach to different settings, situations and groups.

The findings would suggest that the course was well received, delivered to a high standard and

had an impact on skills, confidence and to some extent practice.

30

Appendix 1: Profile of sample

Wave 1 respondents gave details for the following variables:

Employment sector

o 83% worked in the statutory sector

o 10% worked in the voluntary sector

o 6% worked in the private sector

Organisational footprint

o 44% worked in adult services

o 19% worked in children’s services

o 37% worked in a role that covered both adult and children’s services

Job role (respondents were allowed to tick more than one box hence totals do not add

up to 100%)

o 29% were workforce development officers or managers

o 20% worked as a senior social worker

o 20% had a role as a practice educator

o 14% had a team leader or manager role

o 14% had a learning and development role

o 14% were social workers

o 10% worked as an independent practice educator

o 10% were advanced practitioners.

This is the only wave this level of detail was collected for as there were concerns over attrition

by repeating a number of questions for respondents with knowledge that the variables were not

going to be used in the analysis of data.

31

Appendix 2: Knowledge of different action learning techniques

Action learning Column1 Column2 Column3 Column4 Column5 Column6

1 or 2 3 or 4 5 or 6 7 or 8 9 or 10 Have used

Wave 1 29% 39% 10% 18% 2% 2%

Wave 2 0% 7% 26% 57% 11% 0%

Wave 3 4% 2% 16% 60% 11% 5%

Learning process


Wave 1 0% 22% 24% 48% 4% 2%

Wave 2 2% 2% 19% 60% 17% 0%

Wave 3 2% 4% 18% 37% 35% 2%

Complexity mapping


Wave 1 55% 16% 16% 12% 0% 2%

Wave 2 2% 6% 21% 55% 13% 2%

Wave 3 5% 2% 31% 36% 16% 10%

Five questions


Wave 1 76% 12% 10% 0% 0% 0%

Wave 2 4% 11% 36% 38% 9%

Wave 3 4% 7% 39% 35% 7% 7%

Gossip


Wave 1 85% 10% 6% 0% 0% 0%

Wave 2 4% 12% 25% 42% 14% 2%

Wave 3 7% 9% 35% 32% 11% 5%

Constellations


Wave 1 86% 8% 6% 0% 0% 0%

Wave 2 4% 8% 26% 51% 10% 0%

Wave 3 7% 7% 46% 23% 12% 5%

Story telling


Wave 1 70% 8% 18% 2% 0% 0%

Wave 2 4% 8% 21% 55% 12% 0%

Wave 3 4% 7% 42% 30% 9% 7%

Learning from success


Wave 1 76% 2% 17% 2% 0% 2%

32

Wave 2 4% 2% 24% 55% 13% 2%

Wave 3 5% 4% 35% 30% 16% 9%

Open space technology


Wave 1 81% 8% 12% 0% 0% 0%

Wave 2 2% 6% 35% 43% 13% 0%

Wave 3 9% 7% 44% 23% 14% 2%

World café


Wave 1 79% 6% 6% 4% 2% 4%

Wave 2 0% 4% 27% 52% 17% 0%

Wave 3 7% 5% 42% 23% 16% 7%

Talking stick


Wave 1 82% 10% 6% 2% 0% 0%

Wave 2 12% 4% 14% 49% 12% 6%

Wave 3 12% 14% 45% 14% 12% 2%

Skills for CareWest Gate6 Grace StreetLeedsLS1 2RP

telephone 0113 245 1716fax 0113 243 6417email [email protected] www.skillsforcare.org.uk© Skills for Care 2013

evaluation of the facilitator development programme-workforce...evaluation of the facilitator...

Documents