shcr review 2015 - appendix 3 impact measurement
TRANSCRIPT
1 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
The School for Health and Care Radicals: what impact has it had?
Appendix 3: Measuring the impact of the SHCR
Ksenia Zheltoukhova, CIPD
2 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Contents Contents .................................................................................................................................................. 2
Note ......................................................................................................................................................... 2
Introduction ............................................................................................................................................. 3
Methodology ....................................................................................................................................... 3
Measures ............................................................................................................................................. 7
Results ..................................................................................................................................................... 8
Refining the SHCR impact model ......................................................................................................... 8
Engagement with the School and feedback on the SHCR experience ............................................... 13
Self-reported outcomes ..................................................................................................................... 14
Change in individuals’ attitudes and behaviours ............................................................................... 15
Change in energy ............................................................................................................................... 17
Change in organisation-level outcomes ............................................................................................. 19
Inter-group comparisons ................................................................................................................... 22
References ......................................................................................................................................... 37
Note This appendix forms part of the review of the 2015 School for Health and Care Radicals run by NHS Improving Quality. The review was conducted by the research arm of the CIPD between November 2014 and September 2015. The main report, which presents an overview of the findings, is also available.
3 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Introduction The launch of the School for Health and Care Radicals (SHCR) in 2014 was met with strong enthusiasm by the learners within the NHS and abroad, with a number noting positive changes in their attitudes to change, and ability to lead radical change in their organisations. Although students’ reflections provided some evidence on the value added by SHCR, the impact of the School was not measured systematically in its first iteration. So, one of the aims of evaluating SHCR in 2015 was to quantify a range of outcomes at individual, community, and organisational level, that result from learners’ participation in the School.
To measure the impact of the School the CIPD conducted two surveys – one immediately before the School started and one 6 months later. This approach aimed to:
• Assist in refining the model of SHCR impact (see Appendix 2). • Collect feedback on learners’ experience of participating in the School. • Compare individual-level outcomes before and after learners took part in the School. • Compare organisation-level outcomes before and after learners took part in the
School. • Appreciate the link between change in individuals’ behaviours and attitudes and
change in organisation-level outcomes.
This document outlines the methodology of measuring the impact of SHCR 2015 and presents the findings of the two surveys.
Methodology As the main purpose of this research was to evaluate the change in individual- and organisation-level outcomes as a result of taking part in the School of Health Care Radicals (SHCR), the evaluation survey relied on a “before-and-after” design. This approach (also referred to as quasi-experimental design) is a common technique for understanding the effect of an intervention, such as taking part in a course (Rossi et al., 2004).
Gathering survey data before students started the School helped establish a baseline of their attitudes and behaviours, as well as their experiences of leading change in their organisation. The second wave of the survey was conducted 6 months later to allow individuals time to develop their behaviours and implement change in their organisations. The responses to this wave then gave us an understanding of the incremental change since the start of the School.
One of the challenges of the “before-and-after” design is in distinguishing the impact associated with the intervention itself from any natural changes that can happen in individuals’ behaviours and attitudes, as well as in organisational environments over time. To differentiate between the two, a parallel “control” group is sometimes formed (Rossi et al, 2004). Members of the control group do not take part in the intervention but their behaviours are measured for between-group comparisons. However, drawing a control group for this evaluation was difficult, as individuals only contacted the School with the purpose of taking part, so the population of “non-learners” was not accessible. Even where individuals did not take part in the School, despite signing up for it, they were no longer engaged with SHCR enough to be encouraged to take part in the evaluation.
Although the “before-and-after” measures were the core focus of the impact evaluation, the second wave of the survey also assisted in collecting some after-only measures, for example, feedback on the learning experience. Similarly, the first wave of the survey was used to refine the evaluation model. Table 1 below summarises the purpose of the two waves of the survey.
4 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 1. SHCR evaluation objectives and survey waves
Objective Survey wave Assist in refining the model of SHCR impact (see Appendix 2). Wave 1 Collect feedback on learners’ experience of participating in the School. Wave 2 Compare individual-level outcomes before and after learners took part in the School.
Waves 1 and 2
Compare organisation-level outcomes before and after learners took part in the School.
Waves 1 and 2
Appreciate the link between change in individuals’ behaviours and attitudes and change in organisation-level outcomes.
Waves 1 and 2
Sample and data collection The survey population came from the NHS IQ database of all participants who had signed up to the SHCR by 27 January 2015. The first wave survey was a census of these participants – in other words, attempting to survey them all instead of selecting a probability sample (Harding 2006). All participants were invited to complete the first wave of the survey via an email, containing information about the survey and a link to the online questionnaire. In total 1,232 individuals were invited. A reminder requesting to complete the survey was sent within the following week to all non-respondents.
The first wave of the survey gathered 639 valid responses, with the response rate of 51.9%. Of the respondents 71.7% were female and 26.3% were male. The average age was 42.81 (see Figure 1). This sample profile is typical of the healthcare workforce (see, for example, HSCIC, 2014).
Figure 1. Age distribution of Wave 1 survey respondents.
More than half of the respondents worked for a public health care provider (52.9%) – this was also the largest group of the respondents (see Table 2). 58.5% of respondents worked for the NHS.
3.5
18.0
33.9 32.6
12.0
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
Up to 24 25-34 35-44 45-54 55 and over
5 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 2. Type of organisation respondents worked for.
Type of organisation %
Public health care provider 52.9
Public Health/Healthcare improvement 6.9
Community care provider 6.1
Other public sector organisation 5.3
Care commissioning 5.2
Health care education and training 4.5
Consultancy (e.g. change management consultancy)
2.7
Private health care provider 1.7
Health care research 1.4
Patient advocacy 1.3
Other private sector organisation 1.3
Other 10.0
Don’t know .8
The second wave survey sample was drawn from the list of individuals who were registered with SHCR 2015 on 15 June 2015. This list could include individuals who were signed up to take part in the School before it started, as well as those who joined the School later. Once again the invitation to participate in the survey was sent via an email with the link to the questionnaire. Two reminders were sent one week and three weeks after the original email. However, as some of the potential survey participants did not complete the first wave of the survey, two different invites were sent:
1. A survey repeating questions from wave 1, as well as questions about the learning experience and self-reported outcomes – sent to those who took part in Wave 1.
2. A survey only containing questions about the learning experience and self-reported outcomes – sent to those who did not take part in Wave 1.
The first invite for Wave 2 was sent to the 639 individuals who took part in Wave 1 survey, returning 113 responses, which represents a response rate of 17.7%. The data from both survey waves was matched using unique identifying numbers assigned to the respondents by the researchers prior to surveying, and communicated via the invite emails. Only the matched responses were analysed when comparing changes in individual- and organisation-level outcomes, resulting from the School.
Of the respondents who completed both waves of the survey 72.1% were female and 25.6% were male. The average age was 43.81 (see Figure 2). 58.9% of respondents worked in the NHS.
6 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Figure 2. Age distribution of “before-and-after” survey respondents.
The second invite was sent to 1,951 individuals who did not take part in Wave 1 survey, returning 65 valid responses, which represents a response rate of 3.3%. These responses were combined with the 113 of responses from the “before-and-after” sample for analysis of the learning experience and self-reported outcomes of the School. The total sample for this “after-only” analysis, therefore, was 178.
Of the respondents who completed both waves of the survey 70.9% were female and 27.8% were male. The average age was 44.37 (see Figure 3).
The respondents were invited to complete the survey on voluntary basis, and those who completed both waves of the survey were entered in a draw for one of eight non-cash prizes provided by NHS IQ.
Figure 4 below illustrates the types of questions and the samples used in the two waves of the survey.
Note on methods These methods complement the other methods used in this review, including case study research (see Appendix 5). Overall, we collected different types of data that, by the principle of triangulation, help us build a more complete picture of the impact of the SHCR. Thus, the before and after survey gives us our most rigorous assessment of the school’s impact across the board; yet while this focuses at the level of individuals’ capability as change agents, the in-depth case studies go further to investigate the types of impact the school can have at the level of how organisations function and the quality of healthcare services.
The evaluation survey emphasises the impact on individuals as change agents because this is the easiest level at which to obtain convincing evidence. By contrast, assessment of organisational impact of learning is complicated. Firstly, because the SHCR focuses on helping people lead change, the potential organisational outcomes are hugely diverse. Secondly, it is very difficult to attribute any change in the organisation outcomes to the experiences of participants in the school.Figure 3. Age distribution in the total Wave 2 survey sample.
1.8
18.2
33.6 31.8
14.5
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
Up to 24 25-34 35-44 45-54 55 and over
7 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Figure 4. Questions used in survey Waves 1 and 2
Measures Engagement with the School was assesses with a single question, where participants could select activities that they intended to take part in during the School (Wave 1), or the ones they did take part in (Wave 2).
1.2
18.7
31.6 31.0
17.5
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
Up to 24 25-34 35-44 45-54 55 and over
Individual behaviours and attitudes
Energy
Organisation-level outcomes
Individual behaviours and attitudes
Energy
Organisation-level outcomes
Self-reported impact
Feedback on the School
Group 1
Group 2
Wave 1, N= 639 Wave 2, N = 113 (before-and-after)
Wave 2, N = 178 (after-only)
8 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Feedback on the learning experience was measured with 11 questions, asking about participants satisfaction with different aspects of the School (such as quality of resources, communications and others), as well as with each of the five modules delivered in the course of the School.
Individual behaviours and attitudes towards leading change were measured with 25 statements, using a 7-point agree/disagree Likert scale. Example of statements include “I am knowledgeable about models of organisational change” and “If I can see how to make a positive change, I will do it without waiting for permission”.
Energy was measured with 16 statements developed by SHCR, using a 7-point agree/disagree Likert scale. Three components of energy were measured: spiritual energy (5 items, α=.809), psychological energy (6 items, α=.739), and social energy (5 items, α=.847). Example of statements include “I feel depleted of energy when others express doubt about change in my organisation” and “I feel safe enough to do things differently”.
Self-reported outcomes were measured with two sets of questions, assessed by respondents retrospectively. One question asked the respondents about the different actions taken as a result of the School (for example, whether they “shared learning with others in your organisation”). Another set of questions asked the participants to reflect on the extent to which the School improved a range of behaviours and attitudes (such as Openness to other people’s points of view on change, Ability to challenge, and others).
Organisational outcomes were measured with 6 statements, using a 7-point agree/disagree Likert scale. Example of statements included “In my organisation it is acceptable to challenge the status quo” and “I respond to change in a way which increases organisation performance”.
Alongside other background questions (age, gender etc) level of activity on social media networks (Twitter, LinkedIn, blogs) was measured with three multiple-choice questions: “How often do you contribute original posts/content?”, “How often do you read posts/content contributed by others?”, and “How often do you connect/have conversations with other members?”.
Several free-text questions were included. In Wave 1 survey respondents were asked to record their expectations of the School in a free text format, as well as select the activities they intend to take part in as part of the School from the list of activities provided. In Wave 2 survey individuals were also able to submit any comments on their experience of SHCR.
Results This section presents the results of each stage of the data analysis. For the discussion of the findings see the main evaluation report.
Refining the SHCR impact model The first step of the analysis was to assess the strength of the SHCR evaluation model (see Appendix 2), and to reduce the 25 items, measuring individual behaviours and attitudes towards change, into a set of individual-level outcomes enabled by participation in the School.
Principal components analysis was applied on the original 25 items to identify any emerging components of the School’s impact model. KMO measure and Bartlett’s test of sphericity, as well as exploration of between-item correlations indicated that the data was appropriate for factor reduction. Components were first extracted, based on eigenvalues 1 and above, and a rotation applied to increase factor loadings on items. Both Varimax rotation with Kaiser normalisation and Direct oblimin
9 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
rotation models were attempted. The components were not strongly correlated with each other, so a model resulting from Varimax rotation was chosen to represent factor loadings (see Table 3).
Table 3. Factor loadings
Rotated Component Matrix
Component
1 2 3 4 5
I make my work visible in ways that help others. .755
I collect, organise and share useful resources. .728
I build support for change at work. .583
I inspire others to a shared purpose. .535
I help others find solutions to challenges in leading change. .521 .479
I get people interested in change initiatives. .486
I use storytelling to inspire change. .462
I am an effective change agent. .434 .420 .418
I am very aware of my natural leadership style. .378
If I can see how to make a positive change, I will do it without waiting for permission.
.796
I encourage others to lead change without waiting for permission. .792
I mobilise others to act on change. .548 .562
I challenge traditional ways of working when I think they can be improved.
.358 .510 .460
I lead change informally, by connecting with and influencing people.
.322 .446
I receive encouragement or support from like-minded people trying to lead change.
.362 .364
I know when to use planned or emergent approaches to change. .870
I am knowledgeable about models of organisational change. .834
I am naturally motivated to improve how we work. .782
I feel a sense of purpose in my day-to-day work. .626
I reflect on my work experience to help improve practice. .428 .556
I make suggestions on new ways of thinking or working. .441 .510
I clash with colleagues about how we should work. .744
10 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
I prefer to lead change by myself rather than collaborating with others.
.655
I tend to be careless in how my behaviour or words affect others. .615
I am careful to frame what I want to say so that it has a positive impact.
.407 .357 .434
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 9 iterations.
Two items were removed from the model at this stage: “I receive encouragement or support from like-minded people trying to lead change” demonstrated poor factor loadings, and “I am an effective change agent” had similar but relatively low loadings across three components. It is likely that the first statement represented a construct different from individual behaviours and attitudes towards change, while the second statement was too general to be definitively associated with a single component. Factor loadings on other items were interpreted in the context of the School’s objectives to form five components (see Table 4).
Table 5. Five individual-level outcomes of the School
Factor Items α ICC(1) Ability to connect with others to build support for change
I make my work visible in ways that help others. I collect, organise and share useful resources. I build support for change at work. I inspire others to a shared purpose. I help others find solutions to challenges in leading change. I get people interested in change initiatives. I use storytelling to inspire change. I am very aware of my natural leadership style.
.851 .401
Rocking the boat If I can see how to make a positive change, I will do it without waiting for permission. I encourage others to lead change without waiting for permission. I mobilise others to act on change. I challenge traditional ways of working when I think they can be improved. I lead change informally, by connecting with and influencing people.
.780 .352
Knowledge about change
I know when to use planned or emergent approaches to change. I am knowledgeable about models of organisational change.
.843 .631
Sense of purpose and motivation
I am naturally motivated to improve how we work. I feel a sense of purpose in my day-to-day work. I reflect on my work experience to help improve practice. I make suggestions on new ways of thinking or
.647 .287
11 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
working. Staying in the boat I clash with colleagues about how we should
work. I prefer to lead change by myself rather than collaborating with others. I tend to be careless in how my behaviour or words affect others. I am careful to frame what I want to say so that it has a positive impact.
.595 .197
Table 6. Confirmatory factor analysis of the SHCR impact model
Chi sq. (213) = 760.413***, RMSEA = 0.066, CFI=0.900, TLI=0.881, CD=0.998
Factor Items Ability to connect with others to build support for change
I make my work visible in ways that help others (0.588) I collect, organise and share useful resources (0.515) I build support for change at work (0.713) I inspire others to a shared purpose (0.694) I help others find solutions to challenges in leading change (0.769). I get people interested in change initiatives (0.719). I use storytelling to inspire change (0.564). I am very aware of my natural leadership style (0.611).
Rocking the boat
If I can see how to make a positive change, I will do it without waiting for permission (0.380). I encourage others to lead change without waiting for permission (0.632). I mobilise others to act on change (0.857). I challenge traditional ways of working when I think they can be improved (0.658). I lead change informally, by connecting with and influencing people (0.501).
Knowledge about change
I know when to use planned or emergent approaches to change (0.822). I am knowledgeable about models of organisational change (0.888).
Sense of purpose and motivation
I am naturally motivated to improve how we work (0.482). I feel a sense of purpose in my day-to-day work (0.382). I reflect on my work experience to help improve practice (0.515). I make suggestions on new ways of thinking or working (0.698).
Staying in the boat
I clash with colleagues about how we should work (0.269). I prefer to lead change by myself rather than collaborating with others (0.208). I tend to be careless in how my behaviour or words affect others (0.356). I am careful to frame what I want to say so that it has a positive impact (0.923).
Item covariances
I make my work visible in ways that help others / I collect, organise and share useful resources (0.315) I build support for change at work / I make my work visible in ways that help others (0.226) If I can see how to make a positive change, I will do it without waiting for permission / I challenge traditional ways of working when I think they can be improved (0.221) If I can see how to make a positive change, I will do it without waiting for permission / I encourage others to lead change without waiting for permission (0.531) I challenge traditional ways of working when I think they can be improved / I make suggestions on new ways of thinking or working (0.410) I am naturally motivated to improve how we work/ I feel a sense of purpose in my day-to-day work (0.306)
12 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
I prefer to lead change by myself rather than collaborating with others / I clash with colleagues about how we should work (0.265)
Factor covariances
Ability to connect with others/Rocking the boat (0.963) Ability to connect with others/Knowledge about change (0.634) Ability to connect with others/Sense of purpose and motivation (0.896) Ability to connect with others/Staying in the boat (0.664) Rocking the boat/Knowledge about change (0.507) Rocking the boat/Sense of purpose and motivation (0.840) Rocking the boat/Staying in the boat (0.506) Knowledge about change/Sense of purpose and motivation (0.520) Knowledge about change/Staying in the boat (0.412) Sense of purpose and motivation/Staying in the boat (0.468)
13 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Engagement with the School and feedback on the SHCR experience
This section reports on the intended and actual levels of engagement with the School, as well as feedback on various aspects of the School experience.
Table 7. Level of intended engagement with the School before the start of the course (N=639)
Activity % intending to take part in activity Download the materials 92.3% Participate in the online modules 90.8% Connect with other participants 74.6% Participate in other opportunities offered by the School
66.5%
Table 8. Actual levels of engagement with the School (post-SHCR)
Engagement level – actual (N= 178)
% of those who planned to do activity before the start of the School (N=112)
Downloaded the materials
81.5 83.8% of those who said they would download the materials
Listened to the online modules LIVE and/or RECORDINGS
78.7 95.2% of those who said they would participate in the online modules
Listened to the online modules LIVE
65.2 71.2% of those who said they would participate in the online modules
Listened to the RECORDINGS of the online modules
59.0 61.5% of those who said they would participate in the online modules
Took part in the online chats during the modules
51.7 60.7% of those who said they would connect with other participants
Connected with other participants via social media
48.9 61.9% of those who said they would connect with other participants
Other 6.2
Table 9. Actions taken as a result of the School
Action % Shared learning with others in your organisation 75.8 Shared learning outside of your organisation (e.g. via social media) 43.8 Recommended the School to others 82.6 Sought other types of training for change management and leadership
40.4
Became a certified change agent 44.9 None of these 5.6
14 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 10. Satisfaction with the online modules
N Didn’t access this module
Dissatisfied or very dissatisfied
Neither satisfied nor dissatisfied
Satisfied or very satisfied
Module 1. Being a health & care radical: change starts with me 175 1.1 1.1 5.1 92.6
Module 2. Building alliances for change 174 4.6 1.1 4.6 89.7
Module 3. Rolling with resistance 170 5.3 1.2 4.1 89.4 Module 4. Making change happen 170 8.8 0.0 4.7 86.5 Module 5. Moving beyond the edge 169 13.6 0 5.3 81.1
Table 11. Satisfaction with different aspects of the SHCR experience
N Dissatisfied or very dissatisfied
Neither satisfied nor dissatisfied
Satisfied or very satisfied
Quality of additional resources (e.g. study guides) 171 0.6 4.1 95.3 Quality of community activities (e.g. live chats, Twitter, Facebook) 155 2.6 16.1 81.3
Technical aspects of the webinar platform 167 4.8 17.5 77.7 Timing of the modules (i.e. Friday morning UK time) 173 16.2 22.2 61.7
The way the School was run 128 0.0 4.0 96.0 The certification process 171 6.3 14.1 79.7
Self-reported outcomes In the second wave of the survey the participants were asked to retrospectively assess whether they had experienced change in any individual-level outcomes as a result of the School. Table 11 below presents participants’ responses against ten types of such outcomes.
Table 12. As a result of the School, to what extent has the following increased?
Not at all To little extent
To some extent
To a great extent
Your sense of purpose and motivation to improve practice 4.5 6.2 37.6 50.6
Your knowledge about organisational change 3.4 12.4 48.3 34.3 Your ability to challenge 6.2 19.1 48.9 23.6 Your ability to inspire and build support for change 6.2 19.1 48.9 23.6
Your ability to maintain relationships and keep people onside 4.5 10.7 51.7 29.8 Your perseverance and resilience 3.9 21.3 41.6 30.3 Your sense of being supported by others 5.1 17.4 38.2 37.1 How often you seek other people’s views on change or the need for change 8.4 16.3 43.8 28.7
15 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Your openness to other people’s points of view on change 5.6 20.8 45.5 24.7
Your confidence talking about the need for change 5.1 15.2 36.0 41.0
Change in individuals’ attitudes and behaviours Individual-level outcomes were measured in both survey waves. This section presents the results by individual item, as well as by component, identified in the first phase of the analysis.
Table 13. Individual behaviours and attitudes, before and after taking part in the School, % of respondents agreeing/disagreeing
Statement Wave 1 (N=638) Wave 2 (N=178)
% agreeing or strongly agreeing
% neither agreeing nor disagreeing
% disagreeing or strongly disagreeing
% agreeing or strongly agreeing
% neither agreeing nor disagreeing
% disagreeing or strongly disagreeing
I am naturally motivated to improve how we work.
98.4 .5 .9 100.0 0.0 0.0
I feel a sense of purpose in my day-to-day work.
91.5 4.7 3.3 93.8 2.8 3.4
I am knowledgeable about models of organisational change.
64.6 16.0 18.6 92.7 6.2 1.1
I know when to use planned or emergent approaches to change.
41.2 22.1 34.0 81.9 13.0 5.1
I reflect on my work experience to help improve practice.
97.0 2.0 .9 98.3 .6 1.1
I am very aware of my natural leadership style.
85.8 8.0 5.6 93.2 5.6 1.1
I tend to be careless in how my behaviour or words affect others.
9.4 7.8 82.5 6.8 5.6 87.6
I lead change informally, by connecting with and influencing people.
87.3 8.3 3.9 92.1 5.1 2.8
I prefer to lead change by myself rather than collaborating with others.
13.3 14.9 71.4 12.6 12.0 75.4
16 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
If I can see how to make a positive change, I will do it without waiting for permission.
64.5 13.5 21.6 73.3 11.4 15.3
I challenge traditional ways of working when I think they can be improved.
91.5 5.2 3.3 96.6 2.8 .6
I get people interested in change initiatives.
84.7 12.5 2.7 94.9 4.5 .6
I make suggestions on new ways of thinking or working.
95.9 3.1 .9 98.9 .6 .6
I clash with colleagues about how we should work.
32.7 23.3 43.7 29.5 21.6 48.9
I inspire others to a shared purpose.
79.8 15.3 3.3 89.7 7.4 2.9
I use storytelling to inspire change. 68.5 16.4 13.6
83.5 10.2 6.3
I am careful to frame what I want to say so that it has a positive impact.
81.5 13.6 4.7 94.3 2.8 2.8
I build support for change at work.
81.5 13.8 3.3 94.3 5.1 .6
I make my work visible in ways that help others.
81.7 12.2 5.0 93.2 5.1 1.7
I collect, organise and share useful resources.
89.0 8.3 2.5 96.1 2.2 1.7
I encourage others to lead change without waiting for permission.
64.5 17.7 16.6 75.0 17.0 8.0
I help others find solutions to challenges in leading change.
84.8 11.3 3.1 92.7 5.6 1.7
I mobilise others to act on change. 80.9 13.8 4.4
90.4 8.5 1.1
I receive encouragement or support from like-minded people trying to lead change.
77.0 11.6 10.8 83.1 5.6 11.3
I am an effective change agent. 62.1 26.4 9.9 84.6 13.1 2.3
17 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 14 below provides the results of the statistical comparison in the levels of individual outcomes before and after the School. Wilcoxon signed ranks test for paired samples showed significant differences in ‘before’ and ‘after’ levels for ability to connect with others to build support for change, rocking the boat, sense of purpose and motivation, knowledge about change, and responses to the statement “I am an effective change agent”. However, there were no significant differences for staying in the boat and responses to the statement “I receive encouragement of support from like-minded people trying to lead change”.
Table 14. Change in individual-level, by component, average scores (matched sample)
Outcome Before (N=113) After (N=112) Wilcoxon signed ranks test
Effect size
Mean SD Mean SD Z r=Z/√N Ability to connect with others to build support for change
5.40 .67 5.83 .62 -6.33*** 0.60
Rocking the boat 5.15 .90 5.58 .75 -5.41*** 0.51 Staying in the boat 5.09 .77 5.17 .75 -1.28 0.12 Sense of purpose and motivation 6.20 .54 6.31 .54 -1.99* 0.19 Knowledge about change 4.53 1.25 5.59 .77 -7.13*** 0.67 I receive encouragement of support from like-minded people trying to lead change
5.24 1.25 5.38 1.31 -1.521 0.14
I am an effective change agent 4.79 1.21 5.54 .92 -6.004*** 0.57 *p<0.05; **p<0.01; ***p<0.001
Change in energy Spiritual, social and psychological energy were measured in both survey waves. This section presents the results by individual item, as well as by energy scale.
Table 15. Energy, before and after taking part in the School, % of respondents agreeing/disagreeing
Statement Wave 1 (N=638) Wave 2 (N=178)
% agreeing or strongly agreeing
% neither agreeing nor disagreeing
% disagreeing or strongly disagreeing
% agreeing or strongly agreeing
% neither agreeing nor disagreeing
% disagreeing or strongly disagreeing
Social energy
I feel a sense of solidarity with those around me.
72.9 14.7 11.9 75.3 13.5 11.2
I don’t feel appreciated by others at work.
23.9 15.5 59.6 25.6 10.2 64.2
I feel isolated from others. 20.0 13.3 66.5 22.0 13.6 64.4
I feel disconnected from others.
20.2 13.0 66.7 20.8 10.1 69.1
18 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
I feel part of a group engaged in a change.
72.9 14.7 11.9 73.4 9.6 16.9
Psychological energy
I feel safe enough to do things differently.
80.9 13.8 4.4 78.1 10.7 11.2
I will be blamed if I try something new and it fails.
35.5 17.7 45.2 28.0 15.4 56.6
I sense openness about the potential to change.
57.7 19.2 22.2 68.2 14.2 17.6
I am able to keep expressing hope for change in my organisation when presented with setbacks.
80.3 10.5 6.9 92.4 4.7 2.9
I feel depleted of energy when others express doubt about change in my organisation.
45.9 16.9 35.8 44.8 14.9 40.2
I feel fearful about change in my organisation.
12.8 9.9 76.2 10.9 9.1 80.0
Spiritual energy
I am committed to our common vision for the future.
80.0 13.8 3.6 87.9 8.6 3.4
I am driven by shared values. 85.6 11.0 3.0 91.4 5.1 3.4
I am not driven by a shared purpose for change.
5.9 17.2 74.6 6.3 12.6 81.0
Current change in my organisation does not fit with my sense of purpose.
25.7 17.5 53.5 19.8 16.9 63.4
I feel current change in my organisation may conflict with my values.
24.6 15.8 57.0 20.5 15.8 63.7
Table 16 below provides the results of the statistical comparison in the levels of energy before and after the School. Wilcoxon signed ranks test for paired samples showed significant differences in ‘before’ and ‘after’ levels for social, psychological and spiritual energy.
19 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 16. Change in energy, by scale, average scores (matched sample)
Scale Before (N=113) After (N=112) Wilcoxon signed ranks test
Effect size
Mean SD Mean SD Z r=Z/√N Social energy 4.80 1.18 5.04 1.18 -3.09** 0.29 Psychological energy 4.67 .94 5.10 .89 -5.04*** 0.48 Spiritual energy 5.07 .97 5.39 1.04 -3.20** 0.30 *p<0.05; **p<0.01; ***p<0.001
Change in organisation-level outcomes Organisation-level outcomes and organisational context were measured in both survey waves. This section presents the results by individual item, as well as correlations between change in organisation-level outcomes and change in individual-level outcomes.
Table 17. Organisation-level outcomes, before and after taking part in the School, % of respondents agreeing/disagreeing
Statement Wave 1 (N=638) Wave 2 (N=178)
% agreeing or strongly agreeing
% neither agreeing nor disagreeing
% disagreeing or strongly disagreeing
% agreeing or strongly agreeing
% neither agreeing nor disagreeing
% disagreeing or strongly disagreeing
I respond to change in a way which increases organisation performance.
93.0 4.4 1.1 97.7 1.1 1.1
I would recommend care at my place of work to friends and family.
62.6 12.2 7.2 83.5 11.5 5.0
My organisation regularly reviews its performance to identify areas for improvement and change.
67.9 12.4 16.6 74.4 8.9 16.7
In my organisation it is acceptable to challenge the status quo.
56.0 14.7 26.8 65.1 16.9 18
My organisation is resistant to change.
37.9 13.8 45.9 49.4 18.8 31.8
My organisation is good at building employee commitment to its change programmes.
38.5 19.2 38.2 33.5 28.8 37.6
20 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 18 below provides the results of the statistical comparison in the levels of energy before and after the School. Wilcoxon signed ranks test for paired samples showed significant differences in ‘before’ and ‘after’ levels for four types of organisation-level outcomes highlighted in the table.
Table 18. Change in organisation-level outcomes, average scores (matched sample)
Statement Before (N=113)
After (N=112) Wilcoxon signed ranks test
Effect size
N Mean SD N Mean SD Z r=Z/√N I respond to change in a way which increases organisation performance.
101 5.96 .71 103 6.05 .73 -8.50*** 0.85
I would recommend care at my place of work to friends and family.
83 5.48 1.36 78 5.81 1.21 -6.02*** 0.68
My organisation regularly reviews its performance to identify areas for improvement and change.
99 5.02 1.42 97 5.26 1.57 -6.09*** 0.62
In my organisation it is acceptable to challenge the status quo.
98 4.44 1.57 100 4.86 1.54 -3.55*** 0.36
My organisation is resistant to change.
98 4.20 1.46 98 4.33 1.52 -0.53 0.05
My organisation is good at building employee commitment to its change programmes.
97 3.97 1.40 98 4.08 1.29 -0.42 0.04
*p<0.05; **p<0.01; ***p<0.001
Table 19 below reports Pearson r for the different pairs of individual-level and organisation-level outcomes. The change in the outcome level (the score reported after the School minus the score reported before the School) is used.
Table 19. Correlations between changes in organisation-level outcomes, and changes in individual-level outcomes (matched sample)
Key: Org1: I respond to change in a way which increases organisation performance. Org2: I would recommend care at my place of work to friends and family. Org3: My organisation regularly reviews its performance to identify areas for improvement and change. Org4: In my organisation it is acceptable to challenge the status quo. Org5: My organisation is resistant to change. Org6: My organisation is good at building employee commitment to its change programmes.
Org1 Org2 Org3 Org4 Org5 Org6
Ability to connect with others to build support for change
Pearson Correlation .435** -.133 -.004 .132 -.027 .185
Sig. (2-tailed) .000 .255 .968 .199 .797 .075
N 101 75 96 97 95 94
Rocking the boat Pearson Correlation .317** -.026 .014 .055 .027 .189
Sig. (2-tailed) .001 .826 .895 .591 .794 .068
21 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
N 101 75 96 97 95 94
Staying in the boat
Pearson Correlation .096 -.006 .088 .026 .263** .014
Sig. (2-tailed) .338 .958 .392 .802 .010 .895
N 101 75 96 97 95 94
Sense of purpose and motivation
Pearson Correlation .342** -.049 .161 .098 .104 .127
Sig. (2-tailed) .000 .677 .117 .338 .317 .224
N 101 75 96 97 95 94
Knowledge about change
Pearson Correlation .152 -.116 .208* .167 .044 .058
Sig. (2-tailed) .130 .321 .042 .103 .669 .576
N 101 75 96 97 95 94
Social energy
Pearson Correlation .106 -.027 .079 .216* .169 .139
Sig. (2-tailed) .291 .816 .447 .034 .102 .183
N 101 75 96 97 95 94
Psychological energy
Pearson Correlation .184 -.110 .170 .145 .274** .158
Sig. (2-tailed) .066 .346 .099 .155 .007 .128
N 101 75 96 97 95 94
Spiritual energy
Pearson Correlation .335** -.302** -.028 .094 .136 .050
Sig. (2-tailed) .001 .008 .784 .360 .188 .629
N 101 75 96 97 95 94
I am an effective change agent
Pearson Correlation .262** .001 .093 .115 .040 .255*
Sig. (2-tailed) .009 .994 .371 .268 .699 .014
N 98 75 94 94 94 93
I receive encouragement or support from like-minded people trying to lead change
Pearson Correlation .221* -.221 .097 .221* .041 .138
Sig. (2-tailed) .026 .057 .345 .030 .693 .186
N 101 75 96 97 95 94
*p<0.05; **p<0.01; ***p<0.001
22 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Inter-group comparisons This section reports the results of inter-group comparison between the changes in individual-level and organisation-level outcomes before and after the School. Because of the non-normal distribution of variables in the sample non-parametric tests were applied. For comparison of 2 groups this was Wilcoxon Mann-Whitney test, and for 3 and more groups - Kruskal-Wallis Test, with a follow-up Games-Howell post-hoc test appropriate for small groups with unequal variances.
In the tables below significant differences in outcomes are highlighted in bold, and further details on the average outcomes for each of the comparison groups are provided.
Age Table 20. Age groups
Age group % Up to 34 20.0% 35-44 33.6% 45+ 46.4% Total 100.0
Table 21. Kruskal-Wallis test for individual- and organisation-level outcomes by age group
Chi-Square
df Asymp. Sig.
Ability to connect with others to build support for change 1.918 2 .383
Rocking the boat 6.354 2 .042
Staying in the boat .126 2 .939
Sense of purpose and motivation 4.337 2 .114
Knowledge about change 3.070 2 .215
Social energy .619 2 .734
Psychological energy .696 2 .706
Spiritual energy .640 2 .726
I respond to change in a way which increases organisation performance. .904 2 .636
I would recommend care at my place of work to friends and family. .538 2 .764
My organisation regularly reviews its performance to identify areas for improvement and change. 1.030 2 .598
In my organisation it is acceptable to challenge the status quo. .660 2 .719
My organisation is resistant to change. .269 2 .874
23 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
My organisation is good at building employee commitment to its change programmes. .585 2 .746
Results of post-hoc test and means reported in Tables 22 and 23 below show that the significant differences in “Rocking the boat” outcome are between groups of respondents aged 35-44 and 45 and over, with the latter group experiencing least change in “Rocking the boat” since the start of the School.
Table 22. Post-hoc test for “Rocking the boat” by age group
Post-hoc test for Rocking the boat (I) Age group 1
(J) Age group 2
Mean Difference (I-J)
Std. Error Sig. 95% Confidence Interval Lower Bound Upper Bound
Up to 34 35-44 .00393 .25642 1.000 -.6228 .6307 45+ .41381 .23502 .202 -.1688 .9964
35-44 Up to 34 -.00393 .25642 1.000 -.6307 .6228 45+ .40988* .15537 .028 .0369 .7828
45+ Up to 34 -.41381 .23502 .202 -.9964 .1688 35-44 -.40988* .15537 .028 -.7828 -.0369
Table 23. Descriptive statistics for change in “Rocking the boat” levels by age group
Rocking the boat Age group Change in score (After-Before) N Std. Deviation Up to 34 .6364 22 1.03212
35-44 .6324 37 .80071 45+ .2225 51 .58935
Gender Table 24. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by gender
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 613.000 2566.000 -.702 .483
Rocking the boat 673.500 926.500 -.087 .931
Staying in the boat 617.000 2570.000 -.665 .506
Sense of purpose and motivation 650.000 2603.000 -.329 .742
Knowledge about change 642.000 895.000 -.412 .680
Social energy 583.000 2536.000 -1.012 .312
24 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Psychological energy 674.000 927.000 -.082 .935
Spiritual energy 631.000 2584.000 -.521 .602
I respond to change in a way which increases organisation performance. 426.000 657.000 -2.440 .015
I would recommend care at my place of work to friends and family. 361.500 1586.500 -.547 .585
My organisation regularly reviews its performance to identify areas for improvement and change. 575.000 806.000 -.617 .537
In my organisation it is acceptable to challenge the status quo. 621.000 874.000 -.307 .759
My organisation is resistant to change. 514.500 2284.500 -1.177 .239
My organisation is good at building employee commitment to its change programmes. 515.500 2345.500 -.979 .328
There was a significant difference between males and females’ responses to the statement “I respond to change in a way which increases organisation performance” before and after the School. The average scores reported by males have went down since the start of the School, while the scores submitted by females went up.
Table 23. Descriptive statistics for change in “I respond to change in a way which increases organisation performance” levels by gender
I respond to change in a way which increases organisation performance. Gender Change in score (After-
Before) N Std. Deviation
Female .1639 61 .87902 Male -.3333 21 .65828
Working for the NHS No significant differences in individual- and organisation-level outcomes were found between those working for the NHS, and those working for other organisations.
Table 24. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by employment in the NHS
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1208.000 2289.000 -1.834 .067
Rocking the boat 1489.500 2570.500 -.169 .866
25 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Staying in the boat 1438.500 3649.500 -.473 .636
Sense of purpose and motivation 1483.000 3694.000 -.209 .834
Knowledge about change 1354.000 2435.000 -.980 .327
Social energy 1416.500 3627.500 -.602 .547
Psychological energy 1332.000 2413.000 -1.102 .270
Spiritual energy 1492.500 3703.500 -.151 .880
I respond to change in a way which increases organisation performance. 1445.000 3590.000 -.113 .910
I would recommend care at my place of work to friends and family. 758.500 1223.500 -.075 .941
My organisation regularly reviews its performance to identify areas for improvement and change. 1333.500 3413.500 -.071 .944
In my organisation it is acceptable to challenge the status quo. 1351.000 3496.000 -.092 .926
My organisation is resistant to change. 1099.500 3179.500 -1.428 .153
My organisation is good at building employee commitment to its change programmes. 1221.500 3237.500 -.485 .628
Participation in the School as a team Only 17 out of 113 respondents (15%) participated in SHCR as a team, which is insufficient for meaningful comparisons of the outcomes experienced by the respondents in this group with the outcomes reported by those who participated on their own.
CEO distance The number of respondents in each group (by level from CEO) was too small for inter-group comparisons (under 20).
Time working in the current organisation No significant differences in individual- and organisation-level outcomes were found. Table 25. Time spent working in the current organisation Time % Up to 2 years 30.2% 2-5 years 22.6% 5-10 years 22.6% 10+ years 24.5%
26 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 26. Kruskal-Wallis test for individual- and organisation-level outcomes by time working in the current organisation
(Only those who stayed in the same organisation through both survey waves)
Chi-Square
df Asymp. Sig.
Ability to connect with others to build support for change 5.480 3 .140
Rocking the boat 4.856 3 .183
Staying in the boat 7.189 3 .066
Sense of purpose and motivation 4.324 3 .229
Knowledge about change 6.986 3 .072
Social energy 2.263 3 .520
Psychological energy .996 3 .802
Spiritual energy .719 3 .869
I respond to change in a way which increases organisation performance. 2.248 3 .522
I would recommend care at my place of work to friends and family. 2.048 3 .563
My organisation regularly reviews its performance to identify areas for improvement and change. .672 3 .880
In my organisation it is acceptable to challenge the status quo. 3.519 3 .318
My organisation is resistant to change. 2.664 3 .446
My organisation is good at building employee commitment to its change programmes. 1.665 3 .645
Time working in the current role No significant differences in individual- and organisation-level outcomes were found. Table 27. Time spent working in the current role Time % Up to 1 year 30.0% 1-2 years 20.0% 2-5 years 24.0% 5+ years 26.0%
27 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 28. Kruskal-Wallis test for individual- and organisation-level outcomes by time working in the current role (only those who stayed in the same organisation through both survey waves)
Chi-Square
df Asymp. Sig.
Ability to connect with others to build support for change .688 3 .876
Rocking the boat 2.016 3 .569
Staying in the boat 6.892 3 .075
Sense of purpose and motivation .660 3 .883
Knowledge about change 3.683 3 .298
Social energy 7.337 3 .062
Psychological energy .150 3 .985
Spiritual energy 1.663 3 .645
I respond to change in a way which increases organisation performance. 2.513 3 .473
I would recommend care at my place of work to friends and family. 2.735 3 .434
My organisation regularly reviews its performance to identify areas for improvement and change. 1.181 3 .757
In my organisation it is acceptable to challenge the status quo. 2.268 3 .519
My organisation is resistant to change. 4.599 3 .204
My organisation is good at building employee commitment to its change programmes. 1.429 3 .699
Level of engagement with the School Table 29. Summary of the levels of engagement with the School and group sizes
Engagement with the School Yes(N) No (N) Can comparisons be made?
Downloaded the materials 95 18 No Listened to modules live 80 33 Yes Took part in the online chats during the modules
68 45 Yes
Listened to the recordings of online modules 67 46 Yes Connected with other participants via social media
66 47 Yes
Listened to modules live or listened to the recordings
106 7 No
28 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Listened to modules live
Table 30. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Listened to modules live”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 923.000 1484.000 -2.429 .015
Rocking the boat 1174.000 1735.000 -.830 .407
Staying in the boat 1197.500 1758.500 -.681 .496
Sense of purpose and motivation 1223.000 1784.000 -.520 .603
Knowledge about change 1195.500 1756.500 -.697 .486
Social energy 1025.000 1586.000 -1.783 .075
Psychological energy 1072.000 4232.000 -1.481 .139
Spiritual energy 1227.000 4387.000 -.490 .624
I respond to change in a way which increases organisation performance. 1106.000 1667.000 -1.143 .253
I would recommend care at my place of work to friends and family. 627.500 2338.500 -.486 .627
My organisation regularly reviews its performance to identify areas for improvement and change. 1120.500 1648.500 -.454 .650
In my organisation it is acceptable to challenge the status quo. 1194.000 1755.000 -.189 .850
My organisation is resistant to change. 1159.500 3860.500 -.061 .952
My organisation is good at building employee commitment to its change programmes. 1138.000 3766.000 -.103 .918
Those who listened to the modules live reported a significantly greater change in their ability to connect with others to build support for change.
Table 31. Descriptive statistics for change in “Ability to connect with others to build support for change” levels by “Listened to modules live”
Ability to connect with others to build support for change I listened to modules LIVE Change in score (After-
Before) N Std. Deviation
No .2165 33 .61131 Yes .5217 79 .61863
29 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Took part in the online chats during the modules
Table 32. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Took part in the online chats during the modules”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1410.000 2445.000 -.579 .563
Rocking the boat 1427.000 3705.000 -.480 .632
Staying in the boat 1298.000 2333.000 -1.251 .211
Sense of purpose and motivation 1368.500 2403.500 -.834 .404
Knowledge about change 1391.000 3669.000 -.699 .485
Social energy 1089.500 2124.500 -2.489 .013
Psychological energy 1392.500 3670.500 -.684 .494
Spiritual energy 1458.500 3736.500 -.292 .771
I respond to change in a way which increases organisation performance. 1238.000 2273.000 -1.454 .146
I would recommend care at my place of work to friends and family. 764.000 1229.000 -.011 .991
My organisation regularly reviews its performance to identify areas for improvement and change. 1034.000 1937.000 -2.082 .037
In my organisation it is acceptable to challenge the status quo. 1136.500 2082.500 -1.576 .115
My organisation is resistant to change. 1084.000 1987.000 -1.599 .110
My organisation is good at building employee commitment to its change programmes. 1206.500 2109.500 -.659 .510
Those who took part in the online chats during the modules reported an increase in their agreement with the statement “My organisation regularly reviews its performance to identify areas for improvement and change”. On the other hand, those who didn’t take part in the online chats reported slight decrease in this outcome.
30 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 33. Descriptive statistics for change in “My organisation regularly reviews its performance to identify areas for improvement and change” levels by “Took part in the online chats during the modules”
My organisation regularly reviews its performance to identify areas for improvement and change. I took part in the online chats during the modules
Change in score (After-Before) N Std. Deviation
No -.0952 42 1.10010 Yes .3594 64 1.28936
Listened to the recordings of online modules
Table 34. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Listened to the recordings of online modules”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1377.500 3655.500 -.772 .440
Rocking the boat 1449.500 3727.500 -.346 .730
Staying in the boat 1486.000 2521.000 -.128 .898
Sense of purpose and motivation 1449.500 3727.500 -.348 .728
Knowledge about change 1410.500 3688.500 -.582 .561
Social energy 1435.000 3713.000 -.432 .666
Psychological energy 1421.000 3699.000 -.514 .607
Spiritual energy 1416.000 3694.000 -.545 .586
I respond to change in a way which increases organisation performance. 1295.000 2330.000 -1.085 .278
I would recommend care at my place of work to friends and family. 680.000 1905.000 -1.180 .238
My organisation regularly reviews its performance to identify areas for improvement and change. 1172.500 3188.500 -1.218 .223
In my organisation it is acceptable to challenge the status quo. 1208.000 3353.000 -1.037 .300
My organisation is resistant to change. 1245.500 2106.500 -.447 .655
My organisation is good at building employee commitment to its change programmes. 1255.500 2075.500 -.170 .865
31 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Connected with other participants via social media
Table 35. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Connected with other participants via social media”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1291.500 2419.500 -1.392 .164
Rocking the boat 1224.500 2352.500 -1.793 .073
Staying in the boat 1519.500 2647.500 -.047 .962
Sense of purpose and motivation 1259.000 2387.000 -1.601 .109
Knowledge about change 1435.000 3580.000 -.551 .582
Social energy 1322.500 2450.500 -1.213 .225
Psychological energy 1304.500 3449.500 -1.317 .188
Spiritual energy 1453.000 3598.000 -.440 .660
I respond to change in a way which increases organisation performance. 1184.500 2265.500 -1.856 .063
I would recommend care at my place of work to friends and family. 780.500 1341.500 -.130 .897
My organisation regularly reviews its performance to identify areas for improvement and change. 1294.000 3185.000 -.522 .602
In my organisation it is acceptable to challenge the status quo. 1350.500 2431.500 -.342 .732
My organisation is resistant to change. 1205.500 3096.500 -.907 .364
My organisation is good at building employee commitment to its change programmes. 1246.000 3016.000 -.557 .578
Actions taken as a result of the School
Table 36. Summary of the types of actions taken as a result of the School and group sizes
Outcome Yes(N) No (N) Can comparisons be made?
Shared learning with others in your organisation 89 24 Yes Shared learning outside of your organisation (e.g. via social media)
61 52 Yes
Recommended the School to others 99 14 No
32 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Sought other types of training for change management and leadership
52 61 Yes
Become a certified change agent 57 56 Yes
Shared learning with others in your organisation
Table 37. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Shared learning with others in your organisation”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 880.000 1180.000 -1.248 .212
Rocking the boat 954.000 1254.000 -.726 .468
Staying in the boat 960.000 1260.000 -.685 .493
Sense of purpose and motivation 1010.500 1310.500 -.326 .744
Knowledge about change 918.000 1218.000 -.989 .323
Social energy 722.500 1022.500 -2.373 .018
Psychological energy 1036.500 4952.500 -.139 .890
Spiritual energy 906.500 4822.500 -1.063 .288
I respond to change in a way which increases organisation performance. 857.000 1157.000 -1.349 .177
I would recommend care at my place of work to friends and family. 452.000 557.000 -.249 .803
My organisation regularly reviews its performance to identify areas for improvement and change. 719.500 950.500 -1.426 .154
In my organisation it is acceptable to challenge the status quo. 936.000 1212.000 -.236 .814
My organisation is resistant to change. 706.500 4276.500 -1.438 .150
My organisation is good at building employee commitment to its change programmes. 757.000 988.000 -.965 .335
Those who shared learning with others in their organisation reported an increase in social energy, while those who didn’t share learning reported a decrease.
33 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
Table 38. Descriptive statistics for change in “Social energy” levels by “Shared learning with others in your organisation”
Social energy Shared learning with others in your organisation
Change in score (After-Before) N Std. Deviation
No -.1292 24 1.40325 Yes .3394 88 .77401
Shared learning outside of your organisation
Table 39. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Shared learning outside of your organisation”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1298.000 2624.000 -1.505 .132
Rocking the boat 1285.000 2611.000 -1.586 .113
Staying in the boat 1513.500 3404.500 -.247 .805
Sense of purpose and motivation 1424.500 2750.500 -.774 .439
Knowledge about change 1481.500 2807.500 -.437 .662
Social energy 1464.500 2790.500 -.533 .594
Psychological energy 1550.000 3441.000 -.032 .974
Spiritual energy 1366.500 2692.500 -1.107 .268
I respond to change in a way which increases organisation performance. 1102.500 2428.500 -2.567 .010
I would recommend care at my place of work to friends and family. 706.500 1567.500 -1.259 .208
My organisation regularly reviews its performance to identify areas for improvement and change. 1196.500 2471.500 -1.339 .181
In my organisation it is acceptable to challenge the status quo. 1421.000 2696.000 -.026 .979
My organisation is resistant to change. 1337.500 2562.500 -.227 .821
My organisation is good at building employee commitment to its change programmes. 1305.500 2530.500 -.285 .776
Those who shared learning with others outside their organisation reported an increase in their agreement with the statement “I respond to change in a way which increases organisation
34 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
performance”. On the other hand, those who didn’t share learning reported slight decrease in this outcome.
Table 40. Descriptive statistics for change in “I respond to change in a way which increases organisation performance” levels by “Shared learning outside of your organisation”
I respond to change in a way which increases organisation performance. Shared learning outside of your organisation (e.g. via social media)
Change in score (After-Before) N Std. Deviation
No -.1765 51 .86501 Yes .2881 59 .81051
Sought other types of training for change management and leadership
Table 41. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Sought other types of training for change management and leadership”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1412.500 3303.500 -.836 .403
Rocking the boat 1215.000 3106.000 -1.997 .046
Staying in the boat 1225.500 3116.500 -1.940 .052
Sense of purpose and motivation 1270.000 3161.000 -1.687 .092
Knowledge about change 1269.000 3160.000 -1.692 .091
Social energy 1496.000 3387.000 -.349 .727
Psychological energy 1443.500 2769.500 -.656 .512
Spiritual energy 1534.500 2860.500 -.123 .902
I respond to change in a way which increases organisation performance. 1395.000 3225.000 -.672 .502
I would recommend care at my place of work to friends and family. 777.000 1767.000 -.412 .680
My organisation regularly reviews its performance to identify areas for improvement and change. 1365.000 3018.000 -.208 .836
In my organisation it is acceptable to challenge the status quo. 1416.000 3069.000 -.058 .954
My organisation is resistant to change. 1330.500 2605.500 -.292 .770
35 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
My organisation is good at building employee commitment to its change programmes. 1268.000 2753.000 -.555 .579
Those who sought other types of training for change management and leadership reported a greater increase in their ability to rock the boat, compared with those who didn’t seek additional training.
Table 42. Descriptive statistics for change in “Rocking the boat” levels by “Sought other types of training for change management and leadership”
Rocking the boat Sought other types of training for change management and leadership
Change in score (After-Before)
N Std. Deviation
No .3426 61 .84389 Yes .5657 51 .68356
Become a certified change agent
Table 43. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “Become a certified change agent”
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change 1252.500 2848.500 -1.836 .066
Rocking the boat 1206.000 2802.000 -2.115 .034
Staying in the boat 1272.500 2868.500 -1.730 .084
Sense of purpose and motivation 1124.000 2720.000 -2.613 .009
Knowledge about change 1319.000 2915.000 -1.464 .143
Social energy 1282.000 2878.000 -1.670 .095
Psychological energy 1326.500 2922.500 -1.408 .159
Spiritual energy 1371.000 2967.000 -1.150 .250
I respond to change in a way which increases organisation performance. 1265.500 2750.500 -1.570 .116
I would recommend care at my place of work to friends and family. 693.500 1323.500 -1.248 .212
My organisation regularly reviews its performance to identify areas for improvement and change. 1065.500 2443.500 -2.224 .026
In my organisation it is acceptable to challenge the 1396.000 2881.000 -.226 .821
36 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
status quo.
My organisation is resistant to change. 1281.000 2607.000 -.630 .529
My organisation is good at building employee commitment to its change programmes. 1137.000 2568.000 -1.452 .147
Those who became certified change agents reported greater change in their ability to rock the boat, compared with those who didn’t become certified. The former groups also reported an increase in their sense of purpose and motivation, and agreement with the statement “My organisation regularly reviews its performance to identify areas for improvement and change”. In contrast, those who didn’t become certified reported a decrease in those scores.
Table 44. Descriptive statistics for change in “Rocking the boat” levels by “Become a certified change agent”
Rocking the boat Become a certified change agent Change in score (After-
Before) N Std. Deviation
No .2705 56 .69379 Yes .6179 56 .82707
Table 45. Descriptive statistics for change in “Sense of purpose and motivation” levels by “Become a certified change agent”
Sense of purpose and motivation Become a certified change agent Change in score (After-Before) N Std. Deviation No -.0313 56 .51800 Yes .2589 56 .58768
Table 46. Descriptive statistics for change in “My organisation regularly reviews its performance to identify areas for improvement and change” levels by “Become a certified change agent”
My organisation regularly reviews its performance to identify areas for improvement and change. Become a certified change agent Change in score (After-Before) N Std. Deviation No -.0577 52 1.34912 Yes .4074 54 1.07315
Organisational context Table 47. Summary of organisational context statements and group sizes
Organisational context Agree or Strongly agree
Disagree or Strongly disagree
Can comparisons be made?
In my organisation it is acceptable to challenge the status quo.
26 34 Yes
37 | S H C R i m p a c t r e v i e w – A p p e n d i x 3
My organisation is resistant to change.
12 42 No
My organisation is good at building employee commitment to its change programmes.
15 51 No
In my organisation it is acceptable to challenge the status quo.
Table 48. Wilcoxon Mann-Whitney test for individual- and organisation-level outcomes by “In my organisation it is acceptable to challenge the status quo”
(Only those who stayed in the same organisation during both survey waves)
Mann-Whitney U
Wilcoxon W
Z Asymp. Sig. (2-tailed)
Ability to connect with others to build support for change
366.000 717.000 -.593 .553
Rocking the boat 378.000 729.000 -.404 .686
Staying in the boat 400.500 751.500 -.040 .968
Sense of purpose and motivation 347.500 698.500 -.906 .365
Knowledge about change 323.500 674.500 -1.288 .198
Social energy 362.500 858.500 -.653 .514
Psychological energy 383.000 734.000 -.321 .748
Spiritual energy 350.000 846.000 -.853 .394
References Harding, J. (2006) Census. In Jupp, V (ed) The SAGE Dictionary of Social Research Methods. London: Sage
Health and Social Care Information Centre (HSCIC) (2014). NHS Workforce: Summary of staff in the NHS: Results from September 2013 Census.
Rossi, Peter Henry; Mark W. Lipsey; Howard E. Freeman (2004). Evaluation: A Systematic Approach.