evaluating the impact of coaching for dyslexic adults
TRANSCRIPT
Academic excellence for business and the professions
Nancy Doyle MSc. C. Psychol, AFBPsS
Is coaching a ‘reasonable adjustment’ for dyslexic adults in employment?
Background to the field of research
• 3000 dyslexic clients PA through Access to Work (Gifford 2011)• Employers obliged to make reasonable adjustment• 8% of the working population (Rose, 2009)• 20% of entrepreneurs and 1% of corporate managers (Logan,
2006)• Coaching and Assistive Tech commonly recommended by
psychologists and lay assessors• No evidence base for reasonable adjustments• Wide variations in coaching pedagogy and coach training• No reporting on ROI, longitudinal evaluation or content
benchmarking
Scoping study of the literatureStep 1 Step 2 Step 3 Step 4 Step 5
Journals and basic terms
dyslexi* orreading disabilit*
Adult orAdults or19+ added to search parameters
Excl. HE, child, student or education
Interven*;Improv*; remedial; remediation; support; treatment; achievement; success; evaluation
Additional termsEmploymentcareer
Psych InfoBusiness Source CompleteCJ AbstractsBehavioural Science collection1995+English
11,117* 2010 802 463 41**
*A sample of 100 papers revealed 61% neuro based**18 education based
Pilot Study: practitioner basedGenius Within CIC, providing 1:1 coaching to around 1000 individuals per year
1) Recorded which work related topics were raised by coachees and managers to be addressed through coaching.
Memory (92%)Organisation (82%)
Time Management (78%)
Stress management (67%)Spelling (67%)Reading (54%)
2) Compared coachee and manager ratings of performance on the above topics before and 2-4 months after coaching intervention
Coachees: t (92) = 19.35, p < .001, d = 1.94;
Line managers: t (40) = 10.72, p < .001, d = 0.85
Doyle and McDowall (2015)
Study 2: sample"Coaching: this is a partnership and more androgogical approach, in which the learner ultimately takes control of their own learning and progression. The aim is to help and increase the individuals' awareness of what they need to do to improve their performance or develop a particular skill.“
(McLoughlin & Leather, 2013)
Hampshire County Council: Over 40,000 employees covering a range of unskilled, semi-skilled, administrative, professional and managerial staff. Volunteers recruited following an short workshop and briefing from all staff grades and disciplines.
Wave 1: May 2014 – Jan 2015 Control and group coaching intervention
Wave 2: Nov 2014 – July 2015 Group coaching and 1:1 intervention
Wave 3: May 2015 – Nov 2015 1:1, group and control
Study 2: double blind control, QE design3 conditions: G1 control group; G2 group coaching; G3 1:1 coaching
Neuro-cognitive testing
Behavioural Psycho-social3 intervals:
T1 Before
T2 Immediately after
T3 3 months after
Working memory (T1) and full WAIS profile
(Weschler, 2008)
T1, T2, T3 Working memory rating scales (participant & manager rated)
(WMRS, Alloway et al., 2008)
T1, T2, T3 Individual Self-Efficacy
(Judge et al., 1998)
Working memory (T2, T3)
TOMAL 2 & WRAML 2
T1, T2, T3 Job performance(participant & manager rated)(Based on Mcloughlin & Leather,
2013)
T1, T2, T3 Job Satisfaction(control variable)
(Greenhaus, 1990)
Study 2: Working memory rating scalesexampleAdapted, adult-focused items
I need help to stay on track with activities that have lots of steps
I find group discussions difficult and can interrupt too much, or I stay quiet because I don’t know when to speakI find it hard to remember instructions
I abandon activities or get distracted before I finish I find it hard to find the ‘right’ word when asked direct questions, particularly during interviews or in busy environments. My ideas jump around from one thought to another
I have difficulty concentrating in busy environments – I prefer quiet space and smaller offices for talking and working
T1: baseline control variables
Control (n=16) 1:1 (n=14) Action Learning (n=17)
Parametric assumption
1 way ANOVA
M SD M SD M SDAge (yrs) 41 10 38 10 42 9 F (2, 45) = .837, p = .439
Gender 2M 15F 6M 9F 7M 11F Weighted towards women, more so in control group
Age left education (yrs)
19 3.7 20 2.5 19 3.8 F (2, 44) = .427, p = .655
VCIQ 99 11.7 103 11.8 103 12 F (2, 44) = .714, p = .495
WMIQ 95 14.4 92 13 91 10.6 F (2, 44) = .528 , p = .594
PRIQ 104 11.5 107 12 106 7.9 F (2, 44) = .370 p = .693
PSIQ 92 12.5 91 9.4 92 11.3 F (2, 44) = .085, p = .918
T1: baseline Control ( n = 16) 1:1 ( n = 14 ) Action Learning
( n = 17 )Parametric assumption
1 way ANOVA
M SD M SD M SD
Coachee WMRS 1.41 .53 1.40 .55 1.41 .49 F (2, 42) = .237, p = .993
Coachee performance
2.89 .43 2.86 .40 2.94 .67 F (2, 40) = .113, p = .894
Line manager WMRS
.77 .45 .98 .62 1.05 .56 F (2, 31) = .813, p = .453
Line manager performance
3.64 .68 3.25 .51 3.26 .79 F (2, 33) = 1.312, p = .283
Self efficacy 2.58 .49 2.34 .41 2.62 .70 F (2, 42) = 1.081, p = .348
Job satisfaction 3.48 .52 3.41 .53 3.14 .57 F (2, 41) = 1.665, p = .202
Study 2: results so far, Baseline – T2 – T3Coachee ratings of own work performance
Baseline T2 T32
2.2
2.4
2.6
2.8
3
3.2
3.4
3.6
3.8
4
Control1:1Group
T2 compared to baseline: groups t (15) = 5.392, p = <.001, d = 1.3T3 compared to baseline: group t (15) = 5.758, p = <.001, d = 1.53T2 compared to baseline: 1:1 t (9) = 2.380, p = .04, d = 0.73T3 compared to baseline: 1:1 N.S. too many dropouts to calculate
Between groupsF (2,26)= 5.911, p = .008
Study 2: results so far, Baseline – T2 – T3Line manager ratings of coachee work performance
Major departure from study 1: This MAY be a reflection of line manager engagement rather than a review of performance.
Baseline T2 T32
2.5
3
3.5
4
Control1:1Group
Study 2: results so far, Baseline – T2 – T3Coachee ratings of working memory behaviour impact
Baseline T2 T30.4
0.6
0.8
1
1.2
1.4
1.6
1.8
Control1:1Group
T2 compared to baseline: groups t (15) = 2.389, p = .031, d = .6T3 compared to baseline: group t (11) = 2.88, p = .015, d = .83T2 compared to baseline: 1:1 t (12) = 2.312, p = .039, d = 0.64T3 compared to baseline: 1:1 N.S. too many dropouts to calculate
Between groups 1 way ANOVA at T3F (2,25)= 4.387, p = .023
Study 2: results so far, Baseline – T2 – T3Line manager ratings of working memory behaviour impact
Baseline T2 T30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Control1:1Group
Study 2: results so far, Baseline – T2 – T3Neuro-cognitive (working memory) comparisons
Baseline T2 T3 6
6.5
7
7.5
8
8.5
9
9.5
10
Control 1:1Group
• Groups only T3 compared to baseline: t (16) = 2.161, p = <.05, d = 0.53• Not significantly different from control at end, but significantly different to 1:1!
Study 2: results so far, Baseline – T2 – T3Coachee ratings of self-efficacy
Baseline T2 T30
0.5
1
1.5
2
2.5
3
Control1:1Group
Study 2: results so far, Baseline – T2 – T3Coachee ratings of job satisfaction
Baseline T2 T32
2.5
3
3.5
4
Control1:1Group
Summary of results
• Methodological issues – sample sizes not generating enough power in study 2
• Volunteers rather than people who need coaching
Coaching in a QE design not as effective as longitudinal dyad
study. Why?
• Social Cognitive Learning Theory?• However no impact noted on self –efficacy – a more specific
self-efficacy measure required, general not sensitive enough
Group coaching demonstrating a greater effect than 1:1. Why?
• Is the ‘coaching alliance’ as vital for line managers as it is for coachees?
• Manager buy in – employee doesn’t need coachingManagers’ results insignificant
• Event the control group are improving on self ratings – why? Metacognitive influence of having a discussion with a psychologist?
Hawthorne Effect
Next questions
• It’s not generalised self-efficacy!• Work related / WM related self-efficacy? • Metacognitive awareness
What is happening in group coaching that
isn’t happening in 1:1?
• Before, during and after• Written feedback / face-to-face• Training for LMs or just feedback on coachee
What kind of manager involvement is
required?
• 4 sessions 1:1 vs 6 sessions group – reflects practice
Does the contact time make a difference?
Is coaching a reasonable adjustment for dyslexia?
TimeCost
Likely improvement on productivityMore evidence
required!