evaluating the impact of continuing professional development national cpd coordinators network...
TRANSCRIPT
Evaluating the Impact of Continuing Professional Development
National CPD coordinators network meeting1st February 2008
Gillian Brydson
"Not for the first time, Larry asked himself why he had picked a goat for a coach"
Why is evaluating CPD important
‘Over the years, a lot of good things have been done in the name of professional development. So have a lot of rotten things.
What professional developers have not done is provide evidence to document the difference between the good and the rotten.’
leading to….
‘False or exaggerated claims of success are the basis of many school reform strategies – in part because we lack better and more timely evaluations of new practices and programmes’
Consortium for Policy Research in Education (1996)
2 key questions…
What is the quality of the CPD?
Has the money and time invested in a particular CPD activity made a difference to pupils?
Why is evaluating CPD difficult?
Discussions of evaluation issues often are seen as unwelcome and unnecessary intrusions into the important work that needs to be done.
Thomas Guskey
Why is evaluating CPD difficult?
1. What is quality?
‘What is required for success in professional development is a clear and compelling vision of the improvements needed, combined with explicit ideas on the organisational characteristics and attributes necessary for success’
Guskey (1999)
Why is evaluating CPD difficult?
2. What is ‘evidence’?
Did the earth
move for you?
Behaviour
Instruction
Institution
Psychomotor
Affective Cognitive
Content
Organisation
Method
Facilities
Cost
Pupil Teacher Head Specialists Family Community
So many variables…..
What is your general impression of our
evaluation of CPD?
What types of information do you consider particularly important or meaningful in an evaluation?
(Very Quick )Group Discussion
What are the limitations of our evaluations…
Too brief documentation not evaluation
Too shallow need to probe deeper
Too quick expect too much too soon
Not a self-contained exercise Integral to self-evaluation being
undertaken by teacher, school and local authority
Should be explicit and planned from the outset
Begin with the end in mind.
Overcoming the limitations?
Begin with the end in mind…
If you don't know where you are going you will probably end up somewhere else.
The Peter Principle.
It’s hard but….
We understand what CPD is… We know the major models of
professional development… We have clear purposes and goals
for Scottish Education We agree on our outcomes…
Naomi Model
Kirkpatrick Model
1. reaction 2. learning3. behaviour4. results
Guskey Model
1. Participant reaction2. Participant learning3. Organisational support and change4. Use of new knowledge and skills5. Student learning outcomes
Tools to gather information?
Reaction – post course questionnaire
Learning – reflections / portfolio
Organisational Change – school self-
evaluation,focus groups, stakeholder surveys
Use of new knowledge and skills – interviews, observations
Pupil Learning – achievement, participation, attainment
Professional Discussion
Consider these models in providing an evaluation of the impact of a CPD programme. What questions are addressed? What tools could you use? What measure is assessed? How is the information used?
Is an evaluation framework useful?
If so, to what level?
What measures of outcome are already out there that are relevant to the evaluation of CPD?
Recommendations
Paper considered by CPD coordinators
Work taken forward by national group
When you discover that you are riding a dead horse, the best strategy is to dismount. Dakota tribe proverb
Professional Courage
However…in government, education and the corporate world,
more advanced strategies are often employed, such as:
Giving horse and rider a good bollocking Buying a stronger whip. Changing riders. Appointing a committee to study the horse. Arranging to visit other countries to see how other
cultures ride horses. Convening a dead horse productivity
improvement workshop Reclassifying the dead horse as living-impaired. Hiring outside contractors to ride the dead horse. Outsourcing the management of the dead horse.
Harnessing several dead horses together to increase speed.
Doing a productivity study to see if lighter riders would improve the dead horse's performance.
Declaring that as the dead horse does not have to be fed, it is less costly, carries lower overhead and therefore contributes substantially more to the bottom line of the economy than do some other horses.
Rewriting the expected performance requirements for all horses.
Providing additional funding and/or training to increase dead horse's performance… and the highly effective...
Promoting the dead horse to a supervisory position.
Thank you