v2 canons park aggregating november 2016
TRANSCRIPT
+Bridging the knowing-doing gap
Bridging the GapCanons Park 12 November, [email protected]@gmail.comhttp://evidencebasededucationalleadership.blogspot.com
© Gary Jones
+By the end of this session I hope to have Outlined a model of evidence-based practice which
uses multiple sources of evidence Introduced the concept of ‘narrative synthesis’ to help
frame how we can aggregate different sources of evidence
Shared three tools (though only work-in-progress) to help aggregate different sources of evidence
Used an analytical tool developed in in Improvement Science to help analyse your context when making a decision about scale of implementation
+
+Evidence-based practice and four sources of evidence
+Activity 1
Red
Little or No Evidence Available
Amber
Some EvidenceAvailable
Green
ComprehensiveEvidenceAvailable
Dark Green
Secure evidence of what work for who …
Research
School Data
Stakeholder
Practitioner Expertise
RedLittle or No Evidence Available
AmberSome EvidenceAvailable
GreenComprehensiveEvidenceAvailable
Dark GreenSecure evidence of what work for who …
ResearchBackground information – expert opinion
Randomised control trialsCohort data and Case-studies
Critically appraised topics – critically appraised articles
Systematic reviews, best evidence syntheses
School DataLittle or no school data available or merely anecdotal
Some evidence available – mainly from individual departments
Relevant school-wide data available for between 1 and 2 years.
Comprehensive and relevant data available for a minimum of three to five years
StakeholderNone Some
stakeholders obtained
Views obtained from all relevant stakeholders – or representative group of stakeholders
A range of diverse often contradictory views expressed
Practitioner Expertise
NoneSome experience of something similar
Experience of ‘intervention’ in the same context
Extensive successful experience in a range of contexts
+The pyramid of evidence
+Systematic Review “. a review of the evidence on a clearly formulated
question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review.”
Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.
+Narrative synthesis
An approach to systematic review and synthesis of findings from multiple studies that relies primarily on words and text to summarise and explain the findings of the synthesis
+The main elements in a narrative synthesis Developing a theoretical models of how the
interventions work, why and for who. Develop a preliminary synthesis Exploring the relationships in the data Assessing the robustness of the synthesised product
+Three tools to help with evidence synthesis Data Tabulation Table of components Weighting of evidence
Source Reference Intervention/Action
Participants
Setting/context
Outcomes
Results Methods/quality
Other Notes
Research Jones (2016)
Content Duration Delivered
Secondary GCSEs Nos Withdrawals
Practitioner Experience
Stakeholder Views
School data
Preliminary Synthesis & Data Tabulation
Exploring relationshipsTable of components
CPD Handouts Slides One to one
supportGroup support
Information Technology
External support
External funding
Research
Practitioner Experience
Stakeholder Views
School data
+ The Weight of Evidence
Trustworthiness Transparency - Clarity of purpose Accessibility Accuracy
Purposivity Fit for purpose method
Relevance Provides relevant answers Ethical concerns
+Weight of evidence
Source Trustworthiness Appropriateness
Relevance Overall weight
RE Medium High Low LowSVSDPE
+Activity 2
+Task
In pairs describe an evidence-based decision you have recently taken
How did you aggregate the evidence on which you made the decision?
Would one of the three ‘aggregation’ tools have helped with the decision-making process - if so, which one and why?
+Learning to Improve – Three things to consider Critical assessment of available know how Organisational capacity and human capabilities The politics of change – good will and the engagement
of people
Framework for analysis of context
Participants’ will
Sizing up a contextNo Commitment
Some Commitment
Strong Commitment
Extant know-how
limited
Limited capacity
Very small-scale test
Very small-scale test
Very small-scale test
Good capacity
Very small-scale test
Very small-scale test
Small-scale test
Substantial know-how exists
Limited capacity
Very small-scale test
Small-scale test
Large-scale test
Good capacity
Small-scale test
Large-scale test
Implement
BRYK, A. S., GOMEZ, L. M., GRUNOW, A. & LEMAHIEU, P. G. 2015. Learning to improve: How America's schools can get better at getting better. p120
+Question Let’s consider an intervention you are currently looking
into introducing into your school. How would you describe the following:
external available know how to support the intervention; school capacity and human capabilities; goodwill and the engagement of participants;towards the intervention/innovations.
What level of intervention are you considering Volunteers sat 2 or 3 Departmental – Year group/Key stage Whole school.
Framework for analysis of context
Participants’ will
Sizing up a contextNo Commitment
Some Commitment
Strong Commitment
Extant know-how
limited
Limited capacity
Very small-scale test
Very small-scale test
Very small-scale test
Good capacity
Very small-scale test
Very small-scale test
Small-scale test
Substantial know-how exists
Limited capacity
Very small-scale test
Small-scale test
Large-scale test
Good capacity
Small-scale test
Large-scale test
Implement
BRYK, A. S., GOMEZ, L. M., GRUNOW, A. & LEMAHIEU, P. G. 2015. Learning to improve: How America's schools can get better at getting better. p120
+So at the end of this session I hope I have Introduced a model of evidence-based practice which
uses multiple sources of evidence Shown how the concept of ‘narrative synthesis’ to help
frame how we can aggregate different sources of evidence
Shared three tools (though only work-in-progress) to help aggregate different sources of evidence
Used an analytical tool developed in in Improvement Science to help you analyse your context when making a decision about scale of implementation
+
@[email protected]://evidencebasededucationalleadership.blogspot.com