bridging the research-practice gap: teacher preparation and evidence-based education
DESCRIPTION
Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education. Ronnie Detrich Wing Institute. Goals for Today. Discuss basic characteristics and processes of evidence-based education. Review the legal and ethical basis for evidence-based education. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/1.jpg)
Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education
Ronnie DetrichWing Institute
![Page 2: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/2.jpg)
Goals for Today
• Discuss basic characteristics and processes of evidence-based education.
• Review the legal and ethical basis for evidence-based education.
• Highlight issues for higher education.
![Page 3: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/3.jpg)
October 1957
USSR launched Sputnik.
U. S. Education quickly blamed.
Modern reform efforts began.
![Page 4: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/4.jpg)
1983 A Nation at Risk
American students not performing well.
Education quickly blamed.
The Nations Report Card created.
![Page 5: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/5.jpg)
1994 Goals 2000
All students will start school ready to learn.
High school graduation rate ≥ 90%.
All students in grades 4, 8, & 12 will demonstrate competency in challenging subjects.
![Page 6: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/6.jpg)
2001 No Child Left Behind
By 2014 every student will be at grade level.
Instructional methods will be scientifically based.
Educators will be held accountable for outcomes.
![Page 7: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/7.jpg)
Everybody’s Talking
• Teacher accountability often offered as solution for education troubles.
• Numerous proposals to pay and evaluate teachers based on student performance.o Proposals usually linked to performance on high stakes
tests.
![Page 8: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/8.jpg)
Function of Accountability Systems
• Assure that all students are benefitting from educational services.
• Student performance primary source of feedback to educators about effects of interventions.
![Page 9: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/9.jpg)
Scope of the Problem
Research to Practice• Gap concern in many
disciplines.• Education is not excluded.• Scientist/Practitioner model
aimed to close the gap.
Gap or Chasm?
![Page 10: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/10.jpg)
James Lancaster first experiment demonstrating how to prevent scurvy.
1601
John Lind again experimentally demonstrated the effectiveness of citrus in preventing scurvy.
1747 1795
British Navy adopted policy to have citrus on all ships in the Royal Navy.
Scurvy in the British Royal Navy:An Example of the Research to Practice Gap
![Page 11: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/11.jpg)
Research to Practice Issues
• The lag time from efficacy research to effectiveness research to dissemination is 10-20 years. (Hoagwood, Burns & Weisz, 2002)
• Only 4 of 10 Blueprint Violence Prevention programs had the capacity to disseminate to 10+ sites in a year. (Elliott & Mihalic, 2004)
![Page 12: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/12.jpg)
How Big is the Gap?
550 named interventions for children and adolescents
BehavioralCognitive-behavioral
Empirically evaluated
Evidence-based interventions are less likely to be used than interventions for which there is no evidence or there is evidence about lack of impact.
Kazdin (2000)
![Page 13: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/13.jpg)
Goals for Evidence-based Practice in Education
• At its core the EBP movement is a consumer protection movement.o It is not about science per se.o It is a policy to use science for the benefit of
consumers.o “The ultimate goal of the ‘evidence-based movement’
is to make better use of research findings in typical service settings, to benefit consumers and society….” (Fixsen, 2008)
![Page 14: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/14.jpg)
How is Education to Increase the Use of Evidence-based Practices?
• No Child Left Behind (NCLB) requires that interventions used to improve educational performance are based on scientific research.o In NCLB there are over 100 references to scientific
research.
• Individuals with Disabilities Education Improvement Act [IDEIA] (2004) requires interventions that are scientifically based instructional practices.
![Page 15: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/15.jpg)
How is Education to Increase the Use of Evidence-based Practices?
• Specific requirements of IDEIA include:o Pre-service and professional development for all who
work with students with disabilities to ensure such personnel have the skills and knowledge necessary to improve the academic achievement and functional performance of children with disabilities, including the use of scientifically based instructional practices, to the maximum extent possible.
![Page 16: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/16.jpg)
How is Education to Increase the Use of Evidence-based Practices?
• Scientifically based early reading programs, positive behavioral interventions and supports, and early intervention services to reduce the need to label children as disabled in order to address the learning and behavioral needs of such children.
![Page 17: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/17.jpg)
How is Education to Increase the Use of Evidence-based Practices?
• The Individualized Education Program (IEP) shall include a statement of the special education and related services and supplementary aids and services, based on peer-reviewed research to the extent practicable, to be provided to the child, or on behalf of the child, and a statement of the program modifications or supports for school personnel that will be provided for the child.
![Page 18: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/18.jpg)
How is Education to Increase the Use of Evidence-based Practices?
• In determining if a child has a specific learning disability, a local education agency may use a process that determines if a child responds to a scientific, research-based intervention as part of the evaluation procedures.
![Page 19: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/19.jpg)
The Ethical Basis
• Most national psychological and educational organizations have ethical standards requiring science-based practices to address problems.o American Psychological Association Ethical Standard 2.04:
Psychologists’ work is based on the established scientific and professional knowledge of the discipline.
![Page 20: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/20.jpg)
The Ethical Basis
• National Association of School Psychologistso Standard III F 4.
School psychology faculty members and clinical or field supervisors uphold recognized standards of the profession by providing training related to high quality, responsible, and research-based school psychology services.
![Page 21: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/21.jpg)
The Ethical Basis
• National Association of School Psychologistso Standard IV 4.
School psychologists use assessment techniques, counseling and therapy procedures, consultation techniques, and other direct and indirect service methods that the profession considers to be responsible, research-based practice.
![Page 22: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/22.jpg)
The Ethical Basis
• Behavior Analyst Certification Boardo Standard 2.09a
The behavior analyst always has the responsibility to recommend scientifically supported, most effective treatment procedures. Effective treatment procedures have been validated as having both long-term and short-term benefits to clients and society.
o Standard 2.09bClients have a right to effective treatment (i.e., based on the
research literature and adapted to the individual client).
![Page 23: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/23.jpg)
Another Ethical Responsibility
• Education services are largely funded through public dollars (taxpayers dollars).
• There is an implicit assumption that the money will be spent for the public good.
• A fiduciary responsibility exists when one person or organization is charged with managing another person’s money.
![Page 24: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/24.jpg)
Another Ethical Responsibility
• The primary responsibility of a fiduciary is to act solely for the benefit of the other party.
• Being a fiduciary carries the weight of ethical conduct.
• Those of us charged with providing education services have a fiduciary responsibility to assure that the taxpayers are receiving the greatest possible return on their investment.
![Page 25: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/25.jpg)
How Do We Meet Our Fiduciary Responsibility?
• Interventions that have an evidence base are more likely to produce positive effects for students.o Does not assure positive outcomes but increases the
probability.
![Page 26: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/26.jpg)
How Do We Meet our Fiduciary Responsibility?
• The impact of a non-evidence based intervention is unknown.o Using a non-evidence-based intervention when
evidence-based interventions available may constitute unethical practice.
o Use a non-evidence based intervention should be considered research All of the safe-guards afforded research participants and their
families should be in place.o Conducting research with tax-dollars provided for
education services may constitute a violation of our fiduciary responsibility.
![Page 27: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/27.jpg)
Becoming Evidence-based
• Clearly, the intent of Congress, the U.S. Department of Education, and Office of Special Education Programs is to rely on interventions that have a scientific basis.
• Professional organizations place great value on scientific knowledge.
• What does it mean to be evidence-based?
![Page 28: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/28.jpg)
What is Evidence-based Practice?
• Evidence-based practice has its roots in medicine.o Movement has spread across major disciplines in
human services:PsychologySchool PsychologySocial WorkSpeech PathologyOccupational Therapy
![Page 29: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/29.jpg)
What Is Evidence-based Practice?
• EBP is a decision-making approach that places emphasis on evidence to:o guide decisions about which interventions to use;o evaluate the effects of an intervention.
Professional Judgment
Best available evidence
Client Values
Sackett et al (2000)
Professional Professional JudgmentJudgmentBest Available EvidenceBest Available EvidenceClient Client
ValuesValues
![Page 30: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/30.jpg)
IdentifyIdentify
ImplementImplementEvaluateEvaluate
Evidence-basedEvidence-based
InterventionIntervention
IdentifyIdentify
ImplementImplementEvaluateEvaluate
Phases of Evidence-based Intervention
![Page 31: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/31.jpg)
What is Evidence-based Education?
• The term “evidence-based” has become ubiquitous in last decade.o Often used interchangeably with empirically supported
and best practice.o No consensus about what it means.o At issue is what counts as evidence.o Federal definition emphasizes experimental methods.
Preference for randomized trials.Definition has been criticized as being positivistic.
IdentifyIdentify
![Page 32: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/32.jpg)
What Counts as Evidence?
• Ultimately, this depends on the question being asked.o Qualitative methods are best for answering social
validity questions.
• In EBP the goal is to identify causal relations between interventions and outcomes.o Experimental methods do this best.
IdentifyIdentify
![Page 33: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/33.jpg)
What Counts as Evidence?
• Even if we accept causal demonstrations to be evidence, we have no consensus.o Randomized Clinical Trials (RCT) have become the
“gold standard.”o There is controversy about the status of single subject
designs.Most frequently criticized on the basis of external validity. WWC has recently established standards for SSDs .
No well established method for calculating effect sizes.
IdentifyIdentify
![Page 34: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/34.jpg)
Distinguishing Between Evidence-based and Empirically Supported
• Evidence-based refers to practices that have been validated through a systematic review.o Can involve meta-analysis.o WWC reviews are systematic reviews.
• Empirically supported refers to practices that have received support in peer reviewed journals.o Have not been systematically reviewed to establish
strength of evidence.
IdentifyIdentify
![Page 35: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/35.jpg)
Applying Best Available Evidence Construct
• If validated intervention available adopt it.o Assuming it is appropriate to context.
• If no validated intervention select empirically-supported intervention.
• If no empirically-supported interventions develop intervention based on principles.o Principles of scientific reading.o Principles of behavior.
IdentifyIdentify
![Page 36: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/36.jpg)
Applying EBP Framework
• Professional judgment involved at all levels of selection an intervention.
• Interventions must fit competencies of those implementing.
• Interventions must be acceptable to consumers and fit values.
IdentifyIdentify
![Page 37: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/37.jpg)
How Are Evidence-based Interventions Identified?
• Identification is more than finding a study to support an intervention.
• Identification involves distilling a body of knowledge to determine the strength of evidence.
IdentifyIdentify
![Page 38: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/38.jpg)
How Are Evidence-based Interventions Identified?
• Distillation requires standards of evidence for reviewing the literature.o Standards specify:
the quantity of evidence the quality of evidence
IdentifyIdentify
![Page 39: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/39.jpg)
Continua of Evidence
Quality of the Evidence
Personal Observation
Expert Opinion
Current “Gold Standard”High Quality
Randomized Controlled Trial
Uncontrolled Studies
General Consensus
Single Case Designs
Semi-Randomized Trials
Well-conducted Clinical Studies
Quantity of the Evidence
Janet Twyman, 2007
Meta-analysis (systematic review)
Single Case Replication (Direct and Parametric)
Single Study
Various Investigations
Repeated Systematic Measures
Convergent Evidence Threshold of Evidence
IdentifyIdentify
![Page 40: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/40.jpg)
How Are Evidence-based Interventions Identified?
• Two approaches to validating interventionso Threshold approach:
Evidence must be of a specific quantity and quality before an intervention is considered evidence-based.
What Works Clearinghouse» Meets evidence standards. » Meets evidence standards with reservations.» Does not meet standards at this time.
o Hierarchy of evidence approach: Strength of evidence falls along a continuum with each level
having differential standards. National Autism Center
» Established» Emerging» Unestablished» Ineffective/Harmful
IdentifyIdentify
![Page 41: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/41.jpg)
IdentifyIdentify
![Page 42: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/42.jpg)
Evidence-based InterventionEvidence-based InterventionEvidence-based InterventionEvidence-based Intervention
IdentifyIdentify
![Page 43: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/43.jpg)
No Agreed upon Standards
How are consumers to decide?
Validated Not Validated
Standard 1
Standard 2
Intervention X
IdentifyIdentify
![Page 44: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/44.jpg)
Most likely with hierarchy approach
Most likely with threshold approach
Effective IneffectiveEff
ectiv
eIn
effec
tive
True
Positive
True
Negative
False
Positive
False
NegativeAsse
ssed
As
sess
ed
Effec
tiven
ess
Effec
tiven
ess
Actual EffectivenessActual Effectiveness
Effective IneffectiveIn
effec
tive
Effec
tive
IdentifyIdentify
![Page 45: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/45.jpg)
Choosing Between False Positives and False Negatives
• At this stage, it is better to have more false positives than false negatives.
False Negatives:Effective interventions will not be selected for implementation.
As a consequence, less likely to determine that they are actually effective.
False Positives: Progress monitoring will identify interventions that are not effective.
IdentifyIdentify
![Page 46: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/46.jpg)
Role of Higher Ed in Identifying Evidence-based Practices
• Conduct systematic reviews to identify effective practices.o Multiple organizations are have processes for validating
interventions. What Works Clearinghouse Best Evidence Encyclopedia Campbell Collaboration Center for Evidence-based Policy National Autism Center Promising Practices Network CEC developing standards for review.
![Page 47: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/47.jpg)
Role of Higher Ed in Identifying Evidence-based Practices
• Independent reviews for journals:o Evidence-based Communication Assessment and
Interventiono Browder, et al., 2006: Research on Reading Instruction for
Individuals with Significant Cognitive Disabilities, Exceptional Children.
• Expand dissemination efforts.o Specify “audience.”
Different audiences may require different methods.o Expand methods.
![Page 48: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/48.jpg)
Diffusion of InnovationRogers, Diffusion of Innovation, 2003
• Diffusion of innovation is a social process, even more than a technical matter.
• The adoption rate of innovation is a function of its compatibility with the values, beliefs, and past experiences of the individuals in the social system.
![Page 49: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/49.jpg)
Principles for Effective Diffusion:Improving the Odds (Rogers, 2003)
• Innovation has to solve a problem that is important for the “client.”
• Innovation must have relative advantage over current practice.
• It is necessary to gain support of the opinion leaders if adoption is to reach critical mass and become self-sustaining.
• Innovation must be compatible with existing values, experiences and needs of the community.
![Page 50: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/50.jpg)
Principles of Effective Diffusion:Improving the Odds
• Innovation is perceived as being simple to understand and implement.
• Innovation can be implemented on a limited basis prior to broad scale adoption.
• Results of the innovation are observable to others.
![Page 51: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/51.jpg)
Q & A
![Page 52: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/52.jpg)
Implementation: Bridging the Research-Practice Gap
• Is identifying evidence-based interventions sufficient to meet regulatory and ethical requirements?o Intent of both legal and ethical guidelines is to have
positive impact.Evidence-based interventions are assumed to give to give us
that chance.
• Identification is necessary but not sufficient to assure that intervention will be effective.
ImplementImplement
![Page 53: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/53.jpg)
What We Know
• Teachers are primary means of exposure to interventions.
• Students will not benefit from effective practices if they are not exposed to them.
• Data suggest that preparation programs are not preparing trainees to use evidence-based practices.
ImplementImplement
![Page 54: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/54.jpg)
![Page 55: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/55.jpg)
What Are Universities Teaching What Are Universities Teaching About Formative Assessment?About Formative Assessment?
Response to Intervention and Teacher Preparation, Spear-Swerling, 2008
ImplementImplement
![Page 56: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/56.jpg)
Are We Training Educators to Know What to Do?
Survey of School Psychology Directors of Training
29%
Evidence-based interventions
Knowledge
(Shernoff, Kratochwill, & Stoiber, 2003)
Training41% programs
directors
ImplementImplement
![Page 57: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/57.jpg)
Teacher Preparation Programs Under Scrutiny
By almost any standard, many if not most of the nation's 1,450 schools, colleges, and departments of education are doing a mediocre job of preparing teachers for the realities of the 21st century classroom.
Arne Duncan, Secretary of Education
ImplementImplement
![Page 58: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/58.jpg)
Teacher Preparation Programs Under Scrutiny
Teacher education is the Dodge City of the education world…it is unruly and disordered. There is no standard approach to where and how teachers should be prepared.
Levine, 2006
![Page 59: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/59.jpg)
Teacher Preparation Programs Under Scrutiny
The nation’s leading educators concede that there is presently very little empirical evidence to support the methods used to prepare the nation’s teachers
Walsh, 2006
![Page 60: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/60.jpg)
Is this really our choice?
![Page 61: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/61.jpg)
Teacher Preparation Programs Under Scrutiny
• National Council on Teacher Quality reviewing all 1400+ teacher preparation programs in U.S.o Graded on 17 standards.
Classroom management Instructional Practices in reading, math, and ELL. Field work experience. Assessment practices. Instructional Design
ImplementImplement
![Page 62: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/62.jpg)
Scientific Methods of Instruction
![Page 63: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/63.jpg)
Assessment and Data Interpretation
![Page 64: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/64.jpg)
Scientific Basis of Early Reading
![Page 65: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/65.jpg)
University Accountability
![Page 66: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/66.jpg)
Implementation: Where Good Interventions Go to Die
• Implementation is not important unless it is done with integrity.
• Evidence-based drug education programs are implemented with integrity only 19% of the time. (Hallfors & Godette, 2002)o This may be a generous estimate.o No reason to assume that other interventions fare better.
ImplementImplement
![Page 67: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/67.jpg)
Why Treatment Integrity is Important?
• Kovaleski, Gickling, Morrow, & Swank (1999)o Evaluated high vs low implementation of Instructional
Support Teams (IST).School-wide organizational change.Students benefited from IST processes only when implemented
with high fidelity. Implementing with low fidelity resulted in no better outcomes for
students than control group not exposed to IST processes.Having structures in place was not sufficient to assure high fidelity.Fidelity assessed one time per year.
ImplementImplement
![Page 68: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/68.jpg)
Why Treatment Integrity is Important?
• Horner (2005)o Effect of high fidelity vs low fidelity on office discipline
referrals.Schools that implemented with high fidelity had 25% fewer
office referrals for major rule violations than schools that did not meet fidelity criterion.
Fidelity measures taken 2 times per year.
• If we want to increase the integrity of implementation we need a different training model.
ImplementImplement
![Page 69: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/69.jpg)
OUTCOMES
(% of Participants who demonstrate knowledge, demonstrate new skills in a training setting,
and use new skills in the classroom)
TRAININGCOMPONENTS
Knowledge SkillDemonstration
Use in the Classroom
Theory and Discussion
10%
5% 0%
..+Demonstration in Training 30% 20% 0%
…+ Practice & Feedback in Training
60% 60% 5%
…+ Coaching in Classroom 95% 95% 95%
Joyce and Showers, 2002
Effects of Training
![Page 70: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/70.jpg)
Mortenson & Witt, 1998
![Page 71: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/71.jpg)
Challenges for Teacher Preparation Programs
• Adoption or adaptation?o Must we implement exactly as prescribed or can we
adjust to fit local circumstances?
• Research suggests that programs are almost always adapted.
• Presumably adapted to improve outcomes.o Some adaptation for other reasons:
Better fit teaching style.Do not like some elements of program.
ImplementImplement
![Page 72: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/72.jpg)
Challenges for Teacher Preparation Programs
• If adapted is program still research-based?o If teacher’s allowed to adapt then program more
acceptable.o Teacher’s made better adaptation of reading programs if
they were well grounded in principles. (Klingner, Vaughn, Hughes, & Arguellas, 1999).
ImplementImplement
![Page 73: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/73.jpg)
Challenges for Teacher Preparation
• Teach principles of instruction or teach specific methods of instruction.o Principles of reading: 5 elements of scientifically based
instruction.Reading Mastery contains all elements.
o Not all schools use Reading Mastery. If train to Reading Mastery then may not be skilled at
implementing other reading programs. Principles are always expressed as part of a package.
Not all packages created equally. There are more reading programs than is feasible to teach to pre-
service teachers.
ImplementImplement
![Page 74: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/74.jpg)
Challenges for Teacher Preparation
• If emphasize specific curricula:o May not be able to adapt because do not have principles
of instruction to guide decision making.
ImplementImplement
![Page 75: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/75.jpg)
Training Methods for New Teachers
• How do we increase generalization from pre-service to classroom?o Clinical training Model gaining traction.
Will require reorganization of much of training and field work experience.
How skills are taught at pre-service level. How mentors interact with trainees. How mentors are selected. How competencies of mentors are defined and evaluated. Length of training?
![Page 76: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/76.jpg)
Additional Training Content: Seven Questions Parents Should Ask and Why They Are Important
1. What is the evidence-base for this intervention?o What is known about the effects for children like mine?o What is known about effects in settings like this?
![Page 77: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/77.jpg)
Seven Questions Parents Should Ask and Why They Are Important
2. What are the other options?
• Both Picture Exchange Communication System and Sign Language are evidence-based for non-verbal children with developmental disabilities.o Usually practitioners recommend one without discussing
options.
![Page 78: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/78.jpg)
Seven Questions Parents Should Ask and Why They Are Important
3. Why is this intervention being recommended?
• A good answer describes how the recommended intervention is a good fit for a particular child.o A less desirable answer is anything that does not describe
how it is the best match for this child.
![Page 79: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/79.jpg)
Seven Questions Parents Should Ask and Why They Are Important
4. What are the risks?• All interventions have risks.
o A good answer reflects that risks have been considered and steps taken to minimize when possible.
o Both PECS and Sign Language have risks: PECS requires the child always have access to the communication
system which can be difficult to transport. Sign language requires a community of others who sign.
![Page 80: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/80.jpg)
Seven Questions Parents Should Ask and Why They Are Important
5. Do those responsible for implementing have the necessary training, skills, and knowledge to implement with adequate levels of treatment integrity?
![Page 81: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/81.jpg)
Seven Questions Parents Should Ask and Why They Are Important
6. Are the resources available to implement with adequate levels of treatment integrity?
• Most commonly cited reasons for failing to implement with integrity:o Lack of trainingo Lack of materialso Lack of time
![Page 82: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/82.jpg)
Seven Questions Parents Should Ask and Why They Are Important
7. How will the effectiveness of the intervention be evaluated?o How will the family be kept informed about effects?o When shall we meet again to review progress?
![Page 83: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/83.jpg)
Evaluation of Intervention
![Page 84: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/84.jpg)
Evaluating Evidence-based InterventionsProgress Monitoring
• Implementation of evidence-based intervention does not assure success.o Necessary to evaluate impact in local context.
No intervention will be effective for all students.Cannot predict who will benefit.
o Progress monitoring is practice-based evidence about evidence-based practices.
o Consistent with legal requirements and ethical standards.
EvaluateEvaluate
![Page 85: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/85.jpg)
Ethical Standards and Progress Monitoring
• National Association of School Psychologistso Standard IV C 1b.
Decision-making related to assessment and subsequent interventions is primarily data-based.
o Standard IV 6.School psychologists develop interventions that are appropriate to
the presenting problems and are consistent with the data collected. They modify or terminate the treatment plan when the data indicate the plan is not achieving the desired goals.
EvaluateEvaluate
![Page 86: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/86.jpg)
Ethical Standards and Progress Monitoring
• Behavior Analysis Certification Boardo Standard 4.04
The behavior analyst collects data or asks the client, client-surrogate, or designated other to collect data needed to assess progress within the program.
o Standard 4.05The behavior analyst modifies the program on the basis of data.
EvaluateEvaluate
![Page 87: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/87.jpg)
• Fundamental to IEP process.o Must report on same schedule that grades are reported in
general education.• Response to Intervention is accepted as alternative
means for determining eligibility for Learning Disability classification.o Progress monitoring is the heart of RTI.
All students routinely and systematically monitored to assure adequate progress is occurring.
Legal Requirements for Progress MonitoringEvaluateEvaluate
![Page 88: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/88.jpg)
Why Teach Progress Monitoring?
• Progress monitoring 2-5/week in math and reading:o 4 times as effective as 10% increase in per pupil spending;o 6 times as effective as voucher programs;o 64 times as effective as charter schools;o 6 times as effective as increased accountability.
EvaluateEvaluate
Yeh (2007)
![Page 89: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/89.jpg)
Why Teach Formative Assessment?
Hattie, Visible Learning, 2009 Fuchs & Fuchs, 1986
EvaluateEvaluate
![Page 90: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/90.jpg)
Evaluating Evidence-based Interventions
• Curriculum based measurement is a powerful means for evaluating impact of academic interventions.o Scores on CBM correlated with scores on high stakes test.
Can be used to predict how students will perform on state-wide tests.
EvaluateEvaluate
![Page 91: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/91.jpg)
Evidence-based Education, Progress Monitoring and Treatment Integrity
• Student data provides feedback about progress.• If we know about adequacy of treatment integrity
then can make decisions:o Adequacy of interventiono Adequacy of implementation
If implementation is inadequate then focus should be on improving educator behavior.
If implementation is adequate then focus should be on changing intervention so student can succeed.
Decisions can be made about increasing or decreasing intensity of intervention.
EvaluateEvaluate
![Page 92: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/92.jpg)
Grade Level Standard
Aim Line
Trend Line
Visual Aids Can Enhance Data Interpretation
![Page 93: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/93.jpg)
Positive NegativeHi
ghLo
w
Continue Intervention Change Intervention
Unknown reason Unknown reason• Intervention problem?
• Implementation problem?
• Other life changes?
• Unknown intervention?
• Intervention is effective?
OutcomeIn
tegr
ityPositive Negative
High
Low
EvaluateEvaluate
![Page 94: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/94.jpg)
Where are We?
• Being evidence-based is the law and it is ethical conduct; however, it is not as easy as it sounds.
• A common definition of evidence protects consumers.
• The research to practice gap limits the impact of evidence-based education.o The science of implementation is in its infancy.
EvaluateEvaluate
![Page 95: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/95.jpg)
Where are We?
• Pre-service training should change to reflect current policy.o Current methods are not producing desired outcomes.o Changes in both method of training and content.
EvaluateEvaluate
![Page 96: Bridging the Research-Practice Gap: Teacher Preparation and Evidence-based Education](https://reader033.vdocuments.mx/reader033/viewer/2022051700/56815da5550346895dcbd324/html5/thumbnails/96.jpg)
Thank You
Copies May be Downloaded at www.winginstitute.org