resources for evidence-based school social work practice
DESCRIPTION
Resources for Evidence-Based School Social Work Practice. Stephanie Baus Tulane University. Workshop Outline. Overview of Evidence-Based Practice EBP definition and model EBP process (steps) Range of practice questions Resources for learning EBP skills EBP in School Social Work - PowerPoint PPT PresentationTRANSCRIPT
Resources forResources forEvidence-Based Evidence-Based
School Social Work School Social Work PracticePractice
Stephanie BausStephanie BausTulane UniversityTulane University
Workshop OutlineWorkshop Outline Overview of Evidence-Based PracticeOverview of Evidence-Based Practice
– EBP definition and model EBP definition and model – EBP process (steps)EBP process (steps)– Range of practice questionsRange of practice questions– Resources for learning EBP skillsResources for learning EBP skills
EBP in School Social WorkEBP in School Social Work– Relationship to RTI and PBSRelationship to RTI and PBS– Assessment Assessment – Interventions Interventions – Your practice questionsYour practice questions
Workshop Outline (cont.)Workshop Outline (cont.) Formulating practice questionsFormulating practice questions
– Effectiveness/preventionEffectiveness/prevention– Risk/AssessmentRisk/Assessment– DescriptiveDescriptive
Resources for locating evidenceResources for locating evidence– Systematic reviews of studiesSystematic reviews of studies– Electronic databasesElectronic databases– Search strategiesSearch strategies– Specialized sites Specialized sites – Scholarly booksScholarly books
Workshop Outline (cont.)Workshop Outline (cont.) Evidence-informed Assessment in SSWEvidence-informed Assessment in SSW
– Assessment questionsAssessment questions– Locating evidence (searching, resources) Locating evidence (searching, resources) – Critical appraisalCritical appraisal– ApplicationApplication
Evidence-Informed Intervention in SSWEvidence-Informed Intervention in SSW– Effectiveness/prevention questionsEffectiveness/prevention questions– Locating evidence (searching, resources) Locating evidence (searching, resources) – Critical appraisalCritical appraisal– ApplicationApplication– Iatrogenic interventionsIatrogenic interventions
Workshop Outline (cont.)Workshop Outline (cont.) Evaluating OutcomesEvaluating Outcomes
– Group designsGroup designs– Single subject designsSingle subject designs
Overview of Overview of Evidence-Based PracticeEvidence-Based Practice
Definition of Definition of Evidence-Based PracticeEvidence-Based Practice
““Placing the client’s benefits first, evidence-Placing the client’s benefits first, evidence-based practitioners adopt a process of based practitioners adopt a process of lifelong learning that involves continually lifelong learning that involves continually posing specific questions of direct practical posing specific questions of direct practical importance to clients, searching objectively importance to clients, searching objectively and efficiently for the current best evidence and efficiently for the current best evidence relative to each question, and taking relative to each question, and taking appropriate action guided by evidence” appropriate action guided by evidence” (Gibbs, 2003, p 6).(Gibbs, 2003, p 6).
EBP ModelEBP Model
Clinical state and
circumstances
Practitioner's Expertise
Client Preferences and actions
ResearchEvidence
Haynes, Devereaux, and Guyatt, 2002
Steps of EBP*Steps of EBP* Step 1: Convert an information need into Step 1: Convert an information need into
an an answerable practice question. answerable practice question.
Step 2: Efficiently locate the best evidence Step 2: Efficiently locate the best evidence to to answer the question. answer the question.
Step 3: Critically appraise the evidence for Step 3: Critically appraise the evidence for its its validity and usefulness. validity and usefulness.
Steps of EBP (cont.)*Steps of EBP (cont.)* Step 4: Using practice expertise to integrate Step 4: Using practice expertise to integrate
evidence with student characteristics evidence with student characteristics and school context, apply the results of and school context, apply the results of the evidence appraisal to practice. the evidence appraisal to practice.
Step 5: Evaluate the outcome of evidence-Step 5: Evaluate the outcome of evidence-basedbased action. action.
Step 6: Teach others: challenges and Step 6: Teach others: challenges and obstaclesobstacles
*Based on Sackett, et al., 1997, and Gibbs, 2003*Based on Sackett, et al., 1997, and Gibbs, 2003
Range of Practice QuestionsRange of Practice Questions Effectiveness/PreventionEffectiveness/Prevention Assessment/RiskAssessment/Risk DescriptiveDescriptive
TerminologyTerminologyEvidence-Based Practice (EBP)Evidence-Based Practice (EBP)Evidence-Informed PracticeEvidence-Informed PracticeEmpirically-Supported Empirically-Supported Interventions/TreatmentsInterventions/Treatments
Resources for Learning EBP Resources for Learning EBP SkillsSkills
Gibbs, L. E. (2003). Evidence-based practice for the helping professions. Pacific Grove, CA: Brooks/Cole-Thomson Learning.
Kelly, M. S., Raines, J. C, Stone, S. & Frey, A. (2010). School social work: An evidence-informed framework for practice. New York: Oxford University Press.
Raines, J. C. (2008). Evidence-based practice in school mental health. New York: Oxford university Press.
Rubin, A. (2008). Practitioner’s guide to using research for evidence-based practice. Hoboken, NJ: John Wiley & Sons.
Evidence-Based Practice Evidence-Based Practice in in
School Social WorkSchool Social Work
EBP in Response to Intervention EBP in Response to Intervention and Positive Behavior Support Modelsand Positive Behavior Support Models
Efficient and Reliable Assessment Efficient and Reliable Assessment Tools/ProceduresTools/Procedures– Screening to identify prevalence of problems Screening to identify prevalence of problems – Diagnostic evaluation of individual studentsDiagnostic evaluation of individual students– Monitoring of treatment fidelity and student progressMonitoring of treatment fidelity and student progress
Empirically-Supported InterventionsEmpirically-Supported Interventions– Primary Tier: core curriculum to all students/ Primary Tier: core curriculum to all students/
preventivepreventive– Secondary Tier: additional support to at-risk studentsSecondary Tier: additional support to at-risk students– Tertiary Tier: intensive support to individual studentsTertiary Tier: intensive support to individual students
Evidence-Informed School Evidence-Informed School Social Work PracticeSocial Work Practice
Recent survey of over 1600 school social Recent survey of over 1600 school social workers indicates workers indicates – Few use online databases, journals, or Few use online databases, journals, or
scholarly books to inform practice. scholarly books to inform practice. – Primary focus on interventions targeting Primary focus on interventions targeting
individual change and risk factors rather than individual change and risk factors rather than primary prevention.primary prevention. (Kelly, et (Kelly, et al., 2008).al., 2008).
Your practice questions?Your practice questions?
Resources for Resources for Formulating Practice Formulating Practice QuestionsQuestions
COPES QuestionsCOPES Questions
CClient-Orientedlient-Oriented PPracticalractical EEvidence-Searchvidence-Search
Gibbs, L. E. (2003). Evidence-based practice for the helping professions: A practical guide with integrated multimedia. Pacific Grove, CA: Brooks/Cole.
Four Features of a Well-Built QuestionClient Type and Problem:
How would I describe a group of clients of similar type?
What You Might Do:
Apply a treatment; act to prevent a problem; survey or interview clients; measure to assess a problem; screen clients to assess risk.
Alternate Course of Action:
What is the main alternative other than in the box to the left, if any?
What You Want to Accomplish:
Outcome of treatment or prevention; accurate description of client; valid measure; accurate risk estimation.
Gibbs Gibbs (2003) (2003)
COPES QuestionsClient-OrientedPracticalEvidence-Search
Research QuestionResearch Question COPES COPES QuestionQuestionIdentifies specific Identifies specific sample sample
Describes type of Describes type of client and problemclient and problem
Identifies particular Identifies particular instance of instance of intervention, intervention, including what, including what, where, when, howwhere, when, how
May describe May describe particular type(s) of particular type(s) of interventions or ask interventions or ask about “best about “best intervention”intervention”
Identifies specific Identifies specific measurement measurement instrumentsinstruments
Describes intended Describes intended outcomes, not outcomes, not measuresmeasures
Formulating Effectiveness/Formulating Effectiveness/Prevention Questions:Prevention Questions:
a)a) Comparing an intervention to no interventionComparing an intervention to no interventionb) Comparing two interventionsb) Comparing two interventionsc) What is the best intervention?c) What is the best intervention?
ExampleExampless
Client/Client/ProblemProblem
ActionAction Alternate Alternate OutcomeOutcome
Rick Rick Ager’s Ager’s question:question:
If African If African American American adolescentsadolescents
receive receive Motivational Motivational InterviewingInterviewing
or no or no intervention intervention
are the former more are the former more likely to decrease likely to decrease their use of alcohol their use of alcohol and other drugs?and other drugs?
Loretta Loretta Pyles’ Pyles’ question:question:
For clients of For clients of social service social service agenciesagencies
do hierarchical do hierarchical models of models of agency agency managementmanagement
or or collectively collectively structured structured managemenmanagement modelst models
result in greater result in greater client client empowerment?empowerment?
Lynn Lynn Pearl-Pearl-mutter’s mutter’s question:question:
For couples For couples who have who have experienced experienced one partner’s one partner’s marital affairmarital affair
what is the best therapeutic what is the best therapeutic approachapproach
to rebuild trust?to rebuild trust?
Formulating Assessment/Risk Questions:Formulating Assessment/Risk Questions:a) Comparing an assessment instrument/procedure to a) Comparing an assessment instrument/procedure to no assessment no assessmentb) Comparing two assessment instruments/proceduresb) Comparing two assessment instruments/proceduresc) What is the best assessment instrument/procedure?c) What is the best assessment instrument/procedure?
ExamplesExamples Client/Client/ProblemProblem
ActionAction Alternate Alternate OutcomeOutcome
Loretta Loretta Pyles’ Pyles’ question:question:
Among Among battered battered womenwomen
will the will the Danger Danger Assessment Assessment ScaleScale
as opposed to as opposed to no assessmentno assessment
accurately accurately predict predict lethality?lethality?
Judy Lewis’ Judy Lewis’ questionquestion
In an urban In an urban community community devastated by devastated by a natural a natural disasterdisaster
does door-to-does door-to-door contact door contact with residentswith residents
or outreach to or outreach to existing existing institutions in institutions in the area the area
a better means a better means of predicting of predicting the potential the potential viability of a viability of a given given community?community?
Eileen Eileen Ihrig’s’ Ihrig’s’ question:question:
For children in For children in cross-cultural cross-cultural settingssettings
what is the best assessment what is the best assessment
tool?tool?
to assess to assess psychosocial psychosocial well-being?well-being?
Formulating Descriptive (Quantitative) Formulating Descriptive (Quantitative) Questions:Questions:
a) Summarizing characteristics (how much, how a) Summarizing characteristics (how much, how many, what percentage, what is the average?)many, what percentage, what is the average?)b) Asking about relationships between variablesb) Asking about relationships between variables
Examples Client/Problem
Action Alternate Outcome
Cyleste Collins’ question
[If] social work students
[are surveyed] __________
what percentage endorse stereotypical attitudes about victims of domestic violence?
Rick Ager’s question
[If] people who have been affected by a natural disaster
[are surveyed]
__________do they report increased alcohol and other drug use compared to pre-disaster levels?
Formulating Descriptive (Qualitative) Formulating Descriptive (Qualitative) Questions:Questions:
a) Require narrative responsesa) Require narrative responsesb) Suggest in-depth explorations, little knownb) Suggest in-depth explorations, little known phenomena (how would they describe, what is phenomena (how would they describe, what is the process, how do they experience?) the process, how do they experience?)
Examples Client/Problem Action Alternate OutcomeLoretta Pyles’ question:
[If] women who have been battered
[are interviewed in depth]
__________how do they describe what safety means to them?
Eileen Ihrig’s question:
If children in conflict situations who were abducted and became soldiers
[are interviewed in depth]
__________
how do they describe their experiences of reintegrating into the community?
Practice ExercisePractice ExerciseDescribe a practice situation:Describe a practice situation:
Ask a practice question:Ask a practice question:
Identify question type:Identify question type:
Formulate a COPES question:Formulate a COPES question: Client/problem ActionClient/problem Action Alternative Alternative OutcomeOutcome
Resources for Locating Resources for Locating Evidence Evidence
Systematic Reviews and Systematic Reviews and Meta-analysesMeta-analyses
Campbell Collaboration (Social Work, Campbell Collaboration (Social Work, Education & Criminal Justice Education & Criminal Justice http://www.campbellcollaboration.org/
National Registry of Evidence-Based National Registry of Evidence-Based Programs and Practices, SAMHSAPrograms and Practices, SAMHSA http://www.nrepp.samhsa.gov/index.htm
What Works ClearinghouseWhat Works Clearinghouse http://www.ies.ed.gov/ncee/wwc/
On-line DatabasesOn-line Databases Subscription databases: Cochrane Subscription databases: Cochrane
collaboration, SWAB, SSAB, PsycInfo, Criminal collaboration, SWAB, SSAB, PsycInfo, Criminal Justice Ab., Sociological Ab., Medline, CINHAIL, Justice Ab., Sociological Ab., Medline, CINHAIL, etc.etc.
Free access databases: Campbell Collaboration, Free access databases: Campbell Collaboration, ERIC, Pubmed, Google Scholar, PILOTSERIC, Pubmed, Google Scholar, PILOTS
Government and professional sites: National Government and professional sites: National Center for PTSD at Center for PTSD at http://www.ptsd.va.gov/, , Information for Practice at Information for Practice at http://ifp.nyu.edu/
Search Planning WorksheetSearch Planning Worksheet COPES questionCOPES question SynonymsSynonyms ThesaurusThesaurus Search termsSearch terms Boolean logicBoolean logic
– AND, OR, NOTAND, OR, NOT– ““wildcards” (truncation)wildcards” (truncation)
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestion
SynonymSynonymss
ControlleControlled d LanguagLanguage/e/ThesauruThesaurussSearch Search TermsTerms
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestionIf If adjudicated adjudicated delinquent delinquent juvenile juvenile malesmales
complete complete an an Outward Outward Bound Bound type of type of programprogram
as as opposed opposed to no to no programprogram
will the will the former former have lower have lower rates of rates of recidivismrecidivism??
SynonymSynonymssControlleControlled d LanguagLanguage/e/ThesauruThesaurussSearch Search TermsTerms
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestionIf If adjudicated adjudicated delinquent delinquent juvenile juvenile malesmales
complete complete an an Outward Outward Bound Bound type of type of programprogram
as as opposed opposed to no to no programprogram
will the will the former former have lower have lower rates of rates of recidivismrecidivism??
SynonymSynonymss
mandated mandated court court ordered ordered youth youth adolescentadolescent
wilderness wilderness program program adventure/ adventure/ challenge challenge program program
re-arrest re-arrest reoffend reoffend
ControlleControlled d LanguagLanguage/e/ThesauruThesaurussSearch Search TermsTerms
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestionIf If adjudicated adjudicated delinquent delinquent juvenile juvenile malesmales
complete complete an Outward an Outward Bound type Bound type of programof program
as as opposed opposed to no to no programprogram
will the will the former former have lower have lower rates of rates of recidivismrecidivism??
SynonymSynonymss
mandated mandated court court ordered ordered youth youth adolescentadolescent
wilderness wilderness program program adventure/ adventure/ challenge challenge program program
re-arrest re-arrest reoffend reoffend
ControlleControlled d LanguagLanguage/e/ThesauruThesauruss
delinquencydelinquency wilderness wilderness therapy therapy outdoor outdoor educationeducation
Search Search TermsTerms
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestionIf If adjudicated adjudicated delinquent delinquent juvenile juvenile malesmales
complete complete an Outward an Outward Bound type Bound type of programof program
as as opposed opposed to no to no programprogram
will the will the former former have lower have lower rates of rates of recidivismrecidivism??
SynonymSynonymss
mandated mandated court court ordered ordered youth youth adolescentadolescent
wilderness wilderness program program adventure/ adventure/ challenge challenge program program
re-arrest re-arrest reoffend reoffend
ControlleControlled d LanguagLanguage/e/ThesauruThesauruss
delinquencydelinquency wilderness wilderness therapy therapy outdoor outdoor educationeducation
Search Search TermsTerms
(delinquen* (delinquen* or or adjudicated adjudicated or or mandated) mandated) (juvenile* (juvenile* or youth* or or youth* or adolescen*)adolescen*)
““Outward Outward Bound” or Bound” or wilderness wilderness or or “adventure “adventure program*” program*” or or “challenge “challenge program*”program*”
recidivism recidivism or or reoffend reoffend or re-or re-offend or offend or rearrest or rearrest or re-arrestre-arrest
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
MOLESMOLES
MMethodology ethodology OOrienting rienting LLocators for ocators for EEvidence vidence SSearchesearches
(Gibbs, 2003)(Gibbs, 2003)
Search SummarySearch Summary#1: meta-anal* or metaanal* or meta #1: meta-anal* or metaanal* or meta
anal* or systematic review* or anal* or systematic review* or synthesis of studies or study synthesis of studies or study synthesissynthesis (6,807) (6,807)
#2: delinquency prevention#2: delinquency prevention (250) (250)
#3: #1 AND #2 #3: #1 AND #2 (5) (5)
6807
250
5
meta MOLES AND delinquency prevention
Effectiveness Questions
Prevention Questions
Risk/Prognosis Questions
Assessment Questions
Description Questions (With Qualitative Studies a Subset)
Syntheses of Studies
Random*ORControlled
Clinical trial*
ORControl group*OREvaluation stud*ORStudy designORStatistical*
Significan*ORDouble-blindORPlacebo
(Random*ORControlled
Clinical trial*
ORControl Group*OREvaluation Stud*ORStudy DesignORStatistical*
Significan*ORDouble BlindORPlacebo)ANDPrevent*
(Risk AssessmentORPredictive ValidityORPredictive ValueORReceiver Operat*ORROCORSensitivityORSpecificityORFalse positive*ORFalse negative*ORPrognos*)ANDPredict*
(inter-raterORInter-observerORTrue positive*ORSpecificityORFalse Positive*ORFalse negative*ORSensitivtiyORpredict*ORReceiver operat*ORROC)AND(assess*ORdiagnos*)
(Random* Select*ORSurveyORRepresentative
Sample)
Qualitative Studies
QualitativeStudy ORQualitativeAnalysis ORContent AnalysisOR In DepthInterview* OR In-DepthInterview* ORParticipantObservation ORFocus Group*
meta-anal*ORmeta anal*ORmetaanal*ORSystematic
Review*ORsynthesis of
studiesORStudy Synthesis
Gibbs, 2003, p.100
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestionIf If adjudicated adjudicated delinquent delinquent juvenile juvenile malesmales
complete complete an Outward an Outward Bound type Bound type of programof program
as as opposed opposed to no to no programprogram
will the will the former former have lower have lower rates of rates of recidivismrecidivism??
Random* Random* or or controlled controlled clinical clinical trial* or trial* or control control group* or group* or evaluation evaluation stud* or stud* or study study design or design or statistical* statistical* significan* significan* or double or double blind or blind or placeboplacebo
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
SynonymSynonymss
mandated mandated court court ordered ordered youth youth adolescentadolescent
wilderness wilderness program program adventure/ adventure/ challenge challenge program program
re-arrest re-arrest reoffend reoffend
ControlleControlled d LanguagLanguage/e/ThesauruThesauruss
delinquencydelinquency wilderness wilderness therapy therapy outdoor outdoor educationeducation
Search Search TermsTerms
(delinquen* (delinquen* or or adjudicated adjudicated or or mandated) mandated) (juvenile* (juvenile* or youth* or or youth* or adolescen*)adolescen*)
““Outward Outward Bound” or Bound” or wilderness wilderness or or “adventure “adventure program*” program*” or or “challenge “challenge program*” program*”
or or “outdoor “outdoor educatioeducation”n”
recidivism recidivism or or reoffend reoffend or re-or re-offend or offend or rearrest or rearrest or re-arrestre-arrest
Client/Problem ActionClient/Problem Action Alternate Outcome Alternate Outcome MOLES MOLESCOPESCOPES
QuestionQuestionIf If adjudicated adjudicated delinquent delinquent juvenile juvenile malesmales
complete complete an Outward an Outward Bound type Bound type of programof program
as as opposed opposed to no to no programprogram
will the will the former former have lower have lower rates of rates of recidivismrecidivism??
meta-meta-anal* or anal* or metaanal* metaanal* or meta or meta anal or anal or systematic systematic review* or review* or synthesis synthesis of studies of studies or study or study synthesissynthesis
(adapted (adapted from from Gibbs, Gibbs, 2003, p. 2003, p. 140)140)
SynonymSynonymss
mandated mandated court court ordered ordered youth youth adolescentadolescent
wilderness wilderness program program adventure/ adventure/ challenge challenge program program
re-arrest re-arrest reoffend reoffend
ControlleControlled d LanguagLanguage/e/ThesauruThesauruss
delinquencydelinquency wilderness wilderness therapy therapy outdoor outdoor educationeducation
Search Search TermsTerms
(delinquen* (delinquen* or or adjudicated adjudicated or or mandated) mandated) (juvenile* (juvenile* or youth* or or youth* or adolescen*)adolescen*)
““Outward Outward Bound” or Bound” or wilderness wilderness or or “adventure “adventure program*” program*” or or “challenge “challenge program*” program*”
or or “outdoor “outdoor educatioeducation”n”
recidivism recidivism or or reoffend reoffend or re-or re-offend or offend or rearrest or rearrest or re-arrestre-arrest
Descriptive (quantitative) MOLESDescriptive (quantitative) MOLES
random* select*”OR
“random* sampl*”OR
surveyOR
questionnaire*OR
“representative sample”OR
“national sample”OR
“interview schedule”OR
“correlat*”
(Baus)
Revising the SearchRevising the Search Keep a search history (databases, terms, hits, Keep a search history (databases, terms, hits,
relevance)relevance) Too few hits: Broaden search, include all Too few hits: Broaden search, include all
synonyms and combinations, use controlled synonyms and combinations, use controlled language, check spelling, check Boolean language, check spelling, check Boolean structure, change databases. (Removing structure, change databases. (Removing MOLES is last resort.)MOLES is last resort.)
Too many hits: Narrow search, more specific Too many hits: Narrow search, more specific terms, add categories, check Boolean terms, add categories, check Boolean structure.structure.
Hits not relevant: Consider multiple Hits not relevant: Consider multiple meanings and edit terms, use controlled meanings and edit terms, use controlled language, use “not.”language, use “not.”
Evidence-Informed Evidence-Informed Assessment in School Social Assessment in School Social WorkWork
Evidence-Informed AssessmentEvidence-Informed Assessment
Assessment questions Resources
– Kelley. M. L., Noell, G. & Reitman, David. (2003). Practitioner’s guide to empirically based measures of school behavior. New York: Kluwer Academic Publishers.
– Fischer, J & Corcoran, K. (2007). Fischer, J & Corcoran, K. (2007). Measures for clinical Measures for clinical practice and research: A sourcebookpractice and research: A sourcebook (4 (4thth Ed.). New York: Ed.). New York: Oxford University Oxford University Press.
– www.ets.org/test_link/find_tests/
Searching
Step 3: Critically Step 3: Critically Appraising Appraising
the Evidence the Evidence
Criteria for Evaluating Criteria for Evaluating Effectiveness StudiesEffectiveness Studies
Relevance (importance to serving clients, common to Relevance (importance to serving clients, common to practice, feasible, may lead to change)practice, feasible, may lead to change)
Evidence quality (control group, randomization, attrition, Evidence quality (control group, randomization, attrition, subjects and raters “blind,” equivalent groups)subjects and raters “blind,” equivalent groups)
Statistical significance (p Statistical significance (p << .05 - unlikely due to chance) .05 - unlikely due to chance)
Impact of intervention (effect size, absolute risk Impact of intervention (effect size, absolute risk reduction, number needed to treat, number needed to reduction, number needed to treat, number needed to harm)harm)
(Armstrong, cited in Gibbs, 2003)(Armstrong, cited in Gibbs, 2003)
Meta-Analyses and Meta-Analyses and Systematic ReviewsSystematic Reviews
Differences between meta-analyses, Differences between meta-analyses, systematic reviews and narrative systematic reviews and narrative reviewsreviews
Locating meta-analyses and systematic Locating meta-analyses and systematic reviewsreviews
Critically appraising meta-analyses and Critically appraising meta-analyses and systematic reviewssystematic reviews
Criteria for Evaluating Criteria for Evaluating Descriptive StudiesDescriptive Studies
Relevance (provides important information about Relevance (provides important information about client needs, characteristics, and perceptions)client needs, characteristics, and perceptions)
Evidence quality (random selection or representative Evidence quality (random selection or representative sample, sample size, clearly written pretested sample, sample size, clearly written pretested questions, response rate, appropriate generalization)questions, response rate, appropriate generalization)
Statistical significance for relationships (p Statistical significance for relationships (p << .05 - .05 - unlikely due to chance)unlikely due to chance)
Strength of relationships/effect size Strength of relationships/effect size
Criteria for Evaluating Criteria for Evaluating Assessment/Risk StudiesAssessment/Risk Studies
Relevance (importance to clients, common to Relevance (importance to clients, common to practice, feasible, may lead to change)practice, feasible, may lead to change)
Ease of use (administration, scoring)Ease of use (administration, scoring)
Evidence quality Evidence quality – Reliability: interrater agreement and/or Reliability: interrater agreement and/or
Cronbach’s alpha Cronbach’s alpha >> .70 .70– Validity: comparison with gold standard > .70, Validity: comparison with gold standard > .70,
positive predictive value, negative predictive valuepositive predictive value, negative predictive value
(Armstrong, cited in Gibbs, 2003)(Armstrong, cited in Gibbs, 2003)
Criteria for Evaluating Criteria for Evaluating Qualitative StudiesQualitative Studies
Relevance (provides important understanding of client Relevance (provides important understanding of client experiences, perspectives, processes)experiences, perspectives, processes)
Evidence qualityEvidence quality– appropriate, clearly described methodsappropriate, clearly described methods– theoretical consistency theoretical consistency – procedures to enhance trustworthiness* (credibility, procedures to enhance trustworthiness* (credibility,
dependability, confirmability, transferability) dependability, confirmability, transferability) – systematic analysis systematic analysis – no unfounded generalizationno unfounded generalization
*Lincoln & Guba, 1985*Lincoln & Guba, 1985
Use of Checklists/Rating Use of Checklists/Rating FormsForms
Rate study qualityRate study quality
Applying explicit criteria increases Applying explicit criteria increases reliabilityreliability
Use summary score or not?Use summary score or not?
Step 4: Applying the Step 4: Applying the Evidence to Evidence to
PracticePractice
Application to PracticeApplication to Practice
Applicability to client and practice Applicability to client and practice contextcontext
No evidence is an answer – role of No evidence is an answer – role of theory, informed consenttheory, informed consent
Practice wisdom – integrating art and Practice wisdom – integrating art and science, experience, relationship, science, experience, relationship, intuition, and evidenceintuition, and evidence
Application to Specific Clients Application to Specific Clients and Agency Contextsand Agency Contexts
(Slide from Jim Barber, based on Sackett, et al., 2000)(Slide from Jim Barber, based on Sackett, et al., 2000)Application QuestionsApplication Questions Decision Aids & PromptsDecision Aids & Prompts
1. Is this case so different from those in the 1. Is this case so different from those in the literature that you cannot apply their literature that you cannot apply their
results?results?
Differences can be of: Differences can be of: Degree (quantitative) and/or Degree (quantitative) and/or Type (qualitative) Type (qualitative)
2. Is the intervention feasible in our 2. Is the intervention feasible in our setting? setting?
Potential obstacles include: Potential obstacles include: Agency policy, ethics or mandateAgency policy, ethics or mandateCost (including time)Cost (including time)Agency skillAgency skill
3. What are the potential benefits and 3. What are the potential benefits and harms harms
to our client?to our client?
Taking account of cost, ‘outcome valence’ and Taking account of cost, ‘outcome valence’ and NNT/H:NNT/H:Weigh the seriousness if you Weigh the seriousness if you dodoWeigh the seriousness if you Weigh the seriousness if you don’tdon’t
4. What are our client’s values & 4. What are our client’s values & expectations for both the outcome we expectations for both the outcome we are trying to achieve and the are trying to achieve and the intervention we are offering?intervention we are offering?
Assist the client to weigh the intervention’s:Assist the client to weigh the intervention’s:AcceptabilityAcceptabilityDesirability Desirability
5. Taking account of the above, does the 5. Taking account of the above, does the intervention need to be modified?intervention need to be modified?
Modifications can be of:Modifications can be of:Degree (length or intensity)Degree (length or intensity)Type Type
Step 5: Teaching OthersStep 5: Teaching Others
Challenges and ObstaclesChallenges and Obstacles
Misconceptions about EBPMisconceptions about EBP
Agency/colleague supportAgency/colleague support
Availability of databases/articlesAvailability of databases/articles
Potential consequences of knowledgePotential consequences of knowledge
ReferencesReferences
Gibbs, L. E. (2003). Gibbs, L. E. (2003). Evidence-based practice for the helping professions: A Evidence-based practice for the helping professions: A practical guide with integrated multimedia. practical guide with integrated multimedia. Pacific Grove, CA: Thompson Pacific Grove, CA: Thompson Brooks/Cole.Brooks/Cole.
Haynes, R. B., Devereaux, P. J. & Guyatt, G. H. (2002). Clinical expertise in Haynes, R. B., Devereaux, P. J. & Guyatt, G. H. (2002). Clinical expertise in the era of evidence-based medicine and patient choice. the era of evidence-based medicine and patient choice. Vox Sanguinis, Vox Sanguinis, 83,83, Suppl 1, 383-386. Suppl 1, 383-386.
Haynes, R. B., Sackett, D. L., Gray, J. M., Cook, D. J., & Guyatt, G. H. (1996). Haynes, R. B., Sackett, D. L., Gray, J. M., Cook, D. J., & Guyatt, G. H. (1996). Transferring evidence from research into practice: 1. The role of clinical Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions. care research evidence in clinical decisions. ACP Journal Club, 125 ACP Journal Club, 125 (3), (3), A14-A16.A14-A16.
Lincoln, I. & Guba, G. (1985). Lincoln, I. & Guba, G. (1985). Naturalistic inquiryNaturalistic inquiry. Beverly Hills, CA: Sage.. Beverly Hills, CA: Sage.Sackett, D. L., Richardson, W. S., Rosenberg, W. & Haynes, R. B. (1997). Sackett, D. L., Richardson, W. S., Rosenberg, W. & Haynes, R. B. (1997).
Evidence-based medicine: How to practice and teach EBM. Evidence-based medicine: How to practice and teach EBM. New York: New York: Churchill Livingstone.Churchill Livingstone.
Sackett, D. L., Rosenberg, W., Gray, J. M., Haynes, R. B., & Richardson, W. S., Sackett, D. L., Rosenberg, W., Gray, J. M., Haynes, R. B., & Richardson, W. S., (1996), (1996), British Medical Journal, 312British Medical Journal, 312, 71-72., 71-72.
Shlonsky, A., & Gibbs, L. (2004). Will the real evidence-based practice please Shlonsky, A., & Gibbs, L. (2004). Will the real evidence-based practice please stand up? Teaching the process of evidence-based practice to the helping stand up? Teaching the process of evidence-based practice to the helping professions. professions. Brief Treatment and Crisis Intervention, 4Brief Treatment and Crisis Intervention, 4 (2), 137-153. (2), 137-153.
Stephanie BausStephanie Baus
[email protected]@tulane.edu