identifying and assessing benchmarks in the sequence of psychology education and training

of 42/42
Identifying and Identifying and Assessing Benchmarks Assessing Benchmarks in the Sequence of in the Sequence of Psychology Education Psychology Education and Training and Training Nadya A. Fouad, Ph.D. University of Wisconsin- Milwaukee

Post on 08-Feb-2016

18 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training. Nadya A. Fouad, Ph.D. University of Wisconsin-Milwaukee. THANK YOU. - PowerPoint PPT Presentation

TRANSCRIPT

  • Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training

    Nadya A. Fouad, Ph.D.University of Wisconsin-Milwaukee

  • THANK YOUMany thanks to Cathi Grus, APA Education Directorate and Nadine Kaslow, Emory University and the members of the Benchmarks Workgroup and Assessment Workgroup for their hard work and contributions to competency movement

  • CasesStudent A: Applying for internship. Faculty in the program have had concerns about her from the middle of first year. Concerns have included ethical decision making, boundaries with clients, and poor self-reflection. Academic grades are in the A and B range, though, so the faculty vote to allow her to apply for internship. Letters are vaguely positive, because faculty cannot totally identify her strengths or challenges.

  • CasesStudent B: 2nd Year doctoral student end of 2nd year practicum. Faculty instructor has had some concerns about student, but all reports from on-site supervisor have been glowing. End of April, on-site supervisor asks for a meeting to discuss serious reservations about the student, including violations of confidentiality, seeking a personal, sexual relationship with a client, and manipulating the evaluation process.

  • CasesStudent C: 1st year doctoral student in first practicum notes in journal of fear of African-American clients. Responses to ethics vignette indicate lack of knowledge about ethics code and poor ethical decision-making. Initial consultation with supervisor indicates that student had been misrepresenting client contact.

  • Overview

    Overview of Competency MovementCompetency BenchmarksCompetency Assessment Toolkit

  • CONTEXT

  • a culture of competenceRoberts, Borden, Christiansen & Lopez (2005)

    a shift within professional psychology toward an emphasis on the acquisition and maintenance of competence as a primary goal

  • Culture of CompetenceRecent years have witnessed a burgeoning interest in a competency-based approach to student learning outcomes in professional psychologyEducational programs are expected to produce competenceProfessional credentialing bodies are expected to certify individuals as competentPolicy makers laud competenceConsumers increasingly demand it

  • Are all equal?

  • Culture of CompetenceIt is time to embrace a culture of the assessment of competencefosters learningevaluates progressassists in determining curriculum and training program effectivenessadvances the fieldprotects the public

  • Goal: Competent PsychologistsChoosing A Therapist That's Right for You!

  • A Pedagogical ShiftTraditional models of education and training focus on learning objectivesObjective: aim or goalCurriculum is designed to meet goalsCompetency models focus on outcomesOutcome: result, final state, achievementMeasurement of student learning

  • Culture of CompetenceWe can learn a lot from the efforts of our colleagues in education and the other health professions (medicine, pharmacy, nursing, dentistry)Assessment of Competence Toolbox (American College of Graduate Medical Education (ACGME) APA Guidelines for the Undergraduate Psychology Major (APA, 2007)

  • What is competence in professional psychology? What does it look like? How is it best assessed?

  • Key Competency Initiatives in Professional Psychology 1990s 2000 2002 2006CoA revises G&PModel Curriculum for Training in Counseling PsychologyAPA Ethics code revisedAPPIC Competencies Conf.ADPTC CCTC Practicum CompetenciesBEA Task Force on Assessment of CompetenceNSCPP Core Competencies

  • **Rodolfa, E. R., Bent, R. J., Eisman, E., Nelson, P. D., Rehm, L., & Ritchie, P. (2005). A cube model for competency development: Implications for psychology educators and regulators. Professional Psychology: Research and Practice, 36, 347-354.

  • Key Competency Initiatives in Professional Psychology 1990s 2000 2002 2006CoA revises G&PModel Curriculum for Training in Counseling PsychologyAPA Ethics code revisedAPPIC Competencies Conf.ADPTC CCTC Practicum CompetenciesCompetency Benchmarks Work GroupBEA Task Force on Assessment of CompetenceNSCPP Core CompetenciesCompetency Assessment Toolkit

  • Competency Cube General Consensus 12 competency areasBUT what does each competency look like across a developmental trajectory?

  • Assessment of Competency Benchmarks Work GroupChair: Nadya Fouad, Ph.D.Two-day meeting September 2006Gathered individuals knowledgeable about domains of competence32 member work group

  • GoalsBuild on existing knowledge and advance the shift to a culture of competencePromote excellence in professional education and trainingOperationalize a developmental model of competence in professional psychologyTo better inform understanding of entry level to practice in light of the APA policy on Education and Training Leading to Licensure

  • CaveatsApplies most directly to those preparing for the practice of health service provision, i.e., those who will seek licensureNot meant to be prescriptive, a tool for programs to implement if they chose and in accordance with their model of education and trainingAssessment of competence must be balanced with the primary mission of the program: education and training

  • Product of the GroupA document that delineates competency benchmarks, or measurable standards of performance, that are developmental and integrated through the sequence of professional education and training

  • How are the Benchmarks Organized?Core Foundational and Functional CompetenciesEssential Component: what are the critical elements of/what knowledge/skills/attitudes that make up this competency? Behavioral Anchor: what would it look like if you saw it (essential component)?

  • Essential Components

  • Essential Component Readiness for Entry to Practice:Consistently conducts self in a professional manner across and settings and situations

  • Behavioral Anchors

  • Behavioral AnchorsProfessionalism B. DeportmentReadiness for Entry to PracticeVerbal and nonverbal communications are appropriate to the professional context including in challenging interactions

  • Benchmarks

    Benchmarks document underwent period public comment (fall 2007)Group charged by APA Board of Educational Affairs to review comment and made recommendations (spring/summer 2008) (e.g 15 competencies)Final version went to BEA Fall 2008Manuscript in press in TEPP

  • Competency Assessment for Toolkit for Professional Psychology Chair: Nadine Kaslow, Ph.D.Six members in work groupCharge from APA Board of Educational Affairs: Develop a Toolkit for professional psychologyPurpose: Promote broader implementation of competence assessment and provide information about application of assessment methods to the assessment of competenceCoordinated effort with Benchmarks Work Group

  • Components of ToolkitBackground and IntroductionAssessment Method Fact SheetsDescriptionUse specific to core competencies, formative vs. summative, developmental levelImplementationPsychometricsStrengths/ChallengesFuture Directions

  • Assessment Methods in Toolkit360 evaluationPortfolioOSCEStructured Written & Oral ExamsCase PresentationSimulation/Role PlayCompetence Evaluation Rating FormSelf-AssessmentRatings of live or recorded performanceStandardized Client InterviewClient/Patient Process/Outcome MeasureConsumer Satisfaction SurveyEnd of Rotation Performance Review

  • CompetencyProfessionalism: DeportmentNote: 1 = very useful method, 2 = useful, 3 = potentially useful

    Annual reviewCase reviewsProcess/outcome dataRating FormsConsumer surveysOSCEPortfolio reviewsRatings PerformanceRecord reviewsSelf-assessmentSimulations/role playsStand. patient nterviewsStand. oral examsWritten exams360 Evaluations1211123122211

  • Table 1Toolkit Assessment Measures, Foundational and Functional Competencies, Formative and Summative Evaluation, and Developmental Level

    MethodOverall Broad Competencies UsefulEssential Components of Competencies UsefulFormative and Summative Evaluation (notes types of evaluation most useful for most competenciesDevelopmental Level (notes levels most useful for)360-Degree EvaluationFoundationalProfessionalismReflective practiceRelationshipsEthical and legal standards and policyInterdisciplinary systemsFunctionalSupervisionTeachingAdministrationAdvocacyFoundationalIndividual and cultural diversityFunctionalInterventionFormativeSummativeReadiness for internshipEntry level to practiceAdvanced credentialing

  • Components of Toolkit (cont.)Grid of Assessment Methods and Competencies Best Used forGlossary of TermsReference

  • Toolkit: Next StepsPlan for DisseminationOn-line, downloadable resources for education and training programs methods to assess competenceManuscript in press TEPPPresentation at education and training council meetingshttp://www.apa.org/ed/graduate/competency.html

  • Challenges?fostering an environment that supports enhancing excellence through assessment at the institutional, programmatic, and individual level; leadership to use results to strengthenTraining in difficult conversations

  • Challenges?Recognize and articulate multiple roles trainers engage in and potential impact on evaluationKnowledge of a range of assessment tools, access to resources tool kits

  • CasesStudent A: Applying for internship. Faculty in the program have had concerns about her from the middle of first year. Concerns have included ethical decision making, boundaries with clients, and poor self-reflection. Academic grades are in the A and B range, though, so the faculty vote to allow her to apply for internship. Letters are vaguely positive, because faculty cannot totally identify her strengths or challenges.

  • CasesStudent B: 2nd Year doctoral student end of 2nd year practicum. Faculty instructor has had some concerns about student, but all reports from on-site supervisor have been glowing. End of April, on-site supervisor asks for a meeting to discuss serious reservations about the student, including violations of confidentiality, seeking a personal, sexual relationship with a client, and manipulating the evaluation process.

  • CasesStudent C: 1st year doctoral student in first practicum notes in journal of fear of African-American clients. Responses to ethics vignette indicate lack of knowledge about ethics code and poor ethical decision-making. Initial consultation with supervisor indicates that student had been misrepresenting client contact.

  • Questions/CommentsPreparing.

    for many years the doctoral degree has been linked with the construct of entry-level to practice, the latter has been poorly defined at best in terms of the level of competence and nature of competencies expected. Currently, entry-level to practice is defined by documentation of completion of required coursework, including requisite number of hours of supervised training. The correlation between these measures and actual competence as a professional psychologist is unknown, and thus arguably this is a poor proxy for actual evaluation of competence. In addition, external groups such as the U.S. Department of Education, regional accrediting bodies, and regulatory bodies, are discussing incorporating rules and regulations that would measure education and training outcomes in terms of specific competencies that students acquire.