university of kentucky division of communication sciences ......division of communication sciences...

CSD MS Assessment Plan

1

University of Kentucky College of Health Sciences Department of Rehabilitation Sciences Division of Communication Sciences & Disorders Graduate Program

Assessment Plan for the Masters of Science in Communication Sciences & Disorders

College of Health Sciences

Spring 2016

Assessment Plan for the Masters of Science in Communication Sciences & Disorders 2

Communication Sciences & Disorders MS Assessment Plan-Graduate

1. Introduction and Program Overview The Division of Communication Sciences and Disorders (CSD) is comprised of two academic degree programs: 1)

Communication Sciences and Disorders BHS; and 2) Communication Sciences and Disorders MS. In keeping with the

standards of the American Speech-Language-Hearing Association (ASHA), the undergraduate program in Communication

Sciences and Disorders (CSD) is considered a pre-professional degree program for both speech-language pathology (SLP)

and audiology. In order to meet Kentucky licensure and American Speech-Language-Hearing Association certification

requirements, it is necessary to complete the Master’s degree for SLP certification or the clinical doctorate (AuD) to

receive certification in audiology. The Master’s degree is considered the entry-level point for professional speech-

language pathologists, while the AuD is needed to become a practicing audiologist. Undergraduate students intent on

majoring in CSD while at UK should plan on 6 years to complete both the Bachelor of Health Sciences and Master of

Science/Arts programs, and 7-8 years to complete the Bachelor of Health Sciences and the AuD in audiology. The CSD

graduate program is accredited by the Council on Academic Accreditation in Audiology and Speech-Language Pathology

(CAA).

1.1. Mission Statement The mission of the Division of Communication Sciences and Disorders is to serve the citizens of Kentucky and beyond

by preparing graduates with the knowledge and skills needed to provide quality clinical services and assume

leadership roles; by serving the community and the professions through provision of evidence-based clinical

services, continuing education, and exchange of information among practitioners in communication disorders; and

by promoting, conducting, and supporting research that contributes to the understanding and management of

persons with communication disorders.

1.2. Basic Assessment Approach Assess 3-5 learning outcomes within a 3 year cycle, using direct and indirect methods for representative courses in

the curriculum. Please see the Curriculum Map and Learning Outcomes Matrix included in this document.

1.3. Definition of Key Terms

CSD – Division of Communication Sciences & Disorders

RHB – Department of Rehabilitation Sciences

CHS – College of Health Sciences

DD – Division Director – Administrative chief of the CSD program. The CSD DD reports directly to the Chair of

RHB.

AA – CSD Administrative Associate - The AA provides assistance to the program faculty by maintaining records of

undergraduate and graduate students in the major. The AA collects artifacts and data and compiles the data into

a spreadsheet for analysis by the CSD Assessment Liaison.

CSD-AL – CSD Assessment Liaison – CSD faculty responsible for the organization, coordination, collection and

reporting of assessment data to the Office of Assessment in the CHS.

ASHA – American Speech-Language-Hearing Association – national clinical and academic accrediting

organization for the professions of Speech-Language Pathology and Audiology.



Director of Graduate Studies (DGS) - The Director of Graduate Studies (DGS) is the official representative of the

graduate program. The DGS administers the graduate program and operates all advising. The DGS reports

directly to the CSD Division Director in RHB and to the Graduate School.

Curriculum Map: A visual depiction of how learning outcomes and/or professional standards are translated into

individual courses taught within a program

Direct Evidence: Students show achievement of learning goals through performance of knowledge and skills.

Includes, but is not limited to: capstone experiences, score gains between entry and exit, portfolios, and

substantial course assignments that require performance of learning.

Indirect Evidence: Data from which you can make inferences about learning but do not demonstrate actual

learning, such as perception or comparison data. Includes, but is not limited to: surveys, focus groups, exit

interviews, grades, and institutional performance indicators.

Student Learning Outcomes (SLOs): Statements of learning expectations.

2. Assessment Oversight & Resources

2.1 College Learning Outcomes Assessment Coordinator College of Health Sciences (CHS) Office of Assessment.

2.2. Unit Assessment Coordinator Director of CSD Graduate Studies performs the duties of the unit’s assessment liaison and is responsible for

coordination of efforts with the College’s Office of Assessment.

2.3. Other Assessment Resources The CSD program administrative assistant provides clerical support to implement this Assessment Plan. In addition,

CSD faculty members provide input and support during planned CSD faculty meetings in the early segment of the

Fall semester at which time the annual student learning outcome assessment reports/improvement action plans are

reviewed and discussed.

3. Program-Level Learning Outcomes 3.1 Learning Outcomes

Students will demonstrate the knowledge needed to effectively practice as an entry-level speech-language

pathologist (SLP).

Students will construct appropriate therapeutic and evidence-based decisions based on case-based scenarios,

theoretical arguments, and practice-based concepts.

Students will design and implement assessment and treatment plans for adults and/or children serviced by SLPs.

3.2 Accreditation Standards/Outcomes by Program [if applicable]

The graduate program in Communication Sciences and Disorders is accredited by Council on Academic

Accreditation in Audiology and Speech-Language Pathology (CAA) – the national clinical and academic

accrediting organization for the professions of Speech-Language Pathology and Audiology. While the assessment



plans may be used as supporting documentation for the accreditation self-study, the processes are conducted

separately.

4. Curriculum Mapping - (I= introduce, R = reinforce, E = emphasize) The Communication Sciences and Disorders graduate program is lockstep, requiring six to eight semesters for

completion. Students may enter the master’s program as a traditional student who has a bachelor’s degree in the area

of CSD, or as a ‘pre-requisite’ student who has a bachelor’s degree in a different area. Traditional students finish the

program in 6 semesters, while pre-requisite track students take an additional two semesters. Over the last three years,

about 20% of entering students are pre-requisite students. Earned student credit hours at time of graduation for

traditional six-semester students vary between 64-70 hours depending on elective and thesis options.

SLO #1 SLO #2 SLO #3

Students will demonstrate

the knowledge needed to

effectively practice as an

entry-level speech-

language pathologist (SLP).

Students will construct

appropriate therapeutic and

evidence-based decisions based

on case-based scenarios,

theoretical arguments, and

practice-based concepts.

Students will design and

implement assessment

and treatment plans for

adults and/or children

serviced by SLPs.

Course Course Type Key: Introduced, Reinforced, Expanded

CSD 621 – Aug & Alternative Com Didactic I,R,E R,E R

CSD 647 – Lang Disord in Dev Didactic I,R,E R,E R

CSD 648 – Lang in School Age Child Didactic I,R,E R,E R

CSD 654 – Clinical Orient. Practicum I,R,E - I,R,E

CSD 657 – Clinical Pract Practicum I,R,E - I,R,E

CSD 659 – Clinical Rotation Practicum I,R,E - I,R,E

CSD 661 - Phonology Didactic I,R,E R,E R

CSD 670 – Voice Disorders Didactic I,R,E R,E R

CSD 676 – Low Incidence Comm Dis Didactic I,R,E - R

CSD 677 - Aphasia Didactic I,R,E R,E R

CSD 701 – Research in CSD Didactic I,R,E R,E -

CSD 710 – Cognitive Comm Dis Didactic I,R,E R,E R

CSD 744 – Adult Swallow & MS Didactic I,R,E R,E R

CSD 745 – Peds Swallow & MS Didactic I,R,E R,E R



5. Assessment Methods & Measures

5.1 Direct Methods/Measures

5.2 Indirect Methods/Measures

All student learning outcomes use direct measures of student learning.

6. Data Collection & Review 6.1 Data Collection Process/Procedures

6.1.1 When will data be collected for each outcome?

6.1.2 How will data be collected for each outcome?

6.1.3 What will be the benchmark/target for each outcome?

SLO #1 SLO #2 SLO #3

Students will demonstrate the knowledge

needed to effectively practice as an entry-

level speech-language pathologist (SLP).

Students will construct appropriate therapeutic and

evidence-based decisions based on case-based scenarios,

theoretical arguments, and practice-based concepts.

Students will design and implement

assessment and treatment plans for

adults and/or children serviced by SLPs.

NTE Praxis Examination (Pass Rate) Comprehensive examinations graded by a rubric and faculty assessment teams; theses will be evaluated by thesis committee members using grading rubric (Pass Rate)

Supervisory assessment form that

notes proficiency in diagnostic,

intervention, and documentation skills.

Learning Outcome

Date – CYCLE 1 Date – CYCLE 2

SLO #1 Spring 2016 Spring 2019



Learning Outcome

Collection of data

SLO #1 Compilation of NTE Praxis Examination Scores

SLO #2 Compilation of comprehensive examinations scores and faculty evaluations

SLO #3 Collection of supervisory assessment data from CALIPSO electronic clinical tracking software.

Learning Outcome

Benchmark/Target Rational for Benchmark/Target

SLO #1

80% of the students completing the graduate program will pass the Praxis national examination in Speech-Language Pathology (passing score) on the first administration.

Praxis national exam provides a uniform and consistent metric that can be used to compare the preparation of our students to national norms. 80% was chosen to maintain consistency with our program’s academic metric of a B or better.

SLO #2 95% of the students completing the graduate program will pass the comprehensive examination or successfully defend a Master's thesis.

95% benchmark was developed by the faculty based upon an historical review of comprehensive exam pass rates over the past 5 years.



6.1.4 What individuals/groups will be responsible for data collection SLO #1 - DGS w/ help from the program AA

SLO #2 – DGS w/ help from the program AA

SLO #3 – Clinical Coordinator w/ help from the program AA

7. Assessment Cycle and Data Analysis

7.1 Assessment Cycle (3 years) The CSD Program follows a 3-year assessment cycle, with one outcome being assessed in each year. Faculty in each

representative course are asked to maintain records and artifacts specific to the outcome measure specified above.

Data are compiled and analyzed in the early summer each year for the outcomes being assessed. Drafts of the

results are analyzed and interpreted by the Unit Assessment Coordinator in conjunction with the College Learning

Outcomes Assessment Coordinator in the late summer/early fall semester. Results are presented to the CSD faculty

members during the September faculty meeting of each year. Assessment reports are completed by October 1st of

every year and submitted to the College Learning Outcomes Assessment Coordinator for review. Final reports are

sent to the university’s assessment office no later than October 31st of every year.

Performance Criteria Courses Assessor/Assessment Method

Year Performed – CYCLE 1 / CYCLE 2

Students will demonstrate the knowledge needed to effectively practice as an entry-level speech-language pathologist (SLP).

Across all classes in Course Map

Deem & Olson – Praxis Scores

YEAR 1/Cycle 1 - Collect data during Spring of 2016. Report Due 10/1/16 YEAR 1/Cycle 2 - Collect data during Spring of 2019. Report Due 10/1/19

Students will construct appropriate therapeutic and evidence-based decisions based on case-based scenarios, theoretical arguments, and practice-based concepts.

Across all classes in Course Map

Deem & Olson – Comprehensive exam scores


Students will design and implement assessment and treatment plans for adults and/or children serviced by SLPs.

CSD 659 – Clinical Rotations

Morris – CALIPSO ratings from clinical supervisors


SLO #3

80% of enrolled graduate students will receive an average rating of a Level 4 – Beginning Professional on the supervisory assessment and tracking system during their last two semesters of clinical rotations.

Generally, our students are expected to reach at least a Level 4 – Beginning Professional proficiency. We expect students to demonstrate adequate competency, consistency and independence in the execution of the clinical skill.



7.2 Data Collection Process/Procedures

7.2.1 How will the data and findings be shared with faculty? Results are presented to the CSD faculty members during the September faculty meeting of each year. This

opportunity allows for faculty members to reflect on the outcomes of the annual assessment of the SLOs.

Faculty members develop strategies for addressing concerns raised during the assessment process.

7.2.2 Who was involved in analyzing the results? The AA is responsible for organizing assessment data and putting it into a workable format. The assessment is

conducted by the faculty of each assessed course in consultation with and with feedback given by the

Assessment Liaison, the DGS, and the Program Director.

7.2.3 How are results aligned to outcomes and benchmarks/targets given? When assessment results are presented to faculty for review and reflection, they are shared in comparison to

the benchmark/target for the learning outcome under review. As part of this process, the benchmark/target is

also reviewed to determine whether it should be altered or remain the same based on the assessment results.

7.2.4 How will the data be used for making programmatic improvements? Feedback reports are shared with the faculty in the spring semester upon receipt of the evaluations from the

university assessment committee. During a mid-spring faculty meeting and after adequate time to review the

feedback, the faculty identify strategies to address identified deficiencies. Faculty discuss if strategies can be

universally implemented to affect more than just the representative SLO that was analyzed in the preceding fall

semester. Changes that are needed are then planned for and conducted.

Learning Outcome

Analysis Method

SLO #1 Praxis scores are obtained from testing service. Statistically summarized.

SLO #2

Faculty teams of three use rubric (See appendix for SLO #2) to assess each question on the exam. Final grade for each question is the average score across all reviewers on a team. The decision to Pass/Fail the comprehensive exam on the whole rests on the number of questions failed. Failed questions are given the opportunity for re-write as per the graduate school rules. A final Pass Rate for the cohort is calculated after all exam attempts have been completed.

SLO #3

Clinical supervisors are required to evaluate each student’s clinical performance at the mid-point and again at the end of the semester using CALIPSO (commercially-available electronic clinical assessment tool). The supervisor assesses work of a student along the following dimensions of creating a treatment plan: proficiency in diagnostic, intervention/treatment and documentation and patient interaction skills. Each skill area contains multiple sub-categories in which students are evaluated; the sub-categories are averaged to establish a mean for each clinical skill area for each clinical rotation. Supervisors are trained by UK clinical faculty to correctly use the instrument before the student arrives. The clinical performance tool uses a rating scale of 1-5 with each numeric rating defined in CALIPSO. The scale values are as follows: 1-Beginning Graduate Clinician; 2-Developing; 3-Advanced; 4-Beginning Professional; 5-Advanced Beginning Professional.



7.3 Data Analysis Report Process/Procedures Each degree-granting academic program within the College submits a six-year assessment plan as part of the UK

periodic review process. The assessment plan template is provided by the UK Office of Assessment. The assessment

plans contain two three-year assessment cycles of student learning outcomes, curriculum maps and assessment

tools. The completion of the annual assessment reports and improvement action plans is a priority for the College

and has been included as a metric in previous versions of the college strategic plan.

8. Teaching Effectiveness 8.1 Identify Measures of Teaching Effectiveness All instructors will use the University Teacher Course Evaluation (TCE) process to be evaluated by their students each

semester. Through the annual and biennial review processes of faculty performance, each instructor will be asked to

provide a self-reflection which will include areas of improvement. New faculty members are peer reviewed at least

once during their first academic year of teaching. The Program Director and Department Chair review the TCE

results, any available peer review forms, and the self-reflection with the instructors and provides feedback to the

instructor.

8.2 What efforts to improve teaching effectiveness will be pursued based on these measures? A key measure of teaching effectiveness is the performance on the annual faculty performance review. The review

includes evaluation of the teaching portfolio, including the teacher course evaluation data. This information is used

to develop individualized faculty intervention plans and/or College-wide faculty development initiatives as needed.

9. Plans to evaluate post-graduation success Three forms of post-graduation success are used by our Masters program. First, employment rates are tracked and

historically indicate strong and high rates of employment within one year of graduation. Second, alumni surveys are

conducted every three years to review student’s long-term impressions of their preparation for the job market. Finally,

employers are directly surveyed to determine the perceived and actual quality of our trainees from their perspective.

Employers report a high degree of satisfaction with our graduates.

10. Appendices SLO #1

The Praxis Exam in Speech-Language Pathology measures knowledge important for independent practice as an SLP in

schools, hospitals, clinics, private practice, etc. The exam is taken by examinees who are in or who have completed a

master’s degree program. Recognized as the national examination in speech-language pathology, the test is one of

several requirements for the Certificate of Clinical Competence issued by the American Speech-Language-Hearing

Association (ASHA). The 132 selected-response multiple choice test questions assess foundational knowledge,

knowledge of professional practice, and specialized knowledge as it is applied across practice ranging from screening

and assessment to treatment and treatment evaluation. The questions test knowledge and its application across the “big

nine” areas of practice: speech sound production; fluency; voice, resonance, and motor speech; receptive and

expressive language; social aspects of communication, including pragmatics; cognitive aspects of communication;

augmentative and alternative communication; hearing; and feeding and swallowing. Test content is based on a practice

and curriculum analysis commissioned by ASHA.



SLO # 2 – Comprehensive Examination Scoring Rubric*.

*Relevancy of Information Category – Faculty meet beforehand to review the rubric to ensure that everyone is using the same

criteria to evaluate and score each section. The rubric is revised yearly based upon user feedback and the consistency of scoring.



SLO # 3 – (A) Criteria for Clinical Student Evaluations and (B) CALIPSO Performance Rating Scale

A.

1

Beginning Graduate Clinician Demonstrates competency, resourcefulness and creativity rarely or seldom, and or requires

considerable guidance and assistance in the execution of this specific clinical skill, regardless of the disorder. The graduate

clinician contributes little to the learning process. The graduate clinician could visit with a person who presents with a

communication impairment.

2

Developing Graduate Clinician Demonstrates inconsistent competency, independence, resourcefulness, and creativity in the

execution of this specific clinical skill across disorders; level of competency displayed is disorder specific. The graduate clinician

requires regular direction and feedback contributing some to the learning process. The graduate clinician could function as a

one-to-one aide for a person with communication impairment.

3

Advanced Graduate Clinician Demonstrates general competency and consistency across disorders but is still developing

independence, resourcefulness and creativity in the execution of this specific clinical skill. The degree of direction and feedback

required is dictated by the disorder with the graduate clinician contributing much to the learning process. The graduate

clinician could function as a speech-language pathology assistant.

4

Beginning Professional Demonstrates adequate competency, consistency and independence in the execution of the clinical skill

regardless of disorder but lacks resourcefulness and creativity. The graduate clinician requires periodic guidance and feedback

about the execution of this specific clinical skill with low incidence disorders requiring more guidance. The graduate clinician

directs his/her learning, remediating areas of need. The graduate clinician could effectively manage a caseload while benefiting

from mentoring.

5

Advanced Beginning Professional Demonstrates competency, consistency, creativity, resourcefulness and independence;

requiring minimal to no feedback. Implementation of the clinical skill is above that expected of a beginning clinician. The

graduate clinician’s learning is influenced by other disciplines and areas of study. The graduate clinician could mentor another

beginning speech-language pathologist on this specific clinical skill.



B.

Page 1



Page 2

university of kentucky division of communication sciences ......division of communication sciences...

Documents