faculty unplugged? empowering faculty in today’s assessment … › uploadeddocuments › events...

ACCSC Faculty Development Workshop

February 11, 2015

Arlington, Texas

Faculty Unplugged? Empowering Faculty in Today’s Assessment Culture

Presented by:

Julie Basler, PhD (Vice President of Academic Affairs-Platt College)

Juanita L. Gurubatham, PhD (Director of Institutional Review and Development-ACCSC)

• Defining the Loop: Defining measureable course objectives, program objectives, and how to assess it all both internally and externally

• Creating the Loop: Tying course objectives to program objectives

• Enforcing the Loop: Using valid, clear course activities to achieve objectives

• Closing the Loop: Reaching program outcomes at the end of a program

Today’s Agenda

Defining the Loop

• Course objective: Statements of expectation written in measurable terms that express what a student will know, do, or think at the end of a learning experience.

• Course objectives are measureable learner-oriented abilities that are consistent with standards of a professional practice.

Breaking it Down in Terms

• Program Objectives: Two types: Internal Measurements and External Measurements

• Internal Measurements: Performance indicators that reflect the extent to which the purposes of the program are achieved and by which program effectiveness is documented.

• External Measurements: Consumer-Oriented indexes designed to evaluate the degree to which the program is achieving its mission and goals.


• Competencies: Measurable behaviors, knowledge, actions, and skills essential to the practice of a profession.

• Institutional Effectiveness: Written document based on measurable standards that reflect the process of ongoing comprehensive assessment of all program components, including program objectives, etc.


• A fun and meaningful way to teach all of these terms is to use a map. • A map provides major interstates as the Program

Outcomes • By-passes and highways as the competencies

and so on • Choose a map of the area of each person is from

and let that person explain using his/her own home town area

Taking it Back Home

• Course Objective • Competency • Program Outcome(s) • Faculty the “key” to this involvement How is faculty the key and the starting point

to this whole process?

How Does this Tie Together?

• Students can see how material is related to their educational goals • Exams correspond to the stated learning

objectives (Once faculty have written learning objectives, they have defined their assessment materials.)

• Students know what they are expected to be able to do after the instruction is complete daily, quarterly, etc.

Why are Course Objectives Important?

• Course is organized (With objectives, the topics fit together and have direction.)

• In short, course objectives communicate what the instructor is trying to teach; what the students are to be expected to be able to do; how their achievement will be measured; and what will be accepted as evidence that they have achieved the goals.

Why are Course Objectives Important?

• Step One: Determine where the course lies in the curriculum.

• Step Two: Determine what knowledge or skills your students have from previous courses.

• Step Three: Establish or Determine a Course Goal • Step Four: Break down the course goal into subcomponents. Ask yourself what the specific knowledge, skills, and attitudes you expect the students to have at the end of the course, based on the course goal and where the course appears in the curriculum.

Creating a Course Objective

• Step Five: Rewrite these subcomponents into measurable units (aka learning objectives). These measurable units should contain the following criteria:

• A = Audience. Who will be completing the subcomponent? In the case of your class, it is always the student, therefore most people leave audience out of the learning objective.

• B = Behavior. What knowledge, skill or attitude do you expect the student to have? You can use Bloom’s taxonomy as a guide to determine the level of behavior.

• C = Condition. Any special circumstances needed to complete the objective.

Creating a Course Objective

Course goal: Students will be able to correctly identify anatomical structures in the human body.

Specific learning objectives: At the completion of the course, students will be able to:

• Recall the attachments of the muscles of the arm.

• Identify the muscles of the arm on a model, picture, diagram, drawing, or specimen.

Example Course Objective

Course goal: Students will learn common theories of learning and apply them to teaching preschool students.

Specific learning objectives: At the completion of the course, students will be able to: • Compare and contrast the learning theories of

Vygotsky and Piaget. • Design an appropriate playtime activity for three year

olds using Piaget’s theory From the VCU Center for Teaching Excellence

Example Course Objective

Don’t ask for people to write course objectives on the spot:

It causes faculty to freak out Instead, have them evaluate ones that you pull and them have them

submit revised ones privately, not publicly Think of ways to praise faculty for well written course objectives: free things exist too: Car wash tokens Lottery scratch offs

Taking it Back Home

• Using Bloom’s Taxonomy- Student Learning Outcomes A committee of colleges, led by Benjamin Bloom (1956),

identified three domains of educational activities: Cognitive: mental skills (Knowledge) Psychomotor: manual or physical skills (Skills) Affective: growth in feelings or emotional areas

(Attitude)

Using Course Objectives

• Writing Student Learning Outcomes Using the Domains

1. Remembering 2. Understanding 3. Applying 4. Analyzing 5. Evaluating 6. Creating

Categories of Bloom’s Taxonomy

• Have Faculty Members Draw an Illustration in Stick People for Each of These!

1. Remembering 2. Understanding 3. Applying 4. Analyzing 5. Evaluating 6. Creating

Taking It Back Home

Remembering: Recall previous learned information. Examples: Recite the safety rules in administering

insulin. Knows the steps of wrapping a sprained ankle.

Key Words: defines, describes, identifies, knows,

labels, lists, matches, names, outlines, recalls, recognizes, reproduces, selects, states.


Understanding: Comprehending the meaning, translation, and interpretation of instructions and problems. State a problem in one's own words.

Examples: Explain in one's own words the steps

for creating an excel spreadsheet. Key Words: comprehends, converts, defends,

distinguishes, estimates, explains, extends, generalizes, gives an example, infers, interprets, paraphrases, predicts, rewrites, summarizes, translates.


Applying: Use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom into novel situations in the work place.

Examples: Apply laws of statistics to evaluate the reliability of a written test.

Key Words: applies, changes, computes, constructs, demonstrates, discovers, manipulates, modifies, operates, predicts, prepares, produces, relates, shows, solves, uses.


Analyzing: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences.

Examples: Troubleshoot a piece of equipment by using logical deduction.

Key Words: analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, relates, selects, separates.


Evaluating: Make judgments about the value of ideas or materials.

Examples: Select the most effective solution.

Explain and justify a new budget. Key Words: appraises, compares, concludes,

contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, explains, interprets, justifies, relates, summarizes, supports.


Creating: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.

Examples: Write a company operations manual. Integrates training from several sources to solve a problem.

Key Words: categorizes, combines, compiles, composes, creates, devises, designs, explains, generates, modifies, organizes, plans, rearranges, reconstructs, relates, reorganizes, revises, rewrites, summarizes, tells, writes.


• Measureable behaviors, knowledge, actions, and skills essential to the practice of any profession.

Competency Types: • Knowledge Competencies - practical or

theoretical understanding of subjects. • Skill and Ability Competencies - natural or

learned capacities to perform acts. • Behavioral Competencies - patterns of action

or conduct.

What are Competencies?

• Achievement of the Program Outcomes for the Bachelor of Science in Nursing will be measured using the following five competencies based on the national standards of the Essentials of Baccalaureate Education and the Prelicensure knowledge, skills, and attitudes delineated by QSEN: (1) Nursing Process, (2) Nursing Informatics, (3) Evidence-Based Practice

Sample Competencies

Lunch

More importantly -- Dessert

Creating the Loop

• What do I want my graduate to look like when s/he graduates?

• What broad headers do I want them to fall under?

• Internal vs External-What’s the Difference?

Course Objective To Program Objectives

• What do I want my graduate to look like when s/he graduates?

• Have people draw this out • Crayons, paper, etc.

Taking It Back Home

• The end result of what the graduate looks like at the end of the entire program on graduation day

Program Outcomes

• What the graduate looks like-Internal • Also, the program indices play a part

in the external measurement (i.e. pass rates, placement rates, etc.)

Program Outcomes

A program outcome must flow directly from, and support, the school and program/department mission. The connection between the mission and the outcome should be clear. A program outcome must be directly related to the discipline of the program.

Consider these examples: • General: Graduates of the Criminal Justice program will be critical

thinkers. • Program-specific: Graduates of the Criminal Justice program will

analyze a current issue in criminal justice, evaluate evidence, and construct an argument.

Program Outcomes

Enforcing the Loop

ASSESSMENT FOR LEARNING ASSESSMENT OF LEARNING USING CLASSROOM ASSESSMENT TO ENGAGE STUDENTS

Assessment FOR Learning

• Turns the classroom assessment process and results into an instructional intervention

• Students and instructors work as a team • Provide students with a clear vision of the learning

target • Provide students with descriptive feedback-show

them how to improve. • Accuracy (of Information) + descriptive feedback

+ student involvement = achievement gains Stiggins, R. “Assessment FOR Learning Defined” and Classroom Assessment for Student Learning

7 Strategies of Assessment For Learning

Where am I going? 1. Provide a clear and understandable vision of the learning

target. 2. Use examples of models of strong and weak work. Where am I now? 3. Offer regular descriptive feedback. 4. Teach students to self-assess and set goals. How can I close the gap? 5. Design lessons to focus on one aspect of quality at a time. 6. Teach students focused revision. 7. Engage students in self-reflection, and let them keep track of

and share their learning.

Three Basic Questions for Evaluating Courses

Instruments of Evaluation

• Are the learning objectives of the course being met? Are students being inspired and motivated to think analytically and creatively and to develop habits of mind appropriate to the discipline?

• Measures of student learning based on students’ in-course papers, projects, or exams evaluated by the faculty member’s explicit standards and criteria

• Students’ performance on standardized tests

• Students’ performance in subsequent courses

• Are the course material, concepts, and activities rigorous, current, relevant for students’ needs, and consonant with the announced course description?

• Colleague examination of course syllabus, exams, and other material

• Do students perceive themselves to be well taught?

• Student course evaluations, surveys, focus groups

• Alumni surveys

Walvoord, B.E., Assessment Clear and Simple

• Using a “MAP” at the Course Level. • Remind faculty that course objectives/outcomes

should not be a secret. • A map provides major interstates as the Course

Objectives • By-passes and highways as the competencies

and so on • Encourage faculty to set the “destination” in the

first class period and continue to evaluate progress

Taking it Back Home

Tools: Grades

• Exams and assignments must actually measure learning goals

• Establish criteria for assessment • Develop systematic ways of to communicate

student weaknesses and strengths to decision makers

Tools: Rubrics

• Allows for consist evaluation • Provides various criteria and standards that

may be used to evaluate student work • May be used at course level and program level

Sample Rubric

Trait 14: Sentence Construction Level 5 Clear and concise sentences vary, with the degree of complexity reflecting the audience and purpose. 4 Sentences vary, with the degree of complexity reflecting the audience and purpose. 3 Sentence variety is limited but attempts complex structure. 2 Complex structure is attempted without success and/or sentence structure is simplistic, but not throughout the text 1 Sentences are simple and repetitive. Cognitive Level and Quality of Writing Assessment : CLAQWA Online, Cross-Disciplinary. Retrieved from http://claqwa.com

Tools: E-Portfolio

• Portfolios contain authentic evidence of learning that students produce in the everyday course of their studies, both in and out of the classroom (e.g., in internships, independent research projects, and other “real-world” experiences).

• Enables students to collect and reflect on evidence of what they are learning.

• May be used at course level or program level

Banta, T., et al, Three Promising Alternatives for Assessing College Students’ Knowledge and Skills

Tools: E-Portfolio

These basic questions need to be answered when undertaking an ePortfolio project. They include the following:

• When and from where will student work be collected?

• Are there specific assignments that correspond to specific outcomes?

• Is there a grading rubric? • Who will evaluate the student ePortfolios (if

Programmatic Assessment)

• Break faculty into groups by

subject area • Discuss/identify best tools for

assessing competencies

Taking it Back Home

Assessment OF Learning

• Assessments that happen after learning occurred to determine if it did.

• Primary users include policy makers, program planners, faculty, etc.

• Used to “certify” student competence

Tools: Standardized Exams Assessment OF Learning

• Useful in specific content areas • Quantitative measure with established

benchmarks • Has limitations • Licensure and Certification Exams

Program Assessment

• Now you have a program curriculum—is it right?

• Assessment is an ongoing process • Develop a plan

– Keep it simple • Collect and evaluate your data • Remember to close the loop

– Use results to improve curriculum

• Course-level assessments of student learning should map to program assessment

• Program faculty should think collectively about how course assessments contribute to the program review.

• Keep it Simple • Consider preset dates for annual program review • Limit the number of objectives reviewed each year • Put someone in “charge”

Taking it Back Home

Building Blocks for Program Assessment

• Program Mission – Statement of purpose and intentions – Should connect to the institutional mission – Provides foundation for development of goals and

objectives


• Goals and Objectives – What do you want a graduate to “look like” – Must be measureable – More is not always better


• Evaluation – Determine what you will assess – Evaluation tools

• Direct Assessment • Indirect Assessment

– Who – When – Close the loop—documentation!

Goals/Outcomes

Curricular Design

Pedagogy

Assessment

Mission

AAC& U (2004), Taking Responsibility for the Quality of the Baccalaureate Degree

• Planning your Day • Incentive for participating • Don’t overwhelm participants • Keep it simple

Taking it Back Home

faculty unplugged? empowering faculty in today’s assessment … › uploadeddocuments › events...

Documents