using learning assessment techniques throughout your …...using learning assessment techniques...

Using Learning Assessment Techniques

Throughout Your Course

A workshop for

Santa Monica College FSI 2017

August 9, 2017 - AM

Kevin Kelly, EdD This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

http://creativecommons.org/licenses/by-sa/4.0/



About me Since 1999 • Teaching at San Francisco State University

• Current class: UG, Gen Ed course "How 2 Lrn w ur Mobile Device"

• Faculty development/Leadership development • San Francisco State (several hats) • Wiley Learning Institute • Association of College and University Educators (ACUE) • Higher education consulting

2 FSI 2017 Workshop - Assessment

About this workshop What we'll do today: • assess and adjust learning outcomes from one or more

courses • use the Learning Goals Inventory to identify pedagogical

areas of emphasis • discuss structuring activities that promote both

assessment for learning and assessment of learning • work in small groups to explore different categories of

assessment techniques that support common goals • practice analyzing the results from assessments • commit to trying at least one assessment technique in Fall

2017

FSI 2017 Workshop - Assessment 3


NOTE: Assessment is a big topic! • We WILL NOT attempt to drink

straight from a fire hydrant. • We WILL pick a few practical

strategies to use in your course over time.

Image: "fire hydrant" CC BY Renee

About you - setting personal goals

What is one goal you want to achieve during this workshop about assessment?

For the next 2 to 3 minutes: 1. Form small groups & assign a "tech scribe" *

• * someone with a laptop or mobile device 2. Visit Today's Meet:

https://todaysmeet.com/FSI2017-assessment 3. Enter each person's goal as a new entry


https://todaysmeet.com/FSI2017-assessment




About you - setting personal goals

What is one goal you want to achieve during this workshop about assessment?

Common goals:


WHAT IS ASSESSMENT?


About assessment

What do you associate with assessment in your class(es)?

For the next 5 minutes: 1. 2 min: On your own, list 5 to 7 things that you

associate with assessment in your class(es). 2. 3 min: Turn to a neighbor and compare your

lists. Identify any items you have in common.


About assessment

What do you associate with assessment in your class(es)?

Common: option to work in pairs, rubrics, check for understanding, disconnect btw what we think they know and test results, stress, scaffold group activities (guided questions), sufficient time, goals for both students/teachers, retention (in context of the test), essays, pre-assessment, student questions,


Pulling back the curtain • Learning Goal Listing (LAT 47)

• Goal: Help learners become aware of their own goals.

• Timing: Use at the start of a course or unit. • Tips: Help learners connect their goals to

course goals and/or student learning outcomes.

• Focused Listing Activity (CAT 2) • Goal: Determine what learners know or recall

about a topic. • Timing: Use at the start of a class meeting. • Tips: Ask small groups to share common items.

Get a show of hands for each item. Share missing items from your own list.

10

What is assessment? • Barkley & Major (2016, p. 24) define "assessment" as

"appraisal of knowledge, skills, attitudes and beliefs that students have acquired, most often as a result of learning in their courses."

• Assessment is different from grading. Among other things, assessment focuses on a group of students, while grading focuses on an individual student.


What is assessment? • For the purposes of this workshop, we'll look at

assessment from a number of angles • What can we actually assess? • Why use a specific assessment strategy? • When should we use different assessment strategies? • Who should facilitate the assessment? • How do we implement those assessment strategies? • Can we implement the assessment strategies with technology? • What happens after we conduct an assessment?

• We'll focus on ways to engage learners with practical assessment activities you can use right in the classroom.


Assessment references (and resources) Learning Assessment Techniques

by Barkley & Major (2016) Classroom Assessment Techniques

by Angelo & Cross (1993)


Assessment references • Since I'll refer to both resources throughout this workshop,

let's see what they are: • Learning Assessment Techniques (LATs) are comprised of three

parts, reflecting and incorporating effective teaching elements—identify goals, facilitate activities, and assess outcomes.

• Classroom Assessment Techniques (CATs) "are generally simple, non-graded, anonymous, in-class activities designed to give you and your students useful feedback on the teaching-learning process as it is happening" (Vanderbilt CFT, n.d.).

• Both books outline a process AND provide 50 practical techniques!


What can we assess? Learning Assessment Techniques

Cognitive Domains • Foundational Knowledge • Application • Integration Humanistic Domains • Human Dimension • Caring • Learning How to Learn

Classroom Assessment Techniques

Knowledge & Skills • Prior Knowledge, Recall • Analysis, Critical Thinking • Synthesis, Creative Thinking • Problem Solving • Application, Performance Learner Attitudes, Values, Reactions • Awareness of Their Attitudes & Values • Self-Awareness as Learners • Study Skills, Strategies & Behaviors • Reactions to Teachers & Teaching • Reactions to Class Elements


Why should we assess? • Determine for ourselves how well students are learning

• So we can make any necessary changes

• Give learners feedback on their progress • So they can make necessary changes before high-stakes exams

• Improve our profession through the Scholarship of Teaching and Learning • Conversations with colleagues, publish an article

• Provide information to institutional and external stakeholders on how well students are learning • E.g., Program review, Accreditation

(Barkley & Major, 2016, pp. 26-28)


STRETCH BREAK (5 MIN)

Image CC BY-SA Quinn Dombrowski

For the next activity, we'll create different groupings…

Time to move! (discipline clusters)

• Art, Arts, Design • Humanities • Languages

• Social Sciences • Comm & Media • Psychology

• Sciences • CTE programs

• Technology

• Business • Professional Training

• Workforce Dev


WHERE TO START? SETTING THE STAGE FOR ASSESSMENT • (Re)write or add learning outcomes for 1 or more

modules • Plan assessments for each outcomes (along with

activities and materials)


“The key to success is to appreciate how people learn, understand the

thought process that goes into instructional design, what works well,

and a range of different ways of achieving goals.”

- Tim Buff


Activity “The key to success is to appreciate how people learn, understand the thought process that goes into instructional design, what works well, and a

range of different ways of achieving goals.” - Tim Buff

• Paraphrase: Rephrase the passage in your own words. • Interpret: Explain the meaning of this passage. • Comment: Express your opinion of this passage. • Cite: Write out how you would cite the quote using APA,

MLA or other.


Activity “The key to success is to appreciate how people learn, understand the thought process that goes into instructional design, what works well, and a

range of different ways of achieving goals.” - Tim Buff

From a volunteer • Paraphrase: • Interpret: • Comment: • Cite:


Activity “Think about what your learners need to do with that information

after the course is finished and design around that.”

- Matthew Guyan • Summarize this quote in one word. • Explain: In 1-2 sentences, explain why you chose that

word.


Activity “Think about what your learners need to do with that information

after the course is finished and design around that.”

- Matthew Guyan From a volunteer • Summarize: • Explain:


Pulling back the curtain • Quotation Commentaries (LAT 13)

• Goal: Help learners work with quoted material from assigned readings.

• Timing: Use to prepare for writing assignments. • Tips: Take time to showcase the value in what

learners are asked to read for your class.

• Word Journal (CAT 14) • Goal: Stimulate synthesis and creative thinking. • Timing: Use when learners should read carefully. • Tips: This strategy works best for reading short text

(essays, poems, short stories, articles, case studies).

25

Where to start? Objectives vs Outcomes

Learning objectives

• Objectives are a target—what we hope students will do

Learning outcomes

• Outcomes are an end result–students showing what they did • Observable, measurable • Examples: Student Learning

Outcomes, Institutional Learning Outcomes


(Barkley & Major, 2016)

(Re)writing learning objectives/outcomes

• Start with the usual: What should students be able to do by the end of the course?

Example objectives:

• Use metacognitive strategies designed to improve your learning

• Explore different technologies specifically for learning



• Add/revise objectives/outcomes for scope, specificity (granular): • course level • module/unit level • activity level (?)



• Add/revise outcomes for scope, specificity: • Example: Explore different technologies

specifically for learning • Through a quiz, show that you know what a technology

is, what it does, and how it can be used for learning. • In a discussion forum, describe how you used a

technology for specific learning activities. • Given a curated set of links, complete an evaluation

sheet outlining how effective a specific technology is for your own learning.

• Write an essay that reflects on the effectiveness of using a technology for learning in one of your other classes or for work-related training.

29

Course objective

Module outcomes

FSI 2017 Workshop - Assessment

(Re)writing learning objectives/outcomes • Rethink the level

(of thinking, of achievement): • Bloom's taxonomy

(updated)


http://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy/revised-blooms-taxonomy-flash-version

http://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy/revised-blooms-taxonomy-flash-version


• Identifying types of objectives/outcomes • Cognitive (knowledge)

• Example verbs: define, explain, predict, compare, evaluate, design

• Psychomotor (skills) • Example verbs: watch, operate, write, perform

• Affective (attitudes or feelings) • Example verbs: select, discuss, justify, integrate, revise


•Cognitive (knowledge) - Bloom’s Taxonomy (revised) •Remembering: Recall/recognize specific information •Understanding: Lowest level of understanding •Applying: Implement (use) information •Analyzing: Break information into parts and describe the relationship •Evaluating: Make a judgment about materials or methods •Creating: Generate new ideas, products and ways of viewing things


See http://itc.utk.edu/~bobannon/ classifications.html#cognitive_domain

Image source: http://www.iftf.org


http://itc.utk.edu/%7Ebobannon/classifications.html%23cognitive_domain



•Psychomotor (skills) •Observing: Active mental attending of a physical event •Imitating: Attempt to copy a physical behavior •Practicing: Try a specific physical activity over and over •Adapting: Fine tuning, make minor adjustments in a physical activity


See http://ets.tlt.psu.edu/learningdesign/ objectives/psychomotor

Image source: http://www.sharjah.ac.ae


http://ets.tlt.psu.edu/learningdesign/objectives/psychomotor



•Affective (attitudes or feelings) •Receiving: Shows awareness. •Responding: Shows attention and motivation. •Valuing: Demonstrates commitment. •Organizing: Integrates and prioritizes new values. •Characterizing by value: Behavior reflects new values.


http://www.uwsp.edu/education/lwilson/ curric/affectiv.htm

Image source: Karin Kirk, SERC http://serc.carleton.edu


http://www.uwsp.edu/education/lwilson/curric/affectiv.htm



Activity Task (3-5 min): • Review your course objectives or outcomes • (Re)write one course-level objective • Add one or more unit or module-level outcomes


Activity: Share • 3 to 5 faculty will share before/after versions of their objectives


Backward course design • Now that you have your objectives,

it's time to determine how students will show what they know.

• Then select activities that will prepare them to succeed with the assessment strategies.

• Last, identify the content that they'll need for all of it.

37

Image: UC Davis

FSI 2017 Workshop - Assessment

https://canvas.ucdavis.edu/courses/34528/pages/course-planning-with-backward-design

BIO-BREAK (5 MIN)

Image CC BY Rishabh Mishra

WHY ASSESS? Assessment FOR learning

or Assessment OF learning?


Why should we assess? • Determine for ourselves how well students are learning

• So we can make any necessary changes to curriculum

• Give learners feedback on their progress • So they can make necessary changes before high-stakes exams

• Improve our profession through the Scholarship of Teaching and Learning • Conversations with colleagues, publish an article

• Provide information to institutional and external stakeholders on how well students are learning • E.g., Program review, Accreditation

(Barkley & Major, 2016, pp. 26-28)


When should we assess? • Formative: Checking for student understanding during

the learning process • E.g., low-stakes quizzes or assignments

• Summative: Evaluating what students learned after • E.g., midterm or final exams, final projects or essays

• Confirmative: Checking to see what students remember over time, well after the learning took place • E.g., At the start of a term, use a Background Knowledge Probe

(CAT 1) to see what prerequisite information students remember


Prerequisite class – Final exam

(Summative) Break between

semesters

First class – Background knowledge

probe (Confirmative)

Quizzes and assignments (Formative)

In-class assessment strategies

(Formative)

Midterm or Final exam

(Summative)

When the cook tastes the soup, that’s formative; when the guests taste the

soup, that’s summative. - R. Stake


Why and when are we assessing?

Assessment FOR learning

• Formative • Supports/advances

student learning

Assessment OF learning

• Summative • Confirms student

achievement


Activity • Draw a concept map showing the connections between

the major ideas from this workshop. • Incorporate other concepts you have learned at FSI this week.

OR • Map a sequence of assessment events for your class.

• Show which activities are assessment OF learning and which are assessment FOR learning.


Activity • Explain your sequence chains or concept maps to a

neighbor (2 min each).


Pulling back the curtain • Sequence Chains (LAT 22)

• Goal: Help learners apply what they've learned to arrange items in order.

• Timing: Use as either formative or summative assessments.

• Tips: Take time to showcase the connections. Promote the end products as study aids.

• Concept Maps (CAT 16, LAT 23) • Goal: Synthesize, organize and show connections. • Timing: Use to assess prior knowledge or estimate

learning growth. • Tips:.

46

WHICH ONE? SELECTING AN ASSESSMENT TECHNIQUE


Which assessment technique to use? Learning Assessment Techniques

by Barkley & Major (2016)

• Learning Goals Inventory correlates to Fink's Significant Learning Taxonomy • Helps teachers 1) articulate

goals for student learning, 2) choose appropriate LATs, and 3) foster discussions re: teaching & learning with colleagues

Classroom Assessment Techniques by Angelo & Cross (1993)

• Teaching Goals Inventory correlates to the types of skills, knowledge & attitudes you believe are essential • Helps teachers 1) identify

what they want to achieve in courses, 2) choose appropriate CATs, and 3) foster discussions re: teaching & learning with colleagues


Activity Today we'll use the Learning Goals Inventory: • Part 1: Select a course that you'll teach in the Fall.

• Feel free to work on the same course for which you revised learning objectives or outcomes.

• Part 2: For that course, rate the importance of each item in the Learning Goals Inventory: • Rating scale: 5 high, 4 Moderately high, 3 Moderately low, 2 low, 1

N/A

• Part 3: Answer the Learning Goal Self-Assessment questions


Activity Now let's score the Learning Goals Inventory: • Add up the subtotal scores for each domain to determine

which domains have the highest value to you. • Identify the most important cognitive and humanistic

aspects of learning you rate for your students.


Time to move! (LGI clusters)

• Foundational knowledge (front, left)

• Application (middle, left)

• Integration

(back, left)

• Human dimension (front, right)

• Caring (middle, right)

• Learning how to learn (back, right)


Join your peers based on the most important domain identified from the LGI

Activity Using the activity worksheet: • First, read the description of the Learning Assessment

Technique for the most important Domain you identified through the Learning Goals Inventory (2-3 minutes)

• Then, begin work on the task designed to prepare you for implementing the technique (5-10 minutes)

• Last, each person presents his/her draft to the group (2 minutes each person)


Time to move again! (LGI clusters)

• Foundational knowledge (front, left)

• Application (middle, left)

• Integration

(back, left)

• Human dimension (front, right)

• Caring (middle, right)

• Learning how to learn (back, right)


Join your peers based on your second most important domain (or one that interests you)

Activity Using the activity worksheet: • First, read the description of the Learning Assessment

Technique for the most important Domain you identified through the Learning Goals Inventory (2-3 minutes)

• Then, begin work on the task designed to prepare you for implementing the technique (5-10 minutes)

• Last, each person presents his/her draft to the group (2 minutes each person)


55

Image: Buffet - CC BY SA Prinsotel Hotels

LUNCH BREAK FSI 2017 Workshop - Assessment

using learning assessment techniques throughout your …...using learning assessment techniques...

Documents