showing growth through integrated performance assessments karen lichtman, ph.d. klichtman@niu.edu...

1

Showing Growth through Integrated Performance

Assessments

Karen Lichtman, Ph.D.klichtman@niu.eduAssistant Professor of Spanish LinguisticsDirector of Teacher LicensureDepartment of Foreign Languages & LiteraturesNorthern Illinois University

New Ideas in Foreign Language: Teaching Strategies November 6, 2015

2

FL 513 Foreign Language AssessmentSouthern Oregon University

Guanajuato, Mexico

3

Level of comfort with IPA (Integrated Performance Assessment)?

1. I have developed and used IPAs2. I know what one is, but I haven’t really used

them in my class3. I think I’ve heard of them4. Are you talking about beer?

4

Discuss with your table

• How do you currently assess students? • What goes into their grades?

5

1a. Backward Design:Assessment is at the center of planning

6Brown & Abeywickrama

Plan the unit around a summative performance assessment

7

Integrated Performance Assessment (IPA)

• What does “performance assessment” mean?

• Knowledge: Do you know it? • Performance: Can you use it? • Proficiency: Can you use it without any

preparation, with strangers?

KnowledgePerformance

Proficiency

8

Teachers will use both knowledge and performance assessments.

• Performance assessment: application of what students have learned in a unit to a real-life or “authentic” communication situation.– Realistic reflection of real-world situation– Require judgment and innovation– Task requires meaningful target language use– Simulate work, civic or personal life with an audience beyond the

teacher– Students use a variety of skills and knowledge to complete a

complex task– Integration of language skills– Includes opportunities for feedback

(Shrum & Glisan, 2010, p. 410)

9

So what is an Integrated Performance Assessment?

• Integrating Communication with the other goal areas of the standards

• Integrating the 3 modes of communication– Each task in the

assessment builds on the previous task

Assessment Framework by Foreign Language National Assessment of Educational Progress (FLNAEP)

10

1b. 3 modes of communication

InterpretiveListen to or read an

authentic text; assess comprehension

InterpersonalSpontaneous

communication with someone

PresentationalSharing research/ideas in

presentations, posters, brochures, websites, etc.

THEME

11

1b. 3 modes of communication

InterpretiveListen to or read an

authentic text; assess comprehension

PresentationalSharing research/ideas in

presentations, posters, brochures, websites, etc.

• Feedback• Clear expectations• Discuss strategies

• Feedback• Clear expectations

(seeing/using rubrics, samples)

• Feedback• Clear expectations

InterpersonalSpontaneous

communication with someone

THEME

12

1c. ACTFL proficiency guidelines

• Used to select appropriate materials & describe target performance on an IPA

• http://www.actfl.org/publications/guidelines-and-manuals/actfl-proficiency-guidelines-2012

• Page 2 of handout

13

Sounds great, but… isn’t it time-consuming?

• Yes• However, teachers often feel that each

assessment should be new anddifferent

• Showing growth is actuallyeasier if you reuse the SAME assessment… ideas from the researchprocess

14

Discuss

• How much does student growth currently count in your evaluation?

• What kind of assessments is your program using to show student growth?

15

2. The research process

• Teachers do research all the time!

16

Quantitative research

• Research questions– Is this teaching method better than that teaching

method? – How proficient are these students? – (How much) do students improve after ___?

• Common research designs: – Comparing two or more groups to each other – Comparing a group to itself (pretest/posttest)– Both of these designs can tell you whether there are

statistically significant differences between the groups.

17

Research example 1: Ashley Soriano’s masters thesis• “Using tablet-based instruction to improve

communicative responses in the world language classroom”

• Research questions:1. Will tablet-based instruction increase the use of

vocabulary in communicative responses? 2. Can self-assessment and reflection aid in student

awareness and improvement of communicative skills?3. Will fluidity increase as students are exposed to more

input and produce more output in the target language through the use of apps and other websites?

18

• Recordings every other Monday• Prompt: What did you do on the weekend? • Spanish II and III• Meeting standard = 3 out of 4 points;

Exceeding standard = 3.5 out of 4 points

Research example 1: Ashley Soriano’s masters thesis

19

• Students significantly improved both fluidity and ability to elaborate by the end of the semester (3rd hour = p < .05, 5th hour = p < .05, 8th hour = p < .05)

Research example 1: Ashley Soriano’s masters thesis

20

Digression: what is statistical significance?

PretestBob: 85Jenny: 89Phil: 90Jane: 64Elizabeth: 75Jonny: 62, Steve: 88, Missy: 70, Kara: 84,

Paul: 81, Mark: 84

PosttestBob: 80Jenny: 93Phil: 99Jane: 96Elizabeth: 91Jonny: 100, Steve: 94, Missy: 93, Kara: 99,

Paul: 91, Mark: 93

21

Digression: what is statistical significance?

• You use different statistical tests depending on your data.

• Each test spits out a “p-value”• The p-value is the probability that your result is

not due to random chance.

p = .05

95% chance I’m right 5% chance my result isdue to random chance

22

Digression: what is statistical significance?

PretestBob: 85Jenny: 89Phil: 90Jane: 64Elizabeth: 75Jonny: 62, Steve: 88, Missy: 70, Kara: 84,

Paul: 81, Mark: 84

PosttestBob: 80Jenny: 93Phil: 99Jane: 96Elizabeth: 91Jonny: 100, Steve: 94, Missy: 93, Kara: 99,

Paul: 91, Mark: 93

p=.46

Goal: p < .05

p=.00 *

p=.07

23

Research example 2:Julie Weitzel’s masters thesis

• “The Effect of Communication Strategy Training on Students’ Willingness to Communicate in the Interpersonal Mode”

• Research questions:1. Does communication strategy training improve

students’ willingness to communicate in the interpersonal mode?

2. Which communication strategies do students find the most useful?

24

Research example 2:Julie Weitzel’s masters thesis• Same questionnaire in August and

December

• Likert scale questions1 2 3 4

Strongly StronglyDisagree Agree

• Open-ended questions

25

Research example 2:Julie Weitzel’s masters thesis

• Students significantly improvedtheir willingness to communicate

• By far the most helpful strategy was circumlocution

01234

PrePost

26

3. Creating IPAs and rubrics

• Creating an IPA:• http://www.carla.umn.edu/assessment/vac/in

dex.html > Create an assessment unit step-by-step

• Today we will all come up with an IPA idea!

27

Creating your IPA idea

• 1 minute: Think of a unit that you currently teach, but you are not 100% happy with the current final exam or final project. Write down the topic. Example: airport unit

• 1 minute: Describe the unit to a partner

28

Creating your IPA idea

• 2 minutes: write one “Students Will Be Able To (SWBAT)” statement for the content of your unit and one “SWBAT” statement for the language of your unitExample: middle school market unit on CARLA

29

Creating your IPA idea

• 2 minutes: with your partner, brainstorm an interpretive task for the unit.

• Students will read/listen to _____________ and answer questions/ complete an information chart/ other activity

30

Creating your IPA idea

• 2 minutes: with your partner, brainstorm an interpersonal speaking task where students compare their opinions about their information from the interpretive task

• 2 minutes: with your partner, brainstorm a presentational task for your theme. If you can’t think of one, have your students create a “webpage” (real or on paper!).

31

Creating a rubric (Sandrock)

Do include on a rubric• A description of a quality

performance– Clear expectations– Realistic expectations for

language use at proficiency level– Observable indicators– Criteria appropriate for the task– Criteria appropriate for the

mode of communication– List of elements that you want

students to think about in doing the task

Don’t include on a rubric• Non-negotiables (followed

directions, completed task)• Things that are not about

language (has nice drawings)

32

Creating a rubric

• Sample “master rubric” from Jefferson public schools

33

What is realistic?

34

Creating a rubric

• Splitting the master rubric for level 1 vs. level 3

35

3a. Scoring rubrics

• Handout p. 3 (Lackey & Weitzel, 2015)https://sites.google.com/site/assess mentmatterssbc/standards-based-learning

1. Create your rubric2. Put points in each box (1, 2, 3, 4)3. Figure out the minimum and maximum possible

points4. Put every possible point value across the top5. Assign 50% to the minimum value and 100% to the

maximum value

36

3a. Scoring rubrics

6. Time for some math!

50 divided by the number of jumps between possible point values = how many percentage points you go up between possible point values

50 / 9 = 5.55

50 + 5.55 = 55.6 55.6 + 5.55 = 61.1, etc.

37

4. Using rubrics multiple times to show growth

• Remember… it’s good to reuse the SAME RUBRIC!

• By using a rubric tied to the ACTFL proficiency levels, you can show growth over time

• Grading criteria can become more stringent without having to change the rubric itself

38

Last page of handout: different expectations for different levels

39

In conclusion…

• If you can invest the time to create 3 rubrics that cover the 3 modes of communication for all your proficiency levels, you can keep using them for YEARS!

40

Questions?

• klichtman@niu.edu Karen Lichtman