![Page 1: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/1.jpg)
1
Showing Growth through Integrated Performance
Assessments
Karen Lichtman, [email protected] Professor of Spanish LinguisticsDirector of Teacher LicensureDepartment of Foreign Languages & LiteraturesNorthern Illinois University
New Ideas in Foreign Language: Teaching Strategies November 6, 2015
![Page 2: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/2.jpg)
2
FL 513 Foreign Language AssessmentSouthern Oregon University
Guanajuato, Mexico
![Page 3: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/3.jpg)
3
Level of comfort with IPA (Integrated Performance Assessment)?
1. I have developed and used IPAs2. I know what one is, but I haven’t really used
them in my class3. I think I’ve heard of them4. Are you talking about beer?
![Page 4: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/4.jpg)
4
Discuss with your table
• How do you currently assess students? • What goes into their grades?
![Page 5: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/5.jpg)
5
1a. Backward Design:Assessment is at the center of planning
![Page 6: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/6.jpg)
6Brown & Abeywickrama
Plan the unit around a summative performance assessment
![Page 7: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/7.jpg)
7
Integrated Performance Assessment (IPA)
• What does “performance assessment” mean?
• Knowledge: Do you know it? • Performance: Can you use it? • Proficiency: Can you use it without any
preparation, with strangers?
KnowledgePerformance
Proficiency
![Page 8: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/8.jpg)
8
Teachers will use both knowledge and performance assessments.
• Performance assessment: application of what students have learned in a unit to a real-life or “authentic” communication situation.– Realistic reflection of real-world situation– Require judgment and innovation– Task requires meaningful target language use– Simulate work, civic or personal life with an audience beyond the
teacher– Students use a variety of skills and knowledge to complete a
complex task– Integration of language skills– Includes opportunities for feedback
(Shrum & Glisan, 2010, p. 410)
![Page 9: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/9.jpg)
9
So what is an Integrated Performance Assessment?
• Integrating Communication with the other goal areas of the standards
• Integrating the 3 modes of communication– Each task in the
assessment builds on the previous task
Assessment Framework by Foreign Language National Assessment of Educational Progress (FLNAEP)
![Page 10: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/10.jpg)
10
1b. 3 modes of communication
InterpretiveListen to or read an
authentic text; assess comprehension
InterpersonalSpontaneous
communication with someone
PresentationalSharing research/ideas in
presentations, posters, brochures, websites, etc.
THEME
![Page 11: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/11.jpg)
11
1b. 3 modes of communication
InterpretiveListen to or read an
authentic text; assess comprehension
PresentationalSharing research/ideas in
presentations, posters, brochures, websites, etc.
• Feedback• Clear expectations• Discuss strategies
• Feedback• Clear expectations
(seeing/using rubrics, samples)
• Feedback• Clear expectations
InterpersonalSpontaneous
communication with someone
THEME
![Page 12: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/12.jpg)
12
1c. ACTFL proficiency guidelines
• Used to select appropriate materials & describe target performance on an IPA
• http://www.actfl.org/publications/guidelines-and-manuals/actfl-proficiency-guidelines-2012
• Page 2 of handout
![Page 13: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/13.jpg)
13
Sounds great, but… isn’t it time-consuming?
• Yes• However, teachers often feel that each
assessment should be new anddifferent
• Showing growth is actuallyeasier if you reuse the SAME assessment… ideas from the researchprocess
![Page 14: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/14.jpg)
14
Discuss
• How much does student growth currently count in your evaluation?
• What kind of assessments is your program using to show student growth?
![Page 15: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/15.jpg)
15
2. The research process
• Teachers do research all the time!
![Page 16: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/16.jpg)
16
Quantitative research
• Research questions– Is this teaching method better than that teaching
method? – How proficient are these students? – (How much) do students improve after ___?
• Common research designs: – Comparing two or more groups to each other – Comparing a group to itself (pretest/posttest)– Both of these designs can tell you whether there are
statistically significant differences between the groups.
![Page 17: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/17.jpg)
17
Research example 1: Ashley Soriano’s masters thesis• “Using tablet-based instruction to improve
communicative responses in the world language classroom”
• Research questions:1. Will tablet-based instruction increase the use of
vocabulary in communicative responses? 2. Can self-assessment and reflection aid in student
awareness and improvement of communicative skills?3. Will fluidity increase as students are exposed to more
input and produce more output in the target language through the use of apps and other websites?
![Page 18: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/18.jpg)
18
• Recordings every other Monday• Prompt: What did you do on the weekend? • Spanish II and III• Meeting standard = 3 out of 4 points;
Exceeding standard = 3.5 out of 4 points
Research example 1: Ashley Soriano’s masters thesis
![Page 19: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/19.jpg)
19
• Students significantly improved both fluidity and ability to elaborate by the end of the semester (3rd hour = p < .05, 5th hour = p < .05, 8th hour = p < .05)
Research example 1: Ashley Soriano’s masters thesis
![Page 20: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/20.jpg)
20
Digression: what is statistical significance?
PretestBob: 85Jenny: 89Phil: 90Jane: 64Elizabeth: 75Jonny: 62, Steve: 88, Missy: 70, Kara: 84,
Paul: 81, Mark: 84
PosttestBob: 80Jenny: 93Phil: 99Jane: 96Elizabeth: 91Jonny: 100, Steve: 94, Missy: 93, Kara: 99,
Paul: 91, Mark: 93
![Page 21: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/21.jpg)
21
Digression: what is statistical significance?
• You use different statistical tests depending on your data.
• Each test spits out a “p-value”• The p-value is the probability that your result is
not due to random chance.
p = .05
95% chance I’m right 5% chance my result isdue to random chance
![Page 22: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/22.jpg)
22
Digression: what is statistical significance?
PretestBob: 85Jenny: 89Phil: 90Jane: 64Elizabeth: 75Jonny: 62, Steve: 88, Missy: 70, Kara: 84,
Paul: 81, Mark: 84
PosttestBob: 80Jenny: 93Phil: 99Jane: 96Elizabeth: 91Jonny: 100, Steve: 94, Missy: 93, Kara: 99,
Paul: 91, Mark: 93
p=.46
Goal: p < .05
p=.00 *
p=.07
![Page 23: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/23.jpg)
23
Research example 2:Julie Weitzel’s masters thesis
• “The Effect of Communication Strategy Training on Students’ Willingness to Communicate in the Interpersonal Mode”
• Research questions:1. Does communication strategy training improve
students’ willingness to communicate in the interpersonal mode?
2. Which communication strategies do students find the most useful?
![Page 24: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/24.jpg)
24
Research example 2:Julie Weitzel’s masters thesis• Same questionnaire in August and
December
• Likert scale questions1 2 3 4
Strongly StronglyDisagree Agree
• Open-ended questions
![Page 25: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/25.jpg)
25
Research example 2:Julie Weitzel’s masters thesis
• Students significantly improvedtheir willingness to communicate
• By far the most helpful strategy was circumlocution
01234
PrePost
![Page 26: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/26.jpg)
26
3. Creating IPAs and rubrics
• Creating an IPA:• http://www.carla.umn.edu/assessment/vac/in
dex.html > Create an assessment unit step-by-step
• Today we will all come up with an IPA idea!
![Page 27: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/27.jpg)
27
Creating your IPA idea
• 1 minute: Think of a unit that you currently teach, but you are not 100% happy with the current final exam or final project. Write down the topic. Example: airport unit
• 1 minute: Describe the unit to a partner
![Page 28: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/28.jpg)
28
Creating your IPA idea
• 2 minutes: write one “Students Will Be Able To (SWBAT)” statement for the content of your unit and one “SWBAT” statement for the language of your unitExample: middle school market unit on CARLA
![Page 29: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/29.jpg)
29
Creating your IPA idea
• 2 minutes: with your partner, brainstorm an interpretive task for the unit.
• Students will read/listen to _____________ and answer questions/ complete an information chart/ other activity
![Page 30: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/30.jpg)
30
Creating your IPA idea
• 2 minutes: with your partner, brainstorm an interpersonal speaking task where students compare their opinions about their information from the interpretive task
• 2 minutes: with your partner, brainstorm a presentational task for your theme. If you can’t think of one, have your students create a “webpage” (real or on paper!).
![Page 31: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/31.jpg)
31
Creating a rubric (Sandrock)
Do include on a rubric• A description of a quality
performance– Clear expectations– Realistic expectations for
language use at proficiency level– Observable indicators– Criteria appropriate for the task– Criteria appropriate for the
mode of communication– List of elements that you want
students to think about in doing the task
Don’t include on a rubric• Non-negotiables (followed
directions, completed task)• Things that are not about
language (has nice drawings)
![Page 32: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/32.jpg)
32
Creating a rubric
• Sample “master rubric” from Jefferson public schools
![Page 33: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/33.jpg)
33
What is realistic?
![Page 34: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/34.jpg)
34
Creating a rubric
• Splitting the master rubric for level 1 vs. level 3
![Page 35: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/35.jpg)
35
3a. Scoring rubrics
• Handout p. 3 (Lackey & Weitzel, 2015)https://sites.google.com/site/assess mentmatterssbc/standards-based-learning
1. Create your rubric2. Put points in each box (1, 2, 3, 4)3. Figure out the minimum and maximum possible
points4. Put every possible point value across the top5. Assign 50% to the minimum value and 100% to the
maximum value
![Page 36: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/36.jpg)
36
3a. Scoring rubrics
6. Time for some math!
50 divided by the number of jumps between possible point values = how many percentage points you go up between possible point values
50 / 9 = 5.55
50 + 5.55 = 55.6 55.6 + 5.55 = 61.1, etc.
![Page 37: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/37.jpg)
37
4. Using rubrics multiple times to show growth
• Remember… it’s good to reuse the SAME RUBRIC!
• By using a rubric tied to the ACTFL proficiency levels, you can show growth over time
• Grading criteria can become more stringent without having to change the rubric itself
![Page 38: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/38.jpg)
38
Last page of handout: different expectations for different levels
![Page 39: Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of](https://reader033.vdocuments.mx/reader033/viewer/2022051622/5697c0221a28abf838cd335b/html5/thumbnails/39.jpg)
39
In conclusion…
• If you can invest the time to create 3 rubrics that cover the 3 modes of communication for all your proficiency levels, you can keep using them for YEARS!