showing growth through integrated performance assessments karen lichtman, ph.d. [email protected]...
TRANSCRIPT
1
Showing Growth through Integrated Performance
Assessments
Karen Lichtman, [email protected] Professor of Spanish LinguisticsDirector of Teacher LicensureDepartment of Foreign Languages & LiteraturesNorthern Illinois University
New Ideas in Foreign Language: Teaching Strategies November 6, 2015
2
FL 513 Foreign Language AssessmentSouthern Oregon University
Guanajuato, Mexico
3
Level of comfort with IPA (Integrated Performance Assessment)?
1. I have developed and used IPAs2. I know what one is, but I haven’t really used
them in my class3. I think I’ve heard of them4. Are you talking about beer?
4
Discuss with your table
• How do you currently assess students? • What goes into their grades?
5
1a. Backward Design:Assessment is at the center of planning
6Brown & Abeywickrama
Plan the unit around a summative performance assessment
7
Integrated Performance Assessment (IPA)
• What does “performance assessment” mean?
• Knowledge: Do you know it? • Performance: Can you use it? • Proficiency: Can you use it without any
preparation, with strangers?
KnowledgePerformance
Proficiency
8
Teachers will use both knowledge and performance assessments.
• Performance assessment: application of what students have learned in a unit to a real-life or “authentic” communication situation.– Realistic reflection of real-world situation– Require judgment and innovation– Task requires meaningful target language use– Simulate work, civic or personal life with an audience beyond the
teacher– Students use a variety of skills and knowledge to complete a
complex task– Integration of language skills– Includes opportunities for feedback
(Shrum & Glisan, 2010, p. 410)
9
So what is an Integrated Performance Assessment?
• Integrating Communication with the other goal areas of the standards
• Integrating the 3 modes of communication– Each task in the
assessment builds on the previous task
Assessment Framework by Foreign Language National Assessment of Educational Progress (FLNAEP)
10
1b. 3 modes of communication
InterpretiveListen to or read an
authentic text; assess comprehension
InterpersonalSpontaneous
communication with someone
PresentationalSharing research/ideas in
presentations, posters, brochures, websites, etc.
THEME
11
1b. 3 modes of communication
InterpretiveListen to or read an
authentic text; assess comprehension
PresentationalSharing research/ideas in
presentations, posters, brochures, websites, etc.
• Feedback• Clear expectations• Discuss strategies
• Feedback• Clear expectations
(seeing/using rubrics, samples)
• Feedback• Clear expectations
InterpersonalSpontaneous
communication with someone
THEME
12
1c. ACTFL proficiency guidelines
• Used to select appropriate materials & describe target performance on an IPA
• http://www.actfl.org/publications/guidelines-and-manuals/actfl-proficiency-guidelines-2012
• Page 2 of handout
13
Sounds great, but… isn’t it time-consuming?
• Yes• However, teachers often feel that each
assessment should be new anddifferent
• Showing growth is actuallyeasier if you reuse the SAME assessment… ideas from the researchprocess
14
Discuss
• How much does student growth currently count in your evaluation?
• What kind of assessments is your program using to show student growth?
15
2. The research process
• Teachers do research all the time!
16
Quantitative research
• Research questions– Is this teaching method better than that teaching
method? – How proficient are these students? – (How much) do students improve after ___?
• Common research designs: – Comparing two or more groups to each other – Comparing a group to itself (pretest/posttest)– Both of these designs can tell you whether there are
statistically significant differences between the groups.
17
Research example 1: Ashley Soriano’s masters thesis• “Using tablet-based instruction to improve
communicative responses in the world language classroom”
• Research questions:1. Will tablet-based instruction increase the use of
vocabulary in communicative responses? 2. Can self-assessment and reflection aid in student
awareness and improvement of communicative skills?3. Will fluidity increase as students are exposed to more
input and produce more output in the target language through the use of apps and other websites?
18
• Recordings every other Monday• Prompt: What did you do on the weekend? • Spanish II and III• Meeting standard = 3 out of 4 points;
Exceeding standard = 3.5 out of 4 points
Research example 1: Ashley Soriano’s masters thesis
19
• Students significantly improved both fluidity and ability to elaborate by the end of the semester (3rd hour = p < .05, 5th hour = p < .05, 8th hour = p < .05)
Research example 1: Ashley Soriano’s masters thesis
20
Digression: what is statistical significance?
PretestBob: 85Jenny: 89Phil: 90Jane: 64Elizabeth: 75Jonny: 62, Steve: 88, Missy: 70, Kara: 84,
Paul: 81, Mark: 84
PosttestBob: 80Jenny: 93Phil: 99Jane: 96Elizabeth: 91Jonny: 100, Steve: 94, Missy: 93, Kara: 99,
Paul: 91, Mark: 93
21
Digression: what is statistical significance?
• You use different statistical tests depending on your data.
• Each test spits out a “p-value”• The p-value is the probability that your result is
not due to random chance.
p = .05
95% chance I’m right 5% chance my result isdue to random chance
22
Digression: what is statistical significance?
PretestBob: 85Jenny: 89Phil: 90Jane: 64Elizabeth: 75Jonny: 62, Steve: 88, Missy: 70, Kara: 84,
Paul: 81, Mark: 84
PosttestBob: 80Jenny: 93Phil: 99Jane: 96Elizabeth: 91Jonny: 100, Steve: 94, Missy: 93, Kara: 99,
Paul: 91, Mark: 93
p=.46
Goal: p < .05
p=.00 *
p=.07
23
Research example 2:Julie Weitzel’s masters thesis
• “The Effect of Communication Strategy Training on Students’ Willingness to Communicate in the Interpersonal Mode”
• Research questions:1. Does communication strategy training improve
students’ willingness to communicate in the interpersonal mode?
2. Which communication strategies do students find the most useful?
24
Research example 2:Julie Weitzel’s masters thesis• Same questionnaire in August and
December
• Likert scale questions1 2 3 4
Strongly StronglyDisagree Agree
• Open-ended questions
25
Research example 2:Julie Weitzel’s masters thesis
• Students significantly improvedtheir willingness to communicate
• By far the most helpful strategy was circumlocution
01234
PrePost
26
3. Creating IPAs and rubrics
• Creating an IPA:• http://www.carla.umn.edu/assessment/vac/in
dex.html > Create an assessment unit step-by-step
• Today we will all come up with an IPA idea!
27
Creating your IPA idea
• 1 minute: Think of a unit that you currently teach, but you are not 100% happy with the current final exam or final project. Write down the topic. Example: airport unit
• 1 minute: Describe the unit to a partner
28
Creating your IPA idea
• 2 minutes: write one “Students Will Be Able To (SWBAT)” statement for the content of your unit and one “SWBAT” statement for the language of your unitExample: middle school market unit on CARLA
29
Creating your IPA idea
• 2 minutes: with your partner, brainstorm an interpretive task for the unit.
• Students will read/listen to _____________ and answer questions/ complete an information chart/ other activity
30
Creating your IPA idea
• 2 minutes: with your partner, brainstorm an interpersonal speaking task where students compare their opinions about their information from the interpretive task
• 2 minutes: with your partner, brainstorm a presentational task for your theme. If you can’t think of one, have your students create a “webpage” (real or on paper!).
31
Creating a rubric (Sandrock)
Do include on a rubric• A description of a quality
performance– Clear expectations– Realistic expectations for
language use at proficiency level– Observable indicators– Criteria appropriate for the task– Criteria appropriate for the
mode of communication– List of elements that you want
students to think about in doing the task
Don’t include on a rubric• Non-negotiables (followed
directions, completed task)• Things that are not about
language (has nice drawings)
32
Creating a rubric
• Sample “master rubric” from Jefferson public schools
33
What is realistic?
34
Creating a rubric
• Splitting the master rubric for level 1 vs. level 3
35
3a. Scoring rubrics
• Handout p. 3 (Lackey & Weitzel, 2015)https://sites.google.com/site/assess mentmatterssbc/standards-based-learning
1. Create your rubric2. Put points in each box (1, 2, 3, 4)3. Figure out the minimum and maximum possible
points4. Put every possible point value across the top5. Assign 50% to the minimum value and 100% to the
maximum value
36
3a. Scoring rubrics
6. Time for some math!
50 divided by the number of jumps between possible point values = how many percentage points you go up between possible point values
50 / 9 = 5.55
50 + 5.55 = 55.6 55.6 + 5.55 = 61.1, etc.
37
4. Using rubrics multiple times to show growth
• Remember… it’s good to reuse the SAME RUBRIC!
• By using a rubric tied to the ACTFL proficiency levels, you can show growth over time
• Grading criteria can become more stringent without having to change the rubric itself
38
Last page of handout: different expectations for different levels
39
In conclusion…
• If you can invest the time to create 3 rubrics that cover the 3 modes of communication for all your proficiency levels, you can keep using them for YEARS!