1 developing assessments for the second language classroom carla, summer institute, 2006 standards,...
TRANSCRIPT
![Page 1: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/1.jpg)
1
Developing Assessments Developing Assessments for the Second Language for the Second Language Classroom Classroom
•
CARLA, Summer Institute, 2006
Standards, Guidelines, Rating
![Page 2: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/2.jpg)
2
Defining and measuring Defining and measuring outcomesoutcomes
Guidelines, standards, and objectives
![Page 3: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/3.jpg)
3
Standard/Guideline TypesStandard/Guideline Types• ACTFL proficiency Guidelines - four skills• Language Specific Guidelines (Provisional)• ILR (Interagency Language Roundtable) • CEF (Common European Framework of
Reference for Languages)Linguafolio
• National Standards for Language Learning for the 21st Century (K-12/16)
• Performance Guidelines for K-12 Learners (ACTFL)
• ESL Standards
![Page 4: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/4.jpg)
4
StandardsStandards
• s
![Page 5: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/5.jpg)
5
Communication StandardCommunication Standard
Modes of Communication• 1.1 Interpersonal mode (spontaneous,
negotiated, two-way)
Speaking, writing (e-mail, text messaging)
• 1.2 Interpretive (one-way, understand and interpret)
• 1.3 Presentational (one-way, audience, formal)
Speaking, writing
![Page 6: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/6.jpg)
6
![Page 7: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/7.jpg)
7
![Page 8: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/8.jpg)
8
Source: Participants, CARLA Summer Institute, 2003
![Page 9: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/9.jpg)
9
Level - length Level - length of study of study relationship in relationship in K-12 classesK-12 classes
![Page 10: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/10.jpg)
10
Defining Expectations: Defining Expectations: Benchmarks/levelsBenchmarks/levels
Year 1
Year 2
Year 3
Year 4
![Page 11: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/11.jpg)
11
Measuring ExpectationsMeasuring Expectations
• Appropriate assessments – provide evidence that students have achieved what we want them to know and be able to do
• Appropriate rating criteria –fair, students know what is expected, provide feedback to students and stakeholders
![Page 12: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/12.jpg)
12
Rating methodsRating methods
Objective and subjective evaluation
If we view objectivity and subjectivity of evaluation along a continuum, we can represent various assessment and scoring methods along its length.
![Page 13: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/13.jpg)
13
• Objective – traditional assessments
Pros Cons
• Subjective – authentic assessments
Pros Cons
![Page 14: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/14.jpg)
14
Rater reliabilityRater reliability
In assessment, we strive to ensure two types of reliability:
inter-rater (raters agree with each other) and
intra-rater (a rater gives the same score to a performance rated on separate occasions)
![Page 15: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/15.jpg)
15
Rubrics & ScalesRubrics & Scales
• Scales and rubrics differ in their specificity: scales generally can be applied to more than one task; rubrics are more closely aligned with particular tasks.
• Holistic rubrics and scales are used to assign one integrated evaluation to a performance, e.g., ACTFL Proficiency Guidelines.
• Analytic rubrics and scales are used to evaluate the various dimensions of a performance.
![Page 16: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/16.jpg)
16
ChecklistsChecklists
• Less informative than scaled rubrics• Efficient• Record observed performance• Use simplifies rubric construction by
specifying non-negotiables (Donna Clementi, 2002)
![Page 17: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/17.jpg)
17
ChecklistsChecklists
Use checklists to:
• Record observed performance.
• Keep track of progress over time, e.g., inventory of skills at beginning and end of courses.
![Page 18: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/18.jpg)
18
Receptive oral skills – sample Receptive oral skills – sample checklistchecklist
Understands simple directions. Understands simple sentences. Understands simple yes/no questions. Understands vocabulary appropriate to age. Understands meaning of different intonation
patterns. Understands more complex directions. Understands rapid speech. Understands language in classroom situation. Understands language of peers.
http://www.carla.umn.edu/assessment/vacAdapted from Genesee, F. & Upshur, J.A. (1996). Classroom-based evaluation in
second-language education. Cambridge: Cambridge University Press, p. 88.
![Page 19: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/19.jpg)
19
Advantages and DisadvantagesAdvantages and Disadvantages
Advantages
• Easy to construct and use. • Align closely with tasks. • Effective for self and peer assessment. • Make learners aware of task requirements,
allowing them to self-monitor progress. • Useful for sharing information with students
and other stakeholders.
(Brindley, 1989; Genesee & Upshur, 1996: Tedick, 2002; Underhill, 1987)
![Page 20: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/20.jpg)
20
DisadvantagesDisadvantages
• Provide limited information about how to improve performance.
• Do not indicate relative quality of performance.
![Page 21: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/21.jpg)
21
RubricsRubrics
• rating scales used with performance assessments
• increasingly used to evaluate performance assessments.
• provide an indication of quality of performance/student work
• Provide detailed feedback to learners
![Page 22: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/22.jpg)
22
Why use rubrics?Why use rubrics?
• Set anchor points along a quality continuum rather than right or wrong
• Increase construct validity
• Align assessment to curriculum and instruction
• Focus on the most salient goals
• Expectations are clearer to students (and to yourself)
![Page 23: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/23.jpg)
23
• Provide specific feedback to students
• Well-designed rubrics increase assessment reliability by setting criteria that can be consistently applied by raters
• The Virtual Assessment Center at CARLA provides an extensive tutorial on rubrics
http://www.carla.umn.edu/assessment/vac/Evaluation/p_4.html
![Page 24: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/24.jpg)
24
Rubrics should be:Rubrics should be:
• Clear - define excellent, good or work needing improvement
• Democratic – consistent, consistently reinforced, understood by students that grading is not subjective
• Involving – students should be involved in creating rubrics
• Engaging – allow students to evaluate own workFrom Blaz, C. (2001). A collection of performance assessment tasks and rubrics.
Eye on Education: Larchmont, NY.
![Page 25: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/25.jpg)
25
Types of RubricsTypes of Rubrics
![Page 26: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/26.jpg)
26
Compare:Compare:ACTFL Performance Guidelines for K-12 Learners
• Comprehensibility
• Comprehension of Language
• Control Vocabulary
• Cultural Awareness
• Communication Strategies
![Page 27: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/27.jpg)
27
To:To:Analytic Writing Scale for the Spanish FLIP Program, University of Minnesota*
Content (30 pts.)
Organization (20 pts.)
Language use/Grammar/ Morphology (25 pts.)
Vocabulary/Word usage (20 pts.)
Mechanics (5 pts.)
![Page 28: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/28.jpg)
28
Multitrait Rubric for Recipe Presentation
•Organization and Clarity
• Fluency and Pronunciation
• Use of the Imperative (or other form)
![Page 29: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/29.jpg)
29
Primary trait rubrics
• Performance is scored on the main criterion for success on the task.
Example: Task*: Write a persuasive letter to the editor of the school newspaper.
![Page 30: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/30.jpg)
30
Primary Trait: Persuading an audience*Primary Trait: Persuading an audience*
0 Fails to persuade the audience
1 Attempts to persuade but does not provide sufficient support.
2 Presents a somewhat persuasive argument but without consistent development and support
3 Develops a persuasive argument that is well developed and supported.* Tasks/rubrics can be found in Minnesota Articulation Project. (2002). Proficiency-oriented language instruction and assessment: A curriculum handbook for teachers (Rev Ed.). CARLA Working Paper Series. D. J. Tedick (Ed.). Minneapolis, MN: University of Minnesota, The Center for Advanced Research on Language Acquisition.
![Page 31: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/31.jpg)
31
Creating a RubricCreating a Rubric
Step 1: Generate potential dimensions• Rank order the potential dimensions
from most to least important; eliminate the "non-negotiables".
![Page 32: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/32.jpg)
32
Step 2Step 2:: Select a reasonable number of Select a reasonable number of dimensionsdimensions
3-7?, 4-8?• "How many is enough? … There's no one correct
answer, but it might help if you consider your purpose for this measurement. If it's diagnostic and formative, err on the side of more dimensions rather than fewer.
• If you just want to be able to give a summative
evaluation of your students' performance for this particular lesson, fewer dimensions are OK." (Triton/Patterns Summer Symposium. (1999). Creating A Rubric for a Given Task. San Diego City Schools.
Available online at http://projects.edtech.sandi.net/staffdev/tpss99/rubrics/rubrics.html
![Page 33: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/33.jpg)
33
DimensionsDimensions
Dimensions
Accomplished
Exceeds expectations
3
Excellent
Developing
Meets expectations
2
Average
Beginning
Not there yet
1
Needs work
![Page 34: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/34.jpg)
34
Write Benchmark descriptors
Dimensions
Accomplished
Exceeds expectations
3
Excellent
Developing
Meets expectations
2
Average
Beginning
Not there yet
1
Needs work
Benchmark DescriptionsBenchmark Descriptions
![Page 35: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/35.jpg)
35
Avoid:Avoid:• "Squishy" descriptors. For example, these
descriptors were used to evaluate essays:
3
shows depth
2
lacks depth
1
no depth
5
well-balanced
4
moderately well-balanced
3
not so well-
balanced
2
lack of balance
1
total lack of balance
![Page 36: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/36.jpg)
36
Unnecessary negative language:
3
Creative introduction
2
Adequate
introduction
1
Boring
introduction
![Page 37: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/37.jpg)
37
American Heritage American Heritage ® Dictionary® Dictionaryhttp://education.yahoo.com/reference/dictionary/entry/whine
• whine (hw n, w n)
VERB: whined , whin·ing , whines
• To utter a plaintive, high-pitched, protracted sound, as in pain, fear, supplication, or complaint.
• To complain or protest in a childish fashion.
• NOUN:
• The act of whining. • A whining sound. • A complaint uttered in a plaintive tone.
![Page 38: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/38.jpg)
38
Evaluating Rubrics
Scoring Rubric for Whining
• Language• Criteria• Dimensions• Practicality
An entry from the electronic portfolio of Joseph A. Braun, Jr., Curriculum and Instruction, Coe College
• http://coe.ilstu.edu/jabraun/professional/rubrics/whinrub.html
• *Source Tillamook School District
![Page 39: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/39.jpg)
39
Constructing a rubric to evaluate Constructing a rubric to evaluate an ESL speaking assessmentan ESL speaking assessment
According to the number on your sheet, in groups of three or four:
• please read the purpose for which you will use this speaking assessment (see section 4, binder)
• Read the ESL speaking assessment on the handout
• make a rubric with criteria that meet the assigned purpose(to consider level, consult ACTFL guidelines in your binder, if needed)
• Listen to the student, and rate the sample using the rubric your group produced.
![Page 40: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/40.jpg)
40
English CoSA/SOPI – Speaking
Meeting your new tandem partner
It's the second week of school, and the ESL program has arranged a reception for international students to meet their tandem partners. You are paired up with Paula, who is from southern Iowa, and studies linguistics.
Task 1
Paula wonders about you: where you're from, how old you are, what you're studying, and so on. Wait until you hear Paula's question, then tell her about yourself.
![Page 41: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/41.jpg)
41
Task 2 Paula asks you about your classes: she's curious about
the ESL program. You tell her which classes you're taking and your opinion of each class. You also tell her a few things that you do in your classes. Wait until you hear Paula's question, then describe your classes to her, and include the activities students do in each class.
Task 3You and Paula begin talking about dating in the United
States. She wants to know how you meet people in your home country, and what dating customs are like there: are parents involved, do people go out in groups, and so on. Wait until you hear Paula's question, then tell her about dating in your country.
![Page 42: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/42.jpg)
42
Task 4Later that day, you run into Paula in the library—you're
both tired of studying, and you start to talk about families. She tells you all about her family in Iowa, and then wants to know about yours: your sisters, brothers, parents, etc.: their ages, where they work, what they do in their spare time, and so on. Wait until you hear Paula's question, then describe your family to her.
Task 5You would like to learn more about Paula and her life in
Iowa: her friends, her interests, the town she lives in, etc. Wait until you hear Paula speak, then ask her at least five questions to find out more about her life in Iowa.
![Page 43: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/43.jpg)
43
Task 6You start thinking ahead to the weekend, and Paula
wonders what there is to do here at the University of Minnesota and in the Twin Cities. She wants to get to know the area and make some new friends. Wait until you hear Paula's question, then make at least 4 suggestions as to what she might do this weekend.
Task 7You and Paula decide that you will meet later for lunch to
make plans. Wait until you hear Paula's question, then tell her where you go to eat lunch, what you like to eat, and that you will see her later.
![Page 44: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/44.jpg)
44
Appropriateness of the Appropriateness of the instrumentinstrumentIn your group, answer the following:
How appropriate is this speaking assessment for your group’s defined goal?
How appropriate are the tasks?
What about face and construct validity?
Discuss in groups
![Page 45: 1 Developing Assessments for the Second Language Classroom CARLA, Summer Institute, 2006 Standards, Guidelines, Rating](https://reader034.vdocuments.mx/reader034/viewer/2022051621/56649c985503460f949540f0/html5/thumbnails/45.jpg)
45
Resources:Resources:
Rubrics and grades:
Fairfax County Public Schools
http://www.fcps.k12.va.us/DIS/OHSICS/forlang/PALS/rubrics/index.htm
Nebraska
http://www.nde.state.ne.us/FORLG/Frameworks/FLFAsses.pdf
Virtual Assessment Center (VAC)• http://www.carla.umn.edu/assessment/vac