from summative to formative assessment in a traditional elt institute
TRANSCRIPT
FROM SUMMATIVE TO FORMATIVE ASSESSMENT IN A TRADITIONAL ELT
INSTITUTE
Isabela Villas BoasCláudio Fleury Sasse
Kátia Falcomer
OverviewContextTerminologyExperience / examplesQuestions / discussion
Casa Thomas Jefferson at a glance
Founded in 1963
17,000 + students
Six branches + school contracts + in-company
260 + teachers
Children - adults
Basic - post-advanced
Course Supervision - standardization
The “Flex” coursesAround 3,000 Ss
Started in 2004
Only adults
Four ten-week modules a year
A written and an oral test at the end
Typical adult students
Summative X Formative
Traditional X Alternative
Feedback
Alignment
Washback
Coombe et al. (2007, p. xix),
Tests or tasks administered at the end of the course to determine if students have achieved the objectives set out in the curriculum are called summative assessments. They are often used to decide which students move on to a higher level. Formative assessments, however, are carried out with the aims of using the results to improve instruction, so they are given during a course and feedback is provided to students.
Brown (2004): Formative assessment focuses on the process of “forming” students’ competencies and skills and aims at leading them to continued growth. Summative assessment measures, or summarizes, what has been learned at the end of a course or unit of instruction. It is about looking back and taking stock of what has been accomplished.
Wren (2008): The main distinction between the two types of assessment lies in the purpose of the assessment and how the results will be used.
Alternative assessment:
1) is built around topics or issues of interest to the students;
2) replicates real-world communication contexts and situations;
3) involves multi-stage tasks and real problems that require creative use of language rather than simple repetition;
4) requires learners to produce a quality product or performance;
5) includes evaluation criteria and standards which are known to the student;
6) involves interaction between assessor (instructor, peers, self) and person assessed;
7) allows for self-evaluation and self-correction as they proceed.
The National Capital Learning Resource Center (2004)
Alternative assessment asks students to show what they can do; students are evaluated on what they integrate and produce rather than on what they are able to recall and reproduce (Huerta-Macias, 1995, as cited in Coombe et al, 2007, p. xix)
It differs from traditional testing in that it:
- does not intrude on regular classroom activities;
- reflects the curriculum actually being implemented in the classroom;
- provides information on the strengths and weaknesses of each individual student;
- provides multiple indices that can be used to gauge student progress.
Timely Specific Understandable Actionable
Wiggins, G. (1998)
Effective Feedback
Washback: the effect of testing on teaching and learning
- Negative
- Positive
LEARNING OBJECTIVES
ASSESSMENTS
INSTRUCTIONAL ACTIVITIES
A L I G N M E N T
Eberly Center for Teaching Excellence, Carnegie Mellon University
Traditional assessment system
Written test + oral test at the end of ten-week module
Ten weeks of instruction generates a lot of content
Focus on grammar and written exercises
Student anxiety
More selected-response items on the test
Lack of alignment
Negative washback effect
Summative Assessment
Formative Assessment
Traditional Assessment
Alternative Assessment
Aims of the new assessment system
Very little feedback
Actionable feedback
Poor alignment Better alignment
Image by Stuart Miles @ freedigitalimages.net
New assessment system
1 2 3 4 5 6
Short assessments - 10 to 20 minutes# of points ranges from 12 to 202 oral assessments, necessarilyAt least 1 reading, 1 listening, and 1 writingMostly performance assessment of grammar
100 points
Last day of class: Final assessment daySs take the assessments they missed during the module or that they would like to retake (to improve grade)
Pilot groups
Phasing in
MonitoringFeedback
Adjustments
Steps in implementing the project
Building a shared vision - E-teacher course
Assessment
Summative and Formative Practices in Language Learning and Teaching
Three examples of assessments
Results of survey
Example of assessment: Speaking
Students will be able to talk about past experiences related to music and expand the conversation by providing details on the experience.
Ask your partner some questions about musical experience. Begin with a “have you ever” question and then ask some follow-up questions. Below are some ideas, but you can create your own questions about experiences related to music.
Have you ever been to a rock/jazz/pop concert?
Have you ever played in a band?
Have you ever traveled just to go to a concert?
_____________________________________________________________?
Instructions
Students practice as much as they need until they feel ready;Teacher can:- Walk around the classroom and assess
pairs- Ask pairs to present to the class when
they are ready- Have pairs record their dialogue; they
can listen to the recording and decide whether they want to redo it or not; different pairs can give feedback on each other’s recording and redo the assignment.
- Teacher assesses students by way of “real-time, almost surreptitious recording of student verbal and nonverbal behavior”(Brown, 2004. p. 267).
Example of assessment: Writing
Having learned how to provide basic personal information such as name, nickname, nationality, marital status, address, and phone number, and having analyzed a model paragraph in which this information is provided, students will write a paragraph about themselves.
Peer revision
Content:
4 - The paragraph addresses all the information requested in the prompt. 3 - The paragraph addresses some of the information requested in the prompt. 2 - The paragraph addresses little of the information requested in the prompt. Language use:
4 - Correct use of verb “to be” and articles. 3 – No more than three mistakes in the use of verb “to be” and articles. 2 - More than three mistakes in the use of verb “to be” and articles. Vocabulary:
4 - Correct use of vocabulary related to personal information. 3 – No more than three mistakes in the target vocabulary 2 - More than three mistakes in the target vocabulary. Mechanics:
4 - Correct spelling, punctuation, and capitalization. 3 – No more than three mistakes in spelling, punctuation, and capitalization. 2 - More than three mistakes in spelling, punctuation, and capitalization.
Example of assessment: Reading
Having worked on the reading about rapid urbanization and practiced locating and reading statistics to understand a writer’s point of view as a reading strategy, students will read a text about urbanization and megacities, complete a sentence about the topic of the text, and decide what the overall point of view of the author is.
Survey with intermediate-level students after first semester of experience (n=170)
A lot more in NS
A little more in NS
The same A little more in OS
A lot more in OS
Anxiety 4% 15% 36% 12% 19%Need to dedicate more time
20% 15% 34% 11% 13%
More oral communication
44% 19% 15% 4% 3%
Which is your favorite assessment system? (n=152) 2
New systemOld system
More communication in classContent is not accumulatedNo need to memorize rulesNeed to study more frequently = more effective learningLess stressfulMore accurate assessment of the student because it is ongoingThe oral assessments are less stressfulNo last-minute studying onlyLighter“Forces” students to come to classMakes me feel more comfortable in class
Image by photostock available at freedigitalphotos..com
Fewer assessments
A final test is more effective in measuring knowledge
Have the six assessments + final test
Image by Ambro available at freedigitalphotos.com
Survey with intermediate-level teachers after first semester of experience (n=18)
NS a lot more
NS a little more
The same OS a little more
OS a lot more
Effective for learning
78% 11% 6% 0% 0%
Practical for teacher
50% 17% 0% 11% 6%
Survey with intermediate-level teachers after first semester of experience (n=18)
Fully Satisfactorily Partiallly No
Understand reasons for change
89% 6% 0% 0%
Support change 61% 17% 6% 0%
No accumulation of content to assess.
Allows for remedial work and teacher self-assessment.
Student progress is more visible.
Immediate and personalized feedback to students.
Encourages more teacher reflection.
Students are assessed at their best.
Less stressful for students.
Possibility to re-teach and re-assess.
Image by photostock available at freedigitalphotos..com
Too many assessments.
Students keep asking if there will be an assessment the next class.
A little more time-consuming for teachers.
Difficulty with the oral assessments.
Image by Ambro available at freedigitalphotos.com
Stephen Stoynoff, ELT Journal – The Janus Papers (2012, pp. 527-528):
With the emerging dominance of a sociocultural paradigm in which learning is seen as a developmental, socially-constructed, interactive, and reflective process, classroom-based assessment will (among others):
- integrate the teacher fully into the assessment process;- yield multiple samples of learner performance that are collected over time and by
means of multiple assessment procedures and activities;- integrate learners into the assessment process and utilize self- and peer-assessment in
addition to teacher-assessment of learning;- offer learners immediate and constructive feedback;- monitor, evaluate, and modify procedures to optimize teaching and learning.
Brown, H.D. (2004). Language Assessment – Principles and Classroom Practices. White Plains, NY: Pearson Education.
Coombe, C., Folse, K., and Hubley, N. (2007). A Practical Guide to Assessing English Language Learners. Ann Arbor, Michigan: The University of Michigan Press.
National Capital Language Resource Center (NCLRC). (2004) Assessing learning: Alternative assessment. In The essentials of language teaching. Retrieved fromhttp://www.nclrc.org/essentials/assessing/alternative.htm
Stoynoff, S. (2012). Looking backward and forward at classroom-based language assessment. In ELT Journal, V. 66/4 – Special Issue: The Janus Papers, pp. 523-532.
Wren, D. (2008, November 6). Using Formative Assessment to Increase Learning.Research Brief: Report from the Department of Research, Evaluation and Assessment. Virginia Beach City Public Schools. Retrieved October 15, 2012 from http://www.vbschools.com/accountability/research_briefs/ResearchBriefFormAssmtFinal.pdf