from summative to formative assessment in a traditional elt institute

FROM SUMMATIVE TO FORMATIVE ASSESSMENT IN A TRADITIONAL ELT

INSTITUTE

Isabela Villas BoasCláudio Fleury Sasse

Kátia Falcomer

OverviewContextTerminologyExperience / examplesQuestions / discussion

Casa Thomas Jefferson at a glance

Founded in 1963

17,000 + students

Six branches + school contracts + in-company

260 + teachers

Children - adults

Basic - post-advanced

Course Supervision - standardization

The “Flex” coursesAround 3,000 Ss

Started in 2004

Only adults

Four ten-week modules a year

A written and an oral test at the end

Typical adult students

Summative X Formative

Traditional X Alternative

Feedback

Alignment

Washback

Coombe et al. (2007, p. xix),

Tests or tasks administered at the end of the course to determine if students have achieved the objectives set out in the curriculum are called summative assessments. They are often used to decide which students move on to a higher level. Formative assessments, however, are carried out with the aims of using the results to improve instruction, so they are given during a course and feedback is provided to students.

Brown (2004): Formative assessment focuses on the process of “forming” students’ competencies and skills and aims at leading them to continued growth. Summative assessment measures, or summarizes, what has been learned at the end of a course or unit of instruction. It is about looking back and taking stock of what has been accomplished.

Wren (2008): The main distinction between the two types of assessment lies in the purpose of the assessment and how the results will be used.

Alternative assessment:

1) is built around topics or issues of interest to the students;

2) replicates real-world communication contexts and situations;

3) involves multi-stage tasks and real problems that require creative use of language rather than simple repetition;

4) requires learners to produce a quality product or performance;

5) includes evaluation criteria and standards which are known to the student;

6) involves interaction between assessor (instructor, peers, self) and person assessed;

7) allows for self-evaluation and self-correction as they proceed.

The National Capital Learning Resource Center (2004)

Alternative assessment asks students to show what they can do; students are evaluated on what they integrate and produce rather than on what they are able to recall and reproduce (Huerta-Macias, 1995, as cited in Coombe et al, 2007, p. xix)

It differs from traditional testing in that it:

- does not intrude on regular classroom activities;

- reflects the curriculum actually being implemented in the classroom;

- provides information on the strengths and weaknesses of each individual student;

- provides multiple indices that can be used to gauge student progress.

Timely Specific Understandable Actionable

Wiggins, G. (1998)

Effective Feedback

Washback: the effect of testing on teaching and learning

- Negative

- Positive

LEARNING OBJECTIVES

ASSESSMENTS

INSTRUCTIONAL ACTIVITIES

A L I G N M E N T

Eberly Center for Teaching Excellence, Carnegie Mellon University

Traditional assessment system

Written test + oral test at the end of ten-week module

Ten weeks of instruction generates a lot of content

Focus on grammar and written exercises

Student anxiety

More selected-response items on the test

Lack of alignment

Negative washback effect

Summative Assessment

Formative Assessment

Traditional Assessment

Alternative Assessment

Aims of the new assessment system

Very little feedback

Actionable feedback

Poor alignment Better alignment

Image by Stuart Miles @ freedigitalimages.net

New assessment system

1 2 3 4 5 6

Short assessments - 10 to 20 minutes# of points ranges from 12 to 202 oral assessments, necessarilyAt least 1 reading, 1 listening, and 1 writingMostly performance assessment of grammar

100 points

Last day of class: Final assessment daySs take the assessments they missed during the module or that they would like to retake (to improve grade)

Pilot groups

Phasing in

MonitoringFeedback

Adjustments

Steps in implementing the project

Building a shared vision - E-teacher course

Assessment

Summative and Formative Practices in Language Learning and Teaching

Three examples of assessments

Results of survey

Example of assessment: Speaking

Students will be able to talk about past experiences related to music and expand the conversation by providing details on the experience.

Ask your partner some questions about musical experience. Begin with a “have you ever” question and then ask some follow-up questions. Below are some ideas, but you can create your own questions about experiences related to music.

Have you ever been to a rock/jazz/pop concert?

Have you ever played in a band?

Have you ever traveled just to go to a concert?

_____________________________________________________________?

Instructions

Students practice as much as they need until they feel ready;Teacher can:- Walk around the classroom and assess

pairs- Ask pairs to present to the class when

they are ready- Have pairs record their dialogue; they

can listen to the recording and decide whether they want to redo it or not; different pairs can give feedback on each other’s recording and redo the assignment.

- Teacher assesses students by way of “real-time, almost surreptitious recording of student verbal and nonverbal behavior”(Brown, 2004. p. 267).

Example of assessment: Writing

Having learned how to provide basic personal information such as name, nickname, nationality, marital status, address, and phone number, and having analyzed a model paragraph in which this information is provided, students will write a paragraph about themselves.

Peer revision

Content:

4 - The paragraph addresses all the information requested in the prompt. 3 - The paragraph addresses some of the information requested in the prompt. 2 - The paragraph addresses little of the information requested in the prompt. Language use:

4 - Correct use of verb “to be” and articles. 3 – No more than three mistakes in the use of verb “to be” and articles. 2 - More than three mistakes in the use of verb “to be” and articles. Vocabulary:

4 - Correct use of vocabulary related to personal information. 3 – No more than three mistakes in the target vocabulary 2 - More than three mistakes in the target vocabulary. Mechanics:

4 - Correct spelling, punctuation, and capitalization. 3 – No more than three mistakes in spelling, punctuation, and capitalization. 2 - More than three mistakes in spelling, punctuation, and capitalization.

Example of assessment: Reading

Having worked on the reading about rapid urbanization and practiced locating and reading statistics to understand a writer’s point of view as a reading strategy, students will read a text about urbanization and megacities, complete a sentence about the topic of the text, and decide what the overall point of view of the author is.

Survey with intermediate-level students after first semester of experience (n=170)

A lot more in NS

A little more in NS

The same A little more in OS

A lot more in OS

Anxiety 4% 15% 36% 12% 19%Need to dedicate more time

20% 15% 34% 11% 13%

More oral communication

44% 19% 15% 4% 3%

Which is your favorite assessment system? (n=152) 2

New systemOld system

More communication in classContent is not accumulatedNo need to memorize rulesNeed to study more frequently = more effective learningLess stressfulMore accurate assessment of the student because it is ongoingThe oral assessments are less stressfulNo last-minute studying onlyLighter“Forces” students to come to classMakes me feel more comfortable in class

Image by photostock available at freedigitalphotos..com

Fewer assessments

A final test is more effective in measuring knowledge

Have the six assessments + final test

Image by Ambro available at freedigitalphotos.com

Survey with intermediate-level teachers after first semester of experience (n=18)

NS a lot more

NS a little more

The same OS a little more

OS a lot more

Effective for learning

78% 11% 6% 0% 0%

Practical for teacher

50% 17% 0% 11% 6%

Survey with intermediate-level teachers after first semester of experience (n=18)

Fully Satisfactorily Partiallly No

Understand reasons for change

89% 6% 0% 0%

Support change 61% 17% 6% 0%

No accumulation of content to assess.

Allows for remedial work and teacher self-assessment.

Student progress is more visible.

Immediate and personalized feedback to students.

Encourages more teacher reflection.

Students are assessed at their best.

Less stressful for students.

Possibility to re-teach and re-assess.

Image by photostock available at freedigitalphotos..com

Too many assessments.

Students keep asking if there will be an assessment the next class.

A little more time-consuming for teachers.

Difficulty with the oral assessments.

Image by Ambro available at freedigitalphotos.com

Stephen Stoynoff, ELT Journal – The Janus Papers (2012, pp. 527-528):

With the emerging dominance of a sociocultural paradigm in which learning is seen as a developmental, socially-constructed, interactive, and reflective process, classroom-based assessment will (among others):

- integrate the teacher fully into the assessment process;- yield multiple samples of learner performance that are collected over time and by

means of multiple assessment procedures and activities;- integrate learners into the assessment process and utilize self- and peer-assessment in

addition to teacher-assessment of learning;- offer learners immediate and constructive feedback;- monitor, evaluate, and modify procedures to optimize teaching and learning.

Brown, H.D. (2004). Language Assessment – Principles and Classroom Practices. White Plains, NY: Pearson Education.

Coombe, C., Folse, K., and Hubley, N. (2007). A Practical Guide to Assessing English Language Learners. Ann Arbor, Michigan: The University of Michigan Press.

National Capital Language Resource Center (NCLRC). (2004) Assessing learning: Alternative assessment. In The essentials of language teaching. Retrieved fromhttp://www.nclrc.org/essentials/assessing/alternative.htm

Stoynoff, S. (2012). Looking backward and forward at classroom-based language assessment. In ELT Journal, V. 66/4 – Special Issue: The Janus Papers, pp. 523-532.

Wren, D. (2008, November 6). Using Formative Assessment to Increase Learning.Research Brief: Report from the Department of Research, Evaluation and Assessment. Virginia Beach City Public Schools. Retrieved October 15, 2012 from http://www.vbschools.com/accountability/research_briefs/ResearchBriefFormAssmtFinal.pdf

isabela.villasboas@thomas.org.br

claudios@thomas.org.br