reading: assessment and techniques prepared by marina solnyshkina and olga safonkina

44
READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonki

Upload: marianna-tucker

Post on 15-Jan-2016

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

READING:ASSESSMENT AND TECHNIQUES

Prepared by Marina Solnyshkina and Olga Safonkina

Page 2: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

WHY DO WE READ?WHAT DOES READING INVOLVE?TASK DEVELOPMENT ISSUES

Page 3: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Overview

• Why do we read?• What does reading involve?• Task development issues

Page 4: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Why do we read?

• For survival: depends on the day-to-day needs of the reader and often involves an immediate response to a situation.

• For learning: used in the classroom and is goal orientated.

• For pleasure: is something that does not have to be done.

• For assessment: to check the validity of something.

Page 5: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

• Why we read (and what we read) has an important influence on HOW we read.

• Reading is goal-driven: the goal of reading affects the process of reading.

Why do we read?

Page 6: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Reading as activity

is “…an enjoyable, intense, private activity, from which much pleasure can be derived, and in which one can become totally absorbed” (Alderson, 2000:28)

is “the activity of reconstructing the messages that reside in printed text” (Carroll 1964)

Page 7: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Reading as process

Page 8: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

The central ideas behind reading:

• the idea of meaning;• the transfer of meaning from one mind to

another; • the transfer of a message from writer to

reader; • how we get meaning by reading; • how the reader, the writer and the text all

contribute to the process. (Nuttall, 1996)

Page 9: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

What does reading involve?

The type of reading varies according to a number of factors: • purpose for reading (pleasure vs work vs survival);• main ideas comprehension (global) vs. identification of

specific information (local);• our level of reading ability (different approaches);• the difficulty level of grammar and syntax and vocabulary;• the text topic (our familiarity with the topic - regardless of

language ability);• the type of text we are engaged with (eye movements are

different and processing is different when we read narrative rather than argumentative texts).

Page 10: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

• eye movement – linear fashion vs. zigzagging through a text;

• different levels of coverage – grazing vs. detailed reading;

• the working memory keeps new information active 1-2 seconds while it carries out the appropriate process;

• level of fluency;• automatic vs. controlled reading.

What does reading involve?

Page 11: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

• fluent reading comprehension involves automatic word recognition;

• fluent L1 readers can recognize almost all the words they encounter (98 – 100%); 4-5 words per seconds;

• a fluent reader manages the following at the same time:- rapid recognition of words;- analysis of the structure of sentences;- building up a main idea model of text comprehension;- monitoring what has been comprehended;- calling in appropriate schemata from long-term memory.

• reading speed.

Fluency:

Page 12: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

What does reading involve?

Controlled vs. automatic processing:• “…automatic processing requires little/no attention and as such is less

likely to interfere with other processes at work; controlled processes require attention and it is difficult to deal with more than one source of info at a time” (Nagle, S. J., & Sanders, S. L. (1986). Comprehension theory and second language pedagogy. TESOL Quarterly, 20(1), 9-26.)

Automatic vs. controlled reading:• when something is new, a learner pays it conscious attention,

therefore processing is slow;• as the input becomes more and more familiar, processing becomes

faster and eventually automatic;• processing periodically breaks down causing the reader to have to re-

read, employ guessing strategic etc.

Page 13: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

• Recently, there has been a move towards a more interactive, hermeneutic approach, one that assumes a degree of bidirectionality in these processes (Hudson, 1998).

• Reading is an interactive process (a conversation between writer/reader, even though the writer is not present) and for it to occur two processes are necessary, top-down to predict the meaning and bottom-up to check it.

What does reading involve?

Page 14: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

• Bottom-up processes: the reader is operating from the visual data in the text = “information processing” (with a focus on the processing of more micro-level constituents of texts – letter, words, phrases, sentences etc.);

• Top-down processes: readers call upon their background knowledge and /or use contextual information = “analysis-by-synthesis” (with a focus more on macro-level constituents – genre, text structure, as well as the role of background schematic knowledge etc.).

What does reading involve?

Page 15: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Comprehension through decoding (word-level) and focusing on vocabulary or grammar. For example:

• identifying the sound/symbol correspondence in English, and its varieties;

• recognizing “sight words” - common words that are read quickly and easily;

• recognizing what part of speech a word functions as and the systems of tense, agreement, and pluralization;

• recognizing cohesive devices and their function in signaling relationships in texts.

How can we assess these skills?

The bottom-up skills

Page 16: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Comprehension through using prior content knowledge / knowledge of the genre (textual schemata) / using knowledge of language. For example:

• recognizing rhetorical conventions of writing;• recognizing the function of types of writing;• inferring context and connections between ideas from writing;• distinguishing between literal and implied meanings;• recognizing and interpreting culturally specific references in

texts;• using reading skills (skimming, scanning, guessing meaning from

context) to help in the interpretation of texts.How can we assess these skills?

The top-down skills

Page 17: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

What does reading involve?TYPE OF READING Creating a text level structure

Construct an organized representation of the text

Text structure knowledgeGenreRhetorical tasks

Careful reading LocalUnderstand sentence

GlobalComprehend main ideascomprehend overall textcomprehend overall texts

Building a mental modelIntegrating new informationEnriching the proposition

Expeditious reading LocalScan/search for specificsGlobalSkim for gistSearch for main ideas and important details

Inferencing General knowledge of the worldTopic knowledgeEstablishing propositional meaning

At clause and sentence level

Parsing Syntactic knowledgeLexical access Word recognition Lexicon:

LemmaMeaning/word classFormOrthography/phonology/morphology

Visual input

Kevin
All lines are capitalised except two under "Careful reading"
Page 18: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

WHAT MAKES READING EASY OR DIFFICULT?

Page 19: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

What makes reading difficult?

• Grammar and syntax• Vocabulary• Topic• Background knowledge• The length of text

Page 20: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Text requirements

Three main criteria when choosing texts for tests to be used in the classroom:

• suitability of content• exploitability• readability.

Page 21: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Suitability of content

The reading material should be interesting for students and relevant to their needs and it should motivate:

• Does the text interest the student? • Is it relevant to the student’s needs? • Does it represent the type of material that

the student will use outside of the classroom?

Page 22: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Exploitability

How the text can be used to develop the students’ competence as readers (can be exploited for teaching purposes to be useful in the classroom):

• Can the text be exploited for teaching purposes? • For what purpose should the text be exploited? • What skills/strategies can be developed by

exploiting the text?

Page 23: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Readability

• Used to describe the combination of structural and lexical difficulty of a text, as well as referring to the amount of new vocabulary and any new grammatical forms present.

• It is important to assess the right level for the right students.

Page 24: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Factors to measure readability:• speed of perception;• perceptibility at a distance;• perceptibility in peripheral vision;• visibility;• the reflex blink technique;• rate of work (e.g. speed of reading);• eye movement;• fatigue in reading.

http://www.instantfundas.com/2011/09/test-your-document-readability-in.html

Readability

Page 25: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Text requirements (Cont.)

Authentic material: “…real-life texts, not written for pedagogic purposes” (Wallace 1992:145) “…materials that have been produced to fulfill some social purpose in the language community.” (Peacock (1997), in contrast to non-authentic texts that are especially designed for language learning purposes. •Texts written for native speakers and contain “real” language. •Topics within students’ experience, but not too familiar. •Topics not biased or upsetting for any student.•No more than 5%-10% unknown words for good level of comprehension.

Page 26: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

READING AT DIFFERENT ABILITY

LEVELS

Page 27: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

CEFR Levels: A1

Page 28: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

CEFR Levels: B1

Page 29: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

CEFR Levels: C1

Page 30: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

TESTING READING AT DIFFERENT LEVELS

Khalifa and Weir (2009)

Page 31: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina
Page 32: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

32

The cognitive demands imposed by relative text complexity at each stage

Overall Number of words

Lexis Structure

KET

Approximately 740-800words

Restricted to common items which normally occur in the everyday vocabulary of native speakers.

Mainly simple sentences

PET

Approximately 1460-1590 words

General vocabulary sufficient for most topics in everyday life.

Mostly simple sentences but some use of relative and other subordinate clauses.

FCE

Approximately 2000 words

Good range of vocabulary. Topics are addressed in detail and with precision.

A range of sentence patterns– from the simple to the complex.

CAE

Approximately 3000 words

Broad range of vocabulary including idiomatic expressions and colloquialisms as well as language relating to opinion, persuasion and ideas.

This level is typified by: many complex sentences Frequent use of modals Some use of ellipsis Complex approaches to referencing – range of pronouns and adverbials, as well as use of synonymy.

CPE

Approximately 3000 words

Very wide range of vocabulary including idiomatic expressions and colloquialisms as well as language relating to opinion, persuasion and abstract ideas.

Most sentences are long and complex. No restriction on the types of structure employed by the text. Many examples of structures typically used for effect in writing – sentences with several subordinate clauses, for example.

Kevin
No gap in top left box in 800words.
Kevin
On line for CAE in final column, the sentences all start shifted to the right compared to the rest of the table.
Page 33: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

33

Types of reading tested at levels A2 to C2

KET A2 PET B1 FCE B2 CAE C1 CPE C2 Word recognition * * * * * Lexical access * * * * * Parsing * * * * * Establishing propositional meaning * * * * * Inferencing (*) * * * * Building a mental model (*) * * * * Creating a text level structure * * Creating an organised representation of several texts

*

Page 34: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

34

Text length

Examination Overall Number of words Number of texts

Maximum for any single text

KET (A2) Approximately740-800 words 4 250 PET (B1) Approximately 1450-1600 words 5 550 FCE (B2) Approximately 2000 words 3 700 CAE (C1) Approximately 3000 words 6 1100 CPE (C2) Approximately 3000 words 9 1100

Page 35: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

35

Examination Discourse Mode KET (A2) Genre : public signs & notices,(such as those found on roads, railway stations, airport);

newspapers & magazines (e.g., reviews, letters, consumer information, advertisements); informational sources (e.g., encyclopaedias, leaflets, brochures) Rhetorical Task : Descriptive, narrative, instructive

PET (B1) Genre : public signs & notices,(such as those found in shops, banks, restaurants); personal messages (text messages, notes, postcards, emails); newspapers & magazines (e.g., reviews, letters, consumer information, advertisements); informational sources (e.g., web pages, simple encyclopaedias, leaflets, brochures) Rhetorical Task: Descriptive, narrative, expository, instructive.

FCE (B2)

Genre: newspapers & magazines (e.g., articles, reports), fiction books (extracts), informational sources (e.g., guides, manuals) Rhetorical Task: Descriptive, narrative, expository, argumentative, instructive

CAE (C1) Genre : newspapers, magazines & journals (e.g., articles, reports), fiction & non-fiction books (extracts), promotional and informational sources (e.g., guides, manuals) Rhetorical Task: Descriptive, narrative, expository, argumentative. Instructive

CPE (C2) Genre : newspapers, magazines & journals (e.g., articles, reports, editorials), fiction & non-fiction books (extracts), promotional and informational sources (e.g., guides, manuals) Rhetorical Task: Descriptive, narrative, expository, argumentative, instructive

Kevin
Mixed capitalisation after colons. Commas after etc. for no apparent reason.
Kevin
"Rhetorical task" looks like it belongs to the following section, not the previous due to gap/proximity to following element.
Page 36: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

1. Choose a text:• it can be any genre that is relevant to your

students;• the length and difficulty will be determined by

the level of the students’ language proficiency.2. Choose a task that students can complete after

reading the text and that matches a realistic purpose for reading the text.

Steps in reading assessment

Page 37: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Test construction: what do we want to test?3. Decide the different types of reading to assess:

• skimming, scanning, or intensive reading.4. Decide the types of meaning:

• grammatical (syntax), informational, discourse, or pragmatic (meaning from the writer’s tone).

5. Define the types of comprehension required:• recognition, identification, analysis, interpretation,

inference, synthesis, or evaluation.This is the most common way to assess reading.

Steps in reading assessment

Page 38: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Level of engagement

GLOBAL Larger textual units (e.g. a series of paragraphs, or a whole text)

LOCALSections of a text (e.g. single sentences, or groups of sentences)

This refers to how much of a text (or texts) a reader is required to engage with in the performing of a prescribed task.

Page 39: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Type of engagementCAREFULUnderstanding the text - Explicitly stated main idea(s) and / or distinguishing that from supporting details - Locating, identifying, understanding and comparing facts, opinions, definitions (this includes search reading strategies) - Logical organisation of the text, e.g. understanding relationships among ideas in a text (problem – solution, cause – effect, temporal sequence, etc.) -Cohesive relationships (reference, ellipsis, substitution, conjunction, lexical cohesion) - Making propositional inferences (deducing information that is not explicitly stated from information that is explicitly stated)

EXPEDITIOUSSkimming / reading for gist - Identifying text type - Identifying text topic - Identifying text purpose

Scanning / reading for detail - Finding specific details (e.g. names, figures, dates, any other surface-level information)

Page 40: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Macro skills:• reading quickly to skim for

gist, scan for detail;• establish general

organization;• reading thoroughly for

main ideas, supporting details, argument, purpose, relationship of paragraphs, fact vs. opinion, etc;

• information transfer from nonlinear texts.

Assessment strategies

Micro skills:• understanding at the

sentence level - syntax, vocabulary, cohesive markers;

• understanding at inter-sentence level - reference, discourse markers;

• understanding components of nonlinear texts - labels, captions, symbols.

Skills to assess

Page 41: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

As reading is a receptive skill, it is difficult to assess in isolation, without using other skills.It is important to recognize when other skills, such as writing, are also being assessed with reading:•For example, if a student reads a text and then writes a summary, the skills of reading and writing are utilized and therefore assessed.Scoring can either:•focus only on the reading and only assess the content of the response, not the correctness of the writing, or•focus on both reading and writing, calling this an integrated assessment.

Integrated nature of reading

Page 42: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Comprehension questions:•Multiple-choice questions are the most practical but it is difficult to write these types of questions reliably.•Short answer questions are easier to write but also assess students’ writing ability and can be more difficult to score.Summary writing:•Students can choose the best summary of the reading from 3 or 4 options. This only assesses their reading skill.•Students can write a summary, which also assesses their writing ability (and writing summaries is hard!).

Tasks for reading assessment

Page 43: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

True/False Statements The number of the line giving supporting evidence.Matching Words with their explanations, summarising sentences with passages.

Tasks for reading assessment

Page 44: READING: ASSESSMENT AND TECHNIQUES Prepared by Marina Solnyshkina and Olga Safonkina

Metacognitive judgments:•Students determine which retelling is appropriate for different audiences.•This helps assess their ability to read tone and register of a text.•This is a very difficult skill!Question selection or writing:•Students choose questions that will help a peer understand the important ideas in the text.•Students write questions that will help a peer understand the important ideas in the text. This also assesses the students’ writing ability.

Tasks for reading assessment