University of Akureyri

Teachers’ Professional Development Conference

April 2007

Assessment for Learning Workshop: Part 2

The quality of questioning

Val Brooks

Why is questioning so important?

• Identified by Black and Wiliam (1998) as one of 4 key areas for further research & development

• OFSTED (1996) described questions as “the single most important factor in students’ achievement of high standards where questions were used to assess pupils’ knowledge and challenge their thinking”

• “Judicious questioning is nearly half the learning” (Jerome Bruner)

• “our most important intellectual tool” (Postman)

BUT• On average teachers ask 300+

questions each day• “Teachers do not generally review the

assessment questions that they use and do not discuss them critically with peers, so there is little reflection on what is being assessed”

(Black & Wiliam 1998: 17)

Key issues

• Questioning is a major feature of classroom life, occupying high proportion of time

• A powerful strategy: potential to enhance teaching, learning and assessment

• Questioning is employed in an intuitive and naturalistic manner and its true potential is not harnessed

Objectives for Workshop: Part 2

To extend your awareness of key research findings:– The problems and pitfalls associated with

conventional approaches– Strategies which are being developed to

improve the quality of questioning

• To identify strategies with which you may wish to experiment

Ways of conceptualising this session

Components of Questioning

• The question itself

• Posing the question

• Providing an answer

• Providing feedback

Issues to Consider

What kinds of questions

Who does the asking? How?

Who answers? How?

Who provides feedback? How?

Task

• Describe the approach (i.e. identify its component parts)

• In what ways is this approach helpful to teachers?

• What are the potential pitfalls with this approach?

Commonly used approach

The Approach• T broadcasts

question to entire class

• T accepts answer from a pupil, usually one who responds quickly

• T responds (i. e. repeats answer or modifies it)

• The lesson moves on

Research Findings• IRF cycle: one of

most commonly used forms of classroom discourse

• Initiation (T’s question)

• Response (Student’s answer)

• Feedback (T’s evaluation of answer)

Benefits

• Enables teachers to involve pupils & keep them engaged

• Gives teachers conversational control - who speaks, when, about what, for how long etc

• Teachers able to accomplish their plans but without relying on a didactic style of teaching

Consequences

• “[some] pupils don’t even try to think out a response – if you know that the answer, or another question, will come along in a few seconds, there is no point in trying. It is also common that only a few pupils in a class answer teachers’ questions. The rest then leave it to those few, knowing that they cannot respond as quickly and being unwilling to risk making mistakes in public. So the teacher… by accepting answers from a few can keep the lesson going but is out of touch with the understanding of most of the class – the question-answer dialogue becomes a ritual, one in which all connive and thoughtful involvement suffers.”

(Black and Wiliams 1998, pp. 11-12)

Potential Pitfalls• Feedback is obtained from a small, possibly

unrepresentative, sample of the class (e.g. John’s teacher missed a subtle form of feedback conveying confusion)

• Lesson moves on but how many are left behind???• When an answer is not provided quickly teachers either

– Re-phrase the question– Answer it themselves!

• Teachers provide insufficient wait time for students to process their ideas and formulate thoughtful answers

• “too often teachers seem scared of silence, so that they fail to allow pupils sufficient time to think” (OFSTED 1998: 80)

Consequences

• Lessons move at the pace of quickest-thinking children

• Others are left behind• Those who fail to keep pace are liable to

become disengaged and are more likely to become disruptive

• Feedback obtained from only a small group of children so the teacher loses touch with learning of many in the class

• T dominates the ‘talk’, talking far more than pupils

Categorising QuestionsClosed• Limited number of

permissible responses• Answers usually short• Require knowledge and

recall• Accelerates the pace of a

lesson• E.g. What is your name?

Open• A wider range of

possible responses• Elaborated answers• Require pupils to

explain or justify their thinking

• Encourage slower, more reflective pace

• E.g. How do you feel about your name?

Closed questions are useful for

Checking factual accuracy– Establishing how much pupils already

know about a topic before starting to teach

– Reviewing learning at the end of a lesson– Testing recall of prior learning at a later

stage– Formal tests and examinations

Benjamin Bloom’s Taxonomy: A Hierarchy of Thinking Skills

Higher order

• Evaluation

• Synthesis

• Analysis

Lower order

• Application

• Comprehension

• Knowledge

Research findings

• Closed/recall questions used more frequently than open questions

• Over-exposure to closed questions promotes lower order thinking

• Teachers are asking too many questions• Over-questioning encourages a less thoughtful,

more passive and dependent response in students

• Attainment is raised where pupils are given regular practice in higher order thinking (Black and Wiliam 1998)

Ways of categorising questions

Closed Open

Recall Thought

Response Information

-seeking -seeking

Response-seeking

T: Can anyone tell me why the litmus paper turned blue?

P1: Because it’s wet.

T: (Doubtfully) Ummm … not quite right.

P2: Because it was put into acid.

T: (Looks doubtful)

P3: Because it was put into alkaline.

T: Good! It turned blue when it was exposed to alkaline. Now, can someone tell me something you might find at home that is alkaline?

Response-seeking

T: What is a frog?

Silence

T: It’s an am …

Silence

T: It’s an amph …

Silence

T: It’s an amphib …

Silence

T: An amphibian!

“Assessment must therefore extend beyond the simple determination of the extent to which (students) have learned as intended to the discovery of what they have actually learned, right or wrong. Teachers, like marksmen, may have clear objectives, but if they are to improve the quality of their performance then - like marksmen - they will want to know where all their shots went, and not merely how many find their target. Indeed, the patterns of ‘wrong’ learning, like the distribution of off-target shots, will provide the clearest cues to improvement.“(Mary Simpson, 1990)

Information-seeking questions are useful for:

• Eliciting extended answers in which pupils have to explain or justify their thinking

• Discouraging guessing• Promoting higher order thinking• Close monitoring of progress during a

lesson• Diagnostic assessment

Giky Martables

The protruberances of the giky martables are perfunctory and enable them to search and easily locate their talibands. When the talibands are disard, they perform this ritual more often, sometimes in groups but more often in pairs or singly. They have even been known to excalibate the twigs and plants in their path, leaving chaos in their wake. In the early spring, if the temperature rises, they bimimer their dresarwars, sometimes travelling large distances before they macoca and cafuffle in woods or hedges.

(Black et al, Kings College, London)

Response-seeking

Pupils • focus on regurgitating

the required response• Shift of focus away

from learning and on to winning teacher approval

• Can often work out answers with little or no understanding of meaning

• Resort to rote/surface approaches to learning

• Resort to guesswork

Teachers• Are misled (These

questions “do not necessarily tell teachers what they need to know about their students’ learning” (B+W, 1998))

- May think that pupils really understand when they don’t

- May think that pupils can’t do something when they can

Feedback: Handling Inappropriate Answers

• Wragg (1997): over a third of answers received a non-verbal response or no reply

• OFSTED (1998:80) noted a tendency to be: ‘undemanding or over-sensitive to pupils’ feelings, accepting a wrong or partial answer rather than pressing the pupil to reflect and refine or go a stage further’.

• Torrance and Pryor (1998:18) also found that teachers, anxious to utilise infants’ contributions, accepted partial or irrelevant answers and avoided explicit correction of errors

Task • Analyse this transcript from Torrance and

Pryor’s investigation into formative assessment at KS1. (The appendix explains the transcribing conventions.)

• The transcript: a Year 2 whole-class Mathematics lesson. The teacher is revising work on tens and units before extending the work by introducing the concept of ‘rounding up’ to the nearest appropriate number.

• Evaluate the teacher’s questioning. How effective is it?

• How many of the pitfalls that we have identified can you detect?

Look out for• Too many questions/ unthinking use of

questioning • Over-use of closed/response-seeking and

insufficient use of open/information-seeking questions

• Pupils become passive/dependent/resort to guessing

• Tendency to ignore/provide inadequate feedback on inappropriate responses

• T dominating discourse• IRF pattern• T re-phrasing question or answering it• Insufficient wait time

Review of Session Objectives

To extend your awareness of:

• Some key research findings– The problems and pitfalls associated with

conventional approaches– Strategies which are being developed to

improve the quality of questioning (continued in next session)

• To identify strategies you may wish to experiment with

Assumptions underlying innovations in questioning

• Techniques are best suited to – open questions requiring higher order

thinking– An information-seeking approach– Being used in conjunction with one

another rather than in isolation

• Techniques not well-suited to – closed questions/lower order thinking – a response-seeking approach

Benjamin Bloom’s Taxonomy: A Hierarchy of Thinking Skills

Higher order

• Evaluation

• Synthesis

• Analysis

Lower order

• Application

• Comprehension

• Knowledge

Strategy 1: Applying Bloom’s Taxonomy

• KNOWLEDGE ( requires accurate recall – remembering information learnt previously )

• COMPREHENSION ( requires understanding of material – expressing ideas in their own words)

• APPLICATION ( requires transfer of ideas to a new context – understanding a general principle and applying it in a new situation)

• ANALYSIS (requires breaking a subject down into its parts - exploring the nature of the parts and their relationships with one another)

• SYNTHESIS (requires bringing ideas together for a new purpose – to create a theory, plan, experiment, forecast)

• EVALUATION (requires making judgements and explaining reasons for them – weighing the evidence and reaching conclusions)

Task

• Use Bloom’s taxonomy to categorise these questions

If you have time

• Pick a topic and devise 6 questions on it, one for each of the thinking skills

Strategy 2: Encouraging Thoughtful Behaviour

• Studies have found an average wait time of between 0.9 and 3 seconds

• Requiring an almost instant response only works well with knowledge/recall questions

• Some teachers are increasing their wait time, to allow students to formulate more thoughtful answers

Effects of increasing the wait time after a question:

• In one study the average wait time was about a second but: ‘where a longer silence was left – even as short as three seconds – the quality and extent of pupils’ responses improved dramatically… not only longer but also more thoughtful’ (Woods 1998:176).

• More pupils offer answers• The number of ‘I don’t know’ responses decreases• The number of hypothetical answers increases• The frequency of answers from less able students

increases• Students more likely to challenge and/or improve

each other’s answers

One teacher’s observation on his former approach to questioning

• “There must have been times (still are?) where an outside observer would see my lessons as a small discussion group surrounded by many sleepy onlookers”

• (James, Two Bishops School)

Strategy 3: Involving more learners

No hands’ policy

• Pupils are trained not to raise their hands to answer a question

• This strategy works well with increased thinking time/questions requiring more thought

• Everyone is expected to provide an answer if called upon

Requirements

• Well-thought out, clearly worded questions

• Sensitivity/attention to differentiation

• Classroom climate where admitting to uncertainty and making mistakes are seen as a natural part of learning

Other Strategies for Obtaining fuller feedback

• Pupil votes

• Answers written on hand-held dry-wipe boards

• True/false cards

• Traffic lights

• Thumbs

• Response partners

• P-I-P (Private-Intimate-Public)

Who does the asking?• One study found that teachers asked

between 6 and 12 times as many questions as pupils

• Pupils’ questions are often about trivial or procedural issues

• Pupils ask surprisingly few questions and surprisingly few of their questions are about learning!

Encouraging Learners to become Questioners

• B+W (1998) report an experimental study involving students in HE who were studying Physics.

• Treatment group: trained to generate their own thought-provoking questions and then try to answer them.

• Control group: relied on tutors’ questions.

• Which group performed best? Why?

Encouraging Learners to become Questioners

• Study of 2 outstanding teachers (B+W, 1998)

• Lessons characterised by unusually high frequency of questioning

• Over 60% of questions asked by pupils

Implications?

Developing a spirit of enquiry in the classroom

• Planning opportunities for pupil questioning into lessons (e.g. by ‘chunking’)

• Use of response groups• Mehrabian looked at features of

communication (7% - words, 38% - tone of voice, 55% - non-verbal communication)

• Welcoming body language (nodding, leaning forward, good eye contact, active listening)

• Making use of pupil questions in discussion, tests, homework etc

Developing a spirit of enquiry in the classroom

Eyre(1997) (Able Children in Ordinary Schools)

• In one school - at the end of a topic pupils were asked to write a list of questions about the topic to which they did not know the answer

• Some pupils’ questions went way beyond the level expected for their age

• Strategy capable of identifying the very able as well as those who are struggling

Task

• Contrast this example with the Torrance and Pryor example– How does the questioning differ?– How does the discourse differ?

Review of Session Objectives

To extend your awareness of:

• Some key research findings– The problems and pitfalls associated with

conventional approaches– Strategies which are being developed to

improve the quality of questioning

• To identify strategies with which you may wish to experiment