Effectiveness and improvement

Teacher effectiveness (HayMcBer, 2000)

Research involved classroom observation, in-depth interviews, questionnaires, focus groups, as well as the collection of personal and school data. Pupil progress data were collected and analysed taking account of school context.

Teacher effectiveness (HayMcBer, 2000)

Findings:

Pupil Progress

ProfessionalCharacteristics Teaching

Skills

ClassroomClimate

Teacher effectiveness (HayMcBer, 2000)

Teaching skills

HomeworkHomework

High ExpectationsHigh Expectations

PlanningPlanning

Methods and Strategies

Methods and Strategies

Pupil Management/

Discipline

Pupil Management/

DisciplineTime & Resource

Management

Time & Resource Management

AssessmentAssessment

Lessonflow

Time ontask

Teacher effectiveness (HayMcBer, 2000)

Professionalcharacteristics

PROFESSIONALISMChallenge and Support -

Confidence - Creating Trust - Respect for Others

PROFESSIONALISMChallenge and Support -

Confidence - Creating Trust - Respect for Others

THINKINGAnalytical Thinking - Conceptual Thinking

THINKINGAnalytical Thinking - Conceptual Thinking

LEADINGFlexibility - Holding People

Accountable - Managing Pupils - Passion for Learning

LEADINGFlexibility - Holding People

Accountable - Managing Pupils - Passion for Learning

RELATING TO OTHERSImpact and Influence -

Teamworking - Understanding Others

RELATING TO OTHERSImpact and Influence -

Teamworking - Understanding Others

PLANNING & SETTING

EXPECTATIONSDrive for Improvement - Information Seeking -

Initiative

PLANNING & SETTING

EXPECTATIONSDrive for Improvement - Information Seeking -

Initiative

Teacher effectiveness (HayMcBer, 2000)

“Classroom climate is defined as the collective perceptions by pupils of what it feels like to be a pupil in any particular teacher's classroom, where those perceptions influence every student's motivation to learn and perform to the best of his or her ability.”

Teacher effectiveness (HayMcBer, 2000)

“Our findings suggest that, taken together, teaching skills, professional characteristics and classroom climate will predict well over 30% of the variance in pupil progress.”

What works (Barber, 2009)“Every system needs to create an environment in schools in which every lesson is a good one, and teaching is - in Michael Fullan's well-chosen word - "deprivatised" so teachers continuously improve their pedagogy. Experimenting on the basis of deep knowledge, continuous professional dialogue and rapid feedback will drive improvement, lead to bottom-up innovation, and enhance professional satisfaction.”

What doesn’t work (Barber, 2009)“Obsessions with policies that are wrong and expensive, such as continuing marginal reductions in class size or protecting teachers' "rights" to teach as they wish in the citadel of their own classrooms, is widespread. Many still cling to the demonstrably false view that creativity consists of each teacher making it up in the classroom. This is not creativity, it is betrayal.”

Definition of an effective school

“One in which pupils progress further than might be expected from consideration of its intake” (Mortimore, 1991, p.9)

How to judge performance “Over the past decade, increasingly complex

measures for gauging pupil attainment and progress in secondary schools have been introduced in England: from simple threshold measures of raw academic attainment at age 16 to contextualised value-added (CVA) models that take account of prior attainment and a wide range of contextual factors outside the control of schools” (Kelly and Downey, 2011, p. 415)

“Generally the evidence suggests that schools are having difficulty acting on the huge amount of data available and in particular turning it to the task of raising pupil outcomes” (Kelly and Downey, 2011, pp. 416-7)

Critique of measurements It is difficult to measure the impact of

“selection effects” ie families moving into the area in order to send their children to the most effective school or other factors, such as paid tutoring to support students (Timmermans et al, 2011).

Different types of school effects appear to measure the same underlying construct – so some value-added measures reflect raw results (Timmermans et al, 2011).

Approach to measurements

Timmermans et al (2011, p. 410) “advise the value-added models to include prior achievement, some indicators of the students’ socioeconomic background, and compositional characteristics of student population” – eg, student mobility, urban/rural, EAL.

CritiqueThis confirms Van de Grift’s (2009) findings: The use of different value-added models can

lead to different conclusions. Compositional effects need good quality data

on individual pupil’s background and achievements.

Schools in challenging circumstances are more likely to be ineffective and thus reinforce social disadvantage.

Identifying poor and outstanding schools is better than ranking all schools.

Critique

School outcomes are quite similar when adjusted for social factors (Ouston, 1999). Croxford and Cowie’s (1996) study of 38 Grampian secondary schools showed a difference of one grade between the average leaver in the most effective school to the least effective after adjusting for social factors.

What doesn’t work (Barber, 2009)

“The capacity of educators to stumble into a false dichotomy and debate it (vigorously and at length to the benefit of no one) is legendary - for example, the widely held but absurd view that because some things can't be measured, we should measure nothing.”

Do teachers make use of pupil performance data?

Kelly and Downey’s (2011) large scale online survey aimed to address this question. The survey was sent to 1346 schools. There were 813 teacher responses from 178 different schools.

Questions: How much confidence do you have in

your skills to access, utilise and interpret data?

What are the main reasons currently for collecting pupil attainment data?

To assist schools in the process of self-evaluation

To enable pupils to make better academic progress

To enable teachers to be more effective in their roles

To hold schools accountable to the public for the job they do

To hold teachers accountable for the job they do

To enable authorities and/or the media to rank schools based on performance

To enable parents to choose the best school for their children

To identify the relative performance of different groups of students within schools

Kelly and Downey (2011)“The responses from deputies and Key Stage leaders were in stark contrast to each other: 100% of the former felt that pupil performance data are presently collected to assist schools in the process of self-evaluation; 100% of the latter felt that it is done to enable students to make better progress” (pp. 427-8)

Kelly and Downey (2011)“Responsibility for the management, analysis and interpretation of data lies overwhelmingly within senior management teams and, while this approach may be expedient given the volume of data coming through to schools, it tends to lock up access and expertise within a narrow set of ‘gatekeepers’.” (p. 432)

Pressure to perform Former Chief Inspector of Schools, Chris

Woodhead, wrote: “How does one learn as a human being except through pressure and threat?” (Spectator, 1995).

“When pressure is applied to complex human systems such as school, two things seem to happen: first, pressure is usually passed on down the hierarchical levels, second, the operation of classrooms reverts to earlier more teacher-directed forms” (Watkins, 2010, p.3)

Pressure to perform

“A focus on learning can enhance performance whereas a focus on performance (alone) can depress performance” (Watkins, 2010, p. 4)

Factors present in effective schools High expectations by teachers of their learners. In

less effective schools there is a tendency to blame the intake of learners for poor performance, whereas other schools with a similar intake of learners which had a more positive view of their learners’ capabilities were more effective.

Academic emphasis: in these schools teachers give a high priority to preparation of learners, to giving helpful feedback to learners, and to monitoring and assessing/marking work.

Factors present in effective schools Shared vision and goals: effective schools had

shared views on issues such as marking, homework. The goals were clear and shared by all the staff.

Positive reinforcement: greater consistency of practice in learner behaviour and discipline was linked with more effective learning.

Professional leadership at the level not only of principals, but also other levels throughout the school, eg. Departmental level. This gives strong emphasis to teamwork and involves teachers in decision‑making, but provides a clear sense of direction and purpose.

Factors present in effective schools Quality in teaching: factors here include: work focus

in lessons, clarity of goals in learning, planning, challenge for learners of all ability levels, teacher enthusiasm, prompt start/finish to lessons. Staff experience, qualifications and knowledge of the subject, monitoring progress, and involvement on curriculum development were also features of the more effective schools.

A learning environment: an orderly atmosphere and an attractive working environment.

Monitoring progress: monitoring pupil performance; evaluating school performance.

Factors present in effective schools Parental support/involvement: effective schools

attempted to involve parents more than less effective schools, where parents were sometimes seen in a more negative light.

Pupil rights and responsibilities: raising pupil self-esteem; positions of responsibility; control of work. 

A learning organisation: including school-based staff development.

Sammons, Hillman and Mortimore (1995).

Critique

“If schools don’t differ, or we can’t measure these differences reliably, then the lists of the features of effective schools have little justification” (Ouston, 1999, p.169).

Critique

“Are the features of effective schools the result of being effective or the cause of effectiveness? A trivial example to illustrate this issue is that of the house-plants. The effective schools were pleasant places to be in. They were clean and cared for and had plants in the classrooms. If one had cleaned the classrooms of the less effective schools and given each teacher a house-plant, would the exam results have improved? I doubt it: the house-plants would probably have died.” (Ouston, 1999, p.168)

Critique Most studies in the west have been within

disadvantaged or deprived contexts, rather than with schools in advantaged areas.

There has generally been less of a focus on classroom processes.

There have only been rudimentary attempts at theory generation.

Reynolds et al, 1996, cited in Bush and Coleman, 2000, p. 52

Newspaper articles

Read the 3 articles and then in your groups discuss the following questions:

1. How do the views expressed in the articles compare with research evidence about effective schools?

2. What do the articles imply are the important factors in schools achieving success?

3. What might be the outcomes of an inappropriate assessment of a school’s effectiveness?

Barber’s (2002) analysisKnowledge poor Knowledge rich

External/national prescription

Uninformed prescription (eg when accountability and standards were first introduced in the ‘80s)

Informed prescription (eg literacy and numeracy hours in the ‘90s)

Professional judgement

Uninformed professional judgement (eg “permissive individualism” of the 70s (Hargreaves, 2003, p.129))

Informed professional judgement

School improvement

School improvement is a series of concurrent and recurring processes in which a school:

enhances learner outcomes focuses on teaching and learning assesses its current culture and works to

develop positive cultural norms builds the capacity to take charge of change

regardless of its source defines its own direction

School improvement has strategies to achieve its goals addresses the internal conditions that

enhance change maintains momentum during periods of

turbulence monitors and evaluates its process, progress,

achievement and development.

Stoll and Fink (1996, p.43)

The Leadership Purpose Improvement, then, is change with direction, sustained

over time, that moves entire systems, raising the average level of quality and performance while at the same time decreasing the variation among units, and engaging people in analysis and understanding of why some actions seem to work and others don’t.

Leadership is the guidance and direction of instructional improvement. This is a deliberately de-romanticised, focussed and instrumental definition.(Elmore, 2004, p.57)

IMPROVING DECLINING

EFFECTIVEMoving Cruising

Strolling

INEFFECTIVEStruggling Sinking

Stoll and Fink’s model (1996, p. 85)

Moving schools·     We have a clear and stated picture of

where we’re going·     We work together to respond to the

changing educational context·     We work together to keep developing·     We have the will to get there·     We have the skill to get there

Cruising schools·     We appear to possess many ‘school effectiveness’

qualities·     We appear effective to our parents, school

community, inspectors·     We are usually located in a socially and

economically advantaged area·     We do well in league table where ‘value added’ is

not measured·     Our pupils do well without too much effort on our

part·     We are marking time

Strolling schools

·     We don’t stand out as particularly effective or ineffective

·     Our aims are woolly or even conflicting

·     Our pace of improvement is inadequate to keep ahead of what’s required

Struggling schools

·     We know we’re not effective at the moment

·     We put a lot of effort into trying to improve

·     We’ll try anything that might make a difference

·     We’re open to ideas and help·     We’ll get there

Sinking schools

·     We’re isolated in a culture of self reliance and blame

·     Our staff are not prepared or able to change

·     We’re often located in a socially and economically disadvantaged area

·     We blame poor parenting and unprepared children

·     We’re failing

Towards large scale sustainable reformTowards large scale sustainable reform

Every School a Every School a Great SchoolGreat School

National National PrescriptionPrescription

Schools Leading ReformSchools Leading Reform

Building Capacity PrescriptionPrescription ProfessionalismProfessionalism

System Leadership

Developing the Organisation: Highly Differentiated Improvement Strategies (Hopkins, 2006)

Type of School Key strategies – responsive to context and need

Leading Schools- Become leading practitioners- Formal federation with lower-performing schools

Succeeding, self-improving schools

- Regular local networking for school leaders- Between school curriculum development

Succeeding schools with internal

variations

- Consistency interventions: such as AfL.- Subject specialist support to particular depts.

Underperforming schools

- Linked school support for underperforming depts. - Underperforming pupil programmes, e.g. catch-up.

Low attaining schools

- Formal support in Federation structure- Consultancy in core subjects and best practice

Below floor target- Intensive Support Programme- New provider: eg: Academy.

Critique

School improvement models assume that schools are rational organisations, which follow rational and strategic plans. But is this the case? Perhaps chaos theory would be more appropriate (Ouston, 1996).

How can you tell if a school is currently improving?

1. Recent improvement may be evident in the data, but that is no guarantee that improvement is currently taking place.

2. In primary schools, outstanding teachers made no significant difference to their pupils’ progress.

3. Teachers report some success in using pupils’ views of classroom climate to make improvements (Flecknoe, 2005).

 

School Improvement

Only about 10% of schools nationally improved consistently over a 3-year period (Gray et al, 1999).

In a study of 3200 Californian schools, only 16 maintained above average achievement for 3 years running (Joyce, Calhoun and Hopkins, 1999).

School Improvement

“The notion of effective methods has great political appeal, especially when linked to ideas of evidence-based practice imported from the field of medicine. But not more than 20 per cent of medical decisions and virtually no teaching decisions can be made on the basis of gold standard evidence from meta-analyses and randomised control trials. Educational diagnoses and treatments show so much natural variation that the construct of an effective method is highly questionable” (Eraut, 1999, p.119).

  

Role of leaders in promoting effectivenessThree of the ‘seven strong claims’ about successful

school leadership (Leithwood et al, 2008, pp.27-28)

1. School leadership is second only to classroom teaching as an influence on pupil learning.

3. The way in which leaders apply basic leadership practices demonstrate responsiveness to, rather than dictation by, the contexts in which they work.

4. School leaders improve teaching and learning indirectly and most powerfully through their influence on staff motivation, commitment and working conditions.

How strong are the 7 strong claims? Learning for which purposes.

Attainment? Achievement? Other kinds of learning?

What does being context-specific mean in practice?

How much empirical evidence is there for the 7 strong claims? (Morrison, 2009)

Michael Barber’s (2009) conclusions “There are many educators and leaders who

simply don't believe that successful change is possible, and academics who use sophisticated statistical techniques to support the view that social background remorselessly determines outcomes, regardless of what education systems do. Along with the former Russian prime minister Viktor Chernomyrdin, who left office after a frustrating year in the 1990s, they cry: "We tried to do better, but everything turned out as usual." The truth is that all too often we have defeated ourselves in our own heads before we've even begun.”

Michael Barber’s (2009) conclusions

“Evidence shows that there are ways of making systems work better. It's difficult, but not impossible, and requires a relentless focus on what works.”

Activity: Education Endowment Foundation Tool-kit

Look at the list of approaches, costs, evidence and impact. Choose two approaches from the overall list you think would have a positive impact and so worth investigating for your school and two which might be worth dropping.

Look at the additional information for each of these approaches and assess the evidence. a) What further questions would you ask? (ie, what more do you

need to know?) b) How might you develop this into a strategy for your school.

Readings for next week Eacott, S. (2011) Leadership strategies: re-

conceptualising strategy for educational leadership. School Leadership and Management, vol. 31, no. 1 , pp: 35-46

Bell, L. (2002). Strategic planning and school management: full of sound and fury, signifying nothing? Journal of Educational Administration, vol. 40, no. 5, pp. 407-424

Davies, B. (2006). Processes not plans are the key to strategic development. Management in Education, vol. 20, no. 2, pp. 11-15.

ReferencesBarber, M. (2002) From good to great: large-scale reform in England. Paper

presented at Futures of Education conference, Universitat Zurich, Zurich, Switzerland

Barber, M. (2009) How the school system should respond to a shrinking budget. The Guardian, 30 June 2009

Bush, T. and Coleman, M. (2000) Leadership and Strategic Management. London: Sage

Croxford, L. and Cowie, M. (1996) The effectiveness of Grampian Secondary Schools. Report of a research programme undertaken by the Grampian Regional Council and the Centre for Educational Sociology, Edinburgh. Edinburgh: CES

Eraut, M. (1999) ‘Headteacher’s knowledge, practice and code of cognition’. In Bush, T., Bell, L., Bolam, R., Glatter, R and Robbins, P. (eds) Educational management: redefining theory, policy and practice. London: Paul Chapman

Joyce, B., Calhoun, E. and Hopkins, D. (1999) The new structure of school improvement. Inquiring schools and achieving students. Buckingham: Open University Press

ReferencesElmore R (2004) School Reform from the Inside Out, Harvard: Harvard

Education PressFlecknoe, M. (2005) ‘The changes that count in securing school

improvement’ School effectiveness and improvement. Vol.16, no.4, pp425-443)

Hargreaves, A. (2003) Teaching in the Knowledge Society. Maidenhead: Open University Press

Hay McBer (2000) Research into Teacher Effectiveness. London: DfEE

Hopkins, D. (2006) ‘Leadership for 2020 – System Leadership and the Challenge of Personalisation’ Keynote presentation to Leadership 2020 conference, Augusta, Georgia, USA: 26 April 2006

Gray, J., Hopkins, D., Reynolds, D., Wilcox, B., Farrell, S. and Jesson, D. (1999). Improving schools: performance and potential. Buckingham: Open University Press

ReferencesKelly, A. and Downey, C. (2011) Professional attitudes to the use of pupil

performance data in English secondary schools. School effectiveness and school improvement, vol. 22, no. 4, pp. 415-437

Leithwood, K., Harris, A. and Hopkins, D. (2008) Seven strong claims about successful school leadership. School Leadership and Management, vol. 28, no.1, pp.27-42

Morrison, M. (2009) Leadership and Learning. Matters of Social Justice. Charlotte NC: IAP Publishing

Mortimore, P. (1991) The nature and findings of research ons school effectiveness in the primary sector. In S. Riddell and S. Brown (eds) School Effectiveness Research: Its Messages for School Improvement. Edinburgh: HMSO

Ouston, J (1999), “School effectiveness and school improvement: critique of a movement” in Bush, T, Bell, L, Bolam, R, Glatter, R, Ribbins, P Educational management: redefining theory, policy and practice London: Paul Chapman Publishing

ReferencesReynolds, D. and Farrell (1996) Worlds Apart? A review of educational

achievement involving England (OFSTED Reviews of Research) London: HMSO

Sammons, P., Hillman, J. and Mortimore, P. (1996) Key characteristics of effective schools: a review of school effectiveness research. London: Office for Standards in Education

Stoll, L. and Fink, D. (1996) Changing Our Schools Buckingham: Open University Press

Timmermans, A.C., Doolaard, S. and de Wolf, I. (2011) Conceptual and empirical differences among various value-added models for accountability. School Effectiveness and School Improvement, vol. 22, no. 4, pp. 393-413

van de Grift, W. (2009) Reliability and validity in measuring the value added of schools. School Effectiveness and School Improvement, vol. 20, no. 2, pp. 269-285

Watkins, C. (2010) Research Matters: Learning, Performance and Improvement. http://www.ioe.ac.uk/about/documents/Watkins_10_Lng_Perf_Imp_ev.pdf Accessed 16.2.2012