Download - School and Instruction Effects on Mathematics Achievement

This article was downloaded by: [University of Birmingham]On: 12 November 2014, At: 06:26Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

School Effectiveness and SchoolImprovement: An InternationalJournal of Research, Policy andPracticePublication details, including instructions for authorsand subscription information:http://www.tandfonline.com/loi/nses20

School and Instruction Effectson Mathematics AchievementRoel J. Bosker a , Ed J.J. Kremers b & Els Lugthart ca Department of Education , University of Twenteb Cito , National Institute for EducationalMeasurementc RION , Institute for Educational ResearchPublished online: 03 Aug 2006.

To cite this article: Roel J. Bosker , Ed J.J. Kremers & Els Lugthart (1990) School andInstruction Effects on Mathematics Achievement, School Effectiveness and SchoolImprovement: An International Journal of Research, Policy and Practice, 1:4, 233-248,DOI: 10.1080/0924345900010401

To link to this article: http://dx.doi.org/10.1080/0924345900010401

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages, andother liabilities whatsoever or howsoever caused arising directly or indirectlyin connection with, in relation to or arising out of the use of the Content.

http://www.tandfonline.com/loi/nses20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/0924345900010401

http://dx.doi.org/10.1080/0924345900010401

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14

http://www.tandfonline.com/page/terms-and-conditions

School Effectiveness and School Improvement 0924-3453/90/0104-0233S3.001990, Vol. 1, No. 4, pp. 233-248 © Swets & Zeitlinger

School and Instruction Effects on MathematicsAchievement

Roel J. BoskerDepartment of Education, University of Twente

Ed J.J. KremersCito, National Institute for Educational Measurement

Els LugthartRION, Institute for Educational ResearchUniversity of Groningen

ABSTRACT

Much research into school effectiveness fails to distinguish between pupil and class-room or teacher effects on the one hand, and 'real' school effects on the other. MoreoverDutch research into effective secondary schools is primarily concerned with pupilattainment whereas most Anglo-Saxon literature on this topic deals with pupil cogni-tive achievement. In this article an attempt is made to contribute to these topics byusing a large scale Dutch data set on pupil achievement in secondary education. Aftera short descriptive paragraph on pupil achievement in Dutch secondary education, amultilevel instructional and school effects model of pupil achievement is developedand tested. The results show that it is hard to distinguish instructional and teachereffects from school effects and that there are complicated cross-level interaction effectson achievement. For some pupils, instructional factors are more important than forothers; some instructional features only play a significant role in specifically organizedschools.

This article is based on a paper presented to the Third International Congress for SchoolEffectiveness in Jerusalem in January 1990. The research reported in this article wascarried out under the authority of the Ministry of Education and Science.

Correspondence: Dr R.J. Bosker, University of Twente, Department of Education, Divi-sion of Educational Administration, P.O. Box 217, 7500 AE Enschede, the Netherlands.E-mail address: TOBOSKER@HENUT5.

Manuscript submitted: August 14, 1990Accepted for publication: October 12,1990

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14

234 ROELJ.BOSKERETAL.

INTRODUCTION

In this article we report the findings of a research project into effective schoolscarried out in secondary education in the Netherlands.

This research was part of more extensive research that had been carried outwithin the framework of innovation plans for Dutch secondary education. Recentlythe Dutch government proposed a common core curriculum in the lower years ofsecondary education. At the moment this consists of separate types of schools,varying from junior vocational to secondary grammar (for more details see Fig-ure 1). The main objectives of the core curriculum are to increase the cognitivelevel of pupils in general and that of disadvantaged pupils in particular (Ministryof Education and Science, 1987; WRR, 1987).

This extensive research was essentially a baseline study. In a national sampleof 650 secondary schools, the Dutch National Institute for Educational Measurement(Cito) and the Institute for Educational Research of the University of Groningen(RION) assessed characteristics of school, curriculum and classroom organizationand the cognitive and social performance of pupils. The results of the baselinestudy will be compared with the performance of pupils after the introduction of.the core curriculum in the near future (Peschar, 1988). The main aim of assessinginstructional and organizational characteristics also is to gain more insight intothe possible educational effects of the core curriculum on the school and itsteachers. Failure or success of the innovation can then be explained by theseintervening variables, or success might be demonstrated under certain organiza-tional and instructional conditions (Lugthart et al., 1989; Peschar, 1988).

Inequality of educational opportunity in secondary education is a predominanttopic in Dutch educational sociological research. But most studies in this areadeal with educational and occupational attainment (cf. Bakker, Dronkers & Meijnen,1989), and do not pay attention to pupil cognitive achievement. Through thisbaseline study an explicit relation with pupil achievement can be established.

We begin this article with a short explanation of the design of the baselinestudy and of the main results. After that we will elaborate on our research intoeffective schools. In relation to this, we will discuss the following four questions:1. Do schools differ in the cognitive achievement of their pupils?2. Do schools differ in the relations between educational level of the parents, sex,

ethnicity and cognitive achievement?3. Can these differences be explained by characteristics of effective instruction

and effective schools, like 'opportunity to learn', 'direct instruction' and'press to achieve'?

4. How far are these relations contingent on factors like 'school type', 'schoolsize' and 'school organization'?

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14

SCHOOL AND INSTRUCTION EFFECTS ON MATHEMATICS ACHIEVEMENT 235

DESIGN OF BASELINE STUDY

InstrumentsTests for cognitive achievementTests for the subjects of Biology, English, Dutch (the mother tongue) and Math-ematics were taken. The tests referred to those subject areas that could be testedwith pen and paper and they were composed of both open and multiple choicequestions. The test duration was 100 minutes (two timetable hours). The test wasthe same for pupils of the various school types and they therefore containedquestions of varying degrees of difficulty. The tests contained sub-tests withquestions that fitted in with the present curriculum as well as questions thatbelonged to the proposed core curriculum. In this new core curriculum practicalapplication will have more emphasis.

Tests for non-cognitive skillsInformation was collected from pupils about their attitudes towards school andabout their social skills. In this article we do not refer to these skills (see Boskeret al., 1990 for this information).

Background information on pupilsThe following data were recorded: sex, age, length of stay in secondary education(i.e. repeating a class), level of education reached by parents, and ethnicity.

Questionnaire for school managementThis questionnaire for the principal deals with aspects of school organization,such as the transition period and schoolcareer of pupils, innovation activities,pupil counselling and certain aspects of school culture (e.g. rules on homework,acceptance of truancy, pupil assessment provision).

Questionnaire for teachersTeachers of classes that participated were requested to provide information abouttheir teaching method. They were also asked to comment on the content of thetests in relation to their own teaching.

Research populationThe research population included pupils who were taught in the three schooltypes for general secondary education within the Dutch educational system, thatis secondary grammar, senior secondary schools, and junior secondary. Furthermore,the target population consisted of the two largest school types within juniorvocational education: technical and domestic science schools. Within these fiveschool types we looked specifically at pupils in their third year. The plans forinnovation at the lower stage also presuppose a period of three years secondaryschooling. The research population included all pupils in the third grade of sec-ondary education: pupils that had never repeated a class as well as pupils that hadrepeated a class once or more often.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


Sample and designThe research population was composed of 20 subpopulations (4 subjects x 5school types). For each subpopulation a representative sample was taken, follow-ing a three-stage sampling procedure: schools were sampled at random, withineach school two classes were sampled where possible, and within each class allpupils were 'sampled in time'. In total more than 27,000 pupils from 1,308classes participated in the baseline study. The allocation of test-booklets to pupilswithin classes was completely at random.

In order to limit the workload for schools and pupils, all pupils participated inthe tests for one of the four subjects only. It was only for that subject that aschool average could be calculated.

Pre-tests and baseline studies were carried out simultaneously because therewas no time to do a separate pre-test of the test material for the subjects. Thus thenumber of items that could be pre-tested was in fact greater than needed. Also thenumber of items were too numerous to be answered by a pupil within the avail-able testing time. We decided therefore to choose a matrix sampling design inwhich the items of each school subject are divided over a number of test bookletsaccording to the principle of 'nominal equivalence'. This principle means thatwe tried to create comparable test booklets on the basis of content specifications(for example distribution of questions on subject area, type of question, estimateddifficulty). The allocation of booklets to pupils was done at random.1 For a de-tailed description of sample and design we refer to Kremers (1990). For thecomparison, the scores of the pupils on the separate booklets have been madeequivalent.2

SOME RESULTS OF THE BASELINE STUDY

Our results show a remarkable gap between the level of pupil achievement fromjunior vocational schools and general secondary schools:3 the first group achievesmuch less than the second group. Within junior vocational education, theachievement of technical pupils and domestic science pupils does not differ agreat deal. Within general secondary education we see a level of achievementrising from junior secondary to senior secondary and secondary grammar schools.

The differences in achievement between boys and girls are very small. ForBiology, English and Dutch boys and girls in all five types of schools achievedabout the same results. It is only for Mathematics that boys do better in all typesof schools than girls.

On average about a quarter of the pupils have repeated a grade once or moreoften at the end of the third year. The percentage of repeaters varies enormouslyaccording to school type. The range varies from 32% for junior secondary to 11%for secondary grammar education. For all school types boys repeat grades moreoften than girls. Pupils who repeated a grade achieved the same results as thosepupils who did not repeat a grade. This goes for each school subject and eachschool type and also for subpopulations (subject * school type).

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


It appears that the higher the educational level of the parents, the better theachievements of the pupils on the tests. This trend can be seen for each subject, ifwe take the results of the pupils from all school types together. This trend iscaused by the unequal participation of certain pupils in the higher valued generalsecondary school types. In the Dutch education system these pupils' parents tendto have a high level of education themselves.

Within school types we do not see this trend. There is no systematic relationbetween the level of education of the parents and achievements on the tests.

Ethnic minorities have lower test results than pupils of Dutch origin. Thistrend can be noted in each subject, again if we take the results of the pupils fromall school types together. The differences are the greatest for Dutch (mother-tongue) pupils. Within ethnic minorities those pupils of Turkish or Moroccanorigin have the lowest test results. It is rather striking that this trend can also beseen within the school types; so we can say that there is still a systematic relationbetween ethnicity and test results.

STATISTICAL CONSIDERATIONS

Before answering specific questions on school effectiveness we have to dwellsomewhat on the statistical aspects of this study. As outlined in the sample anddesign paragraph, pupils in this study were sampled using a three-stage samplingstrategy: first schools were sampled, then within each of these schools, if possi-ble, two classes in the third grade were sampled and finally all of the pupils inthose classes made up the final sample. It makes sense to conceptualize this laststage (all pupils within the sampled classes) as a sample as well. It can be seen asa sample in terms of time, since we could just as well have done our study a yearearlier in which case we would have had a different sample of pupils, althoughwe would have had the same schools and the same classes and teachers. As aconsequence of our sampling strategy the observations cannot be thought of asbeing statistically independent, since there is natural clustering in the data (cf.Aitkin et al., 1981). Pupils within one class are alike in that they have experiencedthe same instruction and the same school environment. This idea forms the basisfor our statistical model (cf. Aitkin & Longford, 1986; Goldstein, 1987; Raudenbush& Bryk, 1986). This model can be written as (underlined symbols are used toindicate random variables)

withyijk : the test score for pupil i in class j of school kpo-k : the class specific interceptPjjk : the class specific coefficient for the regression of y

on xx j k : a pupil characteristic like sexe ^ : the residual term on the pupil level with variance o2

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


Furthermore

+ Xiok^k+<2> Bosk = Boot ^ o i

withPOOk : the school specific intercepty1Ok : the school specific coefficient for the regression of

Pook ( a n^ thus y) on zz k : a class variable like press to achieveuo.k : the residual term on the class level with variance a^0

And of course

(3) fiook = Pooo + 5iowk + %)k

withPQOQ : the grand mean810 : the coefficient for the regression of P , ^ (and thus

y) on wwk : a school variable\mk : the residual term on the school level with variance o^0

Analogous to (2) and (3) Pjjk and P1Ok can be predicted with class and schoollevel variables as can y1Ok be predicted with school level variables. In the resultsto be presented we only needed

with uljk having variance a ^ and v1Ok having variance a2vV

The first important aspect of this set of equations that are estimated simultaneouslyis the distinction between pupil level, class level and school level variation inachievement (o^, o^0, oj0 respectively). The second point worth mentioning isthe random regression: in this model the within school relation betweenachievement for instance and sex may vary between schools. In some schools theachievement gap between boys and girls may be larger than in other schools.

DIFFERENCES BETWEEN SCHOOLS: GENERAL ANALYSIS

In this paragraph we will answer the following two questions:1. Do schools differ in the cognitive achievement of their pupils?

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


2. Do schools differ in the relation between educational level of the parents, sex,ethnicity and cognitive achievement?

In this article we have restricted ourselves to the achievements for mathematicsin junior secondary schools. We selected the subject mathematics because theachievements in this particular subject are of great importance for the rest of apupil's schooling. Junior secondary schools were chosen because this schooltype has the largest numbers of pupils in the Dutch education system.

For this analysis we selected 19 schools with 2 classes and 6 schools with onlyone class. Within these classes 707 pupils were selected.4 The dependent vari-able is the score on the mathematics achievement test. Although there were 10different tests, they could be treated after standardization as (nominal) equivalenttests.5 In order to find an answer to our first question we used a three-levelstatistical model, as implemented in the VARCL-programme (Longford, 1986).

Table 1. Variance components of mathematics achievement (standard errors, when appli-cable, between brackets).

level variance

pupil a2e : .801

class G*0 : .055

school a* : .105 (.053)

The results from Table 1 show that there is considerable variation between classesand schools with regard to pupil achievement. The intra-class correlation at theclass level is .17, whereas this correlation is .11 at the school level. In otherwords: 11 per cent of the variance in mathematics achievement is accounted forby the schools that the pupils attend. The standard error for the classroom variancecomponent suggests that the class and pupil level almost coincide since it doesnot seem to make much sense to distinguish a separate class level as there is nosignificant unique class level variation in achievement. We will return to thisobservation later.

In Table 2 we give the posterior effects (cf. Aitkin & Longford, 1986) for thetop 20 per cent and bottom 20 per cent of schools. The posterior effect can bethought of as the achievement level of the average pupil attending a given schoolwith a conservative correction depending, among other things, on the samplesize of pupils within that school: the smaller the sample the greater the correc-tion.

From the figures presented below we see that the average pupil attendingschool 1 performs almost two standard deviations better than the average pupilattending school 25, whereas the difference between the schools at P(80) andP(20) is still more than half of a standard deviation.6

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


Table 2. School effects on mathematics achievement.

school posteriorrankno. effect

P(80)

P(20)

12345

2122232425

.92

.81

.62

.59

.51

-.14-.19-.43-.44-.96

Table 3. Pupil effects on mathematics achievement (standard errors for the regressioncoefficients and the variance components are within brackets).

variable

ethnicity

socio-economicstatussex

fixedeffect

Pioo:

P200'

Pscxr

-.148(.174).003(.019)

-.359(.072)

randomeffects

o2vl: .00002

(.0003)a2 : .050

(.158)a2

v3: .008(.026)

Our second question concerns (in)equality of educational opportunity. In Ta-ble 3 we summarize our findings.

The results in the first column show that there is no significant relation betweenmathematics achievement and socio-economic status or ethnicity, although theeffect of ethnicity seems relevant enough (only a very small fraction -4 per cent-of the pupils in the selected sample belongs to an ethnic minority group, thereforewe find this large standard error). Boys perform approximately one third of astandard deviation better in mathematics than girls. The last column of Table 3shows that there are no (significant) school - pupil interactions. In other wordsthe relationship between gender and mathematics achievement within schools doesnot differ between schools.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


SCHOOL AND CLASS EFFECTS ON ACHIEVEMENT

Now having assessed the size of the differences between schools and classes intheir mathematics achievement, we tried to predict these differences by using amultilevel school effects model as outlined in Figure 1.

The essence of this multilevel model is the incorporation of the effects ofthree different levels and of cross-level interaction effects on achievement. Inshort, achievement can be thought of as being a function of pupil's sex, classroomorganization and school organization. For instance boys achieve more than girls;the more opportunity to learn the higher the achievement; the larger the schoolthe lower the achievement. Furthermore the sex-effect on achievement is contingenton classroom and school organization. The more opportunity to learn, for instance,the wider the gap between the achievement of boys and girls. Following the ideasof organizational contingency theory (see: Mintzberg, 1979; Morgan, 1986) theeffect of classroom organization might be contingent on school organization, e.g.classroom climate might only matter for large schools.

Five classroom and three school level variables are included in the model (seeTable 4).

level 1

level 2

level 3

pupilcharacteristics

classroomorganization

schoolorganization

mathematicsachievement

Fig. 1. A multilevel educational effects model.

Table 4. Class and school variables.

class school

opportunity to learn (OTL)press to achieve

(PTA)class climate

(CLCL)use of evaluative tests (UET)effective instruction (EFIN)

size (SIZE)comprehensive or

categorical (COM)standardizationof rules (STAN)

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


OTL is the rating by the teacher of the items in the achievement test that arecovered by the curriculum. Classroom climate is measured by a scale developedby Stoel as teacher's satisfaction with his/her working conditions in school. Thisscale is internally consistent (a: .80; more details in Bosker, van der Velden &Hofman, 1985). Press to achieve is assessed by a test developed by Meijnen (a : .73,see also: Bosker, Van der Velden & Hofman, 1985). Effective instruction is theratio of the time used for instruction to the total time available for classroominstruction. Evaluative feedback is measured as the number of different feedbackmethods and the frequency of their use. School size is measured by the number ofclasses within the school. Furthermore schools can be categorical or combinedwith other school types (e.g. junior vocational education) into comprehensives.The last variable at school level is adopted from a study by Bosker et al. (seealso: Bosker,Van der Velden & Hofman, 1985) and measures the standardizationof school organization, i.e. the degree to which teachers have to behave accord-ing to prescribed rules. The homogeneity-coefficient for this Mokken-scale is.36. Some details of the variables to be included in the model are given in Table5.

Table 5. Summary statistics of the variables.

Variable Mean S.D.

ETNISEXSES

OTLCLCLPTAUETEFIN

SIZECOMSTAN

4% are from ethnic minorities48% girls

3.13

.713.272.682.63

.85

15.0016% comprehensive schools

2.44

1.93

.11

.45

.37

.55

.17

11.97

1.90

In Table 6 the results of the multilevel analyses are summarized.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


Table 6. Regression of mathematics achievement on pupil, class and school level vari-ables (N(pupils)=707, N(classes)=44, N(schools)=25); standard errors withinparentheses.

REGRESSION-COEF.GRAND MEAN ( p ^ )

OTL (Yl0)

SEX

CLCL

PTA

EFIN

CLCL * SEX

EFIN * SEX

STAN

STAN * EFIN

VARIANCE-COMPONENTS

pupil level

classroom level

school level

increase in var.accounted for

(P300)

(Y30)

(Y20)

(Yso)

(Y33)

(Y53)

(830)

(M

(°D

(°v2o)

MODELSTATISTICSDeviance-deer.

d.f.-decrease

p-value

model 1

-.851.22(.64)

.800

.047

.098

1.5%.

4

1

<.O5

model 2

-.651.20(.61)

-.36(.07)

.773

.034

.110

2.9%

26

1

<.01

model 3

-3.03.98

(.61)

-.35(.07)

.39(.16)

1.75(.75)

.773

.072

.020

5.5%

6

2

<05

model 4

-3.03.99

(.61)

-.49(.83)

.29(.19).39(.17).65(.90)

-.44(.18)

1 Gfil.oo

(-89)

.765

.066

.022

1.1%

8

3

<.05

model 5

-4.981.11(.54)

-.13(.83)

.18(.18).24(.16)

3.71(1.21)

-.51(.18)

1.73(.89)

4.14(1.09)-4.82(1.28)

.766

.023

.035

3.2%

13

2

<.O1

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


In column three the results for the three-level main effects model are given.Opportunity to learn is maintained in the model although this effect is no longersignificant. By so doing the other effects make somewhat more sense since theycan be interpreted as school and instruction effects on mathematics achievement,after OTL and SEX are taken into account. Press to achieve and effective instruc-tion account for approximately a quarter unique school level variance. There areno significant effects of school level variables, however. In the last column theresults of the model with main effects and cross-level interaction effects aregiven. Since the interaction effects are difficult to interpret, we present twotables with more details on these effects.

Table 7. Interaction of pupil and classroom effects (expressed as the CLCL and EFINregression coefficients for boys and girls).

boys girls

Classroom climate .18 -.33Effective instruction 3.71 5.44

From the figures presented in Table 7 we can conclude that a positive classroomclimate has a positive effect for boys only, whereas there is a negative effect ofthis variable for girls. Effective instruction on the other hand has a positive effectfor boys as well as for girls, but this effect is somewhat more pronounced forgirls. In other words: girls especially benefit from effective instruction; onlyboys benefit from a positive classroom climate.

Table 8. Interaction of school and classroom effects (expressed as the EFIN regressioncoefficients for schools that score below vs. above the average on STAN).

below above

Effective instruction 3.71 -1.11

In Table 8 we see that the effect of effective instruction is contingent on thedegree to which the school's organization is standardized: effective instruction isimportant only for those schools that have a below average formalization ofschool rules.Our final educational effects model is depicted in Figure 2.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


level 1 sex j j * mathematicsachievement

level 2 e n n ^"^ < ^^* otl

level 3 stand

Fig. 2. School, instruction and pupil effects: results.

CONCLUSIONS

Summarizing the results of the effects of stratification on mathematics achieve-ment we note the important finding that socio-economic status and ethnicityhave no effect, whereas the sex effect is confirmed for junior secondary schools:boys perform better in mathematics than girls. Furthermore mathematicsachievement is dependent on effective instruction and teacher's press to achieve,a finding consistent with earlier school effects research (see Good & Brophy,1986). We may note, however, that these are instruction instead of school variables!There are no main school effects on mathematics achievement in junior second-ary schools, but this may be an effect of model underspecification, since theproportion of variance between schools is greater than the unique variance pro-portion between classrooms. But, as we stated earlier, the distinction of a separateclassroom level seems to be artificial. Firstly pupils of different classes within agiven school reach almost the same achievement level.7 Secondly, as we just noted,teacher variables account for school level variation in achievement. Put differ-ently: what is common to the instruction for maths teaching within a schoolseems to be more important for mathematics achievement than the differencesbetween the maths teachers. How can this common instructional behaviour beachieved? Three explanations are plausible. Schools may select their personnelaccording to a clear professional standard. Or schools succeed in coordinatingtheir personnel in such a way that they all instruct their pupils in the same way.This coordination may take place in different ways: direct supervision by theschool leader, clearcut achievement standards, a well-developed school mission,or informal communication between teachers. These hypotheses call for the studyof within and between school variation in teacher behaviour, and of the factorsthat cause common behaviour.

The most important finding of this study is the existence of cross-level inter-action effects: sex effects are contingent on instruction, and instruction effectsare contingent on school organization. The practical implications of the interac-tion effect between classroom climate and sex are hard to guess: Should we worktowards conditions that lower the teachers' satisfaction with his/her work in

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


order to improve the performance of girls? The implications of the other interac-tion effects are easier to follow: all pupils benefit from increasing the efficiencyof instruction, especially girls. In general one could conclude that teachers inschools with less clear cut rules concerning standards and pupil behaviour shouldbe concerned about the net time available for instruction. The main point, how-ever, is that these results clearly point to the need for a more complexconceptualization of school effects than is usually done (Barr & Dreeben, 1983;Bidwell & Kasarda, 1980).

NOTES

1. It is on the basis of the pre-test data that the 'definitive tests' have been constructedusing item response theory. Through a renewed analysis of the pre-test data, on thebasis of the definitive tests, the baseline was then mapped out. In future comparisons(after the introduction of the core curriculum) these tests will be used again. In thisarticle, however, we will work with the raw test data to overcome the problems inherentin the matrix sampling design. Each pupil has responded to only a few items in thedefinitive test. Working with the raw scores, however, leads to an inflation of pupillevel variance, but it is conservative in statistical terms, since we are primarily con-cerned with class and school level variance.

2. The scores have been equated by transforming them to z-scores per booklet. By doingso, the inevitable inter-booklet variance has been reduced to zero.

3. In this paragraph where we will present the main results of the baseline study, our basewas formed by the results on the definitive tests. The scores in this case were madeequivalent by transforming all pupil scores to mean p-values. In the sub study intoeffective schools we go back to the results on the (pre-test) booklets.

4. The selection procedure was based on the following criteria: a) pupils had to have nomissing values on the variables sex, ethnicity and socio-economic status; b) it must bepossible to assign the teacher-scores to one class only. This last criterion is crucialsince teachers were asked to respond to the items of the questionnaire for one particularclass. In case they taught both classes, they were asked to differentiate their answersbetween those two classes. We decided to reject those teachers from the analysis whofailed to do so, since if we had not done this it would have drastically deflated betweenclass variation in instruction variables. Once again this procedure is conservative,because now instruction variables can reduce within school between class variation inachievement, whereas otherwise they could only have reduced between school variation.

5. This was achieved by transforming them to z-scores per booklet (see also note 3).6. The shrinkage factor is n.o*^(/(o2 + n.o^o). In this case where n. is on the average 28, o2

is approximately .87 (including class level variance!) and a20 i s . 11, the shrinkage is .8.

The real differences between the schools are even somewhat greater.7. We must be cautious, however, since little is known about the power of tests of variance

components (Raudenbush, 1988).

REFERENCES

Aitkin, M., Anderson, D. & Hinde, J. (1981). Statistical Modelling of Data on TeachingStyles. The Journal of the Royal Statistical Society, Series A (General) 144, 419-461.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14


Aitkin, M. & Longford, N. (1986). Statistical Modelling Issues in School EffectivenessStudies. The Journal of the Royal Statistical Society, Series A (General) 149, Part1, 1-43.

Bakker, B.F.M., Dronkers, J. & Meijnen, G.W. (Eds.) (1989). Educational opportunitiesin the welfare state: longitudinal studies in educational and occupational attain-ment in the Netherlands. Nijmegen: ITS/OoMO.

Ban, R. & Dreeben, R. (1983). How schools work. Chicago: The University of ChicagoPress.

Barr, R. & Dreeben, R. (1977). Instruction in Classrooms. Review of Research in Educa-tion, 5, 89-162.

Bidwell, C.E. & Kasarda, J.D. (1980). Conceptualizing and measuring the effects ofschool and schooling. American Journal of Education, 88, 4, 401-430.

Bosker, R.J., Velden, van der, R. & Hofman, A. (1985). Een generatie geselecteerd. Deel3: technisch rapport scholen. Groningen: RION.

Bosker, R.J., Haanstra, F., Lugthart, E. & Roeders, P.J.B. (1990). School- eninstructiekenmerken en non-cognitieve vaardigheden in het voortgezet onderwijs.Groningen: RION.

Goldstein, H. (1987). Multilevel models in educational and social research. London: CharlesGriffin & Co.

Good, J.L. & Brophy, J.E. (1986). School effects. In: M.C. Wittrock (Ed.): Handbook ofresearch on teaching. New York: McMillan Inc.. pp. 570-602.

Kremers, E.J.J. (Ed.) (1990). Overzicht van leerresultaten aan het einde van de eerstefasevoortgezet onderwijs. Arnhem: CITO.

Longford, N.T. (1986). Variance component analysis: manual. University of Lancaster.Lugthart, E., Roeders, P.J.B., Bosker, R.J. & Bos, K.T. (1989). Effectieve schoolkenmerken

in het voortgezet onderwijs: een literatuuroverzicht. Groningen: RION.Ministry of Education and Science (Ministerie van Onderwijs en Wetenschappen) (1987).

Wetsvoorstel betreffende basisvorming in het voortgezet onderwijs. ' s Gravenhage:Staatsuitgeverij.

Mintzberg, H. (1979). The structuring of organizations. Englewood Cliffs: Prentice Hall,Inc.

Morgan, G. (1986). Images of organization. London: Sage.Oakes, J. (1985). Keeping Track. How Schools Structure Inequality. New Haven and

London: Yale University Press.Peschar, J.L. (1988). Evaluatie van de basisvorming. Kader voor het uitvoeringsplan. 's

Gravenhage: SVO.Raudenbush, S.W. (1988). Educational applications of hierarchical linear models: a review.

Journal of Educational Statistics, 13, 85-116.Raudenbush, S.W. & Bryk, A.S. (1986). A hierarchical model for studying school effects.

Sociology of Education, 59, 1-17.W.R.R. (Netherlands Scientific Council for Government Policy) (1987). Basic Education.

's Gravenhage: Staatsuitgeverij.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14

248 ROEL J. BOSKER ET AL.

APPENDIX(reprinted from W.R.R., 1987, 7)

Structure of full-time education in the NetherlandstWMdt IliMMl/MCOKd i M i

tagmtnp/towwr stags

hogaratrap/

Dow

nloa

ded

by [

Uni

vers

ity o

f B

irm

ingh

am]

at 0

6:26

12

Nov

embe

r 20

14

Download - School and Instruction Effects on Mathematics Achievement

Top Related