PREFACE

We were interested in writing comprehensive, object.ive, and quantitative reviews of educational research findings before the term meta-analysis was invented. Our publications on Keller’s Personalized System of Instruction (PSI) document our progress toward this goal during the last 15 years. In 1974 we published a review of 15 evaluations of the effectiveness of the Keller plan in science teaching (J. Kulik, Kulik, & Carmichael, 1974). We laid out findings from the studies in tables, but the review was still essentially a conventional narrative review. In 1976, at a national conference on personalized instruction, we presented a fuller, more sophisticated, and more quantitative review of findings on the Keller plan (J. Kulik, 1976). This review covered 31 articles and included box-score comparisons of findings from Keller and conventional teaching. We reported that in 97% of the studies the Keller Plan had positive effects on student examination performance, in 75% of the studies it had positive effects on student attitudes, and so on. Finally, in a review published later in 1976, we expanded our pool of studies to 39, and we expressed the difference in examination scores between Keller and conventional classes in standard deviation terms (J. Kulik, Kulik, & Smith, 1976). We reported that the average examination score in the Keller classes was 0.67 standard deviations higher than the average score in the conventional class. This was the first review of ours in which we reported an effect size!

Although our reviews were growing in sophistication, they were still far from being true meta-analyses. Then Gene Glass spoke at the 1976 meetings of the American Educational Research Association about his conception of what quantitative reviews could be and changed forever our thinking about the way reviews should be done. When we came back to the subject of the effectiveness of the Personalized System of Instruction in 1979, our approach was for the first time truly meta-analytic (J. Kulik, Kulik, & Cohen, 1979). Our most recent work on the effectiveness of the Keller plan shows how our ideas of meta-analysis have evolved even further (J. Kulik, Kulik, & Bangert-Drowns, 1988).

This volume is meant to convey much of what we think is important to know about meta-analysis in education. The first part of the book covers meta- analytic methodology and the second part goes into what we have learned from actual meta-analyses of educational findings, both those carried out by ourselves and those carried out by other meta-analysts throughout the world.

Chapter 1 covers definitional matters: the 1976 address by Glass that led to the flurry of meta-analytic activity of the last twelve years, the features that distinguish meta-analyses from other reviews, concrete examples of meta- analyses, and the major criticisms and contributions of the methodology. Chapter 2 describes some historical antecedents of meta-analysis that still exert an influence on meta-analytic practice today. Chapter 3 presents current

223

224 JAMES A. KULIK and CHEN-LIN C. KULIK

approaches to meta-analysis, along with our evaluation of the methods. And in Chapter 4, we present our guidelines for meta-analyses. We hope that the guidelines will be useful both to those planning to do meta-analyses and to those who read meta-analytic reviews in the literature.

In the second part. of this issue, we turn to the substantive findings from meta-analytic research. In Chapter 5 we examine design and publication features that are frequently thought to influence results of educational studies, and we try to determine how important these factors really are. In Chapters 6 through 11 we summarize the results of meta-analyses that have been carried out in six major areas of educational research. Finally, in Chapter 12, we describe briefly several major lessons that we have learned from meta-analytic research.