Educational Evaluation: Classic Works of Ralph W. Tyler

Educational Evaluation: Classic Works of Ralph W. Tyler

Compiled and Edited by George F. Madaus and Daniel L. Stufflebeam

~.

" Kluwer Academic Publishers Boston Dordrecht London

Distributors

for the United States and Canada: Kluwer Academic Publishers, 101 Philip Drive, Assinippi Park, Norwell, MA 02061.

for the UK and Ireland: Kluwer Academic Publishers, Falcon House, Queen Square, Lancaster LAI IRN, UK.

for all other countries: Kluwer Academic Publishers Group, Distribution Centre, P. O. Box 322, 3300 AH Dordrecht, The Netherlands

Library of Congress Cataloging in Publication Data

Tyler, Ralph Winfred, 1902-Educational evaluation: classical works of Ralph W. Tyler/

edited by George F. Madaus and Daniel Stufflebeam. p. cm. - (Evaluation in education and human services)

Bibliography: p. Includes index.

ISBN-13: 978-94-010-7708-8 DOl: 10.1007/978-94-009-2679-0

e-ISBN-13: 978-94-009-2679-0

1. Educational tests and measurements-United States­Collected works. 2. Students-United States-Rating of­Collected works. I. Madaus, George F. II. Stufflebeam, Daniel L. III. Title. IV. Series. LB3051. T93 1988 371.2'6-dc19 88-833

© 1989 by Kluwer Academic Publishers, Boston

Softcover reprint of the hardcover 1 st edition 1989

CIP

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without the prior written permission of the publishers. Kluwer Academic Publishers, 101 Philip Drive, Assinippi Park, Norwell, MA 02061

Contents

Foreword

Preface

Part I The Service Study Years: 1929 to 1938 1 Overview Key Points

vii

xi

1 3

2 7 Service Studies in Higher Education Introduction The Construction of Examinations in Botany and Zoology 3 17 Constructing Achievement Tests Measuring the Results of College Instruction A Generalized Technique for Constructing Achievement Tests Formulating Objectives for Tests Ability To Use Scientific Method Explanation Test Measuring the Ability To Infer A Test of Skill in Using the Microscope The Master-List as a Device Improving Test Materials in the Social Studies Assumptions Involved in Achievement-Test Construction What Is Statistical Significance? Making a Co-operative Test Service Effective Techniques for Evaluating Behavior Notes

Part II Appraising and Recording Student Progress: The Eight-Year 87 Study 1 M Overview

v

vi

Key Points 2 Appraising and Recording Student Progress Preface Commission on the Relation of School and College Purposes and Procedures of the Evaluation Staff Aspects of Thinking Evaluation of Social Sensitivity Aspects of Appreciation Interpretation and Uses of Evaluation Data Philosophy and Objectives Teachers' Reports and Reports to the Home Notes

CONTENTS

93

Part III Tyler's Rationale for Curriculum Development 1

197 199

Overview Key Points 2 201 New Dimensions in Curriculum Development

Part IV National Testing Programs 209

Part V

Part VI

Index

1 211 Overview Key Points 2 215 Appraisal of Edu'cational Achievement Gained in the Armed Forces 3 223 The Objectives and Plans for a National Assessment of Educational Progress 4 227 National Assessment - Some Valuable By-Products for Schools Note

Tyler's Recent Reflections on His Work 239 1 241 Overview Key Points 2 243 An Interview with Ralph Tyler The Preface Interview 3 273 Appendix: Vitae of Ralph Winfred Tyler

A Chronological Bibliography 275

307

Foreword

I personally learned to know Ralph Tyler rather late in his career when, in the 1960s, I spent a year as a Fellow at the Center for Advanced Study in the Behavioral Sciences at Stanford. His term of office as Director of the Center was then approaching its end. This would seem to disqualify me thoroughly from preparing a Foreword to this "Classic Works." Many of his colleagues and, not least, of his students at his dear Alma Mater, the University of Chicago, are certainly better prepared than I to put his role in American education in proper perspective. The reason for inviting me is, I assume, to bring out the influence that Tyler has had on the international educational scene. I am writing this Foreword on a personal note. Ralph Tyler's accomplishments in his roles as a scholar, policy maker, educational leader, and statesman have been amply put on record in this book, not least in the editors' Preface. My reflections are those of an observer from abroad but who, over the last 25 years, has been close enough to overcome the aloofness of the foreigner.

Tyler has over many years been criss-crossing the North American con­tinent generously giving advice to agencies at the federal, state, and local levels, lecturing, and serving on many committees and task forces that have been instrumental in shaping American education. Several years ago, Tyler'S famous student, Benjamin Bloom, told me that he and a group of colleagues had ventured a guess of how much of his life Tyler had spent in airplanes. They came up with an estimate of two years! One Saturday morn­ing in the mid-1960s when I drove up to the Stanford Center I ran across Ralph whom I had not seen for the whole week. I knew that he - as usual - had been busy with many things at many places, but in particular with the preparatory work for the National Assessment of Educational Pro­gress. I asked him: "Ralph, how often do you cross the American conti­nent?" With the typical sense of humor pouring through his peering eyes

vii

viii FOREWORD

he replied: "I am trying to keep it down to one tour per week." Ralph Tyler's fellow countrymen are perhaps less familiar with his inter­

national than his national impact. As a graduate student in the early 1940s (during the war when scholarly communications were scarce) I obtained a copy of a report from the Eight-Year Study that, along with his subsequent book on the basic principles of curriculum and instruction, soon began to be regarded as a classic. These and other publications from the 1940s pro­vided a solid empirical foundation for those of us who conducted studies in educational psychology.

During the early stages of his career, Ralph· Tyler was instrumental in developing relevant examinations in specific subject matter fields and he laid the ground for better methods of achievement testing. Psychometricians had, around 1930, developed techniques of measuring certain psychological traits, but Tyler adapted these techniques to suit achievement testing. He widened the horizon with his famous rationales for both evaluation and cur­riculum construction. It was certainly part of the logic of the events that he was the one who, in due time, provided the rationale for a national assess­ment of educational progress.

What strikes me, both in reading this book and in reflecting on my close contacts with American educators, is how seminal Tyler has been with his ideas, not least through his own students. Tyler's bibliography at the end of this book carries a large number of articles in a variety of professional journals (educational and others) in which he acts as a builder of bridges between theory and practice. No wonder, then, that he has been a much sought after consultant and has been asked to nominate superintendents, deans, and other leading people in the educational enterprise.

To me - and, I think to many of my colleagues around the world -Ralph Tyler stands out as the leading figure in American educational research and evaluation - indeed as Mr. Education himself. During the "classical period" epitomized in this book, he advanced his evaluation theory and its applications as well as the principles of curriculum and instruction. It was therefore no surprise to me during my first year at his Center to find that he was the central figure and the idea injector in the enterprise later known as the National Assessment of Educational Progress. Apart from the intellectual leadership required to establish this new system of national evaluation, he had to develop a skillful diplomacy in order to overcome the deep suspicion that many people held against centrally organized assessment. It smelled of federal intrusion.

Neither was it a surprise to me to learn that Tyler emerged as the leading figure in the establishment of a U.S. National Academy of Education. He served as its president for quite some time and his contribution gave it

FOREWORD ix

a high status both within and outside the educational community. As a Foreign Associate of the Academy, I had ample opportunity to observe how, in conducting business during our meetings, he could draw upon an inexhaustible body of experience and could sprinkle the sessions with his witty remarks and anecdotes.

The International Association for the Evaluation of Educational Achievement (lEA), on Ben Bloom's initiative, decided in 1971 to organize an international seminar in Sweden on curriculum development and evalua­tion. The seminar went on for six weeks with some 125 participants from more than 23 nations. Ralph Tyler was the self-evident choice as the key lec­turer. We succeeded in holding him for the whole 6-week period in the little town of Granna in Sweden. This was, indeed, quite an achievement. I wondered whether Ralph had stayed for such a long time at the same place at any time in his adult life!

When a person comes to our mind, we tend to imagine him or her in concrete situations. In my case, what particularly stands out in recalling encounters with Tyler is a walk with him late one evening from the Inter­national House to the Quadrangle Club (where we were both staying) at the University of Chicago - that great institution where Tyler had spent so many productive years and had been instrumental in bringing its Education Department to the forefront, not only in the United States but in the whole world. I heard him often use the expression "cutting edge." He wanted to be part of it. As we passed Judd Hall that evening, Ralph gave me a detailed and vivid account of how education was established at the University during the Dewey era around the turn of the century. This was a fascinating piece of history of education.

Hardly have we on many occasions seized on any topic of conversation before Ralph's immense knowledge of individuals and his interest in their biographies flourished. He is a living Who's Who in American education, and has an encyclopedic grasp of the educational enterprise. Many of us have experienced how he, without even a piece of paper with its main points in his hands, could give a well-structured lecture on almost any topic in education. The Annual Meeting Programs of AERA still carry Tyler's name in several sessions and symposia. He is, indeed, a great generalist in a field where short-sighted specialization, as in other scholarly areas, is gradually creeping in.

We are all, to varying degrees, prisoners of prevalent "paradigms," a term that (it is well known) was coined at the University of Chicago in the early 1960s. Ralph Tyler's contributions to modern thinking in education as epitomized in this volume have, in a remarkable way, survived the change of time and paradigms. He has been able to cut through generations of

x FOREWORD

educational scholars, which is no surprise to those of us who have seen the rapport that he is able to establish with young people. This timelessness is, in my view, the main reason why it has been considered fitting to collate his lasting contributions to educational scholarship. They have, indeed, withstood the test of time.

The emphasis in the present collection is naturally on publications from Tyler's early and mid-career - periods when he apparantly had more time to devote to the writing of major, monographic pieces. He had not yet been submerged in the many activities that took him around his home country and the world at large. His creativity and accomplishments as an innovator did by no means diminish after he left Chicago for the Stanford Center in the early 1950s. The Bibliography attached to this book bears witness to this.

American public education has been intermittently under fire and sub­jected to severe criticism during the post-war years. Those of us on the other side of the Atlantic who in various capacities have been involved in reform­ing the school systems in our home countries have, since the 1940s, been looking at the American public school as a model because of its role as a pillar of American democracy. Ralph Tyler has, during his entire career, always held a strong commitment to free public education that takes care of all children from all walks of life and all ethnic backgrounds - a school providing the same conditions and having them under a common roof. Tyler has seen his task as one of giving service to that school.

Dr. Tyler now lives in Milpitas, California. When I last visited California, somebody gave me a clipping from the San Jose Mercury News of an article published on his 85th birthday. He was presented as a person whose "main problem" since he was young had been "excess energy," something that was behind the opposition that brought him into trouble in high school. He is quoted as saying, after consulting his doctor, that as long as he is active he has no reason to worry about health. He has kept so much of his youthfulness and a combination of seriousness and levity of mind that has been behind his inspiring achievements.

Dr. Torsten Husen, President International Academy of Education

Preface

For nearly 60 years, Ralph W. Tyler has been one of the most influential figures in education, at all levels both nationally and internationally. He has made substantial contributions to the fields of curriculum, testing, evalua­tion, and educational policy. Directly through his work, and indirectly through his many famous students, he has deeply influenced many note­worthy developments in education, including objective-referenced testing, objectives-based program evaluation, mastery learning, achievement test construction, item banking, the taxonomic classification of educational out­comes, and cooperative test development. He has been instrumental in the development of several national testing programs including the General Educational Development Program, the Cooperative Testing Program, and the National Assessment of Educational Progress.

During the past half-century, Tyler's role in education may easily be labeled that of educational statesman. He has been an advisor to pre­sidents, legislators, and national and international commissions. He has helped local schools, school districts, colleges and universities, and state departments of education in the United States and in numerous overseas countries. His work and influence have always been grounded in practical experience, oriented to service, and dedicated to improvement and progress in education. His work reflects his brilliance and vision. His life's work has been governed by a keen sense of values, equity, and fairness. Throughout his career his work shows his humility and sense of humor - the latter often at his own expense.

Tyler's writings chronicle his ideas and projects over the years. His writings are filled with rich and instructive examples that still illuminate many contemporary theories, practices, and policies in education. Students of education can greatly increase their understanding of education's history and progress since the 1930s through a review of Tyler's work. Some of his

xi

xii PREFACE

writings address issues in important debates that have occurred during the past 60 years in education. For example, in the early thirties he directly challenged the ideas of the most prominent figure in testing, Ben Wood. Wood argued that it was sufficient to limit testing to measures of recall and recognition of facts and knowledge. If students had the necessary knowledge, they could apply it to solve problems. Tyler argued and then demonstrated that measures of higher order objectives such as application did not correlate highly with measures of mastery of factual knowledge. He concluded that if higher order cognitive skills were the objectives of instruc­tion they had to be measured directly. His ideas about this - excerpted here in Part I - have influenced achievement test construction to the present day.

Another example of Tyler's past work that still influences the fields of curriculum development and program evaluation is his work in the Eight­Year Study. The Eight-Year Study, as its name suggests, was a longitudinal assessment of students from thirty progressive and traditional high schools through four years of secondary school and four years of college. The very existence of private and public progressive schools was threatened in the mid-thirties when many prestigious colleges changed their entrance require­ments. They began to require courses in specified subjects like American History and Physics. Since the progressive schools were not organized around courses but around learner outcomes, this new requirement posed a real difficulty for them. Under a grant from the Carnegie Corporation, the comparative Eight-Year Study was undertaken. Tyler's work on the study is still the best available description of how evaluators can work coopera­tively with teachers to clarify instructional objectives and develop indicators of students' continuous progress toward the mastery of a whole range of learning outcomes. His description of what was done - excerpted at length in Part II - is a model of sound practical advice for those interested in improving teaching and learning at the classroom level.

Still another example of Tyler's foresight is his persuasive argument against the practice employed in developing many norm-referenced achieve­ment tests of discarding test questions that fail to discriminate among examinees. Tyler recognized that items that were answered by most students as well as those missed by most could, if they were good items to begin with, provide valuable information about instruction and student progress. Infor­mation of this kind was more valuable than including items for the sole pur­pose of differentiating among examinees in order to rank them. Items selected to meet an individual differences model, using technology originally developed in intelligence testing, typically did not provide the necessary information needed to evaluate the full range of educational objectives.

PREFACE xiii

Today, the objectives-referenced testing movement owes a debt to Tyler for his observations about item selection criteria.

Unfortunately, however, the vast majority of Tyler's writings are out of print, and not easily accessible in libraries. The difficulty of getting at this important source of information is the main reason we are bringing out this collection of the classic contributions of Ralph Tyler. But there is another important reason for this book. Tyler's ideas still influence many people in testing, evaluation, and curriculum development. However, all too fre­quently his ideas have been misinterpreted or misapplied. Unwittingly or not, some interpreters have translated and applied his ideas with a simplicity and expediency he never intended.

For example, some educationists have imposed a rigid and mechanistic construction on his recommendations for defining course objectives. They have often assessed course outcomes too narrowly and have not sought to define or assess the full range of educational objectives. They have directed teachers to write hundreds of behavioral objectives without the benefit of discussion with the constituents of educators about the purposes of schooling. They have gravitated to the paper-and-pencil, multiple-choice standardized test and have not realized the essential need for more direct measures or indicators of student progress. They have lost sight of the cen­trality of the classroom teacher in defining, teaching, and assessing student outcomes. They have advocated evaluating student performance only at the end of courses, avoiding implementing programs that continuously monitor student progress. Unfortunately, Tyler has often been blamed for these kinds of misapplications of his ideas.

Still another reason for this book, perhaps less important than those described above but nevertheless we feel important, is that many recent innovations in curriculum, testing, and evaluation have not been properly associated to the degree they deserve with Tyler'S seminal ideas. It is our belief that Tyler's ideas helped motivate and spawn many developments by others. Some investigators, of course, have acknowledged their great debt to Tyler and have built on and expanded his ideas. For example, ideas such as the taxonomic classification of learning outcomes, the need to validate indirect measures against direct indicators of the trait of interest, the development of scoring techniques that provide teachers with information that is helpful in directing instruction, the concept of formative evaluation, content mastery, decision-oriented evaluation, criterion-referenced and objectives-referenced tests, and criticisms of standardized testing all owe a debt to Tyler's influence.

Others coming out of a different tradition have not realized that what they suggest as new or innovative had already been described sometimes in

xiv PREFACE

great detail by Tyler. An example is Mortimer Adler's call in the Paideia Proposal for the development by teachers of different types of learning ob­jectives and of direct indicators of their attainment. These ideas are expli­cated in great detail in Tyler's work in the Ohio State Service Studies and in Appraising and Recording Student Progress. Here we need to point out that Tyler's ideas were, of course, influenced by his predecessors and men­tors including, among others, Percy Bridgeman, John Watson, John Dewey, E. L. Thorndike, C. H. Judd, and W. W. Charters.

One other example of how contemporary educators could benefit from reading Tyler should suffice. Currently there is intense interest in teaching, and in evaluating students' higher order thinking skills (euphemistically referred to as HOTS). The work of Tyler and his colleagues in the Eight­Year Study in defining and developing indicators of what they called critical thinking skills should provide a rich source of ideas about HOTS and their measurement. We feel that at the very least Tyler's early writings provide an extensive set of advance organizers for developments that are commonly attributed to more recent authors.

Depending on one's values and philosophy of education, a few of Tyler's ideas are viewed as having led education in the wrong direction. For exam­ple, there was always a tension in his writings between effectively serving local needs for data and finding efficient ways to serve broad societal needs for accountability. His contribution to testing served the latter but probably detracted from localized internal evaluation services such as those illustrat­ed in his service studies at Ohio State and in the Eight-Year Study. In those cases there was a realization of the idiosyncratic nature of instruction and learning. Tyler worked closely with instructors in team efforts to use testing and evaluation to improve their courses and ultimately student learning. But the classroom and school were the units of interest. Unfortunately, we feel - and this, of course, is a value judgment on our part - the approach was not sustained, possibly due in part to its high costs. His later influence and leadership was in the direction of developing efficient national tests, such as the National Assessment of Educational Progress (NAEP). NAEP has cost the American public heavily. Unfortunately, while the results have been used extensively for accountability and to inform public policy debates - legitimate societal concerns - they have not yet impacted on instruction at the classroom level.

Our aim in this book is not to critically evaluate the works of Ralph Tyler. That requires a different kind of volume. In fact, our comments have necessarily been kept to a minimum so that we could include as much of Tyler's work as possible within the page limitations. Therefore, what we have attempted to do is to set the record straight by presenting large excerpts from a select body of his extensive works. By and large, we present

PREFACE xv

a small, but what we feel is an important, sample of Tyler's writings over the last 55 years. Tyler's bibliography spans 60 years and hundreds of works. Obviously we had to be very selective when choosing among Tyler's writings, as well as editing within a particular work. We were guided by several criteria in this selection process. First, we wanted the sample to be representative of his major involvements in testing and evaluation. Second, we tried to use works that Tyler himself recommended be included. Third, we chose to excerpt from works that are generally thought of as classic but difficult to find, e.g., Appraising and Recording Student Progress, or what we felt was the best brief description available of an important contribution of Tyler's, e.g., the early descriptions of National Assessment. Finally, we must confess that we gravitated to works that we read as graduate students and that influenced our careers and thinking, works we have recommended to our students over the years but which they found difficult, if not im­possible, to obtain. Therefore, we readily admit that what follows is our subjective judgment of what was important to include and what had to be edited out because of space limitations.

Readers interested in Tyler's complete works are referred to the Ralph Tyler Project of The National Foundation for the Improvement of Educa­tion of the National Education Association. The purpose of that project was to document and archive all of Tyler's work.

The book is divided into six sections. Each section is prefaced by a brief description of its importance and the major points the reader should look for. We hope you will see the richness, the concreteness, and the creativity that we have found in the writings. Those who contribute to advancements in any profession must stand on the shoulders of giants; clearly Ralph Tyler is a giant in the fields of educational testing, curriculum development, and evaluation.

In preparing this book we owe a great debt of gratitude to many people. First, we would like to thank our editors at Kluwer-Nijhoff, Jeffrey Smith and Zachary Rolnik, for their vision in agreeing to publish this collection of Tyler'S work and for their support and patience during the project. Second, our thanks to Helen Kolodziey of the Ralph Tyler Project for her suggestions and for sending us copies of hard-to-obtain articles for con­sideration, and for the Tyler bibliography. Sandra Ryan of Western Michigan University, and Amelia Kreitzer and Katherine Spinos of Boston College, read drafts and gave us their impressions from a student's view of the material. Sally Veeder, of Western Michigan University helped produce and proof the final manuscript. Finally, we will never be able to thank Ralph Tyler sufficiently for his permission to undertake the work, his encouragement and suggestions during the process, and for the influence he has had on our careers.

Educational Evaluation: Classic Works of Ralph W. Tyler