1. HANDBOOK ofPSYCHOLOGY VOLUME 2RESEARCH METHODS IN PSYCHOLOGYJohn A. SchinkaWayne F. VelicerVolume EditorsIrving B. WeinerEditor-in-ChiefJohn Wiley & Sons, Inc.

2. HANDBOOK ofPSYCHOLOGY 3. HANDBOOK ofPSYCHOLOGY VOLUME 2RESEARCH METHODS IN PSYCHOLOGYJohn A. SchinkaWayne F. VelicerVolume EditorsIrving B. WeinerEditor-in-ChiefJohn Wiley & Sons, Inc. 4. This book is printed on acid-free paper.Copyright 2003 by John Wiley & Sons, Inc., Hoboken, New Jersey. All rights reserved.Published simultaneously in Canada.No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic,mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 UnitedStates Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of theappropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400,fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to thePermissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008,e-mail: permcoordinator@wiley.com.Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, theymake no representations or warranties with respect to the accuracy or completeness of the contents of this book and specificallydisclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended bysales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation.You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit orany other commercial damages, including but not limited to special, incidental, consequential, or other damages.This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is soldwith the understanding that the publisher is not engaged in rendering professional services. If legal, accounting, medical,psychological or any other expert assistance is required, the services of a competent professional person should be sought.Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where John Wiley &Sons, Inc. is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact theappropriate companies for more complete information regarding trademarks and registration.For general information on our other products and services please contact our Customer Care Department within the U.S. at(800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available inelectronic books.Library of Congress Cataloging-in-Publication Data:Handbook of psychology / Irving B. Weiner, editor-in-chief. p. cm.Includes bibliographical references and indexes.Contents: v. 1. History of psychology / edited by Donald K. Freedheim v. 2. Research methods in psychology / edited by John A. Schinka, Wayne F. Velicer v. 3. Biological psychology / edited by Michela Gallagher, Randy J. Nelson v. 4. Experimental psychology / edited by Alice F. Healy, Robert W. Proctor v. 5. Personality and social psychology / edited by Theodore Millon, Melvin J. Lerner v. 6. Developmental psychology / edited by Richard M. Lerner, M. Ann Easterbrooks, Jayanthi Mistry v. 7. Educational psychology / edited by William M. Reynolds, Gloria E. Miller v. 8. Clinical psychology / edited by George Stricker, Thomas A. Widiger v. 9. Health psychology / edited by Arthur M. Nezu, Christine Maguth Nezu, Pamela A. Geller v. 10. Assessment psychology / edited by John R. Graham, Jack A. Naglieri v. 11. Forensic psychology / edited by Alan M. Goldstein v. 12. Industrial and organizational psychology / edited by Walter C. Borman, Daniel R. Ilgen, Richard J. Klimoski.ISBN 0-471-17669-9 (set) ISBN 0-471-38320-1 (cloth : alk. paper : v. 1) ISBN 0-471-38513-1 (cloth : alk. paper : v. 2) ISBN 0-471-38403-8 (cloth : alk. paper : v. 3) ISBN 0-471-39262-6 (cloth : alk. paper : v. 4) ISBN 0-471-38404-6 (cloth : alk. paper : v. 5) ISBN 0-471-38405-4 (cloth : alk. paper : v. 6) ISBN 0-471-38406-2 (cloth : alk. paper : v. 7) ISBN 0-471-39263-4 (cloth : alk. paper : v. 8) ISBN 0-471-38514-X (cloth : alk. paper : v. 9) ISBN 0-471-38407-0 (cloth : alk. paper : v. 10) ISBN 0-471-38321-X (cloth : alk. paper : v. 11) ISBN 0-471-38408-9 (cloth : alk. paper : v. 12)1. Psychology. I. Weiner, Irving B. BF121.H1955 2003 150dc21 2002066380Printed in the United States of America.10 9 8 7 6 5 4 3 2 1 5. Editorial BoardVolume 1Volume 5 Volume 9History of Psychology Personality and Social PsychologyHealth PsychologyDonald K. Freedheim, PhDTheodore Millon, PhD Arthur M. Nezu, PhDCase Western Reserve University Institute for Advanced Studies inChristine Maguth Nezu, PhDCleveland, OhioPersonology and Psychopathology Pamela A. Geller, PhDCoral Gables, Florida Drexel UniversityMelvin J. Lerner, PhDPhiladelphia, PennsylvaniaVolume 2Florida Atlantic UniversityResearch Methods in PsychologyBoca Raton, FloridaVolume 10 Assessment PsychologyJohn A. Schinka, PhDUniversity of South Florida Volume 6 John R. Graham, PhDTampa, FloridaDevelopmental Psychology Kent State UniversityRichard M. Lerner, PhD Kent, OhioWayne F. Velicer, PhDUniversity of Rhode IslandM. Ann Easterbrooks, PhD Jack A. Naglieri, PhDKingston, Rhode IslandJayanthi Mistry, PhD George Mason UniversityTufts University Fairfax, VirginiaMedford, Massachusetts Volume 11Volume 3 Forensic PsychologyBiological Psychology Volume 7Educational Psychology Alan M. Goldstein, PhDMichela Gallagher, PhD John Jay College of CriminalJohns Hopkins UniversityWilliam M. Reynolds, PhDJusticeCUNYBaltimore, Maryland Humboldt State UniversityNew York, New YorkRandy J. Nelson, PhDArcata, CaliforniaOhio State University Gloria E. Miller, PhDVolume 12Columbus, OhioUniversity of Denver Industrial and OrganizationalDenver, Colorado Psychology Walter C. Borman, PhDVolume 4Volume 8 University of South FloridaExperimental Psychology Clinical PsychologyTampa, FloridaAlice F. Healy, PhD George Stricker, PhD Daniel R. Ilgen, PhDUniversity of ColoradoAdelphi University Michigan State UniversityBoulder, Colorado Garden City, New YorkEast Lansing, MichiganRobert W. Proctor, PhDThomas A. Widiger, PhD Richard J. Klimoski, PhDPurdue University University of Kentucky George Mason UniversityWest Lafayette, Indiana Lexington, KentuckyFairfax, Virginia v 6. My efforts in this work are proudly dedicated to Katherine, Christy, and John C. Schinka. J. A. S. This work is dedicated to Sue, the perfect companion for lifes many journeysand the center of my personal universe.W. F. V. 7. Handbook of Psychology PrefacePsychology at the beginning of the twenty-rst century has A second unifying thread in psychology is a commitmentbecome a highly diverse eld of scientic study and appliedto the development and utilization of research methodstechnology. Psychologists commonly regard their discipline suitable for collecting and analyzing behavioral data. Withas the science of behavior, and the American Psychological attention both to specific procedures and their applicationAssociation has formally designated 2000 to 2010 as thein particular settings, Volume 2 addresses research methodsDecade of Behavior. The pursuits of behavioral scientistsin psychology.range from the natural sciences to the social sciences and em- Volumes 3 through 7 of the Handbook present the sub-brace a wide variety of objects of investigation. Some psy-stantive content of psychological knowledge in five broadchologists have more in common with biologists than with areas of study: biological psychology (Volume 3), experi-most other psychologists, and some have more in common mental psychology (Volume 4), personality and social psy-with sociologists than with most of their psychological col- chology (Volume 5), developmental psychology (Volume 6),leagues. Some psychologists are interested primarily in the be-and educational psychology (Volume 7). Volumes 8 throughhavior of animals, some in the behavior of people, and others12 address the application of psychological knowledge inin the behavior of organizations. These and other dimensions ve broad areas of professional practice: clinical psychologyof difference among psychological scientists are matched by(Volume 8), health psychology (Volume 9), assessment psy-equal if not greater heterogeneity among psychological practi- chology (Volume 10), forensic psychology (Volume 11), andtioners, who currently apply a vast array of methods in many industrial and organizational psychology (Volume 12). Eachdifferent settings to achieve highly varied purposes.of these volumes reviews what is currently known in thesePsychology has been rich in comprehensive encyclope- areas of study and application and identies pertinent sourcesdias and in handbooks devoted to specic topics in the eld. of information in the literature. Each discusses unresolved is-However, there has not previously been any single handbook sues and unanswered questions and proposes future direc-designed to cover the broad scope of psychological science tions in conceptualization, research, and practice. Each of theand practice. The present 12-volume Handbook of Psychol- volumes also reects the investment of scientic psycholo-ogy was conceived to occupy this place in the literature.gists in practical applications of their ndings and the atten-Leading national and international scholars and practitionerstion of applied psychologists to the scientic basis of theirhave collaborated to produce 297 authoritative and detailedmethods.chapters covering all fundamental facets of the discipline,The Handbook of Psychology was prepared for the pur-and the Handbook has been organized to capture the breadth pose of educating and informing readers about the presentand diversity of psychology and to encompass interests and state of psychological knowledge and about anticipated ad-concerns shared by psychologists in all branches of the eld.vances in behavioral science research and practice. With thisTwo unifying threads run through the science of behavior.purpose in mind, the individual Handbook volumes addressThe rst is a common history rooted in conceptual and em-the needs and interests of three groups. First, for graduate stu-pirical approaches to understanding the nature of behavior.dents in behavioral science, the volumes provide advancedThe specific histories of all specialty areas in psychologyinstruction in the basic concepts and methods that dene thetrace their origins to the formulations of the classical philoso-elds they cover, together with a review of current knowl-phers and the methodology of the early experimentalists, and edge, core literature, and likely future developments. Second,appreciation for the historical evolution of psychology in all in addition to serving as graduate textbooks, the volumesof its variations transcends individual identities as being oneoffer professional psychologists an opportunity to read andkind of psychologist or another. Accordingly, Volume 1 incontemplate the views of distinguished colleagues concern-the Handbook is devoted to the history of psychology asing the central thrusts of research and leading edges of prac-it emerged in many areas of scientific study and applied tice in their respective elds. Third, for psychologists seekingtechnology.to become conversant with elds outside their own specialtyix 8. x Handbook of Psychology Prefaceand for persons outside of psychology seeking informa- valuable contributions to the literature. I would like nally totion about psychological matters, the Handbook volumes express my appreciation to the editorial staff of John Wileyserve as a reference source for expanding their knowledgeand Sons for the opportunity to share in the development ofand directing them to additional sources in the literature.this project and its pursuit to fruition, most particularly to The preparation of this Handbook was made possible by Jennifer Simon, Senior Editor, and her two assistants, Marythe diligence and scholarly sophistication of the 25 volumePortereld and Isabel Pratt. Without Jennifers vision of theeditors and co-editors who constituted the Editorial Board.Handbook and her keen judgment and unagging support inAs Editor-in-Chief, I want to thank each of them for the plea- producing it, the occasion to write this preface would notsure of their collaboration in this project. I compliment them have arrived.for having recruited an outstanding cast of contributors totheir volumes and then working closely with these authors toIRVING B. WEINERachieve chapters that will stand each in their own right as Tampa, Florida 9. Volume PrefaceA scientific discipline is defined in many ways by the re- computers to perform complex methods of data analysis, in-search methods it employs. These methods can be said to rep- creased computer capacity allowing for more intense analysisresent the common language of the disciplines researchers.of larger datasets, computer simulations that permit the eval-Consistent with the evolution of a lexicon, new research uation of procedures across a wide variety of situations, newmethods frequently arise from the development of new approaches to data analysis and statistical control, and ad-content areas. By every available measurenumber of re-vances in companion sciences that opened pathways to thesearchers, number of publications, number of journals, num-exploration of behavior and created new areas of researchber of new subdisciplinespsychology has undergone a specialization and collaboration.tremendous growth over the last half-century. This growth is Consider the advances since the publication of thereected in a parallel increase in the number of new researchfirst edition of Kirks (1968) text on experimental design.methods available. At that time most studies were relatively small N experiments As we were planning and editing this volume, we dis-that were conducted in psychology laboratories. Research ac-cussed on many occasions the extent to which psychologytivity has subsequently exploded in applied and clinicaland the available research methods have become increasingareas, with a proliferation of new journals largely dedicatedcomplex over the course of our careers. When our generationto quasi-experimental studies and studies in the natural envi-of researchers began their careers in the late 1960s and early ronment (e.g., in neuropsychology and health psychology).1970s, experimental design was largely limited to simple Techniques such as polymerase chain reaction allow psychol-between-group designs, and data analysis was dominated byogists to test specic genes as risk candidates for behaviorala single method, the analysis of variance. A few other ap- disorders. These studies rely on statistical procedures that areproaches were employed, but by a limited number of re- still largely ignored by many researchers (e.g., logistic re-searchers. Multivariate statistics had been developed, but gression, structural equation modeling). Brain imagingmultiple regression analysis was the only method that wasprocedures such as magnetic resonance imaging, magnetoen-applied with any frequency. Factor analysis was used almostcephalography, and positron-emission tomography provideexclusively as a method in scale development. Classical test cognitive psychologists and neuropsychologists the opportu-theory was the basis of most psychological and educational nity to study cortical activity on-line. Clinical trials involvingmeasures. Analysis of data from studies that did not meetbehavioral interventions applied to large, representative sam-either the design or measurement assumptions required for an ples are commonplace in health psychology. Research em-analysis of variance was covered for most researchers by a ploying each of these procedures requires not only highlysingle book on nonparametric statistics by Siegel (1956). As specific and rigorous research methods, but also speciala review of the contents of this volume illustrates, the choicemethods for handling and analyzing extremely large volumesof experimental and analytic methods available to theof data. Even in more traditional areas of research that con-present-day researcher is much broader. It would be fair totinue to rely on group experimental designs, issues of mea-say that the researcher in the 1960s had to formulate research suring practical significance, determination of sample sizequestions to t the available methods. Currently, there are re-and power, and procedures for handling nuisance variablessearch methods available to address most research questions. are now important concerns. Not surprisingly, the third edi- In the history of science, an explosion of knowledge is tion of Kirks (1995) text has grown in page length by 60%.usually the result of an advance in technology, new theoreti-Our review of these trends leads to several conclusions,cal models, or unexpected empirical ndings. Advances in which are reected in the selection of topics covered by theresearch methods have occurred as the result of all three fac- chapters in this volume. Six features appear to characterizetors, typically in an interactive manner. Some of the specicthe evolution in research methodology in psychology.factors include advances in instrumentation and measure- First, there has been a focus on the development of proce-ment technology, the availability of inexpensive desktop dures that employ statistical control rather than experimentalxi 10. xii Volume Prefacecontrol. Because most of the recent growth involves research to focus on the individual and model individual differences.in areas that preclude direct control of independent variables,This becomes increasingly important as we recognize that in-multivariate statistics and the development of methods suchterventions do not affect everyone in exactly the same waysas path analysis and structural equation modeling have beenand that interventions become more and more tailored to thecritical developments. The use of statistical control has al-individual.lowed psychology to move from the carefully controlled con-The text is organized into four parts. The rst part, titlednes of the laboratory to the natural environment. Foundations of Research, addresses issues that are funda-Second, there has been an increasing focus on construct- mental to all behavioral science research. The focus is ondriven, or latent-variable, research. A construct is dened by study design, data management, data reduction, and data syn-multiple observed variables. Constructs can be viewed as thesis. The rst chapter, Experimental Design by Roger E.more reliable and more generalizable than a single observedKirk, provides an overview of the basic considerations thatvariable. Constructs serve to organize a large set of observed go into the design of a study. Once, a chapter on this topicvariables, resulting in parsimony. Constructs are also theoret-would have had to devote a great deal of attention to compu-ically based. This theory-based approach serves to guide tational procedures. The availability of computers permits astudy design, the choice of variables, the data analysis, andshift in focus to the conceptual rather than the computationalthe data interpretation. issues. The second chapter, Exploratory Data Analysis byThird, there has been an increasing emphasis on the de-John T. Behrens and Chong-ho Yu, reminds us of the funda-velopment of new measures and new measurement models.mental importance of looking at data in the most basic waysThis is not a new trend but an acceleration of an old trend. as a rst step in any data analysis. In some ways this repre-The behavioral sciences have always placed the most empha- sents a back to the future chapter. Advances in computer-sis on the issue of measurement. With the movement of thebased graphical methods have brought a great deal of sophis-eld out of the laboratory combined with advances in tech- tication to this very basic rst step.nology, the repertoire of measures, the quality of the mea-The third chapter, Power: Basics, Practical Problems,sures, and the sophistication of the measurement models have and Possible Solutions by Rand R. Wilcox, reects the crit-all increased dramatically.ical change in focus for psychological research. Originally,Fourth, there is increasing recognition of the importance of the central focus of a test of signicance was on controllingthe temporal dimension in understanding a broad range of psy-Type I error rates. The late Jacob Cohen emphasized that re-chological phenomena. We have become a more intervention-searchers should be equally concerned by Type II errors.oriented science, recognizing not only the complexity of This resulted in an emphasis on the careful planning of atreatment effects but also the importance of the change in pat-study and a concern with effect size and selecting the appro-terns of the effects over time. The effects of an intervention priate sample size. Wilcox updates and extends these con-may be very different at different points in time. New statisti- cepts. Chapter 4, Methods for Handling Missing Data bycal models for modeling temporal data have resulted. John W. Graham, Patricio E. Cumsille, and Elvira Elek-Fisk,Fifth, new methods of analysis have been developed describes the impressive statistical advances in addressingthat no longer require the assumption of a continuous, equal-the common practical problem of missing observations.interval, normally distributed variable. Previously, re- Previously, researchers had relied on a series of ad hoc pro-searchers had the choice between very simple but limited cedures, often resulting in very inaccurate estimates. The newmethods of data analysis that corresponded to the properties statistical procedures allow the researcher to articulate theof the measure or more complex sophisticated methods ofassumptions about the reason the data is missing and makeanalysis that assumed, often inappropriately, that the measure very sophisticated estimates of the missing value based on allmet very rigid assumptions. New methods have been devel- the available information. This topic has taken on even moreoped for categorical, ordinal, or simply nonnormal variables importance with the increasing emphasis on longitudinalthat can perform an equally sophisticated analysis.studies and the inevitable problem of attrition.Sixth, the importance of individual differences is increas-The fth chapter, Preparatory Data Analysis by Linda S.ingly emphasized in intervention studies. Psychology has Fidell and Barbara G. Tabachnick, describes methods of pre-always been interested in individual differences, but meth-processing data before the application of other methods ofods of data analysis have focused almost entirely on the rela- statistical analysis. Extreme values can distort the results oftionships between variables. Individuals were studied as the data analysis if not addressed. Diagnostic methods canmembers of groups, and individual differences served only to preprocess the data so that complex procedures are not un-inate the error variance. New techniques permit researchers duly affected by a limited number of cases that often are the 11. Volume Preface xiiiresult of some type of error. The last two chapters in this part, ndings that are of interest to psychologists in many elds.Factor Analysis by Richard L. Gorsuch and Clustering The major goal of Chapter 11, Animal Learning by Russelland Classification Methods by Glenn W. Milligan andM. Church, is to transfer what is fairly common knowledge inStephen C. Hirtle, describe two widely employed parsimony experimental animal psychology to investigators with limitedmethods. Factor analysis operates in the variable domain andexposure to this area of research. In Chapter 12, Neuropsy-attempts to reduce a set of p observed variables to a smaller chology, Russell M. Bauer, Elizabeth C. Leritz, and Dawnset of m factors. These factors, or latent variables, are moreBowers provide a discussion of neuropsychological inference,easily interpreted and thus facilitate interpretation. Clusteran overview of major approaches to neuropsychological re-analysis operates in the person domain and attempts to reduce search, and a review of newer techniques, including functionala set of N individuals to a set of k clusters. Cluster analysis neuroimaging, electrophysiology, magnetoencephalography,serves to explore the relationships among individuals and or- and reversible lesion methods. In each section, they describeganize the set of individuals into a limited number of sub- the conceptual basis of the technique, outline its strengths andtypes that share essential features. These methods are basic to weaknesses, and cite examples of how it has been used inthe development of construct-driven methods and the focus addressing conceptual problems in neuropsychology.on individual differences.Whatever their specialty area, when psychologists evalu-The second part, Research Methods in Specic Content ate a program or policy, the question of impact is often at cen-Areas, addresses research methods and issues as they apply ter stage. The last chapter in this part, Program Evaluationto specic content areas. Content areas were chosen in part toby Melvin M. Mark, focuses on key methods for estimatingparallel the other volumes of the Handbook. More important, the effects of policies and programs in the context of evalua-however, we attempted to sample content areas from a broadtion. Additionally, Mark addresses several noncausal formsspectrum of specialization with the hope that these chaptersof program evaluation research that are infrequently ad-would provide insights into methodological concerns and dressed in methodological treatises.solutions that would generalize to other areas. Chapter 8,The third part is titled Measurement Issues. Advances inClinical Forensic Psychology by Kevin S. Douglas, Randy measurement typically combine innovation in technologyK. Otto, and Randy Borum, addresses research methods andand progress in theory. As our measures become more so-issues that occur in assessment and treatment contexts. For phisticated, the areas of application also increase.each task that is unique to clinical forensic psychologyMood emerged as a seminal concept within psychologyresearch, they provide examples of the clinical challengesduring the 1980s, and its prominence has continued unabatedconfronting the psychologist, identify problems faced whenever since. In Chapter 14, Mood Measurement: Currentresearching the issues or constructs, and describe not only re- Status and Future Directions, David Watson and Jatin Vaidyasearch strategies that have been employed but also theirexamine current research regarding the underlying structurestrengths and limitations. In Chapter 9, Psychotherapy Out-of mood, describe and evaluate many of the most importantcome Research, Evelyn S. Behar and Thomas D. Borkovecmood measures, and discuss several issues related to theaddress the methodological issues that need to be consideredreliability and construct validity of mood measurement. Infor investigators to draw the strongest and most specific Chapter 15, Measuring Personality and Psychopathology,cause-and-effect conclusions about the active components of Leslie C. Morey uses objective self-report methods of mea-treatments, human behavior, and the effectiveness of thera- surement to illustrate contemporary procedures for scalepeutic interventions. development and validation, addressing issues critical to allThe eld of health psychology is largely dened by threemeasurement methods such as theoretical articulation, situa-topics: the role of behavior (e.g., smoking) in the develop-tional context, and the need for discriminant validity.ment and prevention of disease, the role of stress and emotionThe appeal of circular models lies in the combination of aas psychobiological inuences on disease, and psychological circles aesthetic (organizational) simplicity and its powerfulaspects of acute and chronic illness and medical care. Insightpotential to describe data in uniquely compelling substantiveinto the methodological issues and solutions for research inand geometric ways, as has been demonstrated in describ-each of these topical areas is provided by Timothy W. Smith ing interpersonal behavior and occupational interests. Inin Chapter 10, Health Psychology. Chapter 16, The Circumplex Model: Methods and ResearchAt one time, most behavioral experimentation was con- Applications, Michael B. Gurtman and Aaron L. Pincus dis-ducted by individuals whose training focused heavily on ani-cuss the application of the circumplex model to the descrip-mal research. Now many neuroscientists, trained in varioustions of individuals, comparisons of groups, and evaluationsfields, conduct research in animal learning and publish of constructs and their measures. 12. xiv Volume PrefaceChapter 17, Item Response Theory and Measuring Abili-Velicer and Joseph L. Fava describe a method for studyingties by Karen M. Schmidt and Susan E. Embretson, de- the change in a single individual over time. Instead of a sin-scribes the types of formal models that have been designed to gle observation on many subjects, this method relies on manyguide measure development. For many years, most tests ofobservations on a single subject. In many ways, this methodability and achievement have relied on classical test theory as is the prime exemplar of longitudinal research methods.a framework to guide both measure development and mea-Chapter 24, Structural Equation Modeling by Jodie B.sure evaluation. Item response theory updates this model in Ullman and Peter M. Bentler, describes a very generalmany important ways, permitting the development of a newmethod that combines three key themes: constructs or latentgeneration of measures of abilities and achievement that arevariables, statistical control, and theory to guide data analy-particularly appropriate for a more interactive model of as-sis. First employed as an analytic method little more thansessment. The last chapter of this part, Growth Curve Analy- 20 years ago, the method is now widely disseminated in thesis in Contemporary Psychological Research by John J.behavioral sciences. Chapter 25, Ordinal Analysis of Behav-McArdle and John R. Nesselroade, describes new quantita-ioral Data by Jeffrey D. Long, Du Feng, and Norman Cliff,tive methods for the study of change in development psy-discusses the assumptions that underlie many of the widelychology. The methods permit the researcher to model a wideused statistical methods and describes a parallel series ofvariety of different patterns of developmental change overmethods of analysis that only assume that the measure pro-time. vides ordinal information. The last chapter, Latent Class andThe nal part, Data Analysis Methods, addresses statis- Latent Transition Analysis by Stephanie L. Lanza, Brian P.tical procedures that have been developed recently and areFlaherty, and Linda M. Collins, describes a new method forstill not widely employed by many researchers. They are typ-analyzing change over time. It is particularly appropriateically dependent on the availability of high-speed computerswhen the change process can be conceptualized as a series ofand permit researchers to investigate novel and complex re- discrete states.search questions. Chapter 19, Multiple Linear RegressionIn completing this project, we realized that we were veryby Leona Aiken, Stephen G. West, and Steven C. Pitts, de- fortunate in several ways. Irving Weiners performance asscribes the advances in multiple linear regression that permiteditor-in-chief was simply wonderful. He applied just the rightapplications of this very basic method to the analysis of com-mix of obsessive concern and responsive support to keep thingsplex data sets and the incorporation of conceptual models toon schedule. His comments on issues of emphasis, perspective,guide the analysis. The testing of theoretical predictions andand quality were insightful and inevitably on target.the identification of implementation problems are the two We continue to be impressed with the professionalism ofmajor foci of this chapter. Chapter 20, Logistic Regression the authors that we were able to recruit into this effort.by Alfred DeMaris, describes a parallel method to multipleConsistent with their reputations, these individuals deliv-regression analysis for categorical variables. The procedureered chapters of exceptional quality, making our burdenhas been developed primarily outside of psychology and is pale in comparison to other editorial experiences. Because ofnow being used much more frequently to address psycholog- the length of the project, we shared many contributorsical questions. Chapter 21, Meta-Analysis by Frank L. experiences-marriages, births, illnesses, family crises. A def-Schmidt and John E. Hunter, describes procedures that haveinite plus for us has been the formation of new friendshipsbeen developed for the quantitative integration of research and professional liaisons.ndings across multiple studies. Previously, research ndings Our editorial tasks were also aided greatly by the generouswere integrated in narrative form and were subject to the bi- assistance of our reviewers, most of whom will be quicklyases of the reviewer. The method also focuses attention on therecognized by our readers for their own expertise in researchimportance of effect size estimation. methodology. We are pleased to thank James Algina, PhippsChapter 22, Survival Analysis by Judith D. Singer and Arabie, Patti Barrows, Betsy Jane Becker, Lisa M. Brown,John B. Willett, describes a recently developed method forBarbara M. Byrne, William F. Chaplin, Pat Cohen, Patrick J.analyzing longitudinal data. One approach is to code whetherCurren, Glenn Curtiss, Richard B. Darlington, Susanan event has occurred at a given occasion. By switching the Duncan, Brian Everitt, Kerry Evers, Ron Gironda, Lisafocus on the time to the occurrence of the event, a much more Harlow, Michael R. Harwell, Don Hedeker, David Charlespowerful and sophisticated analysis can be performed. Again,Howell, Lawrence J. Hubert, Bradley E. Huitema, Beththe development of this procedure has occurred largely out- Jenkins, Herbert W. Marsh, Rosemarie A. Martin, Scott E.side psychology but is being employed much more fre-Maxwell, Kevin R. Murphy, Gregory Norman, Daniel J.quently. In Chapter 23, Time Series Analysis, Wayne Ozer, Melanie Page, Mark D. Reckase, Charles S. Reichardt, 13. Volume Preface xvSteven Reise, Joseph L. Rogers, Joseph Rossi, Jamesor supported by differing approaches. For aws in the text,Rounds, Shlomo S. Sawilowsky, Ian Spence, James H. however, the usual rule applies: We assume all responsibility.Steiger, Xiaowu Sun, Randall C. Swaim, David Thissen,Bruce Thompson, Terence J. G. Tracey, Rod Vanderploeg, JOHN A. SCHINKAPaul F. Velleman, Howard Wainer, Douglas Williams, and WAYNE F. VELICERseveral anonymous reviewers for their thorough work andgood counsel. We finish this preface with a caveat. Readers will in-REFERENCESevitably discover several contradictions or disagreementsacross the chapter offerings. Inevitably, researchers in differ- Kirk, Roger E. (1968). Experimental design: Procedures for theent areas solve similar methodological problems in differentbehavioral sciences. Pacic Grove, CA: Brooks/Cole.ways. These differences are reected in the offerings of thisKirk, Roger E. (1995). Experimental design: Procedures for thetext, and we have not attempted to mediate these differingbehavioral sciences (3rd ed.). Pacic Grove, CA: Brooks/Cole.viewpoints. Rather, we believe that the serious researcher Siegel, S. (1956). Nonparametric statistics for the behavioralwill welcome the opportunity to review solutions suggestedsciences. New York: McGraw-Hill. 14. ContentsHandbook of Psychology Preface ixIrving B. WeinerVolume Preface xi John A. Schinka and Wayne F. VelicerContributors xxiPA RT O N EFOUNDATIONS OF RESEARCH ISSUES: STUDY DESIGN, DATA MANAGEMENT, DATA REDUCTION, AND DATA SYNTHESIS1 EXPERIMENTAL DESIGN 3Roger E. Kirk2 EXPLORATORY DATA ANALYSIS 33John T. Behrens and Chong-ho Yu3 POWER: BASICS, PRACTICAL PROBLEMS, AND POSSIBLE SOLUTIONS 65Rand R. Wilcox4 METHODS FOR HANDLING MISSING DATA 87John W. Graham, Patricio E. Cumsille, and Elvira Elek-Fisk5 PREPARATORY DATA ANALYSIS 115Linda S. Fidell and Barbara G. Tabachnick6 FACTOR ANALYSIS 143Richard L. Gorsuch7 CLUSTERING AND CLASSIFICATION METHODS 165Glenn W. Milligan and Stephen C. HirtlePA RT T W ORESEARCH METHODS IN SPECIFIC CONTENT AREAS8 CLINICAL FORENSIC PSYCHOLOGY 189Kevin S. Douglas, Randy K. Otto, and Randy Borum9 PSYCHOTHERAPY OUTCOME RESEARCH 213Evelyn S. Behar and Thomas D. Borkovecxvii 15. xviii Contents10 HEALTH PSYCHOLOGY 241 Timothy W. Smith11 ANIMAL LEARNING 271 Russell M. Church12 NEUROPSYCHOLOGY 289 Russell M. Bauer, Elizabeth C. Leritz, and Dawn Bowers13 PROGRAM EVALUATION 323 Melvin M. Mark PA RT T H R E E MEASUREMENT ISSUES14 MOOD MEASUREMENT: CURRENT STATUS AND FUTURE DIRECTIONS 351 David Watson and Jatin Vaidya15 MEASURING PERSONALITY AND PSYCHOPATHOLOGY 377 Leslie C. Morey16 THE CIRCUMPLEX MODEL: METHODS AND RESEARCH APPLICATIONS 407 Michael B. Gurtman and Aaron L. Pincus17 ITEM RESPONSE THEORY AND MEASURING ABILITIES 429 Karen M. Schmidt and Susan E. Embretson18 GROWTH CURVE ANALYSIS IN CONTEMPORARY PSYCHOLOGICAL RESEARCH 447 John J. McArdle and John R. NesselroadePA RT F O U RDATA ANALYSIS METHODS19 MULTIPLE LINEAR REGRESSION 483 Leona S. Aiken, Stephen G. West, and Steven C. Pitts20 LOGISTIC REGRESSION 509 Alfred DeMaris21 META-ANALYSIS 533 Frank L. Schmidt and John E. Hunter22 SURVIVAL ANALYSIS 555 Judith D. Singer and John B. Willett23 TIME SERIES ANALYSIS 581 Wayne F. Velicer and Joseph L. Fava 16. Contents xix24STRUCTURAL EQUATION MODELING 607Jodie B. Ullman and Peter M. Bentler25ORDINAL ANALYSIS OF BEHAVIORAL DATA 635Jeffrey D. Long, Du Feng, and Norman Cliff26LATENT CLASS AND LATENT TRANSITION ANALYSIS 663Stephanie T. Lanza, Brian P. Flaherty, and Linda M. CollinsAuthor Index 687Subject Index 703 17. ContributorsLeona S. Aiken, PhDNorman Cliff, PhDDepartment of Psychology Professor of Psychology EmeritusArizona State University University of Southern CaliforniaTempe, Arizona Los Angeles, CaliforniaRussell M. Bauer, PhDLinda M. Collins, PhDDepartment of Clinical and Health Psychology The Methodology CenterUniversity of FloridaPennsylvania State UniversityGainesville, Florida University Park, PennsylvaniaEvelyn S. Behar, MSPatricio E. Cumsille, PhDDepartment of Psychology Escuela de PsicologiaPennsylvania State UniversityUniversidad Catlica de ChileUniversity Park, PennsylvaniaSantiago, ChileJohn T. Behrens, PhD Alfred DeMaris, PhDCisco Networking Academy Program Department of SociologyCisco Systems, Inc.Bowling Green State UniversityPhoenix, Arizona Bowling Green, OhioPeter M. Bentler, PhDKevin S. Douglas, PhD, LLBDepartment of Psychology Department of Mental Health Law & PolicyUniversity of California Florida Mental Health InstituteLos Angeles, CaliforniaUniversity of South Florida Tampa, FloridaThomas D. Borkovec, PhDDepartment of Psychology Du Feng, PhDPennsylvania State UniversityHuman Development and Family StudiesUniversity Park, PennsylvaniaTexas Tech University Lubbock, TexasRandy Borum, PsyDDepartment of Mental Health Law & Policy Elvira Elek-Fisk, PhDFlorida Mental Health InstituteThe Methodology CenterUniversity of South FloridaPennsylvania State UniversityTampa, Florida University Park, PennsylvaniaDawn Bowers, PhD Susan E. Embretson, PhDDepartment of Clinical and Health Psychology Department of PsychologyUniversity of FloridaUniversity of KansasGainesville, Florida Lawrence, KansasRussell M. Church, PhD Joseph L. Fava, PhDDepartment of Psychology Cancer Prevention Research CenterBrown University University of Rhode IslandProvidence, Rhode Island Kingston, Rhode Island xxi 18. xxii ContributorsLinda S. Fidell, PhD Melvin M. Mark, PhDDepartment of Psychology Department of PsychologyCalifornia State UniversityPennsylvania State UniversityNorthridge, California University Park, PennsylvaniaBrian P. Flaherty, MSJohn J. McArdle, PhDThe Methodology Center Department of PsychologyPennsylvania State UniversityUniversity of VirginiaUniversity Park, PennsylvaniaCharlottesville, VirginiaRichard L. Gorsuch, PhDGlenn W. Milligan, PhDGraduate School of PsychologyDepartment of Management SciencesFuller Theological SeminaryOhio State UniversityPasadena, California Columbus, OhioJohn W. Graham, PhDLeslie C. Morey, PhDDepartment of Biobehavioral Health Department of PsychologyPennsylvania State UniversityTexas A&M UniversityUniversity Park, PennsylvaniaCollege Station, TexasMichael B. Gurtman, PhDJohn R. Nesselroade, PhDDepartment of Psychology Department of PsychologyUniversity of Wisconsin-Parkside University of VirginiaKenosha, Wisconsin Charlottesville, VirginiaStephen C. Hirtle, PhD Randy K. Otto, PhDSchool of Information Sciences Department of Mental Health Law & PolicyUniversity of Pittsburgh Florida Mental Health InstitutePittsburgh, Pennsylvania University of South Florida Tampa, FloridaJohn E. Hunter, PhD Aaron L. Pincus, PhDDepartment of Psychology Department of PsychologyMichigan State University Pennsylvania State UniversityEast Lansing, Michigan University Park, PennsylvaniaRoger E. Kirk, PhD Steven C. Pitts, PhDDepartment of Psychology and Neuroscience Department of PsychologyBaylor University University of Maryland, Baltimore CountyWaco, Texas Baltimore, MarylandStephanie T. Lanza, MS Karen M. Schmidt, PhDThe Methodology Center Department of PsychologyPennsylvania State University University of VirginiaUniversity Park, Pennsylvania Charlottesville, VirginiaElizabeth C. Leritz, MSFrank L. Schmidt, PhDDepartment of Clinical and Health Psychology Department of Management and OrganizationUniversity of FloridaUniversity of IowaGainesville, Florida Iowa City, IowaJeffrey D. Long, PhD Judith D. Singer, PhDDepartment of Educational Psychology Graduate School of EducationUniversity of MinnesotaHarvard UniversityMinneapolis, Minnesota Cambridge, Massachusetts 19. Contributors xxiiiTimothy W. Smith, PhD David Watson, PhDDepartment of PsychologyDepartment of PsychologyUniversity of UtahUniversity of IowaSalt Lake City, UtahIowa City, IowaBarbara G. Tabachnick, PhDStephen G. West, PhDDepartment of PsychologyDepartment of PsychologyCalifornia State University Arizona State UniversityNorthridge, CaliforniaTempe, ArizonaJodie B. Ullman, PhDRand R. Wilcox, PhDDepartment of PsychologyDepartment of PsychologyCalifornia State University University of Southern CaliforniaSan Bernadino, California Los Angeles, CaliforniaJatin VaidyaJohn B. Willett, PhDDepartment of PsychologyGraduate School of EducationUniversity of IowaHarvard UniversityIowa City, Iowa Cambridge, MassachusettsWayne F. Velicer, PhD Chong-ho Yu, PhDCancer Prevention Research Center Cisco Networking Academy ProgramUniversity of Rhode IslandCisco Systems, Inc.Kingston, Rhode IslandChandler, Arizona 20. PA R T O N E FOUNDATIONS OF RESEARCH ISSUES: STUDY DESIGN, DATA MANAGEMENT,DATA REDUCTION, AND DATA SYNTHESIS 21. CHAPTER 1Experimental DesignROGER E. KIRKSOME BASIC EXPERIMENTAL DESIGN CONCEPTS 3 FACTORIAL DESIGNS WITH CONFOUNDING 21THREE BUILDING BLOCK DESIGNS 4Split-Plot Factorial Design 21Completely Randomized Design 4Confounded Factorial Designs 24Randomized Block Design 6 Fractional Factorial Designs 25Latin Square Design 9 HIERARCHICAL DESIGNS 27CLASSIFICATION OF EXPERIMENTAL DESIGNS 10 Hierarchical Designs With One orFACTORIAL DESIGNS 11 Two Nested Treatments 27Completely Randomized Factorial Design 11 Hierarchical Design With CrossedAlternative Models 14and Nested Treatments 28Randomized Block Factorial Design 19EXPERIMENTAL DESIGNS WITH A COVARIATE 29REFERENCES 31SOME BASIC EXPERIMENTALabout (a) one or more parameters of a population or (b) theDESIGN CONCEPTSfunctional form of a population. Statistical hypotheses are rarely identical to scientic hypothesesthey areExperimental design is concerned with the skillful interroga-testable formulations of scientic hypotheses.tion of nature. Unfortunately, nature is reluctant to reveal2. Determination of the experimental conditions (independenther secrets. Joan Fisher Box (1978) observed in her autobiog-variable) to be manipulated, the measurement (dependentraphy of her father, Ronald A. Fisher, Far from behavingvariable) to be recorded, and the extraneous conditionsconsistently, however, Nature appears vacillating, coy, and(nuisance variables) that must be controlled.ambiguous in her answers (p. 140). Her most effective3. Specication of the number of participants required andtool for confusing researchers is variabilityin particular, the population from which they will be sampled.variability among participants or experimental units. But4. Specication of the procedure for assigning the partici-two can play the variability game. By comparing the variabil- pants to the experimental conditions.ity among participants treated differently to the variability5.Determination of the statistical analysis that will beamong participants treated alike, researchers can make in- performed.formed choices between competing hypotheses in scienceand technology.In short, an experimental design identies the independent, We must never underestimate natureshe is a formidabledependent, and nuisance variables and indicates the way infoe. Carefully designed and executed experiments are re-which the randomization and statistical aspects of an experi-quired to learn her secrets. An experimental design is a planment are to be carried out.for assigning participants to experimental conditions and thestatistical analysis associated with the plan (Kirk, 1995, p. 1).The design of an experiment involves a number of inter- Analysis of Variancerelated activities:Analysis of variance (ANOVA) is a useful tool for under-1. Formulation of statistical hypotheses that are germane to thestanding the variability in designed experiments. The seminal scientic hypothesis. A statistical hypothesis is a statementideas for both ANOVA and experimental design can be traced3 22. 4 Experimental Designto Ronald A. Fisher, a statistician who worked at the Rotham- Fisher popularized two other principles of good experi-sted Experimental Station. According to Box (1978, p. 100), mentation: replication and local control or blocking. Replica-Fisher developed the basic ideas ofANOVAbetween 1919 andtion is the observation of two or more participants under1925. The rst hint of what was to come appeared in a 1918identical experimental conditions. Fisher observed that repli-paper in which Fisher partitioned the total variance of a human cation enables a researcher to estimate error effects andattribute into portions attributed to heredity, environment, andobtain a more precise estimate of treatment effects. Blocking,other factors. The analysis of variance table for a two-treat-on the other hand, is an experimental procedure for isolatingment factorial design appeared in a 1923 paper published with variation attributable to a nuisance variable. As the nameM. A. Mackenzie (Fisher & Mackenzie, 1923). Fisher referred suggests, nuisance variables are undesired sources of varia-to the table as a convenient way of arranging the arithmetic. Intion that can affect the dependent variable. There are many1924 Fisher (1925) introduced the Latin square design in con- sources of nuisance variation. Differences among partici-nection with a forest nursery experiment. The publication inpants comprise one source. Other sources include variation1925 of his classic textbook Statistical Methods for Research in the presentation of instructions to participants, changes inWorkers and a short paper the following year (Fisher, 1926) environmental conditions, and the effects of fatigue andpresented all the essential ideas of analysis of variance. Thelearning when participants are observed several times. Threetextbook (Fisher, 1925, pp. 244249) included a table of theexperimental approaches are used to deal with nuisancecritical values of the ANOVA test statistic in terms of a func- variables:tion called z, where z = 1 (ln 2 2 Treatment ln 2 ). The statis- Error22tics Treatment and Error denote, respectively, treatment and1. Holding the variable constant.error variance. A more convenient form of Fishers z table that 2. Assigning participants randomly to the treatment levels sodid not require looking up log values was developed by that known and unsuspected sources of variation amongGeorge Snedecor (1934). His critical values are expressed in the participants are distributed over the entire experimentterms of the function F = 2 Treatment /2 Error that is obtainedand do not affect just one or a limited number of treatmentdirectly from the ANOVA calculations. He named it F in honor levels.of Fisher. Fishers eld of experimentationagriculture3. Including the nuisance variable as one of the factors in thewas a fortunate choice because results had immediate applica-experiment.tion with assessable economic value, because simplifyingassumptions such as normality and independence of errorsThe last experimental approach uses local control or blockingwere usually tenable, and because the cost of conductingto isolate variation attributable to the nuisance variable soexperiments was modest. that it does not appear in estimates of treatment and erroreffects. A statistical approach also can be used to deal withThree Principles of Good Experimental Designnuisance variables. The approach is called analysis of covari-ance and is described in the last section of this chapter.The publication of Fishers Statistical Methods for ResearchThe three principles that Fisher vigorously championedWorkers and his 1935 The Design of Experiments graduallyrandomization, replication, and local controlremain theled to the acceptance of what today is considered to be the cornerstones of good experimental design.cornerstone of good experimental design: randomization.It is hard to imagine the hostility that greeted the suggestionthat participants or experimental units should be randomlyTHREE BUILDING BLOCK DESIGNSassigned to treatment levels. Before Fishers work, mostresearchers used systematic schemes, not subject to the lawsCompletely Randomized Designof chance, to assign participants. According to Fisher, ran-dom assignment has several purposes. It helps to distribute One of the simplest experimental designs is the randomizationthe idiosyncratic characteristics of participants over the treat- and analysis plan that is used with a t statistic for independentment levels so that they do not selectively bias the outcome of samples. Consider an experiment to compare the effectivenessthe experiment. Also, random assignment permits the com-of two diets for obese teenagers. The independent variable isputation of an unbiased estimate of error effectsthose the two kinds of diets; the dependent variable is the amount ofeffects not attributable to the manipulation of the independent weight loss two months after going on a diet. For notationalvariableand it helps to ensure that the error effects areconvenience, the two diets are called treatment A. The levelsstatistically independent.of treatment A corresponding to the specic diets are denoted 23. Three Building Block Designs5by the lowercase letter a and a subscript: a1 denotes one dietand a2 denotes the other. A particular but unspecied level oftreatment A is denoted by aj, where j ranges over the values 1and 2. The amount of weight loss in pounds 2 months afterparticipant i went on diet j is denoted by Yij. The null and alternative hypotheses for the weight-lossexperiment are, respectively, H0: 1 2 = 0 H1: 1 2 = 0 ,where 1 and 2 denote the mean weight loss of the respec-tive populations. Assume that 30 girls who want to loseweight are available to participate in the experiment. Theresearcher assigns n = 15 girls to each of the p = 2 diets so Figure 1.2 Layout for a completely randomized design (CR-3 design).that each of the (np)!/(n!) p = 155,117,520 possible assign- Forty-ve girls are randomly assigned to three levels of treatment A with thements has the same probability. This is accomplished byrestriction that 15 girls are assigned to each level. The mean weight loss innumbering the girls from 1 to 30 and drawing numbers frompounds for the girls in treatment levels a1, a2, and a3 is denoted by Y 1 , Y 2 , and Y 3 , respectively.a random numbers table. The rst 15 numbers drawn between1 and 30 are assigned to treatment level a1; the remaining 15numbers are assigned to a2. The layout for this experiment isthree diets. The null and alternative hypotheses for theshown in Figure 1.1. The girls who were assigned to treat- experiment are, respectively,ment level a1 are called Group1; those assigned to treatmentlevel a2 are called Group2. The mean weight losses of the twoH0: 1 = 2 = 3groups of girls are denoted by Y 1 and Y 2 . H1: j = j for some j and j .The t independent-samples design involves randomlyassigning participants to two levels of a treatment. A com-Assume that 45 girls who want to lose weight are available topletely randomized design, which is described next, extendsparticipate in the experiment. The girls are randomly as-this design strategy to two or more treatment levels. The com- signed to the three diets with the restriction that 15 girls arepletely randomized design is denoted by the letters CR-p,assigned to each diet. The layout for the experiment is shownwhere CR stands for completely randomized and p is the in Figure 1.2. A comparison of the layout in this gure withnumber of levels of the treatment. that in Figure 1.1 for a t independent-samples design revealsAgain, consider the weight-loss experiment and suppose that they are the same except that the completely randomizedthat the researcher wants to evaluate the effectiveness of design has three treatment levels. The t independent-samples design can be thought of as a special case of a completely randomized design. When p is equal to two, the layouts and randomization plans for the designs are identical. Thus far I have identied the null hypothesis that the researcher wants to test, 1 = 2 = 3 , and described the manner in which the participants are assigned to the three treatment levels. In the following paragraphs I discuss the com- posite nature of an observation, describe the classical model equation for a CR-p design, and examine the meaning of the terms treatment effect and error effect. An observation, which is a measure of the dependent vari- able, can be thought of as a composite that reects the effects of the (a) independent variable, (b) individual charac-Figure 1.1 Layout for a t independent-samples design. Thirty girls are ran-teristics of the participant or experimental unit, (c) chancedomly assigned to two levels of treatment A with the restriction that 15 girlsare assigned to each level. The mean weight loss in pounds for the girls inuctuations in the participants performance, (d) measure-treatment levels a1 and a2 is denoted by Y 1 and Y 2 , respectively. ment and recording errors that occur during data collection, 24. 6 Experimental Designand (e) any other nuisance variables such as environmentallevel are different because the error effects, i( j)s, for theconditions that have not been controlled. Consider the weight observations are different. Recall that error effects reect idio-loss of the fth participant in treatment level a2. Suppose thatsyncratic characteristics of the participantsthose character-two months after beginning the diet this participant has lost istics that differ from one participant to anotherand any13 pounds (Y52 = 13). What factors have affected the value of other variables that have not been controlled. Researchers at-Y52? One factor is the effectiveness of the diet. Other factors tempt to minimize the size of error effects by holding sourcesare her weight prior to starting the diet, the degree to whichof variation that might contribute to the error effects constantshe stayed on the diet, and the amount she exercised during and by the judicial choice of an experimental design. Designsthe two-month trial, to mention only a few. In summary, Y52 isthat are described next permit a researcher to isolate and re-a composite that reects (a) the effects of treatment level a2, move some sources of variation that would ordinarily be in-(b) effects unique to the participant, (c) effects attributable tocluded in the error effects.chance uctuations in the participants behavior, (d) errors inmeasuring and recording the participants weight loss, and(e) any other effects that have not been controlled. Our con- Randomized Block Designjectures about Y52 or any of the other 44 observations can beThe two designs just described use independent samples. Twoexpressed more formally by a model equation. The classicalsamples are independent if, for example, a researcher ran-model equation for the weight-loss experiment isdomly samples from two populations or randomly assigns par-ticipants to p groups. Dependent samples, on the other hand,Yi j = + j + i( j) (i = 1, . . . , n; j = 1, . . . , p), can be obtained by any of the following procedures.where 1. Observe each participant under each treatment level in the experimentthat is, obtain repeated measures on theYi j is the weight loss for participant i in treatment participants. level aj.2. Form sets of participants who are similar with respect tois the grand mean of the three weight-loss popula-a variable that is correlated with the dependent variable. tion means. This procedure is called participant matching.j is the treatment effect for population j and is equal to 3. Obtain sets of identical twins or littermates in which case j . It reects the effects of diet aj. the participants have similar genetic characteristics.i( j) is the within-groups error effect associated with Yi j 4. Obtain participants who are matched by mutual selection, and is equal to Yi j j . It reects all effects for example, husband and wife pairs or business partners. not attributable to treatment level aj. The notation i( j) indicates that the ith participant appears only in In the behavioral and social sciences, the participants are treatment level j. Participant i is said to be nestedoften people whose aptitudes and experiences differ markedly. within the jth treatment level. Nesting is discussed Individual differences are inevitable, but it is often possible in the section titled Hierarchical Designs.to isolate or partition out a portion of these effects so thatthey do not appear in estimates of the error effects. One designAccording to the equation for this completely randomizedfor accomplishing this is the design used with a t statistic fordesign, each observation is the sum of three parameters dependent samples. As the name suggests, the design uses, j , and i( j) . The values of the parameters in the equationdependent samples. A t dependent-samples design also uses aare unknown but can be estimated from sample data.more complex randomization and analysis plan than does a t The meanings of the terms grand mean, , and treatment independent-samples design. However, the added complexityeffect, j , in the model equation seem fairly clear; the mean-is often accompanied by greater powera point that I will de-ing of error effect, i( j) , requires a bit more explanation. Whyvelop later in connection with a randomized block design.do observations, Yi j s, in the same treatment level vary fromLets reconsider the weight-loss experiment. It is reason-one participant to the next? This variation must be due to dif- able to assume that ease of losing weight is related to theferences among the participants and to other uncontrolled amount by which a girl is overweight. The design of the exper-variables because the parameters and j in the model equa- iment can be improved by isolating this nuisance variable.tion are constants for all participants in the same treatment Suppose that instead of randomly assigning 30 participants tolevel. To put it another way, observations in the same treatmentthe treatment levels, the researcher formed pairs of participants 25. Three Building Block Designs 7 participants who have been exposed to only one treatment level. Some writers reserve the designation randomized block design for this latter case. They refer to a design with repeated measurements in which the order of administration of the treatment levels is randomized independently for each participant as a subjects-by-treatments design. A design with repeated measurements in which the order of administration of the treatment levels is the same for all participants is referred to as a subject-by-trials design. I use the designationFigure 1.3 Layout for a t dependent-samples design. Each block containsrandomized block design for all three cases.two girls who are overweight by about the same amount. The two girls in aOf the four ways of obtaining dependent samples, the useblock are randomly assigned to the treatment levels. The mean weight loss in of repeated measures on the participants typically results inpounds for the girls in treatment levels a1 and a2 is denoted by Y 1 and Y 2 ,respectively.the greatest homogeneity within the blocks. However, if re- peated measures are used, the effects of one treatment level should dissipate before the participant is observed under an-so that prior to going on a diet the participants in each pair are other treatment level. Otherwise the subsequent observationsoverweight by about the same amount. The participants in eachwill reect the cumulative effects of the preceding treatmentpair constitute a block or set of matched participants. A simple levels. There is no such restriction, of course, if carryover ef-way to form blocks of matched participants is to rank them fects such as learning or fatigue are the researchers principalfrom least to most overweight. The participants ranked 1 and 2 interest. If blocks are composed of identical twins or litter-are assigned to block one, those ranked 3 and 4 are assigned tomates, it is assumed that the performance of participants hav-block two, and so on. In this example, 15 blocks of dependenting identical or similar heredities will be more homogeneoussamples can be formed from the 30 participants. After all of the than the performance of participants having dissimilar hered-blocks have been formed, the two participants in each blockities. If blocks are composed of participants who are matchedare randomly assigned to the two diets. The layout for this ex-by mutual selection (e.g., husband and wife pairs or businessperiment is shown in Figure 1.3. If the researchers hunch ispartners), a researcher should ascertain that the participantscorrect that ease in losing weight is related to the amount by in a block are in fact more homogeneous with respect to thewhich a girl is overweight, this design should result in a moredependent variable than are unmatched participants. A hus-powerful test of the null hypothesis, 1 2 = 0, than wouldband and wife often have similar political attitudes; the cou-a t test for independent samples. As we will see, the increasedple is less likely to have similar mechanical aptitudes.power results from isolating the nuisance variable (the amount Suppose that in the weight-loss experiment the researcherby which the girls are overweight) so that it does not appear in wants to evaluate the effectiveness of three diets, denotedthe estimate of the error effects. by a1, a2, and a3. The researcher suspects that ease of losingEarlier we saw that the layout and randomization proce-weight is related to the amount by which a girl is overweight.dures for a t independent-samples design and a completelyIf a sample of 45 girls is available, the blocking procedurerandomized design are the same except that a completely ran- described in connection with a t dependent-samples designdomized design can have more than two treatment levels.can be used to form 15 blocks of participants. The three par-The same comparison can be drawn between a t dependent-ticipants in a block are matched with respect to the nuisancesamples design and a randomized block design. A random-variable, the amount by which a girl is overweight. The lay-ized block design is denoted by the letters RB-p, where RB out for this experiment is shown in Figure 1.4. A comparisonstands for randomized block and p is the number of levelsof the layout in this gure with that in Figure 1.3 for a tof the treatment. The four procedures for obtaining depen- dependent-samples design reveals that they are the same ex-dent samples that were described earlier can be used to form cept that the randomized block design has p = 3 treatmentthe blocks in a randomized block design. The procedure thatlevels. When p = 2, the layouts and randomization plans foris used does not affect the computation of signicance tests,the designs are identical. In this and later examples, I assumebut the procedure does affect the interpretation of the results. that all of the treatment levels and blocks of interest are rep-The results of an experiment with repeated measures general- resented in the experiment. In other words, the treatment lev-ize to a population of participants who have been exposed to els and blocks represent xed effects. A discussion of the caseall of the treatment levels. However, the results of an experi-in which either the treatment levels or blocks or both are ran-ment with matched participants generalize to a population of domly sampled from a population of levels, the mixed and 26. 8Experimental Designi jis the residual error effect associated with Yi j and isequal to Yi j j i . It reects all effects notattributable to treatment level aj and Blocki. According to the model equation for this randomized block design, each observation is the sum of four parameters: , j , i , and i j . A residual error effect is that portion of an observation that remains after the grand mean, treatment effect, and block effect have been subtracted from it; that is, i j = Yi j j i . The sum of the squared errorFigure 1.4 Layout for a randomized block design (RB-3 design). Eacheffects for this randomized block design,block contains three girls who are overweight by about the same amount.The three girls in a block are randomly assigned to the treatment levels. Thei2j = (Yi j j i )2 ,mean weight loss in pounds for the girls in treatment levels a1, a2, and a3 isdenoted by Y 1 , Y 2 , and Y 3 , respectively. The mean weight loss for thegirls in Block1, Block2, . . . , Block15 is denoted by Y 1 , Y 2 , . . . , Y 15 , will be smaller than the sum for the completely randomizedrespectively.design,i( j) = 2 (Yi j j )2 ,random effects cases, is beyond the scope of this chapter. Thereader is referred to Kirk (1995, pp. 256257, 265268). if i2 is not equal to zero for one or more blocks. This idea is A randomized block design enables a researcher to test illustrated in Figure 1.5, where the total sum of squares andtwo null hypotheses. degrees of freedom for the two designs are partitioned. The F H0: 1 = 2 = 3 statistic that is used to test the null hypothesis can be thought (Treatment population means are equal.) of as a ratio of error and treatment effects, H0: 1 = 2 = = 15 f (error effects) + f (treatment effects) F= (Block population means are equal.) f (error effects)The second hypothesis, which is usually of little interest,where f ( ) denotes a function of the effects in parentheses. Itstates that the population weight-loss means for the 15 levels is apparent from an examination of this ratio that the smallerof the nuisance variable are equal. The researcher expects a the sum of the squared error effects, the larger the F statistictest of this null hypothesis to be signicant. If the nuisance and, hence, the greater the probability of rejecting a false nullvariable represented by the blocks does not account for an ap-preciable proportion of the total variation in the experiment,little has been gained by isolating the effects of the variable.Before exploring this point, I describe the model equation foran RB-p design.The classical model equation for the weight-loss experi- SSWGment is p(n 1) 42Yi j = + j + i + i j (i = 1, . . . , n; j = 1, . . . , p),whereSSRES Yi j is the weight loss for the participant in Block i and (n 1)(p 1) 28treatment level aj. Figure 1.5 Partition of the total sum of squares (SSTOTAL) and degrees of is the grand mean of the three weight-loss popula- freedom (np 1 = 44) for CR-3 and RB-3 designs. The treatment andtion means.within-groups sums of squares are denoted by, respectively, SSA and SSWG. The block and residual sums of squares are denoted by, respectively, SSBL j is the treatment effect for population j and is equal to and SSRES. The shaded rectangles indicate the sums of squares that are used j . It reects the effect of diet aj.to compute the error variance for each design: MSWG = SSWG/ p(n 1) i is the block effect for population i and is equal to and MSRES = SSRES/(n 1)( p 1). If the nuisance variable (SSBL) in the randomized block design accounts for an appreciable portion of the total sumi . It reects the effect of the nuisance variableof squares, the design will have a smaller error variance and, hence, greaterin Blocki. power than the completely randomized design. 27. Three Building Block Designs 9hypothesis. Thus, by isolating a nuisance variable that ac-counts for an appreciable portion of the total variation in arandomized block design, a researcher is rewarded with amore powerful test of a false null hypothesis. As we have seen, blocking with respect to the nuisancevariable (the amount by which the girls are overweight)enables the researcher to isolate this variable and remove itfrom the error effects. But what if the nuisance variableFigure 1.6 Three-by-three Latin square, where aj denotes one of thedoesnt account for any of the variation in the experiment? In j = 1, . . . , p levels of treatment A; bk denotes one of the k = 1, . . . , p levels of nuisance variable B; and cl denotes one of the l = 1, . . . , p levels of nui-other words, what if all of the block effects in the experimentsance variable C. Each level of treatment A appears once in each row andare equal to zero? In this unlikely case, the sum of the squared once in each column as required for a Latin square.error effects for the randomized block and completely ran-domized designs will be equal. In this case, the randomized Latin square: b1 is less than 15 pounds, b2 is 15 to 25 pounds,block design will be less powerful than the completely ran- and b3 is more than 25 pounds. The advantage of being able todomized design because its error variance, the denominator isolate two nuisance variables comes at a price. The ran-of the F statistic, has n 1 fewer degrees of freedom than domization procedures for a Latin square design are morethe error variance for the completely randomized design. It complex than those for a randomized block design. Also, theshould be obvious that the nuisance variable should be se- number of rows and columns of a Latin square must eachlected with care. The larger the correlation between the nui- equal the number of treatment levels, which is three in the ex-sance variable and the dependent variable, the more likely it ample. This requirement can be very restrictive. For example,is that the block effects will account for an appreciable it was necessary to restrict the continuous variable of theproportion of the total variation in the experiment. amount by which girls are overweight to only three levels. The layout of the LS-3 design is shown in Figure 1.7.Latin Square DesignThe Latin square design described in this section derives itsname from an ancient puzzle that was concerned with thenumber of different ways that Latin letters can be arranged ina square matrix so that each letter appears once in each rowand once in each column. An example of a 3 3 Latin squareis shown in Figure 1.6. In this gure I have used the letter awith subscripts in place of Latin letters. The Latin square de-sign is denoted by the letters LS-p, where LS stands forLatin square and p is the number of levels of the treatment.A Latin square design enables a researcher to isolate the ef-fects of not one but two nuisance variables. The levels of onenuisance variable are assigned to the rows of the square; thelevels of the other nuisance variable are assigned to thecolumns. The levels of the treatment are assigned to the cellsof the square.Lets return to the weight-loss experiment. With a Latinsquare design the researcher can isolate the effects of theamount by which girls are overweight and the effects of a sec-ond nuisance variable, for example, genetic predisposition tobe overweight. A rough measure of the second nuisance vari- Figure 1.7 Layout for a Latin square design (LS-3 design) that is based onable can be obtained by asking a girls parents whether they the Latin square in Figure 1.6. Treatment A represents three kinds of diets;were overweight as teenagers: c1 denotes neither parent over-nuisance variable B represents amount by which the girls are overweight;weight, c2 denotes one parent overweight, and c3 denotes bothand nuisance variable C represents genetic predisposition to be overweight.parents overweight. This nuisance variable can be assigned toThe girls in Group1, for example, received diet a1, were less than fteen pounds overweight (b1), and neither parent had been overweight as athe columns of the Latin square. Three levels of the amount by teenager (c1). The mean weight loss in pounds for the girls in the nine groupswhich girls are overweight can be assigned to the rows of theis denoted by Y 111 , Y 123 , . . . , Y 331 . 28. 10Experimental Design The design in Figure 1.7 enables the researcher to testif the combined effects of 2 , l2 , andk2 arejklthree null hypotheses:greater thani2 . The benets of isolating two nuisancevariables are a smaller error variance and increased power. H0: 1 = 2 = 3 Thus far I have described three of the simplest experimen-(Treatment population means are equal.) tal designs: the completely randomized design, randomized H0: 1 = 2 = 3 block design, and Latin square design. The three designs are(Row population means are equal.) called building block designs because complex experimental H0: 1 = 2 = 3 designs can be constructed by combining two or more of these(Column population means are equal.)simple designs (Kirk, 1995, p. 40). Furthermore, the random-ization procedures, data analysis, and model assumptions forThe rst hypothesis states that the population means for thecomplex designs represent extensions of those for the threethree diets are equal. The second and third hypotheses make building block designs. The three designs provide the organi-similar assertions about the population means for the two zational structure for the design nomenclature and classica-nuisance variables. Tests of these nuisance variables are ex- tion scheme that is described next.pected to be signicant. As discussed earlier, if the nuisancevariables do not account for an appreciable proportion of thetotal variation in the experiment, little has been gained by iso- CLASSIFICATION OF EXPERIMENTAL DESIGNSlating the effects of the variables.The classical model equation for this version of theA classication scheme for experimental designs is given inweight-loss experiment is Table 1.1. The designs in the category systematic designs donot use random assignment of participants or experimental Yi jkl = + j + k + l + jkl + i( jkl)units and are of historical interest only. According to Leonard(i = 1, . . . , n; j = 1, . . . , p; k = 1, . . . , p; l = 1, . . . , p), and Clark (1939), agricultural eld research employing sys-tematic designs on a practical scale dates back to 1834. Overwhere the last 80 years systematic designs have fallen into disuse be-cause designs employing random assignment are more likely Yi jklis the weight loss for the ith participant in treat-to provide valid estimates of treatment and error effects and ment level aj, row bk, and column cl.can be analyzed using the powerful tools of statistical infer- jis the treatment effect for population j and is equalence such as analysis of variance. Experimental designs using to j . It reects the effect of diet aj.random assignment are called randomized designs. The ran- kis the row effect for population k and is equaldomized designs in Table 1.1 are subdivided into categories to k . It reects the effect of nuisance vari- based on (a) the number of treatments, (b) whether participants able bk. are assigned to relatively homogeneous blocks prior to random lis the column effect for population l and is equal assignment, (c) presence or absence of confounding, (d) use of to l . It reects the effects of nuisance vari-crossed or nested treatments, and (e) use of a covariate. able cl. The letters p and q in the abbreviated designations denote jkl is the residual effect that is equal to jkl j the number of levels of treatments A and B, respectively. If a k l + 2.design includes a third and fourth treatment, say treatments C i( jkl)is the within-cell error effect associated with Yijkland D, the number of their levels is denoted by r and t, and is equal to Yi jkl j k l jkl . respectively. In general, the designation for designs with twoor more treatments includes the letters CR, RB, or LS toAccording to the model equation for this Latin square design, indicate the building block design. The letter F or H is addedeach observation is the sum of six parameters: , j , k ,to the designation to indicate that the design is, respectively, al , jkl , and i( jkl) . The sum of the squared within-cell errorfactorial design or a hierarchical design. For example, the F ineffects for the Latin square design,the designation CRF-pq indicates that it is a factorial design;the CR and pq indicate that the design was constructed byi( jkl) = 2(Yi jkl j k l jkl )2 , combining two completely randomized designs with p and qtreatment levels. The letters CF, PF, FF, and AC are added towill be smaller than the sum for the randomized block design, the designation if the design is, respectively, a confoundedfactorial design, partially confounded factorial design, frac- i2j =(Yi j j i )2 ,tional factorial design, or analysis of covariance design. 29. Factorial Designs 11TABLE 1.1 Classication of Experimental Designs AbbreviatedAbbreviatedExperimental DesignDesignationa Experimental Design DesignationaI. Systematic Designs (selected examples).b. Randomized block completely confounded RBCF-pk 1. Beavans chessboard design.factorial design. 2. Beavans half-drill strip design. c. Randomized block partially confoundedRBPF-pk 3. Diagonal square design.factorial design. 4. Knut Vik square design.4. Designs with treatment-interaction confounding. II. Randomized Designs With One Treatment. a. Completely randomized fractional CRFF-pki A. Experimental units randomly assigned tofactorial design.treatment levels. b. Graeco-Latin square fractional factorial design.GLSFF-pk1. Completely randomized design.CR-pc. Latin square fractional factorial design.LSFF-pk B. Experimental units assigned to relatively d. Randomized block fractional factorial design.RBFF-pkihomogeneous blocks or groups prior to B. Hierarchical designs: designs in which one orrandom assignment. more treatments are nested. 1. Balanced incomplete block design. BIB-p1. Designs with complete nesting. 2. Cross-over design. CO-p a. Completely randomized hierarchical design.CRH-pq(A) 3. Generalized randomized block design. GRB-pb. Randomized block hierarchical design. RBH-pq(A) 4. Graeco-Latin square design. GLS-p2. Designs with partial nesting. 5. Hyper-Graeco-Latin square design. HGLS-pa. Completely randomized partialCRPH-pq(A)r 6. Latin square design. LS-phierarchical design. 7. Lattice balanced incomplete block design.LBIB-p b. Randomized block partial hierarchical design. RBPH-pq(A)r 8. Lattice partially balanced incomplete LPBIB-p c. Split-plot partial hierarchical design.SPH-pqr(B)block design. IV. Randomized Designs With One or More Covariates. 9. Lattice unbalanced incomplete block design. LUBIB-p A. Designs that include a covariate have10. Partially balanced incomplete block design.PBIB-pthe letters AC added to the abbreviated11. Randomized block design.RB-p designation as in the following examples.12. Youden square design.YBIB-p1. Completely randomized analysis of covarianceCRAC-pIII. Randomized Designs With Two or More Treatments.design. A. Factorial designs: designs in which all treatments 2. Completely randomized factorial analysisCRFAC-pqare crossed.of covariance design.1. Designs without confounding.3. Latin square analysis of covariance design. LSAC-p a. Completely randomized factorial design. CRF-pq 4. Randomized block analysis of covariance design. RBAC-p b. Generalized randomized block factorial design. GRBF-pq 5. Split-plot factorial analysis of covariance design.SPFAC-pq c. Randomized block factorial design.RBF-pq V. Miscellaneous Designs (select examples).2. Design with group-treatment confounding. 1. Solomon four-group design. a. Split-plot factorial design.SPF- pq2. Interrupted time-series design.3. Designs with group-interaction confounding. a. Latin square confounded factorial design. LSCF-pkaThe abbreviated designations are discussed later.Three of these designs are described later. Because of spacechoose. Because of the wide variety of designs available, it islimitations, I cannot describe all of the designs in Table 1.1. important to identify them clearly in research reports. OneI will focus on those designs that are potentially the most often sees statements such as a two-treatment factorial de-useful in the behavioral and social sciences. sign was used. It should be evident that a more preciseIt is apparent from Table 1.1 that a wide array of designsdescription is required. This description could refer to 10 ofis available to researchers. Unfortunately, there is no univer- the 11 factorial designs in Table 1.1.sally accepted designation for the various designssomeThus far, the discussion has been limited to designs withdesigns have as many as ve different names. For example, one treatment and one or two nuisance variables. In the fol-the completely randomized design has been called a one-waylowing sections I describe designs with two or more treat-classication design, single-factor design, randomized groupments that are constructed by combining several buildingdesign, simple randomized design, and single variable exper-block designs.iment. Also, a variety of design classication schemes havebeen proposed. The classication scheme in Table 1.1 owes FACTORIAL DESIGNSmuch to Cochran and Cox (1957, chaps. 413) and Federer(1955, pp. 1112).Completely Randomized Factorial DesignA quick perusal of Table 1.1 reveals why researcherssometimes have difculty selecting an appropriate experi- Factorial designs differ from those described previously inmental designthere are a lot of designs from which tothat two or more treatments can be evaluated simultaneously 30. 12Experimental Designin an experiment. The simplest factorial design from thestandpoint of randomization, data analysis, and model as-sumptions is based on a completely randomized design and,hence, is called a completely randomized factorial design. Atwo-treatment completely randomized factorial design is de-noted by the letters CRF-pq, where p and q denote the num-ber of levels, respectively, of treatments A and B.In the weight-loss experiment, a researcher might be inter-ested in knowing also whether walking on a treadmill for20 minutes a day would contribute to losing weight, as wellas whether the difference between the effects of walking ornot walking on the treadmill would be the same for each ofthe three diets. To answer these additional questions, a re-searcher can use a two-treatment completely randomized fac-torial design. Let treatment A consist of the three diets (a1, a2,and a3) and treatment B consist of no exercise on the tread-mill (b1) and exercise for 20 minutes a day on the treadmill(b2). This design is a CRF-32 design, where 3 is the numberof levels of treatment A and 2 is the number of levels of treat-ment B. The layout for the design is obtained by combiningthe treatment levels of a CR-3 design with those of a CR-2design so that each treatment level of the CR-3 design ap-Figure 1.8 Layout for a two-treatment completely randomized factorialpears once with each level of the CR-2 design and vice versa. design (CRF-32 design). Thirty girls are randomly assigned to six combina-tions of treatments A and B with the restriction that ve girls are assigned toThe resulting design has 3 2 = 6 treatment combinations each combination. The mean weight loss in pounds for girls in the six groupsas follows: a1b1, a1b2, a2b1, a2b2, a3b1, a3b2. When treatmentis denoted by Y 11 , Y 12 , . . . , Y 32 .levels are combined in this way, the treatments are said to becrossed. The use of crossed treatments is a characteristic ofi( jk)is the within-cell error effect associated with Yi jkall factorial designs. The layout of the design with 30 girls and is equal to Yi jk j k () jk . Itrandomly assigned to the six treatment combinations isreects all effects not attributable to treatmentshown in Figure 1.8.level aj, treatment level bk, and the interaction of ajThe classical model equation for the weight-loss experi-and bk.ment isThe CRF-32 design enables a researcher to test three null Yi jk = + j + k + () jk + i( jk) hypotheses:(i = 1, . . . , n; j = 1, . . . , p; k = 1, . . . , q), H0: 1 = 2 = 3 (Treatment A population means are equal.)whereH0: 1 = 2 Yi jk is the weight loss for participant i in treatment (Treatment B population means are equal.) combination aj bk. H0: jk jk j k + j k = 0 for all j and k is the grand mean of the six weight-loss popula- (All A B interaction effects equal zero.) tion means.The last hypothesis is unique to factorial designs. It states that jis the treatment effect for population aj and is the joint effects (interaction) of treatments A and B are equal equal to j . It reects the effect of diet aj. to zero for all combinations of the two treatments. Two treat- kis the treatment effect for population bk and is ments are said to interact if any difference in the dependent equal to k . It reects the effects of exercise variable for one treatment is different at two or more levels of condition bk.the other treatment. () jk is the interaction effect for populations aj and bk Thirty girls are available to participate in the weight-loss ex- and is equal to jk j k . Interaction periment and have been randomly assigned to the six treatment effects are discussed later. combinations with the restriction that ve girls are assigned to 31. Factorial Designs13TABLE 1.2 Weight-Loss Data for the Diet (aj) and ExerciseTABLE 1.4Analysis of Variance for the Weight-Loss DataConditions (bk) SourceSSdf MSFpa1b1a1b2 a2b1 a2b2 a3b1 a3b2 Treatment A (Diet) 131.66672 65.83344.25 .026 7 79 101513 Treatment B (Exercise)67.50001 67.50004.35 .0481314451016 AB 35.00002 17.50001.13 .340 911771220 Within cell372.0000 24 15.5000 5 4 14 15 519 1 9 11 13 812 Total606.1667 29each combination. The data, weight loss for each girl, are given the hypotheses are false. As John Tukey (1991) wrote, thein Table 1.2. A descriptive summary of the datasample effects of A and B are always differentin some decimalmeans and standard deviationsis given in Table 1.3. placefor any A and B. Thus asking Are the effects differ- An examination of Table 1.3 suggests that diet a3 resultedent? is foolish (p. 100). Furthermore, rejection of a nullin more weight loss than did the other diets and 20 minutes ahypothesis tells us nothing about the size of the treatmentday on the treadmill was benecial. The analysis of variance effects or whether they are important or large enough to befor the weight-loss data is summarized in Table 1.4, which usefulthat is, their practical signicance. In spite of numer-shows that the null hypotheses for treatments A and B can be ous criticisms of null hypothesis signicance testing, re-rejected. We know that at least one contrast or difference searchers continue to focus on null hypotheses and p values.among the diet population means is not equal to zero. Also,The focus should be on the data and on what the data tell thefrom Tables 1.3 and 1.4 we know that 20 minutes a day on researcher about the scientic hypothesis. This is not a newthe treadmill resulted in greater weight loss than did the idea. It was originally touched on by Karl Pearson in 1901no-exercise condition. The A B interaction test is not and more explicitly by Fisher in 1925. Fisher (1925) pro-signicant. When two treatments interact, a graph in which posed t