the research imagination€¦ · the role of the literature in research design types of information...

The Research Imagination

Paul S. Gray, BOSTON COLLEGE

John B. Williamson, BOSTON COLLEGE

David A. Karp, BOSTON COLLEGE

John R. Dalphin, MERRIMACK COLLEGE

with the collaboration of

Karen Bettez Halnon and James Carritte

H I CAMBRIDGEUNIVERSITY PRESS

CONTENTS

Preface page xix

1 RESEARCH PROCESS 1

INTRODUCTION

What Is Social Research?Data Collection and Analysis

ADDING TO KNOWLEDGE

Sources of Data - l

Social SignificanceWhat Is Methodology?

THE SCIENTIFIC METHODThe Research Cycle - TheoryObservation and TestingConclusions and FindingsA Short History of Social Science

Common Sense and Social ScienceWhy common sense fails us, 8 • Distance, 8 • Familiarity, notunderstanding, 8 • Emotions, 9

Principles of Scientific Investigation

Objectivity, 10 • Replication and reliability, 11 •Precision, 12 • Validity, 12

The Critical PerspectiveRemaining Questions

SUMMARYKEY TERMS

EXERCISESSUGGESTED READINGS

REFERENCES

THEORY AND METHOD

11

222

334

45567

10

13131414151515

17

INTRODUCTION 17THEORY AND SOCIAL RESEARCH 18

What Theory Looks Like 18

Taxonomies, 18 • Models, 20 • Typologies, 21 •Paradigms, 22

Description and Explanation 23The Verification and Discovery of Theory 23

Deduction, 23 • Induction, 24

vii

VIII

THE DEVELOPMENT OF RESEARCH IDEAS

Curiosity

Pure and Applied Research

Public Sociology

Already Existing Theory

Training and Experience

INTELLECTUAL CRAFTSMANSHIP

Learning the Craft

Using Research Imagination

SUMMARY

KEY TERMS

EXERCISES

SUGGESTED READINGS

REFERENCES

RESEARCH DESIGN i

INTRODUCTION

The Research Cookbook

Menus and recipes, 34 • Budgeting, 35 • From kitchen to

table, 36

THE COMPONENTS OF RESEARCH

Selecting a Topic

Conceptualizing a Topic

Concepts and Variables

The Role of the Literature in Research Design

Types of Information

Quantitative and Qualitative Techniques ,

Strategies of Data Collection

Spending Time and Resources Wisely

Collecting Data

Analyzing and Processing Data

Making Inferences and Recommendations

PUTTING IT ALL TOGETHER

The Final Report

The Ups and Downs of Research

An Example: A Study of Undergraduate Life

How was the topic selected?, 51 • How was the topic

conceptualized?, 51 • How was a strategy for data collection

selected?, 52 • Issues of data collection, 53

SUMMARY

KEY TERMS

EXERCISES

SUGGESTED READINGS AND SOURCES

REFERENCES

MEASUREMENT

Contents

2525252626282929303031313131

33

3434

3737373839

4042

43464748494949

5051

545455

5556

57

INTRODUCTION 58

LEVELS OF MEASUREMENT 59

Contents ix

Nominal Measures 59

Ordinal Measures 59

Interval Measures 59

Ratio Measures 60

Precision and Accuracy 60

OPERATIONALIZATION 61

Example: Assessing Excellence in Education 61

Example: Assessing Happiness 62

Behavioral Indicators 63

EVALUATING RELIABILITY AND VALIDITY 63

The Criterion of Reliability 64

The Criterion of Validity 64

SOURCES OF MEASUREMENT ERROR 69

Random Error ' 69

Systematic Error . 69

Situational Error ' ' 69

Problems of Face Validity 70

Problems of Reliability in the Research Instrument 70

Researcher Bias 71

Respondent Bias 71

Lying as a Source of Error 71

Error Arising from Special Circumstances 72

Errors in Data Processing 72

THE SITUATIONAL NATURE OF VARIABLES 73

Measurement in the Physical and Social Sciences ' 73

Lack of Consensus in Measurement 74

IMPROVING THE QUALITY OF MEASUREMENT 75

Triangulation 75

Using a Variety of Methods 75

SUMMARY 76

KEYTERMS 77

EXERCISES 77

SUGGESTED READINGS 78

REFERENCES • 78

5 ETHICAL AND POLITICAL ISSUES 81

INTRODUCTION 82

OBJECTIVITY 82

The Strict Position 82

Criticism of Objectivity 83

FORMULATION OF THE RESEARCH QUESTION 84

Topic Selection - 84

Choice of Theory - 84

Funding 85

DATA COLLECTION 86

Impartiality 86

Confidentiality 87

Disguised Observation 87

Ethics and Experimentation

Informed Consent

Research on Children r-

ANALYSIS, INTERPRETATION, AND PRESENTATION OF RESULTS

The Fudging Effect

Appraisals and Characterizations

APPLICATION OF RESEARCH RESULTS

Co-optation or Potency?

Misuse of Information

SUMMARY

KEY TERMS

EXERCISES


REFERENCES

SAMPLING i

INTRODUCTION

Populations

SAMPLING PLANS

Nonp'robability Sampling

Accidental sampling, 104 • Quota sampling, 104 • Purposive

Contents

88909 1 •

9393949494

959596969798

101

102

103104104

sampling, 105

Probability Sampling 105

Random sampling, 105 • Probability theory, 106 • Systematic

sampling, 108 • Stratified sampling, 108 • Cluster

sampling, 110 • Multistage cluster sampling, 110 •

Combining Sampling Plans 111

Polling Research 112

PROBLEMS AND ISSUES IN SAMPLING 112

Sample Size 112

Confidence Levels 113

Nonsampling Error 113

Poor Sampling Frame 114

Nonresponse 114

Bias in Panels - 114

Internet Polling - 115

Bias in Respondent Selection 115

SAMPLING IN QUALITATIVE RESEARCH 115

Time Sampling 116

Place Sampling 116

Event Sampling 116

Status, Role, and Relationship Sampling 117

Sampling and the Creation of Theory 117

Snowball Sampling 117

SUMMARY 118

KEYTERMS 118

EXERCISES 119

Contents xi

SUGGESTED READINGS AND SOURCES 119

REFERENCES 120i r -

7 SURVEY RESEARCH 121

INTRODUCTION 1'22

SURVEY DESIGN 123

Formulating Objectives and Hypotheses 123

Choosing a Time Frame 124

Cross-sectional design, 124 • Longitudinal design, 125 •

Panel studies, 125

Planning a Sampling Strategy 126

Choosing a Data-Gathering Technique 127

Self-administered questionnaires, 127 • Face-to-face

interviews, 128 • Telephone surveys, 129 \

Formulating Questionnaire Items 130

Question content, 130 • Multiple indicators, 132 • Structured

versus unstructured questions, 132 • Question wording, 134

Constructing the Questionnaire 135

Introduction, 135' • Instructions, 136 • Sequence of

questions, 136 • Layout and response format, 137 •

Pretesting, 138

SURVEY EXECUTION 139

The Self-Administered Survey 139

The Interview Survey 139

Preparation, 140 • Developing rapport, 140 • Providing a

uniform stimulus, 140 • Probing, 141 • Recording

responses, 141

Preparing the Data for Processing 141

Precoding, 141 • Coding, 142 • Preparing a codebook, 142 •

Entering the data, 142

A Hypothetical Survey 143

DATA ANALYSIS 143

Code Checking and Cleaning 143

Statistical Analysis 144

Secondary Analysis . ' 144

Data banks, 145

THE STRENGTHS AND LIMITATIONS OF SURVEY RESEARCH 145

SUMMARY 146

KEY TERMS 147

EXERCISES 147


REFERENCES 149

8 INTENSIVE INTERVIEWING 151

INTRODUCTION 152

APPLICATIONS OF INTENSIVE INTERVIEWING 153

xii Contents

DISTINCTIVE CHARACTERISTICS OF INTENSIVE INTERVIEWS 155

Commitment and Reciprocity 156

Shared Meanings ' 157 .

Flexibility 159

Assimilation 159

Interviewing and Induction 160

Interviewing, Reliability, and Validity 160

CONDUCTING AN INTERVIEW 161

Degrees of Structure 161

Access to Respondents . 161

Preparation 163

Execution 163

Sensitive Topics: Emotionality 166

Guidelines for Questioning >. 166

Recording Information 166

Remaining in Control r 170

Avoiding Overrapport 170

Nonverbal Signs 171

LIMITATIONS OF INTENSIVE INTERVIEWING 171

CURRENT DEVELOPMENTS IN THE TECHNIQUE 173

Telephone Intensive Interviewing 173

Using E-mail 174

Using the World Wide Web 174

SUMMARY 175

KEY TERMS 176

EXERCISES 176


REFERENCES 177

9 OBSERVATIONAL FIELD RESEARCH 179

INTRODUCTION 180

The Chicago School and Beyond 180

Examples from the Literature 181

Promise and Problems 181

DOING FIELDWORK 183

Beginning the Research: A Question and a Research Site 183

Gaining Access 186

Taking a Role • 187

Some Practical Suggestions 190

Data Collection and Note Taking 191

Descriptions 192

Explanations 193

Leaving the Field 194

Formulating an Analysis 194

Conceptual Categories 195

Data Coding ' 197

From Analysis to Theory 198

Contents xiii

LIMITATIONS OF OBSERVATIONAL FIELD RESEARCHNEW DEVELOPMENTS IN QUALITATIVE FIELDWORK

Cyberspace as a Field SettingComputer-Assisted Qualitative Data Analysis

Qualitative Data and the Postmodern CritiqueSUMMARY

KEY TERMSEXERCISESSUGGESTED READINGSREFERENCES

FEMINIST METHODS

INTRODUCTION

Multiple Disciplines and MethodologiesPATRIARCHAL SCIENCEBASICS OF FEMINIST EPISTEMOLOGYFEMINIST STANDPOINT THEORY

"Outsider Within" StatusPatricia Hill Collins's Black Feminist ThoughtDorothy Smith: Women's Standpoint

Sandra Harding's Critique of Scientific ObjectivityFEMINIST METHODOLOGY

Grassroots SociologyMaking Women's Invisible Experiences VisibleA View from Below

Feminist Activism as Product of ResearchWebbed AccountsPutting Women Researchers in the ResearchCollaboration with Research SubjectsAcceptance of Alternative Sources of Women's Knowledge

ClaimsMinimizing the Harms of Social Research

ISSUES AND CRITICISMSAre Feminist Methods Exclusionary?Can Men Do Feminist Research?

Do Most Feminists Avoid Feminist Methods?Why Bother Using Feminist Methods?

EXAMPLES OF FEMINIST RESEARCHIncreasing Validity with Feminist MethodsExample: A Feminist Reinterpretation of the Bible

NEWER FEMINIST METHODSSUMMARY

KEY TERMSEXERCISES

SUGGESTED READINGSREFERENCES

198200200201'202202

203203204207

211

212213215216218218218220222224224

225226

226227227

227

228228228228229230230231231233233234235235

236237

xiv

1 1 HISTORICAL ANALYSIS

INTRODUCTION

HISTORY AND CULTURE

HISTORY AND THE GROWTH OF KNOWLEDGE

SOURCES OF HISTORICAL DATA

Primary and Secondary Sources

Using Data versus Generating Data

Generalizability

Postmodernism and Historical Analysis

USES OF HISTORY: SOME EXAMPLES

Analyzing Evolutionary Trends

Changes in Urban Life

The Historical Case Study

Personal Documents •

Life History Reports

The Use of Available Records '

Example: Changing Attitudes toward Abortion

Example: The Persistence of Inequality

HISTORICAL ANALYSIS IN PERSPECTIVE

SUMMARY

KEY TERMS

EXERCISES

SUGGESTED READINGS

REFERENCES

EXPERIMENTAL RESEARCH

INTRODUCTION

THE ELEMENTS OF TRUE EXPERIMENTATION

Independent and Dependent Variables

Experimental and Control Groups

Experimental Procedure

Establishing Causality

THREATS TO VALIDITY IN EXPERIMENTAL RESEARCH

Internal Validity

The Solomon Four-Group Design

External Validity

Modeling Effects

Sampling and Generalizability

A LABORATORY EXPERIMENT

FIELD EXPERIMENTATION

Quasi-Field Experiments

A Study of Group Culture

More Quasi-Field Experiments

DEMONSTRATION EXPERIMENTS

A FINAL WORD

SUMMARY

Contents

241

241243243245'

245246246247247248248250251252252254

255255256256257257259

263

264

265265265266267268268269

270271271272272275275

276277

278279

Contents xv

KEY TERMS 280

EXERCISES 280


REFERENCES 281

13 CONTENT ANALYSIS 283

INTRODUCTION

AN OVERVIEW OF THE METHOD

Objective Analysis

Systematic Analysis

Quantitative and Qualitative Analysis

Manifest and Latent Content

APPLICATIONS OF CONTENT ANALYSIS

Inferences to the Source of Communications \

Inferences to Populations: Communication Content

and Social Values I '

Evaluating the Effects of Communications

PERFORMING CONTENT ANALYSIS

Specification of the Research Problem

Sampling Items

Choosing the Unit of Analysis

Category Construction

COMPUTER-ASSISTED CONTENT ANALYSIS

THE TECHNIQUE IN PERSPECTIVE

SUMMARY

KEY TERMS

EXERCISES

SUGGESTED READINGS

REFERENCES

AGGREGATE DATA ANALYSIS

283284284

285285286286286

288290

291291292

293294296297

298299299299

301

305

INTRODUCTION 305

APPLICATIONS OF AGGREGATE DATA ANALYSIS 306

Using Census Materials to Study Race Relations 307

Estimated Rates and True Rates: The Case

of Crime Statistics 310

Development of Social Indicators 31.1

Forecasting • 313

FALLACIES IN THE INTERPRETATION OF AGGREGATE DATA 316

The Ecological Fallacy 317

The Atomistic Fallacy 318

SUMMARY ' 318

KEY TERMS 319

EXERCISES 319

SUGGESTED READINGS AND SOURCES 320

REFERENCES 322

XVI

16

COMPARATIVE RESEARCH METHODS

INTRODUCTION

A Brief History

WHY DO COMPARATIVE RESEARCH?

Testing and Qualifying Existing Theories

Testing Theory Cross-Culturally

Specifying the Conditions under Which Theory Applies

Discovering the Relationships among Macrolevel Variables

RELIABILITY AND VALIDITY ISSUES

Fieldwork

Ethnographic Data Files

Historical-Comparative Research

COMPARATIVE SURVEY RESEARCH

Conceptual and Measurement Equivalence

Comparative Sampling and Interviewing

Secondary Data Analysis '

THE NATION AS A UNIT OF AGGREGATION

Two Examples

Warning: Use with Care

OTHER COMPARATIVE TECHNIQUES

SUMMARY

KEY TERMS

EXERCISES


REFERENCES

EVALUATION RESEARCH

INTRODUCTION

An Example: Pedestrian Safety

The Social Significance of Evaluation Research

THE EVALUATOR'S ROLE

Formative Evaluation

Summative Evaluation

Resistance to the Evaluation: Outsiders and Insiders

EVALUATION RESEARCH PROCESS

Formulation of the Problem

Research Design

Sources of Data

Sampling

FOCUS GROUPS

MEASUREMENT AND VARIABLES

ONE-SHOT AND BEFORE-AND-AFTER STUDIES

EXPERIMENTATION AND EVALUATION

DIFFICULTIES IN IMPLEMENTING RESEARCH DESIGN

ACTION RESEARCH

OUTCOME MAPPING

Contents

325

326326 •

327327

329329330331331332333335335337337

338338339340341

342 •

342343

345

349

350350352

• 353

353354354

356356

358359360361362362

363364366366

Contents xvii

UTILIZATION OF RESULTS FOR DECISION MAKING 368

SUMMARY 369

KEY TERMS •' 370

EXERCISES 370


REFERENCES 372

17 INDEXES AND SCALES 375

INTRODUCTION

An Example: The Consumer Price Index

INDEX CONSTRUCTION

Item Selection

Face Validity

Unidimensionality

Achieving Conceptual Balance

Statistical Relationship among Items

INDEX SCORING

The Range and Numbering of Response Categories

Weighting of Index Items

Coping with Missing Data

INDEX VALIDATION

Internal Validation

External Validation

THE SCALING OF RESPONSES

Measuring Intensity and Response Patterns

Some Weil-Known Scaling Techniques

The Bogardus Social Distance Scale, 388 • Thurstone

scaling, 389 • Guttman scaling, 390

STANDARDIZATION OF COMPOSITE MEASURES

The Politics of Measurement

SUMMARY

KEY TERMS

EXERCISES


REFERENCES

BASIC STATISTICAL ANALYSIS

376377378378378379379379

381

381383384

385385386386387

388

392393394394395395

396

39718INTRODUCTION ' 397

UNIVARIATE ANALYSIS 398

Marginals 398

Central Tendency and Levels of Measurement 399

Measures of Variability 401

Grouping and Recoding Data 403

BIVARIATE ANALYSIS 404

Cross-Tabulation 405

Computing Percentages for Cross-Tabulations 405

xviii Contents

Measures of Association . 406Correlation, 407

Other Measures of Association 409

STATISTICAL CONTROL 410The Elaboration Paradigm 410

Explanation, 411 • Interpretation, 412 • Specification, 412 •Suppressor variables, 414

Partial Correlation 415SUMMARY 417KEYTERMS 418EXERCISES 418SUGGESTED READINGS' 419REFERENCES 419

19 MULTIVARIATE ANALYSIS AND STATISTICALSIGNIFICANCE jj 4 2 1

INTRODUCTION . 421MULTIVARIATE ANALYSIS 421

Regression Analysis 421

Path Analysis 424Other Multivariate Techniques 425

STATISTICAL INFERENCE 426Tests of Statistical Significance 426The Misuse of Tests of Significance 428

SUMMARY 429KEYTERMS 430EXERCISES 430SUGGESTED READINGS 431REFERENCES 432

EPILOGUE: THE VALUE AND LIMITS OF SOCIAL SCIENCEKNOWLEDGE 433

A CRITICAL PERSPECTIVE 433THE IMPORTANCE OF THE RULES OF INVESTIGATION 433SCIENCE AS A BLUEPRINT; IMAGINATION AS INSPIRATION 434

TOLERANCE FOR AMBIGUITY: LIMITS TO POSITIVISM? 434SOLVING SOCIAL PROBLEMS: DATA AND DEMOCRACY 436

Appendix A: A Precoded Questionnaire 439

Appendix B: Excerpt from a Codebook 442

Author Index 445

Subject Index 449