understanding race data from vehicle stops...dec 31, 2001 · safety. to all of these people, i...

Understanding Race Data from Vehicle Stops:A Stakeholder’s Guide

Lorie A. Fridell

This project, conducted by the Police Executive Research Forum, wassupported by Cooperative Agreement #2001-CK-WX-K046 by the U.S.Department of Justice Office of Community Oriented Policing Services(COPS). Points of view or opinions contained in this document are thoseof the author and do not necessarily represent the official position of theU.S. Department of Justice or the members of PERF.

Police Executive Research Forum, Washington, D.C. 20036

Published 2005Library of Congress Number 2005905945ISBN 1-878734-89-X

Cover art adapted from design by Marnie KenneyInterior design by David Williams, adapted from design by AutomatedGraphic Systems

Chapter 1:Introduction . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 2:The Benchmarking Challenge . . . . . . . . . . 5

Chapter 3:Getting Started . . . . . . . . . . . . . . . . . . . . . 21

Chapter 4:Data Analysis Guidelines forAll Benchmarking Methods. . . . . . . . . . . 29

Chapter 5:Methods for BenchmarkingStop Data . . . . . . . . . . . . . . . . . . . . . . . . . 35

Chapter 6:Guidelines for AnalyzingPoststop Activities by Police . . . . . . . . . . 55

Chapter 7:Drawing Conclusionsfrom the Results . . . . . . . . . . . . . . . . . . . . 65

Chapter 8:Using the Results for Reform. . . . . . . . . . 79

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Acknowledgements. . . . . . . . . . . . . . . . . . . . . vii

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

About the Author . . . . . . . . . . . . . . . . . . . . . . 93

About COPS . . . . . . . . . . . . . . . . . . . . . . . . . . 94

About PERF . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Contents

iv Understanding Race Data from Vehicle Stops: A Stakeholder’s Guide

v

Longstanding tensions between police andminority groups within the community re-ceived renewed attention in the late 1990swith charges that law enforcement engages in“racial profiling.” Although allegations of dis-parate treatment of minority citizens by policeare not new, the police response in the 21st

Century to these issues is superior to that wit-nessed in previous eras. Around the country,executives of law enforcement agencies(public safety directors, chiefs and sheriffs)are joining with concerned residents to ad-dress the important issues of racially biasedpolicing and the perceptions of its practice.

The Police Executive Research Forum(PERF) and the U.S. Department of Justice Of-fice of Community Oriented Policing Services(COPS Office) have partnered to provide re-sources to support these efforts. With COPSOffice funds, PERF first published RaciallyBiased Policing: A Principled Response, whichprovides law enforcement executives withguidance in implementing a comprehensiveresponse to these important issues. Chaptersaddress change efforts that can occur in therealms of policy, supervision and accounta-bility, education and training, recruitmentand hiring, and outreach to diverse commu-nities. The final chapter in the book ad-dresses data collection on vehicle stops.Many agencies around the country are col-lecting data on the stops of drivers made bytheir officers. Officers record information—usually on paper forms—regarding therace/ethnicity and other demographic charac-teristics of the person they have stopped, thereason for the stop, whether a search wasconducted, the disposition of the stop (e.g.,arrest, ticket), and so forth. The primary pur-pose of these data collection efforts is to

assess whether racially biased policing is oc-curring in the jurisdiction.

PERF and the COPS Office joined to pro-duce two resources to guide data collectionefforts. The first book is entitled By the Num-bers: A Guide for Analyzing Race Data fromVehicle Stops and the second is this one,Understanding Race Data from Vehicle Stops:A Stakeholder’s Guide. By the Numbers is adetailed how-to guide on data collection andanalysis. It is written for the people—usuallysocial scientists—who will actually be con-ducting the analyses and issuing the reports.In contrast, this guide addresses the sametopics, but is written for the stakeholders whowill make or otherwise have an impact on de-cisions regarding data collection, and whowill be the consumers of the reports ema-nating from those efforts. This includes lawenforcement chief executives; local, state andfederal policy makers; advocacy groups; themedia; and other concerned communitymembers.

There were two main purposes for writingboth books. One was to temper people’s ex-pectations regarding the conclusions that canbe reached with vehicle stop data. Many con-cerned stakeholders have been too optimisticregarding the ability of these data to deter-mine the existence or lack of racially biasedpolicing. Both books describe the potentialand constraints of these data for measuringracially biased policing. The second purposewas to describe the various ways that datacan be analyzed and the conclusions that canand cannot be drawn from the results.

We hope that this guide will assist stake-holders in making informed decisions re-garding data collection efforts, understandingthe conclusions that can and cannot be

Foreword

vi Understanding Race Data from Vehicle Stops: A Stakeholder’s Guide

drawn from the results, and using the resultsfor constructive change. We also hope thatthis book will lead stakeholders to under-stand that data collection is only one responseto the critical issue of racially biased policing.Data collection seeks to measure racially bi-ased policing (and is limited in its ability toachieve that objective). Chiefs and otherstakeholders should implement measures toaddress (not just measure) the problem of

racially biased policing and perceptions of itspractice (for instance, through policy,training, supervision, and community out-reach). Police and other stakeholders mustcollaborate to identify concerns about law en-forcement practices and think comprehen-sively about how they will be resolved. PERFand the COPS Office hope this document,along with the others in the series, will signif-icantly advance these efforts.

Chuck WexlerExecutive DirectorPERF

Carl PeedDirectorCOPS Office

vii

On behalf of PERF, I convey great appreci-ation to Director Carl Peed, Deputy DirectorPam Cammarata, and other leaders in the U.S.Department of Justice Office of CommunityOriented Policing Services (COPS Office).They shared our belief that there was a signif-icant need for By the Numbers: A Guide forAnalyzing Race Data from Vehicle Stops andfor this companion stakeholder’s guide. Theyprovided the necessary funding to bring thesedocuments to fruition. I am particularlygrateful for the very constructive guidance andunwavering support I received throughout thisproject from Tamara Clark Lucas and AmySchapiro, our COPS Office monitors.

Since this Stakeholder’s Guide is an ab-breviated version of By the Numbers, I ac-knowledge here the people who wereinstrumental in making By the Numbers a re-ality and, by extension, the Stakeholder’sGuide as well. An advisory board made up ofboth practitioners and social scientists metearly on in the project to guide the contentand form of By the Numbers and the Stake-holder’s Guide. Many read chapters or sec-tions of By the Numbers as they werecompleted. Members of this board includedDr. Gary Cordner, Eastern Kentucky Univer-sity; Dr. Scott Decker, University of Missouri,St. Louis; Assistant Chief John Diaz, Seattle(WA) Police Department; Captain Thomas Di-done, Montgomery County (MD) Police De-partment; Dr. Don Faggiani, PERF; Dr. AmyFarrell, Northeastern University; Secretary EdFlynn, Massachusetts Executive Office ofPublic Safety; Assistant Chief John Gallagher,Miami (FL) Police Department; Former ChiefMark Kroeker, Portland (OR) Police Bureau;Dr. John Lamberth, Lamberth Consulting; Dr.Catherine Lawson, State University of NewYork at Albany; Dr. Steve Mastrofski, George

Mason University; Dr. Jack McDevitt, North-eastern University; Antony Pate, Private Con-sultant; Former Assistant Deputy Super-intendent Karen Rowan, Chicago (IL) PoliceDepartment; Professor Margo Schlanger, Har-vard University; Dr. Ellen Scrivner, FormerDeputy Director of the COPS Office; Dr. Deb-orah Thomas, University of Colorado atDenver; Dr. Sam Walker, University of Ne-braska at Omaha; Dr. Charles Wellford, Uni-versity of Maryland; and Dr. Matt Zingraff,North Carolina State University.

A group of academics—most of themmembers of the advisory board—stand out fortheir contributions to the content of bothbooks. They are the key social scientistsaround the country who are analyzing and in-terpreting police-citizen contact data for var-ious law enforcement entities and who havebeen instrumental in advancing the methodsused to assess racial bias. Members of thisgroup were generous with their time andknowledge. They shared their draft and com-pleted reports from their own research, re-sponded to phone and email inquiries, servedas a “sounding board” for particularly vexingissues, and reviewed critically the variouschapters of By the Numbers that reported ontheir work and/or otherwise reflected mostclosely their experience and expertise. In al-phabetical order, these people are

Geoff Alpert, University of South CarolinaGary Cordner, Eastern Kentucky UniversityScott Decker, University of Missouri atSt. LouisRobin Engle, University of CincinnatiAmy Farrell, Northeastern UniversityDavid Harris, University of ToledoJohn Lamberth, Lamberth Consulting

Acknowledgements

viii Understanding Race Data from Vehicle Stops: A Stakeholder’s Guide

Catherine Lawson, State University ofNew York at AlbanyJack McDevitt, Northeastern UniversityMichael Smith, University of SouthCarolinaSamuel Walker, University of Nebraska atOmahaMatthew Zingraff, North Carolina StateUniversity

It is no exaggeration to say that therewould be no content for these documentswithout their research efforts in the field andno summaries of the promising approacheswithout their direct assistance.

Also assisting me in various ways wereIan Ayres of Yale University; R. RichardBanks of Stanford Law School; Deputy ChiefMichael Berkow of the Los Angeles Police De-partment; Jennifer Calnon of PennsylvaniaState University; Jerry Clayton and Karl Lam-berth of Lamberth Consulting; Rod Covey,Former Assistant Director of the Arizona De-partment of Public Safety; Scott Henson ofthe ACLU of Texas; Chief Ellen Hanson ofthe Lenexa Police Department; Chief GlennLadd of the North Kansas City Police Depart-ment; Nicholas Lovrich of Washington StateUniversity; Mike Maltz of the University ofIllinois at Chicago; Ken Novak of the Univer-sity of Missouri, Kansas City; Tim Oettmeier,

Executive Assistant Chief of the HoustonPolice Department; Nicola Persico of theUniversity of Pennsylvania; Jeff Rojek ofSt. Louis University; Matt Salazar of theAustin Police Department; and Jane Wisemanof the Massachusetts Department of PublicSafety. To all of these people, I convey mysincere appreciation.

I owe a tremendous debt to Barbara de-Boinville who provided expert editing of bothbooks—ensuring that complicated, some-times dense material was organized, acces-sible and clear. At PERF my thanks go toExecutive Director Chuck Wexler whopushed me to complete the documents whilebeing completely supportive with the encour-agement and resources I needed to do so. I’mgrateful to Martha Plotkin and David Edelsonwho provided their usual high-quality, expertadvice on both the form and content of bothdocuments and oversaw all aspects of theirpublication. Also at PERF, colleague Don Fag-giani, reviewed various sections of the docu-ments and was on call day and night toprovide advanced methodological guidance.Finally, special thanks to all of my staff in theResearch Unit at PERF who are wonderful inmany, many ways and, for this particularproject, thoughtfully and patiently let mehole up without interruption for days at atime to write.

Lorie A. Fridell

IIntroduction

Racially biased policing and the percep-tions of its practice are critical issues facingjurisdictions across the country. The issuesinvolved in “racial profiling” and racially bi-ased policing are not new. They are the latestmanifestations of a long history of sometimestense, and even volatile, relations betweenpolice and minorities. This need not beviewed, however, as proof of the problem’sintractability. Police are more capable thanever of effectively addressing police racialbias in their ranks. In the past few decadesthere has been a revolution in the quality andquantity of police training, the standards forhiring officers, procedures and accountabilitymechanisms, and the widespread adoption ofcommunity policing.

In the Police Executive Research Forum’sfirst book on this topic, Racially BiasedPolicing: A Principled Response (Fridell et al.2001),1 the various ways that police can andshould respond to racially biased policingand the perceptions of its practice were set

forth.2 Specifically, the book discussedmethods of reform and prevention in theareas of accountability and supervision, poli-cies to address racially biased policing, re-cruitment and hiring, education and training,outreach to minorities in the community, andthe collection of data on police-minority con-tacts. Interestingly, this latter intervention,the collection of vehicle stop data, was the re-sponse of choice for many jurisdictions.Many jurisdictions have started collectingdata on the race/ethnicity of drivers stoppedand/or searched by police.

The collection of data reflects positivelyon policy makers in several ways. It showstheir commitment that biased policing willnot be tolerated, and it conveys a willingnesson the part of the law enforcement agency tobe held accountable to the public for its ac-tions. In addition to wanting to convey con-cern and accountability, policy makerspromote data collection because they wantto determine the nature and extent of racially

1 This book, funded by the U.S. Department of JusticeOffice of Community Oriented Policing Services (COPS)(Grant 1999-CK-WX-0076), is available as a free down-load from the PERF website at www.policeforum.org.

2 Racially biased policing is defined as the inappro-priate consideration by law enforcement of race or eth-nicity in deciding with whom and how to intervene inan enforcement capacity. Mirroring the U.S. Census weuse “ethnicity” to refer to whether a person is of His-panic or non-Hispanic origin.

www.policeforum.org

2 Understanding Race Data from Vehicle Stops: A Stakeholder’s Guide

biased policing. Although many jurisdictionsare eager to analyze data on vehicle stops toachieve this goal, little information has beenavailable to them regarding how to proceed.There also have been overly optimistic expec-tations regarding the ability of social sciencemethods to turn these data into meaningfulconclusions regarding the existence ofracially biased policing.

To address these issues, PERF withfunding from the COPS Office has publishedBy the Numbers: A Guide for Analyzing RaceData from Vehicle Stops (Fridell 2004). By theNumbers is written for the people who are ac-tually conducting the analyses of the data—whether they are associated with policedepartments, academic institutions, or stake-holder groups. This technical guide providespractical, hands-on instruction for analyzingboth stop and poststop data, and it helps theresearcher understand what conclusions canand cannot be drawn based on the results.

Understanding Race Data from VehicleStops: A Stakeholder’s Guide, addresses thesame issues as By the Numbers, but is di-rected toward a different audience. Thisbook is written for the policy makers who areconsidering data collection as a response toconcerns about racial bias in policing in theirjurisdiction or who are otherwise linked todata collection efforts, either as consumers ofthe reports generated by researchers or as par-ticipants of jurisdiction task forces on racialprofiling. This target population includes ex-ecutives of state or local law enforcementagencies (for instance, chiefs, sheriffs, super-intendents), other state or local policy makers(for instance, mayors, city council members,staff of Attorney’s General offices, state orfederal legislators), representatives of themedia, advocacy groups (for instance, theAmerican Civil Liberties Union, Amnesty In-ternational, the NAACP, La Raza, the UrbanLeague), and concerned residents unaffiliatedwith organized groups.

THE RESPONSIBILITIESOF STAKEHOLDERS

The purpose of this guide is to help thetargeted stakeholder groups meet two impor-tant responsibilities. First, they need to de-velop an understanding of both the potentialand constraints associated with data collec-tion; stakeholders need to know what conclu-sions can and cannot be drawn from the data.

Many different people in different profes-sions share a desire to become knowledgeableabout the analysis and interpretation of ve-hicle stop data. They want to gain this infor-mation for varied reasons. Agency executivesand other policy makers seek information tohelp them decide whether to collect vehiclestop data, what data to collect, and how to“benchmark” the data. They want to knowhow to interpret and act upon the results.Media representatives want to know how tomake sense of competing interpretations ofresults. Advocacy groups want objective in-formation regarding all of these issues so thatthey can be constructive in what they conveyto their constituencies. Reflecting on this re-sponsibility of advocacy groups, Ron Davis(quoted in McMahon et al. 2002, 96) makesthese comments:

Civil rights and community-based or-ganizations … have the responsibilityof obtaining ‘expert’ knowledge andunderstanding about racial profiling,biased-based policing, and data collec-tion and analysis before launchingdiscrimination allegations. It is a dis-service to the community for reputableorganizations, whether civil rights orcommunity-based, to accuse law en-forcement of racism and/or discrimi-nation based on statistical disparitiesor the implementation of non-biastraffic enforcement programs.

Although stakeholders should be knowl-edgeable about the potential and constraintsassociated with data collection, this is not

Introduction 3

their only responsibility. A second responsi-bility of stakeholders is to work with eachother. Law enforcement executives shouldreach out to other stakeholders for help in de-signing and implementing the data collectionsystem and for their views on ways theagency can make progress on this long-standing issue affecting police and the com-munity. Residents, local officials—indeed, allparties—need to put aside notions of “we-they” and instead come together in the spiritof cooperation to further the best interests ofthe jurisdiction as a whole.

This guide will help stakeholders meetthe important responsibilities we have de-scribed. It provides a clear explanation of thesocial science challenges associated with datacollection initiatives so that readers can sepa-rate meaning from myth with regard to po-lice-citizen contact data. This book alsoexplains how police and other stakeholderscan come together constructively to conductdata collection/analysis and, more impor-tantly, to address the critical problems ofracially biased policing and the perceptionsof its practice.

The technical information presented inBy the Numbers is summarized here so thatpeople who have a stake in data analysis butwho are not themselves conducting it can un-derstand the material. The book discussesthe challenge of benchmarking, how to assessthe quality of benchmarks, various bench-marking options that jurisdictions canchoose, and how to interpret the research re-sults responsibly.

Readers will come to appreciate that ana-lyzing vehicle stop data is a complicatedbusiness. The information will affirm whatmany have already recognized: qualityanalysis of vehicle stop data is not as simpleas comparing vehicle stop information tobasic census information. In fact, some frus-tration will be generated by a major themeof this book: data collection cannot provideunequivocal answers to questions about the

existence or lack of racial bias by police in ajurisdiction. This fact does not, however,negate the value that can be accrued from col-lecting data on vehicle stops. Even equivocalresults can serve as a basis for constructivedialogue between the police and residents—producing enhanced trust and cooperation aswell as action plans for reform. Equippedwith user-friendly knowledge regarding datacollection and analysis, police and otherstakeholders can join forces to addressracially biased policing and the perceptionsof its practice.

CONTRIBUTIONS OFSOCIAL SCIENTISTS

In producing this document and By theNumbers, PERF relied upon the valuable ex-pertise of an advisory board. Its members—listed in the acknowledgements section—include the key social scientists around thecountry who are analyzing and interpretingpolice-citizen contact data, experienced lawenforcement practitioners, and personnelwithin research units. PERF also has beenaided by members of advocacy groups com-mitted to fair and impartial law enforcement.Therefore, the pronoun “we” is used in bothbooks to acknowledge that their contents re-flect this collective wisdom.

CONTENTS OF THE BOOKChapter 2 describes the social science

challenges associated with analyzing andinterpreting the police-citizen contact datacollected to measure racially biased policing;specifically, it explores the goal, the potential,and the limitations of what has come to becalled “benchmarking” the data. Chapter 3,“Getting Started,” explains the steps agenciesshould take when they begin to collect dataon police-citizen contacts. For example, itdiscusses how to develop a data collectionplan, how and why to involve residentsand police personnel from all levels of theagency in planning and implementing the


data collection system, and how to select abenchmarking method. Chapter 4 examinesissues that are relevant to every bench-marking method, such as the need to reviewdata quality, select a reference period, and an-alyze subsets of data on the jurisdiction.Chapter 5 presents information on methodsthat can be used to address the first of two re-search questions: “Does a driver’s race/ethnicity have an impact on vehicle stoppingbehavior by police?” In considering this ques-tion, a researcher is attempting to understandwhether racially biased policing is manifestedin the decisions of officers regarding whom to

stop. Chapter 6 addresses a second researchquestion: “Does a driver’s race/ethnicity havean impact on police behaviors/activitiesduring the stop?” It describes how to assessthe impact of race/ethnicity on searches anddispositions (for instance, arrest, citation, noaction)—activities that occur after the stop ismade. In Chapter 7 we describe and comparethe various calculations a researcher mightuse to measure disparity between racial/ethnic groups. Finally, in Chapter 8 we dis-cuss how police and other stakeholders cancome together to use the results from data col-lection to achieve reform.

Jurisdictions collecting police-citizencontact data are calling upon social science todetermine whether there is a cause-and-effectrelationship between a driver’s race/ethnicityand vehicle stopping behavior by police. Inanalyzing the data, researchers have at-tempted to develop comparison groups toproduce a “benchmark” against which tomeasure their stop data. If an agency deter-mines that, say, 25 percent of its vehicle stopsare of racial/ethnic minorities, to what shouldthis be compared? In other words, what per-centage would indicate racially biasedpolicing? This is the question at the core ofbenchmarking. To determine an answer, re-searchers have compared the demographicprofiles of people stopped by police to the de-mographic profiles of the residential popula-tion of the jurisdiction, to the demographicprofiles of residents with a driver’s license,and to the demographic profiles of peopleobserved driving on jurisdiction roads—to name a few comparison groups.

THE OBJECTIVEOF BENCHMARKING

Before we discuss the various methods forbenchmarking, it is constructive to considerour objectives when analyzing police-citizencontact data. Then we can outline howbenchmarks vary in their ability to achieve

these objectives. We start with two concep-tual models. Figure 2.1 shows a model of thefirst research question: Does a driver’s race/ethnicity have an impact on the decisionspolice make with regard to whom to stop?We want to know if X (driver race/ethnicity)has any causal impact on Y (police decisionsto stop drivers). To determine causality, how-ever, we must exclude or “control for” rivalcausal factors—factors other than therace/ethnicity of the driver—that could ex-plain police stopping decisions (see themodel in Figure 2.2). In attempting to testwhether X causes Y, we need to rule out alter-native hypotheses that A, B, C, and Z—eitheralone or together or in interaction with X—cause Y.

IIThe Benchmarking Challenge

Figure 2.1. Model of First Research Ques-tion: Does Driver Race/Ethnicity Affect VehicleStopping Decisions Made by Police?

DriverRace/Ethnicity

Police StoppingDecisions

Variable X Variable Y

The following example clarifies why rivalcausal factors must be ruled out in anyanalysis of police-citizen contact data. Let ussay that parents are concerned that thegrading by math teachers at a high school re-flects teachers’ bias against females. The par-ents’ allegation is that these math teachersbelieve boys are better than girls at math andthat—consciously or unconsciously—theseattitudes are reflected in the grades beinggiven to the students.

Our basic conceptual model is that gender(X) has a causal impact on the grades given byteachers (Y). To test this scientifically, how-ever, we cannot conduct analyses that con-sider only X and Y. We cannot, for instance,look only at the percent of females who got A’sand B’s in a course and the percent of maleswho got A’s and B’s in that course and drawany conclusions regarding teachers’ genderbias. Instead, we must consider other factorsthat affect grading behavior. A key variable, ofcourse, would be students’ math performance.Our analyses must control for math perform-ance (for example, scores on objective tests).In other words, our research design or statis-tical techniques must remove or “neutralize”

the impact of performance on grades. If, afterwe have controlled for math performance, westill find that males get better math gradesthan do females, then we must seriously con-sider the possibility of gender bias byteachers.

Now let us return to the first researchquestion concerning who is stopped by po-lice. Police can have various legitimate rea-sons for deciding to stop a vehicle. Thesereasons are the rival causal factors that wouldbecome the A, B, and C of Figure 2.2. Let’sagain consider gender but in the context ofanalyzing police stopping behavior, not mathgrades.

The reports of most jurisdictions re-garding their police-citizen contact data statethat men are stopped by police more thanwomen. For instance, a jurisdiction may findthat 65 percent of its vehicle stops by policeare of male drivers and 35 percent are of fe-male drivers. Does this indicate gender biason the part of the police? It is unclear fromthese data, but most of us are disinclinedto jump to that conclusion because factorsother than police bias could account for thedisproportionate stopping of male drivers.


Variable YPolice Stopping

Decisions

AOther PossibleCausal Factor

InterveningVariable Z

Variable XDriver Race/

Ethnicity

BOther PossibleCausal Factor C

Other PossibleCausal Factor

Figure 2.2. Model of Factors, Other than Bias, that MightAffect Stopping Decisions Made by Police

The Benchmarking Challenge 7

That is, alternative hypotheses for the resultsexist. Men may drive more than women (thequantity of driving factor). Or men may vio-late traffic laws more often than women do(the quality of driving factor). A third possi-bility is that more males than females drive inthe areas where stopping activity by policetends to occur (the location of driving factor).We do not know if these possibilities are true,but we must consider these alternative expla-nations in our research design because it islogical to assume that

l

people who drive more should bemore at risk of being stopped bypolice,

l

people who drive poorly should bemore at risk of being stopped bypolice, and

l

people who drive in locations wherestopping activity by police is highshould be more at risk of beingstopped by police.

The objective of benchmarking in our ex-ample is to see if gender bias is at work. If wecould develop a gender profile of the peoplewho should be more at risk of being stoppedby police, we could compare it to the genderprofile of the people who are being stoppedby police. That is, if we managed through ourresearch design to determine that men shouldcomprise 65 percent of the police stops be-cause of their driving quantity, quality, andlocation, and if indeed they do comprise 65percent of the police stops (based on the stopdata collected), then we could report to thejurisdiction that gender bias did not appear toaffect stopping behavior by police.

Let us review now the key principlesstated above. Researchers’ goal is to develop aracial/ethnic profile of the people who shouldbe at risk of being stopped by police in a ju-risdiction, assuming no bias. Benchmarkingis the essential tool used by researchers toachieve this goal. Benchmarks vary inquality. Their quality is directly related to

how closely each benchmark represents thegroup of people who should be at risk ofbeing stopped by police if no bias exists.

The following example will help clarifywhat we mean by benchmark quality. If a re-searcher uses road-side observers to developa demographic profile of drivers who violatetraffic laws, the researcher has produced abenchmark that represents fairly well thegroup of people who should be at risk ofbeing stopped by police if no bias exists (atleast at the observed location). On the otherhand, if that same researcher used insteadU.S. Decennial Census data to develop a de-mographic profile of people who live in thejurisdiction, the researcher has produced abenchmark that does not represent well thepeople at risk of being stopped by police if nobias exists. The next section on the bias hy-pothesis and the alternative hypotheses ex-pands upon this discussion of benchmarkquality. As we will demonstrate in this re-port, the variation in quality across bench-marks is great. This means that there aresignificant differences in what researcherscan conclude about racial bias and policingin a jurisdiction. Findings based on a high-quality benchmark are more legitimate thanfindings based on a low-quality benchmark.

THE BIAS HYPOTHESIS AND THEALTERNATIVE HYPOTHESES

Alternative hypotheses are hypothesesother than the one that reflects the possibilityof police bias. Law enforcement agencies andtheir partners must consider these hy-potheses when analyzing their jurisdiction’spolice-citizen contact data. These hy-potheses reflect drivers’ driving quantity,quality, and location—the key factors thatcould legitimately influence whom policestop.

The benchmarking method chosen by ajurisdiction must be evaluated in terms of itsability to address the alternative hypotheses.Findings that are based on a method that doesnot address these hypotheses are less valid


than those based on a method that does ad-dress them. The results of benchmarkinganalysis must be interpreted cautiously. Be-fore drawing conclusions about racial bias,researchers must evaluate how successfulthey were in achieving their goal. Their goalis to find out what the racial/ethnic profile ofdrivers stopped by police would look likeassuming no bias.

To make our point we begin with a simpleexample. We ask why—in a jurisdictionmade up of Caucasians, African Americans,Hispanics, and Asians—the police do not re-port that 25 percent of their traffic stops are ofCaucasians, 25 percent are of African Ameri-cans, 25 percent are of Hispanics, and 25 per-cent are of Asians? One hypothesis is thatpolice are racially/ethnically biased in theirdecisions regarding whom to stop. Competingalternative hypotheses are as follows:

l

Racial/ethnic groups are not equallyrepresented as residents in thejurisdiction.

l

Racial/ethnic groups are not equallyrepresented as drivers on jurisdictionroads.

l

Racial/ethnic groups are not equivalentin the nature and extent of their trafficlaw-violating behavior.

l

Racial/ethnic groups are not equallyrepresented as drivers on roads wherestopping activity by police is high.

In order to draw valid conclusions re-garding whether racial bias is occurring inthis hypothetical jurisdiction, we would needto rule out the other possible, legitimate ex-planations for disparity. We refer here to thedisparity between the racial/ethnic profile ofthe people stopped by police for traffic viola-tions and the racial/ethnic profile of thebenchmark population. Ideally, our analysisand interpretation of stop data would encom-pass all of the factors reflected in those alter-native hypotheses.1

Researchers cannot presume that no dif-ferences exist between racial/ethnic groups inthe quantity, quality, and location of theirdriving. (That is, they cannot presume thenull hypothesis.) Below we consider each ofthe four alternative hypotheses every juris-diction must address. For each of the hy-potheses, there is evidence that differencesdo exist between groups, or at least there isinsufficient information to prove to any ac-ceptable degree of certainty that no differ-ences exist. Unless research shows there areno differences between groups as pertains tothese hypotheses, we must assume that thereare differences. Again this requires re-searchers to use methods that consider thefactors encompassed in the alternative hy-potheses or, at the very least, interpret theirresults responsibly in light of any deficien-cies in their chosen methodology.

1 If we address the second hypothesis—racial/ethnicgroups are not equally represented as drivers on jurisdic-tion roads—we need not concern ourselves with thefirst hypothesis—racial/ethnic groups are not equallyrepresented as residents in the jurisdiction. That is, forpurposes of identifying who is at risk of being stoppedby police in a vehicle, if we know who is driving on ju-risdiction roads, we do not need to know who lives inthat jurisdiction. Similarly, addressing the third hy-pothesis—racial/ethnic groups are not equivalent in the

nature and extent of their traffic law-violatingbehavior—arguably negates the need to address the firsttwo. It can be argued that knowing who is engaging inlaw-violating behavior negates the need to know who ison the road. Police are not told to pull over “people onthe road” but rather “people who are violating laws.”The fourth hypothesis—racial/ethnic groups are notequally represented as drivers on roads where stoppingactivity by police is high—stands alone and must be ad-dressed independently of the other three.


Hypothesis 1: Racial/ethnic groups arenot equally represented as residents inthe jurisdiction.It goes without saying: the demographic pro-file of people who live in a jurisdiction willaffect the demographic profile of drivers onthe jurisdiction’s roads. Thus, the above hy-pothesis is indirectly related to the “quantity”factor, and we need to include it in anticipa-tion of our later discussion of census bench-marking (a comparison of the demographicprofile of people stopped by police to the de-mographic profile of jurisdiction residents asmeasured by the U.S. Census Bureau). Thatracial/ethnic groups are not equally repre-sented among residents in jurisdictions is, ofcourse, quite obvious to all. According to the2000 Decennial Census, 75.1 percent of theU.S. population is White, 12.3 percent isBlack or African American,2 and 3.6 percentis Asian; 9.0 percent of the population self-identify as being of more than one race. Justover 12 percent (12.5 percent) of U.S. resi-dents (of all races) are of Hispanic origin. Al-though figures for different jurisdictions willdeviate from this breakdown of the total U.S.population, we can confidently state that nojurisdiction has equal representation in itspopulation of racial/ethnic groups.

Hypothesis 2: Racial/ethnic groups arenot equally represented as drivers onjurisdiction roads.Not only are racial/ethnic groups not equallyrepresented among residents in the jurisdic-tion (Alternative Hypothesis 1), but their rep-resentation as residents might not match theirrepresentation as drivers using jurisdiction

roads for two reasons: (1) racial/ethnic differ-ences in driving quantity and/or (2)racial/ethnic differences in the population ofpeople who do not live in the jurisdiction butdrive in it. This is relevant to the analysis ofvehicle stops by police. If one demographicgroup has more presence on the road than an-other, it should be more at risk of beingstopped.

Driving QuantityThere is evidence that racial/ethnic groupsdiffer in the amount of their driving. Na-tional data from the U.S. Decennial Censusand from the National Household Transporta-tion Survey (NHTS) indicate that racial/ethnic minorities are under-represented asdrivers relative to their residential popula-tions.3 The U.S. Decennial Census providesdata on the percent of households that do notown vehicles, an indirect measure of drivingquantity. In his comprehensive report oncommuting patterns based on 1990 Censusdata, Pisarski (1996, xv) reports that “on av-erage, more than 30 percent of Black house-holds do not own vehicles, and in centralcities the number is over 37 percent.”4 Na-tionally, 19 percent of Hispanic householdsdo not own vehicles; in central cities thatnumber rises to 27 percent. In contrast, justunder 9 percent of White non-Hispanichouseholds are without vehicles, with a cor-responding figure of 15 percent for centralcities (Pisarski 1996, 36).

Vehicle ownership is an indirect measureof driving quantity. Information from the Na-tional Household Transportation Survey pro-vides more direct measures of driving

2 The terms African Americans and Blacks are used in-terchangeably in this book.3 The National Household Transportation Survey (pre-viously called the Nationwide Personal TransportationSurvey and the American Travel Survey) is conductedby the U.S. Department of Transportation. Seewww.bts.gov/nhts.

4 Some cities have “extraordinary levels of Black house-holds without vehicles” (Pisarski 1996, 36). In NewYork, 61 percent of Black households are without vehi-cles. The corresponding figures for Philadelphia,Chicago, and Washington, D.C., are 47 percent, 43 per-cent, and 43 percent, respectively.

www.bts.gov/nhts


quantity. Its data indicate that nonminoritiesdrive more than minorities. For instance, the1995 NHTS indicated that African Americansaverage fewer “trips per day” (includingfewer vehicle trips) than do Caucasians andthat Hispanics are twice as likely as non-Hispanics to use public transportation (in-stead of privately owned vehicles).

While the 2000 Census data on vehicleownership and NHTS data on driving quan-tity both imply that minorities are under-represented as drivers relative to theirrepresentation in the U.S. population, otherresearch reminds us that this is not going tobe true in all places at all times. For instance,research conducted by the United Kingdom’sHome Office (MVA and Miller 2000) foundthat minorities were over-represented asdrivers relative to their representation in theresidential populations in the areas studied.5

In Sacramento, California, Howard Green-wald compared the demographic profiles ofdrivers at various intersections (using obser-vation) to the demographic profiles of resi-dents in the same areas (using census data);he found over-representation of minorities asdrivers in some areas and under-representa-tion of minority drivers in others (Greenwald2001). These two small-scale studies, al-though of less weight than the large-scaleresearch findings of the NHTS and U.S.Census, nonetheless support our simplepoint: jurisdiction-level studies of raciallybiased policing must consider the possibilitythat racial/ethnic groups are not equally

represented as drivers on jurisdiction roadsbecause of differences in their quantity ofdriving.

The extent to which racial/ethnic groupsdrive on roads will vary across locationswithin a jurisdiction. Because of this fact, werecommend that researchers not conduct asingle analysis for the entire jurisdiction butnumerous analyses within various geo-graphic subareas of the jurisdiction (seeChapter 4).

Driving by NonresidentsThere is another reason—other than differ-ences in driving quantity of jurisdiction resi-dents—that racial/ethnic groups may not beequally represented as drivers on jurisdictionroads (and why their representation on theroads may not reflect their representation asresidents). Racial/ethnic groups may not beequally represented among the nonresidentswho drive in the jurisdiction; that is, racial/ethnic groups may not be equally representedamong the people who live outside of the ju-risdiction but drive into it.6 The extent towhich nonresidents drive within the jurisdic-tions that are collecting police-citizen contactdata will vary greatly, as might the demo-graphic profile of those drivers. The influx ofnonresident drivers will be particularly sig-nificant in the big cities that draw commutersin from surrounding jurisdictions, especiallythe suburbs, during the daytime hours.7

Additionally, nonresidents will drive into the“target jurisdiction” (the jurisdiction that is

5 The Home Office of the United Kingdom is the gov-ernment department responsible for promoting safecommunities. Its closest equivalent in the UnitedStates is the National Institute of Justice.6 In its first annual report regarding police–citizen con-tact data, the Denver Police Department (Thomas 2002)revealed that 62.5 percent of the Whites stopped intheir vehicles by police were nonresidents compared to32.8 percent of the Blacks who were stopped and 35.2percent of the Hispanics who were stopped.

7 In 1993, 43 percent of the traffic tickets given inSeattle were given to nonresidents (Scales 2001). TheDenver Police Department (Thomas 2001) reported thatfrom June 2001 through May 2002 (the reference periodfor its second summary report) over one-half of itstraffic stops were of nonresidents. In Louisville (Ed-wards et al. 2002a) and Iowa City (Edwards et al.2002b), fewer than two-thirds of all drivers stoppedwere city residents.


the subject of police-citizen contact dataanalysis) to shop, seek entertainment, vaca-tion, travel on to another jurisdiction, and forother reasons. These nonresident drivers willaffect the demographic profile of drivers onthe roads of the target jurisdiction.

The frequency with which drivers arestopped by police is affected by driving quan-tity. Racial/ethnic groups are not equally rep-resented as drivers on jurisdiction roads. Thisis a viable alternative hypothesis that shouldbe accounted for in the analysis of police-citizen contact data. This book will describehow jurisdictions can incorporate this alter-native hypothesis into their study design.

Hypothesis 3: Racial/ethnic groups arenot equivalent in the nature and extentof their traffic law-violating behavior. Police are directed to stop drivers because oftheir driving behavior. Therefore, researchersmust recognize this variable (traffic law–violating behavior) in their research unless weare quite confident that there are no differ-ences across racial/ethnic groups. Excludingdriving behavior from the model is equivalentto excluding math performance from the ear-lier analysis that tested gender bias in mathteachers.

Vehicle stopping behavior by police maynot be equivalent across racial/ethnic groupsbecause racial/ethnic groups violate trafficlaws at different rates or at different levels ofseriousness. These possibilities must be rec-ognized. Concerned stakeholders have ques-tioned the inclusion in our analysis of thethird hypothesis (racial/ethnic groups are notequivalent in the nature and extent of theirtraffic law-violating behavior). They haveasked the author whether the unstated impli-cation is that minorities violate more. Indeed,no direction is implied by its inclusion.Minorities may violate traffic laws with lessfrequency than do majority populations. (Infact, this could be the case in light of minori-ties’ concern about racial profiling and the

increased attention they perceive they getfrom police.) If minorities do violate less,then it is important that this information beincorporated into the analysis to appropri-ately determine the rate at which they shouldbe stopped by police in light of their drivingquality. Driving behavior cannot be removedfrom our analysis unless there is clear evi-dence in support of the null hypothesis (nodifferences between racial/ethnic groupsexist). The following information calls thenull hypothesis into question.

Information on theEquivalence of Driving BehaviorLarge-scale, quality research on driving be-havior and race/ethnicity is scarce, but thisdoes not negate the importance and viabilityof Alternative Hypothesis 3. In fact, it doesjust the opposite: what is important for ourpurposes is the absence of sufficient researchto rule out the possibility of racial/ethnicdifferences in the nature and extent of law-violating behavior. Again, even if we had na-tional data pointing to equivalent driving be-havior or pointing to one particular directionor the other, we could not presume that thoseresults were applicable to all times and allplaces.

The information on the equivalence ofdriving behavior across racial/ethnic groupsis mixed. Research in the transportationfield, albeit not substantial, indicates somedifferences across racial groups with regardto certain traffic violations. For instance,Feest (1968) found that Whites were morelikely than minorities not to stop at stopsigns. Other researchers analyzing police-citizen contact data have produced informa-tion indicating other differences in violatingbehavior across racial/ethnic groups. For in-stance, Lange, Blackman, and Johnson (2001)found that along segments of the New Jerseyturnpike where the speed limit was 65 milesper hour rather than 55 miles per hour,African Americans were disproportionately


represented among the few speeders.8 Otherresearchers have also identified differences inspeeding behavior across races (Engel,Calnon, Liu and Johnson 2004 and Smith etal. 2003). In contrast, Lamberth (1996a,1996b) conducted research in New Jersey andMaryland and found no differences in the de-mographics of speeders versus nonspeeders.He reports that all racial/ethnic groups werespeeding in high, and similar, proportions.9

In citing these mixed findings, we are nottrying to argue that there are differences in vi-olating behavior across racial/ethnic groups.Quite the contrary: we do not know whetherdifferences exist or not. Because the researchdoes not allow us to rule out the possibility ofdifferences in driving quality across racial/ethnic groups, we contend that research ana-lyzing police-citizen contact data should ad-dress the alternative hypothesis that racial/ethnic groups are not equivalent in the natureand extent of their traffic law-violatingbehavior.10

Youthfulness and Driving BehaviorYouthfulness has been linked to law-violatingbehavior. If a racial/ethnic group has propor-tionately more young people than anotherracial/ethnic group, age becomes an impor-tant “intervening variable”11 in the analysismodel. (It is a potential “Variable Z” in Figure2.2.) We must consider whether the break-down of age groups in a jurisdiction (or in thesubareas being analyzed) varies across racial/ethnic groups. For example, if 30 percent of

the minority population of an area is young(24 years of age or less) and only 20 percentof the Caucasian population is young, thisphenomenon would lead to more drivers whoviolate the law in the minority populationthan in the nonminority population, as-suming the link between poor driving andage.

When researchers use police-citizen con-tact data to measure racially biased policingin a jurisdiction, they may get results thatsuggest bias when none exists. Dispropor-tionate numbers of young drivers in racial/ethnic groups in a jurisdiction can producemisleading results. We illustrate this hazardwith hypothetical data in Table 2.1 andFigure 2.3. In our example, two assumptionsare made: (1) there are equal numbers ofCaucasian and minority drivers on the roadin hypothetical Jurisdiction Q, and (2) thereis equivalence of driving behavior acrossthese two racial/ethnic groups.

As shown in Table 2.1, Caucasians andminorities each made up 50 percent of thedriving population. (There are 1,000 driversin each group.) Among the Caucasiandrivers, 300 or 30 percent were between theages of 15 and 24, and 700 or 70 percent were25 or older. (We use age 15 as the lower cut-off point to include only people of drivingage.) The corresponding percentages for theminority group of drivers were 60 percentand 40 percent. That is, 600 of the driverswere between the ages of 15 and 24, and 400were 25 years of age or older.

8 This study was criticized for various aspects of itsmethodology and the high proportion of missing dataproduced by those methods.9 These studies defined speeding so broadly (1 mile perhour over the speed limit in Maryland and 5 miles perhour over the speed limit in New Jersey) that speedersincluded most drivers. This broad definition reducedthe researcher’s ability to detect any existing, finer dis-tinctions in driving behavior across groups.

10 See By the Numbers (Fridell 2004), Appendix D,which challenges this view in the context of discussingthe observation method of benchmarking. Appendix Esummarizes arguments for and against the methodolog-ical consideration of variations in driving quality. 11 We use the term “intervening variable” to refer to avariable (measured or unmeasured) that is linkedcausally to one or more other variables in an equationor model.


The police in hypothetical Jurisdiction Qare completely devoid of racial/ethnic bias,and they legitimately stop, as a result of theyoung drivers’ poorer quality driving, twiceas many drivers between the ages of 15 and24 as drivers 25 years of age and older.Twenty percent of the young Caucasians werestopped (0.2 x 300 = 60), and 20 percent ofthe young minorities were stopped (0.2 x 600= 120). Police stopped 10 percent of the Cau-casian drivers age 25 or above (producing 70stops) and 10 percent of the minority driversage 25 or above (producing 40 stops).

The effect of the differential representa-tion of young people among the minoritydrivers can be seen when we look at theoverall representation of Caucasians and mi-norities among the drivers stopped by police(Figure 2.3). Caucasians made up 50 percentof the drivers (1,000 of the total 2,000) andonly 46 percent of the stops. Minorities madeup the other 50 percent of the drivers but 56percent of the stops. Even though racial biasis not manifested by the police (equivalentstopping decisions across racial/ethnicgroups), our data indicate (falsely) that dis-parity exists. If the researcher for JurisdictionQ did not, as we did, analyze the data withinage groups to confirm a lack of disparity, theresearcher would have mistakenly concluded

that there was disparity across racial groups.The disproportionate representation of youthin the minority population and the increasedlikelihood of young people being stopped bypolice produced the misleading resultsshown in Figure 2.3: minorities appeared tobe over-represented among people stoppedrelative to minorities’ representation in thedriving population.

In sum, the strongest research methodolo-gies will address the alternative hypothesisthat racial/ethnic groups are not equivalent inthe nature and extent of their traffic law-violating behavior. Theoretically, driving be-havior is quite relevant to decisions by policeto stop drivers, and the research that has beenconducted on the relationship betweendriving quality and race/ethnicity is not suffi-cient for us to assume no differences acrossgroups. Complicating matters as pertains tothis “quality of driving” factor is the link be-tween age and driving behavior.

Hypothesis 4: Racial/ethnic groups arenot equally represented as drivers onjurisdiction roads where stoppingactivity by police is high. Vehicle stops by police may occur more insome areas of a jurisdiction than in others.Indeed, the level of stops may vary quite

Table 2.1. Representation of Caucasian and Minority Drivers in the Driving Population andPopulation of Stopped Drivers, by Age, Hypothetical Jurisdiction Q

Age Group

Caucasians (n=1,000) Minorities (n=1,000)

Number of Drivers Percent Stopped Number Stopped Number of Drivers Percent Stopped Number Stopped

15-24 300 20% 60 600 20% 120

25+ 700 10% 70 400 10% 40

Total 1,000 13% 130 1,000 16% 160

Percentage of all stops: 45.61% Percentage of all stops: 56.14%


legitimately from area to area.12 For example,citizens may be concerned about the highnumber of accidents in a particular area, andthey may ask the local police department tocrack down on speeders there. People whodrive in areas where stopping activity by po-lice is high are at greater risk of being stoppedthan are people who drive in areas with lowstopping activity.13 If the demographic com-position of these areas also varies, the police-citizen contact data gathered by the jurisdic-tion can appear to indicate racial bias bypolice where none exists. Therefore, westrongly recommend that researchers conductanalyses within subareas of the jurisdiction.If they do not—if they analyze data on stopsfor the jurisdiction as a whole—results thatindicate disparity may reflect not racial/

ethnic bias but very legitimate variations inpolice practices.

Table 2.2 and Figure 2.4, analogous to theearlier example that focused on differencesin age demographics across racial/ethnicgroups, illustrate how misleading indicatorsof racial/ethnic disparity can easily emerge.The racial/ethnic profile of driving-age resi-dents and the racial/ethnic profile of thedrivers stopped in hypothetical Jurisdiction R(composed of Area A and Area B) are shownin Table 2.2. There are an equal number ofpeople of driving age in each area (1,000 each),but 80 percent (800) of driving-age residentsin Area A are Caucasians, and 80 percent(800) of driving-age residents in Area B areminorities. In each area, the demographicprofile of the drivers stopped by policematches the demographic profile of the

12 These variations in stops across areas within a juris-diction would not be legitimate if the differential en-forcement were based on inappropriate factors such asracial/ethnic bias. To discern whether bias is a factor,the researcher could assess whether legitimate factors(such as calls for service, traffic accidents) adequatelypredict levels of stops.

13 Heavy levels of police deployment will not neces-sarily coincide with high levels of vehicle stops fortraffic violations. In fact, in some high-crime areaswhere police deployment is likely to be correspondinglyhigh, traffic enforcement may be a low priority in lightof the more critical problems that need to be addressed.

Figure 2.3. False Indication of Racial/Ethnic Bias Based on AgeDifferences of Drivers in Hypothetical Jurisdiction Q

70%

65%

60%

55%

50%

45%

40%

35%

30%Caucasians

DriversStops by Police

Source: Based on Table 2.150.00%

45.61%

Minorities

50.00%

56.14%


driving-age adults in the area. That is, inArea A, 80 percent of the residents are Cau-casians, and 20 percent are minorities; simi-larly, 80 percent of the drivers stopped bypolice are Caucasians and 20 percent areminorities.14

In Area B, like Area A, the demographicprofile of the drivers stopped by policematches the demographic profile of the resi-dents. In short, the results as analyzed withinArea A and Area B indicate no disparity.Note, however, that more traffic stops aremade in Area B than in Area A. In our hypo-thetical example, the reason is legitimate.Concerned citizens have requested greaterenforcement of speeding laws in this partof town. Because of this, twice as many stopsare made in Area B (200 stops) than in Area A(100 stops). If the researcher had not con-trolled for police activity within the two areas(by conducting separate analyses for Area Aand Area B) but instead had presented datafor the whole jurisdiction, the results wouldhave appeared to signal racially biased stop-ping decisions. In the lower, far right col-umn of Table 2.2, note the disproportionate

representation of minorities among driversstopped by police relative to their representa-tion among residents (60 percent versus 50percent, respectively). Those misleading re-sults are obtained when the absolute num-bers of stops across areas are summed, andthe demographic profile of the drivers whoare stopped is compared to the demographicprofile of the residential population. Minori-ties comprise 50 percent of the jurisdictionpopulation but 60 percent of all stops. This isshown in Figure 2.4. Anyone looking at thisfigure could easily jump to the conclusionthat police were picking on minorities, stop-ping them in numbers that were dispropor-tionate to minorities’ representation in thepopulation of the jurisdiction. But was thisthe case? No. Even if all officers’ decisionswere based on legitimate factors, and even ifthe increased traffic enforcement activity inArea B was completely legitimate, the resultsshown in Figure 2.4 would be the same. Suchresults will mislead researchers unless theytake into account Alternative Hypothesis 4.

In sum, people who drive in areas wherestopping activity by police is high are at

Table 2.2. Representation of Caucasian and Minority Drivers in the Driving Populationand Population of Stopped Drivers, by Subarea, Hypothetical Jurisdiction R

Types ofDrivers

Area A Area B Total Jurisdiction

No. ofDriving-AgeResidents

Percent ofResidents

No. ofStops

Percentof Stops


Percent ofResidents

No. ofStops

Percentof Stops


Percent ofResidents

No. ofStops

Percentof Stops

Caucasians 800 80% 80 80% 200 20% 40 20% 1,000 50% 120 40%

Minorities 200 20% 20 20% 800 80% 160 80% 1,000 50% 180 60%

Total 1,000 100% 100 100% 1,000 100% 200 100% 2,000 100% 300 100%

14 We use this particular benchmark, residential popu-lation, for purposes of making our point—not to pro-mote it as a method.


greater risk of being stopped than those whodrive in areas where stopping activity is low.Stops may legitimately vary across geo-graphic areas where the demographic compo-sition also varies. Therefore, analyses ofpolice-citizen contact data should reflect con-sideration of the hypothesis that racial/ethnicgroups are not equally represented as driverson roads where stopping activity by police ishigh. The example given in this section high-lights why researchers should conductanalyses within geographic subareas of theirjurisdiction, and they should select thosesubareas in a way that allows them to holdconstant (or “control for”) the exposure ofdrivers to stopping activity by police.

SUMMARY OF THEBENCHMARKING CHALLENGE

To measure whether racially biased policingis occurring in a jurisdiction, researchersmust develop a “benchmark” against whichto compare the racial/ethnic profile of driversstopped by police. This benchmark is theracial/ethnic composition of the drivers whoare at risk of being stopped, assuming no biasby police. The key factors that influence this

risk are driving quantity, driving quality, andthe location of driving. Stopping activity bypolice is affected by these three causal fac-tors. In order to determine whether there is acause-and-effect relationship between therace/ethnicity of drivers and police stops (thebias hypothesis), researchers must be able torule out alternative hypotheses that reflectthe factors that increase the risk of beingstopped. The alternative hypotheses are

l

racial/ethnic groups are not equallyrepresented as residents in thejurisdiction,

l

racial/ethnic groups are not equallyrepresented as drivers on jurisdictionroads,

l

racial/ethnic groups are not equivalentin the nature and extent of their trafficlaw-violating behavior, and

l

racial/ethnic groups are not equallyrepresented as drivers on roads wherestopping activity by police is high.

It is not difficult to measure whether po-lice stop drivers of one racial/ethnic groupmore or less than another; the difficulty

70%

60%

50%

40%

30%

20%

10%

0%Caucasians

ResidentsStops by Police

Source: Based on Table 2.2

50%

40%

Minorities

50%

60%

Figure 2.4. False Indication of Racial/Ethnic Bias Based onDifferential Stopping Activity by Police across Subareas inHypothetical Jurisdiction R


comes in identifying the causes for disparity.The alternative hypotheses present potentialcauses that need to be ruled out before a re-searcher can claim that any identified dis-parity is likely the result of police bias. Aftercontrolling for driving quantity, drivingquality, and driving location (as pertains tolevels of police stopping activity), a re-searcher who finds that minorities are dispro-portionately represented among driversstopped by police can conclude with reason-able confidence that the disparity reflects po-lice bias in their decision making. If nodisparity was found, the researcher can fairlyconfidently conclude that bias was not a partof police decision making. If, on the otherhand, the researcher finds disparity in the re-sults after controlling for only driving quan-tity and driving location, s/he can report thatdisparity exists and that the results can be ex-plained either by police bias or differentialdriving quality. That is, the researcher couldnot pinpoint a single cause (for example,bias) but must report that (at least) two pos-sible explanations for the disparity remain.

Even results showing no disparity wouldneed to be qualified if all factors were notcontrolled for. If, for instance, results indi-cated no disparity in stops, but drivingquality had not been considered, the re-searcher cannot rule out the possibility ofracial/ethnic bias in stopping behavior. Weexplore this possibility further in our discus-sion below of “masking.”

A benchmark’s value depends on theextent to which it addresses the alternativehypotheses. The higher the quality of thebenchmark, the more confidence a researchercan have in the results. The need to rule outalternative hypotheses shows how muchmore complex benchmarking is than manyhave previously thought.

THE PROBLEM OFINCONCLUSIVE RESULTS:

A CENSUS BENCHMARKING EXAMPLEResearchers’ failure to address the alternativehypotheses can lead to inconclusive results.In this section we use the census bench-marking method of analyzing police-citizencontact data to illustrate this point. In censusbenchmarking, a jurisdiction compares thedemographic profile of the drivers stoppedby police to the demographic profile of theresidents of the jurisdiction as measuredby the U.S. Decennial Census. Regardless ofthe results of this comparison (minoritiesare over-represented, minorities are under-represented, minorities are proportionatelyrepresented), researchers can draw no defini-tive conclusions regarding racially biasedpolicing.

Let us suppose that a law enforcementagency finds that minorities are over-repre-sented among drivers stopped by police rela-tive to minorities’ representation amongjurisdiction residents. The racial/ethnic dis-parities manifested in this comparison mightreflect racially biased policing, or they mightreflect variation in the demographic profilesof (1) drivers on jurisdiction roads, (2) trafficlaw violators, or (3) drivers driving in loca-tions where stopping activity by police ishigh. A comparison of stop data to censusdata indicates disparity, but the causes of thatdisparity have not been identified. We knowthat we have “disparate impact” (using the so-cial science rather than the legal definition ofthe phrase), but we do not know if we haveunjustified disparate impact in the form ofracially biased policing. Because of theselimitations, no conclusions can be drawnwith regard to the existence or absence ofracially biased policing in the jurisdiction.

Census benchmarking (assuming no ad-justments of the census data)15 takes into

15 Chapter 5 discusses ways that census data can be ad-justed by researchers in an attempt to encompass fac-tors related to driving quantity.


consideration only one of the four alternativehypotheses presented in this chapter—thehypothesis that racial/ethnic groups are notequally represented as residents in the juris-diction. Census benchmarking does not ad-dress hypotheses related to demographicvariations across driving quantity, quality, orlocation. Unaware of these shortcomings ofthe methodology, public officials, law en-forcement executives, civil rights group rep-resentatives, journalists, and other stake-holders often draw wrong conclusions fromthe results of census benchmarking. Some ofthose false conclusions are expressed in thebenchmarking “myths” described below.

BENCHMARKING MYTHS

Myth 1: No racial/ethnic disparitymeans no racially biased policing.The results produced by benchmarking withunadjusted census data, regardless ofwhether they show that minorities are under-represented, over-represented, or proportion-ately represented among drivers stopped bypolice, cannot enable researchers to drawconclusions about racially biased policing.This is an important truth, but some havecontradicted it. In some reports, the authorscorrectly acknowledge that census bench-marking cannot produce conclusions re-garding the existence of racially biasedpolicing (because the alternative hypotheseshave not been ruled out), they argue, how-ever, that it can prove the absence of raciallybiased policing. A finding of disproportion-ately high minority representation amongpersons stopped does not prove raciallybiased policing, they say, but a finding ofdisproportionately low minority representa-tion or proportionate minority representationdoes prove that racially biased policing does

not exist. This argument—that a method isvalid for one result although not for an-other—is not true.

The adequacy of a law enforcementagency’s benchmark is the same for all re-sults. The researchers who put forth the ar-gument that, regardless of benchmark quality,a showing of no disparity means no raciallybiased policing fail to recognize that an inad-equate benchmark can “mask” (or hide) dis-parity. The following example shows how.

Let us say that a jurisdiction uses censusbenchmarking and finds that the racial/ethnicprofile of residents matches perfectly theracial/ethnic profile of people stopped by po-lice. It is still possible that policing in the ju-risdiction is racially biased. This is surprisingbut true. How can it be possible? It is pos-sible if minorities’ representation in the resi-dential population is a higher percentagethan minorities’ representation in the popula-tion of drivers or a higher percentage than mi-norities’ representation in the population ofdrivers violating the law. Then a finding thatminorities are stopped proportionate to theirresidential representation may indicateracially biased policing. Indeed, the existenceof racially biased policing may be masked byflaws inherent in the benchmark. For in-stance, a researcher conducting censusbenchmarking would not have the informa-tion needed on violating behavior and there-fore could easily misinterpret the results.

Figure 2.5 presents a finding of no dis-parity between minorities and nonminoritiesthat some mistakenly argue indicates an ab-sence of racially biased policing. It showsthat 25 percent of the residents are racial/ethnic minorities (left bar of three showingminority information) as are 25 percent of thepeople who are stopped by police for traffic


violations (right bar of three showing mi-nority information).

The proportion of minorities and nonmi-norities who are traffic violators (center bar ofthree showing minority information) is infor-mation that would not be available to theresearcher who conducted only censusbenchmarking. This information indicatesthat minorities are over-represented amongthe drivers who are stopped (the right bar ishigher than the center bar). If minoritiescomprise only 10 percent of the traffic viola-tors (that is, 10 percent of the populationlegitimately at risk of being stopped by police),but 25 percent of the population that isstopped by police, racial bias may be a factor.The key here is that the researcher con-ducting census benchmarking would nothave had the information (on violating be-havior that is shown with the center bar) nec-essary to interpret either results that showeddisparity or results that showed no disparity.

Researchers who are assessing police-citizen contact data should remember that(1) a weak benchmark is weak for all results,

and (2) their benchmarking method can maskracially biased policing.

Myth 2: Results from a weak method-ology become more worthy over time. It is not true that results from a weak method-ology, or benchmark, can become a worthybaseline for interpreting data in subsequentyears—at least not for the purpose of as-sessing the existence of racially biasedpolicing. An example will help dispel thismyth. Let’s say that a jurisdiction usescensus benchmarking and determines thatracial/ethnic minorities are over-representedamong people stopped by police relative totheir representation in the residential popula-tion as measured by the census. As explainedabove, these results indicate the existence ofa disparity but not its cause. The temptationfor stakeholders, and even some researchers,is to equate the disparity with racially biasedpolicing and to desire a reduction in that dis-parity in subsequent years. That is, theymight acknowledge that their benchmark is

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%Minorities

Jurisdiction ResidentsTraffic ViolatorsStops by Police

Nonminorities

Figure 2.5. Racially Biased Policing Masked in HypotheticalJurisdiction S


weak but falsely claim that the results pro-duced during the first year of analysis can beused to assess and evaluate change in subse-quent years. Because of the weak methodused, the researcher cannot equate the dis-parity with racially biased policing and there-fore should not presume that a reduction indisparity the following year would be desir-able and that it would indicate reduced bias.Similarly, a jurisdiction that finds no dis-parity as a result of its census benchmarkinganalysis the first year and does find disparitythe second year should not blame the policedepartment. Again, because of the methodsused, this disparity cannot be equated withpolice bias.

In sum, a benchmark that cannot pinpointcause cannot produce explanations of causeover time. A reduction in disparity is not al-ways a legitimate goal. Disparity may reflectwholly legitimate factors at work, but thiscannot be known with some benchmarkingmethods.

Myth 3: Results from a weak method-ology become strong if replicated inmultiple geographic areas.A police department that conducts censusbenchmarking within multiple subareas ofthe city (say, within each police district) andfinds no evidence of racial/ethnic disparity ineach one might conclude that the city as awhole is not encountering biased policing.The police spokesperson might acknowledgethe weaknesses of census benchmarking butdiscount those weaknesses and claim that be-cause the results are consistent throughoutthe city, this proves policing in the city is notracially biased. Such a claim would be inerror. The results from a weak methodologyare not validated if the results are consistentacross multiple geographic areas.

If a methodology can measure only dis-parity and not the cause of that disparity, this

limitation persists even when the method-ology is used over and over again in multipleareas. In a contrasting example, a researchermay find disparity in all or most of the sub-areas within a jurisdiction. Again, however,multiple measures of disparity do not accu-mulate to provide a cause for that disparity;they continue to represent only multiplemeasures of disparity.

CONCLUSIONThe challenge of analyzing stop data is to de-termine a cause-and-effect relationship be-tween drivers’ race/ethnicity and stoppingdecisions by police. This requires that re-searchers develop a racial/ethnic profile ofdrivers at risk of being stopped by police, as-suming no bias. Several factors can legiti-mately increase or decrease the likelihoodthat drivers will be stopped. The “alternativehypotheses” to the “bias hypothesis” take intoaccount these factors. The strength of abenchmark depends on the degree to which itencompasses the factors (driving quantity,quality, and location) associated with the al-ternative hypotheses. In Chapter 5 we discussthe major benchmarking methods: adjustedcensus benchmarking, benchmarking basedon a comparison of licensed drivers anddrivers stopped by police, benchmarkingbased on blind versus not-blind enforcementmechanisms, internal benchmarking, andobservation-based benchmarking. Each bench-marking method’s ability to address the alter-native hypotheses is explained. Relatedly, wemake recommendations regarding how theresults of the police-citizen contact dataanalysis can be responsibly conveyed to thepublic. However, before we turn to these var-ious benchmarking methods, we discuss howagencies mandated or choosing to collect datainitiate collection (Chapter 3) and prepare thedata for analysis (Chapter 4).

IIIGetting Started

This chapter describes the preliminarysteps a jurisdiction should take when col-lecting police-citizen contact data. It also ex-plains how and why a jurisdiction mightinvolve residents, police personnel from alllevels of the department, and independent so-cial scientists in these efforts. The factorsthese jurisdiction stakeholders should con-sider before choosing a benchmark for ana-lyzing the data are specified.

Any jurisdiction team that is planning tocollect data needs to address the followingquestions:

l

On what law enforcement activitiesshould data be collected?

l

What information should be collectedregarding those activities?

l

How should the data be analyzed andinterpreted?

Building upon the work of Ramirez,McDevitt, and Farrell (2000), Fridell et al.

(2001, Chap. 8) discuss the options availableto agencies regarding the first two questions.1

For instance, the 2001 PERF book reviews theconsiderations for deciding whether to col-lect data on traffic stops only, all vehiclestops, or all detentions (including pedestrianstops). Also discussed are the data elementsthat agencies should consider for inclusion intheir protocol (for example, the date, time,and reason for the vehicle stop; the race, eth-nicity, age, and gender of the person stopped;information regarding stop dispositions andsearch activity). We do not repeat those dis-cussions here. Agencies in the first stages ofplanning data collection will find these previ-ously published sources helpful. (Again, theFridell 2001 document can be downloadedfrom www.policeforum.org.) It also may beconstructive for them to contact peer agen-cies and request to review their “forms.”2 Besure to ask relevant personnel what, in hind-sight, they would change about their forms.

1 For the sake of simplicity, we refer to law enforcementagencies as the primary actor in setting up a data collec-tion system. As discussed in Chapter 1 and later in thischapter, we recommend that the agency work with res-ident stakeholders in making these decisions.

2 Not all agencies are using paper forms to collect theirdata. Some agencies ask their officers to submit data byusing handheld or in-car computers; in other agencies,officers verbally submit the stop information over theradio. The word “forms” used throughout this reportdenotes all methods of data submission.

www.policeforum.org


DEVELOPING THEDATA COLLECTION PROTOCOL:

TWO RECOMMENDATIONSWe offer two important recommendations re-lated to developing the data collection pro-tocol. First, plans for how an agency willanalyze its data should be developed, if fea-sible, at the same time the decision makersdevelop the overall strategy for collection.Uninformed or after-the-fact decisions inthese matters can lead to unnecessary ten-sions between residents (particularly racial/ethnic minority residents) and policy makersand/or between police officers and policymakers. Both jurisdiction residents and offi-cers have a strong stake in the highest qualityanalyses of the data. Officers, in particular,can be legitimately skeptical of—evenstrongly opposed to—data collection efforts ifthey lack assurances that the data will be an-alyzed using the best social science methodsavailable or, at least, responsibly interpreted.An early designation of the method ofanalysis and a commitment to responsibleinterpretation can mitigate these concerns.In the same vein, it is important for theagency and other jurisdiction stakeholders toconfirm early on that sufficient resources areavailable to meet their objectives. Otherwise,a jurisdiction may make a significant invest-ment in a data collection system only to findout that analyses of the quality it desirescannot be implemented. Some of the methodsthat can be chosen to analyze police-citizencontact data rely on particular data elementsin the forms that officers complete. This isanother reason for comprehensive, earlyplanning.

Second, we strongly advise that, in iden-tifying which activities a jurisdiction willtarget for data collection, the decision makersselect all traffic stops, all vehicle stops,and/or all detentions and not a subset ofany of these categories as defined by theiroutcomes.3

Some agencies (indeed, some states) arecollecting and analyzing data only from thetraffic stops that result in citations. (That is,instead of collecting and analyzing data fromall traffic stops, these jurisdictions are fo-cusing on a subset of traffic stops as definedby the outcome, a citation.) This commonpractice is convenient because it does not addpaperwork for the officers. They can useexisting, albeit possibly modified, forms. Butthe practice is not recommended. The re-sulting data exclude stops by police thatmay be at heightened risk of being raciallymotivated. A data collection system based oncitation stops alone excludes stops of law-violating drivers who should have receiveda citation but did not, and it may include law-abiding drivers who should not have beenstopped in the first place. These drivers—thefortunate drivers and the illegitimatelystopped drivers—could have been “selected”by police based on the drivers’ race/ethnicity.By excluding drivers who do not receive cita-tions, a jurisdiction severely jeopardizes itsability to assess the existence of racially bi-ased policing, regardless of the strength of thebenchmark used. The researcher could, withthese limited data, identify bias where noneexists or conclude there is no bias when, infact, there is.

This faulty methodology (the limitation ofdata to traffic stops that result in citations) is

3 We use the term “vehicle stop” to denote any stopmade by police of a person in a vehicle; we use “trafficstop” to denote a vehicle stop the stated purpose ofwhich is to respond to a violation of traffic laws(including codes related to quality/maintenance of

vehicles). We use “investigative (vehicle) stop” to de-note police stops of people in vehicles when there is atleast reasonable suspicion of criminal activity. Theterm “detentions” includes both vehicle and pedestrianstops.

Getting Started 23

analogous to assessing the impact of race onprison sentences by focusing only on thosewho are in prison. For example, by exam-ining only the racial makeup of the prisonpopulation and comparing length of prisonsentences across races, a jurisdiction will beunable to reach sound conclusions. It mustalso assess whether or not there are racial dif-ferences with regard to who gets sentenced toprison (versus sentenced to jail or to proba-tion, for example).

If a jurisdiction is collecting data only onsubsets of stops, the report released to thepublic needs to include a strongly statedcaveat regarding the stops that are excludedfrom its research. This limitation on the dataconcerning who is stopped will also affect theanalysis of poststop activities and outcomes.This is because some people who werestopped by police—some of whom weresearched and maybe even detained for longperiods of time—will not be included in thedata set being analyzed.

INVOLVING RESIDENTS ANDPOLICE PERSONNEL IN PLANNINGDATA COLLECTION AND ANALYSIS

It is advantageous for jurisdictions to involveresidents and a cross-section of law enforce-ment agency employees in planning how thedata will be collected and analyzed. (Re-garding the latter, we note that even if a juris-diction did not involve residents and policein planning the data collection system, itcould still involve them in discussions aboutthe data’s analysis and interpretation.)

Police personnel—particularly line per-sonnel and representatives of labor associa-tions—can bring valuable information andan important perspective to the table. Theseagency representatives have a critical stake inensuring a high-quality initiative, and theyshould have the opportunity to raise any oftheir concerns about the integrity and fair-ness of the data collection and analysissystem. Employees’ involvement can also

facilitate “buy in” by the line officers uponwhom the agency will rely to collect the data.

The involvement of residents (particu-larly minority residents) in data collectionplanning can improve police-citizen rela-tions, enhance the credibility of the researchefforts, and increase the likelihood that thewhole community will view the outcome aslegitimate. Involving jurisdiction residents indiscussions regarding data analysis/interpre-tation has the additional advantage of edu-cating a core group within the communityabout the complexities and constraints of theprocess. These residents can serve as impor-tant voices affirming the integrity of theanalysis and the sound interpretation of theresults when reports are released to thepublic.

In the interest of responsible social sci-ence, the caveats associated with variousbenchmarking methods should be includedin jurisdiction reports. The caveats shouldconvey why the results may not provide de-finitive proof of racially biased policing or itsabsence in the jurisdiction. Coming onlyfrom the police department spokesperson,these caveats may be interpreted by skepticalresidents as defensive excuses for why resultsshowing disparity (if they do) are not proof ofracial bias. Although the use of independentsocial scientists to conduct analyses will addcredibility to these caveats, the additionalvoices of respected residents who understandthe methodological constraints will increasethe likelihood that the results and the conclu-sions drawn from them will be viewed as le-gitimate by the general public and the media.“If the community understands benchmarksand the variables that skew aggregate datathere is less likelihood the information willbe misinterpreted and misused,” writesMcMahon et al. (2002, 94). One way to makesure residents understand data analyses is toset up a local task force on racial profiling oran advisory committee.


As recommended in PERF’s first report onthe topic of racially biased policing, thesetask forces should be composed of fifteen totwenty-five people with representatives fromboth the department and the community(Fridell et al. 2001, Chap. 7). In selectingcommunity members, decision makersshould focus on those people who are mostconcerned about racial bias by police. Thetask force should include representativesfrom the jurisdiction’s various minoritygroups and representatives from civil rightsgroups. Consideration should be given tomedia representatives as well because theseprofessionals will be in the important posi-tion of conveying the results to jurisdictionresidents. Police personnel selected for thetask force should represent all departmentallevels, particularly patrol.

Citizens and police should be involvedbecause they can contribute information ofunique value in planning the data analysesand interpreting the results. What they knowabout the jurisdiction’s characteristics, resi-dents, and police activities can be of greathelp to the researchers charged with actuallyimplementing the analysis plan. For in-stance, their knowledge of jurisdiction roadsmay be helpful to a researcher trying tochoose representative intersections where ob-servers will document the race/ethnicity ofdrivers. (See discussion of the observationmethod of benchmarking in Chapter 5.) Ortheir knowledge that a particular downtownentertainment area with a large number ofminority residents draws many white subur-banites on Saturday nights because of its en-tertainment venues can help a researcherinterpret the results for that area.

PARTNERING WITHSOCIAL SCIENTISTS

If resources allow, a jurisdiction should con-sider obtaining the assistance of independentsocial scientists for analyzing its police-

citizen contact data. There are two major rea-sons for partnering with social scientists:

l

Partnering with an individual or ateam external to the law enforcementagency (and independent of otherstakeholders) can add credibility tothe process and thus to the results.

l

The skills of trained social scientistscan supplement the research skills/resources of law enforcement agenciesand/or stakeholder groups.

Data collection to assess racially biasedpolicing is a social science research endeavorand a political endeavor. Thus, a law en-forcement agency that chooses to collect ve-hicle stop data, or is mandated to do so, mustattend to both social science and political ob-jectives in developing and implementing ananalysis plan. An agency could use internalstaff to conduct a high-quality analysis butlose in the political arena because the juris-diction’s residents did not consider the inter-nally conducted analysis to be credible.

Many law enforcement agencies (espe-cially small and medium-size ones) do nothave the in-house expertise to analyze and in-terpret police-citizen contact data. A socialscience partner may be essential to supple-ment agency resources and perform thesefunctions. The analyst(s) should be trained insocial science methods and understand (thatis, should have demonstrated knowledge of)the specific issues associated with analyzingpolice-citizen contact data (Fridell et al. 2001,Chap. 8). Ideally, this “demonstrated knowl-edge” would come from having conductedsimilar analyses for other jurisdictions. Ca-pable analysts are most likely to be associatedwith a college or university or with an inde-pendent research firm. The individual socialscientist or the research team will play amajor role in educating jurisdiction residentsabout the benchmarking methods that can beused for analysis and the strengths and weak-nesses of each.

Getting Started 25

Importantly, the social scientists become“partners” with the agency or, preferably, withthe jurisdiction task force in the data collec-tion/analysis effort. They are not just handedthe data to analyze as they see fit in the pri-vacy of their university or agency offices.The analysis plan should be agreed upon byall parties, and the social scientists shouldcommunicate with their agency and/or taskforce partners throughout their work. The re-searchers should share preliminary results,soliciting perspectives from their police andresident partners who will have superiorknowledge regarding local conditions thatmay be pertinent to the interpretation of thedata.

SELECTING BENCHMARKSChapter 5 describes the various benchmarksthat law enforcement agencies and their part-ners can use to analyze and interpret vehiclestop data. These benchmarks vary consider-ably in terms of their ability to address the al-ternative hypotheses discussed in Chapter 2.In deciding which benchmark(s) to use, deci-sion makers should consider the followingfactors: the level of measurement precisionthey desire, the financial and personnel re-sources that are available, the data elementsthat must be collected, and the availability ofother data that may be required for using aparticular benchmark.

Level of MeasurementPrecision DesiredThe higher the quality of the benchmark, thegreater the ability of the researcher to“measure” racially biased policing and drawconclusions from the data. High-qualityanalysis can provide meaningful informationnot only on whether a problem is indicated,but also on the nature of the problem and thespecifics of its manifestation (in terms of par-ticular geographic areas, shifts, or officers).Imperfect data, however, can still provide asolid base for constructive dialogue between

police and other stakeholders. Resultsshowing “disparity” that cannot be linked toa particular “cause” (such as bias) can stilllead to a meaningful discussion of possiblecauses and desirable reforms. Importantly,these discussions can lead to the collection ofother forms of “data,” including that whichcomes from an open and frank sharing of con-cerns by citizens.

The institution conducting the analysisneed not pick one of the most precisemethodologies (coming as these do with gen-erally higher complications and sometimeshigher costs) in order to make its data collec-tion system successful and constructive.The keys to success for a jurisdiction pickinga benchmark are (1) responsible interpreta-tion and (2) constructive discussion amongstakeholders concerning the weaknesses ofthe benchmark that is selected.

For each benchmark described in Chapter5, we provide information related to itsstrengths and weaknesses. Because the extentto which each benchmark addresses the alter-native hypotheses will determine the legiti-macy of conclusions about police bias,reports must include information on the al-ternative hypotheses to ensure responsibleinterpretation of the data. This means thatthe jurisdiction’s report must include infor-mation on whether and how the bench-marking method takes into account drivingquantity, quality, and location—factors otherthan bias that can explain stopping decisionsby police.

In short, what can be known about thepossibility of police bias in a jurisdiction de-pends on the benchmarking method that ischosen. Because of the difficulty of addressingall of the alternatives to the bias hypothesis,jurisdictions may not be able to pinpointcause of disparity. Nevertheless, the analysisof police-citizen contact data can yield verypositive fruit. Commenting on the value ofpolice-citizen contact data for facilitatingpolice-citizen dialogue, Farrell, McDevitt, and


Buerger (2002, 365) report: “The most effec-tive and productive use of racial profilingdata is not its ability to determine if racialprofiling exists but rather its ability to pro-vide concrete information to ground police-community discussions about patterns ofstops, searches, and arrests throughout localcommunities." This important dialogue is amajor focus of Chapter 8, “Using the Resultsfor Reform.”

Required Agency ResourcesIn selecting a benchmark for analyzing po-lice-citizen contact data, an agency or juris-diction should consider not only the level ofmeasurement precision it desires but also theresources it has available. Not surprisingly,the most effective benchmarks usually (butnot necessarily) require the most resources interms of finances and personnel. A jurisdic-tion will want to select the most effectivemethod given its resources and objectives.4

An important responsibility of stake-holders is to ensure that the police depart-ment or other entity responsible for theanalyses is provided with sufficient re-sources. A number of concerned stake-holders across the nation (most notably localand state legislators) have mandated that lawenforcement agencies collect and analyze ve-hicle stop data, but they have not providedthe resources to ensure quality processes andproducts. Without appropriate resources, ju-risdictions cannot conduct high-qualityanalyses. Jurisdiction stakeholders who pushfor data collection should also push for re-sources to fund the analyses of those data.

Data ElementsThe use of some benchmarks is dependent onthe inclusion of particular elements on thedata collection form completed by police offi-cers. If the jurisdiction is in the early stagesof developing the data collection protocol, de-cisions regarding how to analyze/interpret thedata should be made in conjunction with de-cisions about the content of the form (that is,what data elements to include). If a jurisdic-tion has already developed the form, decisionmakers will need to ensure that the data re-quested on the form match the data that areneeded to conduct the benchmarking methodselected. For example, as noted in Chapter 5,some jurisdictions have compared the demo-graphic profiles of drivers stopped forspeeding by police unaided by radar to thedemographic profiles of drivers stopped be-cause of radar measurements of their speed.(The radar stops are conducted in a mannerso that the radar operator cannot discern thedriver’s race/ethnicity.) To compare these twosets of profiles, the jurisdiction must be ableto identify, from data on the forms, whichstops were conducted with and withoutradar. This is only one example of the neces-sity of advance planning.

For all benchmarking methods we advo-cate analyses of jurisdiction-level data withinspecific geographic subareas.5 Therefore, thelocation of the stop is an important dataelement to include on the police-citizencontact data form. For purposes of reviewingand monitoring data for quality, a uniqueidentifier (number) on the form also ishelpful. Most advantageous is an incidentnumber or similar identifier that corresponds

4 We do not have reliable information regarding thecosts that are associated with the various benchmarkingmethods. Many jurisdictions seeking to hire outsideanalysts issue requests for proposals and then reviewthe proposals. In the review they balance the strengthof the methodology against the resources required toconduct the analysis.

5 Analyzing data within subareas of jurisdictions (forinstance, counties, municipalities) is unwieldy for theresearcher who is charged with analyzing state-widedata.

Getting Started 27

to information about the event that is con-tained in other data sets, such as computer-aided-dispatch (CAD) data and citation data.

The Availability of Other DataSome benchmarking methods are dependentupon the availability of information from out-side sources. Jurisdictions should make surethey can get the data a particular benchmarkrequires before going full steam ahead withefforts to gather police-citizen contact data.

For example, benchmarking with blindenforcement mechanisms (say, enforcementcameras) is a method that would be availableonly to jurisdictions that (1) have enforce-ment cameras in place at controlled intersec-tions to detect and ticket red-light violators orspeeders and (2) are in states that haveracial/ethnic information for owners of regis-tered vehicles. Clearly, a jurisdiction thatchooses this benchmarking method is relianton data from a source outside the law en-forcement agency. With the data provided bythe cameras, the jurisdiction can compare theracial/ethnic profile of drivers who are identi-fied as traffic violators by enforcement cam-eras to the racial/ethnic profile of drivers whoare identified as violators by officers on patrolin the same area as the cameras. The cameraprovides a photo of the license plate number,which enables the jurisdiction to determinethe race/ethnicity of the vehicle owner fromDMV data. (See Chapter 5 for a more detailedexplanation of this benchmarking method.)

Other ConsiderationsA jurisdiction may decide to use multiplebenchmarks. For example, it might imple-ment “internal” benchmarking and some “ex-ternal” method as well. Internal bench-marking is a strong benchmark for identifyingwhich police officers or units may be stop-ping minorities at higher rates than their“similarly situated” counterpart officers orunits. A drawback to internal benchmarking,however, is that it only compares parts of the

law enforcement agency to itself. For thisreason, the jurisdiction might choose—in ad-dition—to compare the agency’s performanceto some outside benchmark, such as that pro-vided by the blind versus not-blind enforce-ment method, or the observation method.Thus, a jurisdiction might implement bothinternal benchmarking and some externalmethod as well.

A jurisdiction might also decide to imple-ment a relatively simple benchmark (for ex-ample, benchmarking with adjusted censusdata) in all the subareas of its jurisdiction andthen invest in a more complicated and moreeffective benchmark (for example, the obser-vation methodology) in those subareas identi-fied by the simpler benchmark as having thegreatest racial/ethnic disparities.

INFORMING THE PUBLIC OFDATA COLLECTION EFFORTS

Some law enforcement executives, when an-nouncing the agency’s data collection efforts,have referred to the initiative as an opportu-nity to “prove” that policing in their jurisdic-tion is not racially biased. Such a predictionof research results is inappropriate. While aparticular executive might be justified inhaving confidence that racially biasedpolicing is neither systematic nor widespreadwithin his or her jurisdiction, the executive isnaïve to claim absolutely that it never occurs.Such a statement is almost certain to offendracial/ethnic minorities who perceive other-wise. Our society has serious racial/ethnicbiases, and the police profession—like everyother profession—hires from a populationwith these prejudices. Even in a departmentin which racial bias is neither systematic norwidespread, it is likely that biased decisionsoccur in some places, at some times, by someindividual officers. Finally, such a strongclaim (the police executive’s use of the word“prove”) conveys to the public that police-citizen contact data can provide definitiveanswers—which they cannot. As is true of


social science in general, even strongmethods of benchmarking will not providedefinitive proof of the existence or lack ofracially biased policing.

Although data collection as a response toracially biased policing has had importantbenefits, one side effect has been negative:the inherent implication that some agenciesare “guilty” of racial bias and others are “in-nocent.” Equally unfortunate is the relatedimplication that the “guilty” agencies are theonly ones that should implement reforms. Infact, all agencies committed to democraticpolicing, not only agencies supposedly“proven guilty” of bias through data collec-tion, can make progress on this longstandingissue; all agencies can implement actionplans to help them move closer to the ideal ofbias-free policing by every officer. Because ofthe inability to provide definitive answersabout the existence of racial bias in a jurisdic-tion and because all agencies can makeprogress on this issue, data collection andanalysis should not be viewed by law en-forcement executives and other stakeholdersas a pass-fail test but rather as a means ofidentifying priorities for change.

CONCLUSIONThis chapter reviewed important considera-tions for jurisdictions that are getting startedwith data collection. It offered concrete sug-gestions that will help jurisdictions develop auseful form for recording police-citizen con-tact data. It also provided guidelines for de-ciding which types of activities to target fordata collection. We encourage the involve-ment of residents and police personnel fromall levels in making decisions regarding thedata collection system, and we explain thecircumstances in which jurisdictions mightwant to involve independent social scientists.

The selection of a benchmarking methodshould be based on several factors: the levelof measurement precision desired, the finan-cial and personnel resources of the jurisdic-tion, the data elements specified on the formfor recording police-citizen contacts, and theavailability of other data. A police executiveannouncing data collection plans to thepublic should not vow that the initiative will“prove innocence” before the fact. Like so-ciety at large, an agency is rarely bias free.Neither should that agency executive, andthose residents and other stakeholders part-nering with the agency, await the results ofdata collection—whatever they might be—toimplement reforms. It is never too soon toaddress racially biased policing or percep-tions of its practice.

IVData Analysis Guidelines forAll Benchmarking Methods

Researchers, regardless of their bench-marking method for evaluating whetherpolicing is racially biased in a jurisdiction,should follow certain guidelines when ana-lyzing data on police-citizen contacts. Thischapter summarizes the guidelines discussedin much more depth in By the Numbers(Fridell 2004, Chapter 4).

Data that have been collected from offi-cers should be checked for quality by the re-searcher or under the supervision of theresearcher. This is an important first step inany type of social science research and notunique to the analysis of police-citizen con-tact data, but it is a step that has been over-looked in many jurisdictions. In this chapterwe also discuss “reference periods,” thelength of time that agencies should collectdata before they begin analyzing it. Thechapter explains why it is advisable for re-searchers to analyze portions or “subsets” ofthe full data set. Subsets based on the type ofstop (proactive or reactive), whether the of-ficer could discern the driver’s race/ethnicitybefore the stop, and the geographic locationof the stop are recommended. The final sec-tion of the chapter describes necessary ad-justments for comparing the stop data and the

benchmarking data in any analysis, or whatwe call “matching the numerator and the de-nominator.”

REVIEW OF DATA QUALITYQuality data are a prerequisite for quality re-search. For accurate results, social scientistsin any research endeavor carefully reviewtheir data to check for and, if possible, correcterrors before analyzing it. Once data collec-tion is under way, researchers who are at-tempting to measure racial bias should“audit” the incoming data from officers forquality. Even if the department has been col-lecting data for a while, researchers are stilladvised to implement a data review and mon-itoring system.

Although there is no cost-effective way toensure that the data are 100 percent accurate,researchers can use various auditing methodsto improve the quality of the data. These au-dits have two objectives: (1) to ascertainwhether line personnel in the police depart-ment are submitting data collection forms forall stops targeted for data collection (for ex-ample, all vehicle stops or all traffic stops)and (2) to ensure that officers are filling outthe forms fully and accurately.1

1 As noted in Chapter 3, not all agencies use paperforms to collect their data. Some officers submit data

by using handheld or in-car computers, or they relay in-formation on vehicle stops over the radio.


To achieve the first objective, researchersshould try to identify a second source of datathat tracks some or all of the stops that aretargeted for data collection. That secondsource of data may be “computer aided dis-patch” (CAD) data, citation data, writtenwarning data, videotapes, or other depart-mental data. Researchers then can compareaggregate totals from the two data sets. Forexample, researchers could check whetherthere are as many police-citizen contactforms with citation dispositions as there arecitation incidents in the citation database.Researchers also can compare the two datasources on a stop-by-stop basis. The lattermethod is preferable because it has the addedadvantage of identifying the source of anyproblems and can lead to interventions to im-prove data quality.

Agencies will not be able to compare twodata sources on a stop-by-stop basis unlessthey institute at the start of the data collec-tion system a mechanism for linking the stopdata to secondary sources. (An example ofthis would be including a citation number onthe vehicle stop form.) Most agencies, how-ever, will have second sources of data to im-plement the first auditing method describedabove. For instance, most will be able to com-pare the number of citations given by policeas recorded in the vehicle contact data to an-other source of citation information.

In addition to checking that forms arebeing submitted for all of the targeted stops,researchers should review the data to detectmissing or potentially erroneous data. Thisreview is particularly important during thefirst two months of data collection. If this re-view identifies significant amounts ofmissing data for particular variables or nu-merous apparent errors on forms, the agencyshould implement, early in the data collec-tion process, certain corrective measures (forexample, a remedial training program to en-sure that officers know how to fill out the

forms and understand the importance offilling out the forms correctly).

It is impossible for researchers to detectall errors merely by reviewing the data thathave been submitted. For example, a reviewof the data is unlikely to detect that the cor-rect disposition of a traffic stop was awarning when the officer erroneously indi-cates on the form that a citation was given.However, a review is still worthwhile becauseit will improve the quality of the data, whichin turn will improve the quality of the re-search results.

REFERENCE PERIOD FORANALYSIS OF DATA

A key question for many social sciencestudies is “How much data should be col-lected before analysis begins?” We recom-mend that analysis be based upon the stopsoccurring within a full twelve-month period,if feasible. This reference period length willlessen the impact on the data of specialevents or circumstances, it will eliminate sea-sonal effects (since all seasons will be in-cluded), and it will increase the reliability ofthe data. A twelve-month reference period,however, may not be economically feasible orpolitically viable. Regarding the latter, resi-dents may not expect to wait more than oneyear for the results of the analysis. If re-searchers choose a reference period of lessthan one year (for example, six months), theirreport to the public should include a caveatthat the results do not necessarily generalizeto the rest of the year for which data were notanalyzed.

It is advisable to delay the start of the ref-erence period for the analysis until officershave become accustomed to the data collec-tion process. (That is, the first one or twomonths of data collection should not be in-cluded in the analysis.) As noted above,these first few months of data should be re-viewed to identify problems (such as large

Data Analysis Guidelines for All Benchmarking Methods 31

amounts of missing data on particular vari-ables), and these problems should be re-solved through communications with officersor other retraining. Once the problems ap-pear to be resolved, the reference periodshould begin.

REASONS FOR ANALYZINGSUBSETS OF DATA

For many reasons explained in this section,researchers should analyze subsets of the po-lice-citizen contact data rather than thewhole data set. Below we discuss subsetsbased on (1) whether stops are proactive orreactive, (2) whether the officer could discernthe driver’s race/ethnicity, (3) geographic lo-cation of the stops, and (4) whether the stopsare for traffic violations or for the purpose ofinvestigating crime.

Type of Stop: Proactive or Reactive Researchers analyze police-citizen contactdata to try to find out whether or not indi-vidual officers are making decisions to stopdrivers based on racial/ethnic bias or basedonly on legitimate factors that should affectthe selection of drivers they stop. Re-searchers can evaluate whether bias was afactor only on stop decisions where the policehad a choice (proactive stops). On reactivestops by police (that is, stops in response to atraffic accident or stops in response to anorder to stop drivers at a highway check-point), police have less discretion (often nodiscretion) in the selection of “who isstopped.” Those reactive decisions to stopdrivers are unlikely to be influenced by bias.

Law enforcement agencies that have de-signed their data collection process to targetonly proactive stops do not need to whittledown their data set. Agencies, however, thatare mandated to collect or voluntarily collectdata on proactive and reactive stops shouldinclude only proactive stops in the data givento researchers for analysis—if the agenciesare able to separate the two groups of stopsbased on information provided on the forms.2

The ability to create this subset of data is de-pendent on the inclusion of information onthe form regarding the type of stop conductedby the officer.

Prestop Observability of theDriver’s Race/Ethnicity“Was the driver’s race/ethnicity observable bypolice before the stop?” The answer to thisquestion has significant relevance to an as-sessment of whether or not stopping deci-sions are based on bias. An officer whocannot discern the racial/ethnic characteris-tics of a driver cannot make a (biased) deci-sion based on those characteristics.Therefore, it makes sense to exclude those in-cidents in which the officers could not dis-cern (at the time the decision was made tostop the vehicle) the driver’s race/ethnicity.3

The decision, however, to exclude data forstops for which officers said they could notdiscern the driver’s racial/ethnic characteris-tics can have negative effects, political as wellas statistical. In some jurisdictions, residentshave questioned the inclusion of this variable(regarding whether the officer can discerndriver characteristics) on police-citizen

2 Both proactive and reactive stops should be includedin the analysis of poststop variables (for example, lengthof stop, whether a search is conducted). Although offi-cers had little discretion deciding whom to stop in reac-tive situations, they have considerable discretion indeciding how to proceed once the stop is made.

3 Both “observable characteristics” stops and “not ob-servable characteristics” stops (like proactive and reac-tive stops) should be included in the analysis ofpoststop factors.


contact data forms because they doubt the va-lidity of officers’ responses. Those who mis-trust the data submitted for this variable arelikely to be skeptical of a decision to excludeall data for stops with “not observable charac-teristics.” Another potential drawback is thatthe exclusion of stops for which officers reportthat they could not discern the race/ethnicityof the driver may reduce the size of the dataset dramatically. We recommend that, if polit-ically and numerically feasible, researchersinclude in their analysis of data regarding“who is stopped” only those stops for whichthe driver’s racial/ethnic characteristics canbe discerned. The ability to create this subsetrequires an element on the form soliciting in-formation from the officer about his or herability to discern demographic characteristics.

Geographic Location of StopIf possible, researchers should create subsetsof data based on type of stop (proactive stopsonly) and on whether the officer could ob-serve the driver’s race/ethnicity before thestop was made (“observable characteristics”stops only). A third subset of data is relatedto the geographic location of the stop.

Because it is likely that racial/ethnic groupsare not equally represented as drivers on juris-diction roads where stopping activity by policeis high (see Chapter 2, Alternative Hypothesis4), researchers should analyze data for geo-graphic subareas of the jurisdiction.

To illustrate our point we use censusbenchmarking as an example. We recom-mend that researchers not compare the racial/ethnic profile of all drivers stopped in the ju-risdiction to the racial/ethnic profile of alldriving-age residents in the jurisdiction. In-stead it is preferable to compare stop data andbenchmark data within smaller geographic

areas of the jurisdiction. These subareas be-come subsets of the analyses.

Subarea analysis “controls for” thevolume of stopping activity by police. Someareas may have many more stops per capitathan others because these areas have a highrate of calls for service or a large volumeof accidents. As suggested earlier, the race/ethnicity of the resident population in thesehigh-activity and low-activity areas may varyconsiderably. Because there may be greatervehicle stop activity in Area A than in Area B,researchers should compare the racial/ethnicprofile of drivers stopped in Area A to theracial/ethnic profile of drivers in the Area Abenchmark group. Similarly, the demo-graphics of drivers stopped within Area Bshould be compared to the demographics ofthe Area B benchmark group.

Type of Stop: Traffic or Investigative The term “traffic stop” refers to a vehicle stopthe stated purpose of which is to respond to aviolation of traffic laws, including codes re-lated to quality/maintenance of the vehicle;the term “investigative stop”—in a vehiclecontext—denotes police stops of people in ve-hicles when there is a reasonable suspicion ofcriminal activity. We now consider whetheragencies collecting data on all vehicle stopsshould analyze traffic and investigative stopstogether as a group or separately.4

In answering this question, we distin-guish between what is theoretically appro-priate and what is practical in terms ofmeasurement capabilities. At a theoreticallevel, traffic stops and investigative stopsshould be analyzed separately and alternativehypotheses developed for both categories.The factors that put a person at risk of beinglegitimately stopped by police for a traffic

4 This discussion is not relevant to agencies that arecollecting data only on traffic stops.

Data Analysis Guidelines for All Benchmarking Methods 33

violation are different from the factors thatput a person at risk of being legitimatelystopped by police for purposes of investi-gating criminal activity.

The police-citizen contact data that arecollected, however, do not enable agencies todistinguish between the stops made for thepurpose of enforcing traffic laws and thosemade for the purpose of investigating crime.Police can and do stop vehicles on the basisof legitimate traffic violations but for the pur-pose of investigating crime. In most agenciesthese “pretext stops”—as they are called—will be coded as traffic stops, even though attheir core they are investigative stops. Be-cause researchers cannot successfully distin-guish the two types of stops, we recommendthat all vehicle stops—those for traffic and forinvestigative purposes—be analyzed together.This is a necessary and practical means of re-solving the research problem just described.It does, however, reduce the precision of theanalyses.

Although pretext stops hinder re-searchers’ ability to determine which stopsthat were coded in the data as “traffic stops”were, in fact, “traffic stops” and not “inves-tigative stops,” the converse is not true. Re-searchers can be confident that the stopscoded as investigative stops are truly inves-tigative stops and not a “ruse” for catching atraffic violator. For this reason, after re-searchers conduct their major analysis usingall traffic and investigative stops (see discus-sion above), they can conduct an additionalanalysis of only investigative stops using ap-propriate crime-related benchmarks (seeChapter 5).

MATCHING THE NUMERATORAND THE DENOMINATOR

Social scientists analyzing police-citizen con-tact data to measure racially biased policingemphasize the importance of “matching thenumerator and the denominator.” In theirspecialized lingo, the “numerator” refers to

the data collected on stops by the police, andthe “denominator” refers to the data collectedto produce the benchmark or comparisondata. To “match the numerator and the de-nominator” means the researcher should ad-just the stop data to correspond to anylimiting parameters of the benchmark or viceversa.

For example, the researcher bench-marking with census data adjusted for ve-hicle ownership should include in his or heranalysis only the stops by police involvingdrivers who are residents of the jurisdiction.In this method of analysis, the researcher ad-justs the census data on the demographics ofresidents to take into consideration who,among those residents, owns a vehicle. Thatis, the researcher compares the racial/ethnicprofile of the people stopped by police to theracial/ethnic profile of people who not onlylive in the jurisdiction but who also have ac-cess to vehicles, according to the U.S.Census. The “numerator” is the stop data col-lected by police, and the “denominator” is theadjusted U.S. Census data. The denominatorin this situation is restricted: it only includespeople who live inside the jurisdiction. Thisparameter on the denominator must be ap-plied to the numerator data. That is, the re-searcher must compare the census data onlyto the stops by police of jurisdiction residents.The researcher must select out of the numer-ator data all of the stops of drivers who do notlive inside the jurisdiction. Nonresidents ofthe jurisdiction are excluded from the de-nominator, and therefore they must be simi-larly excluded from the numerator.

Sometimes, however, data must be ex-cluded from the denominator. For example,with adjusted census benchmarking, there isan inherent limitation on the numerator (thestop data); only people of driving age will beincluded. The drivers stopped by police willusually be of legal driving age. Because onlypeople of driving age will be represented inthe numerator, the researcher must limit the


denominator data to people of driving age.Thus, in the example of census bench-marking, the researcher will not calculate therace/ethnicity of all residents of the jurisdic-tion but only of those residents who are ofdriving age (for example, age 15 and older).

Regardless of the benchmarking method,researchers must match the numerator dataand the denominator data: the parametersthat apply to one must be applied to the other.Let us consider how that works in the obser-vation method. Placed at various locationsin the jurisdiction—often at intersections,observers count drivers in various race/ethnicity categories. The result is a racial/ethnic profile of drivers around that intersec-tion (the denominator data). Therefore, thenumerator data (drivers stopped by police)must be limited to the same intersection. Toconduct observation benchmarking, the re-searcher will compare the demographics ofthe people who are observed driving throughIntersection A to the demographics of thepeople stopped by police in and around Inter-section A. This type of analysis will be con-ducted separately for each intersection understudy in the jurisdiction.

“Matching the numerator and the denom-inator” applies to the time period duringwhich the data are collected as well. In thisobservation methodology example, if the ob-servation data are collected during Januarythrough May 2002, the analysis will involve

only those police stops that occurred duringthat same (or reasonably similar) time period.If the researchers collected observation dataonly during daylight hours because of visi-bility issues, then the analysis should includein the numerator only those stops that oc-curred during daylight hours.

CONCLUSIONThe preceding guidelines for data analysisapply to all benchmarking methods. Re-searchers need to review data quality, choosea reference period, analyze subsets of data,and match the numerator data and the de-nominator data. Some of these guidelinesalso apply to the analyses of poststop data(see Chapter 6).

Jurisdictions that follow these guidelineswill improve the validity and usefulness oftheir findings. They also will increase theircosts. It is important for stakeholders to beaware that some of the steps recommendedabove will increase significantly the labor in-volved in the analysis of the vehicle stopdata. For instance, conducting the recom-mended subarea analyses instead of a single,jurisdiction-wide analysis multiplies severaltimes over the time needed for researchers tocomplete their work. Stakeholders aspiringto high-quality data and meaningful analysesneed to ensure that there are sufficient re-sources available to finish well what theybegan.

VMethods for Benchmarking Stop Data

A number of benchmarking methods havebeen created to help jurisdictions analyzeand interpret vehicle stop data collected tomeasure racial bias. This chapter reviews thefollowing methods and assesses theirstrengths and weaknesses: benchmarkingwith adjusted census data, benchmarkingwith DMV data, benchmarking with datafrom “blind” enforcement mechanisms,benchmarking with data for matched officersor matched groups of officers, observationbenchmarking, and several other bench-marking methods and tools. For more de-tailed information on how to implement eachof these methods, see By the Numbers: AGuide for Analyzing Race Data from VehicleStops (Fridell 2004), Chapters 5 through 10.

Before explaining the first method, we re-view several general principles introduced inChapter 2, “The Benchmarking Challenge.”In any benchmarking method, the researchercompares the racial/ethnic breakdown ofdrivers stopped by police (the stop data) tothe racial/ethnic breakdown of people at riskof being stopped by police, assuming nobias (the benchmark data). The purpose ofthis comparison is to determine if disparityexists that might indicate racial bias. In cre-ating a benchmark group that representsthe people at risk of being stopped by police,the researcher considers important factors

influencing police decisions to stop someone.These factors are related to driving quantity,driving quality, and driving location. Ideally,the researcher’s benchmark would take intoaccount that people who drive more should bemore at risk of being stopped by police,people who drive poorly should be more atrisk of being stopped by police, and peoplewho drive in locations where stopping ac-tivity by police is high should be more at riskof being stopped by police. These factors un-derlie the alternative hypotheses described inChapter 2.

BENCHMARKING WITHADJUSTED CENSUS DATA

In census benchmarking, law enforcementagencies compare the demographic profile ofdrivers stopped by police to the demographicprofile of jurisdiction residents as measuredby the U.S. Census Bureau. A straight com-parison between the demographics of thesetwo groups is called “unadjusted” censusbenchmarking. The weaknesses of thismethod in ruling out alternative hypotheseswere discussed in Chapter 2. Most jurisdic-tions appear to be benchmarking their police-citizen contact data against unadjustedcensus data. This is because their resourcesare limited, and unadjusted census bench-marking is the simplest and least costly


benchmarking method. That said, we do notrecommend unadjusted census bench-marking. Agencies that must rely on censusmethods should use one of the various adjust-ment techniques described below.

Researchers should adjust the census databy incorporating into their benchmarkingmethod information pertaining to one ormore of the alternative hypotheses (such asquantity of driving). They can do this, for ex-ample, by taking into consideration who,among the residents listed in the census datafor the jurisdiction, has access to a vehicle.People without access to a vehicle are clearlyat less risk of being stopped in vehicles by po-lice than are people with vehicle access.Census benchmarking with this adjustment isa stronger method than unadjusted censusbenchmarking for assessing the nature andextent of racially biased policing.

Adjusting Census Data on JurisdictionResidents to Account for Vehicle AccessResearchers who adjust census data to ac-count for vehicle access are improving thequality of their research by taking into ac-count the alternative hypothesis that racial/ethnic groups are not equally represented asdrivers on jurisdiction roads. To make thisbeneficial adjustment to the census data, theywould subtract from the census populationdata for each racial/ethnic group in the juris-diction the estimated number of peoplewithin each of those groups who do not haveaccess to vehicles. To do this, the researcherwould obtain the census information for thejurisdiction on vehicle-less households byrace and ethnicity.

Figure 5.1 compares a hypothetical racialprofile of people stopped by police in Area Ato the benchmark data produced by adjustedcensus information. There is little racial dis-parity indicated. Caucasians represent 65percent of the drivers stopped by police and67 percent of jurisdiction residents with ac-cess to vehicles. African Americans repre-sent 19 percent of drivers stopped by police

and a corresponding 19 percent of jurisdic-tion residents with access to vehicles.

Another recommended way that re-searchers can adjust census data on the juris-diction is to take into account the drivers onjurisdiction roads who come from neigh-boring jurisdictions.

Adjusting Census Data on JurisdictionResidents to Account for the Influx ofNonresident DriversThe race/ethnicity of drivers on jurisdictionroads may not match the race/ethnicity of ju-risdiction residents because of (1) racial/ethnic differences in driving quantity and/or(2) racial/ethnic differences in the populationof people who do not live in the jurisdictionbut drive in it. By adjusting for vehicle ac-cess, researchers can address indirectly thefirst possibility; by adjusting for the influx ofnonresident drivers, researchers can focus onthe second possibility.

Not all drivers on jurisdiction roads areresidents of that jurisdiction, and the influxof nonresidents can affect the racial/ethnicprofile of drivers “available” to be stopped bypolice.

Consider, for example, a municipal areawith a substantial minority population; duringthe day many Caucasians from the suburbsdrive into this jurisdiction for work. As aresult, the percentage of drivers on jurisdic-tion roads who are Caucasian is higher thanthe percentage that would be predicted to beon jurisdiction roads based on residentialpopulation data alone. Of course, nonresi-dents also might enter a jurisdiction for otherreasons—to shop, go to school, to seek enter-tainment, to travel on to another jurisdiction,or for other reasons.

Because of the influx of nonresidents,the racial/ethnic profile of residents producedby unadjusted census data is likely to bean inaccurate estimate of drivers who couldbe stopped by police, assuming no bias.Therefore, the adjustments described beloware needed.

Methods for Benchmarking Stop Data 37

Several researchers have come up withways to estimate the race/ethnicity of residentand nonresident drivers in a jurisdiction.Here we describe the method developedby Novak (2004). First, Novak looked at thestop data in the jurisdiction (the so-called“target jurisdiction”) to estimate the extentto which nonresidents from various “outsidejurisdictions” were represented on target juris-diction roads. (He was able to do this because

the stop data for the jurisdiction included in-formation on the jurisdiction-of-residence ofthe driver.) With this information, he pro-duced an estimate of the extent to which res-idents and nonresidents drove on jurisdictionroads. Second, he used the census data forthe target jurisdiction and for each of thoseoutside jurisdictions to estimate the racial/ethnic profiles of these resident and non-resident drivers. Finally, he compared the

80%

70%

60%

50%

40%

30%

20%

10%

0% Caucasians

65% 67%

African Americans

19% 19%

Asians

12%9%

Other

4% 5%

Stopped Drivers

Residents (Census Data Adjusted for Vehicle Access)

Figure 5.1. Drivers Stopped by Police in Hypothetical Area Aand Area A Residents with Vehicles, by Race

80%

70%

60%

50%

40%

30%

20%

10%

0% Caucasians

64%

68%

African Americans

22%19%

Hispanics

9%9%

Other Races

5% 4%

Stopped Drivers

All Drivers

Figure 5.2. Drivers Stopped by Police in the Target City and All Drivers(Resident and Nonresident) in the Target City, by Race


racial/ethnic profile of drivers stopped by po-lice in the target jurisdiction to the racial/ethnic profile of all drivers (residents andnonresidents) in the target jurisdiction. Hy-pothetical results are shown in Figure 5.2(previous page).

Drawing Conclusions from the ResultsAdjusted census benchmarking can incorpo-rate two kinds of valuable information relatedto driving quantity: information on vehicleaccess and information on the influx of non-residents into the jurisdiction. This methodaddresses, in part, the alternative hypothesisthat racial/ethnic groups are not equally repre-sented as drivers on jurisdiction roads. Re-searchers conducting adjusted census bench-marking who are able to analyze subareas ofthe jurisdiction incorporate useful informa-tion related to driving location; subareaanalyses enable these researchers to addressthe alternative hypothesis that racial/ethnicgroups are not equally represented as driverson roads where stopping activity by police ishigh.

Another alternative hypothesis—racial/ethnic groups are not equivalent in the natureand extent of their traffic law–violating be-havior—is not addressed, however, by themethod of benchmarking with adjusted censusdata. Driving quality is an important factor in-fluencing police decisions to stop drivers, butit is not taken into account by this method. Forthis reason, researchers benchmarking withadjusted census data cannot draw definitiveconclusions regarding the causal link betweenthe race/ethnicity of drivers and stopping be-havior by police. Because of its importance,this statement bears repeating: researchers(and the law enforcement agencies and other

stakeholder groups citing the researchers’ re-ports) cannot draw conclusions aboutwhether policing in a jurisdiction is raciallybiased.1

Despite these weaknesses of adjustedcensus benchmarking as a diagnostic tool, re-searchers limited by resources or time mayhave no option other than to use this method.In particular, researchers who are chargedwith analyzing data from all of the jurisdic-tions within a single state or many of themmay have to rely on this method, despite itsweaknesses. The obligation of the researcherin this position is to ensure that the results areconveyed in a responsible fashion. In fact,this obligation also falls to all stakeholders,including concerned residents, civil rightsgroups, local/state policymakers, and themedia. No one interpreting results based onbenchmarking with adjusted census data canlegitimately draw conclusions regarding theexistence or lack of racially biased policing.

BENCHMARKINGWITH DMV DATA

Benchmarking with census data adjusted forvehicle access and benchmarking with datafrom the Department of Motor Vehicles(DMV) are similar methods. In one the re-searcher creates a comparison group based onpeople who live in the jurisdiction and haveaccess to a vehicle. In the other the re-searcher creates a comparison group based onpeople who live in the jurisdiction and havea driver’s license. Like adjusting census datafor vehicle ownership, benchmarking withDMV data produces an indirect measure ofdriving quantity. It accounts, in part, for thealternative hypothesis that racial/ethnic

1 What they can do is mention the disparities or lack ofdisparities shown by the data, and they can referencepossible explanations for the results—using the alterna-tive hypotheses as a guide.


groups are not equally represented as driverson jurisdiction roads.2

Implementing the MethodTo implement this method in its simplestform, researchers restrict their analysis to ju-risdiction residents; they compare theracial/ethnic profile of licensed drivers wholive in the jurisdiction to the racial/ethnicprofile of the jurisdiction residents stoppedby police.3 To implement a preferable andmore sophisticated version of this method, re-searchers conduct subarea analyses and/ortake into account the influx of nonresidents.

Earlier we described how Novak (2004)adjusted census data by measuring the influxof drivers from outside the jurisdiction. Usingstop or citation information and census data,he was able to estimate the racial/ethnic pro-files of residents and nonresidents on juris-diction roads. These same techniques can beapplied to the DMV benchmarking method bysubstituting the driver’s license demographicdata for the census demographic data.

Drawing Conclusions from the ResultsBenchmarking with DMV data, like bench-marking with adjusted census data that takesinto account vehicle ownership, imperfectlyassesses who is driving on jurisdiction roads.The caveats associated with this methodreflect three truths: not everyone with adriver’s license drives, some people driveeven though they do not have a driver’s li-cense, and some jurisdiction residents (par-ticularly students and military personnel)have a driver’s license from another state.Most importantly, having a driver’s license is

a very crude measure of driving quantity—residents of various racial/ethnic groups whohave a driver’s license may drive in differentamounts.

Using DMV data to benchmark police-citizen contact data is similar to using censusdata that have been adjusted for vehicle own-ership. Both benchmarking methods producea proxy measure for driving quantity bytrying to determine who is and who is notdriving on jurisdiction roads. The bench-marking method that uses adjusted censusdata considers a person a driver if the personhas access to a vehicle. The method de-scribed in this section considers a person adriver if the person has a driver’s license.This method will not produce conclusionsregarding the existence or lack of raciallybiased policing in a target jurisdiction.Nonetheless, the results can be valuable asthe basis for discussions between police andcitizens about racially biased policing and theperceptions of its practice. We discuss howthe results can be used to stimulate these dis-cussions in Chapter 8.

BENCHMARKING WITH DATA FROM“BLIND” ENFORCEMENT MECHANISMSLaw enforcement agencies can use “blind”enforcement mechanisms (for example, redlight cameras, radar, air patrols) to createbenchmark data against which they cancompare their data on stops by patrol officers.In this method the racial/ethnic profile oftechnology-selected drivers is compared tothe racial/ethnic profile of human-selecteddrivers (that is, traffic law-violating driversstopped by police). Some agencies compare

2 To use this benchmarking method, a law enforcementagency must be able to obtain from the Department ofMotor Vehicles (DMV) information on the race and/orethnicity of the licensed drivers in the target jurisdic-tion (the jurisdiction being analyzed based on police-citizen contact forms). Additionally, the informationfrom the DMV regarding race and/or ethnicity must be

compatible with the measurement of race and/or eth-nicity on the law enforcement agency’s data collectionform.3 Comparing residents with drivers’ licenses to residentsstopped by police reflects “matching the numerator andthe denominator” as described in Chapter 4.


stops in which officers exercise a high degreeof discretion to low-discretion stops. At leastone jurisdiction benchmarked with “blind”data from a nontechnological source; theycompared “daylight stops” and “darknessstops.” These benchmarking methods also areexplained in this section.

Benchmarking with Data fromRed Light CamerasEnforcement using red light cameras is blindbecause traffic law violators are detected and“ticketed” in a manner that does not allow forthe intrusion of bias. These cameras areplaced at selected intersections that have atraffic light. A driver who runs the red lighttrips the camera, which takes a picture of theviolator’s license plate.

In this benchmarking method researcherscompare the racial/ethnic profile of thedrivers “ticketed” by the camera technologyto the racial/ethnic profile of the driversstopped by police.4 The goal of bench-marking is to create a comparison group ofpeople at risk of being stopped by police,assuming no bias. People “ticketed” by redlight cameras do represent such a group. Ifofficers are as “blind” to race/ethnicity as arethe cameras, the demographic profile of thepeople stopped for red light violations (orcomparable violations) by the officers shouldmatch the demographic profile of the people“ticketed” by the cameras in the same area.If, however, officers are targeting minoritiesfor stops, minorities may compose a largerpercentage of stops by the humans than bythe technology.

Matching the Numerator andthe DenominatorThe camera data can provide researchers withinformation on the race/ethnicity of people vi-olating red light laws in certain locations.5

The researchers, however, must carefullymatch the numerator data and the denomi-nator data (see Chapter 4). Specifically, theymust match the stop data (numerator) to thered light data (denominator) in terms of loca-tion, time, and violations detected.6

This careful matching, while necessary,inevitably narrows the scope of the racial biasassessment. An example will help to conveythis point. To maximize the match of the twogroups, the researcher might use the same in-tersection (for example, Intersection A) to col-lect both camera data and officer stop data.Alternatively, in a near-ideal design, the re-searcher might compare red-light-cameradata from Intersection B to officer stop datafor red light violations at Intersection A. In-tersections A and B would need to be similarin terms of (1) the race/ethnicity of the drivers(this might be more likely if the intersectionsare near each other) and (2) driving behaviorbecause both intersections have the sametype of traffic—residential, not commercial.In this way the researcher has maximized thematch between the stop data and the bench-mark data. As will be explained later, thelaudable rigor of this match comes at a cost:the scope of the analysis is narrowed.

Having maximized the match betweenthe stop data and the benchmark data, re-searchers can begin to conduct the compar-ison. They can compare the racial/ethnicprofile of the people “ticketed” by the red

4 Actually, the person “ticketed” by the camera is theperson to whom the vehicle is registered, not neces-sarily the driver. This is one drawback of this method.5 To benchmark against red-light-camera data, a lawenforcement agency must, of course, have red-light-camera technology in place. It also must be able to

access Department of Motor Vehicle (DMV) data thatcan link the license plate photographed by the camerato the race and/or ethnicity of the owner of the vehicle.6 See Chapter 4 for a detailed explanation of the con-cepts of “numerator” and “denominator.”


light cameras to the racial/ethnic profile ofthe people stopped by police for red light vi-olations in the matched geographic area (seeFigure 5.3). They will conduct these analysesfor each red-light-camera intersection and itsmatched geographic area.

Drawing Conclusions from the ResultsThis benchmarking method has an importantstrength: it can create a comparison group, orbenchmark, that reflects the people at risk ofbeing stopped by police, assuming no bias.This method, however, has several importantdrawbacks.

First, the measure of race/ethnicity withinthe benchmark group is suspect. The bench-mark group is composed of the owners of theviolating vehicles, not necessarily the drivers.Therefore, a law enforcement agency cannotbe sure that it has accurately measured therace/ethnicity of the people “ticketed” by thered light cameras. Second, any assessment ofracially biased policing with this method islimited to certain locations for certain types ofstops. The results in which an agency canhave confidence relate only to the particulartypes of stops studied (red light and equiva-lent violations) and only to the specific inter-sections studied. To generalize from these

“spot checks” to other types of stops/violationsrequires an assumption without validity—namely, that the racial/ethnic profile of peoplewho violate red light laws matches the racial/ethnic profile of people who commit allmoving violations. Similarly, to generalizebeyond the geographic test areas to the entirejurisdiction, a law enforcement agency mustassume that those areas are representative ofall areas of the target jurisdiction. This also isa shaky assumption for a number of reasons,including the likelihood that red light cam-eras are placed at intersections with higherthan average traffic volume, violation be-havior, and/or accidents.

This benchmarking method has ad-dressed the following alternative hypothesesto the extent that it has created a match be-tween the numerator and denominator data:

l


l


l


60%

50%

40%

30%

20%

10%

0%Caucasians African Americans Hispanics Other

3%5%

12%8%

35%32%

50%55% Drivers Stopped by Police

Drivers “Ticketed” by Cameras

Figure 5.3. Drivers Stopped by Police for Red Light and StopSign Violations and Drivers “Ticketed” by Red Light Cameras, by Race/Ethnicity of Drivers (Hypothetical Data)


l


Although a good match in terms ofdriving quantity, driving quality, and drivinglocation has been created between the popu-lation of drivers stopped by patrol officersand the population of drivers ticketed by thered light cameras, this method has not as-sessed racially biased policing for all types ofstops in the entire target jurisdiction—onlyfor certain moving violations that occur intested areas (or in areas where the results canbe reasonably generalized).

Benchmarking with Radar DataRadar enforcement, like red-light-camera en-forcement, can be “blind” to the racial/ethniccharacteristics of traffic law violators.7 Thesame implementation procedures and re-quirements apply to both methods. For ex-ample, the nonradar police stops of vehiclesincluded in the numerator data should be asequivalent as possible to the radar stops thatcomprise the denominator data. The numer-ator and denominator data are already equiv-alent with regard to type of traffic offense—both sets of data include speeders. The re-searcher still would need to produce equiva-lence or a match in terms of the geographiclocations of the stops.

The strength of benchmarking with“blind” enforcement data (whether it be radardata or red-light-camera data) is its potentialto develop a strong match between the bench-mark population and the people at risk ofbeing stopped by patrol officers. Again, to theextent that this match is maximized, the fac-tors related to the four competing hypothesesare addressed. Both the red-light-camera

method and the radar method have an impor-tant limitation: the rigor of the match comesat a cost in terms of scope. Conclusions canbe made about specific areas and about en-forcement of certain traffic laws but not aboutthe target jurisdiction as a whole or enforce-ment of all traffic laws.

Benchmarking with Data fromLow-Discretion StopsRacial/ethnic bias is more likely to manifestitself when officers have discretion in de-ciding whether to stop someone than whenthey have little choice in the matter. When adriver runs a red light in a busy intersection,most officers feel a strong need to respond.This is an example of a low-discretion stop.An officer is likely to respond to all violationsof this kind, and any racial/ethnic biases anofficer might have are not likely to enter intohis or her decision to ticket the red light vio-lator. On the other hand, officers have greatdiscretion in deciding whether to stopsomeone who is going 5 miles per hour overthe speed limit (a violation at the other end ofthe degree-of-discretion continuum). If an of-ficer has biases, they are more likely to influ-ence high-discretion decisions such as thisone.

In an attempt to measure racial bias in ajurisdiction, some law enforcement agencieshave compared high- and low-discretiontraffic stops. They use the low-discretionstops (the denominator) as a benchmark forthe high-discretion stops (the numerator).Agencies adopting this method compare theracial/ethnic profile of the people stopped inhigh-discretion situations to the racial/ethnicprofile of those stopped in low-discretionsituations to see if the latter produces higherproportions of minority drivers.

7 Radar enforcement is not always conducted in amanner that makes it “blind” to the race and ethnicityof drivers. Radar enforcement is not “blind” if the

officer targeting the radar at cars can determine fromhis/her vantage point the race or ethnicity of thedrivers.


This method has a serious shortcoming.Unlike benchmarking with data from “blind”enforcement mechanisms, benchmarking withlow-discretion stops cannot produce a goodmatch in traffic law-violating behavior be-tween the people at risk of being stopped andthe people who are stopped. Recall that thecomparison between technological enforce-ment using red light cameras and enforcementby patrol officers matched stops of red light vi-olators (the denominator) to stops of red lightviolators (the numerator)—comparable of-fenses. But the use of low-discretion stops(the denominator) as a benchmark for high-discretion stops (the numerator) involves acomparison of drivers who commit very dif-ferent violations. Benchmarking with datafrom “blind” enforcement mechanisms (for in-stance, red light cameras, radar) addresses thealternative hypothesis that racial/ethnicgroups are not equivalent in the nature andextent of their traffic law-violating behavior; itdoes this by making driving behavior equiva-lent in the two groups. Benchmarking withdata from low-discretion stops does notachieve equivalence across groups in drivingbehavior.

This method, however, does address twoother alternative hypotheses: racial/ethnicgroups are not equally represented as resi-dents in the jurisdiction and racial/ethnicgroups are not equally represented as driverson jurisdiction roads. A third hypothesis—racial/ethnic groups are not equally repre-sented as drivers on roads where stoppingactivity by police is high—is addressed if theresearchers conduct analyses, as we recom-mend, within subareas of the jurisdiction.

Benchmarking with “Blind” Datafrom a Nontechnological SourceResearchers from The RAND Corporationstudied vehicle stop data collected by theOakland (CA) Police Department, and they de-veloped a benchmarking method that is basedon comparing “blind” and “not blind” stops(Ridgeway, G., K.J. Riley and J. Grogger 2004).The RAND researchers considered variousmethods that would allow them to comparethe stops in which officers could discern race/ethnicity to stops in which officers were“blind” as to the race/ethnicity of the driver.They ultimately decided to use time of day todifferentiate between stops where officers hadgreater and lesser visibility.

The researchers did not simply comparedaytime and nighttime stops because thepeople at risk of being stopped by policeduring the day are different from the peopleat risk of being stopped by police at night.(The demographic makeup of drivers on theroad in any jurisdiction or subarea can varyconsiderably across times of day.) Instead theRAND researchers compared stops during alimited time period (approximately 5 P.M. to9 P.M.) on the assumption that the visibility ofdriver demographics changes during thesehours, but the demographics of the drivingpopulation does not change then signifi-cantly. The numerator data covered stops thatoccurred between 5:19 and sunset (“daylightstops”) and the denominator covered stopsthat occurred between the end of civil twi-light8 and 9:06 P.M. (“darkness stops”). Again,with these two groups they were able to com-pare stops that occurred when presumably of-ficers could see the race/ethnicity of thedrivers in the cars to stops that occurredwhen presumably the officers could not. Theresearchers conducted their analysis on a

8 “Technically, this occurs when the center of the sun is6 degrees below the horizon, but practically it is whenone can see the brightest stars and artificial light is

needed to perform most outside activities” (Ridgeway,G., K.J. Riley and J. Grogger 2004: 40).


subset of stops to increase the strength oftheir comparison; they conducted theiranalysis on only evening stops and not onstops that occurred at other times of day.Once again the greater rigor of the matchcame at the cost of the scope of the analysis.

Drawing Conclusions from the ResultsUsing data obtained from red light camerasand radar, law enforcement agencies cancompare the racial/ethnic profile of tech-nology-selected drivers to the racial/ethnicprofile of human-selected drivers (driversstopped by patrol officers). This comparisonbenchmarks the data on drivers stopped byenforcement methods that are devoid of dis-cretion (the “blind” technology) against thedata on drivers stopped by methods that in-volve the exercise of discretion (stops by pa-trol officers). If officers’ stopping decisionsare made without racial/ethnic bias, then theracial/ethnic profile of the drivers they stopwill match the racial/ethnic profile of thedrivers stopped by the technology. In a vari-ation of this method, a researcher couldbenchmark high discretion stops against lowdiscretion stops or compare groups of stopsthat differ in terms of the prestop observ-ability of a driver’s race/ethnicity.

When implemented in accordance withour recommendations, benchmarking withdata from “blind” enforcement mechanismssuch as red light cameras and radar enables ajurisdiction to conduct a strong assessment ofbiased policing. The results, however, arestrong only for specific locations and for par-ticular types of stops. In other words, therigor of the methodology comes at the cost ofscope.

Benchmarking with data from low-discretion stops or with data from stopswhere officers could not discern racial/ethniccharacteristics has limitations as well. Be-cause the types of stops represented in thenumerators (high discretion stops, stops in

which characteristics are observable) are, ormay be, dissimilar from the types of stops inthe denominator (low discretion stops, stopsin which characteristics are not observable),this method does not address the alternativehypothesis that racial/ethnic groups are notequivalent in the nature and extent of theirtraffic law-violating behavior. Consequently,the bias hypothesis cannot be tested directly.

BENCHMARKING WITHDATA FOR MATCHED OFFICERS ORMATCHED GROUPS OF OFFICERS

Law enforcement agencies can compare stopsby individual officers to stops by other offi-cers, or they can compare stops by a group ofofficers to stops by other groups of officers.These comparisons must be made across“matched” sets of officers or groups of officersto control for the factors reflected in the alter-native hypotheses described in Chapter 2.For instance, an agency might compare theracial/ethnic profile of people stopped by in-dividual patrol officers who work the sameshift in the same precinct. If a particular of-ficer stops proportionately more minority cit-izens than does his or her matched peers,further exploration of this officer’s policingactivities and decisions would be warranted.This method has been referred to by SamuelWalker as “internal benchmarking” (2001,2002, 2003).

To implement internal benchmarking, theagency must be able to link stop data to indi-vidual officers or to groups of officers. Com-paring officers to each other is preferable tocomparing groups of officers. Analysis at theindividual level allows the agency to identifyparticular officers whose stopping activity isdifferent from his or her colleagues’ stoppingactivity and intervene if appropriate. (Someagencies cannot match at the individual of-ficer level because the agency’s stop datacannot be linked to individual officers.)


The Matching ProcessThe strength of this method is directly linkedto the quality of the match between the offi-cers or groups of officers being compared.That is, the researcher wants to maximize thesimilarity among the officers being comparedor among the groups being compared. To as-sess whether Officer A, for example, ismaking decisions to stop vehicles based ondrivers’ race or ethnicity, an agency can com-pare Officer A’s stop data to the stop data ofother officers who are policing essentially thesame population in essentially the same way.The goal is to compare officers who are sim-ilar to one another in terms of the people atrisk of being stopped by them. For instance,officers on the same shift, in the same geo-graphic area, with the same assignmentwould be exposed to a similar population ofdrivers. Because the selected officers policesimilar populations, all of the factors relatedto the alternate hypotheses (driving quantity,driving quality, driving location) are heldconstant. The racial/ethnic profile of driverson the road, as well as the racial/ethnic pro-file of law violators, are roughly equivalentfor these matched officers. Since all of the

factors related to the alternative hypothesesare held constant in this comparison of indi-vidual officers, the racial/ethnic profile of thedrivers they stop should be about the sameunless one officer (or possibly several) ismore inclined to stop drivers of particularracial/ethnic groups than are the others.

Figure 5.4 illustrates benchmarking withdata for matched officers. For nine of the tenmatched officers, the percentage of stops ofminorities is in the range of 13 percent to 21percent. (That is, for these officers, between13 and 21 percent of the drivers they stop areminorities.) The percentage for Officer 8,however, is much higher—37 percent. Thisfinding of disparate results does not provethat the officer is acting in a racially biasedmanner, but it should prompt a review of thepolicing activities of this officer.

An agency unable to link stop data toindividual officers can still implement in-ternal benchmarking if it can identify groupsof officers that are similarly situated. That is,the unit of analysis would be the group notthe individual. The numerator is the aggre-gate racial/ethnic profile of the driversstopped by all of the officers in the group;

Figure 5.4. Matched Officers’ Stops of Minority Drivers (Hypothetical Data)

40%

35%

30%

25%

20%

15%

10%

5%

0%

Officers

16 16

1

18

2

20

3

14

4 5

17

6

20

7

37

8

21

9

15

10

Perc

ent


the denominator, or benchmark, is the racial/ethnic profile of the drivers stopped by thecorresponding comparison groups. That is,the racial/ethnic profile of drivers stopped byGroup A is compared to the racial/ethnic pro-files of the drivers stopped by the officers inthe matched Group B, matched Group C, andso forth.

Drawing Conclusions from the ResultsBenchmarking with data for matched officersor matched groups of officers enables ana-lysts to identify “outliers,” officers or groupsof officers who stop racial/ethnic minoritiesat higher rates than do their matched counter-parts. The degree of confidence analysts canhave that policing by these officers is raciallybiased is entirely dependent upon thestrength of the match. Perfect matches wouldfully account for the factors reflected in thealternative hypotheses and enable the analystto test the bias hypothesis. But no match isperfect. For instance, in a large geographicarea within which officers are being com-pared, the racial/ethnic profile of drivers towhich particular officers are exposed maydiffer. Even officers with the same general as-signment of “patrol” may be directed towarddifferent activities in the course of their work.Therefore, they would not be exposed toidentical populations.

In sum, definitive conclusions aboutracial profiling cannot be drawn from thisbenchmarking method because the match be-tween individuals or between groups is im-perfect: that is, the racial/ethnic profile of thedrivers to which an officer or group of officersis exposed is not exactly the same as theracial/ethnic profile of the drivers to whichthe matched officers or matched group of of-ficers are exposed. This internal bench-marking method can pinpoint outliers, butfurther review is essential to assess whetherthe disparity may be the result of bias.

There is another major drawback associ-ated with this method: the relativity of the

findings. This method uses information onstopping behavior by police as both the nu-merator and denominator. In an officer-levelmatch, the numerator is one officer’s stopdata, and the denominator is the same type ofdata from other similarly situated officers inthe same department. Although this methodof analysis can identify outliers, it cannot de-termine whether or not all units used in thecomparison (all officers in an officer-levelanalysis or all groups in a group-levelanalysis) are practicing biased policing.

For example, it is clear from Figure 5.4that Officer 8 is stopping minorities at a ratedisproportionate to the rate of minority stopsby his or her peers. But an analyst cannotconclude that the other nine officers in thematch are stopping minorities in proportionsthat reflect legitimate stopping criteria: they,too, might be making decisions based onracial bias. Indeed, every officer in thismatched group of ten officers could be prac-ticing biased policing. In that case Officer 8is only the officer whose stopping decisionsappear to manifest bias most strongly. Simi-larly, in a group-level analysis, all of thegroups in the comparison could be biased.From the analysis, however, the researchercannot determine whether the matchedgroups are fair or biased in their policing.The analyst is able to identify only the offi-cers or groups that stop the highest propor-tion of minorities.

To overcome this major obstacle (that is,the relativity of the findings), an agencycould (if resources allowed) supplement in-ternal benchmarking with other methodssuch as benchmarking with data from “blind”enforcement mechanisms or with observationdata. In using internal benchmarking in con-junction with other methods, the researchercan take advantage of the great strengths ofthe internal benchmarking method andcounter its greatest weakness as well.


Taking Appropriate Action againstOfficers or GroupsThe data for outliers (whether individual offi-cers or a group of officers) should not be con-sidered proof of racially biased policing bythem. The results of high-quality-matchmethods, however, do raise legitimate redflags that can and should prompt further in-vestigation by the law enforcement agency.Specifically, the results justify a comprehen-sive inquiry into the officer’s or group’s stop-ping activity. The high rate of minority stopsmight have a legitimate explanation. For in-stance, the officer or group of officers mighthave a special assignment to a “hot spot” inthe geographic area where minorities arepresent in numbers disproportionately higherthan their representation in the rest of the ge-ographic area.

Samuel Walker (2002, 86) advocates cau-tion when law enforcement agencies inter-pret the results of this benchmarking method:

Where the data analysis identifies po-tential problem officers or supervisors,the [internal benchmarking] approachmoves to the intervention stage. Inter-vention begins with a review of an of-ficer’s performance by supervisors.There may be extenuating circum-stances that explain a particular pat-tern of traffic stops. The officer underreview should enjoy a presumption ofinnocence until a full performance re-view is completed. The importantpoint is that the data represent astarting point, the beginning of a de-partmental inquiry, and are not in andof themselves conclusive. Thus, no of-ficer is automatically presumed guiltysimply because he or she has made ahigh number of stops of minoritydrivers. A flexible system involving acommand review of performance canaccommodate officers who may bedoing professional, proactive policework (emphasis in original).

OBSERVATIONBENCHMARKING

Using the observation method, researcherscompare the racial/ethnic profile of driversobserved at selected sites to the racial/ethnicprofile of drivers stopped by police in thesame vicinity. The observation data (the de-nominator) is used as a benchmark for thestop data (the numerator). Agencies usuallyhire one or several researchers to help themwith this assessment. Observations are con-ducted by individuals trained by the re-searchers to be the observers. Researchershave four choices to make in observationbenchmarking:

l

How should the observations beconducted?

l

What should be observed?

l

What locations should be selectedfor observation?

l

When should the observations beconducted?

Each of these questions is discussedbelow.

Methods of ObservationObservations can be conducted from sta-tionary or mobile positions. With stationarymethods, the researcher places observers atlocations beside roadways; with mobilemethods (also called “rolling” or “carousel”methods), the observers are placed in vehi-cles that move with traffic.

Stationary methods have been used mostfrequently to observe the demographic char-acteristics of drivers on urban and suburbanroads. For instance, the researcher places ob-servers at carefully selected intersections,and the observers record the race/ethnicity ofthe drivers passing through those intersec-tions. The demographic profile of the peoplepassing through the intersections is com-pared to the demographic profile of peoplestopped by police in the same geographic


areas. Stationery methods have also beenused to create benchmarks for highway stops(see for instance, Engel, Calnon, and Dutill2003, and Lange, Blackman, and Johnson2001).

Using mobile methods, Smith et al. (2003)conducted a comprehensive study of stopsmade by the North Carolina State HighwayPatrol. The research team placed a driver andthree observers inside two “observer vans”that moved along selected roadway segmentsat the speed limit. One observer recorded thedemographic characteristics of the drivers ofvehicles passing the van (along with informa-tion regarding the vehicle), and the other twoobservers measured the speed of these vehi-cles using stopwatches. (The two speedmeasures were averaged.) With this informa-tion the researchers were able to compare thedemographic profile of speeders to the demo-graphic profile of drivers stopped by police.

Focus of ObservationsResearchers using observation benchmarkingneed to decide whether to compare the stopdata (the numerator) against demographicdata for all drivers regardless of drivingquality and/or for traffic law–violatingdrivers. The former entails collecting data onthe race/ethnicity of drivers on the roadways;the latter entails collecting data on the race/ethnicity of drivers who are violating specifictraffic laws. If demographic data are col-lected on all drivers (without distinguishingbetween nonviolating and violating drivers),the agency has addressed the alternative hy-pothesis that racial/ethnic groups are notequally represented as drivers on jurisdictionroads. It has not, however, addressed the al-ternative hypothesis that racial/ethnic groupsare not equivalent in the nature and extent oftheir traffic law–violating behavior.

Measuring Race/EthnicityThe assessment of race/ethnicity for thebenchmark data relies upon the perception of

the observers—and their perception, presum-ably, will be in error some unknown propor-tion of the time. Similarly, observation is thepreferred way for officers to measure race/ethnicity for purposes of filling out their datacollection forms; to the extent that officersmake stopping decisions based on race/ethnicity, they do so based on their percep-tions of race/ethnicity, not on the basis of, forinstance, information on the driver’s license.Since observation by officers is the preferredmethod for identifying race/ethnicity for thenumerator data, observation by trained ob-servers is equally viable as the method for ob-taining the denominator data.

It is difficult for both police and observersto make fine distinctions between racial andethnic groups. In the context of imple-menting the observation method, this diffi-culty has ramifications for the categories ofrace and ethnicity used for data collection.Particularly problematic is identifying eth-nicity through observation. It also is difficultfor observers—particularly stationary ob-servers collecting data on fast-moving vehi-cles—to distinguish among, for instance,Middle Easterners, Hispanics, and NativeAmericans. The ability to discern race/ethnicity can be impeded by the time of dayas well as by the speed of vehicles under ob-servation. Because of the difficulty of per-ceiving accurately a driver’s race/ethnicity,many of the researchers implementing obser-vation benchmarking use two or three ob-servers. Additionally, some researchers haveaddressed the problem of discerning the de-mographic characteristics of drivers bybroadening categories of race/ethnicity tomore closely match what observers can see(for instance, “Caucasian” and “not Cau-casian” or “Black” and “non-Black”).

Location of ObservationsFor both stationary and mobile methods of ob-servation benchmarking, researchers must de-termine the type of locations to be observed,


the number of locations, and their geographicarea. The ability of the observers to discernthe race/ethnicity of drivers is affected by thespeed of the vehicles, lighting at the site, andwhere observers stand. In some urban or sub-urban locations, it may not be safe for ob-servers to stand too near the traffic.Conducting observations at intersections witha stop sign or traffic light may be safer andprovides better visibility of drivers becausetraffic is slowed and the lighting may be betterat night. Like stationary methods of observa-tion benchmarking, mobile methods must bestructured to ensure visibility. Observationsmust take place on thoroughfares with at leasttwo lanes in each direction so that cars canpass the observer vehicle and be passed by it.Lighting may also be a consideration duringevenings and nights, but vehicle speed is lesslikely to be a factor since the observer vehicleis moving with the traffic. In making site se-lections, researchers also should consider thevolume of activity. The selected sites musthave sufficient numbers of both police stops(numerator data) and cars and/or violatorspassing by the sites (denominator data) to pro-duce reliable results.

Timing of ObservationsObservation benchmarking requires re-searchers to make decisions not only aboutthe method, focus, and location of observa-tions but also about their timing (the days ofthe week, the times of the day, and the lengthof the reference period). Decisions related totiming are important because the racial/ethnic composition of drivers on the road-ways may vary considerably across days ofweek, times of day, or even seasons of theyear. Choices related to the timing of obser-vations (the denominator data) will affecttime-related choices with regard to the stop(numerator) data.

In selecting days of the week for sched-uling observations, researchers strive for“representativeness” in the nature and extent

of traffic behavior. Observations could coverall days of the week or, to be more efficient,researchers could develop “categories ofdays.” For example, a researcher might makethe reasonable assumption that traffic onMondays, Tuesdays, Wednesdays, and Thurs-days is essentially similar in the area beingstudied but traffic on Fridays, Saturdays, andSundays are each unique. Similarly, thetimes of day for collecting observation datashould also reflect the goal of representative-ness. For example, researchers would notconduct observations from 6 P.M. to midnightif they wanted to benchmark stops madeduring all times of the day.

In some jurisdictions traffic varies duringdifferent times of the year. (For example, asouthwestern city may experience an influxof northern tourists during the wintermonths, and a university town with a popularfootball team may have more traffic duringthe fall.) This seasonal variation will affectthe population of drivers on the roadwaysand thus the racial/ethnic profile of drivers.To account for seasonal variation in traffic, re-searchers can conduct observations at var-ious points throughout a twelve-monthperiod. Researchers who choose a referenceperiod of less than one year (for example, sixmonths) should include in the report a caveatthat the results do not necessarily apply tothe parts of the year for which data were notanalyzed.

Conducting the AnalysisResearchers match the police stops (the nu-merator data) with the observation (or de-nominator) data at each site with regard tothe types of violations observed (for instance,speeding), the geographic location of thestops, the time of day, and the reference pe-riod. In hypothetical City A, for example, ob-servers collected demographic data at fifteensites (intersections) for all drivers violatingspeeding, red light, and/or stop sign laws.The observation data were collected during


randomly selected time blocks between 7 A.M.and 7 P.M. on all days of the week for the pe-riod January through June. Around each sitethe researcher identified a perimeter withinwhich the traffic resembled the traffic goingthrough the intersection. Then, within eachof these fifteen geographic areas, the re-searcher selected the police stops that oc-curred from January to June for speeding, redlight, and/or stop sign violations between thehours of 7 A.M. and 7 P.M. For each of the fif-teen sites, the researcher compared the demo-graphic profile of the people stopped to theprofile of the people observed.

Drawing Conclusions from the ResultsThe observation method, conducted in accor-dance with standard social science methods,can provide meaningful information for a ju-risdiction exploring the existence of raciallybiased policing. The assessment, however, islimited because the researcher is only able toconduct “spot checks” of racially biasedpolicing. For example, in hypothetical City Amentioned above, the researcher will have astrong assessment of racially biased policingbut only in the geographic areas, during thetime periods, and for the violations understudy.

With observation benchmarking, likeother benchmarking methods, the strongerthe “match” between the numerator and de-nominator data, the greater the confidencethe researcher can have in the results. How-ever, this increased confidence comes at acost in terms of the scope of the assessment.Another potential drawback of the matchingprocess is that it may reduce the numbers ofstops in the numerator and/or the number ofobservations in the denominator to the pointthat some analyses become unreliable.

The observation benchmarking methodaddresses the alternative hypotheses (ex-plained in Chapter 2) that racial/ethnicgroups are not equally represented as resi-dents in the jurisdiction and that racial/ethnic

groups are not equally represented as driverson jurisdiction roads. If analyses are con-ducted separately for specific geographic lo-cations, the method addresses the hypothesisthat racial/ethnic groups are not equally repre-sented as drivers on roads where stopping ac-tivity by police is high. If observations aremade of drivers rather than violators, themethod does not address the alternative hy-pothesis that racial/ethnic groups are notequivalent in the nature and extent of theirtraffic law-violating behavior. If, however, theobservations are made of drivers violatingparticular traffic laws, the method addressesthis final hypothesis.

OTHERBENCHMARKING METHODS

This chapter has explained how researcherscan compare data on stopping activity by po-lice to adjusted census data, Department ofMotor Vehicle data, data from blind enforce-ment mechanisms (such as red light camerasand radar), peer officers or units, and obser-vation data. In addition to these bench-marking methods, researchers can conduct

l

crime data benchmarking,

l

crash (auto accident) databenchmarking,

l

transportation data benchmarking, and

l

survey data benchmarking.

These four methods, although they havereceived less attention from researchers, arenoteworthy. We briefly summarize them herein order to communicate the full spectrum ofmethodology options now being consideredby jurisdictions nationwide.

Crime Data BenchmarkingIn crime data benchmarking, unlike the otherbenchmarking methods we have described,the benchmark data are not compared to all“vehicle stops” (stops made by police of aperson in a vehicle) or to “traffic stops”


(vehicle stops the stated purpose of which isto respond to a violation of traffic laws).Rather, the benchmark data are compared toa subset of vehicle stops: the “investigativestops” (police stops of people in vehicleswhen there is at least reasonable suspicion ofcriminal activity).9, 10

Researchers conducting crime databenchmarking compare the racial/ethnic pro-file of drivers stopped by police in an investi-gation of possible criminal activity (thenumerator or investigative stop data) to theracial/ethnic profile of people who appear inrecorded data on crime in the jurisdiction(the denominator or crime data). In thismethod, the measures of crime (1) must belinked to the race/ethnicity of the suspect orperpetrator and (2) must reflect as closely aspossible actual crime as opposed to crime re-sponded to by police.

Arrest data from the Uniform Crime Re-ports (UCR) contain race/ethnicity informa-tion (satisfying the first criterion). But crimedata that meet the second criterion are diffi-cult for researchers to obtain. The demo-graphic profile of people arrested in aparticular jurisdiction (as reported in theUCR) reflects two factors: (1) who commitscrime and (2) whom the police identify andtarget for arrest. The decisions made by po-lice regarding whom to target for arrest couldbe affected by racial bias. If a law enforce-ment agency is racially biased in both its ar-rests and investigative stops of vehicles and ifit uses arrest data as a benchmark for inves-tigative stops, its bias will not be revealed inthe results.

To lessen the problem of police biasskewing the results, some researchers havecompared investigative stop data to carefullyselected subsets of arrest data (for instance,arrests in which there is minimal police dis-cretion).11 The research team that conductedanalyses in San Diego (Cordner, Williams,and Zuniga 2001; Cordner, Williams, and Ve-lasco 2002) developed a racial/ethnic profileof criminals based on crime reports in whichvictims and witnesses provided descriptionsof the race/ethnicity of the perpetrators. TheSan Diego team selected this measure ofcrime because of its strength with regard tothe criterion stated earlier: the measure isminimally affected by police discretion.

Using viable measures of crime, a re-searcher can compare the racial/ethnic profileof drivers stopped by police to investigatecrime (so-called investigative stops) to theracial/ethnic profile of suspected criminalswithin subareas of the jurisdiction.

Crash Data BenchmarkingIn crash data benchmarking, researcherscompare the racial/ethnic profile of driversstopped by police (the numerator) to theracial/ethnic profile of drivers involved incrashes (the denominator). Most researchershave used crash data to estimate “who isdriving” rather than “who is driving poorly.”In the two major studies that used crash data,the North Carolina team (Smith et al. 2003)developed its benchmark using data on allpeople involved in crashes; the Miami-Dadeteam (The Alpert Group 2003) used data onlyon the drivers adjudged not to be at fault in

9 Using crime data to benchmark traffic stops would re-quire one to make a tenuous assumption—namely, thatthe same people who commit traffic violations are theones who commit crimes and vice versa.10 Also refer back to the discussion in Chapter 4 re-garding the analysis of subsets of data based on whether

the stops are for traffic violations or for suspicion ofcriminal activity. 11 Serious crimes are examples of crimes where there isminimal police discretion to arrest. For instance, if po-lice have probable cause to link a person to a murder orrobbery, it is very likely an arrest will be made.


the crashes. The latter decision was based ontransportation literature that conjectures thatnot-at-fault drivers in two-car crashes are arepresentative subset of drivers on the road.

If a jurisdiction decides to use crash dataas a benchmark, it should make sure that

(1) Race and/or ethnicity information onthe drivers involved in the crashes isavailable.

(2) The researchers have reasonable con-fidence that racial/ethnic groups inthe jurisdiction report crashes at sim-ilar rates.

(3) The researchers have reasonable con-fidence that the filing of accident re-ports by officers is systematic (that is,filed for all crashes reported to policeor filed for some clearly definedsubset of crashes).12

(4) The crashes in the data set can belinked to their geographic locationswithin the jurisdiction so that re-searchers can conduct subareaanalyses (see Chapter 4).

Working with crash data will be particu-larly challenging for researchers if the dataare not computerized. Another problem re-lates to sample size. Although there may benumerous crashes within a jurisdiction as awhole, within particular subareas thenumber of crashes available for analysis maybe too few to provide reliable assessments.

Transportation Data BenchmarkingThe National Highway Traffic Safety Admin-istration (NHTSA) collects demographicdata—including race but not ethnicity—on drivers. While valuable for measuring the

demographics of drivers, these national dataare not useful to individual jurisdictions ana-lyzing their police–citizen contact data be-cause it is not viable to assume that thedemographic profile of drivers produced by anational study will mirror the demographicprofile of drivers in a particular jurisdiction.State and local transportation data, however,can help researchers compare the racial/ethnic profile of drivers stopped by police tothe racial/ethnic profile of drivers driving onjurisdiction roads (or violating traffic laws onjurisdiction roads).

The U.S. Department of Transportationgathers travel behavior information throughsurveys of randomly selected households.One such survey is the National HouseholdTransportation Survey (NHTS) conductedevery five years. Respondents to the NHTSreport their race and ethnicity and other de-mographic characteristics. Unfortunately,most jurisdictions cannot use the data fromthe NHTS to benchmark their vehicle stopdata. Again, national data may not accuratelyreflect the demographic profile of drivers in aparticular jurisdiction. Moreover, there aregenerally not enough respondents within in-dividual jurisdictions to produce a reliableprofile.

The national data from the NHTS, how-ever, can be produced at the local level;a jurisdiction can “purchase” a sufficientsample to produce valid jurisdiction-level re-sults. That is, a jurisdiction (for instance, city,county, state) can request an NHTS “add on”that directs the U.S. Department of Trans-portation to sample more residents in thejurisdiction than it would have for purposes ofthe national survey. Sufficient numbersof residents are then surveyed to produce a

12 Officers working in high-crime areas where they arekept very busy may be less likely to take reports onminor accidents than are officers working in areaswhere fewer problems requiring their attention arise.

If these two types of areas also vary by racial/ethniccomposition, the accident reports for the jurisdictionwill not be representative of people involved inaccidents.


reliable assessment of the transportation be-haviors of residents within those jurisdictions.

Some local jurisdictions and states con-duct their own travel behavior surveys inde-pendent of the NHTS. The data produced bythese surveys—if race/ethnicity data are in-cluded—can be used by a researcher to de-velop a benchmark for “who is driving.”

Survey Data BenchmarkingSome researchers have used survey data totry to assess whether policing in a particularjurisdiction is racially biased. These re-searchers have conducted surveys (writtensurveys, telephone interviews, or face-to-faceinterviews) of scientifically selected residentsof the jurisdiction. The respondents areasked about (1) incidents over a specifiedtime period in which they were stopped intheir vehicles by police and (2) the quantity,quality, and location of their driving. In ef-fect, these surveys collect both numeratorand denominator data. The information onstops can be used instead of police-collecteddata to measure the nature and extent of ve-hicle stopping behavior. The information ondriving quantity, quality, and location pro-vides researchers with valuable informationon the various factors, referenced throughoutthis report, that can affect a driver’s risk ofbeing stopped by police.

Survey data benchmarking has a uniqueadvantage: the numerator data includepeople the police don’t stop as well as thosethey do stop. This is because the survey goesto a scientifically selected sample of resi-dents, some of whom have been stopped bypolice during the designated reference periodand some of whom have not. All of the otherbenchmarking methods discussed in this re-port rely upon the police-citizen contact datacollected by police; that numerator data in-clude information only on people who werestopped.

This benchmarking method also hasseveral drawbacks. Respondents may not

accurately report the information that is so-licited because their memories fail. In thecase of vehicle stop information, they mayforget some stops by police or provide faultyreports of driving quantity or quality basedon their less-than-perfect memories. Thefaulty memories may lead to “telescoping,” aterm to describe the reporting of an incidentas occurring during the reference periodwhen, in fact, it occurred before the start ofthe reference period. Some answers may notbe fully accurate—not because of the faultymemories of the respondents—but becausethe respondents want to “look good” or “saythe right thing.” This “social desirability ef-fect” could be particularly applicable to thequestions in a survey to measure racially bi-ased policing. Some respondents may under-report stops by police because they areembarrassed about them or want others tothink that their driving quality is better thanit truly is. If faulty memories (including tele-scoping) and the social desirability effect donot manifest equally across racial/ethnicgroups, the survey method will produce dis-torted assessments of racially biased policing.

Despite its shortcomings, this bench-marking method has been useful. With datafrom survey respondents regarding whetheror not they were stopped by police, re-searchers can determine if there is disparityin the level of stops of various racial/ethnicgroups in the target jurisdiction. A survey isvaluable because it can link that disparity tocauses by collecting from respondents infor-mation pertaining to the alternative hy-potheses (that is, information regardingdriving quantity, quality, and location).

CONCLUSIONThis chapter has described methods that arebeing used around the country to benchmarkstop data. Researchers can compare theirdata on stopping activity by police to ad-justed census data, Department of Motor Ve-hicle data, data from blind enforcement


mechanisms, peer officers or units, observa-tion data, crash data, and other transportationdata. Researchers also can benchmark inves-tigative stops against crime data and conductsurveys to collect “numerator” and “denomi-nator” data simultaneously.

All of these methods have strengths andweaknesses; none of them can prove or dis-prove the existence of racially biased policingin a jurisdiction. A causal connection be-tween drivers’ race/ethnicity and police stop-ping behavior cannot be proven, but this doesnot mean that data collection is for naught.By collecting police-citizen contact data, alaw enforcement agency conveys an impor-tant message to residents: it shows that theagency is concerned about racially biasedpolicing, is open to scrutiny, and is account-able to its constituency. Even if the results donot provide definitive conclusions regardingracial bias, they can serve as a basis for con-structive discussions between police andcommunity members regarding ways to re-duce racial bias and/or perceptions of racialbias.

If an agency chooses to collect data, thiseffort should be only one component of itscomprehensive response to the issues of bi-ased policing and the perceptions of its prac-tice. Reforms in the realms of supervision,policy, training, community outreach to mi-norities, and recruitment also should be con-sidered (see Fridell et al. 2001). Further, if theagency chooses to gather information tomeasure racial bias, it might consider sourcesother than—or in addition to—stop data. Itcan hold police-citizen forums to learn aboutcitizens’ concerns and perceptions, scrutinizecomplaints by the public, and organize meet-ings with supervisors to assess/discuss poten-tial problems. Multiple responses to theissues of racial/ethnic bias are possible, andmultiple sources of information are availableto guide agency reforms. Before we explainhow police-citizen contact data can be usedin constructive ways (Chapter 8), we discussactions police take after a driver has beenstopped (Chapter 6) and how the results ofbenchmarking methods can be conveyedand interpreted (Chapter 7).

VIGuidelines for AnalyzingPoststop Activities by Police

Two questions have interested researchersanalyzing data on vehicle stops:

l

Does a driver’s race/ethnicity have animpact on vehicle stopping behaviorby police?

l

Does a driver’s race/ethnicity have animpact on police behaviors/activitiesduring the stop?

The benchmarking methods discussed inChapter 5 addressed the first question. Herewe consider the second. Attention to poststopactivities is important. Some stakeholdershave expressed concern that poststop activi-ties by police are more likely than stop deci-sions to be influenced by racial bias.

The poststop activities most commonlyexamined by jurisdictions are searches andstop dispositions (the officer’s decision to ar-rest, ticket, warn, or provide no disposition).Other aspects of the stop (for example, lengthof stop and whether a person was asked toexit the vehicle) also have been examined byresearchers interested in assessing whetherpolicing in a jurisdiction is racially biased.

ANALYZING SEARCHESSome agencies collect information onsearches of vehicles and their occupants. Of-ficers in these agencies record search infor-mation on the police-citizen contact forms.The search data collected on the forms canbe analyzed by researchers in two ways:researchers can calculate the “percentsearched” for each racial/ethnic group, andresearchers can calculate “hit rates” (the per-cent of searches in which the officers findsomething) for each racial/ethnic group.Searches are intrusive behaviors by police,and search data can help researchers explorewhether policing in a jurisdiction is biased.To analyze search data, however, requires cer-tain resources.

Resources RequiredFor effective analysis of search data, jurisdic-tions must make sure that officers collect cer-tain information on the forms they fill out.The form should include an item indicatingwhether or not a search was conducted.In addition, the form should solicit informa-tion on the legal authorization for the search.1

1 Example responses include probable cause, reason-able suspicion that a person is armed, incident to arrest,

“plain view,” warrant, inventory, consent, and probation/parole waiver.


To examine “hit rates,” the agency must in-clude on its form an item to indicate searchresults are either “positive” (somethingfound) or “negative” (nothing found). Toallow for a full examination of consentsearches requires an item on the form thatasks: “Did you request consent to search?Yes or No.”

Types of Search Data Analyses

“Percent Searched” Measures “Percent searched” measures are produced bycalculating for each racial/ethnic group thepercentage of stopped drivers who aresearched. If during a specified period, 100minorities were stopped in their vehicles and20 of them were searched, then the percentsearched is 20 (20/100 x 100). If 200 Cau-casians were stopped in their vehicles and 35of them were searched, then the percentsearched is 17.5 (35/200 x 100).

These percentages are often used erro-neously to draw conclusions regarding racialbias. In many jurisdictions higher propor-tions of stopped minorities are searched thanstopped Caucasians. Analysts, stakeholders,reporters, and even expert witnesses havemistakenly concluded that this disparity be-tween the frequency of searches of minoritiesand searches of Caucasians necessarily indi-cates bias on the part of police. Such conclu-sions are not supported by “percentsearched” information.

“Percent searched” information may showdisparity, but it cannot identify the cause ofdisparity between searches of racial/ethnicgroups or, relatedly, whether or not the dis-parity is justified. Not every person who isdetained is at equal risk of being searched bypolice; there are very legitimate reasons whysome persons are at greater risk of beingsearched than other persons. Indeed, thepublic should not expect equal search pro-portions across stopped groups. Virtually allagencies report that stopped men are

searched in greater proportions than stoppedwomen. Does this finding indicate policebias against men? Not necessarily. It could bethat more men are at greater legitimate risk ofbeing searched by police than women be-cause men, more than women, manifest be-haviors that provide legal grounds for asearch.

Figure 6.1 provides data from a hypothet-ical jurisdiction showing “percent searched”data for racial/ethnic groups by gender. Thisfigure indicates that 16 percent of the Cau-casian males who were stopped by policewere searched. Corresponding figures forAfrican American males, Hispanic males, and“Other” males were 24, 21, and 15 percent, re-spectively. Similar information is providedfor the females who were stopped by police.These data indicate disparity; detained mi-norities (particularly detained African Amer-ican and Hispanic males) are searched morefrequently than Caucasians. These results donot provide information regarding the causeor causes of that disparity.

Search “Hit Rates” A hit rate is the percent of searches in whichthe officers find something upon the peoplebeing searched. Officers might find contra-band (for instance, drugs, illegal weapons) orother evidence of a crime. Lower hit rates forminorities than for Caucasians for certain cat-egories of searches are cause for concern.These results are a warning signal or “redflag” requiring the serious attention of law en-forcement agencies. They are, however, notproof of racially biased policing.

A hypothetical example will help explainhit rates. If, during a specified reference pe-riod, police in an agency searched 100 ofthe stopped Caucasians, 80 of the stoppedAfrican Americans, and 60 of the stopped His-panics and found evidence on 10 of the Cau-casians, 4 of the African Americans, and 4 ofthe Hispanics. The hit rates would be 10 per-cent for Caucasians (10/100 x 100), 5 percent

Guidelines for Analyzing Poststop Activities by Police 57

for African Americans (4/80 x 100), and 7 per-cent for Hispanics (4/60 x 100).

For all types of searches, hit rates providedescriptive information regarding whether ornot there is disparity in “productivity.” If, forinstance, 22 percent of the searches incidentto arrest of African Americans produced hitscompared to 30 percent of the searches inci-dent to arrests of Caucasians, the Caucasiansearches of this type are more productive.This is valuable information warranting ex-ploration by the jurisdiction, but bias maynot be the cause of this disparity. Legitimatefactors may account for the differential hitrates.

Evidence-Based Searches. For “evidence-based searches,” however, researchers cansay with reasonable confidence that any iden-tified disparity is unjustified and likelycaused by bias. For this subset of searches,search hit rates can rule out (not definitivelybut with an acceptable degree of confidence)the alternative hypotheses (hypotheses thatfactors other than bias influence police be-havior). An economic theory called the “out-come test” will help us understand how.

The outcome test can be applied onlywhen decision makers claim that their deci-sions are based on the probability of a partic-ular outcome. First proposed by NobelPrize–winning economist Gary S. Becker

(1993), the outcome test was applied by himto outcomes related to money lending. As-sume a bank claims to make loan decisionsbased on the likelihood that the borrower willbe able to pay the loan back. If the bank ap-plies this criteria (probability of loan repay-ment) equitably across all racial/ethnicgroups, then the default rates should be equalacross groups. In other words, racial/ethnicgroups should succeed in their loan repay-ment at the same rates. If, in fact, the mi-nority borrowers default on their loans at alower rate than their Caucasian counterparts,researchers can infer that the criteria for de-termining who would get a loan were not thesame for Caucasians and for minorities; re-searchers can infer that minority borrowerswere held to a higher standard by those de-ciding to make the loans.

The above example pertains to the differ-ential allocation of benefits (that is, loans)across racial groups. The same test can beused to assess a decision maker’s allocation ofdetriments (for example, searches) acrossracial/ethnic groups. As Ian Ayres (2001, 406)explains, if police decisions to search minori-ties are “systematically less productive” thanpolice decisions to search whites, one mightinfer that undeserving minorities are beingsubjected to searches. From these resultsone might infer that different standards were

25%

20%

15%

10%

5%

0%Caucasians

16%

24%

13% 13%15%

21%

14%

8%

African Americans Hispanics Others

Males

Females

Figure 6.1. Searches as a Percentage of Vehicle Stops, by Race/Ethnicity and Gender of Detained Group, Hypothetical Jurisdiction


utilized in selecting Caucasians and minori-ties for searches. Specifically, it appears thata lower standard of proof was appliedto searches of minorities than to searches ofCaucasians.

The outcome test does not focus onwhether different proportions of minoritiesand Caucasians are searched. Different pro-portions of Caucasians and minorities mightmeet legitimate, unbiased criteria for asearch. The outcome test focuses on the poolof people that the decision maker “deemedqualified” for a search (or for a loan, in theearlier example).

Another way to restate the example is byusing a hypothetical construct, “units of evi-dence.” Imagine an officer who searches allminorities he detains for whom he has 50units of evidence that they are carrying con-traband or other evidence. He searches allCaucasians he detains for whom he has a cor-responding 80 units of evidence. He has set alower standard for searching minorities com-pared to Caucasians. The result will be thathe is “wrong” more often with his minoritysearches; the officer is less likely to find evi-dence on the minorities, because he settledfor a low level of evidence to initiate thesearch. He will have more “hits” in hissearches of Caucasians because he didn’tsearch them unless he was highly confidentthat they were carrying contraband/evidence.2 This produces a lower hit rate forminority searches. As Ayres explains (2002,133), “A finding that minority searches aresystematically less productive than whitesearches is accordingly evidence that police

require less [evidence] when searchingminorities.”

As Ayres (2002, 134) explains, “The deci-sion maker in an outcome test by her own de-cisions defines what she thinks the qualifiedpool is, and the outcome test then directly as-sesses whether the minorities and nonminori-ties so chosen are in fact equally qualified.”The bankers will claim that they make loandecisions based only on the probability of de-fault. The corresponding circumstance forpolice is when they make searches based onthe probability of finding contraband/evidence. This is true when the police con-duct probable cause searches, frisks forweapons, searches based on “plain view” ordrug odors, and, arguably, canine alertsearches.3 These types of searches are “evi-dence-based searches.” The requirement ofthe outcome test (decisions must be based onthe probability of a certain outcome) is notmet with other types of searches, such assearches incident to a lawful arrest, inventorysearches, or warrant searches.4

Table 6.1 provides sample results showinghit rates for evidence-based searches forgroups defined by their race, age, and gender.These hypothetical data indicate that hit ratesfor evidence-based searches of young mi-nority males are lower than for any othergroup.5 For young African American malesand for young Hispanic males, the hit ratesare 8 percent and 6 percent, respectively.All other groups have hit rates of at least 13percent. Results such as these should promptlaw enforcement agencies to examinetheir searches more closely and/or implement

2 A “wrong” decision does not imply that the searchwas unjustified; similarly, a “hit” does not imply thatthe basis for the search was legitimate.3 We discuss consent searches separately below. 4 This means that hit rate analyses will be conductedseparately for different subsets of searches. The hit rateanalysis conducted on the subset of evidence-based

searches can be interpreted in accordance with the out-come test.5 Again, a small number of searches in a jurisdictionmay preclude breakdowns of the data within categoriessuch as race, gender, and type. Analyses with smallnumbers are unreliable.


interventions to reduce this apparent (albeitnot proven) bias in searches. For instance,law enforcement agencies could expand col-lection of quantitative or qualitative data onsearches to gather more information or imple-ment interventions to eliminate or decreasepotential bias in search decisions.6

The Special Case of Consent Searches.With nonconsent evidence-based searches,7

the decision of the officer that is evaluated inthe outcome test is the decision to conductthe search; in every instance the researcherwill know whether or not the officer was rightor wrong about whether the person was car-rying contraband or other evidence. If the of-ficer wants to conduct 100 nonconsentevidence-based searches, he will conduct 100of them, and the researcher will know fromthe form filled out for each one whether ornot there was a “hit.”

With consent searches, however, the deci-sion of the officer that is evaluated is the de-cision to request consent to search. Theresearcher wants to know if the officer, be-cause of bias, requests consent to search from

minorities more than from Caucasians. Theofficer may want to conduct 100 consentsearches but be able to conduct only 85 be-cause consent is withheld by 15 people. Toproperly evaluate the officer’s decision usingthe outcome test, the researcher would needto know for all 100 people from whom con-sent was requested who was and was not car-rying contraband or other evidence. Thisinformation is known only for 85 of the 100.The researcher cannot assume that the 85 arerepresentative of the 100. It is plausible thatthe 15 who refused to provide consent arecarrying evidence/contraband at a different(likely higher) rate than the 85 who con-sented, and it is possible that the relationshipbetween refusal and carrying differs acrossdemographic groups.

If an agency has a large number of peoplewho refuse to provide consent, the agencycannot include consent searches in the cate-gory of evidence-based searches for purposesof conducting hit rate analysis. There is noclear rule of thumb for when the level ofmissing “consent search” data is sufficiently

6 See Chapter 8 for actions agencies can take to reducethe potential for bias in consent searches and otherhigh-discretion activities.

7 Again, these are probable cause searches, frisks forweapons, searches based on “plain view” or drugodors, and canine alert searches.

Source: Based on a table in Council on Crime and Justice and Institute on Race and Poverty(2003, 29).

Table 6.1. Evidence-Based Search "Hit Rates," by Race/Ethnicity,Gender, and Age, Hypothetical Jurisdiction

Female Male

Race/Ethnicity <24 25+ <24 25+

Caucasian 17% 14% 15% 16%

African American 13% 15% 8% 15%

Hispanic 15% 16% 6% 17%

Other 15% 13% 14% 15%


low to determine unjustified disparate im-pact. However, we maintain that a researcherwho has at least 95 percent agreement to theconsent searches within each of the racial/ethnic groups can analyze the hit rates forconsent searches and interpret them in accor-dance with the outcome test.8

Lower hit rates for minorities for evi-dence-based searches signal the possibility ofracial bias. These findings are sufficientgrounds for further exploration by a depart-ment, but they are not conclusive evidence ofbias. Similarly, equal hit rates for minoritiesand Caucasians are not conclusive evidencethat search decisions are bias-free. Legitimatefactors unrelated to bias can produce lowerhit rates for minorities, and biased police ac-tions can produce equal hit rates in certain

circumstances (see Fridell 2004, chap. 11).Researchers should consider these factorsor circumstances when interpreting hit rateresults.

Other Ways to Examine SearchesIn addition to analyzing “percent searched”data for racial/ethnic groups and “hit rates” forracial/ethnic groups, researchers can analyzesearches in other ways. One way is to conduct“internal benchmarking” with search data.

Recall from Chapter 5 how the internalbenchmarking method is implemented: Toanalyze stopping behavior by police, agenciescompare stops by individual officers to stopsby other similarly situated officers, or theycompare stops by a group of officers to stopsby other similarly situated groups of officers.9

8 To conduct this analysis, the law enforcement agencyfirst must include in the data collection form an item re-garding whether or not the person was asked for con-sent to search.

9 For instance, they compare officers who are assignedto the same geographic area, the same shift, and whohave the same mission (such as patrol). These similarlysituated officers are exposed to the same group ofpeople at risk of being stopped by police.

Figure 6.2. A Comparison of Twelve Officers Who Are Similarly Situated (“Matched”): Percent of Drivers Searched Who Are Minorities

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

Officer

2624

2927 26

2220

45

2830

2830

1 2 3 4 5 6 7 8 9 10 11 12


In the same way, agencies can comparesimilarly situated officers with regard to thepercent of drivers searched who are minori-ties. Figure 6.2 (previous page) illustrateshow this is done. In this hypothetical juris-diction, between 20 and 30 percent of thedrivers searched by officers (Officers 1through 9 and Officers 11 and 12 in thefigure) are minorities. In contrast, 45 percentof the drivers searched by Officer No. 10 areminorities. Officer No. 10 is an “outlier” (insocial science terminology), and this officer’sdecisions to search should be reviewed by thedepartment to see if bias is influencing them.

ANALYZINGSTOP DISPOSITIONS

Does a driver’s race/ethnicity have an impacton what happens during a vehicle stop? Toaddress this question, jurisdictions can ana-lyze search data. They also can analyze dataon stop dispositions (for instance, arrest, cita-tion, warning, no action).

Jurisdictions do not agree on what disposi-tion results indicate racial bias by police. Re-searchers for the Montgomery County (MD)Police Department (2001) have held that dis-proportionate representation of minoritiesamong drivers given the most serious disposi-tions (arrests or citations) is an indication ofbias. Other analysts have claimed racial biasis indicated by the disproportionate represen-tation of minorities among those receiving theleast serious dispositions such as warnings orno disposition. Such “low-level” outcomesare not viewed by them as a sign of policebenevolence but as evidence that there may

have been no legitimate reasons for thesestops in the first place. More low-level dispo-sitions for minorities than for Caucasiansis seen by this group as evidence of police“fishing” for evidence of crime amongminorities.

These varied interpretations of disposi-tion information reflect the challenge re-searchers face when analyzing this type ofdata. In their analysis of disposition data,like vehicle stop data, researchers can iden-tify “disparity” in police actions or the lackthereof. They can calculate the percentage ofvarious dispositions across drivers withinvarious racial groups. The results in Table6.2 for a hypothetical jurisdiction show thatminorities are over-represented amongdrivers receiving “no disposition.” Like the“percent searched” data, disposition data canidentify disparity in police actions but not thecause of that disparity.

Not all drivers are at equal risk of beingsearched; similarly, not all stopped driversare at equal risk of receiving the various dis-positions. In disposition data analysis, themore legitimate factors the researcher canrule out for the officers’ choice of disposition,the more confidence the researcher can havethat disparity in police decisions is due tobias.

The team analyzing the vehicle stop datafor the Washington State Patrol (WSP)(Lovrich et al. 2003) determined that thequantity and seriousness of the violations bythe stopped driver appear to be key variablesthat influence police disposition decisions.Other variables that might influence the

Race Arrest Citations Warning No Disposition Total

Minorities 6% 59% 23% 12% 100%

Caucasians 5% 62% 25% 8% 100%

Table 6.2. Stop Dispositions for Caucasians and Minorities


dispositions police choose include (but cer-tainly are not limited to) the stopped driver’sdemeanor, the prior driving record of thestopped driver, and the geographic location ofthe stop. An example will illustrate the im-portance of stop location. An officer mightconsider speeding 10 miles per hour over thespeed limit in a school zone as a more seriousoffense than 10 miles per hour over the speedlimit on a highway.

All of these factors can legitimately influ-ence the dispositions chosen by police, and re-searchers analyzing disposition data shouldattempt to take them into account. This type ofanalysis, however, cannot be performed unlessthe agency has certain resources available.

Resources RequiredThe form that officers fill out should includean item regarding the disposition of the stop.Common options are arrest, ticket/citation,verbal warning, written warning, and noaction. Information related to the reasonsfor stopping the vehicle are relevant to ana-lyzing the dispositions of those stops. There-fore, data collection forms should includea field for “reason for the stop.” There is alot of variation across agencies with regard tothe specificity of the “reason” options; anagency may have as few as five or as manyas twenty “reasons” for the stop. As noted

earlier, information on the stop form re-garding the quantity and seriousness of viola-tions can be very useful.

Analysis of Dispositions withinCategories of StopsOne of the factors that legitimately influencesthe choice of dispositions is the seriousnessof the offense. For this reason, researchers tryto control for or isolate this factor. If re-searchers examined dispositions for data thatincluded all possible offenses, they would notknow, for instance, if a finding that AfricanAmericans received harsher dispositionsthan Caucasians was due to bias or to the pos-sibility that they committed more seriousdriving violations. Instead of doing oneanalysis of dispositions for all violations com-bined, researchers are encouraged to look atdispositions across races within offense cate-gories such as speeding violations, red lightviolations, failure to yield violations, and soforth.

Table 6.3 provides hypothetical dispositiondata for moving violations in Jurisdiction A byrace and ethnicity. Relative to the othergroups, African Americans are slightly under-represented among detained persons who re-ceive a citation for moving violations (71.82percent) and slightly over-represented amongpeople who are arrested (6.28 percent). Even

RaceNumberDetained

Percent ofDetained

Disposition

Arrest Citation Written Warning No Disposition

African Americans 6,405 15.65% 402 6.28% 4,600 71.82% 801 12.51% 602 9.40%

Hispanics 1,700 4.15% 54 3.18% 1,267 74.53% 234 13.76% 145 8.53%

Other Minority 8,182 20.00% 402 4.91% 5,998 73.31% 1,035 12.65% 747 9.13%

Caucasians 24,629 60.19% 623 2.53% 18,772 76.22% 2,997 12.17% 2,237 9.08%

Total 40,916 100.00% 1,481 3.62% 30,637 74.88% 5,067 12.38% 3,731 9.12%

Table 6.3. Dispositions for Moving Violations, by Race/Ethnicity, Hypothetical Jurisdiction A


if these differences were larger, conclusionsabout racial bias could not be drawn. This isbecause proportionately more African Ameri-cans than the other groups could have pre-sented behaviors (or been linked to other basesfor an arrest such as outstanding warrants)that legitimately led to the arrest disposition.

Within the broad “reason for a stop” cate-gory of “speeding,” a researcher could refinethe analysis even further. The researchercould subdivide this category based on infor-mation on the stop form regarding how manymiles per hour the person was speeding. Forexample, the researcher might produce atable similar to Table 6.3 for each of the fol-lowing categories of speeding: less than 10mph over the speed limit, 10 to 15 mph overthe speed limit, 16 to 20 mph over the speedlimit, and greater than 20 mph over the speedlimit.

Other Ways to Examine DispositionsDispositions can be analyzed within cate-gories of stops based on the seriousness of theoffense as described above. Researchers alsocan analyze dispositions by matching driversof one race to “similarly situated” drivers ofanother race. In an analysis of vehicle stopdata for the Oakland Police Department, thedrivers were considered similar if theymatched on variables such as location of thestop, time of the stop, whether the driver wasan Oakland resident, age of the driver, reasonfor the stop, and driver gender (Ridgeway,Riley, and Grogger 2004). By comparingthese similarly situated drivers, the researchteam could assess differences in dispositionsacross race while eliminating the possibilitythat some variables besides race (for instance,stop location, driver age) might be the causeof observed differences.

Another way to examine dispositions isthrough multivariate analyses, a topic cov-ered in the next chapter.

ANALYZING OTHERASPECTS OF A STOP

In this chapter we have discussed analysis ofsearch data and analysis of stop dispositiondata. Other aspects of a stop can be analyzedas well. Some jurisdictions, for instance, col-lect information on the duration of the searchor the duration of the entire stop; they mightcollect information regarding whether thedriver (or passengers) were asked to exit thevehicle, whether canines were brought to thescene, and whether firearms were drawn. Anagency may decide to include one or more ofthese variables in its analysis to understandmore fully what happens during traffic stopsin its jurisdiction.

The general analysis concepts presentedabove, indeed throughout the book, apply tothese and any other variables. Researcherswill attempt to identify the factors other thanracial bias that might account for disparitywith respect to any of these variables and ei-ther control for them with their methods or ref-erence their omission in interpreting results.

CONCLUSIONResearchers can analyze search data in var-ious ways. They can calculate and compare“percent searched” for racial/ethnic groups toindicate whether disparity exists, but theycannot draw conclusions from the data aboutthe existence or lack of racial bias in the juris-diction. Similarly, hit rates for all types ofsearches can provide an indication ofwhether disparity exists. Hit rates that meetthe assumptions of the outcome test can indi-cate the existence of unjustified disparate im-pact. The term “unjustified disparate impact”means that the disparity is not easily ex-plained by legitimate (nonbias) factors. Thesearches that meet the assumptions of theoutcome test are evidence-based searches(those where the decision to search is basedon the probability of finding contraband/evidence). Lower hit rates for minorities thanfor Caucasians for evidence-based searches


signal that racial bias may be influencing po-lice decisions, and the agency should con-sider additional assessments of searches orreform measures (see Chapter 8). Consentsearches cannot be analyzed with the out-come test unless high proportions of subjectswithin each of the racial/ethnic groups acqui-esced to requests to search.

From disposition data, agencies can iden-tify disparities across races, but they cannotdraw firm conclusions regarding bias by po-lice because of the challenges associated withcontrolling for the key factors that might le-gitimately affect the selection of dispositions.

The analysis of poststop data is compli-cated, and most methods can indicate onlywhether disparity exists, not the cause.

Despite these constraints, researchers shouldanalyze poststop data and report to the lawenforcement agency and other stakeholderscomprehensive information regarding whathappens after stops are made. These poststopactivities are vulnerable to racial bias bypolice, and they could have great negativeconsequences for the driver subject to them.It is important for police executives to knowwhat is happening during vehicle stops sincethese incidents comprise the most frequentinteraction between police and citizens. Aswe discuss in Chapter 8, a finding of dis-parity, even if the cause of the disparitycannot be identified, may provide impetus forconstructive changes in law enforcementpolicies or practices.

VIIDrawing Conclusions from the Results

Previous chapters have explained ways inwhich data on vehicle stops by police anddata on poststop activity by police (for ex-ample, searches and dispositions) can be an-alyzed. Jurisdictions are trying to determinewhether there is a cause-and-effect relation-ship between a driver’s race/ethnicity and po-lice behavior.

To examine police decisions to stop, forinstance, researchers compare the racial/ethnic profile of thepeople identified in thepolice-citizen contactdata and the racial/ethnicprofile of a “benchmarkpopulation.” This popu-lation might be com-posed of residents of thejurisdiction with accessto vehicles; drivers witha license; drivers identi-fied by red light cameras,radar, or air patrols;drivers stopped by“matched” officers orgroups of officers; driversobserved on the road byresearchers; or driversidentified through otherbenchmarking methods.

Figure 7.1 illustrates disparity in ananalysis of stop data: minorities are over-represented among drivers stopped relative totheir representation in the benchmark popu-lation. They represent 19.06 percent of thestopped drivers and 15.60 percent of thebenchmark population. In this hypotheticaljurisdiction, there is disparity betweenracial/ethnic groups in terms of stops madeby police.

Figure 7.1. Disparity between Drivers Stopped by Police inHypothetical Area A and the Benchmark Population for Area A,by Two Racial/Ethnic Groups

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%Caucasians

Stopped DriversBenchmark80.94%

84.40%

19.06%15.60%

Minorities


Disparity—such as that shown in Figure7.1—can be conveyed in four ways: throughabsolute differences in percentages betweenthose stopped by police and the benchmarkpopulation, relative differences in percent-ages, disparity indexes, and ratios of dis-parity. This chapter focuses on these fourways that disparity can be interpreted andconveyed to the public. In their analysisof stop, search, and disposition data,researchers can choose one or more of thesemeasures of disparity. Two additional toolsfor assessing and conveying disparity—contingency analysis and multivariateanalyses—are described as well.

When does disparity equate to bias?There is no simple answer to this question.Some researchers set a cut-off point: they de-cide that disparity levels above this point in-dicate racial bias. Others believe it isimpossible, and therefore inappropriate, toset a cut-off point. We evaluate these opin-ions and explain useful tools that researcherscan use to interpret data. This chapter ex-plains measures of disparity and how theycan be calculated. It does not provide defini-tive answers about when policing in a juris-diction is characterized by racial bias. Atheme of this book is that researcherscan measure disparity easily, but identifyingthe cause of disparity presents a challenge.That theme continues through this chapter.No calculations of measures of disparity—however advanced—will themselves over-come this challenge. Those who have a stakein the results of benchmarking analysis—residents, local officials, members of themedia, advocates for minorities, and others—seek definitive answers about whetherpolicing in their jurisdiction is racially bi-ased, but those definitive answers cannot be

given. The reason is the impossibility ofruling out all of the legitimate (nonbias) fac-tors influencing police decisions to stop a ve-hicle, conduct a search, or give a disposition(that is, arrest the driver, ticket the driver,warn the driver, or provide no disposition tothe stopped driver). Benchmarking analysiscan signal the possibility of biased policing,motivate jurisdictions to explore policingpractices, and improve relations between po-lice and the community. Definitive conclu-sions, however, cannot be drawn from theresults.

FOUR MEASURESOF DISPARITY

If benchmarking analysis reveals a disparitybetween the racial/ethnic profile of stoppeddrivers and the racial/ethnic profile of thebenchmark population, researchers have achoice of four ways to measure and conveythat disparity: absolute differences in percent-ages between those stopped by police and thebenchmark population, relative differences inpercentages, disparity indexes, and ratios ofdisparity. Table 7.1 explains how each ofthose measures can be calculated. The tableis based on the disparity shown in Figure 7.1between stopped drivers and the benchmarkpopulation for Area A in the hypotheticaljurisdiction. To simplify the explanation, citi-zens are separated into just two groups:Caucasians and minorities.

Column A presents the number of stops ofminorities and Caucasians across the refer-ence period (for instance, one year). Re-searchers who are calculating measures ofdisparity should include in their tables thenumber of stops so that the discerning readercan assess whether this number is sufficientto produce reliable results.1

1 Analyses with small numbers of stops are less reliablethan those with larger numbers of stops.

Drawing Conclusions from the Results 67

Column B presents the percentage of thestops by police that were of minorities and ofCaucasians (summing to 100 percent). Thus,for instance, the percentage of stops that wereof minorities is 19.06 [(15,492/81,281) x(100)]. Column C presents from Figure 7.1the percentage of minorities and Caucasiansin the benchmark population. If the jurisdic-tion were implementing benchmarking withadjusted census data (for instance, adjustedfor access to vehicles), Column C would indi-cate that 15.60 percent of the jurisdiction’sresidential population with access to vehicleswere minorities, 84.40 percent were Cau-casians. If the benchmark represented peopleobserved violating speeding laws (as opposedto jurisdiction residents), Column C wouldindicate that 15.60 percent of the peoplespeeding on the jurisdiction’s roads were mi-norities and 84.40 percent were Caucasians.

For this example, to calculate absolute dif-ferences in percentages between those stoppedby police and the benchmark population,subtract Column C (representation of thegroup among the benchmark population)from Column B (representation of the groupamong the drivers stopped by police). For theminority group, the absolute percentage dif-ference is 3.46 percent (19.06% – 15.60%).This result can be conveyed in the following

language: “there are 3.46 percent more mi-norities among the people who are stoppedthan are represented in the benchmarkgroup.”

A second way that researchers can conveydisparity is through relative differences in per-centages between those stopped by police andthe benchmark population. For the minoritygroup in Table 7.1, the relative percentage dif-ference is 22.18 percent or [(19.06 – 15.60)/15.60] x 100. In other words, 19.06 percent is22.18 percent greater than 15.60 percent.This difference could be expressed as follows:“there are 22.18 percent more minoritiesamong the people who are stopped than arerepresented in the benchmark group.” Or,“minorities are over-represented amongpeople stopped by 22.18 percent relative totheir representation among the benchmarkgroup. Similarly, whites are under-representedamong people stopped by 4.10 percent rela-tive to their representation among the bench-mark group.” The wording used to describeabsolute and relative differences in percent-ages is the same. There is no particular lan-guage for conveying the results thatdistinguishes the figures that are absolutepercentage differences and relative per-centage differences. Researchers shouldconvey the meaning of the disparity by

Note: These data produced the summary results presented in Figure 7.1.

Table 7.1. Four Disparity Measures to Describe Stops in Hypothetical Area A, for Two Racial/Ethnic Groups

A B C D E F G

Numberof Stops

Percentof Stops

Percent ofBenchmark

Absolute% Difference

Relative% Difference

DisparityIndex

Ratio ofDisparity

Equation [A(m or c)/t] x 100 B-C [(B-C)/C] x 100 B/C F(m)/F(c)

Minorities (m) 15,492 19.06% 15.60% 3.46% 22.18% 1.22 1.27

Caucasians (c) 65,789 80.94% 84.40% -3.46% -4.10% 0.96

Total (t) 81,281 100.00% 100.00%


describing in the report the equation used:either B – C or [B – C/C] x 100 (see Table 7.1).

A third way to convey disparity is a “dis-parity index.” For the minority group in Table7.1, the disparity index is 1.22, which is cal-culated by dividing Column B (group per-centage among drivers stopped) by Column C(group percentage among benchmark popula-tion). A value of 1 would indicate no dis-parity; that value would be obtained in ourexample if 19.06 percent of the stops were ofminorities, and minorities comprised 19.06percent of the benchmark population. Avalue greater than 1 indicates over-represen-tation among drivers stopped relative to thebenchmark, and a value less than 1 indicatesunder-representation among drivers stoppedrelative to the benchmark. The results inTable 7.1 indicate an over-representation ofminorities among stops relative to their repre-sentation in the benchmarked group.2

A “ratio of disparity” (referred to by someresearchers as an “odds ratio”) is the fourthway a finding of disparity can be conveyed.The disparity index for one group is dividedby the disparity index for another group. Thegroup in the denominator is the “referencegroup” to which the other group is compared.In our example, we use the disparity index togauge how minorities (the numerator in theequation) fare relative to Caucasians (the de-nominator in the equation).

For the minority group in Table 7.1, theratio of disparity is 1.27 (1.22/0.96). The dis-parity index for minorities is divided by thedisparity index for Caucasians to produce asingle number. A number greater than 1 indi-cates over-representation, and a number lessthan 1 indicates under-representation. Re-searchers could explain the ratio of disparity

shown in Table 7.1 in any of the followingways:

l

“Minorities are stopped 1.27 timesmore than Caucasians.”

l

“If you are a minority, you are 1.27times more likely to be stopped bypolice than if you are Caucasian.”

l

“For every Caucasian stopped, 1.27minorities are stopped.”

Table 7.2 shows how to calculate ratios ofdisparity when there are more than two racial/ethnic groups. Because Hispanics comprised8.24 percent of the stops and a very similarpercent of the benchmark population (8.20percent), the disparity index for Hispanics is1.00 (8.24/8.20), indicating no disparity. Thedisparity indexes for African Americans andCaucasians show over-representation ofAfrican Americans relative to the benchmark(1.46) and under-representation of Cau-casians (0.96). Recall that to produce theratio of disparity for the two groups in Table7.1, we divided the disparity index for mi-norities by the disparity index for Caucasians(1.22/0.96 = 1.27). To calculate the ratio ofdisparity with three racial/ethnic groups, re-searchers again must identify which of thethree groups is the “reference group.” Thedisparity index for this chosen referencegroup becomes the denominator for the ratioof disparity calculations for the other two.We suggest that the relevant group in any cal-culation of a ratio of disparity for vehicle stopanalysis be the Caucasian group. This is be-cause the main question we are trying to an-swer is as follows: “Are minority residentstreated differently from Caucasian residentsbecause of their racial/ethnic status?”

2 Consistent with our caveat that small sample sizesproduce unreliable results, note that all of these meas-ures are unstable when sample sizes are small.


THE CHALLENGE OF SELECTINGMEASURES OF DISPARITY

Above we explained four different ways thatresearchers can convey disparity: absolutepercentage difference, relative percentage dif-ference, disparity index, and ratio of dis-parity. These measures can be used todescribe disparity in stops, dispositions,searches and other aspects of vehicle stops.We turn now to a new question: Whichmeasure or measures of disparity should re-searchers select to present their data?

Social scientists analyzing vehicle stopdata have differences of opinion regardingwhether researchers should report multiplemeasures of disparity or just one. Those whoadvocate the selection and reporting of asingle measure (for instance, the disparityindex) point out that multiple measurescould confuse the residents, policy makers,and other stakeholders who read the agency’sreport. Multiple measures, they say, mightlead the various stakeholders with differentconcerns or agendas to pick and choose thefigures in the report that confirm their viewsor preconceived expectations regarding theresults.

Other social scientists favor reportingtwo, three, or even all four of the measures ofdisparity. They claim it is better to provide

report consumers with more information, notless, including information on how variousmeasures can produce different results in dif-ferent circumstances.

Indeed, different measures do producedifferent results, and researchers and juris-diction stakeholders need to understand thisimportant fact. Care must be exercised in theinterpretation of the findings. If the percent-ages of minorities (or of Caucasians) in thepopulation of stopped drivers or in thebenchmark population are not very high orvery low, the researcher’s choice of onemeasure of disparity over another will nothave strong ramifications for the results. Onthe other hand, when a researcher is dealingwith very high or very low percentages of mi-norities (or of Caucasians), the selection ofone measure over another will lead to verydifferent interpretations of the results.

DIFFERENT MEASURES OF DISPARITY:DIFFERENT INTERPRETATIONS

Table 7.3 shows how the four measures ofdisparity explained in the first section of thischapter can convey very different results.The table presents four measures of disparityfor three hypothetical police departments:A, B, and C. Which department has the most

Table 7.2. Disparity Indexes and Ratios of Disparity to Describe Stops in Hypothetical Area A, for Three Racial/Ethnic Groups

A B C F G

Numberof Stops

Percentof Stops

Percent ofBenchmark

Disparity Index(B/C)

Ratio of Disparity

Formula Result

African Americans (a) 8,798 10.82% 7.40% 1.46 F(a)/F(w) 1.53

Hispanics (h) 6,694 8.24% 8.20% 1.00 F(h)/F(w) 1.05

Caucasians (w) 65,789 80.94% 84.40% 0.96

81,281 100.00% 100.00%


disparity? Well, the answer depends on themeasure of disparity we consider.

In terms of the absolute percentage differ-ence, Department C has the most disparity:African Americans are over-represented inthe stop data relative to the benchmark databy 13.0 percent. In terms of the other threemeasures of disparity, Department B has themost disparity. Although Department B hasan absolute percentage difference of only 0.7,it has a relative percentage difference of 117.The disparity index and ratio of disparity forDepartment B are both 2.2. Department Ahas the second highest disparity when dis-parity is calculated as the relative percentagedifference (56 percent) or disparity index(1.6); Department C has the second highestdisparity (1.7) when calculated as ratios ofdisparity. Clearly, the measure chosen makesa difference in terms of the level of disparityindicated.

Recall the important point made earlier: ifthe population of stopped drivers or the pop-ulation of the benchmark population has veryhigh or very low percentages of minorities (orof Caucasians), the researcher’s selection ofone measure over another could make a big

difference in the interpretation of results.Table 7.3 shows that the percentage of mi-norities in both the stopped driver populationand the benchmark population is low for De-partment B; as a result, the variation betweentwo of the measures of disparity (the absolutepercentage difference and the relative per-centage difference) is extreme.3 Minoritiesrepresent only 1.3 percent of the personsstopped and only 0.6 percent of the bench-mark population; the absolute percentage dif-ference is tiny (0.7 percent), but the relativepercentage difference is large (117 percent).

This extreme variation is even more evi-dent in Table 7.4. In order to highlight theeffects of low levels of minorities in the stopand benchmark populations on the fourmeasures of disparity, we arbitrarily set theabsolute percentage difference at 2 percentfor thirty-five hypothetical departments. Forlow levels of minority representation (the topof Table 7.4), the relative percentage differ-ence can be very high—misleadingly high—even when the absolute percentage differenceis low (in these cases, 2 percent). For Depart-ment 2, minorities comprise 3 percent ofthe drivers stopped and 1 percent of the

3 Here we focus on the situation when the percentageof minorities is low in the stop and/or benchmarkpopulations. The same problems would occur if

Caucasians were the group with low percentagerepresentation.

Table 7.3. Various Measures of Disparity for Hypothetical Departments A, B, and C

Department

Representation ofAfrican Americans

Among Stops

Representation ofAfrican AmericansAmong Benchmark

Absolute% Difference

Relative% Difference

DisparityIndex

Ratio ofDisparity

A 14.0% 9.0% 5.0% 56.0% 1.6 1.6

B 1.3% 0.6% 0.7% 117.0% 2.2 2.2

C 67.0% 54.0% 13.0% 24.0% 1.2 1.7

Source: Farrell 2004


Table 7.4 Disparity Measures for Multiple Departments When Absolute Percentage Difference is Set at Two

Dept.

Percent of Stops Percent of Benchmark Percentage Difference Disparity Index Ratio ofDisparityCaucasians Minorities Caucasians Minorities Absolute Relative Minority Caucasian

1 98 2 100 0 2.0 NA* NA* 0.98 1.02

2 97 3 99 1 2.0 200.00% 3.00 0.98 3.06

3 96 4 98 2 2.0 100.00% 2.00 0.98 2.04

4 95 5 97 3 2.0 66.67% 1.67 0.98 1.70

5 94 6 96 4 2.0 50.00% 1.50 0.98 1.53

6 93 7 95 5 2.0 40.00% 1.40 0.98 1.43

7 92 8 94 6 2.0 33.33% 1.33 0.98 1.36

8 91 9 93 7 2.0 28.57% 1.29 0.98 1.31

9 90 10 92 8 2.0 25.00% 1.25 0.98 1.28

10 89 11 91 9 2.0 22.22% 1.22 0.98 1.25

11 88 12 90 10 2.0 20.00% 1.20 0.98 1.23

12 83 17 85 15 2.0 13.33% 1.13 0.98 1.16

13 78 22 80 20 2.0 10.00% 1.10 0.98 1.13

14 73 27 75 25 2.0 8.00% 1.08 0.97 1.11

15 68 32 70 30 2.0 6.67% 1.07 0.97 1.10

16 63 37 65 35 2.0 5.71% 1.06 0.97 1.09

17 58 42 60 40 2.0 5.00% 1.05 0.97 1.09

18 53 47 55 45 2.0 4.44% 1.04 0.96 1.08

19 48 52 50 50 2.0 4.00% 1.04 0.96 1.08

20 43 57 45 55 2.0 3.64% 1.04 0.96 1.08

21 38 62 40 60 2.0 3.33% 1.03 0.95 1.09

22 33 67 35 65 2.0 3.08% 1.03 0.94 1.09

23 28 72 30 70 2.0 2.86% 1.03 0.93 1.10

24 23 77 25 75 2.0 2.67% 1.03 0.92 1.12

25 18 82 20 80 2.0 2.50% 1.03 0.90 1.14

26 13 87 15 85 2.0 2.35% 1.02 0.87 1.18

27 8 92 10 90 2.0 2.22% 1.02 0.80 1.28

28 7 93 9 91 2.0 2.20% 1.02 0.78 1.31

29 6 94 8 92 2.0 2.17% 1.02 0.75 1.36

30 5 95 7 93 2.0 2.15% 1.02 0.71 1.43

31 4 96 6 94 2.0 2.13% 1.02 0.67 1.53

32 3 97 5 95 2.0 2.11% 1.02 0.60 1.70

33 2 98 4 96 2.0 2.08% 1.02 0.50 2.04

34 1 99 3 97 2.0 2.06% 1.02 0.33 3.06

35 0 100 2 98 2.0 2.04% 1.02 NA* NA*

*Not applicable because formula places a zero in the denominator of the equation.


benchmark population; the absolute per-centage difference of 2 percent is paired witha relative percentage difference of 200 per-cent. Similarly, the disparity index for mi-norities and ratio of disparity are very high at3.0 and 3.06, respectively.

Stakeholders need to understand that dif-ferent measures of disparity can, in some cir-cumstances, lead to different interpretationsof the results. With this knowledge they canengage in a discussion with the researcherprior to the production of results regardingsome key decisions such as how many meas-ures of disparity will be produced and whichone or ones will be selected. Even if stake-holders are not involved in those decisions,this knowledge will help them understand re-sults that include multiple measures of dis-parity that produce various interpretations. Ifstakeholders are presented with results con-veyed with a single measure they will under-stand that the results might have beendifferent if the researcher had made an alter-native selection.

USING CONTINGENCY TABLESTO IDENTIFY DISPARITY

The relationship, if any, between the race/ethnicity of drivers and various actions bypolice (such as stops, searches, and disposi-tions) can be assessed using contingency ta-bles. These tables have a consistent format:the independent variable defines thecolumns, and the dependent variable definesthe rows. Table 7.5 portrays hypotheticalsearch data in contingency table format. Theindependent variable is the race/ethnicity ofthe driver, and the dependent variable is

whether or not a search was conducted.4

Column percentages sum to 100 percent, andthe table is read across. Searches were con-ducted of 17.61 percent of the stoppedAfrican Americans, 11.58 percent of thestopped Hispanics, and 7.00 percent of thestopped Caucasians.

These results indicate that African Amer-icans were more likely than Hispanics andCaucasians to be searched. But what doesthis finding mean? It means only that a dis-parity exists. Researchers cannot concludethat bias influenced search decisions becauseother factors could have caused the disparity.

Researchers can use statistical programsto assess the strength of a relationship that isindicated by the data in a contingency table.Importantly, measures of association (andtests of statistical significance) provide infor-mation regarding disparity, not bias. For in-stance, if we had found a strong associationindicating that African Americans were dis-proportionately represented among driverssearched, we would know only that a dis-parity exists, not why it exists. We cannotconclude that bias influenced search deci-sions because other factors could havecaused that disparity.

USING MULTIVARIATE ANALYSISTO IDENTIFY DISPARITY

Multivariate analysis examines the impactof multiple factors (independent variables)on an outcome (the dependent variable).5 Itcan provide a more thorough and accurateinterpretation of vehicle stop data than bi-variate analysis.

4 A dependent variable is the outcome variable, or thesubject of the analysis. An independent variable is thepredictor variable that is hypothesized to cause changesin the dependent variable. In this example, we aretesting whether race/ethnicity of a driver (the inde-pendent variable) will impact on whether or not asearch is conducted (the dependent variable).

5 In bivariate analysis, researchers look at the relation-ship between two variables. In multivariate analysis,multiple variables are taken into consideration, and thestrength of the relationship between each independentvariable and the dependent variable is determinedwhile controlling for the impact of the other variables inthe equation.


For example, Smith et al. (2003) andTomaskovic-Devey, Wright, and Czaja (2003)analyzed information from a survey of driversin North Carolina, including information onthe extent to which the drivers were stoppedby police. These researchers wanted to findout whether the driver race/ethnicity affectedthe extent to which people were stopped.The frequency of being stopped during thereference period was the dependent variable.A bivariate analysis with these data wouldlook at the relationship between the race/eth-nicity of the survey respondents and thenumber of stops by police they reported. Re-searchers would not know from this bivariateanalysis, however, if variables like drivingquantity, quality, or location had affected thestopping decisions by police. Researcherscould show whether disparity existed (for in-stance, they might find that minorities werestopped more than Caucasians), but theywould not know if race—or alternative, legit-imate factors—produced that disparity. If asurvey data set on stopped drivers in a juris-diction included information on drivingquantity, quality, and location (and the NorthCarolina survey did), researchers conductingmultivariate analysis could look at the effectof race on the frequency of being stopped,controlling for these other factors.

The Key Limitation ofMultivariate AnalysisMultivariate analysis is an important tool forsocial science and can have value for an ex-amination of racial bias in policing. It doesnot, however, overcome the challenges asso-ciated with analyzing vehicle stop data—particularly those challenges associated withidentifying and measuring the alternativelegitimate factors that can influence policedecision making. Multivariate analysis isbased on certain assumptions, and a key oneis “no specification error.” This is a fancyphrase used by statisticians to reference a keytheme of this book: for a method to be mosteffective it must take into consideration all ofthe alternative legitimate factors that mighthave an impact on police behavior. For mul-tivariate analysis to be effective in deter-mining whether driver race/ethnicity has acausal impact on police behavior, it must in-clude independent variables that reflect thealternative legitimate factors that affect policebehavior.

A researcher might find a significant rela-tionship between independent variable X anddependent variable Y that would disappear ifthe researcher had included variable C in themodel. A simple example illustrates thispoint. Let us imagine that a researcher finds

Note: The Contingency Coefficient is 0.121.

Table 7.5. Contingency Table to Assess Relationship Between Driver Race/Ethnicity and Police Searches

Driver Race/Ethnicity

Search AfricanActivity Americans Hispanics Caucasians Total

No Search 7,249 5,919 61,184 74,35282.39% 88.42% 93.00% 91.48%

Search 1,549 775 4,605 6,92917.61% 11.58% 7.00% 8.52%

Total 8,798 6,694 65,789 81,281100.00% 100.00% 100.00% 100.00%


a significant positive relationship betweenthe consumption of high-grade coffee and thesquare footage of homes. Subjects who drinkhigh-grade coffee, the researcher finds, aremore likely to live in large houses. Clearly,drinking high-grade coffee does not cause aperson to have a large house. The “omittedvariable” C, which is wealth, leads to both thedrinking of high-grade coffee and the pur-chase of large houses. Without the inde-pendent variable C in the model, the resultsare misleading: the results indicate a directrelationship where none exists. With wealthin the model, the multivariate methodswould indicate a relationship between wealth(not high-grade coffee) and large houses.

Applied to vehicle stops, multivariateanalysis can similarly identify a misleadingrelationship between the dependent variableand the independent variable. It is mis-leading because the inclusion of a previouslyomitted variable can make the relationship orcorrelation disappear. For example, multi-variate analysis might find a relationship be-tween race/ethnicity and police dispositionsthat would have disappeared (as it did in theanalysis of the Washington State Patrol data)if the researcher had included number of vio-lations or seriousness of offense(s) as inde-pendent variables.

Not including key variables in a multi-variate equation can also serve to “mask”racial bias. A researcher may, for instance,find no indication of racial disparity in searchdecisions—where, in fact, it exists—becausethe researcher fails to include in the equationcrucial independent variables.

For multivariate analysis to be most effec-tive in determining whether driver race/ethnicity has a causal impact on police be-havior, it would include all of the inde-pendent variables that reflect the alternativelegitimate factors that affect police behavior.Quite frequently, however, social scientistscannot identify or measure all of the factors

that they should or would like to include asindependent variables. This is not unique tothe analysis of vehicle stop data.

Researchers should not lead stakeholdersto believe that the use of fancy multivariatestatistical techniques, however beneficial,overcomes all the challenges associated withanalyzing vehicle stop data. The researchershould make explicit reference to the poten-tially relevant variables that were not in-cluded in the equation and report that theseomissions could have had an impact on theresults.

WHEN DOESDISPARITY MEAN BIAS?

Isolating the causes of disparity presents aformidable challenge for researchers. Anidentified “amount” of disparity in stoppingbehavior by police could be caused by any ofthe following: bias on the part of police; de-mographic variations in the quantity, quality,and location of driving; demographic varia-tions in other legitimate factors that have animpact on police behavior; and/or othermeasurement error. Researchers don’t knowwhat proportion of the disparity comes fromwhat source. With strong benchmarkingmethods, researchers can reduce the numberof plausible causes, but only in a perfectworld where they can control for all alterna-tive, legitimate factors and achieve perfectmeasurement could they equate a disparitymeasure or measures with police bias. Forthis reason, there is no agreed upon “brightline” that researchers can set whereby dis-parity levels above it indicate racial bias anddisparity levels below it indicate lack of bias.

Note also that disparity does not indicatebias just because the results are “statisticallysignificant.” Researchers can use tests ofstatistical significance in their analysis ofvehicle stop data for descriptive purposes toshow that a finding is robust.6 For instance, aresearcher might report that the difference


between the representation of minorities inthe stop population and among the bench-mark population is “statistically significant.”This shows that the numerical differences areworthy of notice. Whether this disparity iscaused by bias cannot be discerned by thistest.

THE BRIGHT LINECONTROVERSY

Some researchers argue that conclusionsabout the existence or absence of racial biascan and should be drawn from disparitymeasure calculations in order to provide theclarity needed to guide jurisdiction policy andpractice. Other researchers claim that anycut-off point is arbitrary—providing a falsesense of clarity where none exists. They notethat even large amounts of disparity could bewholly explained by nonbias factors.

Important for stakeholders to understandis that there is no mathematical formula thatcan produce a legitimate cut-off point abovewhich one can say with confidence that dis-parity is equal to bias. A researcher may havedetected disparity between the racial/ethnicprofile of drivers stopped by police and theracial/ethnic profile of the benchmark popu-lation, but the researcher has no way ofknowing how much of this disparity is due tomeasurement error and unmeasured vari-ables that influence police behavior. The re-searchers who advocate cut-off points are, ineffect, arguing that if the disparity is particu-larly large, then chances are, the alternativefactors cannot explain all of it. Certainly, it isprobably safe to say that the larger disparitiesare more likely than the smaller disparities toencompass many causes, including bias. It isimportant to note, however, another possi-bility: a large disparity could be produced

entirely by alternative legitimate factors, anda small disparity could be entirely producedby bias.

For the researchers who choose to select acut-off point, we suggest they select the cut-off point before analyzing the results if fea-sible and set the cut-off point in conjunctionwith a police-resident advisory board aftereducating that board about the challenges ofdrawing conclusions about police bias fromcalculations of measures of disparity. Mostimportantly, researchers need to convey tothe consumers of their reports the constraintsassociated with setting cut-off points.

A researcher might reasonably choose notto select a cut-off point, believing it unwise toselect a point above which “a problem” is in-dicated. The quote below expresses the rea-sons why one group of researchers decidednot to set a cut-off point:

As with other studies, we faced aproblem of establishing a “bright line”above which the conclusion is that alldepartments are engaged in disparatecitation practices that constitute racialprofiling and below which all depart-ments are not engaged in disparate ci-tation practices. . . . In studies ofdisparity, regardless of topic area, it isgenerally inappropriate to concludethat any difference between thestudied population and the compara-tive population automatically consti-tutes a meaningful disparity or racialbias. Such differences may be the re-sult of real differences or may be aproduct of sampling or measurementerror (Farrell et al. 2004, 15).

These researchers conclude, “How muchdisparity is acceptable to a community is

6 More often tests of statistical significance are usedin research to make inferences about whether the re-sults from a sample can be generalized to the popula-tion from which that sample was randomly drawn.

However, most data that are studied to assess the exis-tence of racial bias represent information (gleaned fromforms) on all police stops made in a jurisdiction, not arandom sample.


fundamentally a question that should be ad-dressed by stakeholders and policy makers ineach jurisdiction” (Farrell et al. 2004, 16).

THE GOOD NEWSWhen grappling with the question of “howmuch [disparity] is too much,” researcherscan avail themselves of two important tools.First, they can compare disparities. Suchcomparisons will help them interpret theirdata and provide stakeholders with mean-ingful feedback and guidance. As an example,an agency using internal benchmarking iden-tifies the officers (or units of officers) with the“most disparity” and initiates a review thatwill determine whether there are explanationsother than bias for the disparity (see Chapter5). Similarly, other benchmarking methodscan identify geographic areas within a juris-diction, agencies within a state, and/or unitswithin a department with the highest levels ofdisparity. The researcher could rank thosesubareas, agencies, and departmental unitsbased on measures of disparity. A descriptivecut-off point could then be selected. A unitabove the designated level is considered onlyto have high disparity but no interpretation ismade as to the cause of that disparity. Suchan identification can serve to help policymakers identify the high priority targets foradditional review or for change efforts as dis-cussed more fully in the next chapter.

A second tool to help researchers inter-pret and report findings of disparity is whatwe call a “qualitative review of quantitativedata.” By meeting with law enforcementagency personnel and with other stake-holders, researchers can gain insight and per-spective on the quantitative results. Thesereviews can help ensure that jurisdiction dataare correctly and responsibly interpreted.Two reviews are advisable: (1) a review anddiscussion of the results by researchers andlaw enforcement agencies, and (2) a review

and discussion of the results by law enforce-ment personnel and resident stakeholders.

The independent researcher or researcheremployed by the law enforcement agencyshould discuss the results of vehicle stop dataanalysis with sworn personnel before pub-lishing them. The purpose of this discussionis to gather information from a “street per-spective” regarding what the data mean. Thepurpose is not to “explain away” any dis-parity that may have been identified but tobetter understand what factors—legitimate orotherwise—might be producing the results.

The Northeastern University team, inboth its Massachusetts (Farrell et al. 2004)and Rhode Island (Farrell et al. 2003) reports,indicates that the ultimate interpretation ofthe results comes during discussions betweenpolice and citizens. One benefit of includingresidents in discussions of results is the freshand helpful perspective they bring to under-standing what the data mean. Like the police,residents have information about the jurisdic-tion that can add perspective and context tothe numbers produced by the researcher. Butdiscussions between police and residents areabout more than how to interpret data. Theissue of racially biased policing has, in manycommunities, exacerbated the “divide” be-tween police and residents, particularly resi-dents who are racial/ethnic minorities. Datacollection has the potential to help heal thedivide and provide direction for joint reformefforts by police and community members.Police-resident discussions of data become apart of the change process. We discuss in thenext chapter how police and residents cancome together to use these data for the pur-poses of reform.

CONCLUSIONIn this chapter we discussed four ways topresent the results of vehicle stop analyses:absolute percentage differences, relative per-centage differences, disparity indexes, and


ratios of disparity. These measures can beused to present results on stops, searches,dispositions, and other types of vehicle stopdata. Researchers can choose one or more ofthese measures to convey the results of theirbenchmarking analysis to the public. Thechallenge is not in producing these measuresof disparity but in deciding which one orones to use and present. Some social scien-tists use just one measure of disparity in theirreports to reduce ambiguity and avoid mul-tiple interpretations of results. Others preferto report multiple measures of disparity.Under some circumstances, the interpreta-tions drawn from one measure might be verydifferent from those drawn from another. Re-searchers might also develop contingency ta-bles to convey results or use multivariatemethods to analyze the data.

This chapter also explained why defini-tive conclusions about bias cannot be drawnfrom calculations of measures of disparity.Stakeholders must evaluate the extent towhich nonbias factors (factors related todriving quantity, quality, and location) havebeen addressed by the jurisdiction’s bench-marking method. Conclusions about racial/ethnic bias as the cause of disparity are sus-pect because every benchmarking methodimperfectly addresses the alternatives to thebias hypothesis.

In the next chapter we describe how policeand stakeholders can come together, reflectupon the vehicle stop data analyzed by socialscience researchers, and identify methods forimproving policing practices and the relation-ships police have with local residents.

VIIIUsing the Results for Reform

Vehicle stop data have benefits and con-straints as a means of measuring whetherpolicing in a jurisdiction is racially biased.The limits of social science preclude re-searchers from drawing definitive conclu-sions from the data regarding the existence orlack of racial bias. Faced with this fact, ex-plained at length in previous chapters, thereader well might ask: of what value are theresults if researchers cannot report, with con-fidence, the existence or lack of racial bias inthe jurisdiction?

The answer is that the results of bench-marking analysis can be of significant value.These results can serve as a basis for con-structive dialogue between police and resi-dents, which can lead to (1) increased trustand cooperation and (2) action plans for re-form.1 In its report on traffic stop data for thestate of Rhode Island, the Northeastern Uni-versity team wrote: “We do not view thisanalysis as an end of the discussion about theexistence and extent of racial profiling in

Rhode Island, but rather it will provide . . . in-formation to begin an important dialogue. . . .[A] well conceived and implemented study ofracial disparities in traffic stops can serve asa very useful springboard for communitylevel conversations about the issues of racialprofiling” (Farrell et al. 2003, 6).

Below we describe various ways that po-lice and resident stakeholders2 can come to-gether to reflect on the results of datacollection. The ultimate aim of these meet-ings is mutual understanding and reform.Specifically, we describe in this chapter

l

who should be brought together;

l

what information—including vehiclestop and poststop results—this groupmight explore; and

l

the types of changes the group mightrecommend.

As articulated by Chief John Timoney(2004) of the Miami Police Department, the

1 This should not be construed as an endorsement ofmandatory data collection. As indicated in the firstPERF publication on the subject (Fridell et al. 2001),there are pros and cons of data collection that a local ju-risdiction or state should consider before making a de-cision regarding whether to collect data.

2 In this chapter the term “resident stakeholders” refersto citizens, journalists, advocacy group members, gov-ernment officials, and others who reside in the commu-nity and have a particular stake in the outcome ofresearchers’ race data analysis.


reality is that “race is a factor in policing.”Every police executive needs to consider andaddress the issues of racially biased policingand the perceptions of its practice. Becauseall agencies can make progress on this issueand because the data will never “prove” or“disprove” racially biased policing, vehiclestop data collection and analysis shouldnever be viewed—either by police or residentstakeholders—as a “pass-fail test” (Farrell2004). Instead, it should be viewed as a diag-nostic tool to help the agency, in concert withconcerned residents, set priorities for ad-dressing the problem or perception of racialprofiling. The collection and analysis of ve-hicle stop data can pinpoint geographic sub-areas of a jurisdiction or particular policingprocedures that warrant further study. Inorder to make full use of researchers’ analysisof vehicle stop data, jurisdictions are encour-aged to convene a local task force on racialprofiling.

THE TASK FORCEAND ITS MEMBERSHIP

In Chapter 3, “Getting Started,” we recom-mended that jurisdictions create a local racialprofiling task force to guide police depart-ments in the development of their data col-lection system.3 This task force, composed offifteen to twenty-five people, could plan howdata would be collected and analyzed. Thetask force would bring credibility to the datacollection system, and its members wouldunderstand both the limits and the potentialof vehicle stop data analysis. We recommendincluding people in the community who aremost concerned about racial bias and policepersonnel representing all departmentallevels, particularly patrol.

It is preferable, but not essential, that thetask force be convened before data collectionbegins. If it is formulated after data collectionhas started, however, it still has an importantmission—engaging in constructive dialogueto identify where change is needed. Thisgroup should meet and begin its work beforethe report of findings on the vehicle stop dataanalysis is publicly released.

A group with equal representation of lawenforcement personnel and resident stake-holders should review and discuss the data.Nonresident stakeholders also could be in-cluded. They could be representatives fromstate or national groups, such as the Amer-ican Civil Liberties Union (ACLU), the Na-tional Association for the Advancement ofColored People (NAACP), and the UrbanLeague; nonresident commuters to the juris-diction; and nonresident owners of busi-nesses located in the jurisdiction.

It is usually appropriate for the agency ex-ecutive to call for and develop this task force.It then serves in an advisory capacity to theexecutive and makes recommendations thathe or she will consider adopting. The agencyexecutive should not be a member of thegroup since it has been convened to providehim or her with advice on what actions totake. We recommend, however, that the exec-utive attend the task force meetings. By at-tending the meetings, the executive canconvey to task force members, the executive’sstaff, and the wider community the impor-tance of the issue. There may be circum-stances when another official or groupdevelops the task force rather than the law en-forcement agency executive. For instance, amayor or city council might call for a taskforce for a jurisdiction or a governor mightconvene a statewide task force. The executive

3 Because data collection was organized at the statelevel, the Northeastern University team had a state-level

task force advising it. The team, however, advocatesthat discussions of the data occur at the local level.

Using the Results for Reform 81

should be a member of the task force if it wasnot set up and overseen by the executive; themembers would make recommendations tothe person or organization that developedtheir group.

The local racial profiling task forceshould meet on an ongoing basis. For some ofthe early discussions described below (for in-stance, on trust-building and on general is-sues and concerns related to racially biasedpolicing), we advise the use of a trained, neu-tral (nonpolice, nonstakeholder) facilitator.This facilitator should have experienceworking with groups on issues that provokeemotions and passions and have knowledgeof the topic of racially biased policing. Thisfacilitator might be retained to oversee thelong-term work of the task force or, after theearly sessions, turn over meeting facilitationto a task force chair or to co-chairs. For theco-chair model, the group may elect, or haveappointed, one co-chair who is an internalstakeholder (that is, affiliated with the lawenforcement agency) and another who is anexternal stakeholder. This group may have afinite tenure, or it may become a permanentfixture in the jurisdiction.4

THE AGENDA OF THE POLICE-STAKEHOLDER TASK FORCE

The first few sessions of the task force (ses-sions led by a neutral facilitator, as explainedabove) should be devoted to developing trustbetween police and resident members. Thetask force then would

l

discuss general concerns related toracially biased policing,

l

review the vehicle stop data,

l

review other sources of informationabout racial bias and perceptions ofracial bias, and

l

consider possible reforms.

Developing TrustIn a rare situation, a stakeholder group maybe able to begin its discussions of racially bi-ased policing at the first meeting; mostgroups, however, will be well served by en-gaging in some exercises and discussions ontopics other than racial bias before delvinginto the volatile topic that brings them to-gether. A group in Lowell, Massachusetts—not a task force but a group formed for aone-time discussion—began immediatelytalking about racial bias. After some finger-pointing, raised voices, accusations by citi-zens against police, and defensiveness on thepart of police, the group turned its attentionto developing ways to resolve the particularproblems it had identified. On their own,without prompting from the facilitator, thegroup members agreed that they needed tomeet regularly to continue the process ofsharing, listening, and resolving problems.Ed Davis, Superintendent of the Lowell PoliceDepartment, continued the group as the“Race Relations Council,” which the mayorlater described as “the best thing that hashappened in Lowell in a long time.”

Although this particular group during asingle session was able to move from theheated and angry exchanges at the beginningof the meeting on the controversial issue ofrace to a sober and rational discussion of aconstructive plan of action, most groupscannot. We recommend that task forces en-gage in activities that will develop trust

4 For various reasons, a jurisdiction may be unable (orunwilling) to convene a task force of police and stake-holders. In such circumstances, the department shouldconvene personnel to discuss key topics outlined below,

including general issues related to racially biasedpolicing, the vehicle stop results, other sources of infor-mation, and needed reforms.


among members before tackling the chal-lenging topics that define their existence.This trust-building may require a number ofmeetings.

The Chicago ForumsOne trust-building model comes fromChicago. The former superintendent of theChicago Police Department, Terry Hillard,sponsored a series of forums for police andminority residents of the community. Com-munity activists were recruited to aid the po-lice department in its search for solutions toracial tensions. Department staff of all rankswere also invited to participate. Before thefirst forum was convened, participants weresurveyed for their opinions about racially bi-ased policing and the department’s strengthsand weaknesses regarding minority outreach.In the survey, respondents also were asked fortheir ideas on how to improve relations be-tween police and minorities and for theirthoughts on how to resolve issues. A facili-tator moderated the initial sessions.

During the morning session of the firstforum, community members were asked totalk about strengths and weaknesses of theirinteractions with the police, and police staffwere asked to listen and hold their responsesuntil later in the day. Lunch was structuredas a mixer, with informal discussions. In theafternoon, police staff shared their thoughtsand reactions to the morning session, and res-idents were instructed to listen and not re-spond. Then there was an opportunity fordiscussion. While the issue of racially biasedpolicing was raised by both groups duringthis first meeting, it was just one of many is-sues raised. Race issues became a more cen-tral focus in subsequent forums and, duringthose gatherings, the group identified specificactions to be taken by both police and com-munity members to address them. Superin-tendent Phil Cline who succeeded Hillard hascontinued these forums.

The Lamberth WorkshopsJohn Lamberth’s consulting team uses a two-session workshop to “enhance the trust be-tween law enforcement and the localcommunity” and to develop “collaborativecommunity-based racial profiling solutions”(Clayton 2004). For the first gathering, theLamberth team holds separate sessions withthe police and resident stakeholder partici-pants. The purpose of these separate discus-sions is to “enhance participants’ under-standing of the issues” surrounding raciallybiased policing.

Discussions within these separate groupsaddress the definition of racial profiling, dif-fering perceptions of the issue on the part oflaw enforcement and resident stakeholders,and the expectations and responsibilities ofpolice and drivers during vehicle stops. Bythe end of the first session, each group hasidentified

l

safety issues that concern police,

l

concerns or fears that drivers mighthave when stopped by police,

l

ways racial profiling harms police-community relations, and

l

the group’s expectations when makingcontact with the other group.

During the second session of the work-shop, the group composed of police and thegroup composed of resident stakeholders arebrought together for small- and large-groupdiscussions and activities. The police groupand the resident stakeholder group reviewtheir separate discussions from session oneand identify the areas where their expecta-tions and perceptions are shared and wherethey are different. Together, the first sessionand first half of the second session serve toinitiate constructive dialogue, develop trustbetween participating police and residentstakeholders, and identify common concernsand expectations. These sessions set thestage for the rest of the workshop during


which the participants develop a plan of ac-tion for addressing issues related to raciallybiased policing and the perceptions ofracially biased policing.

Reviewing General Concerns Relatedto Racially Biased PolicingWe have discussed the first item on theagenda of a local racial profiling task force: de-veloping trust. We have described twomethods for developing trust and enhancingcommunication during non-stress times:forums convened in Chicago for policeand minority residents and two-session work-shops developed by John Lamberth’s con-sulting team. Following trust-buildinggatherings similar to those we have described,members of the task force should review gen-eral concerns and perceptions related toracially biased policing.

To provide structure to the potentiallyheated conversation, the facilitator might in-vite resident stakeholders to share their con-cerns—allowing them to voice theirperspective without defensive responses bythe police. While the police might feel in-clined to “explain away” all the concernsvoiced by citizens (and, indeed, there will beincidents described by residents where thepolice feel strongly—and maybe correctly—that there is a race-neutral explanation), itwill ultimately be more valuable for the po-lice to just listen to the residents’ concerns.Residents need to be heard on this issue andtaken seriously. This discussion also canhighlight for police how important it is todeal with perceptions that police in the juris-diction are racially biased. Then the facili-tator could ask police on the task force toshare their concerns related to accusations or

perceptions that bias is influencing theirpolicing decisions.5

Reviewing the Vehicle Stop ResultsAfter a general airing of concerns, the taskforce should be ready to conduct a qualitative(that is, nonempirical) review of the quantita-tive data on vehicle stops (see Chapter 7).This review is a continuation of the dataanalysis. During the researcher’s earlier em-pirical examination of the stop data, all of thefactors that might have influenced stoppingdecisions by police could not have been con-sidered. A “qualitative” review allows for aconstructive assessment of the factors, otherthan bias, that might account in whole or inpart for findings of disparity (or lack thereof)between the racial/ethnic profile of the popu-lation of stopped drivers and the racial/ethnicprofile of the benchmark population. The po-lice and residents who have been brought to-gether on the task force have an importantcontribution to make. They have valuableknowledge researchers don’t have about lawenforcement activities and geographic areasin the jurisdiction. Therefore, they can pro-vide a unique and helpful perspective for un-derstanding the empirical results obtained bythe researchers.

The goal of the qualitative review of quan-titative data is not to determine whether theagency “passed” or “failed” a racial profilingtest. As stated earlier, the goal is to identifygeographic areas, procedures, and decisionsthat should get the highest priority when thepolice department initiates efforts to addresscommunity concerns. Even though the quan-titative data cannot provide the whole pictureor a perfect picture, the data, if carefully in-terpreted, can direct the task force toward

5 The task force should include police leaders at allranks who are open to exploring the issue of policeracial bias and committed to identifying ways of doing

business that can reduce or prevent the problem andperceptions of the problem. These people should beproblem solvers and consensus builders.


particular reform targets such as stops of mi-norities for equipment violations, consentsearches of young African American males,or vehicle stops on the “south side” of thecity.

Before reviewing the data, members of thetask force should become informed aboutwhat can and cannot be understood from theanalysis of vehicle stop data. They could beencouraged to read this book, which waswritten to clarify these issues for stake-holders, or they could be educated someother way (perhaps by the researcher) aboutthe meaning of key terms, such as “bench-marking,” “disparity,” and the difference be-tween disparity and bias. Once all membersof the group have a good preliminary under-standing of vehicle stop analysis, they can re-view the stop and poststop data. Thefollowing questions can help guide this dis-cussion of the data:

l

Are there indications of disparity inthe stop or poststop results?

l

Are there reasons, other than racialbias, that might have led to thesedisparities?

l

For what activities (for example, stops,searches, choice of disposition) isracial bias a possible or probablecause?

l

Regardless of whether or not bias is acause, what is the impact of particulardisparities on residents and on rela-tions between police and residents ofthe jurisdiction?

l

Do the costs of certain policing prac-tices that produce disparities—practicesthat may be race-neutral—outweigh thelaw enforcement benefits?

Each of these questions will now be ex-amined in greater detail.

An appropriate first question to launchthe discussion of the vehicle stop data is “Arethere indications of disparity?” The groupmust keep in mind that indications of “dis-parity” not “bias” are being discussed.6 Thisconversation about disparity may be shorterthan later conversations about the other ques-tions listed. The key is to summarize whatdisparities were identified by the empiricalanalyses.

The more interesting, challenging, andlonger discussion will focus on the reasons,other than racial bias, that might have led tothese disparities. This conversation mightstart by focusing on each specific finding ofdisparity (for instance, disparity in stopsacross racial groups in Area A). Participantsmight reflect on how the methods used toproduce the measure did or did not capturecertain important factors. For instance, a res-ident participant might point out that racialdisparity in stops around a stadium that wasidentified using census benchmarking mightreflect the high volume of nonresident, multi-racial/ethnic traffic on game days. An officermight report that the high level of stops in aparticular minority area is the result, at leastin part, of requests from residents in that areafor strong enforcement of the speed limit.

Indeed, the purpose of the discussion ofthese on-the-ground realities is not to “ex-plain away” disparities but to examine legiti-mate factors that might account, at least inpart, for them. The task force will also con-sider the possibility that certain identifieddisparities could be the result of biased deci-sions by police. The group should considerthe possibility that bias has caused disparity

6 The group should also be reminded that the samemethodological challenges that keep researchers fromequating disparity with bias can produce results

showing no disparity when racial bias does, in fact,exist (see Myth 1 in Chapter 2).


if it cannot identify alternative, legitimate ex-planations for findings of disparity; if there isan accumulation of disparity findings or verylarge levels of disparity; and/or if a particularpolice activity is highly discretionary andthus vulnerable to bias.

The results—whether quantitative or qual-itative or both—will never lead to definitiveconclusions, a key point repeated often in thepreceding chapters. Despite these inevitableconstraints, the conversation between policeand stakeholders and researchers is worth-while and should continue. The task forceis not looking for “proof” of racial bias. (If it is,it will not find it.) Instead the task force istrying to identify priorities for its initialchange efforts.

Even if task force members do not viewracial bias as the cause for particular identi-fied disparities in vehicle stop data, their de-liberations may reveal the need for somechanges in police procedures. Law enforce-ment activities may not be influenced bybias, but they may be detrimental nonethe-less. It is constructive for the group to dis-cuss the potential negative impact on thejurisdiction of even (potentially) race-neutraldisparities and target efforts to change them.

For instance, data on poststop activity bypolice may indicate that African Americansare much more likely than Caucasians to beasked to consent to a search; the data alsomay show that these consent searches arevery unproductive (as measured by hit rates).Although it may not be possible to determinewhether bias produced this disparity (seeChapter 7), the group may decide to recom-mend some changes nonetheless. Such a rec-ommendation may make sense if minorities

in the community perceive racial bias in theserequests by police. This disparity insearches—regardless of whether it is causedby bias—may be too costly in terms of rela-tions between police and minorities. Thefrustration and anger of minorities may be toohigh a price to pay for whatever crime controlvalue is derived.

Reviewing Other Sources ofInformation about Racial Biasand Perceptions of Racial BiasIn addition to vehicle stop data, there areother sources of information that task forcesshould consider when trying to identify posi-tive steps the jurisdiction can take to addressracially biased policing and perceptions of itspractice. These alternative sources could in-clude conventional wisdom regarding thetypes of law enforcement activities that mightbe most vulnerable to officer biases, surveysof jurisdiction residents to assess their per-ceptions of policing, and results of focusgroups held around the jurisdiction.7 Thetask force also might want to review othersources of data within the department (for ex-ample, aggregate data on official complaintsagainst officers, data on the use of force, andarrest data).8 Selected tapes from in-car videocameras might be another valuable source ofinformation.

Considering Possible ReformsThe discussions outlined above canstrengthen the police-community relation-ship and promote trust, as well as highlightareas of concern to guide reform efforts.These benefits, however, can be lost if themove from discussing results to discussing

7 In some jurisdictions, focus groups of residents mightbe supplemented by focus groups of nonresidents (forexample, business owners and commuters) with a stakein the professional performance of police.

8 The department researcher within the Las Vegas Met-ropolitan Police Department examined force reports. Inone analysis, he looked at the race and ethnicity of sub-jects who were cuffed during a stop and then releasedwith no arrest.


reform is predicated on a forced “confessionof guilt” on the part of the law enforcementdepartment.

Following a discussion of the vehicle stopdata by the task force, resident stakeholdersin the group (including government leaders)may demand a confession of guilt. This is amistake. A confession of guilt should not bea criterion for moving the discussions for-ward because vehicle stop data collection/analysis is not a pass-fail test. As conveyedthroughout this book, a jurisdiction will nothave “proof” of racial bias (or the lackthereof). Moreover, “proof” of racial bias isnot a prerequisite for decisions that reformsare worthwhile. All agencies can move closerto the ideal of bias-free policing. Perhapsmost importantly, exploring reform without aforced confession of guilt is the most con-structive and effective way to proceed.

Police-stakeholder discussions of “racialprofiling” that involve finger-pointing by res-idents and defensiveness by police are nothelpful. Discussions when resident stake-holders accuse police of “widespread racism”and of frequently “stopping people solely onthe basis of race” are not constructive. Thesetypes of accusations inevitably lead to defen-sive responses on the part of police. In amore constructive dialogue, the stakeholderswould acknowledge how racial/ethnic biasstill is pervasive in their community and howeven well-meaning people (including, but notlimited to, police officers) might make deci-sions that manifest bias. The police wouldacknowledge the concerns of the communityand express a willingness to engage con-cerned citizens in discussions about how tomove forward.

Without making a confession, a chief canstill acknowledge the need to address theconcerns of residents, local officials, policymakers, and other stakeholders. The chiefsmight say that, while they cannot provewhether or not their agencies have a problemwith racially biased policing, they do know

that some residents have very real concernsand perceptions of a problem that must betaken seriously. The chief could acknowl-edge that these concerns and perceptionsharm the relationship between the police andthe racial/ethnic minorities in the communityand could welcome a dialogue that leads topositive change.

No agency executive should declare his orher agency “innocent of” or “immune from”racial bias. The many caveats in this book re-garding vehicle stop data make clear whysuch a declaration is unwise. The results ofvehicle stop data analysis will never supportsuch a strong statement of innocence and, be-sides, it’s very unlikely that any agency iswithout room for improvement on this issue.A statement of innocence would anger con-stituencies that have strong concerns andperceptions of police bias, and it could signif-icantly undermine police relations with mi-norities. Furthermore, this chief could neverimplement reform measures with any degreeof acceptance from agency personnel since heor she has previously declared publicly thatthere is no problem to address.

CHARTINGCHANGE INITIATIVES

Having agreed to move forward without apublic declaration of guilt or innocence bythe law enforcement agency, the local racialprofiling task force can begin outlining spe-cific change initiatives. In this endeavor itcan use as a guide its discussions of generalconcerns regarding racial bias, vehicle stopdata that may indicate bias and/or deleteriousdisparity, and other sources of information.The interventions the task force identifiesmight be specific to a particular “finding,” orthey might be of a general nature.

First, we will consider examples of spe-cific findings that could lead to reforms. Thetask force might find in the data a largenumber of consent searches of minorities thatare unproductive (no contraband or other


evidence is found) or a curiously large pro-portion of minority stops with unproductivesearches and “no action” dispositions. To ad-dress the specific problem of many consentsearches of minorities that are unproductive,the task force might suggest that the chiefadopt an agency policy requiring citizens tosign a consent form before being searched.This consent form would inform residents oftheir right to refuse. Alternatively or addi-tionally, the task force might suggest that thechief implement a minimum “level of proof”for consent searches, such as reasonable sus-picion.9 In response to the finding of a largeproportion of minority stops with unproduc-tive searches and “no action” dispositions,the task force might suggest that the agencyexecutive revise policies or retrain officers toensure that stops are made only for legitimatereasons. The chief could establish means forcommending officers whose searches are themost productive (as measured by their hitrates).10 To reduce questionable stops, thetask force might suggest that the agency adopta policy that prohibits pretext stops.

These are a few specific changes thatcould be recommended. Broader initiativesare outlined in Racially Biased Policing: APrincipled Response (Fridell et al. 2001). Inthat book the authors argue that all agen-cies—whether they have collected vehiclestop data or not—should consider reforms inthe following areas:

l

Supervision/accountability,

l

Policies,

l

Recruitment and hiring,

l

Education and training, and

l

Minority community outreach.

Community members should be full part-ners in implementing the solutions. For in-stance, residents could help develop theagency’s policy on antibiased policing, assistwith efforts to recruit minority officers, par-ticipate in the development of a recruit or in-service training curriculum, support agencyoutreach efforts to racially diverse communi-ties, or identify external funds for the pur-chase of equipment and software that mightpromote good policing practices and greatertransparency of police decision making.

Specific findings or general conclusionsbased on the vehicle stop data might promptthe task force to recommend the collection ofmore information by the jurisdiction. For ex-ample, an agency that conducted analyses ofthe jurisdiction as a whole might choose toconduct subarea analyses to determinewhether there are particular geographic areaswhere disparities are very high. An agencythat used a relatively weak benchmark andfound areas with large disparities might im-plement a stronger benchmark in the identi-fied areas. An agency that compared itsvehicle stop data to an “external benchmark”(for instance, agencies using observationbenchmarking, benchmarking with adjustedcensus data, or benchmarking with blindversus not-blind enforcement mechanisms)might choose to implement internal bench-marking (see Chapter 5). The agency thencould identify the particular officers who

9 The reforms in this example were implemented byChief Stanley Knee in Austin after vehicle stop datashowed that greater proportions of minorities than Cau-casians were subject to consent searches. The consentsearches of minorities were not very productive, andresident stakeholders perceived that racial bias wasthe cause of this identified disparity. The chief imple-mented a consent form and a policy requiring reasonable

suspicion on the part of the officer prior to requestingconsent to search. He set a goal of decreasing consentsearches by 40 percent over two years; within one yearhe reported a 63 percent decrease (2,141 consentsearches in 2003; 804 in 2004). 10 Hit rates should not be examined in isolation, butrather within the context of other performance or pro-ductivity measures.


produced the disparity so that their policingdecisions could be subject to further review.Alternatively or additionally, an agencymight decide that additional data elementsneed to be included on its forms for recordingpolice-citizen contacts. With more informa-tion it then could further explore a potentialproblem area (for instance, consent searches).

To better understand some aspect of thedata, an agency might choose to conductfocus groups of officers, a community surveyof perceptions of racially biased policing, or aconsumer survey (for instance, a survey ofdrivers stopped by police). All of these initia-tives would help the agency to obtain positiveand negative feedback regarding communitymembers’ interactions with officers. How-ever, the police and resident stakeholders onthe task force should not emphasize data col-lection and measurement to such an extentthat the most important work—implementingchange—is neglected or postponed.

CONCLUSIONThis book has set forth both the benefits andthe limits associated with the use of vehiclestop data to measure whether policing in a ju-risdiction is racially biased. “Benchmarking”is the method of analysis used to make thismeasurement, and, as noted in Chapter 2,benchmarking presents a real challenge forresearchers because they must consider thefollowing four alternatives to the bias hypoth-esis when analyzing data on drivers stoppedby police:

l


l


l


l


Researchers must similarly consider alter-natives to the bias hypotheses when analyzingsearch, disposition, and other poststop data.Identifying and ruling out the “alternative,legitimate factors” that can influence policedecisions concerning stops, searches, or dis-positions is a complex and painstaking task.Nevertheless, many departments have takenon the challenge of data collection andanalysis. By the Numbers: A Guide for Ana-lyzing Race Data from Vehicle Stops (Fridell2004) was written to guide researchers—inside and outside of departments—in this en-deavor. This book summarizes that informa-tion for the non-researcher stakeholder.

We expect that some frustration will begenerated by our message that data collectioncannot provide unequivocal answers to ques-tions about the existence of racial bias by po-lice in a jurisdiction. Despite the sincerity ofmost people posing the questions, answersthat are definitive cannot be offered. Dataanalysis is not as easy as comparing stop datato jurisdiction-level census data, although po-lice departments and concerned residentsmay well wish it were. We hope, however,that the frustrations that may be experiencedare offset somewhat by concrete and usefuladvice. This book (and its companion, By theNumbers) provides previously lacking infor-mation concerning how data can be analyzedand the results reported responsibly. We alsohope frustrations are offset by the knowledgethat even equivocal data can provide guid-ance for useful changes in a jurisdiction. Akey value of these data is their potential tobring police and residents of the communitytogether around a table to identify what mightbe done to make progress in the jurisdictionon the issues of racially biased policing andthe perceptions of its practice.

89

Alpert Group. 2003. Miami-Dade RacialProfiling Study. Draft of the methodssection of the report to the Miami-DadeCounty (FL) Police Department.

Ayres, Ian. 2001. Pervasive Prejudice?Unconventional Evidence of Raceand Gender Discrimination. Chicago:University of Chicago Press.

——. 2002. Outcome Tests of Racial Dispari-ties in Police Practices. Justice Researchand Policy, 4: 131–142.

Becker, Gary S. 1993. The Economics of Dis-crimination. Chicago: University ofChicago Press.

Clayton, Jerry. 2004. Enhancing Law Enforce-ment and Community Trust. PowerPointpresentation at the conference “By theNumbers: How to Analyze Race Datafrom Vehicle Stops,” sponsored by PERFand the Office of Community OrientedPolicing Services, Las Vegas, NV, July13–14.

Cordner, Gary, Brian Williams, and AlfredoVelasco. 2002. Vehicle Stops in SanDiego: 2001. Report submitted to theSan Diego Police Department.

Cordner, Gary, Brian Williams, and MariaZuniga. 2001. Vehicle Stops for the Year2000: Annual Report. Report submittedto the San Diego Police Department.

Council on Crime and Justice and Instituteon Race and Poverty. 2003. The Min-nesota Racial Profiling Study. A reportsubmitted to the Minnesota Legislature.Available on the PERF website athttp://www.policeforum.org. (On thePERF website, enter as a guest, click“Areas of Interest,” “Racially BiasedPolicing,” “Supplemental Resources,”

“Collecting Data,” and “JurisdictionReports.”)

Edwards, Terry D., Elizabeth L. Grossi, Gen-naro F. Vito, and Angela D. West. 2002a.Traffic Stop Practices of the Louisville Po-lice Department: January 15 – December31, 2001. Report submitted to theLouisville Division of Police.

——. 2002b. Traffic Stop Practices of theIowa City Police Department: April 1 –December 31, 2001. Report submittedto the Iowa City Police Department.

Engel, Robin Shepard, Jennifer M. Calnon,and Joshua R. Dutill. 2003. Project on Po-lice-Citizen Contacts, Six-Month Report.Report prepared for the Office of theCommissioner of the Pennsylvania StatePolice by the Population Research Insti-tute of the Pennsylvania State University.

Engel, Robin, Jennifer M. Calnon, Lin Liu,and Richard Johnson. 2004. Project onPolice-Citizen Contacts, Year 1 Final Re-port. Report prepared for the Office ofthe Commissioner of the PennsylvaniaState Police.

Farrell, Amy. 2004. Drawing Conclusionsfrom Results. PowerPoint presentation atthe conference “By the Numbers: How toAnalyze Race Data from Vehicle Stops,”sponsored by PERF and the Office ofCommunity Oriented Policing Services,Las Vegas, NV, July 13–14.

Farrell, Amy, Jack McDevitt, Lisa Bailey,Carsten Andresen, and Erica Pierce.2004. Massachusetts Racial and GenderProfiling Study, Final Report. Submittedto the Massachusetts Department ofPublic Safety. Available on the PERFwebsite at http://www.policeforum.org.

References

www.policeforum.org

www.policeforum.org


Farrell, Amy, Jack McDevitt, and Michael E.Buerger. 2002. Moving Police and Com-munity Dialogues Forward through DataCollection Task Forces. Police Quarterly,5(3): 359–379.

Farrell, Amy, Jack McDevitt, Shea Cronin,and Erica Pierce. 2003. Rhode IslandTraffic Stop Statistics Act: Final Report.Report submitted to the State of RhodeIsland by the Northeastern UniversityInstitute on Race and Justice, June 30.

Feest, J. 1968. Compliance with Legal Regula-tions: Observation of Stop Sign Behavior.Law and Society Review, II: 447–461.

Fridell, Lorie. 2004. By the Numbers:A Guide for Analyzing Race Data fromVehicle Stops. Washington, D.C.: PoliceExecutive Research Forum.

Fridell, Lorie, Robert Lunney, Drew Dia-mond, and Bruce Kubu. 2001. RaciallyBiased Policing: A Principled Response.Washington, D.C.: Police ExecutiveResearch Forum.

Glassbrenner, Donna. 2003. Safety Belt Use in2002: Demographic Characteristics (Na-tional Highway Traffic Safety Administra-tion Research Note [DOT HS 809 557]).Washington, D.C.: U.S. Department ofTransportation, National Highway TrafficSafety Administration, March.

Greenwald, Howard P. 2001. Final Report:Police Vehicle Stops in Sacramento Cali-fornia. Report to the Sacramento PoliceDepartment, October 31. Available athttp://www.cityofsacramento.org/spdata/pdf/data_collection_report_2001.pdf.

Lamberth, John. 1996a. Revised StatisticalAnalysis of the Incidence of Police Stopsand Arrests of Black Drivers/Travelers onthe New Jersey Turnpike between Inter-changes 1 and 3 from the Years 1988through 1991. Report of defendant’s ex-pert in State v. Pedro Soto, 734 A.2d 350(N.J. Super. Ct. Law. Div. 1996).

——. 1996b. Report of plaintiff ’s expert inWilkins v. Maryland State Police et al.,Civil No. MJG-93-468 (D. Md. 1996).

Lange, James E., Kenneth O. Blackman, andMark B. Johnson. 2001. Speed ViolationSurvey of the New Jersey Turnpike: FinalReport. Report submitted by the PacificInstitute for Research and Evaluation tothe Office of the Attorney General, Stateof New Jersey, December 13.

Lovrich, Nicholas, Michael Gaffney, ClayMosher, Mitchell Pickerill, and MichaelR. Smith. 2003. Washington State PatrolStop Data Analysis Project: Data AnalysisProject Report, June 1, 2003. A reportsubmitted to the Washington State Patrolby Washington State University. Pullman,WA: The Division of GovernmentalStudies and Services.

MVA and Joel Miller. 2000. Profiling Popula-tions Available for Stops and Searches(Police Research Series Paper 131).London: Home Office.

McMahon, Joyce, Joel Garner, CaptainRonald Davis, and Amanda Kraus. 2002.How to Correctly Collect and AnalyzeRacial Profiling Data: Your ReputationDepends on It! Final Report for RacialProfiling–Data Collection and Analysis.Washington, D.C.: Government PrintingOffice. Available online at http://www.cops.usdoj.gov/Default.asp?Open=True&Item=770.

Montgomery County (MD) Department ofPolice. 2001. Traffic Stop Data CollectionAnalysis (2nd report).

Novak, Kenneth. 2004. Disparity and RacialProfiling in Traffic Enforcement. PoliceQuarterly, 7(1): 65–96.

Pisarski, Alan E. 1996. Commuting inAmerica II: The Second National Reporton Commuting Patterns and Trends. Lans-downe, VA: Eno Transportation Founda-tion, Inc.

http://www.cityofsacramento.org/spdata/pdf/data_collection_report_2001.pdf

http://www.cops.usdoj.gov/Default.asp?Open=True&Item=770

References 91

Ramirez, D., J. McDevitt, and A. Farrell.2000. A Resource Guide on Racial Pro-filing Data Collection Systems: PromisingPractices and Lessons Learned. Wash-ington, D.C.: U.S. Department of Justice.

Ridgeway, G., K.J. Riley, and J. Grogger.2004. Analysis of Oakland’s Stop andSearch Data, in Promoting CooperativeStrategies to Reduce Racial Profiling:A Technical Guide (Chapter 9), OaklandPolice Department.

Scales, Robert. 2001. Racial Profiling:Seattle’s Community Involved Approach.Presentation at the 2001 InternationalProblem Oriented Policing Conferencesponsored by the Police Executive Re-search Forum, San Diego, CA, December.

Smith, Terry. 2003. Review, Critique and Rec-ommendations for Improving Racial Pro-filing Studies. Document developed bythe Service Improvement Analyst of theCity of Eugene Police Department.

Smith, William R., Donald Tomaskovic-Devey, Matthew T. Zingraff, H. MarcindaMason, Patricia Y. Warren, and CynthiaPfaff Wright. 2003. The North CarolinaHighway Traffic Study. Final report sub-mitted to the National Institute of Jus-tice, Grant No. 1999-MU-CX-0022.Washington, D.C.: National Instituteof Justice.

Thomas, Deborah. 2001. Preliminary Sum-mary Report: Denver Police DepartmentContact Card Data, June 1, 2001 throughAugust 31, 2001. Report provided to theDenver Police Department. Available athttp://admin.denvergov.org/admin/template3/forms/DPDPreliminaryReport3monthNov2001.pdf.

——. 2002. First Annual Report, DenverPolice Department Contact Card DataAnalysis, June 1, 2001 through May 31,2002. Report provided to the DenverPolice Department, October. Available at

http://admin.denvergov.org/admin/template3/forms/DPDContactCardAnnualReport102902.pdf.

Timoney, John. 2004. Panelist at “LawEnforcement Use of Force” webcast dis-cussion sponsored by the Office of Com-munity Oriented Policing Services attheir 2004 National Community PolicingConference, June 22.

Tomaskovic-Devey, Donald, Cynthia PfaffWright, and Ronald Czaja. 2003. Self-Reports of Police Speeding Stops by Race:Results from the North Carolina ReverseRecord Check Survey. Unpublished man-uscript, Department of Sociology andAnthropology, North Carolina StateUniversity.

U.S. Department of Transportation, FederalHighway Administration. 1995. Nation-wide Personal Transportation Survey(Microdata Files CD-ROM). Washington,D.C.: Author.

Walker, Samuel. 2001. Searching for the De-nominator: Problems with Police TrafficStop Data and an Early Warning SystemSolution. Justice Research and Policy,3(2): 63–95.

——. 2002. The Citizen’s Guide to Inter-preting Traffic Stop Data: Unraveling theRacial Profiling Controversy. Unpublishedmanuscript.

——. 2003. Internal Benchmarking for TrafficStop Data: An Early Intervention SystemApproach. Discussion paper available athttp://www.policeforum.org (On thePERF website, enter as a guest, click“Areas of Interest,” “Racially BiasedPolicing,” “Supplemental Resources,”“Collecting Data,” and “Articles andCommentary.”)

www.policeforum.org

http://admin.denvergov.org/admin/template3/forms/DPDPreliminaryReport3monthNov2001.pdf

http://admin.denvergov.org/admin/template3/forms/DPDContactCardAnnualReport102902.pdf

92

Fridell, Lorie. 2004. By the Numbers: AGuide for Analyzing Race Data from VehicleStops. Washington, D.C.: Police ExecutiveResearch Forum. Available online atwww.policeforum.org.

Fridell, Lorie, Robert Lunney, Drew Dia-mond, and Bruce Kubu. 2001. RaciallyBiased Policing: A Principled Response.Washington, D.C.: Police ExecutiveResearch Forum. Available online atwww.policeforum.org.

McMahon, Joyce, Joel Garner, CaptainRonald Davis, and Amanda Kraus. 2002.How to Correctly Collect and Analyze RacialProfiling Data: Your Reputation Depends on

It! Final Report for Racial Profiling–DataCollection and Analysis. Washington, D.C.:Government Printing Office. Available on-line at http://www.cops.usdoj.gov/Default.asp?Open=True&Item=770.

McMahon, Joyce and Amanda Kraus. 2005.A Suggested Approach to Analyzing RacialProfiling: Sample Templates for AnalyzingCar-Stop Data. Washington, D.C.: Govern-ment Printing Office. Available online at:http://www.cops.usdoj.gov/mime/open.pdf?Item=1462.

Northwestern University Racial ProfilingData Collection Resource Center. Online athttp://www.racialprofilinganalysis.neu.edu/.

Resources

www.policeforum.org

www.policeforum.org

http://www.cops.usdoj.gov/Default.asp?Open=True&Item=770

http://www.cops.usdoj.gov/mime/open.pdf?Item=1462

http://www.racialprofilinganalysis.neu.edu/

93

Dr. Lorie A. Fridell is Director of Researchfor the Police Executive Research Forum(PERF) and a social scientist by training.Prior to joining PERF in 1999, she was a pro-fessor of criminology and criminal justicefirst at the University of Nebraska and then atFlorida State University. She has been con-ducting research on law enforcement for

more than 15 years and is a national experton racial profiling. The lead author ofRacially Biased Policing: A Principled Re-sponse (PERF 2001), Fridell also has writtenextensively on such topics as police use offorce, citizen complaints, police pursuits, vi-olence against police, and problem-orientedpolicing.

About the Author

94

The Office of Community OrientedPolicing Services (COPS) was created in 1994and has the unique mission to directly servethe needs of state and local law enforcement.The COPS Office has been the driving forcein advancing the concept of communitypolicing, and is responsible for one of thegreatest infusions of resources into state,local, and tribal law enforcement in our na-tion’s history.

Since 1994, COPS has invested over $11.4billion to add community policing officers tothe nation’s streets, enhance crime fightingtechnology, support crime prevention initia-tives, and provide training and technical as-sistance to help advance communitypolicing. COPS funding has furthered the ad-vancement of community policing throughcommunity policing innovation conferences,the development of best practices, pilot com-munity policing programs, and applied re-search and evaluation initiatives. COPS hasalso positioned itself to respond directly toemerging law enforcement needs. Examplesinclude working in partnership with depart-ments to enhance police integrity, promotingsafe schools, combating the methampheta-mine drug problem, and supporting home-land security efforts.

Through its grant programs, COPS is as-sisting and encouraging state, local, and triballaw enforcement agencies to enhance theirhomeland security efforts using proven com-munity policing strategies. COPS programssuch as the Universal Hiring Program (UHP)has helped agencies address terrorism pre-paredness or response through communitypolicing. The COPS in Schools (CIS) programhas a mandatory training component that in-cludes topics on terrorism prevention, emer-gency response, and the critical role schools

can play in community response. COPS alsodeveloped the Homeland Security OvertimeProgram (HSOP) to increase the amount ofovertime funding available to support com-munity policing and homeland security ef-forts. Finally, COPS has implemented grantprograms intended to develop interoperablevoice and data communications networksamong emergency response agencies that willassist in addressing local homeland securitydemands.

The COPS Office has made substantial in-vestments in law enforcement training. COPScreated a national network of Regional Com-munity Policing Institutes (RCPIs) that areavailable to state, local, and tribal law en-forcement, elected officials and communityleaders for training opportunities on a widerange of community policing topics. Recentlythe RCPIs have been focusing their efforts ondeveloping and delivering homeland securitytraining. COPS also supports the advance-ment of community policing strategiesthrough the Community Policing Consor-tium. Additionally, COPS has made a majorinvestment in applied research which makespossible the growing body of substantiveknowledge covering all aspects of communitypolicing.

These substantial investments have pro-duced a significant community policing in-frastructure across the country as evidencedby the fact that at the present time, approxi-mately 86 percent of the nation’s populationis served by law enforcement agencies prac-ticing community policing. The COPS Officecontinues to respond proactively by pro-viding critical resources, training, and tech-nical assistance to help state, local, and triballaw enforcement implement innovative andeffective community policing strategies.

About the Office of Community Oriented Policing Services (COPS)

U.S. Department of Justice

95

The Police Executive Research Forum(PERF) is a national professional associationof chief executives of large city, county andstate law enforcement agencies. PERF’s ob-jective is to improve the delivery of policeservices and the effectiveness of crime con-trol through several means:

• the exercise of strong nationalleadership,

• the public debate of police andcriminal justice issues,

• the development of research andpolicy, and

• the provision of vital management andleadership services to police agencies.

PERF members are selected on the basisof their commitment to PERF’s objectives andprinciples. PERF operates under the fol-lowing tenets:

• Research, experimentation and ex-change of ideas through public discus-sion and debate are paths for thedevelopment of a comprehensive bodyof knowledge about policing.

• Substantial and purposeful academicstudy is a prerequisite for acquiring,understanding and adding to that bodyof knowledge.

• Maintenance of the highest standardsof ethics and integrity is imperative inthe improvement of policing.

• The police must, within the limits ofthe law, be responsible and account-able to citizens as the ultimate sourceof police authority.

• The principles embodied in the Consti-tution are the foundation of policing.

About PERF

understanding race data from vehicle stops...dec 31, 2001 · safety. to all of these people, i...

Documents