frankly awful

2
LETTERS Frankly Awful To the editor, This letter concerns the article written by Robert Lerner and Althea Nagai titled "Reverse Discrimination by the Numbers," appearing in your summer 2000 issue. I must express my disappointment in the level of scholarship found in that ar- ticle. I think that the authors are prob- ably correct in the conclusions that are drawn from the data, however, the repre- sentation of science and statistics is sim- ply not professional. Speaking as someone who has taught graduate methods and sta- tistics for 15 years and edited books and a journal dealing with these issues, the rep- resentations were frankly awful. I will mention four of the more serious errors in the text. The problem is that all the errors cast a very negative reflection on the quality of scholarship and call now into question the accuracy and confidence anyone can have in the analysis. First, the mathematical representations about causality are exceedingly strange (pages 74 and 81) and simply incorrect. The idea that a "large correlation" indi- cates a less ordered or causal system as opposed to a "small" correlation is incor- rect. Take the following model, which in- dicates a "spurious" relationship between A and B. There is a correlation between A and B, but the correlation is not causal because actually A and B are correlated because they are caused by a common underlying event X. Basically, X causes A and B to occur, but if you measured only A and B you would find a correlation existing be- B X 1 tween A and B but neither variable would cause the other. Now mathematically, you can calculate the correlation between A and B by mul- tiplying the correlations (X with A and X with B). Suppose X and A are correlated at .90 and X and B are correlated at .90 as well. The correlation between A and B would be .81 (very large using the Cohen system the authors mention) but it would not be causal, it would be extraneous. This simple example, one I use to teach un- dergraduates, is an examination of what happens when you try to use mathemati- cal methods to substitute for theoretical methods. The second issue is the "signature" is- sue they use on page 75. Basically, signa- ture evidence is established when you have either base rate information or an estab- lished "footprint" for the process and can seek to verify the existence of the foot- print. The problem is that the authors in this piece do not have such a process; they are seeking to establish the process rather than using an existing process and verify- ing various components. This is simply a misuse of the example. The third issue deals with the problems of multicollinearity. Several of the vari- ables (verbal SAT, math SAT, HS rank) are collinear (that is, they are correlated and indicators of the same underlying con- struct--this is a central argument made by Hernnstein and Murray, whom the au- thors cite). As a result of multicollinearity, that outcome is a regression equation where the multiple R is accurate but the individual estimates for the predictors are unstable and usable (any basic book on multiple regression will establish that). The problem is that the outcomes com- paring various features are suspect and useless. Had the authors published a zero order correlation matrix among all the variables, this could have been assessed. As it is, I cannot evaluate the problem, but

Upload: mike-allen

Post on 25-Aug-2016

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Frankly awful

L E T T E R S

F r a n k l y A w f u l

To the editor, This letter concerns the article written

by Robert Lerner and Althea Nagai titled "Reverse Discrimination by the Numbers," appearing in your summer 2000 issue.

I must express my disappointment in the level of scholarship found in that ar- ticle. I think that the authors are prob- ably correct in the conclusions that are drawn from the data, however, the repre- sentation of science and statistics is sim- ply not professional. Speaking as someone who has taught graduate methods and sta- tistics for 15 years and edited books and a journal dealing with these issues, the rep- resentations were frankly awful.

I will ment ion four of the more serious errors in the text. The problem is that all the errors cast a very negative reflection on the quality of scholarship and call now into question the accuracy and confidence anyone can have in the analysis.

First, the mathematical representations about causality are exceedingly strange (pages 74 and 81) and simply incorrect. The idea that a "large correlation" indi- cates a less ordered or causal system as opposed to a "small" correlation is incor- rect. Take the following model, which in- dicates a "spurious" relationship between A and B.

There is a correlation between A and B, but the correlation is not causal because actually A and B are correlated because they are caused by a common underlying event X. Basically, X causes A and B to occur, but if you measured only A and B you would find a correlation existing be-

B

X 1

tween A and B but nei ther variable would cause the other.

Now mathematically, you can calculate the correlation between A and B by mul- tiplying the correlations (X with A and X with B). Suppose X and A are correlated at .90 and X and B are correlated at .90 as well. The correlat ion between A and B would be .81 (very large using the Cohen system the authors mention) but it would not be causal, it would be extraneous. This simple example, one I use to teach un- dergraduates, is an examination of what happens when you try to use mathemati- cal methods to substitute for theoretical methods.

The second issue is the "signature" is- sue they use on page 75. Basically, signa- ture evidence is established when you have either base rate information or an estab- lished "footprint" for the process and can seek to verify the existence of the foot- print. The problem is that the authors in this piece do not have such a process; they are seeking to establish the process rather than using an existing process and verify- ing various components . This is simply a misuse of the example.

The third issue deals with the problems of multicollinearity. Several of the vari- ables (verbal SAT, math SAT, HS rank) are collinear (that is, they are correlated and indicators of the same under ly ing con- struct-- this is a central a rgument made by Hernnstein and Murray, whom the au- thors cite). As a result of multicollinearity, that o u t c o m e is a regression equa t ion where the multiple R is accurate but the individual estimates for the predictors are unstable and usable (any basic book on multiple regression will establish that). The problem is that the outcomes com- paring various features are suspect and useless. Had the authors published a zero o rder correlat ion matrix a m o n g all the variables, this could have been assessed. As it is, I cannot evaluate the problem, but

Page 2: Frankly awful

6 Academic Questions / Winter 2000-01

it is obviously present and no conclusions can be drawn f rom the analysis-- in fact it is inappropr ia te .

The last issue deals with under ly ing causal issues a m o n g the predictors. Again the Bell Curve (which they cite) argues for causality a m o n g the racial issues that gen- erate outcomes on the scores. A critical assumption is that there are no causal or structural relations a m o n g the predic tor variables. The authors make no case on this point and in some ways they contra- dict citations in the references that would assume otherwise. By the way, this poin t is made in Davis, one of their citations.

My po in t is that I would not recom- m e n d reliance on the conclusions of the article. There are a n u m b e r of o ther is- sues I have no t c o m m e n t e d on. Wha t pains me most is that these are all avoid- able with appropr ia te writing and statisti- cal analysis. The errors are needless and appall ing at the same time. The cur rent article can frankly be dismissed due to the nature of the technical errors, alas.

Mike Allen Depa r tmen t of Communica t ion University of Wisconsin Milwaukee, Wisconsin

Honest Analysis o f Data

To the editor, I was delighted to see in AQLerne r and

Nagai 's "Reverse Discr iminat ion by the Numbers ." This is an age of massive and mal ignant dishonesty by university admin- istrators about the influence of race on college admissions. More often than not, university administrators attack the char- acter and reputa t ion of those who suggest that admissions policies smack of reverse d iscr iminat ion. Moreover, they actively h a m p e r investigations that have the po- tential of chal lenging the politically cor-

rect spin that they cast abou t their insti- tutions.

Within this context, Lerner and Nagai's work is particularly valuable. They note that discrimination, the denial o f " a privi- lege or reward that he would have other- wise received except for the fact o f his [racial] g roup member sh ip" (71), applies to m e m b e r s of any race. Moreover, dis- cr imination is often difficult to detect con- clusively. T h e i r d e l i n e a t i o n o f t he statistical criteria for establishing causal- ity merits acclaim. In particular, their dis- cussion of the absence of third variable causation is clear and effective.

But what is mos t dramat ic in L e r n e r a n d Nagai ' s r e sea rch is the p red ic t ive equa t ion using logistic regression. The odds ratio is familiar to those who bet on horse races. Suppose your horse has a 1 in 10 chance of winning while my horse has a 1 in 2 chance of winning. The odds ratio for my horse c o m p a r e d to yours is 5 (i.e., 10/2=5) while the odds ratio for your horse c o m p a r e d to mine is .2 (i.e., 2 /10 = .2). At even money, a rational person will bet on my horse ra ther than yours.

Logistic regression enabled Lerne r and Nagai to calculate the odds ratios for ad- mission to the University of Virginia us- ing merit-based characteristics (e.g., SAT scores) a long with demograph i c charac- teristics (e.g., race and gender) . This pro- cedure allowed them to predic t the odds of admission using the s tandard statisti- cal protocol of "control l ing for the effects of the o ther predictors." The joy of this a p p r o a c h is tha t the odds ra t io is the statistician's best guess at what the odds w o u l d be i f the p r e d i c t o r w e r e uncorre la ted with all o ther predictors.

I f race is irrelevant to the admissions process , the odds rat ios for the racial d u m m y var iab les used by L e r n e r a n d Nagai, controll ing for all o ther predictors, would equal 1.0. Those who argue that university admissions policies are biased