semiparametric receiver operating characteristic analysis to evaluate biomarkers for disease

This article was downloaded by: [Southern Illinois University]On: 21 December 2014, At: 09:35Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of the American Statistical AssociationPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/uasa20

Semiparametric Receiver Operating CharacteristicAnalysis to Evaluate Biomarkers for DiseaseTianxi Caia & Margaret Sullivan Pepea

a Tianxi Cai is Assistant Professor and Margaret Sullivan Pepe is Professor ,Department of Biostatistics, University of Washington, Seattle, WA 98195. Theauthors thank the editor, the associate editor, and the reviewers for valuablecomments. Support for this research was provided by National Institutes of Healthgrants AI29168 and GM54438.Published online: 01 Jan 2012.

To cite this article: Tianxi Cai & Margaret Sullivan Pepe (2002) Semiparametric Receiver Operating CharacteristicAnalysis to Evaluate Biomarkers for Disease, Journal of the American Statistical Association, 97:460, 1099-1107, DOI:10.1198/016214502388618915

To link to this article: http://dx.doi.org/10.1198/016214502388618915

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy, completeness, or suitabilityfor any purpose of the Content. Any opinions and views expressed in this publication are the opinionsand views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy ofthe Content should not be relied upon and should be independently verified with primary sources ofinformation. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands,costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial orsystematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distributionin any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/uasa20

http://www.tandfonline.com/action/showCitFormats?doi=10.1198/016214502388618915

http://dx.doi.org/10.1198/016214502388618915

http://www.tandfonline.com/page/terms-and-conditions

Semiparametric Receiver Operating CharacteristicAnalysis to Evaluate Biomarkers for Disease

Tianxi Cai and Margaret Sullivan Pepe

The receiver operating characteristic (ROC) curve is a popular method for characterizing the accuracy of diagnostic tests when testresults are not binary. Various methodologies for estimating and comparing ROC curves have been developed. One approach, due toPepe, uses a parametric regression model ROCx4u5 D g4h04u5Cˆ0

0x5 with the baseline function h04u5 speci� ed up to a � nite-dimensionalparameter. In this article we extend the regression models by allowing arbitrary nonparametric baseline functions. We also provideasymptotic distribution theory and procedures for making statistical inference. We illustrate our approach with dataset from a prostatecancer biomarker study. Simulation studies suggest that the extra � exibility inherent in the semiparametric method is gained with littleloss in statistical ef� ciency.

KEY WORDS: Diagnostic tests; Disease screening; Estimating equation; Generalized linear model; Sensitivity.

1. INTRODUCTION

New technologies, such as gene expression microarrays andprotein mass spectrometry, promise to yield a multitude ofbiomarkers for disease. Once a biomarker is identi� ed, it canbe used as the basis for diagnosing disease (Srivastava andKramer 2000) or for monitoring patients during and after treat-ment. Biomarkers such as CA-125 and prostate-speci� c anti-gen (PSA) are used for these purposes in the managementof ovarian cancer and prostate cancer, respectively. Anotherapplication for biomarker research is in the development oftreatment strategies, where disease-speci� c biomarkers cansuggest new targets for therapeutic drugs (Elmer-Dewitt et al.2001). Statistical methods are needed to evaluate a biomarker’scapacity for distinguishing subjects with disease from thosewithout and for comparing biomarkers (Pepe et al. 2001). Onepotentially useful statistical tool in this setting is the receiveroperating characteristic (ROC) curve, which has long beenpopular in medical diagnostic research, particularly in radiol-ogy (Hanley 1989). In this article we consider new statisti-cal methodology for making inference about ROC curves andapply the methods to data from a prostate cancer biomarkerstudy.

Let D be a binary variable taking the value 1 for diseasedsubjects and 0 for nondiseased subjects. Let the variable Y

denote the biomarker, and use the convention that higher val-ues of Y are considered more indicative of disease. The ROCcurve is motivated as follows: If a threshold value c is usedto classify subjects as diseased or not on the basis of Y , thenthe true-positive and false-positive rates can be written as

SD4c5 D P4Y ¶ c—D D 15

andSSD4c5 D P4Y ¶ c—D D 050

A perfect biomarker is one for which at some threshold c ü ,SD4c ü 5 D 1 and SSD4c ü 5 D 0. More usually, there is a trade-off between SD and SSD displayed through the ROC curve, aplot of the true-positive rates versus the false-positive rates,

Tianxi Cai is Assistant Professor (E-mail: [email protected]) andMargaret Sullivan Pepe is Professor (E-mail: [email protected]),Department of Biostatistics, University of Washington, Seattle, WA 98195.The authors thank the editor, the associate editor, and the reviewers for valu-able comments. Support for this research was provided by National Institutesof Health grants AI29168 and GM54438.

84SSD4c51 SD4c551 c 2 4ƒˆ1ˆ59, or, equivalently, the function84u1ROC4u551u 2 401159 D 84u1 SD4Sƒ1

SD 4u5551 u 2 401159.Higher values of the true-positive rate SD4c5 obtained by

lowering the threshold are achieved at the expense of increas-ing the false-positive rate SSD4c5. Biomarkers that better dis-criminate disease from nondisease have ROC curves that arehigher and to the left of the positive unit quadrant. Swetsand Picket (1982), Hanley (1989), and, more recently, Pepe(2000b) have discussed the attributes of ROC curves for eval-uating diagnostic markers of disease.

Regression models for ROC curves can be used to exam-ine covariates that affect the discriminatory capacity of abiomarker. Covariates can include factors related to the envi-ronment in which the biomarker is measured, the protocol forobtaining and processing samples, subject characteristics, andeven disease characteristics. It is important to assess factorsthat in� uence the performance of a biomarker to optimize useof the biomarker in practice.

We have previously proposed parametric ROC regressionmodels (Pepe 1997, 2000a) of the generalized linear model(GLM) form,

ROCx4u5 D g8ˆ 0x C h� 4u591 0 µ u µ 11

where x denotes covariates, gƒ1 is a link function, and h isa baseline function speci� ed up to parameters � . Choices forg and h� used in our applications were g D ê and h� 4u5 D�0

C �1êƒ14u5, where ê denotes the cumulative normal dis-

tribution function. This corresponds to extending the classicbinormal model for the ROC curve (Metz 1986) to includecovariates. The baseline function h� essentially de� nes thelocation and shape of the ROC curve, whereas ˆ quanti� escovariate effects. In this article we extend the regression mod-els by allowing arbitrary nonparametric baseline functions forh� . It turns out that the extra � exibility is gained at little costin ef� ciency of estimating ˆ.

In Section 2 we describe the regression modeling frame-work and procedures for estimating ˆ and the baseline ROC

© 2002 American Statistical AssociationJournal of the American Statistical Association

December 2002, Vol. 97, No. 460, Theory and MethodsDOI 10.1198/016214502388618915

1099

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14

1100 Journal of the American Statistical Association, December 2002

function. We outline asymptotic distribution theory and pro-cedures for making statistical inference in Section 3. We per-formed simulation studies to illustrate the advantage gained interms of robustness of inference by allowing a nonparametricform for h and to determine whether the new semiparametricprocedure has reduced ef� ciency relative to the parametricapproach. We summarize results of the simulation studies inSection 4, and present an illustrative example in Section 5. Thedataset is derived from a study to evaluate PSA as a biomarkerfor the early detection of prostate cancer. We give some clos-ing remarks in Section 6.

2. ESTIMATION

Suppose that the data for analysis are organized as ND datarecords for nD subjects with disease,

84Yik1 Zik1ZDik51 k D 11 : : : 1Ki1 i D 11 : : : 1 nD91

and NSD data records for nSD subjects without disease,

84Yjl1Zjl51 l D 11 : : : 1Kj1 j D nD C 11 : : : 1 nD C nSD91

where each subject may have more than one data record,ND

D PnDiD1 Ki and NSD D PnDCnSD

jDnDC1 Kj . The covariates denotedby Z are relevant to both diseased and nondiseased sub-jects. Examples might include the subject’s age or the type ofbiomarker represented by Y (see Pepe 2000a, sec. 3.3). Weinclude another category of covariates, denoted by ZD , thatare speci� c to subjects with disease but are not relevant tonondiseased subjects. Examples include be severity of diseaseor timing of biomarker measurement before onset of clinicalsymptoms (see Sec. 5).

The ROC curve that compares the biomarker distributionsin diseased subjects who have covariates 4Z1 ZD5 with nondis-eased subjects who have covariate level Z is modeled as

ROCZ1ZD4u5 D g8h04u5 C ‚0Z C ‚0

DZD91 (1)

where g is a monotone increasing function mapping 4ƒˆ1ˆ5to 40115 and h0 is an unspeci� ed monotone increasing func-tion from 401 15 to 4ƒˆ1 ˆ5. We have previously noted that,conditional on the covariates 8Z1 ZD9 and D D 1, the expectedvalue of I 8Y ¶ Sƒ1

SD1 Z4u59 is ROCZ1 ZD

4u5 D g8h04u5 C ‚0Z C‚0

DZD9. This motivates the following class of estimating equa-tions for ˆ0

D 4‚1‚D5 based on binary indicator variables(Pepe 1997):

nDX

iD1

KiX

kD1

Z b

a

w4Xik1 u5Xik

£I©Yik ¶ Sƒ1

SD1 Z4u5ª

ƒ g8ˆ 0XikC h4u59

¤dOv4u5 D 01

where the prespeci� ed constants 4a1 b5 are chosen such thatP8Y11 < Sƒ1

SD1 Z114a59 and P8Y11 > Sƒ1

SD1 Z114b59 are positive; to

simplify notation, we write X D 6Z01Z0D70, so that ‚0Z C

‚0DZD D ˆ 0

0X, w is a positive bounded uniformly continuousweight function; and for now we assume that h04¢5 and Sƒ1

SD1 Z4¢5

are known and that Ov4¢5 is a known increasing but possiblydata-dependent function. In practice, neither h04¢5 nor Sƒ1

SD1 Z4¢5

is known and also need to be estimated.

Our semiparametric approach to estimation involves twosteps. First, for SSD1 Z we use a semiparametric location model(Pepe 1998; Heagerty and Pepe 1999),

SSD1 Z4c5 D S04c ƒ ƒ 00Z50

We estimate the parameter ƒ0 as the solution to

nDCnSDX

jDnDC1

KjX

lD1

Zjl4Yjlƒ ƒ 0Zjl5 D 01 (2)

which is denoted by Oƒ, and the survivor function S0 with theempirical distribution of the residuals

bS04c5 D 1NSD

nDCnSDX

jDnDC1

KjX

lD1

I 4Yjlƒ Oƒ 0Zjl ¶ c50 (3)

We then estimate the baseline ROC function h0 and the param-eter ˆ0 simultaneously as solutions to

nSDX

iD1

KiX

kD1

w4Xik1 u5£I©Yik ¶ bSƒ1

SD1 Z4u5ª

ƒ g©ˆ0Xik

C h4u5ª¤

D 01

for u 2 6a1 b7 (4)

and

nDX

iD1

KiX

kD1

Z b

a

w4Xik1 u5Xik

£I©Yik ¶ bSƒ1

SD1 Z4u5ª

ƒ g©ˆ0Xik

C h4u5ª¤

dOv4u5 D 01 (5)

where bSƒ1SD1 Z

4u5 D bSƒ10 4u5 C Oƒ 0Z.

Remark 1. Although biomarkers are typically measuredon continuous scales, the methodology is equally applicableto biomarkers with discrete or ordinal distributions. Such dataoften arise in diagnostic radiology and in psychology studies,where ROC analysis is already a well-accepted approach toevaluating new procedures.

Remark 2. It follows from (4) and the monotone increas-ing property of the function g that the estimator Oh is a mono-tone increasing function. In this article we assume that h04¢5has a continuous � rst derivative.

Remark 3. In practice, let a µ u1 < u2 < ¢ ¢ ¢ < uL µ b

be the set of distinct jump points of bSƒ1SD1 Zik

4u5 on 6a1 b7. Also,let Ovl

D Ov4tlC15 ƒ Ov4tl5 and hlD h4ul5. Then solving the two

sets of estimating equations is equivalent to solving the p C Lequations

nDX

iD1

KiX

kD1

w4Xik1 ul5£I©Yik ¶ bSƒ1

SD1 Zik4ul5

ªƒ g4ˆ 0Xik

C hl5¤

D 01

l D 11 : : : 1 L

and

LX

lD1

nDX

iD1

KiX

kD1

w4Xik1 ul5Xik

£I©Yik ¶ bSƒ1

SD1 Zik4ul5

ª

ƒ g4ˆ0XikC hl5

¤Ovl

D 00

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14

Cai and Pepe: Receiver Operating Characteristic Analysis 1101

The Newton–Raphson algorithm can be used to solve theforegoing p C L equations. It is not hard to see that in thiscase, inverting the 4p C L5� 4p C L5 Jacobian matrix can beobtained by inverting a diagonal L � L matrix and a p � p

matrix.

Remark 4. The covariates Z and ZD are assumed to bebounded. If the covariate Z takes on only a few values, acompletely nonparametric approach to estimating SSD1 Z maybe preferred. For example, if the problem is to compare twobiomarkers and Z is an indicator of the type of biomarkerquanti� ed by Y , then bSSD1 Z4¢5 can be estimated separately foreach Z using the empirical survivor function for the corre-sponding biomarker among the nondiseased subjects.

Remark 5. A parametric formulation for h04u5, such ash04u5 D �1 C �2ê

ƒ14u5 say, can be accommodated by replac-ing (4) with an equation identical in form to (5) with61êƒ14u570 substituted for Xik in the integrand. Then (4)and (5) are solved simultaneously. (See Pepe (1997) for thisparametric approach.)

Remark 6. As indicated by our notation, the data recordsmay be clustered. For example, if multiple biomarkers aremeasured on a subject or if the same biomarker is measuredat different times, then each subject may have several datarecords in the analysis. The probability distributions and con-sequent ROC curves are de� ned in a marginal sense. There-fore, our estimating equations provide valid estimates for clus-tered data. In the next section we account for correlationbetween data records in the same cluster in making statisticalinference.

Remark 7. The function Ov4u5 can be data dependent, butwe assume that it converges to a deterministic function uni-formly in u. For example, we can choose Ov4u5 to be the count-ing process

PnDiD1

PKi

kD1 I4Yik ¶ bSƒ1SD1 Zik

4u55. We can also restrictOv4u5 to be 0 in certain ranges of u that might not be of par-ticular interest. For example, if large false-positive rates arenot of particular interest, we might model the ROC curve onlyover a restricted region, say, u µ 20%. In that case, we chooseOv4u5 D 0 for u > 02. (See Weiand, Gail, James, and James1989 and Pepe 1998 for further discussion about restrictinginference about ROC curves to clinically relevant ranges offalse-positive rates.) We let the weight function w4x1 u5 be 1in the analyses presented in this article, although more gen-eral choices are possible. Note that we do not allow w to bedependent on ˆ or h, a condition necessary to guarantee theuniqueness of the solution 8 O 1 Oh4u59.

Remark 8. The form of the ROC model ROCx4u5 Dg8h04u5 C ˆ0

0x9 suggests that the covariate effect is the sameat all false-positive rates u. This can be relaxed by includinginteractions between functions of u and x in the regressionmodel. In general, we can write

ROCx4u5 D g8ˆ00x C � 0

1‡14u5x C ¢ ¢ ¢ C � 0q‡q 4u5x C h04u591

where ‡k4u5 are speci� ed functions. This is analogous tothe standard procedure for relaxing the proportional hazardsassumption in the Cox model for survival data, where covari-ate effects can change with time by including interactions

between time and the covariate in the relative risk function.Our estimation procedures are appropriate for the more gen-eral model as well.

3. INFERENCE IN LARGE SAMPLES

3.1 Asymptotic Properties of O

Let 8 O 1 Oh4u59 denote the solution to (4) and (5) and letOh4u1ˆ5 denote the solution to (4) for any given ˆ. We showin Appendix A that

Theorem 1. 8 O 1 Oh4u59 are unique for large n and are con-sistent for u 2 6a1 b7, where 0 < a < b < 1.

To obtain large-sample distributions for 8 O 1 Oh4u59, we � rstshow in Appendix B that

Lemma 1. n12SD 6SSD1 z8bSƒ1

SD1 z4u59 ƒ u7 is asymptotically equiv-

alent to

e°4u1z5 D ƒn

12SD

NSD

nDCnSDX

jDnDC1

KjX

lD1

£I©Yjl

ƒ ƒ 00Zjl > Sƒ1

0 4u5ª

ƒ u

C©4 SD1 1

ƒ z50 ƒ1SD1 2Zjl

ª4Yjl

ƒ ƒ 00Zjl5

PS0

©Sƒ1

0 4u5ª¤

1

where for any function f4x5, Pf 4x5 denotes df4x5=dx, SD1 1

is the limit of SZSD1 1 D PnDCnSDjDnDC1

PKj

lD1 Zjl=nSD1 SD1 2 is the limit

of SZSD1 2D PnDCnSD

jDnDC1

PKj

lD1 ZjlZ0jl=nSD, and e°4u1z5 converges in

distribution to a Gaussian process.

To obtain interval estimates of speci� c components of ‚0,in Appendix C we show that

Theorem 2. n12D4 O ƒ ˆ05 is asymptotically equivalent to

nƒ 1

2D áƒ1

(nDX

iD1

Ui4ˆ05 CnD CnSDX

jDnDC1

Uj4ˆ05

)1

where á is de� ned in (A.7),

Ui4ˆ05 DKiX

kD1

Z b

a

w4Xik1 u58Xikƒ 4u3ˆ059eik4u5dv4u5

for i D 11 : : : 1 nD1

Uj4ˆ05 D ƒ nD

NSD

KjX

lD1

Z b

aâ4u5 ƒ1

SD1 2Zjl4Yjlƒ ƒ 0

0Zjl5

� PS0

©Sƒ1

0 4u5ª

dv4u5 for j D nDC 11 : : : 1 nD

C nSD1

4u3ˆ5 is the limit of SX4u3 ˆ5, de� ned in (A.5), eik4u5 DI 8Yik > Sƒ1

SD1 Zik4u59ƒg8h04u5Cˆ0

0Xik9, v4u5 is the limit of Ov4u5,

aik4u5 is de� ned in (A.8), and â4u5 is the limit of bâ4u5 Dnƒ1

D

PnDiD1

PKi

kD18Xikƒ 4u3ˆ059aik4u54 SD1 1

ƒ Zik50 Ph04u5.

Therefore, the distribution of n12D4 O ƒ ˆ05 can be

approximated by a normal random vector with mean 0and covariance matrix bè D nƒ1

Dbáƒ14 O 58

PnDiD1

bUi4O 5 bUi4

O 50 CPnDCnSDjDnDC1

bUj4O 5bUj4

O 509báƒ14 O 5, where báƒ14ˆ5 is de� ned in

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14


(A.6) and bUi and bUj are obtained by replacing all the-oretical quantities in Ui and Uj by their empirical coun-terparts. One possible approach to obtain an estimate forPS04¢5 and Ph04¢5 is to use bPS04c5 D 1

bs

R c2

c1«s4

cƒxbs

5 dbS04x5 andOPh4u5 D 1

bh

R u2

u1«h4 uƒx

bh5 dOh4x5, where bs and bh are some posi-

tive bandwidth parameters and the kernel functions «s and «h

are bounded functions with support 6ƒ11 17 and that integrateto 1. The Epanechnikov kernel function K4x5 D 07541 ƒ x25�I 4—x— µ 15 was used for the kernel estimates of both PS04¢5 andPh04¢5 through our numerical studies. The bandwidth was cho-sen to be bs

D bhD 4=max8min4nD1 nSD54=51509.

3.2 Estimating the Receiver OperatingCharacteristic Curve

Based on 8 O 1 Oh4u59, the ROC curve for a test with covariatesx can be estimated by dROC4u3 x5 D g8 O 0x C Oh4u59. To obtainthe distribution of dROC4u3 x5, in Appendix D we prove thefollowing theorem.

Theorem 3. Q4u3x5 D n12D 6gƒ18 dROC4u3x59 ƒ gƒ18ROC

4u3x597 has the same asymptotic distribution as

eQ4u3 x5 D nƒ 1

2D

(nDX

iD1

eQi4u3x5 CnDCnSDX

jDnDC1

eQj4u3x5

)1

and eQ4u3x5 converges weakly to a zero-mean Gaus-sian process, where eQi4u3x5 D 8x ƒ 4u3 ˆ059

0áƒ1Ui4ˆ05 CPKi

kD1 w4Xik1 u5eik4u5=a4u5,

eQj4u3 x5 D 8x ƒ 4u3 ˆ0590áƒ1Uj4ˆ05

ƒ nD

NSD

Ph04u5

KjX

lD1

"I©Yjl

ƒ ƒ00Zjl > Sƒ1

0 4u5ª

ƒ u

C az4u50

a4u5ƒ1SD1 2Zjl4Yjl

ƒ ƒ 00Zjl5

PS0

©Sƒ1

0 4u5ª#

1

a4u5 is the limit of nƒ1D

PnDiD1

PKi

kD1 aik4u5, and az4u5 is thelimit of nƒ1

D

PnDiD1

PKi

kD1 aik4u54 SD1 1 ƒ Zik5.

In practice, to approximate the distribution of bQ4u3 x5, weconsider the following resampling technique (Parzen, Wei, andYing 1994). First, let

bQ4u3x5 D nƒ 1

2D

(nDX

iD1

bQi4u3 x5§iC

nDCnSDX

jDnDC1

bQj4u3x5§j

)1 (6)

where bQi4u3 x5 and bQj4u3 x5 are obtained by replacing allof the theoretical quantities in eQi4u3x5 and eQj4u3 x5 bytheir empirical counterparts. Conditional on the observations84Yik1 Zik1ZDik51 k D 11 : : : 1 Ki3 i D 11 : : : 1 nD

C nSD9, the pro-cess bQ4u3x5 has the same limiting covariance function as thatof eQ4u3x5. Furthermore, conditional on the data, the processbQ4u3x5 is tight (Billingsley 1986). It follows that the distri-bution of Q4u3x5 can be approximated by the conditional dis-tribution of bQ4u3x5.

We can then approximate the distribution of Q4u3x5 by gen-erating M independent samples of 8§i1 i D 11 : : : 1 nD

C nSD9.

For the mth sample, we obtain a realization Oq4m54u3x5 ofbQ4u3x5, m D 11 : : : 1 M . For any given u, we can use

O‘ 24u—x5 D 1M

MX

mD1

Oq4m54u3 x52

to estimate the variance of Q4u3 x5. Then a 41ƒ †5 pointwisecon� dence interval for ROC4u3x5 is given by

g£gƒ1

© dROC4u3 x5ª

c†=2nƒ1=2D

O‘ 4u3x5¤1

where c† is the 100† upper percentile point of the standardnormal distribution. To construct a 41 ƒ †5 simultaneous con-� dence band for 8ROC4u3x51a µ u µ b9, we � rst � nd d† suchthat

pr

Àsup

u26a1b7

—bQ4u3x5—O‘ 4u3x5

µ d†

ÁD 1 ƒ †0

Then a 41 ƒ †5 con� dence band for ROC4u3 x5 for a µ u µ b

is given by

g£gƒ1

© dROC4u3x5ª

d† nƒ1=2D

O‘ 4u3x5¤0

The probability and the quantile d† can be approximated withthese M realizations of bQ4u3 x5.

4. SIMULATION STUDIES

4.1 Flexible Modeling Confers Robustness

We would expect the newly proposed semiparametric pro-cedure to be more robust than a procedure that speci� es aparametric form for h04¢5. In particular, we would expect infer-ence about covariate effects and the baseline ROC curve to bemore reliable in the model that does not require speci� cationof a parametric form for h04¢5. To investigate this, we simu-late data and � t a misspeci� ed parametric model to these data.For nD diseased subjects, we generate random variables z and… following uniform 401105 and standard normal distributions,and let

YiD –4‚zi

C …i5 C 2zi1 i D 11 : : : 1 nD0

Similarly, we generate z and … for nSD diseased subjects andconstruct

YjD 6C 2zj

C …j1 j D nDC 11 : : : 1 nD

C nSD1

where –4x5 D 6 C log8ƒ6 logê4ƒx59=2, ê4¢5 is the stan-dard normal cumulative distribution function, and ‚ D 1. Theinduced ROC curve is then

ROC4u3T 1 z5 D ê6ƒ–ƒ186 ƒ êƒ14u59 C ‚z71

which follows the GLM model with

h04u5 D ƒ–ƒ186 ƒ êƒ14u590

We � t the following parametric model to the data, using themethod of Pepe (1997):

ROCz4u5 D ê8�0C �1ê

ƒ14u5 C ‚z91

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Semi-parametricMisspecified Parametrictruth

False Positive Rate

True

Pos

itive

Rat

e

Figure 1. Estimated ROC Curve at Baseline Using the Semipara-metric ( ) and the Misspeci’ ed Parametric Approaches (- - - -)(– – –, Truth).

thus misspecifying the baseline function as having theform h04t5 D �0 C �1ê

ƒ14u5. The semiparametric proceduredescribed in this article is also implemented. Figure 1 dis-plays � tted baseline ROC curves (at z D 0), based on one sim-ulated dataset. The semiparametric method provides an esti-mated ROC curve much closer to the truth than the parametricapproach, suggesting that it is more robust.

Table 1 presents the empirical bias and mean squared errorof O‚ based on 500 simulated datasets. The parametric estima-tor is severely biased on average. In contrast, the semipara-metric approach yields an estimator with essentially no bias.This example, albeit somewhat extreme, demonstrates that theless-restrictive model assumptions made in the semiparamet-ric approach confer some robustness to the estimates of boththe ROC curve and of the covariate effect. This robustnessis particularly important given that formal procedures are notyet available for checking the � t of the parametric model. Wenext investigate whether the added robustness in the semipara-metric approach is gained at the expense of reduced statisticalef� ciency.

4.2 Relative Ef’ ciency

We simulated data to include both a covariate common todiseased and nondiseased subjects and a covariate relevantonly for diseased subjects. These are denoted by z and T . Inparticular, data for diseased subjects were generated from thelinear model

YiD 12 C ‚tTi

C 42 C ‚z5ziC …i1 i D 11 : : : 1 nD1

Table 1. Estimates of ‚ Compared With Its Actual Value, ‚ D 1, Basedon a Semiparametric Model and on a Misspeci’ ed Parametric Model

Semiparametric Parametric

Bias 0031 0128Mean squared error 0019 0042

NOTE: Results are based on 500 simulated datasets of nD D nSD D 200.

Table 2. Estimates of Covariate Effects From Correctly Speci’ edParametric and Semiparametric Models

Semiparametric Parametric

nD D nSD ‚t D ƒ1 ‚z D 3 ‚t D ƒ1 ‚z D 3

Bias 0082 0375 0055 0320100

Mean squared error 0056 10522 0047 10486

Bias 0027 0108 0012 0082200

Mean squared error 0019 0184 0017 0170

and data for diseased subjects followed the model

YjD 10 C 2zj

C …j1 j D nD C 11 : : : 1 nD C nSD1

where …, T , and z are random variables with probability dis-tribution that are standard normal, exponential (rate D 1) andBernoulli (probability D .5). This con� guration for the datainduces the ROC curve

ROC4u3 T 1 z5 D ê8êƒ14u5 C 2 C ‚tt C ‚zz90

We use an ROC curve model of this form for the PSA dataanalysis (Sec. 5), where z represents subject age and T repre-sents the time between serum sampling for the PSA biomarkerand onset of clinical symptoms of cancer.

For each simulated dataset, we obtained point estimates of‚t and ‚z with our semiparametric approach and the paramet-ric approach of Pepe (1997). Table 2 presents the bias andmean squared error based on 500 simulations.

The results in Table 2 show that even though estimates fromthe semiparametric model are not as ef� cient as those cal-culated using the fully parametric approach, their ef� ciencyis very close. The empirical ef� ciency of the semiparametricmethod relative to the parametric method is 95% for ‚z and90% for ‚t at a sample size of 200.

4.3 Asymptotic Inference in Finite Samples

We also conducted simulation studies to examine the valid-ity of the large-sample approximations for making inferencein � nite sample sizes. We simulated 1,000 sets of data withnD

D nSD D n D 50, 100, 200, and 400 from the same modelsdescribed in Section 4.2. Here we let ‚t

D ƒ1 and ‚zD 2.

Table 3 presents the bias, average of the standard error esti-mators, the sampling standard error, and the coverage proba-bility of the 95% con� dence intervals for ‚t and ‚z. The stan-dard error estimator are reasonably close to the true samplingstandard errors, at least for sample size n ¶ 100. In addition,for con� dence intervals, the empirical coverage probabilitiesare close to their nominal counterparts. For a small sample,n D 50, it appears that the estimated standard errors based onlarge-sample approximations tend to be smaller than the sam-pling standard errors. The bootstrap standard error may beused instead when the sample size is small.

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14


Table 3. Bias, Average of the Standard Error Estimator (Ave(cSE)), Standard Error (SE), and the CoverageProbability (Coverage) of the 95% Con’ dence Interval

‚t D ƒ1 ‚z D 2

nD D nSD Bias Ave(cSE) SE Coverage Bias Ave(cSE) SE Coverage

50 0101 0256 0316 0914 0245 0604 0828 0908100 0034 0161 0175 0932 0072 0397 0433 0932200 0026 0113 0112 0947 0052 0267 0280 0941400 0014 0078 0078 0945 0010 0183 0186 0945

NOTE: Each entry is based on 1,000 simulation samples.

5. EXAMPLE: EARLY DETECTION OF PROSTATECANCER WITH SERUM PROSTATE-SPECIFIC

ANTIGEN LEVELS

PSA levels in serum are used to screen men for prostatecancer. However, considerable controversy exists as to itsvalue. A longitudinal case-control study of PSA as a screeningmarker for prostate cancer was nested in the Beta-Caroteneand Retinol Ef� cacy Trial (CARET) (Thornquist et al. 1993),in an effort to quantify the capacity of PSA for discriminatingmen with prostate cancer from those without before the onsetof clinical symptoms.

Brie� y, CARET enrolled more than 12,000 men and ran-domized them to intervention or placebo to prevent lung can-cer. As part of the protocol for the trial, serum was periodicallydrawn and stored from study participants. All 88 subjects whodeveloped prostate cancer over the course of the trial wereincluded in the PSA case-control study. Serum samples storedbefore diagnosis with cancer were analyzed for PSA. An age-matched set of 88 control subjects also had their stored serumsamples analyzed for PSA. Etzioni, Pepe, Longton, Hu, andGoodman (1999) have provided a complete description of thestudy design and plots of the data.

Increasing age is associated with increasing serum PSAlevel and can potentially affect the discriminatory capacity ofPSA. Thus we used z D age (years) as a covariate in our ROCmodel for PSA. In addition, among subjects who develop can-cer, it is likely that PSA measured closer to the time of onsetof clinical symptoms is more predictive of disease than aremeasures taken earlier in time. We included a covariate T ,de� ned as the time between the onset of symptoms and thetime at which the serum sample was drawn. We then � t thefollowing model to the data:

ROCT 1 z4u5 D ê8h04u5 C ‚tT C ‚zz90

Using our semiparametric approach, the estimate of ‚t isƒ0120 per year with standard error .041, and the coef� cientfor age, ‚z, is estimated as ƒ0014 per year of age with stan-dard error .020. The negative coef� cient for T implies thatdiscrimination improves as T decreases, that is, when PSAis measured closer to diagnosis. The negative coef� cient forage suggests that discrimination is better in younger men thanin older men, but the evidence is not conclusive (p value D.484). Figure 2 shows the estimated ROC curves and their 95%con� dence bands at different times for patients who were 60years old when the serum for measuring PSA was obtained.

Also shown are curves estimated using the parametric binor-mal model

ROCT 1 z4u5 D ê8�0 C �1êƒ14u5 C ‚tT C ‚zz90

Observe that the curves are similar for the parametric and semi-parametric methods. The regression coef� cients and their esti-mated standard errors for the parametric method in this exampleare almost identical to the semiparametric ones, O‚t

D ƒ0119,se4 O‚t5 D 0041 and O‚z

D ƒ0014, se4 O‚z5 D 0019. It appears thatthe binormal model does � t the data adequately in this exam-ple and that the semiparametric methods � t the model with ef� -ciency comparable to that of the fully parametric approach.

6. DISCUSSION

This article extends the parametric ROC regression methodof Pepe (1997, 2000a) to a semiparametric approach. The re-ductions in requirements for model speci� cation and increasedrobustness are attractive features. Other approaches to ROCregression have been proposed. One popular method is to modelthe distributions of test results (Tosteson and Begg 1998; Pepe1998). Covariates whose associations with Y differ in diseasedand non-diseased populations (i.e., interactions with D) induceeffects on the ROC curve. Another approach is to model asummary measure of the ROC curve and to use derived vari-ables, estimated summary measures, for � tting regression mod-els (Thompson and Zucchini 1989; Dorfman, Berbaum, andMetz 1992). Pepe (1998) contrasted these regression modelswith those considered in this article that directly model the ROCcurve itself. Our preference is for this latter approach, primarilybecause it directly models the quantities of interest.

Asymptotic distribution theory has been derived. In con-trast to earlier work (Pepe 1997), the theory allows clustereddata which in practice arises frequently, as evidenced by ourtwo examples. We proposed a simulation method for makinginference about ROC curves. This technique avoids the needto derive explicit analytic expressions for variance-covarianceprocesses, which seem intractable in our setting. Moreover,relative to other resampling methods such as the bootstrap, thecomputational burden is minimal.

The application presented in this article concerns abiomarker for prostate cancer. We used a probit link functionin our model,

ROCx4u5 D g8h04u5 C ˆx91

where g D ê. Other choices of link function might be pre-ferred. A logistic link allows the interpretation of exp4ˆ5 as

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

(a)

Semi-parametric : EstimateParametric : EstimateSemi-parametric : 95% confidence band

Parametric : 95% confidence band

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

(b)



0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

(c)



0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

(d)



u

RO

C(u

)

Figure 2. Estimated ROC Curve Using Semiparametric ( ) and Parametric (– – –) Approaches for PSA as a Biomarker of Prostate Cancerin 60-Year-Old Men. Shown also are 95% Con’ dence Bands (thin solid lines, semiparametric; shaded regions, parametric) (a) Serum sampletaken at diagnosis; (b) Serum sample taken 2 years before diagnosis; (c) Serum sample taken 4 years before diagnosis; (d) Sample taken 8 yearsbefore diagnosis.

an odds ratio, for example. A log link implies that exp4ˆ5

is the increase in the true-positive rate per unit increase inx when the false-positive rate is held constant (Pepe 1997).These interpretations do not depend on the baseline functionh0, and this method now enables inference about ˆ withoutspecifying a form for h0.

APPENDIX A: PROOF OF THEOREM 1

The strong law of large numbers ensures consistency of Oƒ and thatbSƒ1

0 4u5 converges to Sƒ10 4u5 uniformly in u 2 6a1 b7. It follows that

bSƒ1SD1 z

4u5 D bSƒ10 4u5C Oƒ 0z converges to Sƒ1

SD1 z4u5 uniformly in u 2 6a1b7

and z 2 Dz‡

D 8Z 2 ˜Z˜ µ ‡9 for any ‡ ¶ 0, almost surely, as n ! ˆ.It then follows from the uniform law of large numbers that for any‡ ¶ 0, … > 0, uniformly in u 2 6a1 b7 and ˆ 2 Dˆ

‡D 8ˆ 2 ˜ˆƒˆ0˜ µ ‡9,

1nD

nDX

iD1

KiX

kD1

w4Xik1 u5£I©Yik > bSƒ1

SD1 Zik4u5

ªƒg8h04u5Cˆ

0Xik ƒ …9¤

! kDE£w4X1k1 u58g4h04u5Cˆ0X1k5 ƒg4h04u5CˆX1k

ƒ …59¤

almost surely as nD ! ˆ and nSD ! ˆ, where kD is the limit ofPnDiD1 Ki=nD. It follows that for large nD and nSD, u 2 6a1b7, ˆ 2 Dˆ

‡ ,and suf� ciently large …,

1nD

nDX

iD1

KiX

kD1


SD1 Zik4u5

ªƒg8h04u5C ˆ0Xik

ƒ …9¤

> 0

(A.1)

and

1nD

nDX

iD1

KiX

kD1


SD1 Zik4u5

ªƒg8h04u5Cˆ0Xik

C …9¤

< 00

(A.2)

This, coupled with the monotonicity and continuity of g for large nD

and nSD, u 2 6a1b7, and ˆ 2 Dˆ‡ , implies that there exists a unique

Oh4u3ˆ5 such that

1nD

nDX

iD1

KiX

kD1

w4Xik1 u5£I©Yik > bSSD1 Zik

4u5ª

ƒg8Oh4u3ˆ5 Cˆ0Xik9¤

D 00

(A.3)

By differentiating both sides of (A.3) with respect to ˆ, we obtainthe identity

ƒ ¡

¡ˆOh4u3ˆ5 D SX4u3ˆ51 (A.4)

where

SX4u3ˆ5 DPnD

iD1

PKikD1 w4Xik1 u5 Pg8Oh4u3 ˆ5 Cˆ0Xik9XikPnD

iD1

PKi

kD1 w4Xik1 u5 Pg8Oh4u3 ˆ5 Cˆ0Xik90 (A.5)

To show the existence and uniqueness of O , we let V 4ˆ5 be the leftside of (5), with h04u5 replaced by Oh4t1ˆ5. It follows from (A.4) that

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14


1nD

¡¡ˆ

V4ˆ5 D ƒbá4ˆ5, where

bá4ˆ5 D 1nD

nDX

iD1

KiX

kD1

Z b

aw4Xik1 u58Xik

ƒSX4u3ˆ59†2

� Pg8Oh4u3ˆ5 Cˆ0Xik9 dOv4u51 (A.6)

for any vector or matrix b, b†0 D 1, b†1 D b, and b†2 is de� ned asbb0. Furthermore, because only when ˆ D ˆ0 do (A.1) and (A.2) holdfor any … > 0, we have that Oh4u3ˆ05 ! h04u5 uniformly in u 2 6a1b7

and 1nD

¡¡ˆ

V 4ˆ05 ! ƒá, where

á D EZ b

a8Xik

ƒ 4u3ˆ059†2aik4u5dv4u51 (A.7)

aik4u5 D w4Xik1 u5 Pg8h04u5C ˆ00Xik91 (A.8)

and 4u3ˆ5 is the limit of SX4u3ˆ5. It is easy to see that á is positivede� nite.

Now, because 1nV 4ˆ05 ! 0, by the standard inverse function the-

orem there exists a unique solution O to the equation V4ˆ5 D 0 ina neighborhood of ˆ0. This, coupled with the nonnegativity of bá4ˆ5

for large n, ensures uniqueness of the root of V 4ˆ5 D 0 in the entiredomain of ˆ asymptotically. The foregoing proof also implies thatO is strongly consistent and that Oh4u3 O 5 ! h04u5 almost surely uni-formly in u 2 6a1b7.

APPENDIX B: PROOF OF LEMMA 1

By the standard central limit theorem, n12SD 4 Oƒ ƒ ƒ05 D

nƒ 1

2SD

ƒ1SD1 2

PnDCnSDjDnDC1

PKj

lD1 Zjl4Yjlƒ ƒ 0

0Zjl5 converges in distribution toa mean 0 multivariate normal random variable. It follows from thefunctional central limit theorem (Pollard 1990) that for any ‡ > 0,

nƒ 1

2SD

nDCnSDX

jDnDC1

KjX

lD1

£I 4Yjl ƒƒ

0Zjl > c5 ƒS08c C 4ƒ ƒ ƒ050Zjl9¤1

4c1ƒ5 2 6Sƒ10 4b51Sƒ1

0 4a57� Dƒ‡ 1

converges in distribution to a Gaussian process, where Dƒ‡

D 8ƒ 2˜ƒ ƒƒ0˜ µ ‡9. It then follows from the equicontinuity (Pollard 1990)of the foregoing process and the consistency of Oƒ that

supc26Sƒ1

0 4b51Sƒ10 4a57

n

ƒ 12

SD

nDCnSDX

jDnDC1

KjX

lD1

£I 4Yjl

ƒ Oƒ 0Zjl >c5

ƒS08cC4 Oƒ ƒƒ050Zjl9

¤

ƒnƒ 1

2SD

nDCnSDX

jDnDC1

KjX

lD1

8I4Yjlƒƒ 0

0Zjl >c5ƒS04c59

!0 (B.1)

in probability. This implies that

n12SD

©bS04c5ƒS04c5ª

ºn

12SD

NSD

nDCnSDX

jDnDC1

KjX

lD1

£I 4Yjl

ƒƒ00Zjl > c5 ƒS04c5

C S08c C 4 Oƒ ƒƒ050Zjl9 ƒS04c5¤

ºn

12SD

NSD

nDCnSDX

jDnDC1

KjX

lD1

nI4Yjl

ƒ ƒ00Zjl > c5 ƒS04c5

C PS04c5¡ 0

SD1 1ƒ1SD1 2Zjl

¢4Yjl ƒƒ

00Zj l5

o0

From the functional central limit theorem, we see that n12SD 8bS04c5 ƒ

S04c59 converges in distribution to a mean 0 Gaussian process. It thenfollows from a Taylor series expansion and the stochastic equiconti-

nuity of n12SD 8bS04c5ƒ S04c59 that

n12SD

£SSD1 z

©bSƒ1SD1 z

4u5ª

ƒu¤

º n12SD

£S0

©bSƒ10 4u5

ªƒu

¤

C n12SD

PS08bSƒ10 4u594 Oƒ ƒƒ050z º e°4u1 z51

where

e°4u1z5 D ƒn

12SD

NSD

nDCnSDX

jDnDC1

KjX

lD1

hI©Yj l ƒ ƒ0

0Zjl > Sƒ10 4u5

ªƒu

C PS08Sƒ10 4u59

©4 SD1 1 ƒ z50 ƒ1

SD1 2Zj l

ª4Yjl

ƒƒ 00Zjl5

i0

The functional central limit theorem implies that e°4u1z5 converges in

distribution to a Gaussian process and hence that n12SD 8SSD1 z4bSƒ1

SD1 z4u55ƒ

u9 converges in distribution to a Gaussian process.

APPENDIX C: PROOF OF THEOREM 2

By the consistency of O and a Taylor series expansion of V 4 O 5

around ˆ0 , we obtain

n12D 4 O ƒ ˆ05 º báƒ14ˆ05n

ƒ 12

D V 4ˆ050 (C.1)

Note that

nƒ 1

2D V 4ˆ05 º n

ƒ 12

D

nDX

iD1

KiX

kD1

Z b

a

£w4Xik1 u5Xik Oeik4u5

CXik8Oh4u1ˆ05ƒ h04u59aik4u5¤dv4u51

where Oeik4u5 D I8Yik > bSƒ1SD1 Zik

4u59ƒg8h04u5Cˆ00Xik9. Using a Taylor

series expansion of Oh4u3ˆ05 around h04u5,

nƒ 1

2D V 4ˆ05 º n

ƒ 12

D

nDX

iD1

KiX

kD1

Z b

aw4Xik1 u5

(

Xik ƒPnD

‰D1

PK‰ŠD1 a‰Š4u5X‰ŠPnD

‰D1

PK‰ŠD1 a‰Š4u5

)

� Oeik4u5dv4u50

It follows from a Taylor series expansion and (B.1) that

nƒ 1

2D

nDX

iD1

KiX

kD1

w4Xik1 u5Oeik4u5 º nƒ 1

2D

nDX

iD1

KiX

kD1

w4Xik1 u5eik4u5

Cp1210n

ƒ1D

nDX

iD1

KiX

kD1

aik4u5Ph04u5e°4u1 Zik51

where p10 D nD=nSD. This, coupled with the uniform convergence

of SX4u3ˆ05 and the weak convergence of n12D 8SSD1 z4bSƒ1

SD1 z4u55 ƒ u9,

ensures that

nƒ 1

2D V 4ˆ05 º n

ƒ 12

D

nDX

iD1

Ui4ˆ05C Op1210nƒ1

D

nDX

iD1

KiX

kD1

Z b

a8Xik ƒ 4u3ˆ059

� aik4u5 Ph04u5e°4u1Zik5 dv4u50

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14


Because supu26a1b7

nƒ1D

PnDiD1

PKikD18Xik

ƒ 4u3 ˆ059aik4u5! 0 almost

surely, we have

Op1210nƒ1

D

nDX

iD1

KiX

kD1

Z b

a8Xik

ƒ 4u3ˆ059aik4u5 Ph04u5e°4u1Zik5 dv4u5

º nƒ 1

2D

nDCnSDX

jDnDC1

Uj 4ˆ050

It follows from the almost sure convergence of bá4ˆ05 ! á that

n12D 4 O ƒˆ05 º n

ƒ 12

D áƒ1

(nDX

iD1

Ui4ˆ05 CnDCnSDX

jDnDC1

Uj4ˆ05

)1

From the central limit theorem, we see that the distribution ofn

12D 4 O ƒ ˆ05 can be approximated by a normal vector with mean 0

and covariance matrix

nƒ1D áƒ1

(nDX

iD1

Ui4ˆ05Ui4ˆ050 CnDCnSDX

jDnDC1

Uj4ˆ05Uj4ˆ050

)áƒ10

APPENDIX D: PROOF OF THEOREM 3

Note that Q4u3x5 D n12D 4 O ƒ ˆ05

0x C n12D 8Oh4u3 O 5 ƒ h04u59. By a

Taylor series expansion of Oh4u3 O 5 around ˆ0 , we obtain

n12D 8Oh4u3 O 5ƒ h04u59

º ƒ 4u3ˆ050áƒ1nƒ 1

2D V 4ˆ05 Cn

12D 8Oh4u3ˆ05 ƒh04u590

Furthermore, by expanding the estimating function in (4), evaluatedat 8ˆ01 Oh4u3 ˆ059, around h04u5, it follows that

Q4u3x5 º 8x ƒ 4u3 ˆ0590áƒ1nƒ 1

2D V 4ˆ05 Cn

12D 8Oh4u3ˆ05ƒ h04u59

º 8x ƒ 4u3 ˆ0590áƒ1nƒ 1

2D V 4ˆ05 C n

ƒ 12

D

a4u5

�nDX

iD1

KiX

kD1

nw4Xik1 u5eik4u5C n

ƒ 12

SD aik4u5 Ph04u5e°4u1Zik5o0

Therefore, Q4u3x5 has the same asymptotic distribution as

eQ4u3 x5 D nƒ 1

2D

(nDX

iD1

eQi4u3 x5 CnDCnSDX

jDnDC1

eQj4u3x5

)0

To show that eQ4u3 x5 converges weakly to a zero-mean Gaussianprocess, let

Q1 D nƒ 1

2D

(nDX

iD1

Ui4ˆ05 CnDCnSDX

jDnDC1

Uj4ˆ05

)1

Q24u5 D nƒ 1

2D

nDX

iD1

KiX

kD1

w4Xik1 u5eik4u51

and

Q34u5 D ƒ n12D

NSD

nDCnSDX

jDnDC1

KjX

lD1

µI©Yjl

ƒƒ00Zjl > Sƒ1

0 4u5ª

ƒu

C az4u50

a4u5ƒ1SD1 2Zjl4Yjl

ƒƒ 00Zjl5

PS04Sƒ10 4u55

¶0

Then eQ4u3x5 D 8x ƒ 4u3ˆ0590áƒ1Q1 C Q24u5=a4u5C Ph04u5Q34u5.

It is straightforward to show that for any � nite number of points8u11 : : : 1 um1

9, the joint distribution of 8eQ4uk3x51k D 11 : : : 1m190 is

asymptotically normal with mean 0. To prove that eQ4u3x5 is tight(Billingsley 1986), because 8x ƒ 4u3ˆ059

0áƒ1 and a4u5 and Ph04u5

are nonrandom functions, it is suf� cient to show that Q1, Q24u5, andQ34u5 are tight. Now because Q1 does not involve u, Q1 is tight. Thetightness of Q24u5 follows from some basic properties of empiricalprocesses (Shorack and Wellner 1986, p. 109). The functional centrallimit theorem implies the weak convergence of e°4u1z5 and hencethe tightness of Q34u5. The � nite-dimensional convergence and tight-ness of eQ4u3x5 imply that eQ4u3x5 converges to a mean 0 Gaussianprocess.

[Received April 2001. Revised March 2002.]

REFERENCES

Billingsley, P. (1986), Probability and Measure (2nd ed.), New York: Wiley.Dorfman, D. D., Berbaum, K. S., and Metz, C. E. (1992), “Receiver Oper-

ating Characteristic Rating Analysis: Generalization to the Population ofReaders and Patients With the Jackknife Method,” Statistics in Radiology,27, 723–731.

Elmer-Dewitt, P., Lemonick, M., Park, A., Nash, M. (2001), “Medicine: TheFuture of Drugs,” Time, 157, 56–102.

Etzioni, R., Pepe, M., Longton, G., Hu, C., and Goodman, G. (1999), “Incor-porating the Time Dimension in Receiver Operating Characteristic Curves:A Case Study of Prostate Cancer,” Medical Decision Making, 19, 242–251.

Hanley, H. A. (1989), “Receiver Operating Characteristic (ROC) Methodol-ogy: The State of the Art,” Clinical Reviews in Diagnostic Imaging, 29,307–335.

Heagerty, P. J., and Pepe, M. S. (1999), “Semiparametric Estimation ofRegression Quantiles With Application to Standardizing Weight for Heightand Age in U.S. Children,” Applied Statistics, 48, 533–551.

Metz, C. E. (1986), “ROC Methodology in Radiologic Imaging,” InvestigativeRadiology, 21, 720–733.

Parzen, M. I., Wei, L. J., and Ying, Z. (1994), “A Resampling Method Basedon Pivotal Estimating Functions,” Biometrika, 81, 341–350.

Pepe, M. S. (1997), “A Regression Modelling Framework for Receiver Oper-ating Characteristic Curves in Medical Diagnostic Testing,” Biometrika, 84,595–608.

(1998), “Three Approaches to Regression Analysis of Receiver Oper-ating Characteristic Curves for Continuous Test Results,” Biometrics, 54,124–135.

(2000a), “An Interpretation for the ROC Curve and Inference UsingGLM Procedures,” Biometrics, 56, 352–359.

(2000b), “Receiver Operating Characteristic Methodology,” Journalof the American Statistical Association, 95, 308–311.

Pepe, M. S., Etzioni, R., Feng, Z. D., Potter, J. D., Thompson, M. L., Thorn-quist, M., Winget, M., Yasui, Y. (2001), “Phases of Biomarker Developmentfor Early Detection of Cancer,” Journal of the National Cancer Institute,93, 1054–1061.

Pollard, D. (1990), Empirical Processes: Theory and Applications, Hayward,CA: Institute of Mathematical Statistics.

Shorack, G. R., and Wellner, J. A. (1986), Empirical Processes With Appli-cations to Statistics, New York: Wiley.

Srivastava, S., and Kramer, B. S. (2000), “Early Detection Cancer ResearchNetwork,” Laboratory Investigation, 80, 1147–1148.

Swets, J. A., and Pickett, R. M. (1982), Evaluation of Diagnostic Systems:Methods From Signal Detection Theory, New York: Academy Press.

Thompson, M. L., and Zucchini, W. (1989), “On the Statistical Analysis ofROC Curves,” Statistics in Medicine, 8, 1277–1290.

Thornquist, M. D., Omenn, G. S., Goodman, G. E., Grizzle, J. E., Rosen-stock, L., Barnhart, S., Anderson, G. L., Hammar, S., Balmes, J., Cherniack,M., Cone, J., Cullen, M. R., Glass, A., Keogh, J. P., Meyskens, F., Vala-nis, B., Williams, J. H. (1993), “Statistical Design and Monitoring of theCarotene and Retinol Ef� cacy Trial,” Controlled Clinical Trials, 14, 308–324.

Tosteson, A. N., and Begg, C. B. (1988), “A General Regression Methodologyfor ROC Curve Estimation,” Medical Decision Making, 8, 204–215.

Wieand, S., Gail, M. H., James, B. R., and James, K. L. (1989), “A Familyof Nonparametric Statistics for Comparing Diagnostic Markers With Pairedor Unpaired Data,” Biometrika, 76, 585–592.

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 0

9:35

21

Dec

embe

r 20

14

semiparametric receiver operating characteristic analysis to evaluate biomarkers for disease

Documents