Using the Gamma Distribution to Determine Using the Gamma Distribution to Determine Anti-drug antibody Screening Assay Cut Points Anti-drug antibody Screening Assay Cut Points

with Non-normal Datawith Non-normal Data

Brian SchlainBrian Schlain

Nonclinical StatisticsNonclinical Statistics

for

Midwest Biopharmaceutical Statistics WorkshopMay 24-26, 2010

high volume?

ANTI-DRUG ANTIBODY (ADA) TESTING

yes

screening assay

confirmationassay

no

positive?yes

positive? titration assay output titer

yes

NEUTRALIZING ANTIBODY ASSAY (NAB) TESTING

high volume? screening assayyes

titration assay/output titer positive?

yesno

Anti-drug Anti- bodies (ADA) detected?

yes

Types of Immunogenicity Assays

Radioimmunoprecipitation Assay (RIPA) Enzyme-Linked Immunosorbent Assay (ELISA)

*Standard sandwich ELISA*Bridging ELISA*ElectrochemiluminescenceElectrochemiluminescence (ECL; IGEN)

Optical Sensor-based* Surface Plasmon Resonance (SPR; Biacore)*Guided Mode Resonance Filter (BIND; BD Biosci)

Anti-Drug Antibody (ADA) Testing with Binding Assays

Neutralizing Antibody (NAB) Testing with Blocking Assays

Related to drug mechanism of actionConverted PK assaysCell based: Some of the reagents genetically engineered and stored as cell lines

Example ADA Assay Positive Control Example ADA Assay Positive Control Response CurveResponse Curve

Example NAB Assay Positive Control Example NAB Assay Positive Control Response CurveResponse Curve

8×12 Example Screening/Confirmation Assay Plate Layout

  1 2 3 4 5 6 7 8 9 10 11 12

A 1S 1C 5S 5C NC LPC 9S 9C 13S 13C 17S 17C

B 1S 1C 5S 5C NC LPC 9S 9C 13S 13C 17S 17C

C 2S 2C 6S 6C NC LPC 10S 10C 14S 14C 18S 18C

D 2S 2C 6S 6C NC HPC 10S 10C 14S 14C 18S 18C

E 3S 3C 7S 7C NC HPC 11S 11C 15S 15C 19S 19C

F 3S 3C 7C 7C NC HPC 11S 11C 15S 15C 19S 19C

G 4S 4C 8S 8C NC HPC-C

12S 12C 16S 16C 20S 20C

H 4S 4C 8S 8C NC HPC-C

12S 12C 16S 16C 20S 20C

Screening Assay Cut PointsScreening Assay Cut PointsFixedFixed: mean and sd do not change across plates.: mean and sd do not change across plates.– CP=meanCP=meanXX ±± a a×sd×sdXX

– X=Sample assay signal responseX=Sample assay signal response

FloatingFloating: mean changes, but sd remains constant : mean changes, but sd remains constant across plates.across plates.– Multiplicative: Multiplicative: CF=meanCF=meanYY ±± a a×sd×sdYY

Y=X/NCY=X/NCCP=CFCP=CF××NCNC

– Additive: Additive: CF=meanCF=meanZZ ±± a a×sd×sdZZ

Z=X – NCZ=X – NCCP=CF + NCCP=CF + NC

DynamicDynamic: mean and sd both change with every plate.: mean and sd both change with every plate.

Screening Multiplicative Cut Point CF DeterminationScreening Multiplicative Cut Point CF Determination

• Screen data for outliers

• Normalize:Y= Sample/NC

Model Model distribution of of normalized valuesnormalized values

non-normal normal or log-normal

• Gamma

• Weibull

• Nonparametric percentile

Validation Phase CP=CF×NC

Estimate CF

CUT POINT RULEsample>cut point, judge screening positive.

cut point=CF avg(NC).

NC: pooled negative sera control   NORMAL THEORY CORRECTION FACTOR (CF) ESTIMATION(Guttman I, Statistical Tolerance Regions: Classical and Bayesian, 1970)

n negative sera samples.R (or normalized value)=Sample/Avg(NC).

  mean (MR) and sd (SR) of R.

CF=MR + tdf,α SR×(1 + 1/n)0.5 no transformation .

CF=exp{MR + tdf,α SR×(1 + 1/n) 0.5 } log transformation.

 df=n – 1.α=targeted false positive rate=0.05.

 R

ADA Screening Assay Cut Point CF Estimation ADA Screening Assay Cut Point CF Estimation under Normal Theoryunder Normal Theory

CUT POINT RULEsample < cut point, judge screening positive.

cut point=CF avg(NC).

NC: pooled negative sera control   NORMAL THEORY CORRECTION FACTOR (CF) ESTIMATION(Guttman I, Statistical Tolerance Regions: Classical and Bayesian, 1970)

n negative sera samples.R (or normalized value)=Sample/Avg(NC).

  mean (MR) and sd (SR) of R.

CF=MR - tdf,α SR×(1 + 1/n)0.5 no transformation .

CF=exp{MR - tdf,α SR×(1 + 1/n) 0.5 } log transformation.

 df=n – 1.α=targeted false positive rate=0.05.

 R

NAB Screening Assay Cut Point CF Estimation NAB Screening Assay Cut Point CF Estimation under Normal Theoryunder Normal Theory

2-Parameter Gamma Density 2-Parameter Gamma Density FunctionsFunctions

Gamma approaches normal as k increases.

2-parameter Gamma2-parameter Gamma

3-parameter Gamma3-parameter Gamma

kk

xsk

sxskxf

)(

]/)exp[(1

)(),,|(

sx (threshold parameter)

Some common gamma fitting methods:maximum likelihood estimation (MLE)

-SAS UNIVARIATE, R OPTIM.-becomes unstable when k approaches 1

method of moments (ME)modified moment estimation (MME) (Cohen and Whitten)

Maximum Likelihood Estimation (MLE) of 2-Maximum Likelihood Estimation (MLE) of 2-parameter Gammaparameter Gamma

Unstable when estimate of k close to 1.ADA distributions tend to be unimodal with k>1.NAB distributions tend to be unimodal with k<1.

-Fit gamma to 1/R=Avg(NC)/Sample.

Estimation of Gamma Parameters Estimation of Gamma Parameters by Method of Moments (ME)by Method of Moments (ME)

Gamma central Gamma central momentmoment

Empirical central Empirical central momentmoment

MeanMean S + S + θθkk =m=m11 (mean) (mean)

VarianceVariance θθ22kk =m=m22 (variance)(variance)

Third standard Third standard momentmoment =m=m33 /(m /(m22 ) )3/23/2

(std. (std. skewness)skewness)

k

23

ME for 3-parameter GammaME for 3-parameter Gamma

Gamma Gamma parameterparameter

Moment estimatorMoment estimator

k (shape)k (shape) =4m=4m2233

/m/m33 22

θθ (scale) (scale) =m=m33 / (2m / (2m22 ) )

s (threshold)s (threshold) =m=m11 - (2m - (2m2222 )/m )/m33

Modified Moment Estimators (MME) for 3-parameter Modified Moment Estimators (MME) for 3-parameter Gamma (using SAS PROBGAM function)Gamma (using SAS PROBGAM function)

23

3

3

21

33111

231

23331

ˆ/4ˆ

ˆ/2ˆ

2/ˆˆ

...

)2/ˆ/()ˆ/2(ˆ/)ˆ(

1

1)ˆ/4,()ˆ/4;2/ˆ;ˆ/2|(

k

sdmeans

sd

WWWsd

meanXW

WsWU

nUPROBGAMksWG

n

Cohen AC, Whitten BJ, Modified moment estimation for the 3-parameter Gamma distribution, J. of Qual. Tech., 18, 1, 1986, 53-62

1

1)ˆ,ˆ,ˆ|( 1 nksXG

Calculating the Cut Point Correction Factor as a Calculating the Cut Point Correction Factor as a Gamma PercentileGamma Percentile

Cut point correction factor (CF) calculated using SAS GAMINV Cut point correction factor (CF) calculated using SAS GAMINV function:function:

P=.95 if false positive rate targeted at 5% (ADA assay)-NAB assay: Model reciprocal of normalized values so that skewness > 0.

θ=scale parameters=threshold parameterk=shape parameter

skPGAMINVCF ˆ)ˆ,(ˆ

CP=CF×NCCP=CF×NC

3-parameter Gamma Estimation3-parameter Gamma Estimation

MLE preferred to MME or ME.MLE preferred to MME or ME.– SAS UNIVARIATESAS UNIVARIATE– R OPTIMR OPTIM

MME generally better than ME.MME generally better than ME.

MLE unstable for k near 1. (Johnson and Kotz recommend k>2.5).MLE unstable for k near 1. (Johnson and Kotz recommend k>2.5).

MME can be calculated for any k. MME can be calculated for any k.

MME comparable to MLE with increasing n or MME comparable to MLE with increasing n or αα33 (Cohen and (Cohen and Whitten).Whitten).

For < 0.10, consider normal distribution (Cohen and For < 0.10, consider normal distribution (Cohen and Whitten).Whitten).k

23

Standard Gamma Simulations Comparing MLE and Standard Gamma Simulations Comparing MLE and Nonparametric Percentile Cut Point Estimators Nonparametric Percentile Cut Point Estimators

(targeted false positive rate=5%)(targeted false positive rate=5%)

Shape Param.

n MLE se FP rate

se Nonpar. Est.

se FP rate

se

2 30 5.085 1.029 .050 .037 4.719 .924 .065 .043 60 4.827 .557 .051 .023 4.744 .648 .057 .028 120 4.794 .379 .050 .016 4.739 .471 .054 .020 240 4.786 .264 .049 .011 4.743 .337 .052 .014 4 30 8.010 1.206 .054 .036 7.701 1.161 .065 .044 60 7.856 .695 .051 .023 7.750 .799 .057 .028 120 7.834 .487 .050 .016 7.754 .582 .053 .020 240 7.829 .335 .049 .011 7.752 .415 .052 .014 8 30 13.458 1.469 .053 .035 13.085 1.482 .065 .044 60 13.314 . 888 .050 .023 13.153 1.043 .056 .029 120 13.289 .610 .049 .015 13.141 .735 .053 .020 240 12.278 .427 .048 .011 13.147 .532 .052 .014 16 30 23.564 1.862 .051 .033 23.005 1.920 .065 .044 60 23.390 1.128 .048 .022 23.089 1.308 .056 .028 120 23.363 .780 .047 .015 23.106 .958 .053 .020 240 23.355 .545 .046 .010 23.105 .684 .051 .014

Abbreviations: n=sample size; η=gamma shape parameter; MLE=maximum likelihood estimator; se=standard error; FP rate=false positive rate; Nonpar. Est.=nonparametric estimator.

Standard Gamma Simulations Comparing MLE with Normal Standard Gamma Simulations Comparing MLE with Normal Based Cut Point Estimators (targeted false Based Cut Point Estimators (targeted false

positive rate=5%)positive rate=5%)

Shape Param.

n MLE se FP rate

se Normal z-method

se FP rate

se Normal predict. interval

se FP rate

se

2 30 5.085 1.029 .050 .037 4.272 .640 .083 .041 4.385 .662 .076 .039 60 4.827 .557 .051 .023 4.301 .466 .077 .028 4.357 .473 .073 .027 120 4.794 .379 .050 .016 4.306 .332 .074 .020 4.334 .335 .072 .019 240 4.786 .264 .049 .011 4.322 .237 .072 .014 4.322 .237 .072 .014 4 30 8.010 1.206 .054 .036 7.234 .808 .079 .039 7.395 .834 .072 .037 60 7.856 .695 .051 .023 7.259 .572 .073 .026 7.338 .581 .070 .026 120 7.834 .487 .050 .016 7.273 .412 .071 .019 7.312 .415 .069 .018 240 7.829 .335 .049 .011 7.283 .286 .069 .013 7.302 .287 .068 .013 8 30 13.458 1.469 .053 .035 12.604 1.044 .074 .038 12.834 1.076 .067 .035 60 13.314 .888 .050 .023 12.628 .734 .069 .025 12.740 .745 .066 .025 120 13.289 .610 .049 .015 12.641 .517 .067 .017 12.697 .521 .065 .017 240 12.278 .427 .048 .011 12.650 .367 .066 .012 12.677 .369 .065 .012 16 30 23.564 1.862 .051 .033 22.525 1.357 .070 .036 22.851 1.397 .062 .033 60 23.390 1.128 .048 .022 22.547 .948 .066 .024 22.706 .961 .062 .023 120 23.363 .780 .047 .015 22.569 .677 .064 .017 22.648 .681 .062 .016 240 23.355 .545 .046 .010 22.584 .477 .062 .012 22.623 .478 .061 .012

Abbreviations: n=sample size; η=gamma shape parameter; MLE=maximum likelihood estimator; se=standard error; FP rate=false positive rate; normal z-method=mean +1.645×sd; Normal Predict. Interval=upper limit of a 1-sided prediction interval based on normal theory.

Standard Gamma Distribution Simulations Comparing Standard Gamma Distribution Simulations Comparing MLE with Log-normal Cut Point Estimators MLE with Log-normal Cut Point Estimators

(targeted false positive rate=5%)(targeted false positive rate=5%)

Histogram of ADA ELISA Trial Pre-dose Histogram of ADA ELISA Trial Pre-dose Screening Assay Normalized Values (n=175)Screening Assay Normalized Values (n=175)

Sample#29

Sample#34

outlyingvalues

MME Gamma Q-Q Plot (with outlying MME Gamma Q-Q Plot (with outlying sample 34 excluded)sample 34 excluded)

Sample # 29

outlying value

MME Gamma Q-Q Plot MME Gamma Q-Q Plot (with outlying samples 29 and 34 excluded)(with outlying samples 29 and 34 excluded)

Histogram of ADA Elisa Trial Pre-dose Normalized Values Histogram of ADA Elisa Trial Pre-dose Normalized Values (with outlying samples 29 and 34 excluded)(with outlying samples 29 and 34 excluded)

skewed to theright

SAS UNIVARIATE EDF Goodness of Fit Test p-values SAS UNIVARIATE EDF Goodness of Fit Test p-values (outlying samples 29 and 34 excluded)(outlying samples 29 and 34 excluded)

testtest gammagamma normalnormal log-normallog-normal

Shapiro-WilkShapiro-Wilk -- <.0001<.0001 .031.031

Kolmogorov-Kolmogorov-SmirnovSmirnov

.146.146 .010.010 .025.025

Cramer-von Cramer-von MisesMises

.089.089 .0050.0050 .073.073

Anderson-Anderson-DarlingDarling

.151.151 .0050.0050 .060.060

Gamma Parameter and CF Estimates Gamma Parameter and CF Estimates (outlying samples 29 and 34 excluded)(outlying samples 29 and 34 excluded)

MMEMME MLEMLE

mean (of norm. values)mean (of norm. values) 1.2701.270

sd (of norm. values)sd (of norm. values) 0.1700.170

skewness (of norm. values)skewness (of norm. values) 0.9020.902

WW11 (smallest stand. value) (smallest stand. value) -2.102-2.102

nn 173173

αα33 .474.474 .727.727

θθ (scale) (scale) .040.040 .061.061

s (threshold)s (threshold) .553.553 .809.809

k (shape)k (shape) 17.79617.796 7.5787.578

CF (upper 3.9% percentile of CF (upper 3.9% percentile of gamma)gamma)

1.601.60 1.601.60

CP=CF×NCCP=CF×NC

target upper gamma percentile=target upper gamma percentile=

5% - 100%×(2/175)=3.9%5% - 100%×(2/175)=3.9%

Gamma Distribution CF DeterminationGamma Distribution CF DeterminationTarget false positive (fp) rate=5%Target false positive (fp) rate=5%

Estimated percentage of outlying samples is 1.1% (=100%Estimated percentage of outlying samples is 1.1% (=100%×2/175)×2/175)

Target fp rate – percentage of outlying samples =5% - 1.1%=3.9%Target fp rate – percentage of outlying samples =5% - 1.1%=3.9%

CF=upper 3.9% percentile of fitted gamma=1.60CF=upper 3.9% percentile of fitted gamma=1.60

– CF=CF=θθ×GAMINV(p,k) + s=1.60.×GAMINV(p,k) + s=1.60.

p=1- 0.039=.961; k= p=1- 0.039=.961; k= 17.796; 17.796; θθ==.040; s=.553..040; s=.553.

CP=CF×NCCP=CF×NC

Empirical fp rate=5.7% (=100%×10/175)Empirical fp rate=5.7% (=100%×10/175)

ReferencesReferencesSchlain B et al., A novel gamma-fitting statistical method for anti-drug Schlain B et al., A novel gamma-fitting statistical method for anti-drug antibody assays to establish assay cut points for data with non-normal antibody assays to establish assay cut points for data with non-normal distribution, JIM, V. 352, Issues 1-2, 31Jan. 2010, pp. 161-168.distribution, JIM, V. 352, Issues 1-2, 31Jan. 2010, pp. 161-168.Guttman I, Statistical Tolerance Regions: Classical and Bayesian, Guttman I, Statistical Tolerance Regions: Classical and Bayesian, 1970).1970).Cohen AC, Whitten BJ, Modified moment estimation for the 3-Cohen AC, Whitten BJ, Modified moment estimation for the 3-parameter Gamma distribution, J. of Qual. Tech., 18, 1, 1986, 53-62.parameter Gamma distribution, J. of Qual. Tech., 18, 1, 1986, 53-62.Cohen AC, Whitten BJ, Modified moment and maximum likelihood Cohen AC, Whitten BJ, Modified moment and maximum likelihood estimators for parameters of the three-parameter gamma distribution, estimators for parameters of the three-parameter gamma distribution, commun. Statist.-Simula. Computa., 11(2), 197-216 (1982).commun. Statist.-Simula. Computa., 11(2), 197-216 (1982).Bowman KO and Shenton LR, Properties of Estimators for the Gamma Bowman KO and Shenton LR, Properties of Estimators for the Gamma Distribution, Marcel Dekker, 1988.Distribution, Marcel Dekker, 1988.Johnson NL and Kotz S, continuous Univariate Distributions, Vol. 1, Johnson NL and Kotz S, continuous Univariate Distributions, Vol. 1, Houghton Mifflin company, 1970.Houghton Mifflin company, 1970.Krishnamoorthy K, Mathew T, and Mukherjee S, Normal-based Krishnamoorthy K, Mathew T, and Mukherjee S, Normal-based methods for a gamma distribution: prediction and tolerance intervals methods for a gamma distribution: prediction and tolerance intervals and stress-strength reliability, Technometrics, Vol. 50, No. 1, pp. 69-and stress-strength reliability, Technometrics, Vol. 50, No. 1, pp. 69-78.78.

BACKUP SLIDESBACKUP SLIDES

Standard Gamma Simulations Comparing MLE, Standard Gamma Simulations Comparing MLE, Nonparametric, and WH Cut Point Estimators Nonparametric, and WH Cut Point Estimators

(targeted false positive rate=5%)(targeted false positive rate=5%)

Shape Param.

n MLE se FP rate

se Nonpar. Est.

se FP rate

se WH (cube root)

se FP rate

se

2 30 5.085 1.029 .050 .037 4.719 .924 .065 .043 4.910 .738 .051 .031 60 4.827 .557 .051 .023 4.744 .648 .057 .028 4.803 .506 .052 .021 120 4.794 .379 .050 .016 4.739 .471 .054 .020 4.746 .353 .052 .015 240 4.786 .264 .049 .011 4.743 .337 .052 .014 4.730 .249 .052 .011 4 30 8.010 1.206 .054 .036 7.701 1.161 .065 .044 7.975 .918 .051 .030 60 7.856 .695 .051 .023 7.750 .799 .057 .028 7.851 .632 .051 .021 120 7.834 .487 .050 .016 7.754 .582 .053 .020 7.789 .443 .051 .015 240 7.829 .335 .049 .011 7.752 .415 .052 .014 7.765 .306 .051 .010 8 30 13.458 1.469 .053 .035 13.085 1.482 .065 .044 13.447 1.174 .050 .030 60 13.314 . 888 .050 .023 13.153 1.043 .056 .029 13.291 .802 .050 .021 120 13.289 .610 .049 .015 13.141 .735 .053 .020 13.214 .560 .050 .015 240 12.278 .427 .048 .011 13.147 .532 .052 .014 13.180 .393 .050 .010 16 30 23.564 1.862 .051 .033 23.005 1.920 .065 .044 23.487 1.503 .050 .030 60 23.390 1.128 .048 .022 23.089 1.308 .056 .028 23.274 1.024 .050 .021 120 23.363 .780 .047 .015 23.106 .958 .053 .020 23.190 .723 .050 .015 240 23.355 .545 .046 .010 23.105 .684 .051 .014 23.152 .505 .050 .010

Abbreviations: n=sample size; η=gamma shape parameter; MLE=maximum likelihood estimator; se=standard error; FP rate=false positive rate; Nonpar. Est.=nonparametric estimator; WH=Wilson-Hilferty estimator.

Need for further research on WHNeed for further research on WH

How well does it perform when the real How well does it perform when the real distribution is not quite a gamma, distribution is not quite a gamma, but the gamma is the best but the gamma is the best approximation that can be found?approximation that can be found?