interpretation of immunogenicity results and …...after aggregation of data, cumulative profile can...

V.Devanarayan,Ph.D.AbbVie,Inc.

EuropeanBioanalysisForum(EBF)9th OpenMeeting:ReachingUtopia-TheKaleidoscopeofBioanalysis,November16-18,2016Barcelona,Spain

InterpretationofImmunogenicityResultsandEvaluationofClinicalAssociations

2InterpretationofImmunogenicityResultsandEvaluationofClinicalAssociations|EBF9th OpenMeeting,November16-18,2016|V.Devanarayan

RecommendationspresentedherewereinfluencedbyourcollaborationsontheseAAPSwhite-papers:

AssessmentandReportingoftheClinicalImmunogenicityofTherapeuticProteinsandPeptides- HarmonizedTerminologyandTacticalRecommendations,AAPSJournal,2014

Thequintessenceofimmunogenicityreportingforbiotherapeutics,NatureBiotechnology,2015

Authors:• GopiShankar,Ph.D.(JanssenR&D/Johnson&Johnson)• ArnoKromminga,Ph.D.(EuropeanImmunogenicityPlatform)• StevenArkin,MD(Pfizer)•MeenaSubramanyam,Ph.D.(Biogen-Idec)• Viswanath Devanarayan,Ph.D.(AbbVie)• LaurentCocea,MD,Ph.D.(HealthCanada,CERB/ClinicalEvaluationDivision)• DanielaVerthelyi,MD,Ph.D.(FDA,CDER/OfficeofTherapeuticProteins)• SusanKirshner,Ph.D.(FDA,CDER/OfficeofTherapeuticProteins)•MarkBorgini,MD(FDA,CBER/OfficeofCellularandGeneTherapies)• SarahYim,MD(FDA,DivisionofPulmonary,AllergyandRheumatologyProducts)• ChristianSchneider,MD(EMA,DanishNationalHealthandMedicinesAuthority)


ClinicalADAdataevaluation

DescribecharacteristicsoftheADAimmuneresponse:pre-existingantibodies,ADAIncidence,Titer,NAb,Kinetics,any

endogenouscross-reactivities

Descriptiveillustrations Descriptive/ExploratoryStatistics

DeterminationofclinicallyimpactfulADAthresholdofADAattributes

DetermineSubjectADAstatusbasedonsampleADAresults

DeterminesampleADAstatus


Terms&Definitions

Simpleterminologyrequiringclarification:• ADA,BindingADA,NeutralizingADA,Non-neutralizingADA,sustainingADA,

clearingADA,HAMA,HACA,HAHA,Titer,etc.

TermsusedtodescribeADAstatusofasample:• ADAPositiveSample: whenADAisdetectedinasample,thesampleisconsidered

positive

• ADANegativeSample: whenADAisnotdetectedinasample,thesampleisconsiderednegative

• ADAInconclusiveSample: whenADAisnotdetectedinasamplebutdrugispresentinthesamesampleatalevelthatcanproduceinterferenceintheADAdetectionmethod,thenthenegativeADAresultcannotbeincontrovertiblyconfirmedandthesampleclassificationforADAstatusshouldbeconsideredinconclusive.

• UnevaluableSample:whenasamplecouldnotbetestedforADAduesampleloss,mishandling,orerrorsinsamplecollection,processing,storage,etc.


Terms&Definitions(contd.)

BaselineADA(pre-existingantibodies): referstoantibodiesreactivewiththebiologicdrugmoleculebeforeinitiationoftreatment.

Treatment-inducedADA: ADAdevelopeddenovo(seroconversion)followingbiologicdrugadministrationinasubjectwithoutpre-existingADA).

Treatment-boostedADA:Pre-existingADAthatwereboostedtoahigherlevelfollowingbiologicdrugadministration(i.e.,anytimeaftertheinitialdrugadministrationtheADAtiterisgreaterthanthebaselinetiterbyascientificallyreasonablemargin.

TermsusedtodescribeADAstatusofaSubject:

• ADAPositiveSubject:Subjectwithatleast1treatment-inducedortreatment-boostedADApositivesampleatanytimeduringthetreatmentorfollow-upobservationperiod.

• BaselineADApositiveSubject: AnADApositivesubjectwithbaselinepositivesample(s),regardlessofboostingafterbiologicdrugadministration.


SamplingRecommendations

ADAshouldbeevaluatedinallstudypatientsandnotonlyinasymptom-drivenmanner

Always collectabaseline sample

Samplingfrequency duringtreatmentshouldbedesignedtomaximizetheopportunityofdetectingtreatment-inducedADAand,whenapplicable,tounderstandthekinetics.

Atleastonesampleshouldbecollectedfollowinganappropriateperiodoftimeafterthelastdrugadministration.

Fortheregistration(BLA/MAA)ofchronictreatments,regulatoryauthoritiestypicallyexpectimmunogenicitydatathroughthefirstyear.Dependingonclinicalstudylength,collectsamplesfortestingat:2weeks(optional),1mo,2mo,3mo,6mo,9mo,12mo,18mo,24mo,andeveryyearthereafterduringtreatment,andatleastonemoresampleafterthelastdrugadministrationwhenoff-treatmentpersistenceofADAneedsevaluation.


Pre-existingAntibodies

ADAatbaseline(“prevalence”)mayhaveclinicalconsequences

Reportseparatelytoenableanalysisofassociationwithclinicaloutcomesafterfirstdose

Assesstreatment-boostingofthepre-existingantibodiestounderstandassociationwithclinicaloutcomesafterrepeatdosing

“boostedbyascientificallyreasonablemargin”• Basedonareasonabletiterincrease;notlog-fold…• Titration with2-3foldserialdilutionschemeisencouraged;greater

dilutionisnotagoodidea.• titerincreasesuchasa4-fold(for2xdilutionscheme)or9-fold(for3x

dilutionscheme)isareasonablemarginforchangeintiter• Optionally,butcertainlyforhigherriskbiologics,a2-foldincreaseis

consideredsignificant.


DataAnalysis&Presentation


DeterminationofclinicallyimpactfulADAthreshold/cut-offofADAattributes




DescriptivecharacteristicsofanADAimmuneresponse

ADAPrevalence (baseline/pre-existing):o BaselineADApositivesubjectsasapercentageofthetotalnumberofsubjects

whosebaselinesamplesweretestedforADA.o Titerrange(median,IQR)ofthebaselineADApositivesamples

ADAIncidence:o OverallADAincidence:combinedresultsoftreatment-boostedADApositive

subjectsandtreatment-inducedADApositivesubjects.Computeasapercentageofthetotalnumberofevaluablesubjects,excludingbaselinepositivesubjectswithoutanysamplesavailableafterdrugadministration.

o Treatment-inducedADAincidence:computedasapercentageofthetotalnumberofevaluablesubjectsthatwereADAnegativeatbaseline.Alsoreportpeakpositivetiterandrange(median,IQR)forthisgroupofsubjects.

o Treatment-boostedADAincidence:computedasapercentageofthetotalnumberofevaluablesubjectsthatwereADApositiveatbaseline.Alsocomputethefold-increaseintiter(ratioofpeakpost-administrationtitertobaselinetiter)andrangeoftiterincreases(median,IQR).


ADAKinetics:o Onset:thetimeperiodbetweentheinitialadministrationofthebiologicdrug(ina

study)andthefirstinstanceoftreatment-inducedADA.o Computethe“mediantimetoADAdevelopment”andtheinter-quartilevalues

Q1andQ3,whichcanenableanunderstandingofonsetofADAinhalf,25%,and75%ofthetreatedsubjects,respectively.

o Duration:referstothelongevityoftreatment-inducedADA.Criteriafor“Transient”vs.“Persistent”shouldbebasedonthenaturalclearanceofendogenoushumanIg.o IfADAarepredominantlyIgG1/2/4,ADAresponselasting>16weekscanbeconsidered

asPersistent.Thisisbecauset1/2is21-25days,So97%oftheIgGwouldhavebeeneliminatedafter5half-lives(16weeks).

o IfADAarepredominantlyIgM,IgAorIgG3,use5weeks.o Donotbasedthisonwhensamplingwasstopped(i.e.,“transient”becauseseronegativity

wasachievedduringstudyiswrong)

o “Transient”vs.“Persistent”describethelongevityoftheADA. Itdoesnotimplynegativeorpositiveassociationwithclinicalconsequences!



TransientADA:– Treatment-inducedADAdetectedonlyatonesamplingtimepointduringthetreatmentorfollow-upobservationperiod(excludingthelastsamplingtimepoint,asaconservativemeasure)

or– Treatment-inducedADAdetectedat2ormoresamplingtimepointsduringthetreatment(includingfollow-upperiod,ifany),wherethefirstandlastADApositivesamples(irrespectiveofanynegativesamplesin-between)areseparatedbyaperiodlessthan16weeks,andthesubject’slastsamplingtimepointisADAnegative.



PersistentADA:• Treatment-inducedADAdetectedat2ormoresamplingtimepoints

duringthetreatment(includingfollow-upperiod,ifany),wherethefirstandlastADApositivesamples(irrespectiveofanynegativesamplesinbetween)areseparatedby>=16weeksor

• Treatment-inducedADAincidenceonlyinthelastsamplingtimepointofthetreatmentstudyperiod,oratasamplingtimepointwithlessthan16weeksbeforeanADAnegativelast sample.[Byconservativeinference]

• Althoughrare,whenapatientpopulationinastudyispredominantlyfoundtodevelopIgG3orIgA,a5weekperiodshouldbeappliedtomodifythedefinitionsoftransientandpersistentADA(insteadofthe16weeks).ThisisbecauseIgG3andIgAhaveshorterhalf-livesthanotherIgGs (IgG3,7days;IgM andIgA,5days).



NAb Incidenceandkinetics: whenstudyresultsindicatedistinctNAb-containingversusnon-Nabcontainingsubjectgroups,itisusefultolookatNAb incidenceandkineticsseparatelyforeachgroupinthesamemannerasdescribedaboveforADA.

Cross-reactivity: whenabiologicdrugmoleculeisidenticalornearlyidenticaltoanendogenousprotein(wholeorinpart),itisimportantthatthecross-reactivityoftheADAwiththeendogenousproteinbeevaluatedbecauseofthepotentialtocauseanautoimmune-likesyndrome.• Comparethekineticsandtitersofthecross-reactiveADAversusthe

whole-drugADA,andrelatethistotheworseningofdisease.



Treatment-inducedADAincidence&kinetics


ThisplotincludesADApositivepatientsforwhomtheADAonsettimeisatleast120daysbeforethelastvisitORtheywereADAnegativebythelastvisit.Referencelinesrepresentthequartiles.

Treatment-inducedADAincidence&kinesticsIllustrationwithexampleclinicaldata

ADADu

ratio

n(days)

ADAonsettime(days)


Cumulativetreatment-inducedADAincidence&kinetics

Afteraggregationofdata,cumulativeprofilecanbeshownasbelow:


DescriptiveillustrationofADATiterKinetics


DescriptiveillustrationofADATiterKineticsIllustrationwithexampleclinicaldata

84 168 196 225 280 364 532 6161

5/500(1%)

29/485(6%)

29/298(8.7%)

40/291(13.7%)

43/472(9.1%)

47/468(10%)

9/187(4.8%)


DescriptivestatisticsforADAKineticsComments

ThisapproachobviatesdefinitionsforADAimmuneresponsekinetics– theonset(early/late)orduration(transient/persistent).

Theseevaluationsarehardtointerpretforverysmallsamplesizeand/orincidencerate.

Complexstudieswithmultiplearmsandsatellitestudiesmaynotallowforstatisticalassessments.

AppliesonlytoTreatment-InducedADA(treatment-boostedADAaredifferentmechanistically).

ADAOnset:reportthequartiles(Q2is“Mediantimetoantibodyformation”)• Interpretation:“WhenhalfoftheADA+subjectsseroconverted”OR“when75%

(majority)oftheADA+subjectsseroconverted”

ADADuration:reportthequartiles(Q2is“Mediantimeofantibodyduration”)• Interpretation:“TheADAslastedQ2monthsinhalfofthesubjects andQ3monthsin

75%ofthesubjects”


AssessmentofADAvs.clinicaloutcomes

Requiresanintegratedanalysis ofPK(serumconc.,clearancerate),PD(whenapplicable),Efficacy,andADRs(acuteandnon-acute),inrelationtotheintendeddosage(testedinpivotaltrials).

Fordeterminingaclinicallyrelevant“threshold”,• ROCandTree-basedanalysisoftheADAcharacteristicsvs.PK/PD,

changesindiseaseparameters(efficacy),andlevelsofADR,maybeused.§ Notfeasible/relevantforsmallsamplesizeand/orincidencerate.


InfluenceofADAonClinicalEfficacy%ofP

atientsw

ithdesire

defficacy


ADAvs.ClinicalEfficacyIllustrationwithexampleclinicaldata

25

50

75

100

300 600 900scheduledvisitday

median patient

ADA

nonADA

MedianlongitudinalefficacyprofilesforADApositive&negativepatientsareplottedseparately.Errorbarsrepresentstandarderrorofmedian(estimatedusingMAD).EfficacyisreducedinADApositivepatientsatseveraltimepoints.

Med

ianEfficacy

Scheduledvisitdayanalyticalissues


ADAvs.ClinicalEfficacyIllustrationwithexampleclinicaldata(Titer,Duration,Onset)

Relationshipbetweenthe52weekefficacy(%reductionindiseaseactivityscore)vs.maximumtiterduringthe52-weekperiod,ADAdurationandADAonsettimearesummarizedhere.EachpointinthegraphisauniqueADApositivepatient.PatientswithADAonsetlaterthan300dayswereexcluded.Trendisapproximatedusingsmoothingspline.EfficacytrendslowerwithlongerADAduration(p<0.05),andalsowithhighertiterwithabreakpointataround120.

%Efficacy

Log(MaxTiter)

%Efficacy

ADAduration

%Efficacy

ADAonsettime


StatisticalevaluationofADAvs.ClinicalendpointsSomeimportantconsiderations

Trendsareusuallynon-linear,oftenwithastep-wiseassociation.

Therefore,linear-basedmodelssuchasLogisticorLinearRegressionmaynotcapturetheassociationswell.

Needflexiblenonparametricand/ornonlinearmodelstoevaluatetheseassociations.

TheADAparameters(onset,titer,duration)aretypicallynotindependent.Needtomodeltheirinteractions.

Theymaycorrelateinacollectivemanner(multivariate).

Whatstatisticalapproachaddressestheseneeds?


StatisticalevaluationofADAvs.ClinicalendpointsSomeconsiderations(contd.)

ForanalyzingtheimpactofeachADAparameterindependentlyonClinicalendpoints(univariate),ROCanalysisorunivariateTreemodelisuseful.• Nonparametric&nonlinear• Readilyprovidesathresholdestimatethatoptimizesthe

association(e.g.,sensitivity/specificity).

“GeneralizedAdditiveModels”(GAM)maybeusefulforvisualizingandmodelingthecollectiveassociationofADAparametersonclinicaldata.

ForcharacterizingthecollectiveimpactofADAparametersonclinicalendpoints,MultivariateTree-basedmodelmaybeuseful.• AccountsforinteractionbetweenADAparameters,inaddition

tomodelingthenonlinearityandbeingnonparametric.• Output:“DecisionThresholdsonmultipleADAparameters”


EachADAparametervs.Clinical outcomeIllustrationofROCanalysis(univariate)

Titerthresholdat120provides~76%Specificity&~74%Sensitivity.

i.e.,74%ofpatientswithfavorableefficacyhaveTiter<120,and76%ofpatientswithpoorefficacyhaveTiter>120.

IncreasingtheTiterthresholdto240,resultsin~68%Specificityand~83%Sensitivity.

Thisanalysispermitstheassociationofonlyonefeatureatatime.NeedMultivariateTree-basedmodelforassessingjointeffectofmultipleADAparameters.


ADAvs.ClinicalSafety(AEincidence)Visualizing&Modeling(GeneralizedAdditiveModels)

12345

6MRP x 1E3123

456MDR x 1E3

00.20.40.60.81

Probability of Refractory

0123456

MRP x 1E3

01

23

45

6MDR x 1E3

ColorshadesdenoteprobabilityofclinicalAE.Probabilityincreasesasthecolorchangesfromdarktolightblue.

ClinicalAE

WhenADAdurationis<4months(16weeks)andTiter<15,thereisnoimpactonClinicalAE.NotethesharpcorrelationtoclinicalAEwiththesteepincreaseintheslope/surfacewhenADAduration>16weeks&Titer>15.

Maxlog2(Titer)

ADAdu

ratio

n(m

onths)


ADAvs.ClinicalSafety(AEincidence)Illustrationof Tree-basedModel(multivariate)

18%ofpatientshaveAE(n=16)

HigherAEincidence(30%)forpatientswithADAtiter>20andonsetwithin6months.

NoAEincidenceforpatientswithlowADAtiter(<20)

*Significanceofonset-timeisnotstrong.


ADAvs.ClinicalEfficacyIllustrationof Tree-basedModel

*LogWorth >1impliesp<0.01(aftermultiplicityadjustment),

69.4%efficacy(n=61patients)

Efficacydropsto37%forpatientswithADAduration>442days(~15months)

Efficacystronger(80%)forpatientswithADAduration<16weeks.


NAb NegativeMean=86,n=19

NAb PositiveMean=71,n=12

ADA&NAb vs.ClinicalEfficacy (RoleofNAb?)Illustrationof Tree-basedModel(multivariate)

69.4%efficacy(n=61patients)

Efficacydropsto37%forpatientswithADAduration>442days(~15months)

Efficacystronger(80%)forpatientswithADAduration<16weeks.

Efficacydropsto71%forNAb +patients


LimitationswithROC/Tree-basedanalysis,andproposedimprovements

Notverystable.• SignificanceofADAparametersmayvarygreatlyduetominorchangesin

thedata.

ADAThresholdestimatesmaybehighlynoisy/variable,especiallyforsmallersamplesizeorincidence (typical).

Toobtainmorerobustthresholdestimatesandevaluatevariability(confidenceinterval),wenowproposemethodstoextendtheunivariateROCandmultivariateTreeanalysisviaaboostrap resamplingscheme.• (nexttwoslides)


Original Data

Tree/ROC 1

>= C1< C1

Tree/ROC 2

>= C2< C2 ……...

Tree/ROC B

>= CB< CB

Aggregate Thresholds (C1, C2, …., CB)

Robust Threshold = Median of this distributionSpread of this distribution reflects the variability.

Bootstrapping (sampling with replacement)

Data 1 Data 2 Data B… … ...

Threshold estimated from this approach is robust to small perturbations in data, outliers, etc.

ForincreasingrobustnessofunivariatethresholdsfromROCorTreeanalysis,weproposethisresamplingmethod:

BATTing:Bootstrapping&AggregatingThresholdsfromROC/Trees

Devanarayan,1999


Formorerobustmultivariatethresholds(signature),weproposethis“sequentialresampling”extension.

Model Growing within the potential Sig+ group• Get the BATTing threshold for each unused ADA parameter.• The best predictor is selected to split the current sig+ group• This procedure continues in the new Sig+ group

Stopping Rule:• The new added predictor goes through the likelihood ratio test for

significance.

WholePopulation (Sig+)

Sig-

(Sig+) (Sig+) Sig+

Sig- Sig- Sig-

Titer>20 Duration>16wks Onset<4wks

Sig+:subgroupofpatientswithloweredefficacy/PKorAE

SequentialBATTing:Huang,Sun,…,Devanarayan,2015


AdditionalcommentsonthestatisticalevaluationsofADAparametersvs.ClinicalPK/Efficacy/Safety

Asevidentfromthemethodsandillustrations,thisanalysisshouldideallybeperformedbyexperiencedbiostatisticians.

InadequateanalysesmayresultinfalsenegativeandfalsepositiveclaimsonADAassociationtoclinicaloutcomes!

False-NegativeAssociations:• Duetoinherentnon-linearity,linear-basedmodelsareinadequate.Mayleadto

incorrectconclusionsonlackofornoassociation.• Needflexiblenonparametric&nonlinearmethodsthataccountforstepwise

nonlineartrends&interactionsbetweenADAfeatures(Tree,GAM,etc.).• Needresampling-basedmethodstorobustify results&evaluatevariability.

False-PositiveAssociations:• Bewareofover-fitting.Performanceclaimsshouldnotbebasedonthesamedata

thatwereusedtoderivetheassociations&thresholds.• Needrigorousinternalcross-validationtoreducebiasintheestimatesofmodel

accuracyandperformance.Follow-upwithexternalvalidation.


Summary:ClinicalADAdataevaluation




DeterminationofclinicallyimpactfulADAthresholdofADAattributes

DetermineSubjectADAstatusbasedonsampleADAresults

DeterminesampleADAstatus

interpretation of immunogenicity results and …...after aggregation of data, cumulative profile can...

Documents