spatial architecture and arrangement of tumor-inﬁltrating ...her2þ breast cancer h&e images...

Precision Medicine and Imaging

Spatial Architecture and Arrangement ofTumor-Infiltrating Lymphocytes for PredictingLikelihood of Recurrence in Early-StageNon–Small Cell Lung CancerGerm�an Corredor1,2, Xiangxue Wang1, Yu Zhou1, Cheng Lu1, Pingfu Fu3,Konstantinos Syrigos4, David L. Rimm5, Michael Yang6, Eduardo Romero2,Kurt A. Schalper5, Vamsidhar Velcheti7, and Anant Madabhushi1

Abstract

Purpose: The presence of a high degree of tumor-infiltratinglymphocytes (TIL) has been proven to be associated withoutcome inpatientswithnon–small cell lung cancer (NSCLC).However, recent evidence indicates that tissue architecture isalso prognostic of disease-specific survival and recurrence. Weshow a set of descriptors (spatial TIL, SpaTIL) that capturedensity, and spatial colocalization of TILs and tumor cellsacross digital images that can predict likelihood of recurrencein early-stage NSCLC.

Experimental Design: The association between recurrencein early-stage NSCLC and SpaTIL features was explored on 301patients across four different cohorts. Cohort D1 (n¼ 70) wasused to identify the most prognostic SpaTIL features and totrain a classifier to predict the likelihood of recurrence. Theclassifier performance was evaluated in cohorts D2 (n ¼ 119),D3 (n¼ 112), andD4 (n¼ 112). Two pathologists graded each

sample of D1 andD2; intraobserver agreement and associationbetween manual grading and likelihood of recurrence wereanalyzed.

Results: SpaTIL was associated with likelihood of recurrencein all test sets (log-rank P < 0.02). A multivariateCox proportional hazards analysis revealed an HR of 3.08(95%confidence interval, 2.1–4.5,P¼ 7.3� 10�5). In contrast,agreement among expert pathologists using tumor gradewas moderate (Kappa¼ 0.5), and the manual TIL grading wasonly prognostic for one reader in D2 (P ¼ 8.0 � 10�3).

Conclusions: A set of features related to density and spatialarchitecture of TILs was found to be associated with a likeli-hood of recurrence of early-stage NSCLC. This informationcould potentially be used for helping in treatment planningand management of early-stage NSCLC.

See related commentary by Peled et al., p. 1449

IntroductionEarly-stage (stages I and II) non–small cell lung cancer (NSCLC;

refs. 1, 2) is typically treated with complete surgical excision.

However, even after resecting the entire tumor mass, 30% to 55%of patients develop disease recurrence within first 5 years follow-ing surgery (3). The ability to identify patients who are at a higherrisk of recurrence could help selecting those patients who maygain maximum benefit with further treatment including adjuvantchemotherapy following standard-of-care surgery.

NSCLC histopathology is characterized by a complex interplayof tumor cells, immune cells (lymphocytes, plasma cells, macro-phages, and granulocytes), fibroblasts, and pericytes/endothelialcells. Recent evidence suggests that the interaction of tumor cellswith immune cells has a high association with likelihood ofdisease progression and influences tumor development, invasion,metastasis, and patient outcome (1, 2). Several independentstudies (2, 4–6) meanwhile have also shown an associationbetween patient survival and treatment response with anincreased density of tumor-infiltrating lymphocytes (TIL) indiverse solid tumor types. In addition, there is substantial evi-dence supporting the fact that increased TIL density is associatedwith better chemotherapeutic response (7–10).

Studies have also found substantial interreader variability inestimating TIL density using hematoxylin and eosin (H&E)–stained slides. This has limited the routine use of TIL density asametric in the clinic as a prognostic marker for NSCLC. Brambillaand colleagues (6), for instance, determined that interpathologistagreement was at best moderate (Kappa ¼ 0.59) in quantifyingTILs on tissue slides. Although attempts have been made

1Center for Computational Imaging and Personalized Diagnostics, CaseWesternReserve University, Cleveland, Ohio. 2Computer Imaging and Medical Applica-tions Laboratory, Universidad Nacional de Colombia, Bogot�a, Colombia.3Department of Population and Quantitative Health Sciences, Case WesternReserve University, Cleveland, Ohio. 4Department of Medicine, University ofAthens, Sotiria General Hospital, Athens, Greece. 5Department of Pathology,Yale University School of Medicine, New Haven, Connecticut. 6Department ofPathology–Anatomic, University Hospitals, Cleveland, Ohio. 7Hematology andMedical Oncology Department, Cleveland Clinic, Cleveland, Ohio.

Note: Supplementary data for this article are available at Clinical CancerResearch Online (http://clincancerres.aacrjournals.org/).

G. Corredor and X. Wang contributed equally to this article.

Prior presentation: Poster presentation at the USCAP 106th Annual Meeting,March 4–10, 2017.

Current address for V. Velcheti: Thoracic Oncology Program, NYU Langone -Laura and Isaac Perlmutter Cancer Center, New York, New York.

Corresponding Author: Anant Madabhushi, Case Western Reserve University,2071 Martin Luther King Jr. Drive, Cleveland, OH 44106. Phone: 216-368-8519;Fax: 216-368-4969; E-mail: [email protected]

doi: 10.1158/1078-0432.CCR-18-2013

�2018 American Association for Cancer Research.

ClinicalCancerResearch

Clin Cancer Res; 25(5) March 1, 20191526

on July 21, 2020. © 2019 American Association for Cancer Research. clincancerres.aacrjournals.org Downloaded from

Published OnlineFirst September 10, 2018; DOI: 10.1158/1078-0432.CCR-18-2013

http://crossmark.crossref.org/dialog/?doi=10.1158/1078-0432.CCR-18-2013&domain=pdf&date_stamp=2019-2-6

http://clincancerres.aacrjournals.org/

to establish guidelines for standardizing TIL grading in breastcancer (11), these efforts have been lagging in lung cancer.

Over the last few years, there has been increasing interest indeveloping automated cell segmentation and detection algo-rithms for identifying and quantifying the extent of TILs fromroutine H&E pathology slide images (12–14). These approachescan be broadly categorized into (i) involving extraction of visualfeatures (12), (ii) employing morphologic operations (15, 16),and (iii) using deep learning–based models (17). However, theseapproaches have primarily focused on either counting the indi-vidual TILs (13, 14) or estimating the TIL grade, typically as beinglow, moderate, or high (12, 15, 16).

Interestingly, there has also been a recent surge in looking atspatial patterns of TILs and investigating their relationship withdisease outcome. Multiplexed quantitative immunofluorescence(QIF) and IHC-based methods have been employed for objec-tively identifying TIL subtypes and correlate the spatial arrange-ment and density of these TIL subtypes with overall survival (OS)inNSCLC (4, 18). For instance, Schalper and colleagues (4) foundout that increased levels of CD3 and CD8 TILs were significantlyassociated with improved 5-year OS. Similarly, Barua and col-leagues (18) showed that spatial interplay between tumor andregulatory T cells was associated with OS inNSCLC. Furthermore,Liu and colleagues (19) demonstrated that the presence of CD8þ

and FOXP3þ TILs was correlated with the response of platinum-based neoadjuvant chemotherapy in advanced NSCLC.

Interestingly, computer-extracted features of spatial patternsand morphologic attributes of TILs from routine H&E slidesalso appeared to be prognostic. Basavanhally and colleagues(15) explored the use of graph network algorithms to spatiallycharacterize the arrangement of machine-identified TILs inHER2þ breast cancer H&E images to predict TIL grade (i.e.,high or low). Yu and colleagues (20) and Luo and colleagues(21) extracted quantitative morphologic features of nuclei andthe surrounding cytoplasm from H&E tissue images of early-stage NSCLC patients (e.g., area, shape, intensity, texture,density) for predicting survival. Saltz and colleagues (22) used

a deep learning model to identify patches of TILs in images,which were subsequently clustered using different similaritymetrics. From such patch clusters, different indices were com-puted (Ball and Hall, Banfield and Raftery, C, determinantratio, etc.) which was then found to be correlated with patientsurvival across different tumor types.

Given the recent evidence that the colocalization of immuneand cancer nuclei is prognostic of disease outcome (20–23), thiswork aims to develop and evaluate new computer-extractedspatial TIL (SpaTIL) features relating to (i) the spatial architectureof TIL clusters, (ii) colocalization of clusters of both TILs andcancer nuclei, and (iii) variation in density of TIL clusters acrossthe tissue slide image. Specifically, our goal was to evaluate theassociation between disease recurrence and the SpaTIL featureson patients with stage I and II of NSCLC. In addition, we alsosought to compare the SpaTIL features in terms of their abilityto predict recurrence in these patients with NSCLC against themanually estimated degree of TILs by two thoracic pathologists.

Materials and MethodsDatasets

Tissue microarrays (TMA) obtained from H&E-stained slidescollated from three independent and well-characterized early-stage NSCLC cohorts were included in this study, representinga total of 301 patients. The three cohorts are represented by D1

(n¼ 70),D2 (n¼119), andD3 (n¼ 112). A fourth dataset, namedD4 (n ¼ 112), was also included, containing tissue cores corre-sponding to the same patients in D3 but extracted from differentregions of the tumor. The corresponding clinicopathologic andoutcome information from patients in D1 to D4 was obtainedfrom retrospective chart review from the institutions at which thedatasets were collated after obtaining the respective InstitutionalReview Board approval.

Patients from D1 and D2 comprised formalin-fixed paraffin-embedded tumor sections from previously reported retrospec-tive collections of NSCLC patients (4). Note that 0.6-mm coresfrom each tumor were then arrayed to make up the TMA. Afurther 116 patients provided two cores from the same tumor,to make up TMA datasets D3 and D4. The standard TMApreparation protocol is described in ref. (24). D1 and D2

were scanned and digitized using an Aperio Scanscope CSwhole-slide imager at 20x magnification. D3 and D4 werescanned and digitized at 20x using a Ventana iScan HT Scanner(serial #: BI15N7205). Finally, a 1500 � 1500 pixel image at20� magnification was extracted and used to represent aunique tumor sample derived from each patient.

D1was employed for feature discovery andmodel training. Thisdataset included samples from 350 patients and was collectedindependently at Sotiria General Hospital and Patras UniversityGeneral Hospital between 1991 and 2001. D2 and D3 were usedfor independently validating the trained classifier. D2 comprisedsamples from 202 patients and was collected at Yale Pathologybetween1988 and2003.D3 andD4 (tissue cores corresponding tosame patients in D3 but from a different portion of the tumor)comprised tissue images from 189 patients and was collected atthe Cleveland Clinic between 2004 and 2014. D3 and D4 wereused to quantitatively assess the ability of the approach to dealwith intratumoral heterogeneity. Supplementary Fig. S1 illus-trates the inclusion and exclusion criteria for patient selectionfor this study.

Translational Relevance

The presence of a high degree of tumor-infiltrating lympho-cytes (TIL) has been found to be associated with better prog-nosis in non–small cell lung cancer (NSCLC). There is amoderate agreement between pathologists when assessing thedegree of TIL presence from histopathologic tissue specimens.In this study, we present a new set of computer-extractedquantitative features (spatial TIL, SpaTIL) related to the spatialarchitecture of TILs, the colocalization of TILs and cancernuclei, and the density variation of TIL clusters from hema-toxylin and eosin images. We demonstrate that SpaTIL canpredict the likelihood of recurrence in early-stage NSCLC.Compared with clinicopathologic features, these features wereindependently prognostic of disease recurrence. Followingadditional, independent, multisite validation, this SpaTIL testcould be used in a manner similar to genomic-based com-panion diagnostic tests to identify those patients who have ahigher chance of recurrence and who might thus potentiallyreceive added benefit from adjuvant chemotherapy followingsurgical resection.

Spatial Architecture of TILs, Recurrence in NSCLC

www.aacrjournals.org Clin Cancer Res; 25(5) March 1, 2019 1527




Automatic characterization of TILsIdentification of lymphocytes. The first step in the process was toidentify the spatial location of the TILs ondigitizedH&E images. Awatershed-based algorithm (25) was used for automaticallydetecting the nuclei. This method applies a set of mathematicaloperations (fast radial symmetry transform and regionalminima)at different scales to identify candidate locations for nuclei. Thismethodwas chosen based on its documented advantages, includ-ing simplicity, speed, and the ease with which parameters can beadjusted and fine-tuned.

Once nuclei were detected, the approach described in ref.(26) was used to distinguish lymphocytes from nonlympho-cytes. This approach takes advantage of the fact that TILs tend tobe smaller compared with cancerous nuclei, and they also tendto be more rounded and with a darker, more homogeneousstaining. Once all the candidate nuclei were identified via thewatershed approach described above, a machine classifier using7 image-derived features related to texture, shape, and colorattributes of the segmented nuclei was used to classify theindividual nuclei as corresponding either to TILs or to non-TILs (Fig. 1B and E).

Quantitative evaluation of spatial arrangement of TILs. Spatial TILgraph construction A graph is a mathematical construct comprisingof a finite sets of objects (nodes) that capture global and localrelationships via pairwise connections (edges) between thenodes.Graphs have been previously used to quantitatively characterize

nuclear architecture in histopathologic images by representing thenuclei as nodes and subsequently quantifying neighborhoodrelationships (e.g., proximity) and spatial arrangement betweenthe nodes (15, 27, 28).

In order to evaluate a spatial network of TILs and to extractthe corresponding SpaTIL features, we first identified sets ofclusters of proximal TILs and non-TILs, respectively. We repre-sented the centroids of each of the individual TILs and non-TILsas nodes of a graph. Using the approach described in refs.(27, 28), each node was connected to others based on theweighted Euclidean norm where a weighting function favorsthe connectivity between proximal nodes. Utilizing this processresulted in multiple disconnected subgraphs or clusters of TILsbeing generated. This process was also repeated separately forall the non-TILs.

SpaTIL features We extracted two separate sets of SpaTIL features.The first set comprised of 20 features related to spatial arrange-ment of TILs, extracted from the TIL cluster graphs. These featuresincluded first-order statistics (e.g., mean, mode, median) of thefollowing representative descriptors: number of lymphocyteswithin the clusters, ratio between the area of the TIL clusters andarea of the TMA spot, and ratio between the numbers of TILswithin the cluster and the cluster area. The second set included 65features describing the relationship between TIL and non-TILclusters extracted for each image. These included the ratio betweenthe density (ratio between the number of nuclei within the cluster

Figure 1.

Representative TMA tissue spots of recurrent (top row) and nonrecurrent (bottom row) early-stageNSCLC cases. The first column (A andD) shows the original H&E-stained images. Identification of TILs (yellow) and non-TILs (cyan) is presented in the second column (B and E). The third column (C and F) illustrates thequalitative representation of one of the SpaTIL features overlaid on the original images, specifically, the variation in the density of lymphocyte clusters. The color barsrepresent the density measurement (H stands for highly dense clusters, whereas L stands for low-density or sparse clusters). Nonrecurrence cases arecharacterized by the presence of more high-density clusters, whereas recurrence cases were comprised of a larger number of low-density clusters.

Corredor et al.

Clin Cancer Res; 25(5) March 1, 2019 Clinical Cancer Research1528




and the cluster area) of anonlymphocyte cluster and thedensity ofits closest lymphocyte cluster, the intersecting areas of the lym-phocyte and nonlymphocyte clusters, a value indicating if thenearest neighbor of a lymphocyte cluster is either a lymphocyte ora nonlymphocyte cluster. The 85 features are listed in the Sup-plementary Material (see Supplementary Table S1).

Feature selection. Theminimum redundancymaximum relevancefeature selection method (29) was employed to identify theSpaTIL features that most correlated with recurrence in the dis-covery set D1 while also eliminating features which were grosslysimilar to each other to prevent redundancy.

Comparative strategiesInterreader variability in TIL estimation by human readers. Twoexpert thoracic pathologists experienced in grading TILs wereasked to determine the infiltration grade for each of the imagesin D1 and D2 via visual evaluation of digitized H&E-stainedimages. An in-house custom web application was used by thereaders to assign an infiltration score to each image. Infiltrationoptions were defined based on findings reported in refs. (4, 6) as(0) no infiltration (virtual absence of TILs), (1) low (1%–33%),(2) moderate (34%–66%), and (3) high (67%–100%).

The agreement among pathologists during the TIL-grading taskwas measured via two quantitative indices: Spearman correlationcoefficient (30) and Cohen Kappa coefficient (31). The Kappaindex is awidely usedmeasure to determine the agreement amonga set of experts making categorical judgments, considering agree-ment may occur by chance.

Computer-based estimation of TIL density. We also extracted TILdensity-based (DenTIL) features and compared the prognosticperformance of these DenTIL features against the SpaTIL features.The DenTIL features included ratio between the number of TILsand the TMA spot area, ratio between the total regions of the TMAspot covered by TILS to the total area of the corresponding TMAspot, ratio between the number of TILs and the number of non-TILswithin a TMAspot, and a grouping value indicating how closethe TILs are to each other (computed as the sum of the inversedistances between TILs).

Statistical analysisClassification. A quadratic discriminant analysis (QDA) classifierwas trained using the top SpaTIL features (QS) identified from D1

to separate the patients into two classes: recurrence and nonre-currence. We chose this classifier owing to the fact that it has nohyper parameters to tune and is able to learn quadratic bound-aries. Therefore, it has been shown to be more flexible comparedwith linear classifiers. In addition, another QDA classifier wastrained using DenTIL features (QD) on the training set D1. Fol-lowing parameter optimization, the classifier was locked downusing D1.

The performance of the locked downQDAclassifiersQS andQD

in distinguishing between early-stage NSCLC patients whodid and did not have recurrence was evaluated on the indepen-dent validation setsD2,D3, andD4.ClassifiersQS andQD assigneda probability of recurrence to each image in the test sets.Classifier performance was evaluated via the concordance statisticor C-index. The recurrence and nonrecurrence labels predictedby QS and QD were compared with the ground truth labels

(true patient outcomes) to determine classifier accuracy andC-index. The C-index obtained for QS on D3 and D4 wasquantitatively compared to evaluate the effect of spatial tissuesampling on the classifier performance.

Statistical and survival analysis. Recurrence-free survival (RFS)was defined as the time interval between the date of diagnosisand the date of death or recurrence (whichever happened first).Patients who were still alive without recurrence at the lastreported date were labeled as censored. Kaplan–Meier survivalanalysis was used to examine the difference of RFS betweendifferent patient groups categorized by the classifier output, andthe difference of RFS in each group was assessed by the log-ranktest. Multivariate Cox regression analysis was employed toexamine the predictive ability of the QS and QD classifierswhen controlling for the effects of clinical and pathologicparameters including gender, age, T stage, and N stage. P valueswere two-sided, and all values under 0.05 were considered to bestatistically significant.

A Kaplan–Meier analysis was also carried out for the DenTILapproach and for the TIL density estimation by the human read-ers. In the case of manual estimation of TIL density, patients weresplit into two groups:High-TIL and Low-TIL. Because pathologistsgraded each case from 0 to 3, case patients with TIL categories of 0to 2 were considered as being part of the low-TIL group and thosewith scores of 3 were grouped in as part of the high-TIL tumorcategory. This strategy was based on the approaches described inrefs. (4, 6).

ResultsClinicopathologic features of the patient cohorts

The median follow-up for patients was 40.91 months, 45.33months, and 70.92 months for D1, D2, and D3, respectively. Bythe end of the study/follow-up, 34 of 70 patients (48.6%) inD1, 54 of 119 (45.4%) in D2, and 34 of 112 (30.4%) in D3 haddeveloped recurrence. Table 1 presents a summary of clinicaland pathologic features for the whole dataset (D1, D2, D3, andD4). Supplementary Table S3 presents the clinical and patho-logic features of cohorts D1, D2, D3, and D4 individually.

The images are all pretreatment tissue slides. The analysis isperformed during the diagnosis phase (no treatment at all).Obviously, the treatment may influence the survival variablebecause of the inherent biological variability, i.e., each individual

Table 1. Summary of clinical and pathologic features for the patients in thewhole dataset

Variable Subvariables N (%)

Number of patients 301Average age 64.3 � 10.5Sex Male 176 (58.5)

Female 125 (41.5)N–pathologic 0 205 (68.1)

1 96 (31.9)T–pathologic 1 135 (44.9)

2 166 (55.1)Stage I/IA/IB 221 (73.4)

II/IIA/IIB 80 (26.6)Recurrence Nonrecurrence 179 (59.5)

Recurrence 122 (40.5)Tumor types Adenocarcinoma 135 (44.9)

Squamous cell carcinoma 89 (29.6)Others 77 (25.6)






response is different, but once the patient is categorized, thetreatment is the same for that particular category. The kind oftreatment is shown in Supplementary Table S5.

Experiment 1: Prognostic ability of SpaTIL in early-stageNSCLCSupplementary Table S2 and Supplementary Fig. S3 illustrate

the top SpaTIL features identified via feature selection and theircorresponding boxplots, respectively.

Figures 2C, 3C, and 4A and D illustrate the ROC curves andcorresponding C-indices for the SpaTIL classifier (Qs) for predict-ing recurrence in NSCLC on D1, D2, D3, and D4. C-indices of thebinary classifier for the 4 datasets were 0.70, 0.73, 0.70, and 0.71,respectively.

Figures 2D, 3D, and 4B and E illustrate the Kaplan–Meierplots corresponding to the SpaTIL features for D1, D2, D3, andD4, respectively. Qs was found to be prognostic for D2 (P ¼5.0 � 10�4, HR, 2.80; 95% confidence interval, 1.64–4.80), D3

(P value ¼ 1.4 � 10�3; HR, 4.45; 95% confidence interval,1.76–11.25), and D4 (P value ¼ 0.02; HR, 4.45; 95% confi-dence interval, 1.26–8.46).

Significance of clinical and pathologic variables with patients'survival time in the test sets was evaluated via the log-rank test(Supplementary Fig. S2). A multivariate analysis, controlling theeffect of major pathologic and clinical variables, was performedfor the three test cohorts (D2, D3, and D4). Table 2 presents theresults of the analysis for the three cohorts together, whereasSupplementary Table S4 shows the results for each cohort indi-vidually. Patients identified by the Qs classifier as having poorprognosis had statistically significantly worse disease-specificsurvival. The HR was 3.08 (95% confidence interval, 2.1–4.5,P ¼ 7.3 � 10�5), meaning that patients with recurrence were

approximately 3 times more likely to develop disease recurrenceand die from the disease.

Experiment 2: Comparison of human and machine-basedassessment of TIL density for predicting recurrence in early-stage NSCLC

Figures 2A and 3A show the computed Spearman correlationand Kappa index values for the two pathologists for D1 and D2,respectively. The overall computed Kappa (considering bothanalyzed datasets) was 0.50. When computed independently foreach dataset, Kappa indices were 0.38 and 0.57 for D1 and D2,respectively. On the other hand, the correlation coefficients were0.61 for D1 and 0.79 for D2.

Figures 2B and 3B illustrate the Kaplan–Meier plots forboth pathologists on D1 and D2, respectively. For reader 1,no statistically significant correlation between TIL estimationand outcome was observed for D1 (P ¼ 0.14) nor D2 (P ¼0.26). Conversely, for reader 2, a significant statistical cor-relation was observed for set D1 (P ¼ 0.01) but not for setD2 (P ¼ 0.07).

Figures 2E, 3E, and 4C and F illustrate the Kaplan–Meier plotscorresponding to the QD classifier on D1, D2, D3, and D4, respec-tively. TheQD classifier was found to have a statistically significantcorrelation between the classifier and patient outcome forD2 (P¼3.4 � 10�4); the predictions, however, were not statisticallysignificant for D3 (P ¼ 0.36) and D4 (P ¼ 0.36), respectively.

DiscussionDifferent studies reported the importance of immune response

in different cancers (32). Such studies have demonstrated a highcorrelation between TILs density and both disease outcome and

Figure 2.

Prognostic prediction results for human readers,QD, andQS forD1.A,Bar plot illustrating theKappa index and correlation coefficient computedbetween readers 1 and2. B, Kaplan–Meier curves for readers 1 and 2. C, ROC curve and corresponding confidence interval (CI) for QS. D, Kaplan–Meier plot for QS classifier using RFS asendpoint. E, Kaplan–Meier plot for QD classifier. The number of cases in each category is indicated in the charts.

Corredor et al.





treatment response, particularly for lung, breast, ovarian, pancre-atic, colorectal, and skin cancer, to name a few (1, 2, 6, 11–13, 22,32, 33). Pathologists have aimed to quantify the number ofimmune cells and their relationships within a tumor. For exam-

ple, Galon and colleagues (34) classified tumors with an Immuno-score that relates the density and location of immune cells withinthe tumor. They showed this score yields a prognostic value thatcan complement or even replace the standard tumor–node–

Figure 3.

Prognostic prediction results for human readers, QD, and QS for D2. A, Bar plot illustrating the Kappa index and correlation coefficient computed between readers1 and 2. B, Kaplan–Meier curves for readers 1 and 2. C, ROC curve and corresponding confidence interval (CI) for QS. D, Kaplan–Meier plot for QS classifierusing RFS as endpoint. Nonrecurr., nonrecurrence; Pred., predicted. E, Kaplan–Meier plot for QD classifier. The number of cases in each category is indicated inthe charts. Nonrecurr., nonrecurrence; Pred., predicted.

Figure 4.

Prognostic prediction results forQD andQS for D3 (top row) andD4 (bottom row). First column (A andD) shows theROC curve and corresponding confidence interval(CI) forQS, second column (B and E) presents the Kaplan–Meier plots forQS classifier using RFS as endpoint, and third column (C and F) illustrates the Kaplan–Meierplot for QD classifier. The number of cases in each category is indicated in the charts. Nonrecurr., nonrecurrence; Pred., predicted.






metastasis classification in colorectal cancer. The authors havesubsequently made a dedicated effort to promote the Immuno-score in routine clinical settings (see http://www.immunoscore.org/). Likewise, Salgado and colleagues (11), as part of theInternational Immuno-oncology Biomarker Working Group onBreast Cancer (see https://www.tilsinbreastcancer.org/), have per-formed the largest effort to date to establish a TIL-based quanti-fication protocol. The authors constructed a set of guides thatstandardize the methodology for visual assessment of TILs onH&Eslide sections. In lung cancer, however, there is a conspicuouslack of a standardized guideline for TIL scoring/use. Althoughdifferent works have demonstrated the prognostic value of TILs inlung cancer, in each of these studies (4, 19), the degree of TILs wasestimated by visual observation, a time-consuming, subjective,and often error-prone task. Moreover, several studies have showna limited reproducibility in visual estimation of TILs (6, 12).

Computer-based approaches for automatic estimation ofTILs (12–15) have helped to mitigate the subjectivity and lowreproducibility associated with human TIL grading. However,different recent works have shown that, besides TIL density, thespatial location of immune cells is useful in predicting patientprognosis (32). Such studies appear to suggest that the spatialorganization of lymphocytic infiltration in the context of near-by cancer cells is an important prognostic hallmark of certaintypes of tumors. This suggests that the study of the immuneresponse with respect to patient outcome should take intoaccount not only the quantity of immune cells, but also thespatial arrangement of the cancerous and surrounding immunecells (20, 21, 23, 35). Previous related studies have found astrong association between the spatial location of nuclei andsurrounding cytoplasmic features with OS (20, 21) and RFS(4, 23) in patients with early-stage NSCLC.

In this work, we presented a set of features based on the spatialarchitecture of TILs (SpaTIL), devised to capture the TIL localdensity and variation in the density, architecture, and colocaliza-tion of TIL and cancerous cells.

On two TMA datasets comprising of 119 and 112 patients, theSpaTIL classifier yielded confidence intervals of 0.73 and 0.70,respectively. A Kaplan–Meier analysis along utilizing the log-ranktest showed a strong association between the predictions of theSpaTIL classifier and recurrence for D2 (P ¼ 5.0 � 10�4), D3 (P ¼1.0 � 10�3), and D4 (P ¼ 0.01). Likewise, a multivariate Coxproportional survival analysis revealed an HR of 3.08 (95%confidence interval: 2.1–4.5, P ¼ 7.3 � 10�5).

Notably, the cohorts were obtained from different institutionswith local (and presumably variable) tissue processing and pres-ervation protocols. In addition, slides were stained in differentinstitutions and digitalized with two different instruments. Thelatter appears to reflect the robustness of the SpaTIL classifier, itsrelative resilience to image, and color variance on account ofmajor preanalytical variables and with samples from multipledifferent sites.

We also compared the prognostic performance of TIL esti-mation carried out by two human readers. A Kaplan–Meieranalysis was conducted for each pathologist; results showedthat no significant statistical correlation was found betweenPathologist 1 and prognosis for any dataset (P > 0.05), whereasthere was a significant statistical correlation between TIL gradeestimation of Pathologist 2 and patient outcome for D1 (P ¼6.0 � 10�3). In addition, the agreement among expert pathol-ogists for D1 and D2 was found to be moderate (K ¼ 0.50)and comparable with the values previously reported for NSCLC(K ¼ 0.59; ref. 6) and breast pathology (K ¼ 0.72; ref. 36).This moderate agreement might be due to the fact that TILgrading in lung pathology lacks a standardized scoring system;hence, each pathologist might preferentially focus on differentareas of the tissue during examination (e.g., epithelium orstroma) or consider different cell populations within the "TIL"infiltration (e.g., mononuclear cells beyond lymphocytes suchas plasmocytes and myeloid cells). Finally, different patholo-gists may have variable expertise evaluating immune cell infil-trates or natural individual variation in their perception ofcolors, shapes, and relative amounts/proportions. In contrast,the results obtained by using SpaTIL features are objectivelymeasured and suggest that the spatial arrangement of TILs andtumor cells was strongly associated with recurrence in early-stage NSCLC (P < 0.05).

The two papers most closely related to the work presentedhere are that of Khan and colleagues (37) and Saltz andcolleagues (22). Khan and colleagues (37) measured the degreeof TIL infiltration for each core of a TMA as the ratio oflymphocytes to all cells. The infiltration score was found tobe associated with poor prognosis in breast cancer, specificallyin HER2-amplified/positive cases (P ¼ 0.02). Saltz and collea-gues (22) used a deep learning model to identify patches fromwhole slide images (WSI) containing TILs. From these patchclusters, cluster indices were calculated, namely Ball and Hall(38), Banfield and Raftery (39), C (40), determinant ratio (41),among others. Five associations between these cluster indicesand patient survival were found to be significant for differenttumor types: Ball–Hall for breast invasive carcinoma (P ¼ 7 �10�3), C-index for lung adenocarcinoma (P ¼ 3.0 � 10�3),Banfield–Raftery for prostate adenocarcinoma (P ¼ 0.01),Determinant Ratio for prostate adenocarcinoma (P ¼ 0.01),and Banfield–Raftery for skin cutaneous melanoma (P ¼ 1.0 �10�3). Although the study by Saltz and colleagues (22) clearlysuggests the prognostic relevance of spatial arrangement ofTILs, the SpaTIL features capture not just the spatial arrange-ment of TILs, but also the spatial interplay and colocalization ofTILs and cancer nuclei. Although studies involving IHC and QIFimages (4, 18, 19) have shown the importance of looking at thespatial architecture and interplay of different TIL families, to thebest of our knowledge, the SpaTIL approach represents the firstattempt to capture the spatial architecture and arrangement ofTILs and non-TILs from routine H&E images alone.

Table 2. Multivariable survival analysis performed on the test sets (D2, D3, andD4) including SpaTIL

VariableHR(95% confidence interval) P value

GenderMale vs. female 1.18 (0.85–1.65) 0.32

T-stageT1 vs. T2 0.98 (0.68–1.40) 0.90

N-stageN0 vs. N1 0.91 (0.59–1.41) 0.68

StageStage I vs. stage II 1.01 (0.65–1.58) 0.96

Tumor subtypesADCs vs. SCC vs. others 0.92 (0.72–1.19) 0.53

SpaTILRecurrence vs. nonrecurrence 3.08 (2.10–4.51) 7.3 � 10�5

NOTE: Two-sided P < 0.05 (in bold) was considered as statistically significant.Abbreviations: ADC, adenocarcinoma; SCC; squamous cell carcinoma.

Corredor et al.




http://www.immunoscore.org/

http://www.immunoscore.org/

https://www.tilsinbreastcancer.org/


In addition, the approaches of both Khan and colleagues (37)and Saltz and colleagues (22) are actually more similar to theDenTIL classifier which we employed to compare against theSpaTIL features, one that invoked the ratio of lymphocytes to allcells in a TMA core and also included other TIL-based densityfeatures. Although the results of the DenTIL classifier were foundto be statistically significant in predicting recurrence for D2, theresults were not significant for either D3 or D4, hence showing thelack of reproducibility. The corresponding HR and confidenceinterval values for the DenTIL classifier were also found to belower comparedwith the SpaTIL features across the validation setsD2, D3, and D4.

Our work did however have a few limitations. First, unlike thestudy of Saltz and colleagues (22), our approachwas evaluated onTMAs and not using WSIs. The SpaTIL features were howeverevaluated and compared from tissue punches from different partsof the same tumor (D3 andD4). The SpaTIL featureswere found toprognostic of recurrence in both D3 and D4 (P � 0.01), and thecorresponding C-index and HRs were also found to be compa-rable (confidence interval ¼ 0.70, HR: 4.45 for D3; confidenceinterval ¼ 0.70, HR: 3.26 D4). These findings appear to be inconcordance with previous studies involving TIL-based biomar-kers that have suggested that results using TMAs are concordantwith findings fromWSIs (4, 11, 37). However, additional studiescomparing the results in TMAs versus WSI will help establish theminimum amount of tumor tissue required for optimally calcu-lating SpaTIL features that are prognostic of recurrence.

Future work will include additional analysis among largerpatient cohorts and eventually using prospectively collected sam-ples. Although our cohorts were not characterized using alterna-tive prognostic molecular signatures for NSCLC (4, 18, 19), weanticipate the SpaTIL to be complementary with such tests.Additional studies will be required to determine the relativecontributions, feasibility, and cost-effectiveness of each approach;and the possible role of such metrics to predict sensitivity/resis-tance to novel anticancer immunostimulatory therapies.

In summary, we presented a novel computational image-anal-ysis–based approach that exploits a set of density and spatialtopological features related to the arrangement of clusters of TILsand nonlymphocytes within the tumor area using standard H&Ehistology preparations. We identified a metric that was consis-tently associated with recurrence/prognosis in early-stage NSCLC.This work represents the first preliminary step in the developmentof an image-based companion diagnostic test to help identifyearly-stage lung cancer patients at increased risk for recurrence andhence potential candidates for adjuvant therapy (e.g., chemother-apy or immunotherapy) following standard-of-care surgery.

Disclosure of Potential Conflicts of InterestD.L. Rimm is a consultant/advisory board member for PAIGE. K.A. Schalper

reports receiving commercial research grants fromModerna Therapeutics, PierreFabre, Surface Oncology, Takeda, Tesaro, and Vasculox, speakers bureau hon-oraria from Merck and Takeda, and is a consultant/advisory board member forCelgene, Moderna Therapeutics, and Shattuck Labs. A. Madabhushi holdsownership interest (including patents) in and is a consultant/advisory boardmember for Inspirata Inc. Nopotential conflicts of interest were disclosed by theother authors.

DisclaimerThe content is solely the responsibility of the authors and does not neces-

sarily represent the official views of the NIH.

Authors' ContributionsConception and design: G. Corredor, X. Wang, K. Syrigos, E. Romero,V. Velcheti, A. MadabhushiDevelopment of methodology: G. Corredor, X. Wang, Y. Zhou, C. Lu,E. Romero, V. Velcheti, A. MadabhushiAcquisition of data (provided animals, acquired and managed patients,provided facilities, etc.): X. Wang, K. Syrigos, D.L. Rimm, M. Yang,K.A. Schalper, V. Velcheti, A. MadabhushiAnalysis and interpretation of data (e.g., statistical analysis, biostatistics,computational analysis):G. Corredor, X. Wang, Y. Zhou, C. Lu, P. Fu, M. Yang,E. Romero, V. Velcheti, A. MadabhushiWriting, review, and/or revision of the manuscript: G. Corredor, X. Wang,P. Fu, K. Syrigos, D.L. Rimm, E. Romero, K.A. Schalper, V. Velcheti,A. MadabhushiAdministrative, technical, or material support (i.e., reporting or organizingdata, constructing databases):X.Wang,D.L. Rimm,V. Velcheti, A.MadabhushiStudy supervision: E. Romero, A. Madabhushi

AcknowledgmentsThis study was supported by the NCI of the NIH under award

numbers 1U24CA199374-01, R01CA202752-01A1, R01CA208236-01A1,R01 CA216579-01A1, and R01 CA220581-01A1; the National Center forResearch Resources under award number 1 C06 RR12463-01; the Departmentof Defense (DOD) Prostate Cancer Idea Development Award; the DOD LungCancer Idea Development Award; the DOD Peer Reviewed Cancer ResearchProgram (W81XWH-16-1-0329); the Ohio Third Frontier Technology Valida-tion Fund; the Wallace H. Coulter Foundation Program in the Department ofBiomedical Engineering; and the Clinical and Translational Science AwardProgram (CTSA) at Case Western Reserve University.

The costs of publication of this article were defrayed in part by thepayment of page charges. This article must therefore be hereby markedadvertisement in accordance with 18 U.S.C. Section 1734 solely to indicatethis fact.

Received June 26, 2018; revisedAugust 16, 2018; accepted September 6, 2018;published first September 10, 2018.

References1. Geng Y, Shao Y, HeW, HuW, Xu Y, Chen J, et al. Prognostic role of tumor-

infiltrating lymphocytes in lung cancer: a meta-analysis. Cell PhysiolBiochem 2015;37:1560–71.

2. Bremnes RM, Donnem T, Busund LT. Importance of tumor infil-trating lymphocytes in non-small cell lung cancer? Ann Transl Med2016;4:142.

3. Uramoto H, Tanaka F. Recurrence after surgery in patients with NSCLC.Transl Lung Cancer Res 2014;3:242–9.

4. Schalper KA, Brown J, Carvajal-Hausdorf D, McLaughlin J, Velcheti V,Syrigos KN, et al. Objective measurement and clinical significanceof TILs in non–small cell lung cancer. J Natl Cancer Inst 2015;107.pii: dju435.

5. Donnem T, Hald SM, Paulsen EE, Richardsen E, Al-Saad S, KilvaerTK, et al. Stromal CD8þ T-cell density - A promising supplementto TNM staging in non-small cell lung cancer. Clin Cancer Res2015;21:2635–43.

6. Brambilla E, Teuff GL, Marguet S, Lantuejoul S, Dunant A,Graziano S, et al. Prognostic effect of tumor lymphocytic infiltra-tion in resectable non–small-cell lung cancer. J Clin Oncol 2016;34:1223–30.

7. Brown JR, Wimberly H, Lannin DR, Nixon C, Rimm DL, Bossuyt V.Multiplexed quantitative analysis of CD3, CD8, and CD20 predictsresponse to neoadjuvant chemotherapy in breast cancer. Clin Cancer Res2014;20:5995–6005.






8. Wang K, Xu J, Zhang T, Xue D. Tumor-infiltrating lymphocytes in breastcancer predict the response to chemotherapy and survival outcome: ameta-analysis. Oncotarget 2016;7:44288–98.

9. Kochi M, Iwamoto T, Niikura N, Bianchini G, Masuda S, Mizoo T, et al.Tumour-infiltrating lymphocytes (TILs)-related genomic signature predictschemotherapy response in breast cancer. Breast Cancer Res Treat 2017;167:39–47.

10. Kashiwagi S, Asano Y, Goto W, Takada K, Takahashi K, Noda S, et al. Useof tumor-infiltrating lymphocytes (TILs) to predict the treatmentresponse to eribulin chemotherapy in breast cancer. PLoS One 2017;12:e0170634.

11. Salgado R, Denkert C, Demaria S, Sirtaine N, Klauschen F, Pruneri G, et al.The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer:recommendations by an International TILs Working Group 2014. AnnOncol 2015;26:259–71.

12. AllardMA, Bachet JB, Beauchet A, Julie C,Malafosse R, PennaC, et al. Linearquantification of lymphoid infiltration of the tumor margin: a reproduc-iblemethod, developedwith colorectal cancer tissues, for assessing ahighlyvariable prognostic factor. Diagn Pathol 2012;7:156.

13. RobinsHS, EricsonNG,Guenthoer J, O'Briant KC, TewariM,Drescher CW,et al. Digital quantification of tumor infiltrating lymphocytes. Sci TranslMed 2013;5:214ra169.

14. Eriksen AC, Andersen JB, KristenssonM, dePont Christensen R, Hansen TF,Kjær-Frifeldt S, et al. Computer-assisted stereology and automated imageanalysis for quantification of tumor infiltrating lymphocytes in coloncancer. Diagn Pathol 2017;12:65.

15. Basavanhally AN, Ganesan S, Agner S, Monaco JP, Feldman MD, Tomas-zewski JE, et al. Computerized image-based detection and grading oflymphocytic infiltration in HER2þ breast cancer histopathology. IEEETrans Biomed Eng 2010;57:642–53.

16. Fatakdawala H, Xu J, Basavanhally A, Bhanot G, Ganesan S, Feldman M,et al. Expectation-maximization-driven geodesic active contour withoverlap resolution (EMaGACOR): application to lymphocyte segmen-tation on breast cancer histopathology. IEEE Trans Biomed Eng 2010;57:1676–89.

17. Janowczyk A, Madabhushi A. Deep learning for digital pathology imageanalysis: a comprehensive tutorial with selected use cases. J Pathol Inform2016;7:29.

18. Barua S, Fang P, Sharma A, Fujimoto J, Wistuba I, Rao AUK, et al. Spatialinteraction of tumor cells and regulatory T cells correlates with survival innon-small cell lung cancer. Lung Cancer 2018;117:73–9.

19. Liu H, Zhang T, Ye J, Li H, Huang J, Li X, et al. Tumor-infiltratinglymphocytes predict response to chemotherapy in patients withadvance non-small cell lung cancer. Cancer Immunol Immunother2012;61:1849–56.

20. YuKH, ZhangC, BerryGJ, AltmanRB, R�eC, RubinDL, et al. Predicting non-small cell lung cancer prognosis by fully automatedmicroscopic pathologyimage features. Nat Commun 2016;7:12474.

21. Luo X, Zang X, Yang L, Huang J, Liang F, Rodriguez-Canales J, et al.Comprehensive computational pathological image analysis predicts lungcancer prognosis. J Thorac Oncol 2017;12:501–9.

22. Saltz J, Gupta R, Hou L, Kurc T, Singh P, Nguyen V, et al. Spatialorganization and molecular correlation of tumor-infiltrating lympho-cytes using deep learning on pathology images. Cell Rep 2018;23:181–93.e7.

23. WangX, JanowczykA, ZhouY, Thawani R, Fu P, Schalper K, et al. Predictionof recurrence in early stage non-small cell lung cancer using computerextracted nuclear features from digital H&E images. Sci Rep 2017;7:13543.

24. Camp RL, Chung GG, Rimm DL. Automated subcellular localizationand quantification of protein expression in tissue microarrays. Nat Med2002;8:1323–7.

25. Veta M, van Diest PJ, Kornegoor R, Huisman A, Viergever MA, Pluim JPW.Automatic nuclei segmentation in H&E stained breast cancer histopathol-ogy images. PLoS One 2013;8:e70221.

26. Corredor G, Wang X, Lu C, Velcheti V, Romero E, Madabhushi A.A watershed and feature-based approach for automated detection oflymphocytes on lung cancer images. In: Angelini ED, LandmanBA.Medicalimaging 2018: image processing. Proceedings of SPIE Medical Imaging;2018 Feb 10–15;Houston, TX. Bellingham (WA): Society of Photo-OpticalInstrumentation Engineers (SPIE); 2018.

27. Ali S, Lewis J, Madabhushi A. Spatially aware cell cluster(spACC1) graphs:predicting outcome in oropharyngeal pl6þ tumors. In: Mori K, Sakuma I,Sato Y, Barillot C, Navab N, editors. Medical image computing andcomputer assisted intervention — MICCAI 2013. Proceedings of the 21stInternational Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI); 2013 Sep 22–26; Nagoya, Japan. Berlin(Germany): Springer. p. 412–9.

28. Ali S, Veltri R, Epstein JA, Christudass C, Madabhushi A. Cell clustergraph for prediction of biochemical recurrence in prostate cancerpatients from tissue microarrays. In: Gurcan MN, Madabhushi A,editors. Medical imaging 2013: digital pathology. Proceedings ofSPIE Medical Imaging; 2013 Feb 9–14; Lake Buena Vista, FL. Bellingham(WA): Society of Photo-Optical Instrumentation Engineers (SPIE);2013. p. 86760H.

29. Peng H, Long F, Ding C. Feature selection based on mutual informationcriteria of max-dependency, max-relevance, and min-redundancy. IEEETrans Pattern Anal Mach Intell 2005;27:1226–38.

30. Myers L, Sirois MJ. Spearman correlation coefficients, differences between.Encyclopedia of statistical sciences. Hoboken (NJ): Wiley; 2006.

31. Carletta J. Assessing agreement on classification tasks: the Kappa statistic.Comput Linguist 1996;22:249–54.

32. Yuan Y. Modelling the spatial heterogeneity and molecular correlates oflymphocytic infiltration in triple-negative breast cancer. J R Soc Interface2015;12.

33. Savas P, Salgado R, Denkert C, Sotiriou C, Darcy PK, Smyth MJ, et al.Clinical relevance of host immunity in breast cancer: fromTILs to the clinic.Nat Rev Clin Oncol 2016;13:228–41.

34. Galon J, Costes A, Sanchez-Cabo F, Kirilovsky A, Mlecnik B, Lagorce-Pag�esC, et al. Type, density, and location of immune cells within humancolorectal tumors predict clinical outcome. Science 2006;313:1960–4.

35. Nawaz S, Yuan Y. Computational pathology: exploring the spatial dimen-sion of tumor ecology. Cancer Lett 2016;380:296–303.

36. Buisseret L, Desmedt C, Garaud S, Fornili M, Wang X, Van den Eyden G,et al. Reliability of tumor-infiltrating lymphocyte and tertiary lymphoidstructure assessment in human breast cancer. Mod Pathol 2017;30:1204.

37. Khan AM, Yuan Y. Biopsy variability of lymphocytic infiltration in breastcancer subtypes and the ImmunoSkew score. Sci Reps 2016;6:36231.

38. Ball GH, Hall DJ. Isodata, a novel method of data analysis and patternclassification. Menlo Park (CA): Stanford Research Institute; 1965.

39. Banfield JD, Raftery AE. Model-based Gaussian and non-Gaussian clus-tering. Biometrics 1993;49:803–21.

40. Hubert L, Schultz J. Quadratic assignment as a general data analysisstrategy. Br J Math Stat Psychol 1976;29:190–241.

41. Scott AJ, SymonsMJ. Clusteringmethods based on likelihood ratio criteria.Biometrics 1971;27:387–97.


Corredor et al.




2019;25:1526-1534. Published OnlineFirst September 10, 2018.Clin Cancer Res Germán Corredor, Xiangxue Wang, Yu Zhou, et al.

Small Cell Lung Cancer−Early-Stage Non Lymphocytes for Predicting Likelihood of Recurrence in

Spatial Architecture and Arrangement of Tumor-Infiltrating

Updated version

10.1158/1078-0432.CCR-18-2013doi:

Access the most recent version of this article at:

Material

Supplementary

http://clincancerres.aacrjournals.org/content/suppl/2018/09/08/1078-0432.CCR-18-2013.DC1

Access the most recent supplemental material at:

Cited articles

http://clincancerres.aacrjournals.org/content/25/5/1526.full#ref-list-1

This article cites 35 articles, 5 of which you can access for free at:

Citing articles

http://clincancerres.aacrjournals.org/content/25/5/1526.full#related-urls

This article has been cited by 5 HighWire-hosted articles. Access the articles at:

E-mail alerts related to this article or journal.Sign up to receive free email-alerts

Subscriptions

Reprints and

[email protected]

To order reprints of this article or to subscribe to the journal, contact the AACR Publications Department at

Permissions

Rightslink site. Click on "Request Permissions" which will take you to the Copyright Clearance Center's (CCC)

.http://clincancerres.aacrjournals.org/content/25/5/1526To request permission to re-use all or part of this article, use this link



http://clincancerres.aacrjournals.org/lookup/doi/10.1158/1078-0432.CCR-18-2013

http://clincancerres.aacrjournals.org/content/suppl/2018/09/08/1078-0432.CCR-18-2013.DC1

http://clincancerres.aacrjournals.org/content/25/5/1526.full#ref-list-1

http://clincancerres.aacrjournals.org/content/25/5/1526.full#related-urls

http://clincancerres.aacrjournals.org/cgi/alerts

mailto:[email protected]

http://clincancerres.aacrjournals.org/content/25/5/1526


spatial architecture and arrangement of tumor-inﬁltrating ...her2þ breast cancer h&e images...

Documents