Multiplexed immunohistochemical quantitative molecular phenotyping viaMultiplexed immunohistochemical quantitative molecular phenotyping viaMultiplexed immunohistochemical quantitative molecular phenotyping via multispectral imaging and automated segmentationmultispectral imaging and automated segmentationmultispectral imaging and automated segmentation

C Hoyt1 K Gossage1 T Hope1 R Levenson1 R Bandaru 2 H Gardner 2Seeing life in a new light Seeing life in a new lightC Hoyt1, K Gossage1, T Hope1, R Levenson1, R Bandaru 2 , H Gardner 2.Seeing life in a new light. Seeing life in a new light.y g p1 Cambridge Research and Instrumentation Inc Woburn Massachusetts USA 2 Oncology Translational Laboratories Novartis Institutes for Biomedical Research Cambridge Massachusetts USACambridge Research and Instrumentation, Inc, Woburn, Massachusetts, USA, Oncology Translational Laboratories, Novartis Institutes for Biomedical Research, Cambridge, Massachusetts, USA.

Background. A l i i hi t h i t (IHC) th d t hi h Semi-Automated Assessment of Histochemical Stains Quantitative Molecular Phenotyping using Multispectral ImagingApplying immunohistochemistry (IHC) methods to high-

l li i l d li i l l l ti b d t i l iSemi-Automated Assessment of Histochemical Stains Quantitative Molecular Phenotyping using Multispectral Imaging

volume preclinical and clinical molecular tissue-based trials is a challenge Traditional visual assessments area challenge. Traditional visual assessments are slow subjective and semi quantitative at best Concordance Multiplexed IHC: lung carcinoma tissue microarray (TMA)Image Processing Steps S t l I i I t t tislow, subjective and semi-quantitative at best. Concordance (r) among pathologists ranges between 0 6 and

Multiplexed IHC: lung carcinoma tissue microarray (TMA)double-stained for pAKT and pMEK

Image Processing Steps Spectral Imaging Instrumentation(r) among pathologists ranges between 0.6 and 0 95 depending on the antigen and typically no more than

double stained for pAKT and pMEKStep 1: Find Cancer Areas at Low Power (2x) Step 2: Extract Several 20x Fields Step 3: Classify Cancer and 0.95, depending on the antigen, and typically, no more than

100 slides per day per pathologist can be analyzed ReliableDiscussion

p ( ) p p ySegment Nuclei, NuanceTM Multispectral 100 slides per day per pathologist can be analyzed. Reliable

assessment of more than one molecular marker per cellular• Detection of phospho-epitopes in fixed

tissue requires high sensitivity

pImaging Systemsassessment of more than one molecular marker per cellular

compartment (nucleus, cytoplasm, membrane) is essentiallytissue requires high sensitivity

• Multispectral imaging elimination ofcompartment (nucleus, cytoplasm, membrane) is essentially impossible by visual means alone.

• Multispectral imaging, elimination of autofluorescence, and the use of Qdots p y ,enables multiplexed detection of low-b d tiObjective. Babundance antigensj

To develop computer-based automated assessment tools that AAimprove accuracy, precision and speed of evaluation of AA B. Spectral

Libraryclinical markers in tissue sections, combining flexible

RGB Image (20x) of 1 of 40 TMA CoresLibrary

machine-learning segmentation, subcellular compartment RGB Image (20x) of 1 of 40 TMA Cores

discrimination and multiplexing in a form readily incorporated i t l b t i f ti t

Unmixed component imagesThumbnail 2X Tile Classified (red = cancer Extracted 20x Fields 20x Fields Cancer Mask and Nucleiinto laboratory information systems. Thumbnail 2X Tile Classified (red = cancer,

green = other)Extracted 20x Fields 20x Fields Cancer Mask and Nuclei

Outlined

Methods and Materials

g )

m

Methods and Materials. 1) Machine learning and nuclear segmentation tools were Step 4: Unmix DAB and Step 5: Convert DAB Signal to Step 6: Review / Edit Segmentations, Spectral Data Acquisition 5 Quantum

m m

1) Machine-learning and nuclear segmentation tools were tested and validated using images from 500 patient cases

Step 4: Unmix DAB and Hematoxylin Signals

Step 5: Convert DAB Signal to Optical Density

Step 6: Review / Edit Segmentations, Summarize results per case Benefits of This Approach: Spectral Data Acquisition

T k i t diff tDot vials

om mtested and validated using images from 500 patient cases. For each case four serial sections were stained CC DD EE

y g p y

• Simple ‘learn by example’• Take images at different

wavelengths with a CCD-equipped

co om

For each case, four serial sections were stained immunohistochemically for ki67 progesterone receptor H h t

CC DDMEK (585 Qd t) AKT (655 Qd t)

EE• Simple learn-by-example interface

wavelengths with a CCD equipped Nuance. λc oimmunohistochemically for ki67, progesterone receptor

(PR) cyclinD1 or AIB1 and imaged on an Aperio slideHoechst pMEK (585 Qdot) pAKT (655 Qdot)

C it I

interface• Fast algorithm training

• Assemble the data into a “cube” in memory

λ

c.c c(PR), cyclinD1, or AIB1, and imaged on an Aperio slide

scanning instrument. Machine learning-based segmentationComposite Image• Fast algorithm training

(algorithms trained in minutes) 650 nm

memory.• This creates a spectrum at every c c.

cscanning instrument. Machine learning based segmentation algorithms were applied, first at reduced resolution (at A: RGB representation of a

spectral dataset acquired

(algorithms trained in minutes)• Fast image processing (individual

650 nmp y(x,y) pixel of the image.

n cg pp , (approximately 2X) to locate fields enriched for tumor. These

spectral dataset acquired from a formalin-

• Fast image processing (individual images in < 10 seconds, whole 625 nm

-in npp y )automatically selected fields were then re-segmented at full fixed, paraffin-embedded

lung carcinoma tissue

images in 10 seconds, whole Aperio .svs files in < 3 mins) 600 nmi- iny g

resolution (approximate 20X) to identify tumor cells and lung carcinoma tissue micro-array core, double-• Sample assessment rate of 200 to

600 nm

xr i-perform cell-compartment-based quantitation of stained for pMEK pAKT with Hoechst

p300 samples per hour 575 nm

x

c riimmunohistochemical staining levels. Concordance pMEK, pAKT, with Hoechst counterstain. B: spectral • ‘Light Weight’ - runs on standard 550 y

w.c crcorrelation coefficients were calculated using the Lin method library used to unmix the

image cube into (left to

g glaptops and desktops, versus

550 nm y

w w.c(1989).

(2) S t l ti i f 40 l

image cube into (left to right): Hoechst (C); pMEK 585 Qd t (D) d AKT

expensive servers

ww w(2) Separately, a tissue microarray of 40 lung cancer cores

d bl t i d ith Qd t t ti AKT d MEK585-nm Qdot (D); and pAKT 655-nm Qdot (E). F:

Demonstration of Spectralw wwwas double-stained with Qdots targeting pAKT and pMEK

expression in cancer cell cytoplasm and imaged with a CRi

655 nm Qdot (E). F: composite image of the

t l i lFFDemonstration of Spectral Unmixing of Overlapping w

w wexpression in cancer cell cytoplasm, and imaged with a CRi Nuance multispectral camera Multispectral imaging detected

spectral signals. FFValidation Results: Concordance between Manual and Semi-Automated Assessments Chromogensw w

wNuance multispectral camera. Multispectral imaging detected and isolated Qdot signals from prominent background

Validation Results: Concordance between Manual and Semi-Automated AssessmentsTMA Core Scatter Plots – pMEK vs. pAKT in cytoplasm of cancer cells wand isolated Qdot signals from prominent background

signals and then was used to assess relative signalTMA Core Scatter Plots pMEK vs. pAKT in cytoplasm of cancer cells

signals, and then was used to assess relative signal strengths

Core #1 Core #2 Core #3 Core #4strengths.

Results and Discussion. DiscussionResults and Discussion. (1) The automated image analysis tools required one to two ( ) g y qhours of system training for each IHC staining method. Summary Table

• Performance similar to manual assessments (trainedy g g

Concordance (r) between pathologists and computer-based Summary Table assessments (trained

pathologists agree typically with( ) p g passessments indicate equivalency (r values ranging from 0.68 Marker Concordance

pathologists agree typically with r-values of 0.65 to 0.90)

Included in the plot of Core #1 are data from control slides for each antigen (yellow and red) Across these four cores distinct relative expression levels are detected

to 0.84), but throughput was greatly enhanced. While 12 Ki67 0.82)

• Automated assessments of someand red). Across these four cores, distinct relative expression levels are detected.

samples per hour on average could be analyzed Ki67 0.82 Automated assessments of some

sections (8-10%) were rejected

C l imanually, results of the automated analysis system could be PR 0.84

( ) jupon review due to imperfect

Conclusionsreviewed, edited and accepted, at a rate of 200 to 300 l h O ll it t k b t 20 d t ll

CyclinD 0.68 tumor segmentation at either low hi h • Semi-automated machine-learning-based segmentation, chromogen signal unmixing, and facilitatedsamples per hour. Overall, it took about 20 days to manually

th 2000 ti ti i hil th

y1 or high power Semi automated machine learning based segmentation, chromogen signal unmixing, and facilitated

review can augment manual assessment rates 20-fold Throughput rises from 12 cases per hour (manual)assess the 2000 tissue section images, while the same task with automated assistance was completed in less than AIB1 0.75 • This is a new capability. We

t i ifi t h t review can augment manual assessment rates 20 fold. Throughput rises from 12 cases per hour (manual) to 250 per hour (semi automated)

task, with automated assistance, was completed in less than 2 days Automated segmentations and classifications were

expect significant enhancements with further training that includes to 250 per hour (semi-automated). 2 days. Automated segmentations and classifications were

rejected at a rate 8 to 10% due mainly to failures to findwith further training that includes additional tissue examples, and

• Machine-learning techniques are capable of producing automated tissue classification and segmentationrejected at a rate 8 to 10%, due mainly to failures to find tumor at low power or incorrect classification at high power

additional tissue examples, and with further algorithm Machine learning techniques are capable of producing automated tissue classification and segmentation

algorithms that agree with manual assessment and are reliable across variability of human tissue samplestumor at low power or incorrect classification at high power.(2) Multispectral imaging effectively enables the separation of

gdevelopment. algorithms that agree with manual assessment and are reliable across variability of human tissue samples

and tissue sample preparation (r values of 0 68 0 84)(2) Multispectral imaging effectively enables the separation of weak signals of low abundance phospho-epitopes from and tissue sample preparation (r-values of 0.68 – 0.84)weak signals of low abundance phospho epitopes from stronger autofluorescence and background signals, to allow

• Spectral imaging can be used to perform multiplexed automated assessments of weakly expressedstronger autofluorescence and background signals, to allow quantitation of multiple signaling pathway proteins. This is not Spectral imaging can be used to perform multiplexed automated assessments of weakly expressed

signaling pathway proteins with signals as low as 10% of tissue autofluorescence levelsq p g g p y ppossible with conventional fluorescence microscopy or visual signaling pathway proteins with signals as low as 10% of tissue autofluorescence levels. p pyassessment alone.