2013 Sixth International Conference on Advanced Computational Intelligence

October 19-21,2013, Hangzhou, China

Biologically Inspired Classification of Microvessel Histopathology via Sparse Coding

Quan Wen, Juan Chen, and Wenhao Liu

Abstract-Recently the sparse coding approaches have been

successfully applied to solve the image classification problems.

However, the classification of microvessel regions from

histopathology still rely on the hand designed low-level features.

In this paper, we propose a novel method to classify region of

microvessel by applying sparse coding on biological signals. The

Single- and Double-Opponent signals from human visual cortex

are simulated to capture microvessel properties. The SIFT

(Scale Invariant Feature Transform» descriptors of these

signals are encoded via sparse coding and classified by SVM

(Support Vector Machine) with the linear spatial pyramid

matching kernel. We have carried out extensive experiments on

the classification of microvessel histopathology and the

proposed method achieves satisfactory classification rates.

I. INTRODUCTION

Tumor angiogenesis, also neovascularization, is the generation of new blood vessels triggered by cancerous growths to reach nutrients and remove wastes. The

density of newly formed vessels, i.e. microvessel density [1], is found to be correlated with tumor malignancy [2] and invasion depth of cancer [3]. As an indicator for metastasis and prognosis of cancer, microvessel density plays an important role in histopathological examination. In essence, the microvessel density measurement depends on the classification of microvessel regions from histopathological tissues. In modern cancer research, the tumor tissue on a glass slide is scanned by an automated light microscope, which generates a virtual slide at a resolution of 0.25 /lm/pixel. Due to the diversity of the microvessel histopathology, the attempts for microvessel classification by computer vision technologies are still in the primitive stage. Unfortunately, most histopathologists have to assess microvessel regions manually from mass virtual slides.

The blood vessels are composed of two interacting cell types: 1) endothelial cells forming the inner lining of the vessel wall, and 2) pericytes enveloping the surface of the vascular tube. A sample image of microvessel in cross-section is shown in Fig. 1, with endothelial cells stained in brown and pericytes stained in red. In the pilot study by Mete et al. [4], a framework on low-level features is introduced for region-of-microvessel identification in liver

This work was supported in part by National Natural Science Foundation of China (No. 61073120) and the Fundamental Research Funds for the Central Universities (No. ZYGX2010J067).

Q. Wen is with University of Electronic Science and Technology of China, Chengdu 611731 China (corresponding author; e-mail: quanwen@uestc.edu.cn).

J. Chen is with University of Electronic Science and Technology of China, Chengdu 611731 China (e-mail: chenjuan@uestc.edu.cn).

W. Liu is with Hangzhou Normal University, Zhejiang China (e-mail: whl819 _819@163.com).

978-1-4673-6343-3/13/$3l.00 ©2013 IEEE 114

carcinoma histopathology. Their research depends on the stain color identity of endothelial cells and pericytes. The statistical and fractal features are extracted from cell regions

for microvessel classification. However, the stain color labeling by human observers is time-consuming and labor intensive in the training stage. Researchers from Aperio Technologies Inc. [5] have proposed a microvessel analysis algorithm based on a serial of thresholds, such as light staining and dark staining parameters. But these thresholds need to be re-configured when applied to other microvessel histopathology.

Endothelial cells

Pericytes

Fig. I. Sample image of microvessel in cross-section.

This paper proposed a novel method applying biological signals to capture microvessel properties, i.e. color and texture. It replaces the stain color labeling or thresholding, which is time-consuming and labor intensive. In the proposed method, a spare coding paradigm combined with the linear classification is utilized to classify microvessel regions on biological signals. The salient microvessel descriptors are automatically selected by Spatial Pyramid Matching topped on sparse coding, instead of hand designed low-level features in existing methods.

The rest of the paper is organized as follows. Section II presents the modeling of Single-Opponent (SO) and Double-Opponent (DO) neurons. We give the sparse coding on SO and DO channels in Section III. In Section IV we introduce the linear classification of microvessel histopathology, followed by experiment results in Section V. Finally, Section VI concludes our paper.

II. MODELING OF SINGLE- AND DOUBLE-OPPONENT

NEURONS

Single-Opponent (SO) and Double-Opponent (DO) [6]

neurons are color-sensitive in human cortex, playing an important role in color perception. However, SO neurons and DO neurons respond to different color vision. On one hand, SO neurons are responsive to color patches, areas and interiors. On the other hand, DO neurons are responsive to color patterns, textures and boundaries. Specifically, SO neurons receive signals of opponent wavelength from cone photoreceptors, falling into two main categories: 1) long wavelength cone (L) opposed by middle wavelength cone (M), i.e. L-M or M-L, also called red/green opponents; 2) short wavelength cone (S) opposed by the summation signal (L+M) , i.e. S-(L+M) or (L+M)-S, also called blue/yellow opponents. Accordingly, DO neurons receive opponent signals at two levels: signal composed of opposite-sign elements from different cones (i.e., Cone Opponency) and pairs of opposite-sign signal from different locations in the cell's receptive field (i.e. Spatial Opponency). For example, in the receptive field of a DO cell, it receives +L-M at one place while receiving -L+M at another place. In summary, the receptive field of color-responsive neurons both SO and DO is modeled in a spatio-chromatic sensitivity function [7] as following:

f(x, y, >..) = W RR(>")fR (x, y) + waG(>" )fa (x, y) + W EB(>")fE(x, y),

(1)

where (x, y) is the horizontal and vertical coordinates, >.. is

the wavelength, R(>") , G (>") and B(>") correspond to the

spectral response functions with wavelength >... In simple

practice, we use the R, G and B components from color images to represent spectral response functions in place of LMS (Long/Middle/Short wavelength cone) color space.

fR (x, y) , fa (x, y) and fE (x, y) correspond to the spatial

sensitivity distributions for each individual color component. Spatial sensitivity distributions can be modeled in excitatory/inhibitory center or surround structures, with positive/negative subunits isolated using linear oriented filters. In this paper, we choose Gabor filters to represent spatial sensitivity distributions for computation efficiency.

Furthermore, the opponent signals are generated by combining the R, G and B components with a weight matrix [7] as following:

+1/13 +1/13 . (2)

+1/13

The "+" sign indicates an excitatory opponent centering, while the "-" sign indicates an inhibitory opponent surrounding. Specifically, four pairs of opponent signals are obtained by applying both positive and negative matrix

[w, -w] on RGB components. The combination of channel

Green and Blue makes a new channel Cyan. In addition, the combination of channel Red and Green makes another new channel Yellow. Therefore, the eight channels of opponent

115

signals are composed of: 1) Red-Green channel (R+-G-) and (d-K); 2) Red-Cyan channel (R+-C) and (C-K); 3) Yellow-Blue channel (Y-B") and (B+-Y); 4) White-Black channel Wh and BI. Specifically, the receptive field model of SO is a 3D convolution of weight matrix, RGB components and Gabor filters:

SO(x, y) = [w, -w ] (g) [R, G, B] (g) Gabor(x, y), (3)

where (g) is the convolution operator.

Similarly, the 2D convolution of Gabor filters and SO channels yields the receptive filed model of DO:

DO(x,y) = SO(x,y) (g) Gabor(x,y). (4)

The eight SO channels of the sample microvessel from Fig. 1 are illustrated in Fig. 2. Accordingly, the eight DO channels of the same microvessel are illustrated in Fig. 3. It can be seen that the (R+-C) channel is the best segmentation of microvessel color and texture, from SO and DO respectively.

(h) Bl

Fig. 2. SO channels of sample microvessel.

(h) Bl

Fig. 3. DO channels of sample microvessel.

III. SPARSE CODING ON SO AND DO CHANNELS

Let X be a set of SIFT descriptors extracted from SO and DO channels with D feature attributes, i.e.

X=[Xb " " XM ]T E�MxD. Given codebookY=[vb ... ,vJ( ]T

being the K cluster centers and codes U = [Ub ... , U M ]T

being the cluster membership, encoding X by sparse coding is to optimize the following problem [8]:

mm u.v

M 2 L Ilxm - Urn yll + A Iuml, m=l

S.t. Ilvkll:S;l, \:fk= 1,2, ... ,K.

(5)

where 11.11 denotes the L2-norm of vectors and I I denotes the

L I-norm of vectors. A is a constant. Compared with the

traditional Vector Quantization, sparse coding has the advantage of lower reconstruction errors. Because the

L1-norm regulation in sparse coding allows coefficient Urn to have a small number of nonzero elements, other than one single nonzero element in Vector Quantization. Therefore, the salient prosperities of SO and DO channels are well captured in sparse signals.

Since the optimization of Eq. (5) is convex in Y while

116

fixing U and vice versa, an iterative strategy is adopted to optimize U and V alternatively. With Y fixed, the optimization of U can be solved by optimizing over each

coefficient Urn individually. If the signs of Urn at the optimal

value are known, the nonzero coefficients (urn> 0 or Urn < 0)

reduce Eq. (5) to a standard unconstrained quadratic optimization problem:

mm U

(6)

where 8i E {-1,1}. We use the feature-sign search algorithm

[9] to maintain an active set of potentially nonzero

coefficients and their corresponding signs 8i . In each

iteration step, given a current guess for the active set and the signs, an analytical solution IS computed, l.e.

U : = (yTYrl(yTX - A8 / 2). Then, a discrete line search

is implemented to update the current solution with respect to

the analytical solution U . Finally, the overall algorithm converges to the optimal solution.

Fixing U , the optimization of Y is reduced to a least square problem with quadratic constraints:

mJn IIX - UY II� ,

S.t. Ilvkll:s; 1, \:fk = 1,2, ... K. (7)

Applying the Lagrange dual [9], the optimal Y is

obtained, i.e. yT = (UUT + A)-l(XUT)T, A = diag(;y) with each "Yj ;:::: 0 as the dual variable. Therefore, both U and

Y are optimized by iteratively solving two convex optimization problems: an Ll-regularized least squares problem and an L2-constrained least squares problem.

IV. LINEAR CLASSIFICATION OF MICROVESSEL

HISTOPATHOLOGY

Similar to the Spatial Pyramid Matching (SPM) architecture proposed by Lazebnik et al. [10], local histograms are generated by max pooling on codes element-wise in each image sub-region, over different spatial scales. The final representation of microvessel image is composed of a concatenation of local histograms. The resulted linear SPM kernel [8] is more salient and robust to local translations as following:

where Zi is the final representation of the ilh microvessel

image and z;(s, t) is the local histogram generated by max

pooling on codes in the (s, t)th sub-region with scale level t.

In addition, (Zi' Z j ) = Zr Z j . Hence, a binary SVM with

linear SPM kernel is implemented to classify the microvessel histopathology.

V. EXPERIMENTS AND RESULTS

In the experiments, we implemented and evaluated three classes of image signals on the dataset of microvessel histopathology [4]. The dataset consists of 248 positive and 274 negative subimages from two virtual slides of live cancer histopathology. The size of the subimages is fixed to 300 x

300 pixels, according to the pathologists' best selection. Some examples of both positive and negative subimages from microvessel histopathology are demonstrated in Fig. 4. It can be seen that different sections of microvessel have various color distribution and shape patterns.

(a) Positive subimages

(b) Negative subimages

Fig. 4. Examples of sub images from microvessel histopathology.

Three kinds of image signals are tested on proposed sparse coding and linear classification paradigm, including: 1) SO signal on (R+-C) channel; 2) DO signal on (R+-C) channel; and 3) RGB signal. The SIFT descriptors are extracted from

16 X 16 pixel patches for different image signals on a grid

with step size of 6 pixels. We fixed the codebook size as 1024 for different image signals and tried our algorithm on 30 training images per class. Following the common benchmarking procedures, we repeat the experimental process by 5 times with different random selected training

117

and testing images to obtain reliable results. The averaged recall, precision and F1-measure in 5 times were recorded for different image signals. The detailed comparison results on image signals are shown in Table I. The SO signal has the best classification performance, compared with the DO signal and RGB signal. Its fl-measure is over 9% superior to RGB signal on microvessel subimages.

Besides our own implementations, we also quote some

results directly from the literature for benchmarking. The

classification rates of the method proposed by Mete [4] and

Aperio [5] are averaged and included in TABLE I. With the

arguable classification performance, our method has the

advantages of efficient implementation and flexible

application to other histopathological data. Because our

method utilize the sparse coding paradigm on biological

signals to replace the bottleneck procedures in the existing

methods, including: 1) the time-consuming and labor

intensive stain color labeling or thresholding; and 2) hand

designed low-level features.

TABLE T CLASSIFICATION PERFORMANCE COMPARISON ON THREE IMAGE SIGNALS

Image signals Recall Precision F I-measure

SO signal on (R+-C) channel 0.8829 0.8094 0.8440

DO signal on (R+-C) channel 0.8710 0.8025 0.8352

RGB signal 0.7253 0.7716 0.7475

Mete [4] 0.9069 0.8717 0.8819

Aperio [5] 0.9239 0.8475 0.8837

VI. CONCLUSION

In this paper we proposed a sparse coding approach based on Single- and Double-Opponent signals for classification of microvessel histopathology. Instead of extracting the hand designed features from the virtual slide, we simulate the biological signals from human visual cortex to specify the microvessel properties, such as color and texture. The SO and DO signals are applied to the sparse coding paradigm with linear classification for region-of-microvessel identification, which avoids the time-consuming and labor intensive stain color labeling or thresholding. Our experiments on the classification of a variety of microvessel subimages demonstrated the effectiveness of this approach.

ACKNOWLEDGMENT

We would like to thank Dr. Mutlu Mete for sharing the dataset on microvessel histopathology with us.

REFERENCES

[ I] P.S. Nia, F. Stessels, F. Pezzella, P.B. Vermeulen, EA Van Marek and P. Van Schil, "Growth index is independent of microvessel density in non-small cell lung carcinomas," Human Pathology, vol. 34, no. 9, pp. 959-960, Sep. 2003.

[2] T. Tsuji, Y. Sasaki, M. Tanaka, N. Hanabata, R. Hada and A. Munakata, "Microvessel morphology and vascular endothelial growth factor expression in human colonic carcinoma with or without metastasis," Laboratory investigation, vol. 82, no. 5, pp. 555-562, May 2002.

[3] J.W. Slaton, K. Inoue, P. Perrotte, A.K. EI-Naggar, D.A. Swanson, U. Fidler and C.P.N. Dinney, "Expression levels of genes that regulate

metastasis and angiogenesis correlate with advanced pathological stage of renal cell carcinoma," American Journal of Pathology, vol. 158, no. 2,pp. 735-743,Feb. 2001.

[4] M. Mete, L. Hennings, H. Spencer and U. Topaloglu, "Automatic identification of angiogenesis in double stained images of liver tissue," BMC Bioinformatics, vol. 10, no. Suppl I I, pp. Sl3, Oct. 2009.

[5] S. Potts, K. Lillard, T. Johnson, F. Voelker and R. Valdez, "P26: Performance of a novel automated microvessel analysis algorithm across whole slide digital images," Experimental and Toxicologic Pathology, vol. 61, no. 4, pp. 413-4l3, Jul. 2009.

[6] R. Shapley and M. 1. Hawken, "Color in the Cortex: single-and double-opponent cells," Vision Research, vol. 51, no. 7, pp. 701-717, Apr. 2011.

[7] 1. Zhang, Y. Barhomi, and T. Serrep'A new biologically inspired color image descriptor," in Proc. 12th European Conference on Computer

Vision, 2012, pp. 312-324. [8] J. Yang, K. Yu, Y. Gong, and T. Huang, "Linear spatial pyramid

matching using sparse coding for image classification," in Proc. IEEE

Conference on Computer Vision and Pattern Recognition Workshops, 2009, pp. 1794-1801.

[9] H. Lee, A. Battle, R. Raina, and A. Ng. "Efficient sparse coding algorithms," in Proc. 20th Annual Conference on Neural Information

Processing Systems, 2006, pp. 801-808. [10] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: spatial

pyramid matching for recognizing natural scene categories," in Proc.

iEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 2169-2178.

118