major vessel segmentation on x-ray coronary angiography

Proceedings of Machine Learning Research – Under Review:1–5, 2019 Abstraction – MIDL 2019 submission

Major Vessel Segmentation on X-ray Coronary Angiographyusing Deep Networks with a Novel Penalty Loss Function

Su Yang1 [email protected] Kweon∗1 [email protected] Kim∗1 [email protected] Department of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul,South Korea

Editors: Under Review for MIDL 2019

Abstract

In this study, we proposed a segmentation method of major vessels on X-ray coronaryangiography using fully convolutional networks based on U-Net architecture. A novel lossfunction pGD was introduced by adding a term for penalizing false negative and falsepositive to generalized dice coefficient (GD). DenseNet121 with pGD achieved the highestaverage DSC of 91.9±8.7%, precision of 91.3±8.8%, and recall of 92.6±9.6%, respectivelyand showed improved segmentation performance compared to GD.

Keywords: Deep learning, X-ray coronary angiography, Major vessel segmentation, PenaltyLoss Function.

1. Introduction

X-ray coronary angiography (CAG) is a primary diagnostic imaging modality for coronaryartery diseases. Quantitative coronary angiography (QCA) provides principle morphologicalindices such as diameter stenosis and lesion length to evaluate coronary lesions. However,QCA analysis shows high inter-observer variability and limited reproducibility with manualcorrection (Hermiller et al., 1992) because the vessel segmentation of CAG is hindered byseveral causes (Figure 1). For automated CAG segmentation, although conventional imageprocessing methods (Wan et al., 2018) and deep learning networks (Au et al., 2018; Jo et al.,2019) have been proposed, the segmentation accuracy was not sufficiently high for clinicalapplications. In clinical practice, QCA analysis still takes 5-10 minutes per image set ofa patient for an expert using commercial software with semi-automatic segmentation toolbased on edge-detection method.In this study, a fast and robust method for segmentation of three major vessels is proposedusing deep networks with introducing a novel loss function based on generalized dice coeffi-cient (Sudre et al., 2017) to put an additional weight to both false negative and false positive.Five fully convolutional networks were applied to evaluate the segmentation performanceby combining simple U-Net (Ronneberger et al., 2015), VGG16 (Simonyan and Zisserman,2014), ResNet (He et al., 2016), DenseNet (Huang et al., 2017), and InceptionResNetv2(Szegedy et al., 2017) encoders with U-Net decoder.

∗ Contributed equally

c© 2019 S. Yang, J. Kweon & Y.-H. Kim.

Major Vessel Segmentation on X-ray Coronary Angiography using Deep Networks

Figure 1: Examples of CAG with segmentation label of major vessels (yellow line). Coloredcircles show major causes to hinder the automated segmentation of CAG.

2. Methods

Data description. Patients underwent CAG in Asan Medical Center from February 2016to November 2016 were retrospectively enrolled. Research approval was granted from In-stitutional Review Board with a waiver of patient consent. Angiographic images of majorvessels for 1980 patients were collected and after excluding the images unable to recognizethe coronary structures like total chronic occlusion, a dataset of 5572 images was built. Forimage labeling, lumen area of major vessel was annotated by two experts using commercialsoftware CAAS workstation 7.5 (Pie Medical Imaging, Netherlands). The ratio of training,validation and test sets was 3 : 1 : 1.Network architecture. The raw 512×512 size of CAG images were resized and stacked to3 channels (224×224×3), and the input images were normalized according to 2-dimensionalmin-max normalization technique. In this work, the network architecture motivated fromthe U-Net was consisted of deep convolution layers of backbone and five 2 × 2 up-samplelayers concatenated with skip connection at same level (Appendix A). We used five back-bones with initial weights of ImageNet (Russakovsky et al., 2015) for transfer learning.Penalty Loss Function. For binary segmentation, a novel penalty loss function inspiredby generalized dice coefficient (GD) was introduced. GD is defined as 2(

∑cl=1wl

∑pnGlnPln)

/(∑c

l=1wl∑p

n(Gln + Pln)), where c is the number of classes, p is a total number of pixels,Gln is ground truth and Pln is prediction result. wl = 1/(

∑pn(Gln/c)

2 + ε) is set as a weightto provide invariance to different label properties, where ε = 10−6. By adding a penaltyproportional to loss, 1 −GD, pGD is defined as

pGD =GD

1 + k(1 −GD)(1)

where k is a penalty coefficient. When k = 0, pGD is equivalent to GD, while pGD givesadditional weights to false positive and false negative for k > 0 (Appendix B).Training setup. Mini-batch size was 16 and models were trained for 200 epochs. The dataaugmentation was performed with rotation (−20◦ ∼ 20◦), width and height shift (0 ∼ 0.1),and zoom (0 ∼ 0.1). For training, Adam optimizer (Kingma and Ba, 2014) was used, andthe learning rate which was initially set as 10−3 was reduced by half up to 10−6 when thevalidation loss stayed saturated for 5 epochs on plateau.

3. Results

Segmentation performance of deep learning networks was evaluated with dice similarity co-efficient (DSC), precision, and recall. In our experiments, DenseNet121 achieved the highest

2


Figure 2: Comparison of ground truth (yellow line) and predicted lumen area (red line)with (a) GD and (b) pGD, respectively. (c) DSC comparison with previous reports. (d)Distribution of DSC in test set.

average DSC of 91.9± 8.7% (Table 1). With all the tested networks, pGD showed a higherDSC than GD and the k maximizing DSC was within 2.2 − 2.6. For DenseNet121, pGDconsiderably improved the false prediction in the cases of severe vessel overlap or catheterinterference (Figure 2). The deep learning required about 0.04 seconds per image.

Table 1: Performance of Proposed Networks (%).

Networks k DSC Precision Recall

Simple U-Net 2.3 89.8 ± 9.4 89.8 ± 9.1 90.3 ± 10.8VGG16 2.2 89.3 ± 10.5 89.4 ± 10.0 90.0 ± 12.5

ResNet101 2.5 90.3 ± 9.8 89.5 ± 10.1 91.5 ± 10.5DenseNet121 2.5 91.9± 8.7 91.3± 8.8 92.6± 9.6

InceptionResNetv2 2.6 91.7 ± 9.3 91.0 ± 9.6 92.7 ± 10.1

Table 2: Parameter Search of k (%).

k DSC Precision Recall

0 91.1 ± 9.7 90.5 ± 9.9 92.0 ± 10.71.0 90.7 ± 10.3 90.1 ± 10.5 91.8 ± 11.22.0 91.6 ± 8.2 90.9 ± 8.9 92.6 ± 9.12.5 91.9± 8.7 91.3± 8.8 92.6± 9.63.0 91.7 ± 8.6 91.4 ± 8.5 92.3 ± 9.8

4. Discussion

In lumen segmentation of CAG, the present study showed a higher DSC than the previousreports (Au et al., 2018; Jo et al., 2019). The improved predictability was mainly responsiblefor application of the latest deep learning networks with the large data set augmented withnormal vessel images. We further enhanced the segmentation performance with introducingthe novel loss function pGD. In the lumen prediction with deep learning, over 80% ofimages in the test set had DSC > 90% and most predictive errors appeared at the proximalor distal part of major vessels, which is little relevant to calculate the quantitative measuresof coronary geometry (Figure 3). Deep learning may not only provide a feasible way to berelieved from the QCA analysis that requires labor-intensive manual corrections but alsoallow an automated real-time diagnostics beyond eye estimation in the clinical practice.

Figure 3: Representative examples of Densenet121 prediction (red) versus ground truth(yellow).

3


Acknowledgments

This research was supported by Basic Science Research Program through the NationalResearch Foundation of Korea (NRF) funded by the Ministry of Education (2016R1D1A1A02937565) and the National Research Foundation of Korea (NRF) grant funded by theKorea government (MSTI) (NRF-2017R1A2B3009800).

References

Benjamin Au, Uri Shaham, Sanket Dhruva, Georgios Bouras, Ecaterina Cristea, AndreasCoppi, Fred Warner, Shu-Xia Li, and Harlan Krumholz. Automated characterizationof stenosis in invasive coronary angiography images with convolutional neural networks.arXiv preprint arXiv:1807.10597, 2018.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning forimage recognition. In Proceedings of the IEEE conference on computer vision and patternrecognition, pages 770–778, 2016.

James B Hermiller, Jack T Cusma, Laurence A Spero, Donald F Fortin, Michael B Harding,and Thomas M Bashore. Quantitative and qualitative coronary angiographic analysis:review of methods, utility, and limitations. Catheterization and cardiovascular diagnosis,25(2):110–131, 1992.

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Denselyconnected convolutional networks. In Proceedings of the IEEE conference on computervision and pattern recognition, pages 4700–4708, 2017.

Kyungmin Jo, Jihoon Kweon, Young-Hak Kim, and Jaesoon Choi. Segmentation of the mainvessel of the left anterior descending artery using selective feature mapping in coronaryangiography. IEEE Access, 7:919–930, 2019.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXivpreprint arXiv:1412.6980, 2014.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks forbiomedical image segmentation. In International Conference on Medical image computingand computer-assisted intervention, pages 234–241. Springer, 2015.

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhi-heng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg,and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journalof Computer Vision (IJCV), 115(3):211–252, 2015. doi: 10.1007/s11263-015-0816-y.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scaleimage recognition. arXiv preprint arXiv:1409.1556, 2014.

Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso.Generalised dice overlap as a deep learning loss function for highly unbalanced segmen-tations. In Deep learning in medical image analysis and multimodal learning for clinicaldecision support, pages 240–248. Springer, 2017.

4


Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-FirstAAAI Conference on Artificial Intelligence, 2017.

Tao Wan, Xiaoqing Shang, Weilin Yang, Jianhui Chen, Deyu Li, and Zengchang Qin. Au-tomated coronary artery tree segmentation in x-ray angiography using improved hessianbased enhancement and statistical region merging. Computer methods and programs inbiomedicine, 157:179–190, 2018.

Appendix A. Proposed U-Net Architecture with Backbone Encoder.

Appendix B. Proof of Theorem 1

The∑p

n(1 −Gln)Pln term is summation of false negative and the∑p

nGln(1 − Pln) term issummation of false positive.

pGD = 2

∑cl=1wl

∑pnGlnPln∑c

l=1wl∑p

n(Gln + Pln) + k∑c

l=1wl∑p

n(1 −Gln)Pln + k∑c

l=1wl∑p

nGln(1 − Pln)

= 2

∑cl=1wl

∑pnGlnPln∑c

l=1wl∑p


l=1wl∑p

n((1 −Gln)Pln +Gln(1 − Pln))

= 2

∑cl=1wl

∑pnGlnPln∑c

l=1wl∑p


l=1wl∑p

n(Pln − 2PlnGln +Gln)

=2

∑cl=1 wl

∑pn GlnPln∑c

l=1 wl∑p

n(Gln+Pln)∑cl=1 wl

∑pn(Gln+Pln)∑c

l=1 wl∑p

n(Gln+Pln)+

k∑c

l=1 wl∑p

n(Pln−2PlnGln+Gln)∑cl=1 wl

∑pn(Gln+Pln)

=GD

1 + k(1 −GD)(2)

5

major vessel segmentation on x-ray coronary angiography

Documents