Upload File

deep learning for medical image analysis - university of oulu learning... · 2017-06-11 · deep...

  • Home
  • Documents
  • Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,
28
Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging, Physics and Technology University of Oulu

Upload: others

Post on 10-Mar-2020

5 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

DeepLearningforMedicalImageAnalysis

AlekseiTiulpinResearchUnitofMedicalImaging,PhysicsandTechnology

UniversityofOulu

Page 2: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Outline

• WhatisDeepLearning• MachineLearning• Convolutionalneuralnetworks:computervisionbreakthrough• Applications:Images,Video,Audio• Interpretability• Transferlearning• Limitations

• MedicalImageanalysis• Segmentation• Skincancerdetectionatadermatologistlevel• DiabeticRetinopathy• Ownstudy:KneeOsteoarthritisdiagnosis

2A.Tiulpin,MIPT,UniveristyofOulu

Page 3: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

WhatisMachineLearning?

3

Page 4: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

MachineLearning

Space𝑿 (objects) Space𝒀 (labels)

Mapping𝑎(𝒙)

Object𝒙 Answer𝒚

4A.Tiulpin,MIPT,UniveristyofOulu

Page 5: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Learning(supervised)

𝑎( 𝒙 ≈ 𝑎(𝒙)

LearningLookingforoptimal

parameters𝜃

+

Newdata

Label

Prediction

5A.Tiulpin,MIPT,UniveristyofOulu

Page 6: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Artificialneuron

𝑥,

𝑥-

𝑥.

1

𝜃/𝜃,

𝜃-

𝜃.

𝜽1𝒙

Activation

Dendrite

Synapse

Axon

Cellbody

https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Complete_neuron_cell_diagram_en.svg/1280px-Complete_neuron_cell_diagram_en.svg.png

6A.Tiulpin,MIPT,UniveristyofOulu

Page 7: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

ArtificialNeuralNetworks:non-linerfunctions

0 𝑥,

𝑥-

http://neuralnetworksanddeeplearning.com/images/tikz35.png

7A.Tiulpin,MIPT,UniveristyofOulu

Page 8: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Popularimagefeaturedescriptors

• HistogramofOrientedGradients- HoG• Detectionofeverything

• Person,car,roadsign,face,…

• Shift-InvariantFeaturetransform- SIFT• Key-pointmatching• …

• LBP• Face detection• Texture analysis

A.Tiulpin,MIPT 8

Page 9: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

WhatisDeepLearning

Data Featureextraction Predictionmodel

Data Featureextraction Prediction

ConventionalMachineLearningpipeline

DeepLearning

Featurelearning

Deeppredictionmodel

9A.Tiulpin,MIPT,UniveristyofOulu

Page 10: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

DeepLearningvs.Hand-craftedfeatures

Hand-craftingfeatures……

A.Tiulpin,MIPT,UniveristyofOulu 10

Page 11: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Whydoweneedit?

• ImageNet –generalimagerecognitionchallenge• 1000classes• Millions oftrainingimages

DeepLearning

https://www.quora.com/What-is-the-winning-top-5-error-rate-on-ImageNet-2016

Top-5%accuracy

11A.Tiulpin,MIPT,UniveristyofOulu

Page 12: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

DeepLearningbreaktrough:AlexNet

• AlexNet• Krizhevsky,A.,Sutskever,I.,&Hinton,G.E.(2012).Imagenet classificationwithdeepconvolutionalneuralnetworks.In Advancesinneuralinformationprocessingsystems (pp.1097-1105).• Cited>11800times according toGoogleScholar

http://whatsnext.nuance.com/in-the-labs/convolutional-neural-networks-applied-to-natural-language-understanding-research/

Drasticerrorratedrop(>10%)achievedbylearningfeaturesdirectlyfromdata!

12A.Tiulpin,MIPT,UniveristyofOulu

Page 13: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

ConvNet layerhierarchy:localconnections

13A.Tiulpin,MIPT,UniveristyofOulu

Page 14: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

DeepLearning:conceptsoutofsimpleconcepts

Yann LeCun et.al,doi:10.1038/nature14539 IanGoodfellow et.al, DeepLearningBook

Deeplearningallowscomputationalmodelsthatarecomposedofmultipleprocessinglayerstolearnrepresentationsofdatawithmultiple

levelsofabstraction

Deeplearningallowsthecomputertobuildcomplexconceptsoutofsimplerconcepts.

https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/

14A.Tiulpin,MIPT,UniveristyofOulu

Page 15: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Convnets:model ofthe brain

• InferiorTemporal(IT)cortex• Visualrecognitiontasks

• ConvNet performsequivalentlywelltoSVMtrainedontopoftheactivationsfromITcortex

15A.Tiulpin,MIPT,UniveristyofOulu

Page 16: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Isitreallyablack-box?

DeepLearning

16A.Tiulpin,MIPT,UniveristyofOulu

Page 17: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Isitreallyablack-box?

ConvNets

RandomForest&GBM

KernelMethods

17A.Tiulpin,MIPT,UniveristyofOulu

Page 18: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Attentionmaps&GuidedBackpropagation

• Neuralnetisadifferentiablefunction• Wecanfindagradientwithrespecttotheimageforanyclass• Details:GradCAM

18A.Tiulpin,MIPT,UniveristyofOulu

Page 19: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Transfer learning

• Deepmodelslearnverygenericfeaturesatthefirstlayers• Iftheytrainedonalargedatasethavingmanycategories,e.g.ImageNet

• Canweusedpre-trainedfeaturesinthenewdomains?

ImageNetfeatures Breastcancerdiagnosis

19A.Tiulpin,MIPT,UniveristyofOulu

Page 20: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Limitations

• Data• Modelswithmanyparameterseasilyoverfit• Wisearchitecturedesignisneededtoperformwell• High-endequipmentisneededtotrainthemodels• Linearnatureofthemodelsallowsto“fool”them• See,Adversarialexamples

20A.Tiulpin,MIPT,UniveristyofOulu

Page 21: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

MedicalImagesegmentation

21A.Tiulpin,MIPT,UniveristyofOulu

Page 22: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

SkinCancer:transferlearningsuccess

• InceptionV3network• ImageNet

• Dermatologistlevelclassification• Canbesooninyourmobilephone

22A.Tiulpin,MIPT,UniveristyofOulu

Page 23: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

DiabeticRetinopathy

• Kaggle competition• $100000prize• Kappacoefficient0.85wasachieved

23A.Tiulpin,MIPT,UniveristyofOulu

Page 24: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Summary

• WithDeeplearningwecandopracticallyanything• Classify• Regress• Segment

• Weneedalotofdatatotraindeepmodels• Wealsoneedpowerfulhardwaretotrainthenetworks• Pre-trainedmodelscanbeusedtoachieveareallygoodperformanceinclassificationandregression

A.Tiulpin,MIPT 24

Page 25: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

DeepLearningforMedicalImageAnalysis

AlekseiTiulpinResearchUnitofMedicalImaging,PhysicsandTechnology

UniversityofOulu

Page 26: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Imagesused

• http://cdn2-www.superherohype.com/assets/uploads/gallery/terminator-genisys/terminator-genisys-018.jpg

• http://2.bp.blogspot.com/-7BaGjkSq6rc/VKBPTRYzNgI/AAAAAAAANyo/4VsIBTP-NVY/s1600/hog.jpg

• https://pdfs.semanticscholar.org/44bd/549524fb17e3e806723fef2a18ffe9cfba87.pdf?_ga=2.65825965.2083661991.1497171718-775385516.1497171718

• http://2.bp.blogspot.com/-WB8VomB7kLY/UTx6se-U1ZI/AAAAAAAAA5g/kEbN46LHty8/s1600/matched_sift.jpg

• http://www.cavemanlogic.com/i/caveman.jpg• https://commons.wikimedia.org/wiki/File%3AGray1197.png• https://arxiv.org/abs/1610.02391• https://devblogs.nvidia.com/parallelforall/deep-learning-computer-vision-caffe-cudnn/• https://blogs.nvidia.com/wp-content/uploads/2016/09/19-beck-image.jpg

A.Tiulpin,MIPT,UniveristyofOulu 26

Page 27: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Imagesused

• https://www.nature.com/article-assets/npg/srep/2016/160607/srep27327/images/w582/srep27327-f1.jpg• https://www.researchgate.net/profile/Marius_George_Linguraru/publication/262293298/figure/fig2/AS:296674879655946@1447744211805/Fig-2-Examples-of-liver-segmentation-overlayed-in-green-on-axial-views-of-3D-CT-data.png• http://www.jpathinformatics.org/articles/2016/7/1/images/JPatholInform_2016_7_1_29_186902_u11.jpg• https://deepsense.io/wp-content/uploads/2015/09/pred_2.jpg

A.Tiulpin,MIPT,UniveristyofOulu 27

Page 28: Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep Learning for Medical Image Analysis Aleksei Tiulpin Research Unit of Medical Imaging,

Imagesused

• http://5047-presscdn.pagely.netdna-cdn.com/wp-content/uploads/2015/08/DR4_1.png• https://www.kaggle.com/c/diabetic-retinopathy-detection

A.Tiulpin,MIPT,UniveristyofOulu 28


Short C HOW TO INCULCATE DEEP LEARNING IN MEDICAL

Review 3D Deep Learning on Medical Images: A Review

Hello World Deep Learning in Medical Imaging - dbmi.mums.ac.ir · Hello World Deep Learning in Medical Imaging Paras Lakhani1 & Daniel L. Gray2 & Carl R. Pett2 & Paul Nagy3,4 & George

Deep Multimodal Learning for Medical Visual Question Answeringceur-ws.org/Vol-2380/paper_123.pdf · Deep Multimodal Learning for Medical Visual Question Answering1 Lei Shi1, Feifan

Hello World Deep Learning in Medical Imaging · 2018. 5. 18. · Hello World Deep Learning in Medical Imaging Paras Lakhani1 & Daniel L. Gray2 & Carl R. Pett2 & Paul Nagy3,4 & George

Challenges in deep learning methods for medical imaging - Pubrica

Chest X-ray Histology Medical Imaging with Deep Learning ... - Lecture 6.pdf · Medical Imaging with Deep Learning Overview Popular image problems: Chest X-ray Histology Multi-modality/view

Adversarial Attacks Against Medical Deep Learning Systems · 2019-02-05 · Adversarial Attacks Against Medical Deep Learning Systems, Figure 1: Adversarial attacks within the broader

Evidence Requirements of Deep Learning Based Medical ...essay.utwente.nl/79028/1/Müller_MA_Human Factors and Engineerin… · 2.1 Deep Learning-Based Medical Support Systems (DLMSS)

Interactive Medical Image Segmentation Using Deep Learning ... VoR08270673.pdf · Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning Guotai

Deep Learning in Medical Physics—lessons We Learned - …amos3.aapm.org/abstracts/pdf/127-36916-419554-130797.pdf · Deep Learning in Medical Physics ... Lasso RF Logistic Prob

Image Reconstruction by Splitting Deep Learning ...math.sjtu.edu.cn/faculty/xqzhang/Publications/MRI_DL.pdf · Most of the emerged deep learning based medical image reconstruction

Prospects of deep learning for medical imaging · significantly. This review article aims to survey deep learning literature in medical imag-ing and describe its potential for future

Machine Learning and Deep Learning - MathWorks · 3 Deep Learning is Everywhere Computer Vision Pedestrian and traffic sign detection Landmark identification Scene recognition Medical

Deep Learning Predicts Lung Cancer Treatment Response from … · Precision Medicine and Imaging Deep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging

13-185A83-HOLZINGER-Deep-Learning-Medical-2017human-centered.ai/.../2016/...HOLZINGER-Deep-Learning-Medical-2017-3x3.pdf · provides a versatile toolbox for statistical data analysis

Deep Learning for Multi-Task Medical Image Segmentation in ... · Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities ... Deep learning for multi-task medical

Active Deep Learning for Medical Imaging Segmentation · Active Deep Learning for Medical Imaging Segmentation Degree’s Thesis Audiovisual Systems Engineering Author: Marc G orriz

Applying deep learning techniques on medical corpora from

Deep Learning for Computer Vision · 2 Deep Learning is Ubiquitous Computer Vision Pedestrian and traffic sign detection Landmark identification Scene recognition Medical diagnosis

Deep Learning for Medical Imaging - Final Project Report

Deep Feature Learning for Medical Image Analysis with Convolutional Autoencoder …mchen/min_paper/2017/2017... · 2019-11-16 · IEEE Proof 1 Deep Feature Learning for Medical Image

(2017/06)Practical points of deep learning for medical imaging

Deep Learning in Medical Imaging: General Overview...571 Deep Learning in Medical Imaging kjronline.org Korean J Radiol 18(4), Jul/Aug 2017 Deep learning is a part of ML and a special

FDA Perspectives on Machine Learning / Deep … · FDA Perspectives on Machine Learning / Deep Learning Devices for Medical Image Interpretation 1 ... What is a Medical Device?

Concept detection on medical images using Deep Residual Learning …ceur-ws.org/Vol-1866/paper_122.pdf ·  · 2017-07-13Concept detection on medical images using Deep Residual Learning

Deep learning for medical image processing: overview

COMPARATIVE DEEP LEARNING FOR CONTENT- … · COMPARATIVE DEEP LEARNING FOR CONTENT-BASED MEDICAL IMAGE RETRIEVAL DECEMBER 1st, 2016 ... CBIR Benchmarking Multi-Agent ... • In this

RetainVis: Visual Analytics with Interpretable and ... · Deep learning for electronic medical records. Though the most prevalent use of deep learning techniques in medical domains

A Feature Selection Study for Medical Image deep learning

Learning Deep Representations of Medical Images using

Deep Learning in Medical Imaging: General Overview · Deep Learning in Medical Imaging kjronline.org Korean J Radiol 18(4), Jul/Aug 2017 Deep learning is a part of ML and a special

Deep Learning for Medical Image Segmentation - arXiv · Deep Learning for Medical Image Segmentation Matthew ... allows faster training since being linear around 0 means nodes will

A gentle introduction to deep learning in medical image ...€¦ · Keywords: Introduction, Deep learning, Machine learning, Image segmentation, Image registration, Computer-aided

Deep Learning Under the Microscope: Improving the ... · Deep Learning Under the Microscope: Improving the Interpretability of Medical Imaging Neural Networks Magdalini Paschali1?,