u2-netdiabetic macular edema (dme) in 2017, 425 million people worldwide were suffering from...
TRANSCRIPT
U2-Net:ABayesianU-NetModelwithEpistemicUncertaintyFeedbackforPhotoreceptorLayerSegmentationinPathologicalOCTScans
JoséIgnacioOrlando,PhilippSeeböck,Hrvoje Bogunović,SophieKlimscha,ChristophGrechenig,SebastianWaldstein,BiancaS.Gerendas,UrsulaSchmidt-Erfurth
1.3billionpeoplesufferingsomeformofvisualimpairment
Age-RelatedMacularDegeneration(AMD)Maincauseofvisualdeficiencyinindustrializedcountries
Globalprevalenceof8.7%within45-85yearsoldpopulation
DiabeticMacularEdema(DME)In2017,425millionpeopleworldwideweresufferingfromdiabetes
~10%developedvision-threateningDME
RetinalVeinOcclusion(RVO)14-19millionpeopleaffectedworldwide
AMD
DME
RVO
Photoreceptorcelldeath Visualacuityloss
OpticalCoherenceTomography(OCT)State-of-the-artimagingmodalityinAMD,RVOandDME
Allowstoassessphotoreceptorintegrity
InterdigitationZone(IZ)
Outersegmentofphoto-receptors
EllipsoidZone(IS/OS)
InterdigitationZone(IZ)
Outersegmentofphoto-receptors
EllipsoidZone(IS/OS)
Normalphotoreceptors Normalphotoreceptors
Abnormalthinning
Pathologicaldisruption
Ourmid-termgoal
(i)Accuratesegmentation(ii)Interpretablefeedbacktocorrecttheresults
Understandthepathophysiologicalprocessesthatcause damageinphotoreceptorintegrity
Keychallenge
Pathologicalalterations
Ambiguousappearancesturndifficulttoproducereliablesegmentations
Unfeasibletocaptureeverypossiblepathologicalfeatureonatrainingset
Bayesiandeeplearning
BayesiandeeplearningModeluncertainty
Aleatoric Taskuncertainty,whatwedon’tknowandwewillneverlearn
Epistemic Modeluncertainty,whatwedon’tknowbutwecanlearngivenmoretrainingdata
BayesiandeeplearningModeluncertainty
Aleatoric
Epistemic
Taskuncertainty,whatwedon’tknowandwewillneverlearn
Modeluncertainty,whatwedon’tknowbutwecanlearngivenmoretrainingdata
Epistemicuncertainty
BDLisusedtocomputeaposteriordistribution
Approximatedistributionlearnedbyvariational inference
(Galetal.,2015)
Bernoullidistributiontotheweightsofthei-thconvolutionallayerusingDropout
Epistemicuncertainty
MonteCarlosamplingwith
dropoutintesttime
MonteCarlosamplingwith
dropoutintesttime
Samplingmultipleslightlydifferent
outputs
Averagingtheoutcomesresultsinbetterperformance
Standarddeviationallowstoretrieveanepistemicuncertaintyestimate
OurapproachUncertaintyU-shapedNetwork
OurapproachUncertaintyU-shapedNetwork
OurapproachU2-Net
StandardU-Net+Nearestneighborupsampling +LeakyReLUs +Batchnorm+Dropout
MCsamplingwithdropoutintesttimetopredictaveragescoremap&epistemicuncertaintymap
Materials
DatasetA
Trainingset Validation Test
AMD(early,CNV) 10volumes 490B-scans
DME 16volumes 784B-scans
RVO 24volumes 1176B-scans
Total 50volumes 2450B-scans
Splitatapatient-basispreservingdiseaseproportion
31volumes(1519B-scans)
4volumes(196B-scans)
15volumes(735B-scans)
DatasetB
Test
LateAMD(GA) 10volumes 490B-scans
Separatetestset
10volumes(496B-scans)
Evaluationmetrics
Photoreceptors Disruptions- AreaunderPrecision/Recallcurve
- Diceindex- AreaunderPrecision/Recallcurve
(atanA-scanlevel)
Baselines
StandardU-Net(Ronneberger etal.,MICCAI2015)
Batchnormalization,NNupsampling,dropoutinbottleneck
BRU-Net(Apostolopoulos etal.,MICCAI2017)
BranchresidualU-Netwithdilatedconvolutionsandresidualconnections
BU-NetBayesianU2-Netwithaleatoric uncertaintyestimates
(InspiredinNairetal.,MICCAI2018)
Results
Quantitativeevaluation
U2-Net
B-scan
Manual
TestsetA– Dice=0.9624(B-scanlevel)– Meanuncertainty:6.004e-4(B-scanlevel)
Manual / U2-Net
TestsetA– Dice=0.9624(B-scanlevel)– Meanuncertainty:6.004e-4(B-scanlevel)
TestsetA– Dice=0.9624(B-scanlevel)– Meanuncertainty:6.004e-4(B-scanlevel)
Epistemicuncertaintyestimate
B-scan
Manual U2-Net
TestsetA– Dice=0.9196(B-scanlevel)– Meanuncertainty:6.720e-4(B-scanlevel)
Manual / U2-Net
TestsetA– Dice=0.9196(B-scanlevel)– Meanuncertainty:6.720e-4(B-scanlevel)
Epistemicuncertaintyestimate
TestsetA– Dice=0.9196(B-scanlevel)– Meanuncertainty:6.720e-4(B-scanlevel)
B-scan
Manual U2-Net
TestsetA– Dice=0.5400(B-scanlevel)– Meanuncertainty:0.0014(B-scanlevel)
Manual / U2-Net
TestsetA– Dice=0.5400(B-scanlevel)– Meanuncertainty:0.0014(B-scanlevel)
Epistemicuncertaintyestimate
TestsetA– Dice=0.5400(B-scanlevel)– Meanuncertainty:0.0014(B-scanlevel)
Uncertaintyestimatesareinverselycorrelatedwithperformance
TestsetA(earlyAMD,CNV,RVO,DME)
TestsetB(lateAMD,GA)
Uncertaintyestimatesareinverselycorrelatedwithperformance
TestsetA(earlyAMD,CNV,RVO,DME)
TestsetB(lateAMD,GA)
Conclusions
Firstdeeplearningapproach forphotoreceptorsegmentationinpathologicalOCTscans
AveragingmultipleMCsamples allowstoincreaseperformanceinabnormalareas withoutaffectingresultsinhealthyregions
Epistemicuncertaintycanbeusedtoassessresults’qualityandtoidentifyareasthatmightneedformanualcorrection
https://ignaciorlando.github.io
Thanksforyourattention!Doyouhaveanyquestions?
@ignaciorlando
U2-Net:ABayesianU-NetModelwithEpistemicUncertaintyFeedbackforPhotoreceptorLayerSegmentationinPathologicalOCTScans
JoséIgnacioOrlando,PhilippSeeböck,Hrvoje Bogunović,SophieKlimscha,ChristophGrechenig,SebastianWaldstein,BiancaS.Gerendas,UrsulaSchmidt-Erfurth
HowmanyMCsamplesarenecessary?ValidationsetA
Photoreceptors Disruptions
Quantitativeevaluation
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Recall
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Prec
isio
n
Standard U-Net - AUC = 0.8946BRU-Net - AUC = 0.879BU-Net (T=1) - AUC = 0.8316BU-Net (T=10) - AUC = 0.8289U2-Net (T=1) - AUC = 0.9201U2-Net (T=10) - AUC = 0.9249
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Recall
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Prec
isio
n
Standard U-Net - AUC = 0.9566BRU-Net - AUC = 0.9593BU-Net (T=1) - AUC = 0.9466BU-Net (T=10) - AUC = 0.9505U2-Net (T=1) - AUC = 0.9653U2-Net (T=10) - AUC = 0.9669
Onecentralmillimeter FullOCTvolume
TestsetA
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Recall
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Prec
isio
n
Standard U-Net - AUC = 0.4428BRU-Net - AUC = 0.1894BU-Net (T=1) - AUC = 0.1461BU-Net (T=10) - AUC = 0.1495U2-Net (T=1) - AUC = 0.5151U2-Net (T=10) - AUC = 0.5047
Quantitativeevaluation
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Recall
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Prec
isio
n
Standard U-Net - AUC = 0.939BRU-Net - AUC = 0.9295BU-Net (T=1) - AUC = 0.8969BU-Net (T=10) - AUC = 0.8998U2-Net (T=1) - AUC = 0.95U2-Net (T=10) - AUC = 0.9472
Onecentralmillimeter FullOCTvolume
TestsetB
InterdigitationZone(IZ)
Outersegmentofphoto-receptors
EllipsoidZone(IS/OS)
Retinalpigment
Epithelium(RPE)
Myoidzone
Externallimiting
membrane(ELM)