deep convolutional neural networks for accelerated...

Deep convolutional neural networks for accelerated dynamic magnetic resonance imaging Christopher M. Sandino 1 , Neerav Dixit 1 Purpose References Discussion / Conclusions 1 Department of Electrical Engineering, Stanford University, Stanford, CA • Magnetic resonance imaging (MRI) scan times are relatively slow, especially for dynamic acquisitions like in the heart • Scan time can be accelerated by compressed sensing 1 (CS) schemes that exploit data redundancy to reconstruct undersampled MR images • However, CS-based reconstruction times are long because they employ iterative algorithms to solve optimization problems • Critical time between patient exam and diagnosis is extended by hours – making MRI infeasible for urgent clinical situations Goal: Use convolutional neural networks to efficiently and accurately reconstruct highly undersampled dynamic MRI data Background • CS-based image reconstruction 1 is based on iteratively solving non-linear inverse problems of the form: • CNNs are well-suited for modelling this task 2 and have previously been used to learn CT 3 and MRI 4 static CS recon pipelines Input time-series 20 64 64 256 x 256 256 x 256 256 x 256 64 128 2 128 2 128 2 128 128 128 64 2 256 256 64 2 64 2 256 32 2 512 512 32 2 32 2 512 16 2 16 2 1024 16 2 1024 1024 32 2 512 512 32 2 32 2 512 64 2 256 256 64 2 64 2 128 2 128 2 128 128 256 128 2 128 256 x 256 64 64 256 x 256 256 x 256 20 256 x 256 conv 3x3, stride 1, pad 1; batch norm; ReLU max pool 2x2, stride 2 convT 2x2, stride 2 conv 1x1, stride 1, pad 0 copy and concatenate Output time-series Modified U-net 5 architecture: Methodology • Each dataset is expanded into N=896 (2D+time) examples • Each example has 20 time frames (50 ms temporal resolution) • Datasets are not fully sampled, but are diagnostic quality • Acquisition time: ~15 min, CS reconstruction time: ~2 hours 4-D cardiac MRI datasets: Deep Reconstruction Workflow: • Still many questions: Similar performance for dynamic data? Best loss function to train on? Upper limit for undersampling? How to evaluate CNN reconstructions? (Courtesy of Dr. Shreyas Vasanawala) N train = 2688 N val = 896 N test = 896 PSNR (dB) SSIM Speed CS (Truth) - - 2hr19 m Naive 57.792 0.629 3s CNN-L2 62.976 0.734 57s CNN-L1(F) 62.923 0.743 53s CNN-SSIM 62.526 0.720 200s Experiment #2: Does loss function impact image quality? Figure from Lustig et al. 1 [1] M Lustig, et al. "Compressed sensing MRI." IEEE signal processing magazine, 2008. [2] K Gregor, and Y LeCun. "Learning fast approximations of sparse coding." ICML, 2010. [3] KH Jin, et al. "Deep Convolutional Neural Network for Inverse Problems in Imaging." arXiv, 2016. [4] K Hammernik, et al. "Learning a Variational Network for Reconstruction of Accelerated MRI Data." arXiv, 2017. [5] O Ronneberger, et al. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015 • We present a CNN model that can reconstruct dynamic MR images comparable to standard techniques in under one minute (150x faster than CS) • Potential to also accelerate scan time (2x faster than standard) • Not able to resolve temporal dynamics well – look to RNNs? • CNNs provide faster scan and recon times – potential to make MRI cheaper and more feasible in urgent clinical situations Performance Statistics: • L1 / L2 similar performance • SSIM better at preserving structure and edges Experiment #1: Can CNN reconstruct undersampled data better than CS? L2: L1(F): SSIM: ||y - ˆ y || 2 ||F (y - ˆ y )|| 1 1 2 (1 - SSIM(y, ˆ y )) minimize x ||F s x - y || 2 2 + λ||φ(x)|| 1 Results PSNR=56.7 dB PSNR=62.4 dB PSNR=51.0 dB PSNR=57.3 dB • Used L2 loss • Ground truth is diagnostic quality CS recon • CNN outperforms Naïve recon by ~6 dB

Upload: others

Post on 07-Aug-2020

35 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Deep convolutional neural networks for accelerated ...cs231n.stanford.edu/reports/2017/posters/513.pdf · Deep convolutional neural networks for accelerated dynamicmagnetic resonance

Deep convolutional neural networks for accelerated dynamic magnetic resonance imagingChristopher M. Sandino1, Neerav Dixit1

Purpose

References

Discussion / Conclusions

1Department of Electrical Engineering, Stanford University, Stanford, CA

• Magnetic resonance imaging (MRI) scan times are relativelyslow, especially for dynamic acquisitions like in the heart

• Scan time can be accelerated by compressed sensing1 (CS)schemes that exploit data redundancy to reconstructundersampled MR images

• However, CS-based reconstruction times are long becausethey employ iterative algorithms to solve optimization problems

• Critical time between patient exam and diagnosis is extendedby hours – making MRI infeasible for urgent clinical situations

Goal: Use convolutional neural networks to efficiently and accurately reconstruct highly undersampled dynamic MRI data

Background• CS-based image reconstruction1 is based

on iteratively solving non-linear inverseproblems of the form:

• CNNs are well-suited for modelling thistask2 and have previously been used tolearn CT3 and MRI4 static CS reconpipelines

Inputtime-series

20 64 64

256x25

1282

128 128

128

642

256 256

642

256

322

512 512

322

512

162

1024

162

1024

322

512 512

322

512

642

256 256

642

1282

128 128256

1282

128

256x25

64 64

256x25

6conv3x3,stride1,pad1;batchnorm;ReLU

maxpool2x2,stride2

convT 2x2,stride2

conv1x1,stride1,pad0

copyandconcatenate

Outputtime-series

Modified U-net5 architecture:

Methodology

• Each dataset is expanded into N=896 (2D+time) examples• Each example has 20 time frames (50 ms temporal resolution)• Datasets are not fully sampled, but are diagnostic quality• Acquisition time: ~15 min, CS reconstruction time: ~2 hours

4-D cardiac MRI datasets:

Deep Reconstruction Workflow:

• Still many questions: Similar performance for dynamic data?Best loss function to train on? Upper limit for undersampling?How to evaluate CNN reconstructions?

(Courtesy of Dr. Shreyas Vasanawala)

Ntrain = 2688Nval = 896Ntest = 896

PSNR (dB) SSIM SpeedCS (Truth) - - 2hr19 mNaive 57.792 0.629 3sCNN-L2 62.976 0.734 57sCNN-L1(F) 62.923 0.743 53sCNN-SSIM 62.526 0.720 200s

Experiment #2: Does loss function impact image quality?

Figure from Lustig et al.1

[1] M Lustig, et al. "Compressed sensing MRI." IEEE signal processing magazine, 2008. [2] K Gregor, and YLeCun. "Learning fast approximations of sparse coding." ICML, 2010. [3] KH Jin, et al. "Deep ConvolutionalNeural Network for Inverse Problems in Imaging." arXiv, 2016. [4] K Hammernik, et al. "Learning a VariationalNetwork for Reconstruction of Accelerated MRI Data." arXiv, 2017. [5] O Ronneberger, et al. "U-net:Convolutional networks for biomedical image segmentation." International Conference on Medical ImageComputing and Computer-Assisted Intervention. 2015

• We present a CNN model that can reconstruct dynamic MRimages comparable to standard techniques in under oneminute (150x faster than CS)

• Potential to also accelerate scan time (2x faster than standard)• Not able to resolve temporal dynamics well – look to RNNs?• CNNs provide faster scan and recon times – potential to make

MRI cheaper and more feasible in urgent clinical situations

Performance Statistics:

• L1 / L2 similarperformance

• SSIM better atpreservingstructure andedges

Experiment #1: Can CNN reconstruct undersampled data better than CS?

L2:L1(F):SSIM:

= ||y � y||2= ||F(y � y)||1

2(1� SSIM(y, y))

minimizex

||Fs

x� y||22 + �||�(x)||1

Results