na-mic national alliance for medical image computing large-scale computing frameworks for...

NA-MICNational Alliance for Medical Image Computing http://na-mic.org

Large-Scale Computing Frameworks for Developing Image Analysis Tools

Steve Pieper, PhD

Isomics, Inc.

Founder and CEO of Isomics, Inc. a technology development company that works closely with multiple research institutions on projects including NIH grants and software development. Much of this work is done in collaboration with the Surgical Planning Laboratory at BWH.

National Alliance for Medical Image Computing http://na-mic.org

2

Acknowledgments• F. Jolesz, R. Kikinis, C. Tempany, P. Black, S. Wells, CF. Westin, M. Halle, N.

Hata, T. Kapur, A.Tannenbaum, M. Shenton, E. Grimson, P.Golland, W.Schroeder, J. Miller, N. Aucoin, K. Hayes, A. Yarmarkovich, C. Lisle, D. Marcus, J. Miller, R. Gollub, S. Pujol, S. Barre, W. Plesniak, B. Fischl, D. Greve and many more….

V E R I TAS


3

Overview

• Software Infrastructure– NA-MIC Kit Components– 3D Slicer

• Modules and Extensibility• Population Analysis• Organization and Community


Large Scale Computing

• Working at a High Level– Major Functional Components are Readily Available– Tested, Cross Platform, Documented…

• Suite of Tools and Systems for Organizing and Processing– Databases and Batch Computing

• Mechanisms to Translate Research to Clinical Users– Plug New Research into a Usable Platform

• A Talented and Productive Community– Multidisciplinary– National and International

4


5

NA-MIC Kit

• The NA-MIC Kit is a Collection of Software and Methodologies for Medical Image Computing

• Most components of the NA-MIC kit predate NA-MIC (National Alliance for Medical Image Computing)


6

NA-MIC Kit

• Packaging of:– Applications– Algorithms (toolkits)– Methodologies


7

NA-MIC Kit Components

• End User Application– 3D Slicer

• Image Analysis, Visualization, and GUI libraries– ITK, VTK, KWWidgets, Teem

• Archive and Distributed Processing Tools – XNAT, Batchmake, BIRN GRID tools

• Software Engineering Tools– CMake, CDash, CTest, CPack

http://www.na-mic.org/Wiki/index.php/SoftwareInventory

Provided by Pieper, Kikinis


8

Visualization Toolkit - vtk

Open source toolkit for scientific visualization, computer graphics, and image processing


9

With support from several IC’s


10

KWWidgets• Usage

– Cross-platform GUI with open source license– Object Oriented C++ API (very VTK-like)

• Active Development– New widgets– Work flow support– File/directory browser

• Future– Tracing (for testing)– Registration inspection widget– Camera control widget– Interface to ITK (spatial objects)– Cleanup

• kwwidgets.org– cvs, dashboard, bug tracker, wiki…– BSD license


11

NA-MIC Software Process

Source

CDashTesting DashboardVersion Control

Svn, CVS with web access

Developers review results

Developers check-in code

CMake, CTest

Multiplatform Software Compilation, Testing


12

Development Methodology

6 months

4 months

2 months

Release Patch Nightly Continuous

Release X.Y

Release X.Y.1

Release X.Y.2

Release X.Y.3

Extreme lifecycle PrivateSandbox

NA-MICSandbox

Slicer

ITK

Dashboard

CMakeCTestCPack

Testing


13

NA-MIC Software Process

• Emphasizes Cross Platform Testing

• Provides informal– Requirements specification– Design documents

• Which are captured via collaboration tools– Tcon, email, wiki, software documentation

• Traceability– Implicit in logs and collaboration


14

What is 3D Slicer?• A platform for image analysis and

visualization• Current Releases 2.7 and 3.2

– 2.x most features and documentation

– 3.x focus of current activity• A freely-downloadable program

– Source code and executables available for Windows, Linux, and Mac OS X

• Slicer is a research platform:– NOT an FDA approved medical

device – NOT finished – some parts will

work better than others


Slicer 3.2 May 2008

Kitware, Inc. GE Research Isomics, Inc.BWHUCLAUCSDU IowaMITGaTechUNCU UtahMGHUCIHarvardWashU…

NACNCIGTNA-MICmBIRNfBIRN…

http://slicer.org


16

3D Slicer Numbers• Numbers May 2007:

Subversion Commits: 3,407

Lines of Code*: 371,428

Bugs & Features:

154 Submitted

63 Closed

Active Developers with svn access†: 33

Numbers July 2008:

Subversion Commits: 7303

Lines of Code*: 448,175

Developers: 42

Image provided by A. Golby, F. Talos, P. Black

*: find . -iname \*.h -o -iname \*.cxx -o -iname \*.tcl -o -iname \*.java -o -name \*.py | grep -v svn | xargs wc†: svn log | grep "^r" | cut -d " " -f 3 | sort | uniq | wc


17

Slicer3 “Observer MVC” Pattern

• MRML (Model)– For Scene Description and Application State– MRML Nodes are Persistent and Undoable– Scene and Nodes are Observable– XML Serialization, Undo/Redo

• Logic Encapsulate VTK and ITK Pipelines (Controller)– Observe MRML to Configure Pipelines– Help Create/Manage Nodes– No UI Components (no Widgets, Actors, Mappers,

Renderers or RenderWindows)• GUI (View)

– Observe and Edit MRML– Interact with User and Display Hardware

Logic

MRML Nodes

GUI

Widgets Renderers

EditObserve

Observe

Observe

Edit

Edit

“Observe” means generic event mechanisms are used to pass information.“Edit” means code can directly call methods.

Example: GUI can call methods in Logic classes,but Logic cannot call GUI methods.MRML cannot call Logic or GUI methods.

There can be many observers for any event.


18

Scene Description

Provided by D. Gering


19

Image/Scene Management

• XML-Based MRML File Stores Scene Description

– Volumes (Images, Label Maps)

– Models– Hierarchical Affine

Transforms– Scene Data (Cameras,

Colors, Fiducials, etc).• Manipulated in World

Coordinates based on Patient RAS

Provided by S. Pieper


20

Alignment of all pre-operative datasets to the intra-operative images achieved during the neurosurgery.

Provided by Archip, Warfield

Neurosurgery Example

Archip N, Clatz O, Whalen S, Kacher D, Fedorov A, Kot A, Chrisochoides N, Jolesz F, Golby A, Black PM, Warfield SK. Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery. Neuroimage. 2007 Apr 1;35(2):609-24


Modules

21


Slicer3 Command Line Modules

Common architecture for interactive and batch

processing

Slices Courtesy Jim Miller


Each module has …… an entry in the module menu

… a panel of user interface controls


Slicer Modules

• Interactive Modules– Interact Closely with GUI– Volume Rendering, Models, Editor…

• Command Line Modules– “Batch Mode” (Can be run from command line or script)– Registration, Filtering, DTI Tractography…

• Both Interactive and Command Line Modules are Dynamically Loaded– They can be Built and Distributed Independent of Slicer

• Today we will look in detail only at Command Line Modules


25

EM Interactive Module

MRML Logic GUI

•Global Parameters

•Hierarchical parameters

•Image data

•Segmentation output

•Manage MRML nodes

•API for access to parameters

•Manage hierarchy

•Segmentation algorithm

•Window to parameter set

•Wizard

•Interaction with images

Provided by B. Davis


26

Command Line Module

MRML Logic CLI

•Global Parameters

•Hierarchical parameters

•Image data

•Segmentation output

•Manage MRML nodes

•API for access to parameters

•Manage hierarchy

•Segmentation algorithm

•XML Parameter File

•File I/O

•Batchable

Provided by B. Davis


Existing Command Line Modules• Demonstration

– Execution Model Tour– Your new hello world module

• Converters – Create a DICOM Series– Dicom DWI loader– Dicom to Nrrd– GE Dicom to NRRD Converter– Orient Images

• Filtering – Calculate Volume Statistics– CheckerBoard Filter– Extract Skeleton– Histogram Matching– Otsu Threshold– Resample Volume/ Resample Volume 2– Voting Binary Hole Filling– Zero Crossing Based Edge Detection Filter– Filtering.Arithmetic:

– Add Images– Subtract Images


Existing Command Line Modules• Filtering.Denoising

– Curvature Anisotropic Diffusion– Gaussian Blur– Gradient Anisotropic Diffusion

• Segmentation – EMSegment Simple– FreesurferSurfaceSectionExtraction– Otsu Threshold Segmentation– Simple region growing

• Filtering.Morphology – Grayscale Fill Hole– Grayscale Grind Peak

• Model Generation – Grayscale Model Maker– Label Map Smoothing– Model Maker– Multiple models example– Probe Volume With Model (Paint)

• Meshing.VoxelMesh – Voxel Meshing

– Median Filter


Registration Command Line Modules

• Registration – Affine registration

– Deformable BSpline registration

– (Utah) Deformable BSpline registration

– Linear registration

– RealignVolume


Diffusion Command Line Modules

• Diffusion Tensor – Resample DTI Volume

• Diffusion Tensor Estimation• Diffusion Tensor Scalar Measurements• Simple DWI IO Test• Rician LMMSE Image Filter• Tractography.Editor: ROISelect• Tractography.Seeding• Stochastic Tractography

– Generate Connectivity Map

– Stochastic ROI Tract Filter

– Stochastic Stochastic Tractography Filter


Example Moduleshttp://www.nitrc.org/projects/slicer3examples/

http://www.na-mic.org/Wiki/index.php/Slicer3.2:Training


Parameter Example

33

<?xml version="1.0" encoding="utf-8"?><executable> <description> HelloWorld Example</description>…<parameters> <label>Input/Output</label> <description>Input/output parameters</description> <image> <name>helloSlicerInputVolume</name> <label>Input Volume</label> <channel>input</channel> <index>0</index> <default>None</default> <description>Input volume</description> </image>…</parameters><parameters>

<label>Discrete Gaussian Parameters</label> <description>Parameters of the Discrete Gaussian Filter </description><double> <name>variance</name> <longflag>--variance</longflag> <description>Variance ( width of the filter kernel) </description> <label>Variance</label> <default>0.5</default></double></parameters></executable>


C++ Module

34

#include <iostream> #include "HelloSlicerCLP.h" #include "itkImage.h" #include "itkImageFileReader.h" #include "itkImageFileWriter.h" #include "itkDiscreteGaussianImageFilter.h"

int main(int argc, char * argv []) { PARSE_ARGS; typedef itk::Image< short, 3 > ImageType; typedef itk::ImageFileReader< ImageType > ReaderType; typedef itk::ImageFileWriter< ImageType > WriterType; ReaderType::Pointer reader = ReaderType::New(); WriterType::Pointer writer = WriterType::New(); reader->SetFileName( helloSlicerInputVolume.c_str() ); writer->SetFileName (helloSlicerOutputVolume.c_str()); typedef itk::DiscreteGaussianImageFilter <ImageType, ImageType> FilterType; FilterType::Pointer filter = FilterType::New(); try { filter->SetInput(reader->GetOutput()); filter->SetVariance(variance); writer->SetInput(filter->GetOutput()); writer->Update(); } catch (itk::ExceptionObject &excep) { std::cerr << argv[0] << ": exception caught !" << std::endl; return EXIT_FAILURE; } return EXIT_SUCCESS; }


Python ModuleXML = """<?xml version="1.0" encoding="utf-8"?> <executable> <category>Filtering.Denoising</category> ...

def toXML(): return XML;

def Execute ( inputVolume, outputVolume, conductance=1.0, timeStep=0.0625, iterations=1 ): print "Executing Python Demo Application!" Slicer = __import__ ( "Slicer" ); slicer = Slicer.Slicer() in = slicer.MRMLScene.GetNodeByID ( inputVolume ); out = slicer.MRMLScene.GetNodeByID ( outputVolume );

filter = slicer.vtkITKGradientAnisotropicDiffusionImageFilter.New() filter.SetConductanceParameter ( conductance ) filter.SetTimeStep ( timeStep ) filter.SetNumberOfIterations ( iterations ) filter.SetInput ( in.GetImageData() ) filter.Update() out.SetAndObserveImageData(filter.GetOutput()) return


Behind the scenes

• Tasks queued for processing thread• Three types of modules:

– executable,

– shared object, and

– python modules

• Scalar images sent via files for executables and memory* for shared objects and python

• Vector images, tensor images, geometry, tables, transforms sent via files

• Scalars, file names, directories, fiducials, regions sent via command line

* VTK-based modules using scalar images are only supported as executable (command line) modules.


Population Statistics

37


BIRN Integration

BIRNBIRN

acquisitionprotocols

distortioncorrection

tools

local databases

workflows

analysis tools

populationstatistics

.

.

.

.

.

.

PubMedPubMed

IBVDIBVD

BrainInfoBrainInfo

Visualization&

Interpretation

Visualization&

Interpretation

(.xcat)(.xcat)

(.xar)(.xar)


mBIRN Informatics

• BIRN Data Repository (BDR)• eXtensible Neuroimaging

Archive Tool (XNAT) http://www.xnat.org

• XML-Based Clinical Experiment Data Exchange Schema (XCEDE) http://www.xcede.org

• Stores Images, Demographics, Clinical Data, Analysis Results

• 3D Slicer Interoperability (currently read-only, read/write prototype exists)


Query Atlas Module

• BIRN Scientific Interpretation Tool

– Compatible with fMRI and Morphometry Datasets in BIRN-

Standard Formats

• Provides a Link Between Images and Text Databases

– Uses Anatomic Labeling + Ontologies

– Links to Definitions, Publications, Quantifications

– E.g. Wikipedia, PubMed, IBVD, BrainInfo


Query Atlas Features

• Input:.xcat or .qdec from .xar

• Hardware Accelerated

Interactive 3D Annotation

• Ontology Engine

– FreeSurfer, UMLS, BIRNLex,

NeuroNames, IBVD

– Interactive Browser

• Direct Browser Launch to

Search Sites using Selected

Terms


BIRN Population Statistics

• Query, Design, Estimate, Contrast of Population Statistics– XNAT to Select

Subjects– FreeSurfer QDEC

Runs on Server– .QDEC file in .XAR

Web Download– Slicer3 Interactive

Visualization – Integrated with Query

Atlas


43

Community


44

Overall Goals

• Why Medical Research?– To Help Patients!


45

NA-MIC Roots: NIH

• Roadmap Initiative• http://grants1.nih.gov/grants/guide/rfa-files/RFA-

RM-04-003.html– …will create a networked national effort to

build the computational infrastructure for biomedical computing in the nation…

– The establishment of the NIH NCBC was called for in the Biomedical Information Science and Technology Initiative report in 1999


46

NA-MIC Governance

Software Sharing:

http://grants1.nih.gov/grants/guide/rfa-files/RFA-RM-04-003.html • …software should be freely available … • …permit the commercialization of

enhanced or customized versions … • …include the ability of researchers outside

the center and its collaborating projects to modify the source code and to share modifications …


47

NA-MIC Kit Governance

• Free, BSD style Open-Source

• NA-MIC Provides “Reference Standard Distribution”

– Maintain Official Releases

– Avoid Code Forking, Avoid Non-Free Code Creeping into Core

• No “Knowing” Addition of Patented Techniques into NA-MIC Kit

• Technology Base for Suited for both Research and Commercialization

– Okay to build GPL or Closed SW on top of NA-MIC Kit

– But no GPL or Closed SW in NA-MIC Kit

• Emerging community process for contributions http://insight-journal.org• Support Required for Contributions (no “shoot and forget”)

• Copyright to Substantive Works Remains with Creators

– Available under NA-MIC Approved Licenses for Distribution with Kit


48

FOSS – A Public Highway…

• Open-source is like a Public Road System– Provides Infrastructure

for a Variety of Uses– Driveways can Lead to

Anything:• a Public Park• a Private Facility

FOSS= Free Open Source Software



49

NA-MIC Policy

Industry

GPL

NA-MIC Kit

ResearchOnly


50

NA-MIC: A Network of Peers• Leadership:

– BWH: Ron Kikinis, (Overall PI)• Core 1 Algorithms

– Utah: Ross Whitaker (Core 1 PI)– MIT: Eric Grimson– UNC: Guido Gerig– MGH: Bruce Fischl, Dave Kennedy– GaTech: Allen Tannenbaum

• Core 2 Engineering– Kitware: Will Schroeder (Core 2 PI)– GE: Jim Miller– Isomics: Steve Pieper– UCSD: Mark Ellisman– UCLA: Art Toga – WashU: Dan Marcus

• Core 3 DBP 2004-2007– BWH: Martha Shenton– Dartmouth: Andy Saykin– UCI: Steve Potkin– UofT: Jim Kennedy

• DBP 2007– UNC: H. Cody– BWH: M. Kubicki– Mind Institute: J. Bockolt– Queens University: G. Fichtinger

• Core 4 Service– Kitware: Will Schroeder

• Core 5 Training– MGH: Randy Gollub

• Core 6 Dissemination– Isomics: Steve Pieper, Tina Kapur

• Core 7 Management– BWH: S. Manandhar



51

Communication

Daily – e-mail, dashboards, wikis Weekly – telephone conferences Periodic – architecture reviews, workshops Semi-annual– Programmer/Project week Yearly – All Hands Meeting


52

All Hands Meeting


53

Programmer’s Week


NA-MIC Project Week

• June 23-27, 2008– 120 Participants– Dozens of Projects– Hands On Software

Development


55

Dissemination and Training

• National and International Events– MIT, MGH, UNC,

EPFL, NIH, UNM, UCSD…

• All Materials on Wiki• Project Weeks

– Full Week Each Summer

– ½ Week at Winter AHM

• Workshops– MICCAI 2005, 2006,

2007, 2008..– OHBM, RSNA,

Munich, NCI…


Training Courses

• June In Munich– 2 Packed Days, 40 Participants– Cover Spectrum of Use and

Development• Data Visualization• Registration• Tracked Surgical Instruments• Developing Custom Modules

• Upcoming– NIH/NCI, Robarts Inst., Stanford,

MICCAI, RSNA, WashU• > 1,000 Participants Since 2004


NAMIC Training Portfolio http://www.na-mic.org/Wiki/index.php/Slicer:Workshops:User_Training_101

• Google for “Slicer 101”

• Most training material is for Slicer2.x with increasing amounts of Slicer3 content


58

It’s all great, but…

• A few caveats to keep in mind

• It’s a big, distributed group– Good spin: “Wisdom of the Crowds”– Bad spin: “Designed by Committee”

• It’s all research– Constantly evolving– Never enough documentation– Plurality of approaches

• The net result is overwhelmingly positive.


59

Getting Involved

• Winter Project Week 2009– January, Salt Lake City, UT

• Summer Project Week 2009– June at MIT Stata Center, Cambridge MA

• Collaborations PAR-05-063– Automated FE Mesh Development

N. Grosland PI, U Iowa + Isomics, Inc.– Measuring Alcohol and Stress Interaction with Structural and Perfusion MRI

• J. Daunais PI, Wake Forest University and Virginia Polytechnic Institute + K. Pohl BWH

– An Integrated System for Image-Guided Radiofrequency Ablation of Liver Tumors

• K. Cleary PI, Georgetown + N. Hata BWH– http://www.na-mic.org/Wiki/index.php/Collaborator:Resources


Summary

• End-to-End Platform for Translational Image Research• Open Source and Extensible• Standard Methodologies Facilitate Sharing and Communication

60


61

More Information• SPL website:

http://www.spl.harvard.edu

• NA-MIC wiki:

http://wiki.na-mic.org

• 3D Slicer

http://www.slicer.org

na-mic national alliance for medical image computing large-scale computing frameworks for...

Documents

medical image computing

medical image computing

namic national alliance

na mic national alliance

image processing slide

cpack http

community slide

kikinis slide