na-mic national alliance for medical image computing 3d slicer modules and extensions steve pieper,...

NA-MICNational Alliance for Medical Image Computing http://na-mic.org

3D Slicer

Modules and Extensions

Steve Pieper, Ph.D.

National Alliance for Medical Image Computing http://na-mic.org

How Modules Should Work• Slicer 3.4 is a Stable Base for Many Medical

Image Computing Apps– Core File I/O, Visualization, Interface…– Tutorials and Other Resources to Get New Users Up to

Speed• Developers of New Techniques can Draw on the

NA-MIC Kit to Implement Compatible Modules– Consistent Framework for Deployment and User

Interface– Several Styles of Development to Meet Varied

Requirements

National Alliance for Medical Image Computing http://na-mic.org

Provided by D. Gering

Slicer3 data model

Provided by R. Kikinis

National Alliance for Medical Image Computing http://na-mic.org

Slicer3 “Observer MVC” Pattern• MRML (Model)

– For Scene Description and Application State– MRML Nodes are Persistent and Undoable– Scene and Nodes are Observable

• Logic Encapsulate VTK and ITK Pipelines (Controller)– Observe MRML to Configure Pipelines– Help Create/Manage Nodes– No UI Components (no Widgets, Actors,

Mappers, Renderers or RenderWindows)• GUI (View)

– Observe and Edit MRML– Interact with User and Display Hardware

• Modules Should Follow Same Conventions

Logic

MRML Nodes

GUI

Widgets Renderers

EditObserve

Observe

Observe

Edit

Edit

“Observe” means generic event mechanisms are used to pass information.“Edit” means code can directly call methods.

Example: GUI can call methods in Logic classes,but Logic cannot call GUI methods.MRML cannot call Logic or GUI methods.

There can be many observers for any event.

National Alliance for Medical Image Computing http://na-mic.org

Variety of levels of integrationSlicer Libs

• ModuleDescriptionParser• GenerateCLP• vtkITK• MRML

Non-slicer specific support libraries

Slicer Base • Application logic• Widgets

Common infrastructure for Slicer applications

Built in modules• Slice viewers• Models• Fiducials• Transforms

Full access to Slicer internals

Loadable modules

• Query Atlas• QDEC• Volume rendering• ChangeTracker• EMSegment

Full access to Slicer internals

Scripted modules• Editor• Teem Two Tensor Tractography• VMTK

Limited access to Slicer internals

Command line modules • Registration Restricted access to Slicer internals

Daemon • OpenIGTLink• Stochastic Tractography Access to MRML

National Alliance for Medical Image Computing http://na-mic.org

Base Features vs. ModulesFeatures Base ModulesVisualization MRML: Models, Volumes, Volume Rendering, Fiducials,

Continuum Meshes, Labeled DataCan create custom MRML Nodes and behavior

Filtering None Implemented as Modules using ITK or other Libraries

Registration Transform Display and Edit, Save/Restore Calculate Transforms, Resample Data

Segmentation Label Maps, Parcellated Surfaces Segmentation Algorithms in ITK or other Libraries

Quantification Label, Image, Volume Statistics; Numpy access to MRML

Applications in Python or MATLAB

Real-time Integration VTK Rendering, KWWidgets framework, Tracker Support (as Transforms)

Direct Manipulation of the MRML Scene; 2D/3D Widgets; Device Interfaces, OpenIGTLink Module

Diffusion Imaging DWI, DTI, Fiber Bundles Tractography, Clustering, Atlases

Applications “Bundles” of Modules in Distribution: Registration, Editor, some Filters,

Customized Extensions, Domain-specific code, Optimized Interfaces…

National Alliance for Medical Image Computing http://na-mic.org

Each module has …… an entry in the module menu

… a panel of user interface controls

National Alliance for Medical Image Computing http://na-mic.org

Built in module

vtkMRMLNode.h vtkSlicerModuleGUI.h vtkSlicerModuleLogic.h

vtkMRMLYourModuleNode.h•GetNodeTagName()•Copy()•ReadXMLAttributes()•WriteXML()

vtkSlicerYourModuleGUI.h•BuildGUI()•ProcessGUIEvents()•ProcessLogicEvents()•ProcessMRMLEvents()

vtkSlicerYourModuleLogic.h•ProcessMRMLEvents()•ProcessLogicEvents()

Slicer provided

Module writer provided

National Alliance for Medical Image Computing http://na-mic.org

Loadable module

vtkMRMLNode.h vtkSlicerModuleGUI.h vtkSlicerModuleLogic.h

vtkMRMLYourModuleNode.h•GetNodeTagName()•Copy()•ReadXMLAttributes()•WriteXML()

vtkSlicerYourModuleGUI.h•BuildGUI()•ProcessGUIEvents()•ProcessLogicEvents()•ProcessMRMLEvents()

vtkSlicerYourModuleLogic.h•ProcessMRMLEvents()•ProcessLogicEvents()

YourModule.so•GetLoadableModuleDescription()•GetLoadableModuleLogic()•GetLoadableModuleGUI()

Slicer provided

Module writer provided

National Alliance for Medical Image Computing http://na-mic.org

Tcl scripted module

vtkMRMLScriptedModuleNode.h

vtkSlicerScriptedModuleGUI.h

vtkSlicerScriptedModuleLogic.h

YourModuleNode.tcl YourModuleGUI.tcl•proc YourModuleBuildGUI•proc YourModuleProcessGUIEvents•proc YourModuleProcessLogicEvents •proc YourModuleProcessLogicEvents

YourModuleLogic.tcl

YourModule.tcl•package provide YourModule x.y

Slicer provided

Module writer provided

National Alliance for Medical Image Computing http://na-mic.org

Python scripted module

vtkMRMLScriptedModuleNode.h

vtkSlicerScriptedModuleGUI.h

vtkSlicerScriptedModuleLogic.h

YourModuleGUI.py•def BuildGUI()•def ProcessGUIEvents()•def ProcessLogicEvents() •def ProcessLogicEvents()

Slicer provided

Module writer provided

YourModuleNode.py• Optional State Management

YourModuleLogic.py• Optional Helper Code

National Alliance for Medical Image Computing http://na-mic.org

Command line module

vtkMRMLCommandLineModuleNode.h

vtkCommandLineModuleGUI.h

vtkCommandLineModuleLogic.h

YourModule.xml•<image>•<geometry>•<integer>•<double>•<file>

YourModule.so•XMLModuleDescription•GetXMLModuleDescription()•ModelEntryPoint()

Slicer provided

Module writer provided

YourModule (exe)•YourModule –xml

YourModule.py•XML•def toXML()•def Execute()

or or

National Alliance for Medical Image Computing http://na-mic.org

All modules are created equal…

… an entry in the module menu

… a panel of user interface controls

National Alliance for Medical Image Computing http://na-mic.org

Daemons

slicerget.tcl name > image.nrrd

cat image.nrrd | slicerput.tcl name

YourTool(exernal process such as

MATLAB)

OpenIGTLink Protocol YourTool(exernal process such as

Tracker or Scanner)

SlicerDaemon

OpenIGTLinkIF

National Alliance for Medical Image Computing http://na-mic.org

Variety of levels of integrationSlicer Libs

• ModuleDescriptionParser• GenerateCLP• vtkITK• MRML

Non-slicer specific support libraries

Slicer Base • Application logic• Widgets

Common infrastructure for Slicer applications

Built in modules• Slice viewers• Models• Fiducials• Transforms

Full access to Slicer internals

Loadable modules

• Query Atlas• QDEC• Volume rendering• ChangeTracker• EMSegment

Full access to Slicer internals

Scripted modules• Editor• Teem Two Tensor Tractography• VMTK

Limited access to Slicer internals

Command line modules • Registration Restricted access to Slicer internals

Daemon • OpenIGTLink• Stochastic Tractography Access to MRML

National Alliance for Medical Image Computing http://na-mic.org

Extension Goals• Keep the base package “lean and mean”• Modules have individual identity

– Per-module web site, svn, downloads, mailing lists, wiki…• Allow users to assemble their own set of tools

– Customized ‘Bundles’ by task or application• Easy to download compatible extensions

– Analogous to Firefox extensions– Integrate extension builds into developer/nightly/release process

• NITRC Supplement to NA-MIC helping to pay for needed infrastructure (Neuroimaging Informatics Tools and Resources Clearinghouse)

– NITRC can host neuroimaging projects (gforge implementation) • Other repositories can also be used

– NAMICSandBox, Institution’s svn, Google Code, etc…

National Alliance for Medical Image Computing http://na-mic.org

Extension Management Wizard*

* Under development

National Alliance for Medical Image Computing http://na-mic.org

NITRC

http://www.nitrc.org/projects/slicer3examples/Sample Code for CommandLine, Loadable, and Scripted Extension Modules

National Alliance for Medical Image Computing http://na-mic.org

Extension Process

• Write your module following the examples on nitrc.org

• Create .s3ext description file– http://viewvc.slicer.org/viewcvs.cgi/trunk/Extensions/

– Svn path, dependencies, homepage, category, status, description…

• Nightly builds create compatible sets of extension binaries that correspond to the slicer builds (windows, linux, mac)– Correct lib versions, compiler versions, etc

• Users can install from Extension Manager

National Alliance for Medical Image Computing http://na-mic.org

CTK Issues

• Can some or all Slicer Module types be interoperable with other CTK Tools– Automatic Adaptors?

• What will be the CTK Preferred Module Implementation?– Web Services? daemons? OSGi?...