stephen r. aylward, director of medical imaging, kitware
DESCRIPTION
Stephen R. Aylward, Director of Medical Imaging, Kitware. Overview. Open-source toolkits Medical image processing (ITK) Image-guided surgery (IGSTK) Microscopy (FARSIGHT) Image display (VTK) Information visualization (VTK) Scientific computing (VTK) Computer vision (VidTK) Open-API - PowerPoint PPT PresentationTRANSCRIPT
Stephen R. Aylward, Director of Medical Imaging, Kitware
CTK Workshop Heidelberg, June 29/30, 2009
Overview
Open-source toolkits Medical image processing (ITK)
Image-guided surgery (IGSTK) Microscopy (FARSIGHT)
Image display (VTK) Information visualization (VTK) Scientific computing (VTK) Computer vision (VidTK)
Open-API Databases (MIDAS)
Open-source applications Medical images
(VolView) Scientific data
(ParaView)
NEW! Open-source application
building ITK, VTK, Qt (Maverick)
CTK Workshop Heidelberg, June 29/30, 2009
Overview
Open-source toolkits Medical image processing (ITK)
Image-guided surgery (IGSTK) Microscopy (FARSIGHT)
Image display (VTK) Information visualization (VTK) Scientific computing (VTK) Computer vision (VidTK)
Open-API Databases (MIDAS)
Open-source applications Medical images
(VolView) Scientific data
(ParaView)
NEW! Open-source application
building ITK, VTK, Qt (Maverick)
CTK Workshop Heidelberg, June 29/30, 2009
IGSTK Features
PI: Kevin Clary, Georgetown (NIH) Toolkit for application builders
Building blocks for C++ programmers Trackers, display layouts, scenes, …
Scene graph based on ITK’s SpatialObjects Objects are tied to trackers Displays show objects in the scenes
State machine for application control All events, not just GUI
Tracker coordinates, type of data loaded, tracker unplugged
Patient safety is a top priority State machine can be visualized and validated
Confirm application never enters an invalid state Some cost added for application developers.
CTK Workshop Heidelberg, June 29/30, 2009
FARSIGHT Features
PI: Badri Roysam, RPI (NIH) Toolkit for biologists
Python scripting 3D/4D Image analysis and visualization
Scene graph based on ITK’s SpatialObjects Arrangement of cells wrt vasculature Scene spans time and space
Flexibility is a top priority New methods for processing and display
Created by biologists (scripts) and computer scientists (new ITK/VTK methods)
CTK Workshop Heidelberg, June 29/30, 2009
Information Visualization using VTK (OverView/ParaView)
Figure 3 - The OverView application displaying a large graph linking miRNA sequences with the proteins they affect.
Figure 4 - The OverView informatics application is shown retrieving PubMed articles from a search string ("HIV"), clustering them, and displaying the articles in geospatial, graph, landscape, and tabular views.
CTK Workshop Heidelberg, June 29/30, 2009
Maverick
PI: Kitware (AFRL) Toolkit for application development
Qt modules for C++ programmers “Qt designer” to define applications
Signals / slots (GUI-driven)
Scene graph Memory (mavScene – SpatialObjects / MRML) Disk (MRML) Display (mavSceneViewers)
Extensible Slicer Execution Model
GUI (User) is the priority Workflow of interdependent modules (maximize user-interaction re-use) Consistent look-and-feel (GUI design standard)
CTK Workshop Heidelberg, June 29/30, 2009
Roadmap (dreams / milestones)
Maverick is released (weeks) FARSIGHT and IGSTK momentum continues Slicer converted to Qt and supports multiple workflows …CTK evolves and is integrated into above systems
CTK Workshop Heidelberg, June 29/30, 2009
What's most important for open-source?
NO DUPLICATION!- Specialization for an audience
Community support License Transparent Value added User support
Funding beyond delivery
Pick ONE audience Clinical / biomedical research
1. Extensible application 2. Custom workflow applications
Application builder
Algorithms research Scripting
Pharma Integration into existing
workflows
CTK Workshop Heidelberg, June 29/30, 2009 What's most important for open-source medical imaging applications?
Medical imaging We are processing organs,
tumors, vessels; not images. Portable data on disk
Crosses applications Patient record
Data representation in memory ITK Spatial Objects (scene graph)
IsInside ValueAt Transform
GUI scene Present using intuitive names,
hierarchy, etc.
Why don’t commercial CAD companies fear open-source CAD efforts Integration / turn-key GUI Patents Clinical testing Outreach Cross-institution distribution
These are out targets
CTK Workshop Heidelberg, June 29/30, 2009
Dreams
Imagine… Custom application from my institute walks me through a workflow for planning
a brain tumor biopsy procedure Save scene Load scene into Slicer to merge with new DTI data Save scene Load scene into surgical suite running IGSTK (robotics) Biopsy pre-chosen sites Send biopsy to lab Lab processes using a new research module loaded into their FARSIGHT-based
application Multiple biomarkers (multiple scales: MRI, DTI, cell, proteomics) are compared
with a database of cases to predict best course of treatment. Radiation oncology program loads scene and biopsy results...
CTK Workshop Heidelberg, June 29/30, 2009
Reality
Legacy applications, formats, data, expertise, … Legacy funding… Good news: different audiences
Opportunity for sharing
Solution: CTK Scene definition (Disk and Memory) IO Library for scene Scene converters for other types of scenes/data API and classes for methods that will run in Slicer, MITK, OpenMAF, Maverick,
XIP, ...
CTK Workshop Heidelberg, June 29/30, 2009
CTK Implementation
Scenes (Disk, Memory, Display) IGSTK / ITK / MITK (SpatialObjects) Slicer (MRML) XIP (OpenInventor) OpenSG XNAT
Modular methods Slicer Execution Model
Implement as ITK/VTK command-line programs Automatically come available as shared libraries Passes events to parent program Missing: interaction with parent GUI (mouse clicks)
Can be done, but no standard set
GUI Modules?