tools for manipulating jpeg-2000 based whole slide image formats toby cornish, m.d., ph.d
DESCRIPTION
Tools for Manipulating JPEG-2000 Based Whole Slide Image Formats Toby Cornish, M.D., Ph.D. Department of Pathology Johns Hopkins Medical Institutions Baltimore, MD. Overview. JPEG-2000 is a highly-efficient compression standard well-suited to storage of whole slide images - PowerPoint PPT PresentationTRANSCRIPT
Tools for Manipulating JPEG-2000 Based Whole Slide Image Formats
Toby Cornish, M.D., Ph.D.
Department of Pathology
Johns Hopkins Medical Institutions
Baltimore, MD
Overview• JPEG-2000 is a highly-efficient compression
standard well-suited to storage of whole slide images
• JP2 and Aperio SVS are two WSI file formats that employ JPEG2k compression
• Although the file formats are different, the data is essentially stored in the same format
• Few tools exist for manipulating these formats • It should be possible to open, manipulate, and
interconvert these formats using readily available, free and open libraries
JPEG-2000
Compression
Codestreams
File Formats
JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results
File Formats
JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results
JPEG-2000
Compression
Codestreams
JPEG-2000
• A standard for wavelet-based image compression
• Joint Photographic Expert Group as a replacement for JPEG
• ISO/IEC 15444-1 and 15444-2 (parts 1 and 2)
• Part 1 defines a self-contained codestream (JPC) as well as a file format (JP2) that can wrap the codestream
JPEG-2000, cont.
• JPIP (ISO/IEC 15444-9) is a related standard for streaming of large JPEG2k codestreams
• Several encoders/decoders both free and proprietary for JPEG2k– Vary widely in the implementation of the standard, but
most support JPEG2k part 1
• Mostly used for high capacity image storage in videogames, satellite imagery, and, more recently, medical imagery
JPEG-2000, cont.
• Features:– Highly efficient compression
– Native tiling support for large images
– Native multiresolution decompression from a single codestream
– Support for an arbitrary number of image components (i.e. channels)
– JPEG2k codestreams are self-contained and can be embedded in other file formats
File Formats
JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results
JPEG-2000
Compression
Codestreams
JPEG-2k codestreams
• The codestream are capable of indepently being decoded into an image
• All codestreams are tiled, but the simplest and most common form is the single tile codestream
• Single tile images are good for small images
• Multiple tile images better for handling large image data such as WSI
JPEG2k codestream, single tile
1
One codestream – One tile
JPEG2k codestream, single tile
SOC
QCD
COD
SIZ
Main header
Tile header 1
Packet 1
Packet n
SOT
COM
SOD
EOC
start of codestream
image and tile size
coding style defaults
quantization defaults
comments
start of tile
start of data
end of codestream
EOC
Tile data 1
JPEG2kcodestream
JPEG2k codestream, multiple tiles
One codestream – Multiple tiles
11 12
13 14 15 16
1 2 3 4
5 6 7 8
9 10
JPEG2k codestream, multiple tilesSOC
QCD
COD
SIZ
Packet 1
Packet n
SOT
COM
SOD
start of codestream
image and tile sizecoding style defaults
quantization defaultscomments
start of tilestart of data
EOC end of codestream
Packet 1
Packet n
SOTSOD
start of tilestart of data
Main header
Tile header 1
EOC
Tile data 1
JPEG2kcodestream
Tile header n
Tile data n
Design
Access SVS Data
Convert SVS to JP2
Results
JPEG-2000
Compression
Codestreams
File Formats
JP2
Aperio SVS
JP2
• Wrapped JPEG2k codestream
• Composed of mandatory and optional header boxes
• Extensible via UUIDs (Universal Unique Identifiers) and/or XML
JP2 file format, cont.
JPEG2k signature box
File type box
JP2 header box
Contiguous codestream box
JPEG2k codestream
XML boxes
UIUD boxes
Main header
Tile header 1
EOC
Tile data 1
Tile header n
Tile data n
Design
Access SVS Data
Convert SVS to JP2
Results
JPEG-2000
Compression
Codestreams
File Formats
JP2
Aperio SVS
Aperio SVS file format• Developed by Aperio, Inc. (San Diego, CA)
• Essentially, a TIFF file with a tiled layout
• Tile data are JPEG2k codestreams
• Can consist of multiple images (pages) derived from slide scanning with one Image File Directory (IFD) per image
• Image metadata is stored in Tags
• The highest resolution data is always tiled
SVS/TIFF structure
ImagePyramid
Thumbnail
Label
Macro
SVS/TIFF structure, cont.Byte order
Version number
IFD offset
Tag entry count
Tag 1
Tag n
IFD 1
Image data
IFD 2
IFD 3
IFD 4
IFD 5
IFD 6
Tag entry count
Tag 1
Tag n
Image data
SVS/TIFF Tiling
Multiple tiles – one codestream per tile
11 12
13 14 15 16
1 2 3 4
5 6 7 8
9 10
SVS/TIFF tiling, cont.
IFD 1
Tile 1
Tile n
Tile 2
Tile 3
Tiff Tiles
JPEG2k codestream
Tag entry count
Tag 1
Tag n
Image data
Main header
Tile header 1
EOC
Tile data 1
SVS/TIFF v JP2• Differences:
– JP2 contains one JPEG2k codestream, which contains multiple tiles
– SVS/TIFF contains multiple tiles, each of which contains one codestream with exactly one tile
• Similarities:– The tile data is essentially the exact same, but packaged
– Should permit direct copying of tile data between formats
Design
Access SVS Data
Convert SVS to JP2
JPEG-2000
Compression
Codestreams
File Formats
JP2
Aperio SVS
Results
Design
• Open source libraries (LibTiff, ImageMagick/Jasper)
• C programming language for portability
• First two goals:
– Read SVS/TIFF image data using custom software
– Convert SVS/TIFF files to JP2 without decompressing
Read SVS/TIFF
Tile 1
Tile n
Tile 2
Tiff Tiles
Main headerTile header
EOC
Tile data 1
Main headerTile header
EOC
Tile data 2
Main headerTile header
EOC
Tile data n
IFD
SVSImage data
TIFF Header
Decode
Decode
Decode
SVS/TIFF to JP2 conversion
Tile 1
Tile n
Tile 2
Tiff TilesIFD
SVSImage data
TIFF Header
Main headerTile header
EOC
Tile data 1
Main headerTile header
EOC
Tile data 2
Main headerTile header
EOC
Tile data n
Main headerTile header
EOC
Tile data 1
Tile header
Tile data 2
Tile header
Tile data 2
JPEG2kCodestream
JP2 Header
Mergedcodestream
Issues• SVS format “quirks”
– Somewhat unusual component subsampling
• Limits the number of decoders that will read the data without complaining
– SVS tiles are transformed into YCrCb color space
• Requires a transform to RGB color space before use
• A compliant JP2 reader should do this, but not all do
Results
JPEG-2000
Compression
Codestreams
File Formats
JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results
• Several tools were created, allowing:
• Direct access to SVS tile data– Manipulate, extract, and montage tiles– See LOCI Bioformats project
• Direct conversion of SVS files to JP2– Requires the SVS tile dimensions to be multiples of 64
– Embedded ICC profiles not propogated
Future work• Implement software in Java
• Create a JP2 SVS/TIFF converter
• Investigate limitations: – Why must tile dimensions be multiples of 64?– Why do most decoders (notably the JAI decoders) fail to
decode JPEG2k codestreams from SVS?
• Include ICC profiles
• Develop a generic LibTIFF plugin for encoding/decoding JPEG2k image data
AcknowledgementsJohns HopkinsKristen Lecksell
University of Wisconsin-Madison, LOCIMelissa LinkertCurtis Rueden
ReferencesTaubman and Marcellin, JPEG2000 Image Compression
Fundamentals, Standards and PracticeAperio Technical Docs (http://www.aperio.com/documents/)
LibrariesLibTIFF (http://www.remotesensing.org/libtiff/)ImageMagick (http://www.imagemagick.org/script/index.php)Jasper (http://www.ece.uvic.ca/~mdadams/jasper/)