QIBA CT Volumetrics - QIBA CT Volumetrics - Cross-Platform Study (Group 1C) Cross-Platform Study (Group 1C)

September 2, 2009September 2, 2009

Interclinical Comparison of CT VolumetryInterclinical Comparison of CT Volumetry

Quantitative Imaging Biomarker AllianceQuantitative Imaging Biomarker Alliance

Dr. Charles Fenimore, NISTDr. Charles Fenimore, NIST

ChargeCharge1.1. To agree on the To agree on the scanner settings and other scanner settings and other

protocol elementsprotocol elements under which imagery is to under which imagery is to be collected.be collected.

2.2. To agree on requirements of To agree on requirements of phantomsphantoms to be to be imaged/measured.imaged/measured.

3.3. To agree on the To agree on the platforms and centersplatforms and centers to be to be selected for imagery collection.selected for imagery collection.

4.4. To agree on the site(s) and methods for the To agree on the site(s) and methods for the reader studyreader study..

5.5. To identify the To identify the measurements and the measurements and the algorithmsalgorithms for use in image processing. for use in image processing.

6.6. To To specify the analysisspecify the analysis of the measurements. of the measurements.

GoalsGoals

1.1. Measure the volume of nodules on CT imagery collected Measure the volume of nodules on CT imagery collected from several CT scanners and sites. Protocol is to be from several CT scanners and sites. Protocol is to be defined in relation to QIBA Vol CT 2 specifications. Protocol defined in relation to QIBA Vol CT 2 specifications. Protocol is to include a branch specifying performance.is to include a branch specifying performance.

2.2. Measure image noise and other image quality factors and Measure image noise and other image quality factors and determine their impact on the measurement of volume.determine their impact on the measurement of volume.

3.3. Analyze the Analyze the accuracy and precisionaccuracy and precision of volume of volume measurements for all design factors including: site, device, measurements for all design factors including: site, device, scanner settings, & reader/algorithm/software.scanner settings, & reader/algorithm/software.

4.4. Determine the Determine the minimum detectable level of changeminimum detectable level of change that that can be achieved when measuring nodules in can be achieved when measuring nodules in phantom phantom datasetsdatasets..

Goals 1 and 2Goals 1 and 2

1.1. Measure the volume of nodules on CT Measure the volume of nodules on CT imagery collected from several CT imagery collected from several CT scanners and sites, etc.scanners and sites, etc.

2.2. Measure “image noise” and determine its Measure “image noise” and determine its impact on the measurement of volume, impact on the measurement of volume, to facilitate scanner comparison.to facilitate scanner comparison.

Realization of these goals is discussed in the next few slides.Realization of these goals is discussed in the next few slides.

QIBA CT 1-C ProtocolQIBA CT 1-C Protocol

AcquisitionAcquisition– The phantomThe phantom– Image datasetsImage datasets– The imaging sitesThe imaging sites– System parametersSystem parameters– Markup sitesMarkup sites

QIBA CT 1-C Acquisition ProtocolQIBA CT 1-C Acquisition ProtocolAcquisition phantom:Acquisition phantom:

FDA lung phantom + NIST pocket phantomFDA lung phantom + NIST pocket phantom– The FDA phantom accomodates 8 attached nodules, some are 20 The FDA phantom accomodates 8 attached nodules, some are 20

mm in size.mm in size. Left: 10 & 20 mm spheres @ - 10 & +100 HULeft: 10 & 20 mm spheres @ - 10 & +100 HU Right: 10 & 20 mm spiculated @ - 10 & +100 HU, 10 mm Right: 10 & 20 mm spiculated @ - 10 & +100 HU, 10 mm

ovoid, lobulated, spiculatedovoid, lobulated, spiculated Graded nodules coming @ 9 mm – giving 5, 8, 9, 10 mm.Graded nodules coming @ 9 mm – giving 5, 8, 9, 10 mm.

- The - The NIST pocket phantom is a free standing Lego structure with NIST pocket phantom is a free standing Lego structure with teflon balls ~ 6 mm.teflon balls ~ 6 mm.

QIBA CT 1-C imaging sitesQIBA CT 1-C imaging sitesPotential imaging sitesPotential imaging sites

– FDA, Philips 16 row detectorFDA, Philips 16 row detector– Nick Petrick / Marios Gavrelides, contactNick Petrick / Marios Gavrelides, contact

– Duke,Duke,– Ehsan Samei, Division Chief, Clinical Imaging PhysicsEhsan Samei, Division Chief, Clinical Imaging Physics– 1 GE LightSpeed VCT (64 slice), 4 GE LightSpeed 16 (16 slice, 1 1 GE LightSpeed VCT (64 slice), 4 GE LightSpeed 16 (16 slice, 1

with fluoroscopic capability)with fluoroscopic capability) – Johns Hopkins – Siemens, ToshibaJohns Hopkins – Siemens, Toshiba

– Mahadevappa Mahesh, Chief PhysicistMahadevappa Mahesh, Chief Physicist

– UM-MC, Sensation 64 Siemens interestedUM-MC, Sensation 64 Siemens interested– Eliot Seigel / Joseph Chen, radiologistsEliot Seigel / Joseph Chen, radiologists

– UCLA UCLA – – Mike McNitt GrayMike McNitt Gray

– Siemens (Germany), offer of multiple scannersSiemens (Germany), offer of multiple scanners– Mathias ThornMathias Thorn

– MSKCCMSKCC– Rush Rush

CROCRO– Rad Pharm – Initiated discussion of markup.Rad Pharm – Initiated discussion of markup.– Additional sites: Perceptive Informatics, Biomedical Systems Additional sites: Perceptive Informatics, Biomedical Systems

QIBA CT 1-C ProtocolQIBA CT 1-C Protocol11stst branch = a standard protocol branch = a standard protocol

Scan 2 - 3 times. No more the one or two are to be Scan 2 - 3 times. No more the one or two are to be read.read.

Use the developing QIBA protocol (akin to ACRIN Use the developing QIBA protocol (akin to ACRIN 6678) to specify kVp, slice thickness, mAs, 6678) to specify kVp, slice thickness, mAs, rotation time, pitch, reconstruction kernel (affects rotation time, pitch, reconstruction kernel (affects MTF). MTF). Include water and ACR phantoms Include water and ACR phantoms to to characterize the resolution and noise levels under characterize the resolution and noise levels under this branch of the protocol.this branch of the protocol.

Sample Protocol Chart for ACRIN 6678Sample Protocol Chart for ACRIN 6678

QIBA CT 1-C ProfilesQIBA CT 1-C Profiles22ndnd branch = performance specified branch = performance specified

Specify easily implemented PERFORMANCE metrics of Specify easily implemented PERFORMANCE metrics of resolution and noise.resolution and noise.

Guidance provided on:Guidance provided on: kVp (affects contrast difference between materials)kVp (affects contrast difference between materials) Slice thickness, recon interval (affects z-axis resolution & Slice thickness, recon interval (affects z-axis resolution &

noise)noise) Rotation time and pitch (coverage, breath hold, etc.)Rotation time and pitch (coverage, breath hold, etc.)

Performance required on:Performance required on: Recon kernel performance - EXAMPLE: Recon kernel performance - EXAMPLE:

– Choose kernel such that you can see 6 or 7 (but no more than 7) Choose kernel such that you can see 6 or 7 (but no more than 7) lp/cm on ACR phantom….orlp/cm on ACR phantom….or

– 10% MTF should be between 6 and 7 lp/cm10% MTF should be between 6 and 7 lp/cm mA level performancemA level performance

– Choose effective mAs level so that std dev is between 20 and 30 Choose effective mAs level so that std dev is between 20 and 30 HU in a 20 cm water phantomHU in a 20 cm water phantom

QIBA CT 1-C Markup ProcedureQIBA CT 1-C Markup Procedure

Adapt the Rad-Pharm process used in QIBA Adapt the Rad-Pharm process used in QIBA volume-CT 1-A and 1-B to acquire RECIST volume-CT 1-A and 1-B to acquire RECIST (1D), WHO (2D), & volume (3D) reads.(1D), WHO (2D), & volume (3D) reads. 5 - 6 readers5 - 6 readers 2 reads for each nodules (one repeat).2 reads for each nodules (one repeat).

– Can we avoid repeat reads for the full set of series?Can we avoid repeat reads for the full set of series? The 1A study may provide prior estimates of intra- The 1A study may provide prior estimates of intra- and inter-rater variability.and inter-rater variability.

– Is it possible to reduce the number of reads?Is it possible to reduce the number of reads?

QIBA CT 1-C ProfileQIBA CT 1-C ProfileNotional sizing of studyNotional sizing of study

4 sites; 4 sites;

2 branches to protocol (QIBA-2 profile & 2 branches to protocol (QIBA-2 profile & performance-based);performance-based);

8 nodules8 nodules

2 – 3 acquisitions, of which 1 or 2 are read2 – 3 acquisitions, of which 1 or 2 are read

2 repeat reads2 repeat reads

Full factorial design implies 256 nodule reads per Full factorial design implies 256 nodule reads per radiologist - too large. Aim for less than full-radiologist - too large. Aim for less than full-factorial design.factorial design.

Goal 3Goal 3

Compare the Compare the accuracy and precisionaccuracy and precision of of radiologists’ measurements of radiologists’ measurements of RECIST and Volume for these RECIST and Volume for these phantom datasets. (image mask?)phantom datasets. (image mask?)

a)a) RECISTRECIST vs. vs. volume.volume.b)b) Investigate Investigate variance & biasvariance & bias..c)c) Inter-system variation.Inter-system variation.d)d) Intra-system variation.Intra-system variation.

Goal 4Goal 4

4. Determine the 4. Determine the minimum detectable minimum detectable level of changelevel of change that can be achieved that can be achieved when measuring nodules in when measuring nodules in phantom phantom datasetsdatasets..

Required resourcesRequired resources

Use of Use of FDA phantom, water and ACR FDA phantom, water and ACR phantoms. Also NIST pocket phantom.phantoms. Also NIST pocket phantom.

Select clinical image Select clinical image collection sitescollection sites through QIBA-CT group: Offers from FDA, through QIBA-CT group: Offers from FDA, Duke, John’s Hopkins. Other possible Duke, John’s Hopkins. Other possible sites: UMMC, MSKCC, Siemens.sites: UMMC, MSKCC, Siemens.

Mark up similar to QIBA CT 1-A Mark up similar to QIBA CT 1-A proceduresprocedures at Rad-Pharm. Generates at Rad-Pharm. Generates

RECIST (1D)RECIST (1D) WHO (2D)WHO (2D) Segmented volume.Segmented volume.

Next stepsNext steps

Refine Questions and Experimental Design.Refine Questions and Experimental Design. Complete clinic recruitment. Develop Complete clinic recruitment. Develop

protocol with associated medical physicists.protocol with associated medical physicists. Confirm availability and arrange reading with Confirm availability and arrange reading with

Rad-Pharm. Other CROs are possible. Rad-Pharm. Other CROs are possible. Schedule use of FDA phantom.Schedule use of FDA phantom.