INSTITUTE OF PHYSICS PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 51 (2006) 2029–2043 doi:10.1088/0031-9155/51/8/005

A hyperspectral fluorescence system for 3D in vivooptical imaging

Guido Zavattini1, Stefania Vecchi1, Gregory Mitchell1, Ulli Weisser1,Richard M Leahy2, Bernd J Pichler1,3, Desmond J Smith4 andSimon R Cherry1

1 Department of Biomedical Engineering, University of California-Davis, One Shields Avenue,Davis, CA 95616, USA2 Signal and Image Processing Institute, University of Southern California, Los Angeles, CA,USA3 Department of Radiology, University of Tubingen, Tubingen, Germany4 Department of Molecular and Medical Pharmacology, UCLA School of Medicine, Los Angeles,CA, USA

E-mail: srcherry@ucdavis.edu

Received 27 July 2005, in final form 15 December 2005Published 4 April 2006Online at stacks.iop.org/PMB/51/2029

AbstractIn vivo optical instruments designed for small animal imaging generallymeasure the integrated light intensity across a broad band of wavelengths, ormake measurements at a small number of selected wavelengths, and primarilyuse any spectral information to characterize and remove autofluorescence. Wehave developed a flexible hyperspectral imaging instrument to explore the use ofspectral information to determine the 3D source location for in vivo fluorescenceimaging applications. We hypothesize that the spectral distribution of theemitted fluorescence signal can be used to provide additional information to 3Dreconstruction algorithms being developed for optical tomography. To test thishypothesis, we have designed and built an in vivo hyperspectral imaging system,which can acquire data from 400 to 1000 nm with 3 nm spectral resolution andwhich is flexible enough to allow the testing of a wide range of illumination anddetection geometries. It also has the capability to generate a surface contourmap of the animal for input into the reconstruction process. In this paper,we present the design of the system, demonstrate the depth dependence ofthe spectral signal in phantoms and show the ability to reconstruct 3D sourcelocations using the spectral data in a simple phantom. We also characterize thebasic performance of the imaging system.

0031-9155/06/082029+15$30.00 © 2006 IOP Publishing Ltd Printed in the UK 2029

2030 G Zavattini et al

1. Introduction

The ability to image specific genes, molecular targets and molecular pathways in vivo ishaving a profound impact in preclinical research designed to develop new animal modelsof human disease and to evaluate new therapeutic and diagnostic strategies. Several imagingmodalities, notably PET, MRI and optical imaging, have been adapted to facilitate these studiesusing targeted or activatable imaging probes that produce signals roughly proportional to theregional molecular abundance or the rate of specific events in molecular pathways (Weisslederand Mahmood 2001, Cherry 2004). The absorbing and scattering properties of tissue in thevisible and near-infrared (NIR) spectrum limit applications of optical techniques in humansto superficial tissue, and consequently PET and MRI are arguably the preferred modalitiesfor ultimate clinical application. However, in preclinical animal models such as mice, theshorter path lengths allow sufficient photon flux to reach the surface of the animal (especiallyin the NIR part of the spectrum ∼650–950 nm) enabling fluorescence and bioluminescenceemissions to be detected with good signal-to-noise ratios (Troy et al 2004) and using relativelylow-cost instrumentation. Optical imaging is therefore the most commonly used modality forsmall animal in vivo molecular imaging at the present time (Ntziachristos et al 2005).

Optical molecular imaging systems have been developed that utilize both bioluminescence(Rice et al 2001) or fluorescence (Graves et al 2003) signals. Bioluminescent systems typicallyuse the firefly luciferase gene as a reporter (Contag and Bachmann 2002). Oxidation ofluciferin, which is injected into the animal, leads to the emission of light at around ∼610 nmand occurs only in cells in which the luciferase enzyme is expressed by the reporter gene.The major attraction of this approach is that although absolute light levels are low, signalis produced only where luciferase is present, leading to extremely low background signals(Troy et al 2004). In contrast, fluorescence-based imaging requires an external light source tostimulate the emission of light from the probe. Fluorescence imaging usually results in largersignals, but they are also contaminated by autofluorescence throughout the animal.

There are several factors that also make the use of fluorescent probes an appealing approachfor molecular optical imaging: (i) fluorescent reporter proteins are available for imaging celltrafficking, infection, gene delivery and gene expression in transgenic animals (Campbellet al 2002), and fluorescent contrast agents are available for receptor imaging and for molecularimaging of cellular events using locally activatable ‘smart probes’ (Mahmood et al 1999);(ii) signal strength is larger than in bioluminescent systems allowing the use of less expensivecameras and (iii) results are readily cross-validated with traditional ex vivo methods usingfluorescence microscopy.

A typical arrangement for in vivo fluorescence imaging involves placing the animal in ablack box, illuminating with a laser or broadband source and imaging with a CCD camera.Excitation and fluorescent light are differentiated through the use of frequency selectivefilters. The resulting data are a 2D map of the fluorescence intensity at the surface of theanimal. While this leads to simple, high-throughput imaging, in which fluorescent signalsemanating from small volumes of tissue can clearly be detected, these surface maps providelimited information on source location and make quantification of the source strength and datainterpretation difficult in all but the simplest models (e.g., tumour xenograft models where thelikely signal location is known a priori). Meaningful longitudinal studies are also hamperedby the need to precisely reposition the animal so that the same 2D projection is collected eachtime.

For these reasons, there is tremendous interest in developing truly 3D tomographic opticalimaging approaches for in vivo small animal imaging, in which the location and amplitude offluorescent signals can be determined and more complex spatial distributions of fluorescent

3D hyperspectral fluorescence imaging system 2031

sources within the animal can be unfolded. These problems can be directly addressed throughdevelopment of a 3D tomographic system that uses the light collected from the animal toreconstruct the 3D fluorophore distribution.

Despite the complexities of tissue-specific absorption and scattering properties, andthe fact that photons typically undergo several hundred scatters before leaving the animal(Yao and Wang 1999), significant progress has been made in developing instrumentationand reconstruction algorithms, both for 3D fluorescence imaging e.g. (Ntziachristos andWeissleder 2001, Graves et al 2003, Chen et al 2000) and for 3D bioluminescenceimaging (Chaudhari et al 2005, Gu et al 2004, Wang et al 2004). Nonetheless, accuratereconstructions have generally been limited to fairly simple cases. It is likely that furtherimprovements can be made by introducing additional information into the reconstructionproblem. Two obvious opportunities are to use temporal (frequency domain) modulation ofthe excitation light in fluorescence imaging e.g. (Roy and Sevick-Muraca 2001, Godavartyet al 2003), or, as we are proposing in this paper, to use spectral information to aid in 3Dlocalization (Zavattini et al 2003, Swartling et al 2005) and tomographic reconstruction. Wehypothesize that introducing spectral information can significantly aid in 3D fluorescenceimaging in three ways: (1) for relatively broad emitters such as traditional fluorophoresand fluorescent proteins, spectral information provides information on source depth dueto the strong wavelength dependence on absorption and scattering properties of tissue.Generally, shorter wavelengths will be much more readily absorbed than longer (near-infrared) wavelengths, hence spectra are altered based on the amount of tissue light mustpass through to reach the surface. (2) Using narrow emitters such as quantum dots (Gaoet al 2004), spectral information may allow highly multiplexed in vivo imaging; (3) asothers have shown for 2D in vivo imaging applications, spectral information is importantin effectively removing signal components arising from tissue autofluorescence (Levensonand Hoyt 2000).

In this paper, we present the development of an in vivo hyperspectral fluorescence imaginginstrument and show that spectral information can be used for depth discrimination. Bycombining data from this instrument with appropriate reconstruction algorithms (that modelmouse geometry, tissue-dependent optical properties as a function of wavelength, distributionof excitation light within the animal and detection of emission light), we believe that wecan ultimately improve the reconstruction of 3D distributions of fluorescent reporter genesor contrast agents. A number of other groups have investigated biomedical applications ofhyperspectral imaging (Ford et al 2001, Levenson and Hoyt 2000, Schultz et al 2001). Mostof this work considers 2D spatial data. While Ford et al (2001) describe a tomographicapplication, reconstruction is restricted to a 2D focal plane. This paper focuses on thedevelopment of a flexible instrumentation platform that can provide data to these algorithms,on the characterization of this imaging system and on proof-of-principle demonstration thatspectral changes can be related to depth information in phantoms and in animal studies. Theformulation of the reconstruction algorithms to provide 3D reconstructions of these data andvalidation of these approaches are presented separately (Chaudhari et al 2005).

2. Description of hyperspectral imaging system

A prototype hyperspectral imaging (HI) system has been developed that enables us to flexiblyacquire hyperspectral fluorescence data from phantoms and mice. A schematic and photographof the system is shown in figure 1. The object of interest is placed on an imaging stage, whichcan be heated for in vivo mouse studies to keep the animal warm while under anaesthesia. Thestage is made from matte black Delron to minimize light reflections and autofluorescence from

2032 G Zavattini et al

yx

x grid

blue LED

532 or 640 nm laser

optical bench for mounting of components

CCD

spectrograph

objective lens

notch filter

mirror

heated stage

slit

Figure 1. Schematic (top) and photograph (bottom) of prototype hyperspectral optical tomographysystem.

the stage itself. The object is illuminated from underneath the stage (which has a gridof holes drilled through it for this purpose) using a 640 nm, 4.5 mW laser diode (S2011,Thorlabs Inc., Newton, NJ, USA). An interferometric filter (Andover Corp., Salem, NH,USA) is placed in front of the laser to reduce its linewidth. The system is also designed toaccommodate a 7 mW, 532 nm Nd:YAG frequency doubled laser, however all fluorescencemeasurements reported below utilized the 640 nm laser. The illuminating laser is mounted onan x–y translation stage with 1 mm/revolution screws controlled by 400 steps/revolutionstepper motors allowing precise control of the illumination location. The fluorescenceemission light emitted at the top surface of the animal is focused with a 17 mm focallength, f/1.4 objective lens (Schneider Optics, Hauppauge, NY, USA) onto the entranceslit of the imaging spectrograph. This spectrograph (Imspector V10E, Specim, Oulu,Finland) images a line across the object and splits the light into its component wavelengthsproducing an image on the surface of the CCD chip that encodes one spatial dimension(y in figure 1) and the spectral dimension λ. The second spatial dimension (x) is acquiredin subsequent frames through relative motion of the spectrograph in the x-axis using atranslation stage to create a 3D (x, y, λ) dataset for each illumination position. The CCDcamera is a front illuminated 512 × 512 chip (Roper Scientific, Trenton NJ, USA), with24 µm pixels, thermoelectrically cooled to −40 ◦C to reduce dark current noise. A front

3D hyperspectral fluorescence imaging system 2033

Figure 2. Projection of a line pattern onto imaging stage (top) and onto mouse (bottom). Thedeformation of the lines enables the surface contours of the mouse to be calculated. The object onthe right is the nose cone for delivering anaesthetic gas.

illuminated CCD was chosen to avoid etaloning effects present for wavelengths >750 nm inback thinned CCD sensors. Etaloning is created by interference of the light from reflectionsat the chip surface and leads to position-dependent effects on the intensity at differentwavelengths due to small variations in the thickness of the silicon across the CCD sensor.The spectral resolution of the system is governed by the slit width of the spectrograph.In our set-up, we use a slit width of 25 µm, leading to a spectral resolution of ∼3 nm.The spectral acceptance of the spectrograph is from 400 to 1000 nm. The slit length is14.3 mm, and the spectrograph has a numerical aperture of f/2.4. To prevent excitationlight from saturating the CCD, a 2-line (532 nm and 640 nm) interferometric rugate notchfilter (Barr Associates, Westford, MA, USA) with ∼10 nm bands centred at 532 nm and640 nm (corresponding to the emission of the two lasers) is placed in front of the spectrograph.The entire system is built on an optical bench placed inside a light-tight enclosure. Theenclosure has internal power connections, and a pass-through for data cables and anaestheticgas.

The 3D volume (x, y, λ) can be acquired for different illumination positions by movingthe excitation laser and repeating the acquisition procedure. Software control of the camerasettings and automated timing of CCD camera frames with movement of the translation stagesare achieved using WinSpec (Roper Scientific, Trenton, NJ). The translation stages are drivenby VMX controllers (Velmex Inc., NY). Without any significant effort at this stage to optimizethe efficiency of data acquisition, approximately 1 h is required to acquire 3D (x, y, λ) datavolumes that provide information on the light emitted from the top surface of the mouse for40 different illumination (excitation) locations. Reductions in the acquisition time to just a fewminutes can likely be realized by faster scanning and simultaneous illumination at multiplelocations across the body.

The HI system also incorporates a blue LED (as shown in figure 1) that projects a structuredlight pattern onto the surface of the object being imaged. Distortion of the pattern (figure 2)by variations in the height of the object will allow the surface contours of the object to becalculated for use in tomographic reconstruction (Takeda and Mutoh 1983, Rice et al 2003).Because of the short wavelength of the LED light, these data can be acquired simultaneously

2034 G Zavattini et al

with the fluorescence data, which may be useful for detecting any animal motion duringscanning.

3. Experimental methods

3.1. System linearity and calibration

The linearity of the HI system in terms of wavelength, spatial coordinates and signal intensitywas evaluated. To determine the spectral linearity and calibrate the wavelength scale, theknown line emissions from mercury and argon gas in a standard fluorescent lamp were usedto illuminate the objective lens of the imaging system. The location of these discrete spectrallines in the CCD image was plotted against the known wavelength of the emission lines todetermine the wavelength calibration of the system. Each line in the acquired spectrum was fitwith a Gaussian function plus a background component to determine its location in the CCDcoordinate space. The calibration curve was fit by linear regression to determine the linearityof the calibration. A white piece of paper with regularly spaced black lines was also imagedto determine spatial linearity of the HI system signal.

To study signal intensity as a function of fluorochrome concentration, two nominallyidentical dilution series of Vybrant DiD fluorescent dye (Molecular Probes Inc., Eugene, OR)were prepared in ethanol. The solutions, mixed in clear plastic microtubes, and all of finalvolume 190 µl, had the following concentrations: 10 µM, 500 nM, 25 nM, 1.25 nM, 62.5 pM,3.1 pM and 0 M (a solution containing just ethanol). Each set of seven solutions was directlyilluminated by the laser diode and measured twice, where in each instance three consecutive5 s exposures of the centre of the sample tube were acquired. Thus at each concentration,four values (two solutions, for each solution two values, each an average of three individualexposures) for signal intensity were obtained. For the 10 µM solution a 0.5 s exposure wasused to avoid saturation of the CCD pixels, and the results were multiplied by a factor of 10.From each image, a region of interest was defined to include wavelengths from 690 nm to810 nm, and in space over the vertical ∼5 mm of volume of solution. The lower limit in thewavelength region of interest avoids including some directly scattered laser light that was noteliminated by the notch filter. The data were background subtracted (empty microtube) withbackground chosen to be one count per pixel less than the minimum measured value in orderto allow plotting on a log scale.

3.2. Spectral sensitivity

The combined spectral efficiency of the imaging system (including notch filter, lens,spectrograph and CCD) was measured by illuminating the objective lens with light froma calibrated monochromator and measuring signal intensity in the CCD camera as a functionof wavelength. Using the manufacturer-supplied efficiency curve for the monochromator(range 400–1000 nm), the spectral efficiency of the hyperspectral imaging system can beestimated.

3.3. Spectral dependence on source depth in biological tissue

Preliminary experiments were performed with the HI system to characterize the spectralvariation of the detected fluorescent light as a function of thickness of tissue traversed, asthis will ultimately be combined with the intensity of the detected light as the basis forreconstructing source location in vivo. Four slices of muscle (red meat) were used, each about5 mm thick, and fluorescent point sources containing small quantities of Vybrant DiD dye

3D hyperspectral fluorescence imaging system 2035

Figure 3. Schematic diagram showing the location and depth of the five fluorescent samples placedwithin the muscle phantom.

(Molecular Probes, Eugene, OR, USA) with a peak excitation of 644 nm and peak emissionat 665 were placed between the slices at different depths. The total thickness of the musclephantom of 20 mm roughly equals the thickness of a mouse. Slices of red meat were chosento best approximate the complex spectral absorption and scattering characteristics of realtissue, while still providing a controlled laboratory environment for source placement andmeasurement geometry. Two sets of measurements were taken, one with the Vybrant DiDfluorescence samples in place and other without. This second measurement was taken tomeasure the autofluorescence of the muscle sample which turned out to be negligible at anexcitation wavelength of 640 nm. The location of the five fluorescent samples is shownin figure 3.

The slices of muscle were placed on a flat stage, illuminated from underneath and imagedfrom above. The illumination was performed on a 9 × 5 grid of points, each 1 cm apart.For each illumination point, the spectrograph was scanned across the meat phantom in 1 mmsteps for a total distance of 4 cm. For each position, a 512 spatial bins × 120 spectral binsimage was acquired. The spectral bins corresponded to a band from 621.3 nm to 898.4 nm.In this way, a dataset of 9 × 5 hyperspectral images each consisting of 512 × 40 spatial bins× 120 spectral bins was acquired. The set-up used to acquire these data is shown in figure 1.Spectra were extracted for each of the sources and illumination positions, and the spectralshape compared for different source depths.

3.4. In vivo imaging study

A simple proof-of-principle experiment was performed to demonstrate that depth-relatedspectral variations are also seen in vivo and to establish the feasibility of in vivo imaging withthe HI system. Nude mice were injected subcutaneously with 5 × 106 PC3 cells overexpressingthe antigen NCA on one flank and 5 × 106 wild-type PC3 cells on the opposing flank. Thewild-type PC3 tumour acted as an internal control for non-specific binding effects. Tumourswere allowed to grow to a target volume of 0.2 cc. Anti-NCA was labelled with Alexafluor647 dye (Molecular Probes Inc.) at a dye:protein ratio of 6. The excitation maximum ofAlexafluor 647 occurs at 652 nm and the emission maximum occurs at 669 nm. 100 µl of2.24 mg ml−1 labelled antibody (equivalent to a dose of 9 mg kg−1) was injected i.v. intoeach mouse. Mice were imaged using the hyperspectral imaging system at 2, 3, 4 and 5 days

2036 G Zavattini et al

(a) (b)

Figure 4. (a) Image from hyperspectral imaging system of regularly spaced pattern of black lineson a white piece of paper illuminated by fluorescent lamp. Line emissions from Ar and Hg in thelamp are clearly seen. Wavelength calibration achieved using data shown in part (b). (b) Plot ofemission wavelengths of Ar and Hg lines from a fluorescent lamp versus channel number in theCCD image demonstrating the linearity of the spectral information (r = 1.0).

post-injection. For imaging, mice were anaesthetized using 1–2% isoflurane. Hyperspectral(x, y, λ) data were acquired at 32 illumination positions, with excitation at 640 nm (80% ofthe maximum excitation for Alexafluor 647) on an 8 × 4 grid, with 5 mm separation betweenillumination positions in x and y. At each illumination position, data were acquired for 1 s foreach of 40 positions in the y-axis, with a 1 mm sampling distance, thus producing an (x, y, λ)data volume covering the entire mouse body for each illumination position on the surface ofthe mouse. Total acquisition time was approximately 45 min.

4. Results

4.1. System linearity and calibration

An image reflecting the data produced from the HI system is shown in figure 4(a). This showsthe result of imaging a pattern of black lines on white paper under fluorescent room lights,displaying spatial information on one axis and wavelength information on the other axis. Thediscrete emission lines from Ar and Hg in the fluorescent lamp are clearly visualized. Whenthe locations of these lines in the CCD image are fitted against their known wavelengths, linearregression yields a regression coefficient of r = 1.0 (figure 4(b)). The spatial linearity alsoexhibits excellent linearity (regression coefficient of r = 1.0) demonstrating that there are nomajor spatial distortions introduced by the optics.

Figure 5 shows the signal intensity from the HI system as a function of the concentrationof fluorescent dye. Note that the plot has logarithmic scales on both axes, and the points plottedat seven locations on the x-axis correspond to the seven solutions in each dilution series. Thesolution with no dye is plotted at the extreme left of the figure and is labelled ‘no dye’. Ateach concentration there are four points, corresponding to the two measurements of each ofthe two series. The plot shows the average pixel value from the region of interest across eachof the three replicate measurements. The line on the graph has a slope of 1 and is the bestfit to the data from concentration of 10 µM to 62.5 pM. The response of the system as usedhere is linear over four orders of magnitude in dye concentration in ethanol, with a sensitivitydown to below 1 nM. The solution of 10 µM concentration was visibly blue, suggesting that

3D hyperspectral fluorescence imaging system 2037

Figure 5. Log–log plot showing measured signal intensity from HI system (integrated from 690to 810 nm) as a function of fluorophore concentration. At each concentration there are four points,corresponding to two measurements of each of the two dilution series. The points are slightlyoffset for clarity. The line on the graph has a slope of 1 and is the best fit to the data between62.5 pM and 10 µM.

14

12

10

8

6

4

2

0

effic

ienc

y (a

.u.)

1000800600400wavelength (nm)

Figure 6. Efficiency of the imaging system (notch filter, lens, spectrograph and CCD) as afunction of wavelength. The notch filter is designed to block light at 532 and 640 nm, theexcitation wavelengths of the two lasers available in the system to illuminate the sample.

there is likely sufficient dye to lead to quenching or other nonlinear effects. The responseof the system to solutions of the dye in water was similarly linear (data not shown). Thisparticular dye yields a factor of 40 less light for solution in water as compared to ethanol ata given concentration, so the sensitivity for solutions in water only extend down to 10 nMconcentrations.

4.2. Spectral sensitivity

The spectral sensitivity of the HI system is shown in figure 6. The sensitivity is determinedby a combination of the spectral sensitivity of the CCD camera, the transmission properties ofthe optical system and filters, and the efficiency of the diffraction grating in the spectrograph.The location of the notch filter can be clearly seen. Although these interferometric filters donot completely eliminate light at the excitation wavelengths of the lasers in the HI system,the attenuation is sufficient to avoid any saturation effects in the CCD, and any remaining

2038 G Zavattini et al

850800750700650

0 mm depth 5 mm depth 10 mm depth 15 mm depth 20 mm depth

norm

aliz

ed s

igna

l

2.0

1.5

1.0

0.5

0

wavelength (nm)

Short Long

Figure 7. Spectra from fluorescent sources at different depths within the muscle phantom. Allspectra are normalized to the integrated intensity above 710 nm to allow comparison of spectralshapes. The spectra show depth-dependent attenuation of fluorescent light in the wavelength range650–710 nm.

excitation light is simply removed by wavelength selection within the hyperspectral data.Note that efficiency of the HI system is highest across the wavelength range 650–850 nm,which is the spectral region of most interest for in vivo studies.

4.3. Spectral dependence on source depth in tissue

Figure 7 shows the variation in the detected spectrum with source depth for a fluorescentpoint source in a 2 cm thick piece of muscle. These spectra are the raw intensities and are notcorrected for the system efficiency shown in figure 6. For each point source, the spectrum fromthe pixel with the highest intensity is shown. Because attenuation of the light signal is highlydepth dependent, the spectra are normalized to the integrated signal intensity above 710 nm toallow their shapes to be compared on the same scale. In practice, of course, one would makeuse of both intensity and spectral information. The division of the spectrum at 710 nm into‘long’ and ‘short’ wavelength bands is arbitrary and is simply chosen to provide a ratio betweenthe two wavelength bands that varies with depth, and in which there is measurable signal inboth bands for the range of depths of interest. Using this simple approach of normalizing datain the ‘long’ wavelength band, depth-dependent spectral changes in the ‘short’ wavelengthband can clearly be demonstrated.

The data are well described by a simple exponential function

R = A0 + A exp(−τ r) (1)

where R is the ratio of the ‘long’ and the ‘short’ wavelength signals as defined in figure 7,r is the distance from the source to the point on the surface from which the measurement isobtained, A0, A and τ are constants, where τ is related to tissue attenuation (but is not the trueattenuation, as the signal is normalized to the integrated intensity for wavelengths >710 nm).The results of this fitting procedure are shown in figure 8.

This calibration can then be used to reconstruct the position of the sources in the musclephantom. Defining the surface from which the measurement is made as being at z = 0, the

3D hyperspectral fluorescence imaging system 2039

3.0

2.5

2.0

1.5

806040200

ratio R

ra

tio S

hort

/Lon

g

depth, r (relative units)

Fit: R=A0+Aexp (–τr)

Figure 8. Plot of ratio R (see the text for details) as a function of source depth. The observed dataare well described by an exponential fit.

Source depth (mm)

Rec

onst

ruct

ed d

epth

(m

m)

Figure 9. Plot of the reconstructed depth (average of five illumination positions, ±1 s.d.) versusactual depth for the fluorescent point emitters embedded in muscle in the geometry shown infigure 3. The reconstructed depth of the five fluorescent sources are obtained solely using thespectral information and the depth calibration shown in figure 9. The five measurements for eachsource location correspond to measurements from the five laser illumination locations (±1 cm inx and y directions) closest to the source.

distance from a source at location (x0, y0, z0) to a measurement point on the surface (x, y, 0) isgiven by

r =√

(x − x0)2 + (y − y0)2 + (z0)2. (2)

The x and y locations (x0, y0) are determined from the brightest pixel in the hyperspectral data,and using the calibration in figure 8, z0 is computed for each of the sources. The 3D sourcelocation is calculated for each of the five (x, y) excitation positions (±1 cm) closest to thesource. Beyond this distance, there is no significant fluorescence from the source. The resultsare shown in figure 9. Despite the significant variations in r for the five different excitationlocations of the same source, and the fact that only one excitation location was used to

2040 G Zavattini et al

Figure 10. Complete dataset from an in vivo hyperspectral imaging experiment. The imagehas been integrated over the wavelength range 656–748 nm for ease of display. Each mouseimage represents a different illumination location on an 8 × 4 grid with 5 mm spacing betweenillumination points. A structured light source illuminates the mouse for contour determinationas described in figure 2. The NCA positive tumour can be clearly seen when the illumination isdirectly underneath it in the bottom right-hand corner. There are some edge effects and artefactswhen the illumination comes close to the edge of the mouse as seen on the top row and left column.

determine the calibration data (the excitation location closest to the source), the reconstructedlocations for different illumination positions are in very good agreement. The sources arealso reconstructed at approximately the correct depth. There is higher variability for sourcesfar from the excitation source, presumably because the detected signal is much lower andmore subject to camera noise. These data are a proof-of-principle demonstration of the use ofspectral information (crudely separated into just two wavelength bands) to reconstruct the 3Dlocation of a fluorescent point emitter within biological tissue, albeit, tissue that has uniformoptical properties and a simple slab geometry.

4.4. In vivo imaging study

The complete hyperspectral dataset, with the fluorescence intensity integrated across thewavelength range 656–748 nm, is shown in figure 10 for a mouse imaged 5 days afteradministration of the fluorescently labelled antibody. Each image corresponds to a differentposition for the excitation light source. The striped pattern on the mouse is from the structuredline pattern illuminated with a blue LED and will ultimately be used to determine the mousecontours (Takeda and Mutoh 1983, Rice et al 2003) for use in the reconstruction algorithm.Figure 11 shows spectra (normalized to intensities above 730 nm) for fluorescence comingfrom different locations across the animal. Although the antibody localizes in the antigen-positive tumour, there is still some non-specifically localized antibody in the blood and tissuesthroughout the body. These spectra are the raw intensities and are not corrected for the systemefficiency shown in figure 6.

The spectra show similar variations to those observed in the muscle phantom shown infigure 7 and demonstrate that spectral changes can be detected in vivo, although their exact

3D hyperspectral fluorescence imaging system 2041

Wavelength [nm]

Inte

nsity

[a.

u.]

ShoulderIntense thighFaint thighTumour

Figure 11. In vivo hyperspectral data for signal from a fluorescently labelled antibody acquired xdays after injection into a mouse. Top image shows integrated (656–748 nm) spectral images ofthe mouse viewed for four different illumination positions. At each pixel in each image an entirespectrum is collected; sample spectra (normalized for wavelengths >730 nm) for various regionswith antibody signal are shown below the figure. Note the significant variations in spectra atdifferent locations that closely resemble the depth-dependent data shown in figure 8 for the musclephantom.

relationship to source depth in a non-homogeneous optical medium must still be investigated.The data shown in figure 10 (without the collapse of the spectral data required to present itin the figure) form the input data for the 3D reconstruction algorithm that is currently underdevelopment and that incorporates the mouse surface geometry combined with tissue opticalproperties applied to a segmented anatomical MR or CT image of the mouse (Chaudhari et al2005).

5. Conclusions

A hyperspectral imaging system has been assembled and images have been acquireddemonstrating the depth dependence of spectral information both in phantoms and in vivo. In ahomogeneous optical medium, we have shown that it is possible to reconstruct fluorescent point

2042 G Zavattini et al

source locations in 3D using only relative differences in the spectral intensities of the emittedlight, rather than the integrated light signal that has been used by others. We are currentlyworking to incorporate both spectral and intensity information into a 3D reconstructionalgorithm for bioluminescence (Chaudhari et al 2005) and fluorescence imaging. Futurestudies will include a detailed evaluation of optimal excitation patterns for fluorescenceimaging, simultaneous collection of multiple views of the surface of the mouse and morerapid data collection.

Acknowledgments

The authors would like to thank Jed Ross and Mary Cole of Genentech Inc. for help indesigning the animal experiments and for providing the fluorescent antibody. We also thankStephen Rendig for help in carrying out the animal studies and Julie Sutcliffe-Goulden andSven Hausner for assistance in accurate sample preparation.

References

Campbell R E, Tour O, Palmer A E, Steinbach P A, Baird G S, Zacharias D A and Tsien R Y 2002 A monomeric redfluorescent protein Proc. Natl Acad. Sci. 99 7877–82

Chaudhari A J, Darvas F, Bading J R, Moats R A, Smith D J, Cherry S R and Leahy R M 2005 Hyperspectral andmultispectral bioluminescence optical tomography for small animal imaging Phys. Med. Biol. 50 5421–41

Chen K, Perelman L T, Zhang Q G, Dasari R R and Feld M S 2000 Optical computed tomography in a turbid mediumusing early arriving photons J. Biomed. Opt. 5 144–54

Cherry S R 2004 In vivo molecular and genomic imaging: new challenges for imaging physics Phys. Med.Biol. 49 R13–48

Contag C H and Bachmann M H 2002 Advances in in vivo bioluminescence imaging of gene expression (review)Annu. Rev. Biomed. Eng. 4 235–60

Ford B K, Volin C E, Murphy S M, Lynch R M and Descour M R 2001 Computed tomography-based spectral imagingfor fluorescence microscopy Biophys. J. 80 986–93

Gao X, Cui Y, Levenson R M, Chung L W and Nie S 2004 In vivo cancer targeting and imaging with semiconductorquantum dots Nat. Biotechnol. 22 969–76

Godavarty A, Eppstein M J, Zhang C Y, Theru S, Thompson A B, Gurfinkel M and Sevick-Muraca E M 2003Fluorescence-enhanced optical imaging in large tissue volumes using a gain-modulated ICCD camera Phys.Med. Biol. 48 1701–20

Graves E E, Ripoll J, Weissleder R and Ntziachristos V 2003 A submillimeter resolution fluorescence molecularimaging system for small animal imaging Med. Phys. 30 901–11

Gu X J, Zhang Q H, Larcom L and Jiang H B 2004 Three-dimensional bioluminescence tomography with model-basedreconstruction Opt. Express 12 3996–4000

Levenson R M and Hoyt C C 2000 Spectral imaging and microscopy Am. Lab. 32 1–8Mahmood U, Tung C H, Bogdanov A Jr and Weissleder R 1999 Near-infrared optical imaging of protease activity

for tumor detection Radiology 213 866–70Ntziachristos V, Ripoll J, Wang L H V and Weissleder R 2005 Looking and listening to light: the evolution of

whole-body photonic imaging Nat. Biotechnol. 23 313–20Ntziachristos V and Weissleder R 2001 Experimental three-dimensional fluorescence reconstruction of diffuse media

by use of a normalized born approximation Opt. Lett. 26 893–5Rice B W, Cable M D and Nelson M B 2001 In vivo imaging of light-emitting probes J. Biomed. Opt.

6 432–40Rice B W, Coquoz O, Kuo C, Nantel N, Nilson D N, Strearns D G, Troy T L, Zwarg D and Cable M D 2003 Abstracts

of the 2nd Annual Meeting of the Society for Molecular Imaging (San Francisco, CA) p 209Roy R and Sevick-Muraca E M 2001 Three-dimensional unconstrained and constrained image-reconstruction

techniques applied to fluorescence, frequency-domain photon migration Appl. Opt. 40 2206–15Schultz R A, Nielsen T, Zavaleta J R, Ruch R, Wyatt R and Garner H R 2001 Hyperspectral imaging: a novel approach

for microscopic analysis Cytometry 43 239–47Swartling J, Svensson J, Bengtsson D, Terike K and Andersson-Engels S 2005 Fluorescence spectra provide

information on the depth of fluorescent lesions in tissue Appl. Opt. 44 1934–41

3D hyperspectral fluorescence imaging system 2043

Takeda M and Mutoh K 1983 Fourier-transform profilometry for the automatic measurement of 3-d object shapesAppl. Opt. 22 3977–82

Troy T, Jekic-McMullen D, Sambucetti L and Rice B 2004 Quantitative comparison of the sensitivity of detection offluorescent and bioluminescent reporters in animal models Mol. Imaging 3 9–23

Wang G, Li Y and Jiang M 2004 Uniqueness theorems in bioluminescence tomography Med. Phys. 31 2289–99Weissleder R and Mahmood U 2001 Molecular imaging (review) Radiology 219 316–33Yao G and Wang L H V 1999 Monte Carlo simulation of an optical coherence tomography signal in homogeneous

turbid media Phys. Med. Biol. 44 2307–20Zavattini G, Vecchi S, Leahy R M, Smith D J and Cherry S R 2003 A hyperspectral fluorescence imaging system for

biological applications IEEE Nuclear Science Symposium Conference Record vol 2, pp 942–6