3-d reconstruction in optical microscopy by a frequency-domain approach

10
SignalProcessing32(1993)357-366 357 Elsevier 3-Dreconstructioninopticalmicroscopybya frequency-domainapproach TullioTommasi',AlbertoDiaspro2 andBrunoBianco' DepartmentofBiophysicalandElectronicEngineering,University of Genova,ViaOperaPiaIla, 16145 Geneva,Italy Institute of Biophysics,University of Genova,ViaGiotto2, 16153 Genova,Italy Received22August1991 Revised6April1992and8October1992 Abstract .Ageneralandsimplemethodfor3-Dimagerestorationinopticalsectioningmicroscopyispresented .Asetof imagestakenwithanopticalsystemwithaknown3-Dpointspreadfunction(psf)areconsidered .Therestorationtaskis described,pointingouttheoreticalconstraintsonrecoveringtheactualspatialdistributionoftheoriginalobject .Aspatial- frequencyanalysisisperformed,usingnoaprioriinformationaboutthespecimensunderinvestigation .Thefeasibilityand limitationsof3-Dreconstructionsareanalyzed .Moreover,aprocedureisproposedthatallowsonetorecoverobjectprojections withinananglerangestrictlydependentonthepsfofthesystemused .3-Dobjectshavebeensimulatedinordertoproduce asetofimagessimilartothoseprovidedbyanopticalsystem .Restorationhasthenbeenaccomplished,andresultsdemonstrate thatside-viewsofanobjectcanbeobtainedtoasatisfactorydegreeofaccuracy .Finally,adiscussionconcerningthepresence oftransparentandopaqueobjectsinthefieldofviewisalsopresented . Zusammenfassung.WirbeschreibeneineallgemeineandeinfacheMethodezurWiederherstellungvon3-DBildeminder optischenSchnitt-Mikroskopie .DabeiwirdeineMengevonBildernverwendet,diemittelseinesoptischenSystemsmit bekannter3-DPunkt-Verbreitcrungsfunktion(PVF)aufgenommenwurden .WirbeschreibendieWiederherstellungsaufgabe andweisenauftheoretischeEinschrankungenhinsichtlichderWiedergewinnungdertatsachlichenraumlichenVerteilungdes ursprunglichenObjektshin .EswirdeineAnalysebezuglichderraumlichenFrequenzvorgenommen,wobeikeineapriori InformationcaberdieuntersuchtenProbenwerwendetwird .DieDurchfuhrbarkeitandGrenzender3-DRekonstruktion werdenuntersucht .WeiterswirdeineMethodevorgeschlagen,mitderObjekt-ProjektioneninnerhalbeinesWinkelbereichs wiedergewonnenwerdenkonnen,dervonderPVFdesverwendetenSystemsabhangt .3-DObjektewardensimuliert,umeine MengevonBildemzuerzcugen,diedenmiteinemoptischenSystemgewonnenenBildernahnlichsind .DieErgebnisseder damitdurchgefuhrtenWiederherstellungzeigendaBSeitenansichteneinesObjektsmitzufriedenstellenderGenauigkeiterhalten werden .AbschlieliendwirdsuchdieGegenwartdurchsichtigerandundurchsichtigerObjektediskutiert . Resume .Unemethodegeneraleetsimplederestaurationd'images3-Ddanslecontextedumicroscope a sectionoptiqueest presentee .Unensembled'imagesprises a l'aided'unsystemeoptique a reponseimpulsionnelle3-Dconnuesontconsiderees . Latachederestaurationestdecrite,etlescontraintestheoriquessurlerecouvrementdeladistributionspatialereellede('objet originelsentraisesenavant .Uneanalysespatio-frequentielleestefectueesansutiliseraucuneinformationapriorisurles specimensetudies .Lafaisabiliteetleslimitationsdesreconstruction3-Dsootanalysees .Deplusuneprocedurepermettant derecupererlesprojectionsd'objetscionunangledependantstrictementdelareponseimpulsionnelleestproposee .Desobjets 3-Dantetesimulesafindeproduireuneensembled'imagessemblable a cellesfourniesparonsystemeoptique .Larestauration aeteaccomplie,etlesresultatsmontrentquelesvuesdecoted'unobjetpeuventtitreobtenuesavecondegredeprecision satisfaisant .Finalement,unediscussionsurlapresenceTabletstransparentsetopaquesdapslechampdevisionestegalement presentee . Keywords .Fouriertransform ;3-Dreconstruction ;incoherentoptics . Correspondenceto : Prof.B .Bianco,DipartimentodiIngegneriaRiofisicaedElettronica,UniversitydeglistudidiGenova,Via OperaPiaIla,Genova16145,Italia. 0165-1684 1 /93506 .00 U 1993ElsevierSciencePublishersB-V .Allrightsreserved

Upload: tullio-tommasi

Post on 21-Jun-2016

213 views

Category:

Documents


2 download

TRANSCRIPT

Signal Processing 32 (1993) 357-366

357Elsevier

3-D reconstruction in optical microscopy by afrequency-domain approach

Tullio Tommasi', Alberto Diaspro2 and Bruno Bianco'Department of Biophysical and Electronic Engineering, University of Genova, Via Opera Pia Ila, 16145 Geneva, ItalyInstitute of Biophysics, University of Genova, Via Giotto 2, 16153 Genova, Italy

Received 22 August 1991Revised 6 April 1992 and 8 October 1992

Abstract . A general and simple method for 3-D image restoration in optical sectioning microscopy is presented . A set ofimages taken with an optical system with a known 3-D point spread function (psf) are considered . The restoration task isdescribed, pointing out theoretical constraints on recovering the actual spatial distribution of the original object . A spatial-frequency analysis is performed, using no a priori information about the specimens under investigation . The feasibility andlimitations of 3-D reconstructions are analyzed . Moreover, a procedure is proposed that allows one to recover object projectionswithin an angle range strictly dependent on the psf of the system used . 3-D objects have been simulated in order to producea set of images similar to those provided by an optical system . Restoration has then been accomplished, and results demonstratethat side-views of an object can be obtained to a satisfactory degree of accuracy . Finally, a discussion concerning the presenceof transparent and opaque objects in the field of view is also presented .

Zusammenfassung. Wir beschreiben eine allgemeine and einfache Methode zur Wiederherstellung von 3-D Bildem in deroptischen Schnitt-Mikroskopie . Dabei wird eine Menge von Bildern verwendet, die mittels eines optischen Systems mitbekannter 3-D Punkt-Verbreitcrungsfunktion (PVF) aufgenommen wurden . Wir beschreiben die Wiederherstellungsaufgabeand weisen auf theoretische Einschrankungen hinsichtlich der Wiedergewinnung der tatsachlichen raumlichen Verteilung desursprunglichen Objekts hin . Es wird eine Analyse bezuglich der raumlichen Frequenz vorgenommen, wobei keine a prioriInformation caber die untersuchten Proben werwendet wird . Die Durchfuhrbarkeit and Grenzen der 3-D Rekonstruktionwerden untersucht. Weiters wird eine Methode vorgeschlagen, mit der Objekt-Projektionen innerhalb eines Winkelbereichswiedergewonnen werden konnen, der von der PVF des verwendeten Systems abhangt . 3-D Objekte warden simuliert, um eineMenge von Bildem zu erzcugen, die den mit einem optischen System gewonnenen Bildern ahnlich sind . Die Ergebnisse derdamit durchgefuhrten Wiederherstellung zeigen daB Seitenansichten eines Objekts mit zufriedenstellender Genauigkeit erhaltenwerden . Abschlieliend wird such die Gegenwart durchsichtiger and undurchsichtiger Objekte diskutiert .

Resume . Une methode generale et simple de restauration d'images 3-D dans le contexte du microscope a section optique estpresentee. Un ensemble d'images prises a l'aide d'un systeme optique a reponse impulsionnelle 3-D connue sont considerees .La tache de restauration est decrite, et les contraintes theoriques sur le recouvrement de la distribution spatiale reelle de ('objetoriginel sent raises en avant . Une analyse spatio-frequentielle est efectuee sans utiliser aucune information a priori sur lesspecimens etudies . La faisabilite et les limitations des reconstruction 3-D soot analysees . De plus une procedure permettantde recuperer les projections d'objet scion un angle dependant strictement de la reponse impulsionnelle est proposee . Des objets3-D ant ete simules afin de produire une ensemble d'images semblable a celles fournies par on systeme optique . La restaurationa ete accomplie, et les resultats montrent que les vues de cote d'un objet peuvent titre obtenues avec on degre de precisionsatisfaisant . Finalement, une discussion sur la presence Tablets transparents et opaques daps le champ de vision est egalementpresentee .

Keywords . Fourier transform ; 3-D reconstruction ; incoherent optics .

Correspondence to : Prof. B . Bianco, Dipartimento di Ingegneria Riofisica ed Elettronica, University degli studi di Genova, ViaOpera Pia I la, Genova 16145, Italia.

0165-1684 1/93506 .00 U 1993 Elsevier Science Publishers B-V . All rights reserved

358

1. Introduction

A topic related to current image science that hasbeen extensively investigated is `reconstruction' ofa three-dimensional (3-D) object, starting fromtwo-dimensional (2-D) images. This is a veryimportant subject, perhaps not clearly defined bythe above statement for a reader not involved inactual work on images, but certainly well knownto researchers in this field . The reconstruction of a3-D image of a given object, starting from imagestaken with some 2-D acquisition apparatus, lies ina full description of the spatial distribution of somephysical intensive parameter of the object. In thissense, a typical example of 3-D reconstruction isgiven by X-ray tomography, where the intensiveparameter is represented by the local-absorptioncoefficient of the object examined, averaged overthe X-ray spectrum used for illumination [4],Another example is offered by the reconstructionof the 3-D shape of an object, starting from a num-ber of stereoscopic views [4, 12] . In these or similarcases, the images acquired are always two-dimen-sional. This is true even for the technique thatyields almost perfect 3-D images, namely,holography [10] .Within the general framework of 3-D recon-

struction, this paper presents the analysis of a tech-nique that has received considerable attention inthe last decade, that is, 3-D reconstruction of anobject, starting from its images taken with an opti-cal system for different focus conditions [ 1, 3, 5, 6] .This technique is of particular importance in thecase of microscopic objects, such as cells or cellularorganelles [1, 3, 7], as it permits one to analyzebiological specimens without damaging their struc-tures and functions .

From a mathematical perspective, the problem isvery similar to the one of longitudinal tomography,which has been investigated by several authors[12]. However, these reconstruction algorithmshave seldom been utilized for the specific goal ofimage restoration in optical microscopy . We aimto reach this goal by developing an analysis relatedto optical transmission microscopy . A `hardware'signal Processing

T. Totnmasi et al / 3-1) reconstruction in optical microscopy

implementation of computerized tomographymethods in optical microscopy was previouslyaccomplished by rotating the microscope system inorder to obtain images of a sample for differentview directions [11] . We utilize images taken witha conventional microscope (i .e., the view directionis always along the optical axis), and computerizedtechniques commonly applied in tomography allowus to obtain side-views from such images .

The basic idea of the proposed approach is thatwhen a microscope is focused on a particular planeof an object, and an image is acquired, this imageresults from the superimposition of the image ofthe in-focus plane and of blurred images of out-of-focus planes. Each of such images is more or lessdegraded, depending on its distance from thefocused plane. Nevertheless, blurred images are notsimply `noise,' as they provide information aboutthe whole 3-D object structure, modified by theoptical system in a deterministic way . If a set ofimages are taken using various focus positions, inprinciple one can recover the three-dimensionalshape of the object. The knowledge of the impulseresponse and the use of such images allow one torecover the original 3-D object through an inverseconvolution procedure . In terms of experimentalapparatus, it is only necessary that a high-qualitymicroscope be connected to an image-acquisitionsystem and the relative displacements (along theoptical axis) between the object slide and themicroscope objective be precisely measured . Mostwork is then carried out by implementing properalgorithms . The method looks very simple andnon-invasive . However, a deeper analysis of thistechnique shows that it is not so simple as it mayappear as the microscope acts as a low-pass filterfor spatial signals ; this behaviour is determined bythe very nature of any system of lenses, and entailsimportant consequences . In particular, the inverseconvolution procedure turns out to be critical inthe absence of some a priori knowledge about theobject to be reconstructed .

In this paper, the capabilities and limitations ofthe 3-D reconstruction technique are assessed . Itwill be shown that the method exhibits notable

limitations, and that what can be recovered is nota 3-D reconstruction, but something less . One canobtain projections of the object within a solid angledependent on the microscope aperture . The relatedtheory will be developed in the frequency domain,and no a priori information will be considered .

2. The 3-D point spread function and properties ofthe optical transfer function

Let us consider an optical system (e .g., a micro-scope) and a 3-D object of intensity io(x, y, z) asthe input. The object is assumed to be highly trans-parent in order to avoid problems related to hiddensurfaces . This is a real situation, as, in optical miro-scopy, many biological specimens show high lighttransmittance . A more complete analysis concern-ing opaque objects will be carried out later on .

The output is a set of images, i(x, y, z), obtainedby bringing into focus different object planes per-pendicular to the optical z axis of the system . Eachimage is composed of an in-focus plane and of out-of-focus planes . The system is characterized by itspsf, s(x, y, z), which represents the response to apoint source . If we assume the optical system to belinear and stationary, a convolution can describethe relation between the object and the images [4]

i(x,Y . z) = io(x, Y, z) * s(x, R Z),

(1)

where s(x, y, z) is the measured or theoretically cal-culated psf of the system for incoherent illumina-tion ; the z-dependence of i(x, y, z) is related todifferent focused object planes. In the case of digi-tized images, io(x, y, z) represents a set of planesperpendicular to the optical z axis, for somediscrete values of this axis . Similarly, i(x, y, z) is aset of discretized images . Once s and i are known,we have to face a typical inverse problem, and adeconvolution procedure can be applied . To thisend, a spatial-frequency approach can be applied ;in terms of the 3-D Fourier transform, one has

AL ,fIf)=1a(f If,.,f)s(f If,f),

(2)where 1, 1, and S are the 3-D Fourier transforms of

T. Tommasi et al. / 3-D reconstruction in optical microscopy

359

i, io and s, respectively, and f,, f, f: are the spatialfrequencies. A Wiener filter can be utilized torestore the original information, i .e. the object,where the noise-to-signal density power ratio isassumed to be a constant . Unfortunately, somelimitations affect the reconstruction process .Restoration constraints are imposed by thes(x, y, z) function, hence by its transform S, whosebehaviour does not allow us to obtain a completereconstruction .

To analyze the 3-D reconstruction task, we usean approximate expression for s(x, y, z) . We candetermine the limitations and possibilities of therestoration process by calculating its Fouriertransform .

The following considerations are basic to thechoice of s(x, y, z) . In an optical system, if geomet-ric optics is assumed (i .e ., the system is perfectlystigmatic), the 3-D psf is a cone whose axis coin-cides with the optical z axis, and whose aperture issome angle dependent on the entrance and exitpupils of the system. Hence, the equation for thepsf is [3]

JK/z2, if -.x2 + yz < IzI tan 0,s(x, y, °)

0,

otherwise,(3)

where K is a constant and 0 is the angle of apertureof the microscope .

The 3-D psf for a microscope has been exten-sively studied [8, 9, 13, 14] . In these works, diffrac-tion phenomena have been considered, which leadto more accurate expressions than a geometricalcone. But it is worth noting that all these expres-sions for s(x, y, z), though more accurate, are onlyslight improvements over (3) . Nevertheless, all ana-lytical psf models exhibit significant differencesfrom measured psf, owing to the fact that thehypotheses made to calculate the psfs (e .g ., parax-ial approximation, shift invariance, aberration freesystem) are not completely verified [9] . For thisreason, the psf is often measured in order toachieve better restoration results [2, 71, eventhough precise measures cannot easily be obtained .On the other hand, we aim to point out what can

Vol . 32 No . 3 . June 1993

360

T Tommasi et al. / 3-D reconstruction in optical microscopy

really be reconstructed, and what are the limits on

Using (Dirac's) delta pulses, straightforward calcu-3-D reconstruction .

lations give the final result :In this context, our simplified model is accurate

enough, as all experimental and analytical evalua-tions have yielded 3-D psfs conical in shape . Thisfact leads to an important theoretical conclusion :it is impossible to perform a complete 3-D recon-struction in a real noisy situation, if no a prioriinformation is utilized .

S(f, f ,f) =0 if I f I > R tan 0In order to prove this, we calculate the optical

transfer function (OTF) of the system, i .e., theFourier transform of the 3-D psf. We consider apsf of the form

s(x,Y, z)=Izq(t/IzI),

(4)

where t= N x2 +y2 , and q is zero if t>tan BIzI ; 0 isthe angle of aperture of the cone . Equation (4)meets the requirements for light-power conserva-tion and for axial symmetry . Equation (3) is a par-ticular case of the more general equation (4) ; itis sufficient to set q equal to K. A more accurateexpression, that takes into account also diffractioneffects, could be defined by choosing a specific qfunction for the domain where t < tan 0 Izl .

By using the new variables

u=Rizl W, +Yf),

vRIzI (-xJ,+Yfr),

where R=~,/f2+f2, one obtains

S f f z 2 q (Izl/x exp(-2nl(xf,+yf +zf) dx dy dz

= ff fq(,/u'+v2) exp(-21cjRuJzJ)

x exp(-2tcjzf) du dv dz

f fdu dvq(_,/u2+v)

If exp(-2rzj(u(zIR+zf) dz .

Signal Processing

S(f,fr,f)=fRq(

and

S(fx,f,f)2

f,2 2+fY

f 2 + v 2 1 dv./R

q vanishes if its argument is greater than tan 0 ; thisimplies that

,/(tan 0)2(f +fv) -f?

if [fI < R tan 0 .Therefore, S(f ,f„ f) identically vanishes

inside a cone in the Fourier space, with the f. axisas the axis of symmetry and n/2 - 0 as the angularaperture . The present analysis refers to the OTF ofan optical microscope, and is similar to the onecarried out for tomographic problems [161, wherethe psf refers to a planar positron camera .

The fact that S vanishes in a wide region can beexplained by some considerations related to thefrequency domain. The waves coming from anobject under investigation can be decomposed intosuperimposed plane waves travelling in differentdirections. Only the plane waves entering the opti-cal system at angles not higher than the system'sangle of aperture contribute to the image forma-tion . In fact, high frequencies are lost . We pointout that these conclusions also hold for a moregeneral approach that considers diffractioneffects .

3. 3-D reconstruction and side-views

The above conclusions are fundamental to the3-D reconstruction task . A suitable restorationprocedure might be the following: Starting fromthe images and from the calculated or measuredpsf of an optical system, one first calculatesthe Fourier transforms of the images ; then, by

(5)

performing a deconvolution, one obtains Io andconsequently is through an inverse transform . Buta deconvolution is not possible in those regionswhere S is zero, so some frequency information islost. This limitation also affects tomography imag-ing with limited angular input . Many works havebeen devoted to the task of extending frequencyinformation to the cone where the Fourier trans-form is zero [12] . The finite extent of an objectunder investigation permits one to state that itsFourier transform is an entire (analytic) function,hence it can be continued throughout the wholecomplex plane on the basis of the knowledge ofthe function on any finite continuous line segment .This approach cannot be implemented in practice,because of noise problems ; so iterative methodsare usually applied in the Fourier domain, utilizinga priori knowledge about the objects under investi-gation [15, 16] . Other approaches have been devel-oped [12], but improvements in results are alwaysobtained by using some a priori knowledge . Ourmethod does not exploit any a priori information,as we aim to propose a general and simple tech-nique able to restore useful optical-microscopyimages. In our case, the situation differs from thoseusually described in works dealing with tomo-graphic images, where some projections are usedto produce a Fourier transform that covers com-pletely the frequency domain, at least in the regionallowed. In conventional optical microscopy, the3-D Fourier transform is directly computed fromthe psf, while no side-views are available, due tothe limitations of the image-acquisition apparatus .On the other hand, side-views of microscopic speci-mens can be very useful, as they give informationabout the structures of the objects, not easilydetectable from the optical sections, To obtainside-views, one can apply a 3-D Wiener filter inthose regions where S does not vanish . On the basisof the resulting information, one can obtain a 3-Dreconstruction by calculating 2-D projections ofthe object. To this end, the projection theorem,or `slice' theorem [4], can be successfully used tocalculate a projection along a z' direction . Considera 3-D object and the corresponding spatial

T. Tonmasi et al. / 3-D reconstruction in optical microscopy

361

spectrum ; choose a i direction that forms an anglea with the optical z axis ; select the plane II (in thefrequency domain), that passes through the originand forms an angle a between its normal and thef axis. The projection theorem states that the dis-tribution of values of the spatial spectrum on 17 isthe 2-D Fourier transform of the projection of theobject along the z' direction . In our case, we con-clude that, if the zz' angle is less than 0 (where 0is the angular aperture of the system), the projec-tion can be obtained, as the corresponding planeII in the frequency domain is placed in a regionwhere S does not vanish. On the contrary, if za' ishigher than 0, the plane 17 is in a region where Sis equal to zero, and no information can be recov-ered (see Fig . 1) . Hence, the angle range for projec-tion images depends on the angular aperture of theoptical system used .

In analytical terms, the projection theorem canbe expressed as follows : Let us consider two framesof references : (f ,f„ f) for the Fourier domain,and (x, y, z) for the spatial domain . A new refer-ence system (x', y', z') is obtained by adding twoorthogonal axes, x' and y', to z', which is thedirection of the chosen projection, and by leavingthe origin unchanged with respect to (x, y, z) . If Ais the orthonormal matrix that links (x, y, z) to(x', y', z') and if (f-, f,,,, f ,) are the spatial

Fig. 1 . A sight direction, z', and the corresponding plane 77 inthe frequency domain .

Vol . 32, No . 3, June 1993

362

T. Tommasi el al. /3-D reconstruction in optical microscopy

frequencies conjugate to (x', y', z'), then

dx dy dz = det(A) dx' dy' dz'=dx' dy' dz'

and

xf +yf+zf = xf+Y'fy+zfIf one considers the planet = 0, one can write thefollowing equation :

I(f',J>',f-')=F(J i(x',y', z') dz'),

(6)

where F( ) stands for Fourier transform .Equation (6) is the Fourier transform of the pro-

jection of i(x', v , z') in the z' direction, calculatedon the planef =0. The above considerations andthe application of the projection theorem enableone to summarize the reconstruction procedure asfollows : Using a set of images i(x, y, z) and assum-ing s(x, y, z) to be known, a Wiener filter is usedin the frequency domain to recover lo(f , f , f )(i .e ., the Fourier transform of i o(x, y, z)) in thoseregions where S(f,f ,f) is not zero. A z' directionis chosen for a projection of the object, within theconstraint that the angle between z and z' is lessthan 0 . The plane 17 can be calculated by meansof linear interpolations in the Fourier domain . Tothis end, the real and imaginary parts can be inter-polated separately, each treated as a real functionof three variables . An inverse 2-D transform of thisplane gives the projection in the z' direction .

4. Results and discussion

Different three-dimensional structures werecomputer-generated (i.e., a pyramid and a stair-case) through the simulations of 16 optical sec-tions. The point spread function of the imageformation system (i .e., a microscope) was assumedto be a double cone (see (3)) . A 3-D defocusedimage was generated for each testing structure byperforming a convolution between the original setof images and the cone . The resulting blurredimages were analogous to the ones that can beSignal Processing

obtained in the experimental situation in which aset of images taken with an optical microscope(using the technique of optical sectioning) give 3-Dinformation about an object to be studied . A resto-ration process was performed on the blurredimages, and side-views were obtained. Calculationswere made on 128 x 128 x 32 matrices, to whichsome zero points were added to avoid aliasingeffects . The 2-D Fourier transforms of the side-views were obtained by linearly interpolating thedata provided by the Wiener restoration of the 3-DFourier transform of the objects .

Figure 2 shows a set of defocused planes of thesimulated pyramid used to test the reconstructionprocedure. The simulated sections mimic the effectof an optical microscope with a numerical apertureof 0 .5 units and hence a limiting view angle ofabout 60' . Projections of the restored pyramid andof the staircase are displayed in Figs . 3 and 4,respectively, for different view angles .

The objects to be reconstructed have beenassumed to be transparent to avoid the problemof hidden surfaces and to describe the 3-D imageformation by means of a convolution . If an object,or a part of it, does not verify the high-transmitt-ance hypothesis, then the image formation cannotbe modelled through a convolution as (1) is validunder the hypothesis that the psf is independent ofthe object and its position. The psf's dependenceon the object can be easily noticed in Fig . 5, whichshows a section of a 3-D pyramid composed ofnine planes. The half-plane A is opaque, while therest of the object is transparent . It is evident thatif the optical system is focused on the point 0,the object within the psf cone does not entirelycontribute to the image formation, and only thepoints within the part of cone not obscured by thehalf-plane A form the image .

From a theoretical point of view, in this case,the reconstruction process cannot be performedthrough a deconvolution . On the other hand, if theobject has only some limited opaque regions, orhas a low transmittance, one can expect that aWiener filter, implemented using the OTF calcula-ted from a psf conical in shape, can still work well .

T. Tommasi et al. l 3-D reconstruction in optical microscopy

363

Fig . 2 . Eight defocused planes of a pyramid, taken at different focus position .

Fig . 3 . Side-views of a pyramid generated in a 128 x 128 x 32 pixel matrix . The sight direction, z', is determined by two angles : thefirst is the angle between the optical z axis and z', and the second is the angle between z' and the vertical y axis of the plane . From

left to right and from top to bottom the view angles are (I) 0-90°, (2) 20-90°, (3) 20-70°, (4) 20-100° .

In order to test the restoration in the presence of performed a defocusing operation in the spatialan opaque object, some simulations were carried domain, by using a psf dependent on the objectout . We considered a pyramid composed of eight (see Fig . 5). Only a 64 x 64 x 8 pixel matrix wasplanes, each discretized into a 64 x 64 pixel matrix ; considered, due to the very time-consuming opera-a half-plane was assumed to be opaque . We

tion. After generating a set of 16 images of theVol. 32, No . 3.

June1993

364

Fig. 4 . Side-views of a staircase . The related view angles are (from left to right and from top to bottom) (1) 20- (-110)`, (2) 20 80',(3) 20-(-80°), (4) 20-110' . The negative angles of (1) and (3) correspond to a point of view on the left of the staircase .

Fig. 5 . A section of a pyramid with the opaque half-plane A .The psf is dependent on the object .

pyramid, taken for different focus positions, imagerestoration was accomplished with a Wiener filterin the frequency domain, and side-views of theobject were reconstructed . Results of two pro-jections are shown in Fig . 6 . They permit us toSignal Promssing

T. Tommasi et al. / 3-D reconstruction in optical microscopy

conclude that 3-D restoration by a deconvolution,even through mathematically incorrect, yieldssatisfactory results . In particular, projections per-formed for particular view directions give recon-structed views where the dark region due to anopaque does not considerably affect theimages .

In order to evaluate the possible ill-posed natureof the 3-D deconvolution, some tests were per-formed with noisy images. Noise was added to theblurred images used for the pyramid reconstruc-tion, and this operation was performed by ran-domly adding an integer number (over the range-10 to + 10) to the gray-level pixel values (from 0to 200) .

Results allow us to conclude that the reconstruc-tion still works well, as the restored images showthe projections of the pyramid, even if noisereduces their quality (see Fig . 6) .

In the present work, it has been shown that a3-D defocusing technique can yield correct projec-tions of an image object, within the physical con-straints imposed by the characteristics of theimage-formation system . In real experiments, thehypothesis of highly transparent specimens cannotbe completely verified ; however, simulations

confirmed that satisfactory results can be achievedalso in the presence of noise and of absorption dueto a 3-D object . Promising preliminary results wereobtained by using experimental optical sections ofneutrophil cells . Difficulties are generallyencountered in acquiring sets of images, as it is notan easy matter to properly adjust, plane by plane,the defocusing device . Further tests have beenplanned, which will exploit the results of hardwareand software modifications aimed at achieving anautomatic control of optical plane acquisition .Moreover, the use of measured psf can consider-ably improve results .

References

[I] D. Agard and J . Sedat, "Three-dimensional architectureof a polytene nucleus", Nature, Vol. 302, 1983, pp . 676681 .

12] D. Agard, Y. Hiraoka, P . Shaw and J_ Sedat, "Fluor-escence microscopy in three dimensions", in : D .L. Taylorand Y .L. Wang, eds ., Methods in Cell Biology, Vol . 30 :Fluorescence Microscopy of Living Cells in Culture, PartB, Academic Press, New York, 1989 .

[31 B . Bianco and A . Diaspro, "Analysis of three-dimensionalcell imaging obtained with optical microscopy techniquesbased on defocusing" . Cell Biophysics, Vol . 15, 1989,pp.189-199 .

T. Tommasi et al. / 3-D reconstruction in optical microscopy

365

Fig. 6 . Side-view of a pyramid with an opaque region (on the left) or with noise (on the right) . From left to right and from top tobottom the view angles are (1) 20 80°, (2) 20-(-110)°, (3) 20-(-90)°, (4) 20-(-90°) .

[4] K.R . Castleman, Digital Image Processing, Prentice Hall,Englewood Clifs, NJ, 1979 .

[5] A. Diaspro, M. Sartore and C . Nicolini, "Three dimemsional representation of biostructures imaged with anoptical microscope : Part A, Digital optical sectioning" .Image Vision Compul . . Vol . 8. No . 2, 1990. pp.130-134.

[6] A. Erhardt, G . Zinser, D . Komitowski and K . Bille,`Reconstructing 3-D light-microscopic images by digitalimage processing', Appl. Optics, Vol. 24, No . 2, 1985,pp-194-200.

[7] F. Fay, K . Fogarty and l . Coggins, "Analysis of moleculardistribution in single cells using a digital imaging micro-scope", in : P. DeWeer and B . Sazburg. eds., OpticalMethods in Cell Physiology, Wiley, New York, 1986 .

[8] B.R . Frieden, "Optical transfer of the three-dimensionalobject", J. Opt . Sac . Amer ., Vol. 57, 1967, pp . 56-66 .

[9] S . Frisken Gibson and F. Lanni, "Measured and analyticalpoint spread functions of the optical microscope for usein 3-D optical serial sectioning microscopy", in : OpticalMicroscopy far Biology, Wiley-Liss, New York, 1990,pp. 109 118 .

[10] P . Hariharan, Optical Holography, Cambridge Univ .Press, Cambridge . 1983 .

[11] S . Kawata and S . Minami, "The principle and applicationsof optical microscope tomography", Acta Hismchem .Cytochem ., Vol . 19, No . 1, 1986, pp. 73-81 .

[121 R. Rangayyan, A.P. Dhawan and R. Gordon, "Algo-rithms for limited-view computed tomography : An annot-ated bibliography and a challenge", Appl. Opt ., Vol . 24,No. 23, 1985, pp . 4000-4012 .

[13] P.A . Stokseth, "Properties of a defocused optical sys-tem", J. Opt. Soc. Amer., Vol. 59, No . 10, 1969, pp .1314-1321 .

Vol . 32,No .3,June 1993

366

T. Tommasi et al. 3-D reconstruction in optical microscopy

[ 14] N. Streibl, "Three-dimensional imaging by a microscope",

[161 K.C. Tam, V . Pert-Mendez and B . Macdonald, "3-DJ. Opt Sac. Amer ., Vol . 2, 1984, pp. 121-127.

object reconstruction in emission and transmission tomo-[151 K.C. Tam and V. Perez-Mendez, "Limits to image recon-

graphy with limited angular input", IEEE Trans . Nuclearstruction from restricted angular input", IEEE Trans .

Sci., Vol. 26, No . 2, 1979, pp. 2797-2805 .Nuclear Sci., Vol . 28 . No, 1, 1981, pp . 179-183 .

Signal Processing