A microscale camera using direct Fourier-domain scene capture

Download A microscale camera using direct Fourier-domain scene capture

Post on 02-Oct-2016




6 download

Embed Size (px)


  • A microscale camera using directFourier-domain scene capture

    Patrick Robert Gill,1,2,* Changhyuk Lee,1 Dhon-Gue Lee,1 Albert Wang,1 and Alyosha Molnar11School of Electrical and Computer Engineering, Cornell University, 223 Phillips Hall, Ithaca, New York 14853, USA

    2Department of Psychology, Cornell University, 211 Uris Hall, Ithaca, New York 14853, USA*Corresponding author: prg56@cornell.edu

    Received May 13, 2011; revised June 23, 2011; accepted July 3, 2011;posted July 7, 2011 (Doc. ID 147442); published August 1, 2011

    We demonstrate a chip-scale (

  • devices satisfying Eq. (3) exactly for 520 nm (invacuum) light. An array using only devices optimal for520 nm light would leave large gaps in the tiling of b.To better span these gaps, our ensemble of devices in-cludes designs optimal for different wavelengths, butall fairly close to optimal devices for 520 nm light (seered open circles of Fig. 4). We laid out 10 m 10m de-vices in concentric rings around four central devicessimilar to those reported in [11] to capture low frequencyinformation. Each design constitutes one ring of ASPs,with larger rings containing devices of higher b (Fig. 5).The greater number of ASPs for devices with higher bpermits more grating orientations. This is desirable, sincethe number of independent Fourier components a givendesign must observe is proportional to b. Devices oppo-site each other have values [see Eq. (1)] 90 apart tocapture magnitude and phase information of each Four-ier component. We built two complementary arrays(with values differing by 180; otherwise identical) of38 38 unique ASPs (2,888 sensors total) in an unmodi-fied 180 nm CMOS process (Fig. 5).

    Fourier completeness relies on our measurements til-ing Fourier space up to our Nyquist limit. As the range ofallowed incident angles increases, so does the frequencyresolution needed to cover all Fourier space. Specifically,the relationship between the maximum allowable eccen-tricity of incident light (relative to the normal to thedevice) h and the maximum difference in b between con-secutive designs is

    h 1802


    : 4

    The largest gap between implemented devices (open cir-cles of Fig. 4) gives a b of 2.62, such that the ensembletransfer functions of either of the 1,444-sensor arraysyield overcomplete coverage of Fourier space for imageswith h < 48:6.To characterize the array, we presented a series of ran-

    dom calibration images [Fig. 6(a)] to the sensor using asquare CRT area 20 cm on a side, 22:86 cm from the PFCA


    Response vs. Angle

    Fig. 2. (Color online) Angle-sensitive pixels. Incident lightinteracting with a grating composed of an upper metal layerproduces an interference pattern at the depth of the second-layer grating. Light is either passed or blocked depending onthe alignment of the interference pattern and the second layer.This alignment, in turn, is sensitive to changes in incident angle;the net effect is that the light passed by an ASP depends sinu-soidally on incident angle.

    Fig. 3. Decomposition of natural images into Fourier compo-nents. (a) All images can be expressed as a sum of 2D Fourierbasis functions [e.g., (b)] by taking the sum over all values in(c) the basis-scaled image.

    5 10 15 20 25 30 35 400.5


    1.5Optimal for 520nm lightConfigurations Used




    h p


    Effective bFig. 4. (Color online) Selecting devices for the PFCA. Filledcircles indicate manufacturable devices with maximal mfor 520nm light; open circles indicate the suite of devices wemanufactured.

    570 Microns

    Fig. 5. (Color online) Manufactured PFCA. Concentric ringsof ASPs with increasingly high sinusoidal periodicity yield acomplete Fourier description of the light intensity from thefar field. Slowly varying orientation is evident from the enlargedsection, where schematics show different metal layers in differ-ent colors.

    Fig. 6. (Color online) PFCA Calibration. Transfer functions ofeach pixel are found by presenting (a) calibration images on aCRT screen to (b) the array and performing reverse correlationbetween the observed photocurrent of each sensor and the im-age presented. (c) The kernels of three ASPs are shown; theseresemble Fourier components.

    2950 OPTICS LETTERS / Vol. 36, No. 15 / August 1, 2011

  • [12] for an h of 31:7 at the squares corners. With thismaximum eccentricity well under the limit of Fouriercompleteness [48:6; see Eq. (4)], the PFCAs outputsare somewhat redundant. We then used linear systemidentification tools [13] to reconstruct the kernel of eachASP [Fig. 6(c)]. Afterward, we presented various testimages on the CRT [Fig. 7(a)], captured the ASP re-sponses with an accumulation time of 16:7ms, and suc-cessfully reconstructed the images [Fig. 7(b)] up to theNyquist limit of our sensor, set by the highest-b designin our array bmax 39. Our effective resolution onthe active square was approximately 20 20 pixels; thenumber of resolvable pixels scales with b2max.We have demonstrated a PFCA: an ASP array that re-

    lates complete Fourier information (up to a maximumspatial frequency) about the far field without usingfocusing optics. The device is manufactured in an unmo-dified semiconductor process and requires no externaloptics or alignment. Its construction cost and resolu-tion specifications fill gap between the smallest miniatur-ized cameras and single photodiodes (Fig. 1), makingit a suitable choice for a large range of cost- andsize-sensitive applications that cannot be served withfocusing optical systems.

    We would like to thank the Defense Advanced Re-search Projects Agency (DARPA), which supported thisresearch via a Young Faculty Award to A. Molnar, and theNational Institutes of Health (NIH), who helped fund thiswork under R21 grant EB 009841-01.

    References and Notes

    1. B. Batchelor, D. Hill, and H. Hodgson, Automated VisualInspection (Elsevier Science, 1985).

    2. T. Lillesand, R. Kiefer, and J. Chipman, Remote Sensing andImage Interpretation, 5th ed. (Wiley, 2004).

    3. C. Urmson, J. Anhalt, D. Bagnell, C. Baker, R. Bittner,M. Clark, J. Dolan, D. Duggins, T. Galatali, C. Geyer,M. Gittleman, S. Harbaugh, M. Hebert, T. M. Howard,S. Kolski, A. Kelly, M. Likhachev, M. McNaughton, N. Miller,K. Peterson, B. Pilnick, R. Rajkumar, P. Rybski, B. Salesky,Y.-W. Seo, S. Singh, J. Snider, A. Stentz, W. R. Whittaker, Z.Wolkowicki, J. Ziglar, H. Bae, T. Brown, D. Demitrish, B.Litkouhi, J. Nickolaou, V. Sadekar, W. Zhang, J. Struble,M. Taylor, M. Darms, and D. Ferguson, J. Field Robot.25, 425 (2008).

    4. N. Bergstrm, J. Bohg, and D. Kragic, in Proceedings ofthe 7th International Conference on Computer VisionSystems, Vol. 5815 of Lecture Notes in Computer Science(Springer-Verlag, 2009), pp. 245254.

    5. A. Wang, P. Gill, and A. Molnar, Appl. Opt. 48, 5897 (2009).6. A. Wang, P. R. Gill, and A. Molnar, in Proceedings of IEEE

    Sensors, 2010 (IEEE, 2010), pp. 17061709.7. H. Talbot, Phil. Mag. Series 3 9, 401 (1836).8. S. Teng, Y. Tan, and C. Cheng, J. Opt. Soc. Am. A 25, 2945

    (2008).9. J. Escofet, M. Milln, and M. Rall, Appl. Opt. 40, 6170

    (2001).10. p is constrained by the minimum manufacturable linewidth

    of the metal, and dmust correspond to one of four availableinter-layer depths of the five metal layers suitable for mak-ing gratings.

    11. C. Koch, J. Oehm, J. Emde, and W. Budde, IEEE J. Solid-State Circuits 43, 1588 (2008).

    12. As 23 cm 570 m (the PFCAs size), images presentedare in the far-field regime where the light field at each pointin the PFCA is essentially identical.

    13. N. Wiener, Extrapolation, Interpolation, and Smoothing ofStationary Time Series: with Engineering Applications(MIT Press, 1949).

    14. L. Paulson, Computer 43, 19 (2010).

    Fig. 7. Image reconstructions. Using the basis functions ob-tained in the calibration phase (Fig. 6), (a) the image presentedwas (b) reconstructed up to the Nyquist limit of our array. Nooff-chip optics were used.

    August 1, 2011 / Vol. 36, No. 15 / OPTICS LETTERS 2951