Single-Lens Camera Based on a Pyramid Prism Arrayto Capture Four ImagesWen-Shing SUN1, Chuen-Lin TIEN

2�, Chien-Yue CHEN3, and Der-Chin CHEN

2

1Department of Optics and Photonics, National Central University, Chungli, Taiwan 32001, R.O.C.2Department of Electrical Engineering, Feng Chia University, Taichung, Taiwan 40724, R.O.C.3Graduate School of Optoelectronics, National Yunlin University of Science and Technology, Yunlin 64002, Taiwan 40724, R.O.C.

(Received August 12, 2012; revised December 27, 2012; Accepted December 28, 2012)

An optical design with a pyramid prism array is employed to simultaneously capture four images. The apex angledesign of an upper and lower prism array is to form the upper and the lower images of the object. The design of the leftand right prism array is to develop the left and right images. In the prism array, the unwanted image can be eliminatedby blacken process in the plane gap between each prism. The optical design was based on the pyramid prism array and asingle-lens to make the system light, portable and low-cost. # 2013 The Japan Society of Applied Physics

Keywords: optical design, pyramid prism array, stereoscopic camera

1. Introduction

In 19th century, according to reliable documentarymaterials,1) Wheatstone, an English scientist, created thefirst stereoscopic drawings based on binocular vision earlyin 1838 and consequently invented stereoscope in the sameyear. In 1852, Wheatstone2) used two prisms to buildstereoscope that two pictures were simultaneously taken bytwo cameras and fused them together for display.

In the present day, there are miscellaneous stereoscopicdisplay technologies in the world.3) However, a stereo imagepair is always necessary for forming binocular depth.4)

Accordingly, the technique to capture stereoscopic imagealso plays an important role in the stereoscopic displaytechnology. To date, stereoscopic imaging techniquesinclude dual-lens imaging5) and single-lens imaging. Thecomponents are often divided into reflection6–8) andrefraction.9) The reflective components include mirrors,while refractive ones include prisms or lenses. In 2000,Lee and Kweon10) proposed using two prisms in front of theCCD of a single-lens camera to take dual-view pictures.Afterward Xiao et al.9) proposed the pyramid prism tocapture sub-images; however, the disadvantage of theseconfigurations is bulky and hard to combine a CCD into aportable device. Chen et al.11,12) suggested replacing theprism with a prism array that could produce a stereoscopicimage pair on a CCD and use for aerial photographs. Thispaper mainly referred to Refs. 9 and 11 using a pyramidprism array camera synchronously capture four images.Finally, we used three pyramid prism array cameras toobtain four stereoscopic image pairs.

2. Theory

The pyramid prism array and lens were used to designa four-image capturing system. The horizontal beam isrequired to pass through pyramid prism, and then the exitbeam at horizontal half field of view (HHFOV) is thehorizontal HFOV of the incident camera lens. Similarly, the

vertical beam passing through the pyramid-prism, and thenthe exit beam at vertical half field of view (VHFOV) is thehalf VFOV of the incident camera lens.

2.1 HFOV of lensThe optical system consists of the CMOS sensor, lens

and the pyramid-prism array. The CMOS image sensor isOmniVision OV10620, and the specifications are shown inTable 1. The pixels number is 768� 506 pixels. The pixelsize is 6� 6 �m2, and the effective area of sensor is4:608� 3:036mm2. Its diagonal length is 5.52mm. If thehorizontal height is 2.304mm and the vertical height is1.518mm, then the diagonal length is 2.76mm.

Table 2 shows the specification of the lens. The focallength and image height are 4.88 and 2.76mm, respectively.The F-number (F/#) is 2.8. Figure 1 indicates the relation-ship between the HFOV (�) and the image height (h) forimage formation at the far distance.

� ¼ tan�1 h

f

� �ð1Þ

Table 1. Specifications of OV10620.

Item Specification

Sensor OV10620Pixel number 768� 506

Pixel size (�m2) 6� 6

Image area (mm2) 4:608� 3:036

Table 2. Specifications of the lens.

Item Specification

Focal length (mm) 4.88Image height (mm) 2.76F/# 2.8HFOV (deg) 29.49HHFOV (deg) 25.27VHFOV (deg) 17.28

�E-mail address: cltien@fcu.edu.tw

OPTICAL REVIEW Vol. 20, No. 2 (2013) 145–152

145

where f is the focal length of the camera and f is equal to4.88mm. The maximum image height is 2.76mm at thediagonal of the sensor. The HFOV is 29.49�. The maximumhorizontal image height is 2.304mm. The HHFOV is 25.27�.Similarly, the maximum vertical image height is 1.518mmand the VHFOV is 17.28�.

2.2 Sign convention for anglesThe sign convention of a single prism is shown in Fig. 2.

The angle is positive if the ray is rotated clockwise to reachthe normal; meanwhile, the angle is negative for counter-clockwise direction. I1 and I2 are the incident angles atthe first surface and the second surface of the prism,respectively. I01 and I02 are the refracted angles at the firstsurface and the second surface of the prism, respectively. Ais the apex angle of the prism. The prism apex angle ispositive for the angle turning up or right. The prism apexangle is negative for turning down or left. n is the refractiveindex of the prism. The relation between the incident angleand the refractive angle follows the Snell’s law at the firstside of the prism. When the ray comes from the air into theprism, then

sin I1 ¼ n sin I01: ð2ÞSimilarly, the relation between the incident angle and therefraction angle at the second surface is

n sin I2 ¼ sin I02; ð3Þand the apex angle is

A ¼ I01 � I2: ð4ÞConsidering the relationship among I1, I

02, and A, we obtain

I1 ¼ sin�1ðn sin I01Þ¼ sin�1½n sinðI2 þ AÞ�¼ sin�1½sin I02 cosAþ ðn2 � sin2 I02Þ1=2 sinA�: ð5Þ

2.3 Horizontal and vertical FOVs of the pyramid prismFigure 3 is the structure of the pyramid prism. If the

horizontal apex angle is A, and vertical apex angle is B.Rays are passing through the left half of the pyramid prism,the relationship between the incident angle and the exit angleis shown in Fig. 4. We used the red dotted line to indicatethe direction of the optical axis (z-axis). The black dottedline represents the ray on the normal of the surface. If theprism apex angle is deviated to the left, then the prismapex angle (i.e., A0) is negative. The chose material ispoly(methyl methacrylate) (PMMA) and its refractive indexis 1.491. A left ray of an object strikes the left part surfaceof the pyramid prism at an angle of incidence � (namely� ¼ I1), as shown in Fig. 4(a). The ray travels the prism andthen is deviated by an exit angle of a, which is the anglebetween the ray and z-axis, thus the HHFOV is a ¼ 25:27�.The angle of refraction at the second surface is equal toI02 ¼ 25:27� � A0. Substituting this into Eq. (5), we obtain

� ¼ sin�1fsinð25:27� � A0Þ cosA0

þ ½1:4912 � sin2ð25:27� � A0Þ�1=2 sinA0g: ð6ÞSimilarly, a right ray of an object hits the left part surfaceof the pyramid prism at an angle of incidence �0 (namely�0 ¼ ��), as shown in Fig. 4(b). We obtain a ¼ 0 andI02 ¼ �A0, then

�� ¼ sin�1fsinð�A0Þ cosA0

þ ½1:4912 � sin2ð�A0Þ�1=2 sinA0g ð7Þusing Eqs. (6) and (7), we have � ¼ 11:19� and A0 ¼�22:05�.

If a left ray of an object hits the right part surface of thepyramid prism at an angle of incidence � (� > 0) with apositive vertex angle A, as shown in Fig. 5(a). An exit angleof a is equal to zero. The angle of refraction, I02 ¼ �A, islocated at the second surface of the prism, thus we get

Fig. 1. (Color online) Image formation of an object at the fardistance indicates the relationship between the HFOV (�) and theimage height (h).

Fig. 2. Layout of ray passing through the prism.

Fig. 3. (Color online) Structure of the pyramid prism.

(a) (b)

Fig. 4. (Color online) A ray strikes the left part surface of thepyramid prism to indicate the relation between the incidence angleand the exit angle. (a) Left ray incidence. (b) Right ray incidence.

OPTICAL REVIEW Vol. 20, No. 2 (2013)146 W.-S. SUN et al.

� ¼ sin�1fsinð�AÞ cosAþ ½1:4912 � sin2ð�AÞ�1=2 sinAg: ð8Þ

If a right ray of an object hits the left part surface of thepyramid prism at an angle of incidence �0 (�0 ¼ ��) asshown in Fig. 5(b), a ¼ �25:27� and I02 ¼ �25:27� � A areobtained, then

�� ¼ sin�1fsinð�25:27� � AÞ cosAþ ½1:4912 � sin2ð�25:27� � AÞ�1=2 sinAg ð9Þ

using Eqs. (8) and (9), we have � ¼ 11:19� and A ¼ 22:05�.Considering the ray comes from the bottom of the object

and strikes the upper part of the pyramid prism as shownin Fig. 6(a). Assuming that the vertex angle B is positiveand the incident angle is ’. The ray is passing through aprism, an exit angle of b is equal to zero along the z-axisdirection. The angle of refraction, I02 ¼ �B, is located atthe second surface of the prism, thus the equation can beexpressed as

’ ¼ sin�1fsinð�BÞ cosBþ ½1:4912 � sin2ð�BÞ�1=2 sinBg: ð10Þ

If the ray comes from the top of the object and hits the upperpart of the pyramid prism as shown in Fig. 6(b). We assumethat the incident angle is ’0 ¼ �’. The angle between theray and z-axis is b ¼ �17:28� for the vertical HFOV of thecamera. The angle of refraction is I02 ¼ �17:28� � B, thus

�’ ¼ sin�1fsinð�17:28� � BÞ cosBþ ½1:4912 � sin2ð�17:28� � BÞ�1=2 sinBg ð11Þ

by using Eqs. (10) and (11), we have ’ ¼ 8:16� andB ¼ 16:31�.

Considering the ray comes from the bottom of the objectand strikes the lower part of the pyramid prism, as shown inFig. 7(a). Assuming that the vertex angle B0 is negative and

the incident angle is ’. As the ray is passing through theprism, an exit angle of b is equal to 17.28�. The angle ofrefraction, I02 ¼ 17:28� � B0, is located at the second surfaceof the prism, thus the angle is

� ¼ sin�1fsinð17:28� � B0Þ cosB0

þ ½1:4912 � sin2ð17:28� � B0Þ�1=2 sinB0g: ð12ÞSimilarly, when the ray comes from the top of the object andthen strikes the lower part of the pyramid prism, as shownin Fig. 7(b). Assuming that the incident angle is ’0 ¼ �’,then we have b ¼ 0 and I02 ¼ �B0, thus

�� ¼ sin�1fsinð�B0Þ cosB0

þ ½1:4912 � sin2ð�B0Þ�1=2 sinB0g ð13Þusing Eqs. (12) and (13), we get ’ ¼ 8:16� and B0 ¼�16:31�.

As shown in Fig. 8, at the right and left apex angles of thepyramid prism array are 22.05 and �22:05�. At the upperand lower apex angles of the pyramid prism array are 16.31and �16:31�. The maximum exit angle of the ray passingthrough the prism should match the HHFOV and theVHFOV of the camera lens. Figure 8(a) illustrates thehorizontal section of the pyramid prism array. On its leftpart, an incident angle of 11.19� led to an exit angle of25.27� and an incident angle of �11:19� incurred an exitangle of 0�. On its right part, an incident angle of 11.19�

made an exit angle of 0�, meanwhile, an incident angle of�11:19� led to an exit angle of �25:27�. Figure 8(b) showsthe vertical section of the pyramid prism array. On its upperpart, an incident angle of 8.16� led to an exit angle of 0� andan incident angle of �8:16� incurred an exit angle of�17:28�. On its lower portion, an incident angle of 8.16� ledto an exit angle of 17.28�, meanwhile, an incident angle of�8:16� incurred an exit angle of 0�.

The pyramid prism array consists of the PMMA material.The area of the plate for the pyramid prism array is643.87mm2 (20:765� 31mm2) which located in front ofthe lens by 30mm, as shown in Fig. 9(a). Figure 9(b)illustrates the size of a single prism. We have L1 ¼2:077mm, L2 ¼ 1:5mm, and d ¼ 0:608mm. The total areais 311.55mm2 for a total of 200 pieces and occupies about48.41% of the entire board. Figure 9(c) shows a singleplane gap structure. The plane gap surface is blackenedprocessing to make it easy absorption for the incident rays.Its dimension is L1 ¼ 2:077mm, L2 ¼ 1:55mm, L3 ¼0:05mm, and d ¼ 1:0mm. The total area is 332.32mm2

(a) (b)

Fig. 5. (Color online) A ray strikes the right part surface of thepyramid prism to indicate the relation between the incidence angleand the exit angle. (a) Left ray incidence. (b) Right ray incidence.

(a) (b)

Fig. 6. (Color online) A ray strikes the upper surface of thepyramid prism to indicate the relation between the incidence angleand the exit angle. (a) Bottom ray incidence. (b) Top ray incidence.

(a) (b)

Fig. 7. (Color online) A ray strikes the lower surface of thepyramid prism to indicate the relation between the incidence angleand the exit angle. (a) Bottom ray incidence. (b) Top ray incidence.

OPTICAL REVIEW Vol. 20, No. 2 (2013) 147W.-S. SUN et al.

for the plane gap and it occupies 51.59% of the entire board.The pyramid prism array is divided into four areas (I, II, III,and IV). The areas I and IV are completely symmetricalstructure. The areas II and III are also symmetrical structure.

3. Lens Design

The lens layout and design data are shown in Fig. 10 andTable 3, respectively. We totally used six lenses, the last oneis a flat glass to protect the sensor. There are two plasticaspherical lenses and three spherical glass lenses. The imagequality for the camera lenses are listed in Table 4. In theFOV of the camera and spatial frequency of 45 lp/mm, thevalue of the modulation transfer function (MTF) is requestedto be larger than 0.4. The distortion is less than 1% withinthe FOV. The lateral color is less than 1 pixel size (6 �m).Finally, the imaging edge illumination to the centerillumination ratio (relative illumination) is larger than50%. Figures 11 to 14 illustrate its MTF, distortion, lateralcolor and relative illumination, respectively.

Figure 11 indicates the MTF plot. The abscissa representsthe spatial frequency value (we choose 45 lp/mm) and theordinate is the MTF values. The MTF values are greater than46% for all of image height. Figure 12 shows the opticaldistortion plot in terms of three wavelengths. The ordinate isthe image height. The maximum value is one of ten equaldivisions for the maximum image height. The abscissa is an

optical distortion and the units are expressed as a percentage.In this figure, distortion values are less than 1%. Figure 13illustrates the lateral color. The ordinate is the image height.The origin in the abscissa is the imaging position at thecenter wavelength of 0.5876 �m. The abscissa is thevariation amount of the imaging location from the centerwavelength (the unit is micrometer). The red curverepresents the change of the imaging position in thevisible-light wavelength of 0.6563 �m. The blue curverepresents the amount of the imaging changes in visible

(a)

(b)

Fig. 8. (Color online) FOV of the pyramid prism array.(a) Horizontal section of the micro prism array and deviation oflight rays. (b) Vertical section of the micro prism array anddeviation of light rays.

(a)

(b)

(c)

Fig. 9. (Color online) The plate of pyramid prism array.(a) Dimensions of the pyramid prism array. (b) Dimensions of asingle prism. (c) Dimensions of a single plane gap.

OPTICAL REVIEW Vol. 20, No. 2 (2013)148 W.-S. SUN et al.

Table 4. Image quality requirements of the lens.

Item Target

MTF (45 lp/mm) >0:4Distortion <1%Lateral color <6 �mRelative illumination >0:52

Fig. 10. (Color online) Lens layout with six lenses, the last one isa flat glass. There are two plastic aspherical lenses and threespherical glass lenses.

Table 3. Lens data.

Number Surface MediumRadius(mm)

Thickness(mm)

Opticalmaterial

0 ObjectAir

1 11 SphereGlass

�6:0442561.025861 BK72 Sphere

Air23.893262

0.3810003 PlaneGlass

11.514254 LASF334 Sphere

Air�7:635652

0.3474465 PlaneAir

10.4588456 (Stop) Sphere

Glass3.677487

1.573440 N-LAK347 SphereAir

�6:2011450.3903548 Asphere

Plastic�2:635099

0.886775 POLYSTYR9 SphereAir

3.0242930.56685910 Asphere

Plastic5.867991

1.792991 E48R11 SphereAir

�2:9545230.90797612 Plane

Glass1

0.700000 B27013 PlaneAir

11.35257814 Image 1

Order of asphericalcoefficients

Surface 8 Surface 10

4th 1:254956� 10�2 �1:861418� 10�2

6th 1:866073� 10�3 3:742390� 10�3

8th �6:330343� 10�3 �3:529746� 10�4

10th 2:778006� 10�3 1:849479� 10�5

Fig. 11. (Color online) MTF plot of the lens design. Thehorizontal axis represents the spatial frequency value, we chose45 lp/mm; the vertical coordinate is the MTF values.

Fig. 12. (Color online) The distortion plot in terms of threewavelengths. The ordinate is image height. The abscissa isdistortion.

Fig. 13. (Color online) Lateral color plot. The ordinate is imageheight. The abscissa is the variation amount of the imaging locationfrom the center wavelength.

OPTICAL REVIEW Vol. 20, No. 2 (2013) 149W.-S. SUN et al.

light with a medium and short wavelength of 0.4861 �m. Thelateral color is the difference of the changes between thelong-wavelength imaging position and short-wavelengthimaging position. The lateral color is less than 4 �m.Figure 14 indicates the relative illumination. The ordinateis the image height and the abscissa is the relativeillumination. The relative illumination is greater than 0.52.

4. Simulation Results

Figure 15 is the shooting results of the traditional camera.Figure 15(a) is the picture of the object, and Fig. 15(b)shows the simulation result of the optical system without

pyramid prism array, in which the distance from the objectto the lens is 850mm. Afterwards, the pyramid prism arrayis inserted between the lens and the object, and the lensstood 30mm apart. The pyramid prism array is withoutprocessing in the pyramid prism array camera, the simula-tion results have five images, as shown in Fig. 16. Thecentral image is unwanted, because the ray through the prismarray will form images by the lens of the plane gap betweeneach prism. The unwanted image is imaging formation dueto the ray through the pyramid prism array board accountedfor 51.59% of the area of the plane gap. We accounted for48.41% of the area of the pyramid prism to serve asabsorption. A plate glass was arranged in front of the lens.The horizontal incident and the vertical incident angles are11.19 and 8.16�, as shown in Fig. 8. Without considering thepyramid prism array structure, only for the optical system ofthe plane gap structure is shown in Fig. 17. By using Eq. (1),the imaging length can be calculated as 1.931mm and thewidth is 1.399mm, as shown in Fig. 18. If we make theabsorption simulation in the plane gap area between eachprism, then the central image can be eliminated as shownin Fig. 19.

Fig. 14. (Color online) Relative illumination, the ordinate isimage height and the abscissa is the relative illumination.

(a)

(b)

Fig. 15. (Color online) The shooting results of the traditionalcamera. (a) Picture of the object. (b) Simulation result of the imageformation without pyramid prism array.

Fig. 16. (Color online) Five images and pyramid prism arraywithout processing.

Fig. 17. (Color online) Without considering the pyramid prismarray structure, only for the optical system of the plane gapstructure.

OPTICAL REVIEW Vol. 20, No. 2 (2013)150 W.-S. SUN et al.

5. Stereoscopic Image Pair

The shooting result of a pyramid prism array camerais demonstrated in Fig. 19. In these four images, the shapeand direction of the two adjacent images are different andthey do not form the image pair. Although the diagonalimages are similar, but the incident angle is not thesame, they do not form image pairs. If there are threepyramid prism array cameras under an equal objectdistance, the shooting direction is separated by 15�, then

four stereoscopic image pairs can be obtained, as shown inFig. 20.

We exchanged the four images for camera 1 and camera 3as shown in Fig. 21. Camera 1 and camera 2 shooting imagesare separated by an angle of 15�. Upper-Left 1 and Upper-Left 2 are the first image pair. Lower-Left 1 and Lower-Left 2are the second image pair. Similarly, camera 2 and camera 3shooting images are separated by an angle of 15�. Upper-Right 2 and Upper-Right 3 are the third image pair, whileLower-Right 2 and Lower-Right 3 are the fourth image pair.

Fig. 18. (Color online) Unwanted central image.Fig. 19. (Color online) Four images with the absorptionsimulation at the plane gap area between each prism.

Fig. 20. (Color online) Schematic diagram of three pyramid prism array cameras to obtain four stereoscopic image pairs.

OPTICAL REVIEW Vol. 20, No. 2 (2013) 151W.-S. SUN et al.

6. Conclusions

The chromatic aberration and aberration corrections forthe pyramid prism array camera are important issues to besolved. The pyramid prism array camera was simulated bythe LightTools software. The most number of rays areblocked by the pyramid prism array, more ray tracing isnecessary and the simulation time will be longer. Using thepyramid prism array camera, the number of cameras can bereduced by half for the 360� surround photography. Thus thestereoscopic shooting will be future research topics. Thepyramid prism array camera can generate four images, andcan be used for aerial photography map. Using the pyramidprism array camera for four-sided shooting is also anattractive research topic.

Acknowledgements

The author would like to thank the National Science Council ofthe Republic of China, Taiwan. This work was supported in part bythe NSC under contract numbers NSC 101-2221-E-008-107 andNSC 101-2221-E-035-055.

References

1) C. Wheatstone: Philos. Trans. R. Soc. London 128 (1838)371.

2) C. Wheatstone: Philos. Trans. R. Soc. London 142 (1852) 1.3) J. Y. Son, B. Javidi, and K. D. Kwack: Proc. IEEE 94 (2006)

502.4) A. J. Parker: Nat. Rev. Neurosci. 8 (2007) 379.5) S. Y. Park, N. Lee, and S. Kim: Opt. Eng. 43 (2004) 3130.6) T. P. Pachidis and J. N. Lygouras: J. Intell. Robot. Syst. 42

(2005) 135.7) J. Gluckman and S. K. Nayar: IEEE Trans. Pattern Anal.

Mach. Intell. 24 (2002) 224.8) J. Zhu, Y. Li, and S. Ye: Opt. Eng. 45 (2006) 1.9) Y. Xiao and K. B. Lim: Image Vision Comput. 25 (2007)

1725.10) D. S. Lee and I. S. Kweon: IEEE Trans. Robotics Autom. 16

(2000) 528.11) C. Y. Chen, T. T. Yang, and W. S. Sun: Opt. Express 16

(2008) 15495.12) Q. L. Deng, C. Y. Chen, S. W. Cheng, W. S. Sun, and B. S.

Lin: Opt. Commun. 285 (2012) 5001.

Fig. 21. (Color online) Distribution of three camera images through image processing.

OPTICAL REVIEW Vol. 20, No. 2 (2013)152 W.-S. SUN et al.