color stereoscopic images requiring only one color image

Cc

YGBCRIE

SR

ABBRI

1

Ceia�mtpc

nTtrusploode

3a

0

Optical Engineering 46�8�, 087003 �August 2007�

O

olor stereoscopic images requiring only oneolor image

ael Terminal A. Kaminkaar Ilan Universityomputer Science Departmentamat Gan, 52900

srael-mail: [email protected]

arit Semoaanana, Israel

ri Z. Zivotofskyar Ilan Universityrain Sciences Centeramat Gan, 52900

srael

Abstract. Utilizing remote color stereoscopic scenes typically requiresthe acquisition, transmission, and processing of two color images. How-ever, the amount of information transmitted and processed is large, com-pared to either monocular images or monochrome stereo images. Exist-ing approaches to this challenge focus on compression and optimization.This paper introduces an innovative complementary approach to the pre-sentation of a color stereoscopic scene, specialized for human percep-tion. It relies on the hypothesis that a stereo pair consisting of one mono-chromatic image and one color image �a MIX stereo pair� will beperceived by a human observer as a 3-D color scene. Taking advantageof color redundancy, this presentation of a monochromatic-color pair al-lows for a drastic reduction in the required bandwidth, even before anycompression method is employed. Herein we describe controlled psy-chophysical experiments on up to 15 subjects. These experiments testedboth color and depth perception using various combinations of color andmonochromatic images. The results show that subjects perceived 3-Dcolor images even when they were presented with only one color imagein a stereoscopic pair, with no depth perception degradation and onlylimited color degradation. This confirms the hypothesis and validates thenew approach. © 2007 Society of Photo-Optical Instrumentation Engineers.�DOI: 10.1117/1.2772235�

Subject terms: stereoscopic image compression; video bandwidth reduction; bin-ocular vision; stereopsis; color perception; depth perception.

Paper 060586R received Jul. 27, 2006; revised manuscript received Feb. 1,2007; accepted for publication Feb. 9, 2007; published online Aug. 21, 2007.

Introduction

olor and depth perception are important capabilities inveryday human activities and in many applications requir-ng human supervision or control. Both properties, colornd stereoscopic perception, are of significance. Binocularstereoscopic� vision provides us with the ability to deter-ine the 3-D structure and the relative and absolute dis-

ance of objects present in our field of view �FOV�. Colorerception allows segmentation and evaluation of the per-eived objects.

Stereopsis, which is perhaps the most important mecha-ism of depth perception, depends on the use of both eyes.wo disparate images �via the two eyes� are presented to

he brain and combined into a single image that yields ste-eoscopic depth perception. Although many of the cuessed to judge depth involve only one eye �viz., relativeize, parallax, texture, interposition, relative position, linearerspective, shadows�, these monocular depth cues result iness depth-estimation accuracy than disparity, which cannly be obtained from binocular vision. Thus, by using bin-cular disparity information, humans are able to make fineepth judgments that they cannot make when using just oneye.1

Color is additionally important in many applications of-D imaging.2,3 Color is used in segmenting the images,llowing the viewer to distinguish between objects in the

091-3286/2007/$25.00 © 2007 SPIE

ptical Engineering 087003-

image and between objects and their background. It is alsoimportant in distinguishing types of objects or their status�e.g., in medical applications�.

Unfortunately, color stereo imaging requires large ca-pacity of the transmission channel. Some approaches to thischallenge focus on compression and optimization of bothimages by various means.4–7 Other approaches exploit thefact that the left and right images differ only in small areas,and thus improved compression can be achieved by takingadvantage of the redundancies in the stereo pair. Thesetechniques often rely on disparity compensation,8,9 a pro-cess that can be computationally complex. Nearly all of thetechniques used for stereo compression assume at least twocameras that yield similar chromatic information. Section 2includes a review of related work.

This paper evaluates a novel complementary approach inwhich a single color image is combined with a singlemonochromatic �grayscale� image. It relies on the hypoth-esis that the sophisticated human visual perception systemwill fuse the two images that differ in chromatic contentinto one perceived color stereoscopic image. Thus, it isspecifically targeted towards human perception. Using thisapproach, bandwidth requirements may be reduced evenbefore any compression technique is applied.

We experimentally validate the hypothesis by conduct-ing a set of experiments with human subjects, evaluatingtheir perception of depth and color in stereoscopic images
generated from color-color pairs, color-gray pairs �MIX�,
August 2007/Vol. 46�8�1

ae

rmdi5

2Ctmmsaitvoed

deonotc

ccaptccdrosimea

obtc

sctircs

hp

Termin et al.: Color stereoscopic images requiring only one color image

O

nd gray-gray pairs. The image pairs were presented viaither a stereoscope or a head-mounted display.

This paper is organized as follows. Section 2 discusseselated work and motivation for the approach. The experi-ental procedure designed to test the proposed approach is

escribed in Sec. 3. The experimental results are presentedn Sec. 4. We discuss the implications of the results in Sec.. Section 6 concludes and outlines future work.

Background and Motivationolor and depth are both important in real-world applica-

ions. Batavia and Singh2 describe an obstacle detectionethodology for robotics, which combines two comple-entary vision-based methods: color segmentation and

tereo-based color homography. Siegal and Akiya3 presentmethodology of obstacle detection in autonomous robot-

ncorporated systems, based on color and stereo segmenta-ions. Depth and color are tightly coupled also in computerision applications such as tracking multiple 3-D movingbjects �e.g., cars, humans�.10,11 For instance, Darrellt al.12 combined depth from stereo with skin color and faceetection in order to track moving people.

Drascic and Grodski13 show that the addition of a thirdimension to displays in human remote-controlled teleop-ration improves the decision-making capabilities of theperator, while adding color improves the operator’s recog-ition and obstacle detection performance. An integrationf two detection methodologies, one using color segmenta-ion and the other depth from stereo, enhances the effi-iency of both.

A stereo pair can be produced from nearly any kind ofamera �still or video, digital or analog, color or mono-hrome� and viewed in a variety of displays. In order tocquire the stereo pair to be viewed in a stereoscopic dis-lay system, the system needs to mimic the views seen byhe two eyes, thus requiring that two horizontally displacedameras be used. The principles are the same whether videoameras or still images are being used. The stereoscopicisplay system presents the left and right perspective sepa-ately to each eye. These images can be viewed in a varietyf techniques, from a simple computer monitor with activehutter glasses that sequentially blocks one eye from view-ng the monitor while the other eye’s image is displayed, toore sophisticated head-mounted displays �HMDs�, where

ach eye has its own screen. These recent developmentsllow the addition of 3-D realism to 2-D displays.

However, color stereoscopic imaging comes at the costf increasing the raw data to be transmitted in the channeletween the two color cameras and the displays. An addi-ional cost factor is the need for placing two full-resolutionolor cameras at the sensing end of the device.

Two of the most important parameters in the design of atereoscopic device are the memory size and transmissionhannel capacity necessary for the storage and transfer ofhe stereoscopic images. Thus, much research effort todays dedicated to the optimization and compression of a ste-eoscopic image, and this has led to the proposal of variousompression schemes for stereo images and stereo videotreams.

Various monoscopic image-compression approachesave been developed, and are continually refined and im-

14
roved �e.g., the JPEG family of compression standards �.

However, while these methods can successfully reduce thebandwidth requirements of each channel, they fail to takeadvantage of the large redundancy in stereoscopic imagingdue to the fact that both channels contain information aboutthe same objects, albeit taken from slightly different per-spectives.

Information about the disparity between the channels—the amount of shift one needs to perform on the pixelswithin one image �target� to locate the corresponding pixelsin the other image �reference�—can be used to further re-duce the combined bandwidth requirements of the twochannels. This redundancy of information between the twoimages was already noticed by Yaroslavsky more than20 years ago.15 Today, many stereo compression schemesutilize the correlation between the two stereo images, theinter frame redundancy.7,16 These methods exploit the re-dundancy between the two images by using disparity-compensated prediction. Such predictions depend on esti-mating the magnitude and direction of the disparity.

Several stereo compression methods have been devel-oped based on the concept of disparity estimation. Lukacs17

takes advantage of the binocular redundancy by using apixel-based stereo disparity. However, most of the com-pression techniques use fixed block-size based disparity es-timation schemes.

Woo and Ortegs7,9 use a block-based disparity estima-tion method rather than estimation based on pixel or fea-ture, and claim it to be simpler and more effective toimplement.18 Whereas Woo and Ortega7 estimate disparitybased on a small block, Sethuraman et al.4–6 present meth-ods for stereo compression based on the concept of multi-resolution disparity estimation. A single image and a dis-parity map are used to synthesize a second image for thestereoscopic pair. Other methods for the compression of astereoscopic image apply different compression strategiesto each of the images separately, while using informationabout the disparity between the channels.

In fact, any of the techniques used in motion compensa-tion �for example, MPEG,19,20 which works at a blocklevel� are applicable to disparity estimation.8 The disparity-estimation/disparity-compensation approach resemblesmotion-estimation/motion-compensation methods, whichare popular for video coding. Intraframe redundancy can befound as well between any two sequential image frames orbetween left and right 3-D stereoscopic image pairs and canbe used in order to achieve significant data compression.21

Kim et al.22 suggest a scheme for high-resolution 3-Dstereoscopy via monoscopic HDTV bandwidth, using threecameras. They use a monoscopic color camera to constructthe main stream, and two monochromatic low-resolutioncameras to create a disparity map. The compressed dispar-ity map is transmitted as a low-bandwidth auxiliary stream,and a synthesized color pair is computed on the receiverend using the main and auxiliary streams. However, thismethod requires three synchronized cameras.

While the preceding methods take advantage of the spa-tial similarity between the stereo images, other methods23

use frequency as the basis for compression, without relyingon any complex calculation of disparity compensation.

Other investigators have suggested relying on the versa-tile capabilities of the human visual system to further re-
duce the bandwidth requirements of the stereoscopic color
August 2007/Vol. 46�8�2

ipisaccsppdpaf

tp�vtfcceglaandbfrr

litttcisp

pttccctccPrgigsT


O

mage, before compression. The motivation for such ap-roaches is that there is much research that deals with thenteraction impact of binocular vision and color. For in-tance, it is well known24 that chromatic information helpsddress the correspondence problem in stereo vision. Ac-ording to Shevell and Miller,25 only a small difference inhromatic adaptation is caused by introducing a 3-D repre-entation of the stimuli. This implies that color and deptherception are only loosely coupled and therefore might berocessed separately. There is evidence that there is a re-uced contrast threshold for binocular detection as com-ared with monocular detection.26 This means that there issort of interocular facilitation that enhances binocular per-

ormance in detection tasks.27

Thus, investigators have examined approaches in whichwo different images are presented to the two eyes. Such arocedure may result either in a fused stereoscopic imageas in stereograms�, or in binocular rivalry. Binocular ri-alry is a form of multistable perception that occurs whenhe two eyes are presented with visual stimuli that are dif-erent from each other and cannot be fused into a singleoherent percept. Under these conditions, the percept typi-ally alternates between two states corresponding to the leftye’s stimulus and the right eye’s stimulus, or between twoestalts that are formed by combining parts of the monocu-ar stimuli.28 When large stimuli are used, the alternationsre often piecemeal, while for small stimuli the whole im-ge alternates in unison. It has been shown that it is notecessary to have images that the subject can interpret: ran-om dot stereograms in which high spatial frequencies haveeen removed from one image and low spatial frequenciesrom the other are no longer fusible and produce binocularivalry29 Reviews and a summary of work on binocularivalry can be found in Refs. 30–32

Investigators have sought techniques in which fusion isikely to occur. One set of approaches focuses on maintain-ng chromatic content and reducing the resolution in one ofhe images, relying on the fusion process for reconstructinghe image in the original high resolution. Dinstein et al.33

ake advantage of these capabilities of human visual per-eption. They use a stereo image pair in which one of themages is subsampled and compressed, yet the human ob-erver produces from those two images an image that iserceived stereoscopically as a sharp 3-D image.

The underlying hypothesis for our investigation calls forresenting two images that differ in their chromatic contento the two eyes. Previous work has examined rivalry be-ween competing colors, not between color and lack ofolor. Andrews and Lotto report that when two different-olored panels are presented to a human viewer, the per-eption will often alternate between the colors.34 Alterna-ively, Julesz35 reported that two images of different colorsan create the impression of a third, mixed color. This per-ept is known as binocular color mixture.36 Makous andulos37 reported that during binocular rivalry between aed-and-black grating and a perpendicular green-and-blackrating, the colors mixed, so that the observer reported see-ng alternation between perpendicular yellow and blackratings �not even a hint of a plaid�. In other words, there isome form of integration of the colors seen by the two eyes.
o the best of our knowledge, there is no research on ri-

valry between a color and a monochrome image such aspresented here.

Our hypothesis would extend the preceding results byreducing the amount of information coming from 3-D colorspace by reducing one of the images to a monochromeimage, thus minimizing the amount of data transmitted be-fore any compression method is applied. Moreover, thisapproach requires only one of the cameras to be a colorcamera.

We were encouraged in exploring this direction �giventhe question of whether fusion or rivalry will occur in ourcase� by Andrews and Lotto,34 who have shown that whenphysically different monocular stimuli are likely to repre-sent the same object at the same location in space, fusion islikely to result. Moreover, there is evidence that even in thecase of rivalry, information from the suppressed eye is stillprocessed in the brain, i.e., the eye’s sensitivity is reducedbut not shut off. Abruptly increasing the luminance of ormoving an object seen by the suppressed eye will cause itto be detected.38,39

To summarize, the literature makes several importantpoints, which we build on in our approach: �i� if thedisparate-appearing objects are likely to be the same objectat the same location in space, fusion may occur; �ii� inte-gration of the colors between the two eyes may occur; and�iii� information from the nonobserving eye is still gettingthrough the visual processing areas, possibly enabling a3-D percept. By using one achromatic image �gray� insteadof a competing color, we anticipate reconstructing an imagewith depth. If the image from one eye were fully sup-pressed, there could be no fusion and no depth perception.

3 MethodsControlled psychophysical experiments were conducted inorder to validate the hypothesis that a stereo pair consistingof one monochrome image and one color image produce a3-D color perception. This section presents the experimentsthat were conducted in order to confirm this fundamentalprinciple of the proposed technique.

Various combinations of color and monochromatic im-ages were presented to the subjects, using either a stereo-scope �Fig. 1� or a HMD �Fig. 2�. The first experimentmade use of a 1905 stereoscope and two sets of images,selected to have different color contents. One set, domi-nated by yellow colors, included only three objects, posi-tioned at various distances from the camera, against a neu-tral homogeneous background. The second set of imagescontained color-saturated objects presented against a clut-tered background of colorful content, and were mainlygreen and magenta in color.

The second set of experiments tested the hypothesis witha HMD. The importance of testing with an HMD is thatrecent technological developments allow the addition of3-D realism through sophisticated new display techniques.These technologies introduce elements such as refresh ratesthat might cause the visual system to perform differentlythan under conditions of a stereoscope. One popular optionis a sophisticated HMD where each eye has its own minis-creen. In light of that, the second experiment, whose intentwas to establish the phenomenon under more application-like conditions, made use of an nVisor-SX HMD controlled
by a PC. This setting enabled a controlled procedure of
August 2007/Vol. 46�8�3

art

3TFeosmaWm

mcececw

Fe


O

utomatic display of images in random order, with multipleepetitions. The images displayed in this experiment con-ained objects of saturated colors �red, yellow, green, blue�.

.1 Experiment 1he 1905 stereoscope used in the experiments �shown inig. 1� is designed to display two photographs to the view-r’s eyes, using a partition that enables each eye to viewnly one of the two pictures. The two pictures, taken fromlightly different viewpoints, are positioned on a stiffounting, side by side, and viewed through a set of prisms

nd lenses. Since the invention of the stereoscope byheatstone in 1838, it has been utilized in many experi-ents involving fusion, color mixing, and rivalry.40

These experiments were performed on 11 subjects �9ales and 2 females� between the ages of 16 and 45. Ex-

ept for two subjects, all were naive as to the purpose of thexperiments. All subjects were clinically tested for normalolor and depth perception and equal visual acuity in bothyes, as determined by optometric examinations. Stereoa-uity was tested via the Titmus stereo fly test. Color visionas tested using Ishiara plates.

Fig. 1 The 1905 stereos

ig. 2 The nVisor-SX head-mounted display used in the
xperiments.

Two sets of stereo image pairs were used. Each set con-tained different objects at different distances and colors,with the two static images taken from slightly differentviewpoints in order to generate depth perception. One set ofstereo images included objects that were mostly yellow,with low average saturation �Fig. 3�. The other pair con-sisted of objects in saturated colors, such as green and ma-genta. Some of the images were taken by us �Fig. 3�, andsome �Fig. 4� were modified from Refs. 41 and 42.

The two static images presented to the observers were attimes different in their color content, to evaluate our hy-pothesis. The original color stereo images were modified tofit the experiment �e.g., cropped and partially or fully con-verted to grayscale�. In each set, four different image pairswere used, creating four color combinations each:

�a�Left eye: color image; right eye: color image �color-color�

�b�Left eye: grayscale image; right eye: grayscale image�gray-gray�

�c�Left eye: color image; right eye: grayscale image�MIX�

�d�Left eye: grayscale image; right eye: color image�MIX�.

Observers were asked to adjust the distance between thedisplayed stereo pairs and their eyes until the pair of imagesfused into one 3-D image, if possible. The image pairs werepresented in random order; each set was presented twice.For each image pair, subjects were asked to rate their colorperception on a scale of 1 to 10, with 1 denoting a gray-scale image and 10 an image in vivid colors. They werealso asked to rate the depth perception from 1 to 10, with 1corresponding to a flat image with no depth at all, and 10 toa full 3-D image. Subjects were unaware of the color con-tent of the images they were viewing.

3.2 Experiment 2In the second experiment the stereoscope was replaced by aHMD. This nVisor-SX display is usually used for advanced

sed in the experiments.
cope u
virtual reality applications. It incorporates high-resolution

August 2007/Vol. 46�8�4


O

Fig. 3 Four combinations of low-saturation stereo pairs, used in the experiments: �a� color-color; �b�
gray-gray; �c� MIX �color on left�; �d� MIX �color on right�.
Fig. 4 Four combinations of high-saturation stereo pairs used in the experiments: �a� color-color; �b�gray-gray; �c� MIX �color on left�; �d� MIX �color on right�.

Fig. 5 First HMD set: four combinations of stereo pairs used in the HMD experiments: �a� color-color;
�b� gray-gray; �c� MIX �color on left�; �d� MIX �color on right�.
ptical Engineering August 2007/Vol. 46�8�087003-5

csntws1

mspmvp

c


O

olor microdisplays with custom-engineered optics de-igned to deliver high visual acuity in a wide FOV. TheVisor-SX has access to user adjustments, including in-erocular distance, and an eye relief to accommodate usersith eyeglasses. The HMD specifications are: dual channel

upport �stereo�, monocular FOV �diagonal� 60° overlap00%.

These experiments were performed on 15 subjects �10ales and 5 females� between the ages of 21 and 43. All

ubjects were clinically tested for normal color and deptherception and equal visual acuity in both eyes, as deter-ined by optometric examinations �using standard nonin-

asive optical tests�. All subjects were naive as to the pur-ose of the experiments.

Three sets of stereo image pairs were used. Each setontained different objects at different distances and of dif-

Fig. 6 Second HMD set: Four combinations ofcolor; �b� gray-gray; �c� MIX �color on left�; �d� M

Fig. 7 Third HMD set: Four combinations of s
color; �b� gray-gray; �c� MIX �color on left�; �d� MIX �co

ferent colors. Within each pair the two images were takenfrom slightly different viewpoints in order to generatedepth perception. One set of stereo images �Fig. 5� includedfour colored objects �red, green, yellow, and blue� at differ-ent distances positioned in front of a homogeneous mono-chromatic background �uncluttered�. In the second set ofstereo images �Fig. 6�, the same objects were used, withmore objects added in the back, but keeping the back-ground homogeneous and monochromatic. The third set ofthe images �Fig. 7�, contained two greenish objects at twodifferent distances from the camera. Overall, the three im-age sets covered a range of color contents, in terms of huesand saturation. The variety was intended to cover a range ofcolors, distances, interpositions, and saturations. Each im-age set comprised four chromatic combinations of the basic

pairs used in the HMD experiments: �a� color-lor on right�.

airs used in the HMD experiments: �a� color-

stereoIX �co

tereo p
lor on right�.
August 2007/Vol. 46�8�6

sl5

4Ttiimbte

4

4DajeaFjv

rb

M

S


O

tereo pair: �a� color-color, �b� gray-gray, �c� MIX �color oneft�, and �d� MIX �color on right�, as presented in Figs.–7.

Resultshe results show that as a rule, subjects successfully fused

he images in all sets. Irrespective of the color differencesn a given image pair, the subjects perceived a strong 3-Dmage in depth. Color perception was successful in allixed cases, though some degradation of color quality can

e seen in the results. We provide below detailed results ofhe depth and color perception grades for each of the twoxperiments.

.1 Experiment 1

.1.1 Depth perceptionepth was perceived in all four combinations of stereo im-

ges and in both image sets that were presented to the sub-ects. Color rivalry, if it occurred, seemed to have no influ-nce on depth perception. The results across the 11 subjectsre summarized in Table 1, and presented graphically inig. 8. Figure 8�a� shows the results averaged over all sub-

ects. Figure 8�b� shows the average for each subject indi-idually.

In Table 1 the row marked “Mean” includes the meanesults �across 11 subjects, multiple viewings per subject,oth sets of images�. The row marked “Std. dev.” includes

Table 1 Depth perception results �stereoscope�.

Subjective depth perception

Color pair MIX pair Monochromatic pair

ean 9.62 9.35 9.20

td. dev. 0.92 1.38 0.96

Fig. 8 �a� Depth perception of three types of ste
tion �stereoscope�.

the standard deviation for these mean values. Columns re-fer to the content of the presented stereo pairs. The columnlabeled “MIX pair” includes both the color-gray and gray-color presentations.

Figure 8�a� shows the same results graphically. The left-most bar indicates the mean depth perception for the color-color images; the middle bar, for the mixed images; and therightmost bar, for the gray-gray images. The offset indica-tors on each bar show the extent of the standard deviation.

Figure 8�b� shows the individual average results for eachsubject. Each of the 11 triplets of bars corresponds to onesubject. The order of the bars within each triplet is the sameas in Fig. 8�a�. The Y axis denotes the score on the scale of1 to 10 described earlier. The X axis separates the differentsubjects. Here, for each subject, the result is their averageacross both image sets �Figs. 3 and 4�.

In summary, depth perception seems unaffected by theuse of mixed pairs. Average depth scores for all three pre-sentations �color-color, MIX, and gray-gray� show insig-nificant differences �the maximum difference obtained be-tween color pair and monochromatic pair has a paired, two-tailed t-test value p=0.21�.

4.1.2 Color perceptionAll subjects reported that they perceived a color imagewhen they were presented with one color image and onemonochromatic image �MIX pair�, as well as when viewing

Table 2 Color perception results �stereoscope�.

Subjective color perception


Mean 9.56 7.37 1

Std. dev. 0.92 1.55 0

irs �stereoscope�. �b� Subjective depth percep-
reo pa
August 2007/Vol. 46�8�7

tadifjawv

TF

taIipfssraaf

o

M

S


O

wo color images. The mean and standard deviation resultsre shown in Table 2. The average color score is 9.56 �std.ev. 0.92� in the color-color pairs, and 7.37 �std. dev. 1.55�n the mixed pairs. Indeed, some subjects reported a colorading effect when viewing some images. Also, some sub-ects reported that the color of some of the objects withinn image faded in and out during viewing. These reportsere not consistent for the same subject across differentiewings, or for the same image pair across subjects.

The results presented in Table 2 are plotted in Fig. 9�a�.he average results for each of the 11 subjects are shown inig. 9�b�. The axes are the same as in Fig. 8.

The results show that subjects clearly perceived color inhe mixed presentations. There was, however, a certainmount of degradation in the quality of the perceived color.n order to ascertain whether the degraded color perceptionn the mixed condition more closely resembled the colorair or the monochrome pair, average color perception dif-erences were calculated in pairs �i.e., for each subjecteparately� and for the group as a whole. These are pre-ented in Table 3. The table shows that the difference inesults between the color-color images and the mixed im-ges is smaller than the difference between the mixed im-ges and the monochromatic �gray-gray� images. This dif-erence is statistically significant �paired one-tailed t-test,

p=0.0005�. Given that the measures are most likely on anrdinal scale rather than an interval one, from a purist ob-

Table 3 Color difference distance results �stereoscope�.

Difference

Color−MIX MIX−monochromatic

ean 2.2 6.37

td. dev. 1.33 1.55

Fig. 9 �a� Color perception of three types of ster
�stereoscope�.

jective perspective we cannot completely reject the null hy-pothesis that the results from the mixed presentations arenot closer to the color presentations than to the mono-chrome presentation. However, from a subjective impres-sion we think that the comparison indeed holds and that thescale may be interval.

4.2 Experiment 2

4.2.1 Depth perceptionDepth was perceived in all four combinations of stereo im-ages and in all three image sets that were presented to thesubjects. The results here were even more impressive thanin the stereoscope experiments, as can be seen in Table 4and graphically in Fig. 10. All subjects in all viewings ratedthe depth as a 10. Figure 10�a� shows the average resultsfor all subjects. Figure 10�b�, shows the average for eachindividual subject. Table 4 is arranged the same as Table 1and Fig. 10 parallels Fig. 8. In summary, when using anHMD, depth perception is unaffected by the use of mixedpairs of images.

4.2.2 Color perceptionWith the HMD, all subjects reported that they perceived acolor image when they were presented with one color im-age and one monochromatic image, as well as when view-ing two color images. The mean and standard deviation are

Table 4 Depth perception results �HMD�.

Subjective depth perception


Mean 10.00 10.00 10.00

Std. dev. 0.0 0.0 0.0

s �stereoscope�. �b� Subjective color perception
eo pair
August 2007/Vol. 46�8�8

s0tftittcT1a9

aiicg6ctcc

M

S


O

hown in Table 5. The average color score is 9.95 �std. dev..18� in the color-color pairs, and 7.20 �std. dev. 0.92� inhe mixed pairs. Here too, some subjects reported a colorading effect when viewing some mixed pairs or reportedhat the color of some of the objects within an image fadedn and out during viewing. These reports were not consis-ent for the same subject across different viewings, or forhe same image pair across subjects. The averages of theolor perception results for all subjects are presented inable 5 and Fig. 11�a�. The average results for each of the5 subjects are shown in Fig. 11�b�. The axes are the sames in the respective depth perception graphs �Fig. 9�a� and�b��.

As with the stereoscope, the results clearly show that inmixed presentation the subjects perceived a 3-D color

mage, but with color that was of a slightly degraded qual-ty. Here too, average color perception differences were cal-ulated in pairs �i.e., for each subject separately� and for theroup as a whole. The group results are presented in Table. The table shows that the difference in results between theolor-color images and the mixed images is smaller thanhe difference between the mixed images and the mono-hromatic �gray-gray� images. This difference is statisti-ally highly significant �paired one-tailed t-test, p=1e−17�.

Table 5 Color perception results �HMD�.

Subjective color perception


ean 9.95 7.20 1.01

td. dev. 0.18 0.92 0.05

Fig. 10 �a� Depth perception of three types of
�HMD�.

5 DiscussionThe approach presented and tested in this paper is innova-tive, and may prove useful for imaging systems combiningcolor and depth. It allows for the use of two cameras, onlyone of which is color. This technique utilizes the humanvisual system’s innate abilities and the manner in which theimage is perceived in the brain to reduce the redundantcolor information. The experimental results confirm our hy-pothesis that one monochrome and one color image aresufficient for the visual system to generate full color anddepth perception.

By using one monochromatic image instead of a com-peting color, our subjects perceived an image with depth,and one in which the “colors” blend to produce a color 3-Dimage. No degradation in depth perception was measured.However, some degradation in the perception of color isevident, though the subjects still rank their color perceptionof mixed images significantly closer to the full-color im-ages �color-color� than to the monochromatic �gray-gray�images.

The proposed approach for reducing bandwidth and us-ing only one color camera appears to be a viable comple-mentary approach to current techniques used in transmit-ting color 3-D images.

Table 6 Color difference distance results �HMD�.

Difference

Color−MIX MIX−monochromatic

Mean 2.7 6.18

Std. dev. 0.83 0.9

pairs �HMD�. �b� Subjective depth perception
stereo
August 2007/Vol. 46�8�9

atisbpsbilIss

6WgmtTvmitr

AT0cykSHmw


O

There are several potential shortcomings to using thispproach. The most obvious, based on the presented data, ishat although the hypothesis that a mixed pair will be seenn color has been validated, it is clear that the color in thatcene is less vivid than in a color pair. Furthermore, as cane seen from Fig. 9�a�, there is a large variation in colorerception between subjects. Thus, if a system were de-igned based on this approach and were intended for aroad audience, it might be necessary to first ascertain thatndeed the intended user is among that section of the popu-ation that perceives the color more vividly rather than less.nvestigation of the color degradation, variation betweenubjects, and means for improving the color perceptioncore will be explored in future work.

Conclusionse continue to further explore this phenomenon with the

oal of defining the minimal requirements for which aixed stereoscopic image contains enough information for

he human visual system to perceive both color and depth.he results from the experiments described in this paperalidate the basic principle. Future experiments will beade to further quantify the efficacy of this approach and

ts robustness. In addition, attempts will be made to use thisechnique in actual applications, such as teleoperation ofobots.

cknowledgmentshis research was supported in part by ISF grant No. 1211/4. We thank Hezzi Yeshurun for helpful discussions andomments, and Leonid P. Yaroslavsky and Natan Netan-ahu for advice. We thank Yoav Elkoby, Michael Bend-owski, and Avi Termin for assisting in the experiments;hlomo Schrader for optometric assistance; and Fany andaim Ram for technical assistance. We thank the anony-ous reviewers for helpful suggestions on improving this

Fig. 11 �a� Color perception of three types of�HMD�.

ork. As always, thanks to K. Ushi.

ptical Engineering 087003-1

References

1. R. Skuler and R. Blake, Perception, 3rd ed., McGraw-Hill �1994�.2. P. H. Batavia and S. Singh, “Obstacle detection using adaptive color

segmentation and color stereo homography,” in Proc. IEEE Int. Conf.on Robotics and Automation �2001�.

3. Y. T. M. W. Siegal and T. Akiya, “Kinder gentler stereo,” in Stereo-scopic Displays and Virtual Reality Systems VI, Proc. SPIE 3639A,18–27 �Jan. 1999�.

4. A. J. S. Sethuraman and M. Siegel, “Multiresolution based hierarchi-cal disparity estimation for stereo image pair compression,” in Proc.Symp. on Application of Subbands and Wavelets, A. Akansu, Ed.,IEEE �1994�.

5. M. S. S. Sethuraman and A. Jordan, “A multiresolutional regionbased segmentation scheme for stereoscopic image compression,” inDigital Video Compression: Algorithms and Technologies, Proc.SPIE 2419, 265–274 �1995�.

6. M. W. S. S. Sethuraman and A. Jordan, “A multiresolution frame-work for stereoscopic image sequence compression,” in Proc. ICIP-94, Vol. II, pp. 361–365, IEEE �1994�.

7. W. Woo and A. Ortega, “Stereo image compression based on dispar-ity field segmentation,” in VCIP ’97, Proc. SPIE 3024, 391–402�Feb. 1997�.

8. W. Woo and A. Ortega, “Dependent quantization for stereo imagecoding,” in PW-EI-VCIP’98, Proc. SPIE 3309, 902–913 �1998�.

9. W. Woo and A. Ortega, “Overlapped block disparity compensationwith adaptive windows for stereo image coding,” IEEE Trans. Cir-cuits Syst. Video Technol. 10�2�, 194–200 �2000�.

10. J. Rehg, M. Loughlin, and K. Waters, Vision for a smart kiosk, inProc. IEEE Conf. on Computer Vision and Pattern Recognition, pp.690–696 �1997�.

11. C. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, “Pfinder: real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach.Intell. 19�7�, 780–785 �1997�.

12. T. Darrell, G. Gordon, M. Harville, and J. Woodfill, “Integrated per-son tracking using stereo, color, and pattern detection,” in Proc. Conf.on Computer Vision and Pattern Recognition, pp. 601–609, IEEEComputer Soc. �1998�.

13. D. Drascic and J. J. Grodski, “Defence teleoperation and stereoscopicvideo,” in Stereoscopic Displays and Applications IV, Proc. SPIE1915, 58–69 �Feb. 1993�.

14. JPEG Committee, http://www.jpeg.org/committee.html.15. L. P. Yaroslavsky, “On redundancy of stereoscopic pictures,” in Im-

age Science ’85, Proc., Acta Polytechnica Scandinavica, Vol. 1, pp.82–85.

16. M. G. Perkins, “Data compression of stereo pairs,” IEEE Trans.Commun. 40�4�, 684–696 �1992�.

17. M. E. Lukacs, “Predictive coding of multi-viewpoint image sets,” inProc. Int. Conf. on Acoustics Speech, and Signal Processing, pp.521–524, IEEE �1986�.

18. W. Woo and A. Ortega, “Optimal blockwise dependent quantizationfor stereo image coding,” IEEE Trans. Circuits Syst. Video Technol.

pairs �HMD�. �b� Subjective color perception
stereo
9�6�, 861–867 �1999�.

August 2007/Vol. 46�8�0

1

2

2

2

2

2

2

2

2

2

2

3

3

3

3

3

3

3

3

3


O

9. L. Chiariglione, “Short MPEG-1 description,” http://www.chiariglione.org/mpeg/standards/mpeg-1/mpeg-1.htm �1996�.

0. R. Koenen, “Overview of the MPEG-4 standard,” http://www.chiariglione.org/mpeg/standards/mpeg-4/mpeg-4.htm �2002�.

1. M. Siegel, P. Gunatilake, S. Sethuraman, and A. Jordan, “Compres-sion of stereo image pairs and streams,” in Stereoscopic Displays andVirtual Reality Systems, Proc. SPIE 2177, 258–268 �Feb. 1994�.

2. K. T. Kim, M. Siegal, and J. Son, “Synthesis of a high-resolution3d-stereoscopic image pair from a high-resolution monoscopic imageand low-resolution depth map,” in Stereoscopic Displays and VirtualReality Systems, Proc. SPIE 3295, 76–86 �1998�.

3. W. D. Reynolds and R. V. Kenyon, “The wavelet transform and thesuppression theory of binocular vision for stereo image compres-sion,” in IEEE Int. Conf. on Image Processing, Vol. 2, pp. 557–560�1996�.

4. D. R. Simmons and F. A. Kingdom, “Interactions between chromatic-and luminance-contrast-sensitive stereopsis mechanisms,” Vision Res.42�12�, 1535–1545 �2002�.

5. S. K. Shevell and P. R. Miller, “Color perception with test and adapt-ing lights perceived in different depth planes,” Vision Res. 36�7�,949–954 �1996�.

6. D. R. Simmons and F. A. Kingdom, “On the binocular summation ofchromatic contrast,” Vision Res. 38�8�, 1063–1071 �1998�.

7. A. Anzai, J. M. A. Bearse, R. D. Freeman, and D. Cai, “Contrastcoding by cells in the cat’s striate cortex: monocular vs. binoculardetection,” Visual Neurosci. 12�1�, 77–93 �1995�.

8. D. Alais, R. P. O’Shea, C. Mesana-Alais, and I. G. Wilson, “Onbinocular alternation—a translation of Diaz-Caneja �1928�,” Dept. ofPsychology, Univ. of Otago, Dunedin, New Zealand �2000�. Re-trieved December 6, 2006 from http://psy.otago.ac.nz/r_oshea/BR_bibliography/br_DJtrans.html.

9. J. E. Mayhew and J. P. Frisby, “Rivalrous texture stereograms,” Na-ture (London) 264�5581�, 53–56 �1976�.

0. R. Blake, “Primer on binocular rivalry, including controversial is-sues,” Brain Mind 2, 5–38 �2001�.

1. R. Fox, Binocular Vision and Psychophysics, pp. 93–110, MacmillanPress, London �1991�.

2. I. P. Howard and B. J. Rogers, Binocular Vision and Stereopsis, Ox-ford Univ. Press, New York �1995�.

3. I. Dinstein, M. G. Kim, A. Henik, and J. Tzelgov, “Compression ofstereo images using subsampling and transform coding,” J. Soc.Photo-Opt. Instrum. Eng. 30�9�, 1359–1364 �1991�.

4. T. J. Andrews and R. B. Lotto, “Fusion and rivalry are dependent onthe perceptual meaning of visual stimuli,” Ecologist 14�5�, 418–423�2004�.

5. B. Julesz, Foundation of Cyclopean Perception, Univ. of ChicagoPress, Chicago �1971�.

6. C. Erkelens and R. van Ee, “Multi-coloured stereograms unveil twobinocular colour mechanisms in human vision,” Vision Res. 42�9�,1103–1112 �2002�.

7. W. Makous and E. Pulos, “Grating colors mix while their contoursrival,” Invest. Ophthalmol. Visual Sci. 20�Suppl.�, 225 �1981�.

8. R. Fox and R. Check, “Detection of motion during binocular suppres-

ptical Engineering 087003-1

sion,” J. Exp. Psychol. 78�3�, 388–395 �1968�.39. H. Wiesenfelder and R. Blake, “Apparent motion can survive binocu-

lar rivalry suppression,” Vision Res. 31, 1589–1599 �1991�.40. W. L. Gulick and R. B. Lawson, Human Stereopsis. A Psychophysical

Analysis, Oxford Univ. Press, New York �1976�.41. D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense

two-frame stereo correspondence algorithms,” Int. J. Comput. Vis.1–3, 7–42 �Apr.-June 2002�.

42. D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps us-ing structured light,” in Proc. IEEE Computer Soc. Conf. on Com-puter Vision and Pattern Recognition �2003�.

Yael Termin received her BSc in physics and MSc in computerscience from Bar-Ilan University and has recently submitted herPhD dissertation on perception of a 3-D colored stereo image fromone colored and one grayscale images. During her PhD studies shehas received the dean’s award and a doctoral fellowship of excel-lence. She is beginning her postdoc in the the Neural Imaging Labo-ratory at the Gonda Multidisciplinary Brain Research Center at BarIlan University, Israel. This paper is based on her PhD thesis.

Gal A. Kaminka is a senior lecturer in the Computer Science De-partment at Bar Ilan University, Israel. His research expertise in-cludes teamwork and coordination, robotics, behavior and plan rec-ognition, multiagent systems, and modeling social behavior. Hereceived his PhD from the University of Southern California, andspent two years as a post doctoral fellow at Carnegie Mellon Uni-versity. Today, Dr. Kaminka leads the MAVERICK group at Bar Ilan,supervising close to 20 MSc and PhD students—the largest com-puter science group in Israel. He has been awarded an IBM facultyaward and top places at international robotics competitions.

Sarit Semo received her BA in physics and BSc in electrical engi-neering with honors from the Technion Institute of Technology inHaifa, Israel. She is currently completing her MSc in biomedical en-gineering at Tel-Aviv University on color adaptation and color con-stancy in vision research.

Ari Z. Zivotofsky is a senior lecturer in the neuroscience programat Bar Ilan University, Israel. He did his undergraduate degree inelectrical engineering at The Cooper Union and received his PhDfrom Case Western Reserve University in biomedical engineering.Following that, he spent four years as a postdoc in the Laboratory ofSensorimotor Research at the NIH. His research interests are in theareas of ocular motility, visual perception, and cognitive functioning.

August 2007/Vol. 46�8�1

color stereoscopic images requiring only one color image

Documents