review: 3d holographic imaging and display exploiting

ITE Trans. on MTA Vol. 5, No. 3, pp. 78-87 (2017)

78

1. Introduction

Holography is a promising method to realize a three-

dimensional display and imaging 1) 2). Holograms directly

encode the light field by addressing both the amplitude

and phase information 3)-5). Recording and projecting

three-dimensional (3D) images using holography

technology provides natural and realistic 3D images,

unlike stereographic techniques which display two

different images simultaneously on the left and right

eyes exploiting binocular parallax of human eyes 6) 7).

In holography, it is crucial to precisely measure and

reconstruct the phase information of light fields. The

direct measurement of optical phase information is

currently impossible because the optical temporal

frequency is several orders faster than the electronics-

based device. Alternatively, conventional holography

methods exploit interference to record both the

amplitude and phase information of an optical field. A

signal beam of interest is combined with a well-defined

reference beam, resulting in the formation of

interference patterns which encode the light field

information. After the first demonstration by Gabor 4) 8),

various holographic techniques have been proposed to

record and reconstruct static optical fields 9)-11).

Recently, the developments of electronic devices such as

a charge-coupled device (CCD) or a spatial light modulator

(SLM) have enabled digital holography techniques 12)-14).

Since digital holography takes advantages in recording,

reading, and transferring of dynamic optical field

information, it has been extended into various fields such

as biomedical optics 15) 16), holographic microscopy 17)-19), 3D

display 20), data storage 21) and security 22).

Although digital holography for recording and

displaying of 3D optical information has potentially

interesting applications, its implementation has been

limited to mostly laboratory-level demonstrations

because of the requirement for a bulky interferometric

setup and a limited space-bandwidth product (SBP) of

SLMs in holographic displays.

In 3D holographic imaging technique, portability

would be crucial for practical applications. However, the

need for a reference beam in conventional interferometry

significantly limits the realization of a portable device.

In 3D holographic display, 3D optical fields are

generated by a spatial light modulator. The product of

the image size and the viewing angle is directly

proportional to the number of controllable optical modes,

or an SBP of an SLM 23). Since the current state-of-art

Abstract Digital holography has high potentials for future 3D imaging and display technology. Due to the

capability of recording and projecting realistic 3D images, holography has been extensively studied for decades.

However, the requirement of a reference beam in interferometric systems and a limited number of pixels in

existing spatial light modulators have been major obstacles for the practical applications of 3D holography

technology. Recently, the field of wavefront shaping, or the study of controlling multiple scattering of light, has

emerged with numerous interesting applications in digital holography. In this review, we introduce the

principles of multiple light scattering in complex media and highlight recent achievements to overcome the

limitation in conventional 3D holography by exploiting multiple light scattering. The complexity of multiple light

scattering, which had been regarded as a major barrier for conventional optical systems, can provide reference-

free 3D holographic imaging and 3D holographic display with several advantages.

Keywords: 3D holographic imaging, 3D holographic display, complex optics, multiple light scattering, wavefront shaping.

Received March 18, 2017; Revised May 20, 2017; Accepted June 7, 2017†Department of Physics, KAIST(Republic of Korea)

Review: 3D Holographic Imaging and Display ExploitingComplex Optics

Hyeonseung Yu†, YoonSeok Baek†, Jongchan Park†, SeungYoon Han†,

KyeoReh Lee† and YongKeun Park†

Copyright © 2017 by ITE Transactions on Media Technology and Applications (MTA)

technology is incapable of addressing a large number of

optical modes, the performances of 3D holographic

display techniques are still limited to a small size with a

narrow viewing angle range 24).

In order to overcome the limitations of such holographic

imaging and display technology, various methods have

been proposed. For example, diffractive optical elements

can enlarge the image size of a 3D holographic screen by

encoding a large-curvature lens 25). Holographic optical

elements 26) are also used for delivering images in head-

mount displays. Freeform optics 27) or deformable

reflecting surfaces 28) have been used to enhance

controllability of optical information as in the case of a

multifocal projection system. However, still much

advancement is required to project viewer-comfortable

3D images.

In this review, we introduce digital holographic

imaging and display methods exploiting multiple light

scattering. When coherent light transmits through

scattering media, speckle patterns are formed as a result

of interference of multiply scattered light paths. It seems

the multiple light scattering events are undesirable for

imaging and display applications because it results in

the loss of optical information. However, in multiply

scattered light, the optical information is scrambled

rather than lost 29) 30), and the events of multiple light

scattering are deterministic processes although it seems

stochastic. With appropriate characterizations of the

optical transfer property of a disordered medium, it can

be utilized as a powerful optical element, even for

holographic imaging and display.

2. Principles

Conventional optical components, such as a lens or a

mirror, offer great possibilities of controlling the light.

Incident and transmitted rays through a conventional

optical lens can be described by a simple linear matrix

[Fig. 1(a)], which can be readily adapted to an optical

imaging system. Despite their simplicity, there are

certain limitations of conventional optics. For example,

the numerical aperture of an imaging and a display

system is highly limited unless a high-magnification

objective lens is used.

Light scattering in scattering media exhibits

extremely complex behaviors, compared to the

conventional optics. However, the linearity and the

deterministic nature of propagation of optical fields are

still preserved in multiple light scattering [Fig. 1(b)].

Therefore, the light transmission process can also be

expressed in a linear matrix formalism, so called a

transmission matrix (TM) 30). The TM is an operator that

maps an incident (input) field to a transmitted (output)

field. Assuming the linearity of a system and the coherency

of a light source, this relationship is expressed as,

where Ein and Eout are an input and an output field,

respectively. When a TM is measured, the light can be

controlled as desired by exploiting the large degrees of

the freedom contained in scattering media.

Theoretically, this linear and deterministic nature of

multiple light scattering has been known for decades,

the experimental control of multiple scattering was

demonstrated very recently. Vellkoop et al. showed that

an optical focus could be made through a scattering

layer consist of TiO2 nanoparticles 31). Although this

study did not explicitly use TMs, it exploited the

deterministic and linear response properties of multiple

light scattering of coherent light. This pioneering work

has initiated various interesting experiments.

Complex optics enables access to and manipulate

optical information unless cannot be addressed with

conventional optics. For instance, the evanescent near-

field information of light can be coupled to propagating

E TEout in= ( )1

79

Paper, Invited Paper » Review: 3D Holographic Imaging and Display Exploiting Complex Optics

Fig.1 (a) (above) The photograph of an imaging system with a

conventional lens. The input USAF target image is clearly

projected through the lens. (lower) The image transfer is

well described by a simple linear relationship according to

the lens equation. (b) (upper) the photograph of an output

speckle pattern when the image transmits through

scattering media. (lower) the input image is highly distorted

due to complex scattering. The image transfer process is

still described by a deterministic linear relationship.

far-fields though scattering layer. This near-to-far field

conversion is described by a TM, which had been used for

focusing and imaging at the subwavelength scales 32) 33). A

scattering layer, whose TM was calibrated, was used to

extend the field of view with extended resolutions in

microscopic imaging 34).

2.1 The measurement of transmission matrices

TMs can solve technical challenges in holographic

imaging and display. First, a TM of a turbid layer can be

used so that the turbid layer can be exploited as a lens.

When an incident beam transmits through a scattering

layer, the output field exhibits complex speckle patterns

due to multiple scattering. Popoff et al. have

demonstrated that an incident field can be retrieved

from a field passing through a turbid layer, by using the

inverse of a TM of the layer 30). Mathematically, when an

output field Eout is measured, the input beam is

retrieved using the TM of a turbid layer as,

Alternatively, one can tailor output fields by actively

controlling the incident fields if the TM information of

an optical system is known 35). Given the desired output

field Eout, projection, the required input field is calculated by

Experimentally, the input field Eout, required can be

produced by using an SLM, such as a LCoS (Liquid

Crystal on Silicon) SLM, a digital micro-mirror device

(DMD), or a deformable mirror (DM).

In order to measure a TM of a turbid layer, essentially

two processes are required: modulating of incident fields

and recording of corresponding output fields. In the TM

formalism, the m-th column of the TM corresponds to

the output field response of the m-th basis of an incident

field [Fig. 2(a)], which is given as

Thus, by measuring the output fields corresponding to

each input basis field, the whole TM elements can be

experimentally obtained.

The basis of input fields can be arbitrarily chosen in

the spatial domain, the spatial frequency domain, or

independent patterns (such as Hadamard bases). In the

spatial domain, the input bases correspond to point

sources at different spatial locations 32). In the spatial

frequency domain, the input bases correspond to plane

waves with different propagation angles 36). Hadamard

basis is also widely used due to the simplicity of

encoding the basis fields using a digital modulator such

as a LCoS SLM or a DMD 37) 38).

For recording of a TM, output fields responding to

various input bases, complex amplitudes of output fields

must be measured. It should be noted that a TM

describes a field-field relation between the input and

output light. Thus, to fully calibrate a TM, typically

interferometric imaging systems, in which a well-defined

reference beam is used, have been utilized 36). However,

one can also measure a partial TM without using an

external reference beam 30) 39). The TMs measured in

these works only have a relative phase information

between different optical modes. Thus they are lack of

the information about a global phase value. The global

phase information is critical in imaging through a

scattering layer. However, for the projection of tailored

output images, the global phase information is not

necessary. This is because the transmitted light is fully

described by the relative phases between speckles.

2.2 Wavefront shaping techniques

In certain applications, one does not need to know

about the full TM information about a scattering layer.

For example, if focusing at several positions is sufficient

for certain applications, a partial information about a

TM is enough. Optical techniques for coherent control of

light transport in turbid media, without the need for full

characterizations of TMs, is also known as wavefront

shaping.

To generate an optical focus at the m-th position in

the output fields through a turbid media, the required

incident fields is given as,

Then the m-th element of output fields is calculated by

E e e eint t tm m mn=

′− − −arg( ) arg( ) arg( ) .1 2 … (( )5

E

E TE T

in m m m nm

out in m

,

,

.= ′

= =

δ δ δ1 2

0 0 1 0 0

…

… … ′

= ′t t tm m nm1 2

4

… .

( )

E T Ein required out projection, ,= −1 3. ( )

E T Ein imaging out, = −1 2( )

ITE Trans. on MTA Vol. 5, No. 3 (2017)

80

Fig.2 (a) Transmission matrix contains full optical information

about the field-field relationship between input and output

fields. (b) The wavefront shaping technique only utilize a

single row in the transmission matrix, corresponding to the

single output channel.

Therefore, the process for focusing a spot through a

turbid layer in wavefront shaping is identical to

characterize a specific row of the TM of the turbid layer

Fig. 2(b). This process could be performed through

various optimization procedures 31), instead of measuring

a whole TM.

An important parameter in wavefront shaping

techniques is the enhancement factor, defined as the

peak to background intensity ratio, after the

optimization. The enhancement factor EF is a function

of the number of controllable optical modes N in an

SLM. The equations for estimating EF in the field

modulation 40), the phase-only modulation 31), and the

binary amplitude modulation 41) are given as follows,

2.3 Practical consideration of transmission

matrices

Scattering layer exhibits different responses

depending on the wavelength and the polarizations of

light. This enables the use of complex optical elements

as a dynamic wave plate 42), a spectral filter 43), and the

control of spatiotemporal profile of light 44)-46). A TM has

different elements depending on the frequency and

polarization of the light. For reconstructing color images

through a turbid layer, a TM of the layer should be

measured for each wavelength. For color display

purposes, the input pattern corresponding to the desired

output pattern should be found separately for each color

channel.

The number of optical modes contained in scattering

media is also an important issue. If scattering media is

interpreted as a 2-D optical waveguide, the number of

optical modes is proportional to the incident area 47). For

this reason, acquiring a TM for large size scattering

materials requires long acquisition time and heavy

computation. Thus, it is important to appropriately

design a measurement system and measure a TM of the

system, considering the tradeoffs between the

information capability and the optimization load.

3. 3D camera exploiting complex optics

3.1 Conventional holography and its

shortcomings

In order to acquire 3D image information, the phase,

as well as the amplitude information of an image, should

be determined. Previously, several holographic imaging

techniques based on interferometry have been utilized in

order to record the wavefront of light 16) 48) 49). It

measures and analyzes intensity patterns generated by

the interference between a sample and a reference

beam.

The realization of an interferometry-based imaging

system is complicated, compared to a photographic

camera. Because coherence light must be divided into a

sample and a reference beam to construct an

interferometer, an optical instrument becomes large and

complicated, and precise and stable experimental

conditions are strongly required 50).

To overcome these limitations, simplified imaging

techniques which can access 3D information have been

proposed such as Shack-Hartmann wavefront sensors 51) 52),

imaging systems based on the transport-of-intensity

equation 53), ptychographic scanning 54) 55), and iterative

algorithms to extract phase information from measured

intensity patterns 56)-58). However, in order to achieve an

appropriate guess of the incident wavefront without

using a reference beam, these techniques require a

priori knowledge (or assumptions) on an object or

incident field has to be provided as a trade-off.

3.2 An optical diffuser as a holographic lens

Recently, Lee et al. proposed a 3D camera technique

using an optical diffuser as an effective solution to the

phase problem 59). In this method, diffused light through

a diffuser or speckle patterns paradoxically helps in

reconstructing holographic images (Fig. 3).

Light passing through an optical diffuser is scrambled

and produces a speckle pattern. In general, speckle is

known to deteriorate imaging quality 60). Because an

optical diffuser scrambles the input optical information

in a highly complex manner, an image cannot be simply

reconstructed from the diffused field. This is the reason

why an optical diffuser is often called as opaque glass.

As addressed in Section 2, if the TM of a diffuser is

known, the optical diffuser is a diffractive optics, similar to

conventional optics. The calibration of a TM is useful for

imaging systems which inevitably include diffusive media

such as scattering media 30) and multimode fibers 61) 62).

Indeed, in conventional imaging systems without

EFNNN

field= − +

− +

( / )( )( / )( )

,,π

π4 1 1

1 2 1 1pphasebinary,

. ( )7

E t t e eout m m mnt tm mn

,arg( ) arg( )=

′

− −1

1… … ′

= ∑ abs tmii( ).

( )6

81


diffusive obstructions, there is no reason to introduce a

diffuser and complicate the system, even if the TM of the

diffuser can be obtained or calibrated.

3.3 Speckle-correlation scattering matrix

method

For 3D imaging purpose, the use of an optical diffuser

took a principal role in solving the phase problem 59). In

the approach named 'speckle-correlation scattering

matrix (SSM),' the insertion of an optical diffuser in

front of a 2-D image sensor results in the formation of

highly scrambled speckle patterns at the sensor plane

(Fig. 4).

Exploiting the random nature of a speckle pattern, an

intriguing mathematical relation called Isserlis' theorem

(or Wick's theorem) was utilized to extract the phase

information directly. The Isserlis' theorem holds for

random variables, X1, X2, X3, and X4 as,

where ⟨·⟩ indicates the ensemble average. This relation

has been widely used in quantum field theory as a

mathematical tool.

Using Eq. (8), it was theoretically shown that the

incident field information could be extracted from a

single intensity measurement of a speckle pattern

without any additional requirement. Based on the

proposed theory, experimental demonstrations of a

reference-free holographic camera utilizing an optical

diffuser (Fig. 5) was also performed.

The important advantage of the SSM approach is that

it does not require for any assumption about a target or

incident light. However, the SSM technique still has

several limitations to overcome. First, dealing with the

TM of diffusive media is very challenging. Because

image quality is determined by the number of optical

modes, which is limited by the size of the TM, heavier

calculations and handling of enormous matrices are

required for high-resolution image reconstruction. In

Ref. 59), the actual implementation needed 12 GB of data

storage for about 4, 000 optical modes, and it took 1 (2

minutes to compute the input field. Fortunately, this

problem is a matter of computing ability of digital

devices, so that advancements in devices and computers

can resolve such problems. Another restriction of this

technique is the calibration of the TM, which is achieved

through very complicated measurements. However, this

difficulty also can be alleviated by carefully

manufacturing scattering objects. Generation of exact

replicas of a disordered layer would provide the same

TM. It means that the TM of one layer can also be used

for other replicas.

3.4 Discussion

The single-shot 3-D imaging using the SSM method,

promises a number of interesting future works. A typical

digital camera can function to obtain 3-D image

information by attaching a scattering object in front of

its image plane. As the SSM technique does not need

sophisticated apparatus, it can be applied to mobile

smartphone cameras. In addition, the frequency range of

light for 3-D reconstruction can be expanded to an X-ray

regime. This is a strong benefit because 3-D imaging

with a reference beam in an X-ray regime has extremely

limited due to the limitation on X-ray optics. As the SSM

method enables a generalized single-shot 3-D imaging

for any wavelengths, various new applications in X-ray

X X X X X X X X

X X X X X X X X

1 2 1

1 1

3 4 2 3 4

3 2 4 4 2 3

8=

+ +( )


82

Fig.3 The schematic of proposed idea: an optical diffuser as a

holographic lens. The incident field is converted into the

diffused field by the diffuser whose transmission matrix is

known. Exploiting the random nature of the diffused field,

the incident field and be reconstructed from the speckle

intensity snapshot captured by a camera. This figure is

modified from Reference 59).

Fig.4 The photograph of the first demonstrated holographic

camera using as optical diffuser (left). The optical diffuser is

directly installed in front of the camera using conventional

C-mount thread. The schematic shows the detailed

composition of system (right). The aperture and polarizer is

added to block the ambient light. This figure is modified

from Reference 59).

are anticipated in fields such as medicals or molecular-

level researches.

4. 3D display exploiting complex optics

4.1 Conventional 3D displays and its

shortcomings.

Recently 3D display has drawn significant interest

due to emerging technologies in virtual and augmented

reality. Most types of current commercial 3D displays

mainly employ binocular disparity of human eyes: by

projecting two different images on a viewer, they

produce the perception of 3D effects. This approach

exploits the limitation of the human visual system, but

only 2D projection is controlled in physical space.

A holographic display is an ultimate type of 3D

display because it exactly replicates 3D optical fields of

real objects. For projecting 3D images, SLMs are utilized

in general, in order to control the light fields.

If viewing conditions of 3D holographic displays are

analyzed with two parameters, viewing angle and image

size, then the current 3D holographic display only offer

3D images with a centimeter scale with a very narrow

viewing angle up to a few degrees 24). This limitation is

mainly due to the small number of pixels, or space-

bandwidth product of current SLMs 23). Using a

conventional optical lens, one can tune the image size

and viewing angle of 3D holograms generated with an

SLM, as shown in Fig. 6(a). However, as long as the

space-bandwidth product of the SLM is fixed, the

product between the image size and the viewing angle

remains constant regardless of the transfer lens.

To overcome this limitation, various multiplexing

methods have been proposed to enhance the number of

controlled pixels 23) 63)-68). However, these multiplexing

approaches require complex and expensive systems.

Recently large optical degrees of freedom can be encoded

using metasurfaces 69)-71) or graphene surface 72), but

these new types of wavefront modulators only offer the

generation of static images.

4.2 3D display exploiting volume speckles

In Chapter 3, we introduced that complex light

scattering can solve the phase problem in holographic

imaging. Similarly, in 3D displays, Yu et al.

demonstrated that random light scattering could also be

exploited to overcome the conventional limitation of 3D

holographic displays 73).

In this study, two holographic diffusers were inserted

83


Fig.5 Diffused field retrieval. The reflected field from the diffusive surface of dices are reconstructed by the proposed method. Two dices are

separated by 40 cm from each other (upper middle). The measured intensity speckle image (upper left), and the corresponding reconstructed

field result are shown (upper right). From the measured field result, the focal plane can be freely refocused for desired focal plane (lower

part, the first row). For the clearer visualization, 25 holograms are compounded into one clear image (lower part, the second row). This

figure is modified from Reference 59).

Fig.6 (a) Conventional Fourier holographic display system. The

viewing angle and the image size can be adjusted by the

focal length of the transfer lens. The product between the

viewing angle and the image size is fixed given the number

of pixels in the SLM (b) 3D holographic image projection

exploiting complex light scattering. The light scattered from

holographic diffusers forms volume speckle fields with large

viewing angle and image size. Inside this volume, 3D

images can be projected by properly modulating the

incident field. This figure is modified from Reference 73).

after the DM to ensure sufficient mixing of light

scattering paths. Then, the constant product of image

size and viewing angle was readily broken: the highly

scattered light forms volume speckles fields which

produce both large viewing angle and large image size,

as illustrated in Fig. 6(b).

Due to the randomness of volume speckle fields, no 3D

image is projected in the initial state. However, this

complex light pattern can be controlled to produce 3D

images by using the wavefront shaping techniques or

transmission matrices as introduced earlier. As shown

in the Fig. 7, volume speckle patterns are controlled to

generate '3DHD' letter images consisting of micrometer-

size optical foci with 35° viewing angle in a volume of 2

cm x 2 cm x 2 cm.

4.3 Viewing conditions of scattering display

For 3D images projected inside volume speckle, one

needs to characterize the image size and viewing angle.

In the current situation, 3D images are projected as a

point cloud. Therefore, the viewing angle is determined

by the physical property of single focus forming the point

cloud and the image size is also governed by the

projection area where the wavefront optimization of

single focus successfully works.

Firstly, the viewing angle of the 3D focus depends on

the contributing angle range of the diffusers. When 3D

images are decomposed into point clouds, the viewing

angle of a single point is governed by a supporting range

of spatial frequencies in the focus formation. For

example, if the volume speckle fields are formed after a

holographic diffuser with 60° diffuse angle, then the

viewing angle is the same as this angle range.

Experimentally, the viewing angle is measured by the

size of the focus. For an example, numerical aperture of

an optimized focus is NA = sin 30° = 0.5, so the focus size

d is 0.63 µm (d = 0.51 λ/NA) for 633 nm. In Ref. 73), the

focus size was measured as 1.05 µm corresponding to the

viewing angle of 35°.

Secondly, the image size of the 3D volume is

determined by the working range of wavefront shaping.

Because the wavefront shaping is based on the TM

information, which has no restriction on the spatial

position, the working range can be extended arbitrarily.

In Ref. 73), it was demonstrated that the focus could be

created throughout the diffuser area of 4.2 cm. If larger

diffusers are utilized, and the sufficient randomness is

guaranteed, the image size can be easily extended.

Based on the viewing angle and image size, we can

calculate the product of the viewing angle and image

size. The scattering display showed a striking

enhancement factor of 2, 600 in the product over the

DM-only case without the diffusers. However, it should

be noted that the amount of controllable information in

the SLM is not enhanced. As the information capacity

remains same, the restriction on the viewing angle and

the image size is converted into the limitation on the

contrast and the number of projected points.

4.4 Image quality of scattering 3D display

Since new restriction on the image projection arises in

the scatting display, it is also important to consider the

image quality of projected 3D scenes. Because 3D image

projection is based on a point cloud, the image quality

should also be discussed in the context of a single focus.

For the quality of point cloud images, we can mainly

consider two parameters: the contrast of a single focus

and the number of projected points.

The contrast of a single focus is determined by the

enhancement factor achieved in the wavefront shaping

algorithm. In Ref. 73), the EF of a single focus ranged

from 500-1000 for the use of a DM. However, this EF is

not exactly the same with the actual contrast because

the pixel unit in the calculation of the EF was set to the

size of the physical focus ~ 1 µm. In human perception,

the smallest size of the human eye can resolve much

larger than 1 µm 74). Therefore the actual focus contrast

perceived by viewers would be lower than the EFs. This

consideration of human visual system must be studied

for the future study.

Another important image quality metric is the number

of projected points. For a given the number of controlled

pixels in an SLM, the product between the number of

points and the contrast of single point is fixed. In Ref. 73),

the maximum number of projected points in a single


84

Fig.7 '3DHD' letters are sequentially projected inside a volume of

2 cm x 2 cm x 2 cm. Each letter consists of 15 points with

35°viewing angle. This figure is modified from Reference 73).

frame was 15 for the use of the DMD. However, it is still

far from high-quality 3D image projection. In order to

increase the number of points, the number of controlled

pixels should be enhanced. Recently a new wavefront

shaping system was introduced, which utilizes the half

of the full pixels in a DMD 75). In this system, the

enhancement factor of the single focus was measured as

10,531. Again, the unit pixel of the single focus is 10 µm,

which is still much smaller than PSF of human eyes, so

the contrast reduction is expected for the human

perception. We expect that further developments of this

ultrahigh enhancement system may enable high-quality

image projection in the near future.

4.5 Discussion

In order to generate realistic 3D images with the

scattering display, the image quality should be further

improved. Because the limitation on the number of

pixels in SLMs directly affects the image quality,

increasing the number of controlling pixels or spatial

multiplexing of multiple SLMs would be possible

solutions. Considering human perception factor can be

another solution to overcome the limitation of the

scattering display. For example, the speckle background

is effectively suppressed by temporal multiplexing of

independent light patterns 76), so the contrast can be

effectively increased for the same number of controlled

pixels.

One of the major hardships in the current system is

long optimization time. The measurement of a TM and

the calculation of the optimal incident patterns are

required for 3D image projection. However, the

optimization time is linearly proportional to the number

of controlled pixels in SLMs. In the current wavefront

shaping system, total optimization time for a single spot

is 73 min for Megapixel control 75). To generate a point

cloud with a large number of point elements, further

improvement on the wavefront shaping technique is

highly desired. We envision that the development of

high-speed optimization system may enable fast and

practical 3D projection through scattering media.

5. Conclusion

In this review, we introduced recent progress on

overcoming the limitation of conventional imaging and

display devices by exploiting the complexity of multiple

light scattering. It may seem to contradict to the

common belief that only well-designed optical

components can be used for imaging and projection

purposes. However, multiple scattering can offer new

possibilities otherwise inaccessible with conventional

optics.

Although we discussed the applications of imaging

and display separately, these two topics are closely

related to each other. For example, the hologram is used

in both recording and projection. The underlying concept

bridging these two applications is optical phase

conjugation 10). Recently Lee et al. demonstrated that

scattering media could serve as a medium for recording

the phase-conjugated light 77). Extending this concept to

the macroscopic scale, i.e. recording and replaying back

the light fields originating from real-world objects, is

expected to bring new possibilities in 3D imaging and

display using complex optics.

We expect that the advancement of the optics

applications exploiting complex optics will be

accompanied by the progress in device technologies. As

discussed earlier, the current systems still lack high

speed and large data calculation, fast modulation and

acquisition devices. Therefore, further investigation on

optimizing devices would significantly enhance the

capabilities of complex optics.

References

1) B. Lee: "Three-dimensional displays, past and present", PhysToday 66, 36-41 (2013)

2) P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey,V. Sainov and C. von Kopylow: "A survey of 3DTV displays:techniques and technologies", Circuits and Systems for VideoTechnology, IEEE Transactions on 17, 1647-1658 (2007)

3) R. Collier, Optical holography (Elsevier, 2013) 4) D. Gabor: "A new microscopic principle", Nature 161, 777-778

(1948) 5) E.N. Leith and J. Upatnieks: "Reconstructed wavefronts and

communication theory", JOSA 52, 1123-1130 (1962) 6) I.P. Howard and B.J. Rogers, Binocular vision and stereopsis

(Oxford University Press, USA, 1995) 7) C. Wheatstone: "Contributions to the physiology of vision.--Part the

first. On some remarkable and hitherto unobserved, phenomena ofbinocular vision", Philosophical transactions of the Royal Society ofLondon, 371-394 (1838)

8) D. Gabor: "Microscopy by reconstructed wave-fronts", inProceedings of the Royal Society of London A: Mathematical,Physical and Engineering Sciences (The Royal Society, 1949) , 454-487

9) S.A. Benton: "Hologram reconstructions with extended incoherentsources", in Journal of the Optical Society of America (AMER INSTPHYSICS CIRCULATION FULFILLMENT DIV, 500 SUNNYSIDEBLVD, WOODBURY, NY 11797-2999, 1969) , 1545-&

10) P. Hariharan, Basics of holography (Cambridge university press,2002)

11) H. Kogelnik: "Coupled wave theory for thick hologram gratings",Bell Labs Technical Journal 48, 2909-2947 (1969)

12) J.W. Goodman and R. Lawrence: "Digital image formation fromelectronically detected holograms", Appl Phys Lett 11, 77-79 (1967)

13) M. Takeda, H. Ina and S. Kobayashi: "Fourier-transform methodof fringe-pattern analysis for computer-based topography andinterferometry", J. Opt. Soc. Am. 72, 156-160 (1982)

14) I. Yamaguchi: "Phase-shifting digital holography", in Digital

85


Holography and Three-Dimensional Display (Springer, 2006) ,pp.145-171

15) G. Popescu, Quantitative Phase Imaging of Cells and Tissues(McGraw-Hill Professional, 2011)

16) K. Lee, K. Kim, J. Jung, J.H. Heo, S. Cho, S. Lee, G. Chang, Y.J.Jo, H. Park and Y.K. Park: "Quantitative phase imagingtechniques for the study of cell pathophysiology: from principles toapplications", Sensors-Basel 13, 4170-4191 (2013)

17) P. Marquet, B. Rappaz, P.J. Magistretti, E. Cuche, Y. Emery, T.Colomb and C. Depeursinge: "Digital holographic microscopy: Anoninvasive contrast imaging technique allowing quantitativevisualization of living cells with subwavelength axial accuracy",Optics Letters 30, 468-470 (2005)

18) G. Popescu, L.P. Deflores, J.C. Vaughan, K. Badizadegan, H. Iwai,R.R. Dasari and M.S. Feld: "Fourier phase microscopy forinvestigation of biological structures and dynamics", Optics letters29, 2503-2505 (2004)

19) Y. Park, G. Popescu, K. Badizadegan, R.R. Dasari and M.S. Feld:"Diffraction phase and fluorescence microscopy", Opt Express 14,8263-8268 (2006)

20) T.-C. Poon, Digital holography and three-dimensional display:Principles and Applications (Springer Science & Business Media,2006)

21) N. Kukhtarev, V. Markov, S. Odulov, M. Soskin and V. Vinetskii:"Holographic storage in electrooptic crystals.I. Steady state",ferroelectrics 22, 949-960 (1978)

22) S.P. McGrew: "Hologram counterfeiting: problems and solutions",in OE/LASE'90, 14-19 Jan. , Los Angeles, CA (InternationalSociety for Optics and Photonics, 1990) , 66-76

23) J. Hahn, H. Kim, Y. Lim, G. Park and B. Lee: "Wide viewing angledynamic holographic stereogram with a curved array of spatiallight modulators", Optics express 16, 12372-12386 (2008)

24) F. Yaras, H. Kang and L. Onural: "State of the art in holographicdisplays: A survey", Journal of display technology 6, 443-454(2010)

25) K. Wakunami, P.-Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y.Ichihashi, M. Okui, Y.-P. Huang and K. Yamamoto: "Projection-type see-through holographic three-dimensional display", Naturecommunications 7, 12954 (2016)

26) H.-j. Kim, S.-K. Lee, M.-L. Piao, N. Kim and J.-H. Park: "Three-dimensional holographic head mounted display using holographicoptical element", in Consumer Electronics (ICCE) , 2015 IEEEInternational Conference on (IEEE, 2015) , 132-133

27) X. Hu and H. Hua: "High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics", Opticsexpress 22, 13896-13903 (2014)

28) D. Dunn, C. Tippets, K. Torell, P. Kellnhofer, K. Aksit, P. Didyk,K. Myszkowski, D. Luebke and H. Fuchs: "Wide Field Of ViewVarifocal Near-Eye Display Using See-Through DeformableMembrane Mirrors", IEEE Transactions on Visualization andComputer Graphics (2017)

29) A.P. Mosk, A. Lagendijk, G. Lerosey and M. Fink: "Controllingwaves in space and time for imaging and focusing in complexmedia", Nature photonics 6, 283-292 (2012)

30) S. Popoff, G. Lerosey, R. Carminati, M. Fink, A. Boccara and S.Gigan: "Measuring the transmission matrix in optics: an approachto the study and control of light propagation in disordered media",Physical review letters 104, 100601 (2010)

31) I.M. Vellekoop and A. Mosk: "Focusing coherent light throughopaque strongly scattering media", Optics Letters 32, 2309-2311(2007)

32) C. Park, J.-H. Park, C. Rodriguez, H. Yu, M. Kim, K. Jin, S. Han,J. Shin, S.H. Ko and K.T. Nam: "Full-Field SubwavelengthImaging Using a Scattering Superlens", Physical review letters113, 113901 (2014)

33) J.-H. Park, C. Park, H. Yu, J. Park, S. Han, J. Shin, S.H. Ko, K.T.Nam, Y.-H. Cho and Y. Park: "Subwavelength light focusing usingrandom nanoparticles", Nature photonics 7, 454-458 (2013)

34) Y. Choi, T.D. Yang, C. Fang-Yen, P. Kang, K.J. Lee, R.R. Dasari,

M.S. Feld and W. Choi: "Overcoming the diffraction limit usingmultiple light scattering in a highly disordered medium", Physicalreview letters 107, 023902 (2011)

35) D.B. Conkey and R. Piestun: "Color image projection through astrongly scattering wall", Optics express 20, 27312-27318 (2012)

36) H. Yu, T.R. Hillman, W. Choi, J.O. Lee, M.S. Feld, R.R. Dasari andY. Park: "Measuring large optical transmission matrices ofdisordered media", Physical review letters 111, 153902 (2013)

37) X.D. Tao, D. Bodington, M. Reinig and J. Kubby: "High-speedscanning interferometric focusing by fast measurement of binarytransmission matrix for channel demixing", Optics Express 23,14168-14187 (2015)

38) D.B. Conkey, A.M. Caravaca-Aguirre and R. Piestun: "High-speedscattering medium characterization with application to focusinglight through turbid media", Optics express 20, 1733-1740 (2012)

39) J. Yoon, K. Lee, J. Park and Y. Park: "Measuring opticaltransmission matrices by wavefront shaping", Optics Express 23,10158-10167 (2015)

40) I. Vellekoop and A. Mosk: "Universal optimal transmission of lightthrough disordered materials", Physical Review Letters 101,120601 (2008)

41) D. Akbulut, T.J. Huisman, E.G. van Putten, W.L. Vos and A.P.Mosk: "Focusing light through random photonic media by binaryamplitude modulation", Optics Express 19, 4017-4029 (2011)

42) J.-H. Park, C. Park, H. Yu, Y.-H. Cho and Y. Park: "Dynamicactive wave plate using random nanoparticles", Opt. Exp. 20,17010-17016 (2012)

43) J.H. Park, C.H. Park, H. Yu, Y.H. Cho and Y.K. Park: "Activespectral filtering through turbid media", Opt. Lett. 37, 3261-3263(2012)

44) J. Aulbach, B. Gjonaj, P.M. Johnson, A.P. Mosk and A. Lagendijk:"Control of light transmission through opaque scattering media inspace and time", Physical Review Letters 106, 103901 (2011)

45) O. Katz, E. Small, Y. Bromberg and Y. Silberberg: "Focusing andcompression of ultrashort pulses through scattering media",Nature photonics 5, 372-377 (2011)

46) D.J. McCabe, A. Tajalli, D.R. Austin, P. Bondareff, I.A. Walmsley,S. Gigan and B. Chatel: "Spatio-temporal focusing of an ultrafastpulse through a multiply scattering medium", NatureCommunications 2, 447 (2011)

47) D. Gabor: "IV Light and Information", Progress in optics 1, 109-153 (1961)

48) P. Hariharan, Optical Holography: Principles, techniques andapplications (Cambridge University Press, 1996)

49) M. Tegze and G. Faigel: "X-ray holography with atomicresolution", Nature 380, 49-51 (1996)

50) K. Lee and Y. Park: "Exploiting the speckle-correlation scatteringmatrix for a compact reference-free holographic image sensor",Nature Communications 7(2016)

51) X.Q. Cui, J.A. Ren, G.J. Tearney and C.H. Yang: "Wavefront imagesensor chip", Optics Express 18, 16685-16701 (2010)

52) R.V. Shack and B.C. Platt: "Production and Use of a LenticularHartmann Screen", J Opt Soc Am 61, 656-& (1971)

53) D. Paganin and K.A. Nugent: "Noninterferometric phase imagingwith partially coherent light", Phys Rev Lett 80, 2586-2589 (1998)

54) R. Hegerl and W. Hoppe: "Phase evaluation in generalizeddiffraction (ptychography) ", Proc. Fifth Eur. Cong. ElectronMicroscopy, 628-629 (1972)

55) J.M. Rodenburg, A.C. Hurst, A.G. Cullis, B.R. Dobson, F. Pfeiffer,O. Bunk, C. David, K. Jefimovs and I. Johnson: "Hard-x-raylensless imaging of extended objects", Phys Rev Lett 98(2007)

56) J.R. Fienup: "Phase Retrieval Algorithms - a Comparison", Appl.Opt. 21, 2758-2769 (1982)

57) R.W. Gerchberg: "A practical algorithm for the determination ofphase from image and diffraction plane pictures", Optik 35, 237(1972)

58) G.A. Zheng, R. Horstmeyer and C.H. Yang: "Wide-field, high-resolution Fourier ptychographic microscopy", Nature Photonics 7,739-745 (2013)


86

59) K. Lee and Y. Park: "Exploiting the speckle-correlation scatteringmatrix for a compact reference-free holographic image sensor",Nature Communications 7, 13359 (2016)

60) J.W. Goodman: "Statistical properties of laser speckle patterns", inLaser Speckle and Related Phenomena (Springer BerlinHeidelberg, 1975) , pp.9-75

61) Y. Choi, C. Yoon, M. Kim, T.D. Yang, C. Fang-Yen, R.R. Dasari,K.J. Lee and W. Choi: "Scanner-Free and Wide-Field EndoscopicImaging by Using a Single Multimode Optical Fiber", Phys RevLett 109(2012)

62) T. Cizmar and K. Dholakia: "Exploiting multimode waveguides forpure fibre-based imaging", Nature Communications 3(2012)

63) N. Fukaya, K. Maeno, O. Nishikawa, K. Matsumoto, K. Sato andT. Honda: "Expansion of the image size and viewing zone inholographic display using liquid crystal devices", in IS&T/SPIE'sSymposium on Electronic Imaging: Science & Technology(International Society for Optics and Photonics, 1995) , 283-289

64) K. Maeno, N. Fukaya, O. Nishikawa, K. Sato and T. Honda:"Electro-holographic display using 15mega pixels LCD", inElectronic Imaging: Science & Technology (International Societyfor Optics and Photonics, 1996) , 15-23

65) Y.-Z. Liu, X.-N. Pang, S. Jiang and J.-W. Dong: "Viewing-angleenlargement in holographic augmented reality using time divisionand spatial tiling", Optics Express 21, 12068-12076 (2013)

66) Z.M. A. Lum, X. Liang, Y. Pan, R. Zheng and X. Xu: "Increasingpixel count of holograms for three-dimensional holographic displayby optical scan-tiling", OPTICE 52, 015802-015802 (2013)

67) Y. Matsumoto and Y. Takaki: "Improvement of gray-scalerepresentation of horizontally scanning holographic display usingerror diffusion", Optics Letters 39, 3433-3436 (2014)

68) Y. Takaki and K. Fujii: "Viewing-zone scanning holographicdisplay using a MEMS spatial light modulator", Optics Express 22,24713-24721 (2014)

69) S. Larouche, Y.-J. Tsai, T. Tyler, N.M. Jokerst and D.R. Smith:"Infrared metamaterial phase holograms", Nature materials 11,450-454 (2012)

70) L. Huang, X. Chen, H. Mühlenbernd, H. Zhang, S. Chen, B. Bai, Q.Tan, G. Jin, K.-W. Cheah, C.-W. Qiu, J. Li, T. Zentgraf and S.Zhang: "Three-dimensional optical holography using a plasmonicmetasurface", Nat Commun 4(2013)

71) X. Ni, A.V. Kildishev and V.M. Shalaev: "Metasurface hologramsfor visible light", Nature communications 4(2013)

72) X. Li, H. Ren, X. Chen, J. Liu, Q. Li, C. Li, G. Xue, J. Jia, L. Cao,A. Sahu, B. Hu, Y. Wang, G. Jin and M. Gu: "Athermallyphotoreduced graphene oxides for three-dimensional holographicimages", Nat Commun 6(2015)

73) H. Yu, K. Lee, J. Park and Y. Park: "Ultrahigh-definition dynamic3D holographic display by active control of volume speckle fields",Nat Photon 11, 186-192 (2017)

74) M. Yamaguchi and T. Utsugi: "Speckle suppression incomputational hologram generated using ray-sampling plane", in2014) , 91170A-91170A-91110

75) H. Yu, K. Lee and Y. Park: "Ultrahigh enhancement of lightfocusing through disordered media controlled by Megapixelmodes", arXiv preprint arXiv:1701.02933 (2017)

76) J.G. Manni and J.W. Goodman: "Versatile method for achieving1% speckle contrast in large-venue laser projection displays usinga stationary multimode optical fiber", Optics Express 20, 11288-11315 (2012)

77) K. Lee, J. Lee, J.H. Park, J.H. Park and Y. Park: "One-WaveOptical Phase Conjugation Mirror by Actively Coupling ArbitraryLight Fields into a Single-Mode Reflector", Physical ReviewLetters 115(2015)

87


YongKeun Park is Associate Professor ofPhysics at KAIST. He earned a Ph.D. from Harvard-MIT Health Science and Technology. Dr. Park's area ofresearch is wave optics and biophotonics.

KyeoReh Lee received his BS degree in physicsfrom KAIST. He is currently a PhD student in theDepartment of Physics at KAIST.

SeungYoon Han is currently a BS student inthe Department of Physics at KAIST.

Jongchan Park received his BS degree inphysics from KAIST. He is currently a PhD student inthe Department of Physics at KAIST.

YoonSeok Baek received his BS degree inphysics from KAIST. He is currently a PhD student inthe Department of Physics at KAIST.

Hyeonseung Yu received his PhD degree inPhysics from KAIST in 2017. He received his BSdegree in physics and mathematical science fromKAIST. He is currently a postdoctoral associate in MaxPlanck Institute for Informatics.

review: 3d holographic imaging and display exploiting

Documents