3D Display Technology in Medical Imaging Field Yue Zhao1,2, Yongtao Wei1,2,*, Xiaoyu Cui1,2, Luxuan Qu1,2, Lin Liu1,2,Yusong Wang1,2

1. Sino-Dutch Biomedical and Information Engineering School, Northeastern University, Shenyang, Liaoning, 110819,China.E-mail:zhaoyue@bmie.neu.edu.cn

2.�Key Laboratory of Medical Image Computing, Ministry of Education, Northeastern University, Shenyang, Liaoning, 110179, China. E-mail:zhaoyue@bmie.neu.edu.cn

*. Corresponding author: Yongtao Wei,E-mail:weiyt2012@163.com. Abstract: 3D display can reflect the image depth information and make the image present stereoscopic sensation to improve the medical imaging effect and diagnosis accuracy, if the technology can be used in image guide surgery to diagnose lesion location. This paper will mainly elaborate some advanced naked eye 3D display technologies——multi-view auto-stereoscopic display technology, integral photography 3D display technology and holographic 3D display technology. At last, image guide surgery based on interactive 3D navigation system is proposed. Meanwhile, this paper indicates the trend of medical imaging fields. Key words 3D Display, Medical Imaging, Diagnosis, Interactive, Navigation.

1. INTRODUCTION

Medical industry has been developing rapidly in recent years. Medical imaging research becomes a hot research topic. Image guide surgery technology requires higher imaging quality and better vision effect to help diagnose diseases, which is the goal of the study of many researchers and doctors. However, conventional medical equipment usually only provide 2D display images. In this case, the images are lack for the depth cues, so that the images lose the stereoscopic sensation. However, 3D display technology can provide depth cues to make images present stereoscopic sensation, so doctors can accurately diagnose lesion location and cure it. Thus, 3D display has a bright application prospect and will be a trend in medical imaging field [1-5].

This paper is arranged as follows. Firstly, some advanced naked eye 3D display technologies will be successively elaborated, such as multi-view auto-stereoscopic 3D display technology, integral photography 3D display technology and holographic 3D display technology. Then, 3D display technology about image guide surgery will be expounded. Meanwhile a trend of the medical imaging development will be envisaged. 2. Multi-view auto-stereoscopic 3D display

technology [7-15]

Multi-view auto-stereoscopic display technology

mainly includes time division multiplexing technology, space division multiplexing technology and multiple projection technology.

Time division multiplexing technology transmits multi-view of scene to display screen by obeying time division and isolate the visual areas by optics and circuit controlling equipment. Dodgson [13] ,et al, show a kind of time division multiplexing auto-stereoscopic display system. The system control super-speed CRT to display multi-view images, and optics control devices control the shutter location to form the vision area in the screen.3D display will be presented according to the principle of human eye vision. The principle of this technology is showed in Figure 1. However, mechanical inertia and photoelectric delay characteristics of this system will impact view numbers and system scalability.

�

Figure 1.The principle of time division multiplexing

technology

Space division multiplexing technology makes the multi-view images multiplex an image by interleaved

____________________________________978-1-4673-5887-3/13/$31.00 ©2013 IEEE

method and then make the images project to the screen. Optics controlling devices will isolate multi-view images and form vision area by multiplexing technology. Isono[14], et al, design a kind of space division multiplexing system. This system is composed of four cameras, a high resolution LCD projector, lenticular lens raster plane and Fresnel lens screen. The principle of space division multiplexing technology is showed in Figure 2.The resolution is inferior to that of time division multiplexing technology, but the refresh frequency requirement is lower than that of time division multiplexing technology. This technology can improve the resolution and display space by the cooperation between multiple projectors.

�Figure 2.The principle of space division multiplexing

display technology

Multiple projector display technology mainly utilizes multiple projectors to capture multi-view images. This technology isolates the multi-view images by using double lenticular lens panel. Matusik and Annen[15]

design a multiple projector display system, which mainly includes projectors, cameras and computers. The principle of this technology is showed in Figure 3.

�

Figure 3.The principle of Multiple projector display technology.

This technology usually requires that the multiple projector area is consistent with the projected zone. Thus, the location of multiple projectors is difficult to adjust.

Moreover, the optical axis of most multiple projectors is usually dissatisfied with vertical relations with optical screen, so the image deformation is difficult to solve. Meanwhile, it is difficult to expand the vision area further.

3.Integral photography 3D display technology[16-18]

Integral photography 3D display technology utilizes micro-lens array or pinhole array to record and recover the space 3D image scene. Lippmann proposes integral photography composed of recording process and reproduction process. The principle of integral photography is showed in Figure 4.Recording process utilizes a pinhole array to make the object information recorded to the image sensor in Figure 12(a). And each pinhole can form an elemental image on the image sensor according to the principle of small hole imaging. Reproduction process makes these elemental images recovered in the high resolution display according to the reversibility principle of beam path in Figure 12(b).Thus we can see the 3D display images.

Figure 4.The principle of Integral photography 3D display

technology.

Integral photography 3D display technology has some advantages as follows.

(1) There is no need for coherent light source and there is no specific requirement about record environment.

(2) This technology can produce true-color images because the 3D space scene information is the point-to-point correspondence .

(3) There are some continuous viewpoints to meet

many people to watch freely at the same time. (4) This technology can not only be used to the static

3D scene information ,but also to 3D videos. (5) It is easy to transmit and dispose an image primitive

because an elemental image exists in 2D form to record and recover.

However, there are some disadvantages due to micro-lens array or pinhole array and the resolution of devices, such as 3D image inversion, narrow viewing angle range and depth range, lower resolution and so on.

4. Holographic 3D display technology [16,18-20]

The principle of holographic display technology utilizes optical interference method to record the amplitude and phase information of object light-wave. The recovering images of holographic display have the same 3D features as the real object. Thus, holographic 3D display technology is deemed to the optimal 3D display technology. The fundamental principle of digital holographic 3D display technology will be expounded as follows in Figure 5.

Figure 5.The principle of digital holographic 3D display technology

SLM is core for digital holographic 3D display technology to recover real the object images. Holographic images are generated by CCD or computers and recover the 3D object images according to the incident reference beam. Nowadays SLM mainly include LC, DMD, PRC, AOM and so on.

However, holographic 3D display technology is still immature, especially dynamic holographic 3D display. Furthermore, there are some urgent key technology problems to improve recording speed, antiknock characteristic and true-color aspects. Holographic 3D display technology is a trend of 3D display in future. 5. 3D display technology in medical imaging

Nowadays, the typical medical imaging devices mainly include CT, MRI, US, PET and endoscopy. Image guide

surgery[11, 21-32] based on interactive 3D navigation system can apply for various anatomical models and clinical scenarios, such as orthopedic surgery and neurosurgery. The structure chart of interactive 3D navigation system is showed in Figure 6. If medical imaging can realize 3D display, many medical negligence can be avoided, such as misdiagnosis and missed diagnosis. Thus, 3D display is doomed to a trend of medical imaging. Nowadays, 3D display technology usually needs some aided equipment to realize the 3D effects, such as 3D glasses and HMD. On the one hand, that makes us feel visual fatigue ;on the other hand ,that also makes it inconvenient to go on some other activities. If naked eye 3D display in medical imaging field, that will be a great process in the development of medical science.

Figure 6.The structure chart of interactive 3D navigation system [25]

We can understand the combination of the multi-modal medical image fusion and 3D display technology will be a trend of development from the above applications. Now medical image fusion is usually two-modal fusion, such as MRI/CT, PET/CT and PET/MRI. If we will realize three-modal fusion or multi-modal fusion, there are still many technical problems. Moreover, naked eye 3D display technology has not been widely used in medical imaging field for some technical bottlenecks. Meanwhile, this also means there will be a bright prospect in this field. If all various medical imaging equipment can be integrated into an exquisite equipment and realize 3D display of multi-modal image fusion, that will be a milestone event in the development of medical applications. REFERENCE

[1]. X.Y. Cui. Research of prism based endoscopy aided diagnosis system [D].Shenyang Northeastern University,2012.(in Chinese) [2]. J. Li.3D display technology: real world, real sight[J]. Electronic Engineering & Product World, 2012(04). (in Chinese) [3]. Y. Wang, K. Sun, S.X. Sun. The status and development of 3D display technology [J]. Advanced Display, 2012(02).(in Chinese) [4]. R.T. Held, T.T. Hui. A guide to stereoscopic 3D displays in medicine[J], Academic radiology, 2011, 18(8): 1035-1048.[5]. T. Yang, Y. Liu, X. Li. A 3D Perspective Display Cube for Multiview Tracking[J], Advanced Materials

Research, 2011, 271: 144-148.[6]. G. Lawton.3D displays without glasses: coming to a screen near you[J].Computer, 2011, 44(1): 17-19.[7].P. Boher, T. Leroux, T. Bignon, et al. Optical characterization of different types of 3D displays[C]//SPIE OPTO. International Society for Optics and Photonics, 2012: 82800A-82800A-16.[8]. K.H. Qin, J. L. Luo. Techniques for auto-stereoscopic display and its development[J]. Journal of Image and Graphics,2009(10).(in Chinese) [9].Y. Zhu, T. Zhen. 3D multi-view autostereoscopic display and its key technologie[C]//Information Processing, 2009. APCIP 2009. Asia-Pacific Conference on. IEEE, 2009, 2: 31-35.[10]. M. Karsten, S. Aljoscha, D. Kristina, et al. View synthesis for advanced 3D video systems[J]. EURASIP Journal on Image and Video Processing, 2009, 2008.[11]. B. Kim, H. Lee, W.Y. Kim. Rapid eye detection method for non-glasses type 3D display on portable devices[J]. Consumer Electronics, IEEE Transactions on, 2010, 56(4): 2498-2505.[12].A. Schwerdtner. Autostereoscopic 3D display[C] //Proc. SPIE. 2006, 6055: 270-279.[13]. N.A. Dodgson, J. R. Moore, S. R. Lang. Multi-view autostereoscopic 3D display[C]//IBC'99 (International Broadcasting Convention). 1999, 1: 497-502.[14].H. Isono, M. Yasuda, D. Takemori, et al. 50-inch autostereoscopic full-color 3D TV display system[A].In:Proceedings of SPIE[C],San Jose,CA,USA,1992:176-185.

[15]. W. Matusik, H. Pfister. 3D TV: A scalable system for real-time acquisition, transmission and autostereoscopic display of dynamic scenes[J].ACM Transactions on Graphic,2004,23(3):814-824. [16]. Q. H. Wang.3D display technology and devices [M]. Beijing: Science Press,2011.4.(in Chinese) [17]. B. Lee, H. Choi, J. Kim, et al. Status and prospects of autostereoscopic 3D display technologies[C]//Lasers and Electro-Optics Society, 2007. LEOS 2007. The 20th Annual Meeting of the IEEE. IEEE, 2007: 354-355.[18]. D. C. Wang, H. N. Wang. About stereoscopic vision and true 3D volumetric display technology[J].Advanced Display,2009(09).(in Chinese) [19].J. Hong, Y. Kim, H. J. Choi, et al. Three-dimensional display technologies of recent interest: principles, status, and issues [Invited][J]. Applied optics, 2011, 50(34): H87-H115. [20]. S .Reichelt, R. Häussler, G. Fütterer, et al. Depth cues in human visual perception and their realization in 3D displays[C]//SPIE Defense, Security, and Sensing. International Society for Optics and Photonics, 2010: 76900B-76900B-12. [21].R. Sidhu, J. Weir-McCall, F. Cochennec, et al. Evaluation of an electromagnetic 3D navigation system to facilitate endovascular tasks: a feasibility study[J]. European Journal of Vascular and Endovascular Surgery, 2012, 43(1): 22-29.[22].M. Silvestri, T. Ranzani, A. Argiolas, et al. A multi-point of view 3D camera system for minimally invasive surgery[J]. Sensors and Actuators A: Physical, 2013.[23]. P. H. Huang, C. Y. Chen, C. J. Chang, et al. Computer aided ergonomics and visualization for improving 3D display techniques[J]. Human Factors and Ergonomics in Manufacturing & Service Industries, 2012, 22(6): 556-561.[24]. M. R. Schaberg, V. K. Anand, T.H. Schwartz, et al. Microscopic versus endoscopic transnasal pituitary

surgery[J]. Current opinion in otolaryngology & head and neck surgery, 2010, 18(1): 8-14. [25]. H.H. Tran, K. Matsumiya, K. Masamune, et al. Interactive 3D navigation system for image-guided surgery[J]. The International Journal of Virtual Reality, 2009, 8(1): 9-16.[26]. M. Figl, D. Rueckert,E. Edwards. Photo-consistency registration of a 4D cardiac motion model to endoscopic video for image guidance of robotic coronary artery bypass[C]//SPIE Medical Imaging. International Society for Optics and Photonics, 2009: 72611X-72611X-9. [27]. L. Lu, C.Chen, W. Cheng. Medical image visualization using true 3D display technology[C]//Complex Medical Engineering, 2007. CME 2007. IEEE/ICME International Conference on. IEEE, 2007: 914-918.[28]. F. Zafar, J. Dorband, A. Badano. Computational observer approach for the assessment of stereoscopic visualizations for 3D medical images[C]//SPIE Medical Imaging. International Society for Optics and Photonics, 2012: 831806-831806-7.[29]. S. Luan, T. Wang, W. Li, et al. 3D navigation and monitoring for spinal milling operation based on registration between multiplanar fluoroscopy and CT images[J]. Computer methods and programs in biomedicine, 2012.[30]. D. S. F. Magalhães, R. L.Serra, A. L. Vannucci, et al. Glasses-free 3D viewing systems for medical imaging[J]. Optics & Laser Technology, 2012, 44(3): 650-655.[31]. D. A. Auger, X. Zhong, F. H. Epstein, et al. Mapping right ventricular myocardial mechanics using 3D cine DENSE cardiovascular magnetic resonance[J]. Journal of Cardiovascular Magnetic Resonance, 2012, 14(1): 1-9.[32]. M. V. Burri, D. Gupta, R. E. Kerber, et al. Review of novel clinical applications of advanced, real-time, 3-dimensional echocardiography[J]. Translational Research, 2012, 159(3): 149-164.� �