international workshop on advanced image technology...

Held in: Supported by: Technical Sponsor:

Hosted by:Multi-plAtform Game Innovation Centre (MAGIC)

Sponsors:

International Workshop on Advanced Image Technology International Forum on Medical Imaging in Asia

The 2019

&

PROCEEDINGS

Computational Anatomy

Volumetric Brain Model Mapping for Constructing Volume Statistical Shape Model

Shoko Miyauchi¹, Ken’Ichi Morooka¹, Yasushi Miyagi², Takaichi Fukuda³, Ryo Kurazume¹

Kyushu University¹, Fukuoka Mirai Hospital², Kumamoto University³

[email protected], [email protected], [email protected], [email protected], [email protected]

Volume Statistical Shape Model, Model correspondence, volumetric Self-organizing Deformable Model

In order to construct a volume Statistical Shape Model (SSM) of human brains including inner organs with complicated shapes, we propose a new method for mapping a brain volume model onto a target volume. The proposed method is based on our volumetric Self-organizing Deformable Model (vSDM) which achieves a mapping between organ volume models including inner organs while controlling the mapping locations of the inner organs. However, since inner organs of the brain have complicated shapes, the mapped brain contains more self-intersections. To avoid this, we introduce into vSDM a new process of correcting the self-intersection. Moreover, by using the mapped brain models, we construct a volume SSM of human brain which represents the shape variations of not only brain surface but also surfaces of inner organs.

Paper No. 35

2

Investigation of Extracting the Interlobular Septa With Combination Of Hessian Analysis And Radial Structure Tensor In Micro-CT Volume

Xiaotian Zhao, Hirohisa Oda, Shota Nakamura, Yuichiro Hayashi, Hayato Itoh, Masahiro Oda, Kensaku Mori

Nagoya University

[email protected], [email protected], [email protected], [email protected],

[email protected], [email protected]

Structure extraction, Structure Analysis, Plane Structure, Interlobular septa, micro-CT

The spread of Micro-CT uses in medical fields is expected to bring further understanding of the human anatomy by analyzing the three-dimensional microstructure from volumes of the vital specimen. In the topic of microstructure analysis of resected lung specimen, the surface structure such as the interlobular septa and the visceral pleura extraction were not commonly studied. In this paper, we introduce a method to extract sheet structure such as interlobular septa and visceral pleura from micro-CT volume. Our proposed method consists of two methods: Hessian analysis and Radial Structure Tensor (RST). The Hessian analysis method is used to extract surface structure and solid tube structure, while RST is used to extract hollow tube structure. We adopted the proposed method on lung micro-CT volume and confirmed the extraction of the interlobular septa.

Paper No. 180

3

Computer Aided Diagnosis

Automated approach for estimation of sizes of unruptured intracranial aneurysms in MRA images using localized principal component analysis

Zhuangfei Ma¹, Hidetaka Arimura¹, Shingo Kakeda², Yukunori Korogi²

Kyushu University¹, University of Occupational & Environmental Health School of Medicine²

[email protected], [email protected], [email protected], [email protected]

principal component analysis, unruptured intracranial aneurysm, estimation sizes

It is substantially difficult for radiologists to measure intracranial aneurysm sizes because of overlapping structures and/or unusual locations, especially for aneurysms smaller than 7mm. Therefore, we have developed an automated approach for estimation of unruptured intracranial aneurysm sizes in MRA images. The errors of estimated aneurysm sizes in the longest, middle and shortest diameters were 2.53%, 10.79% and 12.62%, respectively.

Paper No. 15

4

Lung CT Computer-Aided Diagnosis using Multi-scale Densely Connected Network

Yao-Sian Huang, Ruey-Feng Chang, Jing-Kai Hong, Yeun-Chung Chang

National Taiwan University


Lung cancer, Computed tomography, Computer-aided diagnosis, Densely connected network, Selective sampling

Lung cancer is the common causes of death today. In recent, the convolution neural network (CNN) has been used in medical imaging for lesion detection, segmentation, and classification and the densely connected convolutional neural network (DenseNet) is one of famous CNN architecture. Hence, in this research, a computer-aided diagnosis (CAD) system, multi-scale DenseNet (MSDN), is proposed for lung nodule classification in computed tomography (CT) image. The proposed MSDN use three different nodule sizes as inputs instead of single input. Furthermore, the selective sampling (SeS) method is adopted on our dataset. There are 138 benign and 70 malignant cases with pathology proven used in this research. In our experiments, the proposed CAD system achieves the high performance with 90.0%, 89.6%, and 91.8% in accuracy (Acc), sensitivity (Sen), and specificity (Spec) repsectively.

Paper No. 24

5

Segmentation of intervertebral disks from videofluorographic images using convolutional neural network

Ayano Fujinaka¹, Yuki Saito¹, Kojiro Mekata², Hotaka Takizawa³, Hiroyuki Kudo³

University of Tsukuba¹, Kobe Red Cross Hospital², University of Tsukuba³


Videofluorography, Swallowing, Intervertebral disks, Deep learning, Convolutional neural network

Swallowing is achieved by a sequence of actions performed by cervical structures. Although a lot of patients suffer from dys-phagia in the world, the mechanism and kinematics of swallow-ing are not elucidated sufficiently. This study aims to segment intervertebral disks (IDs), which are ones of representative cervi-cal structures, in videofluorographic (VF) images by use of convo-lutional neural network (CNN). The proposed method consists of three steps: extraction of cervical masks, CNN-based segmenta-tion of candidate regions of IDs, and post-processing. This seg-mentation method was applied to actual VF images of eleven par-ticipants that have fifty-one not-occluded IDs, and forty-six IDs were segmented successfully. One of the experimental results was shown.

Paper No. 36

6

Computer-aided Diagnosis of Breast Cancer Using Ensemble Convolutional Neural Networks

Yan-Wei Lee, Hao-Hsiang Ke, Ruey-Feng Chang


[email protected], [email protected], [email protected]

Breast cancer, Breast ultrasound, CAD, Deep learning, CNN, Ensemble learning

Breast cancer is the most common malignancy of the total cancer cases in United States females. However, early diagnosis leads to early treatment and reduces mortality rates. In the clinical usage, breast ultrasound and computer aided diagnosis (CAD) is usually used to diagnosis tumors into benignancy or malignancy. In addition, CAD has been used to decrease the diagnosis variation of different physicians and assist to classify or detect the tumors. In our study, we use the convolutional neural network (CNN) for automatic feature extraction and the ensemble method to combine multi CNN models for better diagnostic performance. The CNN-based method proposed in this study includes VGG-model, ResNet-model, and DenseNet-model. Also, we used a fully convolutional network (FCN) method for tumor segmentation and extract tumor shape features automatically. There were total 1687 tumors used in this study, including 953 benign tumors and 734 malignant tumors. The accuracy (ACC), sensitivity (SEN), and the specificity (SPEC) were 88.72%, 81.08%, and 94.71% respectively, and the area under the ROC curve was 0.9585. In conclusions, the ensemble method can improve the performance by using multiple CNN methods and the tumor shape feature can improve the diagnostic capability.

Paper No. 58

7

Automatic Metastatic Bone Tumor Classification with DCNN-based Features Using Treatment-planning CT Images

Haruna Watanabe, Ren Togo, Takahiro Ogawa, Miki Haseyama, Koichi Yasuda, Khin Tha, Kohsuke Kudo, Hiroki Shirato

Hokkaido University

[email protected], [email protected], [email protected], [email protected], [email protected], [email protected],


metastatic bone tumor, Treatment-planning computed tomography, deep convolutional neural network

In this paper, we propose a method to classify metastatic bone tumors using treatment-planning computed tomography images. The proposed method utilizes pre-trained deep convolutional neural network (DCNN) models as feature extractors and enables the classification by using the obtained features. Performance of several state-of-the-art DCNN-based features was compared and evaluated in our experiment.

Paper No. 61

8

Initial Study on The Classification Of Amyotrophic Diseases Using Texture Analysis And Deep Learning In Whole-Body CT Images

Naoki Kamiya¹, Ami Oshima¹, Erika Asano², Xiangrong Zhou², Megumi Yamada², Hiroki Kato², Chisako Muramatsu², Takeshi Hara², Toshiharu Miyoshi², Masayuki Matsuo², Hiroshi Fujita²

Aichi Prefectural University¹, Gifu University²


Computer-aided Diagnosis, amyotrophic diseases, texture analysis, deep learning

The skeletal muscle exists in the whole body and can be observed in many cross sections in various tomographic images. Skeletal muscle atrophy is due to aging and disease, and the abnormality is difficult to distinguish visually. In addition, although skeletal muscle analysis requires a technique for accurate site-specific measurement of skeletal muscle, it is only realized in a limited region. We realized automatic site-specific recognition of skeletal muscle from whole-body CT images using model-based methods. Three-dimensional texture analysis revealed imaging features with statistically significant differences between amyotrophic lateral sclerosis (ALS) and other muscular diseases accompanied by atrophy. In recent years, deep learning technique is also used in the field of computer-aided diagnosis. Therefore, in this initial study, we performed automatic classification of amyotrophic diseases using deep learning for the upper extremity and lower limb regions. The classification accuracy was highest in the right forearm, which was 0.960 at the maximum (0.903 on average). In the future, methods for differentiating more kinds of muscular atrophy and clinical application of ALS detection by analyzing muscular regions must be considered.

Paper No. 64

9

Pilot Study to Generate Image Features by Deep Autoencoder for Computer-Aided Detection Systems

Mitsutaka Nemoto¹, Kazuyuki Ushifusa¹, Yuichi Kimura¹, Naoto Hayashi², Moe Kadosawa, Mitsunori Makino

Kindai University¹, The University of Tokyo²


convolutional autoencoder, anomaly detection, image feature, computer-aided detection

We propose an automatic feature generation by deep convolutional autoencoder (deep CAE) without lesion data. The main idea of the proposed method is based on anomaly detection. Deep CAE is trained by only normal volume patches. Trained deep CAE calculates low-dimensional features and reproduction error from 2.5 dimensional (2.5D) volume patch. The proposed method was evaluated experimentally with 200 chest CT cases. Experimental results showed that meaningful image features were extracted by the proposed method.

Paper No. 99

10

Computer-Aided Liver Cirrhosis Diagnosis via Automatic Liver Segmentation and Machine Learning Algorithm

Ting-Yu Su, Wei-Tse Yang, Tsu-Chi Cheng, Yi-Fei He, Yu Hua Fang

National Cheng Kung University


Liver segmentation, non-contrast phase of CT images, arterial phase of CT images, delay phase of CT images, portal venous phase of CT images, support vector machine algorithm

In this paper, a new computer-aided diagnosis system was proposed to automatically diagnose liver cirrhosis on four-phases CT images, which included non-contrast phase, arterial phase, delay phase and portal venous phase. And it was developed for classifying the normal liver and cirrhosis based on automatic liver segmentation method and classification step using machine learning algorithm. First, the gradient-inverse map of CT images would be calculated to derive the relative-smooth features in local area. Then we compared the centroid and area of each binary labeled groups through each slice to quantitatively extract the volume of interest (VOI) of liver automatically. In classification step, some first-order features and texture features would be calculated to describe the intensity representation of liver parenchyma. Also, some parameters would be used to quantify the distribution of intensity in VOI. By the way, we also quantified the shape of VOI and derived some structural features. Finally, a trained support vector machine (SVM) classifier was applied to classify the subjects into clinical stages of the liver cirrhosis.

Paper No. 107

11

Diagnosis of Lymph Node Metastasis in Non-Small-Cell Lung Cancer (NSCLC) Patient on F-18-FDG PET/CT

Tsu-Chi Cheng, Ting-Yu Su, Yi-Fei He, Wei-Tse Yang, Yu-Hua Fang


[email protected], [email protected], [email protected],[email protected], [email protected]

Non-small cell lung cancer, F-FDG-PET/CT, Lymph node, lymphatic drainage pathway, Metastasis, Texture analysis

Lung Cancer is a leading cause of death worldwide, and about 85% of lung cancer is non-small cell lung cancer (NSCLC). The staging of lymph nodes in NSCLC patients is extremely important because respective stages require different treatments. FDG-PET/CT is a gold standard for lymph node metastasis staging of NSCLC. However, the results of discriminating lymph node staging on F-18- FDG-PET/CT still needs improvement. In addition to the traditional image parameters of FDG-PET/CT such as standardized uptake value (SUV) and short axis diameter, there are many other parameters available from FDG-PET/CT images, for example, the lymphatic drainage pathway. Other than this, texture analysis which distinguishes subtle difference can also be a way to define lymph node staging. For the purpose of a better accuracy on lymph node metastasis diagnosis on NSCLC patient in FDG-PET/CT, this research developed a computer-aided diagnosis (CAD) system to improve the diagnostic efficiency, which achieved 76% accuracy.

Paper No. 109

12

Arteriovenous Classification Method using Convolutional Neural Network for Early Detection of Retinal Vascular Lesion

Hibiki Ikawa¹, Yuji Hatanaka¹, Wataru Sunayama¹, Kazunori Ogohara¹, Chisako Muramatsu², Hiroshi Fujita²

The University of Shiga Prefecture¹, Gifu University²

[email protected], [email protected], [email protected], [email protected], [email protected], [email protected]

retinal image, blood vessel classification, arteries and veins, deep convolutional neural networks

Early detection of hypertension is important because of that affects stroke and cardiovascular diseases. The hypertensive changes on the retina is diagnosed by measuring an arteriovenous ratio near the optic disc. Therefore, classification of arteries and veins is necessary for the ratio measurement, and previous studies classified them by using pixel-based features, which were pixel values, texture features, shape features etc. For simplification of classification process, a convolutional neural network (CNN) was applied in this study. For evaluation of classification process, CNN was tested by using centerlines extracted manually in this study. As a result, CNN classified perfectively blood vessels into arteries and veins in the diagnostic range of 10 retinal images when CNN was trained by 30 retinal images. This result may work as an important processing for abnormalities detection.

Paper No. 130

13

Investigation of the Effect of Image Resolution on Automatic Classification of Mammary Gland Density in Mammography Images using Deep Learning

Ami Oshima¹, Norimitsu Shinohara², Naoki Kamiya¹

Aichi Prefectural University¹, Gifu University of Medical Science²


Computer-aided Diagnosis, mammary gland density, deep learning, mammography

Mammary gland density is used as one of the measures in managing the risk of breast cancer. It can be divided into four categories. In addition, mammography is used for population-based breast cancer screening in Japan. However, mass and calcification are assumed to be hidden in the shadow of the mammary gland as displayed by the mammogram when patients showing heterogeneously dense or extremely dense in the mammary gland density category are scanned with mammography. Therefore, it is necessary to recommend an examination suitable for each category of mammary gland density. In one example, a doctor recommends ultrasonography in addition to mammography for patients with dense breasts. However, mammary gland density is distinguished visually using subjective judgment. Against such a background, we have worked on an automatic classification of mammary gland densities using a deep learning technique. Moreover, we investigated the effect of image resolution on the classification results in the automatic classification of mammary gland density with deep learning. The resolution was varied from 1/100 (474 x 354) to 1/3600 (79 x 59) using 1106 cases of resolution 4740 x 3540 (pixels) obtained with Fuji Computed Radiography (FCR) by Fujifilm Co. Ltd. As a result, the accuracy of automatic classification of mammary gland density exceeded 90% up to a resolution of 1/400 (237 x 177), and was 89% even at the lowest resolution of 1/3600 (79 x 59).

Paper No. 157

14

Quantitative Diagnosis of Rotator Cuff Ruptures Using Sonographic Pattern Recognition System

Chung-Chien Lee¹, Chung-Ming Lo¹, Ruey-Feng Chang²

New Taipei City Hospital¹, National Taiwan University²


Rotator cuff lesions, shoulder ultrasound, rupture, computer-aided diagnosis, texture

The lifetime prevalence of shoulder pain is nearly 70% and is mostly attributable to subacromial disorders. A rotator cuff rupture is the most severe form of subacromial disorders, and most occur in the supraspinatus. For clinical examination, shoulder ultrasound is recommended to detect supraspinatus rupturess. In this study, a computer-aided tear classification (CTC) system was developed to identify supraspinatus ruptures in ultrasound examinations and reduce inter-operator variability. The observed cases included 89 ultrasound images of supraspinatus tendinopathy and 102 of supraspinatus rupture from 136 patients. For each case, intensity and texture features were extracted from the entire lesion and combined in a binary logistic regression classifier for lesion classification. The proposed CTC system achieved an accuracy rate of 92% (176/191) and an area under receiver operating characteristic curve (Az) of 0.9694. Based on its diagnostic performance, the CTC system has promise for clinical use.

Paper No. 160

15

Automatic Hepatocellular Carcinoma Lesion Detection with Dynamic Enhancement Characteristic from Multi-phase CT Images

Gaeun Lee¹, June-Goo Lee²

Asan Medical Center


Liver, CAD, Hepatocellular Carcinoma, Multiphase CT

We propose the Computer-aided detection (CAD) scheme of hepatocellular carcinoma (HCC) lesion which utilizes the dynamic information from the multi-phase abdominal CT.

Paper No. 179

16

False Positive Reduction Scheme for Effective Lung Nodule Detection using Dual 3D Deep Neural Networks

Jeonghwan Gwak, Chang Min Park, Jin Mo Goo

Seoul National University


Lung nodule detection, False positive reduction, 3D CNN, Transfer learning, 3D U-Net

Pulmonary nodules are imaging findings and they are considered as abnormalities of the lung parenchyma. For diagnosis of lung cancers, detection and/or interpretation of lung nodules is the primary step. For lung nodule detection, the main difficulty is high false positive (FP) detection rate. To tackle this issue, we proposed a dual 3D deep network architecture which is consisted of (1) the segmentation network using Deep 3D U-Net for semantic segmentation and (2) the classification network using the 3D CNN model transfer learnt from a 2D pretrained model for effective classification of nodules/non-nodules. For an image slice, the proposed algorithm runs the two networks at the same time, then based on the empirical decision rule (segmentation network output is positive and the classification network sigmoid output > 0.5), lung nodules are detected. In the training phase, if the lung nodule is positive, the segmented lung nodule image output is used to fine-tune the classification network. In this way, the activations of lung nodules can be magnified in the classification network.We verified the proposed dual deep learning architecture could reduce FPs effectively and it is believed that the good performance is mainly derived from the fine-tuning using positive samples, which can be seen as hard positive mining process.The proposed dual 3D network architecture can be used for FP reduction of pulmonary lung nodule detection, and we have identified that hard positive mining is also important for the given task.

Paper No. 242

17

Deep Learning and Radiomics

Quantitative Analysis of Dopamine Transporter Imaging using Generating MR Image from Low Dose CT Image and Segmentation by Deep Learning

Shogo Yokoi¹, Takeshi Hara¹, Tetsuro Katafuchi², Masaki Matsusako³, Xiangrong Zhou¹, Hiroshi Fujita¹

Gifu University¹, Gifu University of Medical Science², St.Luke’s International Hospital³


Dopamine transporter imaging, MR image, SPECT image, Deep Learning

123I-FP-CIT binds to dopamine transporter (DAT) present in dopamine nerve of the nigrostriatal body in the brain, and the distribution of DAT can be visualized by SPECT image. Generally, the Tossici-Bolt method is used for analysis of this SPECT image. However, since the Tossici-Bolt method uses a fixed type of region of interest, there is a problem that it is susceptible to the influence of non-accumulated parts. MR images are effective for recognizing the shape of the striatal region. In this study, we use MR images generated by deep learning from low dose CT images taken with SPECT / CT devices. The purpose of this study is quantitative analysis with high repeatability using striatal region extracted from automatically generated MR image. First, an MR image is generated from a CT image by pix2pix. After that, a striatal region is extracted from the generated MR image by PSPNet. Quantitative analysis by Specific Binding Ratio (SBR) is performed using this region. For the experiments, IRB approved 60 clinical cases of SPECT/CT and MR images were used. A comparison experiment between SBR calculated by this method and by Tossici-Bolt method was carried out. As a result, better results than the Tossici-Bolt method were calculated in 12 cases. Therefore, generating MR images from low dose CT images and segmentation by deep learning may contribute to quantitative analysis with high reproducibility of DAT imaging.

Paper No. 11

18

Automated Segmentation Framework Of Lung Gross Tumor Volumes On 3D Planning CT Images Using Dense V-Net Deep Learning

Risa Nakano, Hidetaka Arimura, Mohammad Haekal, Saiji Ohga

Kyushu University


Deep learning, Segmentation, 3D-medical image, dense V-net

[Purpose] Gross tumor volume (GTV) regions of lung tumors should be determined with repeatability and reproducibility on planning computed tomography (CT) in radiation treatment planning for stereotactic body radiation therapy (SBRT) to reduce intra- and inter-observer variations of GTV regions. In this regard, automated segmentation methods are highly demanded in clinical practice. Therefore, we have attempted to develop an automated segmentation framework of the GTV regions on planning CT images using dense V-Net deep learning (DenseVDL), which is the volumetric, fully convolutional neural network with an objective function based on the Dice similarity coefficient (DSC).[Methods] 198 cases, who received SBRT, were selected for this study. 182 training data sets of the 3D planning CT images and contours of GTVs determined by radiation oncologists were fed into the DenseVDL network as input and teacher data of reference contours, respectively. In the training step, an augmentation technique (flip, rotation and scaling) was employed to increase the number of input data. The trained network was validated to estimate GTVs on planning CT images for sixteen cases including six solid, four ground glass opacity (GGO), and six part solid GGO tumor. In order to evaluate the GTV regions extracted by the DenseVDL network, DSC, which denotes the similarity between the reference region and the GTV region estimated using the DenseVDL network, was used in this study.[Results] The 2D-DSCs of solid, GGO, part solid GGO types were 0.78, 0.70, and 0.70, respectively, whereas 3D-DSCs of solid, GGO, part solid GGO types were 0.81, 0.75, and 0.73, respectively. The proposed framework achieved an average 2D-DSC of 0.73 and 3D-DSC of 0.76 for sixteen cases.[Conclusion] The proposed framework using the DenseVDL may be useful for assisting in radiation treatment planning for lung cancer.

Paper No. 21

19

Analysis of the Effects of Transfer Learning on Opacity Classification of Diffuse Lung Diseases Using Convolutional Neural Network

Ami Atsumo, Shingo Mabu, Shoji Kido, Yasushi Hirano, Takashi Kuremoto

Yamaguchi University


deep learning, convolutional neural network, transfer learning, diffuse lung diseases, computer-aided diagnosis

Research on Computer-Aided Diagnosis (CAD), which discriminates the presence or absence of diseases by machine learning and supports doctors’ diagnosis, has been actively conducted. However, training of machine learning requires many training data with annotations. Since the annotations are done by radiologists manually, annotating hundreds to thousands of images is very hard work. This study proposes classifiers using convolutional neural network (CNN) with transfer learning for efficient opacity classification of diffuse lung diseases, and the effects of transfer learning are analyzed under various conditions. In detail, classifiers with nine different conditions of transfer learning and without transfer learning are compared to show the best conditions.

Paper No. 22

20

Convolutional Neural Networks-Based Anti-Weapon Detection for Safe 3D Printing

Giao N. Pham¹, Suk-Hwan Lee², Ki-Ryong Kwon¹

Pukyong National University¹, Tongmyong University²


3D printing, 3D weapons, Shape distribution, Convolutional neural networks

With the development of 3D printing technology anybody can print weapons with home 3D printer. In this paper, we would like to present an anti-weapon detection algorithm for safe 3D printing using the convolutional neural networks (CNNs) to prevent the printing of weapons in 3D printing industry. The proposed algorithm is based on training the D2 shape distribution of 3D weapon models by the improved CNNs. The D2 shape distribution of 3D weapon model is calculated from geometric features and points on the surface of 3D triangle mesh in order to construct a D2 vector. The D2 vector is then trained by the improved CNNs. The training and testing results show that the proposed algorithm is more accuracy than the conventional works and previous methods.

Paper No. 45

21

Segmentation of lung region from chest X-ray images using U-net

Keigo Furutani


[email protected]

Deep learning, U-net, Convolution neural network, CXR

In recent years, many medical image analysis methods based on the Deep Learning techniques have been proposed. The Deep Learning techniques have been used for various medical applications such as organ segmentation and cancer detection. Segmentation of lung region from chest X-ray (CXR) images is also important task for computer-aided diagnosis (CAD). However many methods based on Deep Learning techniques for this purpose were proposed[1], the regions where the lung and the heart overlap have been excluded from the target to be extracted in spite of the importance for CAD. The aim of this paper is to extract whole lung regions from CRX images based on the U-net[2] based method. As widely known, the U-net which consisted of CNN shows its high performance for various applications. As the result of the experiment, the authors archive 0.82 in the average of the Dice coefficient.

Paper No. 53

22

A Computer-aided Diagnosis system based on Deep Learning for Brain Metastases in MRI

Young Jae Kim¹, Sung Jin Park¹, Leonard Sunwoo², Kwang Gi Kim¹

Gachon University College of Medicine¹, Seoul National University²


Brain metastasis, Deep Learning, AlexNet, VGGNet, Computer-aided Diagnosis

In this study, we aimed to improve the detection performance by applying the deep - learning technique to brain cancer metastatic nodule detection system based on image processing. In a total of 110 patients (584 nodules), we used 80 patients (450 nodules) for training data and 30 patients (114 nodules) for test data. The training data were trained under the conditions of 10 batch size, 200 epoch, 0.0001 learning rate using the 3D AlexNet and 3D VGGNet. As a result, the AlexNet showed an accuracy of 99.96%, and the VGGNet showed an accuracy of 99.92%. In conclusion, we have confirmed that the limitations of existing brain metastasis detection techniques can be overcome through deep learning techniques.

Paper No. 65

23

Radiomics-based malignancy prediction of parotid gland tumor

Hidemi Kamezawa¹, Hidetaka Arimura², Ryuji Yasumatsu², Shu Haseai², Kenta Ninomiya²

Teikyo University¹, Kyushu University²


Radiomics, Malignancy prediction, Parotid gland tumor, Machine learning, Classifier

We have investigated a feasible approach for malignancy prediction of parotid gland tumor (PGT). The tumor region in preoperative magnetic resonance (MR) images of 42 PGT patients were segmented. A total of 972 radiomic features were extracted from tumor regions in T1- and T2-weighted MR images. Radiomic signatures for malignancy prediction of PGTs were generated by using a least absolute shrinkage and selection operator (LASSO). Malignancy of PGTs were predicted by using random forest (RF) and k-nearest neighborhood (k-NN). The 5 features were selected by LASSO for the malignancy prediction of PGTs. The proposed approach was evaluated using the accuracy and the mean area under the receiver operating characteristic curve (AUC) based on a leave-one-out cross validation test. The accuracy and AUC of the malignancy prediction of PGTs were 73.8% and 0.88 for the RF and 88.1% and 0.95 for the k-NN. The k-NN demonstrated the higher accuracy in the malignancy prediction of PGTs. Our results suggested that the radiomics-based k-NN approach using preoperative MR images could be feasible to predict the malignancy of PGT.

Paper No. 87

24

Detection of pulmonary nodules on chest X-ray images using R-CNN

Risa Takemiya, Shoji Kido, Yasushi Hirano, Shingo Mabu



deep learning, region with convolutional neural network, convolutional neural network, selective search, pulmonary nodules, computer-aided diagnosis

Burdens of doctors for chest X-ray (CXR) examination have increased because number of X-ray images increases. Furthermore, since diagnosis is based on the experience and subjectivity of them, there is a possibility that a misdiagnosis may occur. Therefore, we performed Computer-Aided Diagnosis (CAD). In this study, we detected pulmonary nodules using R-CNN (Region with Convolutional Neural Network)[1] which is a kind of Deep Learning. First, we created CNN (Convolutional Neural Network) which classified data into classes of nodule opacities and non-nodule opacities. Next, we detected the object candidate regions from the chest X-ray images by Selective Search[2], and applied the CNN to the candidate regions to classify them and estimate the detailed position of the object. Thus, we propose a method to detect pulmonary nodules from the chest X-ray images.

Paper No. 92

25

Automated classification of histological subtypes of NSCLC using support vector machines with radiomic features

Masahiro Yamada¹, Hidetaka Arimura¹, Mazen Soufi², Kenta Ninomiya¹, Mitsutaka Nemoto¹, Kazuyuki Ushifusa¹

Kyushu University¹, Nara Institute of Science and Technology²


radiomics, LASSO, SVM, classification, NSCLC

Histological classification of non-small cell lung cancer (NSCLC) affects the decision making of treatment policies. However, histological subtypes identified from a single biopsy occasionally differ from those from actual surgical resections in NSCLC. For increasing the classification accuracy, we aim to develop an automated approach for classifying NSCLC into histological subtypes (adenocarcinoma (ADN) and squamous cell carcinoma (SCC)) using Gaussian, linear and polynomial support vector machines (SVMs) with radiomic features. Classification models of Gaussian, linear and polynomial SVMs constructed with radiomic features achieved the areas under the curve of 0.7542, 0.7522 and 0.7531, respectively. NSCLC could be classified into at least ADN and SCC using a Gaussian SVM with radiomic features.

Paper No. 98

26

Ground-glass Nodule Classification with Multiple 2.5-dimentional Deep Convolutional Neural Networks in Chest CT images

So Hyun Byun¹, Julip Jung¹, Helen Hong¹, Yong Sub Song², Hyungjin Kim², Chang Min Park²

Seoul Women’s University¹, Seoul National University²


chest CT image, ground-glass nodule, deep convolution neural network

The malignancy of ground-glass nodule differs depending on the presence and size of the solid component of the nodule. It is thus important to distinguish between pure GGN and part-solid GGN with variable-sized solid components. We propose a GGN classification framework based on multiple 2.5-dimensional deep CNNs with concatenated original input patches and nodule information augmented input patches in chest CT images. Experimental results showed an accuracy of 75.86%, and a 3.45% improvement compared to the 2.5D original input patches.

Paper No. 134

27

How Dependent are CT Radiomic Features on CT Scan Parameters

Hyeongmin Jin, Jong Hyo Kim



radiomics, computed tomography, feature variability, reconstruction kernel, lung cancer

Radiomics is attracting research interests for characterization of the tumor phenotype as well as for prediction of patient outcome. However, many radiomic features are known to be affected by a multitude of variability sources, such as CT acquisition parameters, which might lead to false discovery if unknowingly used. Therefore, in order to avoid such pitfalls, appropriate selection of robust features is an essential task in radiomic studies. We investigate the variability of CT imaging features which were previously reported as radiomic markers in non-small cell lung cancer (NSCLC). We collected 13 CT scans with 2 reconstruction kernels (Standard, Sharp) in the National Lung Screening Trail database. We extracted 47 radiomic features on standardized phantom images scanned with various scan conditions, and homogeneous region and lung cancer in patient images. Feature variability index was measured to evaluate the feature robustness depending on the scan parameters. The proportion of features less effect on kernel was observed to only 32%. Our study revealed a high variability of CT image features depending on technical parameters. These characteristics should be considered in the feature extraction procedure when different protocols are used in the patient dataset. Use of the same CT protocol is preferred. Otherwise, the application of kernel normalization techniques is necessary for the radiomic study.

Paper No. 186

28

Simulated Nodule Generation for Improving Computerized Classification of Lung Nodules in CT

Chisako Murmatsu¹, Takuma Goto¹, Mizuho Nishio², Masahiro Yakami², Hiroshi Fujita¹

Gifu University¹, Kyoto University²


convolutional neural network, generative adversarial network, lung nodules, classification, lung CT

Computerized classification of lung nodules in CT using deep learning techniques has been introduced and shown some promising results. Improvement in the number and quality of training data is one way to achieve higher classification performance; however, collection of clinical data is a demanding task. The purpose of this study is to generate nodule samples using generative adversarial networks (GAN) for possible improvement of computerized schemes for classification of lung nodules in CT. The simulated benign and malignant nodules are generated using GAN from noise input data and blob images. The preliminary result indicate a potential utility of simulated samples over the general augmented samples in improving classification performance.

Paper No. 192

29

Combined Low-dose Simulation and Deep Learning for CT Denoising: Application in Ultra-low-dose Chest CT

Chulkyun Ahn, Jong Hyo Kim



deep learning, denoising, ultra-low-dose chest CT, convolutional neural network, CNN

In this study, we present a deep learning approach for denoising of ultra-low-dose chest CT by combining a low-dose simulation and convolutional neural network (CNN). A total of 15,419 anonymized regular-dose chest CT images from 64 CT scans of RIDER lung CT collection were used for training of the CNN. The training CT images were fed into the low-dose simulation tool to generate a paired set of simulated low-dose CT and synthetic low-dose noise. A modified U-net model with 4x4 kernel size and five layers was trained with these paired datasets to predict the low-dose noise from the given low-dose CT image. Independent 10 ultra-low-dose chest CT scans at 120 kVp and 5 mAs were used for testing the denoising performance of the trained U-net. Denoised CT images were obtained by subtracting the predicted noise image from ultra-low-dose chest CT images. We evaluated the image quality by measuring noise standard deviation of soft tissue and with visual assessment of small vessels, bronchial wall, and lung fissure. For comparison, the image quality was assessed on FBP, VEO, and deep learning-denoised FBP images. Image noise of soft tissue was 101±28HU, 20±5HU, 28±10HU in FBP, VEO, deep learning-denoised images. The visual assessment made with 4 points scale was 1.0, 3.4 and 4.0 in FBP, VEO, and deep learning-denoised FBP images.

Paper No. 199

30

Image Acquisition and Reconstruction

Extracting Multi-View Images from Multi-Focused Plenoptic Camera

Shu Fujita, Sho Mikawa, Mehrdad Panahpourtehrani, Keita Takahashi, Toshiaki Fujii

Nagoya University


Light Field, Plenoptic Camera, Focused Plenoptic Camera, Raytrix, Multi-View Image Rendering

A multi-focused plenoptic camera is a powerful device that can capture the light field (LF), which is interpreted as a set of dense multi-view images. The camera has potential ability such that we can obtain LFs having high spatial/view resolutions and deep depth-of-field. To extract multi-view images, we need a sophisticated rendering process due to its complicated optical system. However, there are few studies on this. Furthermore, the only available rendering software to the best of our knowledge has not worked well in several camera configurations. We therefore introduce an improved rendering method and release the software that uses the method. Our software can produce better LFs than the previous one and work in various camera configurations.

Paper No. 93

31

Improvement of optical setup for microcirculation imaging and flow analysis in septic shock rats

Mami Kawasaki, Ryohei Hashimoto, Kazuya Nakano, Takashi Ohnishi, Masashi Sekine, Eizo Watanabe, Shigeto Oda, Hideaki Haneishi

Chiba University



sidestream dark-filed (SDF) imaging, septic shock, non-contact setup

A quantitative index to evaluate septic shock progression non-invasively and quickly is expected. We constructed a non-contact setup for imaging microcirculation of septic rats. Furthermore, using the obtained motion pictures, we estimated the flow of red blood cells (RBCs) and investigated the relationship between the flow of RBCs and septic shock.

Paper No. 108

32

Non-iterative Method for Metal Artifact Reduction by using a Linearized Beam-hardening Correction Model for Polychromatic X-ray CT

Jin Hur¹, Dongjoon Kim¹, Yeong-Gil Shin¹, Sangchul Lee²

Seoul National University¹, University of California, Los Angeles (UCLA)²


metal artifact reduction (MAR), beam-hardening correction, polychromatic X-ray

The beam-hardening effect is one of the most important factors of metal artifact that degrades CT image quality. In the polychromatic X-ray, this occurs noticeably when scanning metallic materials with large changes in energy-dependent attenuation coefficient. This violates the assumption of a CT reconstruction based on a fixed attenuation coefficient in a monochromatic X-ray, which leads to beam-hardening artifacts such as streaking and cupping shapes. Numerous studies have been researched to reduce the beam-hardening artifacts. Most of the methods need the optimization based on iterative reconstruction, which causes a time-consuming problem. This study aims at an efficient methodology in terms of performance time while providing acceptable correction of beam-hardening artifacts. For this, the attenuation coefficient error due to beam-hardening is modeled with respect to the length of the X-ray passing through the metallic material. And the model is approximated by a linear combination of four basis functions determined by the length. The linearity is also preserved in the reconstruction image, so that the coefficient of each basis function can be obtained by solving the minimization problem of the variance of the homogeneous metallic region in the image. For the evaluation, a phantom including three titanium rods was scanned by a cone-beam CT system (Ray, South Korea) and the images were reconstructed by the standard Feldkamp algorithm. The results showed that the proposed method is superior in terms of speed while delivering acceptable beam-hardening correction compared to recent methods. The proposed model will be effective for the applications where processing speed is important for the beam-hardening correction.

Paper No. 194

33

Adjustment of magnification factor on dental panoramic tomosynthesis

Taejin Kwon, Gyuseong Cho, Seungryong Cho

Korea Advanced Institute of Science and Technology


Dental, panoramic tomosynthesis, magnification, slice image

The purpose of this study is to accurately correct the distorted magnification of dental panoramic tomosynthesis images acquired using the shift and added method. Dental panoramic imaging is widely used in many clinical applications because it can provide a wide range of anatomical visualizations with jaws, teeth, etc. on a single display. And nowadays the digital panoramic systems can produce a number of slice images depending on focal depths along the dental arc using a lot of shift-and-add scheme, it is possible to derive high quality dental panoramic images even if the patient dental anatomy or position are off the ideal geometry. Since the in-focus slice may have different depth profiles along the dental arc, we must choose profer depth that best represent the dental information along the arc. So, through auto focusing process, we compose the best focused slice image to create a single auto focused panoramic image. However, in the process of creating such an auto-focused image, if the magnification between each slice is not correct, it may cause errors in diagnosis and implant production. In this study, to solve the problem of magnification difference between slice images as mentioned above, the magnification of each slice image was adjusted exactly after the auto focusing operation. After that, we applied this to create a single panoramic image considering the actual magnification.magnification difference between slice images as mentioned above, the magnification of each slice image was adjusted exactly after the auto focusing operation. After that, we applied this to create a single panoramic image considering the actual magnification.

Paper No. 224

34

Image Processing and Image Quality

Comparison study of image quality between filtered back projection and iterative reconstruction algorithm for dose reduction in chest CT

Junyoung Son, Donghoon Lee, Pil Hyun Jeon, Sunghoon Choi, Hyemi Kim, Hee Joung Kim

Yonsei University


Low dose CT, Iterative reconstruction, Lung nodule detection, Noise power spectrum, Contrast to noise ratio

Many studies have shown that iterative reconstruction (IR) algorithm is possible to lower the tube current and voltage in CT imaging without a major loss of image quality. The aim of this study was to investigate the image quality of low dose CT images obtained with IR algorithm through noise power spectrum (NPS) and contrast to noise ratio (CNR). Images were reconstructed with filtered back projection (FBP) and iDose hybrid IR algorithm (Philips Healthcare, Cleveland, OH). The three levels of iDose (iDose1, 3 and 5) also used, which could define the strength of the IR in reducing image quantum noise. CTDIvol for routine protocol and low dose protocol were 5.2 mGy and 2 mGy, respectively. The results showed that when the level of iDose was 5, image quality was improved than that of iDose1 and iDose3. When the same low-dose protocol is used, the IR algorithm provided improved imaging performance compared with the FBP algorithm. It also demonstrated that IR algorithm provides potential for maintaining or improving image quality with much less radiation dose than FBP algorithm with routine dose.

Paper No. 19

35

Automatic Teeth Separation in 3D Maxillofacial CT Images through Searching Teeth Separation Planes

Soyoung Lee, Min Jin Lee, Helen Hong

Seoul Women’s University


maxillofacial CBCT images, panorama image, teeth separation

We propose an automatic separation method of teeth through searching optimal teeth separation planes in 3D maxillofacial CBCT images. To observe the overall structure of the individual teeth and reduce the search range for teeth separation, 2D panorama image of the teeth is reconstructed. Then optimal teeth separation lines and planes are searched using intensity-based cost function in 2D panorama image and 3D CBCT images, respectively. Experimental results show that the teeth separation planes in 3D maxillofacial CBCT images are accurately localized without passing through the teeth.

Paper No. 68

36

Improvement of Face Image Super-Resolution by High-Precision Skin Color Detection

Keigo Kano¹, Tomio Goto¹, Satoshi Hirano¹, Son Phumg²

Nagoya Institution of Technology¹, University of Wollongong²


super-resolution, facial correction, skin detection

When super-resolution processing is performed on images such as scenery, edges are emphasized to obtain clear images. However, when super-resolution processing is applied to facial images, the wrinkle and stains of the skin are emphasized, so super-resolution processing on the skin part is not suitable. Therefore, in the previous study, we proposed a method to perform facial correction on skin part. However, we confirmed that there was a problem that the image quality deteriorated according to the skin color detection accuracy. Therefore, in this paper, we study the skin color detection method and try to improve the image quality.

Paper No. 110

37

Artifact reduction using segmentation constrained RPCA for CT

Yejin Kim, Dain Choi, Sunho Lim, Seungryong Cho



computed tomography, RPCA, artifact reduction

In CT images, the quality of the image is degraded by the artifacts. The artifacts are due to various reasons including Poisson noise, sparse data sampling, and other hardware issues. In this investigation, we aim to separate the artifacts from the CT image by using RPCA and thus improve the reconstructed CT images.Conventionally, RPCA method separates the foreground and the background. Often, the background is assumed as static or quasi-static. When applied to CT images, the artifacts are considered as quasi-static background whereas the anatomical structures are considered foreground. Thus, RPCA is performed to segment the foreground from the background. Finally, different post-reconstruction denoising parameters are applied to each foreground and background to effectively remove the artifact.

Paper No. 162

38

A Hardware Implementation of Craik-O’Brien Effect-Based Contrast Improvement for Dichromats

Tomohiro Ono¹, Ryosuke Kubota², Noriaki Suetake³, Hakaru Tamukoh¹

Kyushu Institute of Technology¹, National Institute of Technology, Ube College², Yamaguchi University³


Dichromats, Image processing, Craik-O’Brien effect, T-model filter, Digital hardware, Field-Programmable Gate Array

In this paper, we design a digital hardware circuit for a Field-Programmable Gate Array (FPGA) to realize the contrast improvement algorithm for dichromats. The proposed method employs the Craik-O’Brien (C-O) effect. The C-O effect is an optical illusion effect in which subjective contrast creates from contour information. In the proposed method, the contrast modification is only carried out around the contours of objects in order to realize the C-O effect for dichromats. To extract the contours information of objects, a T-model filter which requires only a one-line buffer is introduced. The proposed method can realize the C-O effect without using dividers and multipliers. Therefore, it is easy to implement in the FPGA. Through some experiments, the effectiveness and the validity of the proposed method are verified.

Paper No. 231

39

Image Segmentation and Registration

Construction of multimodal 3D model of pancreatic cancer tumor

Yushi Goto¹, Hidekata Hontani¹, Tatsuya Yokota¹, Mauricio Kugler¹, Chika Iwamoto², Kenoki Ohuchida², Makoto Hashizume²

Nagoya Institute of Technology¹, Kyushu University², National Chiao Tung University¹, Taipei National University of Arts²

[email protected], [email protected], [email protected], [email protected], [email protected],


multimodal, multiresolution, registration

In this study, we propose a method of registration between an MRI image and a set of pathological microscope images of a pancreatic cancer tumor of a KPC mouse. MRI images can be captured non-invasively but their resolution is low. On the other hand, microscopic images can be obtained only invasively although the resolution is high. Registration of these images is needed for realizing a multi-resolution model representing the correlation between the MRI images and the corresponding pathological images. In order to register the 2D pathological microscope images to the 3D MRI image, the proposed method first reconstructs a 3D pathological microscope image and then register between the 3D pathological microscopic image and the tumor region in the MRI image. The 3D pathological microscope image is reconstructed from a spatial series of 2D microscope images. For the reconstruction of an appropriate 3D image, the microscope images should be non-rigidly registered together. We employed a non-rigid registration method that deforms every image so that the spatial patterns in the resultant image have smooth structures. After the 3D image reconstruction, the proposed method then registers between the MRI image and the reconstructed 3D pathological image by means of a mutual-information based non-rigid registration. Down sampling the 3D microscope image so that the both images have same spatial resolution, the proposed method deforms the microscope images to the MRI image by maximizing their mutual information. We report the result of the registration and of the description of the microstructures in the reconstructed 3D pathological image.

Paper No. 49

40

Automatic Segmentation of Meniscus using Locally-weighted Voting based on Multi-atlas and Edge Classification in Knee MR Images

Soonbeen Kim¹, Hyeonjin Kim¹, Helen Hong¹, Joon Ho Wang²

Seoul Women’s University¹, Samsung Medical Center²


Knee MR image, meniscus segmentation, multi-atlas segmentation, locally-weighted voting, support vector machine

We propose an automatic segmentation of meniscus from knee MR images using multi-atlas segmentation and patch-based edge classification. To robustly segment the meniscus with large shape variations, meniscus is segmented by multi-atlas-based locally-weighted voting (LWV) considering the weights of shape and intensity. To remove leakage to the collateral ligaments, meniscus is refined using patch-based edge classification considering the weights of shape and distance. Experimental result shows that the accuracy of segmentation results using the proposed method is improved compared to LWV based on multi-atlas.

Paper No. 103

41

Automated Segmentation of Hip and Thigh Muscles in Metal Artifact Contaminated CT using CNN

Mitsuki Sakamoto¹, Yuta Hiasa¹, Yoshito Otake¹, Masaki Takao², Yuki Suzuki¹, Nobuhiko Sugano², Yoshinobu Sato¹

Nara Institute of Science and Technology¹, Osaka University²


segmentation, deep learning, metal artifact reduction

In total hip arthroplasty, analysis of postoperative images is important to evaluate surgical outcome and create appropriate rehabilitation plans. Since CT is most prevalent modality in orthopedic surgery, we aimed at the analysis of CT image. Specifically, we focused on the measurement of muscle volume, which is relevant to patients’ activity. The challenge we addressed in this work is the metal artifact in postoperative CT caused by the metallic implant, which reduces the accuracy of segmentation especially in the regions next to the implant. Our goal was to develop an automated muscle segmentation in the postoperative CT images. In this paper, we propose a method that combines NMAR[1] (Normalized Metal Artifact Reduction), which is the state-of-the-art metal artifact reduction method, and a CNN-based segmentation using the U-Net[2] architecture. We conducted experiments using simulated images and real images of lower extremity to evaluate the segmentation accuracy of 19 muscles from CT images that are contaminated with metallic artifact. The training dataset we used is 20 CTs that were manually traced by an expert surgeon. In simulation study, the proposed method, increased the dice coefficient from 0.73 to 0.77 and improved the average surface distance from 5.61 mm to 4.07 mm. Qualitative evaluation of the real image experiment showed improvement of segmentation especially in the vicinity of the implant. Our future work includes the end-to-end convolutional neural network for metal artifact reduction and musculoskeltal segmentation. We also plan to establish a ground truth dataset for quantitative evaluation in real images by performing non-rigid registration between the postoperative CT and the preoperative CT of the same patient that is not contaminated by the artifact.

Paper No. 177

42

Defect Detection using Trainable Segmentation

Bisma Mutiargo, Amin Garbout, Andrew Alexander Malcolm

Advanced Remanufacturing & Technology Centre


X-Ray Computed Tomography (CT), Non-Destructive Evaluation (NDE), Trainable Segmentation, Defect, Porosity, Crack, Random Forest

Additive manufacturing processes create the opportunity for freedom of design as it allows parts to be manufactured where conventional methods would fail. Printing methods such as selective laser sintering (SLM), electron beam melting (EBM) and others produce micro-porosity in the range of 10-15 μm in size (W.E Frazier, 2014). These micro defects induced by the different processes can have a major impact on the functionality and lifetime of the components. X-ray Computed Tomography (XCT) is an image acquisition technique that allows a complete three-dimensional capture of an object including its internal features and structures. This technology is an established method of non-destructive evaluation to detect the presence of cracks and large porosity in additively manufactured components.However, micro - defects and cracks are known to be difficult to detect. The distinction between X-ray artefacts due to scattering and beam hardening make it impossible for simple intensity-based image processing algorithms such as thresholding to reliably detect and quantify the presence of a defect especially as the defect size approaches the imaging resolution.Here, a new approach to improve micro-porosity and crack detection through the use of random forest classifier was developed. This new method was optimized to detect defects that are very close to the voxel size. To achieve this, trainable segmentation with random forest classifier was used with five pre-defined classes (Pore, Cracks, Material, air and edge). Random forest classifier is an ensemble learning method for image classification. It creates a set of decision trees from randomly selected subset of training set. It then compiles the probability aggregate of each layer of the node within the tree to make a decision of the final outcome. A reference artefact was designed with internal micro-holes purposefully cut inside the material to simulate the presence of cracks and micro-defects. Using a computer aided design (CAD) model as an input to the aRTist simulation software - a software developed by Federal Institute for Materials Research and Testing (BAM). virtual XCT data was obtained to perform supervised training. Subsequently, the trained system was used to evaluate real CT data to test the detection accuracy of the trained machine learning model. Verification of the performance of the approach is presented based on comparison between

Paper No. 245

43

simulation, model and experimental data.Results will be presented which show that the accuracy and reliability of the defect detection is significantly enhanced compared to commercial CT analysis tools and that it compares well with conventional destructive sectioning followed by 2D image analysis while generating better representation of the whole component volume.

Others

Study on Visual perception for Beta Movement of annular ring under peripheral vision

Kohei Kajiwara, Hikaru Shibata, Hiroto Inoue, Nobuji Tetsutani

Tokyo Denki University


beta motion, peripheral vision, annular ring

The apparent movement is the illusion that an object that is physically not moving appears to move. As one of the phenomena, the beta motion is a visual phenomenon that a light spot appears to be moving as the aligned light spots blink in order. The authors have found a phenomenon in which the blinking speed appears to be faster when the beta motion is viewed in peripheral view. In this paper, we focused on the beta motion of the annular ring and reveal speeding movement and shape change in peripheral vision.

Paper No. 8

45

VR Bowling for Muscular Rehabilitation

Yiliang Sui, Feng Lin, Hock Soon Seah

Nanyang Technological University


medical imaging, virtual reality, rehabilitation

The goal of this VR system is to simulate a bowling game for adaption in muscular rehabilitation training. The virtual environment allows the user to pick up a bowling ball and hit the pins, followed by an update and display of the score they gained; and the players can alternate between each other to have a competition. Implemented on the Unity engine and SteamVR, the VR Toolkit is employed in modeling and script development. Technical innovations are made in generation of the grabbing and releasing controllers with adjustable colliders, and the respawn detector triggered when the ball hits the back of the bowling alley in the game. We present the specific tasks of muscular rehabilitation, conceptualization of VR techniques and the detailed implementation of the system.

Paper No. 16

46

VR Puzzle Room for Cognitive Rehabilitation

Jian Bin Khiew, Peng Hian Tan, Feng Lin, Hock Soon Seah



medical imaging, virtual reality, rehabilitation

The goal of this VR system is to simulate a puzzle / challenge for adaption in cognitive rehabilitation training. Development of the VR system is inspired by a 1st person puzzle-platform game where the player must navigate and complete through a series of puzzle rooms with each room being more difficult than the last. The unique features in this work include the use of ‘portals’and ‘portal gun’ The portal gun allows the player to shoot two separate ‘portals’on walls that will allow anything to be teleported from one portal to the other. Implemented on the Unity engine and SteamVR, the VR Toolkit is employed in modeling and script development. Technical innovations are made in modeling the animated and self-collision detectable spider; upon being collided with a weapon (bullet or blade) it uses a special dissolve shader to give the effect of disappearing gradually from the game. We present the specific tasks of cognitive rehabilitation, conceptualization of VR techniques and the detailed implementation of the system.

Paper No. 17

47

Ultrasound-based Shear-wave Speed Measurement on a Highly Viscous Embedded Phantom

Masashi Usumura¹, Mikio Suga¹, Riwa Kishimoto², Takayuki Obata²

Chiba university¹, National Institute of Radiological Sciences, QST²


ultrasound, elastography, phantom

We have developed highly viscous phantoms that include ultrasound scatterers in polyacrylamide gels, to be measurable by both magnetic resonance elastography and ultrasound elastography. The purpose of this study is to evaluate whether US-based shear-wave elastography can measure elasticity accurately in a highly viscous embedded phantom. Shear-wave speeds in the embedded part were equivalent to the reference values. The size of the embedded part on velocity-mode was demonstrated to be larger than the part on B-mode image. This phantom has the potential as a quality control phantom to mimic living tissues.

Paper No. 32

48

Effects of Ginger Aroma Under Stress Condition: Perspectives from Biometric Informations

Takuma Kitamoto¹, Takahiro Kosuge¹, Yuka Suzuki², Teruko Ohba³, Eiichi Endo³, Kazuya Miyagawa⁴, Motonobu Hidaka⁵, Hiroshi Hasegawa¹, Masao Kasuga⁵

Utsunomiya University¹, Artos Co., Ltd.², Endo Foods Co., Ltd.³, School of Pharmacy, International University of Health and Welfare⁴, Sakushin Gakuin University⁵



Ginger aroma, salivary amylase activity, heart rate, stress, sedative effect

This paper investigates physiological effects of ginger aroma using salivary amylase activity (SAA) and heart rate under stress condition. We recruited 50 subjects, and the subjects were divided into 2 groups: scented-environment group and unscented-environment group. We measured SAA and heart rate while subjects performed a calculation task for 15 min in this experiment. To clarify influence of individual preferences, we allocated the subjects in scented-environment group to 2 groups-favorite aroma (FA) group and unfavorite aroma (UA) group. As a result, we found that heart rates of FA group were lower than UA and unscented-environment group (p < .01). It was suggested that ginger aroma has a sedative effect for the subjects who prefer the aroma.

Paper No. 43

49

Multi-Scale Speed of Sound Analysis by Comparing of Histological Image and Ultrasonic Microscopic Images at Multiple Frequencies

Takuya Ogawa, Kenji Yoshida, Shu Kashio, Takashi Ohnishi, Hideaki Haneishi, Tadashi Yamaguchi

Chiba University


speed of sound, ultrasound, multi-scale, multi-frequency, registlation

In this study, we report the examination results of speed of sound of sliced rat organs analyzed with multi-frequency ultrasound (80 and 250 MHz) from the acquiring radiofrequency (RF) echo signals observed by our self-made scanning acoustic microscopy (SAM) system. In order to evaluate the speed of sound of each tissue by resolution ability of cell organelle unit, histological image and sound speed images were registered. The frequency dependence of speed of sound was evaluated by analysis method involving filtering considering spatial resolution at each frequency.

Paper No. 163

50

A 3D Printing-based Realistic Anthropomorphic Dental Phantom and its Imaging Evaluation

Do Lee, Jong Kim



Anthropomorphic, Image evaluation, 3D printer, Dental phantom

In this study, we present a 3D-printing based realistic anthropomorphic dental phantom and its imaging evaluation.A real skull phantom was scanned with MDCT at high resolution, and then image segmentation and 3D model were carried out for bones, teeth, and soft tissue. Followed by was 3D printing of bones and teeth with gypsum, with additional 3 teeth being printed with metal separately. For soft tissue, a negative model was first printed with PMMA, and then silicon gel was casted into the negative model with printed bones and teeth set in place. The created phantom was scanned with by using an MDCT and a dental CT scanner for image quality evaluation. Mean HU of bone was comparable between 3D printed and real skull phantoms (1860 vs 1730), and mean HU of soft tissue was 40 in 3D printed phantom. The metal artifacts from metal printed teeth was rated as realistically mimicking the real crown teeth.Our study demonstrated the feasibility of making 3D printing-based making realistic anthropomorphic phantoms which can be used in various dental imaging studies.

Paper No. 164

51

Development of a Patient-specific Plate Design Assistant Software for Cranioplasty

Taeseok Lee, Hyun Chul Cho, Tae-Geun Son, Youngjun Kim

Korea Institute of Science and Technology


cranioplasty, skull fracture, patient-specific implant, 3D CAD software

Craniofacial patients suffer from diverse malformations such as cranial asymmetry, syndrome disorders, and several types of skull fractures by external impact. A patient who experiences a cranial fracture needs proper treatments. Depending on the severity of skull fracture, and most cases of the compound skull fracture, the treatment would be mostly a surgical operation. Recently, the transplant operation of customized implants has been developed owing to the spread of biomedical 3D printing technology. To improve the patient-specific implant design process, we developed an assistant software that can help to design the patient-customized skull implant plate for 3D printing. Compared to existing CAD program which requires users to draw complicate contours of the implant plate model, our software minimized user input. With our automated suggestion algorithm, the user can designate a skull region where the implant plate should be placed by simple click and drag of control points. Also, the proposed method makes surface modification simple, by providing NURBS surface fitting. Therefore, the users can design and modify the detailed surface of the 3D implant plate model as they desired for cranioplasty.

Paper No. 206

52

Special Session on Computational Anatomy

Registration between Histopathological Images with Different Stains and an MRI Image of Pancreatic Cancer Tumor

Hidekata Hontani¹, Yushi Goto¹, Yuki Tamura¹, Tomoshige Shimomura¹, Naoki Kawamura¹, Hirokazu Kobayashi¹, Mauricio Kugler¹, Tatsuya Yokota¹, Chika Iwamoto², Kenoki Ohuchida²,

Makoto Hashizume², Takahiro Katagiri³, Tomonari Sei⁴, Akinobu Shimizu⁵

Nagoya Institute of Technology¹, Kyushu University², Nagoya University³, The University of Tokyo⁴, Tokyo Agriculture and Technology University⁵

[email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected],


Non-rigid registration, Pathology images, Robust template matching

This presentation introduces a fully non-rigid image registration method for 3D image reconstruction which considers the spatial continuity and smoothness of each constituent part of the microstructures in the tissue. Corresponding points (landmarks) are detected along anatomical structures using template matching based on cross-correlation, forming jagged shape trajectories that traverse several slices. The registration process smooths out these trajectories by minimizing their total variation and nonr-rigidly deforms the images accordingly.

Paper No. 248

53

Unsupervised and Semi-Supervised Learning for Efficient Opacity Annotation of Diffuse Lung Diseases

Shingo Mabu, Shoji Kido, Yasushi Hirano, Takashi Kuremoto



deep learning, computer-aided diagnosis, unsupervised learning, semi-supervised learning, diffuse lung diseases

Research on computer-aided diagnosis (CAD) for medical images using machine learning has been actively conducted. However, machine learning, especially deep learning, requires a large number of training data with annotations. Deep learning often requires thousands of training data, but it is tough work for radiologists to give normal and abnormal labels to many images. In this research, aiming the efficient opacity annotation of diffuse lung diseases, unsupervised and semi-supervised opacity annotation algorithms are introduced. Unsupervised learning makes clusters of opacities based on the features of the images without using any opacity information, and semi-supervised learning efficiently uses the small number of training data with annotation for its training. The performance evaluation is carried out by the classification of six kinds of opacities of diffuse lung diseases: consolidation, ground-glass opacity, honeycombing, emphysema, nodular and normal, and the effectiveness of the methods is clarified.

Paper No. 249

54

Multiscale and Functional Modeling of Musculoskeletal System for Diagnosis, Surgical Planning and Prognostic Assessment in Orthopedic Surgery

Yoshito Otake¹, Yuta Hiasa¹, Masaki Takao², Norio Fukuda¹, Nobuhiko Sugano², Yoshinobu Sato¹

Nara Institute of Science and Technology¹, Osaka University²


Muscle modeling, Orthopedic surgery, Segmentation, Muscle fiber arrangement

We aim at the patient-specific modeling of musculoskeletal systems of lower extremity for applications in orthopedic surgery. Our method allows to model the shape of each muscle, extracted from patient-specific CT or MRI using a deep neural network, as well as its internal muscle fiber arrangements, extracted from the texture pattern inside the muscle using registration of a high-fidelity model derived from cadaver data. We combine the 3D anatomical model with the x-ray projection images acquired in various functional positions using a 2D-3D registration algorithm in order to obtain patient-specific functional model.In this presentation, we demonstrate the three key components in our system, semantic segmentation of muscles in a lower extremity CT, muscle fiber arrangement estimation, and an automated 2D-3D registration.

Paper No. 250

55

Spatiotemporal Statistical Models of a Human Embryo

Atsushi Saito¹, Masashi Kishimoto¹, Kazuki Kasahara¹, Masaki Tsujikawa¹, Tetsuya Takakuwa², Shigehito Yamada², Hiroshi Matsuzoe³, Hidekata Hontani³, Akinobu Shimizu¹

Tokyo University of Agriculture and Technology¹, Kyoto University², Nagoya Institute of Technology³

[email protected], [email protected], [email protected], [email protected], [email protected], s


statistical model, human embryo, surface, landmark

This paper presents spatiotemporal statistical models of organ surfaces during human embryonic development, in which size, shape and topology of organs are dynamically changed. A two step modeling scheme was employed; 1) statistical modeling of each temporal stage of an embryo and 2) interpolation of models between neighboring temporal stages. This paper includes optimization of interpolation techniques and the number of dimensions of a feature space for modeling. A novel method for modeling of nested and neighboring shapes, such as brain and ventricular surfaces, is also presented.

Paper No. 253

56

Automated detection of abnormal tumors on FDG-PET/CT images based on anatomical standardization using organ segmentations by deep learning

Takeshi Hara¹, Manami Haga¹, Xiangrong Zhou¹, Masaki Matsusako², Hiroshi Fujita¹

Gifu University¹, St. Luke’s International Hospital²


FDG-PET/CT, CAD, Tumors, Quantitative image analysis

Purpose: Segmentations of tumors on FDG-PET/CT images are very important to evaluate the response of chemotherapies or other treatment. The purpose of this study was to develop a new scheme to evaluate the response automatically based on our developed anatomical standardization technique for FDG-PET/CT images in the torso regions. Methods: Tumor regions were detected and segmented automatically from PET images. Our automated anatomical standardization process was applied to the CT images to determine the regions. Based on the standardization process, accumulations of FDG were standardized pixel-by-pixel. The standardization process requires shapes of organs as segmentation results on CT images from PET/CT devices. A deep learning technique was employed to segment the organ shape. After the shapes were segmented from CT images, the shape information was registered to the PET images. The PET images were deformed to fit a normal model to compare activities of glucose with normal cases pixel-by-pixel. Normal cases were collected from examination programs for healthy people, and were deformed to fit a standard Japanese physique based on the results of organ segmentation and body shape determination. A normal model was created by the anatomical standardization process. The model can represent distributions of normal accumulations pixel-by-pixel. By comparing patient cases with the normal model, abnormal regions were extracted on PET images. Results: Twenty-one cases including 25 obvious and 10 suspected tumors were used. All abnormal cases were verified by a radiologist. The true-positive fraction (77%) in the surface method was better than that (69%) in the bounding box one, with almost the same number of false-positives per case (1.10 in the surface and 1.05 in the bounding box methods). MV, TLG, and the effective dose were automatically measured by using the automated detection methods. Conclusions: Quantitative values from the analysis will be helpful indices for evaluations of treatment responses on FDG-PET/CT images.

Paper No. 257

57

Special Session on Deep Learning in Medical Imaging

Pulmonary nodule detection based on low-dose CT images

Huiqin Jiang¹, Ling Ma¹, Jianbo Gao¹, Hiroshi Fujita²

Zhengzhou University¹, Gifu University/Zhengzhou University²


Deep learning, Low dose CT, Pulmonary nodule detection, Faster R-CNN, Deconvolution

In this paper, we propose an improved Faster R-CNN algorithm for detecting pulmonary nodules based on low-dose CT images. We introduce the deconvolution structure at the last layer of vgg16 to recover more fine-grained features, and design the anchor frame which is more suitable for the size of pulmonary nodules. The experimental results show that the proposed technique can more accurately detect lung nodules and has certain clinical significance for early screening of lung cancer.

Paper No. 115

58

Reconstruction of the Spine Structure with Biplanar X-ray Images Using the Generative Adversarial Network

Chih-Chia Chen, Ting-Yu Su, Wei-Tse Yang, Tsu-Chi Cheng, Yi-Fei He, Yu-Hua Fang

National Cheg Kung University


musculoskeletal-related disorder, EOS X-ray imaging system, anteroposterior view & lateral view, GAN

In this study, a new computer-aided system was proposed to automatically reconstruct the spine model. The biplanar EOS X-ray imaging was adopted as the scanning technology, which is capable of a simultaneous capture of biplanar X-ray images by slot scanning of the whole body using ultra-low radiation doses. High quality and high contrast anteroposterior (AP) and lateral (LAT) X-ray images will be acquire during scanning period and these two radiographs enables a precise three-dimensional reconstruction of vertebrae, pelvis and other parts of the skeletal system. To overcome the time-consuming issue of spine reconstruction using EOS system, a generative adversarial network (GAN) was applied to reconstruct the entire spine model, which is consist of generator and discriminator and training by unsupervised learning approach. Nowadays, GAN model has already been adopted in the transformation from 2D image to 3D scenes. Therefore, our approach represents a potential alternative for EOS reconstruction while still maintaining a clinically acceptable diagnostic accuracy.

Paper No. 124

59

Automatic Liver Segmentation with CT images based on 3D U-net Deep Learning Approach

Ting-Yu Su, Wei-Tse Yang, Tsu-Chi Cheng, Yi-Fei He, Yu-Hua Fang



Liver segmentation, deep learning, computed tomography images, gradient map, k-means clustering

The detection and evaluation of the shape of liver from abdominal computed tomography (CT) images are fundamental tasks in computer-assisted liver surgery planning such as radiation therapy. However, automatic and accurate liver segmentation still remains many challenges to be solve, such as ambiguous boundaries, heterogeneous appearances and highly varied shapes of the liver. To address these difficulties, we developed an automatic liver segmentation model based on 3D U-net network. Some preprocessing steps were done to elevate the performance of our protocol first. Also, an approximate liver map was generated by calculating the gradient of CT images. The area which have high possibility to be liver would be select as the training set to make sure the balance of data. Then, a deep learning U-net structure was applied for the processed training data. Finally, some post-processing methods, which include k-means clustering and morphology algorithms, would be applied in our protocol. Our results indicated that a high structure similarity index (SSIM) and dice score coefficient of liver segmentation model can be achieved, which were 0.9731 and 0.9508 respectively, demonstrating the potential clinical applicability of the proposed approach.

Paper No. 125

60

Automated Breast Ultrasound Computer-Aided Diagnosis Using 3-D Convolutional Neural Network

Ruey-Feng Chang, Yao-Sian Huang, Tsung-Chen Chiang, Hsin-Yi Peng, Chiun-Sheng Huang



Automated breast ultrasound, 3-D CNN, Computer-aided detection and diagnosis, Candidate aggregation

The automated breast ultrasound (ABUS) has been widely used as the popular screening examination in breast. However, reviewing hundreds of ABUS slices and classifying tumor is a time-consuming process for the physiciant. Hence, fast and effective computer-aided detection (CADe) and diagnosis (CADx) systems can help to accelerate the process for the physiciant. Tthe convolution neural network (CNN) has been used in medical imaging for lesion detection, segmentation, and classification recently. Therefor, in this research, a CADe and a CADx systems based on 3-D CNN is proposed for breast tumor detection and classification in ABUS image. The proposed systems locate the tumor position and distinguish the tumor as malignant or benign. In CADe system, the whole ABUS image is scanned and the tumor candicates are detected by using sliding window, 3-D CNN, and prioritized candicate aggregation. After tumor detection, the tumor region is fed as the input of following CADx system for tumor classification. In CADx system, the 3-D texture feature maps are extracted based on the 3-D CNN model. Furthermore, the artificial neural network (NN) classification is employed with extracted feature maps for tumor classification.

Paper No. 137

61

Deep Learning for Breast Cancer Classification with Mammography

Wei-Tse Yang, Ting-Yu Su, Tsu-Chi Cheng, Yi-Fei He, Yu-Hua Fang



Deep Learning, Convolutional Neural Network, Mammography, Transfer Learning, Classification

Current screening of mammography results in a high recall rate. Furthermore, distinguishing between BI- RADS 3 and BI-RADS 4 is a challenge for radiologists. In order to help radiologists’ diagnosis, researches of CAD system recently have shown that methods of deep learning can significantly improve lesion detection, segmentation, and classification. Nonetheless, the beneficence of deep learning in clinics has not been verified. Few researches provided further analysis of equivocal lesions, such as the performance on micro-calcification smaller than 1cm. Thus, we initiate this research to address this problem. We propose to train a convolutional neural network (CNN). The input can be images with various sizes. The prediction of CNN includes images in MLO and CC with left and right sides simultaneously. In addition, we extend the CNN to examine asymmetry with images of previous screening. During the evaluation, we pay more attention to examine the performance on equivocal cases and cases of BI-RADS 3 and 4. Currently, we have achieved 0.86 of AUC with CBIS-DDSM and BCDR.

Paper No. 142

62

Data Enhancement of Deep Learning for Thoracic Imaging

Shoji Kido, Yasushi Hirano, Shingo Mabu



deep learning, data augumentation, generative adversarilal network, unsupervised learning, semi-supervised learning

A lot of medical image data are required for the training of deep learning. However, it is difficult to collect such a lot of image data especially for thoracic images. Because, many kinds of diseases such as lung nodules and diffuse lung diseases are included in thoracic field, and they show a variety of image patterns. And also, these image data should be annotated by radiologists for supervised learning such as deep learning. However, such tasks give burden to radiologists. Therefore, we introduce data enhancement techniques of deep learning for thoracic images.

Paper No. 148

63

A Tissue Classification Method of IVOCT Images Using Rectangle Region Cropped along The Circumferential Direction Based on Deep Learning

Xinbo Ren¹, Haiyuan Wu¹, Qian Chen¹, Takashi Kubo², Takashi Akasaka²

Wakayama University¹, Wakayama Medical University²


IVOCT Images, Deep Learning, Vessel Lesion Tissue, Lipid and Fibrous, Plaque Classification

Coronary artery disease (CAD) as a common disease is now indeed affecting patients’ quality of daily life. Qualification analysis of the causing reasons for this kind of disease needs more vessel inner tissue (healthy or not) information in detail. Recent years, an intravascular OCT technology is starting implemented to the patients for a appropriate treatment. Lesion tissue analysis of thousands of IVOCT image data per patient is time-consuming and lower efficiency on manual analysing. Traditional machine learning methods are always applied to investigate features extracted from the image data with some special feature engineering technologies, but for deeper abstract features it’s still difficult to draw out. Recently, the utility of deep learning method to image target detection and classification tasks has won a great success and it’s generally common to use the deep learning method attack many computer version issues. In this paper, we propose a method based on the Convolutional Neural Network (CNN) to model a VGG-Net-like for category classification of vessel tissue. We preprocess the IVOCT image with catheter and guider-wire removal methods and obtain the lumen boundary. Analyzing the intensity of vessel tissues with light attenuation, we crop rectangle regions with fixed size along the circumferential direction to gain a number of patches as the input samples of CNN. Three kinds of input type, Local Binary Pattern (LBP) based single channel, RGB channels and merging LBP channel with RGB channels, are fed into the model we built. Three kinds of input type, LBP based single channel, RGB channels and merging LBP channel with RGB channels, are fed into the model we built to discuss the prediction results.

Paper No. 158

64

Deep Learning-based Metal Artifact Reduction Technique by Simulating Artificial CT Images with Metallic Implants

Jimin Lee¹, Hyungjoo Cho¹, Hoyeon Lee², Seungryong Cho², Hee-Dong Chae¹, Sung-Joon Ye¹

Seoul National University¹, Korea Advanced Institute of Science and Technology²


Metal Artifact Reduction, MAR, Computed Tomography, Deep Learning, Sinogram

To develop a Deep Learning-based metal artifact reduction technique, we generated training CT image datasets artificially by inserting metallic implants and manipulating their sinograms. The simulated image pairs were used to train our proposed Deep Learning model and demonstrated its feasibility of artifact reduction applied in real patients’ CTimages.

Paper No. 184

65

A Two-stage Organ Classification Network based on 3D ResNet in Chest CT

Jangwoo Kim, Jonghyo Kim



Organ Classification, 3D ResNet, Two-stage, False Positive Reduction

Anatomic detection or organ segmentation is the first preprocessing step to utilize various CAD systems. Various automation technologies have been developed for this purpose, but they have limitations for clinical use. Recently, deep learning research has opened a new horizon of automatic learning method through pattern recognition, and there have been many attempts to introduce it into CAD domain. In this study, we constructed a 3D deep learning based network to recognize human organs in chest CT images. Our network utilized a two-stage approach. In the first stage, learning proceeds for reducing false negatives even if false positives are included. In the second stage, learning proceeds for false positive reduction in order to complement the limitations of the previous step. We tested our network using a publicly available set of data and obtained a promising result.

Paper No. 190

66

Detection of Paroxysmal Atrial Fibrillation by Lorenz Plot Imaging of ECG R-R intervals

Junichiro Hayano¹, Masaya Kisohara¹, Yuto Masuda², Emi Yda¹

Suzuken Co. Ltd.², Nagoya City University¹


artificial intelligence, machine learning, atrial fibrillation, wearable sensor, convolutional neural network, Lorenz plot

To utilize for the prevention of embolic cerebral infarction, we developed machine-learning systems for detecting the episodes of paroxysmal atrial fibrillation (AF) from the interbeat interval (IBI) signals obtained by wearable biometric sensors. Lorenz plot images were generated from overlapping consecutive segment of 600 IBI data and the patterns of image characteristic to AF were discriminated from those of non-AF segments, including sinus rhythm, frequent atrial ectopic beats, and atrial flutter. Lorenz plot images consisting of 8,000 known AF and non-AF samples were provided to the machine learning algorithms of Convolutional Neural Network (CNN), Random Forest, and AdaBoost and of 2,000 samples were used for validating the results. As the results, the CNN developed through the machine learning detected AF with 100% sensitivity and 100% specificity. Lorenz plot imaging of IBI data is useful for effectively discriminating AF from non-AF by artificial intelligence.

Paper No. 216

67

Noise Reduction Methods in Low-dose CT Data Combining Neural Networks and an Iterative Reconstruction Technique

Dahim Choi¹, Juhee Kim¹, Seung-Hoon Chae², Byeongjoon Kim³, Jongduk Baek³, Hyun-Seok Park¹, Jang-Hwan Choi¹

Ewha Womans University¹, Electronics and Telecommunications Research Institute², Yonsei University³

[email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]

CT Noise Reduction, RED-CNN, Deep Neural Networks, Landweber Iteration, Deep Learning, Low-dose CT

Improving image quality from low-dose CT images and keeping diagnostic features is integral to lowering the amount of exposure to radiation and its potential risks. Noise reduction methods using deep neural network have been developed and display impressive performance, but there are limitations on noise remnants, blurring on high frequency edges, and artifact occurrence. To increase noise reduction performance and deal with those issues, we have implemented a block-based REDCNN model and applied Landweber-type iteration to images passed through the REDCNN model. We have also tested the effect of repetition on an iterative reconstruction. As a result, our proposed method outperforms nose reduction by other state-of-the-art deep neural network models.

Paper No. 230

68

Deep Convolutional Neural Network-Based Automated Lesion Detection in Wireless Capsule Endoscopy

Yejin Jeon¹, Eunbyul Cho¹, Seung-Hoon Chae², Hae Young Jo¹, Tae Oh Kim¹, Chang Mo Moon¹, Jang-Hwan Choi¹

Ewha Womans University¹, Electronics and Telecommunications Research Institute²


Deep neural networks, Convolutional neural Networks, Wireless capsule endoscopy, Lesion detection, Small bowel tumors

Because most of the capsule-endoscopic images contain normal mucous membranes, physicians spend most of their reading time observing normal areas. Thus, a significant reduction in their reading time would be possible if only the portion of the image frame for which a particular lesion is suspected can be read intensively. This study aims to develop a deep convolutional neural-network-based model capable of automatically detecting lesions in the capsule-endoscopic images of a small bowel. The proposed model consists of two deep neural networks in parallel, each of which takes in images in RGB and CIELab color spaces, respectively. The neural-networks model is based on transfer-learned GoogLeNet architecture. Our proposed algorithm showed promising results in classifying endoscopic images where lesions exist (98.56% accuracy). If the proposed algorithm is used to screen abnormal images, it is expected to reduce a physician’s reading time and to improve his/her reading accuracy.

Paper No. 238

69

Deep Learning in Medical Imaging: Engineer’s Point of View

June-Goo Lee

Asan Medical Center

[email protected]

Deep Learning, Medical Imaging, Convolutional Neural Network, Image Processing, Image Segmentation

Machine learning (ML) is defined as a set of methods that can automatically detect patterns in data, and then utilize the uncovered patterns to predict future data or conduct other types of decision making under uncertain conditions [1]. The most representative characteristic of ML is that it is driven by data, and the decision process is trained with minimum interventions by a human. The program can learn by repetitively analyzing training data, and can then make a prediction when new data is inputted. In recent years, deep learning, which is a ML algorithm for neural networks with many hidden layers, has shown the state-of-the-art performance in many applications including image recognition, video understanding, speech recognition and natural language processing such as translation and summarization.In previous ML algorithms, the features extracted from raw data were used as input for training. So, defining meaningful and powerful features was an important process. Many domain experts and data scientists have sought to discover and generate handcrafted features after applying diverse evaluation approaches, including statistical analysis and performance tests of ML. However, in deep learning, it could automatically generate meaningful and powerful features from raw data. Because of this reason, it also can be called as ‘feature representation learning’. This learning process of deep learning is surprisingly similar to the process of obtaining knowledge in humans with regard to self-organization. This has led to innovative improvements in performances even in medical imaging nowadays.The convolutional neural network (CNN), which consists of multiple convolutional and pooling layers, has achieved significant improvements in medical image analysis including image enhancement, detection and segmentation etc. The overall learning process of CNN is similar to the organization of the animal visual cortex [2], and a successfully trained CNN can compose hierarchical information during pre-processing, such as an edge-shape-component-object structure in image classification. In this talk, I will briefly explain main characteristics of CNN architecture and compare them to the conventional machine learning methods. Then, I will introduce the research works at my laboratory in medical imaging, which includes CT kernel conversion, automatic lesion detection, airway and multi-organ segmentation methods.[1] Murphy KP. Machine Learning: A Probabilistic Perspective. MIT Press. 2012;25[2] Hubel DH, Wiesel TN. Receptive fields

Paper No. 256

70

and functional architecture of monkey striate cortex. J Physiol. 1968;195(1):215’243

Deep Learning in Medical Imaging and Radiology: Radiologist’s View

Chang Min Park


[email protected]

Machine learning, Deep learning, Medical imaging, Radiology, Clinical application

Medicine, particularly Medical imaging is essentially an information science in which medical experts always try to acquire individualized and context-specific data, and to then iteratively evaluate, and refine this information against a vast database of medical knowledge in order to resolve individual patient’s problems. In this context, machine-learning (ML), particularly deep-learning (DL) techniques are extremely well suited to medical imaging, especially radiology. A lot of researches have been made in radiologic application of DL technique, and there are experts stating that ML and DL is now entering the main stream of medicine including radiology. However, the truth be told, reality is not as optimistic as we have been heard. Few qualified DL based techniques or products have been applied in real-clinical practices, and impactful application of these technologies significantly lags in radiology field. In this talk, I will discuss the current status of ML and DL application in radiology field, the challenges against their adoption in clinical practices and suggestive solutions to those challenges.

Paper No. 263

72

Special Session on Image Reconstruction in Tomographic Medical Imaging

Deep Learning-Based CBCT and PseudoCT Reconstruction for Practical Radiation Therapy

Jaehee Chun¹, Hao Zhang², Sohyun Park¹, John H. Lee¹, Justin C. Park², Jinsung Kim¹

Yonsei University College of Medicine¹, Washington University in St. Louis²


IGRT, Deep Learning, CBCT, Radiation Therapy

Purpose: In image guided radiation therapy (IGRT), cone-beam computed tomography (CBCT) system using kilo-voltage (KV) x-ray source is the most widely-used imaging devices. However, CBCT uses hazardous ionizing x-rays and hence minimizing the imaging dose is desirable based on ALARA principle. In this case, we can apply a deep learning based non-iterative low-dose CBCT reconstruction method for IGRT applications. Moreover, MR-only guided radiation therapy (gRT) can be an attractive goal for future gRT applications that take advantage of eliminating dose as well as improving soft tissue contrast offered by MRI compared to CT. However, the electron density information used in dose calculations is derived from CT images, necessitating that CT simulation be a component of the conventional MR-gRT workflow. In this case, we can consider a deep learning-based method for pseudoCT reconstruction that eliminates CT simulation from the MR-gRT workflow in a move towards MR-only gRT. In this study, we propose a practical application of deep learning based CBCT and pseudoCT reconstruction for radiation therapy.Methods: In this study, we have formulated Low-dose CBCT reconstruction problem as restoring high quality high-dose CBCT projections (100kVp, 1.6 mAs) from noisy low-dose CBCT projections (100kVp, 0.1 mAs). The restoration of CBCT projection was performed using generative adversarial network (GAN) which is a convolutional neural net framework that consists of two models: 1) a generative model that produces high-dose CBCT projection from corresponding low-dose projection, and 2) a discriminative model that distinguishes with a certain probability if a given high-dose image is drawn from the true distribution of high-dose images or generated by the other network. Prior to the training, both images were filtered to eliminate low-frequency information which is not necessary for analytical reconstruction. For evaluation, the trained model was applied to unseen phantom data that was placed randomly on the couch. The restored high-dose projection was reconstructed using simple back-projection method.Results: Training on 700 image pairs took approximately 16 hours to complete using Nvidia GTX 1080. PseudoCTs are produced by the trained model with a throughput time of approximately 80 projections/second. Significant noise reduction was achieved compared

Paper No. 178

73

to original input while maintaining the quality comparable to the CBCT of high-dose projections. Conclusion: The proposed deep learning-based method for CBCT reconstruction offers the ability to reduce the imaging dose without the addition of reconstruction time. This makes our approach potentially useful in an on-line image-guided radiation therapy.

Deep Recurrent Learning of Brain Functional Dynamics for Data-driven Classification of Psychiatric Diagnosis: The Temporal Template Network

Byung-Hoon Kim, Jong Chul Ye



Functional Magnetic Resonance Imaging, Recurrent Neural Network, Brain Dynamics

In this paper, we propose a novel method based on deep neural network for classification of psychiatric diagnosis (schizophrenia) from healthy controls using only the resting-state functional magnetic resonance image data

Paper No. 254

75

Metal artifact reduction of dental CT using optical image of dental plaster model

Yejin Kim, Jihoon Cho, Seungyong Cho, Jinah Park



Dental CT, Metal Artifact Reduction, Dental Optical Image, Medical Image Registration

In dental CT, images are subjected to metal artifact due to metal implant. Typically, metal artifact can reduced by reconstructing metal part separately and put it on the metal-free-reconstructed image. However, segmentation of metal part from initially reconstructed image is challenging task since the initial images are highly contaminated by the metal artifact.For effective metal artifact reduction, we propose to introduce optical images of surgical-model to segment the metal part from the initial image. Unlike to real human dental structure the surgical model consist of single material. Moreover, optical images can provide only surface information of the surgical model not containing intensity infomation. Therefore, dental CT images and surgical-model optical images are registrated using iterative-closest-point algorithm first thus, registrated surgical model can be used for mask image for the tooth and metal part in the dental CT. Surgical-model-aided segmented metal part can be forward projected and re-reconstructed while metal-free images are also forwared projected and re-reconstructed separately. Each reconstructed images can be put together thereby single metal-artifact-free image can be obtained.

Paper No. 255

76

Arterial spin labeling MRI with radiofrequency pulse modulation and Fourier analysis

Hyo-Im Heo¹, Paul Kyu Han², Seung Hong Choi³, Sung-Hong Park¹

Korea Advanced Institute of Science and Technology¹, Harvard University², Seoul National University³


Arterial Spin Labeling, Magnetic Resonance Imaging, Perfusion Imaging

Pseudo-continuous arterial spin labeling (pCASL) is one of the non-invasive perfusion imaging technique through MRI. However, pCASL is susceptible to unintended image corruption due to pair-wise subtraction between control and label scan, and has high specific absorption rate. In this study, we propose new approaches to improve pCASL signal including Fourier analysis instead of pair-wise subtraction, application of recently suggested data acquisition scheme for pCASL with bSSFP pulse sequence, and power-modulated pCASL labeling scheme. Perfusion images with Fourier analysis showed the higher contrast between GM and WM and less blurring than images with pair-wise subtraction. Pseudo-centric PE scheme decreased eddy current artifact in perfusion imaging. Power-modulated pCASL showed perfusion images comparable to those of the conventional pCASL. Under data corruption, power-modulated pCASL maintained the perfusion signals well with no observable effect. Proposed approaches showed possibilities of improving perfusion signals in pCASL and power-modulated pCASL is robust to accidental image corruption and has relatively low instantaneous SAR, so that this approach is potentially advantageous in high-field MRI.

Paper No. 258

77

A Dual-Echo Balanced Steady-State Free Precession Imaging for Proton Resonance Frequency-Based Thermometry

Seohee So, Hyunwook Park


[email protected], [email protected] resonance thermometry, Proton resonance frequency shift, alanced steady-state free precession, Dual-echo

acquisition

Magnetic resonance (MR) thermometry is a promising technique for monitoring and guiding thermal therapy. MR thermometry estimates temperature change by measuring temperature sensitive MR parameters. This study investigated the phase linearization to the frequency shift in the balanced steady-state free precession (bSSFP) sequence. A dual-echo balanced steady-state free precession imaging method is proposed for proton resonance frequency-based thermometry. The high signal intensity from the bSSFP sequence and the phase amplification by combining dual echo signals are strong advantages of the proposed dual-echo bSSFP method.

Paper No. 259

78

Big Data and AI

Analysing the Potentials of Young Researchers through H-Index and Co-Author Networks in JST Biological Science Database

Kenta Ishido¹, Hiroto Inoue¹, Nobuji Tetsutani¹, Masanori Fujita², Takao Terano³

Tokyo Denki University¹, Tokyo Institute of Technology², Chiba University of Commerce³


co-author network, betweenness centrality, h-index

H-Index is often used to evaluate researchers. However, h-Index is not suitable for young researchers, because they have only few the past achievements. In this paper, we evaluate JSPS re-search fellows’ potential using betweenness centrality of co-author networks. As a result of logistic regression analysis, it is shown that the proposed method can evaluate young re-searchers’ potential at early stage compared with h-Index.

Paper No. 41

79

A performance improvement of Mask R-CNN using region proposal expansion

Naoki Degawa, Xin Lu, Akio Kimura

Iwate University


object detection, classification, R-CNN, region proposal, deep learning

It is difficult for the conventional Mask Regions with Convolutional Neural Network (Mask R-CNN) to distinguish different objects with similar features of the shape. In this paper, we improve the object classification performance of Mask R-CNN by expanding the region proposal appropriately and using it for learning. The results of experimental evaluations using our modified 300-W dataset show that the mAP of our proposed method is improved from 0.631 to 0.701, compared with the original Mask R-CNN.

Paper No. 42

80

Squat movement recognition using convolutional neural network

Nantana Rungsawasdisap¹, Xin Lu¹, Adiljan Yimit², Zhen Zhang¹, Masaya Mikami¹, Yoshihiro Hagihara¹

Iwate University¹, Akita University of Art²


squat classification, hidden Markov model, convolution neural network

This paper proposes a novel squat movement recognition method based on convolutional neural network (CNN). It can classify 6 squat patterns with a high accuracy of 93.33\%, and perform better than our previous method using hidden Markov model (HMM) with an accuracy of 78.58\%.

Paper No. 96

81

SSD Net: Toward Deep Network Models based on Dissimilarity Metrics

Satoshi Arai, Tomoharu Nagao

Yokohama National University


Deep Learning, Pattern Recognition, Dissimilarity, Sum of Squared Differences, Image Classification

Most artificial neural networks that originate from the perceptron use the inner product as the basic operation to calculate pattern similarities. Unlike them, we propose a novel hierarchical network model based on a pattern dissimilarity operation using a popular dissimilarity metric: sum of squared differences. Our model is differentiable and end-to-end trainable. We provide a description of the basic formulation and network architecture of the proposed method. Then we apply our method to image classification tasks using public datasets for performance comparison. Although our method does not outperform the same size of convolutional neural network in terms of classification accuracy, it demonstrates that comparable performance can be obtained.

Paper No. 119

82

CG/VR/AR

Development of a Puzzle-box Game with Haptic Feedback

Yoshimasa Tokuyama¹, R.P.C. Janaka Rajapakse², Tsubasa Miyazato³, Kouichi Konno⁴, Yi-Ping Hung³

Tokyo Polytechnic University¹, Tainan National University of the Arts², Fuji Electric IT center Co., Ltd.³, Iwate University⁴


haptic games, rehabilitation, force feedback, healthcare, learning

The main purpose of computer game is to immerse a user in a virtual environment. The perception of immersion experience has been used to develop virtual reality applications for game-based learning, healthcare, rehabilitation, etc. In traditional games, the user immersion is performed through the sense of visuals and sounds with interactions through 2D input devices like mouse and keyboards. Haptic technology can simulate tactile and kinesthetic sensations in virtual environments. The real-time interaction and feedback are very important for development of rehabilitation applications. The main goal of our research is to enable real time simulation of force feedback for rehabilitation and learning games. This paper presents a framework for developing the virtual puzzle-box game for rehabilitation and education based on haptic display. The proposed framework introduces a collision handling method to compute feedback force and provide realistic feedback through the haptic device.

Paper No. 6

83

Automatic generation of 3D faces expressing different utterances by modeling individual variation of shape deformation from expressionless faces

Isseki Miwa, Shigeru Akamatsu

Hosei University

[email protected], [email protected] expression generation, vector representation of 3D object, morphable 3D face model, 3D shape clustering of

faces

In this study, we used Kinect V2 RGB-Depth sensor to measure three-dimensional shape of faces and calculated the information of the differences among several individuals. Based on Principle Component Analysis on the obtained information, we tried to verify the possibility of generating various expressions from an inexpressive face of an arbitrary individual.

Paper No. 13

84

Development of a VR Flood Situation Experience System

Yutaka Hikino¹, Tomoaki Moriya¹, Tokiichiro Takahashi²

Tokyo Denki University¹, Tokyo Denki University/ASTRODESIGN, Inc.²


Flood Disaster, Virtual Reality, Experience System

In Japan, flood disaster is frequently caused by typhoons and torrential rain. However, people’s awareness of flood disaster is not high, and low evacuation rates are in question. Since existing research and contents only display the water depth at flooding, it is difficult to fully tell the danger of flood damage. In this research, we aim to develop a system that can tell the risk of flood disaster through the experience of walking on a flooded road using virtual reality (VR) .

Paper No. 47

85

A Study of Historical Sites Display AR Application using Location-based AR

Yukai Onodera, Kouichi Konno

Iwate University


Augmented Reality, Historical site, Global Positioning System, Location-based method

Descrive in PDF file.

Paper No. 51

86

A Study of Extracting a Hollow Space from Joined Material for Finding Adjoining Stone Flakes

Runxin Jia, Kouichi Konno

Iwate University


Joining Material, Point Cloud, 3D Measurement, Hollow Space

Describe in PDF file.

Paper No. 62

87

Avatar’s Facial Expressions with ‘Manpu (Comic Symbols)’ by Referring Biometric Information

Shu Gemba¹, Tokiichiro Takahashi²

Tokyo Denki University ¹, Tokyo Denki University/ASTRODESIGN, Inc.²


Avatar, facial expression, comic symbol, facial image analysis, emotion, biometric information, heart rate

It is difficult to present natural avatar’s facial expression. We propose a new method for generating avatar’s natural facial expression by combining both manpu (comic symbols) based on emotion classification from user’s facial photograph and emotion details measured by biometric sensor.

Paper No. 63

88

A System to Support Kinematic Analysis by Visualizing Human Motion with CG

Sho Nakadaira¹, Asako Soga¹, Kunihiko Oda²

Ryukoku University¹, Osaka Electro-Communication University²


motion capture, visualization, kinematic analysis

In this study we developed a visualization system of human motion characteristics using motion data. Our system visualizes user-defined characteristics with selected visualized methods using 3DCG. For kinematic analysis, the system can project human motion to frontal plane, sagittal plane. We analyzed ballet motion using the system. It was confirm that this system has possibilities to support kinematic analysis.

Paper No. 66

89

Droplet Simulation for Cerebral Aneurysm Embolization

Takuya Natsume¹, Masamichi Oishi², Nobuhiko Mukai¹, Marie Oshima²

Tokyo City University¹, The University of Tokyo²


Physical Simulation, Computational Fluid Dynamics, Particle Method, Cerebral Aneurysm Embolization

Liquid embolization, which is not allowed in Japan, is used to prevent rupture of cerebral aneurysm overseas. Then, we had been developing liquid injection simulation based on CFD (Computational Fluid Dynamics) in order to evaluate the safety of liquid embolization surgery, and comparing the simulation result with the physical experiment to confirm the simulation accuracy. Particle method was used for the simulation, and the simulation size was smaller than the physical experiment, which limitation was due to computational resource. As a result, the formed droplet contacted the side surface of the water tank so that quantitative assessment of the droplet size was not performed. Therefore, we have performed the simulation with wider size of the environment, and also have compared the aspect ratio of droplet between the simulation and the physical experiment. As a result we have confirmed that the simulation can form the droplet, which size is similar to the physical experiment.

Paper No. 69

90

Proposal of a welding skill training system using VR technology

Shintaro Sakata, Shinji Mizuno

Aichi Institute of Technology


VR, welding, skill training, simulation

In this paper, we propose a welding skill training system for beginners using VR technology. By using this system, the user can observe a welding work of an expert from various places in the VR space, and experience a welding work virtually. In addition, the system can acquire and analyze the posture of the user welding in the VR space, and also present to the user whether the user is working with the correct posture in real time. Experiments suggested that this system would be effective for welding skill training for beginners.

Paper No. 70

91

3D visualization of work using AR technology for welding skill training

Takumi Sakakibara, Shinji Mizuno



AR, welding, skill training, visualization

In this paper, we propose a 3D visualization system of work using AR technology for welding skill training. This system presents information necessary for welding training such as welding speed and posture during welding to the user using AR technology. We implemented a prototype system and verified its operation.

Paper No. 71

92

A Real-time Rendering method based on Contour Drawing Techniques in Japanese Ink Painting

Hina Yumoto¹, Shuhei Kodama¹, Tokiichiro Takahashi²



Non-Photorealistic painting, Japanese ink painting, Real-time Rendering

We propose a real-time rendering method of Japanese ink painting. Especially, we force on methods both ‘mokkotsu’and ‘horinuri’techniques. We developed a real-time rendering system which can render Japanese ink painting images based on two techniques.

Paper No. 75

93

Proposal of an interactive projection mapping using seats of multiple chairs

Kouki Yoshikawa, Shinji Mizuno



projection mapping, interaction, CG

In this paper, we propose an interactive projection mapping using multiple chairs. In this content, images are projected from the ceiling to the seat of the chairs. The projected image would change depending on the arrangement of the chairs interactively. When the chairs are placed apart, a small image is projected on the seat of each chair. When the chairs are gathered, a large image is projected on the entire set of the chair seat. We implemented a miniature prototype system and experimented.

Paper No. 76

94

A Development of Stage Event Management System using Virtual Reality Technique

Jifeng Xu, Yuko Tashiro, Mizuki Nakajima, Tsuyoshi Saitoh



virtual reality, stage event, stage layout, simulation, practice

In order to organize the event such as ceremonies, lectures and academic societies, we have to make plans such as layout of the venue, placement the staff, route of entry, exit and evacuation and so on. In addition, it is also important to check the lighting and sound effects of the venue, and to carry out simulations assuming troubles. On the other hand, a presenter is likely to be in trouble caused by being unfamiliar with the venue. It is also important to check the situation of the venue and the position of the seat as participants. However, from the point of the venue reservation and movement cost, it may be difficult to preview and rehearse at the venue several times. Therefore, the purpose of this research is to develop a stage event support system using virtual reality techniques. By using this system, organizer can simulate placement of the venue and simulate the schedule, presenters can confirm the environment of the venue and practice, and participants can check the situation of the venue. When a system user wears a head mounted display (HMD), she/he can perform rehearsals and simulations before the event such as confirming the environment and practicing events as if she / he is in the virtual space on.

Paper No. 78

95

An AR-based Self-Training System of Chest Compression as CPR

Moe Kadosawa, Mitsunori Makino

Chuo University


augmented reality (AR), cardiopulmonary resuscitation (CPR), chest compression, self-training

This article proposes a self-training system of chest compression, one of the important methods of emergency life-saving, with augmented reality (AR) technology, which decreases barrier of the practical learning and contributes to improvement of the learner’s skill. The system is built on a see-through head-mounted display (HMD) with camera and computer. For a user wearing the system who practices the chest compression with a training manikin by him/herself, it provides feedback on posture, depth and rate superimposed on the HMD.

Paper No. 100

96

An Arm Motion Learning Support System using Virtual Reality

Yoshie Doshi, Mitsunori Makino

Chuo University


Sign language, Virtual reality (VR), Head-mounted display (HMD), Motion capture

For beginners of sign language, this article proposes the arm motion learning support system to analyze the difference between the example of teacher’s motion and user’s motion, and to urge the user to improve his/her motion. Using virtual reality (VR) system composed by head-mounted display and motion tracking hardware/software, the proposed system measures how similar between the example motion and user’s motion, and gives him/her the result of analysis and replay of the motion as visual feedback.

Paper No. 101

97

A Study of Recognizing Flake Surfaces Based on Feature Lines of Stone Tool

Erdenebayar Shurentsetseg, Kouichi Konno

Iwate university


feature line, segmentation, stone tool, flake surface, recognition, point cloud

In recent years, three-dimensional technology is capable to simplify the reassembly process of the fractured objects. The reassembly process of stone tools can be easily done by the refitted flake matching process. The refitted flake matching process is required for highly accurate segmented flakes. This work aims to extract highly accurate flake surfaces and to match the extracted flake surfaces. In addition, this paper studies to evaluate the matching accuracy of the extracted flakes and to compare to other methods. Moreover, this paper presents an algorithm of flake surface recognition based on feature lines of stone tools. The implementation of this work can recognize flake surfaces according to the scale drawing of the stone tool with high accuracy.

Paper No. 104

98

Virtual Museum View in Head Mounted Display

Kohei Takeuchi¹, Masaki Hayashi², Makoto Hirayama¹

Osaka Institute of Technology¹, Uppsala University²


Virtual museum, Unity, GoogleVR, C#

Research and development of virtual museum using real time 3DCG is promoted. Research is being conducted with the aim of minimizing the stress on the viewer and experiencing it in a sense close to viewing at an actual art museum. We will develop virtual museums in VR as a way to further enhance the reality of virtual museums this time. Currently a simple virtual museum is being designed, using the Unity game engine, and GoogleVR for development, C# language for programming was used. Based on the prototype of this VR virtual museum, the current virtual museum is to make use of VR viewing. Then, comparison with the current virtual museum is done from various viewpoints, and we examine which is better in terms of user experience .

Paper No. 106

99

A Fast Method of Iron Rust by Corrosion for High Resolution Voxel Models

Yuta Suzuki¹, Tomoaki Moriya¹, Tokiichiro Takahashi²

Tokyo Denki University¹, Tokyo Denki University/ASTRODESIGN,inc.²


Corrosion, Rust, Voxel, Cellular automaton, Voxel automaton

There have been used several techniques to express how the 3D models change by aging and weathering in computer game world. The actual iron discolors as rust progresses, then becomes brittle. A part of the iron surface eventually peels off, then becomes uneven irregularly. The rust progresses further, holes open in the iron plate. In order to express such changes in the computer game world, it is needed to prepare a large variety of 3D models and textures according to the passage of time. These tasks would certainly be cumbersome and burden for 3D modelers.We have proposed a voxel-based 3D automaton method that enabled to express holes in the iron plate due to rust by corrosion. But, it was hard and slow to handle complicated polygonal models.In this paper, we propose a method that converts such complicated polygonal models into high resolution voxel models. When entering a polygonal model that is before rusted, our fast system automatically generates a rusted and corroded model over time until finally, holes open in the model’s surfaces.

Paper No. 117

100

A User Interface for Walking on VR Space by Swinging Arms and its Evaluation

Kosuke Iwasaki¹, Tokiichiro Takahashi², Tomoaki Moriya¹

Tokyo Denki University¹, Tokyo Denki University/ASTRODESIGN,inc.²


VR, User interface, Position sensor, Arm swing

Without causing VR sickness, we had developed a user interface that allows you to swing your arms and walk around VR space smoothly as you walk. Here we propose a new method that wraps the sensor around the wrists and allows natural arm swinging. We also evaluate how to swing user’s arms and walk around virtual space from its walking speed, routes and different types of sensors.In conclusion, our new method is effective for walking around the virtual space without causing VR sickness.

Paper No. 121

101

A Study on Rescue Operation Support Systems Using 3D Reconstruction of Disaster Scenes with Human Cooperation via Digital Pen and Dot Screen

Wei-Chung Chen, Sayaka Oono, Seiji Ishihara, Makoto Hasegawa



3DCG, 3D Reconstruction, Digital pen, Anoto dot screen, Disaster response

Three-dimensional computer graphics (3DCGs) of disaster scenes can be created to visualize the situation of the disaster areas through the use of Structure from Motion (SfM) technology and the collected photographs or videos taken with digital camera or video camera. And it could be possible to write information onto the surface of these 3DCGs through the handwriting with a digital pen on the dot screen where 3DCGs are projected with a projector. The 3DCG created from images taken by victims or rescue teams is proposed to support disaster relief. This paper presents the simulation of Sona Area Tokyo of The Tokyo Rinkai Disaster Prevention Park.

Paper No. 126

102

A Simulation Model of Changing State of Blood Based on SPH Method

Reo Kumaki, Tomoaki Moriya, Tokiichiro Takahashi



Expression of blood, SPH method, Evaporation model, Coffee ring effect, Clotting, Discoloration

Blood expression is often used in the CG field. Along with improvement of technology, it became possible to express that blood adheres to an object. However, there are few examples showing state changes such as coagulation. In this paper, we extend the SPH method, simulate the state change of the blood focused on the coffee ring effect, and propose a more realistic expression method.

Paper No. 127

103

An Examination of Flake Surface Segmentation Based on Ridge Line Extraction Method from Measured Point Clouds

Keita Murakami, Erdenebayar Shurentsetseg, Kouichi Konno

Iwate University


point cloud, flake surface, ridge line, neighboring points of ridge line, Euclidean distance

Describe in PDF file.

Paper No. 129

104

Using Augmented Reality to Enhance Student’s Understanding of Basic Function Graphing in Mathematics

Suneetha Varahamurthy, Shao Shi, Tsu Soo Tan, Mellissa Dobson, Kong Wai Ming

Nanyang Polytechnic


augmented reality, enhanced learning experience, mathematics

Function graphing is a crucial skill required in all scientific fields. However, some student weak in mathematics are unable to visualise & systematically graph various trigonometric functions to completion within a reasonable amount of time. The inability to complete the graphing process often leads them to give up easily in frustration. This paper describes the development of a prototype of an augmented reality (AR) application that provides step-by-step guide to function graphing and the feedback results from students who used this prototype.

Paper No. 131

105

Surf Disc: A Cruising Interface in 3D Virtual Space

Tsuyoshi Matsudo¹, Tokiichiro Takahashi²



Virtual Reality, VR sickness, VR Bodily Sensation Games, A Cruising Interface, Surf Disc

We propose SurfDisk as an interface for moving the virtual space smoothly. In this paper, we evaluate the operability of SurfDisk from the viewpoint of VR sickness and physical fatigue, and report the results.

Paper No. 133

106

An Examination of Perception with Auditory Stimuli When Grasping a Virtual Soft Object with a Bare Hand

Mie Sato¹, Sato Ishigaki², Sho Kato², Arie E. Kaufman³

Utsunomiya University/Stony Brook University¹, Utsunomiya University², Stony Brook University³

[email protected]

augmented reality, auditory stimuli, perception, interaction

In order to manipulate a virtual soft object with a bare hand, we examine effects of the auditory stimuli on the perception of the virtual object regarding the pseudo-softness, the pseudo-heaviness, and the ease of grasping, holding and moving it.

Paper No. 136

107

Luminance Compensation on Swinging Curtains

Kazuma Yoshimura, Naoki Hashimoto

The University of Electro-Communications


luminace compensation, inter-pixel correspondence, non-rigid surface, response function

In this research, we propose luminance compensation method for swinging non-rigid surfaces. The shift of the inter-pixel correspondence between the projector and the camera due to the shape change of the projection surface, is estimated in real time based on Epipolar geometry.

Paper No. 138

108

THE CONTOUR IMAGE STYLE TRANSFER BASED CONVOLUTIONAL NEURAL NETWORK

Nan Deng¹, Jing Li¹, Xingce Wang¹, Zhong-Ke Wu¹, Wuyang Shui¹, Yan Fu¹, Mingquan Zhou¹, Vladimir Korkhov², Luciano Paschoal Gaspary³

Beijing normal university¹, St. Petersburg State University², Federal University of Rio Grande do Sul³

[email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected],


style transfer, contour processing, deep learning, convolutional neural network

The aim of style transfer is giving the style from one picture to another. The application of neural network in image processing separates the high level features and low level features of the image in the process of style transfer, and derives a variety of methods and optimization for style processing. The style transfer generates new images by separating and recombining the content and style of original images. In this process, various factors such as color and illumination will affect the result. The traditional algorithm only focuses on continuous pixels and the whole image, this paper will extend the process object to the contour of the image, and improves the detail processing from the existing style transfer examples. From the contour of images, the target image retains the contour feature of style image and the content of original image, in other word, gives the contour style of style image to original image. Finally, the style transfer effect based on the original image contour is obtained with some defects. The work can be easily extended to the aspects of video and 3D images.

Paper No. 167

109

Refocusing the Real-world using the Optical See-through Head-mounted Display

Jae Ryun Chung, Dong-Wook Kim, Seung-Won Jung

Dongguk University


head-mounted display, image refocusing, optical see-through, wearable device

Optical see-through (OST) head-mounted display (HMD) enables the user to see the real-world and virtual objects together. Previous OST HMD-based augmented reality applications mainly attempt to render the virtual objects that are geometrically and photometrically well-aligned with the real-world. In this paper, we show that the user’s perception of the real-world can be changed by overlapping the image of the real-world with the real-world. As a case study, we apply image refocusing to the captured real-world image and display the refocused image via the OST HMD such that the user sees the real-world and refocused image together. To this end, we first perform an experiment to find the relationship between the synthetic blur of the augmented object and the perceptual blur of the real-world object. We then apply the depth-adaptive image defocus blur to the image of the real-world to create the perceptual blur of the real-world. Experimental results show that the proposed method can make special visual effects to the real-world that the user sees through the OST HMD.

Paper No. 168

110

Physics-based 3D Ink Diffusion Simulation in Real Time

Yu Zhang, Xingce Wang, Zhongke Wu, Shuaishuai Zhao

Beijing Normal University


Fluid Simulation, 3D Ink Simulation, CUDA, SPH

This paper presents a physics-based method for simulating the phenomenon of ink diffusion in fluid.Here are two contributions in our method: 1) Our method is physics-based method, which can simulate ink diffusing phenomenon realistically. 2) CUDA acceleration and spatial subdivision optimization is applied to our method, thus our method can obtain a realistic simulation result in real time.This method is a grid-particle combined method: We use two different particle systems to represent fluid and ink, and use a improved smoothed particle hydrodynamics(SPH) method to simulate the behavior of fluid, velocities of ink particles are calculated by neighbor fluid particles.In our method, Compute Unified Device Architecture(CUDA) acceleration technique and the space subdivision optimization are applied to achieve a real-time simulation.The rendering of ink particles uses point sprites and blurring techniques. Compared to other ink simulation methods, our method is more realistic and efficient.

Paper No. 170

111

Geometrical Correction for Surrounded Projection by Simplifying a Shape of a Measured Indoor Space

Yuka Nakamura, Naoki Hashimoto



Multi-Projection, Gray Code projection, Phase Shift method, Fish-eye lens camera

In this research, we propose a geometrical correction method for surrounded projection combining a standard lens camera with a fish-eye lens camera and simplifying a shape of a measured space. And we realize simple construction of image projection space in any projection space.

Paper No. 197

112

VR System of Infants’ Behavior for Promotion of Prevention of Accidents

Ryo Tomiyama, Hisashi Sato

Kanagawa Institute of Technology


Virtual Reality, Infants’ Behavior, Prevention of Accidents, HTC VIVE

This paper introduces an infant rearing Virtual Reality Game named “Hai Hai VR - What’s Hai Hai…?” This game allows players who are interested in child care to become familiar with all kinds of accidents that can happen when raising infants as this VR system puts the players into the shoes of infants and allows them to see the world through the eyes of the toddlers. The development environments for this VR system are Unity and Steam VR. During the actual gameplay, players are asked to wear HTC VIVE Head Mounted Display (HMD) while using the ‘Crawling Controllers’ original devices invented by the team. Using “Hai Hai VR - What’s Hai Hai…?” players get to experience the baby’s vision, crawling movements, and learn how to navigate around potential accidents.

Paper No. 204

113

Designer-Friendly Curve Refinement

Shirdxay Chanvisouth¹, Daisuke Taki¹, Takafumi Saito¹, Jianmin Zheng²

Tokyo University of Agriculture and Technology¹, Nanyang Technological University²


B-spline curves, Curve interpolation, Optimization problem, Experiential Design Guidelines

In conventional design CAD systems, to construct an intended curve segment, it is considered time-consuming and labor-intensive due to the nature of the curve. Thus, there exists a rebuild function to help facilitate the design process. However, it is not designer-friendly in the sense of the location of control points, and fairness of the curvature, which makes post-rebuild manipulations difficult. In this study, we proposed a method that is based on the experience of industrial designers. We approach this problem in two ways; by solving curve interpolation, and optimization problems. Numerical examples demonstrate the effectiveness of the methods.

Paper No. 209

114

Augmented Reality Based on the Integration of Mobile Edge Computing and Fiber-Wireless Access Networks

Yaqiong Liu¹, Jing Ling¹, Guochu Shou¹, Hock Soon Seah², Yihong Hu¹

Beijing University of Posts and Telecommunications¹, Nanyang Technological University²


augmented reality, mobile edge computing, fiber-wireless networks

Fiber-wireless (FiWi) access networks are typical hybrid access technologies that combine the high bandwidth of optical access and the flexibility and ubiquitous coverage of wireless access. Mobile edge computing (MEC) provides low latency services and cloud computing capabilities at the edge networks. The convergence of MEC and FiWi access networks can improve network performance and QoS (Quality of Service) of augmented reality (AR) services. Therefore, in this paper, we study the problem of integrating MEC with FiWi to enhance AR services. We first present the integration scheme of MEC with FiWi for AR applications. We then propose our AR applications enhanced by the integration of MEC and FiWi access, which can provide multiple low-latency AR services. Performance evaluation results demonstrate that the integration of MEC with FiWi can support long-distance AR applications with low delay.

Paper No. 212

115

Converging Mobile Edge Computing and Wireless Access for Virtual Reality

Qingtian Wang¹, Guochu Shou¹, Yaqiong Liu¹, Feng Lin², Hock Soon Seah²

Beijing University of Posts and Telecommunications¹, Nanyang Technological University²


Virtual Reality, Mobile Edge Computing, Wireless access

Traditional Virtual Reality(VR) devices which limited to the cable connection are inconvenient for users who enjoy moving in the virtual world freely. To this end, a wireless VR solution is a promising approach for improving user experience. Recently, Mobile Edge Computing(MEC) provides high bandwidth and low latency, which enhances VR wireless solution for supporting high definition and low latency applications. In this paper, we propose a Virtual Reality wireless solution based on MEC to meet the requirement of future VR applications with high definition display and low latency. Compared to existing wireless VR solution, our solution is superior in improving transmission bandwidth and enhancing the quality of the user’s experience.

Paper No. 221

116

Efficient Reconstruction of Light Fields for Consistently Augmented Reality Using Dense Multi-View Systems

Seiya Suda¹, Kazuya Kodama², Takayuki Hamamoto¹

Tokyo University of Science¹, National Institute of Informatics, Research Organization of Information and Systems²


light field, multi-view system, 3d display, view interpolation, augmented reality

We study dense multi-view systems transmitting light fields beyond a visual obstruction as if it were transparent. Multi-view 3d displays appropriately combined with camera arrays or lens arrays provide consistently augmented reality for many users at the same time as a simple solution of such occlusion in the real world.We design our proposed multi-view system, and then, in order to reduce image sensors acquiring light fields for inexpensive systems, efficient interpolation is introduced to reconstruct the entire 4d light field in real time for consistency with the real world. We implement on a GPGPU actually achieves real-time interpolation obtaining enormous data of light fields without severe degradation for various depths.

Paper No. 223

117

Multimodal Interaction on Vertically-Mounted Large Displays

Mohammad Chegini¹, Tobias Schreck¹, Alexei Sourin²

Graz University of Technology¹, Nanyang Technological University²


Visual Analytics, Data Visualisation, Multimodal Interaction

Multimodal interactions on vertically-mounted large multi-touch displays offer various advantages in situations that traditional touch input is not adequate. Due to the nature of this type of input, some issues like hygiene problems and the need to work close to the screen are present. Therefore, in this paper, a system that works based on other types of interactions with the display is proposed. The system utilizes the power of eye-tracking and a second handheld device to help the user interact more efficiently with the display in a safe environment.

Paper No. 239

118

Computer Vision

Conflict Robust Player Tracking Method for Wall-Mounted Multi-Camera Systems

Daisuke Ishii, Osafumi Nakayama, Tohru Tsuruta

Fujitsu Labs. Ltd.


Sports, Stats, Multi-camera, Multi-person tracking, Basketball, Computer vision

Recently, there has been an increase in the use of video and statistical data in sports practice and coaching. A player-tracking function plays an important role in order to obtain players’ trajectories which are essential for performing statistical analyses of players. Usually, in video-based tracking systems, sensor cameras are mounted to ceiling to prevent players from overlapping on the image, because the overlapping makes it difficult to detect overlapped further players, which may lead wrong trajectories. So, the ceiling cameras are useful for tracking but it is expensive to mount on ceiling, thus a system using wall-mounted cameras that can be installed at low cost is desired. However, wall-mounted cameras cause a problem that players may overlap each other on the image. To overcome this problem which is inevitable for such wall-mounted camera systems, we propose a stable tracking method which determines the position of the player having less overlapping ratio earlier and determines ones having more overlapping ratio later with using previously determined players’ positions. The overlapping ratio is defined by using the positional relation between the target player and one’s surrounding players from all of those images where the target player exists. Experimental results with real basketball scenes show that the positional error is less than 15cm, and the processing time is less than 10ms/frame at 3.0 GHz CPU, which proved that our method has enough performance for using real-time player tracking system.

Paper No. 25

119

Iterative Update Method of 3D Map Based on Self-Localization Using Multi-Layer NDT in Dynamic Environment

Tsuyoshi Amano¹, Isao Miyagawa², Kazuhito Murakami¹

Aichi Prefectural University¹, Nippon Telegraph and Telephone²


3D map, multi-layer NDT, self-Localization, dynamic environment, 3D LiDAR, mobile robot

We aim to provide a large-scale point cloud-based 3D map that reflects the internal structure in a building for autonomous mobile robots. We propose a new method that iteratively updates the 3D map based on self-localization in dynamic environment. We assume that a patrol robot collects 3D points required for constructing a 3D map in database. We adopt multi-layer NDT (Normal Distributions Transform), which handles multiple horizontal and multiple vertical scan lines, to robustly estimate the robot’s 3D position in real environments. Our proposed method estimates a specified floor in the building and determines a 2D localization on the floor. Based on the self-localization, the method detects depth variations by taking advantage of 3D LiDAR. Once our method detects some dynamic changes on patrol, it replaces the previous 3D points corresponding to the space in 3D map with the latest 3D points. As our approach estimates an accurate self-localization in the 3D map, the 3D map updated by the method is seamless without giving uncomfortable feeling. We demonstrate that our iterative update method is an effective way of successively renewing the 3D map for inside a building.

Paper No. 29

120

Multi-Postures Human Extraction Using Coarse-to-Fine Method and Human Skeleton Method

Yosuke Kinoshita, Hiroki Takahashi


[email protected], [email protected], SVM, Human Detection, Human Extraction, Pose Estimation, Openpose, Coarse-to-Fine Method, Human

Skeleton Method

This paper proposes a method to extract a precise multi-postures human region by two kinds of approaches which are Coarse-to-Fine Method and Human Skeleton Method.In the Coarse-to-Fine Method, first, human region is detected from the entire image by Coarse Detector.Then, precise human region is extracted from the candidate region by Fine Detector. In the Human Skeleton Method, first, human skeleton is obtained by Openpose. Then, human skeleton is dilated. Finally, human mask is extracted by Grabcut from the dilated skeleton. In the evaluation, F-measure is used for detection accuracy and Intersection over Union is used for extraction precision.Coarse-to-Fine Method gets 0.523 for detection accuracy and gets 0.807 for extraction precision.Human Skeleton Method gets 0.952 for detection accuracy and gets 0.868 for extraction precision.For future work, let the confidence map learn the recumbenting posture and evaluate human extraction result.

Paper No. 59

121

Gaze-based Visual Feature Extraction via DLPCCA for Visual Sentiment Estimation

Taiga Matsui¹, Naoki Saito¹, Takahiro Ogawa¹, Satoshi Asamizu², Miki Haseyama¹

Hokkaido University¹, National Institute of Technology, Kushiro College²


gaze information, visual sentiment estimation, canonical correlation analysis

This paper presents gaze-based visual feature extraction via Discriminative Locality Preserving Canonical Correlation Analysis (DLPCCA) for visual sentiment estimation. The proposed method calculates novel visual features reflecting users’ visual sentiment by applying DLPCCA to gaze and original visual features. Consequently, accurate visual sentiment estimation becomes feasible by utilizing the novel visual features derived by the proposed method.

Paper No. 60

122

Separating Information for Space-Time Hybrid Code

Tomohiro Hidaka, Keita Takahashi, Toshiaki Fujii

Nagoya University


Active Stereo, Depth Estimation, RGB Color Space, Linear Transformation

Active stereo is one of the means for estimating depth, which uses a camera and a projector. The projection patterns are classified into two groups: a time code or a space code. In recent years, a space-time hybrid code was investigated to fully utilize the advantages of both codes. With this method, one needs to separate the information from each captured image, on which the information for both codes are coexisting and mixing. In this paper, we propose to use the color space to successfully construct and extract the information for the time/space code.

Paper No. 67

123

A study on garment wrinkle detection by 3D camera scanning with polygon normal vector image and Hough transform - Differences in properties

depending on materials -

Makoto Takamatsu¹, Eizaburo Iwata², Takao Furukawa³, Makoto Hasegawa¹

Tokyo Denki University¹, Universal Robot Co., Ltd², Kyoritsu Women’s University³


Hough transform, wrinkle analysis, intrinsic material properties, 3D scanning polygon normal vector image

Wrinkles are significant visual features appearing on the surface of Garments. Visual details that appear on the garments for character modeling on computer graphics production. The appearance of wrinkles is different depending on the material; for example, wrinkles are different in position and direction. We discuss the intrinsic properties of wrinkles in the material. A person wears his or her garments made of some materials, and RGB - D camera scans his or her 3D shape of body, and a 3D computer graphics model of the body is obtained. Generating a normal polygonal image applying a baking process. Mapping images of coarse polygons. Wrinkles are described for the normal image. Garment wrinkles are detected by straight-line recognition with Hough transform. It is possible to measure the number of wrinkles, the variation in the direction of the wrinkle, and the distribution of the length of the wrinkle. In our experiments, a person puts on three kinds of materials, and we discuss about the properties depending on the difference in materials.

Paper No. 72

124

FOE-based Regularization for Optical Flow Estimation from an In-vehicle Event Camera

Jun Nagata¹, Yusuke Sekikawa², Kosuke Hara², Yoshimitsu Aoki¹

Keio University¹, Denso IT Laboratory Inc.²


Optical flow, Event camera, Focus of expansion

Optical flow estimation in onboard cameras is an important task in automatic driving and advanced driver-assistance systems. However, there is a problem that calculation is mistakable with high contrast and high speed. Event cameras have great features such as high dynamic range and low latency that can overcome these problems. Event cameras report only the change in the logarithmic intensity per pixel rather than the absolute brightness. There is a method of estimating the optical flow simultaneously with the luminance restoration from the event data. The regularization using the L1 norm of differentiation is insufficient for spatially sparse event data. Therefore, we propose to use the focus of expansion (FOE) for regularization of optical flow estimation in event camera. The FOE is defined as the intersection of the translation vector of the camera and the image plane. The optical flow becomes radial from the FOE excluding the rotational component. Using the property, the optical flow can be regularized in the correct direction in the optimization process. We demonstrated that the optical flow was improved by introducing our regularization using the public dataset.

Paper No. 73

125

Character Recognition of Modern Japanese Official Documents using CNN for Imblanced Learning Data

Zongjhe Yang, Keisuke Doman, Masashi Yamada, Yoshito Mekada

Chukyo University


convolutional neural network, handwritten character recognition, imbalanced learning data

The documents of the government-general of Taiwan recorded from 1895 to 1945 contain the whole of Japanese official documents before WW2, and have great historic value. The characters in the documents, however, are illegible because they were written by hand with a brush. It is labor-intensive work for historians or scholars to understand the documents. We propose a method for character recognition of these documents by using a convolutional neural network, and also conduct to solve the problem on imbalanced learning data. Experimental results show that the top-1 and the top-10 accuracy were 89.48% and 98.10%, respectively.

Paper No. 74

126

Discrimination of Drawing Collapse for Animated Characters by SVM

Jun Sakurai¹, Tomokazu Ishikawa², Yusuke Kameda¹, Ichiro Matsuda¹, Susumu Itoh¹

Tokyo University of science¹, Toyo University²


Support vector machine (SVM), Drawing collapse, Face of animated character

In general, ‘drawing collapse’s a word used when very low quality animated contents are broadcast. For example, perspective of the scene is unnaturally distorted and/or sizes of people and buildings are abnormally unbalanced. In our research, possibility of automatic discrimination of drawing collapse is explored for the purpose of reducing a workload for content check typically done by the animation director. In this paper, we focus only on faces of animated characters as a preliminary task, and distances as well as angles between several feature points on facial parts are used as input data. By training a support vector machine (SVM) using the input data extracted from both positive and negative example images, about 90% of discrimination accuracy is obtained when the same character is tested.

Paper No. 90

127

Improvement of Authenticity Inspection Accuracy Using Logo Region Detection

Ryo Inoue¹, Tomio Goto¹, Satoshi Hirano¹, Son Pung²

Nagoya Institute of Technology¹, University of Wollongong²


Feature point matching, Template matching, Sobel filter

In recent years, manufacturing technology of counterfeit brand products has advanced, and it is becoming very difficult for humans to distinguish many counterfeit products. In this paper, we propose an authenticity inspection method for brand items by utilizing image matching. In the experiment, we confirm the effectiveness of logo region detection processing using edge images as preprocessing of image matching with the aim of improving inspection accuracy of images containing many background components. Experimental results show that adding the logo region detection processing as preprocessing reduces the influence of background components and enables more accurate inspection.

Paper No. 95

128

Horizontal Line Detection using Genetic Algorithm

Uuganbayar Ganbold¹, Takuya Akashi¹, Chao Zhang², Hiroyuki Tomita³, Hitoshi Kubota³

Iwate university¹, University of Fukui², Suzuki Motor Corporation³


horizontal line detection, genetic algorithm, mobile robots, autonomous vehicles

In mobile robots and autonomous vehicles, to determine the region of the ground plane is necessary for the estimation of the environment. In this paper, we propose a method for the detection of Horizontal Line (HL) from image data observed with a single camera. HL separates the ground plane from its surroundings. In this paper, detection of HL is treated as an optimization problem and the genetic algorithm (GA) is adopted to optimize the line. We implement the real-time detection system of HL with GA and evaluate it with multiple video sequences under various environment. The detection results demonstrate the efficiency of the proposed method.

Paper No. 105

129

Accuracy Improvement of Depth Estimation with Tilted Optics by Optimizing Neural Network

Hiroshi Ikeoka¹, Takayuki Hamamoto²

Fukuyama University¹, Tokyo University of Science²


depth estimation, distance estimation, tilted optics, defocus, neural network, deep learning

We propose a depth estimation method, which utilizes a monocular aperture camera with two tilted optics, two image sensors, and a spectroscopic mirror. By using this method, it is possible to achieve depth estimation that satisfies needs such as 1) real-time processing, 2) robustness, and 3) adaptive estimation range and resolution for use in automotive tasks.

Paper No. 112

130

Generating Arbitrarily Focused Images from Sparse Focal Stack Through Light Field Reconstruction

Daiki Ito, Keita Takahashi, Toshiaki Fujii

Nagoya University


Light field, Focal stack, aperture shape

We can generate arbitrarily focused images using light field, while we need specific device such as light field cameras to directly capture the light field. Our method can compute arbitrarily focused images from a sparse focal stack, which can be reasonably captured using conventional camera. Additionally, our method can change aperture shape after taking a photo. Experimental results show that our method can generate images focused on arbitrarily depth using arbitrarily aperture shape.

Paper No. 120

131

Multi-kernel based Deep Residual Networks for Image Super-Resolution

Jae Woong Soh, Gu Yong Park, Nam Ik Cho



Single image super-resolution (SISR) Convolutional Neural Networks (CNN), dilated convolution, residual learning

Single image super-resolution (SISR) is a classical problem in low-level vision task, which aims to find a mapping function from a single low-resolution (LR) input to corresponding high-resolution (HR) output. Recently, deep networks has achieved great success in SISR task. Recent successful deep models mostly consist of stacked same size convolution filters, where their size is 3 by 3. To cope both local dependencies and global contexts between LR and HR images, we propose multi-kernel based deep residual networks for SISR. Since, large size of kernel requires larger parameters, we adopt dilated convolution as our solution to increase size of the receptive field. Also, we adopt local feature fusion, global feature fusion and local residual learning for control the multi scale features and the better performance by accelerating the convergence. Experimental results show that our proposed model yields improved performance.

Paper No. 132

132

3D Scene Reconstruction and Object Recognition for Indoor Scene

Yangping Shen, Yoshitsugu Manabe, Noriko Yata

Chiba University


SLAM, 3D object recognition, scene understanding, semantic mapping, RTAB-map

In recent years, many SLAM (simultaneous localization and mapping) systems have appeared showing impressive dense scene reconstruction. However, the normal SLAM system builds 3D scene in point level without any semantic information. Many computer vision applications require high ability of scene understanding and point-based SLAM shows insufficiency in these applications. This paper studies about fusing 3D object recognition into SLAM system, using hand-held RGB-D camera and RTAB-Map to reconstruct dense point cloud of 3D indoor scene. Then we use supervoxel-based point cloud segmentation approaches to over-segment the scene. 3D object classification model trained by PointNet is added to merge segmentation parts and object recognition. Our experiment on indoor environment shows effectiveness of this system.

Paper No. 152

133

An Investigation of Multiplication Error Tolerance in CNN and SIFT

Chanon Khongprasongsiri, Watcharapan Suwansantisuk, Pinit Kumhom

King Mongkut’s University of Technology Thonburi


Approximate Computing, Approximate Algorithm, Approximate Multiplication

Since the demand of computer vision algorithm computation is steadily increase due to its keys computation which is multiplication is the bottlenecks. To reduce the computation, the approximate multiplication plays role in computer vision algorithm because its multiplication can be approximate. In this article, we investigate the error tolerance of the convolutional neural network and SIFT algorithm due to its application in terms of how it can be approximate by injecting a random value to each of precise multiplication value.

Paper No. 154

134

Range image registration based on the consistency of rigid transformation

Masaki Yoshii, Ikuko Shimizu

Tokyo University of Agriculture and Technology


Range image registration, Rigid transformation consistency, Quaternion

Range image registration is an essential technique for 3D modeling of the real world object from its range images captured by a 3D sensor.In this paper, we propose a method for range image registration based on the consistency of the rigid transformation between corresponding point pairs to improve accuracy. The consistent rigid transformations are near in the parameter space.We generate triplets of corresponding point pairs and evaluate the consistency because at least three corresponding point pairs are required to esitimate a rigid transformation.We select the triplet which have the largest number of consistent triplets and estimate the rigid transformation using the selected triplet and its consistent triplets.Experimental results using the synthetic data with the known rigid transformation shows that results by our method was better than the conventional methods.

Paper No. 181

135

Blind Detection of Fabric Defects Using Multiple Image Features

Karam Park, Sang Hwa Lee, Nam Ik Cho



Fabric defect detection, frequency component, color histogram, edge orientation, machine learning

This paper proposes a blind detection of fabric defects using multiple image features. The aim of proposed method is to detect the new types of defects and fabric structures which have not been learned. This paper first learns the general characteristics of image feature vectors of defect and normal image patches using frequency coefficients, color histogram, and edge orientations. The mean vectors of image features are calculated and the distances between the mean and the feature vectors of image patches are learned using support vector machine (SVM). Since the defect decision boundaries are determined from the general characteristics of image features in the defect and normal patches, the proposed detection method is very robust to the new types of defect and fabric structures. According to the real fabric images that have not been trained, the proposed method detects fabric defect with 96.4% accuracy at 0.03% errors.

Paper No. 182

136

Defective Products Detection using Adversarial AutoEncoder

Shunsuke Nakatsuka, Hiroaki Aizawa, Kunihito Kato

Gifu University


visual inspection, neural networks, small number of defective samples, anomaly detection

In this paper, we aimed at discrimination of defects under conditions where there is a large number of good products and a small number of defective products. By combining AAE, which can extract features following any distribution and Hotelling’s T-Square, which is an effective anomaly detection method when data follows a normal distribution, it is possible to discriminate defects under a small number of defective samples.

Paper No. 195

137

A consideration of writer identification using disentangled features independent of character classes

Tomoki Yamada¹, Mariko Hosoe², Kunihito Kato¹, Kazuhiko Yamamoto¹

Gifu University¹, Gifu Pref. Police H.Q. Forensic Science Lab.²


Writer identification, Deep Learning, Variational AutoEncoder

Handwriting identification is one of the active areas of research. It is important to prepare a large number of characters of the same class for improving the accuracy of handwriting identification. However, it is not always possible to prepare enough characters of the same class. In this case, they compare different classes of characters and analyze using common handwriting parts for each character. However, it is very difficult. Therefore, we assume that handwriting characters written by the same writer have features independent of character classes. In this paper, we propose methods to extract features that independent of character classes using deep learning. We used Conditional Variational AutoEncoder (CVAE) as a learning method. A writer identification experiment shows that these methods can extract independent features of character classes, and extracted features are useful in handwriting identification. Furthermore, we examined the relation between human interpretation of character features and accuracy of writer identification by using character features extracted by disentangled feature extraction methods.

Paper No. 196

138

Finding a Rush-out Human Employing a Human Body Direction Detector

Yuta Ono, Joo Kooi Tan

Kyushu Institute of Technology


MSC HOG, human detection, body direction, rush-out, car vision

Recently, along with rapid development of the image processing technology, image processing has been adopted in various fields for various purposes. Development of an intelligent machine that mounts a camera as an eye is a thriving technology, and it is employed not only in industrial fields but also in the fields involving ordinary citizens. Especially, development of Intelligent Transportation Systems is very active and many methods of detecting human and automobiles have been proposed using laser radars, LIDARs or in-vehicle cameras. However, they remain only on the detection of the presence of such objects and the methods to detect rush-out objects into a road have not been developed yet.In this paper, a method is proposed which detects a human from an image with his/her body direction information. This intends to detect a human who might rush out into a road in front of an ego car. In order that the human model used for extracting the feature may capture the appearance of human rush-out properly, directional human models and classifiers are introduced. The proposed method is examined its performance experimentally.

Paper No. 207

139

Fall-down Event Detection for Elderly Based on Motion History Images and Deep Learning

Wen-Nung Lie, Fang-Yu Hsu, Yuling Hsu

National Chung Cheng University


Fall-down event detection, human action recognition, deep learning, motion history image (MHI)

The goal of this research is to apply the state-of-the-art deep learning approach to human fall-down event detection based on Motion History Images (MHI) from multiple color video sequences captured at different viewing angles. MHI is derived by detecting and combining temporal 2D human contours from surveillance cameras. A human action can then be represented by several continuous MHI images. We then use deep learning approach (CNN + LSTM architectures) to recognize the fall-down behavior from MHI sequences. Our method is capable of not only recognizing the actions of walking, standing, falling down, but also rising after falling down to avoid excessive false alarms. The accuracy of classification into the above 4 short-term actions is capable of achieving 97.66%. We also compare the performances of deep learning architectures that use simple CNN or CNN+LSTM, one or two-stage training, and single or two cameras. Our contributions lie on two aspects: (1) improving the performance on human action recognition based on MHIs and a combination of CNN+LSTM architecture, (2) preventing the false alarm of falling-down events that actually need no help.

Paper No. 213

140

Development of a Human-Robot Cooperative System Based on Visual Information

Akitoshi Sato, Joo Kooi Tan

Kyushu Institute of Technology


Cooperative systems, Human care, Autonomous robots, Depth, Segmentation

In recent years, along with the aging of society worldwide, decrease of working population has become a serious problem. For this reason, robots are expected to substitute human work, automate distribution, and support human daily life, especially, elderly care assistance. In this research, we focus on the support of those who need care in everyday life, and, in this paper, we propose a human-robot cooperative system that supports acquisition of objects in cooperation with a human. The outline of the action of this robot is as follows: (i) It moves to the location designated by a user autonomously (not by remote control): (ii) On arrival, the robot exchanges information by video with the user who is at a remote place, and acquires the objects designated by the user among those placed there. (iii) After the acquisition, the robot moves again autonomously to the user and hands over the objects to the user. In this paper, we focus on step (ii) and show some experimental results.

Paper No. 214

141

Examining Performance of Sketch-To-Image translation models with Multiclass Automatically Generated Paired Training Data

Dichao Hu

Georgia Institute of Technology

[email protected]

image-to-image translation, generative adversarial network, paired data generation

In computer graphics and computer vision, a special type of task involves translating one representation of the scene into another. This type of task can be named as image-to-image translation. Various approaches have been proposed to handle this type of task and have achieved highly desirable results. Nevertheless, the accomplishment of this task requires manually designed paired training data which are expensive to acquire. Therefore, models for image-to-image translation are usually trained on a small set of paired training data which are carefully designed. Our work is focused on automatic generation for a special type of image-to-image translation, which is the sketch-to-image translation. We propose a method to generate fake sketches from images using an adversarial network and then pair the images with corresponding fake sketches to form large-scale multiclass paired training data for training a sketch-to-image translation model. We are also going to use one specific type of sketch-to-image translation model called pix2pix network, which is based on conditional adversarial network and is capable of handling different image-to-image translation tasks. Although pix2pix network has been demonstrated to be able to achieve amazing results, our approach is different since we are testing its performance in another setting where both the scale and the class diversity of training data are significantly increased. The new dataset now contains 61255 image-and-sketch pairs from 256 different categories.

Paper No. 217

142

A 3D Reconstruction Method Using PMVS for a Limited Number of Viewpoints

Junichi Hata, Koichi Ito, Takafumi Aoki

Tohoku University


3D reconstruction, free-viewpoint image, patch-based multi-view stereo, phase-only correlation

Patch-based Multi-View Stereo (PMVS), which is an accurate 3D reconstruction method, can reconstruct the accurate and dense 3D shape from enough number of viewpoints. On the other hand, its accuracy is degraded when using only a limited number of viewpoints. Addressing the above problem, this paper improves the PMVS so as to reconstruct the 3D shape from a limited number of viewpoints. Through a set of experiments, we demonstrate that the proposed method exhibits the efficient performance on 3D reconstruction such difficult situations.

Paper No. 218

143

Predicting Visual Saliency via a Dilated Inception Module-based Model

Sheng Yang, Weisi Lin



Visual attention, saliency detection, eye fixation prediction, convolutional neural networks, dilated inception module

The early computational visual saliency models developed to predict human eye fixations cannot match the capacity of human eyes with high efficiency and efficacy. With the advent of deep convolutional neural networks (DCNN), the improvements in visual saliency prediction research are impressive. Despite this, it is still needed to fully characterize the multi-scale saliency-influential factors into the current deep saliency framework for further improvement. However, the existing approaches aiming at capturing multi-scale contextual features either suffer from the heavy computation or limited performance gain. To overcome this, a lightweight yet powerful module for fully exploiting multi-scale contextual features is desired. In this paper, we propose a DCNN-based visual saliency prediction model to approach this goal. Our model is inspired by the GoogleNet, which use the inception module to capture multi-scale contextual features at various receptive fields. Specifically, we revise the original inception module to have more powerful multi-scale feature extraction capacity and less computation load by utilizing dilated convolutions to replace the original standard ones. The whole model is trained end-to-end and is efficient to achieve real-time performance. Experimental results on several challenging saliency benchmark datasets, including SALICON, MIT1003, and MIT300, demonstrate that our proposed saliency model can achieve state-of-the-art performance with competitive inference time.

Paper No. 219

144

Single Image Dehazing Algorithm Based on GANs

Shi Chengyu¹, Wang Tingting¹ and Zhao Haiying²

Beijing University of Posts and Telecommunications¹, Mobile Media and Cultural Computing Key Laboratory of Beijing.Century College²


Single image dehaze, GANs, perceptual losses

In the construction of specimen library, in order to solve the problem of haze image blurring, the related research on image dehazing is carried out. The paper proposes an image dehazing algorithm based on generative adversarial network (GANs). The generator adds Skip Connection to the residual block (ResBlock), the discriminator adopts PatchGAN, and the perceptual loss is added to the loss function. The paper finally compares and analyzes the experimental results through comparative simulation experiments, then conclude that the proposed algorithm can effectively remove the effect of haze on the image.

Paper No. 226

145

Automated Hand Eye Calibration in Laparoscope Holding Robot for Robot Assisted Surgery

Jiang Shuai¹, Hayashi Yuichiro¹, Oda Masahiro¹, Kitasaka Takayuki², Misawa Kazunari³, Mori Kensaku⁴

Nagoya University¹, Faculty of Information Science, Aichi Institute of Technology², Aichi Cancer Center Hospital³, Nagoya University/National Institute of Informatics⁴



Robot surgery, Hand eye calibration, Laparoscope

In this paper, we describe an automated hand eye calibration in laparoscope holding robot for robot assisted surgery. In minimally invasive surgery, laparoscope holding robot can give more stability of the laparoscope images than the laparoscope assistants. We study on laparoscope holding robot controlled based on anatomical structure information during laparoscopic surgery. In order to operate laparoscope holding robot guided by image, it is necessary to make a vision system for laparoscope holding robot. We compute the position and orientation relationships between laparoscope camera and Tool Center Point (TCP) of robot arm to make a vision system. We utilize Tsai’s method for hand eye calibration to estimate the homogeneous transformation matrix between TCP and laparoscope camera. We attached laparoscope to industrial robot arm. Robot arm is moved to different position and capture calibration board images. Hand eye calibration is performed using recorded TCP positions and calibration board images.

Paper No. 228

146

Vision-based Hand Gesture Recognition from RGB Video Data Using SVM

Fahmid Al Farid, Noramiza Hashim, Junaidi Abdullah

Multimedia University


Vison-based, Gesture Recognition, RGB Video Data, SVM

With the growing number of population in the world nowadays, novel human-computer interaction systems and techniques can be used to help improve their quality of life. A gesture-based technology can help to maintain the safety and needs of the disable as well as the general people. Gesture recognition from video streams is a challenging task due to the high changeability in the features of each gesture with respect to a different person. In this work, we propose a vision-based hand gesture recognition from RGB video data using SVM.Gesture-based interfaces are more natural, spontaneous, and straightforward. Previous works attempted to recognize hand gesture for different scenarios. Throughout our studies, gesture recognition system can be based on wearable sensor or it can be vision based. Our proposed method is applied to a vision-based gesture recognition system. In our proposed system image acquisition starts from RGB videos capture using the Kinect sensor. We convert the image frames from videos to blur for background noise removal. Then, we convert the images into hsv color mode. After that, we do the dilation, erosion, filtering, and thresholding the image for converting to black and white format. Finally, using the prominent classification algorithm SVM, hand gesture has been recognized. In conclusion, the framework aims to create a better vision-based hand gesture recognition system compared to the state-of-the-art methods.

Paper No. 241

147

The depth recovery from multiple optical flow detected by fixational eye movements camera

Yuichiro Deguchi, Norio Tagawa

Tokyo Metropolitan University Graduate School


Three-dimensional shape restoration, optical flow, random camera motions, fixational eye movements

Recent advances in computer vision technology are remarkable. Among them, in the field of three-dimensional shape restoration, stereo stereoscopic view which calculates the depth of an object by the principle of triangulation is the mainstream. Although stereo stereoscopic viewing has high depth restoration accuracy, concealment (occlusion) and change in appearance are likely to occur. Therefore, in this research, we focused attention on the depth restoration method by motion stereopsis with monocular eye which is hard to cause the problem described above. For motion stereopsis, motion parallax between consecutive images needs to be small, and in general, restoration accuracy is low. Therefore, motion stereoscopic viewing using a large number of continuous image pairs obtained by camera motion simulating random eye movements called fixational eye movement is performed, and aim to improve depth restoration accuracy by increasing information of depth corresponding to each pixel.

Paper No. 243

148

Octal gray-code phase unwrapping for high-speed fringe projection profilometry

Xiaoyu He, Kemao Qian



high-speed fringe projection profilometry, Phase unwrapping, Octal gray code, Projector defocusing

The speed of a fringe projection profilometry system is of great importance for the measurement of dynamic objects. The previous quaternary gray-code phase unwrapping method uses only five patterns to recover an absolute phase which is suitable for high-speed measurement. However, this method requires a pre-knowledge of the minimum depth and it limits the object to move in a small range of depth. To extend the measurement range, we proposed an octal gray-code phase unwrapping method, in which five patterns are required. We apply this method to different objects and analysis the experiment results. This method can recover a good-quality phase map for sample objects, while its performance is decreased when the object is complex or with an improper defocusing level.

Paper No. 251

149

Image/video Processing

A Noise Reduction Method for AWGN Images using Bilateral Filter and Image Fusion

Hyunho Choi, Seungwoo Wee, Jechang Jeong

Hanyang University


Additive white gaussian noise, Bilateral filter, Image fusion

In this study, we propose an algorithm that effectively remove additive white gaussian noise (AWGN) from images using a combination of bilateral filter and image fusion. The proposed method uses the bilateral filter that reduces noise by utilizing both domain and range filters during image processing process. The bilateral filter is applied to a source image. The subsequent process applies averaging at sample locations of the bilateral filtering image. Averaging represents the performance of noise reduction and stability where bilateral filtering image contains similar information. To evaluate performance of the algorithm, mean square error (MSE), signal to noise ratio (SNR) and peak signal to noise ratio (PSNR) were used. These results demonstrate superior noise reduction performance compared to the conventional methods.

Paper No. 9

150

FIsh Monitoring in Complex Environment

Ling Jun, Lau Phooi Yee

Universiti Tunku Abdul Rahman


fish monitoring system, complex environment, uneven illumination, contour feature

To satisfy consumer demands for seafood, mainly fishes, aquaculture farm provides a solution for the overfishing demands. However, in order to maintain and monitor big scale farm, requires going through hours of video footages for meaningful information such as the estimated number of fishes and size, which can be a demanding task for human operators, who are also affected by fatigue and error. Therefore, automated image processing techniques could benefit fish farms in their fish monitoring processes. This paper proposes a non-invasive method of counting fishes and estimating their sizes using image processing techniques to extract meaningful information. The method proposed by this paper includes three main parts, to compensate the uneven illumination of input image, region segmentation, and fish detection. Some of the issue that arise in detecting fishes using computer vision is the inconsistent lighting in the background, therefore, input video is needed to go through illumination compensation to compensate for the overall footage illumination, for this paper, an x and y Gaussian technique is used to create a mask, which can be seen in Figure 2b, which is later subtracted from the original footage. After that, region segmentation is done to differentiate between foreground and background objects, by thresholding and morphological operations, which can be seen in Figure 2c. Next is to use visual features to differentiate objects that are not fishes, with objects that are fishes. Some visual features that are found in fishes are used to determine that, the more a detected object’s features matches with the set visual features signifying a fish, the higher its probability of being a fish. There are five visual features that are used, namely: 1) Elongated Structure 2) Downward Curve of Fish 3) Downward Convexity Angle 4) Upper Curve of Fish, and 5) Differenced Pixel Value, A more detailed explanation to which how each featured is implemented and why would be further elaborated within the report. Later on, decision making module proceeds, a counter is used to keep count of the number of visual features a detected object match, when it reaches a certain threshold, the object is considered a fish and is counted and sized. This paper also includes discussion and experimental validation for the proposed method.

Paper No. 10

151

A Stereo Images Generating System Considering Both Translation and Rotation of Objects

Yuan-Mau Lo¹, Chin-Chen Chang², Der-Lor Way³, Zen-Chung Shih¹

National Chiao Tung University¹, National United University², Taipei National University of Arts³


Stereo images, View synthesis, Neural network

In this paper, we propose a system which can generate stereo images from a single image considering both translation and rotation of objects in the image. We segment the image and estimate the depth map to convert the task of view synthesis from a scene to objects. Our modified appearance flow network is more general and suitable for our system. We also use the reference image to improve the inpainting method. The quality of our results is better than those using the traditional warping. Our results can better keep the structure of objects in the input image. In addition, our system does not limit the size of the input image. Most importantly, due to considering the rotation of objects, our results are more stereoscopic while watching them with a device.

Paper No. 23

152

Fast Median Filter with Various Kernel Sizes

Koki Horiuchi¹, Wenbo Jiang², Shuichi Yamagishi², Xiaohua Zhang²

Hiroshima Institute of Technology¹, Xihua University²


Image processing, Median filter, Relative total variation, Various kernel scale, Local histogram

As a best known filter in the category of order statistic filters, median filter has been widely employed in the fields of image processing, computer graphics and computer vision etc. This paper proposes an approach for implementing the median filter with varying kernel sizes at each different pixels. The proposed algorithm consists of two stages. Firstly, the kernel sizes at each pixel are computed using a kernel scale estimation scheme based on directional relative total variation and flatness collection. The flatness is large within flat or textured region and small around structure edges as well as sharp corners. Secondly, by maintaining a column histogram at the current kernel scale for each pixel in the consideration, the kernel histogram is computed by adding and subtracting operations. With comparing to the brute-force implementation, the experiments show that the proposed algorithm is very fast and effective.

Paper No. 26

153

Structure Tensor Based Anisotropic Rolling Filter for Image Smoothing

Kouichirou Yoshimura¹, Yuelan Xin², Ning Xie³, Xiaohua Zhang¹

Hiroshima Institute of Technology¹, Qinghai Normal University², University of Science and Technology of China³


Image processing, Rolling guidance filter, Anisotropic rolling filter, Structure tensor

Filtering image by eliminating irrelevant details as could as possible while preserving structure edge and corner becomes very important in the fields such as image processing and computer vision etc. In this paper, we presents an anisotropic rolling filter for smoothing image while preserving important structure edge and corner. The proposed filter implements an extended cross bilateral filter, in which the range weight is updated in an iterative manner, while the spatial Gaussian weight is computed in anisotropic directions instead of isotropic directions. The anisotropic directions are computed based on structure tensors which are calculated at each pixel to determine structure orientations. Compared with the original rolling guidance filter, it is found that the proposed anisotropic rolling filter has stronger smoothing and structure preserving ability.

Paper No. 27

154

A Weight Calculation Method for Multiscale Normalized Cut Image Segmentation

Koya Kato, Kazu Mishiba, Katsuya Kondo

Tottori University


segmentation, superpixels, MNCut, two-hops, Matting Laplacian

In this paper, we propose weight calculation methods to solve a problem of Multiscale Normalized Cut (MNCut) image segmentation. Since Malutiscale is incorporated with various information, robust segmentation results can be expected for various RGB images. However, MNCut has a problem that if the foreground or background is separated into two or more regions, they may be classified into different labels. To solve this problem, we compute edge weights between not only one-hop but also two-hops superpixels within the same scale. Furthermore, we use Matting Laplacian for edge weight calculation in order to realize better segmentation. In the experimental results, the proposed method obtained results closer to the ground truth.

Paper No. 31

155

Pastel Color Imaging and Its Evaluation using Video Coding

Go Kasahara, Kenji Sugiyama

Seikei University


Imaging, Color Filter Array, Video Coding, Pastel Color

In the color imaging, single sensor system with color filter array (CFA) is popular, especially with Bayer CFA. Recently, the method using white component have been studied to improve the resolution and the sensitivity. On the other hand, the performance of de-mosaicking is improved by using correlation of color planes. However, it is mostly for Bayer CFA. We have reported the CFA that has more green pixels to improve luminance resolution with high level de-mosaicking. In this paper, we do not change the array, but color of filter is changed to pastel color. About the color plane reconstruction, the method that abstract the average component of RAW image is proposed. We also propose the color difference method and high resolution reconstruction of the other color to get full color. In the experiments, we evaluate the reconstruction performance of RGB images and the overall performance include video coding. From the experiment, Y-PSNRs of proposal have the advantage. With the coding, the bitrate reduction is up to 30% at the same PSNR, because of less alias component.

Paper No. 34

156

A Retinex-based Image Enhancement Scheme with Noise Aware Shadow-up Function

Chien-Cheng Chien, Yuma Kinoshita, Sayaka Shiota, Hitoshi Kiya

Tokyo Metropolitan University


Contrast enhancement, Image enhancement, Noise aware, Shadow-up function, Retinex

This paper proposes a novel image contrast enhancement method based on both a noise aware shadow-up function and retinex (retina and cortex) decomposition. Under low light conditions, images taken by digital cameras have low contrast in dark or bright regions. This is due to a limited dynamic range that imaging sensors have. For this reason, various contrast enhancement methods have been proposed. The proposed method can enhance the contrast of images without not only over-enhancement but also noise amplification. In the proposed method, an image is decomposed into illumination layer and reflectance layer based on the retinex theory, and lightness information of the illumination layer is adjusted. A shadow-up function is used for preventing over-enhancement. The proposed mapping function, designed by using a noise aware histogram, allows not only to enhance contrast of dark region, but also to avoid amplifying noise, even under strong noise environments.

Paper No. 37

157

Dense learning by high dimensional SOMs composed of input-output fusion vectors for interactive image segmentation

Hotaka Takizawa

University of Tsukuba

[email protected]

Dense learning, High dimensional self organizing maps, Input-output fusion vectors, Interactive image segmentation

This paper proposes an interactive image segmentation method based on high dimensional self organizing maps (SOMs). The proposed method was applied to color images. The experimental results demonstrated that higher dimensional SOMs were able to achieve more accurate segmentation.

Paper No. 38

158

Study on restoration method of deterioration in photographed images of cultural properties

Kenji Nishikane, Yuka Nakamura, Shuichi Yamagishi, Xiaohua Zhang, Kosuke Kato, Shimpei Matsumoto

Hiroshima Institute of Technology



machine learning, texture synthesis, binarization, restoration, deterioration, photographed image

We propose a method to restore the original figures of cultural properties in their photographed images by using extraction of deteriorated parts and texture synthesis. Each block divided from the image is judged whether it involves a crack or not by using machine learning. The cracks in the selected blocks are extracted by the binarization of the pixel values and restored by texture synthesis technique. It has been confirmed by means of computer simulational experiments that our proposed method has shown good results.

Paper No. 39

159

Post-Processing for Disparity Map Enhancement Using Modified Guided Image Filter

Yong-Jun Chang, Yo-Sung Ho

Gwangju Institute of Science and Technology


stereo matching, disparity error detection, disparity error refinement, guided image filter, hole filling

Stereo matching methods estimate depth information from stereo images using characteristics of binocular disparity. Those methods search corresponding points between left and right viewpoint images. After that, a disparity value of two corresponding points is calculated. The disparity value represents depth information of those corresponding points. Since the disparity value is calculated by searching for corresponding points, it is possible to estimate the accurate disparity value in textured regions. On the other hand, the inaccurate value is estimated in the textureless region. This problem generates disparity errors in that region. In addition, an occlusion problem also makes disparity errors. In this paper, a post-processing method using a modified guided image filter is proposed to enhance the disparity map.

Paper No. 40

160

The Fault Tolerant System for Real-time Video Processing by Software

Isao Nishihara

Toyama Prefectural University

[email protected]

Fault tolerant design, Real-time processing, Non-stop system, Software method, Video processing

In general, there is a service that needs to correspond anytime and anywhere. However, fault tolerant systems with fully redundant hardware used for business are very expensive and cost effective. In addition, problems such as occurrence of down time of the entire system due to maintenance and switching to the backup system occur, and it is a problem that it is difficult to secure a bypass route. In this paper, the method to construct a fault tolerant system at low cost only by software processing is proposed. Especially focused on video processing. Even if arbitrary images are simultaneously displayed by a plurality of processes as the characteristics of the image, the influence on the entire display is not very large. Therefore, images simultaneously processed by a plurality of pipelines are collectively displayed. As a result, we constructed a pipeline in which a series of moving image processing is processed without stopping. As a basic system, we constructed a system that duplicated simple image processing, and confirmed that there is no problem even if one side stops.

Paper No. 54

161

The Non-stop Video Transmission Method with Multipath Communication

Isao Nishihara


[email protected]

Video transmission, Non-stop system, Multipath communication, Real-time processing

As the Internet has become widespread, data transmission is becoming possible anywhere with wired connections, satellite links and mobile lines. On the other hand, in natural disasters and in order to know the situation of places where people cannot enter, it is necessary to take videos at the place and transmit them to the people. However, since the line is best-effort and the cost is high, cost performance is very poor in order to always carry out video transmission. In this paper, the video transmission method is considered using multiple low-speed paths, especially with mobile lines. First, by using a plurality of low-speed lines, securing the transmission band as a whole until it is sufficient for video transmission. Further, by using a plurality of lines, even if one line is disconnected, data transmission can be continued by the remaining lines. Further, there are differences in delay time and transmission band between pluralities of lines. The mechanism to realize stable video transmission under these various conditions is proposed. In the proposed method, Motion JPEG is first used simply. Data of one frame is transmitted per line. Display the data that was completely transmitted at the timing. Incomplete data is displayed waiting for timeout, depending on timing, coded in half. Timing is synchronized with the line with the slowest delay time. The future task is to use JPEG 2000 with higher efficiency. We also consider a method for video codec using inter-frame dependent information such as H.264. In addition, we will consider mechanisms that complement mutual data loss between lines.

Paper No. 55

162

Sign Language Words Annotation Assistance using Binary Action Segmentation based on SVM and Graphcuts

Natsuki Takayama, Hiroki Takahashi



annotation assistance, Graphcuts, Japanese sign language words, SVM

This paper describes one of the assistant methods for annotation tasks of Japanese sign language words using automatic binary motion segmentation of video frames. The proposed method is composed of body parts tracking, feature extraction, and two step-wise segmentation based on Support Vector Machine and Graphcuts. This paper reports the evaluation results of the segmentation performance using a video database.

Paper No. 56

163

Field Position Estimation in Soccer Videos Using Convolutional Neural Network-based Image Features

Genki Suzuki, Sho Takahashi, Takahiro Ogawa, Miki Haseyama

Hokkaido University


Soccer videos analysis, CNN, DELM, deep learning.

This paper presents an estimation method of field positions in soccer videos using Convolutional Neural ¥mbox{Network (CNN)}-based image features. CNN-based features have been known to be effective for tasks in machine learning.Therefore, the proposed method adopts image features based on CNN, one of the deep learning methods.By using the extracted image features, the proposed method enables the estimation of soccer field positions.Specifically, the estimation of the field positions is realized via a classifier using the obtained image features.We show the effectiveness of our method via experiment results using actual soccer videos.

Paper No. 81

164

Fast Image Dehazing Based on Multi-Scale Guided Filtering

Van-Thuong Nguyen, Gia-An Vien, Chul Lee

Pukyong National University


Image dehazing, image enhancement, image restoration

We propose a fast image dehazing algorithm based on multi-scale guided filtering. We first construct an image pyramid from a hazy input image. Then, based on the observation that the most time-consuming procedure is the estimation of the transmission map and atmospheric light, we estimate them only at the coarsest level. Then, we obtain the transmission map at the finest level by upsampling and guided image filtering. Experimental results show that the proposed algorithm provides real-time performance, while providing comparable or even higher performance than the state-of-the-art algorithm.

Paper No. 83

165

Multi-View Image Coding Using Disparity-Compensated and Weighted Binary Patterns

Van-Thuong Nguyen, Gia-An Vien, Chul Lee

Nagoya University


Light-field, Disparity, Binary pattern

We proposed an efficient coding scheme for dense light fields, i.e., a set of multi-view images taken with very small viewpoint intervals. The key idea behind our proposal is that light fields are represented only using some weighted binary images. This coding scheme is completely different from modern video codecs, but it has some advantages. For example, the decoding process is extremely simple, which makes a faster and less power-hungry decoder. Moreover, we found that our scheme can achieve comparable rate-distortion performances to that of modern video codecs when using datasets with small disparities. However, it is difficult to express regions with larger disparities using a few common binary images. Therefore, in this paper, we extend our previous model with disparity-compensation and show its effectiveness.

Paper No. 85

166

Proposal of Lie Detection Method without Sensors

Katsuo Nakade, Takayuki Nakata



lie detection, color camera, emotion recognition

In this research, we propose a new lie detection method. Our method does not require attaching sensors to a subject because we use camera. We measure face color when subject tell a lie and tell a truth. Through experiments, we discovered when humans tell a lie, luminance of face sometimes decrease. However, it did not happen every time. We have to reconstruct the experimental environment. In the future, we need to clarify the relationship between lying and face color.

Paper No. 89

167

A Machine Learning Based Post Filtering Method Utilizing Block Boundary Information in HEVC

Yuya Yamaki, Yusuke Kameda, Ichiro Matsuda, Susumu Itoh

Tokyo University of Science


Post filtering, coding artifacts, machine learning, lossy image coding, support vector machine (SVM)

We previously proposed a machine learning based post filtering method for reducing image artifacts caused by lossy compression. The method classifies reconstructed image samples into three categories using a support vector machine (SVM) to roughly discriminate magnitude of the reconstruction errors. Then, an optimum offset value is added to the samples belonging to each category in a similar way to the sample adaptive offset (SAO) specified in the H.265/HEVC standard. In this paper, two kinds of SVMs are adaptively switched according to block boundaries of transform units (TUs) in HEVC intra-frame coding. Furthermore, feature vectors at the block boundaries are rotated before feeding to the SVM to properly capture local characteristics of the reconstruction errors.

Paper No. 91

168

Image Optimization Method for Large Autostereoscopic Display of Dual Projection Types

Yuta Kuroda, Isao Nishihara, Takayuki Nakata

Toyama Prefecture University


autostereoscopic displays, light fields, additive layer

In this research, in order for autostereoscopic display of dual projection types to display the image with larger parallax, we improved a new optimization method of the two projector images. We achieved that aim by deriving a new update formula in addition system and adjusting the target light field and weight matrix to create the area that saturates light.

Paper No. 94

169

A Development of Content-based Video Summarization System Using Machine-Learning’and its Application to Analysis of Livestock Behavior

Que Zhi¹, Tomoko Saitoh², Mizuki Nakijima¹, Tsuyoshi Saitoh¹, Masahiro Yamada¹, Hidetaka Arimura¹

Tokyo Denki University¹, Obihiro University of Agriculture and Veterinary Medicine², Kyushu University¹


Video Summarization, Deep-Learning, Key-Frame Extraction, Behavioral Analysis of the Cattle

In recent years, with the development of computer hardware, various problems solving using deep learning have been reported in the field of artificial intelligence. In this paper, we propose a video summarization system for digital video contents using deep learning. Next with using the proposed system, we summarize the video recorded with the camera installed at the horn of a calf, and report results of the behavioral analysis of the calf using the summary of the video. In cattle livestock management, it is important to find behavior patterns of individual cattle. However, this is a tedious work for research worker, for a long time and because of relatively gradual change, so researchers have given up so far. So this research is one of the solutions of ‘the most difficult but the most desired thing’in behavioral research.

Paper No. 97

170

Simulation of Image Enhancement by Stochastic Resonace in the Human Vision System

Naoki Kawaguchi, Atsushi Osa



FitzHugh’Nagumo equation, Visual phenomena, Stochastic resonance, Image processing

Abstract’In a highly information-oriented world, elucidation of information processing in our brains is progressing. Noise is inevitably present in both man-made and natural systems. Previously, these elements were removed for signal detection and information processing. However, recent researches have reported that noise plays a major role in brain information processing. One of the focuses of the relationship between noise and the vision system is the stochastic resonance phenomenon, wherein the detection rate of a weak signal is improved by the visual addition of a blinking noise of appropriate intensity. The understanding of the vision system is very useful for the development of imaging technology. This strategy of improving weak signal detection can be applied to digital image processing. In this study, we propose a vision model based on the FitzHugh’Nagumo equation and confirm that the stochastic resonance in brightness perception can be described by the model.

Paper No. 102

171

Shoot Motion Analysis in Basketball Formation Practice

Goki Hasegawa¹, Tokiichiro Takahashi²

Tokyo Denki University¹, Tokyo Denki University/ASTRODESIGN, Inc²


basketball, video analysis, player tracking

We have developed a video analysis method of shoot motion in basketball formation practice. Our method can estimate accurately the positions, movements, and shoot timing of players from formation practice videos.

Paper No. 128

172

A Study on Moving Image Noise Removal Using 3-D and Time-Domain Total Variation Regularization Method

Tsubasa Munezawa¹, Tomio Goto¹, Satoshi Hirano¹, Son Phung²

Nagoya Institute of Technology¹, University of Wollongong²


total variation regularization, noise removal method, BM3D method

In this paper, to avoid the adverse effect of image quality deterioration when emphasizing noise mixed image which is a problem of super resolution processing, we examine a noise removal method before super resolution processing. In our proposed method, Total Variation regularization, which decomposes an image into structure and texture components, is extended in direction of time axis.

Paper No. 135

173

Noise Reduction for 8K Endoscope Images

Aya Kubota¹, Seiichi Gohshi¹, Kenkichi Tanioka², Hiromasa Yamashita³

Kogakuin University¹, Medical Imaging Consortium², Kairos Co. Ltd.³


8K, Endoscope images, Total variation denoising

In medical field diagnosis based on images/videos is becoming increasing important year by year. Recently 8K resolution endoscope appeared. Lighting is used to obtain a sufficient light images/videos when endoscopy exams or operations are conducted. Although sufficient illumination is necessary, the lighting condition of the endoscope is limited due to heat that is generated by the light sources. The poor lighting condition always causes noise. Especially the size of 8K imaging cell is small, the number of photons for an 8K imaging cell is limited. Noise is an important issue for 8K endoscope images. In this paper, Total variation (TV) denoising method is used to reduce noise for 8K endoscope images.

Paper No. 140

174

Smoke Matting and Compositing in Video Sequences

Akihiro Fukuda, Hirotaka Tanaka, Yuji Waizumi, Tadashi Kasezawa

Nihon University


matting, compositing, smoke, video processing

This paper presents a novel method for smoke matting and compositing in video sequences. The smoke matting in the proposed method is realized by using the least squares method, on the assumption that nearby pixels have similar foreground colors and alpha values. Experimental results show that sufficient matting performance can be obtained in the scenes with high background color estimation accuracy. In addition, this paper shows that it is possible to change the color of the extractedsmoke and insert it into another video sequence.

Paper No. 141

175

Parallel Curvature Filter for High Performance Image Processing

Wei Pan, Yuanhao Gong, Guoping Qiu

Shenzhen University


filter, mean curvature, parallel computing, performance enhancing, video processing, real-time

Recently, curvature filter has been developed to implicitly minimize curvature for image processing problems. In this paper, we propose a parallel curvature filter that is much faster than the original curvature filter. The parallel curvature filter is performed on single GPU and can reach 33.2 Giga pixels per second. Such performance allows it to work in the real-time applications such as video processing and biomedical image processing, where high performance is required. Our experiments confirm the efficiency and effectiveness of our parallel curvature filter.

Paper No. 144

176

Compact Algorithm for Noise Level Detection

Naoya Sakuma, Seiichi Gohshi

Kogakuin University


noise detection, general video, noise reducer

Due to advances of imaging devices, high quality video has become easy to obtain. In contrast even though the advances, Gaussian noise is always mixed in images by photo-electric conversion. Noise reducer (NR) is a signal processing method to cope with the issue. However, noise level is required for NR to work effectively. Since signal processing methods for video should work in real-times, the noise level also should be detected in real-times as well. In this paper we propose a fast and accurate noise level detection algorithm.

Paper No. 146

177

A JPEG Decompression Technique based on Compressed Sensing

Naoya Sakuma, Seiichi Gohshi, Toshiyuki Yoshida

University of Fukui

[email protected], [email protected], decompression, total variation minimization, compressed sensing, non-convex programming, proximity

operator, primal-dual splitting method

We propose a new iterative JPEG decompression technique that combines a total variation minimization strategy with the so-called compressed sensing. The experimental results indicate that the proposed technique is advantageous over conventional ones.

Paper No. 147

178

A Color Quantization Method Preserving Infrequent Salient Colors and Its Implementation on Mobile Devices

Yukiya Fukuda¹, Hideaki Misawa¹, Hakaru Tamukoh², Ryosuke Kubota¹, Byungki Cha³, Takashi Aso⁴

National Institute of Technology, Ube College¹, Kyushu Institute of Technology², Kyushu Institute of Information Sciences³, Kogakuin University⁴


Color quantization, Infrequent salient colors, K-means clustering, Mobile devices

In this paper, we propose a new color quantization method that can preserve infrequent salient colors of an original image. The infrequent salient colors mean that they are not dominant globally, but are dominant locally. In the proposed method, color quantization is realized by K-means clustering and an input dataset for the clustering are adaptively and repeatedly modified based on local quantization errors to preserve the infrequent salient colors. The proposed method is implemented as an Android application to verify the feasibility of the use of the proposed method on mobile devices.

Paper No. 149

179

Consideration about Preventing Illegal Film Copy with Re-shooting Screen

Ayumu Wada, Seiichi Gohshi

Kogakuin University


re-shooting prevention, pirated films, theater screen

The damage caused by the pirated films amounts to $1.3 billion a year. The pirated films are mainly created by re-shooting a screen in theaters or duplicating the official DVDs. Films are released in theaters. Before the official DVDs are released, 90% of the films are pirated in the theaters and illegally made DVDs are sold. Although a counter technologies are required, there is no effective proposals. The important requirement for the technology is that it does not degrade the image quality in a theater. It only degrades the re-shot video. In this paper, we report a novel method to make it possible.

Paper No. 150

180

An adaptive cost aggregation method based on bilateral filter and Canny edge detector with segmented area for stereo matching

Wei-Jong Yang, Zi-Shiung Tsai, Pau-Choo Chung, Yao-Teng Cheng



Stereo Matching, Disparity, Bilateral filter, Depth map

In traditional stereo matching, global approach is more accurate, also have high accuracy in occlusion area. However, global algorithm is time consuming and hard to parallelized. On the contrary, local approach is usually fast but have bad performance, and easily influenced by noise. This paper proposed a novel method to compute disparity between two images. It is based on local approach, but our new cost function aggregated the cost in global way. This aggregation is processed by a weight map which created by the bilateral filter. Every pixel transfers its own cost information to all pixels on the same object, but this information would be restricted by the weight map.After finishing preliminary depth map, we use L-R check to find occlusion and mismatch pixels to refined our depth map. These refinement mechanics fix occlusion areas by the smallest disparity nearby. At last, we use bilateral filter clean up whole depth map.All of above computing process can be parallelized on GPU or cloud sever. Although this algorithm is designed for low-level machine, it still exerts high performance in high-level hardware.

Paper No. 151

181

Image Registration Using 2D Projection Transformation Invariant GPT Correlation

Toru Wakahara¹, Shizhi Zhang², Yukihiko Yamashita²

Hosei University¹, Tokyo Institute of Technology²


Image registration, Distortion-tolerant template matching, 2D projection transformation, Adaptive subwindow control

This paper describes a new method of image registration using distortion-tolerant template matching via multi-scale subwindow search. Here, we make full use of the GPT (Global Projection Transformation) correlation technique that maximizes a normalized cross-correlation value between an optimally 2D projection transformed template and a subwindow area of an input image. In particular, we propose to adaptively change the shape of the subwindow area from an original rectangle to its 2D projection transformed one through iterative matching process via the GPT correlation. We name this algorithm: adaptive subwindow control. Experiments made on the well-known datasets, Graffiti and Boat, show that the proposed method achieves a far superior ability of image registration under varying zoom, rotation, and viewpoints to the well-known feature-point based technique: a combination of ASIFT (Affine Scale Invariant Feature Transform) and RANSAC (Random Sample Consensus).

Paper No. 153

182

A support system for archiving antique stereographs using a camera

Tomofumi Dokuni, Ikuko Shimizu



Antique Stereo Image, Archiving, Image Restoration

Antique stereographs were taken by photographers from the end of the 19th century to the beginning of the 20th century.There are many antique sterepgraphs which have historical values.However, some of antique stereographs are curved.Therefore, a system for archiving and restoration of curved antique stereographs.In this paper, we propose a system for archiving antique stereographs by undistorting the picture of the curved stereograph captured by a camera.Our system devides the picture of the captured curved stereograph into small tetragonal regions and transforms these regions to the rectangles of the known size by perspective transformations to generate the uncurved stereograph.

Paper No. 155

183

Detpth From Defocus by Cross-Reblurring with Pixel Disalignment Compensation

Kazumi Takemura, Toshiyuki Yoshida

University of Fukui

[email protected], [email protected] estimation, Depth from defocus, multi-focus images, Pixel disalignment, Error reduction, Gauss-Newton

minimization

The authors have proposed a novel depth from defocus (DFD) technique based on a cross reblurring approach. While the technique gives a precise depth estimation, it is unfortunately affected by a disalignment of pixels in a set of multi-focus images. This paper thus proposes a compensation technique for the disalignment to educe the estimation error. The experimental results given in this paper provide the validity of our approach.

Paper No. 159

184

CNN Based Bitstream Feature Extraction for Codec Classification

Seungwoo Wee, Jechang Jeong

Hanyang University


codec, classification, bitstream feature extraction, convolutional neural network

In this paper, we propose codec classification algorithm based on convolutional neural network (CNN) model. In video compression, codecs, such as H.263, H.264, HEVC, etc., have their own distinctive technologies. They have also the unique structure; therefore, the each bitstream of their encoder has certain feature. The proposed algorithm exploits that characteristics for classifying unknown bitstream into specific codec. According to the fact that codecs have the unique start bit codewords respectively, our proposed algorithm learns the forms of these codewords. We constitute the bitstreams of encoder as inputs and each bitstream as its label indicating codec index. Three standard codecs, MPEG-2, H.263, and H.264, are used in experiment. Experimental results demonstrate that the proposed method classifies a bitstream into corresponding codec in high accuracy.

Paper No. 161

185

Hierarchical motion estimation algorithm using multiple candidates for frame rate up-conversion

Songhyun Yu, Jechang Jeong

Hanyang University


Frame rate up-conversion, Motion estimation, Image pyramid

Motion estimation (ME) has the highest computational complexity in motion-compensated frame rate up-conversion (MC-FRUC). For the real-time implementation of FRUC, a fast ME algorithm is required. In this paper, a new hierarchical ME algorithm for MC-FRUC is proposed. It constructs an image pyramid by dividing the frame into several sub-images according to resolution, and performs ME at the top level to reduce complexity while improving accuracy by selecting multiple motion vector candidates. These candidates are refined at the lower levels, and the final motion vector is selected at the bottom level. Thus, the proposed algorithm obtains an average peak signal-to-noise ratio gain of upto 0.85 dB compared to conventional algorithms with lower computational complexity and yields interpolated images with better visual quality than other methods.

Paper No. 171

186

Classification of speaking activities based on lip movements in video images

Prin Bandisak, Pinit Kumhom, Watcharapan Suwansantisuk


[email protected], [email protected], [email protected] detection, talking expression detection, histogram of oriented gradient(HOG), support vector machine(SVM),

convolutional neural network(CNN)

This research presents the analysis of student speech behavior. To use in developing the learning system. Simulation in the classroom and the information that is used to measure the student’s learning. By definition, the analysis of speech behaviors in the classroom consists of student group simulations can detect if the student has speech behaviors. There are 3 behaviors: 1. Speaking 2. Do not Speak 3. Other , By speaking, it creates unique characteristics and conditions. To set up various experiments. To analyze the conversations of students within the group. By behavioral detection. Use a CCTV video camera recorder to record a picture of a student’s face. Each facial image is taken using face detection techniques. To apply the face image to select only the area and position of the mouth, then use the technique to detect the distance of the lips.And use histogram oriented gradient (HOG) to create characteristic and viable vectors and To find the area of the vector of the shifting of the mouth. By applying speech, speech, distance and motion analysis of the lips. And the information that has been brought into the process support vector machine(SVM)To classify by focus Only the movement of the mouth. Then it will bring the characteristics that analyzed the group. And convolutional neural network (CNN) To create conditions For analyzing speech with each student’s speaking time And in explaining the work, creating conditions. To identify the behavior of each student. During the study period and the results of the experiment, students can evaluate their performance. Attention to the development of teaching and learning

Paper No. 176

187

A Parallax-tolerant Real-time Video Stitching Method for 12Kx2K Ultra Wide Vision

Soon-Heung Jung, Yongju Cho, Jeongil Seo

Electronics and Telecommunications Research Institute (ETRI)


video stitching, image stitching, parallax, panorama, UWV

Video stitching technology is widely used to generate Ultra Wide Vision (UWV) and 360VR. However, preserving spatial-temporal coherence and reducing parallax are still challenging problem. In order to increase stitching quality, we propose a parallax-tolerant real-time video stitching method for 12Kx2K UWV. Our experimental results show that the proposed method reduces the parallax efficiently.

Paper No. 187

188

Image Segmentation with Searching Tree of Superpixel Boundaries

Eisaku Ono, Ikuko Shimizu



Image segmentation, Superpixels, Tree search

Image segmentation is one of the most important techniques in computer vision and image processing.Many image segmentation methods have been proposed for these few decades.Hierarchical Feature Selection (HFS) is a graph-based approach for the image segmentation.It is known as a fast segmentation method that merges oversegmented regions hierarchically.At the first level of the merge, the superpixels are utilized to obtain the oversegmented regions.However, HFS sometimes fails when it is applied for the textured regions. In this paper, we propose a new approach for image segmentation, Searching Tree Segmentation from Superpixel (STSS), by formulating the merge of superpixels as a path searching problem.We construct trees and search the trees whose nodes correspond to the boundary of the superpixels and values of the nodes correspond to the distance between superpixels.The trees are searched recursively by depth-first search.Our algorithm does not check the boundaries of similar superpixels if these are no neighboring boundaries of the distinctively different superpixels to prevent the oversegmentation of the textured regions, while HFS checks all boundaries including quite similar superpixels.

Paper No. 188

189

A Study on Face Detection in Smoothed Video

Takashi Ozeki¹, Eiji Watanabe²

Fukuyama University¹, Konan University²

[email protected]

Human behavior analysis, Privacy, Face detection, Smoothed video analysys, Education

Recently, in education related studies, there are researches to detect the face of students from video of cameras installed in classroom and measure the degree of concentration of students in class from the movement of their faces. However, taking high-resolution images in classroom attended by specific students may lead to infringement of individual privacy. If group behavior analysis is possible even from low resolution images, it is possible to analyze behavior of people while maintaining privacy. In this report, we examined the possibility of human face detection in smoothed video which is difficult to identify individuals. From experimental data, we showed that it is possible to analyze the movement of the face from smoothed video by using Haar-like classifier created from smoothed face images.

Paper No. 191

190

Swimmer position estimation by lane rectification

Tayakuki Tsumita, Hidehiko Shishido, Itaru Kitahara, Yoshinari Kameda

Univerisity of Tsukuba


Computer vision, Water splash, Swimming, Swimmer position, Video analysis

We propose a new method of acquiring swimmer position in swimming pool video by image processing. The video of swimming games is taken from the top place of audience seat area, covering the whole field of a swimming pool. The swimming pool video is transformed so that each lane can be analyzed along with the lane direction. Our method can estimate swimmer position of each style. Estimation of swimmer position is composed of two-steps estimations. In first step, the peripheral region of a swimmer is estimated by extracting foreground region based on background subtraction. Second step uses color distribution. By introducing automated contrast adjustment, the swimmer region is emphasized and extracted. Then, by treating the swimmer region as a swimmer model, the swimmer position is estimated in finer resolution. The feasibility of the proposed method was confirmed by the experiment on the videos taken at national swimming games.

Paper No. 198

191

Correcting Geometric Distortion and Reflection Components in an Image of a Folded Print

Suguru Ariga, Yoshitsugu Manabe, Noriko Yata

Chiba University


Monocular camera, Geometric distortion, Shading, Printed material

When the printed material is imaged by a monocular digital camera, geometric distortion caused by fold may result in a different appearance from the content of the original printed material. This study aims to reproduce accurate appearance by correcting the obtained image. The geometric distortion is corrected by deforming each patch after dividing the printed material image into patches. Also, the brightness change by shading is corrected.

Paper No. 200

192

Manga Character Clustering with DBSCAN using Fine-Tuned CNN Model

Hideaki Yanagisawa, Takuro Yamashita, Hiroshi Watanabe

Waseda University


clustering, manga, DBSCAN, CNN

Manga (Japanese comic) is popular content worldwide. In Japan, e-comic accounts for about 80% of e-book market. In recent years, metadata extraction from manga image has been studied for providing new service in e-comic. Manga character is one of the important information for story understanding. Tubota et al. propose character identification method using character’s face image clustering. However, there are two problems. First, number of characters in target manga image is unknown. Second, since manga includes characters with extremely few appearances, it is difficult to classify characters perfectly. DBSCAN is a suitable clustering method to solve these problems, since it decides cluster number automatically and robust to noise data. As a prior study, we studied about character face clustering using DBSCAN and CNN feature. As a result, we confirmed that the proposed method is effective for main character extraction. However, it is difficult for general CNN model to capture detailed image features of manga characters. In this paper, we apply DBSCAN to CNN fine-tuned with manga character faces for the purpose of highly accurate clustering.

Paper No. 202

193

Image Recognition Using Multi-Layer Sparse Feature Extraction with ADMM

Tomoya Hirakawa¹, Kuntopng Wararatpanya², Yoshimitsu Kuroki¹

National Institute of Technology, Kurume College¹, King Mongkut’s Institute of Technology Ladkrabang²


Saak transform, Multilayer feature extraction, ADMM

Being motivated by the Saak (Subspace approximation with augmented kernels) transform, we propose an image recognition scheme using multi-layer sparse feature extraction with a convex solver ADMM (Alternating Direction Method of Multipliers). The Saak transform consists of a multi-layer PCA (Principal Component Analysis) and S/P (Sign-to-Position) conversion to avoid sign confusion. This paper adopts sparse representation instead of PCA and also compares the S/P conversion with the activation function ReLU (Rectified Linear Unit), which is realized by involving the projection mapping onto the non-negative set in convex formulas. The Saak transform uses PCA not only for feature extraction but also for dimension compression of feature vectors. We expect that our method does not need the dimension compression since sparse representation compresses features more than PCA. Experimental results on the MNIST and Fashion-MNIST dataset show that the proposed method is superior to the Saak transform in recognition accuracy.

Paper No. 208

194

Pure-Color Preserving Multi-Exposure Image Fusion

Artit Visavakitcharoen, Yuma Kinoshita, Hitoshi Kiya



Multi-exposure image, Image fusion, Color distortion

Multi-exposure image fusion is a method for producing an image with a wide dynamic range by fusing multiple images taken under various exposure values. In this paper, we point out two issues regarding the color distortion of fused results, that conventional fusion methods have not considered, and a novel multi-exposure fusion method is proposed to improve the issues. The first issue is that multiple images taken in the same scene have different colors. The second is that the color of fused images is not the same as those of the input images due to the influence of fusion functions. The proposed method enables us to preserve the pure-color of input images, while maintaining the wide dynamic ranges produced by conventional fusion methods. In addition, the proposed method can be applied to any existing fusion method to improve the quality of images fused by the fusion method.

Paper No. 210

195

A convolutional neural network with Sign-to-Position format conversion

Tomohito Mizokami¹, Kuntopng Wararatpanya², Yoshimitsu Kuroki¹

National Institute of Technology, Kurume College¹, King Mongkut’s Institute of Technology Ladkrabang²


Convolutional neural networks, Sign-to-Position format conversion, Rectification loss

This paper tries to improve image recognition accuracy with Convolutional Neural Networks (CNNs). CNNs are a state-of-the-art image recognition framework, and have used Rectified Linear Unit (ReLU) as the activation function. However, ReLU rectifies negative values to zero. This paper applies the Sign-to-Position (S/P) format conversion after convolutional procedures to eliminate the rectification loss. Experimental results show that the proposed method improves the recognition accuracy of the MNIST data set by 0.5% compared with a conventional CNN. The S/P format conversion also contributes to negative image recognition, and results in 12.58% higher accuracy.

Paper No. 211

196

Identification Of The Genus of Stingless Bees Via Machine Learning Technique

Muhammad Afif Nizam


[email protected]

Meliponine Bee, Machine Learning, Image Processing, Neural Network, Object detection

This study presents an interesting approach to identifying the Genus of the Stingless Bees aided by the machine learning technology.The conventional way of identifying the Genus of the Stingless bee or ‘ Lebah Kelulut’relied on the face-to-face meetings with local bee experts. This particular process is considered to be outdated and time-consuming.Thus , the proposed solution will incorporate the machine learning tools called the ‘TensorFlow Object detection API’provided by Google TensorFlow using the Faster Region-based Covolutional Neural Network which incorporate the Region Proposal Network to enhance the current network.

Paper No. 222

197

Classification of human’s activity from a sequence of images from a single camera

Natticha Nichaweerasit, Pinit Kumhom, Watcharapan Suwansantisuk



Algrolithms, Histogram of Orientation Gradient, Hough Transform, Support Vector Machine

Abstract This article presents a way to classify students’ behavior in the classroom at the time of instruction, by studying student behavior that is not related to the use of electronic devices. such as computer, tablet, smartphone The behaviors that we will classify are divided into 4 activities (1) working on books (2) experimenting (3) doing other activities (4) stationary / unidentified In tracking student behavior, we use algorithms to help classify behaviors. The presentations were experiment using a single camera to capture various types of activities. Hough Transform in Object Detection and Histogram of Orientation Gradient (HOG) technique for hand detection. Then, the derived data to generate different vector identifiers is then imported into the Support Vector Machine (SVM) to classify the behavior of students in the classroom. By the way, in the article we can use the information to assess the behavior of students In the past, human use in observation may make the assessment more error. And the information we get from tracking the behavior. Can be used as information to improve the learning and teaching for students.

Paper No. 225

198

Estimation of Objective Understanding Measure based on Student’s Nonverbal Behavior Recognition in a Person-to-Person Teaching Situation

Ryo Miyoshi, Koichi Taguchi, Manabu Hashimoto

Chukyo University


Estimation understanding measure, Nonverbal behavior, Head motion, Gaze, Blink

In this paper, we propose a method to estimate the value of ‘understanding measure’in the person-to-person teaching situation by using image recognition techniques. Here, the word ‘understanding measure’means how strong the teacher feels ‘This student understands about this topic.’Concretely, at first, information obtained from student’s nonverbal behavior are extracted as the feature for the estimation process. Next, by using the nonverbal features, unknown input data as an image sequence of the student will be identified to two classes, “understanding” or “not-understanding” with the kNN classifier. As a result of experiments, it was confirmed that the F-value of the class “understanding” by the proposed method was 0.75, and that of the class “not-under-standing” was 0.60. That indicates that our method improved F-values 0.38 and 0.11 compared with previous methods, respectively.

Paper No. 234

199

High Reality Image Generation for DNN Learning based on Varying Pixel Intensity Value Model Depend on Each Camera -The Last 1% Accuracy

Improvement-

Yusuke Kamiya, Nobuyuki Shinohara, Manabu Hashimoto

Chukyo University


Deep Learning, visual inspection, image generation

In object recognition using the DNN in the field of industry, there is a problem that the recognition accuracy rate decreases due to differences of characteristics existing between the camera for learning and the camera for recognition. In this research, we will solve this problem by modeling the varying pixel intensity value of each recognition camera statistically based on acquired actual image samples. By using the camera model, a huge already-captured learning image sets can be converted to virtual images which are truly captured by the recognition camera. Here, the statistical characteristics of generated images must be very similar to images of recognition camera. Through experiments using actual images, we confirmed that the recognition accuracy rate by our method is at least 1.0% higher than that without our method.

Paper No. 235

200

Integrated Analysis of Position of Gaze/Hand for Skill-up Process Analysis of Assembly Tasks

Yohei Kawase, Koichi Taguchi, Manabu Hashimoto

Chukyo University


Skill-up process analysis, Assembly tasks, Gaze, Movement of hands

In this paper, we propose a method of skill-up process analysis for objectively analyzing the difference of the movement of gaze and hand between different skill levels in assembly tasks. Concretely, the proposed method quantizes position information of gaze and hand into 18 regions and generates the code strings of gaze and hand. And, it extracts pairs of codes of gaze and hand in each frame and calculates the frequency of occurrence of pairs of codes. We call this the ‘gaze-motion integration feature’ As a result of analysis by this feature, we found that the movement of the non-dominant hand is different between the ‘elementary’level and the ‘intermediate’level, but the movement of gaze is different between the ‘intermediate’level and the ‘expert’level.

Paper No. 236

201

A Parallel Pre-Processing for Multiple Objects Tracking System

Adirek Maruaisap, Chanon Khongprasongsiri, Watcharapan Suwansantisuk, Pinit Kumhom



FPGA, hardware implementation, image processing, multiple objects tracking, parallel pre-processing

Object tracking based on image processing algorithm is used in various applications. In many object tracking methods, feature extraction is the key processing for object identification. Image pre-processing which transforms an RGB image to a binary image plays an important role. The conventional pre-processing technique applies on a whole image frame. In this paper, we propose to crop the regions of the tracked objects using their current tracking positions. Then, each cropped region is fed to the pre-processing process. Based on this approach, the interference of uninterested regions is eliminated resulting in improved pre-processed image. However, we need to perform N pre-processing processes, where N is the number of tracked object in the frame. This problem is alleviated in FPGA implementation, which is our target platform. The proposed approach is evaluated by comparing the results with conventional pre-processing method using the same tracking system.

Paper No. 237

202

Distortion-Resistant Spherical Visual Odometry for UAV-Based Bridge Inspection

Sarthak Pathak¹, Alessandro Moro¹, Hiromitsu Fujii², Atsushi Yamashita¹, Hajime Asama¹

The University of Tokyo¹, Chiba Institute of Technology²


Spherical vision, Distortion, Visual odometry

In this research, we propose a novel distortion-resistant visual odometry technique using a spherical camera, in order to provide localization for a UAV-based, bridge inspection support system. We take into account the distortion of the pixels during two essential visual odometry steps - the calculation of the 2-frame essential matrix via feature-point correspondences, and the 3D registration of pointclouds across 3 frames. We demonstrate that this greatly increases the accuracy of localization, resulting in an 8.6 times lower localization error.

Paper No. 246

203

DWT-based Watermarking Algorithm for Multiview Image

Young-Ho Seo¹, Yong-Seok Lee², Dong-Wook Kim¹

Kwangwoon University¹, Korea Electronics Technology Institute²


Watermarking, DWT, DIBR, Free-viewpoint, 3D Image Processing

This paper proposes a digital watermarking scheme to protect the ownership of a free-view 2D or 3D image, such that the viewer views the image(s) by rendering an arbitrary viewpoint image (or multiple images) with the received texture image and its depth image. The free-view image therefore suffers from a viewpoint change attack, even if it is not malicious. This paper focuses not only on this viewpoint change attack, but also on the other traditional pixel-value change attacks and the geometric attacks conducted in addition to the viewpoint change attack. The methodology generates a special map, termed the depth variation prediction map (DVPM), to find locations that are safe from viewpoint change. A 3-level Mallat-tree 2-dimensional discrete wavelet transform (2DDWT) is also used, from which the three horizontally low-pass filtered and vertically high-pass filtered subbands are used as the watermark (WM) embedding regions, in conjunction with the DVPM. Multiples of WM data are embedded into the three subbands, and each WM bit is embedded into a 2DDWT coefficient by a quantization index modulation (QIM) method, where the quantization step is decided by considering the energy of each subband. In extracting the WM data, the multiple WM data are extracted, and the most frequent value in each bit position is taken from the extracted data to form the final data. We experiment the proposed method with various test images for the various attacks, and compare our scheme with the previous methods, to show that the proposed method has excellent performance.

Paper No. 261

204

Medical Imaging and Processing

Feasibility study of virtual monochromatic imaging for metal artifact reduction in spectral CT

Minjae Lee, Donghoon Lee, Dohyeon Kim, Hyemi Kim, Hee-Joung Kim

Yonsei University


Metal artifact, Virtual monochromatic imaging, Photon-counting system

Polychromatic X-ray in computed tomography (CT) can cause metal artifacts and beam hardening artifacts, which are limiting factors in the detection and diagnostic of lesions. Several groups have introduced a virtual monochromatic image using dual-source CT to reduce these artifacts. In this study, we investigated the feasibility of virtual monochromatic imaging in a photon-counting system. A prototype of the spectral CT system, which has 64 line-pixels Cadmium Zinc Telluride (CZT)-based photon-counting detector, was used. The source-to-detector distance and the source-to-center of rotation distance were 1,400 and 1250 mm, respectively. Energy bins were set at 23 - 32, 33 - 42, 43 - 52, 53 ‘ 62, and 63 - 90 keV. Integrating mode is the sum of five energy bins, which is assumed to polychromatic X-ray. The phantom contained the two copper (Cu) cylinders in PMMA. We evaluated the signal difference-to-noise ratio (SDNR). The SDNR were the lowest in 77 keV monochromatic imaging. Our results indicated that virtual monochromatic imaging in the prototype of the photon-counting system effectively eliminates the metal artifact and provides better image quality than integrating mode at 23 - 90 keV.

Paper No. 7

205

Hybrid algorithm of maximum-likelihood expectation-maximization and multiplicative algebraic reconstruction technique for iterative tomographic

image reconstruction

Ryosuke Kasai¹, Yusaku Yamaguchi², Takeshi Kojima², Tetsuya Yoshinaga³

Tokushima University¹, Shikoku Medical Center for Children and Adults², Tokushima University³


Iterative image reconstruction, computed tomography, maximum-likelihood expectation-maximization, multiplicative algebraic reconstruction technique, inverse problem

Maximum-likelihood expectation-maximization (ML-EM) method and multiplicative algebraic reconstruction technique (MART), which are well-known iterative image reconstruction algorithms, produce relatively high-quality performance but each of which has an advantage and disadvantage. In this paper, in order to compensate for both disadvantages, we present a novel iterative algorithm constructed by a nonautonomous iterative system derived from the minimization of an alpha-skew Kullback-Leibler divergence, which is considered as a hybrid objective function for ML-EM and MART. We confirmed effectiveness of the proposed hybrid method through numerical experiments.

Paper No. 12

206

POCS-based restoration algorithm for beam modulation CT acquisition

Dohyeon Kim, Donghoon Lee, Hyemi Kim, Zhen Chao, Minjae Lee, Hee Joung Kim

Yonsei University


Computed tomography, Beam modulation, Projection onto convex sets algorithm

Region-of-interest (ROI) imaging is considered an effective method to reduce the exposure dose. We propose ROI-based beam modulation acquisition to restore the information outside of the ROI. The CT system and 3D voxelized abdominal phantom were simulated using the MATLAB R2017b program. A total of 360 projections were obtained and used for CT reconstruction with a filtered back projection (FBP) algorithm. Beam modulation CT images were reconstructed using 288 truncated and 72 full projections. An interpolation method and our proposed method based on a projection onto convex sets (POCS) algorithm corrected the truncated projections. The image quality of three ROIs was evaluated using the structural similarity index measure (SSIM). The reconstructed image obtained by beam modulation acquisition resulted in a much higher SSIM value for the external information than that obtained by the ROI scan. The proposed method based on a POCS algorithm provides the best image quality in beam modulation acquisition. In conclusion, we have verified the possibility of restoring the ROI external information using beam modulation acquisition.

Paper No. 14

207

Brain image segmentation based on improved BP-Adaboost neural network

Zhen Chao, Dohyeon Kim, Hee Joung Kim

Yonsei University


Back propagation neural network, Adaboost algorithm, Gravitational search algorithm

The segmentation of medical image applying in medical anatomy plays an important role in various application. So, the study of medical image arithmetic is very important and necessary. Due to the presence of noise and complexity of structure, the existing methods have various shortcomings and the performances are not ideal. In this study, we propose a new method which based on back propagation (BP) neural network and Adaboost algorithm. The BP neural network we created is 1-7-1 structure. then we trained the system by Gravitational search algorithm (Here, we use the segmented images which were obtained by classic fuzzy c-means algorithm as the ideal output data). Then, we trained 10 groups of BP neural networks. Based on this, we can obtain the performance weighting of each BP neural network by error calculation, subsequently, we adopted the Adaboost algorithm to adjust the data weighting for getting the best performance. Finally, we made up a new BP-Adaboost system for image segmentation. In this experiment, we used one group of datasets: Brain MRI. Finally, subjective observation and objective evaluation prove the superiority and effectiveness of proposed method.

Paper No. 18

208

Visualize Medical Information using VISTouch Technology

Masasuke Yasumoto¹, Takehiro Teraoka²

Kanagawa Institute of Technology¹, Takushoku University²


Visualize, Multi-display, Interactive, xR, Medical Information, CT, MRI

We constructed a novel system that achieves new operational capability and increases user interest in mobile devices by enabling multiple devices to be used in combination dynamically and spatially. We call the system VISTouch[1] and we realize visualize medical information. We focused on developing a mobile device feature that would enable users to easily move the position of a device connected with another device and keep track of it. We implemented a novel approach with intuitive handling in 3D virtual space. When a smartphone is spatially connected to a horizontally positioned tablet that is displaying a map as viewed from above, these devices dynamically obtain the correct relative position by using VISTouch. The smartphone displays images viewed from its position, direction, and angle in real time as a window that shows virtual 3D space. In addition to the old way of recognizing directions, which depends on virtual 3D images, we use real space information to improve a userʼs spatial perception by combining real space and virtual space. Thus, our VISTouch offers a novel way to interact with multiple devices by moving or inclining a smartphone on the tablet display and uses detailed information of the relative position in real space. This was applied to the visualization of medical information. First of all, we reconstructed images of multiple MRI and CT stereoscopically as point clouds. By using this system, it is possible to visualize images from arbitrary cross section or arbitrary viewpoint in real time. It is also possible to cut it obliquely and see it overlooking. It can be viewed simultaneously by multiple people, and the same human body can be seen from multiple viewpoints. In the future we would like to use it for medical education.

Paper No. 44

209

Automated detection of fundic gland polyps from endoscopic images using SSD

Nagito Shichi¹, Arata Totsuka¹, Junichi Hasegawa¹, Tomoyuki Shibata²

Chukyo University¹, Fujita Health University Hospital²


Endscopic, Convolutional Neural Network, automated detection

In stomach lesion screening, endoscopic images provide the most effective diagnostic information. However, in the most of lesions at the initial stage, the sign of existence is hard to appear on endoscopic images, and also there is the difference in operations of endoscopes and observation of images in real time among individual medical doctors. Therefore, development of a computer aided diagnostic system (CAD system) for endoscopic images is required.In this study, we propose a method for automated detection of fundic gland polyps from endoscopic images using the object detection algorithm SSD (Single Shot MultiBox Detector) which is one of CNN (Convolutional Neural Network). SSD used here has 20 of convolution layers and 6 of pooling layers, and the input image size is 300x300. In the experiment, 73 practical fundic gland polyp images were used. To compensate for lack of training images, augmentation was performed using image rotation and edge enhancement. We trained 8751 training images and 2188 verification images. Also, as a preprocessing, highlight areas were removed automatically from all images including both training and test samples. As a result, 94.7% of TP (true positive) rate for 73 fundic gland polyp images was obtained by using our learned SSD.

Paper No. 46

210

Partial rigid diffeomorphism for measuring temporal change of pancreatic cancer tumor

Yuki Tamura¹, Tatsuya Yokota¹, Mauricio Kugler¹, Valentin Triquet¹, Tomonari Sei², Chika Iwamoto³, Kenoki Ohuchida³, Makoto Hashizume³, Hidekata Hontani¹

Nagoya Institute of Technology¹, The University of Tokyo², Kyushu University³



diffeomorphism, pancreas cancer, temporal change, medical image, MRI

The objective of our research is to describe the isotropic temporal change of the tumor region in the whole body MRI images of a KPC mouse captured at different time. For accurately describing the tumor growth, non-rigid registration should be applied to normalize the body motion. Here, in order to suppress the tumor deformation caused by the registration, we develop a Large Deformation Diffeomorphic Metric Mapping(LDDMM)[1] that transforms the tumor region rigidly. Given a set of pairs of corresponding landmarks on peripheral organs in a source image and a target image, the proposed method first non-rigidly registers the two images by referring to those corresponding landmarks. This non-rigid registration maps the tumor region in the source image non-rigidly. The proposed method next determines the destination locations of landmarks generated in the tumor region in the source image by converting the non-rigid mapping of the tumor region to a rigid mapping. Then, we can have a set of pairs of the corresponding landmarks that consists of (a) the landmarks on the peripheral organs that would be registered non-rigidly and (b) the landmarks in the tumor region that can be registered rigidly. Our method iterate the non-rigid registration of the landmarks and the conversion of the non-rigid mapping of the tumor landmarks to the rigid one until they are converged. We experimented real MRI images and confirmed the effect.

Paper No. 48

211

Kymograph of Continuous MRI Images for Analyzing Speech Articulations

Makoto J. Hirayama¹, Sayoko Takano²

Osaka Institute of Technology¹, Kanazawa Institute of Technology²


MRI, kymograph, speech articulation, tongue

Magnetic Resonance Imaging (MRI) was used for analyzing speech articulations. MRI imaging experiments were done to capture tongue motions during speech. Then, kymograph for tongue motions were made by slicing MRI images and lining up images to the time axis direction. The proposed kymograph method is a useful tool for analyzing speech articulations especially for narrowing positions of vocal tract by tongue motions.

Paper No. 111

212

High-Precision Performance Evaluation for Digital Radiography Detectors

Dong Sik Kim

Hankuk University of Foreign Studies

[email protected]

Detective quantum efficiency, Modulation transfer function, Noise power spectrum, Radiography detector

Performance of digital radiography detectors can be evaluated from measuring the noise power spectrum (NPS), modulation transfer function, and detective quantum efficiency (DQE). Hence, measuring these quantities with high precisions is important. In this paper, we introduce a program, which is called DRDQE (digital radiography DQE). DRDQE supports the IEC62220 standards for various radiography detectors. Furthermore, DRDQE measures NPS and DQE with high precisions based on Small-Subimage Algorithm. Compensated Partial-Sum Algorithm is also adopted to measure a high-precision NPS at zero frequency. We briefly introduce the employed algorithms in DRDQE and then show an example of DRDQE usage for a direct flat-panel mammography detector. The high-precision measurements from DRDQE were compared with the IEC62220 standard.

Paper No. 143

213

Deep Learning based Fully-Automated Segmentation of Abdominal Organs from CT Images

Jieun Kim, June-Goo Lee

University of Ulsan College of Medicine/Asan Medical Center


multiple organ, multi-organ, segmentation, MPR based segmentation

To develop a fully-automated method to segment multiple organs from abdominal CT images and to evaluate its performance on clinical dataset.

Paper No. 166

214

Lumen and Vessel Wall Segmentation on Intravascular Ultrasound Images Using Fully Convolutional Network

Jiyeon Ko, June-Goo Lee

University of Ulsan College of Medicine/Asan Medical Center


Intravascular Ultrasound, Machine Learning, Fully Convolutional Network Medical Image Segmentation

We devised an automatic segmentation algorithm for the lumen and vessel wall on intravascular ultrasound images.

Paper No. 175

215

Accuracy and Precision in Estimating Noise Power Spectrum in Radiography Imaging

Eunae Lee, Dong Sik Kim

Hankuk University of Foreign Studies


Estimate accuracy, Estimate precision, Noise power spectrum, Periodogram, Radiography imaging

In order to observe the noise property of the radiography image detector, the noise power spectrum (NPS) is usually measured. Such an NPS curve is measured from the sample average of the periodogram samples. Here, we can improve the measurement accuracy and precision by increasing the number of periodogram samples and the spectrum resolution. In this paper, we observe the accuracy and precision of the NPS measurement, which is based on the periodograms. For real digital X-ray images, which were acquired from an indirect CsI(Tl)-scintillator radiography detector, we observed the accuracy and precision of the NPS estimates for various resolutions and sample sizes. For a given image size, we could find appropriate sample size and spectral resolution for obtaining a stable NPS curve.

Paper No. 193

216

Many-view under sampling reconstruction in low-dose helical CT

Sanghoon Cho¹, Seoyoung Lee¹, Sunho Lim¹, Seungryong Cho¹, Kou Gyeom Kim², Jong Hyun Ryu², Kil Hwan Jeong²

Korea Advanced Institute of Science and Technology¹, Wonkwang University Hospital²


Helical CT, Low-dose, Many-view under sampling

For the purpose of the assistance in surgery, helical CT is used to check the situation of operations. However, undesirable x-ray exposure could lead to health risks of patients in operation. The purpose of our research was to reconstruct helical CT image with oscillating multi-slit filter in front of the x-ray source. We obtained real sinogram data by multi-row helical CT gantry system and applied expected numerical mask pattern in simulation on to the sinogram. The line collimator was assumed to be in reciprocating linear motion and open area was set to be half of the field of view of x-ray. An ASD POCS algorithm regularized by TV minimization was adopted to deal with sparsely sampled data. Results indicated that the proposed low-dose helical CT scheme successfully could lead to quality reconstruction image.

Paper No. 220

217

A Study on Improvement of Lateral Resolution in Synthetic Aperture Super Resolution Ultrasound imaging

Ryoya Kozai, Jing Zhu, Norio Tagawa



super resolution imaging, high frame rate imaging, beamforming, frequency sweep, F-DMAS beamforming

Ultrasound imaging is applied to various fields because it is noninvasive and real-time imaging is possible. However, in the field of diagnosis imaging, ultrasound imaging is inferior in resolution to other modalities, so researches to improve resolution are actively conducted. We proposed a method called the SCM (Super resolution FM-Chirp correlation Method) that improves the range resolution based on frequency sweep. In order to improve the frame rate, the SCM was extended to the SA-SCM (Synthetic Aperture-SCM) by incorporating synthetic aperture beamforming using divergent waves in the SCM. However, these methods improve only the range resolution. Therefore, in this study, we aim at high lateral resolution by using F-DMAS (Filtered-Delay Multiply and Sum) as the beamforming method.

Paper No. 227

218

Differential phase contrast imaging using the phase retrieval of Hilbert Transform and noise filtering by low rank method

Jae-Suk Yang¹, Myung-Joon Kwack², Soo-Yeul Lee², Jang-Hwan Choi¹

Ewha Womans University¹, Medical Imaging Research Section Electronics and Telecommunications Research Institute (ETRI)²


Differential phase contrast imaging, Hilbert transform, Phase unwrapping, Low-rank

In this paper, to obtain the differential phase contrast imaging, we propose the phase retrieval method by the Hilbert transform (HT) and filtering method by the low-rank. The proposed method has merits: 1. We can implement a single grating system without the mechanical movement of the grating. 2. The complexed computation of the phase retrieval method by the Fast Fourier transform (FFT) can be avoided by the proposed method. 3. Thanks to fringes from the various energy bins, the noise reduction by the low-rank can be handled. Especially, the low-rank is the singular value decomposition (SVD) by the rank-one property. The phase retrieval of the Hilbert transform and noise filtering have been performed to validate the proposed method.

Paper No. 240

219

Deep Neural Networks for Lung Cancer Tumor Region Segmentation

Runze Zhang, Zhong Guan, Shun-Cheung Lai, Jun Xiao, Kin Man Lam

The Hong Kong Polytechnic University


deep learning, segmentation, U-Net, radiomic analysis, lung tumor

In this paper, we investigated the use of deep neural networks to perform segmentation and prediction of lung cancer tumor regions, via screening Computed Tomography (CT) scans. In order to achieve satisfactory results, we have adopted U-Net, a deep neural network, for the segmentation of tumor regions in the CT scans. U-Net was designed for medical image segmentation. With the segmentation results generated by U-Net, some false-positive segments will appear. Therefore, a process similar to radiomic analysis was carried out to rectify this issue. After evaluating the performance of various networks, ResNet-18 can achieve the best performance, in terms of reducing the false positives. Using the extracted deep features for classification, each segmented region generated by U-Net is classified as to whether it contains a real tumor or not. With the use of U-Net only, the four evaluation metrics, i.e., dice coefficient, mean surface distance, Hausdorff distance, and patient-wise accuracy, are 0.5471, 12.505, 29.336 and 90%, respectively. After using the radiomic analysis, the four metrics become 0.592, 9.786, 20.061, and 90%, respectively.

Paper No. 260

220

Rapid On-site ICH Volume Measurement & Stroke Patient Prognosis System

Tiehua Du¹, Keng Wah Choo¹, Eddie Tan², Wai Hoe Ng², Tchoyoson Choie Lim², Wai Ming Kong¹, Tsu Soo Tan¹

Nanyang Polytechnic¹, National Neuroscience Institute²

[email protected]

ICH, 3D, stroke, mobile application, prognosis, on-site, volume measurement

ICH is a type of stroke caused by internal bleed of the brain. Accurate and fast measurement of the blood clot volume is important for neuro surgeon to make an informed decision if the patient need surgery or not. In this paper, we present a mobile application ‘ICH 3D’ which is able to measure blood clot volume on-site using a smart phone with improved accuracy and within a minute.

Paper No. 262

221

Multimedia

Interactive Geo-Temporal Data Visualization for Aircraft Engine

Qian Zhang, Chung Soo Ahn, Jigang Liu, Yan Chao Wang, Feng Lin, Hock Soon Seah



aircraft engine, interactive data visualization, geo-temporal, react-redux web application

The heart of every vehicle is the engine and many factors contribute to the aircraft engine’s durability and lifespan. In this paper, we propose an interactive system to link the engine maintenance event data, flight route history data and volcanic ash eruption data and visualize the relations among them. Several kinds of visualization forms are used to present the data. We use the react-redux frontend framework along with a Python backend and the Elasticsearch engine. This highly tailored system shows how datasets of different sources can be linked and visualized with interactivity. Data analysts and developers can benefit from full-stack development process from data to intuition, meeting the requirements of customized visualization.

Paper No. 3

222

Classification of Tourism Categories Based on Heterogeneous Features Considering Existence of Reliable Results

Naoki Saito¹, Takahiro Ogawa¹, Satoshi Asamizu², Miki Haseyama¹

Hokkaido University¹, National Institute of Technology, Kushiro College²


tourism images, category classification, image sharing services

This paper presents a classification method of tourism categories based on heterogeneous features considering existence of reliable results. The proposed method performs estimation of existence of reliable results based on one-versus-one scheme from three kinds of classification results obtained from tourism images, geotags and textual tags, separately. Then if the reliable result is included in the above results, this result is regarded as a final result. Otherwise, the final result is obtained by the multiple annotator logistic regression. The proposed method realizes accurate classification by estimating the existence of reliable results from more than two kinds of results.

Paper No. 57

223

Projecting Technique of Holographic 3D imgaes With the Use of Fog Screen

Kunihiko Takano¹, Syuntaro Tominaga¹, Tatsuya Ikeuchi¹, Koki Sato², Kikuo Asai³

Tokyo Metropolitan College of Industrial Technology¹, Former Shonan Institute of Technology², The Open University of Japan³

[email protected]

Electro-Holography, Projection in to the space, Fog screen, Digital Micro-mirror Device

As an effective reconstructing technique to observe full-parallax holographic images in the wide viewing area, we have studied a projecting process of 3-D holographic images with the use of the spatial screen. Last year, as one technique to perform a suitable projection of the images, we have reported a projecting process of holographic 3-D images onto the underwater micro-bubble screen.Applying this technique, the stability of the images displayed in the screen seems to have been highly improved, however, as for the flickering of the images, much more improvement seems to be not so easy. Moreover, it requires a lot of water in the water tank, so that a complicated and a more weighty system has been needed.For this reason, we constructed a new spatial screen system employed the fog, in which a complicated structure and a lot of water was not required. In this report, we have discussed its effects to the reconstructed images through the holograms. In addition, in this study, we have seen that the system construction seems to be simple and not so weighty, and effective for suppressing the flickering of the images with higher resolution.

Paper No. 82

224

TV Production Tool to Make Art Programmes Based on a Simple Scenario

Akira Asada¹, Masaki Hayashi²

Osaka Institute of Technology¹, Uppsala University²


Virtual museum, CG animation, TVML

We have developed the virtual museum capable of presenting TV-program-like animations in the Computer Graphics (CG) museum space. To make the animation easily, we propose a production tool based on simple scenarios to make the art TV programmes. When it inputs simple scripts in a text file, the system converts it to a TVML (TV program Making Language) script, then the TVML engine generates a CG animation like TV art programme.

Paper No. 114

225

Playing Digital Music by Waving Hands in the Air

Alexei Sourin, Zhong Cai Chock



digital music, hand tracking, theremin

We study how music can be played with a computer by waving hands in the air. We begin our considerations at analyzing how the theremin is played’an electronic musical instrument which is played without physical contact by the hands of the performer. We then analyze the hand-tracking technologies available for common personal computers and mobile devices and hypothesize that optical tracking devices like Leap Motion controller may be used for simulating the theremin functions with a common personal computer. We then analyze why the theremin is considered as the most difficult to play musical instrument and how its deficiencies can be overcome with the help of computer graphics.

Paper No. 145

226

Efficient Scheduling for Dynamic Streaming of 3D Scene for Mobile Devices

Budianto Tandianus¹, Hock Soon Seah¹, Tuan Dat Vu², Anh Tu Phan²

Nanyang Technological University¹, Hanoi University Of Science and Technology²


streaming, out-of-core, progressive, cloud, Unity, pathfinding, scheduling

We present an efficient resource scheduling scheme for out-of-core dynamic streaming of a 3D scene. The entire scene is stored in the cloud and relevant scene are streamed to a client mobile device in real time based on the camera path in the 3D scene. We analyze the camera path data such as speed and position in order to yield efficient streaming of 3D urban objects. We compare streaming scheduling based on camera path and on only camera’s current location in term of storage, rendering performance, and number of misses. The client application is implemented in Unity game engine and we perform experiment on an Android mobile device.

Paper No. 156

227

Digital Scratch Art Painting Interface

Ken Ishibashi

[email protected]

scratch art, blob, kubelka-munk theory, painting interface

Scratch art is an artwork created by scratching on the black paint. After scratching, a part of colorful background is emerging appear from the scratched region. Several applications have released for creating the digital scratch arts. These applications enable users to create their digital scratch arts easily. However, users cannot draw their own backgrounds. If users can draw the background easily, they may find their own artistic expressions. This study presents a digital scratch art painting interface using a blob-based method proposed by shugrina et al. The interface allows users to create each gradient background by their sketches. Furthermore, users are possible to edit each color blob, hence every blob is movable and changeable the size. The blob is also mergeable each other, and these colors are mixed from one’s color to the other color gradually because the blob-based method uses a metaball function. In addition, the color gradient method adopts the Kubelka-Munk theory. This theory makes beautiful gradient colors.

Paper No. 173

228

Procedural Animation of Waving Cartoon Hair by Emulating Animator’s Techniques

Naoaki Kataoka¹, Tomokazu Ishikawa², Yusuke Kameda¹, Ichiro Matsuda¹, Susumu Itoh¹

Tokyo University of Science¹, Toyo University²


hair waving, cell look animation, procedural animation

This paper describes a method for creating animations of waving cartoon hair.In this method, the gatherings of air are modeled with rigid circles, and hair bundles are modeled with elastic bodies. Deformation of the hair bundles is determined by simulating collision events between these circles and hair bundles. Since the method is based on the animator’s techniques used in creation of hand-drawn works, it is expected to suppress a feeling of strangeness that is often introduces by the previous method.

Paper No. 215

229

Others

Research on Troxler Effect Focusing on Gazing Time for Edge blurred Stripe Image

Takahiro Miyamoto, Hiroto Inoue, Nobuji Tetsutani



visual perception, edge effect, blur, contrast, troxler effect

The Troxler effect is a phenomenon in which a blurry image in the peripheral visual field becomes invisible with gazing time. We found that the Troxler effect is related to the contrast reduction induced by edge blur under the study of gazing time and edge blur phenomenon. In this paper, we clarify the quantitative relationship between the edge blurring phenomenon and the Troxler effect.

Paper No. 4

230

Study on Evaluation Method for 3D Perspective Image

Seiya Iwasaki, Nobuji Tetsutani



3D, perspective, evaluation, incongruity

The method for evaluating stereoscopic effect has not been adequately studied. We proposed “incongruity” as the evaluation item for a size of the face in the 2D image and 3D image. The sense of incongruity that people experienced is a sense that ‘I understand something is different, but I cannot explain it in concrete terms. In this paper, we describe the experimental results by using three evaluation items (incongruity, depth feeling, and stereoscopic effect) focusing on a perspective 3D image, and propose a new evaluation method using 3D reversed images with the left and right images reversed.

Paper No. 5

231

Leader-Follower Formation Control of Multiple Unmanned Aerial Vehicles for Omnidirectional Patrolling

Kosuke Kasugai¹, Isao Miyagawa², Kazuhito Murakami²

Aichi Prefectural University¹, Nippon Telegraph and Telephone²


Formation control, Unmanned aerial vehicle, Patrolling, Leader-follower Structure, Motion capture system

Unmanned aerial vehicles (UAVs) are applied to various applications due to its maneuverable flight. Formation flight composed of multiple UAVs has obvious advantages in accomplishing effectively and speedily a task. We propose a formation control of multiple UAVs for omnidirectional patrolling. The formation flight adopts a leader-follower structure. Assumed that motion capture system detects the 3D configuration of formation flight, our method stably and safely controls the formation based on geometric constraint in 3D space. When we fly the leader UAV by remote control according to given flight trajectory, the position and orientation of follower UAVs are automatically adjusted by employing feedback control so that the leader-follower structure forms a geometric pattern. We demonstrate that our proposed method offers excellent formation flight to provide practical omnidirectional patrolling.

Paper No. 28

232

Three-dimensional Temperature Distribution Estimation around A Heat Source in Water Using Asymmetric Inverse Abel Transformation

Mizuki Kyoda¹, Naoto Kakuta², Kazu Mishiba¹, Yuki Arakawa², Katsuya Kondo²

Tottori University¹, Tokyo Metropolitan University²


3D reconstruction, Near-infrared absorption, Asymmetric temperature distribution, Asymmetric inverse Abel transformation

We propose a method to reconstruct three-dimensional (3D) temperature distribution in water around a heat source. Temperature variation by the near infrared (NIR) temperature imaging method is the integral value of NIR light intensity in a measurement direction. In the conventional method, it is assumed that the temperature is symmetrically distributed with respect to the vertical axis through the center of the heat source. However, in practice, it is asymmetrically distributed. In this paper, asymmetric inverse Abel transformation is used to estimate 3D temperature distribution which is asymmetry.

Paper No. 30

233

Deep Learning Based Style Transfer for Video

Wei-Cheng Chang¹, Der-Lor Way², Chin-Chen Chang³, Zen-Chung Shih¹

National Chiao Tung University¹, Taipei National University of Arts², National United University³


Semantic Segmentation, Neural network, Style transfer, Non Photorealistic Rendering

Neural style transfer is usually suitable for use in abstract styles. When used in styles such as Japanese animation whose foreground is more complex than their background, the results are often not as good as expected. We design a method to automatically transfer the style for video with this type of style. We combine semantic segmentation and spatial control to transfer the specified style to the specified area. By designing the initial image and the loss function, we fixed the distortion of the face and the incomplete style transfer. We propose a method to provide users with the ability to adjust the feature weights of different regions to maintain the artistic conception of the target style, and combine optical flow to ensure coherence from frame to frame in the video.

Paper No. 50

234

Funiture layout design aided system using subjective reward

Tomohiro Matsuno, Yuji Hatanaka, Wataru Sunayama, Kazunori Ogohara

The University of Shiga Prefecture


Layout design, optimization, Q learning, Genetic Algorithm, subjective reward, virtual reality

Many studies for layout design optimization depends on evaluation indexes with necessary passage, spaciousness, etc. It was difficult for everyone to obtain friendly layout by using the conventional methods. The layout design model of this study is the residence space, there are eight kinds of furniture. All furniture were first classified into each rooms by using genetic algorithm. Classified furniture were then decided initial arrangements in each rooms by using Q learning. User checks the initial layout structured by virtual reality, and he evaluates by subjectivity. The layout for a specific user was flexible fixed by applying Q learning added user subjective reward.

Paper No. 77

235

Speech motion feedback system using semi-transparent screen

Sayoko Takano¹, Makoto J. Hirayama²

Kanazawa Institute of Technology¹, Osaka Institute of Technology²


speech motion feedback system, jaw, lip, semi-transparent screen, 3D position sensor

We propose a speech motion feedback system for improving articulation by showing 3D-CG jaw and abstracted lip. The subjects’ motion of the jaw is captured by a 3D position with rotation sensor. The lip motion is measured by four 3D position sensors using infrared emission. The subjects observe his/her own face on LCD screen, and 3D-CG jaw and abstracted lip motion on semi-transparent screen. The subjects reported that they noticed the importance of the motion of the speech organs after the experiment.

Paper No. 84

236

Automatic system for life-stage detection of drosophila with single animal resolution toward high-throughput screening

Taishi Matsumura¹, Ki-Hyeon Seong², Maki Otori³, Mayu Kudo³, Manae Kashimura³, Tetsuya Yuasa³, Siu Kang³

Yamagata University¹, RIKEN Tsukuba Institute², Yamagata University³


segmentation, image processing, signal processing, automatic detection system, deep neural network

Drosophila, as known as fruit fly, is a holometabolous insect typically showing four life-stages’embryo, larva, pupa, and adult. Fruit fly is often used as a model animal in gene manipulation research since it is easily bred and manipulated with genetic operation. However, detection of the life-stages is manually performed in laboratories. In this paper, we propose the automatic life-stage detection system. We trained the neural network specializing in a semantic segmentation task. As a result, the network could detect pixel-wise an animal in a picture. Moreover, simple signal processing supported to achievethe precise detection of the life-stages comparable with manual operation.

Paper No. 86

237

Year of Wine Aromas Classification by Using Principal Component Analysis as Feature Reduction

Sirichai Turmckokksam

Bangkok University

[email protected]

electronic noses, linear regression, percent classification rates

In the area of electronic noses (e-nose), applications in the field of wine aromas detection are uncommon. The number of qualified human wine experts is low and their cost is high. This paper has been developed for the purpose of recognition of typical aromas in red wines at a low cost. We propose simple linear regression analysis to classify typical aromatic compounds in wine by years of an electronic nose and using feature reduction-based method, principal component analysis (PCA) as feature extraction techniques show datasets of this group of compounds are clearly improved the requirement as follows percent classification rates (performance evaluation). The experiment simple linear regression analysis classification results different types of wine grapes percentage of correlation extract and different years of wine grapes the percent classification rates.

Paper No. 88

238

Development of Head Direction Measuring System using Ultrasound for Environmental Education

Yuya Nakata¹, Kantaro Tabiraki², Takayuki Nakata¹

Toyama Prefectural University¹, Matsumoto University²


Ultrasound, Environmental education, Measuring head direction

In environmental education, learning evaluation is generally used questionnaires. However, the questionnaires can only evaluate the set learning goal. Hitherto, the method of learning evaluation for getting bio-information of students in educational activities has been considered. In this research, we use gaze direction for learning evaluation. The environmental education targets a large number of students in outdoor setting. In outdoor environmental, it is difficult to apply conventional gaze estimation technology using cameras because the influence of visibility failure caused sunlight and mist is large. In addition, the wearable device gives the student a physical burden, which may affect the detection result. In this research, we use ultrasound. We equip small ultrasonic speakers to cap and helmet to reduce the physical burden on the students. Three ultrasonic speakers are mounted linearly on a cap or helmet. Head direction is measured from time difference of these speakers, and gaze direction is estimated. To deal with a large number of students, each ultrasonic speaker transmits different M-sequences to improve the spectral efficiency. In the experiment, we attached ultrasonic speakers on both sides on a helmet and measured the horizontal head direction.

Paper No. 116

239

Remarkable point extraction from overlapping analysis of view fields in multiple smartphones shooting environment

Shogo Tokai, Fumiaki Motoyama

University of Fukui


multi-views processing, viewing directions, smartphone, attitude sensor, spectator

In this paper, we propose a method of object tracking using viewing directions of multiple spectators’ cameras. Attitude sensors in a smartphone can measure its shooting direction on time. By collecting the directions and analyzing the overlapping situation of multiple view fields, a remarkable part of the scene situation can be extracted as the most overlapped area of them. We implemented the analysis by using the blending function with a configuration of interested degree map in the view fields. We adopted it to actual sport scene situations, and we got good estimation results close to the ground truth.

Paper No. 123

240

A Modified Hierarchical k-Nearest Neighbor Method with Application to Land Cover Classification

Tatsuya Hayashi¹, Hakaru Tamukoh¹, Ryosuke Kubota²

Kyushu Institute of Technology¹, National Institute of Technology, Ube College²


land cover classification, real remote sensing image, k nearest neighbor

In this paper, we propose a land cover classification method based on a modified hierarchical k-nearest neighbor (MHkNN) in order to realize the accurate classification. The proposed method introduces a reliability of each training data.The reliability means a confidence in belongingness to the class. The proposed method performs majoring vote with considering not only the number of the training data, but also their reliabilities. The classification performance of the proposed methodis compared with the conventional land cover classification methods. The effectiveness and validity of the proposed method are confirmed by applying it to the real remote sensing images.

Paper No. 174

241

Panoramic image generation from a handheld video by combination of attitude sensors and image features

Shinnosuke Yabuuchi, Shogo Tokai

University of Fukui


panoramic image, smartphone, handheld shooting, attitude sensors, projective transformation

In this paper, we propose a method to generate a panoramic image from a video sequence which is shot by a handheld smartphone. The panoramic image can be made by projective transformation with a homography matrix estimation for each frame using corresponding of image features. However, it is difficult to get a precise result because of blurring on the video by handheld operations, and movements of shooting objects in the scene. Therefore, we consider using attitude sensor information together to compensate for the failure of embedding a frame onto the panoramic image. We explain a method for it and show several experimental results with considerations.

Paper No. 185

242

Basic research into applications for consumer to understand the value of recycled products -Increase the Value and Attraction of Waste Polythene Bag

through Product Design-

Huang Sih Ping, Tanaka Takamitsu

Iwate University


Waste polythene bag, Eco-friendly, Eco-friendly application

Polythene bag is commonly used in the daily life but also responsible for causing pollution. The purpose of this study was through product design to increase the value of the waste polythene bag and draw customer’s attention through online shopping applications while people’s shopping habit had changed into online mode instead of store. Conveying the eco-friendly specialty on-screen display is necessary. Therefore, this study used an online survey to figure out the attractive features of the waste polythene bag products such as insulation property and unique patterns. The scale of the points for each feature was ranging from one as lowest and seven as the highest point. The result showed Over 80 % of the people gave ≥6 points to the insulation property and over 60 % of the people gave ≥6 points to the unique pattern feature. In conclusion, applying the polythene bag board on products was attractive to people. Therefore, through product design, the waste polythene bag can be a resource and catch customer’s eye with its eco-friendly specialty through applications.

Paper No. 189

243

Video-haptic communication with motion-tracking device

Song Guo¹, Jia Jing Jeslyn Goh², Alexei Sourin²

Fraunhofer Singapore¹, Nanyang Technological University²


haptic interaction, new user-computer interfaces, internet communication, video communication

Video-conferencing and video calls are frequently used nowadays, but more immersive communications where the participants can also physically feel each other are still to be explored. Different from earlier attempts of connecting haptic devices only, we propose to use more types of devices to track body parts, like a hand or a finger, of one user, and to send it to the other user in the form of coordinates. As the receiving device reads the data and reacts to the tracked motion, the hand or finger becomes tangible to the other party. To host such connection over the Internet, cloud servers can be used to link the two users. In this paper we study how smooth and consistent such haptic communications can be performed, as well as the effect of long distances to the user experience in the connections.

Paper No. 205

244

Physical Condition Analysis of Road Signs using Image Template

Wan-Noorshahida Mohd-Isa, Nurul-Farhana Md-Saffi, Shahbe Mat Desa, Noramiza Hashim, Zarina Che Embi, Junaidi Abdullah, Aziah Ali




road signs, condition analysis, template matching, template difference

Road signs play an essential role in ensuring road safety. However, due to external factors such as bad weather, road signs may not be in their best condition that may hinder effective communication between the road and the road users. Although the study on detection and classification of road signs is becoming more common these days, there is yet to exist a study that analyzes whether a road sign is in a good or bad condition. The basis of this study is to implement image template algorithms on road signs that will be used to analyze their physical condition as good or damaged. In addition to template matching, this study proposes a template difference method for its condition analysis. Analyses on three shapes of road signs: circular, triangular, and diamond give out the average correctly classified detection as 60% and 78%, respectively for template matching and template difference method.

Paper No. 229

245

Real-Time Air-writing Recognition in Motion Stream

Buntueng Yana, Takao Onoye

Osaka University

[email protected], [email protected] recognition, Real-time approach, Motion Stream, Path signature, Long Short-Term Memory, Connectionist

Temporal Classification

The contribution of this work is developing an end-to-end air-writing recognition technique for a real-time application. We assume the user performs the air-writing in a natural and intuitive way without doing an explicit signal. Learn the air-writing trajectory without spotting, the LSTM network with CTC loss is considered. This work model the motion trajectory as a transition of hand movement. We utilize a window-based technique to capture the writing information. From the experiment, an appropriate size of the window for the proposed structure is 0.5 second. The hand position and the path signature features are deployed to train the network. The experiments are examined in a public dataset, namely the air finger writing. From the results, the proposed network can recognize the air-writing word 50.02% without the language model. When considering the processing time of the recognition technique, the air-writing could predict the written word within 4.44 milliseconds.

Paper No. 232

246

Video Coding

Two-Stage Neural Network for Intra Prediction Mode Decision

Yukiya Seki¹, Yoshiaki Shishikui¹, Shunsuke Iwamura²

Meiji University¹, NHK²


HEVC, Image coding, Intra prediction, Neural Network, CNN

In the H.265 / HEVC standard reference software HM, rate distortion (RD) optimization is used for intra prediction mode determination. However, the calculation complexity of RD optimization is high. In this research, instead of HM’s RD optimization, we consider a method to determine the prediction mode in the neural network. Generally, networks with deep layers have high accuracy but high calculation complexity. On the other hand, in networks with shallow layers, the calculation complexity is low but the accuracy decreases. Therefore, we examined a method to reduce computational complexity in slight deterioration of accuracy. We use two types of neural networks differently depending on the frequency of prediction mode.

Paper No. 79

247

Methods of Inactive Region Padding for Rotated Sphere Projection (RSP) of 360 Video

Yong-Uk Yoon, Hyun-Ho Kim, Jae-Gon Kim

Korea Aerospace University


JVET, video coding, VVC, 360 video, projection format

In recent years, 360 videos have been attracting increasing attention as a new type of video that provide an immersive experience. In JVET, which is developing Versatile Video Coding (VVC) as a next generation of video coding standard with capability beyond HEVC, 360 videos are included in the scope along with SDR and HDR content. Existing video codecs are designed considering conventional 2D video. Therefore, in the workflow of 360 video coding of JVET, firstly the 360 video is projected onto the 2D plane with a projection format. Some projection formats have inactive regions in the converted 2D plane such as rotated sphere projection (RSP) and segmented sphere projection (SSP). This paper proposes methods of inactive regions padding to improve coding efficiency and reduce visual artifacts along the face boundaries. The proposed method of inactive region padding gives -0.07% and -0.07% average coding gain of BD-rate in terms of end-to-end S-PSNR and WS-PSNR, respectively. In addition, the visual artifacts along the edges of discontinuous faces could be reduced.

Paper No. 165

248

Block Adaptive CNN/HEVC Interframe Prediction for Video Coding

Satoru Jimbo, Ji Wang, Yoshiyuki Yashima

Chiba Institute of Technology


video coding, CNN, interframe prediction

Recently, video traffic on the Internet continues to increase due to higher definition of images and popularization of SNS, and a lot of video compression technologies that have compression performance beyond that of HEVC are being studied. One of those approaches is to apply machine learning to video coding. The authors have already examined the method of applying CNN to bidirectional interframe prediction. However, the conventional method has a problem that the prediction efficiency for videos with large motion becomes worse than the normal HEVC prediction efficiency. In this paper, we introduce a method to adaptively switch the CNN-based and the HEVC-based bidirectional interframe prediction block by block. Experimental results show that the prediction error using the proposed method can be reduced to 56% to 93% for almost videos with various motions compared to HEVC prediction.

Paper No. 169

249

Boundary Handling For Video Based Light Field Coding With A New Hybrid Scan Order

Thuong Nguyen, Byeungwoo Jeon

Sungkyunkwan University


light field, image coding, video coding, pseudo-sequence, future video coding

In video based light field (LF) coding, sub-aperture images (SAIs) are ordered to form a pseudo video sequence which is encoded by a video compression algorithm, for example, by HEVC. When the size of SAI is not divisible by the minimum coding tree unit size, proper boundary handling method is required. This paper investigates several boundary handling methods which can either increase or decrease the size of SAIs. To maintain a high quality of the central SAI, we combine rotation and u scan to have a new hybrid scan order.

Paper No. 172

250

Joint Super-Resolution and Bit Depth Extension by DNN

Seiya Umeda¹, Hiroshi Watanabe¹, Tomohiro Ikai², Tomonori Hashimoto², Takeshi Chujoh², Norio Ito²

Waseda University¹, Sharp Corporation²


Future Video Coding, Super-resolution, HDR, Bit depth

Many Super-Resolution processes using DNN have been proposed. It expands the image size. On the other hand, bit depth expansion is also proposed independently. In this paper, we consider the image enlargement processing with bit depth expansion by Super-Resolution processing. Experimental results show its effectiveness in both SDR and HDR images.

Paper No. 201

251

Lossless Video Coding Based on Multi-Frame Example Search and Probability Model Optimization

Koji Nemoto, Yusuke Kameda, Ichiro Matsuda, Susumu Itoh

Tokyo University of Science


Lossless video coding, example search, probability model optimization

This paper describes a lossless video coding method that directly estimates a probability distribution of image values pel-by-pel. In the estimation process, several examples, i.e. a set of pels whose neighborhoods are similar to a local texture of the target pel to be encoded, are gathered from search windows located on an already encoded area of the current frame as well as those of the previous frames. Then the probability distribution is modeled as weighted sum of the Gaussian functions whose center positions are given by the individual examples. Furthermore, model parameters that control shapes of the Gaussian functions are numerically optimized so that the resulting coding rate can be a minimum. Simulation results indicate that the coding performance can be improved by increasing the number of reference frames.

Paper No. 233

252