Research ArticleDynamic Self-Occlusion Avoidance Approach Based onthe Depth Image Sequence of Moving Visual Object

Shihui Zhang12 Huan He1 Yucheng Zhang1 Xin Li1 and Yu Sang1

1School of Information Science and Engineering Yanshan University Qinhuangdao 066004 China2The Key Laboratory for Computer Virtual Technology and System Integration of Hebei Province Qinhuangdao 066004 China

Correspondence should be addressed to Shihui Zhang sshhzzysueducn

Received 16 May 2016 Accepted 30 August 2016

Academic Editor Alessandro Gasparetto

Copyright copy 2016 Shihui Zhang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

How to avoid the self-occlusion of a moving object is a challenging problem An approach for dynamically avoiding self-occlusionis proposed based on the depth image sequence of moving visual object Firstly two adjacent depth images of a moving object areacquired and each pixelrsquos 3D coordinates in two adjacent depth images are calculated by utilizing antiprojection transformation Onthis basis the best view model is constructed according to the self-occlusion information in the second depth image Secondly theGaussian curvature feature matrix corresponding to each depth image is calculated by using the pixelsrsquo 3D coordinates Thirdlybased on the characteristic that the Gaussian curvature is the intrinsic invariant of a surface the object motion estimation isimplemented by matching two Gaussian curvature feature matrices and using the coordinatesrsquo changes of the matched 3D pointsFinally combining the best viewmodel and themotion estimation result the optimization theory is adopted for planning the camerabehavior to accomplish dynamic self-occlusion avoidance process Experimental results demonstrate the proposed approach isfeasible and effective

1 Introduction

Self-occlusion avoidance refers to the fact that when the self-occlusion of visual object occurs the visual system changesthe camera observation direction and position by usingcurrent detected self-occlusion cue to observe self-occlusionregion further and obtains more information of the visualobject so as to accomplish the related visual task betterTherefore the main problem to be solved on self-occlusionavoidance is how to determine the best observation directionand position of the camera by utilizing the occlusion cue ofcurrent visual object

According to spatiotemporal attribute difference of thevisual object the self-occlusion avoidance problem can beclassified into static self-occlusion avoidance and dynamicself-occlusion avoidance Static self-occlusion avoidancerefers to the fact that when the visual object is staticthe self-occlusion phenomenon is avoided by moving thecamera to a next best view for observing the self-occlusionregion detected in an image of the visual object Dynamic

self-occlusion avoidance refers to the fact that when the visualobject is moving the self-occlusion phenomenon is avoidedby moving the camera step by step to the best view forobserving the self-occlusion region detected in one framefrom depth image sequence of the moving visual object Bythe concept above we can see themain difference between thetwo cases is that the result of static self-occlusion avoidanceis a next best view under self-occlusion and the cameracan accomplish the task of self-occlusion avoidance in thecalculated view which can be regarded as ldquoone-steprdquo typeWhile the result of dynamic self-occlusion avoidance is aseries of next views the camera needs multiple motions toavoid self-occlusion which can be regarded as ldquostep-by-steprdquotype

Compared with static self-occlusion avoidance thevisual object keeps moving in the process of dynamicself-occlusion avoidance which makes it impossible toaccomplish the dynamic self-occlusion avoidance throughmoving the camera to the calculated next best viewdirectly In order to accomplish the dynamic self-occlusion

Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2016 Article ID 4783794 11 pageshttpdxdoiorg10115520164783794

2 Mathematical Problems in Engineering

avoidance besides calculating the next views the motionestimation about the moving visual object is necessaryThen the camera not only does synchronous motionwith the visual object according to the motion estima-tion result but also executes self-occlusion avoidance func-tion simultaneously In this way the camera moves stepby step until accomplishing the dynamic self-occlusionavoidance

Before self-occlusion avoidance problem emerging thesimilar problemcalled next best viewhas beenwidely studiedCurrently scholars have gained some achievements on thenext best view Connolly [1] as one of the earlier scholarsstudying the next best view used partial octree model todescribe visual object and made different marks to the nodesof different observation situations so as to determine thenext best view By discretizing a fixed surface Pito [2]determined the next best view from plenty of discretizationcandidate views Based on the depth data Whaite and Ferrie[3] constructed parameter similarity model of object andplanned the next best view of the camera according to thedifference between the depth data and the current fittedmodel Li and Liu [4] proposed a method which used B-spline to construct the model of object and determined thenext best view by calculating information gain Trummeret al [5] proposed a next best view method by combiningon-line theory to optimize the 3D reconstruction precisionof any object Unfortunately because of never consideringocclusion factor in thesemethods themore serious the occlu-sion is the more the accuracy of these methods would beaffected

Because ubiquitous occlusion would affect the result ofthe next best view scholars further proposed the next bestview methods taking occlusion into account Banta et al[6] proposed a combination method to determine the nextbest view under occlusion Based on the integration of activeand passive vision Fang and He [7] determined the nextbest view through the shape of the shadow region andthe concept of limit visible surface Combining layered raytracing and octree Vasquez-Gomez et al [8] constructedthe object model and generated candidate views based onsorting of the utility function to determine the next bestview Potthast and Sukhatme [9] determined the next bestview through the difference of information entropies underocclusion Wu et al [10] determined the next best view byusing the algorithm of layered contour fitting (LCF) basedon density Although these methods consider the factor ofocclusion (mutual occlusion or self-occlusion this paperpays attention to self-occlusion) there are limitations in thecamera position [6 7] specific equipment [8 9] a prioriknowledge [7 10] and so forth And besides the self-occlusion region is not modeled properly in these next bestview methods Moreover there exist some differences (suchas problem description and solving cue) between the nextbest view and self-occlusion avoidance so the abovemethodshave some reference significance but cannot be taken as thefinal solution to the problem of self-occlusion avoidanceMost importantly all of the above methods aim at the studyof static visual object and they are not suitable for the

moving visual object That is to say all of the above methodscannot be the solution to dynamic self-occlusion avoidanceHowever in many scientific research fields such as real-time 3D reconstruction object tracking autonomous navi-gation scene recognition of mobile robot robot autonomousoperation and dynamic scene rendering the self-occlusionof moving visual object is a universal phenomenon Mean-while the visual system would be invalid even wrongif it cannot effectively detect and avoid self-occlusion ofmoving visual object thus the visual system would loseits value which makes how to avoid the self-occlusionof moving visual object become an inevitable and urgentproblem

According to investigation result there is no relatedresearch on dynamic self-occlusion avoidance of movingvisual object that is to say the research on dynamic self-occlusion avoidance is still in initial stage at present Mean-while considering the objective fact that the 3D informationof a scene can be better obtained from the depth imagethan the intensity image an approach for avoiding the self-occlusion of the rigid motion object is proposed in this paperbased on the depth image In the process of designing themethod wemainly solve the following three problems firstlyhow to design the solution to the dynamic self-occlusionavoidance problem secondly how to estimate the motionof visual object based on depth image thirdly how to mea-sure the effect of dynamic self-occlusion avoidance methodAiming at the first problem the dynamic self-occlusionavoidance is posed as an optimization problem and motionestimation is merged into the optimization process of bestview model to solve the problem of dynamic self-occlusionavoidance Aiming at the second problem the two Gaussiancurvature feature matrices are matched by SIFT algorithmand the efficient singular value decomposition (SVD) is usedto estimate themotion of visual object For the third problemldquoeffective avoidance raterdquo is proposed to measure the per-formance of the dynamic self-occlusion avoidance methodThe rest of the paper is organized as follows Section 2 isthe method overview Section 3 describes the dynamic self-occlusion avoidancemethod based on depth image Section 4presents our experimental results and Section 5 concludes thepaper

2 Method Overview

21 The Analysis of Dynamic Self-Occlusion AvoidanceThe problem of dynamic self-occlusion avoidance can bedescribed on the premise that the self-occlusion region inan image of visual object image sequence is taken as theresearch object the reasonable next view sequence is plannedfor observing the self-occlusion region bymoving the camerato achieve the goal that the most information about the self-occlusion region can be obtained from the camera final viewBecause the visual object is moving in the process of self-occlusion avoidance the next view of the camera shouldbe adjusted dynamically to observe the vested self-occlusionregion

Mathematical Problems in Engineering 3

Ideal object modelCamera

(a) (b) (c)

Figure 1 The sketch map of camera observing ideal object model(a) Observing effect in initial view (b) Observing effect duringavoiding self-occlusion (c) Observing effect at the end of avoidingself-occlusion

Figure 1 shows the sketch map of camera observing idealobject model Figure 1(a) is the observing effect of camerain initial view and the shadow region is the self-occlusionregion to be avoided In the case of object moving if camerawants to obtain the information of the self-occlusion regionit must do the synchronousmotion with the visual object andmove to the best view for observing self-occlusion region atthe same time Figure 1(b) is the observing effect when thecamera is avoiding self-occlusion Figure 1(c) is the observingeffect when the camera arrives at the final view in whichthe camera can obtain the maximum information of self-occlusion region so the process of dynamic self-occlusionavoidance is accomplished when the camera arrives at thisview

22 The Overall Idea of Dynamic Self-Occlusion AvoidanceBased on the analysis of dynamic self-occlusion avoidanceabove an approach to dynamic self-occlusion avoidance isproposed The overall idea of the approach is as followsFirstly two adjacent depth images of the moving objectare acquired and each pixelrsquos 3D coordinates (all pixelsrsquo 3Dcoordinates are in the same world coordinate system) intwo depth images are calculated by utilizing antiprojectiontransformation and the self-occlusion cue in the seconddepth image is detected On this basis the best view modelis constructed according to the self-occlusion informationin the second depth image Secondly according to the 3Dcoordinates calculated above the Gaussian curvature featurematrices corresponding to the two adjacent depth imagesare calculated by using the pixelsrsquo 3D coordinates Then theGaussian curvature feature matrices corresponding to thetwo adjacent depth images are matched by SIFT algorithmand the motion equation is estimated by using the 3Dcoordinates of the matched points Finally combining thebest view model and the estimated motion equation ofthe visual object a series of next views of the camera areplanned to accomplish dynamic self-occlusion avoidanceprocess

3 The Approach to Dynamic Self-OcclusionAvoidance Based on Depth Image

31 Constructing the Self-Occlusion Region to BeAvoided and the Best View Model

311 Constructing the Self-Occlusion Region to Be AvoidedIn order to solve the problem of dynamic self-occlusionavoidance it is necessary to first construct the self-occlusionregion to be avoided As we know the information of self-occlusion region in current view is unknown that is to saythe geometry information of self-occlusion region could notbe obtained directly according to the depth image acquiredin current view so the specific modeling method should beadopted to describe the self-occlusion region approximatelyFigure 2 shows the local self-occlusion region of visualobject and its approximate description and in Figure 2(a)the red boundary is self-occlusion boundary and the blueboundary is the nether adjacent boundary correspondingto the red one The method in literature [11] is used todetect the self-occlusion boundary in depth image and all thepoints on self-occlusion boundary are organized to form self-occlusion boundary point set119874 As shown in Figure 2(b) theregion between self-occlusion boundary and nether adjacentboundary in 3D space is the unknown self-occlusion regionFigure 2(c) shows the construction of self-occlusion regionmodel by utilizing quadrilateral subdivision on unknownself-occlusion region

The concrete steps of constructing self-occlusion regionmodel are as follows Firstly take out the 119894th self-occlusionboundary point 119900

119894from self-occlusion boundary point set 119874

in turn Mark the unmarked nonocclusion boundary pointcorresponding to themaximum of depth differences betweenthe self-occlusion boundary point 119900

119894and its eight neighbors

as 1199001015840119894 and add 119900

1015840

119894into nether adjacent boundary point set

1198741015840 Secondly use the neighboring self-occlusion boundary

points 119900119895 119900119895+1

in 119874 and their corresponding nether adjacentpoints 1199001015840

119895 1199001015840119895+1

in 1198741015840 to form quadrilateral patch

119895 as shown

in Figure 2(c) 119895 is the integer from 1 to 119873 and the com-bination of all 119873 quadrilaterals is approximate descriptionof self-occlusion region Thirdly combining depth imageand camera parameters use antiprojection transformationto calculate each pixelrsquos 3D coordinates At last use the 3Dcoordinates of quadrilateral patch

119895rsquos four vertices to calculate

its normal and area thus the modeling process of self-occlusion region can be accomplished

On the premise of not considering the normal directionthe formula for calculating the normal k1015840patch119895 of quadrilateralpatch119895can be defined as

k1015840patch119895 = u119895times w119895

or k1015840patch119895 = w119895times u119895

(1)

where u119895is the vector from point 119900

119895to the midpoint of 1199001015840

119895

and 1199001015840

119895+1in 3D space and w

119895is the vector from point 119900

119895to

point 119900119895+1

in 3D space Supposing the initial camera view is

4 Mathematical Problems in Engineering

oj

oj+1

o998400

j

o998400

j+1

patchj

Self-occlusion regionmodelUnknown self-

occlusion regionSelf-occlusion boundary

Upper surface

Nether surface

(a) (b) (c)

Modelling

Figure 2 The local self-occlusion region and its approximate description for visual object (a) The depth image of visual object (b) Spatialstructure of local self-occlusion region (c) Approximate description of self-occlusion region

(xbegin kbegin) in order to ensure that the direction of calcu-lated normal points outside in the process of self-occlusionregion modeling the calculation formula of patch

119895rsquos normal

kpatch119895 is defined as

kpatch119895 = sign (minusk1015840patch119895 sdot kbegin) sdot k1015840

patch119895 (2)

where sign is the sign function it means the function valueis 1 when minusk1015840patch119895 sdot kbegin ge 0 and the function value is minus1when minusk1015840patch119895 sdot kbegin lt 0 The area 119878

119895of quadrilateral patch

119895

is defined as

119878119895=100381710038171003817100381710038171003817kpatch119895

1003817100381710038171003817100381710038172 (3)

In summary the set 119875 consisting of all quadrilateralspatch119895is the modeling result of self-occlusion region so the

self-occlusion information to be avoided is described as

119875 = (119900119895 119900119895+1

1199001015840

119895 1199001015840

119895+1 vpatch119895 119878119895) | 119900

119895 119900119895+1

isin 119874 1199001015840

119895 1199001015840

119895+1isin 1198741015840

(4)

312 Constructing the Best ViewModel The best view modelshould be constructed after obtaining the self-occlusioninformation to be avoided This paper uses the area ofvisible self-occlusion region on next view to construct theobjective function and meanwhile uses the size equality ofreal object corresponding to pixels in different depth imagesas constraint condition So the best view model is defined as

argmaxxBV

119873

sum

119895=1

119878119895cos (120579

119895(xBV))

st 120579119895(xBV) =

120579119895(xBV) 120579

119895(xBV) lt

120587

2

120587

2120579119895(xBV) ge

120587

2

1003817100381710038171003817xmid minus xBV10038171003817100381710038172 =

10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172

(5)

where xBV and xmid minus xBV are respectively the position anddirection of camera for observing the vested self-occlusionregion at maximum level119873 is the number of quadrilaterals120579119895(xBV) is the angle between the normal of 119895th quadrilateral

and the vector from the midpoint of 119895th quadrilateral tocamera xmid is the center of gravity of all visible points ininitial camera view (xbegin kbegin)

32 Estimating theMotion Equation of the Visual Object Basedon Two Adjacent Depth Images Different from the static self-occlusion avoidance the dynamic self-occlusion avoidance iscarried out on the premise that the visual object is movingwhichmakes themotion estimation of the visual object in 3Dspace become a necessary step in the process of dynamic self-occlusion avoidance Because the existing motion estimationmethods based on depth image all estimate the motion ofthe visual object in 2D images this paper designs a feasiblescheme based on the depth image to estimate the motion ofthe visual object in 3D space

321 Calculating the Gaussian Curvature Feature Matricesof the Two Adjacent Depth Images Considering that rigidbody has rotation and translation invariant characteristicsthe surface shape of visual object remains invariant inthe motion process Because the curvature feature reflectsbending degree of the object surface and Gaussian curvatureonly depends on Riemannian metric of the surface which isthe intrinsic invariant of the surface (ie Gaussian curvaturekeeps invariant in rotation and translation) the motion ofvisual object is estimated based onGaussian curvature featurematrices corresponding to the two adjacent depth images

In 3D Euclidean space 1198961and 119896

2are two principal

curvatures of a point on the differentiable surface then theGaussian curvature is defined as the product of two principalcurvatures namely 119896 = 119896

11198962 The method of calculating

the Gaussian curvature in literature [12] is implemented byoptimizing convolution calculation and it is high-efficientand suitable for different scales of curvature value so the

Mathematical Problems in Engineering 5

method in literature [12] is used for calculating the Gaussiancurvature feature matrix of depth image

322 Estimating the Motion Equation of Visual Object Basedon the Matched Feature Points Analysis shows that thematched points can provide reliable information for themotion estimation if the Gaussian curvature feature matricescorresponding to two adjacent depth images of the visualobject can be matched SIFT algorithm in literature [13]is more efficient and it is not only invariant to imagetranslation rotation and scaling but also robust to thechange of visual angle affine transformation and noise sothe SIFT algorithm is adopted in this paper for matching thefeature points

SIFT algorithm mainly consists of two steps the gener-ation of SIFT features and the matching of feature vectorsIn order to generate the SIFT features firstly the scale spaceshould be established Because the Difference of Gaussiansscale-space (DoG scale-space) has good stability in detectionof key points the DoG scale-space is used for detecting thekey points in this paper Secondly the key points which areinstable and sensitive to noise are filtered Then the directionof each feature point is calculated Finally the SIFT featurevector of each key point is generated In the process ofgenerating SIFT feature vector 16 seed points are selectedfor each key point and each seed point has 8 orientationsTherefore the 16 times 8 = 128 dimensions of the SIFT featurevector can be obtained for each key point After obtainingthe SIFT feature vectors of the Gaussian curvature featurematrices corresponding to the two adjacent depth images thetwoGaussian curvature featurematrices can bematched andthe similarity of the vectors ismeasured byEuclidean distancein the process of matching

After the key points of the Gaussian curvature featurematrices corresponding to the two adjacent depth images arematched the motion equation of visual object can be esti-mated by using the pixelsrsquo 3D coordinates which are obtainedby antiprojection transformation The motion equation isdefined as

x1015840119897= Rx119897+ T (6)

where x119897and x1015840119897are respectively the 3D coordinates of points

before and after visual objectmotionR is the unit orthogonalrotation matrix and T is the translation vector then theresult of motion estimation is the solution of minimizing theobjective function 119891(RT) The objective function 119891(RT) isdefined as

119891 (RT) =119872

sum

119897=1

10038171003817100381710038171003817x1015840119897minus (Rx

119897+ T)10038171003817100381710038171003817

2

(7)

where119872 is the number of key points which are matched Inorder to solve the R and T in formula (7) the matrix H isdefined as

H =

119872

sum

119897=1

x1015840clx119879

cl (8)

where x1015840cl = x1015840119897minus (1119872)sum

119872

119897=1x1015840119897 xcl = x

119897minus (1119872)sum

119872

119897=1x119897

By carrying out singular value decomposition on H H canbe expressed as H=UΛV119879 then according to U and V R informula (7) can be deduced as

R = VU119879 (9)

After obtaining R substitute R into formula (10) tocalculate T namely

T = x1015840119897minus Rx119897 (10)

After obtaining R and T the motion equation shown informula (6) can be obtained and themotion estimation of thevisual object is accomplished

33 Planning the Next View Based on Motion Estimationand Best View Model After the motion equation of visualobject is estimated the next view of the camera can beplanned Analysis can be known if substituting the cameraobservation position into the motion equation of visualobject the camera can do the synchronous motion with thevisual object In addition on the basis of the synchronousmotion with the visual object if the camera motion offsetfor avoiding self-occlusion is introduced the whole processof dynamic self-occlusion avoidance can be achieved Byanalyzing the best view model in formula (5) we can knowit is the camera best view that can make formula (11) achievemaximum value Formula (11) is defined as

119891 (x) =119873

sum

119895=1

119878119895cos (120579

119895 (x)) (11)

where x is the camera observation position Since formula (11)is a nonconvex model it is difficult to directly calculate itsglobal optimal solution Meanwhile because the problem ofdynamic self-occlusion avoidance needs to calculate a seriesof next views and the gradient information in formula (11) canprovide basis for determining the cameramotion offset of theself-occlusion avoidance process in the process of dynamicself-occlusion avoidance the gradient descent idea is usedin this paper for defining the formula of camera observationposition x1015840

119873119881as

x1015840119873119881

= Rx119881+ T + 120575nabla119891 (x

119881) (12)

where x119881is the current observation position of the camera

and 120575 is the offset coefficient Based on the solution to formula(12) the formula of next observation direction k

119873119881is defined

as

k119873119881

= xmid minus x1015840119873119881

(13)

According to the constraint condition in formula (5) wecan search one point x

119873119881along the opposite direction of k

119873119881

and make x119873119881

meet1003817100381710038171003817xmid minus x

119873119881

10038171003817100381710038172 =10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172 (14)

By now the obtained view (x119873119881

v119873119881

) can be regardedas the new current view (x

119881 v119881) After acquiring the depth

6 Mathematical Problems in Engineering

image of visual object in the new view (x119881 v119881) the process

from formula (6) to formula (14) is repeated until thedifference between two 119891(x) which are calculated in twoadjacent views according to formula (11) is less than a giventhreshold It means that a series of views (x

119873119881 v119873119881

) can becalculated during the visual object motion so the process ofdynamic self-occlusion avoidance can be accomplished

34 The Algorithm of Dynamic Self-Occlusion Avoidance

Algorithm 1 (dynamic self-occlusion avoidance)

Input The camera internal and external parameters and twoadjacent depth images acquired in initial camera view

Output The set of a series of camera views corresponding tothe dynamic self-occlusion avoidance

Step 1 Calculate the pixelsrsquo 3D coordinates and the Gaussiancurvature feature matrices corresponding to the two adjacentdepth images

Step 2 Detect the self-occlusion boundary in the seconddepth image and establish the self-occlusion region in thesecond depth image according to formula (1) to formula (4)

Step 3 Construct the best view model according to self-occlusion information (formula (5))

Step 4 Match the key points based on Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages by using SIFT algorithm

Step 5 Based on the 3D coordinates of matched key pointssolve the objective function in formula (7) according toformula (8) to formula (10) then the visual objectrsquos motionequation in formula (6) can be obtained

Step 6 Plan the next view of camera according to formula (11)to formula (14)

Step 7 In the obtained next view acquire a depth image andcalculate 119891(x) in formula (11)

Step 8 If the difference between two adjacent 119891(x) is lessthan a given threshold the process of dynamic self-occlusionavoidance can be terminated and otherwise jump to Step 4

to continue the process of dynamic self-occlusion avoidance

4 Experiments and Analysis

41 Experimental Environment In order to validate the effectof the method in this paper the OpenGL is adopted toimplement the process of dynamic self-occlusion avoidanceduring camera observing the moving object based on 3Dobject model in the Stuttgart Range Image Database Theexperimental hardware environment is Intel CPU (R) Core(TM) i7-3770 340GHz and the memory is 800GB Thedynamic self-occlusion avoidance program is implemented

with C++ language In the process of self-occlusion avoid-ance the parameter of the projection matrix in OpenGL is(60 1 200 600) and the window size is 400 times 400

42 The Experimental Results and Analysis To verify thefeasibility of the proposed method we do experiments withdifferent visual objects in different motion modes Duringthe experiments the initial camera observation position isxbegin = [000 minus100 30000]

119879 and the initial observationdirection is kbegin = [000 100 minus30000]

119879 The coordinateunit is millimeter (mm) and the offset coefficient is 120575 = 4Theprocess of dynamic self-occlusion avoidance will terminate ifthe calculated difference of 119891(x) between two adjacent viewsis less than the given threshold 30mm2 The offset coefficientand threshold are chosen by empirical value Due to thelimited space here we show 3 groups of experimental resultscorresponding to the visual object Bunny with relatively largeself-occlusion region and Duck with relatively small self-occlusion region The results are shown in Tables 1 2 and3 respectively Table 1 shows the process of dynamic self-occlusion avoidance in which the visual object Bunny doestranslation along [1 0 0]119879 with the speed of 3mms Table 2shows the process of dynamic self-occlusion avoidance inwhich the visual object Bunnydoes translation along [2 1 0]119879

with the speed ofradic5mms and rotation around [minus4 1 25]119879with the speed of 1∘s Table 3 shows the process of dynamicself-occlusion avoidance inwhich the visual objectDuck doestranslation along [2 1 0]

119879 with the speed of radic5mms androtation around [minus4 1 25]

119879 with the speed of 1∘s Becausethe camera needs multiple motions in the process of dynamicself-occlusion avoidance only partial observing results thematched results of Gaussian curvature feature matrices andrelated camera information are shown in Tables 1 2 and 3after the results of first three images are shown

The first row in Tables 1 2 and 3 shows the depth imagenumber corresponding to different views in the process ofself-occlusion avoidance At the end of the self-occlusionavoidance the image number in Tables 1 2 and 3 is 1716 and 13 respectively Thus we can see that the cameramovement times are different for different visual objects orthe same visual object with different motions in the processof self-occlusion avoidance The second row in Tables 1 2and 3 shows the acquired depth images of visual object indifferent camera views which are determined by proposedmethod From the images in this row we can see that theself-occlusion region is gradually observed more and morein the process of self-occlusion avoidance for example theself-occlusion region mainly caused by Bunnyrsquos ear in thesecond image in Tables 1 and 2 and the self-occlusion regionmainly caused by Duckrsquos mouth in the second image inTable 3 are better observed when the process of self-occlusionavoidance is accomplished The third row in Tables 1 2and 3 shows the matched results of the Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages with SIFT algorithm These matched results are thebasis of motion estimation and self-occlusion avoidance andgood matched results are helpful to obtain accurate motionestimation then the accurate camera motion trajectory can

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

2 Mathematical Problems in Engineering

avoidance besides calculating the next views the motionestimation about the moving visual object is necessaryThen the camera not only does synchronous motionwith the visual object according to the motion estima-tion result but also executes self-occlusion avoidance func-tion simultaneously In this way the camera moves stepby step until accomplishing the dynamic self-occlusionavoidance

Before self-occlusion avoidance problem emerging thesimilar problemcalled next best viewhas beenwidely studiedCurrently scholars have gained some achievements on thenext best view Connolly [1] as one of the earlier scholarsstudying the next best view used partial octree model todescribe visual object and made different marks to the nodesof different observation situations so as to determine thenext best view By discretizing a fixed surface Pito [2]determined the next best view from plenty of discretizationcandidate views Based on the depth data Whaite and Ferrie[3] constructed parameter similarity model of object andplanned the next best view of the camera according to thedifference between the depth data and the current fittedmodel Li and Liu [4] proposed a method which used B-spline to construct the model of object and determined thenext best view by calculating information gain Trummeret al [5] proposed a next best view method by combiningon-line theory to optimize the 3D reconstruction precisionof any object Unfortunately because of never consideringocclusion factor in thesemethods themore serious the occlu-sion is the more the accuracy of these methods would beaffected

Because ubiquitous occlusion would affect the result ofthe next best view scholars further proposed the next bestview methods taking occlusion into account Banta et al[6] proposed a combination method to determine the nextbest view under occlusion Based on the integration of activeand passive vision Fang and He [7] determined the nextbest view through the shape of the shadow region andthe concept of limit visible surface Combining layered raytracing and octree Vasquez-Gomez et al [8] constructedthe object model and generated candidate views based onsorting of the utility function to determine the next bestview Potthast and Sukhatme [9] determined the next bestview through the difference of information entropies underocclusion Wu et al [10] determined the next best view byusing the algorithm of layered contour fitting (LCF) basedon density Although these methods consider the factor ofocclusion (mutual occlusion or self-occlusion this paperpays attention to self-occlusion) there are limitations in thecamera position [6 7] specific equipment [8 9] a prioriknowledge [7 10] and so forth And besides the self-occlusion region is not modeled properly in these next bestview methods Moreover there exist some differences (suchas problem description and solving cue) between the nextbest view and self-occlusion avoidance so the abovemethodshave some reference significance but cannot be taken as thefinal solution to the problem of self-occlusion avoidanceMost importantly all of the above methods aim at the studyof static visual object and they are not suitable for the

moving visual object That is to say all of the above methodscannot be the solution to dynamic self-occlusion avoidanceHowever in many scientific research fields such as real-time 3D reconstruction object tracking autonomous navi-gation scene recognition of mobile robot robot autonomousoperation and dynamic scene rendering the self-occlusionof moving visual object is a universal phenomenon Mean-while the visual system would be invalid even wrongif it cannot effectively detect and avoid self-occlusion ofmoving visual object thus the visual system would loseits value which makes how to avoid the self-occlusionof moving visual object become an inevitable and urgentproblem

According to investigation result there is no relatedresearch on dynamic self-occlusion avoidance of movingvisual object that is to say the research on dynamic self-occlusion avoidance is still in initial stage at present Mean-while considering the objective fact that the 3D informationof a scene can be better obtained from the depth imagethan the intensity image an approach for avoiding the self-occlusion of the rigid motion object is proposed in this paperbased on the depth image In the process of designing themethod wemainly solve the following three problems firstlyhow to design the solution to the dynamic self-occlusionavoidance problem secondly how to estimate the motionof visual object based on depth image thirdly how to mea-sure the effect of dynamic self-occlusion avoidance methodAiming at the first problem the dynamic self-occlusionavoidance is posed as an optimization problem and motionestimation is merged into the optimization process of bestview model to solve the problem of dynamic self-occlusionavoidance Aiming at the second problem the two Gaussiancurvature feature matrices are matched by SIFT algorithmand the efficient singular value decomposition (SVD) is usedto estimate themotion of visual object For the third problemldquoeffective avoidance raterdquo is proposed to measure the per-formance of the dynamic self-occlusion avoidance methodThe rest of the paper is organized as follows Section 2 isthe method overview Section 3 describes the dynamic self-occlusion avoidancemethod based on depth image Section 4presents our experimental results and Section 5 concludes thepaper

2 Method Overview

21 The Analysis of Dynamic Self-Occlusion AvoidanceThe problem of dynamic self-occlusion avoidance can bedescribed on the premise that the self-occlusion region inan image of visual object image sequence is taken as theresearch object the reasonable next view sequence is plannedfor observing the self-occlusion region bymoving the camerato achieve the goal that the most information about the self-occlusion region can be obtained from the camera final viewBecause the visual object is moving in the process of self-occlusion avoidance the next view of the camera shouldbe adjusted dynamically to observe the vested self-occlusionregion

Mathematical Problems in Engineering 3

Ideal object modelCamera

(a) (b) (c)

Figure 1 The sketch map of camera observing ideal object model(a) Observing effect in initial view (b) Observing effect duringavoiding self-occlusion (c) Observing effect at the end of avoidingself-occlusion

Figure 1 shows the sketch map of camera observing idealobject model Figure 1(a) is the observing effect of camerain initial view and the shadow region is the self-occlusionregion to be avoided In the case of object moving if camerawants to obtain the information of the self-occlusion regionit must do the synchronousmotion with the visual object andmove to the best view for observing self-occlusion region atthe same time Figure 1(b) is the observing effect when thecamera is avoiding self-occlusion Figure 1(c) is the observingeffect when the camera arrives at the final view in whichthe camera can obtain the maximum information of self-occlusion region so the process of dynamic self-occlusionavoidance is accomplished when the camera arrives at thisview

22 The Overall Idea of Dynamic Self-Occlusion AvoidanceBased on the analysis of dynamic self-occlusion avoidanceabove an approach to dynamic self-occlusion avoidance isproposed The overall idea of the approach is as followsFirstly two adjacent depth images of the moving objectare acquired and each pixelrsquos 3D coordinates (all pixelsrsquo 3Dcoordinates are in the same world coordinate system) intwo depth images are calculated by utilizing antiprojectiontransformation and the self-occlusion cue in the seconddepth image is detected On this basis the best view modelis constructed according to the self-occlusion informationin the second depth image Secondly according to the 3Dcoordinates calculated above the Gaussian curvature featurematrices corresponding to the two adjacent depth imagesare calculated by using the pixelsrsquo 3D coordinates Then theGaussian curvature feature matrices corresponding to thetwo adjacent depth images are matched by SIFT algorithmand the motion equation is estimated by using the 3Dcoordinates of the matched points Finally combining thebest view model and the estimated motion equation ofthe visual object a series of next views of the camera areplanned to accomplish dynamic self-occlusion avoidanceprocess

3 The Approach to Dynamic Self-OcclusionAvoidance Based on Depth Image

31 Constructing the Self-Occlusion Region to BeAvoided and the Best View Model

311 Constructing the Self-Occlusion Region to Be AvoidedIn order to solve the problem of dynamic self-occlusionavoidance it is necessary to first construct the self-occlusionregion to be avoided As we know the information of self-occlusion region in current view is unknown that is to saythe geometry information of self-occlusion region could notbe obtained directly according to the depth image acquiredin current view so the specific modeling method should beadopted to describe the self-occlusion region approximatelyFigure 2 shows the local self-occlusion region of visualobject and its approximate description and in Figure 2(a)the red boundary is self-occlusion boundary and the blueboundary is the nether adjacent boundary correspondingto the red one The method in literature [11] is used todetect the self-occlusion boundary in depth image and all thepoints on self-occlusion boundary are organized to form self-occlusion boundary point set119874 As shown in Figure 2(b) theregion between self-occlusion boundary and nether adjacentboundary in 3D space is the unknown self-occlusion regionFigure 2(c) shows the construction of self-occlusion regionmodel by utilizing quadrilateral subdivision on unknownself-occlusion region

The concrete steps of constructing self-occlusion regionmodel are as follows Firstly take out the 119894th self-occlusionboundary point 119900

119894from self-occlusion boundary point set 119874

in turn Mark the unmarked nonocclusion boundary pointcorresponding to themaximum of depth differences betweenthe self-occlusion boundary point 119900

119894and its eight neighbors

as 1199001015840119894 and add 119900

1015840

119894into nether adjacent boundary point set

1198741015840 Secondly use the neighboring self-occlusion boundary

points 119900119895 119900119895+1

in 119874 and their corresponding nether adjacentpoints 1199001015840

119895 1199001015840119895+1

in 1198741015840 to form quadrilateral patch

119895 as shown

in Figure 2(c) 119895 is the integer from 1 to 119873 and the com-bination of all 119873 quadrilaterals is approximate descriptionof self-occlusion region Thirdly combining depth imageand camera parameters use antiprojection transformationto calculate each pixelrsquos 3D coordinates At last use the 3Dcoordinates of quadrilateral patch

119895rsquos four vertices to calculate

its normal and area thus the modeling process of self-occlusion region can be accomplished

On the premise of not considering the normal directionthe formula for calculating the normal k1015840patch119895 of quadrilateralpatch119895can be defined as

k1015840patch119895 = u119895times w119895

or k1015840patch119895 = w119895times u119895

(1)

where u119895is the vector from point 119900

119895to the midpoint of 1199001015840

119895

and 1199001015840

119895+1in 3D space and w

119895is the vector from point 119900

119895to

point 119900119895+1

in 3D space Supposing the initial camera view is

4 Mathematical Problems in Engineering

oj

oj+1

o998400

j

o998400

j+1

patchj

Self-occlusion regionmodelUnknown self-

occlusion regionSelf-occlusion boundary

Upper surface

Nether surface

(a) (b) (c)

Modelling

Figure 2 The local self-occlusion region and its approximate description for visual object (a) The depth image of visual object (b) Spatialstructure of local self-occlusion region (c) Approximate description of self-occlusion region

(xbegin kbegin) in order to ensure that the direction of calcu-lated normal points outside in the process of self-occlusionregion modeling the calculation formula of patch

119895rsquos normal

kpatch119895 is defined as

kpatch119895 = sign (minusk1015840patch119895 sdot kbegin) sdot k1015840

patch119895 (2)

where sign is the sign function it means the function valueis 1 when minusk1015840patch119895 sdot kbegin ge 0 and the function value is minus1when minusk1015840patch119895 sdot kbegin lt 0 The area 119878

119895of quadrilateral patch

119895

is defined as

119878119895=100381710038171003817100381710038171003817kpatch119895

1003817100381710038171003817100381710038172 (3)

In summary the set 119875 consisting of all quadrilateralspatch119895is the modeling result of self-occlusion region so the

self-occlusion information to be avoided is described as

119875 = (119900119895 119900119895+1

1199001015840

119895 1199001015840

119895+1 vpatch119895 119878119895) | 119900

119895 119900119895+1

isin 119874 1199001015840

119895 1199001015840

119895+1isin 1198741015840

(4)

312 Constructing the Best ViewModel The best view modelshould be constructed after obtaining the self-occlusioninformation to be avoided This paper uses the area ofvisible self-occlusion region on next view to construct theobjective function and meanwhile uses the size equality ofreal object corresponding to pixels in different depth imagesas constraint condition So the best view model is defined as

argmaxxBV

119873

sum

119895=1

119878119895cos (120579

119895(xBV))

st 120579119895(xBV) =

120579119895(xBV) 120579

119895(xBV) lt

120587

2

120587

2120579119895(xBV) ge

120587

2

1003817100381710038171003817xmid minus xBV10038171003817100381710038172 =

10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172

(5)

where xBV and xmid minus xBV are respectively the position anddirection of camera for observing the vested self-occlusionregion at maximum level119873 is the number of quadrilaterals120579119895(xBV) is the angle between the normal of 119895th quadrilateral

and the vector from the midpoint of 119895th quadrilateral tocamera xmid is the center of gravity of all visible points ininitial camera view (xbegin kbegin)

32 Estimating theMotion Equation of the Visual Object Basedon Two Adjacent Depth Images Different from the static self-occlusion avoidance the dynamic self-occlusion avoidance iscarried out on the premise that the visual object is movingwhichmakes themotion estimation of the visual object in 3Dspace become a necessary step in the process of dynamic self-occlusion avoidance Because the existing motion estimationmethods based on depth image all estimate the motion ofthe visual object in 2D images this paper designs a feasiblescheme based on the depth image to estimate the motion ofthe visual object in 3D space

321 Calculating the Gaussian Curvature Feature Matricesof the Two Adjacent Depth Images Considering that rigidbody has rotation and translation invariant characteristicsthe surface shape of visual object remains invariant inthe motion process Because the curvature feature reflectsbending degree of the object surface and Gaussian curvatureonly depends on Riemannian metric of the surface which isthe intrinsic invariant of the surface (ie Gaussian curvaturekeeps invariant in rotation and translation) the motion ofvisual object is estimated based onGaussian curvature featurematrices corresponding to the two adjacent depth images

In 3D Euclidean space 1198961and 119896

2are two principal

curvatures of a point on the differentiable surface then theGaussian curvature is defined as the product of two principalcurvatures namely 119896 = 119896

11198962 The method of calculating

the Gaussian curvature in literature [12] is implemented byoptimizing convolution calculation and it is high-efficientand suitable for different scales of curvature value so the

Mathematical Problems in Engineering 5

method in literature [12] is used for calculating the Gaussiancurvature feature matrix of depth image

322 Estimating the Motion Equation of Visual Object Basedon the Matched Feature Points Analysis shows that thematched points can provide reliable information for themotion estimation if the Gaussian curvature feature matricescorresponding to two adjacent depth images of the visualobject can be matched SIFT algorithm in literature [13]is more efficient and it is not only invariant to imagetranslation rotation and scaling but also robust to thechange of visual angle affine transformation and noise sothe SIFT algorithm is adopted in this paper for matching thefeature points

SIFT algorithm mainly consists of two steps the gener-ation of SIFT features and the matching of feature vectorsIn order to generate the SIFT features firstly the scale spaceshould be established Because the Difference of Gaussiansscale-space (DoG scale-space) has good stability in detectionof key points the DoG scale-space is used for detecting thekey points in this paper Secondly the key points which areinstable and sensitive to noise are filtered Then the directionof each feature point is calculated Finally the SIFT featurevector of each key point is generated In the process ofgenerating SIFT feature vector 16 seed points are selectedfor each key point and each seed point has 8 orientationsTherefore the 16 times 8 = 128 dimensions of the SIFT featurevector can be obtained for each key point After obtainingthe SIFT feature vectors of the Gaussian curvature featurematrices corresponding to the two adjacent depth images thetwoGaussian curvature featurematrices can bematched andthe similarity of the vectors ismeasured byEuclidean distancein the process of matching

After the key points of the Gaussian curvature featurematrices corresponding to the two adjacent depth images arematched the motion equation of visual object can be esti-mated by using the pixelsrsquo 3D coordinates which are obtainedby antiprojection transformation The motion equation isdefined as

x1015840119897= Rx119897+ T (6)

where x119897and x1015840119897are respectively the 3D coordinates of points

before and after visual objectmotionR is the unit orthogonalrotation matrix and T is the translation vector then theresult of motion estimation is the solution of minimizing theobjective function 119891(RT) The objective function 119891(RT) isdefined as

119891 (RT) =119872

sum

119897=1

10038171003817100381710038171003817x1015840119897minus (Rx

119897+ T)10038171003817100381710038171003817

2

(7)

where119872 is the number of key points which are matched Inorder to solve the R and T in formula (7) the matrix H isdefined as

H =

119872

sum

119897=1

x1015840clx119879

cl (8)

where x1015840cl = x1015840119897minus (1119872)sum

119872

119897=1x1015840119897 xcl = x

119897minus (1119872)sum

119872

119897=1x119897

By carrying out singular value decomposition on H H canbe expressed as H=UΛV119879 then according to U and V R informula (7) can be deduced as

R = VU119879 (9)

After obtaining R substitute R into formula (10) tocalculate T namely

T = x1015840119897minus Rx119897 (10)

After obtaining R and T the motion equation shown informula (6) can be obtained and themotion estimation of thevisual object is accomplished

33 Planning the Next View Based on Motion Estimationand Best View Model After the motion equation of visualobject is estimated the next view of the camera can beplanned Analysis can be known if substituting the cameraobservation position into the motion equation of visualobject the camera can do the synchronous motion with thevisual object In addition on the basis of the synchronousmotion with the visual object if the camera motion offsetfor avoiding self-occlusion is introduced the whole processof dynamic self-occlusion avoidance can be achieved Byanalyzing the best view model in formula (5) we can knowit is the camera best view that can make formula (11) achievemaximum value Formula (11) is defined as

119891 (x) =119873

sum

119895=1

119878119895cos (120579

119895 (x)) (11)

where x is the camera observation position Since formula (11)is a nonconvex model it is difficult to directly calculate itsglobal optimal solution Meanwhile because the problem ofdynamic self-occlusion avoidance needs to calculate a seriesof next views and the gradient information in formula (11) canprovide basis for determining the cameramotion offset of theself-occlusion avoidance process in the process of dynamicself-occlusion avoidance the gradient descent idea is usedin this paper for defining the formula of camera observationposition x1015840

119873119881as

x1015840119873119881

= Rx119881+ T + 120575nabla119891 (x

119881) (12)

where x119881is the current observation position of the camera

and 120575 is the offset coefficient Based on the solution to formula(12) the formula of next observation direction k

119873119881is defined

as

k119873119881

= xmid minus x1015840119873119881

(13)

According to the constraint condition in formula (5) wecan search one point x

119873119881along the opposite direction of k

119873119881

and make x119873119881

meet1003817100381710038171003817xmid minus x

119873119881

10038171003817100381710038172 =10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172 (14)

By now the obtained view (x119873119881

v119873119881

) can be regardedas the new current view (x

119881 v119881) After acquiring the depth

6 Mathematical Problems in Engineering

image of visual object in the new view (x119881 v119881) the process

from formula (6) to formula (14) is repeated until thedifference between two 119891(x) which are calculated in twoadjacent views according to formula (11) is less than a giventhreshold It means that a series of views (x

119873119881 v119873119881

) can becalculated during the visual object motion so the process ofdynamic self-occlusion avoidance can be accomplished

34 The Algorithm of Dynamic Self-Occlusion Avoidance

Algorithm 1 (dynamic self-occlusion avoidance)

Input The camera internal and external parameters and twoadjacent depth images acquired in initial camera view

Output The set of a series of camera views corresponding tothe dynamic self-occlusion avoidance

Step 1 Calculate the pixelsrsquo 3D coordinates and the Gaussiancurvature feature matrices corresponding to the two adjacentdepth images

Step 2 Detect the self-occlusion boundary in the seconddepth image and establish the self-occlusion region in thesecond depth image according to formula (1) to formula (4)

Step 3 Construct the best view model according to self-occlusion information (formula (5))

Step 4 Match the key points based on Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages by using SIFT algorithm

Step 5 Based on the 3D coordinates of matched key pointssolve the objective function in formula (7) according toformula (8) to formula (10) then the visual objectrsquos motionequation in formula (6) can be obtained

Step 6 Plan the next view of camera according to formula (11)to formula (14)

Step 7 In the obtained next view acquire a depth image andcalculate 119891(x) in formula (11)

Step 8 If the difference between two adjacent 119891(x) is lessthan a given threshold the process of dynamic self-occlusionavoidance can be terminated and otherwise jump to Step 4

to continue the process of dynamic self-occlusion avoidance

4 Experiments and Analysis

41 Experimental Environment In order to validate the effectof the method in this paper the OpenGL is adopted toimplement the process of dynamic self-occlusion avoidanceduring camera observing the moving object based on 3Dobject model in the Stuttgart Range Image Database Theexperimental hardware environment is Intel CPU (R) Core(TM) i7-3770 340GHz and the memory is 800GB Thedynamic self-occlusion avoidance program is implemented

with C++ language In the process of self-occlusion avoid-ance the parameter of the projection matrix in OpenGL is(60 1 200 600) and the window size is 400 times 400

42 The Experimental Results and Analysis To verify thefeasibility of the proposed method we do experiments withdifferent visual objects in different motion modes Duringthe experiments the initial camera observation position isxbegin = [000 minus100 30000]

119879 and the initial observationdirection is kbegin = [000 100 minus30000]

119879 The coordinateunit is millimeter (mm) and the offset coefficient is 120575 = 4Theprocess of dynamic self-occlusion avoidance will terminate ifthe calculated difference of 119891(x) between two adjacent viewsis less than the given threshold 30mm2 The offset coefficientand threshold are chosen by empirical value Due to thelimited space here we show 3 groups of experimental resultscorresponding to the visual object Bunny with relatively largeself-occlusion region and Duck with relatively small self-occlusion region The results are shown in Tables 1 2 and3 respectively Table 1 shows the process of dynamic self-occlusion avoidance in which the visual object Bunny doestranslation along [1 0 0]119879 with the speed of 3mms Table 2shows the process of dynamic self-occlusion avoidance inwhich the visual object Bunnydoes translation along [2 1 0]119879

with the speed ofradic5mms and rotation around [minus4 1 25]119879with the speed of 1∘s Table 3 shows the process of dynamicself-occlusion avoidance inwhich the visual objectDuck doestranslation along [2 1 0]

119879 with the speed of radic5mms androtation around [minus4 1 25]

119879 with the speed of 1∘s Becausethe camera needs multiple motions in the process of dynamicself-occlusion avoidance only partial observing results thematched results of Gaussian curvature feature matrices andrelated camera information are shown in Tables 1 2 and 3after the results of first three images are shown

The first row in Tables 1 2 and 3 shows the depth imagenumber corresponding to different views in the process ofself-occlusion avoidance At the end of the self-occlusionavoidance the image number in Tables 1 2 and 3 is 1716 and 13 respectively Thus we can see that the cameramovement times are different for different visual objects orthe same visual object with different motions in the processof self-occlusion avoidance The second row in Tables 1 2and 3 shows the acquired depth images of visual object indifferent camera views which are determined by proposedmethod From the images in this row we can see that theself-occlusion region is gradually observed more and morein the process of self-occlusion avoidance for example theself-occlusion region mainly caused by Bunnyrsquos ear in thesecond image in Tables 1 and 2 and the self-occlusion regionmainly caused by Duckrsquos mouth in the second image inTable 3 are better observed when the process of self-occlusionavoidance is accomplished The third row in Tables 1 2and 3 shows the matched results of the Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages with SIFT algorithm These matched results are thebasis of motion estimation and self-occlusion avoidance andgood matched results are helpful to obtain accurate motionestimation then the accurate camera motion trajectory can

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 3

Ideal object modelCamera

(a) (b) (c)

Figure 1 The sketch map of camera observing ideal object model(a) Observing effect in initial view (b) Observing effect duringavoiding self-occlusion (c) Observing effect at the end of avoidingself-occlusion

Figure 1 shows the sketch map of camera observing idealobject model Figure 1(a) is the observing effect of camerain initial view and the shadow region is the self-occlusionregion to be avoided In the case of object moving if camerawants to obtain the information of the self-occlusion regionit must do the synchronousmotion with the visual object andmove to the best view for observing self-occlusion region atthe same time Figure 1(b) is the observing effect when thecamera is avoiding self-occlusion Figure 1(c) is the observingeffect when the camera arrives at the final view in whichthe camera can obtain the maximum information of self-occlusion region so the process of dynamic self-occlusionavoidance is accomplished when the camera arrives at thisview

22 The Overall Idea of Dynamic Self-Occlusion AvoidanceBased on the analysis of dynamic self-occlusion avoidanceabove an approach to dynamic self-occlusion avoidance isproposed The overall idea of the approach is as followsFirstly two adjacent depth images of the moving objectare acquired and each pixelrsquos 3D coordinates (all pixelsrsquo 3Dcoordinates are in the same world coordinate system) intwo depth images are calculated by utilizing antiprojectiontransformation and the self-occlusion cue in the seconddepth image is detected On this basis the best view modelis constructed according to the self-occlusion informationin the second depth image Secondly according to the 3Dcoordinates calculated above the Gaussian curvature featurematrices corresponding to the two adjacent depth imagesare calculated by using the pixelsrsquo 3D coordinates Then theGaussian curvature feature matrices corresponding to thetwo adjacent depth images are matched by SIFT algorithmand the motion equation is estimated by using the 3Dcoordinates of the matched points Finally combining thebest view model and the estimated motion equation ofthe visual object a series of next views of the camera areplanned to accomplish dynamic self-occlusion avoidanceprocess

3 The Approach to Dynamic Self-OcclusionAvoidance Based on Depth Image

31 Constructing the Self-Occlusion Region to BeAvoided and the Best View Model

311 Constructing the Self-Occlusion Region to Be AvoidedIn order to solve the problem of dynamic self-occlusionavoidance it is necessary to first construct the self-occlusionregion to be avoided As we know the information of self-occlusion region in current view is unknown that is to saythe geometry information of self-occlusion region could notbe obtained directly according to the depth image acquiredin current view so the specific modeling method should beadopted to describe the self-occlusion region approximatelyFigure 2 shows the local self-occlusion region of visualobject and its approximate description and in Figure 2(a)the red boundary is self-occlusion boundary and the blueboundary is the nether adjacent boundary correspondingto the red one The method in literature [11] is used todetect the self-occlusion boundary in depth image and all thepoints on self-occlusion boundary are organized to form self-occlusion boundary point set119874 As shown in Figure 2(b) theregion between self-occlusion boundary and nether adjacentboundary in 3D space is the unknown self-occlusion regionFigure 2(c) shows the construction of self-occlusion regionmodel by utilizing quadrilateral subdivision on unknownself-occlusion region

The concrete steps of constructing self-occlusion regionmodel are as follows Firstly take out the 119894th self-occlusionboundary point 119900

119894from self-occlusion boundary point set 119874

in turn Mark the unmarked nonocclusion boundary pointcorresponding to themaximum of depth differences betweenthe self-occlusion boundary point 119900

119894and its eight neighbors

as 1199001015840119894 and add 119900

1015840

119894into nether adjacent boundary point set

1198741015840 Secondly use the neighboring self-occlusion boundary

points 119900119895 119900119895+1

in 119874 and their corresponding nether adjacentpoints 1199001015840

119895 1199001015840119895+1

in 1198741015840 to form quadrilateral patch

119895 as shown

in Figure 2(c) 119895 is the integer from 1 to 119873 and the com-bination of all 119873 quadrilaterals is approximate descriptionof self-occlusion region Thirdly combining depth imageand camera parameters use antiprojection transformationto calculate each pixelrsquos 3D coordinates At last use the 3Dcoordinates of quadrilateral patch

119895rsquos four vertices to calculate

its normal and area thus the modeling process of self-occlusion region can be accomplished

On the premise of not considering the normal directionthe formula for calculating the normal k1015840patch119895 of quadrilateralpatch119895can be defined as

k1015840patch119895 = u119895times w119895

or k1015840patch119895 = w119895times u119895

(1)

where u119895is the vector from point 119900

119895to the midpoint of 1199001015840

119895

and 1199001015840

119895+1in 3D space and w

119895is the vector from point 119900

119895to

point 119900119895+1

in 3D space Supposing the initial camera view is

4 Mathematical Problems in Engineering

oj

oj+1

o998400

j

o998400

j+1

patchj

Self-occlusion regionmodelUnknown self-

occlusion regionSelf-occlusion boundary

Upper surface

Nether surface

(a) (b) (c)

Modelling

Figure 2 The local self-occlusion region and its approximate description for visual object (a) The depth image of visual object (b) Spatialstructure of local self-occlusion region (c) Approximate description of self-occlusion region

(xbegin kbegin) in order to ensure that the direction of calcu-lated normal points outside in the process of self-occlusionregion modeling the calculation formula of patch

119895rsquos normal

kpatch119895 is defined as

kpatch119895 = sign (minusk1015840patch119895 sdot kbegin) sdot k1015840

patch119895 (2)

where sign is the sign function it means the function valueis 1 when minusk1015840patch119895 sdot kbegin ge 0 and the function value is minus1when minusk1015840patch119895 sdot kbegin lt 0 The area 119878

119895of quadrilateral patch

119895

is defined as

119878119895=100381710038171003817100381710038171003817kpatch119895

1003817100381710038171003817100381710038172 (3)

In summary the set 119875 consisting of all quadrilateralspatch119895is the modeling result of self-occlusion region so the

self-occlusion information to be avoided is described as

119875 = (119900119895 119900119895+1

1199001015840

119895 1199001015840

119895+1 vpatch119895 119878119895) | 119900

119895 119900119895+1

isin 119874 1199001015840

119895 1199001015840

119895+1isin 1198741015840

(4)

312 Constructing the Best ViewModel The best view modelshould be constructed after obtaining the self-occlusioninformation to be avoided This paper uses the area ofvisible self-occlusion region on next view to construct theobjective function and meanwhile uses the size equality ofreal object corresponding to pixels in different depth imagesas constraint condition So the best view model is defined as

argmaxxBV

119873

sum

119895=1

119878119895cos (120579

119895(xBV))

st 120579119895(xBV) =

120579119895(xBV) 120579

119895(xBV) lt

120587

2

120587

2120579119895(xBV) ge

120587

2

1003817100381710038171003817xmid minus xBV10038171003817100381710038172 =

10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172

(5)

where xBV and xmid minus xBV are respectively the position anddirection of camera for observing the vested self-occlusionregion at maximum level119873 is the number of quadrilaterals120579119895(xBV) is the angle between the normal of 119895th quadrilateral

and the vector from the midpoint of 119895th quadrilateral tocamera xmid is the center of gravity of all visible points ininitial camera view (xbegin kbegin)

32 Estimating theMotion Equation of the Visual Object Basedon Two Adjacent Depth Images Different from the static self-occlusion avoidance the dynamic self-occlusion avoidance iscarried out on the premise that the visual object is movingwhichmakes themotion estimation of the visual object in 3Dspace become a necessary step in the process of dynamic self-occlusion avoidance Because the existing motion estimationmethods based on depth image all estimate the motion ofthe visual object in 2D images this paper designs a feasiblescheme based on the depth image to estimate the motion ofthe visual object in 3D space

321 Calculating the Gaussian Curvature Feature Matricesof the Two Adjacent Depth Images Considering that rigidbody has rotation and translation invariant characteristicsthe surface shape of visual object remains invariant inthe motion process Because the curvature feature reflectsbending degree of the object surface and Gaussian curvatureonly depends on Riemannian metric of the surface which isthe intrinsic invariant of the surface (ie Gaussian curvaturekeeps invariant in rotation and translation) the motion ofvisual object is estimated based onGaussian curvature featurematrices corresponding to the two adjacent depth images

In 3D Euclidean space 1198961and 119896

2are two principal

curvatures of a point on the differentiable surface then theGaussian curvature is defined as the product of two principalcurvatures namely 119896 = 119896

11198962 The method of calculating

the Gaussian curvature in literature [12] is implemented byoptimizing convolution calculation and it is high-efficientand suitable for different scales of curvature value so the

Mathematical Problems in Engineering 5

method in literature [12] is used for calculating the Gaussiancurvature feature matrix of depth image

322 Estimating the Motion Equation of Visual Object Basedon the Matched Feature Points Analysis shows that thematched points can provide reliable information for themotion estimation if the Gaussian curvature feature matricescorresponding to two adjacent depth images of the visualobject can be matched SIFT algorithm in literature [13]is more efficient and it is not only invariant to imagetranslation rotation and scaling but also robust to thechange of visual angle affine transformation and noise sothe SIFT algorithm is adopted in this paper for matching thefeature points

SIFT algorithm mainly consists of two steps the gener-ation of SIFT features and the matching of feature vectorsIn order to generate the SIFT features firstly the scale spaceshould be established Because the Difference of Gaussiansscale-space (DoG scale-space) has good stability in detectionof key points the DoG scale-space is used for detecting thekey points in this paper Secondly the key points which areinstable and sensitive to noise are filtered Then the directionof each feature point is calculated Finally the SIFT featurevector of each key point is generated In the process ofgenerating SIFT feature vector 16 seed points are selectedfor each key point and each seed point has 8 orientationsTherefore the 16 times 8 = 128 dimensions of the SIFT featurevector can be obtained for each key point After obtainingthe SIFT feature vectors of the Gaussian curvature featurematrices corresponding to the two adjacent depth images thetwoGaussian curvature featurematrices can bematched andthe similarity of the vectors ismeasured byEuclidean distancein the process of matching

After the key points of the Gaussian curvature featurematrices corresponding to the two adjacent depth images arematched the motion equation of visual object can be esti-mated by using the pixelsrsquo 3D coordinates which are obtainedby antiprojection transformation The motion equation isdefined as

x1015840119897= Rx119897+ T (6)

where x119897and x1015840119897are respectively the 3D coordinates of points

before and after visual objectmotionR is the unit orthogonalrotation matrix and T is the translation vector then theresult of motion estimation is the solution of minimizing theobjective function 119891(RT) The objective function 119891(RT) isdefined as

119891 (RT) =119872

sum

119897=1

10038171003817100381710038171003817x1015840119897minus (Rx

119897+ T)10038171003817100381710038171003817

2

(7)

where119872 is the number of key points which are matched Inorder to solve the R and T in formula (7) the matrix H isdefined as

H =

119872

sum

119897=1

x1015840clx119879

cl (8)

where x1015840cl = x1015840119897minus (1119872)sum

119872

119897=1x1015840119897 xcl = x

119897minus (1119872)sum

119872

119897=1x119897

By carrying out singular value decomposition on H H canbe expressed as H=UΛV119879 then according to U and V R informula (7) can be deduced as

R = VU119879 (9)

After obtaining R substitute R into formula (10) tocalculate T namely

T = x1015840119897minus Rx119897 (10)

After obtaining R and T the motion equation shown informula (6) can be obtained and themotion estimation of thevisual object is accomplished

33 Planning the Next View Based on Motion Estimationand Best View Model After the motion equation of visualobject is estimated the next view of the camera can beplanned Analysis can be known if substituting the cameraobservation position into the motion equation of visualobject the camera can do the synchronous motion with thevisual object In addition on the basis of the synchronousmotion with the visual object if the camera motion offsetfor avoiding self-occlusion is introduced the whole processof dynamic self-occlusion avoidance can be achieved Byanalyzing the best view model in formula (5) we can knowit is the camera best view that can make formula (11) achievemaximum value Formula (11) is defined as

119891 (x) =119873

sum

119895=1

119878119895cos (120579

119895 (x)) (11)

where x is the camera observation position Since formula (11)is a nonconvex model it is difficult to directly calculate itsglobal optimal solution Meanwhile because the problem ofdynamic self-occlusion avoidance needs to calculate a seriesof next views and the gradient information in formula (11) canprovide basis for determining the cameramotion offset of theself-occlusion avoidance process in the process of dynamicself-occlusion avoidance the gradient descent idea is usedin this paper for defining the formula of camera observationposition x1015840

119873119881as

x1015840119873119881

= Rx119881+ T + 120575nabla119891 (x

119881) (12)

where x119881is the current observation position of the camera

and 120575 is the offset coefficient Based on the solution to formula(12) the formula of next observation direction k

119873119881is defined

as

k119873119881

= xmid minus x1015840119873119881

(13)

According to the constraint condition in formula (5) wecan search one point x

119873119881along the opposite direction of k

119873119881

and make x119873119881

meet1003817100381710038171003817xmid minus x

119873119881

10038171003817100381710038172 =10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172 (14)

By now the obtained view (x119873119881

v119873119881

) can be regardedas the new current view (x

119881 v119881) After acquiring the depth

6 Mathematical Problems in Engineering

image of visual object in the new view (x119881 v119881) the process

from formula (6) to formula (14) is repeated until thedifference between two 119891(x) which are calculated in twoadjacent views according to formula (11) is less than a giventhreshold It means that a series of views (x

119873119881 v119873119881

) can becalculated during the visual object motion so the process ofdynamic self-occlusion avoidance can be accomplished

34 The Algorithm of Dynamic Self-Occlusion Avoidance

Algorithm 1 (dynamic self-occlusion avoidance)

Input The camera internal and external parameters and twoadjacent depth images acquired in initial camera view

Output The set of a series of camera views corresponding tothe dynamic self-occlusion avoidance

Step 1 Calculate the pixelsrsquo 3D coordinates and the Gaussiancurvature feature matrices corresponding to the two adjacentdepth images

Step 2 Detect the self-occlusion boundary in the seconddepth image and establish the self-occlusion region in thesecond depth image according to formula (1) to formula (4)

Step 3 Construct the best view model according to self-occlusion information (formula (5))

Step 4 Match the key points based on Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages by using SIFT algorithm

Step 5 Based on the 3D coordinates of matched key pointssolve the objective function in formula (7) according toformula (8) to formula (10) then the visual objectrsquos motionequation in formula (6) can be obtained

Step 6 Plan the next view of camera according to formula (11)to formula (14)

Step 7 In the obtained next view acquire a depth image andcalculate 119891(x) in formula (11)

Step 8 If the difference between two adjacent 119891(x) is lessthan a given threshold the process of dynamic self-occlusionavoidance can be terminated and otherwise jump to Step 4

to continue the process of dynamic self-occlusion avoidance

4 Experiments and Analysis

41 Experimental Environment In order to validate the effectof the method in this paper the OpenGL is adopted toimplement the process of dynamic self-occlusion avoidanceduring camera observing the moving object based on 3Dobject model in the Stuttgart Range Image Database Theexperimental hardware environment is Intel CPU (R) Core(TM) i7-3770 340GHz and the memory is 800GB Thedynamic self-occlusion avoidance program is implemented

with C++ language In the process of self-occlusion avoid-ance the parameter of the projection matrix in OpenGL is(60 1 200 600) and the window size is 400 times 400

42 The Experimental Results and Analysis To verify thefeasibility of the proposed method we do experiments withdifferent visual objects in different motion modes Duringthe experiments the initial camera observation position isxbegin = [000 minus100 30000]

119879 and the initial observationdirection is kbegin = [000 100 minus30000]

119879 The coordinateunit is millimeter (mm) and the offset coefficient is 120575 = 4Theprocess of dynamic self-occlusion avoidance will terminate ifthe calculated difference of 119891(x) between two adjacent viewsis less than the given threshold 30mm2 The offset coefficientand threshold are chosen by empirical value Due to thelimited space here we show 3 groups of experimental resultscorresponding to the visual object Bunny with relatively largeself-occlusion region and Duck with relatively small self-occlusion region The results are shown in Tables 1 2 and3 respectively Table 1 shows the process of dynamic self-occlusion avoidance in which the visual object Bunny doestranslation along [1 0 0]119879 with the speed of 3mms Table 2shows the process of dynamic self-occlusion avoidance inwhich the visual object Bunnydoes translation along [2 1 0]119879

with the speed ofradic5mms and rotation around [minus4 1 25]119879with the speed of 1∘s Table 3 shows the process of dynamicself-occlusion avoidance inwhich the visual objectDuck doestranslation along [2 1 0]

119879 with the speed of radic5mms androtation around [minus4 1 25]

119879 with the speed of 1∘s Becausethe camera needs multiple motions in the process of dynamicself-occlusion avoidance only partial observing results thematched results of Gaussian curvature feature matrices andrelated camera information are shown in Tables 1 2 and 3after the results of first three images are shown

The first row in Tables 1 2 and 3 shows the depth imagenumber corresponding to different views in the process ofself-occlusion avoidance At the end of the self-occlusionavoidance the image number in Tables 1 2 and 3 is 1716 and 13 respectively Thus we can see that the cameramovement times are different for different visual objects orthe same visual object with different motions in the processof self-occlusion avoidance The second row in Tables 1 2and 3 shows the acquired depth images of visual object indifferent camera views which are determined by proposedmethod From the images in this row we can see that theself-occlusion region is gradually observed more and morein the process of self-occlusion avoidance for example theself-occlusion region mainly caused by Bunnyrsquos ear in thesecond image in Tables 1 and 2 and the self-occlusion regionmainly caused by Duckrsquos mouth in the second image inTable 3 are better observed when the process of self-occlusionavoidance is accomplished The third row in Tables 1 2and 3 shows the matched results of the Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages with SIFT algorithm These matched results are thebasis of motion estimation and self-occlusion avoidance andgood matched results are helpful to obtain accurate motionestimation then the accurate camera motion trajectory can

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

4 Mathematical Problems in Engineering

oj

oj+1

o998400

j

o998400

j+1

patchj

Self-occlusion regionmodelUnknown self-

occlusion regionSelf-occlusion boundary

Upper surface

Nether surface

(a) (b) (c)

Modelling

Figure 2 The local self-occlusion region and its approximate description for visual object (a) The depth image of visual object (b) Spatialstructure of local self-occlusion region (c) Approximate description of self-occlusion region

(xbegin kbegin) in order to ensure that the direction of calcu-lated normal points outside in the process of self-occlusionregion modeling the calculation formula of patch

119895rsquos normal

kpatch119895 is defined as

kpatch119895 = sign (minusk1015840patch119895 sdot kbegin) sdot k1015840

patch119895 (2)

where sign is the sign function it means the function valueis 1 when minusk1015840patch119895 sdot kbegin ge 0 and the function value is minus1when minusk1015840patch119895 sdot kbegin lt 0 The area 119878

119895of quadrilateral patch

119895

is defined as

119878119895=100381710038171003817100381710038171003817kpatch119895

1003817100381710038171003817100381710038172 (3)

In summary the set 119875 consisting of all quadrilateralspatch119895is the modeling result of self-occlusion region so the

self-occlusion information to be avoided is described as

119875 = (119900119895 119900119895+1

1199001015840

119895 1199001015840

119895+1 vpatch119895 119878119895) | 119900

119895 119900119895+1

isin 119874 1199001015840

119895 1199001015840

119895+1isin 1198741015840

(4)

312 Constructing the Best ViewModel The best view modelshould be constructed after obtaining the self-occlusioninformation to be avoided This paper uses the area ofvisible self-occlusion region on next view to construct theobjective function and meanwhile uses the size equality ofreal object corresponding to pixels in different depth imagesas constraint condition So the best view model is defined as

argmaxxBV

119873

sum

119895=1

119878119895cos (120579

119895(xBV))

st 120579119895(xBV) =

120579119895(xBV) 120579

119895(xBV) lt

120587

2

120587

2120579119895(xBV) ge

120587

2

1003817100381710038171003817xmid minus xBV10038171003817100381710038172 =

10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172

(5)

where xBV and xmid minus xBV are respectively the position anddirection of camera for observing the vested self-occlusionregion at maximum level119873 is the number of quadrilaterals120579119895(xBV) is the angle between the normal of 119895th quadrilateral

and the vector from the midpoint of 119895th quadrilateral tocamera xmid is the center of gravity of all visible points ininitial camera view (xbegin kbegin)

32 Estimating theMotion Equation of the Visual Object Basedon Two Adjacent Depth Images Different from the static self-occlusion avoidance the dynamic self-occlusion avoidance iscarried out on the premise that the visual object is movingwhichmakes themotion estimation of the visual object in 3Dspace become a necessary step in the process of dynamic self-occlusion avoidance Because the existing motion estimationmethods based on depth image all estimate the motion ofthe visual object in 2D images this paper designs a feasiblescheme based on the depth image to estimate the motion ofthe visual object in 3D space

321 Calculating the Gaussian Curvature Feature Matricesof the Two Adjacent Depth Images Considering that rigidbody has rotation and translation invariant characteristicsthe surface shape of visual object remains invariant inthe motion process Because the curvature feature reflectsbending degree of the object surface and Gaussian curvatureonly depends on Riemannian metric of the surface which isthe intrinsic invariant of the surface (ie Gaussian curvaturekeeps invariant in rotation and translation) the motion ofvisual object is estimated based onGaussian curvature featurematrices corresponding to the two adjacent depth images

In 3D Euclidean space 1198961and 119896

2are two principal

curvatures of a point on the differentiable surface then theGaussian curvature is defined as the product of two principalcurvatures namely 119896 = 119896

11198962 The method of calculating

the Gaussian curvature in literature [12] is implemented byoptimizing convolution calculation and it is high-efficientand suitable for different scales of curvature value so the

Mathematical Problems in Engineering 5

method in literature [12] is used for calculating the Gaussiancurvature feature matrix of depth image

322 Estimating the Motion Equation of Visual Object Basedon the Matched Feature Points Analysis shows that thematched points can provide reliable information for themotion estimation if the Gaussian curvature feature matricescorresponding to two adjacent depth images of the visualobject can be matched SIFT algorithm in literature [13]is more efficient and it is not only invariant to imagetranslation rotation and scaling but also robust to thechange of visual angle affine transformation and noise sothe SIFT algorithm is adopted in this paper for matching thefeature points

SIFT algorithm mainly consists of two steps the gener-ation of SIFT features and the matching of feature vectorsIn order to generate the SIFT features firstly the scale spaceshould be established Because the Difference of Gaussiansscale-space (DoG scale-space) has good stability in detectionof key points the DoG scale-space is used for detecting thekey points in this paper Secondly the key points which areinstable and sensitive to noise are filtered Then the directionof each feature point is calculated Finally the SIFT featurevector of each key point is generated In the process ofgenerating SIFT feature vector 16 seed points are selectedfor each key point and each seed point has 8 orientationsTherefore the 16 times 8 = 128 dimensions of the SIFT featurevector can be obtained for each key point After obtainingthe SIFT feature vectors of the Gaussian curvature featurematrices corresponding to the two adjacent depth images thetwoGaussian curvature featurematrices can bematched andthe similarity of the vectors ismeasured byEuclidean distancein the process of matching

After the key points of the Gaussian curvature featurematrices corresponding to the two adjacent depth images arematched the motion equation of visual object can be esti-mated by using the pixelsrsquo 3D coordinates which are obtainedby antiprojection transformation The motion equation isdefined as

x1015840119897= Rx119897+ T (6)

where x119897and x1015840119897are respectively the 3D coordinates of points

before and after visual objectmotionR is the unit orthogonalrotation matrix and T is the translation vector then theresult of motion estimation is the solution of minimizing theobjective function 119891(RT) The objective function 119891(RT) isdefined as

119891 (RT) =119872

sum

119897=1

10038171003817100381710038171003817x1015840119897minus (Rx

119897+ T)10038171003817100381710038171003817

2

(7)

where119872 is the number of key points which are matched Inorder to solve the R and T in formula (7) the matrix H isdefined as

H =

119872

sum

119897=1

x1015840clx119879

cl (8)

where x1015840cl = x1015840119897minus (1119872)sum

119872

119897=1x1015840119897 xcl = x

119897minus (1119872)sum

119872

119897=1x119897

By carrying out singular value decomposition on H H canbe expressed as H=UΛV119879 then according to U and V R informula (7) can be deduced as

R = VU119879 (9)

After obtaining R substitute R into formula (10) tocalculate T namely

T = x1015840119897minus Rx119897 (10)

After obtaining R and T the motion equation shown informula (6) can be obtained and themotion estimation of thevisual object is accomplished

33 Planning the Next View Based on Motion Estimationand Best View Model After the motion equation of visualobject is estimated the next view of the camera can beplanned Analysis can be known if substituting the cameraobservation position into the motion equation of visualobject the camera can do the synchronous motion with thevisual object In addition on the basis of the synchronousmotion with the visual object if the camera motion offsetfor avoiding self-occlusion is introduced the whole processof dynamic self-occlusion avoidance can be achieved Byanalyzing the best view model in formula (5) we can knowit is the camera best view that can make formula (11) achievemaximum value Formula (11) is defined as

119891 (x) =119873

sum

119895=1

119878119895cos (120579

119895 (x)) (11)

where x is the camera observation position Since formula (11)is a nonconvex model it is difficult to directly calculate itsglobal optimal solution Meanwhile because the problem ofdynamic self-occlusion avoidance needs to calculate a seriesof next views and the gradient information in formula (11) canprovide basis for determining the cameramotion offset of theself-occlusion avoidance process in the process of dynamicself-occlusion avoidance the gradient descent idea is usedin this paper for defining the formula of camera observationposition x1015840

119873119881as

x1015840119873119881

= Rx119881+ T + 120575nabla119891 (x

119881) (12)

where x119881is the current observation position of the camera

and 120575 is the offset coefficient Based on the solution to formula(12) the formula of next observation direction k

119873119881is defined

as

k119873119881

= xmid minus x1015840119873119881

(13)

According to the constraint condition in formula (5) wecan search one point x

119873119881along the opposite direction of k

119873119881

and make x119873119881

meet1003817100381710038171003817xmid minus x

119873119881

10038171003817100381710038172 =10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172 (14)

By now the obtained view (x119873119881

v119873119881

) can be regardedas the new current view (x

119881 v119881) After acquiring the depth

6 Mathematical Problems in Engineering

image of visual object in the new view (x119881 v119881) the process

from formula (6) to formula (14) is repeated until thedifference between two 119891(x) which are calculated in twoadjacent views according to formula (11) is less than a giventhreshold It means that a series of views (x

119873119881 v119873119881

) can becalculated during the visual object motion so the process ofdynamic self-occlusion avoidance can be accomplished

34 The Algorithm of Dynamic Self-Occlusion Avoidance

Algorithm 1 (dynamic self-occlusion avoidance)

Input The camera internal and external parameters and twoadjacent depth images acquired in initial camera view

Output The set of a series of camera views corresponding tothe dynamic self-occlusion avoidance

Step 1 Calculate the pixelsrsquo 3D coordinates and the Gaussiancurvature feature matrices corresponding to the two adjacentdepth images

Step 2 Detect the self-occlusion boundary in the seconddepth image and establish the self-occlusion region in thesecond depth image according to formula (1) to formula (4)

Step 3 Construct the best view model according to self-occlusion information (formula (5))

Step 4 Match the key points based on Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages by using SIFT algorithm

Step 5 Based on the 3D coordinates of matched key pointssolve the objective function in formula (7) according toformula (8) to formula (10) then the visual objectrsquos motionequation in formula (6) can be obtained

Step 6 Plan the next view of camera according to formula (11)to formula (14)

Step 7 In the obtained next view acquire a depth image andcalculate 119891(x) in formula (11)

Step 8 If the difference between two adjacent 119891(x) is lessthan a given threshold the process of dynamic self-occlusionavoidance can be terminated and otherwise jump to Step 4

to continue the process of dynamic self-occlusion avoidance

4 Experiments and Analysis

41 Experimental Environment In order to validate the effectof the method in this paper the OpenGL is adopted toimplement the process of dynamic self-occlusion avoidanceduring camera observing the moving object based on 3Dobject model in the Stuttgart Range Image Database Theexperimental hardware environment is Intel CPU (R) Core(TM) i7-3770 340GHz and the memory is 800GB Thedynamic self-occlusion avoidance program is implemented

with C++ language In the process of self-occlusion avoid-ance the parameter of the projection matrix in OpenGL is(60 1 200 600) and the window size is 400 times 400

42 The Experimental Results and Analysis To verify thefeasibility of the proposed method we do experiments withdifferent visual objects in different motion modes Duringthe experiments the initial camera observation position isxbegin = [000 minus100 30000]

119879 and the initial observationdirection is kbegin = [000 100 minus30000]

119879 The coordinateunit is millimeter (mm) and the offset coefficient is 120575 = 4Theprocess of dynamic self-occlusion avoidance will terminate ifthe calculated difference of 119891(x) between two adjacent viewsis less than the given threshold 30mm2 The offset coefficientand threshold are chosen by empirical value Due to thelimited space here we show 3 groups of experimental resultscorresponding to the visual object Bunny with relatively largeself-occlusion region and Duck with relatively small self-occlusion region The results are shown in Tables 1 2 and3 respectively Table 1 shows the process of dynamic self-occlusion avoidance in which the visual object Bunny doestranslation along [1 0 0]119879 with the speed of 3mms Table 2shows the process of dynamic self-occlusion avoidance inwhich the visual object Bunnydoes translation along [2 1 0]119879

with the speed ofradic5mms and rotation around [minus4 1 25]119879with the speed of 1∘s Table 3 shows the process of dynamicself-occlusion avoidance inwhich the visual objectDuck doestranslation along [2 1 0]

119879 with the speed of radic5mms androtation around [minus4 1 25]

119879 with the speed of 1∘s Becausethe camera needs multiple motions in the process of dynamicself-occlusion avoidance only partial observing results thematched results of Gaussian curvature feature matrices andrelated camera information are shown in Tables 1 2 and 3after the results of first three images are shown

The first row in Tables 1 2 and 3 shows the depth imagenumber corresponding to different views in the process ofself-occlusion avoidance At the end of the self-occlusionavoidance the image number in Tables 1 2 and 3 is 1716 and 13 respectively Thus we can see that the cameramovement times are different for different visual objects orthe same visual object with different motions in the processof self-occlusion avoidance The second row in Tables 1 2and 3 shows the acquired depth images of visual object indifferent camera views which are determined by proposedmethod From the images in this row we can see that theself-occlusion region is gradually observed more and morein the process of self-occlusion avoidance for example theself-occlusion region mainly caused by Bunnyrsquos ear in thesecond image in Tables 1 and 2 and the self-occlusion regionmainly caused by Duckrsquos mouth in the second image inTable 3 are better observed when the process of self-occlusionavoidance is accomplished The third row in Tables 1 2and 3 shows the matched results of the Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages with SIFT algorithm These matched results are thebasis of motion estimation and self-occlusion avoidance andgood matched results are helpful to obtain accurate motionestimation then the accurate camera motion trajectory can

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 5

method in literature [12] is used for calculating the Gaussiancurvature feature matrix of depth image

322 Estimating the Motion Equation of Visual Object Basedon the Matched Feature Points Analysis shows that thematched points can provide reliable information for themotion estimation if the Gaussian curvature feature matricescorresponding to two adjacent depth images of the visualobject can be matched SIFT algorithm in literature [13]is more efficient and it is not only invariant to imagetranslation rotation and scaling but also robust to thechange of visual angle affine transformation and noise sothe SIFT algorithm is adopted in this paper for matching thefeature points

SIFT algorithm mainly consists of two steps the gener-ation of SIFT features and the matching of feature vectorsIn order to generate the SIFT features firstly the scale spaceshould be established Because the Difference of Gaussiansscale-space (DoG scale-space) has good stability in detectionof key points the DoG scale-space is used for detecting thekey points in this paper Secondly the key points which areinstable and sensitive to noise are filtered Then the directionof each feature point is calculated Finally the SIFT featurevector of each key point is generated In the process ofgenerating SIFT feature vector 16 seed points are selectedfor each key point and each seed point has 8 orientationsTherefore the 16 times 8 = 128 dimensions of the SIFT featurevector can be obtained for each key point After obtainingthe SIFT feature vectors of the Gaussian curvature featurematrices corresponding to the two adjacent depth images thetwoGaussian curvature featurematrices can bematched andthe similarity of the vectors ismeasured byEuclidean distancein the process of matching

After the key points of the Gaussian curvature featurematrices corresponding to the two adjacent depth images arematched the motion equation of visual object can be esti-mated by using the pixelsrsquo 3D coordinates which are obtainedby antiprojection transformation The motion equation isdefined as

x1015840119897= Rx119897+ T (6)

where x119897and x1015840119897are respectively the 3D coordinates of points

before and after visual objectmotionR is the unit orthogonalrotation matrix and T is the translation vector then theresult of motion estimation is the solution of minimizing theobjective function 119891(RT) The objective function 119891(RT) isdefined as

119891 (RT) =119872

sum

119897=1

10038171003817100381710038171003817x1015840119897minus (Rx

119897+ T)10038171003817100381710038171003817

2

(7)

where119872 is the number of key points which are matched Inorder to solve the R and T in formula (7) the matrix H isdefined as

H =

119872

sum

119897=1

x1015840clx119879

cl (8)

where x1015840cl = x1015840119897minus (1119872)sum

119872

119897=1x1015840119897 xcl = x

119897minus (1119872)sum

119872

119897=1x119897

By carrying out singular value decomposition on H H canbe expressed as H=UΛV119879 then according to U and V R informula (7) can be deduced as

R = VU119879 (9)

After obtaining R substitute R into formula (10) tocalculate T namely

T = x1015840119897minus Rx119897 (10)

After obtaining R and T the motion equation shown informula (6) can be obtained and themotion estimation of thevisual object is accomplished

33 Planning the Next View Based on Motion Estimationand Best View Model After the motion equation of visualobject is estimated the next view of the camera can beplanned Analysis can be known if substituting the cameraobservation position into the motion equation of visualobject the camera can do the synchronous motion with thevisual object In addition on the basis of the synchronousmotion with the visual object if the camera motion offsetfor avoiding self-occlusion is introduced the whole processof dynamic self-occlusion avoidance can be achieved Byanalyzing the best view model in formula (5) we can knowit is the camera best view that can make formula (11) achievemaximum value Formula (11) is defined as

119891 (x) =119873

sum

119895=1

119878119895cos (120579

119895 (x)) (11)

where x is the camera observation position Since formula (11)is a nonconvex model it is difficult to directly calculate itsglobal optimal solution Meanwhile because the problem ofdynamic self-occlusion avoidance needs to calculate a seriesof next views and the gradient information in formula (11) canprovide basis for determining the cameramotion offset of theself-occlusion avoidance process in the process of dynamicself-occlusion avoidance the gradient descent idea is usedin this paper for defining the formula of camera observationposition x1015840

119873119881as

x1015840119873119881

= Rx119881+ T + 120575nabla119891 (x

119881) (12)

where x119881is the current observation position of the camera

and 120575 is the offset coefficient Based on the solution to formula(12) the formula of next observation direction k

119873119881is defined

as

k119873119881

= xmid minus x1015840119873119881

(13)

According to the constraint condition in formula (5) wecan search one point x

119873119881along the opposite direction of k

119873119881

and make x119873119881

meet1003817100381710038171003817xmid minus x

119873119881

10038171003817100381710038172 =10038171003817100381710038171003817xmid minus xbegin

100381710038171003817100381710038172 (14)

By now the obtained view (x119873119881

v119873119881

) can be regardedas the new current view (x

119881 v119881) After acquiring the depth

6 Mathematical Problems in Engineering

image of visual object in the new view (x119881 v119881) the process

from formula (6) to formula (14) is repeated until thedifference between two 119891(x) which are calculated in twoadjacent views according to formula (11) is less than a giventhreshold It means that a series of views (x

119873119881 v119873119881

) can becalculated during the visual object motion so the process ofdynamic self-occlusion avoidance can be accomplished

34 The Algorithm of Dynamic Self-Occlusion Avoidance

Algorithm 1 (dynamic self-occlusion avoidance)

Input The camera internal and external parameters and twoadjacent depth images acquired in initial camera view

Output The set of a series of camera views corresponding tothe dynamic self-occlusion avoidance

Step 1 Calculate the pixelsrsquo 3D coordinates and the Gaussiancurvature feature matrices corresponding to the two adjacentdepth images

Step 2 Detect the self-occlusion boundary in the seconddepth image and establish the self-occlusion region in thesecond depth image according to formula (1) to formula (4)

Step 3 Construct the best view model according to self-occlusion information (formula (5))

Step 4 Match the key points based on Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages by using SIFT algorithm

Step 5 Based on the 3D coordinates of matched key pointssolve the objective function in formula (7) according toformula (8) to formula (10) then the visual objectrsquos motionequation in formula (6) can be obtained

Step 6 Plan the next view of camera according to formula (11)to formula (14)

Step 7 In the obtained next view acquire a depth image andcalculate 119891(x) in formula (11)

Step 8 If the difference between two adjacent 119891(x) is lessthan a given threshold the process of dynamic self-occlusionavoidance can be terminated and otherwise jump to Step 4

to continue the process of dynamic self-occlusion avoidance

4 Experiments and Analysis

41 Experimental Environment In order to validate the effectof the method in this paper the OpenGL is adopted toimplement the process of dynamic self-occlusion avoidanceduring camera observing the moving object based on 3Dobject model in the Stuttgart Range Image Database Theexperimental hardware environment is Intel CPU (R) Core(TM) i7-3770 340GHz and the memory is 800GB Thedynamic self-occlusion avoidance program is implemented

with C++ language In the process of self-occlusion avoid-ance the parameter of the projection matrix in OpenGL is(60 1 200 600) and the window size is 400 times 400

42 The Experimental Results and Analysis To verify thefeasibility of the proposed method we do experiments withdifferent visual objects in different motion modes Duringthe experiments the initial camera observation position isxbegin = [000 minus100 30000]

119879 and the initial observationdirection is kbegin = [000 100 minus30000]

119879 The coordinateunit is millimeter (mm) and the offset coefficient is 120575 = 4Theprocess of dynamic self-occlusion avoidance will terminate ifthe calculated difference of 119891(x) between two adjacent viewsis less than the given threshold 30mm2 The offset coefficientand threshold are chosen by empirical value Due to thelimited space here we show 3 groups of experimental resultscorresponding to the visual object Bunny with relatively largeself-occlusion region and Duck with relatively small self-occlusion region The results are shown in Tables 1 2 and3 respectively Table 1 shows the process of dynamic self-occlusion avoidance in which the visual object Bunny doestranslation along [1 0 0]119879 with the speed of 3mms Table 2shows the process of dynamic self-occlusion avoidance inwhich the visual object Bunnydoes translation along [2 1 0]119879

with the speed ofradic5mms and rotation around [minus4 1 25]119879with the speed of 1∘s Table 3 shows the process of dynamicself-occlusion avoidance inwhich the visual objectDuck doestranslation along [2 1 0]

119879 with the speed of radic5mms androtation around [minus4 1 25]

119879 with the speed of 1∘s Becausethe camera needs multiple motions in the process of dynamicself-occlusion avoidance only partial observing results thematched results of Gaussian curvature feature matrices andrelated camera information are shown in Tables 1 2 and 3after the results of first three images are shown

The first row in Tables 1 2 and 3 shows the depth imagenumber corresponding to different views in the process ofself-occlusion avoidance At the end of the self-occlusionavoidance the image number in Tables 1 2 and 3 is 1716 and 13 respectively Thus we can see that the cameramovement times are different for different visual objects orthe same visual object with different motions in the processof self-occlusion avoidance The second row in Tables 1 2and 3 shows the acquired depth images of visual object indifferent camera views which are determined by proposedmethod From the images in this row we can see that theself-occlusion region is gradually observed more and morein the process of self-occlusion avoidance for example theself-occlusion region mainly caused by Bunnyrsquos ear in thesecond image in Tables 1 and 2 and the self-occlusion regionmainly caused by Duckrsquos mouth in the second image inTable 3 are better observed when the process of self-occlusionavoidance is accomplished The third row in Tables 1 2and 3 shows the matched results of the Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages with SIFT algorithm These matched results are thebasis of motion estimation and self-occlusion avoidance andgood matched results are helpful to obtain accurate motionestimation then the accurate camera motion trajectory can

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

6 Mathematical Problems in Engineering

image of visual object in the new view (x119881 v119881) the process

from formula (6) to formula (14) is repeated until thedifference between two 119891(x) which are calculated in twoadjacent views according to formula (11) is less than a giventhreshold It means that a series of views (x

119873119881 v119873119881

) can becalculated during the visual object motion so the process ofdynamic self-occlusion avoidance can be accomplished

34 The Algorithm of Dynamic Self-Occlusion Avoidance

Algorithm 1 (dynamic self-occlusion avoidance)

Input The camera internal and external parameters and twoadjacent depth images acquired in initial camera view

Output The set of a series of camera views corresponding tothe dynamic self-occlusion avoidance

Step 1 Calculate the pixelsrsquo 3D coordinates and the Gaussiancurvature feature matrices corresponding to the two adjacentdepth images

Step 2 Detect the self-occlusion boundary in the seconddepth image and establish the self-occlusion region in thesecond depth image according to formula (1) to formula (4)

Step 3 Construct the best view model according to self-occlusion information (formula (5))

Step 4 Match the key points based on Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages by using SIFT algorithm

Step 5 Based on the 3D coordinates of matched key pointssolve the objective function in formula (7) according toformula (8) to formula (10) then the visual objectrsquos motionequation in formula (6) can be obtained

Step 6 Plan the next view of camera according to formula (11)to formula (14)

Step 7 In the obtained next view acquire a depth image andcalculate 119891(x) in formula (11)

Step 8 If the difference between two adjacent 119891(x) is lessthan a given threshold the process of dynamic self-occlusionavoidance can be terminated and otherwise jump to Step 4

to continue the process of dynamic self-occlusion avoidance

4 Experiments and Analysis

41 Experimental Environment In order to validate the effectof the method in this paper the OpenGL is adopted toimplement the process of dynamic self-occlusion avoidanceduring camera observing the moving object based on 3Dobject model in the Stuttgart Range Image Database Theexperimental hardware environment is Intel CPU (R) Core(TM) i7-3770 340GHz and the memory is 800GB Thedynamic self-occlusion avoidance program is implemented

with C++ language In the process of self-occlusion avoid-ance the parameter of the projection matrix in OpenGL is(60 1 200 600) and the window size is 400 times 400

42 The Experimental Results and Analysis To verify thefeasibility of the proposed method we do experiments withdifferent visual objects in different motion modes Duringthe experiments the initial camera observation position isxbegin = [000 minus100 30000]

119879 and the initial observationdirection is kbegin = [000 100 minus30000]

119879 The coordinateunit is millimeter (mm) and the offset coefficient is 120575 = 4Theprocess of dynamic self-occlusion avoidance will terminate ifthe calculated difference of 119891(x) between two adjacent viewsis less than the given threshold 30mm2 The offset coefficientand threshold are chosen by empirical value Due to thelimited space here we show 3 groups of experimental resultscorresponding to the visual object Bunny with relatively largeself-occlusion region and Duck with relatively small self-occlusion region The results are shown in Tables 1 2 and3 respectively Table 1 shows the process of dynamic self-occlusion avoidance in which the visual object Bunny doestranslation along [1 0 0]119879 with the speed of 3mms Table 2shows the process of dynamic self-occlusion avoidance inwhich the visual object Bunnydoes translation along [2 1 0]119879

with the speed ofradic5mms and rotation around [minus4 1 25]119879with the speed of 1∘s Table 3 shows the process of dynamicself-occlusion avoidance inwhich the visual objectDuck doestranslation along [2 1 0]

119879 with the speed of radic5mms androtation around [minus4 1 25]

119879 with the speed of 1∘s Becausethe camera needs multiple motions in the process of dynamicself-occlusion avoidance only partial observing results thematched results of Gaussian curvature feature matrices andrelated camera information are shown in Tables 1 2 and 3after the results of first three images are shown

The first row in Tables 1 2 and 3 shows the depth imagenumber corresponding to different views in the process ofself-occlusion avoidance At the end of the self-occlusionavoidance the image number in Tables 1 2 and 3 is 1716 and 13 respectively Thus we can see that the cameramovement times are different for different visual objects orthe same visual object with different motions in the processof self-occlusion avoidance The second row in Tables 1 2and 3 shows the acquired depth images of visual object indifferent camera views which are determined by proposedmethod From the images in this row we can see that theself-occlusion region is gradually observed more and morein the process of self-occlusion avoidance for example theself-occlusion region mainly caused by Bunnyrsquos ear in thesecond image in Tables 1 and 2 and the self-occlusion regionmainly caused by Duckrsquos mouth in the second image inTable 3 are better observed when the process of self-occlusionavoidance is accomplished The third row in Tables 1 2and 3 shows the matched results of the Gaussian curvaturefeature matrices corresponding to the two adjacent depthimages with SIFT algorithm These matched results are thebasis of motion estimation and self-occlusion avoidance andgood matched results are helpful to obtain accurate motionestimation then the accurate camera motion trajectory can

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 7

Table 1 The self-occlusion avoidance process of visual object Bunny with translation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 10 sdot sdot sdot 16 17

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 221 149 sdot sdot sdot 214 sdot sdot sdot 305 sdot sdot sdot 323 mdash

Error matchedpoints mdash 32 33 sdot sdot sdot 47 sdot sdot sdot 55 sdot sdot sdot 35 mdash

Error rate mdash 145 222 sdot sdot sdot 219 sdot sdot sdot 180 sdot sdot sdot 108 mdash

Observationposition

000 minus10030000

000 minus10030000

6005 minus37129306 sdot sdot sdot

15494minus602024336

sdot sdot sdot

22252minus1040316734

sdot sdot sdot25637

minus12216 941326057

minus12269 8872

Observationdirection

000 100minus30000

000 100minus30000

minus5456 minus173minus29499 sdot sdot sdot

minus16823 9591minus22913 sdot sdot sdot

minus2459712077minus12212

sdot sdot sdotminus25867

12317 minus8900minus26593

12399 minus6251

Observed area mdash 16761 58702 sdot sdot sdot 215860 sdot sdot sdot 262568 sdot sdot sdot 301434 302558

be further calculated Therefore the matched results in thethird row will influence the effect of self-occlusion avoidanceto a certain extent The fourth fifth and sixth rows in Tables1 2 and 3 respectively show the total matched points theerror matched points and the error rate in the process ofmatching with SIFT algorithm where the error rate is equalto the error matched points divided by the total matchedpointsThe seventh eighth and ninth rows in Tables 1 2 and3 respectively show the observation position observationdirection and the observed area of the vested self-occlusionregion in related view when the visual object Bunny andDuck do different motions It can be seen from these dataand their change trends that the process of avoiding self-occlusion dynamically with proposed method accords withthe observing habit of human vision

In view of the fact that there is nomethod of dynamic self-occlusion avoidance to be compared and no Ground Truthabout dynamic self-occlusion avoidance currently it is verydifficult to evaluate our method by comparing with othermethods or Ground Truth In order to make the furtherquantitative analysis of the feasibility and effectiveness of ourmethod after deeply analyzing the characteristics of dynamicself-occlusion avoidance we propose an index named ldquoeffec-tive avoidance raterdquo to measure the performance of dynamicself-occlusion avoidance method Considering the effect ofobject surface the same object will have different self-occlusion regions in different camera views and meanwhilethe distribution of the self-occlusion region is random it is

not realistic to find a camera view in which the camera canobtain all the information of the self-occlusion region so itis inappropriate to take the ratio of the final observed self-occlusion region area to the total self-occlusion region area asevaluation criterion In addition because the goal of dynamicself-occlusion avoidance is that the camera not only can trackthe visual object but also can observe the vested self-occlusionregion maximally the camera will inevitably gradually beclose to the best view in the process of dynamic self-occlusionavoidance namely the objective view described in formula(5) The area in the objective view xBV is also the self-occlusion region area really expected to be observed Basedon this the proposed index of effective avoidance rate isdefined as

120578 =

119878viewsum119873

119895=1119878119895

119878purposesum119873

119895=1119878119895

times 100 =119878view119878purpose

times 100 (15)

where 120578 is the effective avoidance rate 119878view is the observedself-occlusion region area in the view where the process ofdynamic self-occlusion avoidance is accomplished 119878purposeis the self-occlusion region area in the objective viewand sum

119873

119895=1119878119895is the total self-occlusion region area to be

avoidedIn order to make quantitative analysis of the performance

of proposed method based on the effective avoidance ratewe have done multiple groups of experiments by adjust-ing the motion modes of different visual objects Table 4

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

8 Mathematical Problems in Engineering

Table 2 The self-occlusion avoidance process of visual object Bunny with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 5 sdot sdot sdot 9 sdot sdot sdot 15 16

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 224 162 sdot sdot sdot 205 sdot sdot sdot 247 sdot sdot sdot 338 mdash

Error matchedpoints mdash 20 11 sdot sdot sdot 30 sdot sdot sdot 42 sdot sdot sdot 29 mdash

Error rate mdash 89 68 sdot sdot sdot 146 sdot sdot sdot 170 sdot sdot sdot 86 mdash

Observationposition

000 minus10030000

000 minus10030000

5668 minus20029507 sdot sdot sdot

12716 minus457426626 sdot sdot sdot

21141 minus977818011 sdot sdot sdot

25384minus1114710268

26232minus11725 9219

Observationdirection

000 100minus30000

000 100minus30000

minus5909 minus485minus29409 sdot sdot sdot

minus133205028minus26407

sdot sdot sdot

minus2264012939minus14833

sdot sdot sdotminus25515

13429 minus8287minus25698

14529 minus5343

Observed area mdash 16761 62548 sdot sdot sdot 145364 sdot sdot sdot 243137 sdot sdot sdot 264857 265463

Table 3 The self-occlusion avoidance process of visual object Duck with both translation and rotation

Information Image number1 2 3 sdot sdot sdot 6 sdot sdot sdot 9 sdot sdot sdot 12 13

Depth image sdot sdot sdot sdot sdot sdot sdot sdot sdot

MatchingGaussiancurvaturefeature matriceswith SIFTalgorithm

mdash sdot sdot sdot sdot sdot sdot sdot sdot sdot mdash

Total matchedpoints mdash 552 431 sdot sdot sdot 456 sdot sdot sdot 432 sdot sdot sdot 438 mdash

Error matchedpoints mdash 20 20 sdot sdot sdot 38 sdot sdot sdot 22 sdot sdot sdot 71 mdash

Error rate mdash 36 46 sdot sdot sdot 83 sdot sdot sdot 51 sdot sdot sdot 162 mdashObservationposition

000 minus10030000

000 minus10030000

minus018 261329759 sdot sdot sdot

minus932 656229207 sdot sdot sdot

minus198810603 27918 sdot sdot sdot

minus497413568 26574

minus243010582 25845

Observationdirection

000 100minus30000

000 100minus30000

084 minus2599minus29887 sdot sdot sdot

1472 minus6807minus29181 sdot sdot sdot

1758 minus9239minus28488 sdot sdot sdot

2396minus12071minus27360

5605minus11493minus27139

Observed area mdash 8304 22232 sdot sdot sdot 31765 sdot sdot sdot 46284 sdot sdot sdot 70577 73358

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 9

Table 4 The quantitative analysis results for 10 groups of experiments

Groupnumber

Visualobject Initial camera view Motion mode

sum119873

119895 = 1119878119895

(mm2)119878purpose(mm2)

119878view(mm2)

120578

()119879

(s) 119873119879

(s)

1 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879Translation along[1 0 0]119879 with thespeed of 3mms

728926 318912 302558 9487 158 17 093

2 Bunnyxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

728926 318912 265463 8324 155 16 097

3 Duckxbegin = [000 minus100 30000]119879

vbegin = [000 100 minus30000]119879

Translation along[2 1 0]119879 with thespeed ofradic5mms

and rotationaround [minus4 125]119879 with thespeed of 1∘s

191110 102527 73358 7155 133 13 102

4 Bunnyxbegin = [12000 10000 minus25600]119879

vbegin = [minus12000 minus10000 minus25600]119879Rotation around[7 minus3 4]119879 with the

speed of 2∘s502892 323847 301748 9318 136 15 091

5 Bunnyxbegin = [15000 17000 19600]119879

vbegin = [minus15000 minus17000 minus19600]119879

Translation along[1 2 1]119879 with thespeed ofradic6mms

and rotationaround [2 minus1 minus2]119879with the speed of

1∘s

441450 175818 155348 8836 171 18 095

6 Duckxbegin = [10000 minus15000 24000]119879

vbegin = [minus10000 15000 minus24000]119879

Translation along[3 2 minus1]119879 with the

speed ofradic14mms

330746 278837 274835 9856 195 21 093

7 Molexbegin = [minus21000 5000 21000]119879

vbegin = [21000 minus5000 minus21000]119879

Translation along[2 0 2]119879 with thespeed ofradic8mms

and rotationaround [1 minus1 minus2]119879with the speed of

3∘s

32001 22168 13309 6004 112 11 102

8 Rockerxbegin = [minus19600 17000 15000]119879

vbegin = [19600 minus17000 minus15000]119879Translation along[1 1 2]119879 with thespeed ofradic6mms

190509 144092 136345 9462 84 9 093

9 Rockerxbegin = [19000 minus20000 11800]119879

vbegin = [minus19000 20000 minus11800]119879

Translation along[2 minus1 minus1]119879 withthe speed ofradic6mms androtation around[minus1 2 minus3]119879 withthe speed of 2∘s

343354 270733 194798 7195 98 10 098

10 Dragonxbegin = [19000 20000 11800]119879

vbegin = [minus19000 minus20000 minus11800]119879

Translation along[2 4 minus1]119879 with the

speed ofradic21mms androtation around[minus1 2 minus2]119879 withthe speed of 3∘s

573648 194861 148598 7626 165 15 110

Average mdash mdash mdash 406356 215071 186636 8678 141 145 097

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

10 Mathematical Problems in Engineering

shows the quantitative analysis results of total 10 groupsof experiments which include the 3 groups of experimentsdescribed above and the other 7 groups In addition in thelast column of each experiment the average time whichis from the time that the camera arrives at a certain viewto the time that the next view has been calculated isshown

From the 10 groups of experiments in Table 4 we can seethat affected by surface of visual object the different self-occlusion region area sum119873

119895=1119878119895will be generated for the same

(or different) visual object in different (or the same) cameraviews and the distribution of the self-occlusion region israndom so the vested self-occlusion region area sum119873

119895=1119878119895will

not be observed completely even if the camera is in thebest view (the objective view) At the same time the cameracan only infinitely be close to but hardly reach the objectiveview accurately in the process of self-occlusion avoidancethat is to say at the end of self-occlusion avoidance thecamera observed area 119878view can only be close to but notreach the area 119878purpose in the objective view which alsoverifies the rationality of taking the effective avoidance rate120578 as evaluation criterion Besides comparing 10 groups ofexperiments in Table 4 we can know that the visual objectsin groups 1 4 6 and 8 only do translation or rotation indifferent initial views and the motion is relatively simplethen the average effective avoidance rate of these fourgroups is 9529 But in the groups 2 3 5 7 9 and 10 inTable 4 the visual objects do both translation and rotationin different initial views the motion is relatively complexand then the average effective avoidance rate of these sixgroups is 7842 which is obviously lower than that of thefour preceding groups in which the visual objects only dotranslation or rotation The main reason from our analysisis that many steps in the process of dynamic self-occlusionavoidance such as antiprojection transformation matchingpoints based on SIFT andmotion estimation will make errorand the error will accumulate when the camera changes itsview The most important thing is that error accumulationwill become more obvious when the motion of the visualobject is more complex which will lead to the result thatthe effective avoidance rate will be significantly lower whenthe motion of the visual object is complex Objectivelyspeaking it is inevitable that there are error and erroraccumulation in the process of self-occlusion avoidanceand it is not realistic to eliminate the effect of these errorscompletely what different methods can do is to reduce orcut down the effect of these errors The last row data inTable 4 shows that the average effective avoidance rate ofthe proposed method in this paper reaches 8678 thereforein the average sense the better result of dynamic self-occlusion avoidance can be obtained based on the proposedmethod Furthermore from the last column data in Table 4it can be seen that the average time consumption is 097swhich is from the time that the camera arrives at a certainview to the time that the next view has been calculatedIt shows that the proposed method has good real-timeperformance

5 Conclusion

In this paper an approach for avoiding self-occlusion dynam-ically is proposed based on the depth image sequence ofmoving visual object The proposed approach does not needspecific equipment and the a priori knowledge of visual objector limit the observation position of the camera on a fixedsurface This work is distinguished by three contributionsFirstly the problem of dynamic self-occlusion avoidancewhich is still in initial stage at present is studied and thesolution to dynamic self-occlusion avoidance problem is pro-posed and implemented which provides an idea for furtherstudying the dynamic self-occlusion avoidance Secondlybased on the depth images of moving visual object theGaussian curvature feature matrices are matched by SIFTalgorithm and the motion equation is efficiently estimatedby using singular value decompositionThis provides a novelidea about motion estimation in 3D space based on the depthimage of visual object Thirdly considering that there are noevaluation criterions about dynamic self-occlusion avoidanceat present an evaluation criterion named ldquoeffective avoidanceraterdquo is proposed to measure the performance of dynamicself-occlusion avoidance reasonably and it provides thereferable evaluation basis for further studying the problem ofdynamic self-occlusion avoidance The experimental resultsshow that the proposed approach achieves the goal thatthe camera avoids the self-occlusion automatically when thevisual object is moving and the avoidance behavior accordswith the observing habit of human vision

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this article

Acknowledgments

This work is supported by the National Natural ScienceFoundation of China under Grant no 61379065 and theNatural Science Foundation of Hebei province in Chinaunder Grant no F2014203119

References

[1] C Connolly ldquoThe determination of next best viewsrdquo in Pro-ceedings of the 1985 IEEE International Conference on Roboticsand Automation pp 432ndash435 IEEEMissouriMo USAMarch1985

[2] R Pito ldquoA sensor-based solution to the lsquonext best viewrsquoproblemrdquo in Proceedings of the 13th International Conferenceon Pattern Recognition (ICPR rsquo96) vol 1 pp 941ndash945 IEEEVienna Austria August 1996

[3] P Whaite and F P Ferrie ldquoAutonomous exploration drivenby uncertaintyrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 19 no 3 pp 193ndash205 1997

[4] Y F Li and Z G Liu ldquoInformation entropy-based viewpointplanning for 3-D object reconstructionrdquo IEEE Transactions onRobotics vol 21 no 3 pp 324ndash337 2005

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Mathematical Problems in Engineering 11

[5] M Trummer C Munkelt and J Denzler ldquoOnline next-best-view planning for accuracy optimization using an extended E-criterionrdquo in Proceedings of the 20th International Conference onPattern Recognition (ICPR rsquo10) pp 1642ndash1645 IEEE IstanbulTurkey August 2010

[6] J E Banta L M Wong C Dumont and M A Abidi ldquoA next-best-view system for autonomous 3-D object reconstructionrdquoIEEE Transactions on Systems Man and Cybernetics Part ASystems and Humans vol 30 no 5 pp 589ndash598 2000

[7] W Fang and B He ldquoAutomatic view planning for 3D recon-struction and occlusion handling based on the integrationof active and passive visionrdquo in Proceedings of the 21st IEEEInternational Symposium on Industrial Electronics (ISIE rsquo12) pp1116ndash1121 Hangzhou China May 2012

[8] J I Vasquez-Gomez L E Sucar and R Murrieta-Cid ldquoHierar-chical ray tracing for fast volumetric next-best-view planningrdquoin Proceedings of the 10th International Conference on Computerand Robot Vision (CRV rsquo13) pp 181ndash187 IEEE SaskatchewanCanada May 2013

[9] C Potthast and G S Sukhatme ldquoA probabilistic framework fornext best view estimation in a cluttered environmentrdquo Journalof Visual Communication and Image Representation vol 25 no1 pp 148ndash164 2014

[10] B Wu X Sun Q Wu M Yan H Wang and K Fu ldquoBuildingreconstruction from high-resolution multiview aerial imageryrdquoIEEE Geoscience and Remote Sensing Letters vol 12 no 4 pp855ndash859 2015

[11] S Zhang and J Liu ldquoA self-occlusion detection approach basedon depth image using SVMrdquo International Journal of AdvancedRobotic Systems vol 9 article 230 2012

[12] P J Besl and R C Jain ldquoSegmentation through variable-order surface fittingrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 10 no 2 pp 167ndash192 1988

[13] N Bayramoglu and A A Alatan ldquoShape index SIFT rangeimage recognition using local featuresrdquo in Proceedings of the20th International Conference on Pattern Recognition (ICPR rsquo10)pp 352ndash355 IEEE Istanbul Turkey August 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of