a principal component regression strategy for motion estimation iasted 2007 583-110

8/9/2019 A Principal Component Regression Strategy for Motion Estimation Iasted 2007 583-110

http://slidepdf.com/reader/full/a-principal-component-regression-strategy-for-motion-estimation-iasted-2007 1/5

A PRINCIPAL COMPONENT REGRESSION STRATEGY FOR ESTIMAMOTION

Vania V. Estrela1, M.H. Da Silva Bassani2, and J. T. de Assis31Scientific Computing Group (NCC),

State University of Western Rio de Janeiro(UEZO)Rua Manoel Caldeira de Alvarenga nº 1203, Campo Grande, RJ, Brazil, CEP 23070-120

[email protected] 2Mechanical Engineering Department, DEPES, CEFET-RJ

Av. Maracana 229, Rio de Janeiro, RJ, Brazil, CEP [email protected]

3State University of Rio de Janeiro (UERJ),Instituto Politécnico (IPRJ-UERJ)

CP 972825 – CEP 28601-970 – Nova Friburgo, RJ, Brazil [email protected]

ABSTRACTIn this paper, we derive a principal component regression(PCR) method for estimating the optical flow betweenframes of video sequences according to a pel-recursivemanner. This is an easy alternative to dealing withmixtures of motion vectors due to the lack of too much prior information on their statistics (although they aresupposed to be normal). The 2D motion vector estimationtakes into consideration local image properties. The mainadvantage of the developed procedure is that noknowledge of the noise distribution is necessary.Preliminary experiments indicate that this approach provides robust estimates of the optical flow.KEY WORDSMotion estimation, principal component regression, andsurveillance.

1. IntroductionMotion estimation algorithms have found a number of applications ranging from video coding to rendering of scenes in virtual reality environments. The evolution of animage sequence motion field can also help other image processing tasks in multimedia applications such asanalysis, recognition, tracking, restoration, collisionavoidance and segmentation of objects.

In coding applications, a block-based approach isoften used for interpolation of lost information betweenkey frames as shown by Tekalp [14]. The fixedrectangular partitioning of the image used by some block- based approaches often separates visually meaningfulimage features. Pel-recursive schemes ([6], [7], [13], [14])can theoretically overcome some of the limitationsassociated with blocks by assigning a unique motion

vector to each pixel. Intermediate frames are constructed by resampling the image at locadetermined by linear interpolation of the motion vThe pel-recursive approach can also manage motiosub-pixel accuracy. The update of the motion estim based on the minimization of the displaced difference (DFD) at a pixel. In the absence of addassumptions about the pixel motion, this estim problem becomes “ill-posed” because of the fol problems: a) occlusion; b) the solution to the 2D mestimation problem is not unique (aperture problemc) the solution does not continuously depend on thdue to the fact that motion estimation is highly sensthe presence of observation noise in video images.

We propose to solve optical flow (OF) problemmeans of a well-stablished technique for dimensioreduction named principal component analysis (PCis closely related to the EM framework presentedand [15]. Such approach accounts better for the sta properties of the errors present in the scenes thsolution proposed in previous works ([1],[4) relythe assumption that the contaminating noise has Gadistribution.

Most methods assume that there is little ointerference between the individual sample constitu

that all the constituents in the samples are known ahtime. In real world samples, it is very unusual, entirely impossible to know the entire compositiomixture sample. Sometimes, only the quantities ofconstituents in very complex mixtures of muconstituents are of interest ([2],[13],[15]).

In this paper we propose a new method for estimoptical flow which incorporates a general PCR mOur method is based on the works of Jollife [12Wold [16] and involves a much simpler computa

583-110 224



procedure than previous attempts at addressing mixturessuch as the ones found in [2], [13] and [15].

In the next section, we will set up a model for our optical flow estimation problem, and in the followingsection present a brief review of previous work in thisarea. Section 2 states the simplest and most commonform of regression: the ordinary least squares (OLS), aswell as one of its extensions, the regularized leastsquares, RLS, ([5],[8],[9],[10],[11],[12],[14]). Section 3introduces the principal component regression concepts,describing our proposed technique and shows someexperiments used to access the performance of our proposed algorithm. Finally, a conclusion, a discussion of the results and future research plans are presented inSection 4

2. Problem CharacterizationThe displacement of each picture element in each frameforms the displacement vector field (DVF) and itsestimation can be done using at least two successiveframes. The DVF is the 2D motion resulting from theapparent motion of the image brightness (OF). A vector isassigned to each point in the image.

A pixel belongs to a moving area if its intensity haschanged between consecutive frames. Hence, our goal isto find the corresponding intensity value I k (r) of the k-thframe at locationr = [ x , y]T, and d(r) = [d x , d y]T thecorresponding (true) displacement vector (DV) at theworking pointr in the current frame. Pel-recursivealgorithms minimize the DFD function in a small areacontaining the working point assuming constant imageintensity along the motion trajectory. The DFD is defined

byΔ(r;d (r))= I k (r)- I k -1(r-d (r)), (1)

and the perfect registration of frames will result in I k (r)= I k-1(r-d(r)). The DFD represents the error due to thenonlinear temporal prediction of the intensity fieldthrough the DV. The relationship between the DVF andthe intensity field is nonlinear. An estimate of d(r), isobtained by directly minimizing Δ(r,d(r)) or bydetermining a linear relationship between these twovariables through some model. This is accomplished byusing the Taylor series expansion of I k-1(r-d(r)) about thelocation (r-di(r)), wheredi(r) represents a prediction of d(r) in i-th step. This results in

(r,r-di(r))=-uT I k-1(r-di(r))+e(r, d(r)), (2)

where the displacement update vector u=[ux, uy]T = d(r) – di(r), e(r, d(r)) represents the error resulting from thetruncation of the higher order terms (linearization error)and =[δ/δx, δ/δy]T represents the spatial gradient

operator. Applying (2) to all points in a region R of pixelsneighboring the current pixel being studied gives

z =Gu+n, (3)

where the temporal gradients Δ(r, r-di(r)) have beenstacked to form the Nx1 observation vector z containingDFD information on all the pixels in a neighborho R,the Nx2 matrixG is obtained by stacking the spatigradient operators at each observation, and the terms have formed the Nx1 noise vector n which isassumed Gaussian withn~ N (0, σn2I). Each row of G hasentries [gxi, gyi]T, with i = 1, …, N. The spatial gradieof Ik-1 are calculated through a bilinear interpolascheme similar to what is done in [4] and [5].

Once we have stated the problem, we neeinvestigate possible ways of solving (3). This concentrates its attention on regression-like method

The pseudo-inverse or ordinary least -squares (Oestimate of the update vector is

uLS = (GT

G )-1

GT

z, (4)which is given by the minimizer of the functJ(u)=║z-Gu║2 (for more details, see [ 4], [5] and [ From now on,G will be analyzed as being an Nmatrix in order to make the whole theoretical disceasier. SinceG may be very often ill–conditioned, solution given by the previous expression will be uunacceptable due to the noise amplification resultinthe calculation of the inverse matrixGTG. In other words,the data are erroneous or noisy. Therefore, one cexpect an exact solution for the previous equatiorather an approximation according to some proc

The assumptions made aboutn

for least squaresestimation areE(n)=0, and Var ( n )=E(nnT)=σ2IN, whereE(n) is the expected value (mean) of n, IN, is the identitymatrix of order N , andnT is the transpose of n .

The simplest way to stabilize the matrix inverseadd a constant to the diagonal. This is a consequeminimizing the functional resulting in the regulsolution obtained in [5] and [9]. The formal express

uRLS = (GTG+ λ I)-1GT z, (5)

where λ is the regularization coefficient. The resregression coefficients are biased as a result of the regularization. As their variance decrease, howevmodel becomes more stable with respect to the prederror. The variance–bias dilemma is a general probmultivariate regression. The model bias can onreduced at the expense of increased model varianvice versa. The expected prediction error is the stwo components:

E((ˆ

z −z)2) = (model bias)2 + (model variance). (6

225



3. Principal Component Regression (PCR)PCA is a useful method to solve problems includingexploratory data analysis, classification, variabledecorrelation prior to the use of neural networks, patternrecognition, data compression, and noise reduction, for example.

The formulation of PCA implies a Gaussian latentvariable model and can easily lead to Bayesian models.This technique is used whenever uncorrelated linear combinations of variables are wanted, which reduces thedimensions of a set of variables by reconstructing theminto uncorrelated combinations of another set of variables,called principal components (PC´s). The first PCaccounts for the largest part of the variance. The second,for the next largest variance and so on.

PCR is based on principal component analysis (PCA)of the matrixG and it is related to its second statisticalmoment, which is proportional toGTG according to theexpressionG = TPT , where matrixT contains theorthogonal eigenvectors of GTG (also called scores)ordered in descending order by their eigenvalues ([11],[12]). If P has the same rank asG, i.e.,P contains theeigenvectors to all non-zero eigenvalues, thenT =GPis arotation of G. As said before, PCA assumes a Gaussianvariable model and it is simpler than other approachessuch as the one in [2]. In essence, PCR is just multiplelinear regression of PCA scores onz :

uPCR =P(TTT)−1TTz . (7)

In PCR, the inverse matrix is stabilized in an

altogether different way than in regularized least squaresregression (RLS, as seen in [4] and [5]) regression. Themain drawback of PCR is that the largest variation inGmight not correlate withz and therefore the method mayrequire the use of a more complex model. Some nice properties of the PCR are:

1) Using a complete set of PCs, PCR will produce thesame results as the original OLS, but with possibly moreaccuracy if the originalGTG matrix has inversion problems.

2) If GTG is nearly singular, a solution better than the one

given by the OLS can be obtained. If GT

G is singular, thevector associated with the zero root may point towardsremoving one or more of the original variables.

3) The regression coefficients will be uncorrelated and theamounts explained by each PC will be independent and,hence, additive so that the results may be reported in theform of an analysis of variance. If the PCs can be easilyinterpreted, the resultant regression equations may bemore meaningful.

The division of objects into groups can be simpin the space generated by the PCs. Then, the groucharacterizing by the DVs corresponding to the ceeach class. Two measures can be used to deterwhether a sample belongs to a specific class or noMahalanobis distance to the class center and the nthe residual. If a set of observations is plottedrespect to the first two PCs, then one can easily appthat there is a strong suggestion of distinct grouwhich convex hulls and ellipses have been drawn athem. It is likely that the clusters found would corrto different types of displacement vectors.

PCR provides additional information about the being analyzed. The eigenvalues of the correlation of predictor variables play an important role in detmulticollinearity and in analyzing its effects. Theestimates are biased, but may be more accurate thaestimates in terms of mean square error. One shouconsider using OLS on a reduced set of variables, unsatisfactory, then PCR can be used. Beexploring the most obvious approach, it reduce

computer load.

Figure 1. An example of cluster analysis obtainmeans of principal components.

Outliers should not be automatically remo because they are not necessarily bad observations,they can signal some change in the scene context orthe data is not Gaussian or that the model is not linadditional knowledge on the existence of bordeused, then one´s ability to predict the correct motioincrease. To classify a new observation, a “distancthe observation from the hyperplane defined byretained PCs is calculated for each group. If observations are to be assigned to one and only one

226



then assignment is to the cluster from which the distanceis minimized. As it is not close to any of the existinggroups, it may be an outlier or come from a new groupabout which there is currently no information.Conversely, if the classes are not all well separated, somefuture observations may have small distances from morethan one population.

(a)

(b)

(c)

Figure 2. (a) Frame of the Rubik Cube sequence.; (b)DVF for a 31×31 mask obtained by means of PCR withSNR=20 dB; and (c) PCR with discriminant analysis,31×31 mask and edge information with SNR=20 dB.

At this time, we have done an analysis of thePCR for motion estimation purposes and theresults for SNR=20 dB are shown in Fig, 1. In themeantime, they can function as a way of illustrating the performance obtained with our approach. The SNR is defined asSNR = 10log10(σ 2/σ c2), whereσ 2 is the variance of the originalimage and σ c2 is the variance of the noisecorrupted image.

4. ConclusionIn this paper, we extended the PCR framework detection of motion fields. The algorithm devecombines regression and PCA which is primarily a data-analytic method that obtains linear transformationgroup of correlated variables such that certain opconditions are achieved. The resulting transfovariables are uncorrelated.

Unlike other works ([8],[11],[12]), we areinterested in reducing the dimensionality of the fspace describing the different behaviors insidneighborhood surrounding a pixel. Instead, we usin order to validate motion estimates. This can be a simple alternative way of dealing with mixturmotion displacement vectors.

We need to test the proposed algorithm different noisy images and look closely atdiscriminant analysis scheme performance. It isnecessary to incorporate more statistical informatour model and analyze if it will improve the outco

References[1] J. Biemond, L. Looijenga, D.E. Boekee, & R.Plompen, A pel-recursive wiener-based displacestimation algorithm,Signal Proc. , 13, December 1987,399-412.

[2] K. Blekas, A. Likas, N.P. Galatsanos, & I.E. LA spatially-constrained mixture model for isegmentation, IEEE Trans. on Neural Networks , vol. 16,2005, 494-498.

[3] S. Chattefuee, & A.S. Hadi, Regression analysis byexample , (Hoboken, New Jersey: John Wiley & SInc., 2006).

[4] V.V. Estrela, & M.H. da Silva Bassani, Expectamaximization technique and spatial adaptation appl pel-recursive motion estimation, Proc. of the SCI 2004 ,Orlando, Florida, USA, July, 2004, 204-209.

[5] V.V. Estrela, & N.P. Galatsanos, Spatially-adaregularized pel-recursive motion estimation bas

227



cross-validation, Proc. of ICIP-98 (IEEE International Conference on Image Processing) , Vol. III, Chicago, IL,USA, October. 1998, 200-203.

[6] M.O. Franz, & H.G. Krapp, Wide-field, motion-sensitive neurons and matched filters for optic flow fields,

Biological Cybernetics 83 , 2000, 185-197.

[7] M.O. Franz, & J.S. Chahl, Linear combinations of optic flow vectors for estimating self-motion - a real-world test of a neural model, Advances in Neural

Information Processing Systems 15 , 1343-1350. (Eds.)Becker, S., S. Thrun & K. Obermayer, MIT Press,Cambridge, MA 2003).

[8] K. Fukunaga, Introduction to statistical patternrecognition , (San Diego, CA: Computer Science andScientific Computing, Academic Press, 2 ed. 1990).

[9] N.P. Galatsanos, & A.K. Katsaggelos, Methods for choosing the regularization parameter and estimating the

noise variance in image restoration and their relation, IEEE Trans. Image Processing , Vol No , July, 1992,322-336.

[10] S. Haykin, Neural networks: a comprehensive foundation, (Prentice Hall, London, 2 ed.,1999).

[11] Jackson, J.E., A user’s guide to principal components , (John Wiley & Sons, Inc., 1991).

[12] I. Jolliffe, Principal component analysis , (NewYork, NY: Springer-Verlag, 1986).

[13] K. I. Kim, M. Franz, B. Schölkopf, Iterative kernel principal component analysis for image modeling, IEEE Transactions on Pattern Analysis and Machine

Intelligence 27(9), 2005, 1351-1366.

[14] A.M. Tekalp, Digital video processing , (New Jersey, NJ: Prentice-Hall, 1995).

[15] M.E.Tipping, & C.M. Bishop, Probabilistic principalcomponent analysis, J. R. Statist. Soc. B , 61, 1999, 611– 622.

[16] S. Wold, C. Albano, W.J. Dunn, K. Esbensen, S.Hellberg, et. al., Pattern recognition: finding and usingregularities in multivariate data, Food Research and Data

Analysis , Eds. H. Martens and H. Russwurm, London,Applied Science Publishers, 1983, 147–188.

228

a principal component regression strategy for motion estimation iasted 2007 583-110

Documents