a principal component regression strategy for estimating motion eng opt 2008

8/9/2019 A Principal Component Regression Strategy for Estimating Motion Eng Opt 2008

1/8

EngOpt 2008 - International Conference on Engineering OptimizationRio de Janeiro, Brazil, 01 - 05 June 2008.

A Principal Component Regression Strategy for Estimating MotionVania V. Estrela, S. R. Fernandes and J. T. Assis

State University of Rio de Janeiro (UERJ),Instituto Politcnico do Rio de JaneiroCP 972825 CEP 28601-970 Nova Friburgo, RJ, Brazil

[email protected], [email protected], [email protected]

Abstract.In this paper, we derive a principal component regression (PCR) method for estimating the optical flow betweenof video sequences according to a pel-recursive manner. This is an easy alternative to dealing with mixtures of motion vto the lack of too much prior information on their statistics (although they are supposed to be normal). The 2D motion vection takes into consideration local image properties. The main advantage of the developed procedure is that no knowlednoise distribution is necessary. Preliminary experiments indicate that this approach provides robust estimates of the optical Keywords: principal component regression, clustering, motion detection.

1. IntroductionIn a dynamic scene observed by an active sensor, motion provides an important source of information. It signals a dynawhich may be of interest to the observer, it may characterize the interaction of objects, i.e., motion on a collision path docking, it may be obtrusive as during intentional sensor movement, or convey important object properties. Optic flow mosis represents an important family of visual information processing techniques in computer vision. Segmenting an optic into coherent motion groups and estimating each underlying motion is a very challenging task when the optic flow field isfrom a scene of several independently moving objects. The problem is further complicated if the optic flow data are noisy aly incorrect. The use of regression models can suppress a small percentage of gross data errors or outliers. However, segmoptic flow field consisting of a large portion of incorrect data or multiple motion groups requires a very high robustneunattainable by the conventional robust estimators.

Recovering the 3-D motion of rigid objects from image sequences is essential in areas where dynamic scene analyquired, such as stereo-vision. The motion of each object is modeled using 6 parameters that describe object translation anThe parameters are estimated by minimizing the reconstruction error between the original frames and the motion-comframes produced using the 3D motion model.

The main problem of motion analysis is the difficulty of getting accurate motion estimates without prior motion segmenvice-versa. Some researchers tried to address both problems in a unified frameworks which allow simultaneous motion eand segmentation. One method uses a robust estimator which can cope with errors due to noise and scene clutter (multiplobjects). It is based on the Hough transform implemented in a search mode. This has the benefit of identifying the most smoving regions first and, thus, providing an effective focus of attention mechanism. The motion estimator and segmentorvides information about the confidence of motion estimates. This is very important from the point of view of image interThis procedure is very useful not only from the point of view of scene interpretation, but also from the point of other apsuch as video compression and coding.

In coding applications, a block-based approach is often used for interpolation of lost information between key fshown by Tekalp [14]. The fixed rectangular partitioning of the image used by some block-based approaches often separatmeaningful image features. Pel-recursive schemes ([6], [7], [13], [14]) can theoretically overcome some of the limitations with blocks by assigning a unique motion vector to each pixel. Intermediate frames are then constructed by resampling thlocations determined by linear interpolation of the motion vectors. The pel-recursive approach can also manage motion witel accuracy. The update of the motion estimate was based on the minimization of the displaced frame difference (DFD) at the absence of additional assumptions about the pixel motion, this estimation problem becomes ill-posed because of the problems: a) occlusion; b) the solution to the 2D motion estimation problem is not unique (aperture problem); and c) thdoes not continuously depend on the data due to the fact that motion estimation is highly sensitive to the presence of obnoise in video images.We propose to solve optical flow (OF) problems by means of a well-stablished technique for dimensionality renamed principal component analysis (PCA) and is closely related to the EM framework presented in [4] and [15]. Suchaccounts better for the statistical properties of the errors present in the scenes than the solution proposed in previous workrelying on the assumption that the contaminating noise has Gaussian distribution.

Most methods assume that there is little or no interference between the individual sample constituents or thatconstituents in the samples are known ahead of time. In real world samples, it is very unusual, if not entirely impossible toentire composition of a mixture sample. Sometimes, only the quantities of a few constituents in very complex mixtures oconstituents are of interest ([2],[13],[15]).

In this paper we propose a new method for estimating optical flow which incorporates a general PCR model. Our m based on the works of Jollife [12] and Wold [16] and involves a much simpler computational procedure than previous attemdressing mixtures such as the ones found in [2], [13] and [15]. In the next section, we will set up a model for our opticalmation problem, and in the following section present a brief review of previous work in this area, stating the simplest common form of regression: the ordinary least squares (OLS), as well as one of its extensions, the regularized least squa([5],[8],[9],[10],[11],[12],[14]). Section 4 introduces the concepts of PCA, and principal component regression. Section 5


2/8

our proposed technique and shows some experiments used to access the performance of our proposed algorithm. Finally, a of the results and future research plans are presented in Section 6.

2. Problem Formulation

2.1 Displacement EstimationThe displacement of each picture element in each frame forms the displacement vector field (DVF) and its estimation can bing at least two successive frames. The DVF is the 2D motion resulting from the apparent motion of the image brightnesvector is assigned to each point in the image.

A pixel belongs to a moving area if its intensity has changed between consecutive frames. Hence, our goal is to corresponding intensity value I k (r) of thek -th frame at locationr = [ x, y]T, andd(r) = [d x, d y]T the corresponding (true) displacementvector (DV) at the working pointr in the current frame. Pel-recursive algorithms minimize the DFD function in a small area coning the working point assuming constant image intensity along the motion trajectory. The DFD is defined by

(r;d (r))= I k (r)- I k-1(r-d (r)) (1)

and the perfect registration of frames will result in I k (r)= I k-1(r-d(r)). Figure 1 shows some examples of pixel neighborhoods. TheDFD represents the error due to the nonlinear temporal prediction of the intensity field through the DV.

Figure 1. Backward motion estimation problem.

The objective of this work is to determine the motion or displacement vector field (DVF) resulting from the estimthe optical flow (OF) via pel-recursive algorithms. We understand by OF the 2D field of instantaneous velocities or, equ

X Future Pixels (not scanned)u Update vector Pixel position in frame k-1

Past PixelsCurrent Pixel in frame k

P Location of current pixel inframe k projected on frame k-1

Frame k-1

Frame k

Frame k+1


3/8


4/8

Spatial Coherence or Smoothness ConstraintIn general, adjacent pixels in the image are part of a same region. The smoothness constraint states that those neighboring psess the same motion. In other words, it relies on the assumption that the optical flow is constant within the same surface.

This supposition can be violated in planar surfaces and arbitrary translational motion. However, the most serious (aous) problems with its enforcement occurs at motion discontinuities, i.e., shear, occlusion, disocclusion, and nonrigidicity.ly speaking, it can be forced by considering an overdetermined system of equations and/or by means of regularization. Unfemploying this constraint at motion boundaries may result in an over-smoothed DVF, that is, the resulting motion field is This effect shows up as a smooth "blending" of the object(s) into the background. Sharp discontinuities between surfaces pear completely due to oversmoothing.

Temporal Coherence or Temporal ConstraintAccording to the temporal continuity restriction, the motion of a surface patch in a scene changes gradually over time. It rstability and persistence of the image. Techniques that use this constraint are known as spatio-temporal approaches. Tempoence can be disrespected by camera vibrations and bounces, unpredictable change of course of a target or sudden stop of ascene. In applications such as robot navigation, this constraint will be most likely violated. A common source of temporal dity is the abrupt appearance/disappearance of a region. Thus adaptability to those changes must be part of any procedure detemporal continuity. Strict enforcement of this feature can lead to mistakes because the system continues to track a surfacespite the lack of evidence of its presence in the scene and delays the acknowledgement of a new region in the image.

Data Conservation (Coherence) ConstraintThis constraint implies that the image measurements, such as image brightness and gradients, corresponding to small portiimage remain the same, regardless of changes of its location over time. There are several situations that violate this reqAmong them, we can cite sensor noise and uncontrollable changes in illumination derived from shadows and reflections. tions, non-rigidicity and the existence of multiple motions within the same object transgress this assumption as well. The dvation constraint per si is not always able of characterizing the optical flow due to the aperture problem.

3. Ordinary Least SquaresThe pseudo-inverse or ordinary least -squares (OLS) estimate of the update vector is

( )= T -1 TLSu G G G z (7)

which is given by the minimizer of the functional J(u)=z-Gu2 (for more details, see [ 4], [5] and [ 7]). From now on, G will beanalyzed as being an N p matrix in order to make the whole theoretical discussion easier.

SinceG may be very often illconditioned, the solution given by the previous expression will be usually unaccepta

to the noise amplification resulting from the calculation of the inverse matrixGT

G. In other words, the data are erroneous or noisy.Therefore, one cannot expect an exact solution for the previous equation, but rather an approximation according to some p

4. Principal Component Regression (PCR)The main idea behind principal component regression is the principal component analysis of theG data matrix. Several algorithmsare available ([10],[11],[12]). They yield the same result, but differ in memory and computing time required. The classical use the matrixGTG in one way or another. This has dimensions p p.

4.1 Principal Component Analysis (PCA)PCA is a useful method to solve problems including exploratory data analysis, classification, variable decorrelation prior of neural networks, pattern recognition, data compression, and noise reduction, for example. The formulation of PCA Gaussian latent variable model and can easily lead to Bayesian models.

This technique is used whenever uncorrelated linear combinations of variables are wanted which reduces the dimof a set of variables by reconstructing them into uncorrelated combinations. It combines the variables that account for the l

of the variance to form the first PC. The second PC accounts for the next largest amount of variance, and so on, until thesample set variance is combined into progressively smaller uncorrelated component categories. Each successive componen portions of the variance in the total sample. All of the components are uncorrelated with each other. PCA relates to the sectical moment of G, which is proportional toGTG and it partitionsG into two matricesT andP. They are sometimes called scoresand loadings respectively, such that:

= TG TP . (8 )MatrixT contains the eigenvectors of GTG ordered by their eigenvalues with the largest first and in descending order. If P

has the same rank asG, i.e.,P contains the eigenvectors to all nonzero eigenvalues, thenT =GPis a rotation of G. The first columnof P, p1, gives the direction that minimizes the orthogonal distances from the samples to their projection onto this vector. Ththat the first column of T represents the largest possible sum of squares as compared to any other direction in N. It is customary tocenter the variables in matrixG prior to using PCA. This makesGTG proportional to the variance-covariance matrix. The first prin-cipal axis is then the direction in which the data have the largest spread.T andP can be found by means of singular value decompo-

sition. The number of components can be chosen via examination of the eigenvalues or, for instance, considering the resifrom cross-validation ([5].[12]). Due to the nature of our stated motion estimation problem, we will keep the PCs and us


5/8

group displacement vectors inside a neighborhood. The resulting clusters give an idea about the mixture of motion vectorsmask.

4.2 Principal Component RegressionAs its name implies, this method is closely related to principal components analysis.In essence, the method is just multiple linear regression of PCA scores onz using a suitable number of principal components. Theformal solution may be written as:

( )= T -1 TPCR u P T T T z (9)

In principal components regression (PCR) the matrix inverse is stabilized in an altogether different way than in RLsion. The scores vectors (columns inT) of different components are orthogonal. PCR uses a truncated inverse where only the scocorresponding to large eigenvalues are included. The main drawback of PCR is that the largest variation inG might not correlatewithzand therefore the method may require the use of more a more complex model.

Some nice properties of the PCR are:

1) Using a complete set of PCs, PCR will produce the same results as the original OLS, but with possibly more acthe originalGTGmatrix has inversion problems.2) If GTG is nearly singular, a solution better than the one given by the OLS can be obtained by means of a reducedPCs due to the calculated variances . If GTG is singular, the vector associated with the zero root may point towards re-moving one or more of the original variables.

3) Since the PCs are uncorrelated, straightforward significance tests may be employed that do not need be concerthe order in which the PCs were entered into the regression model. The regression coefficients will be uncorrelatedamounts explained by each PC are independent and hence additive so that the results may be reported in the form of ananalysis of variance.4) If the PCs can be easily interpreted, the resultant regression equations may be more meaningful.

The criteria for deciding when PCR estimators are superior to the OLS estimators depend on the values of the trusion coefficients in the model.

5. Proposed Strategy and ExperimentsThe division of objects into groups can be posed as a mathematical problem consisting of finding the boundaries betgroups. This discrimination between objects can be highly nonlinear. An object is, or is not, a member of a class, depewhich side of the class boundary it lies.

Linear discriminant analysis is limited to situations where a sample belongs to exactly one of m classes. Someti

problem is such that a sample may belong to more than one class at the same time, or not belong to any class. In this meclass is modelled by a multivariate normal in the score space from PCA. Two measures are used to determine whether a slongs to a specific class or not. One is the leveragethe Mahalanobis distance to the center of the class, the class boundcomputable as an ellipse (please, see Fig. 3). The other is the norm of the residual, which must be lower than a critical valuIn Fig. 1, a set of observations is plotted with respect to the first two PCs. One can easily apprehend that there is a strong of four distinct groups on which convex hulls and ellipses have been drawn around the four suspected groups. It is likely thclusters shown correspond to four different types of displacement vectors. For a big neighborhood, it could happen that thewould not be readily distinguished using only one variable at a time, but the plot with respect to the two PCs clearly distingfour populations.

PCR provides additional information about the data being analyzed. The eigenvalues of the correlation matrix of pvariables play an important role in detecting multicollinearity and in analyzing its effects. The PCR estimates are biased, bmore accurate than OLS estimates in terms of mean square error. It is impossible to evaluate the gain in accuracy for a spelem since a comparison of the two methods to OLS since for real video sequence, knowledge of the true values of the coerequired. Nevertheless, when severe multicollinearity is suspected, it is recommended that at least one set of estimates in a

the OLS estimates be computed since these estimates may help interpreting the data in a different way. There is no strong justification for using principal components regression. We recommend that the methods be used in the presence of sevcollinearity as a visual diagnostic tool for judging the suitability of the data for least squares analysis.


6/8


7/8

(a)

(b) (c)

Figure 4. Displacement field for the Rubik Cube sequence. (a) Frame of Rubik Cube Sequence; (b) Corresponding displvector field for a 3131 mask obtained by means of PCR with SNR=20 dB; and (c) PCR with discriminant analysis, 3131

edge information with SNR=20 dB.

6. AcknowledgementsThe authors are thankful to Fundao de Amparo Pesquisa do Estado do Rio de Janeiro (FAPERJ) for the financial sceived.

7. ConclusionsIn this paper, we extended the PCR framework to the detection of motion fields. The algorithm developed combines regrePCA which is primarily a data-analytic method that obtains linear transformations of a group of correlated variables such thaoptimal conditions are achieved. The resulting transformed variables are uncorrelated. Unlike other works ([8],[11],[12]not interested in reducing the dimensionality of the feature space describing the different behaviors inside a neighborhooding a pixel. Instead, we use them in order to validate motion estimates. This can be seen as a simple alternative way of demixtures of motion displacement vectors.

We need to test the proposed algorithm with different noisy images and look closely at the discriminant analysis performance. It is also necessary to incorporate more statistical information in our model and analyze if it will improvcome.

References1. Biemond, J., Looijenga, L., Boekee, D. E., Plompen, R. H. J. M., A Pel-Recursive Wiener-Based Displacement Estimat

rithm, Signal Proc., 13, December (1987), 399-412.2. Blekas, K., Likas, A., Galatsanos, N.P., Lagaris. I.E., A Spatially-Constrained Mixture Model for Image Segmentatio

Trans. on Neural Networks,vol. 16, (2005) 494-498.3. Chattefuee, S., . Hadi, A.S., Regression Analysis by Example, John Wiley & Sons, Inc., Hoboken, New Jersey, (2006)4. Estrela, V.V., da Silva Bassani, M.H., Expectation-Maximization Technique and Spatial Adaptation Applied to Pel-R

Motion Estimation, Proc. of the SCI 2004, Orlando, Florida, USA, July (2004), 204-209.

5. Estrela, V.V., Galatsanos, N.P., Spatially-Adaptive Regularized Pel-Recursive Motion Estimation Based on Cross-VaProc. of ICIP-98 (IEEE International Conference on Image Processing), Vol. III, Chicago, IL, USA, October (1998), 20


8/8

a principal component regression strategy for estimating motion eng opt 2008

Documents