1

Kernel Methods on Riemannian Manifolds withGaussian RBF Kernels

Sadeep Jayasumana, Student Member, IEEE, Richard Hartley, Fellow, IEEE,Mathieu Salzmann, Member, IEEE, Hongdong Li, Member, IEEE, and Mehrtash Harandi, Member, IEEE

Abstract—In this paper, we develop an approach to exploiting kernel methods with manifold-valued data. In many computer visionproblems, the data can be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry ofRiemannian manifolds, usual Euclidean computer vision and machine learning algorithms yield inferior results on such data. In thispaper, we define Gaussian radial basis function (RBF)-based positive definite kernels on manifolds that permit us to embed a givenmanifold with a corresponding metric in a high dimensional reproducing kernel Hilbert space. These kernels make it possible to utilizealgorithms developed for linear spaces on nonlinear manifold-valued data. Since the Gaussian RBF defined with any given metric is notalways positive definite, we present a unified framework for analyzing the positive definiteness of the Gaussian RBF on a generic metricspace. We then use the proposed framework to identify positive definite kernels on two specific manifolds commonly encountered incomputer vision: the Riemannian manifold of symmetric positive definite matrices and the Grassmann manifold, i.e., the Riemannianmanifold of linear subspaces of a Euclidean space. We show that many popular algorithms designed for Euclidean spaces, such assupport vector machines, discriminant analysis and principal component analysis can be generalized to Riemannian manifolds with thehelp of such positive definite Gaussian kernels.

Index Terms—Riemannian manifolds, Gaussian RBF kernels, Kernel methods, Positive definite kernels, Symmetric positive definitematrices, Grassmann manifolds.

F

1 INTRODUCTION

MATHEMATICAL entities that do not form Euclideanspaces but lie on nonlinear manifolds are often

encountered in computer vision. Examples include 3Drotation matrices that form the Lie group SO(3), nor-malized histograms that form the unit n-sphere Sn,symmetric positive definite (SPD) matrices and linearsubspaces of a Euclidean space. Recently, the lattertwo manifolds have drawn significant attention in thecomputer vision community due to their widespreadapplications. SPD matrices, which form a Riemannianmanifold when endowed with an appropriate metric [1],are encountered in computer vision in the forms of co-variance region descriptors [2], [3], diffusion tensors [1],[4] and structure tensors [5], [6]. Linear subspaces of aEuclidean space, known to form a Riemannian manifoldnamed the Grassmann manifold, are commonly used tomodel image sets [7], [8] and videos [9].

Since manifolds lack a vector space structure andother Euclidean structures such as norm and innerproduct, many popular computer vision and machinelearning algorithms including support vector machines(SVM), principal component analysis (PCA) and mean-shift clustering cannot be applied in their original formson manifolds. One way of dealing with this difficultyis to neglect the nonlinear geometry of manifold-valueddata and apply Euclidean methods directly. As intuitionsuggests, this approach often yields poor accuracy andundesirable effects, see, e.g., [1], [10] for SPD matrices.

• The authors are with the College of Engineering and Computer Science,Australian National University, Canberra and NICTA, Canberra.Contact E-mail: sadeep.jayasumana@anu.edu.au

• NICTA is funded by the Australian Government as represented by theDepartment of Broadband, Communications and the Digital Economyand the Australian Research Council (ARC) through the ICT Centre ofExcellence program.

When the manifold under consideration is Rieman-nian, another common approach used to cope with itsnonlinearity consists in approximating the manifold-valued data with its projection to a tangent space at aparticular point on the manifold, for example, the meanof the data. Such tangent space approximations are oftencalculated successively as the algorithm proceeds [3].However, mapping data to a tangent space only yieldsa first-order approximation of the data that can be dis-torted, especially in regions far from the origin of thetangent space. Moreover, iteratively mapping back andforth to the tangent spaces significantly increases thecomputational cost of the algorithm. It is also difficultto choose the origin of the tangent space, which heavilyaffects the accuracy of this approximation.

In Euclidean spaces, the success of many computervision algorithms arises from the use of kernel meth-ods [11], [12]. Therefore, one could think of followingthe idea of kernel methods in Rn and embed a manifoldin a high dimensional Reproducing Kernel Hilbert Space(RKHS), where linear geometry applies. Many Euclideanalgorithms can be directly generalized to an RKHS,which is a vector space that possesses an importantstructure: the inner product. Such an embedding, how-ever, requires a kernel function defined on the manifold,which, according to Mercer’s theorem [12], should bepositive definite. While many positive definite kernelsare known for Euclidean spaces, such knowledge re-mains limited for manifold-valued data.

The Gaussian radial basis function (RBF) kernelexp(−γ‖x − y‖2) is perhaps the most popular and ver-satile kernel in Euclidean spaces. It would thereforeseem attractive to generalize this kernel to manifolds byreplacing the Euclidean distance in the RBF by a moreaccurate nonlinear distance measure on the manifold,

arX

iv:1

412.

0265

v2 [

cs.C

V]

17

Mar

201

5

2

such as the geodesic distance. However, a kernel formedin this manner is not positive definite in general.

In this paper, we aim to generalize the successful andpowerful kernel methods to manifold-valued data. Tothis end, we analyze the Gaussian RBF kernel on ageneric manifold and provide necessary and sufficientconditions for the Gaussian RBF kernel generated bya distance function on any nonlinear manifold to bepositive definite. This lets us generalize kernel methodsto manifold-valued data while preserving the favorableproperties of the original algorithms.

We apply our theory to analyze the positive definite-ness of Gaussian kernels defined on two specific man-ifolds: the Riemannian manifold of SPD matrices andthe Grassmann manifold. Given the resulting positivedefinite kernels, we discuss different kernel methods onthese two manifolds including, kernel SVM, multiplekernel learning (MKL) and kernel PCA. Our experimentson a variety of computer vision tasks, such as pedes-trian detection, segmentation, face recognition and actionrecognition, show that our manifold kernel methodsoutperform the corresponding Euclidean algorithms thatneglect the manifold geometry, as well as other state-of-the-art techniques specifically designed for manifolds.

2 RELATED WORK

In this paper, we focus on the Riemannian manifoldof SPD matrices and on the Grassmann manifold. SPDmatrices find a variety of applications in computer vi-sion [13]. For instance, covariance region descriptors areused in object detection [3], texture classification [2], [14],object tracking, action recognition and human recogni-tion [15], [16]. Diffusion Tensor Imaging (DTI) was one ofthe pioneering fields for the development of non-linearalgorithms on SPD matrices [1], [10]. In optical flowestimation and motion segmentation, structure tensorsare often employed to encode important image features,such as texture and motion [5], [6]. Structure tensorshave also been used in single image segmentation [17].

Grassmann manifolds are widely used to encode im-age sets and videos for face recognition [7], [8], activ-ity recognition [9], [18], and motion grouping [18]. Inimage set based face recognition, a set of face imagesof the same person is represented as a linear subspace,hence a point on a Grassmann manifold. In activityrecognition and motion grouping, a subspace is formedeither directly from the sequence of images containinga specific action, or from the parameters of a dynamicmodel obtained from the sequence [9].

In recent years, several optimization algorithms havebeen proposed for Riemannian manifolds. In particular,LogitBoost for classification on Riemannian manifoldswas introduced in [3]. This algorithm has the drawbacksof approximating the manifold by tangent spaces andnot scaling with the number of training samples due tothe heavy use of exponential/logarithmic maps to transitbetween the manifold and the tangent space, as wellas of gradient descent based Karcher mean calculation.

Here, our positive definite kernels enable us to usemore efficient and accurate classification algorithms onmanifolds without requiring tangent space approxima-tions. Furthermore, as shown in [19], [20], extendingexisting kernel-free, manifold-based binary classifiers tothe multi-class case is not straightforward. In contrast,the kernel-based classifiers on manifolds described inthis paper can readily be used in multi-class scenarios.

In [5], dimensionality reduction and clustering meth-ods were extended to manifolds by designing Rieman-nian versions of Laplacian Eigenmaps (LE), Locally Lin-ear Embedding (LLE) and Hessian LLE (HLLE). Cluster-ing was performed after mapping to a low dimensionalspace which does not necessarily preserve all the infor-mation in the original data. Instead, we use our kernelsto perform clustering in a higher dimensional RKHS thatembeds the manifold of interest.

The use of kernels on SPD matrices has previouslybeen advocated for locality preserving projections [21]and sparse coding [15]. In the first case, the kernel,derived from the affine-invariant distance, is not positivedefinite in general [21]. In the second case, the kernel ispositive definite only for some values of the Gaussianbandwidth parameter γ [15]. For all kernel methods,the optimal choice of γ largely depends on the datadistribution and hence constraints on γ are not desir-able. Moreover, many popular automatic model selectionmethods require γ to be continuously variable [22].

In [7], [23], the Projection kernel and its extensionswere introduced and employed for classification onGrassmann manifolds. While those kernels are analo-gous to the linear kernel in Euclidean spaces, our kernelsare analogous to the Gaussian RBF kernel. In Euclideanspaces, the Gaussian kernel has proven more powerfuland versatile than the linear kernel. As shown in ourexperiments, this also holds for kernels on manifolds.

Recently, mean-shift clustering with the heat kernelon Riemannian manifolds was introduced [24]. How-ever, due to the mathematical complexity of the kernelfunction, computing the exact kernel is not tractable andhence only an approximation of the true kernel wasused. Parallel to our work, kernels on SPD matricesand on Grassmann manifolds were used in [25], albeitwithout explicit proof of their positive definiteness. Incontrast, in this paper, we introduce a unified frameworkfor analyzing the positive definiteness of the Gaussiankernel defined on any manifold and use this frameworkto identify provably positive definite kernels on the man-ifold of SPD matrices and on the Grassmann manifold.

Other than for satisfying Mercer’s theorem to generatea valid RKHS, positive definiteness of the kernel is arequired condition for the convergence of many ker-nel based algorithms. For instance, the Support VectorMachine (SVM) learning problem is convex only whenthe kernel is positive definite [26]. Similarly, positivedefiniteness of all participating kernels is required toguarantee the convexity in Multiple Kernel Learning(MKL) [27]. Although theories have been proposed to

3

exploit non-positive definite kernels [28], [29], they havenot experienced widespread success. Many of thesemethods first enforce positive definiteness of the kernelmatrix by flipping or shifting its negative eigenval-ues [29]. As a consequence, they result in a distortionof information and become inapplicable with large sizekernels, which are not uncommon in learning problems.

It is important to note the difference between this workand manifold-learning methods such as [30]. We workwith data sampled from a manifold whose geometryis well known. In contrast, manifold-learning methodsattempt to learn the structure of an underlying un-known manifold from data samples. Furthermore, thosemethods often assume that noise-free data samples lieon a manifold from which noise push them away. Inour study, data points, regardless of their noise content,always lie on the mathematically well-defined manifold.

3 MANIFOLDS IN COMPUTER VISION

A topological manifold, generally referred to as simply amanifold, is a topological space that is locally homeomor-phic to the n-dimensional Euclidean space Rn, for somen. Here, n is referred to as the dimensionality of themanifold. A differentiable manifold is a topological man-ifold that has a globally defined differential structure.The tangent space at a given point on a differentiablemanifold is a vector space that consists of the tangentvectors of all possible curves passing through the point.

A Riemannian manifold is a differentiable manifoldequipped with a smoothly varying inner product oneach tangent space. The family of inner products onall tangent spaces is known as the Riemannian metric ofthe manifold. It enables us to define various geometricnotions on the manifold such as the angle between twocurves and the length of a curve. The geodesic distancebetween two points on the manifold is defined as thelength of the shortest curve connecting the two points.Such shortest curves are known as geodesics and areanalogous to straight lines in Rn.

The geodesic distance induced by the Riemannianmetric is the most natural measure of dissimilaritybetween two points lying on a Riemannian manifold.However, in practice, many other nonlinear distances ormetrics which do not necessarily arise from Riemannianmetrics can also be useful for measuring dissimilarity onmanifolds. It is worth noting that the term Riemannianmetric refers to a family of inner products while the termmetric refers to a distance function that satisfies the fourmetric axioms. A nonempty set endowed with a metricis known as a metric space which is a more abstract spacethan a Riemannian manifold.

In the following, we discuss two important Rieman-nian manifolds commonly found in computer vision.

3.1 The Riemannian Manifold of SPD Matrices

A d × d, real Symmetric Positive Definite (SPD) matrixS has the property: xTSx > 0 for all nonzero x ∈ Rd.

The space of d × d SPD matrices, which we denoteby Sym+

d , is clearly not a vector space since an SPDmatrix when multiplied by a negative scalar is no longerSPD. Instead, Sym+

d forms a convex cone in the d2-dimensional Euclidean space.

The geometry of the space Sym+d is best explained

with a Riemannian metric which induces an infinitedistance between an SPD matrix and a non-SPD ma-trix [1], [31]. Therefore, the geodesic distance inducedby such a Riemannian metric is a more accurate dis-tance measure on Sym+

d than the Euclidean distance inthe d2-dimensional Euclidean space it is embedded in.Two popular Riemannian metrics proposed on Sym+

d

are the affine-invariant Riemannian metric [1] and thelog-Euclidean Riemannian metric [10], [31]. They resultin the affine-invariant geodesic distance and the log-Euclidean geodesic distance, respectively. These two dis-tances are so far the most widely used metrics on Sym+

d .Apart from these two geodesic distances, a number of

other metrics have been proposed for Sym+d to capture

its nonlinearity [32]. For a detailed review of metrics onSym+

d , we refer the reader to [33].

3.2 The Grassmann Manifold

A point on the (n, r) Grassmann manifold, where n > r,is an r-dimensional linear subspace of the n-dimensionalEuclidean space. Such a point is generally represented byan n× r matrix Y whose columns store an orthonormalbasis of the subspace. With this representation, the pointon the Grassmann manifold is the subspace spanned bythe columns of Y or span(Y ) which we denote by [Y ].We use Grn to denote the (n, r) Grassmann manifold.

The set of n × r (n > r) matrices with orthonormalcolumns forms a manifold known as the (n, r) Stiefelmanifold. From its embedding in the nr-dimensional Eu-clidean space, the (n, r) Stiefel manifold inherits a canon-ical Riemannian metric [34]. Points on Grn are equivalenceclasses of n × r matrices with orthonormal columns,where two matrices are equivalent if their columns spanthe same r-dimensional subspace. Thus, the orthogonalgroup Or acts via isometries (change of orthogonal basis)on the Stiefel manifold by multiplication on the right,and Grn can be identified as the set of orbits of thisaction. Since this action is both free and proper, Grnforms a manifold, and it is given a Riemannian structureby equipping it with the standard normal Riemannianmetric derived from the metric on the Stiefel manifold.

A geodesic distance on the Grassmann manifold,called the arc length distance, can be derived fromits canonical geometry described above. The arc lengthdistance between two points on the Grassmann manifoldturns out to be the l2 norm of the vector formed bythe principal angles between the two subspaces. Severalother metrics on this manifold can be derived from theprincipal angles. We refer the reader to [34] for moredetails and properties of different Grassmann metrics.

4

4 HILBERT SPACE EMBEDDING OF MANI-FOLDS

An inner product space is a vector space equipped withan inner product. A Hilbert space is an (often infinite-dimensional) inner product space which is completewith respect to the norm induced by the inner product.A Reproducing Kernel Hilbert Space (RKHS) is a specialkind of Hilbert space of functions on some nonempty setX in which all evaluation functionals are bounded andhence continuous [35]. The inner product of an RKHS offunctions on X can be defined by a bivariate function onX × X , known as the reproducing kernel of the RKHS.

Many useful computer vision and machine learningalgorithms developed for Euclidean spaces depend onlyon the notion of inner product, which allows us tomeasure angles and also distances. Therefore, such algo-rithms can be extended to Hilbert spaces without effort.A notable special case arises with RKHSs where the innerproduct of the Hilbert space can be evaluated using akernel function without computing the actual vectors.This concept, known as the kernel trick, is commonlyutilized in machine learning in the following setting:input data in some n-dimensional Euclidean space Rnare mapped to a high dimensional RKHS where somelearning algorithm, which requires only the inner prod-uct, is applied. We never need to calculate actual vectorsin the RKHS since the learning algorithm only requiresthe inner product of the RKHS, which can be calculatedby means of a kernel function defined on Rn × Rn.A variety of algorithms can be used with the kerneltrick, such as support vector machines (SVM), principalcomponent analysis (PCA), Fisher discriminant analysis(FDA), k-means clustering and ridge regression.

Embedding lower dimensional data in a higher di-mensional RKHS is commonly employed with data thatlies in a Euclidean space. The theoretical concepts ofsuch embeddings can directly be extended to manifolds.Points on a manifold M are mapped to elements in ahigh (possibly infinite) dimensional Hilbert space H, asubspace of the space spanned by real-valued functions1

on M. A kernel function k : (M ×M) → R is usedto define the inner product on H, thus making it anRKHS. The technical difficulty in utilizing Hilbert spaceembeddings with manifold-valued data arises from thefact that, according to Mercer’s theorem, the kernel mustbe positive definite to define a valid RKHS. While manypositive definite kernel functions are known for Rn,generalizing them to manifolds is not straightforward.

Identifying such positive definite kernel functions onmanifolds would, however, be greatly beneficial. Indeed,embedding a manifold in an RKHS has two major ad-vantages: First, the mapping transforms the nonlinearmanifold into a (linear) Hilbert space, thus making it

1. We limit the discussion to real Hilbert spaces and real-valuedkernels, since they are the most useful kind in learning algorithms.However, the theory holds for complex Hilbert spaces and complex-valued kernels as well.

possible to utilize algorithms designed for linear spaceswith manifold-valued data. Second, as evidenced by thetheory of kernel methods in Euclidean spaces, it yieldsa much richer high-dimensional representation of theoriginal data, making tasks such as classification easier.

In the following we build a framework that enablesus to define positive definite kernels on manifolds.

5 THEORY OF POSITIVE AND NEGATIVE DEFI-NITE KERNELS

In this section, we present some general results onpositive and negative definite kernels. These results willbe useful for our derivations in later sections. We startwith the definition of real-valued positive and negativedefinite kernels on a set [36]. Note that by the term kernelwe mean a real-valued bivariate function hereafter.

Definition 5.1. Let X be a nonempty set. A kernel f : (X ×X ) → R is called positive definite if it is symmetric (i.e.,f(x, y) = f(y, x) for all x, y ∈ X ) and

m∑i,j=1

cicjf(xi, xj) ≥ 0

for all m ∈ N, {x1, . . . , xm} ⊆ X and {c1, ..., cm} ⊆ R. Thekernel f is called negative definite if it is symmetric and

m∑i,j=1

cicjf(xi, xj) ≤ 0

for all m ∈ N, {x1, . . . , xm} ⊆ X and {c1, ..., cm} ⊆ R with∑mi=1 ci = 0.

It is important to note the additional constraint on∑ci for the negative definite case. Due to this constraint,

some authors refer to this latter kind as conditionallynegative definite. However, in this paper, we stick to themost common terminology used in the literature.

We next present the following theorem which plays acentral role in this paper. It was introduced by Schoen-berg in 1938 [37], well before the theory of ReproducingKernel Hilbert Spaces was established in 1950 [35].

Theorem 5.2. Let X be a nonempty set and f : (X×X )→ Rbe a kernel. The kernel exp(−γ f(x, y)) is positive definite forall γ > 0 if and only if f is negative definite.

Proof: We refer the reader to Theorem 3.2.2 of [36]for a detailed proof of this theorem.

Note that this theorem describes Gaussian RBF–likeexponential kernels that are positive definite for allγ > 0. One might also be interested in exponentialkernels that are positive definite for only some values ofγ. Such kernels, for instance, were exploited in [15]. Toour knowledge, there is no general result characterizingthis kind of kernels. However, the following result canbe obtained from the above theorem.

Theorem 5.3. Let X be a nonempty set and f : (X×X )→ Rbe a kernel. If the kernel exp(−γ f(x, y)) is positive definitefor all γ ∈ (0, δ) for some δ > 0, then it is positive definitefor all γ > 0.

5

Proof: If exp(−γ f(x, y)) is positive definite for allγ ∈ (0, δ), it directly follows from Definition 5.1 that1− exp(−γ f(x, y)) is negative definite for all γ ∈ (0, δ).Therefore, the pointwise limit

limγ→0+

1− exp(−γ f(x, y))

γ= f(x, y)

is also negative definite. Now, since f(x, y) is negativedefinite, it follows from Theorem 5.2 that exp(−γ f(x, y))is positive definite for all γ > 0.

Next, we highlight an interesting property of neg-ative definite kernels. It is well known that a posi-tive definite kernel represents the inner product of anRKHS [35]. Similarly, a negative definite kernel repre-sents the squared norm of a Hilbert space under someconditions stated by the following theorem.

Theorem 5.4. Let X be a nonempty set and f(x, y) : (X ×X ) → R be a negative definite kernel. Then, there exists aHilbert space H and a mapping ψ : X → H such that,

f(x, y) = ‖ψ(x)− ψ(y)‖2 + h(x) + h(y)

where h : X → R is a function which is nonnegative wheneverf is. Furthermore, if f(x, x) = 0 for all x ∈ X , then h = 0.

Proof: The proof for a more general version of thistheorem can be found in Proposition 3.3.2 of [36].

Now, we state and prove a lemma that will be usefulfor the proof of our main theorem.

Lemma 5.5. Let X be a nonempty set, V be an inner productspace, and ψ : X → V be a function. Then, f : (X ×X )→ Rdefined by f(x, y) := ‖ψ(x)− ψ(y)‖2V is negative definite.

Proof: The kernel f is obviously symmetric. Basedon Definition 5.1, we then need to prove that∑mi,j=1 cicjf(xi, xj) ≤ 0 for all m ∈ N, {x1, . . . , xm} ⊆ X

and {c1, ..., cm} ⊆ R with∑mi=1 ci = 0. Now,

m∑i,j=1

cicjf(xi, xj) =

m∑i,j=1

cicj

∥∥∥ψ(xi)− ψ(xj)∥∥∥2V

=

m∑i,j=1

cicj

⟨ψ(xi)− ψ(xj), ψ(xi)− ψ(xj)

⟩V

=

m∑j=1

cj

m∑i=1

ci

⟨ψ(xi), ψ(xi)

⟩V

− 2

m∑i,j=1

cicj

⟨ψ(xi), ψ(xj)

⟩V

+

m∑i=1

ci

m∑j=1

cj

⟨ψ(xj), ψ(xj)

⟩V

= −2

m∑i,j=1

cicj

⟨ψ(xi), ψ(xj)

⟩V

= −2

∥∥∥∥∥m∑i=1

ciψ(xi)

∥∥∥∥∥2

V

≤ 0.

5.1 Test for the Negative Definiteness of a Kernel

In linear algebra, an m × m real symmetric matrixM is called positive semi-definite if cTMc ≥ 0 for allc ∈ Rm and negative semi-definite if cTMc ≤ 0 for allc ∈ Rm. These conditions are respectively equivalentto M having non-negative eigenvalues and non-positiveeigenvalues. Furthermore, a matrix M which has theproperty cTM c ≤ 0 for all c ∈ Rm with

∑ci = 0,

where cis are the components of the vector c, is termedconditionally negative semi-definite.

Positive (resp. negative) definiteness of a given kernel–a bivariate function–is usually tested by evaluating thepositive semi-definiteness (resp. conditionally negativesemi-definiteness) of kernel matrices generated with thekernel2. Although such a test is not always conclusive ifit passes, it is particularly useful to identify non-positivedefinite and non-negative definite kernels. Given a set ofpoints {xi}mi=1 ⊆ X and a kernel k : (X × X ) → R, thekernel matrix K of the given points has entries Kij =k(xi, xj). Any kernel matrix generated by a positive(resp. negative) definite kernel must be positive semi-definite (resp. conditionally negative semi-definite). Asnoted above, it is straightforward to check the positivesemi-definiteness of a kernel matrix K by checking itseigenvalues. However, due to the additional constraintthat

∑ci = 0, checking the conditionally negative semi-

definiteness is not straightforward. We therefore suggestthe following procedure.

Let P = Im − 1m1m1Tm, where Im is the m × m

identity matrix and 1m is the m-vector of ones. Anygiven c ∈ Rm with

∑ci = 0 can be written as c = Pc

for some c ∈ Rm. Therefore, the condition cTM c ≤ 0is equivalent to cTPMPc ≤ 0. Hence, we concludethat an m×m matrix M is conditionally negative semi-definite if and only if PMP is negative semi-definite (i.e.,has non-positive eigenvalues). This gives a convenienttest for the conditionally negative semi-definiteness of amatrix, which in turn is useful to evaluate the negativedefiniteness of a given kernel.

6 KERNELS ON MANIFOLDS

A number of well-known kernels exist for Rn includingthe linear kernel, polynomial kernels and the GaussianRBF kernel. The key challenge in generalizing kernelmethods from Euclidean spaces to manifolds lies indefining appropriate positive definite kernels on themanifold. There is no straightforward way to generalizeEuclidean kernels such as the linear kernel and polyno-mial kernels to nonlinear manifolds, since these kernelsdepend on the linear geometry of Rn. However, we showthat the popular Gaussian RBF kernel can be generalizedto manifolds under certain conditions.

2. There is an unfortunate confusion of terminology here. A matrixgenerated by a positive definite kernel is positive semi-definite while amatrix generated by a negative definite kernel is conditionally negativesemi-definite.

6

In this section, we first introduce a general theoremthat provides necessary and sufficient conditions to de-fine a positive definite Gaussian RBF kernel on a givenmanifold and then show that some popular metrics onSym+

d and Grn yield positive definite Gaussian RBFs onthe respective manifolds.

6.1 The Gaussian RBF Kernel on Metric SpacesThe Gaussian RBF kernel has proven very effective inEuclidean spaces for a variety of kernel-based algo-rithms. It maps the data points to an infinite dimen-sional Hilbert space, which, intuitively, yields a very richrepresentation of the data. In Rn, the Gaussian kernelcan be expressed as kG(x,y) := exp(−γ‖x−y‖2), whichmakes use of the Euclidean distance between two datapoints x and y. To define a kernel on a manifold, wewould like to replace the Euclidean distance by a moreaccurate distance measure on the manifold. However,not all geodesic distances yield positive definite kernels.For example, in the case of the unit n-sphere embeddedin Rn+1, exp(−γ d2g(x, y)), where dg is the usual geodesicdistance on the manifold, is not positive definite.

We now state our main theorem which provides thenecessary and sufficient conditions to obtain a positivedefinite Gaussian kernel from a given distance functiondefined on a generic space.

Theorem 6.1. Let (M,d) be a metric space and define k :(M ×M)→ R by k(x, y) := exp(−γ d2(x, y)). Then, k is apositive definite kernel for all γ > 0 if and only if there existsan inner product space V and a function ψ : M → V suchthat d(x, y) = ‖ψ(x)− ψ(y)‖V .

Proof: We first note that positive definiteness ofk(., .) for all γ and negative definiteness of d2(., .) areequivalent conditions according to Theorem 5.2.

To prove the forward direction of the present theorem,let us first assume that V and ψ exist such that d(x, y) =‖ψ(x)− ψ(y)‖V . Then, from Lemma 5.5, d2 is negativedefinite and hence k is positive definite for all γ.

On the other hand, if k is positive definite for all γ,then d2 is negative definite. Furthermore, d(x, x) = 0 forall x ∈ M since d is a metric. Following Theorem 5.4,then V and ψ exist such that d(x, y) = ‖ψ(x)− ψ(y)‖V .

6.2 Geodesic Distances and the Gaussian RBFA Riemannian manifold, when considered with itsgeodesic distance, forms a metric space. Given The-orem 6.1, it is then natural to wonder under whichconditions would a geodesic distance on a manifoldyield a Gaussian RBF kernel. We now present and provethe following theorem, which answers this question forcomplete Riemannian manifolds.

Theorem 6.2. Let M be a complete Riemannian manifoldand dg be the geodesic distance induced by its Riemannianmetric. The Gaussian RBF kernel kg : (M ×M) → R :kg(x, y) := exp(−γ d2g(x, y)) is positive definite for all γ > 0

if and only if M is isometric (in the Riemannian sense) tosome Euclidean space Rn.

Proof: If M is isometric to some Rn, the geodesicdistance on M is simply the Euclidean distance in Rn,which can be trivially shown to yield a positive definiteGaussian RBF kernel by setting ψ in Theorem 6.1 to theidentity function.

On the other hand, if kg is positive definite, fromTheorem 6.1, there exists an inner product space Vg anda function ψg : M → Vg such that dg(x, y) = ‖ψg(x) −ψg(y)‖Vg . Let Hg be the completion of Vg . Therefore, Hgis a Hilbert space, in which Vg is dense.

Now, take any two points x0, x1 in M. Since themanifold is complete, from the Hopf-Rinow theorem,there exists a geodesic δ(t) joining them with δ(0) = x0and δ(1) = x1, and realizing the geodesic distance. Bydefinition, δ(t) has a constant speed dg(x0, x1). Therefore,for all xt = δ(t) where t ∈ [0, 1], the following equalityholds

dg(x0, xt) + dg(xt, x1) = dg(x0, x1).

This must also be true for images ψ(xt) in Hg fort ∈ [0, 1]. However, since Hg is a Hilbert space, this isonly possible if all the points ψ(xt) lie on a straight linein Hg . Let ψ(M) be the range of ψ. From the previousargument, for any two points in ψ(M) ⊆ Hg , the straightline segment joining them is also in ψ(M). Therefore,ψ(M) is a convex set in Hg . Now, since M is complete,any geodesic must be extensible indefinitely. Therefore,the corresponding line segment in ψ(M) must also beextensible indefinitely. This proves that ψ(M) is an affinesubspace of Hg , which is isometric to Rn, for some n.Since M is isometric to ψ(M), this proves that M isisometric to the Euclidean space Rn.

According to Theorem 6.2, it is possible to obtaina positive definite Gaussian kernel from the geodesicdistance on a Riemannian manifold only when the man-ifold is made essentially equivalent to some Rn by theRiemannian metric that defines the geodesic distance.Although this is possible for some Riemannian mani-folds, such as the Riemannian manifold of SPD matrices,for some others, it is theoretically impossible.

In particular, if the manifold is compact, it is impossi-ble to find an isometry between the manifold and Rn,since Rn is not compact. Therefore, it is not possibleto obtain a positive definite Gaussian from the geodesicdistance of a compact manifold. In such cases, the besthope is to find a different non-geodesic distance on themanifold that does not differ much from the geodesicdistance, but still satisfies the conditions of Theorem 6.1.

6.3 Kernels on Sym+d

We now discuss different metrics on Sym+d that can

be used to define positive definite Gaussian kernels.Since Sym+

d is not compact, as explained in the previoussection, there is some hope in finding a geodesic distanceon it that also defines a positive definite Gaussian kernel.

7

In this section we show that the log-Euclidean distance,which has been proved to be a geodesic distance onSym+

d [31], is such a distance.In the log-Euclidean framework, a geodesic connecting

S1, S2 ∈ Sym+d is defined as γ(t) = exp((1 − t) log(S1) +

t log(S2)) for t ∈ [0, 1]. The log-Euclidean geodesic dis-tance between S1 and S2 can be expressed as

dLE(S1, S2) = ‖ log(S1)− log(S2)‖F , (1)

where ‖ ·‖F denotes the Frobenius matrix norm inducedby the Frobenius matrix inner product 〈., .〉F .

The log-Euclidean distance has proven an effectivedistance measure on Sym+

d [10], [33]. Furthermore, ityields a positive definite Gaussian kernel as stated inthe following corollary to Theorem 6.1:

Corollary 6.3 (Theorem 6.1). The Log-Euclidean Gaus-sian kernel kLE : (Sym+

d ×Sym+d ) → R :

kLE(S1, S2) := exp(−γ d2LE(S1, S2)), where dLE(S1, S2) isthe log-Euclidean distance between S1 and S2, is a positivedefinite kernel for all γ > 0.

Proof: Directly follows from Theorem 6.1 with theFrobenius matrix inner product.

A number of other metrics have been proposed forSym+

d [33]. The definitions and properties of these met-rics are summarized in Table 1. Note that only someof them were derived by considering the Riemanniangeometry of the manifold and hence define true geodesicdistances. Similarly to the log-Euclidean metric, fromTheorem 6.1, it directly follows that the Cholesky andpower-Euclidean metrics also define positive definiteGaussian kernels for all values of γ. Note that somemetrics may yield a positive definite Gaussian kernelfor some value of γ only. This, for instance, was shownin [32] for the root Stein divergence metric. No such re-sult is known for the affine-invariant metric. Constraintson γ are nonetheless undesirable, since one should beable to freely tune γ to reflect the data distribution, andautomatic model selection algorithms require kernels tobe positive definite for continuous values of γ > 0 [22].

6.4 Kernels on GrnSimilarly to Sym+

d , different metrics can be defined onGrn. Many of these metrics are related to the principalangles between two subspaces. Given two n × r matri-ces Y1 and Y2 with orthonormal columns, representingtwo points on Grn, the principal angles between thecorresponding subspaces are obtained from the singularvalue decomposition of Y T1 Y2 [34]. More specifically, ifUSV T is the singular value decomposition of Y T1 Y2, thenthe entries of the diagonal matrix S are the cosines ofthe principal angles between [Y1] and [Y2]. Let {θi}ri=1

represent those principal angles. Then, the geodesicdistance derived from the canonical geometry of theGrassmann manifold, called the arc length, is given by(∑i θ

2i )

1/2 [34]. Unfortunately, as can be shown witha counter-example, this distance, when squared, is not

negative definite and hence does not yield a positivedefinite Gaussian for all γ > 0. Given Theorem 6.2 andthe discussion that followed, this not a surprising result:Since the Grassmann manifold is a compact manifold, itis not possible to find a geodesic distance that also yieldsa positive definite Gaussian for all γ > 0.

Nevertheless, there exists another widely used met-ric on the Grassmann manifold, namely the projectionmetric, which gives rise to a positive definite Gaussiankernel. The projection distance between two subspaces[Y1], [Y2] is given by

dP ([Y1], [Y2]) = 2−1/2‖Y1Y T1 − Y2Y T2 ‖F . (2)

We now formally introduce this Gaussian RBF kernelon the Grassmann manifold.

Corollary 6.4 (Theorem 6.1). The Projection Gaussiankernel kP : (Grn × Grn) → R : kP ([Y1], [Y2]) :=exp(−γ d2P ([Y1], [Y2])), where dP ([Y1], [Y2]) is the projectiondistance between [Y1] and [Y2], is a positive definite kernel forall γ > 0.

Proof: Follows from Theorem 6.1 with the Frobeniusmatrix inner product.

As shown in Table 2, none of the other popular metricson Grassmann manifolds have this property. Counter-examples exist for these metrics to support our claims.

6.4.1 Calculation of the Projection Gaussian KernelTo calculate the kernel introduced in Corollary 6.4,one needs to calculate the squared projection dis-tance given by d2P ([Y1], [Y2]) = 2−1‖Y1Y T1 − Y2Y T2 ‖

2

F ord2P ([Y1], [Y2]) = (

∑i sin2 θi)

1/2 where both Y1 and Y2 aren × r matrices. The second formula requires a singularvalue decomposition to find the θis, which we would liketo avoid due to its computational complexity. The firstformula requires calculating the Frobenius l2 distancebetween Y1Y

T1 and Y2Y

T2 , both of which are n × n

matrices. For many applications, n is quite large, andtherefore, the direct computation of ‖Y1Y T1 − Y2Y

T2 ‖2F

is inefficient. As a consequence, the cost of computingd2P can be reduced with the following equation, whichmakes use of some properties of the matrix trace and ofthe fact that Y T1 Y1 = Y T2 Y2 = Ir:

d2P ([Y1], [Y2]) = 2−1‖Y1Y T1 − Y2Y T2 ‖2

F = r − ‖Y T1 Y2‖2F .

This implies that it is sufficient to compute only theFrobenius norm of Y T1 Y2, an r × r matrix, to calculated2P ([Y1], [Y2]).

7 KERNEL-BASED ALGORITHMS ON MANI-FOLDS

A major advantage of being able to compute positivedefinite kernels on manifolds is that it directly allowsus to make use of algorithms developed for Rn, whilestill accounting for the geometry of the manifold. Inthis section, we discuss the use of five kernel-based

8

Metric Name Formula Geodesic Distance Positive Definite GaussianKernel for all γ > 0

Log-Euclidean ‖ log(S1)− log(S2)‖F Yes Yes

Affine-Invariant ‖ log(S−1/21 S2S

−1/21 )‖

FYes No

Cholesky ‖ chol(S1)− chol(S2)‖F No Yes

Power-Euclidean 1α‖Sα1 − Sα2 ‖F No Yes

Root Stein Divergence[log det

(12S1 + 1

2S2

)− 1

2log det(S1S2)

]1/2 No No

TABLE 1: Properties of different metrics on Sym+d . We analyze the positive definiteness of Gaussian kernels generated by

different metrics. Theorem 6.1 applies to the metrics claimed to generate positive definite Gaussian kernels. For the other metrics,examples of non-positive definite Gaussian kernels exist.

Metric Name Formula Geodesic Distance Positive Definite GaussianKernel for all γ > 0

Projection 2−1/2‖Y1Y T1 − Y2Y T2 ‖F = (∑i sin

2 θi)1/2 No Yes

Arc length (∑i θ

2i )

1/2 Yes No

Fubini-Study arccos | det(Y T1 Y2)| = arccos(∏i cos θi) No No

Chordal 2-norm ‖Y1U − Y2V ‖2 = 2maxi sin12θi No No

Chordal F-norm ‖Y1U − Y2V ‖F = 2(∑i sin

2 12θi)

1/2 No No

TABLE 2: Properties of different metrics on Grn. Here, USV T is the singular value decomposition of Y T1 Y2, whereas θis arethe the principal angles between the two subspaces [Y1] and [Y2]. Our claims on positive definiteness are supported by eitherTheorem 6.1, or counter-examples.

algorithms on manifolds. The resulting algorithms canbe thought of as generalizations of the original Euclideankernel methods to manifolds. In the following, we useM, k(., .), H and φ(x) to denote a manifold, a positivedefinite kernel defined on M×M, the RKHS generatedby k, and the feature vector in H to which x ∈ M ismapped, respectively. Although we use φ(x) for explana-tion purposes, following the kernel trick, it never needsto be explicitly computed in any of the algorithms.

7.1 Classification on ManifoldsWe first consider the binary classification problem on amanifold. To this end, we propose to extend the popularEuclidean kernel SVM algorithm to manifold-valueddata. Given a set of training examples {(xi, yi)}mi=1,where xi ∈ M and the label yi ∈ {−1, 1}, kernelSVM searches for a hyperplane in H that separates thefeature vectors of the positive and negative classes withmaximum margin. The class of a test point x ∈ M isdetermined by the position of the feature vector φ(x) inH relative to the separating hyperplane. Classificationwith kernel SVM can be done very fast, since it onlyrequires to evaluate the kernel at the support vectors.

The above procedure is equivalent to solving the stan-dard kernel SVM problem with kernel matrix generatedby k. Thus, any existing SVM software package can beutilized for training and classification. Convergence ofstandard SVM optimization algorithms is guaranteed,since k is positive definite.

Kernel SVM on manifolds is much simpler to im-plement and less computationally demanding in bothtraining and testing phases than the current state-of-the-art binary classification algorithms on manifolds,such as LogitBoost on a manifold [3], which involvesiteratively combining weak learners on different tangentspaces. Weighted mean calculation in LogitBoost on a

manifold involves an extremely expensive gradient de-scent procedure at each boosting iteration, which makesthe algorithm scale poorly with the number of trainingsamples. Furthermore, while LogitBoost learns classifierson tangent spaces used as first order Euclidean approx-imations to the manifold, our approach works in a richhigh dimensional feature space. As will be shown in ourexperiments, this yields better classification results.

With manifold-valued data, extending the currentstate-of-the-art binary classification methods to multi-class classification is not straight-forward [19], [20]. Bycontrast, our manifold kernel SVM classification methodcan easily be extended to the multi-class case withstandard one-vs-one or one-vs-all procedures.

7.2 Feature Selection on ManifoldsWe next tackle the problem of combining multi-ple manifold-valued descriptors via a Multiple KernelLearning (MKL) approach. The core idea of MKL is tocombine kernels computed from different descriptors(e.g., image features) to obtain a kernel that optimallyseparates two classes for a given classifier. Here, wefollow the formulation of [27] and make use of anSVM classifier. As a feature selection method, MKL hasproven more effective than conventional techniques suchas wrappers, filters and boosting [38].

More specifically, given training examples {(xi, yi)}m1 ,where xi ∈ X (some nonempty set), yi ∈ {−1, 1}, anda set of descriptor generating functions {gj}N1 wheregj : X → M, we seek to learn a binary classifierf : X → {−1, 1} by selecting and optimally combiningthe different descriptors generated by g1, . . . , gN . LetK(j) be the kernel matrix generated by gj and k asK

(j)pq = k(gj(xp), gj(xq)). The combined kernel can be

expressed as K∗ =∑j λjK

(j), where λj ≥ 0 for all jguarantees the positive definiteness of K∗. The weights

9

λ can be learned using a min-max optimization pro-cedure with an l1 regularizer on λ to obtain a sparsecombination of kernels. The algorithm has two stepsin each iteration: First it solves a conventional SVMproblem with λ, hence K∗, fixed. Then it updates λ withthe SVM parameters fixed. These two steps are repeateduntil convergence. For more details, we refer the readerto [27] and [38]. Note that convergence of MKL is onlyguaranteed if all the kernels are positive definite, whichis satisfied in this setup since k is positive definite.

We also note that MKL on manifolds gives a conve-nient method to combine manifold-valued descriptorswith Euclidean descriptors which is otherwise a difficulttask due to their different geometries.

7.3 Dimensionality Reduction on ManifoldsWe next study the extension of kernel PCA to nonlineardimensionality reduction on manifolds. The usual Eu-clidean kernel PCA has proven successful in many ap-plications [12], [39]. On a manifold, kernel PCA proceedsas follows: All points xi ∈M of a given dataset {xi}mi=1

are mapped to feature vectors in H, thus yielding thetransformed set {φ(xi)}mi=1. The covariance matrix of thistransformed set is then computed, which really amountsto computing the kernel matrix of the original data usingthe function k. An l-dimensional representation of thedata is obtained by computing the eigenvectors of thekernel matrix. This representation can be thought of as aEuclidean representation of the original manifold-valueddata. However, owing to our kernel, it was obtained byaccounting for the geometry of M.

Once the kernel matrix is calculated with k, imple-mentation details of the algorithm are similar to that ofthe Euclidean kernel PCA algorithm. We refer the readerto [39] for more details on the implementation.

7.4 Clustering on ManifoldsFor clustering problems on manifolds, we propose tomake use of kernel k-means. Kernel k-means mapspoints to a high-dimensional Hilbert space and performsk-means clustering in the resulting feature space [39]. Ina manifold setting, a given dataset {xi}mi=1, with eachxi ∈M, is clustered into a pre-defined number of groupsin H, such that the sum of the squared distances fromeach φ(xi) to the nearest cluster center is minimized. Theresulting clusters can then act as classes for the {xi}mi=1.

The unsupervised clustering method on Riemannianmanifolds proposed in [5] clusters points in a low-dimensional space after dimensionality reduction on themanifold. In contrast, our method performs clusteringin a high-dimensional RKHS which, intuitively, betterrepresents the data distribution.

7.5 Discriminant Analysis on ManifoldsThe kernelized version of linear discriminant analysis,known as Kernel Fisher Discriminant Analysis (Kernel

10−5

10−4

10−3

10−2

10−1

0.01

0.02

0.05

0.1

0.2

0.3

False Positives Per Window (FPPW)

Mis

s R

ate

Proposed Method (MKL on Manifold)

MKL with Euclidean Kernel

LogitBoost on Manifold

HOG Kernel SVM

HOG Linear SVM

Fig. 1: Pedestrian detection. Detection-Error tradeoff curvesfor the proposed manifold MKL approach and state-of-the-art methods on the INRIA dataset. Our method outperformsexisting manifold methods and Euclidean kernel methods. Thecurves for the baselines were reproduced from [3].

FDA), can also be extended to manifolds on which a pos-itive definite kernel can be defined. Given {(xi, yi)}mi=1,with each xi ∈M having class label yi, manifold kernelFDA maps each point xi on the manifold to a featurevector in H and finds a new basis in H where the classseparation is maximized. The output of the algorithmis a Euclidean representation of the original manifold-valued data, but with a larger separation between classmeans and a smaller within-class variance. Up to (l− 1)dimensions can be extracted via kernel FDA where lis the number of classes. We refer the reader to [12],[40] for implementation details. In Euclidean spaces,kernel FDA has become an effective pre-processing stepto perform nearest-neighbor classification in the highlydiscriminative, reduced dimensional space.

8 APPLICATIONS AND EXPERIMENTS

We now present two series of experiments on Sym+d

and Grn using the positive definite kernels introduced inSection 6 and the algorithms described in Section 7.

8.1 Experiments on Sym+d

In the following, we use the log-Euclidean Gaussiankernel defined in Corollary 6.3 to apply different kernelmethods to Sym+

d . We compare our kernel methods onSym+

d to other state-of-the-art algorithms on Riemannianmanifolds and to kernel methods on Sym+

d with theusual Euclidean Gaussian kernel that does not accountfor the nonlinearity of the manifold.

8.1.1 Pedestrian DetectionWe first demonstrate the use of our log-EuclideanGaussian kernel for the task of pedestrian detectionwith kernel SVM and MKL on Sym+

d . Let {(Wi, yi)}mi=1

be the training set, where each Wi ∈ Rh×w is an

10

image window and yi ∈ {−1, 1} is the class label(background or person) of Wi. Following [3], we usecovariance descriptors computed from the feature vector[x, y, |Ix|, |Iy|,

√I2x + I2y , |Ixx|, |Iyy|, arctan

(|Ix||Iy|

)],

where x, y are pixel locations and Ix, Iy, . . . are intensityderivatives. The covariance matrix for an image patchof arbitrary size therefore is an 8 × 8 SPD matrix. Inan h × w window W , a large number of covariancedescriptors can be computed from subwindows withdifferent sizes and positions sampled from W . Weconsider N subwindows {wj}Nj=1 of size rangingfrom h/5 × w/5 to h × w, positioned at all possiblelocations. The covariance descriptor of each subwindowis normalized using the covariance descriptor of thefull window to improve robustness against illuminationchanges. Such covariance descriptors can be computedefficiently using integral images [3].

Let X(j)i ∈ Sym+

8 denote the covariance descriptor ofthe jth subwindow of Wi. To reduce this large numberof descriptors, we pick the best 100 subwindows thatdo not mutually overlap by more than 75%, by rankingthem according to their variance across all training sam-ples. The rationale behind this ranking is that a gooddescriptor should have a low variance across all givenpositive detection windows. Since the descriptors lie ona manifold, for each X(j), we compute a variance-likestatistic var(X(j)) across all positive training samples as

var(X(j)) =1

m+

∑i:yi=1

dpg(X(j)i , X(j)), (3)

where m+ is the number of positive training samplesand X is the Karcher mean of {Xi}i:yi=1 given by

X = exp(

1m+

∑i:yi=1 log(Xi)

)under the log-Euclidean

metric. We set p = 1 in Eq.(3) to make the statistic lesssensitive to outliers. We then use the SVM-MKL frame-work described in Section 7.2 to learn the final classifier,where each kernel is defined on one of the 100 selectedsubwindows. At test time, detection is achieved in asliding window manner followed by a non-maximumsuppression step.

To evaluate our approach, we made use of the INRIAperson dataset [41]. Its training set consists of 2,416positive windows and 1,280 person-free negative images,and its test set of 1,237 positive windows and 453negative images. Negative windows are generated bysampling negative images [41]. We first used all posi-tive samples and 12,800 negative samples (10 randomwindows from each negative image) to train an initialclassifier. We used this classifier to find hard negativeexamples in the training images, and re-trained theclassifier by adding these hard examples to the trainingset. Cross validation was used to determine the hyper-parameters including the parameter γ of the kernels. Weused the evaluation methodology of [41].

In Fig. 1, we compare the detection-error tradeoff(DET) curves of our approach and state-of-the-art meth-ods. The curve for our method was generated by contin-

uously varying the decision threshold of the final MKLclassifier. We also evaluated our MKL framework withthe Euclidean Gaussian kernel. Note that the proposedMKL method with our Riemannian kernel outperformsMKL with the Euclidean kernel, as well as LogitBooston the manifold. This demonstrates the importance ofaccounting for the nonlinearity of the manifold using anappropriate positive definite kernel. It is also worth not-ing that LogitBoost on the manifold is significantly morecomplex and harder to implement than our method.

8.1.2 Visual Object CategorizationWe next tackle the problem of unsupervised object cate-gorization. To this end, we used the ETH-80 dataset [42]which contains 8 categories with 10 objects each and 41images per object. We used 21 randomly chosen imagesfrom each object to compute the parameter γ and the restto evaluate clustering accuracy. For each image, we useda single 5 × 5 covariance descriptor calculated from thefeatures [x, y, I , |Ix| , |Iy|], where x, y are pixel locationsand I , Ix, Iy are intensity and derivatives. To obtainobject categories, the kernel k-means algorithm on Sym+

5

described in Section 7.4 was employed.One drawback of k-means and its kernel counterpart

is their sensitivity to initialization. To overcome this, weran each algorithm 20 times with random initializationsand picked the iteration that converged to the minimumsum of point-to-centroid squared distances. For kernel k-means on Sym+

5 , distances in the RKHS were used. Weassumed the number of clusters to be known.

To set a benchmark, we evaluated the performanceof both k-means and kernel k-means on Sym+

5 withdifferent metrics that generate positive definite Gaussiankernels (see Table 1). For the power-Euclidean metric,we used α = 0.5, which, as in the non-kernel methodof [33], we found to yield the best results. For allnon-Euclidean metrics with (non-kernel) k-means, theKarcher mean [33] was used to compute the centroid.The results of the different methods are summarized inTable 3. Manifold kernel k-means with the log-EuclideanGaussian kernel performs significantly better than allother methods in all test cases. These results also out-perform the results with the heat kernel reported in [24].Note, however, that [24] only considered 3 and 4 classeswithout mentioning which classes were used.

8.1.3 Texture RecognitionWe then utilized our log-Euclidean Gaussian kernel todemonstrate the effectiveness of manifold kernel PCAon texture recognition. To this end, we used the Brodatzdataset [43], which consists of 111 different 640 × 640texture images. Each image was divided into four subim-ages of equal size, two of which were used for trainingand the other two for testing.

For each training image, covariance descriptors of 50randomly chosen 128 × 128 windows were computedfrom the feature vector [I, |Ix|, |Iy|, |Ixx| , |Iyy|] [2]. Ker-nel PCA on Sym+

5 with our Riemannian kernel was

11

Nb. of Euclidean Cholesky Power-Euclidean Log-Euclideanclasses KM KKM KM KKM KM KKM KM KKM

3 72.50 79.00 73.17 82.67 71.33 84.33 75.00 94.834 64.88 73.75 69.50 84.62 69.50 83.50 73.00 87.505 54.80 70.30 70.80 82.40 70.20 82.40 74.60 85.906 50.42 69.00 59.83 73.58 59.42 73.17 66.50 74.507 42.57 68.86 50.36 69.79 50.14 69.71 59.64 73.148 40.19 68.00 53.81 69.44 54.62 68.44 58.31 71.44

TABLE 3: Object categorization. Sample images and percentages of correct clustering on the ETH-80 dataset using k-means(KM) and kernel k-means (KKM) with different metrics on Sym+

d . The proposed KKM method with the log-Euclidean metricachieves the best results in all the tests.

KernelClassification Accuracy

l = 10 l = 11 l = 12 l = 15

Log-Euclidean 95.50 95.95 96.40 96.40Euclidean 89.64 90.09 90.99 91.89

TABLE 4: Texture recognition. Recognition accuracies on theBrodatz dataset with k-NN in an l-dimensional Euclideanspace obtained by kernel PCA. The log-Euclidean Gaussiankernel introduced in this paper captures information moreeffectively than the usual Euclidean Gaussian kernel.

then used to extract the top l principal directions in theRKHS, and project the training data along those direc-tions. Given a test image, we computed 100 covariancedescriptors from random windows and projected themto the l principal directions obtained during training.Each such projection was classified using a majority voteover its 5 nearest-neighbors. The class of the test imagewas then decided by majority voting among the 100descriptors. Cross validation on the training set was usedto determine γ. For comparison purposes, we repeatedthe same procedure with the Euclidean Gaussian kernel.Results obtained for these kernels and different valuesof l are presented in Table 4. The better recognitionaccuracy indicates that kernel PCA with the Riemanniankernel more effectively captures the information of themanifold-valued descriptors than the Euclidean kernel.

8.1.4 SegmentationWe now illustrate the use of our kernel to segmentdifferent types of images. First, we consider DTI seg-mentation, which is a key application area of algorithmson Sym+

d . We utilized kernel k-means on Sym+3 with our

Riemannian kernel to segment a real DTI image of thehuman brain. Each pixel of the input DTI image is a 3×3SPD matrix, which can thus directly be used as input tothe algorithm. The k clusters obtained by the algorithmact as classes, thus yielding a segmentation of the image.

Fig. 2 depicts the resulting segmentation along withthe ellipsoid and fractional anisotropy representationsof the original DTI image. We also show the resultsobtained by replacing the Riemannian kernel with theEuclidean one. Note that, up to some noise due to thelack of spatial smoothing, Riemannian kernel k-meanswas able to correctly segment the corpus callosum fromthe rest of the image.

We then followed the same approach to perform 2Dmotion segmentation. To this end, we used a spatio-

Ellipsoids Fractional Anisotropy

Riemannian kernel Euclidean kernel

Fig. 2: DTI segmentation. Segmentation of the corpus callo-sum with kernel k-means on Sym+

3 . The proposed kernel yieldsa cleaner segmentation.

temporal structure tensor directly computed on im-age intensities (i.e., without extracting features such asoptical flow). The spatio-temporal structure tensor foreach pixel is computed as T = K ∗ (∇I∇IT ), where∇I = (Ix, Iy, It) and K∗ indicates convolution withthe regular Gaussian kernel for smoothing purposes.Each pixel is thus represented as a 3 × 3 SPD matrix Tand segmentation can be performed by clustering thesematrices using kernel k-means on Sym+

3 .We applied this strategy to two images taken from the

Hamburg Taxi sequence. Fig. 3 compares the results ofkernel k-means with our log-Euclidean Gaussian kernelwith the results of [5] obtained by first performing LLE,LE, or HLLE on Sym+

3 and then clustering in the low di-mensional space. Note that our approach yields a muchcleaner segmentation than the baselines. This could beattributed to the fact that we perform clustering in a highdimensional feature space, whereas the baselines workin a reduced dimensional space.

8.2 Experiments on GrnWe now present our experimental evaluation of theproposed kernel methods on Grn with the projectionGaussian kernel introduced in Corollary 6.4. We compareour results to those obtained with state-of-the-art classi-fication and clustering methods on Grn, and show that weachieve significantly better classification and clusteringaccuracies in a number of different tasks.

In each of the following experiments, we model animage set with the linear subspace spanned by its princi-pal components. More specifically, let {fi}pi=1, with each

12

Frame 1 Frame 2 KKM on Sym+3

LLE on Sym+3 LE on Sym+

3 HLLE on Sym+3

Fig. 3: 2D motion segmentation. Comparison of the segmenta-tions obtained with kernel k-means with our Riemannian ker-nel (KKM), LLE, LE and HLLE on Sym+

3 . Our KKM algorithmyields a much cleaner segmentation. The baseline results werereproduced from [5].

fi ∈ Rn, be a set of descriptors each representing oneimage in a set of p images. The image set can thenbe represented by the linear subspace spanned by r(r < p, n) principal components of {fi}pi=1. Limiting rhelps reducing noise and other fine variations withinthe image set, which are not useful for classification orclustering. The image set descriptors obtained in thismanner are r-dimensional linear subspaces of the n-dimensional Euclidean space, which lie on the (n, r)Grassmann manifold Grn. We note that the r-principalcomponents of {fi}pi=1 can be efficiently obtained byperforming a Singular Value Decomposition (SVD) onthe n × p matrix F having the fis as columns: If USV T

is the singular value decomposition of F , the columnsof U corresponding to the largest r singular values givethe r principal components of {fi}pi=1.

8.2.1 Video Based Face RecognitionFace recognition from video, which uses an image set foridentification of a person, is a rapidly developing areain computer vision. For this task, the videos are oftenmodeled as linear subspaces, which lie on a Grassmannmanifold [7], [8]. To demonstrate the use of our projec-tion Gaussian kernel in video based face recognition, weused the YouTube Celebrity dataset [44], which contains1910 video clips of 47 different people. Face recognitionon this dataset is challenging since the videos are highlycompressed and most of them have low resolution.

We used the cascaded face detector of [45] to extractface regions from videos and resized them to have acommon size of 96 × 96. Each face image was thenrepresented by a histogram of Local Binary Patterns [46]having 232 equally-spaced bins. We next representedeach image set corresponding to a single video clip bya linear subspace of order 5. We randomly chose 70%of the dataset for training and the remaining 30% fortesting. We report the classification accuracy averagedover 10 different random splits.

We employed both kernel SVM on a manifold andkernel FDA on a manifold with our projection Gaussiankernel. With kernel SVM, a one-vs-all procedure was

MethodFace Recognition

AccuracyAction Recognition

AccuracyDCC [47] 60.21 ± 2.9 41.95 ± 9.6

KAHM [48] 67.49 ± 3.5 70.05 ± 0.9GDA [7] 58.72 ± 3.0 67.33 ± 1.1

GGDA [8] 61.05 ± 2.2 73.54 ± 2.0Linear SVM 64.76 ± 2.1 74.66 ± 1.2

Manifold Kernel FDA 65.32 ± 1.4 76.35 ± 1.0Manifold Kernel SVM 71.78 ± 2.4 76.95 ± 0.9

TABLE 5: Face and action recognition. Recognition accuracieson the YouTube Celebrity and Ballet datasets. Our manifoldkernel SVM method achieves the best results.

used for multiclass classification. For kernel FDA, thetraining data was projected to an (l − 1) dimensionalspace, where l = 47 is the number of classes, andwe used a 1-nearest-neighbor method to predict theclass of a test sample projected to the same space. Wedetermined the hyperparameters of both methods usingcross-validation on the training data.

We compared our approach with several state-of-the-art image set classification methods: Discriminant anal-ysis of Canonical Correlations (DCC) [47], Kernel AffineHull Method (KAHM) [48], Grassmann DiscriminantAnalysis (GDA) [7], and Graph-embedding GrassmannDiscriminant Analysis (GGDA) [8]. As shown in Table 5,manifold kernel SVM achieves the best accuracy. GDAuses the Grassmann projection kernel which correspondsto the linear kernel with FDA in a Euclidean space.Therefore, the results of GDA and Manifold Kernel FDAin Table 5 also provide a nice comparison between thelinear kernel and our projection Gaussian kernel. To ob-tain a similar comparison with SVMs, we also performedone-vs-all SVM classification with the projection kernel,which really amounts to linear SVM in the Projectivespace. This method is denoted by Linear SVM in Table 5.With both FDA and SVM, our Gaussian kernel performsbetter than the projection kernel, thus agreeing with theobservation in Euclidean spaces that the Gaussian kernelperforms better than the linear kernel.

8.2.2 Action RecognitionWe next demonstrate the benefits of our projectionGaussian kernel on action recognition. To this end, weused the Ballet dataset [49], which contains 44 videosof 8 actions performed by 3 different actors. Each videocontains different actions performed by the same actor.Action recognition on this dataset is challenging due tolarge intra-class variations in clothing and movement.

We grouped every 6 subsequent frames containing thesame action, which resulted in 2338 image sets. Eachframe was described by a Histogram of Oriented Gradi-ents (HOG) descriptor [41], and 4 principal componentswere used to represent each image set. The sampleswere randomly split into two equally-sized sets to obtaintraining and test data. We report the average accuracyover 10 different splits.

As in the previous experiment, we used kernel FDAand one-vs-all SVM with the projection Gaussian kernel

13

and compared our methods with DCC, KAHM, GDA,GGDA, and Linear SVM. As evidenced by the resultsin Table 5, Manifold Kernel FDA and Manifold KernelSVM both achieve superior performance compared tothe state-of-the-art algorithms. Out of the two methodsproposed in this paper, Manifold Kernel SVM achievesthe highest accuracy.

8.2.3 Pose Categorization

Finally, we demonstrate the use of our projection Gaus-sian kernel in clustering on the Grassmann manifold.To this end, we used the well-known CMU-PIE facedataset [50], which contains face images of 67 subjectswith 13 different poses and 21 different illuminations.Images were closely cropped to enclose the face regionfollowed by a resizing to 32×32. The vectorized intensityvalues were directly used to describe each image. Weused images of the same subject with the same posebut different illuminations to form an image set, whichwas then represented by a linear subspace of order 6.This resulted in a total of 67 × 13 = 871 Grassmanndescriptors, each lying on G61024.

The goal of the experiment was to cluster togetherimage sets having the same pose. We randomly dividedthe images of the same subject with the same pose intotwo equally sized sets to obtain two collections of 871image sets. The optimum value for γ was determinedwith the first collection and the results are reportedon the second one. We compare our Manifold Kernelk-means (MKKM) with two other algorithms on thesame data. The first algorithm, proposed in [9], is theconventional k-means with the arc length distance onGrn and the corresponding Karcher mean (KM-AL). Thepublicly available code of [9] was used to obtain theresults with KM-AL. Since the Karcher mean with the arclength distance does not have a closed-form solution andhas to be calculated using a gradient descent procedure,KM-AL tends to slow down with the dimensionality ofthe Grassmann descriptors and the number of samples.The second baseline was k-means with the projectionmetric and the corresponding Karcher mean (KM-PM).Although the Karcher mean can be calculated in closed-form in this case, the algorithm becomes slow whenworking with large matrices. In our setup, since theprojection space consisted of symmetric matrices of size1024×1024, projection k-means boils down to performingk-means in a 1024× (1024 + 1)/2 = 524, 800 dimensionalspace.

The results of the three clustering algorithms for dif-ferent numbers of clusters are given in Table 6. As in theexperiment on Sym+

d , each clustering algorithm was run20 times with different random initializations, and theiteration which converged to the minimum energy waspicked. Note that our MKKM algorithm yields the bestperformance in all scenarios. Performance gain with theMKKM algorithm becomes more significant when theproblem gets harder with more clusters.

Nb. of ClustersClustering Accuracy

KM-AL [9] KM-PM MKKM5 88.06 94.62 96.126 85.07 94.52 95.277 85.50 94.88 95.528 85.63 90.93 95.349 73.96 79.10 83.0810 70.30 78.95 81.7911 68.38 78.56 81.4112 64.55 74.75 81.2213 61.65 73.82 80.14

TABLE 6: Pose grouping. Clustering accuracies on the CMU-PIE dataset. The proposed kernel k-means method yields thebest results in all the test cases.

9 CONCLUSIONIn this paper, we have introduced a unified frameworkto analyze the positive definiteness of the Gaussian RBFkernel defined on a manifold or a more general metricspace. We have then used the same framework to deriveprovably positive definite kernels on the Riemannianmanifold of SPD matrices and on the Grassmann mani-fold. These kernels were then utilized to extend popularlearning algorithms designed for Euclidean spaces, suchas SVM and FDA, to manifolds. Our experimental evalu-ation on several challenging computer vision tasks, suchas pedestrian detection, object categorization, segmen-tation, action recognition and face recognition, has evi-denced that identifying positive definite kernel functionson manifolds can be greatly beneficial when workingwith manifold-valued data. In the future, we intend tostudy this problem for other nonlinear manifolds, as wellas to extend our theory to more general kernels than theGaussian RBF kernel.

ACKNOWLEDGEMENTSThis work was supported in part by an ARC grant.The authors would like thank Bob Williamson for usefuldiscussions.REFERENCES[1] X. Pennec, P.Fillard, and N. Ayache, “A Riemannian Framework

for Tensor Computing,” IJCV, 2006.[2] O. Tuzel, F. Porikli, and P. Meer, “Region Covariance: A Fast

Descriptor for Detection and Classification,” in ECCV, 2006.[3] ——, “Pedestrian Detection via Classification on Riemannian

Manifolds,” PAMI, 2008.[4] P. Li and Q. Wang, “Local Log-Euclidean Covariance Matrix

(L2ECM) for Image Representation and Its Applications,” inECCV, 2012.

[5] A. Goh and R. Vidal, “Clustering and Dimensionality Reductionon Riemannian Manifolds,” in CVPR, 2008.

[6] R. Caseiro, J. F. Henriques, P. Martins, and J. Batista, “A Nonpara-metric Riemannian Framework on Tensor Field with Applicationto Foreground Segmentation,” in ICCV, 2011.

[7] J. Hamm and D. D. Lee, “Grassmann Discriminant Analysis: aUnifying View on Subspace-Based Learning,” in ICML, 2008.

[8] M. T. Harandi, C. Sanderson, S. Shirazi, and B. C. Lovell, “GraphEmbedding Discriminant Analysis on Grassmannian Manifoldsfor Improved Image Set Matching,” in CVPR, 2011.

[9] P. Turaga, A. Veeraraghavan, A. Srivastava, and R. Chellappa,“Statistical Computations on Grassmann and Stiefel Manifolds forImage and Video-Based Recognition,” PAMI, 2011.

[10] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, “Log-EuclideanMetrics for Fast and Simple Calculus on Diffusion Tensors,”Magnetic Resonance in Medicine, 2006.

14

[11] J. Shawe-Taylor and N. Cristianini, Kernel Methods for PatternAnalysis. Cambridge University Press, 2004.

[12] B. Scholkopf and A. J. Smola, Learning with Kernels: Support VectorMachines, Regularization, Optimization, and Beyond. MIT Press,2002.

[13] J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu, “SemanticSegmentation with Second-Order Pooling,” in ECCV, 2012.

[14] P. Li, Q. Wang, W. Zuo, and L. Zhang, “Log-Euclidean Kernels forSparse Representation and Dictionary Learning,” in ICCV, 2013.

[15] M. Harandi, C. Sanderson, R. Hartley, and B. Lovel, “SparseCoding and Dictionary Learning for Symmetric Positive DefiniteMatrices: A Kernel Approach,” in ECCV, 2012.

[16] D. Tosato, M. Spera, M. Cristani, and V. Murino, “CharacterizingHumans on Riemannian Manifolds,” PAMI, 2013.

[17] J. Malcolm, Y. Rathi, and A. Tannenbaum, “A Graph Cut Ap-proach to Image Segmentation in Tensor Space,” in CVPR, 2007.

[18] Y. M. Lui, “Advances in Matrix Manifolds for Computer Vision,”Image and Vision Computing, 2012.

[19] D. Tosato, M. Farenzena, M. Cristani, M. Spera, and V. Murino,“Multi-class Classification on Riemannian Manifolds for VideoSurveillance,” in ECCV, 2010.

[20] R. Caseiro, P. Martins, J. F. Henriques, F. S. Leite, and J. Batista,“Rolling Riemannian Manifolds to Solve the Multi-class Classifi-cation Problem,” in CVPR, 2013.

[21] M. Harandi, C. Sanderson, A. Wiliem, and B. Lovell, “KernelAnalysis over Riemannian Manifolds for Visual Recognition ofActions, Pedestrians and Textures,” in WACV, 2012.

[22] O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, “ChoosingMultiple Parameters for Support Vector Machines,” in MachineLearning, 2002.

[23] J. Ham and D. D. Lee, “Extended Grassmann Kernels forSubspace-Based Learning.” in NIPS, 2008.

[24] R. Caseiro, J. F. Henriques, P. Martins, and J. Batista, “Semi-Intrinsic Mean shift on Riemannian Manifolds,” in ECCV, 2012.

[25] R. Vemulapalli, J. K. Pillai, and R. Chellappa, “Kernel Learningfor Extrinsic Classification of Manifold Features,” in CVPR, 2013.

[26] J. Platt, “Fast Training of Support Vector Machines using Se-quential Minimal Optimization,” in Advances in Kernel Methods— Support Vector Learning. MIT Press, 1999.

[27] M. Varma and D. Ray, “Learning The Discriminative Power-Invariance Trade-Off,” in ICCV, 2007.

[28] C. S. Ong, S. Canu, and A. J. Smola, “Learning with Non-PositiveKernels,” in ICML, 2004.

[29] G. Wu, E. Y. Chang, and Z. Zhang, “An Analysis of Transfor-mation on Non-positive Semidefinite Similarity Matrix for KernelMachines,” in ICML, 2005.

[30] M. Belkin and P. Niyogi, “Semi-Supervised Learning on Rieman-nian Manifolds,” Machine Learning, 2004.

[31] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, “Fast and SimpleComputations on Tensors with Log-Euclidean Metrics,” INRIA,Research report RR-5584, 2005.

[32] S. Sra, “Positive Definite Matrices and the Symmetric Stein Di-vergence,” preprint, 2012, [arXiv:1110.1773].

[33] I. L. Dryden, A. Koloydenko, and D. Zhou, “Non-EuclideanStatistics for Covariance Matrices, with Applications to DiffusionTensor Imaging,” The Annals of Applied Statistics, 2009.

[34] A. Edelman, T. A. Arias, and S. T. Smith, “The Geometry ofAlgorithms with Orthogonality Constraints,” SIAM J. Matrix Anal.Appl., 1999.

[35] N. Aronszajn, “Theory of Reproducing Kernels,” Transactions ofthe American Mathematical Society, 1950.

[36] C. Berg, J. P. R. Christensen, and P. Ressel, Harmonic Analysis onSemigroups. Springer, 1984.

[37] I. J. Schoenberg, “Metric Spaces and Positive Definite Functions,”Transactions of the American Mathematical Society, 1938.

[38] M. Varma and B. R. Babu, “More Generality in Efficient MultipleKernel Learning,” in ICML, 2009.

[39] B. Scholkopf, A. Smola, and K.-R. Muller, “Nonlinear ComponentAnalysis as a Kernel Eigenvalue Problem,” Neural Computation,1998.

[40] S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. R. Mullers,“Fisher Discriminant Analysis with Kernels,” in Workshop onNeural Networks for Signal Processing, 1999.

[41] N. Dalal and B. Triggs, “Histograms of Oriented Gradients forHuman Detection,” in CVPR, 2005.

[42] B. Leibe and B. Schiele, “Analyzing Appearance and ContourBased Methods for Object Categorization,” in CVPR, 2003.

[43] T. Randen and J. Husoy, “Filtering for Texture Classification: aComparative Study,” PAMI, 1999.

[44] M. Kim, S. Kumar, V. Pavlovic, and H. A. Rowley, “Face Trackingand Recognition with Visual Constraints in Real-World Videos,”in CVPR, 2008.

[45] P. Viola and M. J. Jones, “Robust Real-Time Face Detection,” IJCV,2004.

[46] T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with localBinary Patterns,” PAMI, 2002.

[47] T.-K. Kim, J. Kittler, and R. Cipolla, “Discriminative Learning andRecognition of Image Set Classes Using Canonical Correlations,”PAMI, 2007.

[48] H. Cevikalp and B. Triggs, “Face Recognition Based on ImageSets,” in CVPR, 2010.

[49] Y. Wang and G. Mori, “Human Action Recognition by SemilatentTopic Models,” PAMI, 2009.

[50] T. Sim, S. Baker, and M. Bsat, “The CMU Pose, Illumination, andExpression (PIE) Database,” in FG, 2002.

Sadeep Jayasumana obtained his BSc degreein Electronics and Telecommunication Engineer-ing from University of Moratuwa, Sri Lanka, in2010. He is now a PhD student at the College ofEngineering and Computer Science, AustralianNational University (ANU). He is also a mem-ber of the Computer Vision Research Group atNICTA, government funded research laboratory.He received CSiRA Best Recognition prize atDICTA 2013. He is a student member of IEEE.

Richard Hartley is a member of the computervision group in the Research School of Engi-neering, at ANU, where he has been since Jan-uary, 2001. He is also a member of the computervision research group in NICTA. He worked atthe GE Research and Development Center from1985 to 2001, working first in VLSI design, andlater in computer vision. He became involvedwith Image Understanding and Scene Recon-struction working with GE’s Simulation and Con-trol Systems Division. He is an author (with A.

Zisserman) of the book Multiple View Geometry in Computer Vision.Mathieu Salzmann obtained his MSc and PhDdegrees from EPFL in 2004 and 2009, respec-tively. He then joined the International ComputerScience Institute and the EECS Department atthe University of California at Berkeley as a post-doctoral fellow, and later the Toyota TechnicalInstitute at Chicago as a research assistant pro-fessor. He is now a senior researcher at NICTAin Canberra.

Hongdong Li is with the computer vision groupat ANU. His research interests include 3D Com-puter Vision, vision geometry, image and pat-tern recognition, and mathematical optimization.Prior to 2010 he was a Senior Research Scien-tist with the NICTA, and a Fellow at the RSISE,ANU. He is a recipient of the CVPR Best Pa-per Award in 2012, Best Student Paper Awardat ICPR 2010, CSiRA Best Recognition PaperPrize at DICTA 2013. He was in Program Com-mittees (Area Chair/Reviewer) for recent ICCV,

CVPR, and ECCV. He is a member of ARC Centre of Excellence forRobotic Vision, a former member of the ”Bionic Vision Australia”.

Mehrtash Harandi received the BSc in Elec-tronics from Sharif University of technology, MScand PhD degrees in Computer Science from theUniversity of Tehran, Iran. Dr. Harandi is a seniorresearcher at Computer Vision Research Group(CVRG), NICTA. His main research interests aretheoretical and computational methods in com-puter vision and machine learning with a focuson Riemannian geometry.