1

A convex model for non-negative matrixfactorization and dimensionality reduction on

physical spaceErnie Esser, Michael Möller, Stanley Osher, Guillermo Sapiro, Jack Xin

Abstract—A collaborative convex framework for factoring adata matrix X into a non-negative product AS, with a sparsecoefficient matrix S, is proposed. We restrict the columns ofthe dictionary matrix A to coincide with certain columns of thedata matrix X , thereby guaranteeing a physically meaningfuldictionary and dimensionality reduction. We use l1,∞ regular-ization to select the dictionary from the data and show thisleads to an exact convex relaxation of l0 in the case of distinctnoise free data. We also show how to relax the restriction-to-Xconstraint by initializing an alternating minimization approachwith the solution of the convex model, obtaining a dictionaryclose to but not necessarily in X . We focus on applicationsof the proposed framework to hyperspectral endmember andabundances identification and also show an application to blindsource separation of NMR data.

Index Terms—Non-negative matrix factorization, dictionarylearning, subset selection, dimensionality reduction, hyperspec-tral endmember detection, blind source separation

I. INTRODUCTION

D IMENSIONALITY reduction has been widely studiedin the signal processing and computational learningcommunities. One of the major drawbacks of virtually allpopular approaches for dimensionality reduction is the lackof physical meaning in the reduced dimension space. Thissignificantly reduces the applicability of such methods. In thiswork we present a framework for dimensionality reduction,based on matrix factorization and sparsity theory, that uses thedata itself (or small variations from it) for the low dimensionalrepresentation, thereby guaranteeing the physical fidelity. Wepropose a new convex method to factor a non-negative datamatrix X into a product AS, for which S is non-negative andsparse and the columns of A coincide with columns from thedata matrix X .

The organization of this paper is as follows. In the remainderof the introduction, we further explain the problem, summarize

Copyright c©2012 IEEE. Personal use of this material is permitted. How-ever, permission to use this material for any other purposes must be obtainedfrom the IEEE by sending a request to pubs-permissions@ieee.org.

EE and JX were supported by NSF grants DMS-0911277, DMS-0928427,MM by the German Academic Exchange Service (DAAD), SO and MM byNSF grants DMS-0835863, DMS-0914561, DMS-0914856 and ONR grantN00014-08-1119, and GS was supported by NSF, NGA, ONR, ARO, DARPA,and NSSEFF.

E. Esser and J. Xin are with the Department of Mathematics, University ofCalifornia Irvine, 340 Rowland Hall, Irvine, CA 92697-3875. M. Möller andS. Osher are with the Department of Mathematics, University of CaliforniaLos Angeles, Box 951555, Los Angeles, CA 90095-1555. G. Sapiro is with theDepartment of Electrical and Computer Engineering, University of Minnesota,200 Union Street SE, 4-174 Keller Hall, Minneapolis, MN 55455-0170.

our approach and discuss applications and related work. InSection II we present our proposed convex model for end-member (dictionary) computation that uses l1,∞ regularizationto select as endmembers a sparse subset of columns of X ,such that sparse non-negative linear combinations of them arecapable of representing all other columns. Section III showsthat in the case of distinct noise free data, l1,∞ regularizationis an exact relaxation of the ideal row-0 norm (number ofnon-zero rows), and furthermore proves the stability of ourmethod in the noisy case. Section IV presents numerical resultsfor both synthetic and real hyperspectral data. In Section Vwe present an extension of our convex endmember detectionmodel that is better able to handle outliers in the data. Wediscuss its numerical optimization, compare its performance tothe basic model and also demonstrate its application to a blindsource separation (BSS) problem based on NMR spectroscopydata.

A. Summary of the problem and geometric interpretation

The underlying general problem of representing X ≈ ASwith A,S ≥ 0 is known as non-negative matrix factorization(NMF). Variational models for solving NMF problems aretypically non-convex and are solved by estimating A and Salternatingly. Although variants of alternating minimizationmethods for NMF often produce good results in practice, theyare not guaranteed to converge to a global minimum.

The problem can be greatly simplified by assuming a partialorthogonality condition on the matrix S as is done in [1],[2]. More precisely, the assumption is that for each row iof S there exists some column j such that Si,j > 0 andSk,j = 0 for k 6= i. Under this assumption, NMF has asimple geometric interpretation. Not only should the columnsof A appear in the data X up to scaling, but the remainingdata should be expressible as non-negative linear combinationsof these columns. Therefore the problem of finding A is tofind columns in X , preferably as few as possible, that span acone containing the rest of the data X . Figure 1 illustrates thegeometry in three dimensions.

The problem we actually want to solve is more difficultthan NMF in a couple respects. One reason is the need todeal with noisy data. Whereas NMF by itself is a difficultproblem already, the identification of the vectors becomes evenmore difficult if the data X contains noise and we need tofind a low dimensional cone that contains most of the data(see lower right image in Figure 1). Notice that in the noisy

2

case, finding vectors such that all data is contained in thecone they span would lead to a drastic overestimation of thenumber of vectors. Arbitrarily small noise at a single datapoint could already lead to including this vector into the setof cone spanning vectors. Thus, the problem is ill-posed andregularization is needed to handle noise. In addition to smallnoise there could also be outliers in the data, namely columnsof X that are not close to being well represented as a non-negative linear combination of other columns, but that we donot wish to include in A. Such outliers could arise from badsensor measurements, non-physical artifacts or any featuresthat for some reason we are not interested in including in ourdictionary A. Another complication that requires additionalmodeling is that for the applications we consider, the matrixS should also be sparse, which means we want the data to berepresented as sparse non-negative linear combinations of thecolumns of A.

Fig. 1: Geometric interpretation of the endmember detectionproblem. First row: two different viewpoints for a data setin three dimensions, second row: same data set with thevectors that can express any data point as a non-negative linearcombination in red, third row left: cone spanned by the threevectors containing all data, third row right: illustration of thecone with Gaussian noise added to the data, in this case notall points lie inside the cone anymore

B. Our proposed approach

We obtain the X = AS factorization by formulating theproblem as finding a sparse non-negative T such that X ≈ XTand as many rows of T as possible are entirely zero. We wantto encourage this so called row sparsity of T in order to selectas few as possible data points as dictionary atoms. We do this

by using l1,∞ regularization. This type of regularization cou-ples the elements in each row of T and is based on the recentideas of collaborative sparsity (see for example [3] and thereferences therein). The particular l1,∞ regularization has beenstudied by Tropp in [4], however, without considering non-negativity constraints and also not in the setting of finding a Tsuch that X ≈ XT for physically meaningful dimensionalityreduction. A strong motivation for using l1,∞ regularizationinstead of other row sparsity regularizers like l1,2 is that inthe case of distinct, noise-free data, the l1,∞ model is an exactrelaxation of minimizing the number of non-zero rows in Tsuch that X = XT . This exact relaxation is independent ofthe coherence of the columns of X . Without the non-negativityconstraint, a low coherence is crucial as is shown in [4]. Thegeneral setting X ≈ XT was proposed by Lin et al. in [5] forlow rank approximation with the nuclear norm. However, thenuclear norm does not lead to row sparsity, and thus there is noobvious way to extract dictionary atoms from the minimizer.Both the nuclear norm and l1,∞ approaches are addressingrelated but different problems. Our main contribution is toapply the joint sparsity regularizer l1,∞ to the non-negativefactorization setting X ≈ XT and thereby implicitly selectcertain columns of X for the description of all columns ofX . We pose this as a convex optimization problem in T . Forpractical reasons, we will need to perform some preliminarydata reduction, explained in Section II, before carrying out theconvex minimization. The main idea, however, is to minimizeover T ≥ 0 the l1,∞ norm of T plus some convex penaltyon X −XT . In the simplest case, we penalize ‖X −XT‖2F .We also propose an advanced noise model to handle the casewhere X contains outliers. Both models also incorporate aweighted l1 penalty to encourage a sparser T so that from thefew columns of X selected to represent the whole data, onlya few are used per sample.

C. Applications and related work

Although we concentrate on hyperspectral imaging (HSI)and briefly discuss an application to blind source separation(BSS), our method is applicable in numerous areas, frombiology to sensor networks.

For instance, one approach to text mining, that is thetechnique of extracting important information from large textdata sets, is to reduce the number of relevant text documentsby clustering them into content dependent categories usingnon-negative matrix factorization (see [6] for details). Ourapproach could potentially be used to not only cluster the largeamount of documents by a similar factorization, but due to thedictionary being a part of the data, it would furthermore leadto a correspondence of every atom in the dictionary to a certaindocument. This correspondence might help a human analyzerjudge the importance of each cluster. Therefore the physicalmeaning of the dictionary atoms would have an immediateadvantage for the further analysis.

As another example, in [7] non-negative matrix factorizationis applied to spectrograms of many different musical sounds inorder to obtain a spectral basis for musical genre classification.Again the physical fidelity of our approach could be interesting

3

since it would provide a correspondence of each spectral basiselement to a certain musical sound.

From the numerous potential applications, we concentrateon two to illustrate the proposed framework. We describe nextthe challenges in these applications. This early description ofthese applications will help to further motivate the work.

1) Introduction to hyperspectral imaging (HSI): HSI sen-sors record up to several hundred different frequencies inthe visible, near-infrared and infrared spectrum. This precisespectral information provides some insight on the material ateach pixel in the image. Due to relatively low spatial resolutionand the presence of multiple materials at a single location (e.g.,tree canopies above ground or water and water sediments),many pixels in the image contain the mixed spectral signaturesof multiple materials. The task of determining the abundances(presence quantity) of different materials in each pixel is calledspectral unmixing. This is clearly an ill-posed problem thatrequires some assumptions and data models.

Unmixing requires a dictionary with the spectral signaturesof the possible materials (often denoted as endmembers). Sincethese dictionaries can be difficult to obtain and might dependon the conditions under which they were recorded, it is some-times desirable to automatically extract suitable endmembersfrom the image one wants to demix in a process called end-member detection. Many different techniques for endmemberdetection have been proposed including variational [8], [9],[10] and Bayesian [11], [12], [13] models as well as convexgeometry-based methods [2], [14], [1]. Related although notyet applied to endmember detection are subset selection meth-ods like the rank-revealing QR (RRQR) decomposition (e.g.[15], [16]) and greedy dictionary selection methods [17], [18].The general principle behind rank-revealing QR for subset se-lection is to find a column permutation of the data matrix suchthat the first few columns are as well conditioned as possible[16]. In a sense, it looks for the most linearly independentcolumns of a matrix.1 However, unlike our approach, QRmethods do not take non-negativity constraints into account.The greedy dictionary selection methods [17], [18] are basedon greedy approximations of combinatorial optimization prob-lems for sparse dictionary selection. These methods come withtheoretical guarantees given incoherence assumptions aboutthe data matrix. Such assumptions do not hold for our data,but because we have a nonnegativity constraint not assumedthere, we can show our model is theoretically sound withoutstrong incoherence assumptions on the data, as demonstratedby the exact convex relaxation discussed in Section III.

Simultaneously detecting endmembers and computing abun-dances can be stated as factoring the data matrix X ∈ Rm,dinto X ≈ AS, A,S ≥ 0, with both A ∈ Rm,n and S ∈ Rn,dbeing unknown. In this notation each column of X is thespectral signature of one pixel in the image. Hence, m is thenumber of spectral bands, d is the total number of pixels, andeach of the n columns of A represents one endmember. Theabundance matrix S contains the amounts of each material in

1Independent of the work here described, Laura Balzano, Rob Nowakand Waheed Bajwa developed a related matrix factorization technique andconnected and compared it to RRQR. We thank Laura for pointing out theirwork [19] and also the possible relationships with RRQR.

A at each pixel in X . The application of NMF to hyperspectralendmember detection can for instance be found in [8], [9].

Considering that while material mixtures in HSI exist, itis unlikely that pixels contain a mixture of all or manyof the materials in A, researchers have recently focused onencouraging sparsity on the abundance matrix S [20], [21].Motivated by the ideas of dictionary learning for sparsecoding, the works [22], [23] proposed explicitly to look forendmember matrices that lead to sparse abundance maps. Wefollow a similar idea, though our method will be fundamentallydifferent in two aspects: First, we restrict columns of ourdictionary A to appear somewhere in the data X . This is acommon working hypothesis for moderate ground samplingdistance images and is called pixel purity assumption. It corre-sponds to the partial orthogonality assumption on S discussedpreviously. In the general context of dictionary learning andnon-negative matrix factorization it guarantees the columnsof A to be physically meaningful. As mentioned above, thelack of physical interpretation has been a critical shortcomingof standard dimensionality reduction and dictionary learningtechniques, and not yet addressed in these areas of research.Second, choosing the dictionary columns from the data enablesus to propose a convex model and hence avoid the problemof saddle points or local minima.

Different areas of applications use different terminology formathematically similar things. Throughout this paper we willuse the terminology of hyperspectral unmixing to explain andmotivate our model, although its application is not exclusivelyin the field of HSI. For instance, an endmember could be anyabstract column of the dictionary matrix A with a differentphysical meaning depending on its context. However, we thinkit aids the clarity and understanding of the model to use theHSI example throughout the explanations. To show that ourmodel is not limited to the hyperspectral case we also presentresults for the problem of blind source separation, which wewill briefly describe in the next subsection.

2) Introduction to blind source separation (BSS): Blindsource separation is the general problem of recovering un-known source signals from measurements of their mixtures,where the mixing process is unknown but often assumed to belinear. Examples include demixing audio signals and nuclearmagnetic resonance (NMR) spectroscopy. Many BSS prob-lems have the same linear mixing model X = AS that we areusing for hyperspectral endmember detection and unmixing,often also with a nonnegativity constraint on S [1]. Here therows of S represent the source signals, A is the unknownmixing matrix and the rows of X are the measured signals.The analogy of the hyperspectral pixel purity assumption canalso be applied to some BSS problems [1]. The interpretationis that for each source there is some place where among allthe sources only that source is nonzero. Thus up to scaling,the columns of A should appear somewhere in the columnsof the data matrix X and we can use the same algorithm weuse for endmember detection.

4

II. CONVEX ENDMEMBER DETECTION MODEL

A. Convexification of matrix factorization using the pixelpurity assumption

As mentioned above, we are assuming at first that theendmembers can be found somewhere in the data X . Thisassumption will enable us to propose a convex model forfactoring the data X into a product AS, a problem usuallytackled by non-convex optimization techniques. We assumethat there exists an index set I such that the columns Xi ofX are endmembers for i ∈ I . Under the assumption of non-negative linear mixing of signals, this means that any columnXj in X can be written as

Xj =∑i∈I

XiTi,j , (1)

for coefficients Ti,j ≥ 0. The problem is that the coefficientsTi,j as well as the index set I are unknown. Hence, we startby using all columns in X to describe X itself, i.e. we lookfor coefficients T ≥ 0 for which

X = XT. (2)

Notice that Equation (2) has many solutions. However, weknow that the desired representation uses as few columns ofX as possible, i.e., only the endmembers. Since not using thejth column in the above formulation corresponds to havingthe entire jth row of T be zero, we can reformulate theendmember detection problem as finding a solution to (2) suchthat as many rows of T as possible are zero. Mathematically,

minT≥0‖T‖row-0 such that XT = X, (3)

where ‖T‖row-0 denotes the number of non-zero rows. Thecolumns of X that correspond to non-zero rows of the mini-mizer T of (3) are the desired endmembers (which define thelowest dimensional subspace where the data resides). Sinceproblem (3) is not convex we relax the above formulationby replacing ‖T‖row-0 by the convex l1,∞ norm ‖T‖1,∞ =∑i maxj |Ti,j |. The l1 part of the norm should encourage

sparsity. Combined with the maximum norm we can expectthat the vector of maxima among each row becomes sparsewhich means we obtain row sparsity. Notice that we could havechosen other row-sparsity encouraging regularizers such asl1,2. However, we will prove in Section III the l1,∞ relaxationis exact in the case of normalized, non-repeating data, whichmakes it clearly preferable.

As mentioned in the previous section, it is important totake noise into account and therefore the equality (2) is toorestrictive for real data. Hence, we will introduce a parameterthat negotiates between having many zero rows (also calledrow sparsity) and good approximation of the data ‖X−XT‖2Fin the Frobenius norm. Furthermore, for T to be a goodsolution, not only should most of its rows be zero, but thenonzero rows should also be sparse, since sparsity of the co-efficient matrix reflects physically reasonable prior knowledge.For hyperspectral unmixing the additional sparsity requirementon T reflects the assumption that most pixels are not mixturesof all the selected endmembers, but rather just a few. Thus,we will add an additional weighted l1 term to incorporate

this second type of sparsity (see [24] for a model combiningstructured and collaborative sparsity with individual sparsity).

B. Data reduction as preprocessing

It is already clear from Equation (2) that the problem istoo large because the unknown matrix T is a d × d matrix,where d is the number of pixels. Thus, before proceedingwith the proposed convex model, we reduce the size of theproblem by using clustering to choose a subset of candidateendmembers Y from the columns of X and a submatrixXs ∈ Rm×ds of X for the data with ds ≤ d. In other words,we want to reformulate the problem as Y T ≈ Xs, whereXs ∈ Rm×ds , Y ∈ Rm×nc , T ∈ Rnc×ds , nc

5

To encourage sparsity of the nonnegative T , we use aweighted l1 norm, 〈RwσCw, T 〉. Here Rw is a diagonal matrixof row weights. We choose Rw to be the identity in ourexperiments, but if for example we wanted to encourageselection of endmembers towards the outside of the conecontaining the data, we could also choose these weights to beproportional to 〈Yj , Ȳ 〉, where Ȳ is the average of the columnsof Y . This would encourage the method to prefer endmebersfurther away from the average Ȳ . The weighting matrix σ hasthe same dimension as T , and the weights are chosen to be

σi,j = ν(1− e−(1−(Y TXs)i,j)

2

2h2 ), (5)

for constants h and ν. This means that σi,j is small when theith column of Y is similar to the jth column of Xs and largerwhen they are dissimilar. This choice of weights encouragessparsity of T without impeding the effectiveness of the otherregularizer. Since the smallest weight in each row occurs atthe most similar data, this helps ensure a large entry in eachnonzero row of T , which makes the row sparsity term moremeaningful. It also still allows elements in a column of T tobe easily shifted between rows corresponding to more similarendmember candidates, which can result in a reduction ofζ∑i maxj(Ti,j) without significantly affecting the weighted

l1 term. Since the weighted l1 penalty here is really just alinear term, it can’t be said to directly enforce sparsity, but byencouraging each column of T to sum to something closer toone, the weighted l1 penalty prefers data to be represented bynearby endmember candidates when possible, and this oftenresults in a sparser matrix T . Overall the proposed convexmodel is given by

minT≥0

ζ∑i

maxj

(Ti,j) + 〈RwσCw, T 〉+β

2‖(Y T −Xs)Cw‖2F .

(6)For our experiments we normalize the columns of X to haveunit l2 norm so that we discriminate based solely on spectralsignatures and not intensity.

D. Refinement of solution

Since we are using a convex model to detect endmem-bers, it cannot distinguish between identical or very similarendmember candidates, which we will discuss from a moretheoretical point of view in Section III. However, the modelworks very reliably when the columns of Y are sufficientlydistinct, which they are by construction. A limitation is thatthe convex model is unable to choose as endmembers anydata not represented in Y . Nonetheless, as is shown in SectionIV, the results of this approach already compare favorably toother methods. Even if some endmembers are not in Y , whichcould be due either to the data reduction preprocessing or to afailure of the pixel purity assumption, the underlying problemof representing the data in terms of the available endmembercandidates is still well posed. The convex model clearly cannotrecover endmembers that are not present in Y , but it shouldstill recover those that are present and represent the data assparse nonnegative linear combinations of a small number ofcolumns of Y . Moreover, it provides an excellent initialization

for the alternating minimization approach to NMF, which canbe used to further refine the solution if desired. Letting à bethe endmembers selected by the convex model, the solution isrefined by alternately minimizing

minA≥0,S≥0,||Aj−Ãj‖2 0.

Note that it is faster to precompute and store the inverseinvolved in the update for Z, but this can be overly memoryintensive if Y has many columns and Cw 6= I . One could usea more explicit variant of ADMM such as the method in [28]to avoid this difficulty if a larger number of columns of Yis desired. Here we have restricted this number nc to be lessthan 150 and ADMM can be reasonably applied.

Also note that the minimization problem for T k+1,

T k+1 = arg minTg≥0(T ) + ζ

∑i

maxj

(Ti,j)

+δ

2‖T − Zk+1 − P

k

δ+RwσCw

δ‖2F , (8)

decouples into separate problems for each row Ti. The Legen-dre transformation of g≥0(Ti) + ζ maxj(Ti,j) is the indicatorfunction for the set Cζ = {Pi ∈ Rds : ‖max(Pi, 0)‖1 ≤ ζ}.Let T̃ k+1 = Zk+1 + P

k

δ −RwσCw

δ . Then by the Moreaudecomposition [29], the minimizer T k+1 is given by

T k+1 = T̃ k+1 −ΠC ζδ

(T̃ k+1),

6

where ΠC ζδ

(T̃ k+1) orthogonally projects each row of T̃ k+1

onto C ζδ

, which is something that can be computed withcomplexity O(ds log(ds)).

The other algorithm parameters we use are δ = 1, ζ = 1,β = 250, ν = 50, and h = 1 − cos(4π/180). In ourexperiments, we also choose Xs = Y . We then define columnweights Cw that weight each column j by the number ofpixels in the jth cluster (the cluster centered at Yj) dividedby the total number of pixels d. To refine the solution ofthe convex model, we note that each alternating step in theminimization of (7) is a convex minimization problem thatcan again be straightforwardly minimized using ADMM andits variants. The update for the abundance S is identical to thesplit Bregman algorithm proposed for hyperspectral demixingin [20], [21], and its connection to ADMM is discussed in[30].

III. THE CONNECTION BETWEEN ROW-0 AND l1,∞In this section we will show that the motivation for our

model comes from a close relation between the convex l1,∞norm and the row-0 norm, which counts the number of non-zero rows.

A. Distinct noise-free data

Let us assume we have data X which is completely noisefree, normalized, obeys the linear mixing model, and containsa pure pixel for each material. Under slight abuse of notationlet us call this data after removing points that occur morethan once in the image, X again. As discussed in Section IIthe endmember detection problem can now be reformulatedas finding the minimizer T of (3), where the true set ofendmembers can then be recovered as the columns of X thatcorrespond to non-zero rows of T .

The fact that problem (3) gives a solution to the endmemberselection problem is not surprising, since (3) is a non-convexproblem and related problems (for instance in compressedsensing) are usually hard to solve. However, due to the non-negativity constraint we will show that l1,∞ minimization isan exact relaxation of the above problem.

For any T ≥ 0 with XT = X the entries of T are less thanor equal to one, Ti,j ≤ 1.3 Furthermore, the endmembers canonly be represented by themselves which means that in eachrow with index i, i ∈ I , we have a coefficient equal to 1. Wecan conclude that the l1,∞ norm of any T ≥ 0 with XT = Xis

‖T‖1,∞ =d∑i=1

maxjTi,j , (9)

≥∑i∈I

maxjTi,j = |I|. (10)

However, it is possible to have equality in the above estimateif and only if Ti,j = 0 for i /∈ I . In this case the rows of thenon-negative l1,∞ minimizer of XT = X are only supported

3This is a simple fact based on the normalization and non-negativity, 1 =

‖Xk‖ = ‖∑

i Ti,kXi‖T,X≥0≥ maxi ‖Ti,kXi‖ = maxi Ti,k

on I , which means it is a minimizer to the row-0 problem (3).Vice versa, any row-0 minimizer T̂ has exactly one entry equalto one in any row corresponding to an endmember and zerorows elsewhere, which means ‖T̂‖1,∞ = |I|, and hence T̂ alsominimizes the l1,∞ norm under the XT = X constraint. Wetherefore have shown the following lemma:

Lemma III.1. If we remove repeated columns of X and havenormalized data, the sets of minimizers of

minT≥0‖T‖row-0 such that XT = X (11)

and

minT≥0‖T‖1,∞ such that XT = X (12)

are the same.

Notice that while generally there are other regularizations,like for instance l1,2, which would also encourage row sparsity,this lemma only holds for l1,∞. For XT = X , the l1,∞norm counts the number of rows and is not influenced byany value that corresponds to a mixed pixel. This property isunique for l1,∞ regularization with a non-negativity constraintand therefore makes it the preferable choice in the proposedframework.

B. Noise in the data

Of course the assumptions above are much too restrictivefor real data. Therefore, let us look at the case of noisy dataof the form Xδ = X+N , where X is the true, noise-free datawith no repetition of endmembers as in the previous section,and N is noise bounded in the Frobenius norm by ‖N‖F ≤ δ.We now consider the model

Jα(T ) = ‖XT −X‖2F + α‖T‖1,∞ such that T ≥ 0. (13)

The following lemma shows that for the right, noise-dependentchoice of regularization parameter α, we converge to thecorrect solution as the noise decreases.

Lemma III.2. Let T̂ be a non-negative ‖ · ‖1,∞-minimumnorm solution of XT = X , Xδ = X +N be noisy data with‖N‖F ≤ δ and T δα denote a minimizer of energy functional(13) with regularization parameter α and replacing X by Xδ .If α is chosen such that

α→ 0, δ2

α→ 0 as δ → 0 (14)

then there exists a convergent subsequence T δnαn and the limitof each convergent subsequence is a ‖ · ‖1,∞-minimum normsolution of XT = X .

For the sake of clarity we moved the proof of this lemmato the appendix. Lemma III.2 shows that our regularization isstable, since for decreasing noise and appropriate choice of theparameter α we converge to a non-negative ‖ · ‖1,∞-minimumnorm solution of XT = X which - as we know from thefirst part - gives the true solution to the endmember detectionproblem when the columns of X are distinct. While identicalpoints are easy to identify and eliminate in the noise free

7

data, determining which points belong to the same endmemberin the noisy data can be very difficult. This is the reasonfor our first data reduction step. Lemma III.2 tells us thatfor our method to be closely related to the row-0 approachwe have to avoid having several noisy versions of the sameendmember in X . We therefore impose an angle constraintas described in Section II-B while clustering the data, whichbasically corresponds to an upper bound on the noise and statesup to which angle signals might correspond to the same point.

IV. NUMERICAL RESULTS FOR HSI

In this section we present numerical results on endmem-ber detection for supervised and real hyperspectral data andcompare our results to existing detection algorithms. Since thegoal of the paper is to present a general new convex frameworkfor matrix factorization which is applicable to multiple areas,the hyperspectral numerical results are intended mainly as aproof of concept. It is therefore encouraging that our methodis competitive with some of the established methods it iscompared to in the examples below.

A. Application to blind hyperspectral unmxing

1) Supervised endmember detection: For comparison pur-poses we extracted nine endmembers from the standard Indianpines dataset (publicly available at https://engineering.purdue.edu/biehl/MultiSpec/hyperspectral.html) by averaging over thecorresponding signals in the ground truth region. Then wecreated 50 data points for each endmember, 30 data points foreach combination of two different endmembers, 10 data pointsfor each combination of three different endmembers, and addi-tionally 30 data points as mixtures of all endmembers. Finally,we add Gaussian noise with zero mean and standard deviation0.006, make sure our data is positive, and normalize it. Weevaluate our method in a comparison to N-findr [2], vertexcomponent analysis (VCA) [14], [31] with code from [32],an NMF method using the alternating minimization schemeof our refinement step with random initial conditions, andthe QR algorithm. For the latter we simply used MATLAB’sQR algorithm to calculate a permutation matrix Π such thatXΠ = QR with decreasing diagonal values in R and chose thefirst nine columns of XΠ as endmembers. Many other methodscould also be applied here, including the additional variational,Bayesian and greedy dictionary selection methods referencedin the introduction. However, comparisons to the simple, wellestablished N-findr, VCA, QR and alternating minimizationalgorithms suffice to demonstrate that our proposed methodworks. Since the success of non-convex methods depends onthe initial conditions or on random projections, we run 15tests with the same general settings and record the average,maximum and minimum angle by which the reconstructedendmember vectors deviate from the true ones, see Table I.Our method, unlike the others in Table I, does not requirethe number of endmembers to be specified in advance. Forthe tests we adjusted the parameters of our method to obtain9 endmembers. Figure 2 shows the original endmembers aswell as an example reconstructions by each method with thecorresponding angle of deviation.

We can see that our method gives the best averageperformance. Due to a high noise level, methods that relyon finding the cone with maximal volume or finding mostlinearly independent vectors, will select outliers as theendmembers and do not yield robust results. Looking atthe minimal and maximal α we see the effect predicted.The non-convex methods like alternating minimizationand VCA can outperform our method on some examplesgiving angles as low as 1.76. However, due to the non-convexity they can sometimes find results which are faroff the true solution with deviation angles of 6.95 or even8.17 degrees. A big benefit of our convex framework isthat we consistently produce good results. The differencebetween the best and the worst reconstruction angledeviation for our method is 0.15 degrees with and 0.17without the refinement, which underlines its high stability.

Method Evaluation on 15 test runsAvg. α Min. α Max. α

Ours refined 3.37 3.30 3.42Ours without refinement 3.93 3.84 4.01

VCA 4.76 1.78 6.95N-findr 10.19 7.12 13.79

QR 9.87 4.71 12.74Alt. Min. 4.50 1.76 8.17

TABLE I: Deviation angle from true endmembers

The computational complexity of our method is highlydependent on the amount of data reduction. There is atrade-off here since more data reduction speeds up thecomputation but limits the number of endmember candidates.For instance, in the above experiment our algorithm (includingthe clustering with 10 k-means iterations and the refinementstep) takes about 100 seconds. This run-time comes fromthe clustering selecting 250 endmember candidates due to arather large noise level, which can make many similar signalslook much more distinct than they really are. Reducing thenoise level by a factor of 1/2, the clustering only selectsbetween 90 and 100 endmember candidates, leading to amuch faster run-time of about 13 seconds. In comparison, thealternating minimization approach needs about 45 seconds inthe case of strong noise and about 29 seconds in the caseof moderate noise. In our approach, the upper bound on thenumber of columns of Y can be adjusted according to howmuch of an issue computational resources are.

2) Results on real hyperspectral data: To show how ourmethod performs on real hyperspectral data we use the urbanimage (publicly available at www.tec.army.mil/hypercube).Figure 3 shows the RGB image of the urban scene, the spectralsignatures of the endmembers our method extracted, and theabundance maps of each endmember, i.e., each row of Twritten back into an image.

The abundance maps are used in HSI to analyze the dis-tribution of materials in the scene. First of all our methodmanaged to select six endmembers from the image, whichis a very reasonable dimension reduction for hyperspectralimage analysis. We can see that the abundance maps basicallysegment the image into clusters of different material categories

8

Fig. 2: Comparison of endmember recostruction methods

such as concrete, house roofs, soil or dirt, grass, and twodifferent types of vegetation, which all seem to be reasonablewhen visually comparing our results to the RGB image. Thespectral signatures our method selected are relatively distinct,but do not look noisy. Furthermore, the abundance maps seemto be sparse which was a goal of our method and reflects thephysically reasonable assumption that only very few materialsshould be present at each pixel.

As a next step we compare our results to the ones obtainedby N-findr, VCA, QR and alternating minimization. Unfortu-nately, we have no ground truth for the urban image, whichis why we will look at the non-negative least squares (NNLS)unmixing results based on the endmembers Am each methodfound. Notice that geometrically NNLS gives the projectionof the data onto the cone spanned by the endmembers. IfSm denotes the NNLS solution of each method, the error‖AmSm − X‖2F gives some insight on how much data iscontained in the cone and therefore, how well the endmembersdescribe the data. However, as discussed earlier, due to noisewe might not be interested in detecting the pixels furthestoutside to be endmembers, although they might describe thedata better. Thus, we will also report the sparsity of eachcone projection Sm. Since any outside point will be projectedonto a face or an edge of the cone, the sparsity will givesome insight into whether the endmember vectors are welllocated. The more the endmembers are towards the inside ofa point cluster, the more points we expect to be projectedonto an edge of the cone rather than a face. Thus, a highsparsity relates to a reasonable position of an endmember.Table II shows the relative number of non-zero coefficients,i.e., ‖Sm‖0 divided by the the total number of entries inSm, as well as the projection error for N-findr, QR, VCA,alternating minimization and our method. Figure 4 shows the

corresponding endmember signals each method found.

Ours N-findr VCA QR Alt. Min.‖AmSm −X‖2 533.9 4185.8 2516.5 1857.6 454.3‖Sm‖0/(d · n) 0.40 0.60 0.47 0.60 0.41

TABLE II: Comparison of different endmember detectionmethods in terms of error and sparsity of the data projectiononto the cone spanned by the endmembers on the urban image.The corresponding endmember signatures are shown in Figure4.

We can see from the projection error that the sparse al-ternating minimization approach and our method found muchbetter representation for the data than N-findr, VCA and QR,with the alternating minimization performing slightly betterthan our approach. Furthermore, the sparsity of the projectionis also higher which indicates more reasonable choices forthe endmembers. Looking at the spectral signatures in Figure4 we can speculate why this is the case. QR as well asN-findr chose very distinct signatures, which are probablyfar outliers in the dataset. Some of the VCA endmemberslook even more extreme and take negative values, whichis clearly unphysical and does not allow these endmembersto be interpreted as any material. On the other hand theendmembers of the alternating minimization approach and ofour method are very similar and look comparably smooth.The average angle of deviation between our method and thealternating minimization approach is only 3.4 degrees, whichlets us conclude that they basically converged to the sameanswer, which is encouraging considering the fact that theseendmembers describe the rest of the data more than three timesmore accurately than the endmembers found by other methods.Furthermore, any signal our method selected differs at most by0.036 degrees from an actual point in the data and is therefore

9

Fig. 3: Results on the urban hyperpectral image. First row, left: RGB image to compare the fraction planes to. First row, middleleft: spectral signatures of the endmembers our method found. First row middle right, right and second row: abundance mapsof the six endmembers.

physically meaningful.

V. AN EXTENDED MODELWe also propose an extended version of the model that

takes into account the normalization of the data in order tobetter distinguish between noise and outliers. We show thisslightly more complicated functional can still be efficientlysolved using convex optimization.

A. Error modelWe continue to work with the reduced set of endmember

candidates Y and a subset of the data Xs, which in practicewe take to be Y . Instead of penalizing ‖(Y T −Xs)Cw‖2F , weimpose the linear constraint Y T−Xs = V−Xs diag(e), whereT ,V and e are the unknowns. T has the same interpretationas before, V ∈ Rm×ds models the noise, and e ∈ Rds modelsthe sparse outliers. Since the columns of Xs are normalized tohave unit l2 norm, we would also like most of the columns ofY T to be approximately normalized. In modeling the noise V ,we therefore restrict it from having large components in thedirection of Xs. To approximate this with a convex constraintV ∈ D, we restrict each column Vj to lie in a hockey puckshaped disk Dj . Decompose Vj = V ⊥j + V

‖j , where V

‖j

is the orthogonal projection of Vj onto the line spanned byXj and V ⊥j is the radial component of Vj perpendicular toV‖j . Then given 0 ≤ rj < 1, we restrict ‖V ⊥j ‖2 ≤ rj and√1− r2j − 1 ≤ V

‖j ≤ 0. The orthogonal projection onto this

set is straightforward to compute since it is a box constraintin cylindrical coordinates. This constraint set for Vj is shownin Figure 5 in the case when ej = 0.

We also allow for a few columns of the data to be outliers.These are columns of X that we don’t expect to be well

Fig. 5: Region of possible values for Xj + Vj

represented as a small error plus a sparse nonnegative linearcombination of other data, but that we also don’t want toconsider as endmembers. Given some γ ≥ 0, this sparse erroris modeled as −Xs diag e with e restricted to the convexset E = {e : e ≥ 0 and

∑j(Cwe)j ≤ γ}. Since E is

the nonnegative region of a weighted l1 ball, the orthogonalprojection onto E can be computed with O(ds log(ds)) com-plexity. Here, since the weights wj sum to one by definition,γ can be roughly interpreted as the fraction of data we expectto be outliers. For non-outlier data Xj , we want ej ≈ 0and for outlier data we want ej ≈ 1. In the latter outliercase, regularization on the matrix T should encourage thecorresponding column Tj to be close to zero, so ‖Y Tj‖2 isencouraged to be small rather than close to one.

Keeping the l1,∞ regularization, the nonnegativity constraintand the weighted l1 penalty from Equation (6), the overallextended model is given by

10

Fig. 4: Spectral signatures of endmembers extracted by different methods. Top row: results of our method and the alternatingminimization approach. Bottom row: endmembers found by N-findr, QR and VCA

minT≥0,Vj∈Dj ,e∈E

ζ∑i

maxj

(Ti,j) + 〈RwσCw, T 〉 (15)

such that Y T −Xs = V −Xs diag(e).

The structure of this model is similar to the robust PCA modelproposed in [33] even though it has a different noise modeland uses l1,∞ regularization instead of the nuclear norm.

B. Numerical optimization

Since the convex functional for the extended model (15) isslightly more complicated, it is convenient to use a variantof ADMM that allows the functional to be split into morethan two parts. The method proposed by He, Tao and Yuanin [34] is appropriate for this application. Again introduce anew variable Z and the constraint Z = T . Also let P1 andP2 be Lagrange multipliers for the constraints Z − T = 0and Y Z − V − Xs + Xs diag(e) = 0 respectively. Then theaugmented Lagrangian is given by

Lδ(Z, T, V, e, P1, P2) =g≥0(T ) + gD(V ) + gE(e)

+ ζ∑i

maxj

(Ti,j) + 〈RwσCw, T 〉

+ 〈P1, Z − T 〉+ 〈P2, Y Z − V −Xs +Xs diag(e)〉

+δ

2‖Z − T‖2F

+δ

2‖Y Z − V −Xs +Xs diag(e)‖2F ,

where gD and gE are indicator functions for the V ∈ D ande ∈ E constraints.

Using the ADMM-like method in [34], a saddle point ofthe augmented Lagrangian can be found by iteratively solvingthe following subproblems with parameters δ > 0 and µ > 2,

Zk+1 = arg minZ

∥∥∥∥[ IY]Z −

[T k

V k −Xs diag(ek) +Xs

]+

1

δ

[P k1P k2

]∥∥∥∥2F

T k+1 = arg minTg≥0(T ) + ζ

∑i

maxj

(Ti,j)

+δµ

2

∥∥∥∥T − T k − 1µ (Zk − T k)− P1δµ + RwσCwδµ∥∥∥∥2F

V k+1 = arg minV

gD(V ) +δµ

2

∥∥V − V k− 1µ

(Y Zk+1 − V k +Xs diag(ek)−Xs)−P k2δµ

∥∥∥∥2F

ek+1 = arg minegE(e) +

δµ

2

ds∑j=1

(ej − ekj +

1

δµ

m∑i=1

(Xs)i,j(Pk2 + δ(Y Z

k+1 − V k +Xs diag(ek)−Xs))i,j)2

P k+11 = Pk1 + δ(Z

k+1 − T k+1)P k+12 = P

k2 + δ(Y Z

k+1 − V k+1 −Xs +Xs diag(ek+1))

Each of these subproblems can be efficiently solved. Thereare closed formulas for the Zk+1 and V k+1 updates, and the

11

ek+1 and T k+1 updates both involve orthogonal projectionsthat can be efficiently computed.

C. Effect of extended model

A helpful example for visualizing the effect of the extendedmodel (15) is to apply it to an RGB image. Even thoughthe low dimensionality makes this significantly different fromhyperspectral data, it’s possible to view a scatter plot of thecolors and how modifying the model parameters affects theselection of endmembers. The NMR data in Section V-E isfour dimensional, so low dimensional data is not inherentlyunreasonable.

For the following RGB experiments, we use the sameparameters as described in Section II-E and use the same k-means with farthest first initialization strategy to reduce thesize of the initial matrix Y . We do not however perform thealternating minimization refinement step. Due to the differentalgorithm used to solve the extended model, there is anadditional numerical parameter µ, which for this applicationmust be greater than two according to [34]. We set µ equal to2.01. There are also model parameters rj and γ for modelingthe noise and outliers. To model the small scale noise V ,we set rj = η + mj , where η is fixed at .07 and mj isthe maximum distance from data in cluster j to the clustercenter Yj . To model the sparse error e, we use several differentvalues of γ, which should roughly correspond to the fractionof data we are allowed to ignore as outliers. We will also usedifferent values of ν, setting ν = 50 to encourage a sparserabundance matrix and setting ν = 0 to remove the weightedl1 penalty from the model. Figure 6 shows the data, which is acolor image of multicolored folders, the selected endmembers(black dots) from four different experiments, and the sparseabundance matrix T for one of the experiments.

Note that in the ν = 0 experiment, sparsity of T is notencouraged, so the dense gray cluster in the center of the cone,which corresponds to the floor in the image, is not selectedas an endmember. Additionally, the selected endmembers aretowards the very outside of the cone of data. In all theother experiments where ν = 50 the floor is selected as anendmember even though it is in the center of the cone of data.This results in a much sparser matrix T . Moreover, the selectedendmembers tend to be in the center of the clusters of colorsthat we see in the scatter plot of the folder data. Finally wenote that as we increase the parameter γ, fewer endmembersare selected and some of the smaller outlying color clustersare ignored.

D. Comparison between the base and the outlier model

To illustrate the difference between the outlier and the basemodel we use the same dataset as for the supervised endmem-ber detection experiment and first repeat the experiment fromSection IV-A1 with 30 data points for each endmember, 20data points for each combination of two different endmem-bers, 10 data points for each combination of three differentendmembers, and additionally 30 data points as mixtures ofall endmembers. We add Gaussian noise with zero mean andstandard deviation 0.005, run the outlier model with ζ = 1,

η = 0.08, γ = 0.01, ν = 40 and run the basic model withζ = 1.3, ν = 40, β = 250. Figure 7 shows the results for bothmethods with their average angle of deviation from the trueendmembers.

We can see that both models give good results close tothe ground truth. Due to the Gaussian noise we added, thebest possible choice of columns of X deviates by an averageangle of 3.29 degrees from the true endmembers. With therefinement step, both methods could achieve an average anglebelow this value. The main remaining deviation is mainly dueto the methods selecting an almost straight line rather then thecorresponding true endmember. The similar results show thatthe extended model can be made to perform like the basic one.The extended model has the advantage, though, of being ableto ignore a specified fraction of outliers.

As a second experiment we simulate outliers in the data.First we again create a mixed pixel data set with nine end-members as above, but without adding Gaussian noise. Next,we create a spike signal and add the spike itself as well asabout 3% data that is mixed with the spike signal. This shallsimulate a small fraction of the data being outliers. Again,we run the basic as well as the outlier model on this datasetand obtain the results shown in Figure 8. The upper left imageshows the true nine endmembers plus the spike signal we usedto create the outliers with. As we can see in the image on theupper right, which shows the result of the basic model, thealgorithm selected the spike as an endmember. This is veryreasonable because although only 3% of the data contains partsof the spike, it is a strong outlier and hence expensive for thefidelity term in the Frobenius norm to exclude. The shown tendetected endmembers deviate only by 3.6 degrees from thenine true endmembers and the spike.

The second row of Figure 8 shows the nine true endmemberson the left and the result of the outlier model on the right. Aswe can see the outlier model was able to handle the smallfraction of signals containing the spike and only selected thenine true endmembers. The average angle of deviation in thiscase is 2.7 degrees and we can confirm that the model behaveslike we expected it to.

E. Application to blind source separation of NMR data

To illustrate how our model can be applied to BSS problems,we use it to recover the four NMR source spectra from ([1]Fig. 4) from four noise free mixtures. The four sources areshown in Figure 9. Let S0 ∈ R4×5000 be the sources and letthe mixtures X0 be generated by X0 = A0S0 with

A0 =

.3162 .6576 .3288 .5000.3162 .3288 .6576 .5000.6325 .1644 .1644 .5000.6325 .6576 .6576 .5000

.We will use the outlier model to recover the mixing matrix

A from X0. Unlike the hyperspectral examples, some columnsof X0 here can be nearly zero if all sources are simultaneouslyzero at the same spectral index. We can see from Figure9 that this is indeed the case. Since our algorithm usesnormalized data, we first remove columns of X0 whose norm

12

original image ν = 0, γ = 0 ν = 50, γ = .005

T for ν = 50, γ = .005 ν = 50, γ = .01 ν = 50, γ = .1

Fig. 6: Results of the extended model applied to RGB image. Upper left: RGB image we apply the blind unmixing algorithmto. Upper middle: 3d plot of the data points in the image in their corresponding color. Shown as black dots are the endmembersdetected without allowing outliers (γ = 0) and without encouraging particular sparsity on the coefficients (ν = 0). Upper right:With allowing some outliers the method removed an endmember in the one of the outside clusters, but included the middlecluster due to the encouraged sparsity. Lower left: Endmember coefficients for the parameter choice ν = 50, γ = 0.005, wherethe brightness corresponds to the coefficient value. We can see that the coefficient matrix is sparse. Lower middle: Increasingthe allowed outliers the red cluster endmember is removed. Increasing the outliers even further leads to decreasing the numberof endmembers to 4.

Fig. 7: Comparison between the basic and the outlier method on Indian pines data with Gaussian noise

is below some threshold, which we take to be .01 maxj ‖Xj‖.We then normalize the remaining columns to get X . Thissimple approach suffices for this example, but in general theparameters rj for the V ∈ D constraint could also be modifiedto account for columns of X0 that have significantly differentnorms.

A minor difficulty in applying our method to this BSSproblem is that we know A should have four columns butthere is no way to constrain the algorithm to produce adictionary with exactly four elements. We therefore adjustparameters until the dimension of the result is correct. This is

straightforward to do and could be automated. For example, tochoose a smaller dictionary, we can reduce ν and/or increase γ.

The parameters used here are identical to those used in theRGB experiments of Section V-C except γ = .01 and ν = 5.Also, for the data reduction step, the angle constraint wasincreased to 〈Yi, Yj〉 < .998. The computed mixing matrixafter permutation is

A =

.3267 .6524 .3327 .4933.3180 .3300 .6544 .5110.6228 .1757 .1658 .4836.6358 .6593 .6585 .5114

.

13

Fig. 8: Comparison between the basic and the outlier method on Indian pines data with outliers

Note that the columns of A are normalized because thealgorithm selects dictionary elements from a normalized ver-sion of the data X . Since for this simple problem, A isinvertible, it is straightforward to recover the sources byS = max(0, A−1X0). More generally, we can recover S byminimizing the convex functional in Equation (7) with respectto S using the un-normalized data matrix X0 and the computedendmembers A.

VI. FUTURE RESEARCH

We have presented a convex method for factoring a datamatrix X into a product AS with S ≥ 0 under the constraintthat the columns of A appear somewhere in the data X . Thistype of factorization ensures the physical meaning of the dic-tionary A, and we have successfully applied it to hyperspectralendmember detection and blind source separation in NMR.For non-repeating noise free data, the l1,∞ regularization wasproven to be an exact relaxation of the row-0 norm. We furtherproposed an extended model that can better handle outliers.Possible future application areas include computational biol-ogy, sensor networks, and in general dimensionality reductionand compact representation applications where the physicalinterpretation of the reduced space is critical. It will also beinteresting to try and extend our convex model to the classof problems discussed in [35] for which the pixel purity ornon-overlapping assumption is approximately but not exactlysatisfied.Acknowledgments - We would like to acknowledge LauraBalzano and Rob Nowak, who are working on a similarselection framework without non-negativity constraints, par-tially motivated by problems in sensor networks, and had

provided important comments and feedback. We would alsolike to thank Nicolas Gillis, John Greer and Todd Wittman forhelpful discussions about endmember detection strategies, andYuanchang Sun for his advice on BSS problems.

APPENDIXPROOF OF LEMMA III.2

Proof: Since T δα is a minimizer of Jα we can conclude

‖T δα‖1,∞ ≤1

αJα(T

δα)

≤ 1αJα(T̂ )

= ‖T̂‖1,∞ +1

α‖(X +N)T̂ − (X +N)‖2F

= ‖T̂‖1,∞ +1

α‖ (XT̂ −X)︸ ︷︷ ︸

=0

+(NT̂ −N)‖2F

≤ ‖T̂‖1,∞ +1

α‖N‖2F ‖T̂ − Id‖2F

= ‖T̂‖1,∞ +δ2

α‖T̂ − Id‖2F . (16)

Since δ2

α → 0, Tδα is bounded in the ‖ · ‖1,∞ norm and

therefore has a convergent subsequence. Let T δnαn denote sucha convergent subsequence and let T̄ be its limit. BecauseT δnαn ≥ 0 we also have T̄ ≥ 0. We can now use the aboveestimate for showing

‖T̄‖1,∞ = limn‖T δnαn‖1,∞

≤ limn

[‖T̂‖1,∞ +

δ2nαn‖T̂ − Id‖2F

]= ‖T̂‖1,∞. (17)

14

Fig. 9: Four NMR source spectra from [1] Fig. 4

Furthermore, we have

‖XT̄ −X‖2F = limn‖XδnT δnαn −X

δn‖2F≤ lim

nJαn(T

δnαn)

≤ limnJαn(T̂ )

≤ limn‖Xδn T̂ −Xδn‖2F + αn‖T̂‖1,∞

≤ limnδ2n‖T̂ − Id‖2F + αn‖T̂‖1,∞

= 0. (18)

Now, by the above estimate (18) we know that XT̄ = X .Furthermore, by the estimate (17) and taking into account thatT̂ was a non-negative ‖ · ‖1,∞-minimum norm solution ofXT = X , we have shown that the limit of our convergentsubsequence is also a non-negative ‖ · ‖1,∞-minimum normsolution of XT = X .

REFERENCES

[1] W. Naanaa and J-M. Nuzillard, “Blind source separation of positive andpartially correlated data,” Signal Processing, vol. 85, pp. 1711–1722,2005.

[2] M. E. Winter, “N-FINDR: an algorithm for fast autonomous spectralend-member determination in hyperspectral data,” in Imaging Spectrom-etry V. 1999, vol. 3753, pp. 266–275, SPIE.

[3] R. Jenatton, J.-Y. Audibert, and F. Bach, “Structured variable selectionwith sparsity-inducing norms,” Tech. Rep., 2009, arXiv:0904.3523v2.

[4] J. A. Tropp, “Algorithms for simultaneous sparse approximation: partII: Convex relaxation,” Signal Process., vol. 86, no. 3, pp. 589–602,2006.

[5] G. Liu, Z. Lin, and Y. Yu, “Robust subspace segmentation by low-rankrepresentation,” in ICML, 2010, pp. 663–670.

[6] W. Xuand X. Liu and Y. Gong, “Document clustering based onnon-negative matrix factorization,” in Proceedings of the 26th annualinternational ACM SIGIR conference on Research and development ininformaion retrieval. 2003, SIGIR ’03, pp. 267–273, ACM.

[7] A. Holzapfel and Y. Stylianou, “Musical genre classification usingnonnegative matrix factorization-based features,” IEEE Trans. on Audio,Speech, and Language Processing, vol. 16, pp. 424–434, 2008.

[8] L. Miao and H. Qi, “Endmember extraction from highly mixed datausing minimum volume constrained nonnegative matrix factorization,”IEEE Trans. Geosci. Rem. Sens., vol. 45, no. 3, pp. 765–777, 2007.

[9] M. Berry, M. Browne, A. Langville, P. Pauca, and R.J. Plemmons,“Algorithms and applications for approximate nonnegative matrix fac-torization,” Computational Statistics and Data Analysis, vol. 52, pp.155–173, 2007.

[10] V. P. Pauca, J. Piper, and R. J. Plemmons, “Nonnegative matrix factor-ization for spectral data analysis,” Linear Algebra and its Applications,vol. 416, no. 1, pp. 29–47, 2006.

[11] A. Zare, “Hyperspectral endmember detectionand band selection using Bayesian methods,” 2008,http://gradworks.umi.com/33/47/3347194.html.

[12] S. Moussaoui, D. Brie, A. Mohammad-Djafari, and C. Carteret, “Sepa-ration of non-negative mixture of non-negative sources using a bayesianapproach and mcmc sampling,” Signal Processing, IEEE Transactionson, vol. 54, no. 11, pp. 4133 –4145, nov. 2006.

[13] N. Dobigeon, S. Moussaoui, M. Coulon, J.-Y. Tourneret, and A.O.Hero, “Joint bayesian endmember extraction and linear unmixing forhyperspectral imagery,” Signal Processing, IEEE Transactions on, vol.57, no. 11, pp. 4355 –4368, nov. 2009.

[14] J.M.P. Nascimento and J.M. Bioucas-Dias, “Blind hyperspectral un-mixing,” in Proceedings of the SPIE Conference on Image and SignalProcessing for Remote Sensing XIII, 2007, vol. 6748.

[15] C. Boutsidis, M. W. Mahoney, and P. Drineas, “An improved approx-imation algorithm for the column subset selection problem,” in SODA’09: Proceedings of the twentieth Annual ACM-SIAM Symposium onDiscrete Algorithms, 2009, pp. 968–977.

[16] T.F. Chan and P.C. Hansen, “Some applications of the rank revealingQR factorization,” SIAM J. Sci. and Stat. Comput., vol. 13, no. 3, pp.727–741, 1992.

[17] A. Krause and V. Cevher, “Submodular dictionary selection for sparserepresentation,” in ICML, 2010.

[18] A. Das and D. Kempe, “Submodular meets spectral: Greedy algorithmsfor subset selection, sparse approximation and dictionary selection,”Tech. Rep., 2011, arXiv:1102.3975.

[19] L. Balzano, R. Nowak, and W.U. Bajwa, “Column subset selectionwith missing data,” in Proc. NIPS Workshop on Low-rank Methods forLarge-scale Machine Learning, 2010.

[20] A. Szlam, Z. Guo, and S. Osher, “A split Bregman method for non-negative sparsity penalized least squares with applications to hyperspec-tral demixing,” Tech. Rep., 2010, UCLA CAM Report [10-06].

[21] Z. Guo, T. Wittman, and S. Osher, “L1 unmixing and its applicationto hyperspectral image enhancement,” Proceedings SPIE Conferenceon Algorithms and Technologies for Multispectral, Hyperspectral, andUltraspectral Imagery XV, vol. 7334, pp. 73341M–73341M–9, 2008.

15

[22] J. Greer, “Sparse demixing,” SPIE proceedings on Algorithms and Tech-nologies for Multispectral, Hyperspectral, and Ultraspectral ImageryXVI, vol. 7695, pp. 76951O–76951O–12, 2010.

[23] A. Castrodad, Z. Xing, J. Greer, E. Bosch, L. Carin, and G. Sapiro,“Learning discriminative sparse models for source separation andmapping of hyperspectral imagery,” Submitted September 2010,http://www.ima.umn.edu/preprints/oct2010/oct2010.html.

[24] P. Sprechmann, I. Ramrez, G. Sapiro, and Y. Eldar, “C-HiLasso: A collaborative hierarchical sparse modeling framework,”http://arxiv.org/abs/1006.1346, June 2010.

[25] J. Mairal, R. Jenatton, G. Obozinski, and F. Bach, “Network flowalgorithms for structured sparsity,” CoRR, vol. abs/1008.5209, 2010.

[26] D. Gabay and B. Mercier, “A dual algorithm for the solution of nonlinearvariational problems via finite-element approximations,” Comp. Math.Appl., vol. 2, pp. 17–40, 1976.

[27] R. Glowinski and A. Marrocco, “Sur lapproximation par elementsfinis dordre un, et la resolution par penalisation-dualite dune classe deproblemes de Dirichlet nonlineaires,” Rev. Francaise dAut Inf. Rech.Oper., vol. R-2, pp. 41–76, 1975.

[28] X. Zhang, M. Burger, and S. Osher, “A unified primal-dual algorithmframework based on Bregman iteration,” Tech. Rep., 2009, UCLA CAMReport [09-99].

[29] J. J. Moreau, “Proximité et dualité dans un espace hilbertien,” Bull.Soc. Math. France, vol. 93, pp. 273–299, 1965.

[30] E. Esser, “Applications of Lagrangian-based alternating direction meth-ods and connections to split Bregman,” Tech. Rep., 2009, UCLA CAMReport [09-31].

[31] J.M.P. Nascimento and J.M. Bioucas-Dias, “Vertex component analysis:A fast algorithm to unmix hyperspectral data,” IEEE Trans. Geosci.Rem. Sens., vol. 43, pp. 898–910, 2004.

[32] “Open source MATLAB hyperspectral toolbox,” 2010. Version 0.04.http://matlabhyperspec.sourceforge.net/.

[33] E. Candes, X. Li, Y. Ma, and J. Wright, “Ro-bust principal component analysis,” 2009,http://arxiv.org/PS cache/arxiv/pdf/0912/0912.3599v1.pdf.

[34] B. He, M. Tao, and X. Yuan, “A splitting method for separate convexprogramming with linking linear constraints,” Tech. Rep., 2010.

[35] Y. Sun, C. Ridge, F. del Rio, A.J. Shaka, and J. Xin, “Postprocessingand sparse blind source separation of positive and partially overlappeddata,” Tech. Rep., 2010.

Ernie Esser Ernie Esser received B.S. degrees inmath and applied and computational mathematicalsciences from the University of Washington in 2003and a Ph.D. in math from UCLA in 2010. He iscurrently a postdoc at UC Irvine. His main researchinterests are convex optimization, image processingand inverse problems.

Michael Möller Michael Möller recieved his dim-ploma in mathematics from the Westfälische Wil-helms University of Münster in 2009, where hecurrently is a Ph.D. student. He spent two years asa visiting researcher at the University of California,Los Angeles. His research interests include convexoptimization, compressed sensing, image processing,and inverse problems.

Stanley Osher Stanley Osher received his B.S. de-gree in mathematics from Brooklyn College, Brook-lyn, NY, in 1962, and M.S. and Ph.D. degrees inmathematics from New York University in 1964 and1966, respectively. He has been with the Univer-sity of California, Los Angeles, since 1977. He iscurrently a Professor in the Department of Mathe-matics and the Director of Special Projects at theInstitute for Pure and Applied Mathematics. He isa member of the National Academy of Sciences,the American Academy of Arts and Sciences and

is one of the top 25 most highly cited researchers in both mathematicsand computer science. He has received numerous academic honors andhas co-founded three successful companies, each based largely on his own(joint) research. His current interests mainly involve information sciencewhich includes image processing, compressed sensing and machine learn-ing. His website is www.math.ucla.edu/˜sjo. His recent papers are on thewebsite for the UCLA Computational and Applied Mathematics Reports athttp://www.math.ucla.edu/applied/cam/index.shtml.

Guillermo Sapiro Guillermo Sapiro was born inMontevideo, Uruguay, on April 3, 1966. He receivedhis B.Sc. (summa cum laude), M.Sc., and Ph.D.from the Department of Electrical Engineering at theTechnion, Israel Institute of Technology, in 1989,1991, and 1993 respectively. After post-doctoralresearch at MIT, Dr. Sapiro became Member ofTechnical Staff at the research facilities of HP Labsin Palo Alto, California. He is currently with theDepartment of Electrical and Computer Engineeringat the University of Minnesota, where he holds the

position of Distinguished McKnight University Professor and VincentineHermes-Luh Chair in Electrical and Computer Engineering.

G. Sapiro works on differential geometry and geometric partial differentialequations, both in theory and applications in computer vision, computergraphics, medical imaging, and image analysis. He recently co-edited a specialissue of IEEE Image Processing in this topic and a second one in the Journalof Visual Communication and Image Representation. He has authored andco-authored numerous papers in this area and has written a book publishedby Cambridge University Press, January 2001.

G. Sapiro was awarded the Gutwirth Scholarship for Special Excellencein Graduate Studies in 1991, the Ollendorff Fellowship for Excellence inVision and Image Understanding Work in 1992, the Rothschild Fellowshipfor Post-Doctoral Studies in 1993, the Office of Naval Research YoungInvestigator Award in 1998, the Presidential Early Career Awards for Scientistand Engineers (PECASE) in 1998, the National Science Foundation CareerAward in 1999, and the National Security Science and Engineering FacultyFellowship in 2010.

G. Sapiro is a member of IEEE and SIAM.G. Sapiro is the funding Editor-in-Chief of the SIAM Journal on Imaging

Sciences.

Jack Xin Jack Xin received his B.S in computationalmathematics at Peking University in 1985, M.S andPh.D in applied mathematics at New York Universityin 1988 and 1990. He was a postdoctoral fellow atBerkeley and Princeton in 1991 and 1992. He wasassistant and associate professor of mathematics atthe University of Arizona from 1991 to 1999. Hewas a professor of mathematics from 1999 to 2005at the University of Texas at Austin. He has beena professor of mathematics in the Department ofMathematics, Center for Hearing Research, Institute

for Mathematical Behavioral Sciences, and Center for Mathematical andComputational Biology at UC Irvine since 2005. He is a fellow of the JohnS. Guggenheim Foundation. His research interests include applied analysis,computational methods and their applications in nonlinear and multiscaledynamics in fluids, and signal processing.