arX

iv:1

506.

0074

9v1

[cs.

IT]

2 Ju

n 20

151

Large-Scale Convex Optimization for DenseWireless Cooperative Networks

Yuanming Shi, Jun Zhang, Brendan O’Donoghue, and Khaled B. Letaief,Fellow, IEEE

Abstract—Convex optimization is a powerful tool for resourceallocation and signal processing in wireless networks. As thenetwork density is expected to drastically increase in order toaccommodate the exponentially growing mobile data traffic,per-formance optimization problems are entering a new era charac-terized by a high dimension and/or a large number of constraints,which poses significant design and computational challenges. Inthis paper, we present a novel two-stage approach to solve large-scale convex optimization problems for dense wireless cooperativenetworks, which can effectively detect infeasibility and enjoymodeling flexibility. In the proposed approach, the originallarge-scale convex problem is transformed into a standard coneprogramming form in the first stage via matrix stuffing, whichonly needs to copy the problem parameters such as channel stateinformation (CSI) and quality-of-service (QoS) requirements tothe pre-stored structure of the standard form. The capability ofyielding infeasibility certificates and enabling parallel computingis achieved by solving the homogeneous self-dual embeddingofthe primal-dual pair of the standard form. In the solving stage,the operator splitting method, namely, the alternating directionmethod of multipliers (ADMM), is adopted to solve the large-scalehomogeneous self-dual embedding. Compared with second-ordermethods, ADMM can solve large-scale problems in parallel withmodest accuracy within a reasonable amount of time. Simulationresults will demonstrate the speedup, scalability, and reliabilityof the proposed framework compared with the state-of-the-artmodeling frameworks and solvers.

Index Terms—Dense wireless networking, large-scale opti-mization, matrix stuffing, operator splitting method, ADMM ,homogeneous self-dual embedding.

I. I NTRODUCTION

T HE proliferation of smart mobile devices, coupled withnew types of wireless applications, has led to an expo-

nential growth of wireless and mobile data traffic. In order toprovide high-volume and diversified data services, ultra-densewireless cooperative network architectures have been proposedfor next generation wireless networks [1], e.g., Cloud-RAN[2], [3], and distributed antenna systems [4]. To enable efficientinterference management and resource allocation, large-scalemulti-entity collaboration will play pivotal roles in dense wire-less networks. For instance, in Cloud-RAN, all the basebandsignal processing is shifted to a single cloud data center with

Copyright (c) 2015 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to pubs-permissions@ieee.org.This work is supported by the Hong Kong Research Grant Council underGrant No. 16200214.

Y. Shi, J. Zhang and K. B. Letaief are with the Department of Electronicand Computer Engineering, Hong Kong University of Science and Technology(e-mail: {yshiac, eejzhang, eekhaled}@ust.hk).

B. O’Donoghue is with the Electrical Engineering Department, StanfordUniversity, Stanford, CA 94305 USA (e-mail: bodono@stanford.edu).

very powerful computational capability. Thus the centralizedsignal processing can be performed to support large-scalecooperative transmission/reception among the radio accessunits (RAUs).

Convex optimization serves as an indispensable tool forresource allocation and signal processing in wireless com-munication systems [5], [6], [7]. For instance, coordinatedbeamforming [8] often yields a direct convex optimizationformulation, i.e., second-order cone programming (SOCP) [9].The network max-min fairness rate optimization [10] canbe solved through the bi-section method [9] in polynomialtime, wherein a sequence of convex subproblems are solved.Furthermore, convex relaxation provides a principled way ofdeveloping polynomial-time algorithms for non-convex or NP-hard problems, e.g., group-sparsity penalty relaxation for theNP-hard mixed integer nonlinear programming problems [3],semidefinite relaxation [6] for NP-hard robust beamforming[11], [12] and multicast beamforming [13], and sequentialconvex approximation to the highly intractable stochasticcoordinated beamforming [14].

Nevertheless, in dense wireless cooperative networks [1],which may possibly need to simultaneously handle hundredsof RAUs, resource allocation and signal processing problemswill be dramatically scaled up. The underlying optimizationproblems will have high dimensions and/or large numbersof constraints (e.g., per-RAU transmit power constraints andper-MU (mobile user) QoS constraints). For instance, for aCloud-RAN with 100 single-antenna RAUs and 100 single-antenna MUs, the dimension of the aggregative coordinatedbeamforming vector (i.e., the optimization variables) will be104. Most advanced off-the-shelf solvers (e.g., SeDuMi [15],SDPT3 [16] and MOSEK [17]) are based on the interior-point method. However, the computational burden of suchsecond-order method makes it inapplicable for large-scaleproblems. For instance, solving convex quadratic programshas cubic complexity [18]. Furthermore, to use these solvers,the original problems need to be transformed to the standardforms supported by the solvers. Although the parser/solvermodeling frameworks like CVX [19] and YALMIP [20] canautomatically transform the original problem instances intostandard forms, it may require substantial time to performsuch transformation [21], especially for problems with a largenumber of constraints [22].

One may also develop custom algorithms to enable efficientcomputation by exploiting the structures of specific problems.For instance, the uplink-downlink duality [8] is exploitedtoextract the structures of the optimal beamformers [23] andenable efficient algorithms. However, such an approach still

2

has the cubic complexity to perform matrix inversion at eachiteration [24]. First-order methods, e.g., the ADMM algorithm[25], have recently attracted attention for their distributed andparallelizable implementation, as well as the capability ofscaling to large problem sizes. However, most existing ADMMbased algorithms cannot provide the certificates of infeasibility[11], [24], [26]. Furthermore, some of them may still fail toscale to large problem sizes, due to the SOCP subproblems[26] or semidefinite programming (SDP) subproblems [11]needed to be solved at each iteration.

Without efficient and scalable algorithms, previous studiesof wireless cooperative networks either only demonstrateperformance in small-size networks, typically with less than10 RAUs, or resort to sub-optimal algorithms, e.g., zero-forcing based approaches [27], [28]. Meanwhile, from theabove discussion, we see that the large-scale optimizationalgorithms to be developed should possess the following twofeatures:

• To scale well to large problem sizes with parallel com-puting capability;

• To effectively detect problem infeasibility, i.e., providecertificates of infeasibility.

To address these two challenges in a unified way, in thispaper, we shall propose a two-stage approach as shown inFig. 1. The proposed framework is capable to solve large-scale convex optimization problems in parallel, as well asproviding certificates of infeasibility. Specifically, theoriginalproblem P will be first transformed into a standard coneprogramming formPcone [18] based on the Smith formreformulation [29], via introducing a new variable for eachsubexpression in the disciplined convex programming form[30] of the original problem. This will eventually transform thecoupled constraints in the original problem into the constraintonly consisting of two convex sets: a subspace and a convex setformed by a Cartesian product of a finite number of standardconvex cones. Such a structure helps to develop efficientparallelizable algorithms and enable the infeasibility detectioncapability simultaneouslyvia solving the homogeneous self-dual embedding [31] of the primal-dual pair of the standardform by the ADMM algorithm.

As the mapping between the standard cone program andthe original problem only depends on the network size (i.e.,the numbers of RAUs, MUs and antennas at each RAU),we can pre-generate and store the structures of the standardforms with different candidate network sizes. Then for eachproblem instance, that is, given the channel coefficients, QoSrequirements, and maximum RAU transmit powers, we onlyneed to copy the original problem parameters to the standardcone programming data. Thus, the transformation procedurecan be very efficient and can avoid repeatedly parsing andre-generating problems [19], [20]. This technique is calledmatrix stuffing[21], [22], which is essential for the proposedframework to scale well to large problem sizes. It may alsohelp rapid prototyping and testing for practical equipmentdevelopment.

Transformation ADMM SolverP Pcone x⋆

Fig. 1. The proposed two-stage approach for large-scale convex optimization.The optimal solution or the certificate of infeasibility canbe extracted fromx⋆ by the ADMM solver.

A. Contributions

The major contributions of the paper are summarized asfollows:

1) We formulate main performance optimization problemsin dense wireless cooperative networks into a generalframework. It is shown that all of them can essentially besolved through solving one or a sequence of large-scaleconvex optimization or convex feasibility problems.

2) To enable both the infeasibility detection capability andparallel computing capability, we propose to transformthe original convex problem to an equivalent standardcone program. The transformation procedure scales verywell to large problem sizes with the matrix stuffingtechnique. Simulation results will demonstrate the ef-fectiveness of the proposed fast transformation approachover the state-of-art parser/solver modeling frameworks.

3) The operator splitting method is then adopted to solvethe large-scale homogeneous self-dual embedding of theprimal-dual pair of the transformed standard cone pro-gram in parallel. This first-order optimization algorithmmakes the second stage scalable. Simulation results willshow that it can speedup several orders of magnitudeover the state-of-art interior-point solvers.

4) The proposed framework enables evaluating variouscooperation strategies in dense wireless networks, andhelps reveal new insightsnumerically. For instance,simulation results demonstrate a significant performancegain of optimal beamforming over sub-optimal schemes,which shows the importance of developing large-scaleoptimal beamforming algorithms.

This work will serve the purpose of providing practicaland theoretical guidelines on designing algorithms for genericlarge-scale optimization problems in dense wireless networks.

B. Organization

The remainder of the paper is organized as follows. SectionII presents the system model and problem formulations. InSection III, a systematic cone programming form transforma-tion procedure is developed. The operator splitting methodispresented in Section IV. The practical implementation issuesare discussed in Section V. Numerical results will be demon-strated in Section VI. Finally, conclusions and discussions arepresented in Section VII. To keep the main text clean and freeof technical details, we divert most of the proofs, derivationsto the appendix.

II. L ARGE-SCALE OPTIMIZATION IN DENSE WIRELESS

COOPERATIVE NETWORKS

In this section, we will first present two representativeoptimization problems in wireless cooperative networks, i.e.,the network power minimization problem and the network

3

Fronthaul Links

BBU Pool

Radio Access Unit

Mobile User

Baseband Unit

Fig. 2. The architecture of Cloud-RAN, in which, all the RAUsare connectedto a BBU pool through high-capacity and low-latency opticalfronthaul links.To enable full cooperation among RAUs, it is assumed that allthe user dataand CSI are available at the BBU pool.

utility maximization problem. We will then provide a unifiedformulation for large-scale optimization problems in densewireless cooperative networks.

A. Signal Model

Consider a dense fully cooperative network1 with L RAUsandK single-antenna MUs, where thel-th RAU is equippedwith Nl antennas. The centralized signal processing is per-formed at a central processor, e.g., the baseband unit poolin Cloud-RAN [2], [3] as shown in Fig.2. The propagationchannel from thel-th RAU to the k-th MU is denoted ashkl ∈ CNl , ∀k, l. In this paper, we focus on the downlinktransmission for illustrative purpose. But our proposed ap-proach can also be applied in the uplink transmission, as weonly need to exploit the convexity of the resulting performanceoptimization problems.The received signalyk ∈ C at MU kis given by

yk =

L∑

l=1

hH

klvlksk +∑

i6=k

L∑

l=1

hH

klvlisi + nk, ∀k, (1)

wheresk is the encoded information symbol for MUk withE[|sk|2] = 1, vlk ∈ CNl is the transmit beamforming vectorfrom the l-th RAU to thek-th MU, andnk ∼ CN (0, σ2

k) isthe additive Gaussian noise at MUk. We assume thatsk ’s andnk ’s are mutually independent and all the users apply singleuser detection. Thus the signal-to-interference-plus-noise ratio(SINR) of MU k is given by

Γk(v) =|hH

kvk|2∑

i6=k |hH

kvi|2 + σ2k

, ∀k, (2)

1The full cooperation among all the RAUs with global CSI and full userdata sharing is used as an illustrative example. The proposed framework canbe extended to more general cooperation scenarios, e.g., with partial user datasharing among RAUs as presented in [7, Section 1.3.1].

wherehk , [hTk1, . . . ,h

TkL]

T ∈ CN with N =∑L

l=1 Nl,vk , [vT

1k,vT2k, . . . ,v

TLk]

T ∈ CN andv , [vT1 , . . . ,v

TK ]T ∈

CNK . We assume that each RAU has its own power constraint,

K∑

k=1

‖vlk‖22 ≤ Pl, ∀l, (3)

wherePl > 0 is the maximum transmit power of thel-thRAU. In this paper, we assume that the full and perfect CSI isavailable at the central processor and all RAUs only provideunicast/broadcast services.

B. Network Power Minimization

Network power consumption is an important performancemetric for the energy efficiency design in wireless cooperativenetworks. Coordinated beamforming is an efficient way todesign energy-efficient systems [8], in which, beamformingvectorsvlk ’s are designed to minimize the total transmit poweramong RAUs while satisfying the QoS requirements for all theMUs. Specifically, given the target SINRsγ = (γ1, . . . , γK)for all the MUs withγk > 0, ∀k, we will solve the followingtotal transmit power minimization problem:

P1(γ) : minimizev∈V

L∑

l=1

K∑

k=1

‖vlk‖22, (4)

whereV is the intersection of the sets formed by transmitpower constraints and QoS constraints, i.e.,

V = P1 ∩ P2 ∩ · · · ∩ PL ∩ Q1 ∩Q2 . . . ,∩QK , (5)

wherePl ’s are feasible sets ofv that satisfy the per-RAUtransmit power constraints, i.e.,

Pl =

{

v ∈ CNK :

K∑

k=1

‖vlk‖22 ≤ Pl

}

, ∀l, (6)

andQk ’s are the feasible sets ofv that satisfy the per-MUQoS constraints, i.e.,

Qk = {v ∈ CNK : Γk(v) ≥ γk}, ∀k. (7)

As all the setsQk’s and Pl’s can be reformulated intosecond-order cones as shown in [3], problemP1(γ) can bereformulated as an SOCP problem.

However, in dense wireless cooperative networks, the mo-bile hauling network consumption can not be ignored. In [3],a two-stage group sparse beamforming (GSBF) frameworkis proposed to minimize the network power consumption forCloud-RAN, including the power consumption of all opticalfronthaul links and the transmit power consumption of allRAUs. Specially, in the first stage, the group-sparsity structureof the aggregated beamformerv is induced by minimizing theweighted mixedℓ1/ℓ2-norm ofv, i.e.,

P2(γ) : minimizev∈V

L∑

l=1

ωl‖vl‖2, (8)

where vl = [vTl1, . . . ,v

TlK ]T ∈ CNlK is the aggregated

beamforming vector at RAUl, andωl > 0 is the correspondingweight for the beamformer coefficient groupvl. Based on

4

the (approximated) group sparse beamformerv⋆, which isthe optimal solution toP2(γ), in the second stage, an RAUselection procedure is performed to switch off some RAUsso as to minimize the network power consumption. In thisprocedure, we need to check if the remaining RAUs cansupport the QoS requirements for all the MUs, i.e., check thefeasibility of problemP1(γ) given the active RAUs. Pleaserefer to [3] for more details on the group sparse beamformingalgorithm.

C. Network Utility Maximization

Network utility maximization is a general approach tooptimize network performance. We consider maximizing anarbitrary network utility functionU(Γ1(v), . . . ,ΓK(v)) thatis strictly increasing in the SINR of each MU [7], i.e.,

P3 : maximizev∈V1

U(Γ1(v), . . . ,ΓK(v)), (9)

where V1 = ∩Ll=1Pl is the intersection of the sets of the

per-RAU transmit power constraints (6). It is generally verydifficult to solve, though there are tremendous research effortson this problem [7]. In particular, Liu et al. in [32] provedthat P3 is NP-hard for many common utility functions, e.g.,weighted sum-rate. Please refer to [7, Table 2.1] for details onclassification of the convexity of utility optimization problems.

Assume that we have the prior knowledge of SINR valuesΓ⋆1, . . . ,Γ

⋆K that can be achieved by the optimal solution to

problemP3. Then the optimal solution to problemP1(γ)with target SINRs asγ = (Γ⋆

1, . . . ,Γ⋆K) is an optimal solution

to problemP3 as well [23]. The difference between problemP1(γ) and problemP3 is that the SINRs inP1(γ) arepre-defined, while the optimal SINRs inP3 need to besearched. For the max-min fairness maximization problem,optimal SINRs can be searched by the bi-section method[22], which can be accomplished in polynomial time. Forthe general increasing utility maximization problemP3, thecorresponding optimal SINRs can be searched as follows

maximizeγ∈R

U(γ1, . . . , γK), (10)

whereR ∈ RK+ is the achievable performance region

R = {(Γ1(v), . . . ,ΓK(v)) : v ∈ V1}. (11)

Problem (10) is a monotonic optimization problem [33] andthus can be solved by the polyblock outer approximationalgorithm [33] or the branch-reduce-and-bound algorithm [7].The general idea of both algorithms is iteratively improvingthe lower-boundUmin and upper-boundUmax of the objectivefunction of problem (10) such that

Umax− Umin ≤ ǫ, (12)

for a given accuracyǫ in finite iterations. In particular, at them-iteration, we need to check the convex feasibility problemof P1(γ

[m]) given the target SINRsγ[m] = (Γ[m]1 , . . . ,Γ

[m]K ).

However, the number of iterations scales exponentially withthe number of MUs [7]. Please refer to the tutorial [7, Section2.3] for more details. Furthermore, the network achievablerate region [34] can also be characterized by the rate profilemethod [35] via solving a sequence of such convex feasibilityproblemsP1(γ).

D. A Unified Framework of Large-Scale Network Optimiza-tion

In dense wireless cooperative networks, the central proces-sor can support hundreds of RAUs for simultaneously trans-mission/reception [2]. Therefore, all the above optimizationproblems are shifted into a new domain with a high problemdimension and a large number of constraints. As presentedpreviously, to solve the performance optimization problems,we essentially need to solve a sequence of the followingconvex optimization problem with different problem instances(e.g., different channel realizations, network sizes and QoStargets)

P : minimizev∈V

f(v), (13)

wheref(v) is convex inv as shown inP1(γ) and P2(γ).Solving problemP means that the corresponding algorithmshould return the optimal solution or the certificate of infea-sibility.

For all the problems discussed above, problemP can bereformulated as an SOCP problem, and thus it can be solvedin polynomial time via the interior-point method, which isimplemented in most advanced off-the-shelf solvers, e.g.,pub-lic software packages like SeDuMi [15] and SDPT3 [16] andcommercial software packages like MOSEK [17]. However,the computational cost of such second-order methods willbe prohibitive for large-scale problems. On the other hand,most custom algorithms, e.g., the uplink-downlink approach[8] and the ADMM based algorithms [11], [24], [26], however,fail to either scale well to large problem sizes or detect theinfeasibility effectively.

To overcome the limitations of the scalability of the state-of-art solvers and the capability of infeasibility detection ofthe custom algorithms, in this paper, we propose to solvethe homogeneous self-dual embedding [31] (which aims atproviding necessary certificates) of problemP via a first-order optimization method [25] (i.e., the operator splittingmethod). This will be presented in SectionIV. To arriveat the homogeneous self-dual embedding and enable parallelcomputing, the original problem will be first transformed intoa standard cone programming form as will be presented inSectionIII . This forms the main idea of the two-stage basedlarge-scale optimization framework as shown in Fig.1.

III. M ATRIX STUFFING FORFAST STANDARD CONE

PROGRAMMING TRANSFORMATION

Although the parser/solver modeling language framework,like CVX [19] and YALMIP [20], can automatically transformthe original problem instance into a standard form, it requiressubstantial time to accomplish this procedure [21], [22]. Inparticular, for each problem instance, the parser/solver model-ing frameworks need to repeatedly parse and canonicalize it.To avoid such modeling overhead of reading problem data andrepeatedly parsing and canonicalizing, we propose to use thematrix stuffing technique [21], [22] to perform fast transfor-mation by exploiting the problem structures. Specifically,wewill first generate the mapping from the original problem tothe cone program, and then the structure of the standard form

5

will be stored. This can be accomplished offline. Therefore,foreach problem instance, we only need to stuff its parameters todata of the corresponding pre-stored structure of the standardcone program. Similar ideas were presented in the emergingparse/generator modeling frameworks like CVXGEN [36] andQCML [21], which aim at embedded applications for somespecific problem families. In this paper, we will demonstratein SectionVI that matrix stuffing is essential to scale to largeproblem sizes for fast transformation at the first stage of theproposed framework.

A. Conic Formulation of Convex Programs

In this section, we describe a systematic way to transformthe original problemP to the standard cone program. Toenable parallel computing, a common way is to replicate somevariables through either exploiting problem structures [11],[24] or using the consensus formulation [25], [26]. How-ever, directly working on these reformulations is difficulttoprovide computable mathematical certificates of infeasibility.Therefore, heuristic criteria are often adopted to detect theinfeasibility, e.g., the underlying problem instance is reportedto be infeasible when the algorithm exceeds the pre-definedmaximum iterations without convergence [24]. To unify therequirements of parallel and scalable computing and to providecomputable mathematical certificates of infeasibility, inthispaper, we propose to transform the original problemP to thefollowing equivalent cone programPcone:

Pcone : minimizeν,µ

cTν

subject to Aν + µ = b (14)

(ν,µ) ∈ Rn ×K, (15)

whereν ∈ Rn andµ ∈ R

m are the optimization variables,K = {0}r×Sm1 ×· · ·×Smq with Sp as the standard second-order cone of dimensionp

Sp = {(y,x) ∈ R× Rp−1|‖x‖ ≤ y}, (16)

andS1 is defined as the cone of nonnegative reals, i.e.,R+.Here, eachSi has dimensionmi such that(r+

∑qi=1 mi) = m,

A ∈ Rm×n, b ∈ Rm, c ∈ Rn. The equivalence means thatthe optimal solution or the certificate of infeasibility of theoriginal problemP can be extracted from the solution to theequivalent cone programPcone. To reduce the storage andmemory overhead, we store the matrixA, vectorsb andc inthe sparse form [37] by only storing the non-zero entries.

The general idea of such transformation is to rewrite theoriginal problemP into a Smith form by introducing a newvariable for each subexpression in disciplined convex program-ming form [30] of problem P. The details are presented inthe Appendix. Working with this transformed standard coneprogramPcone has the following two advantages:

• The homogeneous self-dual embedding of the primal-dual pair of the standard cone program can be induced,thereby providing certificates of infeasibility. This willbepresented in SectionIV-A .

• The feasible setV (5) formed by the intersection of afinite number of constraint setsPl’s and Qk ’s in the

original problemP can be transformed into two setsin Pcone: a subspace (14) and a convex coneK, which isformed by the Cartesian product of second-order cones.This salient feature will be exploited to enable paralleland scalable computing, as will be presented in SectionIV-B.

B. Matrix Stuffing for Fast Transformation

Inspired by the work [21] on fast optimization code deploy-ment for embedding second-order cone program, we proposeto use the matrix stuffing technique [21], [22] to transformthe original problem into the standard cone program quickly.Specifically, for any given network size, we first generateand store the structure that maps the original problemP

to the standard formPcone. Thus, the pre-stored standardform structure includes the problem dimensions (i.e.,m andn), the description ofV (i.e., the array of the cone sizes[r,m1,m2, . . . ,mq]), and the symbolic problem parametersA,b andc. This procedure can be done offline.

Based on the pre-stored structure, for a given probleminstanceP, we only need to copy its parameters (i.e., thechannel coefficientshK ’s, maximum transmit powersPl’s,SINR targetsγk ’s) to the corresponding data in the standardform Pcone (i.e.,A andb). Details of the exact description ofcopying data for transformation are presented in the Appendix.As the procedure for transformation only needs to copy mem-ory, it thus is suitable for fast transformation and can avoidrepeated parsing and generating as in parser/solver modelingframeworks like CVX.

Remark 1:As shown in the Appendix, the dimension ofthe transformed standard cone programPcone becomesm =(L+K)+(2NK+1)+

∑Ll=1(2KNl+1)+K(2K+2), which

is much larger than the dimension of the original problem,i.e., 2NK in the equivalent real-field. But as discussed above,there are unique advantages of working with this standardform, which compensate for the increase in the size, as willbe explicitly presented in later sections.

IV. T HE OPERATORSPLITTING METHOD FOR

LARGE-SCALE HOMOGENEOUSSELF-DUAL EMBEDDING

Although the standard cone programPcone itself is suitablefor parallel computing via the operator splitting method [38],directly working on this problem may fail to provide certifi-cates of infeasibility. To address this limitation, based on therecent work by O’Donoghueet al. [39], we propose to solvethe homogeneous self-dual embedding [31] of the primal-dualpair of the cone programPcone. The resultant homogeneousself-dual embedding is further solved via the operator splittingmethod, a.k.a. the ADMM algorithm [25].

A. Homogeneous Self-Dual Embedding of Cone Programming

The basic idea of the homogeneous self-dual embedding isto embed the primal and dual problems of the cone programPcone into a single feasibility problem (i.e., finding a feasiblepoint of the intersection of a subspace and a convex set) suchthat either the optimal solution or the certificate of infeasibility

6

of the original cone programPcone can be extracted from thesolution of the embedded problem.

The dual problem ofPcone is given by [39]

Dcone : maximizeη,λ

−bTη

subject to −ATη + λ = c

(λ,η) ∈ {0}n ×K∗, (17)

whereλ ∈ Rn andη ∈ Rm are the dual variables,K∗ is thedual cone of the convex coneK. Note thatK = K∗, i.e., Kis self dual. Define the optimal values of the primal programPcone and dual programDcone are p⋆ and d⋆, respectively.Let p⋆ = +∞ and p⋆ = −∞ indicate primal infeasibilityand unboundedness, respectively. Similarly, letd⋆ = −∞ andd⋆ = +∞ indicate the dual infeasibility and unboundedness,respectively. We assume strong duality for the convex coneprogramPcone, i.e., p⋆ = d⋆, including cases when they areinfinite. This is a standard assumption for practically designingsolvers for conic programs, e.g., it is assumed in [15], [16],[17], [31], [39]. Besides, we do not make any regularityassumption on the feasibility and boundedness assumptionson the primal and dual problems.

1) Certificates of Infeasibility:Given the cone programPcone, a main task is to detect feasibility. In [40, Theorem 1], asufficient condition for the existence of strict feasible solutionwas provided for the transmit power minimization problemwithout power constraints. However, for the general problemP with per-MU QoS constraints and per-RAU transmit powerconstraints, it is difficult to obtain such a feasibility conditionanalytically. Therefore, most existing works either assume thatthe underlying problem is feasible [8] or provide heuristicways to handle infeasibility [24].

Nevertheless, the only way to detect infeasibility effectivelyis to provide a certificate or proof of infeasibility as presentedin the following proposition.

Proposition 1: [Certificates of Infeasibility] The followingsystem

Aν + µ = b,µ ∈ K, (18)

is infeasible if and only if the following system is feasible

ATη = 0,η ∈ K⋆,bTη < 0. (19)

Therefore, any dual variableη satisfying the system (19)provides a certificate or proof that the primal programPcone

(equivalently the original problemP) is infeasible.Similarly, any primal variableν satisfying the following

system

−Aν ∈ K, cTν < 0, (20)

is a certificate of the dual programDcone infeasibility.Proof: This result directly follows the theorem of strong

alternatives [9, Section 5.8.2].2) Optimality Conditions:If the transformed standard cone

programPcone is feasible, then (ν⋆,µ⋆,λ⋆,η⋆) are optimalif and only if they satisfy the following Karush-Kuhn-Tucker

(KKT) conditions

Aν⋆ + µ⋆ − b = 0 (21)

ATη⋆ − λ⋆ + c = 0 (22)

(η⋆)Tµ⋆ = 0 (23)

(ν⋆,µ⋆,λ⋆,η⋆) ∈ Rn ×K × {0}n ×K∗. (24)

In particular, the complementary slackness condition (23) canbe rewritten as

cTν⋆ + bTη⋆ = 0, (25)

which explicitly forces the duality gap to be zero.3) Homogeneous Self-Dual Embedding:We can first detect

feasibility by Proposition1, and then solve the KKT system ifthe problem is feasible and bounded. However, the disadvan-tage of such a two-phase method is that two related problems(i.e., checking feasibility and solving KKT conditions) needto be solved sequentially [31]. To avoid such inefficiency,we propose to solve the following homogeneous self-dualembedding [31]:

Aν + µ− bτ = 0 (26)

ATη − λ+ cτ = 0 (27)

cTν + bTη + κ = 0 (28)

(ν,µ,λ,η, τ, κ) ∈ Rn ×K × {0}n ×K∗ × R+ × R+, (29)

to embed all the information on the infeasibility and optimalityinto a single system by introducing two new nonnegativevariables τ and κ, which encode different outcomes. Thehomogeneous self-dual embedding thus can be rewritten asthe following compact form

FHSD : find (x,y)

subject to y = Qx

x ∈ C,y ∈ C∗, (30)

where

λ

µ

κ

︸ ︷︷ ︸

y

=

0 AT c

−A 0 b

−cT −bT 0

︸ ︷︷ ︸

Q

ν

η

τ

︸ ︷︷ ︸

x

, (31)

x ∈ Rm+n+1, y ∈ Rm+n+1, Q ∈ R(m+n+1)×(m+n+1), C =Rn ×K∗ ×R+ andC∗ = {0}n ×K×R+. This system has atrivial solution with all variables as zeros.

The homogeneous self-dual embedding problemFHSD isthus a feasibility problem finding a nonzero solution inthe intersection of a subspace and a convex cone. Let(ν,µ,λ,η, τ, κ) be a non-zero solution of the homogeneousself-dual embedding. We then have the following remarkabletrichotomy derived in [31]:

• Case 1: τ > 0, κ = 0, then

ν = ν/τ, η = η/τ, µ = µ/τ (32)

are the primal and dual solutions to the cone programPcone.

• Case 2: τ = 0, κ > 0; this impliescTν+bTη < 0, then

7

1) If bTη < 0, thenη = η/(−bTη) is a certificate ofthe primal infeasibility as

AT η = 0, η ∈ V⋆,bT η = −1. (33)

2) If cTν < 0, then ν = ν/(−cT ν) is a certificate ofthe dual infeasibility as

−Aν ∈ V , cT ν = −1. (34)

• Case 3: τ = κ = 0; no conclusion can be made aboutthe cone problemPcone.

Therefore, from the solution to the homogeneous self-dualembedding, we can extract either the optimal solution (basedon (60)) or the certificate of infeasibility for the originalproblem. Furthermore, as the set (29) is a Cartesian product ofa finite number of sets, this will enable parallelizable algorithmdesign. With the distinct advantages of the homogeneousself-dual embedding, in the sequel, we focus on developingefficient algorithms to solve the large-scale feasibility problemFHSD via the operator splitting method.

B. The Operator Splitting Method

Conventionally, the convex homogeneous self-dual embed-ding FHSD can be solved via the interior-point method, e.g.,[15], [16], [17], [31]. However, such second-order methodhas cubic computational complexity for the second-order coneprograms [18], and thus the computational cost will be pro-hibitive for large-scale problems. Instead, O’Donoghueet al.[39] develop a first-order optimization algorithm based on theoperator splitting method, i.e., the ADMM algorithm [25], tosolve the large-scale homogeneous self-dual embedding. Thekey observation is that the convex cone constraint inFHSD isthe Cartesian product of standard convex cones (i.e., second-order cones, nonnegative reals and free variables), whichenables parallelizable computing. Furthermore, we will showthat the computation of each iteration in the operator splittingmethod is very cheap and efficient.

Specifically, the homogeneous self-dual embeddingFHSD

can be rewritten as

minimize IC×C∗(x,y) + IQx=y(x,y), (35)

where IS is the indicator function of the setS, i.e., IS(z)is zero for z ∈ S and it is +∞ otherwise. By replicatingvariablesx andy, problem (35) can be transformed into thefollowing consensus form [25, Section 7.1]

PADMM : minimize IC×C∗(x,y) + IQx=y(x, y)

subject to (x,y) = (x, y), (36)

which is readily to be solved by the operator splitting method.Applying the ADMM algorithm [25, Section 3.1] to prob-

lem PADMM and eliminating the dual variables by exploitingthe self-dual property of the problemFHSD (Please refer to[39, Section 3] on how to simplify the ADMM algorithm), thefinal algorithm is shown as follows:

OSADMM :

x[i+1] = (I+Q)−1(x[i] + y[i])

x[i+1] = ΠC(x[i+1] − y[i])

y[i+1] = y[i] − x[i+1] + x[i+1],

(37)

whereΠC(x) denotes the Euclidean projection ofx onto theset C. This algorithm has theO(1/k) convergence rate [41]with k as the iteration counter (i.e., theǫ accuracy can beachieved inO(1/ǫ) iterations) and will not converge to zeroif a nonzero solution exists [39, Section 3.4]. Empirically, thisalgorithm can converge to modest accuracy within a reasonableamount of time. As the last step is computationally trivial,inthe sequel, we will focus on how to solve the first two stepsefficiently.

1) Subspace Projection via Factorization Caching:Thefirst step in the algorithmOSADMM is a subspace projection.After simplification [39, Section 4], we essentially need tosolve the following linear equation at each iteration, i.e.,

[I −AT

−A −I

]

︸ ︷︷ ︸

S

[ν

−η

]

︸ ︷︷ ︸

x

=

[ν[i]

η[i]

]

︸ ︷︷ ︸

b

, (38)

for the givenν [i] and η[i] at iteration i, whereS ∈ Rd×d

with d = m+ n is a symmetric quasidefinitematrix [42]. Toenable quicker inversions and reduce memory overhead viaexploiting the sparsity of the matrixS, the sparse permutedLDL

T factorization [37] method can be adopted. Specifically,such factor-solve method can be carried out by first computingthe sparse permutedLDLT factorization as follows

S = PLDLTPT , (39)

whereL is a lower triangular matrix,D is a diagonal matrix[38] and P with P−1 = PT is a permutation matrix tofill-in of the factorization [37], i.e., the number of nonzeroentries inL. Such factorization exists for any permutationP,as the matrixS is symmetric quasidefinite [42, Theorem 2.1].Computing the factorization costs much less thanO(1/3d3)flops, while the exact value depends ond and the sparsitypattern ofS in a complicated way. Note that such factorizationonly needs to be computed once in the first iteration and canbe cached for re-using in the sequent iterations for subspaceprojections. This is called thefactorization cachingtechnique[39].

Given the cached factorization (39), solving subsequentprojectionsx = S−1b (38) can be carried out by solvingthe following much easier equations:

Px1 = b,Lx2 = x1,Dx3 = x2,LTx4 = x3,P

Tx = x4,(40)

which cost zero flops,O(sd) flops by forward substitution withs as the number of nonzero entries inL, O(d) flops,O(sd)flops by backward substitution, and zero flops, respectively[9,Appendix C].

2) Cone Projection via Proximal Operator Evaluation:Thesecond step in the algorithmOSADMM is to project a pointωonto the coneC. As C is the Cartesian product of the finitenumber of convex conesCi, we can perform projection ontoCby projecting ontoCi separately and in parallel. Furthermore,the projection onto each convex cone can be done with closed-forms. Specifically, for nonnegative realCi = R+, we have that[43, Section 6.3.1]

ΠCi(ω) = ω+, (41)

8

where the nonnegative part operator(·)+ is taken elementwise.For the second-order coneCi = {(y,x) ∈ R×Rp−1|‖x‖ ≤ y},we have that [43, Section 6.3.2]

ΠCi(ω, τ) =

0, ‖ω‖2 ≤ −τ(ω, τ), ‖ω‖2 ≤ τ(1/2)(1 + τ/‖ω‖2)(ω, ‖ω‖2), ‖ω‖2 ≥ |τ |.

(42)

In summary, we have presented that each step in thealgorithmOSADMM can be computed efficiently. In particular,from both (41) and (42), we see that the cone projection canbe carried out very efficiently with closed-forms, leading toparallelizable algorithms.

V. PRACTICAL IMPLEMENTATION ISSUES

In previous sections, we have presented the unified two-stage framework for large-scale convex optimization in densewireless cooperative networks. In this section, we will focuson the implementation issues of the proposed framework.

A. Automatic Code Generation for Fast Transformation

In the Appendix, we describe a systematic way to transformthe original problem to the standard cone programming form.The resultant structure that maps the original problem to thestandard form can be stored and re-used for fast transformingvia matrix stuffing. This can significantly reduce the modelingoverhead compared with the parse/solver modeling frame-works like CVX. However, it requires tedious manual works tofind the mapping and may not be easy to verify the correctnessof the generated mapping. Chuet al. [21] gave such anattempt intending to automatically generate the code for matrixstuffing. However, the corresponding software package QCML[21], so far, is far from complete and may not be suitable forour applications. Extending the numerical-based transforma-tion modeling frameworks like CVX to the symbolic-basedtransformation modeling frameworks like QCML is not trivialand requires tremendous mathematical and technical efforts. Inthis paper, we derive the mapping in the Appendix manuallyand verify the correctness by comparing with CVX throughextensive simulations.

B. Implementation of the Operator Splitting Algorithm

Theoretically, the presented operator splitting algorithmOSADMM is compact, parameter-free, with parallelizable com-puting and linear convergence. Practically, there are typicallyseveral ways to improve the efficiency of the algorithm. Inparticular, there are various tricks that can be employed toim-prove the convergence rate, e.g, over-relaxation, warm-staringand problem data scaling as described in [39]. In the densewireless cooperative networks with multi-entity collaborativearchitecture, we are interested in two particular ways to speedup the subspace projection of the algorithmOSADMM , whichis the main computational bottleneck. Specifically, one wayis to use the parallel algorithms for the factorization (39) byutilizing the distributed computing and memory resources [44].For instance, in the cloud computing environments in Cloud-RAN, all the baseband units share the computing, memoryand storage resources in a single baseband unit pool [2],

[3]. Another way is to leveragesymbolic factorization (39)to speed up the numerical factorization for each problem in-stance, which is a general idea for the code generation systemCVXGEN [36] for realtime convex quadratic optimization [45]and the interior-point method based SOCP solver [46] forembedded systems. Eventually, the ADMM solver in Fig.1can be symbolic based so as to provide numerical solutionsfor each problem instance extremely fast and in a realtimeway. However, this requires further investigation.

VI. N UMERICAL RESULTS

In this section, we simulate the proposed two-stage basedlarge-scale convex optimization framework for performanceoptimization in dense wireless cooperative networks.The cor-responding MATLAB code that can reproduce all the simula-tion results using the proposed large-scale convex optimizationalgorithm is available online2.

We consider the following channel model for the linkbetween thek-th MU and thel-th RAU:

hkl = 10−L(dkl)/20√ϕklsklfkl, ∀k, l, (43)

whereL(dkl) is the path-loss in dB at distancedkl as shownin [3, Table I], skl is the shadowing coefficient,ϕkl is theantenna gain andfkl is the small-scale fading coefficient. Weuse the standard cellular network parameters as showed in [3,Table I]. All the simulations are carried out on a personalcomputer with 3.2 GHz quad-core Intel Core i5 processor and8 GB of RAM running Linux. The reference implementationof the operator splitting algorithm SCS is available online3,which is a general software package for solving large-scaleconvex cone problems based on [39] and can be called by themodeling frameworks CVX and CVXPY [47]. The settings(e.g., the stopping criteria) of SCS can be found in [39].

The proposed two-stage approach framework, termed “Ma-trix Stuffing+SCS”, is compared with the following state-of-art frameworks:

• CVX+SeDuMi/SDPT3/MOSEK: This category adoptssecond-order methods. The modeling framework CVXwill first automatically transform the original probleminstance (e.g., the problemP written in the disciplinedconvex programming form) into the standard cone pro-gramming form and then call an interior-point solver, e.g.,SeDuMi [15], SDPT3 [16] or MOSEK [17].

• CVX+SCS: In this first-order method based framework,CVX first transforms the original problem instance intothe standard form and then calls the operator splittingsolver SCS.

We define the “modeling time” as the transformation timefor the first stage, the “solving time” as the time spent for thesecond stage, and the “total time” as the time of the two stagesfor solving one problem instance. As the large-scale convexoptimization algorithm should scale well to both the modelingpart and the solving part simultaneously, the time comparisonof each individual stage will demonstrate the effectiveness ofthe proposed two-stage approach.

2https://github.com/SHIYUANMING/large-scale-convex-optimization3https://github.com/cvxgrp/scs

9

Given the network size, we first generate and store theproblem structure of the standard formPcone, i.e., the structureof A, b, c and the descriptions ofV . As this procedure canbe done offline for all the candidate network sizes, we thusignore this step for time comparison. We repeat the followingprocedures to solve the large-scale convex optimization prob-lem P with different parameters and sizes using the proposedframework “Matrix Stuffing+SCS”:

1) Copy the parameters in the problem instanceP to thedata in the pre-stored structure of the standard coneprogramPcone.

2) Solve the resultant standard cone programming instancePcone using the solver SCS.

3) Extract the optimal solutions ofP from the solutionsto Pcone by the solver SCS.

Finally, note that all the interior-point solvers are multiplethreaded (i.e., they can utilize multiple threads to gain extraspeedups), while the operator splitting algorithm solver SCSis single threaded. Nevertheless, we will show that SCSperforms much faster than the interior-point solvers. We alsoemphasize that the operator splitting method aims at scalingwell to large problem sizes and thus provides solutions tomodest accuracy within reasonable time, while the interior-point method intends to provide highly accurate solutions.Furthermore, the modeling framework CVX aims at rapidprototyping and providing a user-friendly tool for automati-cally transformations for general problems, while the matrix-stuffing technique targets at scaling to large-scale problems forthe specific problem familyP. Therefore, these frameworksand solvers are not really comparable with different purposesand application capabilities. We mainly use them to verifythe effectiveness and reliability of our proposed framework interms of the solution time and the solution quality.

A. Effectiveness and Reliability of the Proposed Large-ScaleConvex Optimization Framework

Consider a network withL 2-antenna RAUs andK single-antenna MUs uniformly and independently distributed4 inthe square region[−3000, 3000]× [−3000, 3000] meters withL = K. We consider the total transmit power minimizationproblemP1(γ) with the QoS requirements for each MU asγk = 5 dB, ∀k. Table I demonstrates the comparison of therunning time and solutions using different convex optimizationframeworks. Each point of the simulation results is averagedover 100 randomly generated network realizations (i.e., onesmall scaling fading realization for each large-scale fadingrealization).

For the modeling time comparisons, this table shows that thevalue of the proposed matrix stuffing technique ranges between0.01 and 30 seconds5 for different network sizes and canspeedup about 15x to 60x compared to the parser/solver mod-eling framework CVX. In particular, for large-scale problems,

4Consider the CSI acquisition overhead, our proposed approach is mainlysuitable in the low user mobility scenarios.

5This value can be significantly reduced in practical implementations, e.g.,at the BBU pool in Cloud-RAN, which, however, requires substantial furtherinvestigation. Meanwhile, the results effectively confirmthat the proposedmatrix stuffing technique scales well to large-scale problems.

the transformation using CVX is time consuming and becomesthe bottleneck, as the “modeling time” is comparable and evenlarger than the “solving time”. For example, whenL = 150,the “modeling time” using CVX is about 3 minutes, while thematrix stuffing only requires about 10 seconds. Therefore, thematrix stuffing for fast transformation is essential for solvinglarge-scale convex optimization problems quickly.

For the solving time (which can be easily calculated bysubtracting the “modeling time” from the “total time”) usingdifferent solvers, this table shows that the operator splittingsolver can speedup by several orders of magnitude over theinterior-point solvers. For example, forL = 50, it can speedupabout 20x and 130x over MOSEK6 and SDPT3, respectively,while SeDuMi is inapplicable for this problem size as therunning time exceeds the pre-defined maximum value, i.e.,one hour. In particular, all the interior-point solvers fail tosolve large-scale problems (i.e.,L = 100, 150, 200), denotedas “N/A”, while the operator splitting solver SCS can scalewell to large problem sizes. For the largest problems withL = 200, the operator splitting solver can solve them in about5 minutes.

For the quality of the solutions, this table shows that thepropose framework can provide a solution to modest accuracywithin much less time. For the two problem sizes, i.e.,L = 20andL = 50, which can be solved by the interior-point methodbased frameworks, the optimal values attained by the proposedframework are within0.03% of that obtained via the second-order method frameworks.

In summary, the proposed two-stage based large-scale con-vex optimization framework scales well to large-scale problemmodeling and solving simultaneously. Therefore, it provides aneffective way to evaluate the system performance via large-scale optimization in dense wireless networks. However, itsimplementation and performance in practical systems stillneedfurther investigation. In particular, this set of results indicatethat the scale of cooperation in dense wireless networks maybe fundamentally constrained by the computation complex-ity/time.

B. Infeasibility Detection Capability

A unique property of the proposed framework is its infeasi-bility detection capability, which will be verified in this part.Consider a network withL = 50 single-antenna RAUs andK = 50 single-antenna MUs uniformly and independentlydistributed in the square region[−2000, 2000]×[−2000, 2000]meters. The empirical probabilities of feasibility in Fig.3show that the proposed framework can detect the infeasibilityaccurately compared with the second-order method frame-work “CVX+SDPT3” and the first-order method framework“CVX+SCS”. Each point of the simulation results is av-eraged over200 randomly generated network realizations.The average (“total time”, “ solving time”) for obtaining asingle point with “CVX+SDPT3”, “ CVX+SCS” and “Matrix

6Although SeDuMi, SDPT3 and MOSEK (commercial software) areallbased on the interior-point method, the implementation efficiency of the corre-sponding software packages varies substantially. In the following simulations,we mainly compare with the state-of-art public solver SDPT3.

10

TABLE IT IME AND SOLUTION RESULTS FORDIFFERENTCONVEX OPTIMIZATION FRAMEWORKS

Network Size (L = K) 20 50 100 150 200

CVX+SeDuMiTotal Time [sec] 8.1164 N/A N/A N/A N/A

Objective [W] 12.2488 N/A N/A N/A N/A

CVX+SDPT3Total Time [sec] 5.0398 330.6814 N/A N/A N/A

Objective [W] 12.2488 6.5216 N/A N/A N/A

CVX+MOSEKTotal Time [sec] 1.2072 51.6351 N/A N/A N/A

Objective [W] 12.2488 6.5216 N/A N/A N/A

CVX+SCSTotal Time [sec] 0.8501 5.6432 51.0472 227.9894 725.6173

Modeling Time [sec] 0.7563 4.4301 38.6921 178.6794 534.7723

Objective [W] 12.2505 6.5215 3.1303 2.0693 1.5404

Matrix Stuffing+SCSTotal Time [sec] 0.1137 2.7222 26.2242 90.4190 328.2037

Modeling Time [sec] 0.0128 0.2401 2.4154 9.4167 29.5813

Objective [W] 12.2523 6.5193 3.1296 2.0689 1.5403

4 5 6 7 8 9 10 11

0

0.2

0.4

0.6

0.8

1

50 RAUs, 50 MUs100 RAUs, 100 MUs

Target SINR [dB]

Em

piric

alP

roba

bilit

yof

Fea

sbili

ty

CVX+SDPT3

CVX+SCS

Matrix Stuffing+SCS

Fig. 3. The empirical probability of feasibility versus target SINR withdifferent network sizes.

Stuffing+SCS” are (101.7635, 99.1140) seconds, (5.0754,2.3617) seconds and (1.8549, 1.7959) seconds, respectively.This shows that the operator splitting solver can speedup about50x over the interior-point solver.

We further consider a larger-sized network withL = 100single-antenna RAUs andK = 100 single-antenna MUsuniformly and independently distributed in the square region[−2000, 2000] × [−2000, 2000] meters. As the second-ordermethod framework fails to scale to this size, we only comparewith the first-order method framework. Fig.3 demonstratesthat the proposed framework has the same infeasibility de-tection capability as the first-order method framework. Thisverifies the correctness and the reliability of the proposedfast transformation via matrix stuffing. Each point of thesimulation results is averaged over 200 randomly generatednetwork realizations. The average (“solving time”, “ modelingtime”) for obtaining a single point with “CVX+SCS” and“Matrix Stuffing+SCS” are (41.9273, 18.6079) seconds and(31.3660, 0.5028) seconds, respectively. This shows that thematrix stuffing technique can speedup about 40x over thenumerical based parser/solver modeling framework CVX. Wealso note that the solving time of the proposed framework issmaller than the framework “CVX+SCS”, the speedup is dueto the warm-staring [39, Section 4.2].

0 1 2 3 4 5 6

10−1.8

10−1.6

10−1.4

10−1.2

20 RAUs, 40 MUs

50 RAUs, 50 MUs

Target SINR [dB]

Nor

mal

ized

Net

wor

kP

ower

Con

sum

ptio

n CVX+SDPT3

CVX+SCS

Matrix Stuffing+SCS

Fig. 4. Average normalized network power consumption (i.e., the obtainedoptimal total network power consumption over the maximum network powerconsumption with all the RAUs active and full power transmission) versustarget SINR with different network sizes.

C. Group Sparse Beamforming for Network Power Minimiza-tion

In this part, we simulate the network power minimizationproblem using the group sparse beamforming algorithm [3,Algorithm 2]. We set each fronthaul link power consumptionas 5.6W and set the power amplifier efficiency coefficientfor each RAU as25%. In this algorithm, a sequence ofconvex feasibility problems need to be solved to determinethe active RAUs and one convex optimization problem needsto be solved to determine the transmit beamformers. Thisrelies on the infeasibility detection capability of the proposedframework.

Consider a network withL = 20 2-antenna RAUs andK = 40 single-antenna MUs uniformly and independentlydistributed in the square region[−1000, 1000]×[−1000, 1000]meters. Each point of the simulation results is averaged over50 randomly generated network realizations. Fig.4 demon-strates the accuracy of the solutions in the network powerconsumption obtained by the proposed framework comparedwith the second-order method framework “CVX+SDPT3” andthe first-order method framework “CVX+SCS”. The average(“ total time”, “ solving time”) for obtaining a single point with

11

−10 0 10 20 30

0

2

4

6

Optimal Beamforming

RZF

ZFBF

MRT

SNR [dB]

Ave

rage

Min

imum

Rat

e[b

ps/H

z]

CVX+SCS

Matrix Stuffing+SCS

Fig. 5. The minimum network-wide achievable versus transmit SNR with55 single-antenna RAUs and 50 single-antenna MUs.

“CVX+SDPT3”, “ CVX+SCS” and “Matrix Stuffing+SCS”are (48.6916, 41.0316) seconds, (9.4619, 1.7433) secondsand (2.4673, 2.3061) seconds, respectively. This shows thatthe operator splitting solver can speedup about 20x over theinterior-point solver.

We further consider a larger-sized network withL = 50 2-antenna RAUs andK = 50 single-antenna MUs uniformly andindependently distributed in the square region[−3000, 3000]×[−3000, 3000] meters. As the second-order method frameworkis not applicable to this problem size, we only compare withthe first-order method framework. Each point of the simula-tion results is averaged over 50 randomly generated networkrealizations. Fig.4 shows that the proposed framework canachieve the same solutions in network power consumption asthe first-order method framework “CVX+SCS”. The average(“solving time”, “ modeling time”) for obtaining a singlepoint with “CVX+SCS” and “Matrix Stuffing+SCS” are(11.9643, 69.0520) seconds and (14.6559, 2.1567) seconds,respectively. This shows that the matrix stuffing techniquecan speedup about 30x over the numerical based parser/solvermodeling framework CVX.

In summary, Fig.4 demonstrates the capability of infeasi-bility detection (as a sequence of convex feasibility problemsneed to be solved in the RAU selection procedure), the accu-racy of the solutions, and speedups provided by the proposedframework over the existing frameworks.

D. Max-min Rate Optimization

We will simulate the minimum network-wide achievablerate maximization problem using the max-min fairness op-timization algorithm in [22, Algorithm 1] via the bi-sectionmethod, which requires to solve a sequence of convex fea-sibility problems. We will not only show the quality of thesolutions and speedups provided by the proposed framework,but also demonstrate that the optimal coordinated beamform-ers significantly outperform the low-complexity and heuristictransmission strategies, i.e., zero-forcing beamforming(ZFBF)[48], [28], regularized zero-forcing beamforming (RZF) [49]and maximum ratio transmission (MRT) [50].

Consider a network withL = 55 single-antenna RAUs andK = 50 single-antenna MUs uniformly and independently

distributed in the square region[−5000, 5000]×[−5000, 5000]meters. Fig. 5 demonstrates the minimum network-wideachievable rate versus different SNRs (which is defined asthe transmit power at all the RAUs over the receive noisepower at all the MUs) using different algorithms. Each pointofthe simulation results is averaged over 50 randomly generatednetwork realizations. For the optimal beamforming, this figureshows the accuracy of the solutions obtained by the pro-posed framework compared with the first-order method frame-work “CVX+SCS”. The average (“solving time”, “ modelingtime”) for obtaining a single point for the optimal beam-forming with “CVX+SCS” and “Matrix Stuffing+SCS” are(176.3410, 55.1542) seconds and (82.0180, 1.2012) seconds,respectively. This shows that the proposed framework canreduce both the solving time and modelling time via warm-starting and matrix stuffing, respectively.

Furthermore, this figure also shows that the optimal beam-forming can achieve quite an improvement for the per-user ratecompared to suboptimal transmission strategies RZF, ZFBFand MRT, which clearly shows the importance of developingoptimal beamforming algorithms for such networks. The aver-age (“solving time”, “ modeling time”) for a single point using“CVX+SDPT3” for the RZF, ZFBF and MRT are (2.6210,30.2053) seconds, (2.4592, 30.2098) seconds and (2.5966,30.2161) seconds, respectively. Note that the solving timeis very small, which is because we only need to solve asequence of linear programming problems for power controlwhen the directions of the beamformers are fixed during thebi-section search procedure. The main time consuming part isfrom transformation using CVX.

VII. C ONCLUSIONS ANDFURTHER WORKS

In this paper, we proposed a unified two-stage frameworkfor large-scale optimization in dense wireless cooperativenetworks. We showed that various performance optimiza-tion problems can be essentially solved by solving one ora sequence of convex optimization or feasibility problems.The proposed framework only requires the convexity of theunderlying problems (or subproblems) without any otherstructural assumptions, e.g., smooth or separable functions.This is achieved by first transforming the original convexproblem to the standard form via matrix stuffing and thenusing the ADMM algorithm to solve the homogeneous self-dual embedding of the primal-dual pair of the transformedstandard cone program. Simulation results demonstrated theinfeasibility detection capability, the modeling flexibility andcomputing scalability, and the reliability of the proposedframework.

In principle, one may apply the proposed framework toany large-scale convex optimization problems and only needsto focus on the standard form reformulation as shown inAppendix, as well as to compute the proximal operators fordifferent cone projections in (42). However, in practice, weneed to address the following issues to provide a user-friendlyframework and to assist practical implementation:

• Although the parse/solver frameworks like CVX canautomatically transform an original convex problem into

12

the standard formnumericallybased on the graph imple-mentation, extending such an idea to theautomatic andsymbolictransformation, thereby enabling matrix stuffing,is desirable but challenging in terms of reliability andcorrectness verification.

• Efficient projection algorithms are highly desirable. Forthe subspace projection, as discussed in SectionV-B,parallel factorization and symbolic factorization are espe-cially suitable for the cloud computing environments asin Cloud-RAN [2], [3]. For the cone projection, althoughthe projection on the second-order cone is very efficient,as shown in (42), projecting on the semidefinite cone(which is required to solve the semidefinite programmingproblems) is computationally expensive, as it requires toperform eigenvalue decomposition [13]. The structure ofthe cone projection should be exploited to make speedups.

• It is interesting to apply the proposed framework tovarious non-convex optimization problems. For instance,the well-known majorization-minimization optimizationprovides a principled way to solve the general non-convexproblems, whereas a sequence of convex subproblemsneed to be solved at each iteration. Enabling scalablecomputation at each iteration will hopefully lead toscalability of the overall algorithm.

APPENDIX

CONIC FORMULATION FOR CONVEX PROGRAMS

We shall present a systematic way to transform the originalproblem to the standard convex cone programming form. Wefirst take the real-field problemP with the objective functionf(x) = ‖v‖2 as an example. At the end of this subsection,we will show how to extend it to the complex-field.

According to the principle of the disciplined convex pro-gramming [30], the original problemP can be rewritten asthe following disciplined convex programming form [30]

Pcvx : minimize ‖v‖2subject to ‖Dlv‖2 ≤

√

Pl, l = 1, . . . , L (44)

‖Ckv + gk‖2 ≤ βkrTk v, k = 1, . . . ,K,(45)

where Dl = blkdiag{D1l , . . . ,D

Kl } ∈ RNlK×NK with

Dkl =

[

0Nl×∑l−1

i=1Ni, INl×Nl

,0Nl×∑

Li=l+1

Ni

]

∈ RNl×N ,

βk =√

1 + 1/γk, rk =[

0T(k−1)N ,hT

k ,0T(K−k)N

]T

∈ RNK ,

gk = [0TK , σk]

T ∈ RK+1, and Ck = [Ck,0NK ]T ∈R(K+1)×NK with Ck = blkdiag{hk, . . . ,hk} ∈ RNK×K . Itis thus easy to check the convexity of problemPcvx, followingthe disciplined convex programming ruleset [30].

A. Smith Form Reformulation

To arrive at the standard convex cone programPcone, werewrite problemPcvx as the following Smith form [29] byintroducing a new variable for each subexpression inPcvx,

minimize x0

subject to ‖x1‖ = x0,x1 = v

G1(l),G2(k), ∀k, l, (46)

whereG1(l) is the Smith form reformulation for the transmitpower constraint for RAUl (44) as follows

G1(l) :

(yl0,yl1) ∈ QKNl+1

yl0 =√Pl ∈ R

yl1 = Dlv ∈ RKNl ,

(47)

and G2(k) is the Smith form reformulation for the QoSconstraint for MUk (45) as follows

G2(k) :

(tk0 , tk1) ∈ QK+1

tk0 = βkrTk v ∈ R

tk1 = tk2 + tk3 ∈ RK+1

tk2 = Ckv ∈ RK+1

tk3 = gk ∈ RK+1.

(48)

Nevertheless, the Smith form reformulation (46) is notconvex due to the non-convex constraint‖x1‖ = x0. We thusrelax the non-convex constraint as‖x1‖ ≤ x0, yielding thefollowing relaxed Smith form

minimize x0

subject to G0,G1(l),G2(k), ∀k, l, (49)

where

G0 :

{(x0,x1) ∈ QNK+1

x1 = v ∈ RNK .(50)

It can be easily proved that the constraint‖x1‖ ≤ x0 has to beactive at the optimal solution; otherwise, we can always scaledownx0 such that the cost function can be further minimizedwhile still satisfying the constraints. Therefore, we concludethat the relaxed Smith form (49) is equivalent to the originalproblemPcvx.

B. Conic Reformulation

Now, the relaxed Smith form reformulation (49) is readilyto be reformulated as the standard cone programming formPcone. Specifically, define the optimization variables[x0;v]with the same order of equations as inG0, then G0 can berewritten as

M[x0;v] + µ0 = m, (51)

where the slack variables belong to the following convex set

µ0 ∈ QNK+1, (52)

andM ∈ R(NK+1)×(NK+1) andm ∈ RNK+1 are given asfollows

M =

[−1

−INK

]

,m =

[0

0NK

]

, (53)

respectively. Define the optimization variables[yl0;v] with thesame order of equations as inG1(l), thenG1(l) can be rewrittenas

Pl[yl0;v] + µl1 = pl, (54)

where the slack variablesµl1 ∈ RKNl+2 belongs to the

following convex set formed by the Cartesian product of twoconvex sets

µl1 ∈ Q1 ×QKNl+1, (55)

13

andPl ∈ R(KNl+2)×(NK+1) andpl ∈ RKNl+2 are given asfollows

Pl =

1−1

−Dl

,pl =

√Pl

00KNl

, (56)

respectively. Define the optimization variables[tk0 ;v] with thesame order of equations as inG2(k), then G2(k) can berewritten as

Qk[tk0 ;v] + µk2 = qk, (57)

where the slack variablesµk2 ∈ RK+3 belong to the following

convex set formed by the Cartesian product of two convex sets

µk2 ∈ Q1 ×QK+2, (58)

and Qk ∈ R(K+3)×(NK+1) and qk ∈ RK+3 are given asfollows

Qk =

1−βkrTk

−1−Ck

,qk =

00gk

, (59)

respectively.

Therefore, we arrive at the standard formPcone by writingthe optimization variablesν ∈ Rn as follows

ν = [x0; y10 ; . . . ; y

L0 ; t

10; . . . , t

K0 ;v], (60)

and c = [1;0n−1]. The structure of the standard cone pro-grammingPcone is characterized by the following data

n= 1 + L+K +NK, (61)

m= (L+K) + (NK + 1) +

L∑

l=1

(KNl + 1)+K(K + 2),(62)

K=Q1 × · · · × Q1

︸ ︷︷ ︸

L+K

×QNK+1 ×QKN1+1 × · · · × QKNL+1

︸ ︷︷ ︸

L

×

QK+2 × · · · × QK+2

︸ ︷︷ ︸

K

, (63)

whereK is the Cartesian product of2(L+K)+1 second-order

ones, andA andb are given as follows:

A =

1. . .

11 −β1r

T1

. . ....

1 −βKrTK−1

−INK

−1−D1

......

...−1

−DL

−1−C1

......

−1−CK

,b =

√P1

...√PL

0...00

0NK

00KN1

...0

0KN1

0g1

...0gK

,(64)

respectively.

C. Matrix Stuffing

Given a problem instanceP, to arrive at the standard coneprogram form, we only need to copy the parameters of themaximum transmit powerPl’s to the data of the standard form,i.e.,

√Pl’s in b, copy the parameters of the SINR thresholds

γ to the data of the standard form, i.e.,βk ’s in A, and copythe parameters of the channel realizationshk ’s to the data ofthe standard form, i.e.,rk ’s andCk ’s in A. As we only needto perform copying the memory for the transformation, thisprocedure can be very efficient compared to the state-of-the-art numerical based modeling frameworks like CVX.

D. Extension to the Complex Case

For hk ∈ CN ,vi ∈ CN , we have

hH

kvi =⇒[R(hk) −J(hk)J(hk) R(hk)

]

︸ ︷︷ ︸

hk

T [R(vi)J(vi)

]

︸ ︷︷ ︸

vi

, (65)

wherehk ∈ R2N×2 and vi ∈ R2N . Therefore, the complex-field problem can be changed into the real-field problem bythe transformations:hk ⇒ hk andvi ⇒ vi.

ACKNOWLEDGMENT

The authors would like to thank Dr. Eric Chu for hisinsightful discussions and constructive comments relatedtothis work.

14

REFERENCES

[1] Y. Shi, J. Zhang, K. B. Letaief, B. Bai, and W. Chen, “Large-scale convexoptimization for ultra-dense Cloud-RAN,”IEEE Wireless Commun.Mag., to appear, 2015.

[2] China Mobile, “C-RAN: the road towards green RAN,”White Paper,ver. 3.0, Dec. 2013.

[3] Y. Shi, J. Zhang, and K. B. Letaief, “Group sparse beamforming forgreen Cloud-RAN,”IEEE Trans. Wireless Commun., vol. 13, pp. 2809–2823, May 2014.

[4] S. Zhou, M. Zhao, X. Xu, J. Wang, and Y. Yao, “Distributed wirelesscommunication system: a new architecture for future publicwirelessaccess,”IEEE Commun. Mag., vol. 41, no. 3, pp. 108–113, 2003.

[5] A. B. Gershman, N. D. Sidiropoulos, S. Shahbazpanahi, M. Bengts-son, and B. Ottersten, “Convex optimization-based beamforming: Fromreceive to transmit and network designs,”IEEE Signal Process. Mag,vol. 27, no. 3, pp. 62–75, 2010.

[6] Z.-Q. Luo, W.-K. Ma, A.-C. So, Y. Ye, and S. Zhang, “Semidefiniterelaxation of quadratic optimization problems,”IEEE Signal Process.Mag., vol. 27, pp. 20–34, May 2010.

[7] E. Bjornson and E. Jorswieck, “Optimal resource allocation in coordi-nated multi-cell systems,”Found. Trends Commun. Inf. Theory, vol. 9,pp. 113–381, Jan. 2013.

[8] H. Dahrouj and W. Yu, “Coordinated beamforming for the multicellmulti-antenna wireless system,”IEEE Trans. Wireless Commun., vol. 9,pp. 1748–1759, Sep. 2010.

[9] S. P. Boyd and L. Vandenberghe,Convex optimization. Cambridgeuniversity press, 2004.

[10] R. Zakhour and S. V. Hanly, “Base station cooperation on the downlink:Large system analysis,”IEEE Trans. Inf. Theory, vol. 58, pp. 2079–2106,Apr. 2012.

[11] C. Shen, T.-H. Chang, K.-Y. Wang, Z. Qiu, and C.-Y. Chi, “Dis-tributed robust multicell coordinated beamforming with imperfect CSI:an ADMM approach,”IEEE Trans. Signal Process., vol. 60, pp. 2988–3003, Jun. 2012.

[12] Y. Shi, J. Zhang, and K. Letaief, “Robust group sparse beamforming formulticast green Cloud-RAN with imperfect CSI,”IEEE Trans. SignalProcess., to appear, 2015.

[13] J. Cheng, Y. Shi, B. Bai, W. Chen, J. Zhang, and K. Letaief,“Group sparse beamforming for multicast green Cloud-RAN via par-allel semidefinite programming,”IEEE Int. Conf. on Commun. (ICC),London, UK, 2015.

[14] Y. Shi, J. Zhang, and K. Letaief, “Optimal stochastic coordinatedbeamforming for wireless cooperative networks with CSI uncertainty,”IEEE Trans. Signal Process., vol. 63, pp. 960–973, Feb. 2015.

[15] J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimizationover symmetric cones,”Optim. Methods Softw., vol. 11, no. 1-4, pp. 625–653, 1999.

[16] K.-C. Toh, M. J. Todd, and R. H. Tutuncu, “SDPT3—a MATLABsoftware package for semidefinite programming, version 1.3,” Optim.Methods Softw., vol. 11, no. 1-4, pp. 545–581, 1999.

[17] E. D. Andersen and K. D. Andersen, “The mosek interior point opti-mizer for linear programming: an implementation of the homogeneousalgorithm,” in High performance optimization, pp. 197–232, Springer,2000.

[18] Y. Nesterov, A. Nemirovskii, and Y. Ye,Interior-point polynomialalgorithms in convex programming, vol. 13. SIAM, 1994.

[19] CVX Research, Inc., “CVX: Matlab software for disciplined convexprogramming, version 2.0 (beta),” 2013.

[20] J. Lofberg, “YALMIP: A toolbox for modeling and optimization in MAT-LAB,” in IEEE Int. Symp. Computer-Aided Control Systems Design,(Taipei, Taiwan, R.O.C.), pp. 284–289, Sep. 2004.

[21] E. Chu, N. Parikh, A. Domahidi, and S. Boyd, “Code generationforembedded second-order cone programming,” inControl Conference(ECC), 2013 European, pp. 1547–1552, Jul. 2013.

[22] Y. Shi, J. Zhang, and K. B. Letaief, “Scalable coordinated beamformingfor dense wireless cooperative networks,” inProc. IEEE Global Com-munications Conf. (GLOBECOM), Austin, TX, 2014.

[23] E. Bjornson, M. Bengtsson, and B. Ottersten, “Optimal multiuser trans-mit beamforming: A difficult problem with a simple solution structure[lecture notes],”IEEE Signal Process. Mag., vol. 31, pp. 142–148, Jul.2014.

[24] W.-C. Liao, M. Hong, Y.-F. Liu, and Z.-Q. Luo, “Base station activationand linear transceiver design for optimal resource management in het-erogeneous networks,”IEEE Trans. Signal Process., vol. 62, pp. 3939–3952, Aug. 2014.

[25] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributedoptimization and statistical learning via the alternatingdirection methodof multipliers,” Found. Trends in Mach. Learn., vol. 3, pp. 1–122, Jul.2011.

[26] S. K. Joshi, M. Codreanu, and M. Latva-aho, “Distributed resourceallocation for MISO downlink systems via the alternating directionmethod of multipliers,”EURASIP Journal on Wireless Communicationsand Networking, vol. 2014, pp. 1–19, Jan. 2014.

[27] J. Zhang, R. Chen, J. G. Andrews, A. Ghosh, and R. W. Heath, “Net-worked MIMO with clustered linear precoding,”IEEE Trans. WirelessCommun., vol. 8, pp. 1910–1921, Apr. 2009.

[28] T. Yoo and A. Goldsmith, “On the optimality of multiantenna broad-cast scheduling using zero-forcing beamforming,”IEEE J. Sel. AreasCommun., vol. 24, pp. 528–541, Mar. 2006.

[29] E. Smith,On the optimal design of continuous processes. PhD thesis,Imperial College London (University of London), 1996.

[30] M. C. Grant and S. P. Boyd, “Graph implementations for nonsmoothconvex programs,” inRecent advances in learning and control, pp. 95–110, Springer, 2008.

[31] Y. Ye, M. J. Todd, and S. Mizuno, “AnO(√nL)-iteration homogeneous

and self-dual linear programming algorithm,”Math. Oper. Res., vol. 19,no. 1, pp. 53–67, 1994.

[32] Y.-F. Liu, Y.-H. Dai, and Z.-Q. Luo, “Coordinated beamforming forMISO interference channel: Complexity analysis and efficient algo-rithms,” IEEE Trans. Signal Process., vol. 59, pp. 1142–1157, Mar.2011.

[33] H. Tuy, “Monotonic optimization: Problems and solution approaches,”SIAM Journal on Optimization, vol. 11, no. 2, pp. 464–494, 2000.

[34] E. A. Jorswieck, E. G. Larsson, and D. Danev, “Complete characteriza-tion of the pareto boundary for the MISO interference channel,” IEEETrans. Signal Process., vol. 56, pp. 5292–5296, Oct. 2008.

[35] R. Zhang and S. Cui, “Cooperative interference management with MISObeamforming,” IEEE Trans. Signal Process., vol. 58, pp. 5450–5458,Oct. 2010.

[36] J. Mattingley and S. Boyd, “CVXGEN: a code generator for embeddedconvex optimization,”Optimization and Engineering, vol. 13, no. 1,pp. 1–27, 2012.

[37] T. A. Davis, Direct Methods for Sparse Linear Systems. Society forIndustrial and Applied Mathematics, 2006.

[38] E. Chu, B. O’Donoghue, N. Parikh, and S. Boyd, “A primal-dualoper-ator splitting method for conic optimization,” 2013, [Online]. Available:http://web.stanford.edu/∼boyd/papers/pdos.html

[39] B. O’Donoghue, E. Chu, N. Parikh, and S. Boyd, “Conic opti-mization via operator splitting and homogeneous self-dualembed-ding,” arXiv preprint arXiv:1312.3039, 2013, [Online]. Available:http://arxiv.org/abs/1312.3039

[40] A. Wiesel, Y. Eldar, and S. Shamai, “Linear precoding via conicoptimization for fixed MIMO receivers,”IEEE Trans. Signal Process.,vol. 54, pp. 161–176, Jan. 2006.

[41] T. Goldstein, B. O’Donoghue, S. Setzer, and R. Baraniuk, “Fast alternat-ing direction optimization methods,”SIAM Journal on Imaging Sciences,vol. 7, no. 3, pp. 1588–1623, 2014.

[42] R. Vanderbei, “Symmetric quasidefinite matrices,”SIAM J. Optim.,vol. 5, no. 1, pp. 100–113, 1995.

[43] N. Parikh and S. Boyd, “Proximal algorithms,”Foundations and trendsin optimization, vol. 1, Jan. 2014.

[44] J. Poulson, B. Marker, R. A. van de Geijn, J. R. Hammond, and N.A.Romero, “Elemental: A new framework for distributed memorydensematrix computations,”ACM Transactions on Mathematical Software(TOMS), vol. 39, p. 13, Feb. 2013.

[45] J. Mattingley and S. Boyd, “Real-time convex optimization in signalprocessing,”IEEE Signal Process. Mag, vol. 27, no. 3, pp. 50–61, 2010.

[46] A. Domahidi, E. Chu, and S. Boyd, “ECOS: An SOCP solver forembedded systems,” inControl Conference (ECC), 2013 European,pp. 3071–3076, IEEE, 2013.

[47] S. Diamond, E. Chu, and S. Boyd, “CVXPY: A Python-embeddedmodeling language for convex optimization, version 0.2,” May 2014.

[48] J. Zhang and J. G. Andrews, “Adaptive spatial intercell interference can-cellation in multicell wireless networks,”IEEE J. Sel. Areas Commun.,vol. 28, pp. 1455–1468, Dec. 2010.

[49] C. B. Peel, B. M. Hochwald, and A. L. Swindlehurst, “Avector-perturbation technique for near-capacity multiantenna multiusercommunication-part I: channel inversion and regularization,” IEEETrans. Commun., vol. 53, pp. 195–202, Jan. 2005.

[50] F. Rusek, D. Persson, B. K. Lau, E. Larsson, T. Marzetta, O. Edfors,and F. Tufvesson, “Scaling up MIMO: Opportunities and challenges with

15

very large arrays,”IEEE Signal Process. Mag., vol. 30, no. 1, pp. 40–60,2013.