online distributed optimization via dual averaging · 2017. 7. 29. · online distributed...

Online Distributed Optimizationvia Dual Averaging

Saghar Hosseini, Airlie Chapman and Mehran Mesbahi

Robotics, Aerospace, and Information Networks (RAIN) Lab

University of Washington

Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 1 / 13

Motivation: Distributed Sensor Networks

Forest temperature detection Collect atmospheric data

Given accurate observation models (cost functions) and convergence time(offline) the problem is traditionally solved by distributed optimization

What if the observation models are largely uncertain and solutions arerequired in real-time?

... Online Distributed Optimization


Outline

Problem Statement

Previous Work

Online Distributed OptimizationAlgorithm

Main Results

Application: Estimation in aDistributed Sensor Network

Conclusion


Problem Statement

Problem: Minimize a global cost over a network of n agents:

ft(x) =1

n

n

∑i=1

ft,i (x) subject to x ∈ χ

Each ft,i (xi (t)) : Rd → R is a convex cost on agent i ’s x at time t, xi (t), andevolves over time in an unpredictable manner

Easily projectable constraint set χGraph G = (V ,E ) represent the communication constraints withV = {1,2, ...n} agents

How do we quantify performance over time for an agent’s choices of xi (t)?


Regret Definition

The regret is the difference between the cost of the sequence of decisions {xi (t)}generated by the algorithm and the performance of the best fixed decision inhindsight x∗

The regret due to agent i ’s action

RT (x∗,xi ) =

T

∑t=1

(ft(xi (t))− ft(x∗)) =T

∑t=1

n

∑i=1

(ft,i (xi (t))− ft,i (x∗))

Online Algorithm’s Objective:

Sublinear RT orRT/T → 0,

i.e., “on average” (xi (1),xi (2), . . . ,xi (T )) performs as well as (x∗,x∗, . . . ,x∗)


Previous Work

Online Distributed Gradient Descent Method

Yan et. al (2010): Distributed Autonomous Online Learning: Regrets andIntrinsic Privacy-Preserving Properties

Directed weighted graphsStrong convex cost functions : Regret = O(log(T ))Convex functions: Regret = O(

√T )

The effect of new information is diminishing over time

Distributed Dual Averaging Method

Duchi et. al (2012): Dual Averaging for Distributed Optimization:Convergence Analysis and Network Scaling

Offline problemEffect of different types of graph on convergenceNetwork and cost uncertainties

Raginsky et. al (2011): Decentralized Online Convex Programming withLocal Information

Chain graphs with radius of neighborhood r

Regret = O(√T ) for a growing graph (r ≥ (3log(T )+log(2))

log(2∗T 3/2/(2T 3/2−1)) )


Online Distributed Optimization Algorithm

Communication matrix (doubly stochastic): P = [Pj ,i ]

Non-increasing sequence of positive functions: α(t)Proximal function: ψ(x) : χ → R, strongly convex,ψ ≥ 0, and ψ(0) = 0Projection function:

Πψχ (z(t),α(t)) = arg min

x(t)∈χ

{〈z(t),x〉+ 1

α(t)ψ(x)

}

Online Distributed Dual Averaging (ODD) Algorithm

For t = 1 to T , and each agent i , provided a subgradient gi (t) ∈ ∂ ft,i (xi (t))

zi (t + 1) = ∑j∈N(i)

Pj ,izj(t) +gi (t) (Dual update)

xi (t + 1) = Πψχ (zi (t + 1),α(t)) (Primal Projection)

x̂i (t + 1) =1

t + 1

t+1

∑s=1

xi (s) (Optional: running average)


Main Results

Theorem

Given the ψ(x∗)≤ R2 and α(t) = k/√t,

supfT∈F

RT (x∗,xi ) =O

((kL2 +

R2

k+ 2kL2

(3√n

1−σ2(P)+ 6

))√T

),

where σ2(P) is the second largest singular value of P, n is the number of nodes,and fi ’s are L-Lipschitz

For P = I − 1ε diag(v)L(G) is doubly stochastic whereG is strongly connectedvTL(G) = 0 with positive vector v = [v1,v2, . . . ,vn]Tε ∈ (maxi∈V (vidi ) ,∞), where di is the in-degree of G

For special P, 1−σ2(P) ∝ λ2(G) a well known connectivity measure forundirected graphs


Application: Estimation in a Distributed Sensor Network

Sensors are estimating a random vector θ ∈ χ ={

θ ∈ Rd |‖θ‖2 ≤ θmax}

zt,i (θ) : Rd → Rpi is the observation vector for the ith sensor observing θ attime t

The sensor is modeled as hi (θ) = Hiθ where Hi ∈ Rpi×d is the observationmatrix of sensor i , and ‖Hi‖1 ≤ hmax, for all sensors i


Application: Estimation in a Distributed Sensor Network

Goal: Find the argument θ that minimizes thecost function

ft(θ) =1

n

n

∑i=1

ft,i (θ)

where

ft,i (θ) =1

2‖zt,i −Hiθ‖22

∂ ft,i (θ) = HTi (zt,i −Hiθ)

Best fixed strategy: The centralized optimalin hindsight is

θ ∗ =1

T

T

∑t=1

(n

∑i=1

HTi Hi

)−1(n

∑i=1

HTi zt,i

)


Simulation Run

100 sensor nodes distributed across map measuring a collection of local cells

Best Fixed in Hindsight θ ∗ ODD Algorithm θi (t)


temp_profile_sensor_50.aviMedia File (video/avi)

Results

100

101

102

103

104

10−2

10−1

100

101

102

103

T

RT(x

∗,x

1)

SimulationBounds

200 400 600 800 1000 1200 1400 1600 1800 2000

100

101

RT(x

∗ ,x1)/√T

T

PathDirected CycleRandom TreeRandomRandom regular

Graph σ2(P)Path 0.9993

Directed Cycle 0.9990Random Tree 0.9954

Erdos-Renyi, p = 0.08 0.8169Random k-regular, k = 6 0.5610


Conclusion

Extended dual averaging method to an distributed online formulation withO(√T ) regret

Future Work:

Investigate favorable graph characteristics for the online framework improvingthe regret bound

Apply online approach to traditionalproblems in multi-agent networks

Distributed estimation in adversarialenvironments, in the presence ofmistrust and jammingEnergy aware sensingOnline distributed energymanagement/pricing


online distributed optimization via dual averaging · 2017. 7. 29. · online distributed...

Documents