Myopic and Optimal Distributed Routing

Wei Chen and Sean P. MeynECE & CSL University of Illinois

MaxWeight: Issues Raised before and after 2007

Routing requires information. In the MaxWeight policy, this information is obtained through local queue length values. This can lead to irrational behavior when information is scarce

Better performance through geometric insight and better information sharing

Issues addressed by 12/07

• Why does MW work? Answer led to h-MW policies introduced in CTCN – Performance evaluation and approximate optimality

• Improved robustness

• Specialization to routing

Example: (and Leith, 2007 Subramanian, submitted). MaxWeight or Backpressure routing will send packets upstream!

Issues addressed in 2008

• Understanding dynamic multi-commodity flow problem, providing workload relaxation

• Distributed implementation in wireless setting based on message passing

• Simulation and analysis

Context of the reported work

Upper bounding techniques Assumptions

Novel techniques Realistic

New application of old techniques Unrealistic but robust

Unconsummated union – applicable “layers”

Application layer Overhead, feedback

Transport layer Ignored

Routing layer Considered

Scheduling/queuing layer Topologies

Physical layer Canonical problems

Dynamics Infinite / scaling law

Arbitrary movement Through separation of time scales

Finite but arbitrary

Random waypoint Other considerations

No mobility Low latency as well as optimal throughput

Capacity-achieving techniques and architectures Proposed network coding application: Relaxations are required to reduce complexity

Yes, new techniques Adaptation

No, old techniques Mobility a potential extension

Work shows how important global information can be used if available. Generally, amount of global information required for approximate optimization is low

• Performance impact of consensus algorithms, and other learning schemes used in this project

• Simulation study in realistic wireless setting

• Full performance analysis of multiple bottlenecks

• Integration with Network Coding projects

What is the state of the art and what are its limitations? MW routing inflexible with respect to performance improvement; requires global information (wireless); very little flexibility

New h-MW technique requires information, but is highly flexible, and approximately optimal in certain cases

Optimizing MaxWeight for Routing

KEY NEW INSIGHTS:Goal: develop insights of 07

• Distributed implementation via message-passing

• New learning technique based on Oja’s algorithm

• Learning cuts/workload through stochastic approximation techniques

• Construction of h for models with multiple bottlenecks based on DP insights for w. relaxation

• Un-consummated union challenge: Integrate coding and resource allocation

• Generally, solutions to complex decision problems should offer insight

Algorithms for dynamic routing: Visualization and Optimization EN

D-O

F-P

HA

SE G

OA

LC

OM

MU

NIT

Y C

HA

LLEN

GE

ACHIEVEMENT DESCRIPTION

STA

TU

S Q

UO

NEW

IN

SIG

HTS

MAIN RESULT: Decentralized implementation, use of

consensus algorithms

Simulation study for wireless model

Multiple cuts/bottlenecks addressed

HOW IT WORKS: Step 1: Estimation of workload – generalization

of network cuts

Step 2: Estimation of congestion

Step 3: Choice of h0 - piecewise quadratic

Special case: Single dominant destination gives h0 quadratic function of workload, cost, and effective cost w.r.t. workload relaxation

Simulations for Multiple Traffic Streams

• Network approximately 100 nodes. Single destination, multiple sources

• MaxWeight compared to policy based on logarithmic perturbation of

• Simulation for high load: 50% improvement over greedy, 25% over MW

Performance improves for functions h that more closely approximate DP solution

Greedy Approximation of DP solution

Source of performance loss in MW :

Cycling back and forth across bottleneck network cut leads to higher workload values

PERFORMANCE

Without attention to bottlenecks, a decentralized routing algorithm will create inefficiency through cycling.

Stability has been established for logarithmic perturbation - results from simulation studies give optimism. Analysis is required.

LEARNING

Distributed learning of workload in a dynamic setting

Distributed learning of control parameters

Learning of channel characteristics

Summaries and challenges

Research bottlenecks: Learning dynamic bottleneck location and workload

CHALLENGES:

Implementation requires learning on several levels – from global architecture, to local policy parameters.

CONCLUSIONS:

In wireless model with multiple bottlenecks we again see 25% delay improvement in simulation.

Logarithmic perturbation gives universally stabilizing policies, with no evident performance loss in experiments.

Learning insights: 1. Message passing can lead to performance loss - analysis underway. 2. Stochastic approximation Oja’s algorithm completed. Application to network decomposition. 3. Local capacity learning investigated.

SCIENTIFIC FOUNDATIONS

Stochastic Lyapunov theory + workload relaxation to approximate DP solution

References

• S. P. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. To appear, SIAM J. Control Opt. (Preliminary version to appear at the 46th IEEE Conference on Decision and Control, December 2007).

• W. Chen and S. P. Meyn Optimizing MaxWeight For Routing. In preparation.

• S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press, 2007. Chinese edition to appear.

• S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability. CUP Mathematical Library edition, to appear 2008.

• C. Lin, V. V. Veeravalli, and S. P. Meyn. Distributed beamforming with feedback: Convergence analysis. Submitted to the IEEE Transactions on Information Theory. http://arxiv.org/abs/0806.3023, 2008 (adaptation at nodal level)

• V. Borkar and S. Meyn. Oja’s algorithm for graph clustering and Markov spectral decomposition. In Proceedings of VALUETOOLS, Athens, Greece, 2008 (adaptation at network level)

References in on-going research: CTCN and …

• Iterative Scheduling Algorithms, M. Bayati, B. Prabhakar, D. Shah and M. Sharma, Proceedings of IEEE Infocom 2007

• Distributed Subgradient Methods for Multi-agent Optimization Angelia Nedic and Asuman Ozdaglar. To appear in IEEE Transactions on Automatic Control.

• Polynomial Complexity Algorithms for Full Utilization of Multi-hop Wireless Networks Atilla Eryilmaz, Asuman Ozdaglar and Eytan Modiano. INFOCOM 2007.

• Decentralized implementation, use of consensus algorithms

• Wireless models: Apply D. Shah’s insights on maxproduct convergence

• Full analysis of multiple bottlenecks

• Integration with Network Coding projects: Can we code around network hot-spots?

What is the state of the art and what are its limitations? MW routing inflexible with respect to performance improvement

MW corresponds to h-myopic, with h quadratic. Key geometric property of quadratic identified by Meyn prior to July meeting.

Optimizing MaxWeight: From 12/07 Meeting

KEY NEW INSIGHTS:• New perturbation technique:

• Application to routing & refinements for decentralization

• Heavy traffic optimality

• Taylor series approximation gives interpretation as adaptive MaxWeight - Diagonal matrix adapts to varying congestion

• Un-consummated union challenge: Integrate coding and resource allocation

• Generally, solutions to complex decision problems should offer insight

Algorithms for dynamic routing: Visualization and Optimization EN

D-O

F-P

HA

SE G

OA

LC

OM

MU

NIT

Y C

HA

LLEN

GE

ACHIEVEMENT DESCRIPTION

STA

TU

S Q

UO

NEW

IN

SIG

HTS

MAIN RESULT: h-myopic policy is universally stabilizing

Application to policy synthesis for approximately optimal performance (delay or backlog) in heavy traffic, with log regret

HOW IT WORKS: Step 1: Estimation of network cuts

Step 2: Estimation of congestion on either side

Step 3: Choice of h0 - piecewise quadratic

Special case: Single dominant destination gives h0 quadratic function of workload, cost, and effective cost w.r.t. workload relaxation

Decentralized implementation appears feasible.

Numerical study underway

Investigate performance and feasibility: 100 nodes, multiple arrivals. Only wireline models investigated to-date.

Excellent performace as predicted by theory