Multiple-access Communication in Networks A Geometric View

W. Chen & S. MeynDept ECE & CSL University of Illinois

• Implementation – Consensus algorithms & Information distribution

• Adaptation – Reinforcement learning techniques

• Integration with Network Coding projects: Code around network hot-spots

What is the state of the art and what are its limitations? Notes from Austin: MW routing inflexible, and does not easily incorporate multi-access capacity region in wireless.

Workload relaxation techniques: Tremendous value for policy synthesis based on dynamic hot-spots in the network

Can these techniques be extended to wireless models?

Relaxation Techniques for Net Opt W. Chen & S. Meyn

KEY NEW INSIGHTS:• Extend to wireless? YES Geometric picture is very different. Interpretation: The number of resources is infinite

• Structure of optimal solution to relaxation is very simple, even for very complex networks

• New application of relaxation: Q-learning and TD-learning for routing and power control

• Un-consummated union challenge: Integrate coding and resource allocation

• Generally, solutions to complex decision problems should offer insight

Algorithms for dynamic routing: Visualization and Optimization EN

D-O

F-P

HA

SE G

OA

LC

OM

MU

NIT

Y C

HA

LLEN

GE

ACHIEVEMENT DESCRIPTION

STA

TU

S Q

UO

NEW

IN

SIG

HTS

MAIN RESULT:

HOW IT WORKS: Step 1: Estimate capacity region near

estimated allocation rate vector

Step 2: Construct half-space relaxation

Step 3: Optimal policy for relaxation: Buffer priorities, based on coefficients of normal vector

Numerical findings: With many flows, the rate region appears smooth even in a static interference model

Dynamics of 720 queues Half space relaxation provides:

Impact: Network cut is no longer a useful concept

• Lower bound on performance and

• Tools for policy synthesis

Infinite complexity leads to simple solution:

Where to focus attention for coding and routing?

Understanding MANETs – where do we direct genius for coding and control?

Issues addressed:

• Where should effort be directed in coding and control for complex networks?

• Performance evaluation: Lower bounds, and approximate optimality

• Special attention to issues surrounding MANETs:

Multiple access phenomena and fading

Message from 15 years of research: Achieving stability is possible using very simple routing schemes.

Implementation in multiple access settings possible with a bit of genius

Lacking: Methods to improve delay performance, and methods to make appropriate tradeoffs between throughput and delay.

Decision & Control Perspective

Understanding MANETs – Relaxations capture essential constraints and dynamics

D&C Perspective for networks

1.Simple, idealized model is the dynamic fluid model

2.Further simplification to obtain the workload relaxation

3.Lower bounds on performance, and control solutions from the relaxation

4.Lyapunov based design constitutes translation to network (e.g. h-MaxWeight).

D&C Perspective:

Obtain the simplest model that captures essential constraints and dynamics

Design highly robust control solution for the simple model

Translate design to the relatively complex system

Example: Dynamic Power Control

Power control solved using fluid model + reinforcement learning techniques

Dynamic speed scaling

1. Model: Fluid and stochastic model for arrivals, and controlled service rate

2. Further relaxation not required in single link model

3. Solve DP equation for fluid model

4. Solution to fluid model used to construct architecture for reinforcement learning

Approximate Dynamic Programming using Fluid and Diffusion Approximations with Applications to Power Management

Input power as function of queue length. Policy for fluid model closely approximates the optimal average delay policy for the discrete/stochastic model

Optimal control solution for fluid model gives perfect architecture for on-line learning/on-line optimization

Example: h-MaxWeight Policy

Optimal control of complex routing models solved using workload relaxation

h-MaxWeight policy - introduced in ITMANET project

1. Model: Fluid and stochastic model for arrivals, and controlled service rate

2. Workload relaxations – dynamic generalization of network cuts

3. Optimal control for relaxation is simple

4. Breakthrough: Translation using Lyapunov function

Extension to MANETs?

How to cope with infinite complexity in Interference models?

Issues

Complexity from fading – interpreted as infinite resources in a wireless multiple access setting

Relaxation in previous work relied on a dominant face in the capacity region. For MANET models this region is smooth

TDMA – complexity is nearly infinite for multi-hop interference models. Resulting capacity region again appears smooth

Complexity Results in Simplicity

Conclusion: Half space relaxation is more easily justified in MANETs

D&C Approach

Step 1: First identify or approximate rate region near desired operating point. This is the basis of the dynamic fluid model

Step 2: Relaxation is again justified through separation of time scales

Step 3: Policy synthesis and translation as in 2008 result Step 4: Expand capacity

region at hot spots through network coding

Challenges

CAN WE LEARN? Critical information for optimization is easy to identify. How can this information be shared?

CAN WE CODE? With the identification of dynamic bottlenecks, it is then evident where the capacity region can be improved.

Summaries and challenges

KEY CONCLUSION

Complexity in MANETs actually results in a simple model description

References

• S. Meyn. Stability and asymptotic optimality of generalized MaxWeight policies. SIAM J. Con Optim., 47(6):3259–3294, 2009

• W. Chen et. al. Approximate Dynamic Programming using Fluid and Diffusion Approximations with Applications to Power Management. Submitted to the 48th IEEE Conference on Decision and Control, 2009.

• W. Chen, S. P. Meyn and M. Medard. Optimal Control of Stochastic Networks. Plenary Lecture at Erlang Centennial, April 2009. Manuscript in preparation.

• S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press, 2007.

• F. S. Melo, S. Meyn, and M. I. Ribeiro. An analysis of reinforcement learning with function approximation. In Proceedings of ICML, pages 664–671, 2008.