Machine Learning in 6G Systems:

An Overview

Oulu Webinar

Walid SaadElectrical and Computer Engineering Department,

Network sciEnce, Wireless, and Security (NEWS) Group

Virginia Tech

Email: walids@vt.edu

Group: http://www.netsciwis.com

Personal: http://resume.walid-saad.com

2

Why ML in 6G?

Why do we need ML in 6G?

Communication and autonomy are intertwined – ML/AI will

already be there!

Communication must be self-sustaining – system should be able

operate itself by itself: less rigid protocols!

Communication should be proactive – learn the user (beyond

basic caching) or learn the comm. medium!

Three use cases for AI/ML

Data analytics for proactiveness (started in 5G and continues)

Artificially-intelligent optimization and control (theoretical in

5G+ and can potentially replace standardized protocols in 6G)

Edge learning and small distributed data (6G)

3

Use Case 1: Data Analytics

Large-scale wireless networks or…

…………data hurricane!

Massive volumes of data will be generated from sensors,

wearables, surveillance cameras, etc.

How to make sense of this hurricane of chaos?

Analyze, predict, and adapt to the wireless user behavior,

user-centric wireless (e.g., virtual reality)

4

Use Case 2: Autonomy and AI-enabled Optimization

Connected devices (and people) will make use of big data

to act, react, and adapt intelligently

ML/AI are not mere data analytics tools

They will instill intelligence into the wireless network design itself:

AI-driven control and optimization

The idea of SON will no longer be simple optimization, it becomes AI-

driven self-sustaining networks!

AI-enabled devices will be there, let’s use them!

Data may or may not be necessary

Intelligent optimization can be trained numerically in an online

manner, but of course data can enhance performance

Reinforcement learning is the key, let’s see a detailed example

5

Reliability and Low Latency

Communication with connected autonomy

Massive number of devices with services that are

more critical than Facebook/Instagram posts!

6

Communication Requirements

How to effectively connect all those infrastructure and

components of a city?

Communication should be high data rate – terrabytes of

data to be delivered!

Communication should be high-speed – low delay and

response time (few millisecond target!)

Communication should be pervasive – anytime,

anywhere, anyone!

Communication should be reliable – seven 9’s for 6G!

Reliable low latency communications a key in 6G

Beyond ultra reliable low latency communication (URLLC)

7

URLLC: What’s next?

URLLC has been around for a while but prior art…

Focused on IoT sensors (uplink) – autonomous vehicles/drones

are different (downlink?)!

Assumes known models for traffic (M/M/1 etc.)– latency has many

components, hard to model!

Considers slow deep reinforcement learning (DRL) – learning in

URLLC must handle extreme, rare conditions (6G feature)!

Assumes rate can be ignored – 6G applications may need some

form of rate guarantees!

Think of a new service, we can call it highly reliable low

latency communications (HRLLC)

Experienced DRL for Model-free HRLLC

8

System Model

Consider the downlink of a single-cell wireless network

whose base station is sending latency-sensitive control

message to autonomous vehicles

We consider a downlink OFDMA system with resource blocks that must be allocated

Reliability is defined as the probability of end-to-end

packet delay exceeding a threshold

We do not make any assumptions for a delay model

Delay is intrinsically hard to model, most models are often

unrealistic and have some hidden drawbacks

Delay has many components, hard to model their combination

precisely

Our goal is to solve the following problem

Explicit rate guarantees imposed

Challenging to solve because of our model-free assumption

Problem Formulation

9

Reliability

constraint

Feasibility

constraints

Rate

constraint

Total power

10

Handling Model-Free

A wireless network can empirically measure the delay

Network can “learn” the delay once it connects with a user

How? Use case 2: Reinforcement learning is natural but…

….classical solutions cannot handle the large state space

Solution: Deep reinforcement learning

Deep RL used because it is appropriate to handle our large

state space not because it is “fashionable”

Ratio of number of packets with delay excess

and total number of packets

11

Deep-RL for Model-Free HRLLC

State space: number of packets transmitted, packet size, and

channel gains

PPO: Proximal policy optimization determines target rates

Action space reducer: Deep-RL made tractable

12

Deep-RL for Model-Free HRLLC

The reward function used by deep-RL:

Theorem 1: By maximizing this reward, after convergence of

the deep-RL algorithm, the reliability of each user is

guaranteed, such that:

But, is deep-RL reliable and suitable for HRLLC?

No! Can be slow to converge and unreliable extreme cases

Solution? Use generative adversarial networks (GANs)!

Time-varying weight

that control

the reliability

13

Experienced Deep RL

Use GAN to create a “virtual environment” for training

Virtual environment is created by GAN using a mix of

(limited) real data and synthetic (simulated) data

14

Experienced Deep RL

GAN-based refiner

Proposed by Apple for

computer vision

Inputs

Unlabeled real data

Synthetic model data

Output

Refined (and larger) dataset

that includes new network

conditions (extreme events)

that can train your deep RL

15

Simulation Results

We use a real dataset with specific packet sizes and

interarrival times (with some modification)

16

Simulation Results

Experience allows a very smooth handling of extreme

events compared to vanilla deep-RL

See paper: A. T. Z. Kasgari, W. Saad, M. Mozaffari, and H. V. Poor, "Experienced Deep

Reinforcement Learning with Generative Adversarial Networks (GANs) for Model-Free Ultra Reliable

Low Latency Communication", IEEE Transactions on Communications, to appear, 2020.

Open Questions

Use of a neural network instead of PPO to enhance/reduce

possible overhead

More rigorous approach to action space reduction

Extensions to multi-agent scenario is a very interesting

aspect (how to look at scale in that case)

We used a deep Q network

Can we design new deep RL architectures with a more

interesting backbone ANN? (see M. Chen, W. Saad, et al.)

More control on the GAN?

Useof experienced deep RL in other 6G problems17

Device Device Device Device

Federated AI (current)Collaborative AI (future)Classical AI (past)

• Cloud-AI w/ dumb devices

• All data in the cloud

• Classification/inference at the cloud

• No privacy

• No cloud and/orinfrastructure needed

• Collective intelligence

• Privacy-preserving

• Cloud-AI + on-device AI = mobile AI

• Use the cloud but smartly

• Privacy-preserving

Use Case 3: Edge AI

Massive Small Data and Distributed Machine learning

Cloud big data was the norm but data is private and

distributed

Can users collaboratively

learn a task of interest?

Learning (at the edge) to

communicate?

Communication for

learning (joint design)?

Areas ripe for exploration

in 6G

Joint FL-Wireless Design

20

Since the global and local FL model

parameters need wireless link exchanges,

then wireless transmission errors will

impact FL performance

The base station (BS) must update the

global FL model as it receives all of the

local FL model transmitted from the

users. Hence, transmission delay and

energy consumption must be

considered.

Communication to support FL!

FL training process in a wireless network

System Model

21

Consider a cellular network with one BS and U users that

are engaged in an FL algorithm

Uplink transmissions are used for sending the local FL

model parameters

Downlink transmissions are used for updating the global

model parameters from the BS to the users

M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor,

and S. Cui, “A Joint Learning and Communications

Framework for Federated Learning over Wireless

Networks”, IEEE Transactions on Wireless

Communications, to appear, 2020.

System Model

22

Local FL model including transmission errors

where

With packet error rates, the global FL model becomes:

PER over RB n

Packet error rate (PER)

If a user i is

discarded because

of high PER then:

Problem Formulation

23

RB allocation matrix

User association vector

Delay constraint

Energy constraint

Power allocation matrix

Theorem 1: An upper bound on the convergence point

of FL over wireless networks can now be found:

This is a key characterization of FL performance over a

wireless network

Convergence affected by PER and user association: wireless

network must be reliable enough to support effective FL

We used full gradient descent but results extendable

Global FL Model Analysis

24

Total samples

Number of samples

of user iGlobal model at

convergence

FL Optimization

25

Based on Theorem 1, we can solve the original

optimization problem and find the optimal uplink

transmit power of a user i for a given user selection a

and uplink RB allocation vector ri:

where satisfies the equality

We can also find the optimal uplink RB allocation by

using a very simple Hungarian method

This is a different view of learning: We optimize a

wireless network to serve edge FL!

Joint learning and communication design: A new paradigm

Simulation Results

Value of the

loss function

as the number

of samples per

user

varies.

Simulation Results

27

Digit above: Identification result of the proposed FL

Digit below: Identification result of the standard FL

Red digit: Identification error

An example of

implementing

FL algorithms

for handwritten

digit

identification.

Remarks

We now can ask: What can 6G do for FL, rather than the

reverse?

We cannot yet give a direct answer but…

We need reliability and low latency for FL convergence

Data rate may also be needed (some FL models are large!)

Joint learning and communications is hot for beyond 5G

How to maintain high accuracy, low convergence time, and

error-free communications, jointly?

Generative models (brainstorming GANs: A. Ferdowsi and W.

Saad)

Beyond FL: full distributed/collaborative learning 28

29

What’s next in AI?

Issues of reliability (particularly at high frequencies):

Experienced RL is one idea, but are there others

Distributed learning – how accurate can we go?

Beyond standard learning:

Learning is mostly training-based, can we get rid of training?

We are pursuing several ideas in this context: learn one task

and map it to another (like humans do), this includes self-

learning and beyond meta-learning

Theoretical foundations for RL/explainable AI

Topological aspect (persistent homology)

Rich field of application in 6G with specific needs

Acknowledgement Acknowledgement to students/collaborators: Ali Taleb Zadeh Kasgari,

Mingzhe Chen, Tengchan Zeng, Anibal Sanjab, Christina Chaccour,

Omid Semiari, Mohammad Mozaffari, Gilsoo Lee, Mehdi Bennis, and

Merouane Debbah

More on 6G? W. Saad, M. Bennis, and M. Chen, "A Vision of 6G Wireless Systems:

Applications, Trends, Technologies, and Open Research Problems", IEEE Network,

2020 (February 2019 pre-print on Arxiv).

More on AI? M. Chen, U. Challita, W. Saad, C. Yin, and M. Debbah, "Artificial

Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial", IEEE

Communications Surveys and Tutorials, vol. 21, no. 4, pp. 3039-3071, 2019

More on FL? M. Chen, H. V. Poor, W. Saad, and S. Cui, "Wireless

Communications for Collaborative Federated Learning", IEEE Communications

Magazine, to appear, 2020.

More on self-learning? Y. Xiao, G. Shi, Y. Li, W. Saad, and H. V. Poor, "Towards

Self-learning Edge Intelligence in 6G", IEEE Communications Magazine, Special

Issue on Communication Technologies for Efficient Edge Learning, to appear, 2020.30

Finally….

Thank You