machine learning approaches for wireless spectrum and energy …chintha/pdf/thesis/ppt_keyu.pdf ·...

38
PhD Final Oral Defense Machine Learning Approaches for Wireless Spectrum and Energy Intelligence Keyu Wu Department of Electrical and Computer Engineering University of Alberta, Edmonton, Alberta T6G 1H9, Canada September, 2018

Upload: others

Post on 06-Sep-2021

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

PhD Final Oral Defense

Machine Learning Approaches for Wireless Spectrum andEnergy Intelligence

Keyu Wu

Department of Electrical and Computer EngineeringUniversity of Alberta, Edmonton, Alberta T6G 1H9, Canada

September, 2018

Page 2: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Outlines:

1 Background and Motivations

2 Sensing-Probing-Transmitting Control of EH CR

3 Selective Transmission for EH Sensors

4 CSS under Spectrum Heterogeneity

5 Conclusion and Future Research

Page 3: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Outline

1 Background and Motivations

2 Sensing-Probing-Transmitting Control of EH CR

3 Selective Transmission for EH Sensors

4 CSS under Spectrum Heterogeneity

5 Conclusion and Future Research

Page 4: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Spectrum and Energy Consideration of WirelessCommunications

With the increase of data volume, service types and devices, wirelesscommunication face

Spectrum scarcity

Limited spectrum for wireless applications(33% for all commercial applications in 225 to 3700 MHz);Spectrum reallocation is expensive and slow(70 MHz band costs 19.8 billion dollars).

Energy issue

Huge energy consumption without careful design(communication industry may use 51% of global electricity in 2030);Difficult for powering massive amount of IoT devices.

1 / 20

Page 5: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Cognitive Radio and Energy Harvesting

Cognitive radioEnergy harvesting

2 / 20

Page 6: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

ML for wireless spectrum & energy intelligence

Machine learning, a data-driven methodology, is promising for handlingrelevant spectrum and energy management problems.

With ML as a primary tool, three research contributions are made

Joint sensing-probing-transmitting control for EH CR

Optimal transmission for an EH sensor with data priority consideration

Cooperative spectrum sensing under spectrum heterogeneity

3 / 20

Page 7: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Outline

1 Background and Motivations

2 Sensing-Probing-Transmitting Control of EH CR

3 Selective Transmission for EH Sensors

4 CSS under Spectrum Heterogeneity

5 Conclusion and Future Research

Page 8: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Sensing-probing-transmitting

Harvest: energy package arrives each time slot (uncontrollable)

Sense: measure channel output to detect and track PU activity

Probe: estimate CSI via pilot sequence

Transmit: based on CSI, adapt transmission power and send data

4 / 20

Page 9: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Sensing-probing-transmitting

Harvest: energy package arrives each time slot (uncontrollable)

Sense: measure channel output to detect and track PU activity

Probe: estimate CSI via pilot sequence

Transmit: based on CSI, adapt transmission power and send data

4 / 20

Page 10: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Sensing-probing-transmitting

Harvest: energy package arrives each time slot (uncontrollable)

Sense: measure channel output to detect and track PU activity

Probe: estimate CSI via pilot sequence

Transmit: based on CSI, adapt transmission power and send data

4 / 20

Page 11: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Sensing-probing-transmitting

Harvest: energy package arrives each time slot (uncontrollable)

Sense: measure channel output to detect and track PU activity

Probe: estimate CSI via pilot sequence

Transmit: based on CSI, adapt transmission power and send data

4 / 20

Page 12: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Sensing-probing-transmitting

Harvest: energy package arrives each time slot (uncontrollable)

Sense: measure channel output to detect and track PU activity

Probe: estimate CSI via pilot sequence

Transmit: based on CSI, adapt transmission power and send data

Problem

Based on energy status, PU activity and CSI, the node needs to decidewhether or not to sense and probe, and how much power for transmission,in order to maximize long-term throughput.

4 / 20

Page 13: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Two-stage MDP for long-term optimization

Goal: solving a policy π∗ that maximizes expected throughput

π∗ = arg maxπ

{E

[ ∞∑t=0

γtr(st , π(st))

]}

5 / 20

Page 14: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

After-state simplification

s

r

a

s'

Currentstate

After-state Nextstate

π∗(s) = arg maxa∈A(s)

{r(s, a) + γE[V ∗(s ′)|s, a]

}= arg max {immed. reward + expected furture value}= arg max

a∈A(s){r(s, a) + J∗( %(s, a)︸ ︷︷ ︸

after-state β

)}

6 / 20

Page 15: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Solve J∗ with RL

Exactly solving J∗ requires the pdfs of EH and fading processes, which canbe hard to obtain.

7 / 20

Page 16: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Solve J∗ with RL

Exactly solving J∗ requires the pdfs of EH and fading processes, which canbe hard to obtain.

We consider to (approximately) learn J∗ with RL algorithm withoutdistribution information.

EH/fadingsample RL

7 / 20

Page 17: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Learned policies

(a) Sensing-probing sub-policy (b) Transmitting sub-policy

aSP : ‘00’, no sense; ‘10’, sense but no probe; ‘11’, sense and probeEnergy for: sense, 1; probe, 2; transmit, {no tx, 3, 4, 5, 6}.

8 / 20

Page 18: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Outline

1 Background and Motivations

2 Sensing-Probing-Transmitting Control of EH CR

3 Selective Transmission for EH Sensors

4 CSS under Spectrum Heterogeneity

5 Conclusion and Future Research

Page 19: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Selective transmission for EH sensor

Incorporating data-centric consideration

packets associated different priorities

drop low priority packet to save energy

9 / 20

Page 20: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Selective transmission for EH sensor

Problem

Based on EH, energy status, CSI and packet priority, the node needs todecide whether or not to send each packet, in order to maximize the totalpriority values of sent packets.

9 / 20

Page 21: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

MDP formulation with after-state

10 / 20

Page 22: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Structural results of J∗ and π∗

Theorem

After-state value function J∗(p) is differentiable and non-decreasing.

Theorem

The optimal policy π∗ has the following structure

π∗([b, h, d ]) =

{1 if b ≥ h and d ≥ J∗(b)− J∗(b − h),

0 otherwise.

11 / 20

Page 23: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Learn J∗ with monotone neural network

FittedValue

Iteration

z-1Data set

FMNN

new MNN

current MNN

ifCnvg.

12 / 20

Page 24: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 0.2 0.4 0.6 0.8 1

0.5

1

1.5

2

2.5

3

13 / 20

Page 25: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Learning efficiency

5e2 1e3 1e4 1e5 1e6

Consumed data samples

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Aver

aged

rew

ard

s

Online-DIS

FNN

FMNN

5 10

105

0.16

0.18

Learning curve

14 / 20

Page 26: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Outline

1 Background and Motivations

2 Sensing-Probing-Transmitting Control of EH CR

3 Selective Transmission for EH Sensors

4 CSS under Spectrum Heterogeneity

5 Conclusion and Future Research

Page 27: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

CSS under spectrum heterogeneity

Spectrum heterogeneity

SUs at different spatial locations mayexperience different spectrumstatuses.

15 / 20

Page 28: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

CSS under spectrum heterogeneity

Spectrum heterogeneity

SUs at different spatial locations mayexperience different spectrumstatuses.

Problem

Under spectrum heterogeneity, how to exploit neighbor information to fuseSU observations for improving sensing performance.

15 / 20

Page 29: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Fuse data via MAP-MRF

MAP-MRF CSS framework

Compute maximum a posterior estimation

xMAP = arg maxx

{ΦX (x)

N∏i=1

γxi fY |X (yi | xi )

},

weight γ > 0 introduces tradeoff

Existing works in references [99–102] fuse data via solving marginaldistributions.Compared with [99–102], the proposed MAP-MRF can be solved moreflexibly and efficiently.

16 / 20

Page 30: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Three CSS algorithms based on MAP-MRF

Via graph cut theory, GC-CSS algorithm solves xMAP exactly; complexityorder: O(N · |E|2).

17 / 20

Page 31: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Three CSS algorithms based on MAP-MRF

Via dual decomposition theory, DD-CSS estimates xMAP distributedly (atcluster-level); complexity: O(T · Nl · |El |2).

17 / 20

Page 32: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Three CSS algorithms based on MAP-MRF

Distributed network: clusters with size 1, DD1-CSS becomes fullydistributedly message passing algorithm; guaranteed for solving xMAP ;complexity order O(T · |N (i)|3).

17 / 20

Page 33: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Three CSS algorithms based on MAP-MRF

Existing algorithms (based on belief propagation) only work in distributedsetting; complexity: O(T · |N (i)| · 2|N (i)|).

17 / 20

Page 34: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Performance comparison

0 0.1 0.2 0.3 0.40.5

0.6

0.7

0.8

0.9

1

GC-CSS

DD-CSS

DD1-CSS

BP-CSS

Ind-SS

ROC for various algorithms.18 / 20

Page 35: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Outline

1 Background and Motivations

2 Sensing-Probing-Transmitting Control of EH CR

3 Selective Transmission for EH Sensors

4 CSS under Spectrum Heterogeneity

5 Conclusion and Future Research

Page 36: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Conclusion

In EH CR, the joint optimization of sensing, probing and transmittingis modeled as a two-stage MDP, whose structure is exploited forafter-state simplification.

In EH WSNs, the optimal selective transmission policy is investigated,which is proved to be threshold-based and derived by training amonotone neural network.

CSS under spectrum heterogeneity is formulated via MAP-MRF,which can be effectively solved by graph cut theory and dualdecomposition theory with polynomial complexity.

19 / 20

Page 37: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Future research

Optimal sensing-probing policy without primary user model

Multi-link selective transmission for energy-harvesting sensors

Learn MRF model from data

20 / 20

Page 38: Machine Learning Approaches for Wireless Spectrum and Energy …chintha/pdf/thesis/ppt_keyu.pdf · 2018. 10. 2. · Cognitive radio Energy harvesting 2/20. ML for wireless spectrum

Thank you!