zhao swami mil com 08 tutorial

Networking Cognitive Radios forDynamic Spectrum Access

Qing Zhao Ananthram SwamiQing [email protected] of California

Ananthram [email protected] Army Research Lab

Davis, CA Adelphi, MD

MILCOM 2008 Tutorial18 Nov 2008 San Diego

© Q. Zhao, A. Swami, Tutorial at MILCOM 2008

2

Tutorial Outline

Introduction Physical layer issues MAC layer issues Network layer issues Programs, Policies, StandardsConclusion


3

Tutorial Outline

Introduction Physical layer issues MAC layer issues Network layer issues Programs, Policies, StandardsConclusion


Introduction

MotivationTaxonomy Taxonomy Technical challengesApplications

• Q. Zhao, A. Swami, “A Survey of Dynamic Spectrum Access: Signal Processing and Networking Perspectives,” IEEE ICASSP 2007.

Applications

g g p ,

• Q. Zhao, B.M. Sadler, “A Survey of Dynamic Spectrum Access,” IEEE Signal Processing Magazine, May, 2007.


Limit d S pplLimited Supply

5


Gr in D m ndGrowing Demand

6


R l ti P i t 1927 O t AllRegulation Prior to 1927: Open to All

Herbert Hoover

The Secretary of Commerce...and The Secretary of Commerce...and Under-Secretary of Everything Else!

Agency: Department of Commerce.Service: AM radio broadcasting.

7

Service: AM radio broadcasting.Limited power: cannot deny license to anyone.


Sin 1927: Ti ht C ntr l b FCCSince 1927: Tight Control by FCC

Federal Communications Commission (FRC b. 1934).

8

All non-Federal Government use of the spectrum.


C t P li & S t S itCurrent Policy & Spectrum Scarcity

Fixed allocationRigid eq i ements on ho to se

Little Sharing

9

Rigid requirements on how to use


Spectrum Opportunities in Space, Time, & Frequency

(Credit: DARPA XG) (Credit: ACSP Cornell)

10

M r d Sp tr m O p nMeasured Spectrum Occupancy

(Credit: SSC)


Di r Id C nf in T rmDiverse Ideas, Confusing Terms

Dynamic spectrum accessDynamic spectrum accessDynamic spectrum allocationSpectrum property rightsSpectrum property rightsSpectrum commonsOpportunistic spectrum accessSpectrum poolingSpectrum underlaySpectrum overlayCognitive radio

12

…


A T n m f DSAA Taxonomy of DSADynamic

Current Policy

Fixed allocation Little sharing Rigid requirement

ySpectrum

Access

Rigid requirementon how to use

13


Thr DSA M d lThree DSA ModelsDynamic ySpectrum

AccessCurrent Policy

Fixed allocation Little sharing Rigid requirement

Exclusive Use Model

Open Sharing Model

Hierarchical Access Model


g

14


E l i U M d lExclusive Use ModelDynamic ySpectrum


Fixed allocationLittle sharing Rigid req irement

Exclusive Use Model

Open Sharing Model



g

15

Maintains the basic structure: license for exclusive use

Introduces flexibility in allocation and spectrum usage


E l i U M d lDynamic

Exclusive Use Modely

Spectrum Access

Current Policy


Exclusive Use Model

Open Sharing Model



S t D i Spectrum Property

Rights

Dynamic Spectrum Allocation

16

Maintains the basic structure: license for exclusive use

Introduces flexibility in allocation and spectrum usage


Sp tr m Pr p rt Ri htSpectrum Property RightsDynamic ySpectrum


Fixed allocation Little sharing Rigid req irement

Exclusive Use Model

Open Sharing Model




Rights


17

Allows selling and trading spectrum and freely choosing technology

Let economy & market determine the most profitable use of spectrum


N b l Priz Winnin IdNobel Prize Winning Idea

Ronald H. Coase

Nobel Prize Laureate in Economics (1991)

18

R. Coase, “The federal communications commission,” J. Law and Economics, pp. 1–40, 1959.


C Th r mCoase TheoremCoase Theorem: All government gallocations of a public good are equally efficient in the absence of transaction coststransaction costs.

Ronald H. Coase


19

R. Coase, “The federal communications commission,” J. Law and Economics, pp. 1–40, 1959.


C Th r mCoase TheoremCoase Theorem: All government ll i f bli d allocations of a public good are

equally efficient in the absence of transaction costs.

Ronald H. Coase


Milton Friedman George J Stigler

20

Milton Friedman


George J. Stigler



C Th r mCoase TheoremCoase Theorem: All government allocations of a public good are equally efficient in the absence of transaction coststransaction costs.

Government Regulation: not to find the most efficient allocation, but ,to minimize transaction costs.

Spectrum Property Rights: Allow Ronald H. Coase

licensees to sell and trade spectrum and freely choose technology.


21


Sp tr m Pr p rt Ri htSpectrum Property Rights

How To Define and Enforce?

Spatial, Spectral Spillover

How To Define and Enforce?

p , p pMeasured or computed?Tx or Rx’s responsibility?

How to detect trespassing

(D. Hatfield and P. Weiser, 2005 DySpan)( , y p )

Need Accurate Yet Simple Signal Propagation Models

22

Need Accurate Yet Simple Signal Propagation Models

And Filter Design For Effective Spillover Suppression.


D n mi Sp tr m All ti nDynamic Spectrum AllocationDynamic ySpectrum



Exclusive Use Model

Open Sharing Model




Rights


23

Dynamic spectrum assignment to different services

Exploiting spatial and temporal traffic statistics


D n mi Sp tr m All ti nDynamic Spectrum AllocationDemand Frequency

UMTSDVB TDVB-T

Time

DVB-T

Time

UMTS

Traffic statistical modeling, estimation, and predictionCent ali ed and dist ib ted spect m allocation

24

Centralized and distributed spectrum allocation(L. Xu, etal, 2000, P. Leaves, etal, 2004)


Op n Sh rin M d lOpen Sharing ModelDynamic ySpectrum


Fixed allocation Little sharingRigid req irement

Exclusive Use Model

Open Sharing Model




Rights


25

Open sharing among peer users (spectrum commons)

Draws support from the success of unlicensed ISM bands


Hi r r hi l A M d lHierarchical Access ModelDynamic ySpectrum



Exclusive Use Model

Open Sharing Model




Rights


26

Hierarchical access with primary and secondary users

sharing with limited interference to primary users (licensees)


Hi r r hi l A M d lHierarchical Access ModelDynamic ySpectrum



Exclusive Use Model

Open Sharing Model



S t D i Spectrum SpectrumSpectrum Property

Rights


SpectrumUnderlay(UWB)

SpectrumOverlay

(OSA, pooling)

27

Spectrum underlay: constraint on transmission power

Spectrum overlay: constraint on when and where to transmit


Und rl O rlUnderlay vs. OverlaySpectrum Underlay (UWB) Spectrum Overlay (OSA)

PSD PSD

Primary Primary

28Secondary Secondaryf f


A T f DSAA Taxonomy of DSADynamic Dynamic Spectrum

Access

Exclusive Use Model

Open Sharing Model

Hierarchical Access ModelUse Model Sharing Model Access Model

Spectrum Property

Rights


Spectrum Underlay(UWB)

Spectrum Overlay

(OSA, pooling)

29


C niti R diCognitive RadioSoftware Defined RadioSo t a e e ed ad o

Promoted by Mitola in 1991A multiband radio supporting multiple air interfaces and reconfigurable through softwareinterfaces and reconfigurable through software

Cognitive RadioCognitive RadioPromoted by Mitola in 1998Built upon a software defined radio platformContext-aware, autonomous reconfigurableLearning from and adapting to environmentApplications not limited to DSA

30

Applications not limited to DSA


C iti R di Th Ph i l Pl tfCognitive Radio: The Physical Platform

Dynamic Dynamic Spectrum

Access

Exclusive Use Model

Open Sharing Model


Spectrum Property

Rights



Spectrum Overlay

(OSA, pooling)

31Cognitive Radio


T d D i S t AToward Dynamic Spectrum Access

32Overlay(54-700M)

Underlay(3.1-10.6G)

Auction(A-TV,PCS,C)


Sp tr m O rl : Appli ti nSpectrum Overlay: Applications

Opportunistic use based on hierarchical price structuresEmergency response and military applications

33

Integration of emerging applications such as sensor networks


Sp tr m O rl : T hni l ISpectrum Overlay: Technical IssuesPhysical Layery y

Opportunity sensingInterference Aggregation

MAC LayerOpportunity tracking and learningOpportunity tracking and learningOpportunity exploitation with imperfect sensingOpportunity sharing

Network LayerP t l d ti

34

Power control and routing

c©Q. Zhao, A. Swami, Tutorial at MILCOM 2008. 1

Spectrum Sensing and Opportunity Identification

PHY Layer Issues

• Model and detection problem

• How should we sense?

• Cooperative Sensing

• Hardware Challenges

• Waveform Design & Modulation

• Interference Constraints


Channel Sensing Model: Slotted Primary Users

Opportunities

Channel 1

Channel N0 1 2 3 T

S1(1) = 0 S1(2) = 1 S1(3) = 0 S1(T ) = 0

SN(1) = 1 SN(2) = 0 SN(3) = 0 SN(T ) = 0

t

t

N independent channels, each with bandwidth Bi.

Secondary users search for opportunities independently.

Every primary tx interferes with all secondary users (symmetric).

How to detect whitespace?


802.22 Draft DFS Sensing Requirements

Parameter Digital TV Wireless Microphone

(Part 74)

Channel Detection Time ≤ 2 sec ≤ 2 sec

Channel Move Time 2 sec 2 sec

Detection Threshold - 116 dBM - 107 dBm

(required sensitivity) (over 6 MHz) (over 200 KHz)

Probability of detection 0.9 0.9

Probability of false alarm 0.1 0.1

SNR - 21 db - 12 dB

Low SNR regime

FCC ET Docket no. 03-122, November 18, 2003, Cordeiro et al, Ghosh et al, Shellhammer


Spectrum Sensor at PHY

Binary Hypotheses Test:

H0 (idle) vs. H1 (busy)

Two Types of Sensing Errors:

opportunity overlook: H0 → H1 ǫ∆= prob. of overlook

opportunity misidentification: H1 → H0 δ∆= prob. of misidentification

Receiver Operating Characteristics (ROC): 1 − δ vs. ǫ

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of False Alarm ε

Pro

babi

lity

of D

etec

tion

1 −

δ

δε

Which point δ to operate at?

overlook vs. misidentification






opportunity overlook: H0 → H1 ǫ∆= prob. of overlook

opportunity misidentification: H1 → H0 δ∆= prob. of misidentification


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Pro

babi

lity

of D

etec

tion

1 −

δ

δε How to choose operating point δ?

Overlook vs. Misidentification

Which is worse:

false alarm or miss detection?


Spectrum Sensor at PHY: MAC performance



O

ǫ

1 − δ

1 − ζ

δ > ζ δ < ζ

conservative aggressive

MAC Layer Performance

Probability of success PS

(throughput)

Probability of collision PC

Objective: max PS s.t. PC ≤ ζ

How to choose operating

point δ?


Channel Sensing Model: unslotted primary users

Acknowledgement

Secondary users

Slot (L)

Sensing Transmission

Channel 1

Channel N

Ls L − Ls

t

t

Slotted secondary usage, with sensing, data, and ACK periods

Problem: Given measurements during sensing time, detect the channel state

Problem: during transmission time.

Problem: Is a sensed idle channel an opportunity?

Challenge: Even with perfect sensing, opportunity detection is subject to

Challenge: errors.



Binary Hypotheses Test: (channel n in slot k)

H0 (On(k) = 1 : opportunity) vs. H1 (On(k) = 0 : no opportunity)

Let On(k) denote the sensing outcome.


false alarm: H0 → H1 ǫn(k)∆= PrOn(k) = 0|On(k) = 1

miss detection: H1 → H0 δn(k)∆= PrOn(k) = 0|On(k) = 1


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Pro

babi

lity

of D

etec

tion

1 −

δ

δε

How to choose the operating point (ǫn(k), δn(k))

for each channel at each slot?


Spectrum Sensing: Some key questions

How should we sense?

Choice of detectors

Tradeoff SU QoS with PU protection

Detecting spectrum opportunities


Energy Detection

Pros: easily implemented; minimal assumptions

Cons: poor performance with noise uncertainty

Cons: and with multiple secondary users

Cons: Performance ∼ 1/SNR2 at low SNR

H0 : (idle) y(n) = w(n), n = 1, ..., N, AWGN

H1 : (busy) y(n) = w(n) + s(n)

Decide H1 if z =1

N

N∑

n=1

|y(n)|2 > τ(N, σ2

w)

Under H0 : z ∼ N (σ2

w, σ4

w/N)

Under H1 : z ∼ N (σ2

y, σ4

y/N), σ2

y := σ2

w + σ2

s

µ1 − µ0 = σ0Q−1(PFA) − σ1Q

−1(PD)

√NSNR = Q−1(PFA) − (1 + SNR)Q−1(PD)


Choice of Detectors - Cyclic Detectors (2)

Exploit guard bands in frequency, known carriers, data rates, modulation type

Pros: fc, Ts easy to detect via square-law devices, or cyclic approaches

Pros: Cyclic approaches useful when σ2n is unknown (avoid SNR wall)

Test Statistic : S(f ; τ ) =1

N

∑

n

y(n)y(n + τ )e−j2πfn

Pros: Easily implemented via FFTs

Cons: Timing and frequency jitter can be detrimental

Cons: Requires long integration times

Cons: RF non-linearities; Spectral leakage (ACI).

Cabric et al, Asiloamr’04 Cabric-Brodersen, PIMRC’05

Ghozzi et al, Crowncom’06 da Silva-Choi-Kim,ITA Wkhsp ’07

Lee-Yoon-Kim,ICIPC’07 Kim et al, Dyspan’07

Ye et al, SPS Wkshp ’07 Tu et al, PIMRC’07

Sutton-Nolan-Doyle, JSAC’08

General references on cyclic detection: Giannakis; Gardener


Choice of Detectors: Matched Filter (3)

Exploit pilots or sync (PN) sequences in primary (WRAN 802.22)

Test Statistic : z =1

N

N∑

n=1

y(n)s(n)

Pros: Correlation detection is usually better than energy detection.

Pros: Performance ∼ 1/SNR at low SNR

Q−1(PFA) − Q−1(PD) =√

NSNR

Cons: fading may null pilot; need to cope with time and freq syncLi et al, JSAC’07 - Exploits pilots, for interference detection

Kundargi-Tewfik, ICASSP’08 - sequential tests with pilots

Yu-Sung-Lee, ICASSP’08 - exploit PU pilots

Other Detectors based on

Receiver leakage Wild-Ramachandran, Dyspan’05

Signal correlation Zeng et al, PIMRC’07

Fast fading Larson-Regnoli, CommLett’07

Multiple antennas Pandhripande-Linnartz, ICC’07

HMM classifier Kyouwoong et al, Dyspan’07

Wavelet-based Tian-Giannakis, CrownCom’06

Multi-resolution Neihart-Roy-Allstot, ISCAS’07

Compressed sensing Tian-Giannakis, ICASSP’07




Choice of detectors




How long should the sensing time be

• Channel coherence

• Primary’s traffic patterns (e.g., fractional on-time)

• Interference constraints

• Primary and secondary user powers; noise power

• Fading, multipath, shadowing

• Multiple primaries? Spatial distribution

• Multiple secondary users? (aggregate interference)

• SU QoS (rate, reliability, latency) and constraints (power, cooperation)

• Can we exploit PU Modulation, pilots, sync signals,

• Complexity and specifics of algorithms

• Robustness

Detection problem cannot be solved in isolation


Optimizing sensing time for detection

N ≈ 2(SNR)−2[Q−1(PFA) − Q−1(PD)]2

What if we do not know the noise variance?

Could use sample estimate of noise variance,

σ2w ∈ [aσ2

w, bσ2w], a, b ∼ 1/

√N

To ensure desired performance with uncertainty, need

N ≈ 2(SNR − ∆)−2[bQ−1(PFA) − (SNR + a)Q−1(PD)]2

Energy Detector breaks down when SNR ≈ ∆ = b − a, uncertainty

Tandra and Sahai’s SNR wall, JSTSP, 2008

Example: 6 MHz BW, 1 sec. obs time, ∆ ≈ 0.0022, SNR threshold = −23dB,

close to operating SNR of -21 dB in the 802.22 standard

Robustness to model imperfections important at low SNR


Optimizing sensing time for throughput

Trade off sensing accuracy of throughput

Slot size of length N - devote n samples for sensing

Maximize throughput efficiency

η(n) :=N − n

N[1 − PFA(n)]

For specified PD (interference constraint)

PFA(n) = Q(

(1 + SNR)Q−1(PD) + SNR√

n)

n∗ and throughput increase as N ↑

n∗ ↓ and η ↑ as SNR ↑

n∗ ↑ and η ↓ as interference constraint ↓

Does this represent SU performance?

Wang et al, WCNC 2007

Kattepur et al, ICICSP 2007




Choice of detectors



Choosing ‘sensing radius’ or threshold

Interaction with MAC


Whitespace Detection to Opportunity Detection

Is detecting primary signals = detecting spectrum opportunity?

How does PHY performance translate to MAC performance?

We want to detect primary receivers!

PU locations are unknown

1 − PMD

PFA0 1

1 − ζ

If PU is loud but SU is not

listening, is it interference?

SU-TX and SU-TX must

jointly detect opportunities

PS = (1 − PFA) Pr[H0]

PC = PMD


Spectrum Opportunity: Definition

A B

Interference

Primary Tx

Primary Rx

RI: interference range

RI: of primary users

rI: interference range

rI: of secondary users

A channel is an opportunity for A −→ B if

the transmission from A to B can succeed

the interference power to primary is below a prescribed level



A B

Interference

RI

Primary Tx

Primary Rx


RI: of primary users RI ∝ P1/αtx


rI: of secondary users

rI: ∝ p1/αtx






A B

Interference

rI

RI

Primary Tx

Primary Rx




rI: of secondary users rI ∝ p1/αtx





Spectrum Opportunity: Properties

A B

Interference

rI

RI

Primary Tx

Primary Rx




rI: of secondary users rI ∝ p1/αtx

determined by both transmitting and receiving activities of primary users.

Asymmetric (an opportunity for A −→ B may not be one for B −→ A).


Detection of Primary Receivers (LBT)

AXY

Primary TxPrimary Rx

rI

rD

Rp + rI

rI: interference range, Rp: primary tx range, rD: detection range

Detecting primary Rx within rI by detecting primary Tx within rD

False alarms and miss detections occur due to noise and fading


From Detecting Signal to Detecting Opportunity

A B

X

Y

rI

rD

RI

Prob. of Detection (1 − PMD)

Prob. of False Alarm0

1

1

rD ↓

rD ↑

H0: opportunity, H1: alternative.

Even with perfect ears, exposed Tx (X) ⇒ FA, hidden Rx (Y ) ⇒ MD.

Adjusting detection range rD leads to different operating points.


Miss Detection May not Lead to Collision

A B

rI

RI

Interference

Primary Tx

Primary Rx

There is no primary receiver around A

There are primary transmitters around B


Miss Detection May Lead to Success

A B

rI

RI

Primary Tx

Primary Rx

There are primary receivers around A

There is no primary transmitter around B


Correctly Identified Opportunity May Not Lead to Success

A B

rIRI

DATA

Primary Tx

Primary Rx

A B

rI

RI

ACK

Primary Tx

Primary Rx

Successful data transmission and failed ACK


Network Model

Primary users form a Poisson point process with density λ.

Each primary user transmits with probability p in a slot.

Primary receivers are uniformly distributed within Rp of their transmitters.

Rp

Rp

Rp

2 Analytical expressions for

PFA, PMD, PC, PS

2 For LBT and for RTS-CTS enabled

LBT

Zhao-Ren-Swami, Asilomar ’07


Summary of Opportunity Detection

Spectrum Opportunity

Determined by both transmitting and receiving activities of primary users

Asymmetric (an opportunity for A −→ B may not be one for B −→ A)

Equivalence of Detecting Signal and Opportunity

Inevitability of opportunity detection errors

A necessary and sufficient condition

Translation from PHY Performance to MAC Performance

Crucial for choosing optimal detector operating point

Complex dependency on the application type and MAC

Choice of sensor operating point cannot be decoupled from sensing and

accessing policies




Choice of detectors



Cooperative sensing

Hardware challenges

Waveform Design & Modulation

Interference Constraints


Cooperative Schemes

Benefits: combat fading, shadowing, poor sensors

Overhead: control channel? broker?

Trust issues: - jammed links, malicious nodes

Fairness

Time scales - latency

Increased uncertainty due to aggregate interference

Many based on ‘distributed estimation detection’ ideas

Cabric-Mishra-Brodersen, Asilomar’04

Ghasemi-Sousa, Dyspan’05

Mishra-Sahai-Brodersen, ICC’06 - correlated fading; detecting malicious users

Qihang et al, PIMRC’06 - uses Dempster-Shafer theory

Yi et al, PIMRC’07 - relies on multi-hop cooperation

Gandetto-Regazzoni, JSAC’07 - distributed detection

Tahrepour et al, IET Commun, 07 - asymptotic theory, sequential detection

Ma-Li, Globecom’07 - extensions of MRC, EGC

Chen-Wang-Li, ISWCS’07 - learns local ROC parameters

Peh-Liang, WCNC’07 - select users for cooperation

Ganesan et al, TWC’07, JSAC’08- optimal pairing of SU’s to improve detection

Quan-Sayed,JSTSP’08 - linear combining

Unnikrishnan-Veeravalli, JSTSP’08 - linear-quadratic fusion of LLR’s


Hardware Challenges

Large bandwidth and sampling rates

PU load dictates scanning architecture

Dynamic range

Linearity of analog circuits (mixers, etc)

Adjacent channel interference

Adaptive notch filtering

Active interference cancellation

Cabric-Mishra-Brodersen, Asilomar’04

Cabric-Brodersen, PIMRC’05

Mayer et al, ECWT’07, PIMRC’07

Luu-Daneshrad, JSAC 2007

Jia-Zhang-Shen, JSAC 2008


Waveform Design / Modulation

OFDMA has emerged as natural standard

Time-frequency granularity well-suited to filling holes

ACI in sensing

Hardware non-linearities and ACI in transmit

Null subcarriers to protect PU from ACI

( 1440 data carriers out of 2048 in 802.22)

PAPR issues

Dictates pulse shape design

Symbol period dictated by channel and SO coherence times

SU transmit power dictated by allowed interference to PUWeiss-Jondral, Comm Mag, 2004

Berthold-Jondral, Dyspan 2005

Tang, Dyspan 2005

Wright, AccessNets 07


Interference Constraints

Policy Issues: What to impose, How to monitor?

How to impose?

Need to specify allowed probability of interference ζ

at prescribed interference level η to PU

[η, ζ] is a PU-protection / SU-QoS tradeoff

Conditional or joint probability of collision ?

Impose on per-slot basis, or on average ?

From aggregate to node-level parameters?

Requires knowledge of node location, traffic, and channel models

How to monitor?

Qing Zhao, ICASSP’07

c©Q. Zhao, A. Swami, tutorial at MILCOM 2008. 1

MAC Issues in Opportunistic Spectrum Access

PSfrag replacements

Ch 1 Ch 2 Ch 3

NAK

NAKNAK

PSfrag replacements

Ch 1

Ch 2

Ch 3

NAK

ε

1 − δ

1 − ζ

δ > ζ δ < ζ


optimal (δ∗ = ζ)

References

[1 ] Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum

Access in the Presence of Sensing Errors,” IEEE Transactions on Information Theory, May 2008.

[2 ] Q. Zhao, B. Krishnamachari, and K. Liu, “On Myopic Sensing for Multi-Channel Opportunistic Access,”

to appear in IEEE Transactions on Wireless Communications.

[3 ] K. Liu, Q. Zhao, “A Restless Bandit Formulation of Opportunistic Access: Indexablity and Index Policy,”

Proc. of IEEE SDR Workshop, June, 2008.


Basic MAC Issues in OSA

PSfrag replacements

Opportunities

Channel 1

Channel N0 1 2 3 T

S1(1) = 0 S1(2) = 1 S1(3) = 0 S1(T ) = 0

SN(1) = 1 SN(2) = 0 SN(3) = 0 SN(T ) = 0

t

t

I Search for fast-varying opportunities in multiple channels.

I Limited Sensing: can only sense and access a subset of channels in each slot.

Which channels to sense and whether to access?


Outline

I Slotted independent Markov channels with perfect sensing

I (Correlated) channels with imperfect sensing

I Unslotted primary systems

I Energy-constrained opportunistic spectrum access in fading

I Distributed sensing for multiple users

I Opportunistic spectrum access in self-similar primary traffic

I Concluding remarks


Gilbert-Elliot Channel Model

I N independent Gilbert-Elliot channels with rate Bi (i = 1, · · · , N).

PSfrag replacements

Opportunities

Channel 1

Channel N0 1 2 3 T t

t

PSfrag replacements

0 1

(busy) (idle)

p(i)01

p(i)11p

(i)00

p(i)10

0E.N. Gilbert, “Capacity of burst-noise channels,” Bell Syst. Tech. J., vol. 39, pp. 1253-1265, Sept. 1960.


Positive Memory vs. Negative Memory

PSfrag replacements

0 1(busy) (idle)

p0,1

p1,1p0,0

p1,0

Markov Channels with Positive Memory (p11 ≥ p01)

! ! ! ! ! ! ! ! ! ! !

! ! ! ! ! ! ! ! ! ! !

! ! ! ! ! ! ! ! ! ! !

! ! ! ! ! ! ! ! ! ! !

Markov Channel with Negative Memory (p11 < p01)" " "

" " "" " "

# # ## # #

# # #

$ $ $ $ $ $ $

$ $ $ $ $ $ $

$ $ $ $ $ $ $

% % % % % % %

% % % % % % %

% % % % % % %

& & & & &

& & & & &

& & & & &

' ' ' ' '

' ' ' ' '

' ' ' ' '

( ( (( ( (

( ( () ) )

) ) )) ) )

* * * * *

* * * * *

* * * * *

+ + + + +

+ + + + +

+ + + + +

, , , ,

, , , ,

, , , ,

- - - -

- - - -

- - - -


Sensing Policy for Opportunity Tracking

. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .

/ / / / / / / // / / / / / / // / / / / / / // / / / / / / /0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 22 2 2 2 2 2 2 22 2 2 2 2 2 2 22 2 2 2 2 2 2 2

3 3 3 3 3 3 3 33 3 3 3 3 3 3 33 3 3 3 3 3 3 33 3 3 3 3 3 3 3

4 4 4 4 4 4 4 44 4 4 4 4 4 4 44 4 4 4 4 4 4 44 4 4 4 4 4 4 4

5 5 5 5 5 5 5 55 5 5 5 5 5 5 55 5 5 5 5 5 5 55 5 5 5 5 5 5 5

6 6 6 6 6 6 6 66 6 6 6 6 6 6 66 6 6 6 6 6 6 66 6 6 6 6 6 6 6

7 7 7 7 7 7 7 77 7 7 7 7 7 7 77 7 7 7 7 7 7 77 7 7 7 7 7 7 7

8 8 8 8 8 8 8 8 88 8 8 8 8 8 8 8 88 8 8 8 8 8 8 8 88 8 8 8 8 8 8 8 8

9 9 9 9 9 9 9 9 99 9 9 9 9 9 9 9 99 9 9 9 9 9 9 9 99 9 9 9 9 9 9 9 9

: : : : : : : : :: : : : : : : : :: : : : : : : : :: : : : : : : : :

; ; ; ; ; ; ; ; ;; ; ; ; ; ; ; ; ;; ; ; ; ; ; ; ; ;; ; ; ; ; ; ; ; ;

PSfrag replacements

Opportunities

Channel 1


t

I Sensing Policy πs

2 Choose M out of N channels to sense in each slot.

2 Without loss of generality, consider M = 1.


Sensing Policy for Opportunity Tracking

< < < < < < < << < < < < < < << < < < < < < << < < < < < < <

= = = = = = = == = = = = = = == = = = = = = == = = = = = = => > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >

? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ? ?

@ @ @ @ @ @ @ @@ @ @ @ @ @ @ @@ @ @ @ @ @ @ @@ @ @ @ @ @ @ @

A A A A A A A AA A A A A A A AA A A A A A A AA A A A A A A A

B B B B B B B BB B B B B B B BB B B B B B B BB B B B B B B B

C C C C C C C CC C C C C C C CC C C C C C C CC C C C C C C C

D D D D D D D DD D D D D D D DD D D D D D D DD D D D D D D D

E E E E E E E EE E E E E E E EE E E E E E E EE E E E E E E E

F F F F F F F F FF F F F F F F F FF F F F F F F F FF F F F F F F F F

G G G G G G G G GG G G G G G G G GG G G G G G G G GG G G G G G G G G

H H H H H H H H HH H H H H H H H HH H H H H H H H HH H H H H H H H H

I I I I I I I I II I I I I I I I II I I I I I I I II I I I I I I I I

PSfrag replacements

Opportunities

Channel 1


t

Immediate Reward

2 If the chosen channel i is idle, Bi units reward is accrued.

2 If the chosen channel i is busy, no reward; wait until the next slot.

Objective: choose sensing policy πs to

max E[throughput]


Optimal Sensing Policy for Opportunity Tracking

PSfrag replacements

0 t t + 1 TO(1) O(2) O(t − 1) O(t)

R(t) R(t + 1)

· · ·

· · ·

Use entire observation history

Max total remaining reward

I Learn from the observation history.

I Foresighted planning: maximize total remaining reward.

I Optimal action: Gaining immediate reward vs. Gaining spectrum information.


Sensing Policy: Learn from Observations

0 5 10 15 20 25 30

0.5

0.52

0.54

0.56

0.58

0.6

0.62

0.64

0.66

0.68

0.7

Time (slot)

Thro

ughp

ut (b

its p

er s

lot)

Use only statistical info

Optimal Approach

I Cognition: improved performance by learning from accumulating observ.

I Difficulty: exponential complexity; sensitivity to model mismatch.

I Goal: structural policies that are simple, robust, and optimal.


Slotted Independent Markov Channels: outline

I A restless multi-armed bandit formulation

I Indexability and index policies

I When optimality, simplicity, and robustness are achieved simultaneously

I Scaling behavior of the max. throughput w.r.t. the number N of channels


Multi-Armed Bandit

Multi-Armed Bandit Process

I A bandit with N independent arms.

I Fully observable states of all arms Zi(t)

I Activate arm i and get reward Ri(Zi(t)).

I Active arms change state (Markovian).

I Passive arms are frozen.

Objective: Decide which arm to activate in each slot for max long-term reward.

Optimal Policy: Gittins Index (1979)

I Compute an index for each state of each arm.

I Activate the arm whose current state has the largest index.

Advantage: Reduces an N-D problem to N independent 1-D problem.0J.C.Gittins, “Bandit Processes and Dynamic Allocation Indices,” in Journal of the Royal Sttistical Society, Series B (Methodological), Vol.41, No.2 (1979), 148-177.


Restless Multi-Armed Bandit

Restless Multi-armed Bandit Problem

I Passive arms also change states.

I Can active M ≥ 1 arms simultaneously.

Structure of Optimal Policy

I Not yet known.

Complexity

I PSPACE-hard.

0P. Whittle, ”Restless bandits: Activity allocation in a changing world”, in Journal of Applied Probability, Volume 25, 1988.


Restless Multi-Armed Bandit Formulation

J J J J J J J JJ J J J J J J JJ J J J J J J JJ J J J J J J J

K K K K K K K KK K K K K K K KK K K K K K K KK K K K K K K KL L L L L L L L LL L L L L L L L LL L L L L L L L LL L L L L L L L L

M M M M M M M M MM M M M M M M M MM M M M M M M M MM M M M M M M M M

N N N N N N N NN N N N N N N NN N N N N N N NN N N N N N N N

O O O O O O O OO O O O O O O OO O O O O O O OO O O O O O O O

P P P P P P P PP P P P P P P PP P P P P P P PP P P P P P P P

Q Q Q Q Q Q Q QQ Q Q Q Q Q Q QQ Q Q Q Q Q Q QQ Q Q Q Q Q Q Q

R R R R R R R RR R R R R R R RR R R R R R R RR R R R R R R R

S S S S S S S SS S S S S S S SS S S S S S S SS S S S S S S S

T T T T T T T T TT T T T T T T T TT T T T T T T T TT T T T T T T T T

U U U U U U U U UU U U U U U U U UU U U U U U U U UU U U U U U U U U

V V V V V V V V VV V V V V V V V VV V V V V V V V VV V V V V V V V V

W W W W W W W W WW W W W W W W W WW W W W W W W W WW W W W W W W W W

PSfrag replacements

Opportunities

Channel 1


t

I Each channel is considered as an arm.

I If channel i is sensed, then it is “activated”.

I The channel states S1, ...SN are not observable

I ⇒ Cannot use the channel state as the state of each arm

0K. Liu, Q. Zhao, “A Restless Bandit Formulation of Opportunistic Access: Indexablity and Index Policy,” Proc. of IEEE SDR Workshop, June, 2008.


From Unobservable to Observable

I Information state: the state of each arm should be its observation history.

I Sufficient statistic: the a posterior distribution (belief vector) Ω(t) that

I exploits the entire observation history.

Ω(t) = [ω1(t), · · · , ωN (t)]

ωi(t) = Pr[channel i is idle in slot t | O(1), · · · , O(t − 1)︸︷︷︸

observations

]

I The state of arm i in slot t is ωi(t).

I The expected immediate reward obtained when activate arm i is ωi(t) Bi.


Markovian Transition of the Belief StatePSfrag replacements

0 1

(busy) (idle)

p(i)0,1

p(i)1,1p

(i)0,0

p(i)1,0

I If channel i is activated in slot t:

ωi(t + 1) =

p(i)11 , if Si(t) = 1

p(i)01 , if Si(t) = 0

.

I If channel i is made passive in slot t:

ωi(t + 1) = ωi(t)p(i)11 + (1 − ωi(t))p

(i)01 .


Index Policies

Are there simple index policies with good performance?

Myopic policy: maximize immediate reward

I Index of channel i: Ii(t) = ωi(t) Bi

I Action: choose the channel with the largest index

1 2 3 4 5 60.65

0.7

0.75

0.8

0.85

0.9

Time Slot

Trou

ghpu

t(bits

per

slo

t)

Optimal policyMyopic policy


Whittle’s Index Policy

Whittle’s Index Policy

2 Subsidy for passivity: provide a subsidy ν when the arm is made passive.

2 Whittle’s index: the minimum subsidy ν that makes the passive action

optimal at the current state.

Performance

2 Optimal under relaxed constraint on the average number of active arms.

2 Asymptotically optimal (N → ∞ w. MN

fixed) under certain conditions.

2 Near optimal performance observed from extensive numerical examples.

Difficulties

2 Existence of Whittle’s index (indexability) is often difficult to establish.

2 High complexity to compute index for uncountable states of each arm.

0P. Whittle, ”Restless bandits: Activity allocation in a changing world”, in Journal of Applied Probability, Volume 25, 1988.0Richard R. Weber; Gideon Weiss, “On an Index Policy for Restless Bandits,” in Journal of Applied Probability, Vol.27, No.3. (Sep,. 1990), pp. 637-648.


The Optimality of Threshold Policy and Indexability

Indexability: The optimal policy for a single-armed bandit is a threshold policy: there exists a

ω∗(ν) ∈ R such that it is optimal to activate the arm if the current belief ω > ω∗(ν); otherwise it is

optimal to make the arm passive. Furthermore, the threshold ω∗(ν) monotonically increases from

−∞ to ∞ as subsidy ν goes from −∞ to ∞, thus the bandit is indexable.

10

PSfrag replacements

Total remainingreward if active

Total remainingreward if passive

ω∗i (ν)

Passive Activeω < ω∗

i (ν) ω > ω∗i (ν)

ω



Whittle’s Index In Closed-Form

I Positive memory (p11 ≥ p01)

Ii(ω) =

ωBi, if ω ≤ p01 or ω ≥ p11;

ω1−βp11+βω

Bi, if ωo ≤ ω < p11;

ω−βT (ω)+C(1−β)(β(1−βp11)−β(ω−βT (ω)))1−βp11−A(1−β)(β(1−βp11)−β(ω−βT (ω))) Bi, if p01 < ω < ωo;

I Negative memory (p11 < p01)

Ii(ω) =

ωBi, if ω ≤ p11 or ω ≥ p01;

βp01+ω(1−β)1+β(p01−ω) Bi, if T (p11) ≤ ω < p01;

(1−β)(1+βE)(βp01+ω(1−β))1−β(1−p01)−D(1−β)(β2p01+βω−β2ω)

Bi, if ωo ≤ ω < T (p11);

(1−β)(βp01+ω−βT (ω))−E(1−β)β(βT (ω)−βp01−ω)1−β(1−p01)+D(1−β)β(βT (ω)−βp01−ω) Bi, if p11 < ω < ωo;

,



Whittle’s Index in Closed-Form

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Belief ω

Whi

ttle’

s in

dex

W(ω

)

p11

=0.4, p01

=0.8, β=0.9

p11

=0.9, p01

=0.1, β=0.9

I Whittle’s index is an increasing function of the belief state.

I Whittle’s index policy is equivalent to myopic policy for identical channels.


The Performance of Whittle’s Index Policy

1 2 3 4 5 60.65

0.7

0.75

0.8

0.85

0.9

Time Slot

Trou

ghpu

t(bits

per

slo

t)

Optimal policyWhittle’s index policyMyopic policy


The Optimality of Whittle’s Index Policy

For stochastically identical channels,

I Whittle’s index policy = myopic policy

I Myopic policy is proven to be optimal when

2 channels have positive memory (p11 ≥ p01)

2 N = 2 and N = 3 channels with negative memory (p11 < p01)

0Q. Zhao, B. Krishnamachari, K. Liu, “On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance,” to appear in IEEE Trans. Wireless Communications.0T. Javidi, B. Krishnamachari, Q. Zhao, and M. Liu, “Optimality of Myopic Sensing in Multi-Channel Opportunistic Access,” ICC 2008.


Structure of Whittle’s Index Policy: Positive Memory Case

X X X X X X

X X X X X X

X X X X X X

X X X X X X

Y Y Y Y Y Y

Y Y Y Y Y Y

Y Y Y Y Y Y

Y Y Y Y Y Y

Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z




[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [

[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [

[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [

[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [

\ \ \ \ \ \ \ \ \ \ \

\ \ \ \ \ \ \ \ \ \ \

\ \ \ \ \ \ \ \ \ \ \

\ \ \ \ \ \ \ \ \ \ \

] ] ] ] ] ] ] ] ] ] ]

] ] ] ] ] ] ] ] ] ] ]

] ] ] ] ] ] ] ] ] ] ]

] ] ] ] ] ] ] ] ] ] ]I Stay with idle channels and leave busy ones to the end of the queue.

1

2

3

4

2

1

3

4

N

N

PSfrag replacements

Sense“idle”

“idle”

“busy”

t = 1 t = 2

0Q. Zhao, B. Krishnamachari, K. Liu, “On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance,” to appear in IEEE Trans. Wireless Communications.0K. Liu and Q. Zhao, “Channel Probing for Opportunistic Access with Multi-channel Sensing,” IEEE Asilomar Conference on Signals, Systems, and Computers, Oct., 2008.


Structure of Whittle’s Index Policy: Negative Memory Case

^ ^ ^^ ^ ^

^ ^ ^^ ^ ^

_ _ __ _ _

_ _ __ _ _

` ` ` ` ` ` `

` ` ` ` ` ` `

` ` ` ` ` ` `

` ` ` ` ` ` `

a a a a a a a

a a a a a a a

a a a a a a a

a a a a a a a

b b b b b

b b b b b

b b b b b

b b b b b

c c c c c

c c c c c

c c c c c

c c c c c

d d dd d d

d d dd d d

e e ee e e

e e ee e e

f f f f f

f f f f f

f f f f f

f f f f f

g g g g g

g g g g g

g g g g g

g g g g g

h h h h

h h h h

h h h h

h h h h

i i i i

i i i i

i i i i

i i i i

I Stay with busy channels and leave idle ones to the end of the queue.

I Reverse the order of unobserved channels.

4

1

2

3

4

3

2

N

1

N

PSfrag replacements

Sense

“idle”

“idle”

“busy”

t = 1 t = 20Q. Zhao, B. Krishnamachari, K. Liu, “On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance,” to appear in IEEE Trans. Wireless Communications.0K. Liu and Q. Zhao, “Channel Probing for Opportunistic Access with Multi-channel Sensing,” IEEE Asilomar Conference on Signals, Systems, and Computers, Oct., 2008.


Robustness of Whittle’s Index Policy

I No need to know the transition probabilities except the order of p11 and p01.

I Automatically tracks model variations.

1 2 3 4 5 6 7 8 9 100.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

Time slot (T)

Thro

ughp

ut

p11

=0.6, p01

=0,1 (T<=5); p11

=0.9, p01

=0,4 (T>5)

Model Variation


Scaling Behavior of Maximum Throughput with N

I The upper bound is independent of N .

I The lower bound approaches to the upper bound at geometric rate with N .

I Throughput w. single-channel sensing saturates at geometric rate with N .

5 10 15 20 25 300.52

0.54

0.56

0.58

0.6

0.62

Number of channels

Low

er a

nd u

pper

bou

nds

p11

=0.8, p01

=0.1

The upper bound of the throughput limitThe lower bound of the throughput limit

0Q. Zhao, B. Krishnamachari, K. Liu, “On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance,” to appear in IEEE Trans. on Wireless Comm..


Slotted Independent Markov Channels: Summary

I A restless multi-armed bandit formulation

I Indexability and Whittle’s index policy in closed-forms

I When channels are stochastically identical

2 Whittle’s index policy = myopic policy.

2 A semi-universal structure ⇒ optimality + simplicity + robustness.

2 Scaling behavior: max throughput saturates at geometric rate under

limited sensing.


Outline



2 A constrained POMDP formulation for joint PHY-MAC design

2 A separation principle







Correlated Channels

j j j j j j j j

j j j j j j j j

j j j j j j j j

j j j j j j j j

k k k k k k k k

k k k k k k k k

k k k k k k k k

k k k k k k k kl l l l l l l l

l l l l l l l l

l l l l l l l l

l l l l l l l l

m m m m m m m m

m m m m m m m m

m m m m m m m m

m m m m m m m m

n n n n n n n n

n n n n n n n n

n n n n n n n n

n n n n n n n n

o o o o o o o o

o o o o o o o o

o o o o o o o o

o o o o o o o o

p p p p p p p p

p p p p p p p p

p p p p p p p p

p p p p p p p p

q q q q q q q q

q q q q q q q q

q q q q q q q q

q q q q q q q q

r r r r r r r r

r r r r r r r r

r r r r r r r r

r r r r r r r r

s s s s s s s s

s s s s s s s s

s s s s s s s s

s s s s s s s s

t t t t t t t t

t t t t t t t t

t t t t t t t t

t t t t t t t t

u u u u u u u u

u u u u u u u u

u u u u u u u u

u u u u u u u u

v v v v v v v v

v v v v v v v v

v v v v v v v v

v v v v v v v v

w w w w w w w w

w w w w w w w w

w w w w w w w w

w w w w w w w w

PSfrag replacements

Opportunities

Channel 1


t

PSfrag replacements

(0, 0)

(1, 0)

(0, 1)

(1, 1)

I Potentially correlated channels.

I Markov chain with 2N states.

I Imperfect sensing.


Sensing and Access Policies

x x x x x x x xx x x x x x x xx x x x x x x xx x x x x x x x

y y y y y y y yy y y y y y y yy y y y y y y yy y y y y y y yz z z z z z z z zz z z z z z z z zz z z z z z z z zz z z z z z z z z

| | | | | | | || | | | | | | || | | | | | | || | | | | | | |

~ ~ ~ ~ ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ ~

PSfrag replacements

Opportunities

Channel 1

Channel N0 1 2 3 T

S1(1) = 0 S1(2) = 1 S1(3) = 0 S1(T ) = 0

SN(1) = 1 SN(2) = 0 SN(3) = 0 SN(T ) = 0

t

t

Sensing Policy πs

2 Deterministic: choose which channel to sense in each slot

2 Randomized: choose the probability of sensing each channel

Access Policy πc

2 Deterministic: whether to transmit based on the sensing outcome

2 Randomized: transmission probability based on the sensing outcome


Sensing and Access Policies

PSfrag replacements

Opportunities

Channel 1

Channel N0 1 2 3 T

S1(1) = 0 S1(2) = 1 S1(3) = 0 S1(T ) = 0

SN(1) = 1 SN(2) = 0 SN(3) = 0 SN(T ) = 0

t

t

Reward and Collision

2 A reward R(t) = Bi is accrued when access an idle channel i.

2 A collision with primary users occurs when access a busy channel.

2 A successful transmission is acknowledged at the end of the slot.

Objective:

max E[throughput] s.t. collision probability Pc ≤ ζ


Sensing Policy: How to Achieve Synchronous Hopping?

PSfrag replacements

0 t t + 1 TK(1) K(2) K(t − 1) K(t)

R(t) R(t + 1)

· · ·

· · ·

Use entire observ. history (ACK/NAK)

Max total remaining rewarda(t)

I Use common observ. (ACK/NAK) to ensure Tx-Rx synchronous hopping.

I Sufficient statistic: the a posterior distribution (belief vector) Λ(t) that

I exploits the entire observation history.

Λ(t) = [λ1, · · · , λ2N ]

λi = Pr[state is i | K(1), · · · , K(t − 1)︸︷︷︸

common observ.

]


Access Policy: How to Deal with Sensing Errors?

max E[throughput] s.t. collision probability Pc ≤ ζ

Consequences of trusting spectrum sensor:

2 idle sensed as busy ⇒ missed opportunity

2 busy sensed as idle ⇒ collision

Access Policy: when and how much to trust the sensor

tx probability =

p0 if idle

p1 if busy

p0 < 1 : conservative

p1 > 0 : aggressive

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Pro

babi

lity

of D

etec

tion

1 −

δ

δε


Joint PHY-MAC Design: A Constrained POMDP

π∗δ , π

∗s , π

∗c = arg max E[

T∑

t=1

R(t)], subject to Pc ≤ ζ

I Belief vector Λ(t) (conditional distribution)

I is a sufficient statistic

I πδ: Λ(t) → sensor operating point or its PDF

I πs: Λ(t) → sensing action or its PMF

I πc: Λ(t),sensing outcome → access action

I πc: Λ(t),sensing outcome or tx probability 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Pro

babi

lity

of D

etec

tion

1 −

δ

δε

A constrained POMDP often requires randomized policy for optimality.

0Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors,” IEEE Trans. on Info. Theory, May 2008.


Separation Principle

π∗δ , π

∗s , π

∗c = arg max E[

T∑

t=1


Separation principle: πδ and πc can be decoupled from πs

I Choose sensor operating policy πδ and access policy πc

I to maximize immediate reward R(t) and ensure constraint Pc = ζ.

I =⇒ Static optimization problem

I =⇒ Deterministic policy δ∗, π∗c in closed form.

I Choose sensing policy πs to maximize total reward E

[∑T

t=1 R(t)]

.

I =⇒ An unconstrained POMDP

I =⇒ Optimality achieved with deterministic policies.

0Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors,” IEEE Trans. on Info. Theory, May 2008.



π∗δ , π

∗s , π

∗c = arg max E[

T∑

t=1





I =⇒ A static optimization problem

I =⇒ Optimal policies π∗δ, π

∗c with a universal structure in closed form.


[∑T

t=1 R(t)]

.



I =⇒ For i.i.d. channels, myopic has the same semi-universal structure and optimality.


The Optimal Sensor Operating and Access Policies

I when δ > ζ (conservative access)

tx probability =

0 if busyζδ

if idle

I when δ < ζ (aggressive access)

tx probability =

ζ−δ1−δ

if busy

1 if idle

I when δ = ζ (optimal joint design)

tx probability =

0 if busy

1 if idle

PSfrag replacements

ε

1 − δ

1 − ζ

δ > ζ δ < ζ


optimal (δ∗ = ζ)

Optimal policies are deterministic, stationary, and model independent:

δ∗ = ζ, π∗c = trust the sensor.



π∗δ , π

∗s , π

∗c = arg max E[

T∑

t=1





I =⇒ Static optimization problem

I =⇒ Optimal policies π∗δ, π

∗c with a universal structure in closed form.


[∑T

t=1 R(t)]

.



I =⇒ For i.i.d. channels, myopic has the same semi-universal structure and optimality

0Q. Zhao and B. Krishnamachari, “Structure and Optimality of Myopic Policy in Opportunistic Access with Noisy Observations,” submitted to IEEE Transactions on Automatic Control.


Correlated Channels with Imperfect Sensing: Summary

I A constrained POMDP formulation for joint PHY-MAC design

I The separation principle

I Step 1: Design the spectrum sensor and the access policy

2 Being myopic is optimal.

2 A universal structure ⇒ optimality + simplicity + robustness.

I Step 2: Design the sensing policy

2 An unconstrained POMDP.

2 A semi-universal structure ⇒ optimality + simplicity + robustness for

i.i.d. Markov channels.


Outline









Unslotted Primary Systems

PSfrag replacements

White space

Channel 1

Channel N

0

1

2

3

t

t

I N channels, each with bandwidth Bi.

I Channel i: two-state continuous Markov process with transition rates µi, λi.

PSfrag replacements

0 1(busy) (idle)

λi

µi


Reduce to OSA in Slotted Primary Systems

Acknowledgement

¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡

¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢

£ £ £ £ £ £ £ ££ £ £ £ £ £ £ ££ £ £ £ £ £ £ ££ £ £ £ £ £ £ £

¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤¤ ¤ ¤ ¤ ¤ ¤ ¤ ¤

¥ ¥ ¥ ¥ ¥ ¥ ¥ ¥¥ ¥ ¥ ¥ ¥ ¥ ¥ ¥¥ ¥ ¥ ¥ ¥ ¥ ¥ ¥¥ ¥ ¥ ¥ ¥ ¥ ¥ ¥

¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦

§ § § § § § § § § § § §§ § § § § § § § § § § §§ § § § § § § § § § § §§ § § § § § § § § § § §

¨ ¨ ¨ ¨ ¨ ¨¨ ¨ ¨ ¨ ¨ ¨¨ ¨ ¨ ¨ ¨ ¨¨ ¨ ¨ ¨ ¨ ¨

© © © © © ©© © © © © ©© © © © © ©© © © © © ©ª ª ª ª ª ª ª ª ª ªª ª ª ª ª ª ª ª ª ªª ª ª ª ª ª ª ª ª ªª ª ª ª ª ª ª ª ª ª

« « « « « « « « « «« « « « « « « « « «« « « « « « « « « «« « « « « « « « « «

¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬ ¬

® ® ®® ® ®® ® ®® ® ®¯ ¯ ¯¯ ¯ ¯¯ ¯ ¯¯ ¯ ¯

Secondary users

Slot (L)

Sensing Transmission

PSfrag replacements

Opportunities

Channel 1

Channel N

0

1

2

3

Ls L − Ls

t

t

I Secondary users adopt a slotted transmission structure with slot length L.

I A slot is partitioned into sensing time (Ls) and transmission time (L − Ls).

I The chosen channel is an opportunity if it stays idle during the tx period.

I Unslotted tx of primary users absorbed by sensing errors.

I The problem can be reduced to that in a slotted primary system.

0Q. Zhao and K. Liu, “Detecting, Tracking, and Exploiting Spectrum Opportunities in Unslotted Primary Systems,” RWS 2008.


Outline









Energy-Constrained OSA in Fading

Energy Constraint

I Both sensing and access cost energy

I Finite initial energy

Optimal Sensing and Access Policies

I may choose not to sense when the belief vector indicates all channels are

unlikely to be idle

I may choose not to access when channels are in deep fade

0Y. Chen, Q. Zhao, and A. Swami, “Distributed Spectrum Sensing and Access in Cognitive Radio Networks with Energy Constraint” to appear in IEEE Trans. Signal Processing.


Outline









Sharing among Competing Distributed Users

2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

2

2.5

Time (slot)

Nor

mal

ized

net

wor

k th

roug

hput

(Bit

unit

per s

lot)

Multi−user approach in multi−user settingSingle−user approach in multi−user settingSingle−user approach in single−user setting

Tradeoff: choosing the best channel vs. avoiding competing secondary users.

0K. Liu, Q. Zhao, and Y. Chen, “Distributed Sensing and Access in Cognitive Radio Networks,” ISSSTA 2008.


Spatial Opportunity Sharing

° °° °° °± ±± ±± ±

² ²² ²² ²³ ³³ ³³ ³

´ ´´ ´´ ´µ µµ µµ µ

¶ ¶¶ ¶¶ ¶¶ ¶· ·· ·· ·· ·

¸ ¸¸ ¸¸ ¸¹ ¹¹ ¹¹ ¹

PSfrag replacements

CH1 CH2

CH3A(1, 2)

C(3)

B(2)

D(1, 2)

E(1)

I Secondary users cannot use the channels assigned to nearby primary users.

I Neighboring secondary users interfere.

I How to allocate available channels to optimize a certain network utility.

I Tools: graph coloring and game theory.

0H. Zheng and C. Peng, “Collaboration and Fairness in Opportunistic Spectrum Access,” ICC 2005.0W. Wang and X. Liu, “List-coloring based channel allocation for open-spectrum wireless networks,” VTC 2005.


Outline






I Opportunity spectrum access in self-similar primary traffic

2 K. Liu, X. Xiao, and Q. Zhao, “Opportunistic Spectrum Access in Self Similar Primary

Traffic,” IEEE Military Communication Conference (MILCOM), Nov., 2008.



Conclusion

Basic Components:

I Spectrum sensor: opportunity identification (PHY)

I Sensing policy: where in the spectrum to sense (MAC)

I Access policy: whether to tx given that sensing errors may occur (MAC)

Fundamental Tradeoffs

I Spectrum sensor: false alarm vs. miss detection

I Sensing policy: gaining immediate reward vs. gaining spectrum information

I Access policy: conservative vs. aggressive

I Spectrum sharing: choosing the best channel vs. avoiding competing users

c©Q. Zhao and A. Swami, tutorial at MILCOM 2008. 1

Network Layer Issues in Opportunistic Spectrum Access

References

[1 ] W. Ren, Q. Zhao, and A. Swami, “Power Control in Spectrum Overlay Networks: How to Cross AMulti-Lane Highway,” ICASSP 2008.

[2 ] Q. Zhao, W. Ren, and A. Swami, “Spectrum Opportunity Detection: How Good Is Listen-Before-Talk?”Proc. of IEEE Asilomar Conference on Signals, Systems, and Computers, November, 2007.


Long Hop vs. Relaying: We Know This

Whispering: Spatial Reuse Shouting: Fewer Hops


How to Cross A Multi-Lane Highway?

One lane at a time or dash through?


How to Cross A Multi-Lane Highway?

Detecting traffic in multiple lanes is more difficult.


Unique Tradeoffs in Spectrum Overlay

I Transmission power affects how often opportunities occur.

2 Pr[opportunity] decreases exponentially with tx range squared (p2/αtx ).

I Transmission power affects the reliability of opportunity detection.

2 Reliable detection achieved when ptx/Ptx → 0 or ptx/Ptx → ∞.

I Optimal transmission power for transport throughput.

2

0W. Ren, Q. Zhao, and A. Swami, “Power Control in Spectrum Overlay Networks: How to Cross A Multi-Lane Highway,” ICASSP 2008.


Quantification for Poisson Primary Networks






2 p∗tx decreases with the traffic load of primary network.



Network Model

I Primary users form a Poisson point process with density λ.

I Each primary user transmits with probability p in a slot.

I Primary receivers are uniformly distributed within Rp of their transmitters.

PSfrag replacementsRp

Rp

Rp

2 Thinning Thm ⇒ Txs are Poisson.

2 Displacement Thm ⇒ Rxs are Poisson.



PSfrag replacements

A B

Interference

rI

RI

Primary Tx

Primary Rx

I RI: interference rangeI RI: of primary usersI RI: RI ∝ P

1/αtx

I rI: interference rangeI rI: of secondary usersI rI: rI ∝ p

1/αtx


I the transmission from A to B can succeed

I the interference power to primary is below a prescribed level


Spectrum Opportunity: Properties

PSfrag replacements

A B

Interference

rI

RI

Primary Tx

Primary Rx


1/αtx


1/αtx

I Detecting primary signals 6= detecting spectrum opportunity.

I Asymmetric (an opportunity for A −→ B may not be one for B −→ A).

0Q. Zhao, W. Ren, and A. Swami, “Spectrum Opportunity Detection: How Good Is Listen-Before-Talk?” Proc. of IEEE Asilomar Conference on Signals, Systems, and Computers, Nov., 2007.


Probability of Spectrum Opportunity

Pr[opportunity] = Prno rx ≤ rI of A ∩ no tx ≤ RI of B

= exp

−pλ

∫∫

S0(rI+Rp,RI)

SI(r, Rp, rI)

πR2p

rdrdθ + πR2I

PSfrag replacements

A B

rI

RI

d

rI + Rp

S0(rI + Rp, RI)


Impact of Transmission Power on Pr[opportunity]

I Asymptotically Achievable Lower and Upper Bounds:

exp[−pλπ(r2I + R2

I)] < Pr[ opportunity ] ≤ exp(−pλπr2I)

I Pr[opportunity] decreases exponentially with r2I ∝ p

2/αtx .

100 200 300 400 500 6000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

rI

Pr[H

0]

Pr[H0]

exp[−pλπ(rI2+R

I2)]

exp(−pλπrI2)


Spectrum Opportunity Detection

PSfrag replacements

A B

Interference

rI

RI

Primary Tx

Primary Rx


1/αtx


1/αtx

Detecting primary signals 6= detecting spectrum opportunity.


Detecting Primary Signals

PSfrag replacements

A

X

Y B

Primary TxPrimary Rx

rI

rD

Rp + rI

I rD: detection range.

I H0: no primary Tx within rD, H1: alternative.

I False alarms and miss detections occur due to noise and fading.


From Detecting Signal to Detecting Opportunity

! !! !

""" ###

$$ %%

PSfrag replacements

A B

X

YPrimary TxPrimary RxrI

rD

RI

Rp + rI

PSfrag replacements

Prob. of Detection (1 − PMD)

Prob. of False Alarm0

1

1

rD ↓

rD ↑

I H0: opportunity, H1: alternative.

I Even with perfect ears, exposed Tx (X) ⇒ FA, hidden Rx (Y ) ⇒ MD.

I Adjusting detection range rD leads to different operating points.


Detection Performance with Perfect Ears

I False Alarm Probability

PF = 1 − exp

−pλ

πr2

D − SI(d, rD, RI) −

∫∫

SA2

SI(r, Rp, rI)

πR2p

rdrdθ

I Miss Detection Probability

PMD =

exp(−pλπr2D) − exp

[

−pλ

(

π(r2D + R2

I) − SI(d, rD, RI) +∫∫

S0(rI+Rp,RI)−SA2

SI(r,Rp,rI)

πR2p

rdrdθ

)]

1 − exp

[

−pλ

(

∫∫

S0(rI+Rp,RI)

SI(r,Rp,rI)

πR2p

rdrdθ + πR2I

)]


Asymptotic Properties of ROC

I Reliable opportunity detection is achieved in two extreme regimes:

2 The point (PF (rD = RI), PD(rD = RI)) → (0, 1) when ptx/Ptx → 0.

2 The point (PF (rD = rI − RI), PD(rD = rI − RI)) → (0, 1) when ptx/Ptx → ∞.

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

PF

PD

LBT (rI = 50)

LBT (rI = 300)

LBT (rI = 800)


Optimal Power for Transport Throughput

p∗tx = arg max d(ptx) Pr[ success | ptx] s.t. Pr[ collision | ptx] ≤ ζ

& && &' '' '

( (( () )) )*+*,+,

-+- .+.

PSfrag replacements

A B

Interference

rI

RI

Tx

Rx

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

PF

PD

LBT (rI = 50)

LBT (rI = 300)

LBT (rI = 800)

PSfrag replacements

A

B

InterferencerI

RI

TxRx


Numerical Examples

0 0.5 1 1.5 2 2.5 3 3.50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Normalized Hop Length

Nor

mal

ized

Tra

nspo

rt C

apac

ity

low primary traffic loadhigh primary traffic load

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10.5

1

1.5

2

2.5

3

p (λ = 10/2002)

r* I/RI


Conclusion








Programs, Policies, Standards

Programs Programs • Policies

St d d • Standards

1

CN Programs


CN Programs

• NSF Programmable Wireless Networks (ProWiN)www.nsf.gov/cise/cns/prowin.jsp

• European research initiative: end-to-end reconfigurability (E2R) e2r.motlabs.com/front-page

• WINNER: Wireless world initiative new radio www.ist-winner.org• U.S. National Institute of Justice

www.ojp.usdoj.gov/nij/topics/technology/communication/research-priorities.htm

• U.S. Army Spectrum Exploitation Program• DARPA XG and related programs• CERDEC CN StudyCERDEC CN Study • FCC www.fcc.gov/oet/spectrum• ITU www.itu.int/publications/main_pub/frequency_html• …

2


DARPA Programs related to DSA / CR

3Source: Chris Ramming’s presentation at WAND Proposer day. Feb 2006, online atwww.darpa.mil/STO/Solicitations/WAND/pdf/CBMANET_WAND_proposer_day_briefing.pdf


DARPA XG Program

Develop Technology and System Concepts for DoD to Dynamically Access All Available SpectrumDynamically Access All Available Spectrum

• Complexity of planning large-scale dynamic networks• Spectrum regulation policies and processes vary by

country region even by DoD unit country, region, even by DoD unit – Military: Purple problem; Coalition planning, with NGO’s

• Interference prevention and coexistence with QoS and cost constraintscost constraints

Basic Components: • Real-time low-power wideband spectrum sensing• Rapid characterization of signals and waveforms• React and Adapt: rapid formulation of best course of action

Credit: Preston Marhsall, XG Briefings, e.g., SWANS Conf., April 2005

4

http://www.daml.org/meetings/2005/04/pi/DARPA_XG.pdf

Cognitive Network Study at CERDEC


Cognitive Network Study at CERDEC

Dynamic Spectrum Utilization

CNetwork ControlSoftware Radio

Evolution Network Operations

Sources: David Jimenez Director (A)

Network OperationsNetwork Learning /

Reasoning

David Jimenez, Director (A), CERDEC STCD, “STCD Future Efforts & Direction”, 1 AFCEA Luncheon,7 Jan 2008. Derek S. Morris, CERDEC STCD, “Cognitive Networking for Tactical Army Applications” Tekes

5

Army Applications , Tekes Workshop, Mar 2008.



• Programs • Programs Policy – is a deliberate plan of action to guide decisions and achieve to guide decisions and achieve rational outcome(s) St d d • Standards

6


Examples of Policyp y

802.22 90% detection within 2 seconds at prescribed power level

UWBFCC t l kFCC spectral mask

WNaN WNaN Maintain 3dB SNR margin at protected receiversVacate channel within 500 ms

7

Vacate channel within 500 ms


Many sources of policy

Regulatory policy: interference avoidance; protection to emergency services incumbents and licenseesto emergency services, incumbents and licensees

Military policy: Mission needs; C2 priorities; CSI avoidanceavoidance

Radio policy: control radio parameters

Policy implementations may differ e g back off Policy implementations may differ e.g., back-off window durations; reactive-proactive routing …

Policy Reasoning Engine to verify policy, detect Policy Reasoning Engine to verify policy, detect conflicts, negotiate and resolve conflicts

Yes, No, possibly with additional constraints

8


Typical Policy Life CycleTypical Policy Life CycleHigh Level

Specification of Operational

Policy System Translates OP

Policy System Analyzes OPOperational

Policies (OP) I eInto Machine-ReadableOperational Policies

y

for conflicts/errors

Policy System RefinesOP Into

Policy System Validates

Compliance

Graphic Adapted from ITA briefing by Dinesh Verma, Dakshi Agrawal et al

OP Into Deployable Policies

In Post-Mortem Some CR Policy Languages:OWL, CoRaL

Policy System Distributes

Deployable PoliciesTo MANET Devices

MANET Devices EnforcePolicies

Policy Systemupdates

Devices to PoliciesEffective Post Operation

9


Policy Representation

Issues:Must provide rich interfaceExpressiveness vs. applicability to CRInteroperability and testability V&V Interoperability and testability – V&V Security issues with dynamic policy updates

Examples: Examples: UML, XTM, RDF, RDFS, …. OWL – Web-based Ontology Language (BBN in XG) CoRaL – Cognitive Radio Language (SRI in XG)

Berlemann et al, DySPAN 05Chapin-Sicker, IEEE Comm Mag 06; Wilkins et al IEEE Wireless Comm 07

10

Wilkins et al, IEEE Wireless Comm 07Kokar, The Role of Ontologies in Cognitive Radioin Cognitive Radio Technology, ed., B. Fette, 2006.ITA: www.usukita.org

XG Policy Reasoner


XG Policy Reasoner

Source: P Marshall and T Martin “XG Communications Program

11

Source: P. Marshall and T. Martin, XG Communications ProgramOverview”, at WAND Industry Day, 27 Feb 2007, available atwww.darpa.mil/STO/Solicitations/WAND/pdf/XG_overview_for_WAND.pdf



• Programs • Programs • Policy

St d dStandards

12


Commercial Standards

• 802 22 www ieee802/22• 802.22 www.ieee802/22• Also various 802.11x and 802.16x

• IEEE P1900 and SCC41

13


802.22

Wireless Regional Area Networks (WRAN)U TV b d 54 t 862 MH• Uses TV bands 54 to 862 MHz

• Provides 6 – 7 – 8 MHz bands• 4 W EIRP default • Explicit requirements :

– Avoid interference with incumbents – Channel sensing and measurements g– Coexist with DTV and wireless microphones

• Explicit sub-slots to signal p g`Urgent coexistence’ (PU detected) and `Self-coexistence’ (another WRAN cells detected)

14


802.22 PHY/MAC Draft

Largely based on 802.16

• Modulation – OFDMA (IEEE 802.16d/e – WiBro) – QPSK, 16-QAM, 64-QAM– Channel bonding and aggregation optional– Adaptive sub-carrier allocation– Adaptive pilot insertion– Channel coding: LDPC, STBC, Turbo – 2048 carriers: 1440 modulated

• MAC also based on 802.16 standard

15


Other standards with `CR’ featuresOther standards with CR features

802.11h - Dynamic Frequency Selection (DFS) and transmit power control (TPC) to avoid interference with radar and satellite control (TPC) to avoid interference with radar and satellite

• Quiet channels to test for presence of radar• Test channels for radar before and during use

http://standards.ieee.org/getieee802/download/802.11h-2003.pdf

802.11y – 3.65-3.7 GHz band (fixed satellite / radar band)• Provides for spatial exclusion zones• Location-based policy (e.g., near borders)• Must protect incumbent users • Must protect incumbent users

802.16 h – Explicit CR protocol, with co-existence and interference avoidance

802.11j - 802.11 designed specially for to conform to local policiesin Japan; operates in 4.9-5 GHz band.

16


C l i D i S t AConclusion: Dynamic Spectrum Access

Dynamic Dynamic Spectrum

Access

Exclusive Use Model

Open Sharing Model


Spectrum Property

Rights



Spectrum Overlay

(OSA, pooling)

18Cognitive Radio


C n l i n: Sp tr m O rlConclusion: Spectrum OverlayPhysical Layer

Opportunity sensingInterference Aggregation

MAC LayerOpportunity tracking and learningOpportunity exploitation with imperfect sensingOpportunity exploitation with imperfect sensingOpportunity sharing

Network LayerNetwork LayerPower control and routing

R l t P li19

Regulatory Policy


20

IEEE Explore: CR & SDR pubs


21

Google TrendsCR:South Korea ------------------------Taiwan -----India -----China --USA -

SDR: DenmarkS. KoreaCzech RepSwedenRomania


S Op ISome Open IssuesCo-existence issuesHardware Issues: PHY – RF Security and privacy Models and measurements Increased cross-layer interactions Multicast …Policy translators yAll the usual radio issues with a twist True MANET & co-existence still far away ?

20

y

zhao swami mil com 08 tutorial

Documents

spectrum opportunities

spectrum usa

government use

ti p i t

o t allregulation

t rmdiverse ideas

applications g g p

limit d s ppllimited