![Page 1: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/1.jpg)
Soft-Covering ExponentPaul Cuff and Semih Yagli
![Page 2: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/2.jpg)
Soft Covering
- Opposite of Reliable Decoding - What happens when rate is too high?
Noise
X1
Y1
Noise
X2
Y2
Noise
X3
Y3
Noise
X4
Y4
i.i.d.
i.i.d.
Codebook
i.i.d.?
![Page 3: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/3.jpg)
Output Distribution
PY
PY|C
x(1) x(2) x(3)x(4)x(5)
![Page 4: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/4.jpg)
Gaussian Example
![Page 5: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/5.jpg)
Gaussian Example
![Page 6: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/6.jpg)
Soft Covering (origin)
Theorem 6.3 of Wyner’s Common Information Paper
R > I(X;Y) produces the phenomenon
![Page 7: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/7.jpg)
Various Names
Covering [Ahlswede, Winter, Wilde]
Sampling Lemma [Winter]
Resolvability [Han, Verdu]
Cloud Mixing
![Page 8: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/8.jpg)
Applications
Security and Privacy
e.g. wiretap channel
Encoder analysis
Common Information
![Page 9: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/9.jpg)
Main Result
![Page 10: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/10.jpg)
Previous lower bounds
Notable bounds by Hayashi, Han, Verdu, Cuff
Best previous:
![Page 11: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/11.jpg)
TV vs KL-divergence
Inspired by [Parizi, Telatar, Merhav 17]
Solved exponent for KL-divergence
![Page 12: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/12.jpg)
Features of TV
Accepted standard for cryptography
Tractable multi-stage encoder analysis
Stronger concentration result
![Page 13: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/13.jpg)
New Features of TV Proof
“Typical set” approach is not optimal
Poisson codebook size used in converse
(Taylor expansion not possible for absolute value)
![Page 14: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/14.jpg)
Equivalence
Carefully swap min’s and max’s
Massage
[R�D(QXY kPXQY )]+ = max�2[0,1]
�(R�D(QXY kPXQY ))<latexit sha1_base64="(null)">(null)</latexit><latexit sha1_base64="(null)">(null)</latexit><latexit sha1_base64="(null)">(null)</latexit><latexit sha1_base64="(null)">(null)</latexit>
![Page 15: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/15.jpg)
Proof of Error Upper Bound
![Page 16: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/16.jpg)
Illustration of codebook approximation
Ynyn
![Page 17: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/17.jpg)
Conditional Type
0 0 0 0 1 0 0 00 1 0 1 1 1 0 11 1 1 1 1 0 0 0
yn
xn same conditional type
![Page 18: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/18.jpg)
Polynomial Number of Types
Length-n binary types
fraction of 1’s0 1/n 2/n 1
Length-n ternary types
fraction of 1’s
fraction of 2’s
1
1
![Page 19: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/19.jpg)
Key Quantities
![Page 20: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/20.jpg)
Bound each type in one of two ways
![Page 21: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/21.jpg)
Proof of Error Lower Bound
![Page 22: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/22.jpg)
Poisson Trick
Let the number of codewords M be Poisson distributed
Take care of this assumption at the end
![Page 23: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/23.jpg)
Another Look
Need independence for different types
![Page 24: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/24.jpg)
A Simple Lemma{Xi} independent
E[Xi]=0
E�����X
i
Xi
����� � maxi
E |Xi|<latexit sha1_base64="(null)">(null)</latexit><latexit sha1_base64="(null)">(null)</latexit><latexit sha1_base64="(null)">(null)</latexit><latexit sha1_base64="(null)">(null)</latexit>
![Page 25: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/25.jpg)
Initial Details
![Page 26: Soft-Covering Exponent - Princeton Universitycuff/publications/cuff_ita_2019.pdfInspired by [Parizi, Telatar, Merhav 17] Solved exponent for KL-divergence. Features of TV Accepted](https://reader033.vdocuments.mx/reader033/viewer/2022052000/601305915ca1235f3916231a/html5/thumbnails/26.jpg)
Poisson to Fixed Codebook
Codebook Size (M)
codebook size PMFexponentially small
(still too big to be bad)
μ