an enhanced covering lemma for multiterminal source...
TRANSCRIPT
![Page 1: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/1.jpg)
2006-10-26 ITW 2006 1
An Enhanced Covering Lemma for Multiterminal Source Coding
Shengtian Yang and Peiliang QiuDept. of ISEE
Zhejiang University
![Page 2: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/2.jpg)
2006-10-26 ITW 2006 2
Overview
The history of multiterminal source coding
The covering lemma for a Markov chain
Some ideas in the proof of multiterminalsource coding
![Page 3: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/3.jpg)
2006-10-26 ITW 2006 3
The Problem of Multiterminal Source Coding
Encoder 1
Encoder 2
Encoder m
Decoder
nS
nMX
2nX
1nX 1
nY
2nY
nMY
1R
2R
mR
![Page 4: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/4.jpg)
2006-10-26 ITW 2006 4
Slepian and Wolf, 1973, for stationary and memoryless sources
Cover, 1975, for stationary and ergodicsourcesMiyake and Kanaya, 1995, for general sources
The Lossless Case: Slepian-Wolf Coding
![Page 5: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/5.jpg)
2006-10-26 ITW 2006 5
The Lossy Case for Discrete Sources: The Berger-Tung Inner/Outer Bounds
![Page 6: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/6.jpg)
2006-10-26 ITW 2006 6
![Page 7: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/7.jpg)
2006-10-26 ITW 2006 7
Some special cases
Wyner-Ziv coding: Lossy source coding with uncoded side information at the decoder
Wyner and Ziv, 1976, for stationary memorylesssourcesIwata and Muramatsu, 2002, for general sources with maximum distortion criteria
Multiterminal source coding with one distortion criterion, Berger and Yeung, 1988Multiterminal source coding with uncoded side information at the decoder, Gastpar, 2004
![Page 8: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/8.jpg)
2006-10-26 ITW 2006 8
The Covering Lemma for a Markov Chain
Lemma 4.3, Wyner, “On source coding with side information at the decoder”, 1975Wyner and Ziv, “The rate-distortion function for source coding with side information at the decoder”, 1976Iwata and Muramatsu, “An information-spectrum approach to rate-distortion function with side information”, 2002
![Page 9: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/9.jpg)
2006-10-26 ITW 2006 9
![Page 10: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/10.jpg)
2006-10-26 ITW 2006 10
The Covering Lemma in Iwata and Muramatsu’s paper
![Page 11: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/11.jpg)
2006-10-26 ITW 2006 11
![Page 12: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/12.jpg)
2006-10-26 ITW 2006 12
To Understand It
![Page 13: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/13.jpg)
2006-10-26 ITW 2006 13
To Enhance It
![Page 14: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/14.jpg)
2006-10-26 ITW 2006 14
![Page 15: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/15.jpg)
2006-10-26 ITW 2006 15
The main idea of the proof of this enhanced lemma is a combination of the ideas in the proofs of Iwata’s covering lemma and Han’s Theorem 5.5.1 in his book “Information-Spectrum Methods in Information Theory”.
![Page 16: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/16.jpg)
2006-10-26 ITW 2006 16
![Page 17: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/17.jpg)
2006-10-26 ITW 2006 17
Multiterminal Source Coding
By the enhanced lemma, we can then obtain the sufficient and necessary condition for determining the achievability of rate-distortion tuples for general sources with one average distortion criterion and multiple maximum distortion criteria under fixed-length coding.See Theorem 1 and Remark 2 in the paper.
![Page 18: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/18.jpg)
2006-10-26 ITW 2006 18
Average distortion criterion and maximum distortion criterion
![Page 19: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/19.jpg)
2006-10-26 ITW 2006 19
Some ideas in our proof of multiterminalsource coding
1nX
2nX
Q1 SW11nZ
Q2 SW22nZ
SW-1 1nZ
2nZ
h1
h2
Encoder 1
Encoder 2
Decoder
1nY
2nY
Two stages of coding:Quantization (Lemma 3)Slepian-Wolf coding (Random binning)
![Page 20: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/20.jpg)
2006-10-26 ITW 2006 20
![Page 21: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/21.jpg)
2006-10-26 ITW 2006 21
Using the Covering Lemma to Prove Lemma 3
1nX 1
nZ
2nX 2
nZ
1nX (1)
1( )nnF X
2nX 2
nZ
![Page 22: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/22.jpg)
2006-10-26 ITW 2006 22
1nX (1)
1( )nnF X
2nX 2
nZ
1nX (1)
1( )nnF X
2nX (2)
2( )nnF X
![Page 23: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/23.jpg)
2006-10-26 ITW 2006 23
Reference
1. T. Berger, “Multiterminal source coding,” in The Information Theory Approach to Communications. New York: Springer-Verlag, July 1977, pp. 171–231.
2. T. Berger and R. W. Yeung. Multiterminal Source Encoding with One Distortion Criterion. IEEE Trans. Inform. Theory, 35(2):228–236, 1989.
3. T. M. Cover, “A proof of the data compression theorem of Slepianand Wolf for ergodic sources,” IEEE Trans. Inform. Theory, vol. 21, no. 2, pp. 226-228, Mar. 1975.
4. M. Gastpar, “The Wyner-Ziv problem with multiple sources,” IEEE Trans. Inform. Theory, vol. 50, no. 11, pp. 2762–2768, Nov. 2004.
5. T. S. Han, Information-Spectrum Methods in Information Theory. Berlin: Springer, 2003.
![Page 24: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/24.jpg)
2006-10-26 ITW 2006 24
6. K. Iwata and J. Muramatsu, “An information-spectrum approach to rate-distortion function with side information,” IEICE Trans. Fundamentals, vol. E85-A, no. 6, pp. 1387–1395, June 2002.
7. S. Miyake and F. Kanaya, “Coding theorems on correlated general sources,” IEICE Trans. Fundamentals, vol. E78-A, no. 9, pp. 1063-1070, Sept. 1995.
8. D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inform. Theory, vol. 19, no. 4, pp. 471-480, July 1973.
9. A. D. Wyner, “On source coding at the with side information decoder”, IEEE Trans. Inform. Theory, Vol. 21, No. 3, pp. 294-300, May 1975.
10. A. D. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,” IEEE Trans. Inform. Theory, vol. 22, no. 1, pp. 1–10, Jan. 1976.
![Page 25: An Enhanced Covering Lemma for Multiterminal Source Codingarxiv.codlab.net/paper/legacy/YANGST_ITW2006_MSC.pdf · 2006-10-26 ITW 2006 1 An Enhanced Covering Lemma for Multiterminal](https://reader036.vdocuments.mx/reader036/viewer/2022062415/6052264573d5f066d07f8f68/html5/thumbnails/25.jpg)
2006-10-26 ITW 2006 25
Thank you!