Networking Seminar ECE@OSU

Network Information Flow R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung.

Network Information Flow. IEEE Transactions on Information Theory, 46(4):1204–1216, 2000.

(Referred to as this paper throughout the presentation)

Presented byBo Ji (ji@ece.osu.edu)

11-28-2007

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

Some of the slides are from Prof. Chih-Chun Wang@Purdue & Communications and Multimedia Laboratory@NTU

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

Introduction• In existing computer networks, each node functions as a switch

either relays or replicates information received.(Store and forward)

• However, from the information-theoretic point of view, there is no reason to restrict the function of a node to that of a switch.

• Rather, a node can function as an encoder in the sense that it receives information from all the input links, encodes, and sends information to all the output links.

(Store, code and forward)• We refer to coding at a node in a network as network coding.

• Basic idea of network coding- Coding at intermediate nodes

Introduction• Contributions:

– Introduce a new class of problems called network information flow.

– Obtained a simple characterization of the admissible coding rate region for single-source problem.(Max-Flow Min-Cut Theorem for network information flow)

– Show that o the traditional technique for multicasting in a computer network in

general is not optimal.o Rather, we should think of information as being “diffused” through

the network from the source to the sinks by means of network coding. – Show that very simple optimal codes do exist for certain

networks.(Actually, Li, Yeung and Cai [1] have devised a systematic procedure to construct linear codes for acyclic networks. )

[1] Ref: S. Y. R. Li, R. W. Yeung, and N. Cai. Linear Network Coding. IEEE Transactions on Information Theory, 49(2):371 – 381, 2003.

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

Max-Flow Problem

Max-Flow Problem

Ford, Fulkerson

Notations

Properties of Flow

Max-Flow

The Capacity of A Cut

Min-Cut

• The cut with the minimum value of capacity in G.

Example

Example

Example

Example

Max-Flow Min-Cut Theorem

Max-Flow Min-Cut Theorem

• Unicast Case:– MinCut Bound, Menger’s Theorem and Path Packing.

• Broadcast Case:– Edmond’s Theorem and Spanning Tree Packing.

• Multicast Case:– Steiner Tree Packing and Non-Achievability of MinCut

Bound.– Unlike the broadcast case, the upper bound may not

be achievable by routing information through a set of edge-disjoint trees.

Is there any other way being able to achieve the upper bound?

A single-level diversity coding system

A single-level diversity coding system The graph representing the coding system Let X1 ,…, Xi be mutually independent

information sources and the information rate of X1 ,…,Xi is denoted by h1 ,…, hi. We drop the subscripts of X1 and h1, since there is only one-single source.

Let ri be the coding rate of encoder i. In order for a decoder to reconstruct X, it is necessary that the sum of the coding rates of the encoder accessible by this decoder is at least h.

Max-Flow in Multicast

Max-Flow Min-Cut Theorem• Conjecture 1: Let G = (V,E) be a graph with

source s and sinks t1 ,……,tL, and the capacity of an edge (i, j) be denoted by Rij. Then (R,h,G) is admissible if and only if the values of a max-flow from s to tl , l = 1,……,L are greater than or equal to h, the rate of the information source.

• The spirit of this conjecture resembles that of the celebrated Max-flow Min-cut Theorem in graph theory[2].

• Illustrate Conjecture 1 by a few examples[2] Ref : B. Bollobas, Graph Theory, An Introductory Course. New York: Springer-Verlag, 1979.

A one-source one-sink network(L=1)

Capacity Max-Flow

A one-source two-sink network without coding (L=2)

h ≤ min(5,6)

A one-source two-sink network with coding (L=2)

A one-source three-sink network(L=3)

Quantified advantage:•Save bandwidth: 10%•Increase throughput: 1/3

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

Main Result• Let us consider a block code of length n.• We assume that x, the value assumed by X, is obtained

by selecting an index from a set Ω with uniform distribution.

• The elements in Ω are called messages. • For (i,j) є E, node i can send information to node j

which depends only on the information previously received by node i.

• The paper confines its discussion to a class of block codes, called the α-code.

α-Code

• An on a graph G is defined by the following components:

( , ( , ( , ) ), )ijn i j E h code

Construction of α-Code• At the beginning of the coding session, the value of X is available to node

s. • In the coding session, there are K transactions which take place in

chronological order, where each transaction refers to a node sending information to another node.

• In the kth transaction, node u(k) encodes according to fk and sends an index in Ak to node v(k) . The domain of fk is the information received by node u(k) so far, and we distinguish two cases:– If u(k) = s , the domain of fk is Ω. – If u(k) ≠ s , Qk gives the indices of all previous transactions for which

information was sent to node u(k) , so the domain of fk is ∏k’є QkAk’

• The set Tij gives the indices of all transactions for which information is sent from node i to node j, so ηij is the number of possible index-tuples that can be sent from node i to node j during the coding session.

• Finally, Wl gives the indices of all transactions for which information is sent to tl, and gl is the decoding function at tl.

Theorem 1

• Theorem 1: *

, ,h G h G

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

An Example

Show that very simple optimal codes do exist for certain cyclic networks.(A kind of convolutional code)

A Simple Optimal Coding Scheme• Show a coding scheme that can multicast {x0(k), x1(k), x2(k)} from the

source to the sinks. • xl(k) = 0 for k ≤ 0. At time k ≥ 1, information transactions occur in the

following order:

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

Multiple Sources

The multisource problem is not a trivial extension of the single-source problem, and it is extremely difficult in general.

A multilevel diversity coding system The graph G representing the coding system

Multiple Sources

• Assume the sources X1 and X2 are independent, and h1=h2=1.

• For any admissible coding rate triple (r1,r2,r3), for i = 1,2,3, we can writewhere are the subrates associated with sources X1 and X2 respectively.

• X1 is multicast to all the decoders, and X2 is multicast to Decoder 2,3,4. So we have following constraints:

Multiple Sources• [3] shows that the rate triple (1,1,1) is admissible, but

it cannot be decomposed into two sets of subrates as prescribed above. Therefore, coding by superposition is not optimal in general, even when the two information sources are generated at the same node.

• How to characterize multilevel diversity coding systems for which coding by superposition is always optimal is still an open problem.

• Although the multisource problem in general is extremely difficult, there exist special cases which can be readily solved by the results for the single-source problem (Such as video-conferencing scenario).

[3] Ref: R.W. Yeung, “Multilevel diversity coding with distortion,” IEEE Trans. Inform. Theory, vol. 41, pp. 412–422, Mar. 1995.

Outline

• Introduction• Max-Flow Min-Cut Theorem• Main Result• Simple optimal codes• Multiple sources case• Discussion and Conclusion

Conclusion• have characterized the admissible coding rate region

of the single-source problem.• result can be regarded as the Max-flow Min-cut

Theorem for network information flow.• the discussion is based on a class of block codes called

α-codes. Therefore, it is possible that the result can be enhanced by considering more general coding schemes.

• prove that probabilistic coding does not improve performance.

• The problem becomes more complicated when there are more than one source.

Conclusion

• The most important contribution of this paper is to show that – the traditional technique for multicasting in a

computer network in general is not optimal.– Rather, we should think of information as being

“diffused” through the network from the source to the sinks by means of network coding.

• This is a new concept in multicasting in a point-to-point network which may have significant impact on future design of switching systems.

Interesting Problems

• by imposing the constraint that network coding is not allowed, i.e., each node functions as a switch in existing computer networks, whether a rate tuple R is admissible?

• under what condition can optimality be achieved without network coding?

• Besides, the problem of representing codes in graphs has received much attention.

• ……

Discussion

• Question:– How to identify when network coding solution is

needed and feasible?

Two Simple Unicast Sessions

Goal: X1: S1 t1X2: S2 t2

When can we send X1 and X2 simutaneously?(2-2 case)

Ref: C. Wang and N B. Shroff, " Beyond the Butterfly - A Graph-Theoretic Characterization of the Feasibility of Network Coding with Two Simple Unicast Sessions, " IEEE International Symposium on Information Theory, Nice, France, June 2007.

Two Simple Multicast Sessions

• It can be generalized to two simple multicast sessions (2-m case)

– C.-C. Wang, N.B. Shroff, “Intersession Network Coding for Two Simple Multicast Sessions,” in 45-th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, September 26 – 28, 2007

Two Simple Multicast Sessions

• Settings

Two Simple Multicast Sessions

• Main Theorem

Further…

• A more general characterization for problems with more than two multicast sessions is still an on-going research.

Thank you ! !