8/14/2019 10.1.1.137.871

1/12

5 -1

Chapter 5

Lempel-Ziv Codes

To set the stage for Lempel-Ziv codes, suppose we wish to find the best block code for compressing a

datavector X. Then we have to take into account the complexity of the code. We could represent the totalnumber of codebits at the decoder output as:

[# of codebits to describe block code] + [# of codebits from using code on X]

The codebits used to describe the block code that is chosen to compress X form a prefix of the encoderoutput and constitute what is called the overhead of the encoding procedure. If we wish to choose the best

block code for compress ing X, from among block codes of all orders, we would choose the block code inorder to minimize the total of the overhead codebits and the encoded datavector codebits. One could also

adopt this approach to code design in order to choose the best finite memory code for compressing X, or,more generally, the best finite-state code.

EXAMPLE 1. Suppose we wish to compress English text using finite memory codes. A finite memorycode of order zero entails 51 bits of overhead. (Represent the Kraft vector used as a binary tree with 26

terminal nodes and 2 x 26 1 = 51 nodes all together. You have to let the decoder know how to grow this

tree it takes one bit of information at each of the 51 nodes to do that, since the decoder will either grow two

branches at each node, or none.) A fini te memory fi rs t order code for English text wil l entail 27 x 51 = 1377

bi ts of overhead. (You need a codebook of 27 different codes, with 51 bit s to describe each code.) A finitememory second order code for English text can be described with 677 x 51 = 34527 bits of overhead. (There

are 26 x 26 + 1 = 677 codes in the codebook, in this case.) You would keep increasing the order of your finite

memory code until you find the order for which you have minimized the sum of the amount of overhead plus

the length of the encoded English text via the best finite memory code of that order.

It would be nice to have a compression technique that entails no overhead, while performing at least as

well as the block codes, or the finite memory codes, or the finite-state codes (provided the length of the

datavector is long enough). Overhead is caused because statistics of the datavector (consisting of various

frequency counts) are collected first and then used to choose the code. Since the code arrived at depends on

these statistics, overhead is needed to describe the code. Suppose instead that information about the

datavector is collected "on the fly" as you encode the samples in the datavector from left to right in

encoding the current sample (or group of samples), you could use information collected about the

prev ious ly en coded sa mples. A code which op erat es in this way might no t ne ed an y overhe ad to describe it.

Codes like this which require no overhead at the decoder output are called adaptive codes. The Lempel-Ziv

code, the subject of this chapter, will be our first example of an adaptive code. There are quite a number of

variants of the Lempel-Ziv code. The variant we shall describe in this chapter is called LZ78, after the date

of the paper [1].

5.1 Lempel-Ziv Parsing

In block coding, you first partition the datavector into blocks of equal length. In Lempel-Ziv coding, you start by

partitioning the datavector into variable-lengthblocks instead. The procedure via which this partitioning

8/14/2019 10.1.1.137.871

2/12

5 - 2

takes place is called Lempel -Z iv pa rs ing. The first variable-length block arising from the Lempel-Ziv

pa rs ing of the da tavect or X = (X1, X2, .. . , Xn) is the single sample X1. The second block in the parsing is

the shortest prefix of (X2, .. . , Xn) which is not equal to X1. Suppose this second block is (X2, .. . , Xj).

Then, the third block in Lempel-Ziv parsing will be the shortest prefix of (Xj +1 , ... , Xn) which is not equal

to either X1 or (X2, .. . , Xj). In general, suppose the Lempel-Ziv parsing procedure has produced the first k

variable-length blocks B1, B2, .. . , Bk in the parsing, and X(k) is that part left of X after B1, B2, .. . , Bk have

be en re moved. Then the ne xt bloc k Bk +1 in the parsing is the shortest prefix of X(k) which is not equal toany of the preceding blocks B1, B2, .. . , Bk. (If there is no such block, then Bk+1 = X(k) and the Lempel-Ziv

pa rs ing proc ed ure te rmin ates .)

By construction, the sequence of variable-length blocks B1, B2, .. . , B tproduced by the Lempel -Ziv pars -

ing of X are distinct, except that the last block Bt could be equal to one of the preceding ones.

EXAMPLE 2. The Lempel-Ziv parsing of X = (1, 1, 0,1,1, 0, 0, 0,1,1, 0, 1) is

B1 = 1, B2 = 10, B3 = 11, B4 = 0, B5 = 00, Bs = 110, B7 = 1 (1)

This parsing can also be accomplished via MATLAB. Here are the results of a MATLAB session that the

reader can try:

x=[1 1 0 1 1 0 0 0 1 1 0 1];

y=LZparse(x);p r i n t _ b i t s t r i n g s ( y)

1

1 0

1 1

0

0 0

110

1

The MATLAB function LZparse(x) (the m-file of which is given in Section 5.6) gives the indices of

the variable-length blocks in the Lempel-Ziv parsing of the datavector x. Using the MATLAB function

pr int bit st ring s , we were able to print out the blocks in the parsing on the screen.

5.2 Lempel-Ziv Encoder

We suppose that the alphabet from which our datavector X = (X1, X2, .. . , Xn) is formed is A = {0, 1, .. . , k-1}, where k is a positive integer. After obtaining the Lempel-Ziv parsing B1, B2, .. . , B tof X, the next stepis to represent each block in the parsing as a pair of integers. The first block in the parsing, B1, consists of

a single symbol. It is represented as the pair (0, B1). More generally, any block Bj of length one is

represented as the pair (0, Bj). If the block Bj is of length greater than one, then it is represented as the pair

(i, s), where s is the last symbol in Bj and Bi is the block in the parsing which coincides with the block

obtained by removing s from the end of Bj. (By construction of the Lempel-Ziv parsing, there will always

be su ch a bloc k B i.)

EXAMPLE 3. The sequence of pairs corresponding to the parsing (1) is

(0, 1), (1, 0), (1, 1), (0, 0), (4, 0), (3, 0), (0, 1) (2)

For example, (4, 0) corresponds to the block 00 in the parsing. Since the last symbol of 00 is 0, the pair

(4, 0) ends in 0. The 4 in the first entry refers to the fact that B4 = 0 is the preceding block in the parsing

which is equal to what we get by deleting the last symbol of 00.

For our next step, we replace each pair (i, s) by the integer ki+s. Thus, the sequence of pairs (2) becomes the sequence

of integers

8/14/2019 10.1.1.137.871

3/12

2~0+1=1,21+0=2,21+1=3,20+0=0,24+0=8,23+0=6,20+1=1 (3)

5-3

To finish our description of the encoding process in Lempel-Ziv coding, let I1, I2, ... , Itdenote the integers

corresponding to the blocks B1, B2, ... , Bt in the Lempel-Ziv parsing of the datavector X. Each integer Ij is

expanded to base two and these binary expansions are "padded" with zeroes on the left so that the overalllength of the string of bits assigned to Ij is dlog2(kj)e. The reason why this many bits is necessary andsufficient is seen by examining the largest that Ij can possibly be. Let ( i, s) be the pair associated with Ij.Then the biggest that i can be is j 1 and the biggest that s can be is k 1. Thus the biggest that Ij can beis k ~ (j 1) + k 1 = kj 1, and the number of bits in the binary expansion of kj 1 is dlog2(kj)e. LetWjbe the string of bits of length dlog2(kj)e assigned toIj as described in the preceding. Then, the Lempel-Zivencoder output is obtained by concatenating together the strings W1, W2, . .. , Wt.

To illustrate, suppose a binary datavector has seven blocks B1, B2, ... , B7 in its Lempel-Ziv parsing (suchas in Example 2). These blocks are assigned, respectively, strings of codebits W1, W2, W3, W4, W5, W6, W7 oflengths dlog2(2)e = 1 bits, dlog2(4)e = 2 bits, dlog2(6)e = 3 bits, dlog2(8)e = 3 bits, dlog2(10)e = 4 bits,

dlog2(12)e = 4 bits, and dlog2(14)e = 4 bits. Therefore, any binary data vector with seven blocks in its Lempel-

Ziv parsing would result in an encoder output of length 1 + 2 + 3 + 3 + 4 + 4 + 4 = 21 codebits. In particular,

for the datavector in Example 2, the seven strings W1, . .. , W7 are (referring to (3)):

W1 = 1W2 = 10W3 = 011W4 = 000W5 = 1000W6 = 0110

W7 = 0001

Concatenating, we see the encoder output from the Lempel-Ziv coding of the datavector in Example 1 is

110011000100001100001.

5.3 Lempel-Ziv Decoder

Suppose a datavectorX with alphabet f0, 1, 2g was Lempel-Ziv encoded and the encoder output is:

001000010101010110000100000 (4)

Let us decode to getX. For an alphabet of size three, dlog2(3j)e codebits are allocated to thej-th block in the Lempel-Ziv parsing. This gives us the following table of codebit allocations:

codebit allocation table

parsing block number # of codebits

1 2

2 3

3 4

4 4

5 4

6 5

7 5

8 5

Partitioning up the encoder output (4) according to the allocations in the above table, we obtain the partition:

00,100, 0010,1010,1011, 00001, 00000

8/14/2019 10.1.1.137.871

4/12

5 - 4

Converting these to integer form we get:

0, 4, 2, 10, 11, 1, 0

Dividing each of these integers by three and recording quotient and remainder in each case, we get the pairs

(0, 0), (1, 1), (0, 2), (3, 1), (3, 2), (0, 1), (0, 0)

Working backward from these pairs we obtain the Lempel-Ziv parsing

0, 01, 2, 21, 22, 1, 0

and the datavector

X = (0, 0, 1, 2, 2, 1, 2, 2, 1, 0)

5.4 Lempel-Ziv Parsing Tree

In some implementations of Lempel-Ziv coding, both encoder and decoder grow from scratch a tree called

the Lempel-Ziv parsing tree. Here is the Lempel-Ziv parsing tree for the datavector in Example 2:

Figure 1 : Lempel-Ziv Pars ing Tree for Example 2

0

3 2 5

6

We explain to the reader the meaning of this tree. Label each left branch with a "1" and each right branch

with a "0". For each node i (i = 1, .. . , 6) write down the variable-length block consisting of the bitsencountered along the path from the root node (labelled 0) to node i this block B2 is then the i-th block inthe Lempel-Ziv parsing of the datavector. For example, if we follow the path from node 0 to node 6, we

see a left branch, a left branch, and a right branch, which converts to the block 110. Thus, the sixth block

in the Lempel-Ziv parsing of our datavector is 110.

Let the datavector be X = (X1, X2, .. . , Xn). The encoder grows the Lempel-Ziv parsing tree as follows.Suppose there are q distinct blocks in the Lempel-Ziv parsing,B1, B2, .. . , Bq. Then the encoder grows treesT1, T2, .. . , Tq. Tree T1 consists of node 0, node 1, and a single branch going from node 0 to node 1 that islabelled with the symbol B1 = X1. For each i > 1, tree T2 is determined from tree T2_1 as follows:

(a) RemoveB1, .. . , B2_1 from the beginning ofX and let the resulting datavector be called X~2~.

(b) Starting at the root node of T2_1, follow the path driven by X~2~ until a terminal node of T2_1 is reached (the

labels on the resulting path form a prefix of X~2~ which is one of the blocks B~E {B1, B2, .. . , B2_1}, and

the terminal node reached is labelledj).

(c) Let X*be the ne xt symbol in X~2~ to appear after B~. Grow a branch from node j of T2_1, label this

br anch wi th the symbol X*, and label the new node at the end of this branch as "node i". This new tree

is T2.

The decoder can also grow the Lempel-Ziv parsing tree as decoding of the compressed datavector proceeds from left to

right. We will leave it to the reader to see how that is done.

Growing a Lempel-Ziv parsing tree allows the encoding and decoding operations in Lempel-Ziv coding to

be done in a fast manner . Also , there are modifi catio ns of Lempel -Z iv coding (not to be discussed here) in

which enhancements in data compression are obtained by making use of the structure of the parsing tree.

8/14/2019 10.1.1.137.871

5/12

5 - 5

5.5 Redundancy of LZ78

We want to see how much better the Lempel-Ziv code is than the block codes of various orders. We shall do

this by comparing the Lempel-Ziv codeword length for the datavector to the block entropies of the datavector

introduced in Chapter 4. It makes sense to make this comparison because the block entropies tell us how well

the best block codes do.

The simplest case, which we discuss first, would be to compare Lempel-Ziv code performance to thefirst order entropy. Let X = (X1 ;X2 ;::: ; Xn) denote the datavector to be compressed, and let LZ(X) denotethe length of the codeword assigned by the Lempel-Ziv code to X. Comparing LZ(X) to the first orderentropy H1(X), one can derive a bound of the form

LZ(X) nH1(X) + nn (5)

The constant term n, which depends only on the datavector length n, is called the first order redundancy andits units are bits per data sample. The better a data compression algorithm is, the smaller the redundancy will

be. The fo llowing resu lt gives the fi rs t order redundancy fo r the Lempel -Ziv code.

RESULT. The first order redundancy

n= Clog2log2n (6)

log2nis achievable for the Lempel-Ziv code, where C is a positive constant that depends upon the size of the dataalphabet. (In the preceding, we assume that the datavector length n is at least three, so that the redundancywill be well-defined.)

INTERPRETATION. We introduce some notation which makes it more convenient to talk aboutredundancy. If fzng is a sequence of real numbers, and fng is a sequence of re al numbers, we say that znisO(n) if there is a positive constant D such that

zn Djnj

for all sufficiently large positive integers n. Using our new notation, we see that the above RESULT says thatthe first order redundancy of the Lempel-Ziv code is O(log2log2n= log 2n ) (where n denotes the length of thedatavector). What does our redundancy result say? Recall that H1(X) is a lower bound on the compression rate

that results when one compresses X using the best memoryless code that can be designed for X. Thus, theRESULT tells us that the Lempel-Ziv code yields a compression rate on any datavector of length n no worsethan log2 log2n= log2n bi ts per sample more than the compression rate for the best memoryless cod e for thedatavector. Since the quantity log 2 log2 n= log2 n is very small when n is large, we can achieve throughLempel-Ziv coding a compression performance approximately no worse than that achievable by the best

memoryless code for the given datavector.

To show that the RESULT is true, we need the notion of unnormalized entropy. Let (Y1 ;Y2;::: ; Ym) be adatavector. (We allow the case in which each entry Yi is itself a datavector; for example, the Yi's may be

blocks arising from a Lempe l-Ziv parsing.) The unnorma lized entropy H*(Y1 ; ::: ; Ym) of the datavector (Y1;

::: ; Ym) is defined to be m, the length of the datavector, times the first order entropy H1(Y1 ;::: ; Ym) of thedatavector. This gives us the formula

m

H*(Y1;:::; Ym) = log2p(Yi) (7)

i=1

where p is the probability distribution on the set of entries of the datavector which assigns to each entry Y the probabilityp(Y ) defined by

p ( Y ) = # f1 im : Y i = Yg=m

(In other words, p is the first-order empirical distribution for the datavector (Y1 ;:: :;Ym).)In this argument, fix an arbitrary datavector X = (X1 ;X2 ;::: ; Xn). Let (B1 ;B2;::: ; Bt) be the Lempel-Ziv

parsing of the d atavector. From Exercise 4 at the end of this chapter, we have the followin g inequalit y:

H*(B1 ;B2 ; :: : ;B t ) H

*(X1 ; :: : ;Xn) + H

*(jB1j ;jB2j ; ::: ;jB tj )

8/14/2019 10.1.1.137.871

6/12

We know that

LZ(X) = Xt dlog2(ki)e

2 = 1

H*(jB1j;:::;jBtj) log2(1 + logen) + Xt

2=1

log2jB2j

(9)

where jB2j denotes the length of the block B2. Since the blocks B1;B2 ;::: ; B t-1 are distinct,

(t 1) log2(t 1) = H*(B1;::: ; B t-1) H

*(B1 ;::: ; B t)

where k is the size of the data alphabet. Expanding out the right side of the preceding equation, one can see that there is aconstant c1 such that

LZ(X) (c1 + k)t + (t 1) log2(t 1)

for all datavectors X. From Exercise 6 at the end of the chapter,

By concavity of the logarithm function,

Xt log 2jB2j t log2(t-1

XtjB2j) = tlog2(n=t)2=1 2=1

and so

LZ (X)=n H1(X ) + (X)

where

(X) = (c1 + k)(t=n) + n-1

log2(1 + logen) +log2(n=t) (8)(n=t)

By Exercise 8 at the end of the chapter, there is a constant c2 such that

t log2n c2n

Applying this to the first and third terms on the right side of (8), it is seen that

( X ) = O 1 + O l o g 2 l o g 2 n + O l o g 2 l o g 2 nlog2n n log2n

Of the three terms on the right above, the third term is dominant. We have achieved the bound (5) with ngiven by (6). The

RESULT is proved.

We now want to compare the compression performance of the Lempel-Ziv code to the performance of

bloc k co de s of an ar bi tr ar y ord er j. Consider an arbitrary datavector X = (X1 ; ::: ; Xn) of length n amultiple of j. By a complicated argument similar the argu ment given above for j = 1 (which we omit), itcan be shown that there is a constant Cj such that

log2log2 nL Z ( X ) = n H j ( X ) + C j

log2n

The second term on the right above is the j-th order redundancy of the Lempel-Ziv code. In other words, therelation (9) tells us that for any j, thej-th order redundancy of the Lempel-Ziv code is O(log2 log2n= log2n), which becomes very small as n gets large. Recall from Chapter 4 that Hj(X) is a lower bound on thecompression rate of the best j-th order block code for X. We conclude that no matter how large the order ofthe block code that one attempts to use, the Lempel Ziv algorithm will yield a compression rate on an

arbitrary datavector approximately no worse than that of the block code, provided the datavector is long

enough relative to the order of the block code. Hence, one loses nothing in compression rate by using the

Lempel-Ziv code instead of a block code. Also, one is able to compress a datavector faster via the Lempel-

8/14/2019 10.1.1.137.871

7/12

Ziv code than via block coding. To see this, one need only look at memoryless codes. For a datavector of

8/14/2019 10.1.1.137.871

8/12

5 - 7

length n, the overall time for best compression of the datavector via a memoryless code is proportional ton~. (The overall compression time in this case would be the time it takes to design the Huffman code forthe datavector plus the time it takes to compress the datavector with the Huffman code; since the first time

is proportional to n~ and the second time is p roportional to n, the overall compression time is proportionalto n~.) On the other hand, if the Lempel-Ziv code is implemented properly, it will take time proportional ton to compress any datavector of length n. (No time is wasted on design; the Lempel-Ziv code structure is

the same for every datavector.)We conclude:

Lempel-Ziv coding yields a compression performance as good as or better than the best block codes (provided the

datavector is long enough).

Lempel-Ziv coding yields faster compression of the data than does coding via the best block codes, because no time

is wasted on design.

The Lempel-Ziv code has been our first example of a code which does at least as we ll as the block codes

in terms of the redundancy of all orders becoming small with large datavector length. Such codes are called

universal codes. Although the Lempel-Ziv code is a universal code, there are universal codes whose

redundancy goes to zero faster with increasing datavector length than does the redundancy of the Lempel-Ziv

code. This point is discussed further in Chapter 15.

5.6 MATLAB m-files

We present two MATLAB programs in connection with Lempel-Ziv coding:

LZparse

LZcodelength

5.6 .1 LZparse.m

Here is the m-file for the MATLAB function LZparse:

%This m- f i le i s ca l led LZparse.m%It accomplishes Lempel-Ziv parsing of a binary

%datavector

%x is a b inary datavec tor

% y = LZpars e (x ) i s a v ec t o r c ons i s t i ng o f t he i nd i c es

%of the b locks in the Lempel -Z iv pars ing o f x

%

func t ion y = LZparse(x )

N=length(x);

d i c t = [ ] ;

lengthdic t=0;

wh i l e l eng t hd i c t < N

i= lengthd ic t+1;

k=0;while k==0

v =x ( l eng t hd i c t +1 : i ) ;

j =b i t s t r i ng _ t o _ i nd e x(v ) ;

A= (d ic t~= j ) ;

k=prod(A);

i f i ==N

k=1;

else

end

8/14/2019 10.1.1.137.871

9/12

5 - 8

i=i+1;

end

dict=[dict j];

lengthdict=lengthdict + length(v);

end

y=dict;

The function "LZparse" was illustrated in Example 2.

5.6 .2 LZcodelength .m

Here is the m-file for the MATLAB function LZcodelength:

%This m-file is named LZcodelength.m

%x = a binary datavector

%LZcodelength(x) =length in codebits of the encoder

%output resulting from the Lempel-Ziv coding of x

%

function y = LZcodelength(x)

u=LZparse(x);

t=length(u);

S=0;

for i=1:t;

S=S+ceil(log2(2*i));

end

y=S;

To illustrate the MATLAB function LZcodelength , we performed the following MATLAB session:

x=[1 1 0 1 1 0 0 0 1 1 0 1];

LZcodelength(x)

21

As a result of this session, we computed the length of the codeword resulting from the Lempel -Ziv encodingof the datavector in Example 2, and "21" was printed out on the screen. This is the correct length of this

codeword, as computed earlier in these notes.

5.7 Exercises

1. What is the minimum number of variable-length blocks that can appear in the Lempel-Ziv parsing of a binary

datavector of length 28? What is the maximum number?

2. Find the binary codeword that results when the datavector 11101011100101001111 is encoded using the Lempel-Ziv

code.

3. The alphabet of a datavector is {0, 1, 2}. The codeword

10100000001101010100010110010000

results when the datavector is Lempel-Ziv encoded. Find the datavector.

4. LetX = (X1, X2, ... , Xn) be a datavector and letB1, B2, ... , Btbe variable-length blocks into whichX is partitioned(from left to right). Show that

x*(B1,B2,...,Bt)

8/14/2019 10.1.1.137.871

10/12

H*(X1, X2, ... , Xn) log2(1 + logeN) + X

n

i=1log2Xi

5 - 9

5. Let (X1, X2, .. . , Xn) be a datavector and let A be the da ta alph ab et . Show that

H*(X1,X2, . . . ,Xn ) X n l o g 2 p (Xi )

i = 1

for every probability distribution p on A. (Hint: Use the fact that

Xa2A

p1(a) log2~p1(a)l0p2 a J

for any two probability distributions p1, p2 on A; see Exercise 1 of Chapter 3.)

6. Consider a datavector (X1, X2, .. . , Xn) in which each sample Xi is a positive integer less than o r equalto N. Show that

(Hint: First, use the result of Exercise 5 with the probability distribution

1~j

p(j) = 1 + (1~2) + (1~3) + ... + (1~N), j = 1, .. . , N

Then use the inequality

(1~2) + (1~3) + ... + (1~N) ~N(1~x )dx = logeNi

which can be seen by approximating the area under the curve y = 1~x by a sum of areas o f rectangles .)

7. Let A be an arbit ra ry fin it e al ph ab et . De fi ne L*lz (n) to be the minimum Lempel-Ziv codeword length

assigned to the datavectors of length n over the alphabet A. Show that

limn~oo

log2L*

lz(n)log2n

= 1~2

This property points out a hidden defect of the Lempel-Ziv code. Because the limit on the left above is

greater than zero, there exist certain datavectors which the Lempel-Ziv code does not compress very

well.

8. Consider all datavectors of all lengths over a fixed finite alphabet A. If X is such a datavector, let t(X)denote the number of variable-length blocks that appear in the Lempel-Ziv parsing of X. Show thatthere is a constant M (depending on the size of the alphabet A), such that for any integer n 2, andany datavector X of length n,

t(X) M n l og2 n

(Hint: Let t = t(X) and let B1, B2, .. . , B t_ 1be the first t 1 variable-length blocks in the Lempel-Zivpa rs ing of X. Let jB ij denote the length of block B i. In the inequality

jB1j + jB2j + . . . + jB t_1j n

find a lower bound for the left hand side using the fact that the B i 's are distinct.)

9. We discuss a variant of the Lempel-Ziv code which yields shorter codewords for some datavectors than does

LZ78. Encoding is accomplished via three steps. In Step 1, we partition the datavector (X1, ... , Xn) into

variable-length blocks in which the first b lock is of length one, and each succeeding block (except

8/14/2019 10.1.1.137.871

11/12

5-10

for possibly the last block) is the shortest prefix of the rest of the datavector which is not windowed in the

datavector as we slide to the left. To illustrate, the datavector 000110 is partitioned into

0,001,10 (11)

in Step 1. (On the other hand, LZ78 partitions this datavector into four blocks instead of three: 0,

00,1, 10.) In Step 2, each block B in the sequence of blocks from Step 1 is represented as a triple ( i, j,k) in which k is the last symbol in B, i is the length of the block B, and j is the smallest integer such

that if we look at the i1 samples in the datavector starting with sample Xj, we will see windowed theblock ob tained by removing the last symbol from B. (Take j = 0 if B has length one.) For example, forthe blocks in (11), Step 2 gives us the triples

(1, 0, 0), (3,1,1), (2, 4, 0)

In Step 3, the sequence of triples from Step 2 is converted into a binary codeword. There is a clever

way to do this which we shall not discuss here. All we need to know for the purposes of this exercise is

that if there are t triples and the datavector length is n, then the approximate length of the binary

codeword is t log 2 n.

(a) Show that there are infinitely many binary datavectors such that Step 1 yields a partition of the datavector into

5 blocks.

(b) Let X(n) be th e da ta vector co ns is ting of n zeroes. Let LZ*(X(n)) be the length of the binary

codeword which results when X(n) is encoded using the variant of the Lempel-Ziv code. Showthat LZ

*(X

(n))~LZ(X

(n)) converges to zero as n ~ oo.

8/14/2019 10.1.1.137.871

12/12

5-11

References

[1] J. Ziv and A. Lempel, "Compression of individual sequences via variable-rate coding,"IEEE Trans. Inform. Theory,

vol. 24, pp. 530-536, 1978.