arX

iv:1

004.

5157

v2 [

cs.IT

] 19

Aug

201

0SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 1

Deriving Good LDPC Convolutional Codes fromLDPC Block Codes

Ali E. Pusane,Member, IEEE,Roxana Smarandache,Member, IEEE,Pascal O. Vontobel,Member, IEEE,and Daniel J. Costello, Jr.,Life Fellow, IEEE

Abstract—Low-density parity-check (LDPC) convolutionalcodes are capable of achieving excellent performance with lowencoding and decoding complexity. In this paper we discussseveral graph-cover-based methods for deriving families of time-invariant and time-varying LDPC convolutional codes fromLDPC block codes and show how earlier proposed LDPCconvolutional code constructions can be presented within thisframework.

Some of the constructed convolutional codes significantlyoutperform the underlying LDPC block codes. We investigatesome possible reasons for this “convolutional gain” and alsodiscuss the — mostly moderate — decoder cost increase thatis incurred by going from LDPC block to LDPC convolutionalcodes.

Index Terms—Block codes, convolutional codes, low-densityparity-check (LDPC) codes, message-passing iterative decoding,pseudo-codewords, pseudo-weights, quasi-cyclic codes, unwrap-ping, wrapping.

I. I NTRODUCTION

I N THE LAST fifteen years, the area of channel codinghas been revolutionized by the practical realization of

capacity-approaching coding schemes, initiated by the inven-tion of turbo codes and their associated decoding algorithmsin 1993 [2]. A few years after the invention of the turbocoding schemes, researchers became aware that Gallager’slow-density parity-check (LDPC) block codes and message-passing iterative decoding, first introduced in [3], were alsocapable of capacity-approaching performance. The analysisand design of these coding schemes quickly attracted con-siderable attention in the literature, beginning with the workof Wiberg [4], MacKay and Neal [5], and many others. Anirregular version of LDPC codes was first introduced by Luby

Revised version, August 2010. Originally submitted to IEEETransactionson Information Theory, April 2010. The first and fourth authors were partiallysupported by NSF Grants CCR-0205310 and CCF-0830650, and byNASAGrant NNX09AI66G. Additionally, the first author was supported by aGraduate Fellowship from the Center for Applied Mathematics, University ofNotre Dame. The second author was supported by NSF Grants DMS-0708033and TF-0830608, and partially supported by NSF Grant CCR-0205310. Someof the material in this paper previously appeared in the Proceedings of the2007 IEEE International Symposium on Information Theory [1].

A. E. Pusane was with the Department of Electrical Engineering, Universityof Notre Dame, Notre Dame, IN 46556, USA. He is now with the Depart-ment of Electrical and Electronics Engineering, Bogazici University, Bebek,Istanbul 34342, Turkey (email: ali.pusane@boun.edu.tr).

R. Smarandache is with the Department of Mathematics and Statis-tics, San Diego State University, San Diego, CA 92182, USA (e-mail:rsmarand@sciences.sdsu.edu).

P. O. Vontobel is with Hewlett-Packard Laboratories, 1501 Page Mill Road,Palo Alto, CA 94304, USA (e-mail: pascal.vontobel@ieee.org).

D. J. Costello, Jr. is with the Department of Electrical Engineering, Univer-sity of Notre Dame, Notre Dame, IN 46556, USA (e-mail: costello.2@nd.edu).

et al. in [6], [7], and analytical tools were presented in [8],[9] to obtain performance limits for graph-based message-passing iterative decoding algorithms, such as those suggestedby Tanner [10]. For many classes of channels, these tools havebeen successfully employed to design families of irregularLDPC codes that perform very well near capacity [11], [12].Moreover, for the binary erasure channel these tools haveenabled the design of families of irregular LDPC codes that arenot only capacity-approaching but in fact capacity-achieving(see [13] and references therein).

The convolutional counterparts of LDPC block codes areLDPC convolutional codes. Analogous to LDPC block codes,LDPC convolutional codes are defined by sparse parity-checkmatrices, which allow them to be decoded using iterativemessage-passing algorithms. Recent studies have shown thatLDPC convolutional codes are suitable for practical imple-mentation in a number of different communication scenarios,including continuous transmission and block transmissioninframes of arbitrary size [14]–[16].

Two major methods have been proposed in the literature forthe construction of LDPC convolutional codes, two methodsthat in fact started the field of LDPC convolutional codes.The first method was proposed by Tanner [17] (see also[18], [19]) and exploits similarities between quasi-cyclic blockcodes and time-invariant convolutional codes. The secondmethod was presented by Jimenez-Feltstrom and Zigangirovin [20] and relies on a matrix-based unwrapping procedure toobtain the parity-check matrix of a periodically time-varyingconvolutional code from the parity-check matrix of a blockcode.

The aims of this paper are threefold. First, we show thatthese two LDPC convolutional code construction methods,once suitably generalized, are in fact tightly connected. Weestablish this connection with the help of so-called graphcovers.1 A second aim is to discuss a variety of LDPC convolu-tional code constructions. Although the underlying principlesare mathematically quite straightforward, it is importanttounderstand how they can be applied to obtain convolutional

1Note that graph covers have been used in two different ways inthe LDPCcode literature. On the one hand, starting with the work of Tanner [21], theyhave been used to construct Tanner graphs [10] of LDPC codes,and thereforeparity-check matrices of LDPC codes. Codes constructed in this way arenowadays often called proto-graph-based codes, followingthe influential workof Thorpe [22], who formalized this code construction approach. On the otherhand, starting with the work of Koetter and Vontobel [23], [24], finite graphcovers have been used to analyze the behavior of LDPC codes under message-passing iterative decoding. In this paper, we will use graphcovers in the firstway, with the exception of some comments on pseudo-codewords.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 2

Hconv =

H0(0)H1(1) H0(1)

......

. . .Hms

(ms) Hms−1(ms) · · · H0(ms)Hms

(ms + 1) Hms−1(ms + 1) · · · H0(ms + 1). . .

. . .. . .

Hms(t) Hms−1(t) · · · H0(t)

. . .. . .

. . .

. (1)

codes with good performance and attractive encoder anddecoder architectures. A third aim is to make progress towardsa better understanding of where the observed “convolutionalgain” comes from, and what its costs are in terms of decodercomplexity.

The paper is structured as follows. After some notationalremarks in Section I-A, we discuss the basics of LDPCconvolutional codes in Section II. In particular, in that sectionwe give a first exposition of the LDPC convolutional codeconstruction methods due to Tanner and due to Jimenez-Feltstrom and Zigangirov. In Section III we discuss two typesof graph-cover code constructions and show how they can beused to connect the code construction methods due to Tannerand due to Jimenez-Feltstrom and Zigangirov. Based on theseinsights, Section IV presents a variety of LDPC convolutionalcode constructions (along with simulation results), and inSec-tion V we mention some similarities and differences of theseconstructions compared to other recent code constructionsinthe literature. Afterwards, in Section VI we analyze someaspects of the constructed LDPC convolutional codes anddiscuss some possible reasons for the “convolutional gain,”before we conclude the paper in Section VII.

A. Notation

We use the following sets, rings, and fields:Z is the ringof integers;Z>0 is the set of non-negative integers;F2 isthe field of size two;F2[X ] is the ring of polynomials withcoefficients inF2 and indeterminateX ; andF2[X ]/〈Xr−1〉is the ring of polynomials inF2[X ] moduloXr − 1, wherer is a positive integer. We also use the notational short-handF〈r〉2 [X ] for F2[X ]/〈Xr−1〉.

By Fn2 andF

〈r〉2 [X ]n, we mean, respectively, a row vector

overF2 of lengthn and a row vector overF〈r〉2 [X ] of length

n. In the following, ifM is some matrix, then[M ]j,i denotesthe entry in thej-th row andi-th column ofM . Note that weuse the convention that indices of vector entries start at0 (andnot at1), with a similar convention for row and column indicesof matrix entries. (This comment applies also to semi-infinitematrices which are defined such that the row and column indexsets equalZ>0.) The only exception to this convention are bi-infinite matrices, where the row and column index sets equalZ. Finally, M1 ⊗ M2 will denote the Kronecker product ofthe matricesM1 andM2.

II. LDPC CONVOLUTIONAL CODES

This section defines LDPC convolutional codes and dis-cusses why they are interesting from an implementationperspective. Afterwards, we review two popular methods ofobtaining LDPC convolutional codes by unwrapping blockcodes. Later in this paper, namely in Section III, we will usegraph covers to show how these two methods are connected,and in Section IV we will see how these two methods canbe implemented and combined to obtain LDPC convolutionalcodes with very good performance.

A. Definition of LDPC Convolutional Codes

A semi-infinite binary parity-check matrix as in (1) at thetop of this page defines a convolutional codeCconv as follows.Namely, it is the set of semi-infinite sequences given by

Cconv ={

v ∈ F∞2

∣

∣

∣Hconv · v

T = 0T

}

,

where( · )T denotes the transpose of a vector or of a matrix.We comment on several important aspects and properties of

the codeCconv and its parity-check matrixHconv.

• If the submatricesHi(t), i = 0, 1, · · · ,ms, t ∈ Z>0,have size(c − b) × c with b < c, thenCconv is said tohave (design) rateR = b/c.

• The parameterms that appears in (1) is called thesyndrome former memory. It is an important parameterof Cconv because the maximal number of non-zero sub-matrices per block row ofHconv is upper bounded byms + 1.

• The quantityνs = (ms + 1) · c is called the constraintlength of Cconv. It measures the maximal width (insymbols) of the non-zero area ofHconv.2

• We do not require that for a giveni = 0, 1, . . . ,ms thesubmatrices{Hi(t)}t∈Z>0

are independent oft, and soCconv is in general atime-varyingconvolutional code.

• If there is a positive integerTs such thatHi(t) = Hi(t+Ts) for all i = 0, 1, . . . ,ms and all t ∈ Z>0, thenTs iscalled the period ofHconv, and Cconv is periodicallytime-varying.

• If the period Ts equals1, then Hconv is called time-invariant, and the parity-check matrix can be simply

2Strictly speaking, the above formula forνs gives only an upper boundon the maximal width (in symbols) of the non-zero area ofHconv, but thisupper bound will be good enough for our purposes.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 3

written as

Hconv =

H0

H1 H0

......

. . .Hms

Hms−1 . . . H0

HmsHms−1 . . . H0

. . .. . .

. . .

.

(2)

• If the number of ones in each row and column ofHconv

is small compared to the constraint lengthνs, thenCconv

is an LDPC convolutional code.• An LDPC convolutional codeCconv is called(ms, J,K)-

regular if, starting from the zeroth column,Hconv hasJones in each column, and, starting from the(ms+1)·(c−b)-th row, Hconv hasK ones in each row. If, however,there are no integersms, J , andK such thatCconv is(ms, J,K)-regular, thenCconv is called irregular.

Of course, there is some ambiguity in the above definition.Namely, a periodically time-varying LDPC convolutional codewith parametersTs, b, and c can also be considered to be aperiodically time-varying LDPC convolutional code with pa-rametersT ′

s = Ts/ℓ, b′ = ℓ · b, c′ = ℓ · c, andR′ = b′/c′ = b/cfor any integerℓ that dividesTs. In particular, forℓ = Ts weconsider the code to be a time-invariant LDPC convolutionalcode with parametersb′ = Ts · b andc′ = Ts · c.

B. Implementation Aspects of LDPC Convolutional Codes

An advantage of LDPC convolutional codes compared totheir block code counterparts is the so-called “fast encoding”property. As a result of the diagonal shape of their parity-checkmatrices, many LDPC convolutional codes enjoy simple shiftregister based encoders. Even randomly constructed LDPCconvolutional codes can be formed in such a way as to achievethis feature without any loss in performance (see, e.g., [20],[25]). On the other hand, in order to have a simple encodingprocedure for LDPC block codes, either the block code musthave some sort of structure [26] or the parity-check matrixmust be changed to a more easily “encodable” form [27].

The difficulty in constructing and decoding LDPC convolu-tional codes is dealing with the unbounded size of the parity-check matrix. This is overcome at the code construction stepby considering only periodically time-varying or time-invariantcodes. The code construction problem is therefore reducedto designing just one period of the parity-check matrix. Fordecoding, the most obvious approach is to terminate theencoder and employ message-passing iterative decoding basedon the complete parity-check matrix of the code. Althoughthis would be straightforward to implement using a standardLDPC block code decoder, it would be wasteful of resources,since the resulting (very large) block decoder would not betaking advantage of two important aspects of the convolutionalstructure: namely, that decoding can be done continuouslywithout waiting for an entire terminated block to be receivedand that the distance between two variable nodes that areconnected to the same check node is limited by the size ofthe syndrome former memory.

In order to take advantage of the convolutional nature of theparity-check matrix, a continuous sliding window message-passing iterative decoder that operates on a window of sizeI · νs variable nodes, whereI is the number of decodingiterations to be performed, can be implemented, similar to aViterbi decoder with finite path memory [28]. (See also therecent paper by Papaleoet al. [29], which investigates furtherreducing the window size for codes operating on a binaryerasure channel (BEC).) Another simplification is achievedby exploiting the fact that a single decoding iteration of twovariable nodes that are at leastms +1 time units apart can beperformed independently, since the corresponding bits cannotparticipate in the same parity-check equation. This allowstheparallelization of theI iterations by employingI independentidentical processorsworking on different regions of the parity-check matrix simultaneously, resulting in the parallel pipelinedecoding architecture introduced in [20]. The pipeline decoderoutputs a continuous stream of decoded data after an initialdecoding delay ofI · νs · c received symbols. The operationof this decoder on the Tanner graph of a simple time-invariantrate-1/3 convolutional code withms = 2 is illustrated inFigure 1.3 Note that the decoding window includesI constraintlengths of variable nodes since, overI iterations, decodingperformance can be affected by messages passed across awindow of that size.

Although the pipeline decoder is capable of fully paralleliz-ing the iterations by usingI independent identical processors,employing a large number of hardware processors might not bedesirable in some applications. In such cases, fewer processors(even one processor) can be scheduled to perform subsets ofiterations, resulting in a serial looping architecture [30] withreduced throughput. This ability to balance the processor loadand decoding speed dynamically is especially desirable wherevery large LDPC convolutional codes must be decoded withlimited available on-chip memory. Further discussion on theimplementation aspects of the pipeline decoder can be foundin [31].

C. Unwrapping Techniques due to Tanner and due to Jimenez-Feltstrom and Zigangirov (JFZ)

In this subsection we discuss two approaches for derivingconvolutional codes from block codes, in particular for de-riving LDPC convolutional codes from LDPC block codes.The first technique will be the unwrapping due to Tanner andthe second will be the unwrapping due to Jimenez-Feltstromand Zigangirov (JFZ). In Section III we will see, with thehelp of graph covers, how these two – seemingly different –unwrapping techniques are connected with each other.

The term unwrapping, in particular unwrapping a quasi-cyclic block code to obtain a time-invariant convolutionalcode, was first introduced in a paper by Tanner [18] (seealso [17]). That paper describes a link between quasi-cyclicblock codes and time-invariant convolutional codes and shows

3For LDPC convolutional codes the parameterms is usually much largerthan typical values ofms for “classical” convolutional codes. Therefore thevaluems = 2 of the convolutional code shown in Figure 1 is not typical forthe codes considered in this paper.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 4

v0(t)

v1(t)

v2(t)

y0(t)

y1(t)

y2(t)

ms + 1

Processor 2 Processor IProcessor I−1Processor 1

· · ·

· · ·

· · ·

· · ·

· · ·

decoding window

(size = I · (ms + 1))

Fig. 1. Tanner graph of a rate-1/3 convolutional code and an illustration of pipeline decoding. Here,y0(t), y1(t), y2(t) denote the stream of channel outputsymbols, andv0(t), v1(t), v2(t) denote the stream of decoder code bit decisions.

that the free distance of the unwrapped convolutional codecannot be smaller than the minimum distance of the underlyingquasi-cyclic code. This idea was later extended in [32], [33].

Consider the quasi-cyclic block codeC(r)QC defined by the

polynomial parity-check matrixH(r)QC(X) of sizem× n, i.e.,

C(r)QC =

{

v(X) ∈ F〈r〉2 [X ]n

∣

∣

∣H

(r)QC(X) · v(X)T = 0

T

}

.

Here the polynomial operations are performed moduloXr−1.The Tanner unwrapping technique is simply based on droppingthese modulo computations. More precisely, with a quasi-cyclic block codeC(r)

QC we associate the convolutional code

Cconv ={

v(D) ∈ F2[D]n∣

∣

∣Hconv(D) · v(D)T = 0

T

}

with polynomial parity-check matrix

Hconv(D) , H(r)QC(X)

∣

∣

∣

X=D. (3)

Here the change of indeterminate fromX to D indicates thelack of the moduloDr − 1 operations. (Note that in (3) weassume that the exponents that appear in the polynomials inH

(r)QC(X) are between0 andr − 1 inclusive.)It can easily be seen that any codewordv(D) in Cconv maps

to a codewordv(X) in C(r)QC through

v(X) , v(D) mod (Dr − 1)∣

∣

∣

D=X,

a process which was described in [18] as the wrapping aroundof a codeword in the convolutional code into a codeword inthe quasi-cyclic code. The inverse process was described asunwrapping.

Having introduced the unwrapping technique due to Tanner,we move on to discuss the unwrapping technique due toJFZ [20], which is another way to unwrap a block code toobtain a convolutional code. The basic idea is best explainedwith the help of an example.

0

0

1

1

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0 1

1 1 0 0 1 1 1 0 0 1

1 1

1 1

1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

0

1

1

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0 1

1 1 0 0 1 1 0 0 1

1 1

1 1

1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

0

0

1

1

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0

1 1 0 0 1 1 1 0 0 1

1 1

1 1

1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

0

1

1

Fig. 2. Deriving a rateR = 1/2 periodically time-varying convolutionalcode with b = 1, c = 2, ms = 4, νs = 10, andTs = 5 from a rate-1/2block code of length10.

Example 1 Consider the parity-check matrix

1 0 0 1 1 1 0 0 1

1 0 1 1

1 1 0 1 1 0

1 1 0 1

10 0 1 1 0 1 1 1 0

0 1 1 0 0 1

1 0 1 0

0 1 0 10 1

1

,H ,

�with sizem×n = 5×10, of a rate-1/2 block code. As indicatedabove, we can take a pair of scissors and “cut” the parity-check matrix into two pieces, whereby the cutting pattern issuch that we repeatedly movec = 2 units to the right andthenc− b = 1 unit down. Having applied this “diagonal cut,”we repeatedly copy and paste the two parts to obtain the bi-infinite matrix shown in Figure 2. This new matrix can be seenas the parity-check matrix of (in general) a periodically time-varying convolutional code (here the period isTs = 5). It isworth observing that this new matrix has the same row and

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 5

column weights as the matrix that we started with.4

This example can be formalized easily. Namely, startingwith anm×n parity-check matrixH of some block code, letη , gcd(m,n). Then the “diagonal cut” is performed by alter-nately movingc = n/η units to the right and thenc−b , m/ηunits down (i.e.,b =

(

(n−m)/η)

. The resulting convolutionalcode has rateR = b/c, syndrome former memoryms = η−1,constraint lengthνs = (ms + 1) · c = η · c = n, and periodTs = ms + 1 = η.

Analogous to the comment at the end of Section II-A, itis also possible to cut the matrixH in larger step sizes, e.g.,movingc′ = ℓ ·c units to the right andc′−b′ = ℓ ·(c−b) unitsdown, for any integerℓ that dividesTs = η, thereby obtain-ing a periodically time-varying convolutional code with rateR′ = b′/c′ = b/c, syndrome former memorym′

s = (η/ℓ)− 1,constraint lengthν′s = (m′

s + 1) · c′ = η · c = n, and periodT ′s = m′

s+1 = η/ℓ. (See also the discussion in Section IV-B.)In the rest of this paper, the term “JFZ unwrapping tech-

nique” will also stand for the following generalization of theabove procedure. Namely, starting with a length-n block codeC defined by some size-m× n parity-check matrixH, i.e.,

C ={

v ∈ Fn2

∣

∣

∣H · vT = 0

T

}

,

we writeH as the sumH =∑

ℓ∈LHℓ (in Z) of a collectionof matrices{Hℓ}ℓ∈L. The convolutional codeCconv is thendefined to be

Cconv ,{

v ∈ F∞2

∣

∣

∣Hconv · v

T = 0T

}

, (4)

where

Hconv ,

H0

H1 H0

......

. . .H|L|−1 H|L|−2 . . . H0

H|L|−1 H|L|−2 . . . H0

. . .. . .

. . .. . .

.

Referring to the notation introduced in Section II-A, the matrixHconv is the parity-check matrix of a time-invariant convolu-tional code. However, depending on the decomposition ofH

and the internal structure of the terms in that decomposition,the matrixHconv can also be (and very often is) viewed asthe parity-check matrix of a time-varying convolutional codewith non-trivial periodTs.

In order to illustrate the generalization of the JFZ unwrap-ping technique that we have introduced in the last paragraph,observe that decomposingH from Example 1 asH =H0 +H1 (in Z) with

1 0 0 1 1 1 0 0 1

1 0 1

1 1 0 1 0

1 0

0 0 0 0

0 1 0 0

1 0

0 1 0 10

1

0 0 0 0 0 0

0001

0 0 0

0 0

H0 , ,

4In practice, the codewords start at some time, so the convolutional parity-check matrix has effectively the semi-infinite form of (1), and the row weightsof the firstνs − 1 rows are reduced.

0 0 0 0

0 1

0 1 1 0

1 0 1

10 0 1 1 0 1 1 1 0

0 0 0 1

0 1 0

0 0 0 1

,,H1

0 0 0

000

0

0 0

0 0

0 0

0

0 0

yields a convolutional code with parity-check matrixHconv

whose bi-infinite version equals the matrix shown in Figure 2.

III. TANNER GRAPHS FROMGRAPH COVERS

Having formally introduced LDPC convolutional codes inthe previous section, we now turn our attention to the maintool of this paper, namely graph covers.

Definition 2 (see, e.g., [34])A coverof a graphG with vertexset W and edge setE is a graph G with vertex setW andedge setE , along with a surjectionπ : W → W which is agraph homomorphism (i.e.,π takes adjacent vertices ofG toadjacent vertices ofG) such that for each vertexw ∈ W andeach w ∈ π−1(w), the neighborhood∂(w) of w is mappedbijectively to∂(w). A cover is called anM -cover, whereMis a positive integer, if

∣

∣π−1(w)∣

∣ = M for every vertexw inW .5

These graph covers will be used for the construction ofnew Tanner graphs from old Tanner graphs, in particularfor the construction of Tanner graphs that represent LDPCconvolutional codes.

More specifically, this section starts by discussing twosimple methods to specify a graph cover, which will be calledgraph-cover construction 1 (GCC1) and graph-cover con-struction 2 (GCC2). Although they yield isomorphic Tannergraphs, and therefore equivalent codes, it is convenient tohaveboth methods at hand.6 As we will see, interesting classes ofTanner graphs can be obtained by repeatedly applying thesegraph-cover constructions, by mixing them, and by suitablyshortening the resulting codes. Moreover, these two graph-cover constructions will allow us to exhibit a connectionbetween the Tanner and the JFZ unwrapping techniques.

A. Graph-Cover Constructions

Let A be anmA×nA matrix overZ>0. With such a matrixwe can associate a Tanner graphT(A), where we drawnA

variable nodes,mA check nodes, and where there are[A]j,iedges from thei-th variable node to thej-th check node.7

Given the role that the matrixA will play subsequently, wefollow [22] and call the matrixA a proto-matrix and thecorresponding graphT(A) a proto-graph.

5The numberM is also known as the degree of the cover. (Not to beconfused with the degree of a vertex.)

6For a formal definition of code equivalence, see for example [35].7Note that we use a generalized notion of Tanner graphs, whereparallel

edges are allowed and are reflected by corresponding integerentries in theassociated matrix.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 6

The next definition introducesGCC1 andGCC2, two waysto specify graph covers that will be used throughout the restof the paper.8

Definition 3 For some positive integersmA andnA, let A ∈ZmA×nA

>0 be a proto-matrix. We also introduce the followingobjects:

• For some finite setL, let {Aℓ}ℓ∈L be a collection ofmatrices such thatAℓ ∈ Z

mA×nA

>0 , ℓ ∈ L, and such thatA =

∑

ℓ∈L Aℓ (in Z).• For some positive integerr, let {Pℓ}ℓ∈L be a collection

of size-r×r permutation matrices. I.e., for everyℓ ∈ L,the matrixPℓ is such that it contains one “1” per column,one “1” per row, and “0”s otherwise.

Based on the collection of matrices{Aℓ}ℓ∈L and the col-lection of matrices{Pℓ}ℓ∈L, there are two common ways ofdefining a graph cover of the Tanner graphT(A). (In thefollowing expressions,I is the identity matrix of sizer × r.)

• Graph-cover construction 1 (GCC1). Consider the inter-mediary matrix

B′ , A⊗ I =

∑

ℓ∈L

(Aℓ ⊗ I) ,

whose Tanner graphT(B′) consists ofr disconnectedcopies ofT(A). This is anr-fold cover ofT(A), albeita rather trivial one. In order to obtain an interestingr-fold graph cover ofA, for eachℓ ∈ L, we replaceAℓ⊗I

by Aℓ ⊗ Pℓ, i.e., we define

B ,∑

ℓ∈L

(Aℓ ⊗ Pℓ) .

• Graph-cover construction 2 (GCC2) Consider the inter-mediary matrix

B′, I ⊗A =

∑

ℓ∈L

(I ⊗Aℓ) ,

whose Tanner graphT(B′) consists ofr disconnected

copies ofT(A). This is anr-fold cover ofT(A), albeita rather trivial one. In order to obtain an interestingr-fold graph cover ofA, for eachℓ ∈ L, we replaceI⊗Aℓ

by Pℓ ⊗Aℓ, i.e., we define

B ,∑

ℓ∈L

(Pℓ ⊗Aℓ) .

If all the matrices{Pℓ}ℓ∈L are circulant matrices, then thegraph coversT(B) andT(B) will be called cyclic covers ofT(A).

One can verify that the two graph-cover constructions inDefinition 3 are such that the matrixB, after a suitablereordering of the rows and columns, equals the matrixB.9

8We leave it as an exercise for the reader to show that the graphs constructedin GCC1 and GCC2 are indeed two instances of the graph cover definitionin Definition 2.

9Indeed, a possible approach to show this is to use the fact that Aℓ ⊗ Pℓ

andPℓ ⊗Aℓ are permutation equivalent, i.e., there is a pair of permutationmatrices(Q,Q′) such thatAℓ ⊗Pℓ = Q · (Pℓ ⊗Aℓ) ·Q

′. Of course, forthis to work, the pair(Q,Q′) must be independent ofℓ ∈ L, i.e., dependentonly on the size of the matrices{Aℓ}ℓ∈L and {Pℓ}ℓ∈L. Such a(Q,Q′)pair can easily be found.

This implies thatT(B) and T(B) are isomorphic graphs;nevertheless, it is helpful to define both types of constructions.

B. Graph-Cover Construction Special Cases

The following examples will help us to better understandhow GCC1 and GCC2 can be used to obtain interestingclasses of Tanner graphs, and, in particular, how the result-ing graph-cover constructions can be visualized graphically.Although these examples are very concrete, they are writtensuch that they can be generalized easily.

Example 4 (Cyclic cover) Consider the proto-matrix

A ,

[

1 1 11 1 1

]

(5)

with mA = 2 andnA = 3, and whose Tanner graphT(A) isshown in Figure 3(a). LetL , {0, 1} × {0, 1, 2}, and let thecollection of matrices{Aℓ}ℓ∈L be given by{Aj,i}j,i wherefor eachj = 0, . . . ,mA − 1 and eachi = 0, . . . , nA − 1 thematrix Aj,i ∈ Z

mA×nA

>0 is defined as follows

[Aj,i]j′,i′ ,

{

[A]j′,i′ if (j′, i′) = (j, i)

0 otherwise.

Moreover, letr , 7, and let the collection of matrices{Pℓ}ℓ∈L

be given by{Pj,i}j,i whereP0,0 , I1, P0,1 , I2, P0,2 , I4,P1,0 , I6, P1,1 , I5, P1,2 , I3, and whereIs is an s timesleft-shifted identity matrix of sizer × r.

• Using GCC1, we obtain the matrices

B′ = A⊗ I0 =

[

I0 I0 I0

I0 I0 I0

]

,

B =

[

I1 I2 I4

I6 I5 I3

]

, (6)

whose Tanner graphsT(B′) andT(B), respectively, areshown in Figure 3(b).

• Using GCC2, we obtain the matrices

B′= I0 ⊗A =

A 0 0 0 0 0 0

0 A 0 0 0 0 0

0 0 A 0 0 0 0

0 0 0 A 0 0 0

0 0 0 0 A 0 0

0 0 0 0 0 A 0

0 0 0 0 0 0 A

,

B =

0 A1,0 A1,1 A0,2 A1,2 A0,1 A0,0

A0,0 0 A1,0 A1,1 A0,2 A1,2 A0,1

A0,1 A0,0 0 A1,0 A1,1 A0,2 A1,2

A1,2 A0,1 A0,0 0 A1,0 A1,1 A0,2

A0,2 A1,2 A0,1 A0,0 0 A1,0 A1,1

A1,1 A0,2 A1,2 A0,1 A0,0 0 A1,0

A1,0 A1,1 A0,2 A1,2 A0,1 A0,0 0

,

(7)

whose Tanner graphsT(B′) andT(B), respectively, are

shown in Figure 3(c). Note that all the block rows and

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 7

(a) Proto-graphT(A).

(b) GCC1 based onT(A). Top: T(B′). Bottom:T(B).

(c) GCC2 based onT(A). Top: T(B′). Bottom:T(B).

Fig. 3. Proto-graph and graph-covers for the graph-cover constructionsdiscussed in Example 4. (Compare with the corresponding graphs in Figure 4.)

all the block columns sum (inZ) to A. (This observationholds in general, not just for this example.)

We would like to add two comments with respect to theabove example.

First, instead of definingIs to be ans times left-shiftedidentity matrix of sizer×r, we could have definedIs to be ans times right-shifted identity matrix of sizer × r. Comparedto the matrices and graphs described above, such a changein definition would yield (in general) different matrices butisomorphic graphs.

Secondly, let us mention thatGCC2 was termed the “copy-and-permute” construction by Thorpe and his co-workers. Thisterminology stems from the visual appearance of the proce-dure: namely, in going from Figure 3(a) to Figure 3(c)(top)we copy the graph several times, and in going from Fig-ure 3(c)(top) to Figure 3(c)(bottom) we permute the edges

(a) Proto-graphT(A).

· · · · · ·

· · ·

· · ·

· · ·

· · ·

· · ·

· · ·

· · · · · ·

· · · · · ·

· · · · · ·

· · ·· · · · · · · · ·

· · · · · ·

(b) GCC1 based onT(A). Top: T(B′). Bottom:T(B).

· · ·

· · ·

· · ·

· · ·

· · ·

· · ·· · ·

· · ·

· · ·

· · ·

· · ·

· · ·

(c) GCC2 based onT(A). Top: T(B′). Bottom:T(B).

Fig. 4. Proto-graph and graph-covers for the graph-cover constructionsdiscussed in Example 6. (Compare with the corresponding graphs in Figure 3.)

of the graph, whereby the permutations are done within thesets of edges that have the same pre-image in Figure 3(a).

Remark 5 (Quasi-cyclic codes)Consider again the matricesthat were constructed in Example 4, in particular the matrixA in (5) and its r-fold cover matrixB in (6). Because allmatrices in the matrix collection{Pℓ}ℓ∈L are circulant,T(B)represents a cyclic cover ofT(A). Clearly, when seen overF2,the matrixH(r)

QC , B is the parity-check matrix of a quasi-cyclic binary linear block code

C(r)QC =

{

v ∈ FnA·r2

∣

∣

∣H

(r)QC · vT = 0

T

}

.

Using the well-known isomorphism between the addition andmultiplication of circulant matrices overF2 and the additionand multiplication of elements of the ringF〈r〉

2 [X ], this code

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 8

can be written equivalently as

C(r)QC =

{

v(X) ∈ F〈r〉2 [X ]nA

∣

∣

∣H

(r)QC(X) · v(X)T = 0

T

}

with

H(r)QC(X) ,

[

X1 X2 X4

X6 X5 X3

]

.

As noted above, the graphsT(B) and T(B) that areconstructed in Definition 3 are isomorphic. Applying thisobservation to Example 4, the matrixH

(r)

QC , B with B

from (7) is therefore the parity-check matrix of a binary linearblock code

C(r)

QC ={

v ∈ Fr·nA

2

∣

∣

∣H

(r)

QC · vT = 0T

}

that is equivalent toC(r)QC, i.e., the codewords ofC

(r)

QC can be

obtained from the codewords ofC(r)QC by a suitable reorder-

ing of the codeword components. In terms of the matrices{Aj,i}j,i, which also appear in the matrixB in (7), onecan verify that the polynomial parity-check matrixH(r)

QC(X)

can be written asH(r)QC(X) = 0X0 + A0,0X

1 + A0,1X2 +

A1,2X3 +A0,2X

4 +A1,1X5 +A1,0X

6.

Besides defining finite graph covers, we can also defineinfinite graph covers, as illustrated in the following examples.These infinite graph covers will be crucial towards definingTanner graphs of convolutional codes.

Example 6 (Bi-infinite Toeplitz covers) We continue Exam-ple 4. However, besides keeping the proto-matrixA andthe collection of matrices{Aj,i}j,i, we consider a differentcollection of matrices{Pj,i}j,i. Namely, we setP0,0 , T1,P0,1 , T2, P0,2 , T4, P1,0 , T6, P1,1 , T5, P1,2 , T3.HereTs is a bi-infinite Toeplitz matrix with zeros everywhereexcept for ones in thes-th diagonal below the main diagonal,i.e., [Ts]j,i = 1 if j = i+ s and [Ts]j,i = 0 otherwise. E.g.,

T1 =

. . .. . .

. . .. . .

. . . 0 0 0 0

. . . 1 0 0 0. . .

. . . 0 1 0 0. . .

0 0 1 0. . .

. . .. . .

. . .. . .

,

where for clarity we have underlined the entries of the maindiagonal.

• Using GCC1, we obtain the matricesB′ = A⊗ T0 and

B =

[

T1 T2 T4

T6 T5 T3

]

. (8)

The Tanner graphT(B′), which is depicted in Fig-ure 4(b)(top), is similar to the corresponding Tannergraph in Figure 3(b)(top), but with bi-infinitely manyindependent components. Analogously, the Tanner graph

T(B), which is depicted in Figure 4(b)(bottom) is similarto the Tanner graph shown in Figure 3(b)(bottom), butinstead of cyclically wrapped edge connections, the edgeconnections are infinitely continued on both sides.

• Using GCC2, we obtain the matricesB′= T0 ⊗A and

B =

. . .. . .

. . .. . .

. . .. . .

. . .. . .

. . . 0 0 0 0 0 0 0 0

. . . A0,0 0 0 0 0 0 0 0.. .

. . . A0,1 A0,0 0 0 0 0 0 0.. .

. . . A1,2 A0,1 A0,0 0 0 0 0 0.. .

. . . A0,2 A1,2 A0,1 A0,0 0 0 0 0.. .

. . . A1,1 A0,2 A1,2 A0,1 A0,0 0 0 0.. .

. . . A1,0 A1,1 A0,2 A1,2 A0,1 A0,0 0 0.. .

0 A1,0 A1,1 A0,2 A1,2 A0,1 A0,0 0.. .

. . .. . .

. . .. . .

. . .. . .

. . ... .

.

(9)

The Tanner graphT(B′), which is depicted in Fig-

ure 4(c)(top), is similar to the corresponding Tannergraph in Figure 3(c)(top), but with bi-infinitely manyindependent components. Analogously, the Tanner graphT(B), which is depicted in Figure 4(c)(bottom) is similarto the Tanner graph shown in Figure 3(c)(bottom), butinstead of cyclically wrapped edge connections, the edgeconnections are infinitely continued on both sides.

Although it is tempting to replace in Example 6 the bi-infinite Toeplitz matricesTs (whose row and column index setsequalZ) by semi-infinite Toeplitz matrices (whose row andcolumn index sets equalZ>0), note that the resulting TannergraphsT(B) andT(B) would then in general not be graphcovers ofT(A). This follows from the fact that semi-infiniteToeplitz matrices are not permutation matrices (except forT0),and so some vertex degrees ofT(B) and T(B) would notequal the corresponding vertex degrees inT(A).10

Remark 7 It turns out that the Tanner graphs in Figure 4 areinfinite graph covers of the Tanner graphs in Figure 3. Moreprecisely, the Tanner graphsT(B′), T(B), T(B

′), T(B)

in Figure 4 are graph covers of the corresponding Tannergraphs T(B′), T(B), T(B

′), T(B) in Figure 3. For the

Tanner graphsT(B′) in Figures 3(b)(top) and 4(b)(top) andthe Tanner graphsT(B

′) in Figures 3(c)(top) and 4(c)(top)

this statement is easily verified by inspection.To verify that the Tanner graphT(B) in Figure 4(c)(bottom)

is a graph cover ofT(B) in Figure 3(c)(bottom), we apply

10As will be clear from the discussion later on, in this paper wetake anapproach where in a first step we construct bi-infinite Tannergraphs thatare “proper” graph covers and where in a second step we obtainsemi-infinite Tanner graphs by applying a “shortening” procedureto these bi-infinite Tanner graphs. Alternatively, one could have also chosen an approachbased on “improper” graph covers. Both approaches have their advantagesand disadvantages; we preferred to take the first approach.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 9

GCC2 with proto-matrix A, with resulting matrixB, withthe setL, with the collection of matrices{Aℓ}ℓ∈L, and withthe collection of permutation matrices{Pℓ}ℓ∈L as follows.Namely, we let the proto-matrixA be the matrix from(7)(there denoted byB), we let the resulting matrixB be thematrix in (9) (there denoted byB), we defineL , {0, 1}, weselect

A0 =

0 0 0 0 0 0 0

A0,0 0 0 0 0 0 0

A0,1 A0,0 0 0 0 0 0

A1,2 A0,1 A0,0 0 0 0 0

A0,2 A1,2 A0,1 A0,0 0 0 0

A1,1 A0,2 A1,2 A0,1 A0,0 0 0

A1,0 A1,1 A0,2 A1,2 A0,1 A0,0 0

, (10)

A1 =

0 A1,0 A1,1 A0,2 A1,2 A0,1 A0,0

0 0 A1,0 A1,1 A0,2 A1,2 A0,1

0 0 0 A1,0 A1,1 A0,2 A1,2

0 0 0 0 A1,0 A1,1 A0,2

0 0 0 0 0 A1,0 A1,1

0 0 0 0 0 0 A1,0

0 0 0 0 0 0 0

, (11)

and we selectP0 = T0 and P1 = T1, whereTs was definedin Example 6. Clearly,A = A0 +A1 (in Z).11 With this wehave

B = P0 ⊗A0 + P1 ⊗A1 =

. . .. . .

. . ... .

. . . A0 0 0 0

. . . A1 A0 0 0. . .

. . . 0 A1 A0 0. . .

0 0 A1 A0. . .

. . ... .

. . .. . .

,

and one can verify that this matrix equals the matrix in(9)(there denoted byB), which means thatT(B) is indeed aninfinite cover ofT(A). We remark that, interestingly, in thisprocess we have shown how a certainGCC2 graph cover ofa proto-matrix can be written as aGCC2 graph cover of acertain GCC2 graph cover of a proto-matrix.

Finally, a similar argument shows that the Tanner graphT(B) in Figure 4(b)(top) is a graph cover of the Tanner graphin Figure 3(b)(top), also denoted byT(B).

There are many other ways of writing a proto-matrixA asa sum of a collection of matrices{Aℓ}ℓ∈L. The next examplediscusses two such possibilities.

Example 8 Consider the proto-matrix

A ,

0 0 1 1 1 0 1 1 1 00 1 1 1 0 1 0 0 1 11 1 1 0 0 1 1 1 0 01 0 0 1 1 0 0 1 1 11 1 0 0 1 1 1 0 0 1

,

11Note that a non-zero block diagonal ofA would be put inA0.

that is shown in Figure 5(a), and that also appeared inExample 1. Its Tanner graphT(A) is (3, 6)-regular, i.e., allvariable nodes have degree3 and all check nodes have degree6. Let L = {0, 1}, and consider the collection of matrices{Pℓ}ℓ∈L with P0 = T0 andP1 = T1, where the matricesT0

andT1 are defined as in Example 6. In the following, we lookat two different choices of the collection of matrices{Aℓ}ℓ∈L.

• Figure 5(c) shows a typical part of the matrixB that isobtained whenGCC2 is used to construct a graph coverof A with the collection of matrices{Aℓ}ℓ∈L defined asshown in Figure 5(a).

• Figure 5(d) shows a typical part of the matrixB whenGCC2 is used to construct a graph cover ofA withthe collection of matrices{Aℓ}ℓ∈L defined as shown inFigure 5(b).

Overall, because of the choice of the collection{Pℓ}ℓ∈L, thesupport of both matricesB possesses a banded structure.Moreover, the different choices of the collection{Aℓ}ℓ∈L

leads to a somewhat narrower banded structure in the firstcase compared to the second case.

The next example makes a crucial observation; namely, itshows that the above graph-cover constructions can be appliedrepeatedly to obtain additional interesting classes of Tannergraphs.

Example 9 (Iterated Graph-Cover Construction)Consider the proto-matrixA from Example 4. We consideriterated graph-cover constructions. In the first case, we applyGCC1 and thenGCC2, and in the second case we applyGCC2 twice.

• Consider the matrixB obtained from the matrixA usingGCC1, like in Example 4. The resulting matrixB isshown in (6) and will be calledA(1) in this example,since it is considered to be a proto-matrix by itself,cf. Figure 6(a). Based on the “cutting line” shown inFigure 6(a), we define the matricesA(1)

0 and A(1)1 as

follows: the non-zero part ofA(1)0 equals the non-zero

part of the lower triangular part ofA(1) and the non-zero part ofA(1)

1 equals the non-zero part of the uppertriangular part ofA(1). (Clearly, A(1) = A

(1)0 +A

(1)1 .)

Applying the procedure from Example 8, Figure 6(c)shows a typical part of the matrixB

(1)that is obtained

whenGCC2 is used to construct a graph cover ofA(1).• Consider the graph-coverB obtained fromA using

GCC2, like in Example 4. The resulting matrixB isshown in (7) and will be calledA(2) in this example,since it is considered to be a proto-matrix by itself,cf. Figure 6(b). Based on the “cutting line” shown inFigure 6(b), we define the matricesA(2)

0 and A(2)1 as

follows: the non-zero part ofA(2)0 equals the non-zero

part of the lower triangular part ofA(2) and the non-zero part ofA(2)

1 equals the non-zero part of the uppertriangular part ofA(2). (Clearly, A(2) = A

(2)0 +A

(2)1 .)

Applying the procedure from Example 8, Figure 6(d)shows a typical part of the matrixB

(2)that is obtained

whenGCC2 is used to construct a graph cover ofA(2).

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 10

1 1 0 0 1 1 1 0 0 1

0

0

1

1

=

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0 1 0 0

0

0

0 0

0

=

0

0

=

0

0

1

0

0

1

0

0 0 0

0 1 1

0 1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

A

A0

A1

0

1 1 0 0 1 1 1 0 0 1

1 0 1 1

1 1 0 1 1 0

1 1 0 1

1

000

0

0 0 0

0 0

00

0 0

000

000

0

00

0 0

0 0 0

0

0

0

0 0 1 1 0 1 1 1 0

0 1 1 0 0 1

1 0 1 0

0 1 0 10 1

(a) First decomposition of the matrixA into the matricesA0 andA1.

0

0

1

1

=

0

0

1

1

0

1

0

0 0 1

0 1 0 0

0

0

0 0

0

=

0

0

=

0

0

1

0

0

1

0

0 0 0

0 1

0 1 1 0 0

0 0 0

0

A

A0

A1

0

1 1 0 0 1 1 1 0 0 1

1 0 1 1

1 1 0 1 1 0

1 1 0 1

1

00 0 0

00

0

00

00

0

0 0 0

0

0

0 0 1 1 0 1 1 1 0

0 1 1 0 0 1

1 0 1 0

0 1 0 10 1

1 0 0 1 0 01

0

1 1

0 1 1 1

0

0 0 1

0 0 0

0

1 0 0 0

1

1 1 0

1 1 1

01 1

(b) Second decomposition of the matrixA into the matricesA0 andA1.

0

0

0

0

1

0

0

1

0

0 0 0

0 1 1

0 1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

0

000

000

0

00

0 0

0 0 0

0

0

0

0

0

1

1

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0 1 0 0

0

0

0 0

0000

0

0 0 0

0 0

00

0 0

1 1 0 0 1 1 1 0 0 1

0

0

0

0

1

0

0

1

0

0 0 0

0 1 1

0 1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

0

000

000

0

00

0 0

0 0 0

0

0

0

0

0

0

0

1

0

0

1

0

0 0 0

0 1 1

0 1 1 0 0

0

1

1

0 1

0

1

0 1

1 0

1

0

000

000

0

00

0 0

0 0 0

0

0

0

0

0

1

1

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0 1 0 0

0

0

0 0

0000

0

0 0 0

0 0

00

0 0

1 1 0 0 1 1 1 0 0 1

0

0

1

1

0

1

1

0

1

1

0

1

0

1 1 0 0 1

0 1 0 0

0

0

0 0

0000

0

0 0 0

0 0

00

0 0

1 1 0 0 1 1 1 0 0 1

A0

A0

A0

A1

A1

A1

(c) Part of the matrixB based on the first decomposition ofA.

0

0

1

1

0

0

1

1

0

1

0

0 0 1

0 1 0 0

0

0

0 0

000 0 0

00

0

1 0 0 1 0 01

1 1

0 1 1 1

0

0 0 1

0 0 0

0

0

0

0

1

0

0

1

0

0 0 0

0 1

0 1 1 0 0

0 0 0

0

00

00

0

0 0 0

0

0

00

1 0 0 0

1

1 1 0

1 1 1

01 1 0

0

0

0

0

1

0

0

1

0

0 0 0

0 1

0 1 1 0 0

0 0 0

0

00

00

0

0 0 0

0

0

00

1 0 0 0

1

1 1 0

1 1 1

01 1 0

0

0

1

1

0

0

1

1

0

1

0

0 0 1

0 1 0 0

0

0

0 0

000 0 0

00

0

1 0 0 1 0 01

1 1

0 1 1 1

0

0 0 1

0 0 0

0

0

0

0

1

0

0

1

0

0 0 0

0 1

0 1 1 0 0

0 0 0

0

00

00

0

0 0 0

0

0

00

1 0 0 0

1

1 1 0

1 1 1

01 1 0

0

0

1

1

0

0

1

1

0

1

0

0 0 1

0 1 0 0

0

0

0 0

000 0 0

00

0

1 0 0 1 0 01

1 1

0 1 1 1

0

0 0 1

0 0 0

A0

A0

A0

A1

A1

A1

(d) Part of the matrixB based on the second decomposition ofA.

Fig. 5. Matrices appearing in Example 8. (See main text for details.)

We observe a large difference in the positions of the non-zero entries inB

(1)andB

(2).

• In the first case, the two graph-cover constructions are“incompatible” and the positions of the non-zero entriesin B

(1)follow a “non-simple” or “pseudo-random”

pattern. As we will see in Example 18 with the helpof simulation results, such Tanner graphs can lead totime-varying LDPC convolutional codes with very goodperformance.

• In the second case, the two graph-cover constructionsare “compatible” in the sense thatB

(2)can be obtained

from the proto-matrixA by applyingGCC2 with suitablematrix collections{Aℓ}ℓ∈L and {Pℓ}ℓ∈L. As such, the

positions of the non-zero entries ofB(2)

follow a rela-tively “simple” or “non-random” pattern, which leads toa time-invariant LDPC convolutional code.

The above procedure of obtaining two matrices that add upto a matrix is called “cutting a matrix”. Actually, we will usethis term also if there is no simple cutting line as in the above

examples, and also if the matrix is written as the sum of morethan two matrices (cf. Example 1 and the paragraphs after it).

C. Revisiting the Tanner and the JFZ Unwrapping Techniques

In Section II-C we introduced two techniques, termedthe Tanner and the JFZ unwrapping techniques, to deriveconvolutional codes from block codes. In this subsection werevisit these unwrapping techniques. In particular, we showhow they can be cast in terms of graph covers and how thetwo unwrapping techniques are connected.

Because of the importance of the coding-theoretic notionof shortening [35] for this subsection, we briefly revisit thisconcept. LetH be a parity-check matrix that defines somelength-n binary codeC. We say that the length-(n−1) codeC′ is obtained by shorteningC at positioni if

C′=

{

(v0, . . . , vi−1, vi+1, . . . , vn−1) ∈ Fn−12

∣

∣

∣v ∈ C, vi = 0

}

.

In terms of parity-check matrices, a possible parity-checkmatrixH ′ of C′ is obtained by deleting thei-th column ofH .

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 11

0 00 0 0 0 0 00 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 00 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0

0 00 0 0 0 0 0 00 0 0 0

0 00 0 0 0 0 0 0 0 0 0 0

0 0 00 0 0 0 00 0 0 0 0

0 00 0 0 0 0 00 0 0 0 00 0 0 0 0

00 0 0 0 0 00 0 0 0 0 0 00 0 0 0

00 0 0 0 0 0 00 0 0 0 0 00 0 0 0

0 00 0 0 0 0 00 0 0 0 0 00 0 0 0

0 00 0 0 0 0 00 0 0 0 0 0 0 0 0 0

0 00 0 0 0 0 00 0 0 0 00 0 0 0 0

0 00 0 0 0 0 0 0 0 0 0 00 0 0 0 0

0 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

00 0 0 0

00 0 0 0

0 00 0 0

0 00 0 0

1

1

1

1

(a) Matrix A(1)

0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0

0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0

0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0

0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0

0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1

0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1

0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0

0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0

0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0

1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

(b) Matrix A(2)

0 0 0 0 00 0 0 0 0 00 0 0 0

0 0 0 0 0 00 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0

0 00 0 0 0 0 0 00 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 00 0 0 0 0

0 0 00 0 0 0 0

0 0 0 00 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0

0 0 0

1 1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

0 00

0 0

00 0 0 0

00 0 0 0

0 00 0 0 0 00

0 00 0 0 0 00

0 00 0 0 0 0 00 0

00 0 0 0 0 00 0 0

00 0 0 0 0 0 00 0 0 0 0

0 00 0 0 0 0 00 0 0 0 0

0 00 0 0 0 0 00 0 0 0 0 0 0

0 00 0 0 0 0 00 0 0 0 00 0

0 00 0 0 0 0 0 0 0 0 0 00 0 0 0 0

0 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0

1

1

1

1

1

1 1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

(c) Part of matrixB(1)

0 0 0

0 0 0

1 0 0 0 0 0

0 0 0 0 0 0

0 1 0 1 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 1 0 0 0 0 0

0 0 1 0 0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 1 0 1 0 0 0 0 0

0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0

0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0

1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0

1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0 0 1 0

1 0 0 0 1 0 0 0 0 0 0 1 0 0 0

0 0 0 0 0 0 0 0 1 0 0 0

1 0 0 0 1 0 0 0 0 0 0 1

0 0 0 0 0 0 0 0 1

1 0 0 0 1 0 0 0 0

0 0 0 0 0 0

1 0 0 0 1 0

0 0 0

1 0 0

(d) Part of matrixB(2)

Fig. 6. Matrices appearing in Example 9. (See main text for details.)

In terms of Tanner graphs, this means that the Tanner graphT(H ′) is obtained fromT(H) by removing thei-th variablenode, along with removing its incident edges. In the following,we will also use the term “shortening” to denote this graphmodification procedure.

Now, to explain the Tanner unwrapping technique in termsof graph covers, consider the quasi-cyclic block codeC

(r)QC

defined by the polynomial parity-check matrixH(r)QC(X) of

sizemA × nA, i.e.,

C(r)QC =

{

v(X) ∈ F〈r〉2 [X ]nA

∣

∣

∣H

(r)QC(X) · v(X)T = 0

T

}

,

where the polynomial operations are performed moduloXr−1(see also Remark 5). As already mentioned in Section II-C,the Tanner unwrapping technique is simply based on droppingthese modulo computations. More precisely, with a quasi-cyclic block codeC(r)

QC, we associate the convolutional code

Cconv ={

v(D) ∈ F2[D]nA

∣

∣

∣Hconv(D) · v(D)T = 0

T

}

with polynomial parity-check matrix

Hconv(D) , H(r)QC(X)

∣

∣

∣

X=D.

Again, the change of indeterminate fromX to D indicates thelack of moduloDr − 1 operations.

In the following we will give, with the help of an example,two interpretations of the Tanner unwrapping technique interms of graph covers.

Example 10 We continue Remark 5. Unwrapping the quasi-cyclic block codeC(r)

QC that was considered therein, we obtain

a rate-1/3 time-invariant convolutional code

Cconv ={

v(D) ∈ F2[D]nA

∣

∣

∣Hconv(D) · v(D)T = 0

T

}

based on

Hconv(D) ,

[

D1 D2 D4

D6 D5 D3

]

.

Consider now the infinite graph covers that were constructedin Example 6 usingGCC1, in particular T(B). LetC

(

T(B))

be the set of codewords defined by the Tanner graphT(B).Then the convolutional codeCconv is a shortened version ofC(

T(B))

where all codeword bits corresponding to negativetime indices have been shortened. Therefore, the Tanner graphof Cconv is given by the Tanner graph in Figure 4(b)(bottom),where all bit nodes with negative time indices, along with theirincident edges, are removed. Clearly, this bit-node and edgeremoval process implies decreasing the degrees of some checknodes. In fact, some check nodes become obsolete, becausetheir degree is decreased to zero.

Therefore, one interpretation of the Tanner unwrappingtechnique in terms of graph covers is that the Tanner graph ofthe convolutional code is obtained by taking a suitable graphcover of the same proto-graph that was used to construct thequasi-cyclic LDPC code, along with some suitable shortening.

Example 11 We continue Remark 5 and Example 10. Clearly,

in the same way as the block codeC(r)

QC is equivalent to the

block codeC(r)QC, we can define a codeCconv (with parity-check

matrixHconv) that is equivalent toCconv. The observations inRemark 7 and Example 10 can then be used to show that theTanner graph ofHconv equals a graph cover of the TannergraphH

(r)

QC, along with some suitable shortening.

Therefore, the second interpretation of the Tanner unwrap-ping in terms of graph covers is that the Tanner graph of theconvolutional code is obtained by taking a suitable graph coverof the Tanner graph of the quasi-cyclic code, along with somesuitable shortening.

Now turning our attention to the JFZ unwrapping technique,recall from Section II-C that this method is based on writinga parity-check matrixH of some block codeC as the sumH =

∑

ℓ∈LHℓ (in Z) of a collection of matrices{Hℓ}ℓ∈L.The convolutional code is then defined to be

Cconv ,{

v ∈ F∞2

∣

∣

∣Hconv · v

T = 0T

}

, (12)

where

Hconv ,

H0

H1 H0

......

. . .H|L|−1 H|L|−2 . . . H0

H|L|−1 H|L|−2 . . . H0

. . .. . .

. . .. . .

. (13)

With the help of an example, we now explain how the JFZunwrapping technique can be cast in terms of graph-covers.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 12

Example 12 Consider the infinite graph covers that wereconstructed usingGCC2 in Example 6, in particularT(B).Let C

(

T(B))

be the set of valid assignments to the Tannergraph T(B). Moreover, letH , H0 + H1 + · · · + H6 ,0+A0,0 +A0,1 +A1,2 +A0,2 +A1,1 +A1,0, and letCconv

be defined as in(12). Then the codeCconv is a shortenedversion ofC

(

T(B))

, where all codeword bits correspondingto negative time indices have been shortened. Therefore, theTanner graph ofCconv is given by the Tanner graph inFigure 4(c)(bottom), where all the bit nodes with negative timeindices are shortened.

In order to connect the unwrapping techniques due to Tannerand due to JFZ, we show now, with the help of an example,that in fact the unwrapping technique due to Tanner can beseen as aspecial caseof the unwrapping technique due toJFZ.12

Example 13 Consider the quasi-cyclic block code defined bythe parity-check matrixH

(r)

QC , B, whereB was definedin (7). Applying the JFZ unwrapping technique with the matrixdecompositionH

(r)

QC = A0 + A1 (in Z), with A0 definedin (10) and A1 defined in(11), Hconv turns out to equal asubmatrix ofB in (9), namely the submatrix ofB where therow and column index set are equal toZ>0. However, thecode defined byHconv is equivalent to the code defined bythe Tanner unwrapping technique applied to the quasi-cycliccode defined byH

(r)

QC.

Therefore, the unwrapping technique due to JFZ is moregeneral. In fact, whereas the Tanner unwrapping techniqueleads totime-invariant convolutional codes, the unwrappingtechnique due to JFZ can, depending on the parity-checkmatrix decomposition and the internal structure of the termsin the decomposition, lead totime-varyingconvolutional codeswith non-trivial period.13

Despite the fact that the unwrapping technique due to Tanneris a special case of the unwrapping technique due to JFZ, itis nevertheless helpful to have both unwrapping techniquesat hand, because sometimes one framework can be moreconvenient than the other. We will use both perspectives inthe next section.

We conclude this section with the following remarks.

• Although most of the examples in this section haveregular bit node degree2 and regular check node degree3, there is nothing special about this choice of bit andcheck node degrees; any other choice would have workedequally well.

• Although all polynomial parity-check matrices that ap-pear in this section contain only monomials, this is not re-quired, i.e., the developments in this section work equallywell for polynomial parity-check matrices containing the

12We leave it as an exercise for the reader to show the validity of thisconnection beyond this specific example.

13Of course, if the underlying quasi-cyclic block code is suitably chosen,then also the Tanner unwrapping technique can yield a time-varying convo-lutional code; however, we do not consider this option here.

zero polynomial, monomials, binomials, trinomials, andso on.

• It can easily be verified that if the matrixA in Defini-tion 3 contains only zeros and ones, then the graph coversconstructed inGCC1 and GCC2 never have paralleledges. In particular, ifA is the parity-check matrix ofa block code (like in most examples in this paper), thenthe constructed graph covers never have parallel edges.However, ifA contains entries that are larger than one,then there is the potential for the constructed graph coversto have parallel edges; if parallel edges really appeardepends then critically on the choice of the decompositionA =

∑

ℓ∈L Aℓ (in Z) and the choice of the permutationmatrices{Pℓ}ℓ∈L. An example of such a case is the Tan-ner graph construction in Section V-C, whereA ,

[

3 3]

and where{Aℓ}ℓ∈L and {Pℓ}ℓ∈L are chosen such thatparallel edges are avoided in the constructed graph cover.Finally, in the case of iterated graph-cover constructions,it can make sense to have parallel edges in the inter-mediate graph covers. However, in the last graph-coverconstruction stage, parallel edges are usually avoided,because parallel edges in Tanner graphs typically lead toa weakening of the code and/or of the message-passingiterative decoder.

IV. GRAPH-COVER BASED CONSTRUCTIONS OF

LDPC CONVOLUTIONAL CODES

Although the graph-cover constructions and unwrappingtechniques that were discussed in Sections II and III are math-ematically quite straightforward, it is important to understandhow they can be applied to obtain LDPC convolutional codeswith good performance and attractive encoder and decoderarchitectures. To that end, this section explores a varietyofcode design options and comments on some practical issues.It also proposes a new “random” unwrapping technique whichleads to convolutional codes whose performance comparesfavorably to other codes with the same parameters. Of course,other variations than the ones presented here are possible,in particular, by suitably combining some of the exampleconstructions.

The simulation results for the codes in this section plotthe decoded bit error rate (BER) versus the signal-to-noiseratio (SNR)Eb/N0 and were obtained by assuming BPSKmodulation and an additive white Gaussian noise channel(AWGNC). All decoders were based on the sum-productalgorithm [36] and were allowed a maximum of 100 itera-tions, with the block code decoders employing a syndrome-check based stopping rule. For comparing the performance ofunwrapped convolutional codes with their underlying blockcodes we will use the following metric.

Definition 14 For a convolutional code constructed from anunderlying block code, we define its “convolutional gain” tobe the difference in SNR required to achieve a particular BERwith the convolutional code compared to achieving the sameBER with the block code.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 13

100

10-1

10-2

10-3

10-4

10-5

10-6

1 1.5 2 2.5 3 3.5 4 4.5 5

BE

R

Eb/N0 (dB)

[155,64] QC code[240,98] QC code[400,162] QC codeR=2/5 time-invariant conv. code with νs=145R=2/5 time-varying conv. code with νs=155R=2/5 time-varying conv. code with νs=240R=2/5 time-varying conv. code with νs=400

Fig. 7. Performance of three (3,5)-regular quasi-cyclic LDPC block codes andtheir associated time-invariant and time-varying LDPC convolutional codes.

The rest of this section is structured as follows. First wediscuss the construction of sometime-invariant LDPC con-volutional codes based on the Tanner unwrapping technique.In this context we make a simple observation about how thesyndrome former memory can sometimes be reduced withoutchanging the convolutional code. Secondly, we present aconstruction oftime-varyingLDPC convolutional codes basedon iterated graph-cover constructions. An important sub-topichere will be an investigation of the influence of the “diagonalcut” (which is used to define a graph cover) on the decodingperformance.

A. Construction of Time-Invariant LDPC ConvolutionalCodes Based on the Tanner Unwrapping Technique

In this section we revisit a class of quasi-cyclic LDPCcodes and their associated convolutional codes that was studiedin [37]. As we will see, they are instances of the quasi-cyclic code construction in Example 4 and Remark 5, andthe corresponding convolutional code construction based onTanner’s unwrapping technique in Example 10.

Example 15 Consider the regular proto-matrix

A ,

1 1 1 1 11 1 1 1 11 1 1 1 1

(14)

with mA = 3 andnA = 5. We applyGCC1, as in Example 4and Remark 5, with an interesting choice of permutationmatrices first suggested by Tanner [38] that yields the parity-check matrix

H(r)QC ,

I1 I2 I4 I8 I16

I5 I10 I20 I9 I18

I25 I19 I7 I14 I28

, (15)

where as beforeIs is ans times left-circularly shifted identifymatrix of sizer×r andr > 28. The corresponding polynomialparity-check is

H(r)QC(X) ,

X1 X2 X4 X8 X16

X5 X10 X20 X9 X18

X25 X19 X7 X14 X28

.

The resulting quasi-cyclic(3, 5)-regular LDPC block codeshave block lengthn = 5 · r. In particular, for r = 31, r = 48,and r = 80, we obtain codes of length155, 240, and 400,respectively, whose simulated BER performance results areshown in Figure 7. The choicer = 31 yields the well-knownlength-155 quasi-cyclic block code that was first introducedby Tanner [38] (see also the discussion in [19]).

Unwrapping these codes by the Tanner unwrapping tech-nique as in Example 10, we obtain a rate-2/5 time-invariantconvolutional code withνs = 145 defined by the polynomialparity-check matrix

Hconv(D) ,

D1 D2 D4 D8 D16

D5 D10 D20 D9 D18

D25 D19 D7 D14 D28

.

Its decoding performance is also shown in Figure 7 under thelabel “R = 2/5 time-invariant conv. code withνs = 145.” Weconclude this example with a few remarks.

• Figure 7 shows that the convolutional code exhibits a“convolutional gain” of between0.5 dB and 0.7 dBcompared to the[155, 64] quasi-cyclic LDPC block codeat moderate BERs and that the gain remains between0.15 dB and 0.3 dB at lower BERs.

• Note that the polynomial parity-check matrixHconv(D)that is obtained by the Tanner unwrapping technique isindependent of the parameterr of the polynomial parity-check matrixH(r)

QC(X), as long asr is strictly larger than

the largest exponent appearing inH(r)QC(X). Moreover,

for r → ∞, the Tanner graph ofH(r)QC(X) is closely

related to the Tanner graph ofHconv(D), and so it isnot surprising to see that, for largerr, the decodingperformance of quasi-cyclic LDPC block codes based onH

(r)QC(X) tends to the decoding performance of the LDPC

convolutional based onHconv(D), as illustrated by thetwo curves labeled “[240, 98] QC code” and “[400, 162]QC code” in Figure 7.

• The permutation matrices (more precisely, the circulantmatrices) that were used for constructing the quasi-cycliccodes in this example werenot chosen to optimize theHamming distance or the pseudo-weight properties ofthe code. In particular, a different choice of circulantmatrices may result in better high-SNR performance,i.e., in the so-called “error floor” region of the BERcurve. For choices of codes with better Hamming distanceproperties, we refer the reader to [39].

• The remaining curves in Figure 7 will be discussed inExample 18.

We conclude this subsection with some comments on thesyndrome former memoryms of the convolutional codesobtained by the Tanner unwrapping technique, in particularhow this syndrome former memoryms can sometimes bereduced without changing the convolutional code.

Assume that we have obtained a polynomial parity-checkmatrix Hconv(D) from H

(r)QC(X) according to the Tanner

method. Clearly, the syndrome former memoryms is givenby the largest exponent that appears inHconv(D). In some

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 14

instances there is a simple way of reducingms withoutchanging the convolutional code. Namely, ife is theminimalexponent that appears in the polynomials of a given row ofHconv(D), then the polynomials in this row ofHconv(D) canbe divided byDe. We illustrate this syndrome former memoryreduction for the small convolutional code that appeared inExample 10.

Example 16 Applying the Tanner unwrapping technique tothe polynomial parity-check matrixH(r)

QC(X) of the quasi-cyclic LDPC code withr = 7 in Remark 5, we obtainHconv(D) of a rate-1/3 time-invariant LDPC convolutionalcode, as shown in Example 10, with syndrome former memoryms = 6. Following the procedure discussed in the paragraphabove, the first and second rows ofHconv(D) can be dividedby D1 and D3, respectively, to yield an equivalent convo-lutional code with syndrome former memoryms = 3 andpolynomial parity-check matrix

Hconv(D) =

[

D0 D1 D3

D3 D2 D0

]

. (16)

Figure 8 shows parts of the corresponding scalar parity-checkmatrix Hconv for ms = 3, together with the original scalarparity-check matrix forms = 6, and illustrates the equivalenceof the two matrices in the sense that only the ordering ofthe rows is different, which does not affect the correspondingconvolutional code. In this example, the order of the even-numbered rows stays the same, while the odd-numbered rowsare shifted by four positions. The equivalence of the two parity-check matrices can be seen by noting that the parity-checkmatrix, outside of the diagonal structure, is filled with zeros.

B. Construction of Time-Varying LDPC Convolutional CodesBased on Iterated Graph-Cover Constructions

As was seen in Example 9, interesting graph covers canbe obtained by combiningGCC1 with GCC2, or vice-versa.Inspired by that example, this subsection considers iteratedgraph-cover constructions for constructing Tanner graphsofLDPC convolutional codes, in particular of time-varyingLDPC convolutional codes.

Definition 17 Based on a combination ofGCC1, GCC2, andthe code-shortening concept introduced in Section III-C, wepropose the following construction of LDPC convolutionalcodes.

1) We start with a proto-matrixA of sizemA × nA.2) We applyGCC1 to A with finite-size permutation ma-

trices and obtain the matrixA′.3) We applyGCC2 to A

′ with permutation matrices thatare bi-infinite Toeplitz matrices and obtain the matrixA

′′.4) Finally, looking atA′′ as the parity-check matrix of a bi-

infinite convolutional code, we obtain the parity-checkmatrix of a convolutional code by shortening the codebit positions corresponding to negative time indices.

0 0 0

0 0 0

0 0 0

0 0 0

1 0 0

0 0 0

1 0 0

0 0 0

0 1 0

0 0 0

0 0 0

0 0 0

0 0 0

0 1 0

0 0 0

1 0 0

0 0 1

0 0 0

0 0 0

0 0 1

0 1 0

0 0 0

1 0 0

0 0 0

0 0 0

0 0 0

0 0 0

1 0 0

0 0 0

0 1 0

0 0 1

0 0 0

0 0 0

0 0 1

0 1 0

0 0 0

1 0 0

0 0 0

0 0 0

1 0 0

0 0 0

0 1 0

0 0 1

0 0 0

0 0 0

0 0 1

0 1 0

0 0 0

0 0 0

1 0 0

0 0 0

0 1 0

0 0 1

0 0 0

0 0 0

0 0 1

0 0 0

0 1 0

0 0 1

1 0 0

0 1 0

0 0 0

1 0 0

0 0 1

0 0 1

1 0 0

0 0 0

0 1 0

0 1 0

0 0 0

1 0 0

0 0 1

0 0 1

1 0 0

0 0 0

0 1 0

0 1 0

0 0 0

1 0 0

0 0 1

0 0 1

1 0 0

0 0 0

0 1 0

0 1 0

0 0 0

1 0 0

0 0 1

0 0 00 0 0

(a)

(b)

0 0 0

Fig. 8. Parts of the scalar parity-check matrices (see (2)) corresponding to thetwo equivalent LDPC convolutional codes with syndrome former memories(a) ms = 6 and (b)ms = 3.

Here, Steps 3 and 4 can be seen as an application of the JFZunwrapping method.

The following example shows how this construction can beapplied to obtain LDPC convolutional codes with excellentperformance. (In the example, where suitable, we will refertothe analogous matrices of Example 9 and Figure 6 that wereused to illustrate the iterated graph-cover construction.)

Example 18 Based on Definition 17, we construct an LDPCconvolutional code by performing the following steps.

1) We start with the same regular proto-matrixA as inExample 15, for whichmA = 3 andnA = 5.

2) We applyGCC1 toA with permutation matrices that arecirculant matrices of sizer × r and obtain the parity-check matrixA′ = H

(r)QC shown in(15), which is the

analogue ofA(1) in Figure 6(a).3) We apply GCC2 to A

′ = H(r)QC with permutation

matrices that are bi-infinite Toeplitz matrices and obtaina new parity-check matrixA′′. This is analogous to thetransition of the matrixA(1) in Figure 6(a) to the matrixB

(1)in Figure 6(c). The “diagonal cut” is obtained by

alternately movingnA = 5 units to the right and thenmA = 3 units down.

4) Finally, we obtain the desired convolutional code byshortening the code bit positions corresponding to neg-ative time indices.

For the choicesr = 31, 48, 80, this construction resultsin rate-2/5 time-varying convolutional codes with syndromeformer memoryms = 30, 47, 79, respectively, and withconstraint lengthνs = (ms+1) ·nA = 155, 240, 400, respec-tively. The label “time-varying” is indeed justified because theconvolutional codes constructed here can be expressed in theform of the parity-check matrix in(1) with a suitable choiceof syndrome former memoryms, non-trivial period Ts, andsubmatricesHi(t).

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 15

The decoding performance of these codes is shown in Fig-ure 7, labeled “R = 2/5 time-varying convolutional code withνs = . . .”. As originally noted in [1], we observe that thesethree LDPC convolutional codes achieve significantly betterperformance at a BER of10−6 than the other codes shownin this plot, namely with “convolutional gains” of2.0 dB forthe νs = 155 convolutional code,2.4 dB for the νs = 240convolutional code, and2.8 dB for theνs = 400 convolutionalcode, compared to the three respective underlying LDPC blockcodes.

In order to compare these codes based on a given decodingprocessor (hardware) complexity, we consider a block codeof length n = νs (see [40] and [41]). The above time-varying convolutional code forr = 31 has constraint lengthνs = (ms + 1) · c = 155, and hence approximately the sameprocessor complexity as the quasi-cyclic block code of lengthn = 155 in Figure 7 and the time-invariant convolutional codewith νs = 145 in Figure 7, but it achieves large BER gainscompared to both of these codes. We note, in addition, thatthe performance of the time-varying convolutional code withconstraint lengthνs = 400 is quite remarkable, since, at aBER of10−5, it performs within1 dB of the iterative decodingthreshold of 0.965 dB, while having the same processorcomplexity as a block code of length onlyn = 400. In SectionVI-C, we discuss some possible reasons for these “convolu-tional gains,” along with their associated implementationcostsin terms of decoder memory and decoding delay.

We make the following observations with respect to theabove definition and example.

• The LDPC code construction in the above example yieldstime-varying LDPC convolutional codes with syndromeformer memoryms 6 r − 1 and periodTs = r.Most importantly, varyingr in the above constructionleads to different LDPC convolutional codes. This is incontrast to the Tanner unwrapping technique discussedin Section IV-A, where the obtained LDPC convolutionalcode is independent of the parameterr, as long asr isstrictly larger than the largest exponent inH(r)

QC(X).• As mentioned previously in Example 9, the iterated

graph-cover construction based on the combination ofGCC1 and GCC2 yields Tanner graphs that have a“pseudo-random” structure, a structure that seems to bebeneficial as indicated by the above simulation results.(We remark that the improved performance of the time-varying LDPC convolutional codes obtained by unwrap-ping a randomly constructed LDPC block code was firstnoted by Lentmaieret al. in [42].)

• Instead of constructing a first parity-check matrix as inStep 2 of Definition 17, one can also start with anyother (randomly or non-randomly constructed, regularor irregular) parity-check matrix, and still achieve a“convolutional gain.” The next example is an illustrationof this point.

Example 19 As was done in [41], one can replace the parity-check matrix that was constructed in Step 2 of Definition 17by an irregular LDPC block code with optimized iterative

100

10-1

10-2

10-3

10-4

10-5

0 0.5 1 1.5 2 2.5 3 3.5 4

BE

R

Eb/N0 (dB)

n=2500 R=1/2 protographUnwrapped conv. code with νs=2500n=2520 R=2/3 protographUnwrapped conv. code with νs=2520n=2520 R=3/4 protographUnwrapped conv. code with νs=2520n=2508 R=4/5 protographUnwrapped conv. code with νs=2508

Fig. 9. Performance of a family of irregular proto-graph-based LDPC blockcodes and the associated time-varying LDPC convolutional codes.

decoding thresholds. In particular, one can start with theparity-check matrix of the rate-1/2 irregular proto-graph-based code from [43] with an iterative decoding threshold of0.63 dB, and several of its punctured versions. Figure 9 showssimulation results for the obtained block and convolutionalcodes. Each simulated block code had a block length of about2500, with code rates ranging from1/2 to 4/5. We see that“convolutional gains” ranging from0.6 dB to 0.9 dB at aBER of10−5 were obtained.

Similarly, it was shown in [25] that an LDPC convolutionalcode derived from a randomly constructed rate-1/2 irregularLDPC block code with block length2400 outperformed theunderlying code by almost0.8 dB at a BER of10−5. Thedegree distribution of the underlying LDPC block code wasfully optimized and had an iterative decoding threshold of0.3104 dB [12].

Of course, there are other ways of applying the “diagonalcut” in Step 3 of Example 18, and so it is natural to investigatethe influence of different “diagonal cuts” on the decodingperformance. We will do this in the next few paragraphsby extending the discussion that was presented right afterExample 1.

We start by assuming that the matrix after Step 2 ofDefinition 17 has sizem×n, and defineη , gcd(m,n). Then,for any positive integerℓ that dividesη, we can perform a“diagonal cut” where we alternately movec′ = ℓ · (n/η) unitsto the right and thenc′ − b′ , ℓ · (m/η) units down (i.e.,b′ = ℓ ·

(

(n − m)/η)

. With this, the obtained convolutionalcode is a periodically time-varying LDPC convolutional codewith rate R′ = b′/c′ = b/c, syndrome former memorym′

s = (n/c′) − 1 = (η/ℓ) − 1, periodT ′s = m′

s + 1 = n/c′,and constraint lengthν′s = c′ · (m′

s + 1) = n. (Note that thesyndrome former memorym′

s depends onℓ, but the constraintlengthν′s is independent ofℓ.)

Example 20 Here we simulate the performance of someLDPC convolutional codes obtained according to the abovegeneralization of the “diagonal cut.” Namely, we start witha

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 16

10-6

10-5

10-4

10-3

10-2

10-1

0 0.5 1 1.5 2

BE

R

Eb/N0 (dB)

Block code with n=2048Conv. code with b’=ℓ=1, c’=2, m’s=1023Conv. code with b’=ℓ=2, c’=4, m’s=511Conv. code with b’=ℓ=4, c’=8, m’s=255Conv. code with b’=ℓ=8, c’=16, m’s=127Conv. code with b’=ℓ=16, c’=32, m’s=63Conv. code with b’=ℓ=32, c’=64, m’s=31Conv. code with b’=ℓ=64, c’=128, m’s=15Conv. code with b’=ℓ=128, c’=256, m’s=7Conv. code with b’=ℓ=256, c’=512, m’s=3Conv. code with b’=ℓ=512, c’=1024, m’s=1Conv. code with b’=ℓ=1024, c’=2048, m’s=0

Fig. 10. Performance of a family of LDPC convolutional codesobtainedfrom a (3, 6)-regular LDPC block code using different step sizes.

randomly-constructed(3, 6)-regular LDPC block code basedon a parity-check matrix of size1024 × 2048. Thereforem = 1024, n = 2048, and η , gcd(m,n) = 1024. (Notethat c′ = ℓ · (n/η) = 2ℓ and b′ = ℓ ·

(

(n − m)/η)

= ℓ inthis case.) Figure 10 shows the performance of the resultingfamily of LDPC convolutional codes, whereℓ varies in powersof 2 from 1 to 1024, each with constraint lengthν′s = 2048.We make the following observations. First, the caseℓ = 1024is not interesting because it results inm′

s = 0, i.e., it is atrivial concatenation of copies of the block code, and so theBER is the same as for the underlying block code. Secondly,for all other choices ofℓ, the constructed codes performvery similarly, each exhibiting a sizable “convolutional gain”compared to the block code, although the syndrome formermemorym′

s is different in each case.

A special case of the above code construction deservesmention. Whenη = 1, i.e., m and n are relatively prime,the only possible step size is obtained by choosingℓ = η = 1,which results in the above-mentioned uninteresting case oftrivial concatenations of copies of the block code. However,all-zero columns can be inserted in the parity-check matrixsuch that a value ofη > 1 is obtained, which allows a stepsize to be chosen that results in a convolutional code withm′

s > 0. The variable nodes corresponding to the all-zerocolumns are not transmitted, i.e., they are punctured, so thatthe rate corresponds to the size of the original parity-checkmatrix.

For the “diagonal cut” LDPC convolutional code construc-tions discussed above, the unwrapped convolutional codeshave the minimum possible constraint lengthν′s, which is equalto the block length of the underlying block code. Although thisis a desirable property for practical implementation, we donotneed to limit ourselves to diagonal cuts in general.

Inspired by the graph-cover construction of Figures 5(b)and 5(d) in Example 8, instead of a “diagonal cut” we nowconsider a “random cut,” which we define as a partitioning ofthe parity-check matrix into two matrices that add up (overZ) to the parity-check matrix. Despite the randomness of this

10-6

10-5

10-4

10-3

10-2

10-1

0 0.5 1 1.5 2

BE

R

Eb/N0 (dB)

Block code with n=2048Conv. code with b’=1, c’=2 (ℓ=1)

Conv. code with random partitionConv. code with random partitionConv. code with random partitionConv. code with random partitionConv. code with random partition

Fig. 11. Performance of “randomly unwrapped” LDPC convolutional codesobtained from a(3, 6)-regular LDPC block code using random partitions.

approach, several of the key unwrapping properties of the“diagonal cut” are preserved. For example, the computationalcomplexity per decoded bit does not change, since the degreedistributions of the resulting codes are all equal.14 However,the LDPC convolutional codes based on a “random cut”typically require larger decoding processor sizes as a resultof increased code constraint lengths.

Example 21 We continue Example 20; however, instead ofperforming “diagonal cuts,” we perform “random cuts.” Fig-ure 11 shows the performance of five such LDPC convolutionalcodes, each with rate1/2 and constraint lengthν′s = 4096,compared to the underlying block code and the LDPC con-volutional code constructed in Example 20 (with parametersℓ = 1, b′ = 1, c′ = 2, and ν′s = 2048). We note that theincrease in constraint length fromν′s = 2048 to ν′s = 4096due to the “random cut” results in a small additional codinggain in exchange for the increased decoding processor size.

Finally, we note that, for a sizem× n sparse parity-checkmatrix H with p nonzero entries, there are a total of2mn

possible ways of choosing a random cut. However, due tothe sparsity, there are only2p distinct random cuts, wherep ≪ n ·m.

V. CONNECTIONS TOOTHER LDPC CODES BASED ON

GRAPH-COVER CONSTRUCTIONS

In this section we briefly discuss some other graph-coverbased LDPC code constructions proposed in the literature,namely by Ivkovicet al. [44], by Divsalaret al. [43], [45], byLentmaieret al. [46], [47], and by Kudekaret al. [48].

A. LDPC Code Construction by Ivkovic et al.

The LDPC code construction by Ivkovicet al. in [44] canbe seen as an application of the graph-cover construction in

14This is guaranteed by choosing a random partitioning of the block codeparity-check matrix and then using this partitioning to construct one periodof the time-varying convolutional code parity-check matrix.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 17

Figures 5(b) and 5(d) in Example 8. Namely, in terms of ournotation, Ivkovicet al. [44] start with a parity-check matrixH ,choose the setL , {0, 1}, a collection of zero-one matrices{H0,H1} such thatH = H0+H1 (in Z), and the collectionof permutation matrices{P0,P1}ℓ∈L ,

{[

1 00 1

]

,[

0 11 0

]}

. Mostimportantly, the decomposition ofH into H0 andH1 is donesuch that trapping sets that were present in the Tanner graphof H , are not present in the Tanner graph of the new parity-check matrix. In addition, Ivkovicet al. give guarantees onthe relationship between the minimum Hamming distances ofthe old and new code.15

B. LDPC Code Construction by Divsalar et al.

One of the LDPC code constructions by Divsalaret al.in [43], [45] is the so-called rate-1/2 AR4JA LDPC codeconstruction, which was also considered earlier in Example19.A particularly attractive, from an implementation perspective,version of this code construction is obtained by an iteratedgraph-cover construction procedure, where each graph-coverconstruction is based on a cyclic cover, as in the applicationof GCC1 in Example 4. Although cyclic covers result insimplified encoding and decoding circuitry, codes based oncyclic covers are known to have the disadvantage that theminimum Hamming distance is upper bounded by a numberthat is a function of the proto-graph structure [49], [50].However, because the cyclic cover of a cyclic cover of theproto-graph isnot necessarilya cyclic cover of the proto-graph, such disadvantages are avoided to a certain extent intheAR4JA LDPC code construction. Nevertheless, ultimately theminimum Hamming distance of such codes will also be upperbounded by some number; however, these bounds usuallybecome relevant only beyond the code length of interest.16

C. LDPC Code Construction by Lentmaier et al. and Kudekaret al.

The LDPC code constructions by Lentmaieret al. [46], [47]and Kudekaret al. [48] can also be seen as iterated graph-cover constructions. We now describe a specific instance ofthis construction.

• It starts with a proto-matrixA ,[

3 3]

.• The first graph-cover construction is very similar to the

bi-infinite graph-cover construction in Example 6 andFigure 4. Namely, in terms of our notation, we definethe setL , {0, 1, 2, 3, 4, 5}, the collection of matrices{Aℓ}ℓ∈L with A0 = A1 = A2 =

[

1 0]

and A3 =A4 = A5 =

[

0 1]

, and the collection of permutationmatrices{Pℓ}ℓ∈L with P0 , T0, P1 , T1, P2 , T2,P3 , T0, P4 , T1, P5 , T2, where as beforeTs is abi-infinite Toeplitz matrix with zeros everywhere exceptfor ones in thes-th diagonal below the main diagonal.

• The second graph-cover construction is a random graph-cover construction of cover-degreeM .

15See also the discussion of similar results in [49, Appendix J].16For this statement we assume that the degree of the first coveris fixed.

• The code is shortened. Namely, for some positive integerL all codeword indices corresponding to values outsidethe range[−LM,LM ] are shortened.17

We now point out some differences between this code con-struction and the LDPC convolutional code construction inDefinition 17. Namely, the LDPC code ensemble constructedabove has the following properties.

• The first graph-cover construction is based on bi-infiniteToeplitz permutation matrices, and the second graph-cover construction is based on finite-size permutationmatrices.

• The analysis focuses on the case whereM andL go toinfinity (in that order), i.e., for a fixedL the parameterM tends to infinity. Afterwards,L tends to infinity.

• The number of check nodes with degree smaller than6in the Tanner graph is proportional toM .

• In [48], for the binary erasure channel, whenM andLgo to infinity (in that order), Kudekaret al. prove thatthe sum-product algorithm decoding threshold for a slightvariation of the above-mentioned ensemble of codesequals the maximum a-posteriori decoding threshold forthe ensemble of(3, 6)-regular LDPC codes. This is a veryremarkable property! (In [51], using density evolutionmethods, Lentmaieret al. give numerical evidence thatthis statement might also hold for binary-input output-symmetric channels beyond the binary erasure channel.)

On the other hand, the codes constructed in Definition 17 havethe following properties. (We assume that the underlying blockcode is a(3, 6)-regular LDPC code.)

• The first graph-cover construction is based on finite-size permutation matrices, and the second graph-coverconstruction is based on bi-infinite Toeplitz permutationmatrices.

• In a typical application of this construction,r is fixed.• The number of check nodes with degree smaller than6

in the Tanner graph of the LDPC convolutional code isproportional tor.

• For a binary-input output-symmetric channel, the per-formance of the unterminated LDPC convolutional codeunder the continuous sliding window sum-product algo-rithm decoding discussed in Section II-B improves withincreasingr (see, e.g., Fig. 7), but the ultimate asymptoticthreshold of such unterminated decoding is unknown.18

The differences between these two code families come mainlyfrom the fact that the codes constructed by Lentmaieretal. and Kudekaret al. are essentially block codes, althoughsophisticated ones, whereas the codes in Definition 17 are

17Although this code construction method could be presented such that theshortening is done between the two graph-cover construction steps, namelyby shortening all codeword indices that correspond to values outside therange [−L,L], we have opted to present the code construction such thatthe shortening is done after the two graph-cover construction steps. In thisway, the structure of the code construction description matches better thedescription in Definition 17.

18Lentmaieret al. have shown in [46] and [47] that properly terminatedLDPC convolutional codes become equivalent to the LDPC block codesconstructed by Kudekaret al. in [48] and inherit their excellent asymptoticthreshold properties, but whether this is true for unterminated LDPC convo-lutional codes is still an open question.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 18

convolutional codes, along with their advantages and disad-vantages. In particular, the way the limits of the parametersare taken, there is a significant difference in the fraction ofcheck nodes with degree strictly smaller than6. Namely, inthe case of the codes by Lentmaieret al. and Kudekaretal. this fraction is a fixed non-zero function ofL (here weassume fixedL and M → ∞), whereas in the case of thecodes considered in this paper, this fraction is zero (here weassume fixedr and an unterminated convolutional code).

We conclude this section with the following remarks.Namely, although the convolutional codes in Definition 17may not enjoy the same asymptotic thresholds as the blockcode constructions discussed in this subsection, they lendthemselves to a continuous decoding architecture, as describedin Section II-B, which can be advantageous in certain ap-plications, such as data streaming, without a predeterminedframe structure. More importantly, however, it is very en-couraging that the simulation results reported in this paperindicate that sizable “convolutional gains” are already visiblefor very reasonable constraint/code lengths, and in the nextsection we discuss some possible reasons for these gains.Finally, it is worth noting that, as the block lengths andassociated constraint lengths of the constructions presented inthis section become larger, the observed “convolutional gains”will become smaller since the block code results will approachtheir respective thresholds.

VI. A NALYSIS OF DERIVED

LDPC CONVOLUTIONAL CODES

This section collects some analytical results about LDPCconvolutional codes. In particular, we compare the existence /non-existence of cycles in LDPC block and LDPC convolu-tional codes, we present some properties of pseudo-codewords,and we discuss the — mostly moderate — cost increase indecoder complexity that is incurred by going from LDPCblock to LDPC convolutional codes.

A. Graph-Cycle Analysis

It is well known that cycles in the Tanner graph repre-sentation of a sparse code affect message-passing iterativedecoding algorithms, with short cycles generally pushing theperformance further away from optimum. (Indeed, attempts toinvestigate and minimize these effects have been made in [52]and [53], where the authors propose LDPC code constructionprocedures to maximize the connectivity of short cycles to therest of the graph, thus also maximizing the independence ofthe messages flowing through a cycle.) Hence it is commonpractice to design codes that do not contain short cycles, soasto obtain independent messages in at least the initial iterationsof the decoding process.

Avoiding cycles in Tanner graphs also has the benefit ofavoiding pseudo-codewords.19 To see this, let the active part

19Here and in the following, pseudo-codewords refer to pseudo-codewordsas they appear in linear programming (LP) decoding [54], [55] and in thegraph-cover-based analysis of message-passing iterativedecoding in [23],[24]. For other notions of pseudo-codewords, in particularcomputation treepseudo-codewords, we refer to the discussion in [56].

TABLE IAVERAGE (PER BIT NODE) NUMBER Nℓ OF CYCLES OF LENGTHℓ FOR THE

TANNER GRAPHS OF THE BLOCK CODES(BCS) OF BLOCK LENGTHn ANDCONVOLUTIONAL CODES(CCS) OF CONSTRAINT LENGTHνs DISCUSSED

IN EXAMPLE 22. (ALL TANNER GRAPHS HAVE GIRTH8.)

Code N8 N10 N12

BC (n = 155) 3.000 24.000 146.000BC (n = 240) 2.600 14.000 93.400BC (n = 400) 2.200 12.400 70.600

Time-invariant CC (νs = 145) 2.200 12.400 70.200

Time-varying CC (νs = 155) 0.910 8.342 44.813Time-varying CC (νs = 240) 0.917 5.338 30.242Time-varying CC (νs = 400) 0.675 4.705 24.585

of a pseudo-codeword be defined as the set of bit nodescorresponding to the support of the pseudo-codeword, alongwith the adjacent edges and check nodes. With this, it holdsthat the active part of any pseudo-codeword contains at leastone cycle and/or at least one bit node of degree one. Andso, given that the typical Tanner graph under considerationinthis paper does not contain bit nodes of degree one, the activepart of a pseudo-codeword must contain at least one cycle.Therefore, avoiding cycles implicitly means avoiding pseudo-codewords.20

Let H andH be two parity-check matrices such thatT(H)is a graph cover ofT(H). It is a well-known result that anycycle in T(H) can be mapped into a cycle inT(H). Thishas several consequences. In particular, the girth ofT(H) isat least as large as the girth ofT(H), and more generally,T(H) contains fewer short cycles thanT(H).21 For the codesconstructed in this paper, this means that the unwrappingprocess (from block code to convolutional code) can “break”some cycles in the Tanner graph of the block code.

We now revisit some codes that where discussed in earliersections and analyze their graph cycle structure using a brute-force search algorithm.22 Note that, in order to accuratelycompare the graph cycle distributions of two codes with dif-ferent block/constraint lengths, we compute the total numberof cycles of a given cycle length per block/constraint length,and divide this number by the block/constraint length.23

Example 22 Consider the LDPC block and convolutionalcodes that were constructed in Examples 15 and 18 and whoseBER performance was plotted in Figure 7. Table I shows theaverage number of cycles of certain lengths for the Tannergraphs of the quasi-cyclic block codes, for the Tanner graph

20Note that the support of any pseudo-codeword is a stopping set [23], [24],[57].

21This observation has been used in many different contexts over the pastten years in the construction of LDPC and turbo codes; in particular, it wasused in [42], where the authors dealt with bounding the girthof the resultingLDPC convolutional codes.

22The brute-force search technique that we used is based on evaluating thediagonal entries of the powers of the matrixM defined in [34, Eq. (3.1)].Note that this search technique works only for counting cycles of lengthsmaller than twice the girth of the graph. For searching longer cycles, moresophisticated algorithms are needed.

23For LDPC convolutional codes, we have made use of the periodicity ofthe parity-check matrices in order to achieve the search in afinite number ofsteps.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 19

TABLE IIAVERAGE (PER BIT NODE) NUMBER Nℓ OF CYCLES OF LENGTHℓ FOR THE

TANNER GRAPHS OF THE BLOCK CODES(BCS) OF BLOCK LENGTHn ANDCONVOLUTIONAL CODES(CCS) OF CONSTRAINT LENGTHνs DISCUSSED

IN EXAMPLE 23. (ALL TANNER GRAPHS HAVE GIRTH4.)

Code N4 N6

Rate-1/2 BC (n = 2500) 0.013 0.120Rate-2/3 BC (n = 2520) 0.065 0.839Rate-3/4 BC (n = 2520) 0.136 2.710Rate-4/5 BC (n = 2508) 0.250 6.544

Rate-1/2 time-varying CC (νs = 2500) 0.010 0.064Rate-2/3 time-varying CC (νs = 2520) 0.044 0.483Rate-3/4 time-varying CC (νs = 2520) 0.091 1.465Rate-4/5 time-varying CC (νs = 2508) 0.173 3.622

of the corresponding time-invariant convolutional code, andfor the Tanner graph of the time-varying convolutional codes.

Example 23 Table II shows the cycle analysis results forthe rate-1/2 proto-graph-based codes that were discussedin Example 19 and whose BER performance was plotted inFigure 9.

From Examples 22 and 23, we see that many of the shortcycles in the Tanner graphs of the LDPC block codes are“broken” to yield cycles of larger length in the Tanner graphsof the derived LDPC convolutional codes.

B. Pseudo-Codeword Analysis

This section collects some comments concerning thepseudo-codewords of the parity-check matrices under consid-eration in this paper.

We start by observing that many of the statements that weremade in [37] about pseudo-codewords can be extended to thesetup of this paper. In particular, if some parity-check matricesH and H are such thatT(H) is a graph cover ofT(H),then a pseudo-codeword ofH can be “wrapped” to obtain apseudo-codeword ofH , as it is formalized in the next lemma.

Lemma 24 Let the parity-check matricesH andH be suchthatT(H) is anM -fold graph cover ofT(H). More precisely,let H =

∑

ℓ∈LHℓ⊗Pℓ for some setL, for some collection ofparity-check matrices{Hℓ}ℓ∈L such thatH =

∑

ℓ∈L Hℓ (inZ), and for some collection ofM ×M permutation matrices{Pℓ}ℓ∈L. Moreover, letI be the set of column indices ofHand let I × M with M , {0, 1, . . . ,M − 1} be the set ofcolumn indices ofH. With this, if ω = (ω(i,m))(i,m)∈I×M isa pseudo-codeword ofH , thenω = (ωi)i∈I with

ωi ,1

M

∑

m∈M

ω(i,m) (in R) (17)

is a pseudo-codeword ofH .

Proof: (Sketch.) There are different ways to verify thisstatement. One approach is to show that, based on the factthat ω satisfies the inequalities that define the fundamental

polytope ofH [23], [24], [54], [55],ω satisfies the inequalitiesthat define the fundamental polytope ofH . (We omit thedetails.) Another approach is to use the fact that pseudo-codewords with rational entries are given by suitable projec-tions of codewords in graph covers [23], [24]. So, for everypseudo-codewordω of H with rational entries there is somegraph cover ofT(H) with a codeword in it, which, whenprojected down toT(H), givesω. However, that graph coverof T(H) is also a graph cover ofT(H), and so this codeword,when projected down toT(H), givesω as defined in (17). (Weomit the details; see [37] for a similar, but less general, result.)

One can then proceed as in [37] and show that the AWGNC,the BSC, and the BEC pseudo-weights [4], [23], [24], [54],[55], [58] of ω will be at least as large as the correspondingpseudo-weights ofω. As a corollary, the minimum AWGNC,BSC, and BEC pseudo-weights ofH are, respectively, at leastas large as the corresponding minimum pseudo-weights ofH . Similar results can also be obtained for the minimumHamming distance.

Because the high-SNR behavior of linear programmingdecoding is dominated by the minimum pseudo-weight ofthe relevant parity-check matrix, the high-SNR behavior oflinear programming decoding of the code defined byH is atleast as good as the high-SNR behavior of linear programmingdecoding of the code defined byH .24

In general, because of the observations made in Sec-tion VI-A about the “breaking” of cycles and the fact thatthe active part of a pseudo-codeword must contain at leastone cycle, it follows that the unwrapping process is beneficialfor the pseudo-codeword properties of an unwrapped code,in the sense that many pseudo-codewords that exist in thebase code do not map to pseudo-codewords in the unwrappedcode. It is an intriguing challenge to better understand thisprocess and its influence on the low-to-medium SNR behaviorof linear programming and message-passing iterative decoders,in particular, to arrive at a better analytical explanationof thesignificant gains that are visible in the simulation plots thatwere shown in Section IV. To this end, the results of [46]and [48] with respect to some related code families (see thediscussion in Section V) will be very helpful, since theyindicate that some of the features of the fundamental polytopedeserve further analysis.

C. Cost of the “Convolutional Gain”

In this subsection, we investigate the cost of the convolu-tional gain by comparing several aspects of decoders for LDPCblock and convolutional codes. In particular, we considerthe computational complexity, hardware complexity, decodermemory requirements, and decoding delay. More details on thevarious comparisons described in this section can be found in[31], [41], and [40].

LDPC block code decoders and LDPC convolutional codedecoders have the same computational complexity per decodedbit and per iteration since LDPC convolutional codes derived

24We neglect here the influence of the multiplicity of the minimum pseudo-weight pseudo-codewords.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 20

from LDPC block codes have the same node degrees (row andcolumn weights) in their Tanner graph representations, whichdetermines the number of computations required for message-passing decoding.

We adopt the notion ofprocessor sizeto characterize thehardware complexity of implementing the decoder. A de-coder’s processor size is proportional to the maximum numberof variable nodes that can participate in a common checkequation. This is the block lengthn for a block code, sinceany two variable nodes in a block can participate in the samecheck equation. For a convolutional code, this is the constraintlengthνs, since no two variable nodes that are more thanνspositions apart can participate in the same check equation.Theconstraint lengths of the LDPC convolutional codes derivedfrom LDPC block codes of lengthn satisfyνs 6 n. Therefore,the convolutional codes have a processor size less than or equalto that of the underlying block code.

On the other hand, the fully parallel pipeline decodingarchitecture penalizes LDPC convolutional codes in terms ofdecoder memory requirements (and decoding delay/latency)asa result of theI iterations being multiplexed in space ratherthan in time. The pipeline decoder architecture of Figure 1consists ofI identical processors of sizeνs performing Idecoding iterations simultaneously on independent sectionsof a decoding window containingI constraint lengths ofreceived symbols. This requiresI times more decoder memoryelements than an LDPC block code decoder that employsa single processor of sizen = νs performing I decodingiterations successively on the same block of received symbols.Therefore, the decoder memory requirements and the decodingdelay of the pipeline decoder are proportional toνs ·I, whereasthe block decoder’s memory and delay requirements are onlyproportional ton. Another way of comparing the two types ofcodes, preferred by some researchers, is to equate the blocklength of a block code to the memory/delay requirements,rather than the processor size, of a convolutional code, i.e.,to setn = νs · I. In this case the block code, now having ablock length many times larger than the constraint length oftheconvolutional code, will typically (depending onI) outperformthe convolutional code, but at a cost of a much larger hardwareprocessor. Finally, as noted in Section II, the parallel pipelinedecoding architecture for LDPC convolutional codes can bereplaced by a serial looping decoding architecture, resultingin fewer processors but a reduced throughput along with thesame memory and delay requirements.

In summary, the convolutional gain achieved by LDPCconvolutional codes derived from LDPC block codes comesat the expense of increased decoder memory requirements anddecoding delays. Although this does not cause problems forsome applications that are not delay-sensitive (e.g., deep-spacecommunication), for other applications that are delay-sensitive(e.g., real-time voice/video transmission), design specificationsmay be met by deriving LDPC convolutional codes fromshorter LDPC block codes, thus sacrificing some coding gain,but reducing memory and delay requirements, or by employinga reduced window size decoder, as suggested in the recentpaper by Papaleoet al. [29], with some resulting reduction inthe “convolutional gain.”

VII. C ONCLUSIONS

In this paper we showed that it is possible to connecttwo known techniques for deriving LDPC convolutional codesfrom LDPC block codes, namely the techniques due to Tannerand due to Jimenez-Feltstrom and Zigangirov. This connectionwas explained with the help of graph covers, which were alsoused as a tool to present a general approach for constructinginteresting classes of LDPC convolutional codes. Because it isimportant to understand how the presented code constructionmethods can be used — and in particular combined — we thendiscussed a variety of LDPC convolutional code constructions,along with their simulated performance results.

In the future, it will be worthwhile to extend the presentedanalytical results, in particular to obtain a better quantitativeunderstanding of the low-to-medium SNR behavior of LDPCconvolutional codes. In that respect, the insights in the papersby Lentmaieret al. [46], [47] and Kudekaret al. [48] on thebehavior of related code families will give valuable guidelinesfor further investigation.

ACKNOWLEDGMENTS

The authors would like to thank Chris Jones, MichaelLentmaier, David Mitchell, Michael Tanner, and Kamil Zi-gangirov for their valuable discussions and comments. We alsogratefully acknowledge the constructive comments which weremade by the reviewers.

REFERENCES

[1] A. E. Pusane, R. Smarandache, P. O. Vontobel, and D. J. Costello, Jr.,“On deriving good LDPC convolutional codes from QC LDPC blockcodes,” inProc. IEEE Intl. Symposium on Inform. Theory, Nice, France,Jun. 24–29, 2007, pp. 1221–1225.

[2] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limiterror correcting coding and decoding: turbo codes,” inProc. IEEEInternational Conference on Communications, Geneva, Switzerland,May 1993, pp. 1064–1070.

[3] R. G. Gallager, “Low-density parity-check codes,”IRE Trans. Inf.Theory, vol. 8, no. 1, pp. 21–28, Jan. 1962.

[4] N. Wiberg, “Codes and decoding on general graphs,” Ph.D.dissertation,Linkoping University, Sweden, 1996.

[5] D. J. C. MacKay and R. M. Neal, “Near Shannon limit performance oflow density parity check codes,”Electronics Letters, vol. 32, no. 18, pp.1645–1646, Aug. 1996.

[6] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, D. A. Spielman, andV. Stemann, “Practical loss-resilient codes,” inProc. 29th Annual ACMSymp. on Theory of Computing, 1997, pp. 150–159.

[7] M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman,“Improved low-density parity-check codes using irregulargraphs,”IEEETrans. Inf. Theory, vol. 47, pp. 585–598, Feb. 2001.

[8] S. Y. Chung, T. J. Richardson, and R. L. Urbanke, “Analysis of sum-product decoding of low-density parity-check codes using aGaussianapproximation,”IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 657–670,Feb. 2001.

[9] T. J. Richardson and R. L. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,”IEEE Trans. Inf. Theory,vol. 47, no. 2, pp. 599–618, Feb. 2001.

[10] R. M. Tanner, “A recursive approach to low complexity codes,” IEEETrans. Inf. Theory, vol. 27, no. 5, pp. 533–547, Sep. 1981.

[11] S. Y. Chung, G. D. Forney, Jr., T. J. Richardson, and R. L.Urbanke,“On the design of low-density parity-check codes within 0.0045 db ofthe Shannon limit,”IEEE Communications Letters, vol. 5, no. 2, pp.58–60, Feb. 2001.

[12] T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design ofcapacity-approaching irregular low-density parity-check codes,” IEEETrans. Inf. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001.

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 21

[13] P. Oswald and A. Shokrollahi, “Capacity-achieving sequences for theerasure channel,”IEEE Trans. Inf. Theory, vol. 48, no. 12, pp. 3017–3028, Dec. 2002.

[14] S. Bates, D. Elliot, and R. Swamy, “Termination sequence generationcircuits for low-density parity-check convolutional codes,” IEEE Trans.Circuits and Systems I, vol. 53, no. 9, pp. 1909–1917, Sep. 2006.

[15] S. Bates, Z. Chen, and X. Dong, “Low-density parity check con-volutional codes for Ethernet networks,” inProc. IEEE Pacific RimConference on Communications, Computers and Signal Processing,Victoria, BC, Canada, Aug. 2005.

[16] S. Bates, L. Gunthorpe, A. E. Pusane, Z. Chen, K. Sh. Zigangirov, andD. J. Costello, Jr., “Decoders for low-density parity-check convolutionalcodes with large memory,” inProc. 12th NASA Symposium on VLSIDesign, Coeur d’Alene, ID, USA, Oct. 2005.

[17] R. M. Tanner, “Error-correcting coding system,”U.S. Patent # 4,295,218,Oct. 1981.

[18] ——, “Convolutional codes from quasi-cyclic codes: a link between thetheories of block and convolutional codes,”University of California,Santa Cruz, Tech Report UCSC-CRL-87-21, Nov. 1987.

[19] R. M. Tanner, D. Sridhara, A. Sridharan, T. E. Fuja, and D. J. Costello,Jr., “LDPC block and convolutional codes based on circulantmatrices,”IEEE Trans. Inf. Theory, vol. 50, no. 12, pp. 2966–2984, Dec. 2004.

[20] A. Jimenez-Feltstrom and K. Sh. Zigangirov, “Time-varying periodicconvolutional codes with low-density parity-check matrix,” IEEE Trans.Inf. Theory, vol. 45, no. 6, pp. 2181–2191, Sep. 1999.

[21] R. M. Tanner, “On quasi-cyclic repeat-accumulate codes,” in Proc. of the37th Allerton Conference on Communication, Control, and Computing,Allerton House, Monticello, Illinois, USA, Sep. 22-24 1999, pp. 249–259.

[22] J. Thorpe, “Low-density parity-check (LDPC) codes constructed fromprotographs,”JPL INP Progress Report, vol. 42-154, Aug. 2003.

[23] R. Koetter and P. O. Vontobel, “Graph covers and iterative decoding offinite-length codes,” inProc. 3rd Intl. Turbo Symp. on Turbo Codes andRelated Topics, Brest, France, Sep. 1-5 2003.

[24] P. O. Vontobel and R. Koetter, “Graph-cover decoding and finite-lengthanalysis of message-passing iterative decoding of LDPC codes,” CoRR,available online underhttp://www.arxiv.org/abs/cs.IT/0512078, Dec. 2005.

[25] A. E. Pusane, K. Sh. Zigangirov, and D. J. Costello, Jr.,“Constructionof irregular LDPC codes with fast encoding property,” inProc. IEEEIntl. Conference on Commun., Istanbul, Turkey, Jun. 11–15, 2006.

[26] L. Zongwang, C. Lei, Z. Lingqi, S. Lin, and W. H. Fong, “Efficientencoding of quasi-cyclic low-density parity-check codes,” IEEE Trans.Commun., vol. 54, no. 1, pp. 71–81, Jan. 2006.

[27] T. J. Richardson and R. L. Urbanke, “Efficient encoding of low-densityparity-check codes,”IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 638–656, Feb. 2001.

[28] S. Lin and D. J. Costello, Jr.,Error Control Coding, 2nd ed. EnglewoodCliffs, NJ: Prentice-Hall, 2004.

[29] M. Papaleo, A. R. Iyengar, P. H. Siegel, J. Wolf, and G. Corazza,“Windowed erasure decoding of LDPC convolutional codes,” in Proc.IEEE Inform. Theory Workshop, Cairo, Egypt, Jan. 6–8 2010, pp. 78–82.

[30] S. Bates, Z. Chen, L. Gunthorpe, A. E. Pusane, K. Sh. Zigangirov, andD. J. Costello, Jr., “A low-cost serial decoder architecture for low-densityparity-check convolutional codes,”IEEE Trans. Circuits and Systems I,vol. 55, no. 7, pp. 1967–1976, Aug. 2008.

[31] A. E. Pusane, A. Jimenez-Feltstrom, A. Sridharan, M.Lentmaier, K. Sh.Zigangirov, and D. J. Costello, Jr., “Implementation aspects of LDPCconvolutional codes,”IEEE Trans. Commun., vol. 56, no. 7, pp. 1060–1069, Jul. 2008.

[32] Y. Levy and D. J. Costello, Jr., “An algebraic approach to constructingconvolutional codes from quasi-cyclic codes,” inCoding and Quantiza-tion (Piscataway, NJ, 1992), vol. 14 of DIMACS Ser. Discrete Math.Theoret. Comput. Sci., pp. 189–198, Providence, RI: Amer. Math. Soc.,1993.

[33] M. Esmaeili, T. A. Gulliver, N. P. Secord, and S. A. Mahmoud, “A linkbetween quasi-cyclic codes and convolutional codes,”IEEE Trans. Inf.Theory, vol. 44, no. 1, pp. 431–435, Jan. 1998.

[34] H. M. Stark and A. A. Terras, “Zeta functions of finite graphs andcoverings,”Adv. in Math., vol. 121, no. 1, pp. 124–165, Jul. 1996.

[35] F. J. MacWilliams and N. J. A. Sloane,The Theory of Error-CorrectingCodes. New York: North-Holland, 1977.

[36] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs andthe sum-product algorithm,”IEEE Trans. Inf. Theory, vol. 47, no. 2, pp.498–519, Feb. 2001.

[37] R. Smarandache, A. E. Pusane, P. O. Vontobel, and D. J. Costello, Jr.,“Pseudo-codeword performance analysis of LDPC convolutional codes,”IEEE Trans. Inf. Theory, vol. 55, no. 6, pp. 2577–2598, Jun. 2009.

[38] R. M. Tanner, “A [155,64,20] sparse graph (LDPC) code,”in RecentResults Session, IEEE Intl. Symposium on Inform. Theory, Sorrento,Italy, June 2000.

[39] F. Hug, I. Bocharova, R. Johannesson, B. Kudryashov, and R. Satyukov,“New low-density parity-check codes with large girth basedon hyper-graphs,” inProc. IEEE Intl. Symposium on Inform. Theory, Austin, TX,USA, Jun. 13–18 2010.

[40] D. J. Costello, Jr., A. E. Pusane, S. Bates, and K. Sh. Zigangirov, “Acomparison between LDPC block and convolutional codes,” inProc.Information Theory and Applications Workshop, San Diego, CA, USA,Feb. 6–10, 2006.

[41] D. J. Costello, Jr., A. E. Pusane, C. R. Jones, and D. Divsalar, “A com-parison of ARA- and protograph-based LDPC block and convolutionalcodes,” inProc. Information Theory and Applications Workshop, SanDiego, CA, USA, Jan. 29–Feb. 2, 2007.

[42] M. Lentmaier, D. V. Truhachev, and K. Sh. Zigangirov, “On thetheory of low-density convolutional codes II,”Problems of InformationTransmission (Problemy Peredachy Informatsii), vol. 37, pp. 288–306,Oct.-Dec. 2001.

[43] D. Divsalar, C. R. Jones, S. Dolinar, and J. Thorpe, “Protograph basedLDPC codes with minimum distance linearly growing with block size,”in Proc. IEEE Global Telecommun. Conf., vol. 3, St. Louis, MO, USA,Nov. 28 – Dec. 5 2005.

[44] M. Ivkovic, S. K. Chilappagari, and B. Vasic, “Eliminating trapping setsin low-density parity-check codes by using Tanner graph covers,” IEEETrans. Inf. Theory, vol. 54, no. 8, pp. 3763–3768, Aug. 2008.

[45] CCSDS, “Low density parity check codes for use in near-earth and deepspace applications,”Experimental Specification CCSDS 131.1-O-2, Sep.2007.

[46] M. Lentmaier, G. P. Fettweis, K. Sh. Zigangirov, and D. J. Costello,Jr., “Approaching capacity with asymptotically regular LDPC codes,” inProc. Information Theory and Applications Workshop, San Diego, CA,USA, Feb. 8–13 2009.

[47] M. Lentmaier, D. G. M. Mitchell, G. P. Fettweis, and D. J.Costello,Jr., “Asymptotically regular LDPC codes with linear distance growthand thresholds close to capacity,” inProc. Information Theory andApplications Workshop, San Diego, CA, USA, Jan. 1 – Feb. 5 2010.

[48] S. Kudekar, T. Richardson, and R. Urbanke, “Threshold saturation viaspatial coupling: Why convolutional LDPC ensembles perform so wellover the BEC,”available online under http://arxiv.org/abs/1001.1826,Jan. 2010.

[49] R. Smarandache and P. O. Vontobel, “Quasi-cyclic LDPC codes: in-fluence of proto- and Tanner-graph structure on minimum Hammingdistance upper bounds,”submitted to IEEE Trans. Inf. Theory, availableonline underhttp://arxiv.org/abs/0901.4129, Jan. 2009.

[50] B. K. Butler and P. H. Siegel, “On distance properties ofquasi-cyclicprotograph-based LDPC codes,” inProc. IEEE Intl. Symposium onInform. Theory, Austin, TX, USA, Jun. 13–18 2010.

[51] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K. S. Zigangirov,“Iterative decoding threshold analysis for LDPC convolutional codes,”IEEE Trans. Inf. Theory, to appear, Oct. 2010.

[52] T. Tian, C. R. Jones, J. D. Villasenor, and R. D. Wesel, “Selectiveavoidance of cycles in irregular LDPC code construction,”IEEE Trans.Commun., vol. 52, no. 8, pp. 1242–1247, 2004.

[53] A. Ramamoorthy and R. D. Wesel, “Analysis of an algorithm forirregular LDPC code construction,” inProc. IEEE Intl. Symposium onInform. Theory, Chicago, IL, USA, Jun. 27–Jul. 2 2004, p. 69.

[54] J. Feldman, M. J. Wainwright, and D. R. Karger, “Using linear program-ming to decode binary linear codes,”IEEE Trans. Inf. Theory, vol. 51,no. 3, pp. 954–972, Mar. 2005.

[55] J. Feldman, “Decoding error-correcting codes via linear programming,”Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge,MA, 2003.

[56] N. Axvig, D. Dreher, K. Morrison, E. Psota, L. C. Perez, and J. L.Walker, “Analysis of connections between pseudocodewords,” IEEETrans. Inf. Theory, vol. 55, no. 9, pp. 4099–4107, Sep. 2009.

[57] C. A. Kelley and D. Sridhara, “Pseudocodewords of Tanner graphs,”IEEE Trans. Inf. Theory, vol. 53, no. 11, pp. 4013–4038, Nov. 2007.

[58] G. D. Forney, Jr., R. Koetter, F. Kschischang, and A. Reznik, “Onthe effective weights of pseudocodewords for codes defined on graphswith cycles,” inCodes, Systems, and Graphical Models, B. Marcus andJ. Rosenthal, Eds. New York, USA: Springer Verlag, 2001, vol. 123,pp. 101–112.