بسم الله الرحمن الرحيم

An-Najah National University

Department of Mathematics

Introduction to Linear Block Code

&

Correction Coding Error

prepared by

Hisham Salahat

presented to

Dr. Mohammad Assa`d2010/2011

Thanks and acknowledgment

I would like to thank all of those who contributed and helped me

to achieve this modest study.

Thanks to my parents who provided me with what is needed to

achieve my goals and aims in my life.

Thanks to my teachers who paid their efforts and did their utmost

to make me achieve this work.

Thanks, also and especially to Dr. Mohammad Assa`d who made it

easy and possible for me to accomplish this work successfully, with his

supervision, evaluations and encouragement.

Thanks also to my friends and colleagues who shared me all

moments of sadness and happiness in this stage of my life. I say to you

go ahead, my colleagues. I will never forget your friendship. To all of you

I can`t say good bye, but I am waiting for the moment of meeting with

everybody of you.

I hope that this modest study will serve other students and lead

me to achieve my prospects to become a doctor of math in the nearest

future.

i

preface

Translation of information has become one of the most important

fields included in telecommunication.

To translate information from the sender to the receiver we need to

send and receive the message without being discovered or modulated by

any one. So we use what we called information coding which means to

send codes instead of the message itself. Then it requires that the receiver

can check whether the received message is acceptable or that it is

modulated or be noisy and after that he have to decode it.

Coding theory, the science of coding, basically depends of maths.

There are many kinds to deal with in coding theory but linear block code

stays the most simplest one to deal with, that is because the properties of

Linear Algebra which makes it easy not only to code and decode the

message but also to correct the errors.

I have divided this modest paper into three chapters. The first one

discussed groups , cyclic groups, subgroups, cosets, factor groups, fields,

polynomials over the binary field, construction of Galois field and vector

spaces over finite fields. The second one includes linear block codes,

hamming weight, The generator matrix, the parity-check matrix, and an

application. Finally the third one mentions two types of correction errors

methods minimum distance decoding and syndrome & error detection .

I hope that the others will find this study important and useful .

ii

Contents

Thanks and acknowledgment

……………………………………………………........i

Preface …………………………………………………………………….ii

Chapter 1 Groups and Vector Spaces 1 1.1 Introduction ……………………………………………………..…………....11.2 Groups ………………………………………………………..……………….11.3 Permutation Groups………………………………………..………………...41.4 Cyclic Permutations …………………………………………..………………51.5 Cyclic Groups and the Order of an Element…..……………..…………..… 61.6 Cosets ………………………….…………….……………..…………………..61.7 Fields …………...……………………………………………...………………71.8 Polynomials over the Binary Field …..………………………………………81.9 Construction of Galois Field GF(2n) …………………………………..…….101.10 Vector Spaces Over Finite Fields ……………………………….....………11

Chapter 2 Linear Block Codes 16

2.1 Basic Concepts of Block Code ………………………………………….…….16

2.2 Defintion & Properties of the Linear Block Codes ………………………...172.3 Hamming Weight …………………………………………………...………...182.4 Basis for Linear Codes ……………………………………………….……….192.5 The Generator Matrix Description of Linear Block Codes ………….……..202.6 The Parity-Check MatrixH(n-k),n ……………………….…………….………232.7 AN APPLICATION ………………………………….………….……..…….25

Chapter 3 Error detection, Error correction 30

3.1 Minimum Distance Decoding …………………………..............…………….30

3.2 Syndrome & Error Detection / Correction ……..……………..…………….32

References …………………………………………………………………………………………………….………36

Chapter 1

Groups and Vector Spaces

1.1: Introduction

Linear block codes form a group and a vector space. Hence, the study of the

properties of this class of codes benefits from a formal introduction to these concepts.

Our study of groups leads us to cyclic groups, subgroups, cosets, and factor groups.

These concepts, important in their own right, also build insight in understanding the

construction of extension fields which are essential for some coding algorithms to be

developed.

1.2: Groups

A group formalizes some of the basic rules of arithmetic necessary for

cancellation and solution of some simple algebraic equations.

Definition 1.1: A binary operation * on a set is a rule that assigns to each ordered pair

of elements of the set (a , b) some element of the set. (Since the operation returns an

element in the set, this is actually defined as closed binary operation. We assume that

all binary operations are closed.)

Example 1.1: On the set of positive integers, we can define a binary operation * by

a * b = min(a, b).

Example 1.2: On the set of real numbers, we can define a binary operation * by

a * b = a (i.e., the first argument).

Example 1.3: On the set of real numbers, we can define a binary operation * by

a * b = a + b. That is, the binary operation is regular addition.

Definition 1.2: A group (G, *) is a set G together with a binary operation * on G such

that:

G1 The operator is associative: for any a , b, c ∈ G , (a * b) * c = a * (b * c).

G2 There is an element e ∈ G called the identity element such that a * e = e *

a = a for all a ∈ G.

G3 For every a ∈ G, there is an element b ∈G known as the inverse of a such

that a * b = e . The inverse of a is sometimes denoted as a-1 (when the operator

* is multiplication-like) or as -a (when the operator * is addition-like). Where

the operation is clear from context, the group (G, *) may be denoted simply as

G . The particular notation used is modified to fit the concept. Where the

group operation is addition, the operator + is used and the inverse of an

element a is more commonly represented as -a. When the group operation is

multiplication, either . or juxtaposition is used to indicate the operation and the

inverse is denoted as a -l.

Definition 1.3: If G has a finite number of elements, it is said to be a finite group.

The order of a finite group G, denoted │G│, is the number of elements in G.

This definition of order (of a group) is to be distinguished from the order of an

element given below.

Definition 1.4: A group (G, *) is commutative if a * b = b * a for every a , b ∈ G.

Example 1.4: The set (Z, +), which is the set of integers under addition, forms a

group. The identity element is 0, since 0 + a = a + 0 = a for any a ∈ Z. The inverse of

any a ∈ Z is -a. This is a commutative group.

Now , a group that is commutative with an additive-like operator is said to be an

Abelian group .

Example 1.5: The set (a,.), the set of integers under multiplication, does not form a

group. There is a multiplicative identity, 1, but there is not a multiplicative inverse for

every element in Z.

Example 1.6: The set (Q \ {0}, .) the set of rational numbers excluding 0, is a group

with identity 1. element 1. The inverse of an element a is a-l = l/a.

The requirements on a group are strong enough to introduce the idea of

cancellation. In a group G, if a * b = a * c, then b = c (this is left cancellation). To see

this, let a-l be the inverse of a in G. Then

a-1 * (a * b ) = a-1 * (a * c ) = (a-1 * a ) * c = e * c = c

and a-1* (a * b) = (a-1 * a ) * b = e * b = b, by the properties of associativity and

identity.

Under group requirements, we can also verify that solutions to linear equations

of the form a * x = b are unique. Using the group properties we get immediately that

x = a-1b.

If x1 and x2 are two solutions, such that a * x 1 = b = a * x2, then by cancellation we get

immediately that x 1 = x2.

Example 1.7: Let (Z5, +) denote addition on the numbers {0,1,2,3,4} modulo 5. The

operation is demonstrated in tabular form in the table below:

Clearly 0 is the identity element. Since 0 appears in each row and column,

every element has an inverse.

By the uniqueness of solution, we must have every element appearing in

every row and column, as it does. By the symmetry of the table it is clear that the

operation is Abelian (commutative). Thus we verify that (Z5, +) is an Abelian group.

(Typically, when using a table to represent a group operation a * b, the first

operand a is the row and the second operand b is the column in the table).

In general, we denote the set of numbers 0, 1, . . . ,n - 1 with addition modulo n by

>Zn,+ < or, more briefly, Zn .

Example 1.8: Consider the set of numbers {1,2,3,4,5} using the operation of

multiplication modulo 6. The operation is shown in the following table:

The number 1 acts as an identity, but this does not form a group, since not

every element has a multiplicative inverse. In fact, the only elements that have a

multiplicative inverse are those that are relatively prime to 6.

1.3: Permutation Groups

Definition 1.5: A permutation σ of a nonempty set A={1,2,.....,n} is a one-to-one

mapping of a set A onto itself. This permutation σ can be denoted by :

σ =( 1… .. nσ (1 )…… σ (n)),where σ(i) = j ∈ A ∀i , j = 1,2,…,n.

Note that, the order of the columns in this representation σ is immaterial. For example

The set of all permutation on A={1,2,…,n} is denoted by Ѕn.

The composition operation, denoted by о, on Ѕn is defined by σ2 о σ1, where σ1 is

applied first and then σ2 , for any two permutation σ1 , σ2 ∈ Ѕn.

The set (Ѕn ,о) is a non-commutative group.

Remark 1.1: The number of elements of Ѕn = n !

Example 1.9: Let A= {1,2,3}, then the set of all permutation on A is denoted by Ѕ3

and │Ѕ3│= 3!=6 elements, where

Ѕ3 = { 1 =(1 2 31 2 3) , σ 1=(1 2 3

2 3 1), σ 2=(1 2 33 1 2), μ 1=(1 2 3

1 3 2), μ 2=(1 2 33 2 1)

, μ 3=(1 2 32 1 3)}

Defintion 1.6: The pair (H,*) is said to be a subgroup of given group (G,*) if H is a

nonempty subset of G and is itself a group under the same operation * of G.

Example 1.10 : Let H = {1, σ1 ,σ2 }. The pair (H, о) is a subgroup of Ѕ3 in example

1,9 . according to the following table.

○ 1 σ1 σ2

1 1 σ1 σ2

σ1 σ1 σ2 1

σ2 σ2 1 σ1

1.4: Cyclic Permutations

Definition 1.7 : Let σ ϵ Ѕn . if there exists a list of distinct integers a1 a2 … ak such

that : σ( ai ) = ai+1, i = 1,…, k-1, σ( ak ) = a1 and σ (a )=a if ∉ {a1 a2…ak} then σ is

called a cycle of length k or k-cyclic (k≤n) and denoted by ( a1 … ak) .

A cyclic of length 2 is called a transposition.

Remark 1.2 The transposition (a1 a2) is its own inverse.

Example 1.11: σ = (1 2 3 4 54 3 1 2 5) ∈Ѕ5 is the 4-cycle (1 4 2 3)

Remark 1.3 : The composition of disjoint cycles is commutative; i.e, if σ1 ,σ2 are

disjoint cycles then σ1○ σ2 = σ2○ σ1.

Theorem 1.1 : Every permutation in Ѕn can be expressed uniquely (up to order) as a

product of disjoint cycles.

Example 1.12: In Ѕ6 , the permutation σ = (1 2 3 4 5 65 4 3 2 6 1)can be expressed as a

product (1 5 6 )(2 4 ) or (2 4 )( 1 5 6).

Theorem 1.2: Every cycle of length k is a product of k-1 transpositions.

Example 1.13: σ = (1 4 2 3) = (1 3) ( 1 2) ( 1 4 ).

1.5: Cyclic Groups and the Order of an Element

Definition 1.8: For any a ∈G, the set { an │ n ∈Z } generates a

subgroup of G called the cyclic subgroup. The element a is said to

be the generator of the subgroup. The cyclic subgroup generated by

a is denoted as ¿ a ¿.

Definition 1.9: If every element of a group can be generated by a single element,

the group is said to be cyclic.

Example 1.14: The group (Z5,+ ) is cyclic, since every element in the set can be

generated by a = 2

2, 2 + 2 = 4 , 2 + 2 + 2 = 1 , 2 + 2 + 2 + 2 = 3 , 2 + 2 + 2 + 2

+ 2 = 0 .

Definition 1.10: In a group G , with a ∈G , the smallest n such that an is equal to the

identity in G is said to be the order of a. If no such n exists, a is of infinite order.

1.6: Cosets

Definition 1.11: Let H be a subgroup of (G,*) (where G is not necessarily

commutative) and let a ∈G. The left coset of H , a * H , is the set {a * h│ h ∈H } .

The right coset of H is similarly defined, H * a = { h * a │ h∈H } .

Of course, in a commutative group, the left and right cosets are the

same.

Defintion 1.12 : Let H be a subgroup of a finite group G. the number of distinct left

(right) cosets of H in G denoted by [ G : H ]= │G ││ H │

called the index.

Example 1.15: Consider the subgroup Ѕ3 = { 1 , σ 1, σ 2, μ 1 , μ 2, μ 3} as given in

example 1.9 and consider the subgroup H = {1, μ 1}. Then there are [ G : H ]= │G ││ H │

=

62

= 3 left cosets and 3 right cosets.

The left cosets of H are found as follows :

1 H = ( 2 3 ) H = H = {1, μ 1}

( 1 2 3 ) H = (1 2) H ={(1 2 3), (1 2)} = {σ 1 , μ 3 }

And (1 3 2)H = (1 3)H = {(1 3 2), (1 3)} = {σ 2 , μ2 }

Thus the left cosets of H are { H, {σ 1 , μ 3 },{σ 2 , μ2 }}

The right cosets of H are found in similar way, and they are {{1, μ 1}, {σ 1 , μ 3 },

{σ 2 , μ2 }}.

Theorem 1:3: For a subgroup H of a group G, the following hold ( for left or right

cosets).

i. If a ∈G and b ∈aH , then bH = aH.

ii. aH= bH iff a-1b ∈H .

iii. Any two left (right) cosets of H say aH & bH are either equal or disjoint.

iv. G = the union of all distinct cosets of H in G.

1.7: Fields

Defintion 1.13: Let F be a nonempty set with two binary operation addition ‘+’ and

multiplication ‘・’ are defined then the system (F, + , .) is a field if the following

condition are satisfied:

i. ( F , + ) is a commutative group.

ii. (F – {0}, .) is a commutative group.

iii. Multiplication is distributive over addition; that is , for any three elements a, b

and c in F: a.(b + c) = a .b + a .c, , (b + c). a = b .a + c.a.

The elements of the field are called scalars. The field (F, + , .) will be denoted by F

as long as the operation (+) and (.) are understood from context.

Defintion 1.14 : If F has a finite number of elements , it is said to be a finite field.

The number of the element in a field F is called the order of the field F and is denoted

by │F│.

Remark 1.4: The set ZP = {0,1, … ,p-1} (where p is prime) is a field of order p

under modulo-p addition and multiplication. This field is called prime field.

Example 1.16 : The set Z2 = {0,1}. Is a field of order 2 under modulo-2 addition and

modulo-2 multiplication.

This field is called a binary field.

In this binary field , 1 + 1 = 0, so the addition and subtraction are interchangeable.

Basic Properties of Finite Fields :

In the following let F be a finite field of order p, where p is a prime number

i. For every a ∈F , a .0 = 0 . a = 0

ii. For any two nonzero elements a ,b ∈F , a .b ≠ 0 Hence if r * b = 0 ,then

r = 0 or b = 0.

iii. Let 0 ≠ a ∈ Zp then⇒

a .a … a⏟

p−1׿¿ = ap-1 = 1 .

iv. All finite fields are also called Galois field and denoted by GF.

The prime field Zp will be denoted by GF(P), hence , the binary field Z2 is

denoted by GF(2).

1.8: Polynomials Over the Binary Field

A polynomial f (x) of degree n over GF(2) is a polynomial f(x) with coefficients from

GF(2), as in the following form:

f(x) = f0 + f1 x + f2 x2 + … +fn xn , where, f i = 0 or 1 for 0 ≤ i<n, fn = 1.

Theorem 1.4: Over GF(2) there are 2n polynomial of degree n.

Example 1.17: There are 4 polynomial of degree 2 over GF(2) and they are :

X 2, 1 + X 2 , X + X 2 AND 1 + X + X 2.

Addition, multiplication and division of polynomials over GF(2)

Let f(x) = f0 + f1 x + f2 x2 + … +fn xn and h(x) = h0 + h1 x + h2 x2 + … +hm xm

be a polynomial over GF(2).

Then (f + h) (x) = ∑i=0

max {m, n }

(f i+h i ) xi where {hi=0 i>m

f i=0 i>n

And (f . h) (x) = ∑i=0

m+n

¿¿i

When f(x) is divided by h(x) , we obtain a unique pair of polynomials :

The quotient q(x) ,and the remainder r(x) over GF(2) with degree of r(x) is less than

the degree of h(x).

Example 1.18 : Let f(x) = 1 + x 2 + x3 , h(x) = x + x 2

( f + h)(x) = (1 + 0) + (0 + 1)x + (1 + 1)x2 +(1 + 0)x3 = 1 + x + x3

(f. g) (x) = x + x2 + x3 + x5.

Remark 1.5: If r(x) = 0 we say that f(x) is divisible by h(x) or h(x) is a factor of f(x).

Theorem 1.5: If a is a root of a polynomial f(x) then f(x) is divisible by (x-a).

Fact: If we have a polynomial over GF(2) with an even number of terms, then it is

divisible by (x + 1) because this polynomial has the number 1 as a root.

Example 1.19 : Let f(x) = 1 + x + x2 + x4 , then f(1) = 1 + 1 +1 +1 = 0 ⟹f(x) is

divisible by (x + 1).

Definition 1.15: A nonconstant polynomial f (x) ∈GF(2) is irreducible over GF(2) if

f(x) cannot be expressed as a product g(x).h(x) where both g(x) and h(x) are

polynomials of degree less than the degree of f (x) and g (x), h(x) ∈ GF(2) .

Example 1.20 :Let f(x) = x3 + x + 1 be a polynomial over GF(2)

f(x) does not have neither "0" nor "1" as a root. Therefore f(x) is not divisible by any

polynomial of degree 1 : x nor x + 1. Hence it cannot be divisible by a polynomial of

degree 2 . so f(x) is irreducible over GF(2).

Theorem1.6 : Any irreducible polynomial over GF(2) of degree m divides x2m−1+ 1

Example 1.21: f(x) = x3 + x +1 divides x23-1 +1 = x7 +1. Since

x7 + 1 = (x3 + x +1)(x4 + x2 + x + 1).

Definition 1.16 An irreducible polynomial p(x) ∈ GF(2) of degree m is said to be

primitive if the smallest positive integer n for which p(x) divides xn - 1 is n = 2m - 1

Otherwise p(x) is not a primitive.

Theorem 1.7: Let f(x) be a polynomial over GF(2) then for any i≥ 0 we have the

following: [ f (x )]2i

= f(x2i

).

1.9: Construction of Galois Field GF(2n)

We begin with the two elements 0 and 1 from GF(2) and a new symbol α . Then we

define a multiplication ( . ) to introduce a sequence of powers of α as follows :

α 2 = α .α , α 3 = α .α .α , … ,α j = α .α ……. α .⏟

j−¿

Now, we have the following set of elements:{0 ,1 , α ,α2,α 3,…, α j,…}.

Now suppose p(x) is a primitive polynomial of degree n over GF(2) such that

p(α) = 0. Then p(x) divides x2n-1 + 1, i.e, x2n-1 + 1 = q(x)p(x)

replace x by α , we obtain:

α 2n-1 + 1 = q(α ¿ p(α ¿ = q(α ¿ . 0 = 0

This implies α 2n-1 + 1 = 0 . adding 1 to both sides : α 2n-1 = 1 , and hence α 2n

= α

Therefore the set above becomes finite and consist of the 2n elements :

GF(2n) = {0 ,1 , α ,α2,α 3,…, α 2n-2}

Remark 1.6 :

i. In the construction of Galois field GF(2n), we use a primitive polynomial p(x)

of degree n and require that the element α be a root of p(x) . since the powers

of α generate all the nonzero elements of GF(2n) , α is a primitive element.

ii. The elements of GF(2n) have three representation shown in the table given in

the following example.

Example 1.22: Let n = 3 ,the polynomial p(x) = 1 + x + x3 is a primitive polynomial

over GF(2). Set p(α) = 1 +α + α 3 = 0. Then α 3 = 1 + α . using this , we construct

GF(23) = {0 ,1, α ,α 2, … , α 23-2} = {0 ,1, α 2, α 3, … , α 6}.

The element α 3 = 1 + α is used repeatedly to form the polynomial representation for

the elements of GF(23 ): α 4 = α . α 3 = α (1 +α ) = α + α 2

α 5 = α .α 4 = α( α + α 2) = α 2 + α 3 = α 2 +1 +α = 1 +α + α 2

α 6 = α . α 5 ¿α(1 +α + α 2 ) = α + α 2 + α 3 = α + α 2 + 1 +α = 1 + α 2

Table: Three representation for the elements of GF(23) generated by p(x) = 1 + x + x3

Power representation Polynomial representation

in α

3-tuples representation

0 0 ( 0 0 0 )

1 1 ( 1 0 0 )

α α (0 1 0) α 2 α 2 ( 0 0 1 )

α 3 1 + α ( 1 1 0 )

α 4 α + α 2 ( 0 1 1 )

α 5 1 +α + α 2 ( 1 1 1 )

α 6 1 + α 2 ( 1 0 1 )

1.10: Vector Spaces over Finite Fields

Definition 1.17: Let V be a set of elements called vectors and let F be a field of

elements called scalars. An addition operation + is defined between vectors. A scalar

multiplication operation . (or juxtaposition) is defined such that for a scalar a ∈ F and

a vector v ∈ V, a . v ∈ V. Then V is a vector space over F if + and . satisfy the

following:

V1 V forms a commutative group under addition .

V2 For any element a ∈F and v ∈ V, a . v ∈ V

V3 The operations + and . are distribute:

(a + b) . v = a .v + b .v and a . (u + v) = a . u + a .v

for all scalars a, b ∈ F and vectors v, u ∈ V.

V4 The operation . is associative: (a . b) . v = a . (b . v) for all a,

b ∈ F and v∈V

F is called the scalar field of the vector space V.

Remark 1.7 : Let K be an extinction field of afield k , then K can be considered as a

vector space over k.

Since the set GF(2n) is an extension field of GF(2), then GF(2n) can be considered as

a vector space over GF(2). Let Vn denote that set of all 2n distinct n-tuples ( f0 f1 … fn-

1) over GF(2). Then (Vn , + , .) is a vector space with + is vector addition and . is

scalar multiplication.

Example 1.23: Let n =3 . the vector space V3 of all 3-tuples over GF(2) consists of

the following 8 vectors: ( 0 0 0 ), ( 0 0 1 ), ( 0 1 0 ), ( 0 1 1 ) ,( 1 0 0 ), (1 0 1 ),

( 1 1 0 ), ( 1 1 1 ).

Defintion 1.18: Let Ѕ be a nonempty subset of a vector space V over a field F then S

is a subspace of V, if S is its self a vector space over F .

Theorem 1.8 : Let S be a nonempty set of a vector space V over a field F.

Then S is a subspace of V iff the following condition is satisfied:

if u , v ∈S and γ , δ ∈ F , then γu+δv∈ S.

Note that, a necessary and sufficient condition for a nonempty subset of S of a vector

space V over GF(2) to be a subspace is : if x, y ∈ S, then x + y ∈ S.

Example 1.24: The set of the vectors ( 0 0 0 ), ( 0 0 1 ), ( 1 1 0 ), ( 1 1 1 ), satisfies the

condition of theorem1.8 as a subspace of the set V3 given in example 1.23 .

Defintion 1.19: In a vector space V, a sum of the form u = a1v1 + a2v2 +· · ·+akvk ,

where the ai are scalars, is called a linear combination of the vectors v1, . . . , vk .

A set of vectors {v1, . . . , vk} is called linearly dependent if there is a set of

scalars {a1, . . . , ak}, not all zero, such that a1v1 + a2v2 +· · ·+akvk = 0.

A set of vectors that is not linearly dependent is called linearly independent.

No vector in a linearly independent set can be expressed as a linear combination of

the other vectors. Note that the zero vector 0 cannot belong to a linearly independent

set; every set containing 0 is linearly dependent.

Defintion 1.20: A set of vectors is said to span a vector space if every vector in the

space equals at least one linear combination of the vectors in the set. A vector space

that is spanned by a finite set of vectors is called a finite-dimensional vector space.

We are interested primarily in finite-dimensional vector spaces.

Defintion 1.22: The number of linearly independent vectors in a set that spans a

finite-dimensional vector space V is called the dimension of V. A set of k linearly

independent vectors that spans a k-dimensional vector space is called a basis of the

space.

Note : Every set of more than k vectors in a k-dimensional vector space is linearly

dependent.

Theorem 1.9: In any vector space V, the set of all linear combinations of a nonempty

set of vectors {v1, . . . , vk} is a subspace of V.

Theorem 1.10. If W ,a vector subspace of a finite-dimensional vector space V, has

the same dimension as V ,then W = V .

Defintion 1.23: The set B ={v1, . . . , vk} of vectors in a vector space V over a field F

is said to form a basis for V if:

B spans V

B is linearly independent

Remark 1.8

i. If v1, v2 …. , vk form a basis for a vector space V, then they must be nonzero

distinct vectors.

ii. A vector space V over a finite field F can have many basis; but all basis

contain the same number of elements called dim(v)

Theorem 1.11: If S ={v1, . . . , vk} form a basis for a vector space V , then every

vector in V can be written in one and only one way as a linear combination of the

vectors in S.

Example 1.25: Consider the vector space V3 of all 3-tuples over GF(2). Let us form

the following 3-tuples : e0 = (1 0 0 ) , e1 = (0 1 0) , e2 = ( 0 0 1) . then every 3-tuple (

a0 a1 a2 ) in V3 can be expressed as a linear combination of e0, e1 ,e2 as follows:

( a0 a1 a2 ) = a0 . e0 + a1 . e1 + a2 . e2. Therefore , e0, e1 ,e2 span the vector space V3 .

We also see that e0, e1 ,e2 are linearly independent . hence , they form a basis ( called

the standard basis) for V3 with dimension 3.

Theorem 1.12: Let V be a vector space over GF(2) and dim(V) = k , then V has 2k

element.

Corollary: Let V be an n-dimensional vector space, and let B ={v1, . . . , vn} be a set

of n-vectors in V then:

i. If B is linearly independent, then it is a basis for V.

ii. If B spans V , then it is abasis for V .

Defintion 1.24: Let u = ( u0 u1 … un-1) and v = ( v0 v1 … vn-1) be two n-tuples in Vn

over GF(2) then:

i. We define the Euclidean inner product ( also called scalar product or dot

product) of u and v as u .v = ∑i=0

n−1

u i . v i

ii. The two vectors u and v are said to be orthogonal if u .v = 0.

iii. Let C be a nonempty subset of Vn . the orthogonal complement of C is

defined to be : = {v ∈ Vn : v.C = 0 ∀ c ∈ C}

Example 1.26 : Let u = ( 1 1 1 1 ) , v = ( 1 1 1 0 ) , w = ( 1 0 0 1 ) be a vector in V4

over GF(2). Then : u .v = 1.1 + 1.1 + 1.1 +1.0 = 1

u .w = 1.1 + 1.0 + 1.0 + 1.1 = 0

v .w = 1.1 + 1.0 + 1.0 + 0.1 = 1. Hence. u and w are orthogonal .

Example 1. 27: Let C = { ( 0 1 0 0 ) , ( 0 1 0 1 ) } ∈ V4 over GF(2) . To find . let

v = ( v0 v1 v2 v3 ) ∈ then:

v . ( 0 1 0 0 ) = 0 ⟹ v1 = 0 and v .( 0 1 0 1 ) = 0 ⟹ v1 + v3 = 0

hence . we have v1 = v3 = 0 . since v0 and v1 can be either 0 or 1 , we can conclude

that = { ( 0 0 0 0 ).( 0 0 1 0 ) , ( 1 0 0 0 ), ( 1 0 1 0 )}.

Theorem 1 .13: Let C be a subspace of Vn then :

i. is a subspace of Vn

ii. C ∩ = { 0 }

iii. C ∪ = Vn.

Theorem 1.14: Let C be a k-dimensional subspace of Vn. Then we have dim(C) +

dim ( ) = n .

Remark 1.9 : If is an orthogonal complement of C , then C is also an orthogonal

complement of .

Remark 1.10 : If A is a given k×n matrix, we associate the following four

fundamental vector spaces with A : the null space of A , the row space of A , the null

space of AT and the column space of A .

Theorem 1.15 : If A is a given k×n matrix , then:

i. The null space of A is the orthogonal complement of the row space of A, that

is , dim(row space) + dim(null space) = n. and any vector in the row space of

A is orthogonal to the vectors of the null space of A.

ii. The null space of AT is the orthogonal complement of the column space of A .

that is ; dim( column space ) + dim ( null space of AT ) = k . and any vector in

the column space of A is orthogonal to the vectors of the null space o AT .

Chapter 2

Linear Block Codes

2.1: Basic Concepts of Block Code

Consider a source that produces symbols from an alphabet A having q

symbols, where A forms a field. We refer to a tuple (cO cl , . . . , c n-l) An with n

elements as an n-vector or an n-tuple.

Codes are introduced in two important types: instantaneous codes ,which are

codes of variable word lengths decodable symbol per symbol , and block codes,

which are the special case of instantaneous codes with constant word length.

Definition 2.1: An (n, k) block code C over an alphabet of q symbols is a set of qk n-

vectors called codewords or code vectors. Associated with the code is an encoder

which maps a message, a k-tuple m ∈ Ak, to its associated codeword.

For a block code to be useful for error correction purposes, there should be a

one-to-one correspondence between a message m and its codeword c. However, for a

given code C, there may be more than one possible way of mapping messages to

codewords.

NOTE:A binary block code is said to be linear provided that the sum of arbitrary two

code word is a code word.

Defintion 2.2: Given the binary field GF(2) = {0,1}, we define:

i. A binary word w of length n over GF(2) is an n –tuple w =(w0 w1 … wn-1) of

binary digits wi ∈GF(2) ∀ i = 0,…., n-1.

ii. A binary block code of length n is a set C(≠∅ ¿of binary word w each of

length n .

iii. The size of C, denoted by │C │ ,is the number of codewords in C.

Example 2.1: let C ={00,01,10,11}. C is a binary block code of length n = 2 and

size 4.

C = {000, 001, 010, 011}., C is a binary block code of length n =3 and size 4.

C = {000, 001, 010, 011, 100, 101, 110, 111}. C is a binary block code of length n = 3

and size 8.

A set of 2k distinct codewords w of length n each, over the binary field GF(2) = {0,1}

, is called a Binary Block Code (BBC) C(n,k)

2.2: Defintion & Properties of the Linear Block Codes

Defintion 2.3: A BBC C(n,k) of length n and 2k codewords is called linear if its 2k

codeword form a k-dimensional subspace of the vector space V n of all n-tuples over

the field GF(2). A linear combination of codewords in C is also a codeword in C.

Basic Properties of a Linear Block Code C(n,k)

i. The zero word (00 …0), is always a codeword.

ii. If c is a codeword ,then – c is acodeword.

iii. A linear code is invariant under translation by a codeword. That is, if v is a

codeword in linear code C, then C + v = C.

iv. The dimension k of the linear code C(n,k) is the dimension of C as a

subspace of Vn over GF(2), i.e , dim(C) = k

Example 2.2: let C={(λ λ … λ¿:λ∈GF(2)}. Then C is a linear block code often called

the repetition code.

Example 2.3: let C ={ (1 1 1 0 0 ), ( 0 0 1 1 0 ), ( 1 1 1 1 1 ), (1 1 0 10 ), ( 0 0 0 1 1) ,

(1 1 0 0 1 ) ,(0 0 0 0 0 ) , (0 0 1 0 1 )}. Since any linear combination of the codeword

in C is also a codeword in C , then C is a (5,3) linear block code. For instance,

( 0 0 0 1 1 ) + (1 1 0 0 1) = (1 1 0 1 0) ∈C.

( 0 0 0 1 1 ) + ( 1 1 0 1 0) = (1 1 0 0 1 ) ∈C.

(1 1 0 0 1) + (1 1 0 1 0) = (0 0 0 1 1 ) ∈C.

(0 0 0 1 1) + (1 1 0 0 1 ) + ( 1 1 0 1 0 ) = ( 0 0 0 0 0) ∈C.

2.3: Hamming Weight

Definition 2.4: Let x be a word in Fn . The (Hamming) weight of x, denoted by

wt(x), is defined to be the number of nonzero coordinates in x; i.e.,

wt(x) = d(x, 0), where 0 is the zero word.

Remark 2.1: For every element x of Fn , we can define the Hamming weight as

follows:

wt(x) = d(x,0) = {1 if x≠ 00 if x=0

.

writing x ∈ Fn as x = (x1, x2, . . . , xn), the Hamming weight of x can also be

equivalently defined as wt(x) = wt(x1) + wt(x2) + ・・ ・+ wt(xn).

Lemma: If x, y ∈ Fn , then d(x, y) = wt(x − y)

Definition 2.5: Let C be a code (not necessarily linear). The minimum (Hamming)

weight of C, denoted wt(C), is the smallest of the weights of the nonzero codewords

of C.

Example 2.4: Consider the binary linear code C = {0000, 1000, 0100, 1100}. We see

that

wt( 1 0 0 0 ) = 1,

wt( 0 1 0 0 ) = 1,

wt( 1 1 0 0 ) = 2.

Hence, d(C) = 1.

Remark 2.2: (Some advantages of linear code) The following are some of the

reasons why it may be preferable to use linear codes over nonlinear ones:

(i) As a linear code is a vector space, it can be described completely by using

a basis .

(ii) The distance of a linear code is equal to the smallest weight of its nonzero

codewords.

(iii) The encoding and decoding procedures for a linear code are faster and

simpler than those for arbitrary nonlinear codes

2.4: Basis for Linear Codes

Definition 2.6: Let A be a matrix over Fq; an elementary row operation performed

on A is any one of the following three operations:

(i) interchanging two rows,

(ii) multiplying a row by a nonzero scalar,

(iii) replacing a row by its sum with the scalar multiple of another row.

Definition 2.7: Two matrices are row equivalent if one can be obtained from the other

by a sequence of elementary row operations.

The following are well known facts from linear algebra:

(i) Any matrix M over Fq can be put in row echelon form (REF) or reduced row

echelon form (RREF ) by a sequence of elementary row operations. In other words, a

matrix is row equivalent to a matrix in REF or in RREF.

(ii) For a given matrix, its RREF is unique, but it may have different REFs . (Recall

that the difference between the RREF and the REF is that the leading nonzero entry of

a row in the RREF is equal to 1 and it is the only nonzero entry in its column.)

Algorithm for finding a basis:

Input: A nonempty subset S of Fn .

Output: A basis for C = <S>, the linear code generated by S.

Description: Form the matrix A whose column are the words in S. Use elementary

row operations to find an REF of A. Then the colums of A corresponding to the

column of A in REF which have leading 1`s form a basis for S.

Example 2.5 : Find a basis for the linear block code C (5,3) given in example 2.3.

To find a basis for C , we find a set of linearly independent columns of the 5 ×7

Matrix A whose columns are the nonzero codewords of C :

A = [1 0 1 1 0 1 01 0 1 1 0 1 01 1 1 0 0 0 10 1 1 1 1 1 00 0 1 0 1 1 1

] → REF [1 0 1 1 0 1 00 1 0 1 0 1 10 0 1 0 1 1 10 0 0 0 0 0 00 0 0 0 0 0 0

]Since the leading 1,s in the REF are in columns 1,2 & 3 ,then the corresponding

colums of A form a basis ß for C. thus ß =[[11100] , [

00110] ,[

11111] ] is a basis of code C.

2.5: The Generator Matrix Description of Linear Block Codes

Since a linear block code C is ak-dimensional vector space, there exist k linearly

independent vectors which we designate as go, gl, . . . , gk-1 such that every codeword c

in C can be represented as a linear combination of these vectors,

c=mo go +ml gl +...+ m k-l gk-1, (3.1)

where mi∈Fq. (For binary codes, all arithmetic in (3.1) is done modulo 2; for codes of

Fq, the arithmetic is done in Fq ) Thinking of the gi as row vectors' and stacking up,

we form the k x n matrix G.

G = [ g0

g1

⋮gk−1

]

Let

M = [m0 m1 ⋯ mk−1 ]Then (3.1) can be written as

c= m G (3.2).

and every codeword c ∈ C has such a representation for some vector m. Since the

rows of G generate (or span) the (n, k) linear code C, G is called a generator matrix

for C.

Equation (3.2) can be thought of as an encoding operation for the code C.

Representing the code thus requires storing only k vectors of length n (rather than the

qk vectors that would be required to store all codewords of a nonlinear code).

Note that the representation of the code provided by G is not unique. From a given

generator G, another generator G' can be obtained by performing row operations

(nonzero linear combinations of the rows). Then an encoding operation defined by

c = mG' maps the message m to a codeword in C, but it is not necessarily the same

codeword that would be obtained using the generator G.

Example 2.6: The generator matrix for the linear code C(5,3) in example 2.4 is

G = [1 1 1 0 00 0 1 1 01 1 1 1 1 ]

Note ß = {(1 1 1 0 0 ), ( 0 0 1 1 0), (1 1 1 1 1 )} is abasis of C(5,3).

Encoding scheme:

If u = (u0 u1 …uk-1) is the message to be encoded, then the corresponding codeword v

can be given as follows:

v = u.G = (u0 u1 …uk-1) . [ g0

g1

⋮gk−1

] =u0g0 + u1g1 +… + uk-1gk-1..

i.e ∑i=0

k −1

uigiis a codeword of c with coefficients u0, u1 ,… , uk-1.

Remark 2.3 : For each k-tuple (message) u = (u0 ,u1 ,…, uk-1) there corresponds one

and only one codeword v = (v0 ,v1 …,vn-1) . so there are 2k distinct message and

corresponding 2k distinct codeword .

Example 2.7: Let M = { ( 0 0 0 ), ( 0 1 1 ), ( 1 1 0 ), ( 0 1 0 ), ( 0 0 1 ), ( 1 0 0 ),

( 1 0 1), (1 1 1 )}. Be the message set . use the generator matrix

G = [1 1 1 0 00 0 1 1 01 1 1 1 1 ] of linear code C(5,3) given in example 2.3 to obtain the

encoded codewords v of C.

u0 = (0 0 0)u 0. G→

v0 = (0 0 0 0 0) u4 = (0 0 1)u 4. G→

v1 = (1 1 1 1 1)

u1 = (0 1 1)u 1.G→

v1 = (1 1 0 0 1) u5 = (1 0 0 )u 1.G→

v5 = (1 1 1 0 0)

u2 = (1 1 0)u 2. G→

v2 = (1 1 0 1 0) u6 = (1 0 1)u 6. G→

v6 = (0 0 0 1 1)

u3 = (0 1 0)u 3.G→

v1 = (0 0 1 1 0) u7 = (1 1 1)u 7.G→

v7= (0 0 1 0 1)

Definition 2.8: Let C be an ( n , k) block code (not necessarily linear). An encoder is

systematic if the message symbols mo, ml, . . . , mk-1 may be found explicitly and

unchanged in the codeword. That is, there are coordinates io , i 1 , . . . , ik-1

such that ci0 = mo, ci1 = m 1 , . . . , cik-l = mk-1.

For a linear code, the generator for a systematic encoder is called a systematic

generator .

Frequently, a systematic generator is written in the form

G = [p Ik] = [p0,0 p0,1 ⋯ p0 ,n−k−1 1 0 0 ⋯ 0p1,0 p1,1 ⋯ p1 ,n−k−1 0 1 0 ⋯ 0p2,0 p2,1 ⋯ p2 ,n−k−1 0 0 1 ⋯ 0⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋮ ⋱ ⋮

pk−1,0 pk−1,1 ⋯ pk −1 , n−k−1 0 0 0 ⋮ 1]

where I is the k x k identity matrix and P is a k x (n - k) matrix which

generates parity symbols. The encoding operation is

c = m [ P I k ] = [mP m].

The codeword is divided into two parts: the part m consists of the message

symbols, and the part mP consists of the parity check symbols.

Performing elementary row operations (replacing a row with linear

combinations of some rows) does not change the row span, so that the same code is

produced. If two columns of a generator are interchanged, then the corresponding

positions of the code are changed, but the distance structure of the code is preserved.

Example 2.8: Consider the C(5,3) given in example 2.3 with the generator matrix

G = [1 1 1 0 00 0 1 1 01 1 1 1 1 ]

Using elementary row operation and/or column then the generator matrix G can be

written as follows:

G = [1 1 1 0 00 0 1 1 01 1 1 1 1 ] ¿¿ G` = [1 1 1 0 0

0 1 0 1 00 1 0 0 1 ]

Therefore , a linear systematic code C(5,3) is generated now by G`.

To show this , let u = ( u0 u1 u2) be a message to be encoded . the corresponding

codeword is v = ( v0 v1 v2 v3 v4) = ( u0 u1 u2) . [1 1 1 0 00 1 0 1 00 1 0 0 1 ] given by the following

equations:

v0 = u0, v1 = u0 + u1 + u2, v2 = u0 , v3= u1 , v4 = u2

thus, the corresponding codewords for the message set {( 0 0 0 ), ( 0 1 1 ), ( 1 1 0 ),

( 0 1 0 ), ( 0 0 1 ), ( 1 0 0 ), ( 1 0 1 ), (1 1 1 )} are shown in the following table

Table: linear systematic block code with k = 3 and n = 5

Message u Codeword v

( 0 0 0 ) ( 0 0 0 00⏟

u)

( 0 1 1 ) ( 0 0 0 1 1⏟

u)

( 1 1 0 ) ( 1 0 1 10u)

( 0 1 0 ) ( 0 1 0 1 0⏟

u)

( 0 0 1 ) ( 0 1 0 01⏟

u)

( 1 0 0 ) ( 1 1 1 00⏟

u)

( 1 0 1) ( 1 0 1 0 1⏟

u )

(1 1 1 ) ( 1 11 11⏟

u)

2.6: The Parity-Check Matrix H(n-k),n

Another matrix associated with every linear block code is the parity-check

matrix H. the null space of G is orthogonal to the row space of G. so we construct an

( n-k ) × n matrix H whose rows form a basis of the null space of G in this case

G. HT= 0 . an n-tuple v is called a codeword in the code generated by G If and only if

H.vT = 0 . This matrix H is called a parity-check matrix of the code C.

The 2 n-k linear combination of the rows of the matrix H form the dual code

( n, n-k) of C which is defined as follows;

Defintion 2.9: Let C(n,k) be a linear code in Vn. The dual code of C is the

orthogonal complement of the subspace C of Vn. is linear with dim(c) + dim (

¿=n .

Remark 2.4: The dual code is spanned by the null space of the generator matrix G

of C.

Example 2.9: Consider the linear block code C (5,3 ) given in example 2.3 with the

generator matrix G =[1 1 1 0 00 0 1 1 01 1 1 1 1 ] . to find the parity-check matrix H of C , we

find a basis of which forms the rows of H using the following algorithm:

algorithm:

input: A nonempty subset B of Vn.

Output: A basis for the dual code of C

DESCRIPTION: start with the generator matrix G . Writing G in RREF then matrix G

contains k leading columns. Permute the columns of G to form G`= G.P = (X│IK)

where IK denotes the k×k identity matrix .

Form a matrix H` as follows: H`= ( I n-k │- XT) where XT denotes the transpose of X .

apply the inverse of the permutation applied to the columns of G to colums of H` to

form H = H`.PT. then the rows of H form a basis for .

We now apply the above algorithm on the generator matrix given in the example:

G =[1 1 1 0 00 0 1 1 01 1 1 1 1 ] RREF

⇒ [1 1 0 0 1

0 0 1 0 10 0 0 1 1] (X 3,2 │ I 3)

⇒ G` = [1 1 0 0 1

0 0 1 0 10 0 0 1 1]. P

= [1 1 0 0 10 0 1 0 10 0 0 1 1] [

0 0 1 0 01 0 0 0 00 0 0 1 00 0 0 0 10 1 0 0 0

] = (1 1 1 0 00 1 0 1 00 1 0 0 1) (I 2│ XT )

⇒

H` = (1 0 1 0 00 1 1 1 1) → H = H`.PT = (1 0 1 0 0

0 1 1 1 1) [0 1 0 0 00 0 0 0 11 0 0 0 00 0 1 0 00 0 0 1 0

] =

[1 1 0 0 01 0 1 1 1 ]

Therefore , = { ( 1 1 0 0 0 ), ( 1 0 1 1 1 ), ( 0 0 0 0 0 ), ( 0 1 1 1 1)}

Theorem 2.1 : For an (n,k) linear systematic block code C with generator matrix G`

and parity check matrix H` we have G`.H`T = 0

2.7: AN APPLICATION :

Consider the linear block code C(8,5) = { ( 1 1 1 1 1 1 1 1 ), ( 1 1 1 1 1 1 1 0 ), ( 1 0

1 1 1 1 0 1 ), ( 1 0 1 1 1 1 0 0 ), ( 1 1 0 1 1 0 1 1 ), (1 1 0 1 1 0 1 0 ) ,(1 0 0 1 1 0 0 1 ),

(1 0 0 1 1 0 0 0 ), ( 1 1 1 1 0 1 1 1 ), ( 1 1 1 1 0 1 1 0 ), (1 0 1 1 0 1 0 1 ), ( 1 0 1 1 0 1

0 0 ), ( 1 1 0 1 0 0 1 1 ) , ( 1 1 0 1 0 0 1 0 ), ( 1 0 0 1 0 0 0 1 ), ( 1 0 0 1 0 0 0 0 ), ( 0 0

0 0 0 0 0 0 ), ( 0 0 0 0 0 0 0 1), ( 0 1 0 0 0 0 1 0 ) , ( 0 1 0 0 0 0 1 1 ), ( 0 0 1 0 0 1 0 0),

( 0 0 1 0 0 1 0 1) , ( 0 1 1 0 0 1 1 0 ), ( 0 1 1 0 0 1 1 1 ), ( 0 0 0 0 1 0 0 0), ( 0 0 0 0 1 0

0 1 ), ( 0 1 0 0 1 0 1 0 ), ( 0 1 0 0 1 0 1 1 ), ( 0 0 1 0 1 1 0 0), ( 0 0 1 0 1 1 0 1 ), ( 0 1 1

0 1 1 1 0 ), ( 0 1 1 0 1 1 1 1 )}.

The basis for this space is β = {( 1 1 1 1 1 1 1 1 ), ( 1 1 1 1 1 1 1 0 ), ( 1 0 1 1 1 1 0 1)

)1 1 0 1 1 0 1 1 ) ,( 1 1 1 1 0 1 1 1.{(

Forming the generator matrix G` we get

RREF⇒

G = `[1 1 1 1 1 1 1 11 1 1 1 1 1 1 01 0 1 1 1 1 0 11 1 0 1 1 0 1 11 1 1 1 0 1 1 1

] G = ``[

1 0 0 1 0 0 0 00 1 0 0 0 0 1 00 0 1 0 0 1 0 00 0 0 0 1 0 0 00 0 0 0 0 0 0 1

]

P= [0 0 0 1 0 0 0 00 0 0 0 1 0 0 00 0 0 0 0 1 0 01 0 0 0 0 0 0 00 0 0 0 0 0 1 00 0 1 0 0 0 0 00 1 0 0 0 0 0 00 0 0 0 0 0 0 1

] let

Now G = G``*P = [1 0 0 1 0 0 0 00 1 0 0 1 0 0 00 0 1 0 0 1 0 00 0 0 0 0 0 1 00 0 0 0 0 0 0 1

]G is written in a systematic form (X 5,3 │ I 5)

H` = (1 0 0 1 0 0 0 00 1 0 0 1 0 0 00 0 1 0 0 1 0 0)

H = H`. PT = (1 0 0 1 0 0 0 00 1 0 0 0 0 1 00 0 1 0 0 1 0 0)

Let M= {(1 1 1 1 1 ), (1 1 1 1 0 ), (1 1 1 0 1 ), (1 1 1 0 0 ), (1 1 0 1 1 ), (1 1 0 1 0 ) ,

( 1 1 0 0 1 ), (1 1 0 0 0 ), (1 0 1 1 1 ), (1 0 1 1 0 ), (1 0 1 0 1 ), (1 0 1 0 0 ), (1 0 0 1

1 ) , (1 0 0 1 0), (1 0 0 0 1 ), (1 0 0 0 0 ), (0 0 0 0 0 ), (0 0 0 0 1), (0 0 0 1 0 ) , (0 0 0 1

1 ), (0 0 1 0 0), (0 0 1 0 1) , (0 0 1 1 0 ), (0 0 1 1 1 ), (0 1 0 0 0), (0 1 0 0 1 ), (0 1 0 1 0

), (0 1 0 1 1 ), (0 11 0 0), (0 1 1 0 1 ), (0 1 1 1 0 ), (0 1 1 1 1 )}.

If we take m = ( m0 m1 m2 m3 m4) then m.G = ( c0 c1 c2 c3 c4 c5 c6 c7)

Such that c0 = m0 , c1 = m1 , c2 = m2, c3 = m0, c4 = m1, c5 = m2 , c6 = m3 , c7 = m4

Consider the message m= (1 0 0 1 0 1) to be sent then

m.G = c = ( 1 0 0 1 0 0 1 0 1 ) where c is the codeword.

Remark: In any codeword the first three components form the parity check

symbols.

For the codeword to be correct it must satisfy the following condition :

c0 = c3, c1 = c4, c2 = c5.

In the following table we will give every message a message code as it determined.

message Message code message Message code

A ( 0 0 0 0 0 ) Q ( 0 0 0 0 1 )

B ( 0 0 0 1 1 ) R ( 0 0 0 1 0 )

C ( 0 0 1 0 0 ) S ( 0 0 1 0 1 )

D ( 0 0 1 1 1 ) T ( 0 0 1 1 0 )

E ( 0 1 0 0 0 ) U ( 0 1 0 0 1 )

F ( 0 1 0 1 1 ) V ( 0 1 0 1 0 )

G ( 0 1 1 0 0 ) W ( 0 1 1 0 1 )

H ( 0 1 1 1 1 ) X ( 0 1 1 1 0 )

I ( 1 0 0 0 0 ) Y ( 1 0 0 0 1 )

J ( 1 0 0 1 1 ) Z ( 1 0 0 1 0 )

K ( 1 0 1 0 0 ) Space ( 1 0 1 0 1 )

L ( 1 0 1 1 1 ) Enter ( 1 0 1 1 0 )

M ( 1 1 0 0 0 ) Comma "," ( 1 1 0 0 1 )

N ( 1 1 0 1 1 ) Full stop "." ( 1 1 0 1 0 )

O ( 1 1 1 0 0 ) Under score ''-" ( 1 1 1 0 1 )

P ( 1 1 1 1 1 ) @ ( 1 1 1 1 0 )

Every message has its own code. If we want to code any message we just

multiply it by the generator matrix G. for example if we want to send K as a message

then the code word = ( 1 0 1 0 0 )* G = ( 1 0 1 1 0 1 0 0 ) which is the received code

word.

In many cases the message is formed by many letters and marks. To deal with

this we will take every letter and assume it to be a message, since there are many

letters we will form a new matrix called the message matrix consisting of 5 columns

and a number of rows equal the number of the messages (assume n) denoted by M` .

then we multiply it by the generator matrix G. To get a code word matrix consist of 8

columns and n rows. In which every row is a code word to the corresponding message

in M`.

Example: If I want to tell you that " I love math." then I will form M` as it mentioned

before and it will be as follows:

M` = [1 0 0 0 01 0 1 0 11 0 1 1 11 1 1 0 00 1 0 1 00 1 0 0 01 0 1 0 11 1 0 0 00 0 0 0 00 0 1 1 00 1 1 1 11 1 0 1 0

]

Now we find the matrix C = M`.G

C = [1 0 0 0 01 0 1 0 11 0 1 1 11 1 1 0 00 1 0 1 00 1 0 0 01 0 1 0 11 1 0 0 00 0 0 0 00 0 1 1 00 1 1 1 11 1 0 1 0

] .G = [1 0 0 1 0 0 0 01 0 1 1 0 1 0 11 0 1 1 0 1 1 11 1 1 1 1 1 0 00 1 0 0 1 0 1 00 1 0 0 1 0 0 01 0 1 1 0 1 0 11 1 0 1 1 0 0 00 0 0 0 0 0 0 00 0 1 0 0 1 1 00 1 1 0 1 1 1 11 1 0 1 1 0 1 0

]If you received the matrix C first you have to check that the code words are

correct using the remark determine previously . OR using theorem 2.1. for example

the third code word is equal to ( 1 0 1 1 0 1 1 1 ) ⟹

c0 = c3 = 1 , c1 = c4 = 0 , c2 = c5 = 1 .

it satisfies the conditions i.e it is correct.

Now to decode the received matrix you will delete the parity check symbols

and then the remains part forms the sent message. Just you need to determine what it

codes to from the above table.

For example if you want to decode the fifth code word which is

( 0 1 0 0 1 0 1 0 ) first you delete the parity check symbols consisting of the first

three components then the remaining part is ( 0 1 0 1 0 ) which is the sent message.

Back to the table you will find that it is the code of " v ".

i.e v is the message which was sent.

Chapter 3

Error detection, Error correction

In a communication channel a codeword v = (v0 ,v1 …,vn-1) is transmitted and

suppose r = (r0 ,r1 …,rn-1) is received at the output of the channel. If r is a valid

codeword, we may conclude that there is no error in v . otherwise , we know that

some errors have occurred and we need to find the correct codeword that was sent by

using any of the following methods of linear codes decoding:

i. Minimum distance decoding

ii. Syndrome decoding

These methods for finding the most likely codeword sent are known as decoding

methods.

3.1: Minimum distance decoding

In this section an important parameters of linear block codes called the

hamming distance and hamming weight are introduced as well as the minimum

distance decoding:

Defintion 3.1: Let x = ( x0 x1 …xn-1 ) and y = ( y0 y1 …yn-1 ) be two binary words. The

hamming distance or simply (distance) from x to y denote by d( x ,y ) is defined to be

the number of positions that the corresponding elements differ:

d( x ,y ) = ∑i=0

n−1

d (x i , y i)

where: d(x i ,y i ) = {1 if x i≠ y i

0 if x i= y i (3.1)

Example 3.1: Let x = ( 0 0 1 1 1 ) and y = ( 1 1 0 0 1 ) be two codewords in the linear

block C(5 , 2) over GF(2). Then the hamming distance from x to y is

d( x ,y ) = ∑i=0

4

d (x i , y i)= 1 + 1 + 1 + 1 + 0 = 4

Theorem 3.1: Let x , y and z be words of length n over GF(2) then we have:

i. 0 ≤d( x ,y ) ≤ n

ii. d( x ,y ) = 0 ⟺ x = y

iii. d( x ,y ) = d( y ,x)

iv. d( x ,z ) ≤ d( x ,y ) + d( y, z )

Defintion 3.2 Let x = ( x0 x1 … xn ) be a binary n-tuple . the ( hamming) weight of x,

denoted by w(x), is defined to be the number of nonzero components of x; that is,

wt(x) = d(x,0) = ∑i=0

n−1

d (x i , 0).

where 0 is the zero word and d(x i ,0 ) = {1 if x i≠ 00 if x i=0 (3.2)

Example 3.2 The hamming weight of x = ( 1 1 0 0 1 ) is 3.

Lemma 3.1 : If x, y ∈ Vn then d( x ,y ) = wt( x – y ).

Corollary 3.1: Let x, y be two binary n-tuples, then d( x ,y ) = wt( x + y ).

Example 3.3 For x = ( 1 0 0 1 0 1 1 ), y = ( 1 1 1 0 0 1 0 )

d( x ,y ) = 4 and wt(x + y ) = wt ( 0 1 1 1 0 0 1 ) = 4 .

we now explain the minimum distance decoding; suppose the codewords v0,

v1, … v2k-1 from a code C(n,k) are being sent over a BSC.

If a word r is received, the nearest neighbor decoding or ( minimum distance

decoding) will decode r to the codeword vr that is the closest one to the received word

r . such procedures can be realized by an exhaustive search on the set of codewords

which consists of comparing the received word with all codewords and choosing of

the closest codeword. That is; d( r, vr) = min vi ∈ c d( r, vi) ∀ I = 0, 1, …, 2k-1.

Example 3.4 Let C= { ( 0 0 0 0 0 ), ( 1 1 0 0 1 ), ( 1 1 1 1 0), ( 0 1 1 1 1 ) } ⊆V5.

Consider r = ( 1 0 0 0 1 ) is a received codeword. r ∉ C.

d( r, ( 0 0 0 0 0 )) = 2

d( r, ( 1 1 0 0 1 )) = 1

d( r, ( 1 1 1 1 0)) = 4

d( r, ( 0 1 1 1 1 )) = 4

hence, r is decode to ( 1 1 0 0 1) which is nearest one to r.

3.2: Syndrome & Error Detection / Correction

Consider an (n,k) linear code C . let v = (v0 ,v1 …,vn-1) be a codeword that was

transmitted over a noisy channel (BSC). Let r = (r0 ,r1 …,rn-1) be the received vector

at the output of the channel. Because of the channel noise, r may be different from v.

hence, the vector sum e = r + v = ( e0 e1 … en-1) (3.3)

Is an n-tuple where ei =1 for ri ≠vi ∀ i = 0, 1, … n-1.

This n-tuple is called an error vector or ( error pattern). The 1`s in e are the

transmission errors caused by the channel noise.

Defintion 3.3 Let C ⊆ Vn be an (n,k) linear code with parity-check matrix H . then

for a received word r , the syndrome of s, denoted s ( r ) is:

s( r ) = s = r. HT = ( s0 s1 …sn-k-1) (3.4)

note that s is a linear map s : Vn → V n-k

Remark:

i. s( r ) = 0 ⟺ r ∈ C ⟹ r is a codeword and the receiver accepts r as the

transmitted codeword.

ii. when s ≠ 0 , we know that the received word r ∉ C and the presence of error

has been detected.

Defintion 3.4 An error pattern e is called an undetectable error pattern if it is

identical to a codeword.

When a codeword v is transmitted over a noisy channel and undetectable error

e occurred to the transmitted codeword v then, the received word r will be r = v + e ,

which is also a codeword since it`s the sum of tow codewords. Thus the syndrome of

r will be zero.

In this case, the decoder accepts r as transmitted codeword and thus commits

an incorrect decoding, and we say that the decoder makes a decoding error.

Now let H be a parity-check matrix in a systematic form of an (n,k) linear

code. Then the syndrome digits are as follows;

s( r ) = s( r0 r1 …rn -1) = r.HT = ( s0 s1 …sn-k-1)

Example 3.5 Consider the (5,2) linear code whose parity-check matrix

H = [1 0 0 1 10 1 0 1 10 0 1 1 0 ] in systematic form.

let r = (r0 r1 r2 r3 r4) be the received word. Then its syndrome is given by:

s = ( s0 s1 s2 )= (r0 r1 r2 r3 r4). [1 0 00 1 00 0 11 1 11 1 0

]Theorem 3.2 The syndrome s of a received vector r = v + e depends only on the error

pattern e, and not on the transmitted codeword v.

We now use the syndrome for error correction; let H be a parity-check matrix in a

systematic form of an (n,k) linear code C. then the syndrome digits of the received

word r = r = (r0 r1 …rn-1) can be formed as follows:

s( r ) = s( r0 r1 …rn -1) = e.HT = ( s0 s1 …sn-k-1)

the system above of linear equation can be solved for the digits of an error pattern

e = ( e0 e1 … en-1) then compute the decode word v* :

v* = r + e

note that the system above is an (n-k) × n of linear equation, and so, it doesn`t have a

unique solution.

Theorem 3.3 The (n-k) linear equation mentioned above do not have a unique

solution but have 2k solutions.

In other words , there are 2k error patterns that result in the same syndrome,

and the true error pattern e is just one of them.

Theorem 3.4 For the BSC, the most probable error pattern e is the one that has the

smallest number of nonzero digits.

Example 3.6 Again, we consider the code C(5,2) with parity-check matrix

H = [1 0 0 1 10 1 0 1 10 0 1 1 0 ] . let v = ( 0 0 1 1 1 ) be the transmitted codeword over BSC

and r = ( 1 0 1 1 1 ) be the received vector.

The problem is to find the digits of an error pattern e = ( e0 e1 e2 e3 e4 ).

1. Compute the syndrome s = ( s0 s1 s2 ) of r = ( 1 0 1 1 1 )

s = r. HT = ( 1 0 1 1 1 ). [1 0 00 1 00 0 11 1 11 1 0

] = ( 1 0 0 )

2. Solve the system for e = ( e0 e1 e2 e3 e4 ) with s = ( 1 0 0 ) as

HeT = sT ⟹ (1 0 0 1 10 1 0 1 10 0 1 1 0) [

e0

e1

e2

e3

e4

] = [100] ⟹ (1 0 0 1 10 1 0 1 10 0 1 1 0|

100)

⟹ {e0+e3+e4=1e1+e3+e4=0

e2+e3=0

There are 22 = 4 error patterns that satisfy the above system depending on e3 e4 = 0 0 or

0 1 or 1 0 or 1 1 they are :

( 1 0 0 0 0 ), ( 0 1 0 0 1 ), ( 0 1 1 1 0 ), ( 1 0 1 1 1).

Now, since the channel is BSC. then the most probable error pattern that satisfies the

system above is e =( 1 0 0 0 0 ) into the following codeword v*:

v* = r + e = ( 1 0 1 1 1 ) + ( 1 0 0 0 0 ) = ( 0 0 1 1 1 ).

We see that the receiver has made a correct decoding.

References

1. John Wiley & Sons " Foundation of Codes: Theory and Application of Errors –Correction Codes with an Introduction to Cryptography and Information", 1991

2. Kolman B. Introductory Linear Algebra : with Application. 3rd ed. United State of America. 1997

3. Jean Berstel and Dominique Perrin" theory of codes", 2002

4. Todd K Moon " Error Correction Coding : mathematical methods and algorithms", 2005

5. Torleiv Clove "Series on Coding Theory and Cryptology- Vol. 2 codes for error detection " 2007

6. Alain Glavieux "Channel Coding in Communication Networks From Theory to Turbocodes" 2007