1

Linear Block codes

Lecture Notes by Y. N. Trivedi

I. C HANNEL CODING

Communication systems should be efficient and reliable: Efficiency

is attained through source coding and reliability is coming through
channel coding. The objectives of channel coding are as follows.
• fast encoding of information.
• easy transmission of encoded message.
• fast decoding of received message.
• correction of errors introduced in the channel.
• maximum transfer of information per unit time.

II. L INEAR A LGEBRA

• Field: A field is an algebraic system < F ; +; . > such that

1) < F ; + > is an abelian group (Closure, Associativity, Iden-
tity element, Inverse element, Commutativity), where neutral
element is denoted by 0.
2) < F ; . > is an abelian group, where neutral element is de-
noted by 1.
3) For every a; b and c in F, the distributive law a.(b + c) =
(a.b) + (a.c) holds, and multiplication by 0 obeys the rule
a.0 = 0.a = 0.
• Vector Spaces: A vector space is an algebraic system < V; +; .; F ; +; . >
such that
 2

– < F ; +; . > is a field.

– < V; +; . > is an abelian group; (whose neutral element is
denoted by 0);
– . is an operation on pairs consisting of an element of F and an
element of V such that, for all c1; c2 in F and all v1; v2 in V,
∗ (closure) c1.v1 is in V;
∗ (scaling by 1) 1.v1 = v1;
∗ (associativity) c1.(c2.v1) = (c1.c2).v1
∗ (distributivity) (c1 + c2).v1 = (c1.v1) + (c2.v1) and c1.(v1 +
v2) = (c1.v1) + (c1.v2)
– In a vector space < V; +; .; F ; +; . >, the elements of V are
called vectors and the elements of F are called scalars.
– If n is a positive integer and if c1; c2; ..; cn are scalars, then the
vector c1v1 + c2v2 + .. + cnvn is called a linear combination
of the vectors v1; v2; ..; vn. The vectors v1; v2; ...; vn are said to
be linearly independent if the only linear combination of these
vectors that gives 0 is that with c1 = c2 = ... = cn = 0;
otherwise they are said to be linearly dependent.
– Subspaces: If < V; +; .; F ; +; . > is a vector space and if U
is a subset of V such that < U; +; .; F ; +; . > is also a vector
space, then < U; +; .; F ; +; . > is called a subspace of the
vector space < V; +; .; F ; +; . >.
– Vector Space of N-Tuples and basis: For any positive integer
N, the set F N is an N-dimensional vector space with 0 =
[0; 0; ...; 0]. The vectors [1; 0; ..; 0]; [0; 1; ..; 0]; ...; [0; 0; ...; 1] con-
taining a single non-zero component equal to 1 are a basis for
 3

F N.
– If V is any vector space with 0 < n < ∞ (i.e., any non-trivial
finite-dimensional vector space) with the scalar field F and if
e1; e2; ..; en is any basis for V, then every vector v in V can be
written uniquely as a linear combination of the basis vectors,
v = c1e1 + c2e2 + .. + cnen.
– Hamming Distance: The Hamming distance, d(x; y), between
N-tuples x and y with components in an arbitrary non-empty
set is defined as the number of coordinates in which x and y
differ. Properties of Hamming Distance.
∗ d(x; y) ≥ 0 with equality if and only if x = y (positive
definiteness);
∗ d(x; y) = d(y; x) (symmetry)
∗ d(x; z) + d(z; y) ≥ d(x; y) (triangle inequality).
– Hamming weight: The Hamming weight, w(x), of a vector x
in F N is defined to be the number of coordinates in which x
is non-zero. It follows that d(x; y) = w(y − x). Properties of
Hamming weight.
∗ w(x) ≥ 0 with equality if and only if x = 0.
∗ w(x) = w(−x).
∗ w(x) + w(y) ≥ w(x + y).

III. E RROR C ONTROL C ODING

Two valuable resources: Signal power and Channel bandwidth.

For certain power limited (fixed SNR) applications, when modula-
tion schemes can not improve the performance, error control coding
 4

or FEC is the acceptable solution.

Hamming Bound: It is a relationship between the length of data
word (k ) and code word (n), and the number of errors (t) it can
correct.
t
!
X n
n k
2 ≥2 (1)
i=0
i
A code for which the inequality becomes equality is known as the per-
fect code. Binary, single error correcting codes are called Hamming
codes.

A. Linear codes

A linear code is a code which is closed under addition of codewords,

i.e. if v and w are in code C , then v + w ∈ C . A linear code must
contain 0 codeword. For example C = {000, 111} is a linear code,
however C1 = {000, 001, 101} is not.
Determine which of the following codes are linear.
1) C = {101, 111, 011}.
2) C = {000, 001, 010, 011}.
3) C = {0000, 0001, 1110}.
4) C = {0000, 1001, 0110, 1111}.
5) C = {00000, 11111}.
6) C = {00000, 11100, 00111, 11011}.
7) C = {00000, 11110, 01111, 10001}.
8) C = {000000, 101010, 010101, 111111}.
 5

B. Linear Block codes

Generator matrix G = [Ik |P].

For (7,4) Hamming code
 
1 0 0 0 1 0 1
 
0 1 0 0 0 1 1
G= 0 0

 1 0 1 1 0
0 0 0 1 1 1 1
code word (ci) for message signal (mi = [1 1 0 1]) is ci = miG.
Parity check matrix H = [PT |In−k ].
 
1 0 1 1 1 0 0
H=0 1 1 1 0 1 0
 

1 1 0 1 0 0 1
GHT = 0.

Theorem : Let H be a parity check matrix for a linear code C. Then

C has distance d if and only if any set of d − 1 rows of H is linearly
independent, and at least one set of d rows of H is linearly dependent.
Proof: If v is a word, then vHT is a linear combination of exactly
wt(v) rows of H. So if v is in C and wt(v) = d, then since vHT = 0,
some d rows of H are linearly independent. And if vHT = 0 then v is
a codeword so wt(v) ≥ d.
 6

Determine the minimum distance for the following matrix H.

 
1 1 0
0 1 1
 
T
 
H =1 0 0


0 1 0
 

0 0 1
 
0 1 1 1
 
1 1 1 0
 
1 0 0 0
HT = 
 

0 1 0 0
 
0 0 1 0
 
0 0 0 1
 
1 1 1 0
1 1 0 1
 
 
1 0 1 1
 
0 1 1 1
HT = 
 

1 0 0 0
 
0 1 0 0
 
0 0 1 0
 

0 0 0 1
 7

 
1 1 0 1
1 0 1 1
 
 
1 1 1 0
 
T
H =1 0 0 0
 
 
0 1 0 0
 
0 0 1 0
 
0 0 0 1

C. Syndrome decoding

eHT = S, where S is Syndrome matrix.

   
0 0 0 0 0 0 0   0 0 0
1 0 0
 0 0 0 0 0 0 1   1 1 1
   
  0 1 0   
 0 0 0 0 0 1 0 
 0 0 1   1 1 0
 
 
 0 0 0 0 1 0 0   0 1 1
 1 0 1  = 
    
 
 0 0 0 1 0 0 0 
 0 1 1   1 0 1
 
 
 0 0 1 0 0 0 0   0 0 1
 1 1 0  
    
0 1 0 0 0 0 0 0 1 0
 
1 1 1
1 0 0 0 0 0 0 1 0 0
Syndrome is dependent upon the error pattern (e), not the codewords.

D. Cyclic codes

• Subclass of linear clock codes.

• Cyclic shift of a codeword is a new codeword.
• Encoding and syndrome calculation can be easily implemented
using simple shift registers.
 8

• C = [cn−1, cn−2 , ...., c1 p, c0] can be written by a polynomial C(p)

of degree ≤ n − 1.

C(p) = cn−1pn−1 + cn−2 pn−2 + .... + c1 + c0

• cn−1, ...c0 ∈ {0, 1} for binary code.

• pC(p) = cn−1 pn + cn−2pn−1 + c0p.
pC(p) cn−2 pn−1 +.....+cn−1
• pn +1 = c n−1 + pn +1 . Here, the remainder, let us take it as
C1(p) is a shifted version of C(p) or C1(p) = pC(p) mod (pn +1).
• Similarly, piC(p) = Q(p){pn + 1} + Ci(p), where Ci(p) is the
shifted version of C(p) by i number of positions.
Generation of Cyclic codes (n, k):
• data polynomial d(p) with k information bits is d(p) = dk−1pk−1 +
dk−2pk−2 + ... + d1p + d0 or d = [dk−1 dk−2 ...d0 ].
• Generator g(p) is of degree n−k , where g(p) = pn−k +gn−k−1 pn−k−1+
.. + g1p + 1.
• Let us take g(p) as a factor of pn + 1.
• The product d(p)g(p) is a polynomial of degree less or equal to
n − 1, which may represent a codeword.
• cm(p) = dm(p)g(p), where m = 1, 2, ...2k .
Theorem: If g(p) is a factor of n − k degree of pn + 1, then cm(p)
will be the cyclic codes.
Proof:

pc(p) + cn−1(pn + 1) = cn−1pn + cn−2pn−1 + .... + c1p2 + c0p + cn−1pn + cn−1

= cn−2pn−1 + .... + c1p2 + c0p + cn−1
= c1(p)
 9

Hence
c1(p) = pc(p) + cn−1(pn + 1) (-8)

Dividing this equation by g(p), we can say RHS is divisible because

• c(p) = d(p)g(p).
• g(p) is a factor of pn + 1.
Therefore, LHS is also divisible by g(p). However, c1(p) is another
codeword, shifted by 1 bit. So, c1(p) = d1(p)g(p).
Conclusion: A cyclic shift of any codeword cm(p) = dm(p)g(p)
gives another codeword.
Example: (7,4) cyclic codes.
Here, n = 7 and k = 4 and
pn + 1 = p7 + 1 = (p + 1)(p3 + p2 + 1)(p3 + p + 1)
Here g(p) can be either p3 + p2 + 1 or p3 + p + 1.
Let us take g(p) = p3 + p2 + 1.
Now cm(p) = dm(p)g(p)
For d0 = [0000] → c0 = [0000000].
For d1 = [0001] or d(p) = 1 → c1 = [0001101], c(p) = p3 + p2 + 1.
For d2 = [0010] or d(p) = p → c2 = [0011010], c(p) = p4 + p3 + p.

E. Systematic Cyclic codes

Let us take c(p) = pn−k d(p)+ρ(p), where degree of d(p) is ≤ k −1.

Prove that ρ(p) is the remainder from dividing pn−k d(p) by g(p).
 10

pn−k d(p)
 
ρ(p) = Rem
g(p)
pn−k d(p) ρ(p)
= q(p) + , degree(q(p)) ≤ k − 1
g(p) g(p)
pn−k d(p) + ρ(p)
= q(p)
g(p)
g(p)q(p) = pn−k d(p) + ρ(p)

As the degree of q(p) is less or equal to k − 1, g(p)q(p) is a true

codeword. So, RHS is also a valid codeword. Here,
• pn−k d(p) is shifted for n − k bits.
• ρ(p) indicates the parity bits.
Example: (7,4) systematic cyclic code, g(p) = p3 + p2 + 1. Find c(p)
for d(p) = p3 + p i.e. 1010.
pn−k d(p)=p6 + 4
n pn−k. o n 6 4 o
p d(p) p +p
ρ(p) = Rem g(p) = Rem p +p2 +1 = 1
3

It means c(p) = p6 + p4 + 1.

F. Systematic cyclic code generator

 11

If

gi = 0, it is open.
= 1, it is close.

Total n − k shift registers are required.

For the above example if d = 1010, then c = 1010001.
d d’ C0 C1 C2
1 1 0 0 0
0 1 1 0 1
1 0 1 1 1
0 1 0 1 1
1 0 0

G. Systematic cyclic code decoding

For a valid codeword c(p)

 
c(p)
Rem =0
g(p)
However, for erroneous codeword r(p)
 
r(p)
Rem 6= 0
g(p)
r(p) s(p)
= m1(p) +
g(p) g(p)
 
r(p)
s(p) = Rem
g(p)
where s(p) is syndrome polynomial of degree n − k − 1 or less.
 12

If e(p) is an error polynomial then r(p) = c(p) + e(p), where c(p) is

a multiple of g(p).
 
r(p)
s(p) = Rem
g(p)
 
c(p) + e(p)
= Rem
g(p)
 
e(p)
= Rem
g(p)
Thus, syndrome can be generated using shift registers.
Example:
• For received codeword r = 0011010, syndrome is s(p) = 0.
• For received codeword r = 1011010, syndrome is s(p) = p2 + p.