Elements of Coding Theory

(Based on Grimaldi’s Discrete Mathematics)

1 Introduction
Coding theory explores how to transmit data reliably across noisy communication channels.
This theory was inspired by the fundamental paper of Claude Shannon (1948), Marcel Golay
(1949), and Richard Hamming (1950). It uses concepts from algebraic structures, probability,
and combinatorics.

1.1 Message
A message is defined as a finite sequence of characters from a finite alphabet.

1.2 Word
In binary coding theory, the alphabet is usually taken to be the binary set:
B = {0, 1}.
Every character or symbol that we want to transmit is now represented as a sequence of m
elements from B. This binary string is called a word. Thus, our basic unit of information,
called a word, is a sequence of m 0′ s and 1′ s.

1.3 Group Structure of Binary Words

The set B = {0, 1} is a group under addition modulo 2, denoted by ⊕2 . This group is
denoted by (Z2 , ⊕2 ).
For a fixed integer m, the Cartesian product
Zm
2 = Z2 × Z2 × · · · × Z2 (m times)
forms a group under component-wise addition modulo 2, denoted by ⊕:
(x1 , x2 , . . . , xm ) ⊕ (y1 , y2 , . . . , ym ) = (x1 ⊕2 y1 , x2 ⊕2 y2 , . . . , xm ⊕2 ym ).

1.4 Notation
A element (b1 , b2 , b3 , . . . , bm ) in Zm 5
2 is denoted by b1 b2 b3 . . . bm . For example, (1, 0, 1, 1, 0) ∈ Z2
is simply denoted by 10110.

1
1.5 Properties of Zm
2 :
• Identity element: e = (0, 0, . . . , 0)

• Every element is its own inverse.

• Number of elements: 2m

For instance, inverse of 10110 in Z52 is 10110. Since 10110 ⊕ 10110 = 00000

1.6 Transmission and Noise

In digital communication, information is transmitted from a sender to a receiver through a
transmission channel. This channel is essentially the medium or pathway that carries the
signals. However, real-world channels are imperfect and introduce noise, which can corrupt
transmitted data.
When a word x ∈ Zm m
2 is transmitted over a channel, it may be received as xt ∈ Z2 . Due
to noise (e.g., electrical faults, interference), some bits may change, resulting in:

xt ̸= x.

This situation is illustrated below:

Hence, we want to develop a technique to help us detect and perhaps even correct transmis-
sion errors. However, we can only improve the chance of correct transmission; there are no
guarantees.

2 Binary Symmetric Channel (BSC)

The Binary Symmetric Channel (BSC) is a fundamental and widely used theoretical model
that captures the essential characteristic of errors in a simple yet powerful way. A Binary
Symmetric Channel is a model where:

• Each bit transmitted (0 or 1) has a fixed probability p of being received incorrectly.

• Probability of correct transmission is 1 − p.

• The channel is symmetric because the error probability is the same for 0 and 1.

2
2.1 Probability of Errors
Consider the following example:
Let
c = 10110 ∈ Z52
be the transmitted code, and

r = 00110 ∈ Z52

be the received code.

Suppose
e = 10000
We observe that:
c⊕e=r or equivalently r ⊕ c = e
Here, e is called the error pattern.

• The positions of 1’s in e indicate the locations of errors in the received code.

• The number of 1’s in e corresponds to the number of bit errors during transmission.

General Case
Let
c ∈ Zn2
be a transmitted codeword, and
r =c⊕e
be the received codeword, where:

• e is the error vector.

• 1’s in e represent bit positions where errors occurred in r.

• Number of bit errors in transmission is number of 1’s in e.

3
Theorem 2.1. Let c ∈ Zn2 be the transmitted codeword through BSC with probability p of
incorrect transmission.

• The probability of receiving r = c + e (with e having k ones):

P (r = c ⊕ e) = pk (1 − p)n−k

• The probability that exactly k errors are made in the transmission:

 
n k
Pk = p (1 − p)n−k
k

Example 2.2. Let c = 10110, p = 0.05

Probability of no error: (0.95)5 ≈ 0.77

 
5
Probability of 1-bit error: (0.05)(0.95)4 ≈ 0.204
1
 
5
Probability of 2-bit error: (0.05)2 (0.95)3 ≈ 0.021
2

Example 2.3. Let C be a set of codewords, where C ⊆ Zn2 . In each of the following cases,
two of the three values are given: the codeword c, the received word r, and the error pattern
e. Using the relation
r = c ⊕ e,
determine the missing term.

(a) c = 1010110, r = 1011111.

(b) c = 1010110, e = 0101101.

(c) Given: e = 0101111, r = 0000111.

Answer (a):
e = r ⊕ c = 1011111 ⊕ 1010110 = 0001001
Answer (b):
r = c ⊕ e = 1010110 ⊕ 0101101 = 1111011
Answer (c):
c = r ⊕ e = 0000111 ⊕ 0101111 = 0101000

Example 2.4. A binary symmetric channel has probability p = 0.05 of incorrect transmis-
sion per bit. Let the codeword
c = 011011101
be transmitted. What is the probability of the following events?

(a) We receive r = 011111101.

4
 (b) We receive r = 111011100.

(c) A single error occurs.

(d) A double error occurs.

(e) A triple error occurs.

Answer Let n = 9, so the probability of correct transmission per bit is 1 − p = 0.95.

(a): r differs from c in exactly one bit.

P = (0.05)1 · (0.95)8 ≈ 0.315

(b): This differs from c in 3 bits (positions 1, 2, and 9).

P = (0.05)3 · (0.95)6 ≈ 0.00029

9


(c): There are 1
= 9 such patterns.

P = 9 · (0.05)1 · (0.95)8 ≈ 0.315

(d):  
9
P = · (0.05)2 · (0.95)7 = 36 · 0.0025 · 0.698 ≈ 0.0629
2
(e):  
9
P = · (0.05)3 · (0.95)6 = 84 · 0.000125 · 0.735 ≈ 0.0077
3

3 Encoding and Decoding

The basic task in the transmission of information is to reduce the probability of receiving a
word that differs from the word that was sent. This is done as follows:

• Encoding function: Let m, n ∈ Z+ with n > m. Let W ⊆ Zm 2 be the set of

n
messages, and C ⊆ Z2 be the set of code words. Consider and define a one-to-one
encoding function:
E : W → Zn2 with E(w) = c ∈ C.
This function maps each m-bit word in W into an n-bit codeword (obtained by ap-
pending n − m extra bits), introducing redundancy to detect or correct errors.

• Upon transmission, c is received as T (c) may be different at different transmission

times.

• Decoding function: Upon receiving T (c), we apply decoding function

D : Zn2 →, W

and hope to receive original message w.

5
 • Ideally, D ◦ T ◦ E should be the identity on W , but this is not always possible due to
errors. The goal is to design E and D so that the probability of correctly decoding is
high.

• Efficiency (Rate): The ratio

m
n
measures the efficiency of the scheme and is called the rate of the code.

• Block code The ordered pair (n, m) is called block code of E and D.

We illustrate the above idea using the following codes:

• Parity-Check code

• Triple repetition code

3.1 Parity-Check Code

′
Definition 3.1 (Weight of x ∈ Zm m
2 ). For any x ∈ Z2 , the number of 1 s is x is called the
weight of x and is denoted by wt(x). For example, if x = 0111011, then wt(x) = 5.

3.1.1 Even parity check:

The (m+1, m) block code is called even parity check code if for any w = w1 w2 w3 . . . wm ∈ Zm
2 ,
define
E : Zm2 → Z2
m+1
by E(w) = w1 w2 w3 . . . wm wm+1
where
(
0 if wt(w) is even,
wm+1 =
1 if wt(w) is odd.

• Observe that wm+1 is zero if and only if the number of 1’s in w is even. Hence, every
code word E(w) has even weight.

• A single-bit error in the transmission of a code word will change its weight to odd,
which can be detected.

6
 • Similarly, any odd number of bit errors can be detected by this code.

• The scheme does not detect errors if an even number of bits are altered

• If an error is detected, the codeword is retransmitted and continue the process until
the correct code is received or received code has an even number of 1′ s.

Example: Suppose w = 11010110. Then the codeword is

c = E(w) = 110101101.

If the transmission channel sends this as T (c) = 100101101, we compute the weight wt(T (c)) =
5, which is odd. Therefore, we conclude that an error (or an odd number of errors) has oc-
curred.

3.1.2 odd parity check:

The (m+1, m) block code is called odd parity check code if for any w = w1 w2 w3 . . . wm ∈ Zm
2 ,
define
E : Zm2 → Z2
m+1
by E(w) = w1 w2 w3 . . . wm wm+1
where
(
0 if wt(w) is odd,
wm+1 =
1 if wt(w) is even.

Example 3.2. For the given codes, apply (i) the even parity check, (ii) the odd parity check
codes.
(a) w = 1011
(b) w = 1010
(c) w = 101101
(d) w = 110111
Answer:
Label Original Code (w) Even Parity Odd Parity
(a) 1011 10111 10110
(b) 1010 10100 10101
(c) 101101 1011010 1011011
(d) 110111 1101111 1101110

Example 3.3. Apply the odd parity check to the code 101101. Determine if the received
code is correct and whether the error is detected.
(a) r = 1011011
(b) r = 1111011
(c) r = 1001111

7
Answer:

Original code: w = 101101 ⇒ Transmitted (odd parity): c = 1011011

Case Received Code (r) Error in r Error Detected?

(i) 1011011 No —
(ii) 1111011 Yes Yes
(iii) 1001111 Yes No (even number of 1’s)
Example 3.4. Apply the even parity check to the code 110111. Determine if the received
code is correct and whether the error is detected.
(a) r = 1101111
(b) r = 1100111
(c) r = 1001011
Answer:

Original code: w = 110111 ⇒ Transmitted (even parity): c = 1101111

Case Received Code (r) Error in r Error Detected?

(i) 1101111 No —
(ii) 1100111 Yes Yes
(iii) 1001011 Yes No (even number of 1’s)

Probability of Correct Detection

Let p be the bit error probability. The probability of sending the code 110101101 and making
at most one error is:
 
9 9
(1 − p) + p(1 − p)8
1
For p = 0.001, this gives:
 
9 9
(0.999) + (0.001)(0.999)8 ≈ 0.99996417
1
Improved Reliability: If an error is detected, the codeword is retransmitted and con-
tinue the process until the received word has an even number of 1′ s.

Trade-off:
Adding extra bits reduces the chance of undetected error but also reduces the rate of the
code
Example 3.5. Suppose 160 bits are sent in successive strings of length 8. Find the proba-
bility of receiving the correct message:

8
 (i) without any coding scheme,
(ii) with the parity-check method.
Assume p = 0.001.
Solution:

(i) Since there are 160 bits and each string is of length 8, there are 160/8 = 20 strings.
Therefore, Probability of receiving the correct message without any coding scheme

P (correct message) = (0.999)160 ≈ 0.852076

(ii) Probability with the parity-check method: In parity-check, each 8-bit string becomes
a 9-bit code word with error detection.
A 9-bit word is correct if either:
• all 9 bits are received correctly: (0.999)9 ,
• exactly 1 bit is incorrect and it is detected and corrected: 9 · (0.001) · (0.999)8 .
So the total probability per word:

P (correct) = (0.999)9 + 9(0.001)(0.999)8 ≈ 0.999964

For 20 such 9-bit strings:

P (correct message) = (0.999964)20 ≈ 0.9999280

Example 3.6. Suppose the message w = 110101 is sent. Find the probability of receiving
the correct message:
(i) without any coding scheme,
(ii) with the parity-check method.
Assume p = 0.05.
Solution:

(i) Probability of receiving the correct code without any coding method:

P (correct) = (0.95)6 = 0.7351

(ii) In the parity-check method, we append a bit to make the number of 1’s even. For
w = 110101, number of 1’s is 4 (even), so we append a 0:

New code = 1101010

The code is 7 bits long. The probability of correct message:

P = (0.95)7 + 7 · (0.05)(0.95)6 = 0.9552

9
3.2 Triple repetition code
The (3m, m) block code is called triple repetition code if for any w = w1 w2 w3 . . . wm ∈ Zm
2 ,

• Define
E : Zm 3m
2 → Z2 by E(w) = w1 w2 . . . wm w1 w2 . . . wm w1 w2 . . . wm

• The decoding function D : Z3m m

2 → Z2 uses the majority rule.

For example, if w = 10110111, then the encoded word is:

c = E(w) = 101101111011101110111011.

If we receive:
T (c) = 101001110011011110110110,
we decode each group of three bits by majority:

• First triplet: positions 1, 9, 17 = 1, 1, 0 ⇒ 1

• Second triplet: positions 2, 10, 18 = 0, 0, 0 ⇒ 0

• Third triplet: positions 3, 11, 19 = 1, 1, 1 ⇒ 1

• Fourth triplet: 0, 0, 0 ⇒ 0

• Fifth triplet: 0, 1, 1 ⇒ 1

• Sixth triplet: 1, 1, 1 ⇒ 1

• Seventh triplet: 1, 1, 1 ⇒ 1

• Eighth triplet: 0, 1, 0 ⇒ 0

So we recover the correct message:

10110111

Error Analysis
Even though multiple bit errors occurred, as long as at most one error happens in each group
of three, the original bit can be correctly recovered. However, if two or more errors occur in
the same group (e.g., at positions 2, 10, and 18), decoding may fail.

10
Probability of Correct Decoding
Let p = 0.001 be the probability of a bit error. Then, the probability of correctly decoding
a single bit using triple repetition is:
 
3 3
Pcorrect = (1 − p) + (p)(1 − p)2
1
 
3 3
= (0.999) + (0.001)(0.999)2 ≈ 0.99999700.
1

Therefore, the probability of correctly decoding an 8-bit message is:

(0.99999700)8 ≈ 0.99997600.

This is slightly better than the parity-check code, but note that it requires transmitting
24 bits instead of 8, so the transmission rate is 13 . While transmission time increases, we
avoid retransmissions and gain the ability to correct errors—not just detect them.

Conclusion
• The triple repetition code is effective at correcting single-bit errors with high proba-
bility.

• The rate of code is 31 . That is, lower transmission efficiency.

• Retransmission is avoided.

Example 3.7. Let E : Z32 → Z92 be the encoding function for the (9, 3) triple repetition
code.

(a) Let D : Z92 → Z32 be the corresponding decoding function. Apply D to decode the
received words:
(i) r = 111101100
(ii) r = 010011111

(b) Find three different received words r for which D(r) = 000.

(c) For each w ∈ Z32 , what is |D−1 (w)|?

Solution (a):
(i)

1st, 4th, and 7th positions: 1, 1, 1 ⇒ decoded bit: 1

2nd, 5th, and 8th positions: 1, 0, 0 ⇒ decoded bit: 0
3rd, 6th, and 9th positions: 1, 1, 0 ⇒ decoded bit: 1
⇒ Final decoded word: 101

11
 (ii)

1st, 4th, and 7th positions: 0, 0, 1 ⇒ decoded bit: 0

2nd, 5th, and 8th positions: 1, 1, 1 ⇒ decoded bit: 1
3rd, 6th, and 9th positions: 0, 1, 1 ⇒ decoded bit: 1
⇒ Final decoded word: 011

Solution (b):
For D(r) = 000, in each of the positions:

• 1st, 4th, and 7th: majority should be 0

• 2nd, 5th, and 8th: majority should be 0
• 3rd, 6th, and 9th: majority should be 0

Possible received words:

000000000, 100000000, 010000000

Solution (c):

D : Z92 → Z32
|Z92 | = 512, |Z32 | = 8
|Z9 | 512
⇒ D−1 (w) = 32 = = 64
|Z2 | 8

Example 3.8. The (5m, m) five-times repetition code has encoding function E : Zm2 → Z2 ,
5m

where E(w) = wwwww. Decoding with D : Z5m m

2 → Z2 is accomplished by the majority rule
(i.e., we are able to correct single and double errors made in transmission).

(a) With p = 0.05, what is the probability for the transmission and correct decoding of
the signal 0?

(b) Answer part (a) for the message 110 in place of the signal 0.

(c) For m = 2, decode the received word r = 0111001001.

(d) If m = 2, find three received words r where D(r) = 00.

−1
(e) For m = 2 and D : Z10 2 2
2 → Z2 , what is |D (w)| for each w ∈ Z2 ?

Solution:

12
(a) Here w = 0, ⇒ E : Z12 → Z52 , ⇒ E(w) = 00000
then D : Z52 → Z12 .
With p = 0.05, the probability of correctly decoding a single bit is (by majority rule
we can make at most 2 errors out of 5 positions):
   
5 5 4 5
(0.95) + (0.05)(0.95) + (0.05)2 (0.95)3 = 0.998842
1 2

So the probability of receiving and correctly decoding the one-bit message is:

(0.998842)1 = 0.998842

(b) Here w = 110, E : Z32 → Z15

2 , ⇒ T (w) = 111001110111000
15 3
then D : Z2 → Z2
With p = 0.05, the probability of correctly decoding a single bit is:
   
5 5 4 5
(0.95) + (0.05)(0.95) + (0.05)2 (0.95)3 = 0.998842
1 2

So the probability of receiving and correctly decoding the three-bit message is:

(0.998842)3 = 0.996530

(c) With m = 2, received word r = 0111001001.

The 1st, 3rd, 5th, 7th and 9th positions have 0, 1, 0, 1, 0
By majority rule the correct message is 0.
The 2nd, 4th, 6th, 8th and 10th positions have 1, 1, 0, 0, 1
By majority rule the correct message is 1.
The decoded message is 01.

(d) m = 2, and D(r) = 00, the received word r is:

0000000000, 0000000011, 1100000000

(e) m = 2, D : Z10 2
2 → Z2

For each w ∈ Z22 ,

|Z10
2 | 210
|D−1 (w)| = = = 256
|Z22 | 22

13
4 Hamming metric
Definition 4.1 (Weight). For any x = x1 x2 . . . xn ∈ Zn2 , the weight of x, denoted by wt(x),
is the number of 1’s in x: n
X
wt(x) = xi .
i=1

Lemma 4.2. For all x, y ∈ Zn2 , we have:

wt(x ⊕ y) ≤ wt(x) ⊕ wt(y).

Proof. This follows because bitwise addition modulo 2 does not introduce any new 1’s. The
only case where xi ⊕2 yi = 1 is when exactly one of xi or yi is 1.

Example 4.3. We have

• wt(01001) = 2

• wt(11101) = 4

• wt(01001 + 11101) = wt(10100) = 2

Definition 4.4 (Distance). The Hamming distance between x, y ∈ Zn2 is defined as:
n
(
X 0 if xi = yi ,
d(x, y) = d(xi , yi ), where d(xi , yi ) =
i=1
1 if xi ̸= yi .

It counts the number of positions where xi ̸= yi for 1 ≤ i ≤ n..

Example 4.5. We have

• x = 01001, y = 11101 ⇒ d(x, y) = 2

• x = 10101, y = 10101 ⇒ d(x, y) = 0

Note:
d(x, y) = wt(x ⊕ y).
This is because in Z2 , xi ⊕2 yi = 1 if and only if xi ̸= yi .

Properties of d(x, y):

Let d(x, y) be the Hamming distance. Then for all x, y, z ∈ Zn2 :

1. d(x, y) ≥ 0

2. d(x, y) = 0 ⇔ x = y

3. d(x, y) = d(y, x)

4. d(x, z) ≤ d(x, y) + d(y, z) (Triangle Inequality)

14
4.1 Hamming Metric
The function d(x, y) defined on Zn2 is a distance function or metric, and the space (Zn2 , d)
is called a metric space. It captures the idea of closeness between binary vectors.

Definition 4.6 (Sphere of Radius k). For a vector x ∈ Zn2 , the sphere of radius k
centered at x is:
S(x, k) = {y ∈ Zn2 : d(x, y) ≤ k}

Example 4.7. For n = 3, x = 110 ∈ Z32 , we have:

• S(x, 1) = {110, 010, 100, 111}

• S(x, 2) = {110, 010, 100, 111, 000, 101, 011}

4.2 Major Theorem

Theorem 4.8. Let E : W → C be an encoding function with the set of all messages W ⊆ Zm
2
and the set of code words E(W ) = C ⊆ Zn2 , where m < n.

1. We can detect all transmission errors of weight ≤ k iff the minimum distance between
code words is at least k + 1.

2. We can correct all transmission errors of weight ≤ k iff the minimum distance between
code words is at least 2k + 1.

Example 4.9. Let W = Z22 , and let E : W → Z62 be defined by:

E(00) = 000000,
E(10) = 101010,
E(01) = 010101,
E(11) = 111111.

1. Compute Hamming distances between code words and hence find minimum distance
between code words.

2. Discuss error detecting and correcting capabilities.

Solution 1: Let c1 = E(00), c2 = E(10), c3 = E(01), c4 = E(11).

We compute Hamming distances:

d(c1 , c2 ) = 3,
d(c1 , c3 ) = 3,
d(c1 , c4 ) = 6,
d(c2 , c3 ) = 6,
d(c2 , c4 ) = 3,
d(c3 , c4 ) = 3.

15
 The minimum distance between the codewords is 3.
Solution 2: By Theorem 4.8 (1), k + 1 = 3 ⇒ k = 2; and by Theorem 4.8 (2), 2k + 1 =
3 ⇒ k = 1.
Hence, we can:
• Detect up to 2 errors,

• Correct up to 1 error.
Example 4.10. Let W = Z22 and E : W → Z62 be defined by:

E(00) = 000000,
E(10) = 101010,
E(01) = 010101,
E(11) = 111111.

1. List the elements in S(101010, 1) and S(111111, 1).

2. Decode each of the following received words using majority rule.

(i) 110101
(ii) 101011
(iii) 001111
(iv) 110000

Solution 1:
S(101010, 1) = {x ∈ Z62 | d(x, 101010) ≤ 1}:

{101010, 001010, 111010, 100010, 101110, 101000, 101011}

S(111111, 1) = {x ∈ Z62 | d(x, 111111) ≤ 1}:

{111111, 011111, 101111, 110111, 111011, 111101, 111110}

Solution 2:

(i) 110101
Majority at 1st, 3rd, 5th positions: 0
Majority at 2nd, 4th, 6th positions: 1
⇒ Decoded word is: 01

(ii) 101011
Majority at 1st, 3rd, 5th positions: 1
Majority at 2nd, 4th, 6th positions: 0
⇒ Decoded word is: 10

16
(iii) 001111
Majority at 1st, 3rd, 5th positions: 1
Majority at 2nd, 4th, 6th positions: 1
⇒ Decoded word is: 11

(iv) 110000
Majority at 1st, 3rd, 5th positions: 0
Majority at 2nd, 4th, 6th positions: 0
⇒ Decoded word is: 00

Note:
Suppose c = 010101 and T (c) = r = 111101. Then:
• r is not a codeword, so a double error can be detected.
However, if T (c) = r1 = 111111 (a triple error), then:
• We incorrectly decode r1 as the codeword 111111, assuming c = 11 instead of the
correct message 01.
Example 4.11. Answer the following:
1. If x ∈ Z10
2 , determine |S(x, 1)|, |S(x, 2)|, |S(x, 3)|.

2. For n, k ∈ Z+ with 1 ≤ k ≤ n, if x ∈ Zn2 , what is |S(x, k)|?

Solution 1:

S(x, 1) = y ∈ Z10

2 d(x, y) ≤ 1
= y ∈ Z10

2 d(x, y) = 0 or d(x, y) = 1

For d(x, y) = 0, y should be same as x, there is only one such possibility.

For d(x, y) = 1, y should differ from x at only one position, and there are C(10, 1) = 10 such
possibilities. Therefore,
|S(x, 1)| = 1 + 10 = 11

S(x, 2) = y ∈ Z10

2 d(x, y) ≤ 2
= y ∈ Z10

2 d(x, y) = 0 or d(x, y) = 1 or d(x, y) = 2

For d(x, y) = 2, y should differ from x at only two positions, and there are C(10, 2) = 45
possibilities.

|S(x, 2)| = 1 + 10 + 45 = 56
Similarly,
|S(x, 3)| = 1 + C(10, 1) + C(10, 2) + C(10, 3) = 176

17
Solution 2:
With 1 ≤ k ≤ n, for x ∈ Zn2 ,
k
X
|S(x, k)| = 1 + C(n, 1) + C(n, 2) + · · · + C(n, k) = C(n, r)
r=0

Example 4.12. Let E : Z52 → Z25 2 be an encoding function where the minimum distance
between code words is 9. What is the largest value of k such that we can detect errors of
weight ≤ k? If we wish to correct errors of weight ≤ t, what is the maximum value of t?
Soln:
We can use Theorem 4.8 (1) to find k: k+1=9⇒k =8
We can use Theorem 4.8 (2) to find t: 2t + 1 = 9 ⇒ t = 4
Example 4.13. For each of the following encoding functions, find the minimum distance
between the code words. Discuss the error-detecting and error-correcting capabilities of each
code.
1. E : Z22 → Z52

00 → 00001
01 → 01010
10 → 10100
11 → 11111

2. E : Z22 → Z10
2

00 → 0000000000
01 → 0000011111
10 → 1111000000
11 → 1111111111

3. E : Z32 → Z62

000 → 000011
001 → 001001
010 → 010010
011 → 011100
100 → 100100
101 → 101010
110 → 110001
111 → 111000

18
 4. E : Z32 → Z82

000 → 00011111
001 → 00111010
010 → 01010101
011 → 01110000
100 → 10001101
101 → 10101000
110 → 11000100
111 → 11100011

Solution 1:

d(00001, 01010) = 3, d(00001, 10100) = 3, d(00001, 11111) = 4

d(01010, 10100) = 4, d(01010, 11111) = 3, d(10100, 11111) = 3
Minimum distance between code words is 3.
Applying Theorem 4.8 (1): k + 1 = 3 ⇒ k = 2

The code can detect all errors of weight ≤ 2.

Applying Theorem 4.8 (2): 2k + 1 = 3 ⇒ k = 1

The code can correct all errors of weight ≤ 1.

Solution 2: Let the code words be denoted as follows:

r1 = 0000000000,
r2 = 0000011111,
r3 = 1111000000,
r4 = 1111111111
d(r1 , r2 ) = 5,
d(r1 , r3 ) = 5,
d(r1 , r4 ) = 10,
d(r2 , r3 ) = 10,
d(r2 , r4 ) = 5,
d(r3 , r4 ) = 5

Minimum distance between code words is 5.

Applying Theorem 1: k+1=5⇒k =4

19
 The code can detect all errors of weight ≤ 4.
Applying Theorem 2: 2k + 1 = 5 ⇒ k = 2

The code can correct all errors of weight ≤ 2.

Solution 3:
Let the code words be denoted as follows:

r1 = 000111, r2 = 001001, r3 = 010010, r4 = 011100,

r5 = 100100, r6 = 101010, r7 = 110001, r8 = 111000
r2 r3 r4 r5 r6 r7 r8
r1 3 3 4 4 4 4 6
r2 4 3 4 3 3 3
r3 3 4 3 3 3
r4 3 4 4 2
r5 3 3 3
r6 4 2
r7 2
r8

Minimum distance between code words is 2.

Applying Theorem 1: k+1=2⇒k =1

The code can detect all errors of weight ≤ 1.

1
Applying Theorem 2: 2k + 1 = 2 ⇒ k = 2

1
The code can correct all errors of weight ≤ ⇒ the code cannot correct any error.
2

20
Introduction
In coding theory, generator matrices and parity-check matrices help encode and decode
messages using binary linear codes over Z2 . These matrices are useful in locating the nearest
codeword to a received word, and they play a vital role in error detection and correction.

Generator Matrix G
A generator matrix G is used to encode a message vector into a codeword.
Let G be a k × n matrix over Z2 , and let w ∈ Zk2 be a message (row vector). The encoded
codeword is:
E(w) = wG ∈ Zn2

Example
Let  
1 0 0 1 1 0
G = 0 1 0 0 1 1
0 0 1 1 0 1
This is a 3 × 6 matrix. The first 3 columns form the identity matrix I3 , and the last 3
columns form matrix A. Thus,
G = [I3 | A]
This structure ensures systematic encoding, meaning the original message appears
directly in the codeword.

Codewords
Each message w ∈ Z32 produces a codeword in Z62 .

• E(110) = (110)G = 110101

• E(010) = (010)G = 010011

For
W = {000, 100, 010, 001, 110, 101, 011, 111}
The set of codewords is:

C = {000000, 100110, 010011, 001101, 110101, 101011, 011110, 111000}

21
Parity-Check Matrix H
The parity-check matrix H is used to verify if a received word is a valid codeword.
Given A as the right part of G, define:

H = [AT | In−k ]

In our example:
   
1 1 0 1 0 1 1 0 0
A = 0 1 1 ⇒ H = 1 1 0 0 1 0
1 0 1 0 1 1 0 0 1

Syndrome Decoding
Given a received word r ∈ Z62 , compute the syndrome:

syndrome(r) = H · rT

• If H · rT = 0, then r is a valid codeword.

• If H · rT ̸= 0, then r has errors.

Example (Continued...)
Suppose r = 110110. Then,
 
1
  1  
1 0 1 1 0 0  
0 0
T
H · r = 1 1 0 0 1 0 ·   = 1
    
1
0 1 1 0 0 1  1 1
0

This nonzero result indicates an error. Based on the syndrome, we identify and correct
the bit in error.

Error Detection and Correction

Let d be the minimum Hamming distance between any two codewords.

• The code can detect all errors of weight ≤ k if:

d>k

• The code can correct all errors of weight ≤ t if:

d ≥ 2t + 1

22
In Our Example
• Minimum distance, d = 3

• Can detect all errors of weight ≤ 2

• Can correct all errors of weight ≤ 1

 
0
• H · r = 1 is same as second column of H. Therefore, it has single error and error
T 
1
is in second component.

• Thus, transmitted code is 100110 and message is 100.

Multiple Errors
Suppose we receive r = 000111. Then,
 
0
  0  
1 0 1 0 0 0  
0 1
T
H · r = 1 1 0 1 0 0   = 1 ,
1 which is not a column of H.
0 1 1 0 0 1  
1 1
1

Yet

• H · rT can be obtained as the sum of two columns of H.

• If H · rT came from the first and sixth columns of H, correcting these components in
r results in the code word 100110.

• If we sum the third and fifth columns of H to get this syndrome, upon changing the
third and fifth component of r we get a second code word, 001101.

• So we cannot expect it to correct multiple errors.

(This is no surprise, since the minimum distance between code words is 3:

2k + 1 = 3 ⇒ k = 1,

the code can correct errors of at most 1.)

23
Example 4.14. Let E : Z32 → Z62 .

(a) Use the parity-check matrix H:

 
1 0 1 1 0 0
H = 1 1 0 0 1 0
0 1 1 0 0 1

To decode the following received words:

(i) 111101, (ii) 110101, (iii) 001111, (iv) 100100, (v) 110001, (vi) 111111,
(vii) 111100, (viii) 010100.

(b) Are all the results in part (a) uniquely determined?

Solution a):
Let r1 = 111101 Consider H · r1T :
 
1
  1    
1 0 1 1 0 0   1 1
1 1 0 0 1 0 1 = 0 ⇒ H · r1T = 0
 
1
0 1 1 0 0 1  
0 1 1
1
Similarly, we can obtain all other decoded words.
The decoded words are:
Considering
Received Correction
Decoded Word First 3 Bits (De-
Word at Position
coded Word)
(i) 111101 3rd 110101 110
(ii) 110101 nil 110101 110
(iii) 001111 5th 001101 001
(iv) 100100 5th 100110 100
(v) 110001 4th 110101 110
(vi) 111111 1st & 6th 011110 (other possibilities) 011
(vii) 111100 4th 111000 (other possibilities) 111
(viii) 010100 1st & 6th 110101 (other possibilities) 110

Solution b): No, not all results are uniquely determined, (vi) and (viii) received words
have other possibilities.
Example 4.15. The encoding function E : Z22 → Z52 is given by the generator matrix
 
1 0 1 1 0
G=
0 1 0 1 1

24
 (a) Determine all code words. What can we say about the error-detection capability of
this code? What about its error-correction capability?

(b) Find the associated parity-check matrix H.

(c) Use H to decode each of the following received words:

i) 11011
ii) 10101
iii) 11010
iv) 00111
v) 11101
vi) 00110

Solution a):
Let W = {00, 10, 01, 11}. Then
   
00   00000
10 1 0 1 1 0 10110
c = wG =  
01 0 1 0 1 1 =  
01011
11 11101
So the code is: C = {00000, 10110, 01001, 11101}.
Let c1 = 00000, c2 = 10110, c3 = 01001, c4 = 11101
The Hamming distance between code words are:

d(c1 , c2 ) = 3, d(c1 , c3 ) = 3, d(c1 , c4 ) = 4

d(c2 , c3 ) = 4, d(c2 , c4 ) = 3, d(c3 , c4 ) = 3

The minimum distance between the code words is 3. Therefore:

• The code can detect all errors of weight ≤ 2.

• The code can correct all single errors. since 2t + 1 ≤ 3 ⇒ t ≤ 1.

Solution b):
Now find H. Since E : Z22 → Z52 , m = 2, n = 5:
 
  1 0 1 0 0
1 1 0  
G = [I2 | A], where A = ⇒ H = AT | I3 = 1 1 0 1 0
0 1 1
0 1 0 0 1
Solution c): Use H to decode the received words:
Compute syndrome: s = HrT .

i) : r = 11011, s = HrT = 110 ⇒ error at position 1 ⇒ decode to 01011

25
 ii) : r = 10101, s = 011 ⇒ error at position 2 ⇒ decode to 11101
iii) : r = 11010, s = 111 ⇒ error at position 1 and 5 ⇒ decode to 01011
iv) : r = 00111, s = 111 ⇒ error at position 1 and 5 ⇒ decode to 10110
v) : r = 11101, s = 000 ⇒ valid codeword
vi : r = 00110, s = 110 ⇒ error at position 1 ⇒ decode to 10110
Considering the first 2 bits (message), the decoded words are:
01, 11, 01, 10, 11, 10
Example 4.16. Define the encoding function E : Z32 → Z62 by means of the parity-check
matrix  
1 0 1 1 0 0
H = 1 1 0 0 1 0
1 0 1 0 0 1
(a) Determine all code words.
(b) Does this code correct all single errors in transmission?
Solution a):
Any x = x1 x2 x3 x4 x5 x6 ∈ Z62 is a codeword if and only if HxT = 0.
Consider  
x1
  x2   
1 0 1 1 0 0  
x3  0
T
Hx = H = 1 1 0 0 1 0   = 0
x4 
1 0 1 0 0 1  
x5  0
x6
This implies the equations:

x1 + x3 = x4
x1 + x2 = x5
x1 + x 3 = x6
Possible combinations:

x1 x2 x3 x4 x5 x6 Codeword Label
0 0 0 0 0 0 000000 c1
1 0 0 1 1 1 100111 c2
0 1 0 0 1 0 010010 c3
0 0 1 1 0 1 001101 c4
1 1 0 1 0 1 110101 c5
1 0 1 0 1 0 101010 c6
0 1 1 1 1 1 011111 c7
1 1 1 0 0 0 111000 c8

26
Solution b):
Hamming distance table d(ci , cj ):

c1 c2 c3 c4 c5 c6 c7 c8
c1 − 4 2 3 4 3 5 3
c2 4 − 4 3 2 3 3 5
c3 2 4 − 5 4 3 3 3
c4 3 3 5 − 3 4 2 4
c5 4 2 4 3 − 5 3 3
c6 3 3 3 4 5 − 4 2
c7 5 3 3 2 3 4 − 4
c8 3 5 3 4 3 2 4 −
The minimum distance = 2. Therefore:

2t + 1 ≤ 2 ⇒ t ≤ 0 ⇒ This code cannot correct single errors in transmission.

27