DIGITAL IMAGE PROCESSING

18CS741
[As per Choice Based Credit System (CBCS) scheme]
(Effective from the academic year 2018 -2019)
SEMESTER – VII
MODULE 5
Notes

Prepared By
Athmaranjan K
Associate Professor
Dept. of Information Science & Eng.
Srinivas Institute of Technology, Mangaluru

Athmaranjan K Dept of ISE

DIGITAL IMAGE PROCESSING 18CS741 Module 5

MODULE 5
SYLLABUS
Image Compression: Introduction, coding Redundancy, Inter-pixel redundancy, image compression model,
Lossy and Lossless compression, Huffman Coding, Arithmetic Coding, LZW coding, Transform Coding,
Sub-image size selection, blocking, DCT implementation using FFT, Run length coding.
Textbook 1: Rafael C G., Woods R E. and Eddins S L, Digital Image Processing, Prentice Hall, 2nd edition,
2008
Textbook 1: Ch. 8.1 to 8.5

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 2

DIGITAL IMAGE PROCESSING 18CS741 Module 5

IMAGE COMPRESSION
INTRODUCTION:
Image compression addresses the problem of reducing the amount of data required to represent a digital
image. Reduction process is the removal of redundant data. From Mathematical view point reduction process
involves transforming 2D pixel array into a statistically un-correlated data set. The Transformation is applied
before storage or transmission of image. Later compressed image is de-compressed to construct original
image or an approximation of it.
What is image compression?
Image compression is a process to reduce irrelevance and redundancy of the image data to be able to store or
transmit data in an efficient form.
Why do we need image compression?
 To reduce the storage space.
 To reduce the transmission time.
DATA COMPRESSION
 It is the process of reducing the amount of data required to represent a given quantity of information.
 Data is an individual unit that contains raw materials which do not carry any specific meaning.
 Information is a processed data that collectively carries a logical meaning.
Example: Narration of any story.
Information of interest is the story.
Words are the data used to relate the information.
DATA REDUNDANCY (RD)
Data redundancy occurs when the same piece of data exists in multiple places,
Example:
If the two individuals use a different number of words to tell the same basic story, two different versions of
the story are created, and at least one includes non-essential data. That is it contains data (words) that either
provide no relevant information or simply restate that which is already known. Thus the data contains
redundant data.
Data redundancy is a central issue in digital image compression.
Let n1 and n2 denote the number of information-carrying units in two data sets that represents the same
information.
The relative data redundancy RD of the first data set (the one characterized by n1)

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 3

DIGITAL IMAGE PROCESSING 18CS741 Module 5

For example: Redundancy of 0.9 implies that 90% of the data in the first data set is redundant.
COMPRESSION RATIO (CR)
It is the ratio between the uncompressed data size and compressed data size. It is denoted by CR.

A practical compression ratio, such as 10 (10:1), means that the first data set has 10 information carrying
units (Bits) for every 1 unit in the second or compressed data set.
Case 1: When n2 = n1
Then compression ratio CR = 1 and RD = 1 – 1/CR = 1 – 1 = 0, indicating that (relative to the second data set)
first representation of the information contains no redundant data.
Case 2: When n2 << n1
Then compression ratio CR = ∞ and RD = 1 – 1/CR = 1 – 0 = 1, indicating that significant compression and
first representation of the information contains high redundant data.
Case 3: When n2 >> n1
Then compression ratio CR = 0 and RD = 1 – 1/CR = 1 – ∞ = -∞ indicating that second data set contains much
more data than the original representation. This is of course, is the normal undesirable case of data
expansion.
NOTE: In general CR lie in the interval (0, ∞) and RD in the interval (-∞, 1)
TYPES OF DATA REDUNDANCIES IN DIGITAL IMAGE COMPRESSION
In digital image compression the 3 basic types of redundancies can be identified as:
1. Coding Redundancy
2. Inter-pixel redundancy
3. Psycho-visual redundancy
Data compression is achieved when one or more of these redundancies are reduced or eliminated
CODING REDUNDANCY
******Explain coding redundancy by taking suitable example.
If the gray levels of an image are coded in a way that uses more code symbols than absolutely necessary to
represent each gray level then the resulting image is said to contain coding redundancy.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 4

DIGITAL IMAGE PROCESSING 18CS741 Module 5
• Coding redundancy is almost always present when the intensities of an image are represented using a
natural binary code. (fixed length coding)
• Coding redundancy is present when the codes assigned to a set of events do not take full advantage of
the probabilities of the events.
• Coding redundancy can be eliminated by coding the gray levels using variable length coding; in
which assigning fewer bits to the more probable gray levels than to the less probable ones achieves
data compression.
Example:
Let us consider the computer generated 256 x 256 (8-bit) images with the intensity distribution as shown
below:
Intensity value (rk) Probability
87 0.25
128 0.47
186 0.25
255 0.03
All other intensity
value rk for k ≠ 87, 0
128, 186, 255
If the image data (rk) is represented with natural 8-bit binary code [binary number] (Fixed length coding):

Intensity value (rk) Probability Fixed length Code 1 Length of Code1 l1(rk)
87 0.25 01010111 8
128 0.47 10000000 8
186 0.25 11000100 8
255 0.03 11111111 8
All other intensity
value rk for k ≠ 87, 0 - 8
128, 186, 255
Average number of bits required to represent each pixel is:

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 5

DIGITAL IMAGE PROCESSING 18CS741 Module 5
The number of bits used to represent each value of rk is l(rk). For fixed length 8-bit coding ie: equal to 8.
Therefore Lavg = 8 [0.25 + 0.47 + 0.25 + 0.03] = 8 x 1 = 8 bits.
Therefore the total number of bits required to represent the entire image is MNLavg = 256 x 256 x 8
= 524288 bits
If the image data (rk) is represented with variable length coding:

Intensity value (rk) Probability Variable length Code 2 Length of Code2 l2(rk)
87 0.25 01 2
128 0.47 1 1
186 0.25 000 3
255 0.03 001 3
All other intensity
value rk for k ≠ 87, 0 - 0
128, 186, 255
Average number of bits required to represent each pixel is:
Lavg = [0.25 x 2 + 0.47 x 1 + 0.25 x 3 + 0.03 x 3] = 1.8 bits.
Therefore the total number of bits required to represent the entire image is MNLavg = 256 x 256 x 1.81
= 118621 Bits
Compression ratio = CR = 524288/118621 = 4.42
Redundancy = RD = 1 – 1/4.42 = 0.774
Thus 77.4% of the data in the original 8-bit 2-D intensity array is redundant.
A natural binary encoding or fixed length encoding assigns the same number of bits to both the most and
least probable values, failing to minimize average code length and resulting in coding redundancy. Thus the
coding redundancy can be eliminated by coding the gray levels using variable length coding.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 6

DIGITAL IMAGE PROCESSING 18CS741 Module 5
SPATIAL or TEMPORAL or INTER-PIXEL REDUNDANCY
****Explain inter-pixel redundancy
Inter-pixel redundancy is due to the correlation between the neighboring pixels in an image. Most of the
pixels in 2-D intensity arrays are correlated spatially that means each pixel is similar and dependent on
neighboring pixels. The information is unnecessarily replicated in the representation of correlated pixels.
Consider the computer-generated collection of constant intensity lines as shown in figure below:

In the corresponding 2-D intensity array:

1. All 256 intensities are equally probable. The histogram of this image is uniform and is as shown in
below figure.

2. Because the intensity of each line was selected randomly, its pixels are independent of one another in
the vertical direction.
3. Because the pixels along each line are identical, they are maximally correlated (completely dependent
on one another) in the horizontal direction
In most images, pixels are correlated spatially (in both x and y) and in time. Because most pixel intensities
can be predicted reasonably well from neighboring intensities, the information carried by a single pixel is
small. Much of its visual contribution is redundant in the sense that it can be inferred from its neighbors. To

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 7

DIGITAL IMAGE PROCESSING 18CS741 Module 5
reduce the redundancy associated with spatially and temporally correlated pixels, a 2-D intensity array must
be transformed into a more efficient but usually ―non-visual‖ representation.
For example, run-lengths or the differences between adjacent pixels can be used. Transformations of this
type are called mappings. A mapping is said to be reversible if the pixels of the original 2-D intensity array
can be reconstructed without error from the transformed data set; otherwise the mapping is said to be
irreversible.
PSYCHOVISUAL REDUNDANCY or IRRELEVANT INFORMATION
Certain information simply has less relative importance than other information in normal visual processing.
This information is said to be psycho-visually redundant. It can be eliminated without significantly impairing
the quality of image perception.
Human eye and brain do not respond to all visual information with same sensitivity. Some information is
neglected during the processing by the brain. Elimination of this information does not affect the
interpretation of the image by the brain. Example of this computer-generated image is as shown in below
figure.

It appears to be a homogeneous field of gray, can be represented by its average intensity alone—a single 8-bit
value. Its elimination is possible only because the information itself is not essential for normal visual
processing. Since the elimination of psycho-visually redundant data results in a loss of quantitative
information, it is commonly referred to as quantization. This terminology is consistent with normal usage of
the word, which generally means the mapping of a broad range of input values to a limited number of output
values. •As it is an irreversible operation (visual information is lost).quantization results in lossy data
compression.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 8

DIGITAL IMAGE PROCESSING 18CS741 Module 5
IMAGE COMPRESSION MODELS
********Describe the general image compression models with a neat block diagram.
Image compression system consists of two distinct structural blocks:
1. Encoder
2. Decoder

General Image compression system model

 An input image f(x, y) is fed into the encoder, which creates a set of symbols from the input data.
After transmission over the channel, the encoded representation is fed to the decoder, where a re-
constructed output image f^(x, y) is generated.
 In general f^(x, y) may or may be an exact replica of input image f(x, y). If it is the system is error
free; if not some level of distortion is present in re-constructed image.
 The encoder is made up of a source encoder, which removes input redundancies like code, inter-pixel
or psycho-visual redundancies, and a channel encoder, which increases the noise immunity of the
source encoder’s output.
 The decoder includes a channel decoder followed by a source decoder
Both encoder and decoder consist of two relatively independent sub-blocks;
Source Encoder:
 The first stage of source encoding process is the mapper transforms input data into a (usually a non-
visual) format designed to reduce the inter-pixel redundancies in the input image. This operation is
generally reversible and may or may not reduce directly the amount of data required to represent the
image.
 Mapper transforms the image into an array of coefficients, making its inter-pixel redundancies more
accessible for compression in later stages of the encoding process.
 The second stage or quantizer block reduces the accuracy of the mapper’s output in accordance with
some pre-established fidelity criterion. This stage reduces the psycho-visual redundancies.
 The final stage of source encoding process, the symbol decoder creates a fixed or variable length code
to represent the quantizer output and maps the output in-accordance with the code.
ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 9
DIGITAL IMAGE PROCESSING 18CS741 Module 5

Source decoder:
 It contains two blocks as symbol decoder and inverse mapper. These blocks perform, in reverse
order, the inverse operation of the source encoder’s symbol decoder and mapper blocks

Channel Encoder and Decoder

 These two blocks play an important role in the overall encoding- decoding process when the channel
is prone to error or noise.
 They are designed to reduce the impact of channel noise by inserting a controlled form redundancy
into the source encoded data.
 One of the most useful channel encoding techniques was Hamming coding

MEASURING INFORMATION
Entropy:
The entropy or average information of an image is a measure of the degree of randomness in the image.
The entropy is useful in the context of image coding: it is a lower limit for the average coding length in bits
per pixel which can be realized by an optimum coding scheme without any loss of information.

The entropy of the image can be estimated by substituting the intensity probabilities of that image.
Efficiency:
Compression efficiency ɳ = H(S) / Lavg

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 10

DIGITAL IMAGE PROCESSING 18CS741 Module 5
LOSSY AND LOSSLESS COMPRESSION
The two broad categories of image compression are:
1. Lossy image compression algorithm
2. Lossless image compression algorithm
Lossless Image compression:
 The original data can be recovered exactly from the compressed data.
 Generally used for application where any difference between the compressed and the reconstructed
data cannot be allowed.
These techniques generally are composed of relatively two independent operations
1. A representation in which its inter-pixel redundancies are reduced.
2. Codification of the representation to eliminate coding redundancies.
Example: Variable length coding, Huffman coding, Arithmetic coding, LZW coding, Bit plane coding,
Lossless Predictive coding.
Lossy Compression Techniques:
 It involves some loss of information and data cannot be recovered from the same.
 These are used where some loss of data can be acceptable.
Example: Compressed Video signal is different from the original. However, we can generally obtain the
higher compression ratio than the Lossless compression methods.
Commonly used lossy compression techniques are Lossy predictive coding, Transform coding, Zonal coding,
Wavelet coding, Image compression standard. These techniques are much more effective at compression than
the Lossless methods.
HUFFMAN CODING
Most popular coding scheme used for removal of coding redundancy. It is developed by D.A. Huffman in
1952 and these are optimal codes that map one symbol to one code word. Huffman coding is used to
eliminate coding redundancy. It is a variable length coding and is used in lossless compression. When coding
the symbols of an information source individually, Huffman coding yields the smallest possible number of
code symbols per source symbol.
***********Explain the Huffman compression technique
 Huffman coding is used to eliminate coding redundancy. It is a variable length coding and is used in
lossless compression.
 Huffman coding yields the smallest possible number of code symbols per source symbol. These codes
are optimal codes that map one symbol to one code word.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 11

DIGITAL IMAGE PROCESSING 18CS741 Module 5
 Huffman encoding assigns smaller codes for more frequently used symbols (high probability) and
larger codes for less frequently (low probability) used symbols.
Procedure for Huffman coding:
1. Create a series of source reductions by ordering (Descending order) the probabilities of the symbols
under consideration and combining the lowest probability symbols into a single symbol that replaces
them in the next source reduction. This source reduction process will continue until a source with
only two symbols is obtained.
2. Code each reduced source, starting with the smallest source and working back to the original source.
The minimal length binary code for two-symbol source is the symbols 0 and 1.
3. The coding of source is then repeated for each reduced source until the original source is reached.
Example:
Apply Huffman coding technique for the following image data;
Symbol a1 a2 a3 a4 a5 a6
Probability 0.1 0.4 0.06 0.1 0.04 0.3
Ordering the probabilities of the symbols in descending order:
Symbol Probability
a2 0.4
a6 0.3
a1 0.1
a4 0.1
a3 0.06
a5 0.04
Source reduction is created by adding the lowest two probabilities into a single symbol and repeat until a
source with only two symbols is obtained.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 12

DIGITAL IMAGE PROCESSING 18CS741 Module 5
Huffman coding assignment procedure starts from the smallest source obtained in the last step and goes back
to the original source.

Thus the average length of bits required to code the symbols using Huffman coding:

Entropy H(s) = -[0.1 log20.1 + 0.4 log20.4 + 0.06 log20.06 + 0.1 log20.1 + 0.04
log20.04 + 0.3 log20.3]
= 2.1396
Efficiency = Entropy /Lavg = 2.1396 /2.2 = 97.25%

Apply Huffman coding technique to encode the following image of size 5 x 5 (3-bit image).
7 7 3 1 1
6 2 3 1 1
4 3 0 0 7
3 4 3 3 4
5 5 6 2 2
Compute the average length of the code
1. Total no, of bits transmitted
2. Compression ratio
3. Redundancy
4. Also decode the message for encoded string: 0010111011010011

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 13

DIGITAL IMAGE PROCESSING 18CS741 Module 5
Answer:

Ordering the probabilities of the symbols in descending order:

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 14

DIGITAL IMAGE PROCESSING 18CS741 Module 5
Initially for column 6 we assign binary code 0 to upper probability value of 0.56 and 1 to lower value 0.44.
For column 5 probability value 0.44 is due to the summation of 0.24 and 0.20. So first assign the code value
1 of 0.44 to both 0.24 and 0.20 then assign the binary value 0 to upper 0.24 and 1 to lower 0.20 probabilities
respectively. This process will continue until we reach the original source.

Compression ratio C = N1/N2

Where N1 = number of bits used for input image = 5 x5 x 3 = 75 bits
N2 = number of bits used for compressed image = 5 x 5 x 2.92 = 73 bits
C = 75/73 = 1.027
Redundancy R = 1- 1/C = 1 – 1/2.27 = 0.026 = 2.6%
Decode the message for encoded string: 0010111011010011
Left-to-right scan of the encoded string 0010111011010011 reveals that:
The first valid code word is 0010, which is the code for gray level value 5
The next valid code is 111, which corresponds to gray value 0. Continuing in this manner reveals the
completely decoded message to be: 50346

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 15

DIGITAL IMAGE PROCESSING 18CS741 Module 5
Compress the following 8-bit image using Huffman coding. Compute the average length of the code and
compression ratio and redundancy.
21 21 21 95 169 243 243 243
21 21 21 95 169 243 243 243
21 21 21 95 169 243 243 243
21 21 21 95 169 243 243 243
Answer:
Gray level value Frequency Probability
21 12 0.375
95 4 0.125
169 4 0.125
243 12 0.375
Source reduction:

Huffman Coding assignment:

Average length of the code LAvg = (0.375 x 1) + (0.375 x 2) + (0.125 x 3) + (0.125 x 3)

= 0.375 + 0.750 + 0.375 + 0.375
= 1.875 bits
Total Number of bits in original image data N1 = M xN x 8 bits = 8 x 4 x 8 = 256 Bits
Total Number of bits in compressed image data N2 = M xN x LAvg = 8 x 4 x 1.875 = 60 Bits
Compression ratio CR = N1/N2 = 256/60 = 4.27
Redundancy RD = 1 – 1/CR = 1 – 1/4.27 = 1 - 0.2342 = 0.7658 = 76.58%

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 16

DIGITAL IMAGE PROCESSING 18CS741 Module 5
ARITHMETIC CODING
***********Explain arithmetic coding technique with example.
 Arithmetic coding generates non-blocking codes, in which a one-to-one correspondence between
source symbols and code words does not exist. Instead entire sequence of source symbols or message
is assigned a single arithmetic code word.
 The code word itself defines an interval of real numbers between 0 and 1.
 As the number of symbols in the message increases, the interval used to represent it becomes smaller
and the number of bits required to represent the interval becomes larger.
 Each symbol of the message reduces the size of the interval in accordance with its probability of
occurrence.
Example:
Let us construct arithmetic coding for the message (string) ―SWISS‖
Step 1: Compute probability of occurrence of each symbol in message SWISS
Symbol Frequency Probability Initial Sub-
interval
S 3 0.6 [0, 0.6]
W 1 0.2 [0.6, 0.8]
I 1 0.2 [0.8, 1.0]

or the last character is encoded.

At the start of the coding process, the message is assumed to occupy the entire half-open interval [0, 1).As
shown in below figure, this interval is subdivided initially into three regions based on the probabilities of
each source symbol.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 17

DIGITAL IMAGE PROCESSING 18CS741 Module 5

Symbol S for example, is associated with subinterval [0, 0.6). Because it is the first symbol of the message
being coded, the message interval is initially narrowed to [0, 0.6).Thus in below figure the interval [0, 0.6] is
expanded to the full height of the figure and its end points labelled by the values of the narrowed range. The
narrowed range is then subdivided in accordance with the original source symbol probabilities and the
process continues with the next message symbol.

For the first symbol S in the given message:

In the interval [0, 0.6] the different symbols occupies the different range that is calculated using

d = 0.6 – 0 = 0.6
For S: = 0 + 0.6 x P(S) = 0 + 0.6 x 0.6 = 0.36

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 18

DIGITAL IMAGE PROCESSING 18CS741 Module 5
For W: = 0 + 0.6 x P(W) = 0.36 + 0.6 x 0.2 = 0.36 + 0.12 = 0.48
For I: = 0.48 + 0.6 x P(I) = 0.48 + 0.6 x 0.2 = 0.48 + 0.12 = 0.60
For the second symbol W in the given message the interval [0.36, 0.48] is divided into:
d = 0.48 – 0.36 = 0.12
For S: = 0.36 + 0.12 x P(S) = 0.36 + 0.12 x 0.6 = 0.36 + 0.072 = 0.432
For W: = 0.432 + 0.12 x P(W) = 0.432 + 0.12 x 0.2 = 0.432 + 0.024 = 0.456
For I: = 0.456 + 0.12 x P(I) = 0.456 + 0.12 x 0.2 = 0.456 + 0.024 = 0.480
For the third symbol I in the given message the interval [0.456, 0.48] is divided into
d = 0.48 – 0.456 = 0.024
For S: = 0.456 + 0.024 x P(S) = 0.456 + 0.024 x 0.6 = 0.456 + 0.0144 = 0.4704
For W: = 0.4704 + 0.024 x P(W) = 0.4704 + 0.024 x 0.2 = 0.4704 + 0.0048 = 0.4752
For I: = 0.4752 + 0.024 x P(I) = 0.4752 + 0.024 x 0.2 = 0.4752 + 0.0048 = 0.48
For the fourth symbol S in the given message the interval [0.456, 0.4704] is divided into
d = 0.4704 – 0.4560 = 0.0144
For S: = 0.456 + 0.0144 x P(S) = 0.456 + 0.0144 x 0.6 = 0.456 + 0.00864 = 0.46464
For W: = 0.46464 + 0.0144 x P(W) = 0.46464+ 0.0144 x 0.2 = 0.46464 + 0.00288 = 0.46752
For I: = 0.46752 + 0.0144 x P(I) = 0.46752 + 0.0144 x 0.2 = 0.46752+ 0.00288 = 0.4704
For the fifth symbol S in the given message the interval [0.456, 0.46464]
Any number within this sub interval [0.45600, 0.46464] can be used as code word to represent the
message SWISS.

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 19

DIGITAL IMAGE PROCESSING 18CS741 Module 5

QUESTION BANK
MODULE 5
1 What is image compression?
2 Explain coding redundancy by taking suitable example.
3 Explain inter-pixel redundancy
4 Describe the general image compression models with a neat block diagram.
5 Explain the Huffman compression technique with example
6 Explain arithmetic coding technique with example.
7
8
9
10
11

ATHMARANJAN K DEPT OF ISE, SRINIVAS INSTITUTE OF TECHNOLOGY MANGALURU Page 20