ELE5211: Advanced Topics in CE
Module Five
(Multimedia – Compression Algorithms)
Tutor: Dr. Hassan A. Bashir1
Information Theory and
Compression Algorithms
2
Information Theory and
Compression Algorithms
3
Information Theory (Shannon Theory)
4
Information Theory (Shannon Theory)
5
Entropy and Code Length
6
Run-Length Coding (RLC)
7
Run-Length Coding (RLC)
Run length encoding (RLC) is a technique that is not
so widely used these days
but it is a great way to get a feel for some of the
issues around using compression.
Imagine we have the following simple black and
white image.
One very simple way a computer can store this image in binary is by using a
format where ‘1' means white and ‘0' means black
This is a "bitmap", because we've mapped the pixels onto the values of bits
Using this method, the above image would be represented in the following way:
8
Run-Length Coding (RLC)
100111101111001 1, 2, 4, 1, 4, 2, 1
011111000111110 0, 1, 5, 3, 5, 1
111110000011111 5, 5, 5
111100000001111 ….
111000000000111
….
Can we represent the same image using fewer bits,
but still be able to reconstruct the original image?
Yes, we can. One of the many methods is called run length encoding.
Replace each row with numbers that say how many consecutive pixels
are the same colour,
Always starting with the number of white pixels.
For example, the first row in the image above contains one white, two black,
four white, one black, four white, two black, and one white pixel. 9
Decompression of RLC
Exercise 1: 4, 11, 3
4, 9, 2, 1, 2
Can you decompress the following code? 4, 9, 2, 1, 2
How many pixels were there in the original image? 4, 11, 3
How many numbers were used to represent those 4, 9, 5
pixels? 4, 9, 5
How much space have we saved using this alternate 5, 7, 6
representation, and how can we measure it? 0, 17, 1
1, 15, 2
RLC Usage
The main place that black and white scanned images are used now is on fax
machines, which use this approach to compression. One reason that it works so
well with scanned pages is that the number of consecutive white pixels is huge.
In fact, there will be entire scanned lines that are nothing but white pixels. A
typical fax page is 200 pixels across or more, so replacing 200 bits with one
10
number is a big saving.
Variable Length Coding (VLC)
11
Variable Length Coding (VLC)
12
Variable Length Coding (VLC)
13
Variable Length Coding
14
Variable Length Coding
15
Huffman Coding
16
Huffman Coding
17
Huffman Coding
18
Properties of Huffman Coding
19
Fixed vs. Variable Length Coding
20
Exercise: Shannon vs. Huffman Coding
Shannon-Fano: 89 bits
Huffman: 87 bits
Fix Length Coding : 117 bits
21
Extended Huffman Coding
22
Extended Huffman Coding
23
Adaptive Huffman Coding
24
Adaptive Huffman Coding
25
Adaptive Huffman Coding
(Tree Updating)
26
27
Adaptive Huffman Coding
28
Adaptive Huffman Coding
29
Adaptive Huffman Coding
30
31
Adaptive Huffman Coding
32
Dictionary-based Coding
• Lempel–Ziv–Welch is a universal lossless data compression
algorithm created by Abraham Lempel, Jacob Ziv, and Terry Welch.
• LZW compression is the compression of a file into a smaller file
using a table-based lookup algorithm.
• LZW compression works by:
• reading a sequence of symbols,
• grouping the symbols into strings, and
• converting the strings into codes.
• Because the codes take up less space than the strings they
replace, we get compression.
33
Dictionary-based Coding
• Two commonly-used file formats in which LZW compression is
used are
• the GIF image format served from Web sites and
• the TIFF image format.
• Both ZIP and LZW are lossless compression methods.
• That means that no data is being lost in the compression,
unlike a lossy format like JPG.
• You can open and save a TIFF file as many times you like without
degrading the image.
• If you try that with JPG, the image quality will deteriorate more
each time.
34
Dictionary-based Coding
35
36
Dictionary-based Coding
37
38
Dictionary-based Coding
39
40
Dictionary-based Coding
(LZW - Remarks)
41
Arithmetic Coding
• Arithmetic coding (AC) is a form of entropy encoding used in
lossless data compression.
• Normally, a string of characters such as the words "hello there" is
represented using a fixed number of bits per character, as in the
ASCII code.
• Comparison of AC with Huffman
• Arithmetic Coding is superior to Huffman coding in the sense
that it can assign a fractional number of bits for the codewords
of the symbols, whereas
• in Huffman coding an integral number of bits have to be
assigned to a codeword of a symbol.
42
Arithmetic Coding
43
Arithmetic Coding
44
Arithmetic Coding
45
Arithmetic Code for: CAEE$
46
Arithmetic Coding
47
Arithmetic Coding
48
Arithmetic Coding
For the above example, low = 0.33184, high = 0.3322.
If we assign 1 to the 1st binary fraction bit, it would be 0.1 in binary,
and its decimal value is:
value(code) = value(0.1) = 0.5 > high
Hence, we assign 0 to the first bit. Since value(0.0) = 0 < low, the
while loop continues.
Assigning 1 to the 2nd bit makes a binary code 0.01 and value(0.01) =
0.25, which is less than high, so it is accepted. Since it is still true that
value(0.01) < low, the iteration continues.
Eventually, the binary codeword generated is: 0.01010101
which is 2-2+2-4+2-6+2-8 = 0.33203125 49
Arithmetic Coding
50
Arithmetic Coding
51
Lossless Image Compression
52
Lossless Image Compression
53
Lossless JPEG
54
Lossless JPEG
55
Lossless JPEG
56
Lossless JPEG
57
Homework
Use Arithmetic Coding to encode your Surname
58
