GNR602
Advanced Methods in Satellite Image
Processing
Prof. B. Krishna Mohan
CSRE, IIT Bombay
[email protected] Slot 13
Lecture 23 JPEG Image Compression
IIT Bombay Slide 1
Lecture 23 Image Compression
Contents of the Lecture
Principles of Image Compression
Lossy v/s Lossless Image Compression
JPEG Image Compression
Rate Controlled JPEG Compression
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 2
What is Data and Image
Compression?
• Data compression is the representation of
information in a compact form.
• Data is a sequence of symbols taken from a discrete
alphabet.
• Still image data, that is a collection of 2-D arrays (one
for each color plane) of values representing intensity
(color) of the point in corresponding spatial location
(pixel).
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 2a
Why is an image compressible?
• Statistical redundancy:
1) Spatial correlation
a) Local - Pixels at neighboring locations have similar
intensities.
b) Global - Reoccurring patterns.
2) Spectral correlation – between color planes.
3) Temporal correlation – between consecutive frames.
• Tolerance to fidelity:
1) Perceptual redundancy.
2) Limitation of rendering hardware.
GNR602 Lecture 23 B. Krishna Mohan
Compression Ratio
compression
Compression ratio:
Relevant Data Redundancy
Example:
Types of Data Redundancy
(1) Coding Redundancy
(2) Interpixel Redundancy
(3) Psychovisual Redundancy
• Data compression attempts to reduce one or
more of these redundancy types.
Coding - Definitions
• Code: a list of symbols (letters, numbers, bits
etc.)
• Code word: a sequence of symbols used to
represent some information (e.g., gray
levels).
• Code word length: number of symbols in a
code word.
Coding - Definitions (cont’d)
N x M image
rk: k-th gray level
l(rk): # of bits for rk
P(rk): probability of rk
Expected value: E ( X ) xP( X x)
x
Coding Redundancy
• Case 1: l(rk) = constant length
Example:
Coding Redundancy (cont’d)
• Case 2: l(rk) = variable length
variable length
Total number of bits: 2.7NM
Interpixel redundancy
• Interpixel redundancy implies that pixel values are
correlated (i.e., a pixel value can be reasonably predicted
by its neighbors).
histograms
auto-correlation
f ( x) o g ( x ) f ( x) g ( x a ) da
auto-correlation: f(x)=g(x)
Interpixel redundancy (cont’d)
• To reduce interpixel redundancy, some kind of transformation
must be applied on the data (e.g., thresholding, DFT, DWT)
Example:
threshold
original
11 ……………0000……………………..11…..000…..
thresholded
(1+10) bits/pair
Psychovisual redundancy
• The human eye is more sensitive to the lower
frequencies than to the higher frequencies in
the visual spectrum.
• Idea: discard data that is perceptually
insignificant!
Psychovisual redundancy (cont’d)
Example: quantization
256 gray levels 16 gray levels 16 gray levels + random noise
add a small pseudo-random number
C=8/4 = 2:1 to each pixel prior to quantization
Measuring Information
• A key question in image compression is:
“What is the minimum amount of
data that is sufficient to describe
completely an image without
loss of information?”
• How do we measure the information content
of an image?
Measuring Information (cont’d)
• We assume that information generation is a
probabilistic process.
• Idea: associate information with probability!
A random event E with probability P(E)
contains:
Note: I(E)=0 when P(E)=1
How much information
does a pixel value contain?
• Suppose that gray level values are
generated by a random process, then rk
contains:
units of information!
(assume statistically independent random events)
How much information does an image
contain?
• Average information content of an image:
L 1
E I ( rk ) P(rk )
k 0
using
units/pixel
Entropy:
(e.g., bits/pixel)
Redundancy
• Redundancy:
(data vs info)
where:
Note: if Lavg= H, then R=0 (no redundancy)
Entropy Estimation
• It is not easy to estimate H reliably!
image
Entropy Estimation (cont’d)
• First order estimate of H:
Lavg = 8 bits/pixel R= Lavg- H
The first-order estimate provides only a
lower-bound on the compression that can be
achieved.
Estimating Entropy (cont’d)
• Second order estimate of H:
– Use relative frequencies of pixel blocks :
image
Differences in Entropy Estimates
• Differences between higher-order estimates of
entropy and the first-order estimate indicate the
presence of interpixel redundancy!
• Need to apply some transformation to deal with
interpixel redundancy!
Differences in Entropy Estimates
(cont’d)
• Example: consider pixel differences
16
Differences in Entropy Estimates
(cont’d)
• What is the entropy of the pixel differences
image?
(better than the entropy of the original image
H=1.81)
• An even better transformation should be
possible
since the second order entropy estimate is lower:
Image Compression Model
We will focus on the Source Encoder/Decoder only.
Image Compression Model (cont’d)
• Mapper: transforms data to account for interpixel redundancies.
Image Compression Model (cont’d)
• Quantizer: quantizes the data to account for psychovisual
redundancies.
Image Compression Model (cont’d)
• Symbol encoder: encodes the data to account for coding
redundancies.
Image Compression Models
(cont’d)
• The decoder applies the inverse steps.
• Note that quantization is irreversible in
general.
Fidelity Criteria
• How close is to ?
• Criteria
– Subjective: based on human observers
– Objective: mathematically defined criteria
IIT Bombay Slide 3
How are images compressed?
• Lossless compression: reversible, information preserving
• text compression algorithms,
• binary images, palette images
• Lossy compression: irreversible
• grayscale, color, video
• Near-lossless compression:
• medical imaging, remote sensing.
• Questions to answer before compression
• 1) Why do we need lossy compression?
• 2) When do we use only lossless compression?
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 3a
Lossy vs. Lossless compression
• Achievable compression
– Lossless: Modest 1:2 to 1:3 at best
– Lossy: Can be as high as 1:100
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 4
Lossy vs. Lossless compression
100%
Example
80%
60%
40%
53.4 %
20%
6.7 %
4.3 %
0%
IMAGE: CCITT-3 LENA LENA
TYPE: binary gray-scale gray-scale
METHOD: JBIG JPEG JPEG
(lossless) (lossless) (lossy)
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 4a
Rate measures
size of the compressed file C
Bitrate:
bits/pixel pixels in the image N
Compression ratio: size of the original file N k
size of the compressed file C
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 5
Distortion measures
N
1
Mean average error (MAE): MAE
N
y
i 1
i xi
N
1
Mean square error (MSE): MSE
i i
y x 2
N i 1
Signal-to-noise ratio (SNR): SNR 10 log10 2 MSE
(decibels)
Peak-signal-to-noise ratio (PSNR): PSNR 10 log10 A2 MSE
(decibels)
A is amplitude of the signal: A = 28-1=255 for 8-bits signal.
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 5a
Lossy image compression
• DPCM: Prediction error quantization
• Block Truncation Coding (BTC)
• Vector Quantization (VQ)
• Transform Coding (DCT, JPEG)
• Subband Coding
• Wavelet Coding (JPEG2000)
Data Bitstream
Transformation Quantization Encoding
Model
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 6a
Illustration of PCT – transformation highlights low variance
bands for discarding; 1st row original, 2nd row PC 1,2,3
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 6
Transform Domain Compression
•The main goal of transformation is decorrelation of data.
X Y Q(Y) C
Transformation Quantization Encoding
Model
GNR602 Lecture 23 B. Krishna Mohan
Lossy Compression
• Transform the image into some other domain
to reduce interpixel redundancy.
Example: Fourier Transform
Note that the
magnitude of the FT
decreases, as u, v
increase!
K << N
K-1 K-1
Transform Selection
• T(u,v) can be computed using various
transformations, for example:
– DFT
– DCT (Discrete Cosine Transform)
– KLT (Karhunen-Loeve Transformation) or Principal
Component Analysis (PCA)
• JPEG uses DCT for handling interpixel
redundancy.
DCT (Discrete Cosine Transform)
Forward:
Inverse:
if u=0 if v=0
if u>0 if v>0
DCT (cont’d)
• Basis functions for a 4x4 image (i.e., cosines of
different frequencies).
DCT (cont’d)
Using DFT WHT DCT
8 x 8 sub-images
yields 64 coefficients
per sub-image.
Reconstructed images
by truncating
50% of the
coefficients
DCT is a more
compact
transformation!
RMS error: 2.32 1.78 1.13
DCT (cont’d)
• Sub-image size selection:
Reconstructions
original 2 x 2 sub-images 4 x 4 sub-images 8 x 8 sub-images
IIT Bombay Slide 7
2-D DCT
N 1 N 1
2i 1k 2 j 1l
C k , l k l xi , j cos cos
j 0 i 0 2 N 2 N
• The 2-D DCT is performed as two sequential 1-D DCTs
• Complexity of DCT is O(N logN) instead of O(N4)
Image y: 1-D DCT x: 1-D DCT 2-D DCT
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 8
2-D DCT: N=4
v
0 1 2 3
GNR602 2-D Lecture 23 functions
DCT basis B. Krishna Mohan
IIT Bombay Slide 9
Zig-zag DCT coefficients ordering
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 10
Where is compression?
• DCT is reversible transformation:
Y=TX X=T-1Y
• Where is compression?
Data decorrelation and energy compactness
Quantization (lossy operation)
Statistical encoding
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 11
Example
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 12
JPEG
• JPEG = Joint Photographic Experts Group
• Lossy coding of continuous tone still images (color and
grayscale)
• Based on Discrete Cosine Transform (DCT):
0) Image is divided into block NN
1) The blocks are transformed with 2-D DCT
2) DCT coefficients are quantized
3) The quantized coefficients are encoded
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 13
JPEG: Encoding and Decoding
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 14
2-D DCT basis functions: N=8
Low High
Low Low
8x8 block
High High
GNR602
Low Lecture 23 B. High
Krishna Mohan
IIT Bombay Slide 15
2-D Transform Coding
y00 + y23
y01 y10 y12
...
DC term High frequencies
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 16
Zig-zag ordering of DCT coefficients
Converting a 2-D matrix into a 1-D array, so that the
frequency (horizontal and vertical) increases in this order
and the coefficents variance are decreasing in this order.
GNR602 Lecture 23 B. Krishna Mohan
JPEG Steps
4. Quantize the coefficients (i.e., reduce the
amplitude of coefficients that do not
contribute a lot).
Q(u,v): quantization table
One way to generate the
quantization table
• Quantization Table Q[i][j]
Example
IIT Bombay Slide 17
Example of DCT for image block
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 18
Default quantization matrix Q
Examples: 236/16 15
-22/11 -2
yq(k,l)=round[y(k,l)/Q(k,l)]
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 19
Quantization of DCT coefficients:
Example
Ordered DCT coefficients: 15,0,-2,-1,-1,-1,0,0,-1,-1, 54{’0’}.
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 20
Dequantization
z (k,l)=yq(k,l)·Q(k,l)
Examples: 236/16 15
-22/11 -2
GNR602 Lecture 23 B. Krishna
Original Mohan
DCT block
IIT Bombay Slide 21
Inverse DCT
Original
GNR602 Lecture 23 B. Krishna Mohan
Original block
IIT Bombay Slide 22
Encoding of quantized DCT coefficients
• Ordered data: 15,0,-2,-1,-1,-1,0,0,-1,-1, 54{’0’}.
• Encoding: DC: ? AC: ?
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 23
Encoding of quantized DCT
coefficients
• DC coefficient for the current block is predicted of
that of the previous block, and error is coded using
Huffman coding
• AC coefficients:
(a) Huffman code, arithmetic code for non-zeroes
(b) run-length encoding: (number of ’0’s, non-’0’-symbol)
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 24
Suitability of JPEG Method
• The disadvantage with the present JPEG
compression technique is that the
compression rate is arbitrary.
• As the main source of compression is the
quantization table, an optimal quantization
table can help to achieve a desired
compression rate with good image quality.
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 25a
JPEG LOSSY COMPRESSION
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 25
Performance of JPEG algorithm
8 bpp 0.6 bpp
0.37 bpp 0.22 bpp
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 26
Rate control in JPEG Method
• When is a precise compression rate essential?
– Maximum quality to be preserved
– Full capacity of communication channel to be
utilized
– Data rate should not exceed the channel capacity
Key requirement in space applications
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 26a
How to control rate of
compression with JPEG?
• Quality factor is the only way to compress less
or more with JPEG.
• How can the desired rate of compression be
specified in JPEG?
• What is the implication when the rate of
compression is user specified on JPEG
technique?
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 27
Quality Factor in JPEG
• The information loss happens during the
quantization phase
• In normal case quantization table coefficients
are scaled up or down based on user-specified
• The precise rate cannot be predicted based on
the quality factor
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 33
Uncompressed CR = 3.5
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 34
Uncompressed CR =3.5
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 35
Uncompressed CR = 3.5
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 36
Uncompressed CR = 3.5
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 37
Uncompressed CR = 3.5
GNR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 39
POWAI (8 BIT)
(Large Size)
NR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 40
POWAI (8 BIT); (Large Size) CR=6
NR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 41
Original CR = 6
NR602 Lecture 23 B. Krishna Mohan
IIT Bombay Slide 42
Sample Quantization Table CR=6
2 8 9 13 13 21 22 41
8 10 12 13 21 22 40 43
10 12 15 20 23 40 44 55
11 15 20 23 39 46 54 55
17 17 25 35 46 54 56 83
17 25 35 49 51 57 63 88
28 32 50 51 59 61 104 115
30 50 50 59 61 105 112 127
NR602 Lecture 23 B. Krishna Mohan
1 1 1 1 1 2 3 5
1 1 1 1 2 3 5 5
1 1 2 2 3 5 5 7
1 2 2 3 5 5 6 7
2 2 3 4 6 6 7 23
2 3 4 6 6 7 20 32
3 4 6 6 9 16 38 68
Uncompressed MUMBAI(11 BIT) CR = 3.5 4 6 6 12 15 65 67 71
2 8 9 13 13 21 22 41
8 10 12 13 21 22 40 43
10 12 15 20 23 40 44 55
11 15 20 23 39 46 54 55
17 17 25 35 46 54 56 83
17 25 35 49 51 57 63 88
28 32 50 51 59 61 104 115
30 50 50 59 61 105 112 127
CR = 6 CR = 9.5
IIT Bombay Slide 43
Summary
• JPEG compressors and decompressors can be used for
compressing remotely sensed data as the Compression ratio
(CR) will seldom be high and hence JPEG gives satisfactory
performance in terms of maintaining image characteristics.
• As JPEG works with 8 by 8 block of pixels at a time the
implementation in hardware for JPEG is amenable to onboard
satellite platforms
• The JPEG process maintains a small Mean Squared Error
(MSE) at different compression ratios tested in the range
from 3.0 to 15.0.
NR602 Lecture 23 B. Krishna Mohan
Contd…
