IMAGE FEATURES USING
WAVELETS AND APPLICATIONS TO
DOCUMENT IMAGE PROCESSING
Dr. S A Angadi
Professor
Department of Computer Science and Engineering
PG Centre,
Visvesvaraya Technological University, Belgaum
Contents
Document Image Processing : An Introduction
Wavelets : An Overview
Discrete Wavelet Transform and Images
Image Features using Wavelets
Applications
Document Image Processing: An Introduction
DIP/DIA is the theory and practice of recovering the symbol
structures of digital images scanned from paper or produced by
computer
DIA is a subfield of Digital Image processing
Digital images of natural objects: X-rays, fingerprints, faces, scenery,
etc. are NOT part of DIA
Digital images of symbolic objects: Postal addresses, printed articles,
forms, music sheets, engineering drawings, topographic maps belong to
DIA
Source: Scanners, printers, fax machines, hand!
Incidental text: license plates, billboards, subtitles, in photos and video
WWW ??
DIA’s grand goal is take us to the land of paperless office
Document Image Processing
200 dpi images 400 dpi images
Document Image Examples
Document Image Processing
Meter
Mark
Digital
ender’s Address Endorsem Post Mark
ent
In Case of Undeliverable as Addressed Return to Sender
Linear
Code
Delivery Address
Document Image Examples
-Postal documents
Document Image Processing
Document Image Examples-Forms
Document Image Processing
Document Image Examples-Unconstrained Text
Document Image Processing
Document Image
Examples-Graphics
Document Image Processing
Document Image
Analysis
Textual Processing Graphical Processing
Optical Page Region and
Line
Character Layout Symbol
Processing
Recognition Analysis Processing
Text Skew, blocks, Lines, curves, Filled
paragraphs corners regions
Document Image Processing
The ultimate solution of Doc. Image Processing would be for computers
to deal with paper documents as they deal with other forms of computer
media.
A file of picture elements/ pixels is the raw input data to Doc. IP/ Doc.
IA systems
The first step in document analysis is to perform processing on this
image to prepare it for further analysis. Such processing includes:
thresholding to reduce a gray-scale or color image to a binary image,
reduction of noise to reduce extraneous data, and
thinning and region detection to enable easier subsequent detection of
pertinent features and objects of interest.
This is also called pixel-level processing (also called preprocessing and
low-level processing in literature)
Document Image Processing
Document
Data Capture
Pixel Level Processing
Feature Level Analysis
Text Analysis and Recognition Graphics Analysis and Recognition
Document Image Processing
The next major step is extraction of intermediate
features that aid in the final recognition
The features include features from individual pixels or from
collection of pixels or by transforming the information in
collection of pixels (eg: RLC features, normalization and
feature extraction, shape descriptors, compactness,
asymmetry, topology, contour smoothness, Hough
transform features, DCT features DWT features etc)
The features extracted from the image are subjected to
processing, for deriving the information from text/
graphics
Documents Image Processing
There are two main types of analysis that are
applied to text in documents (Text processing)
One is optical character recognition (OCR) to
derive the meaning of the characters and words
from their bit-mapped images, and the other is page
layout analysis to discover formatting of the text,
and from that to derive meaning associated with the
positional and functional blocks in which the text is
located
Document Image Processing
Graphics recognition and interpretation is an
important topic in document image analysis since
graphics elements pervade textual material, with
diagrams illustrating concepts in the text, company
logos heading business letters, and lines separating
fields in tables and sections of text (graphics
Processing)
The regions may be intermixed
For different type of processing, different features are
essential
Document Image Features
For Document Image Analysis features employed are categorized
into,
Image Features
Textual Features
Structural Features
Image features are either extracted directly from the image (e.g. the
Density of black pixels in a region) or extracted from a Segmented
image (e.g. the number of horizontal lines in a segmented block)
Image features extracted at the level of a whole image are called
global image features;
Image features extracted from the regions of an image are called
local image features
Document Image Features
Structural features (e.g. relationships between
objects in the page) are obtained from physical or
logical layout analysis
Textual features (e.g. presence of keywords) may
be computed from OCR output or directly from
document images
Some classifiers use only image features, only
structural features, or only textual features; others
use a combination of features from several groups
Document Image Features
Image Features Structural Features Textual Features
Various levels of Density Physical Layout Textual features
Attributes of connected Logical Structures Textual features obtained
components Results of functional before layout
Column/ row gaps labeling Textual features from
Text histogram Spatial relations OCR results analysis
Location and size of cells
Line and text features
Physical layout features
All these features can be subjected to transforms to obtain more meaningful information
Wavelet Transform: An Overview
What are Transforms
Transforms are applied to raw signal/data to obtain
further information from signal/ data which is not
readily available from raw data.
Fourier Transform gives frequency content of time
domain signal.
It will also give point value representation of a
polynomial
Why do we need frequency content of a signal/ ECG
Wavelet Transform: An Overview
FT provides a signal which contains only the frequency
domain information
It does not give any information of the signal in the
time domain
When time localization of the frequency components is
needed we require a transform giving time frequency
representation
STFT, Wigner Distributions and Wavelet Transforms
provide time frequency representation
Both FT and WT are reversible transforms- one can go
from raw data to processed data and back
Wavelet Transform: An Overview
“The wavelet transform is a tool that cuts up data,
functions or operators into different frequency
components, and then studies each component with
a resolution matched to its scale”
Wavelet Transform: An Overview
tau and s are translation and scale parameters
psi(t) is the transforming function and is called the
mother wavelet
there are two main differences between the STFT and
the CWT
1. The Fourier transforms of the windowed signals are not taken,
and therefore single peak will be seen corresponding to a
sinusoid, i.e., negative frequencies are not computed
2. The width of the window is changed as the transform is
computed for every single spectral component, which is probably
the most significant characteristic of the wavelet transform
Wavelet Transform: An Overview
the CWT can be thought of as the inner product of
the test signal with the basis functions psi_(tau ,s)
(t):
Where
Wavelet Transform: An Overview
Wavelet transform is referred to as “Fourier Transform of 20th Century”
Wavelets are wavelike oscillatory signals of finite bandwidth both in Time
and in Frequency
Wavelets are basis functions of spaces with certain properties
Wavelets provide time scale(frequency) representation of non stationary
signals
Based on multiresolution approximation (MRA)
• Approximate a function at various resolutions using a scaling function,
(t)
• Keep track of details lost using wavelet functions, (t)
• Reconstruct the original signal by adding approximation and detail
coeff
Implemented by using a series of lowpass and highpass filters
• Lowpass filters are associated with the scaling function and provide approximation
• Highpass filters are associated with the wavelet function and provide detail lost in approximating the signal
Discrete Wavelet Transform
The foundations of the DWT go back to 1976 when Croiser, Esteban,
and Galand devised a technique to decompose discrete time signals.
Crochiere, Weber, and Flanagan did a similar work on coding of speech
signals in the same year, they named their analysis scheme as subband
coding
A Discrete Wavelet Transform is any Transform for which
wavelets are discretely sampled
It transforms a discrete time signal to a discrete wavelet
representation
It converts an input series x0, x1, ..xm, into one high-pass
wavelet coefficient series and one low-pass wavelet coefficient
series (of length n/2 each)
Discrete Wavelet Transform
signal
lowpass highpass
filters
Approximation Details
(a) (d)
Discrete Wavelet Transform
The procedure starts with passing this signal
(sequence) through a half band digital lowpass
filter with impulse response h[n]
Filtering a signal corresponds to the mathematical
operation of convolution of the signal with the
impulse response of the filter.
Discrete Wavelet Transform
The DWT analyzes the signal at different frequency
bands with different resolutions by decomposing the
signal into a coarse approximation and detail information
DWT employs two sets of functions, called scaling
functions and wavelet functions, which are associated
with low pass and highpass filters, respectively.
The decomposition of the signal into different frequency
bands is simply obtained by successive highpass and
lowpass filtering of the time domain signal
The original signal x[n] is first passed through a
halfband highpass filter g[n] and a lowpass filter h[n]
Discrete Wavelet Transform
After the filtering, half of the samples can be
eliminated according to the Nyquist’s rule, since
the signal now has a highest frequency of p/2
radians instead of p
The signal can therefore be subsampled by 2,
simply by discarding every other sample. This
constitutes one level of decomposition and can
mathematically be expressed as follows:
Discrete Wavelet Transform
This decomposition halves the time resolution since
only half the number of samples now characterizes the
entire signal. However, this operation doubles the
frequency resolution, since the frequency band of the
signal now spans only half the previous frequency
band, effectively reducing the uncertainty in the
frequency by half
At every level, the filtering and subsampling will
result in half the number of samples (and hence half
the time resolution) and half the frequency band
spanned (and hence double the frequency resolution)
Discrete Wavelet Transform
Discrete Wavelet Transform
In practice, such transformation will be applied
recursively on the low-pass series until the desired
number of iterations is reached
The frequencies that are most prominent in the original signal
will appear as high amplitudes in that region of the DWT
signal that includes those particular frequencies
The difference of this transform from the Fourier transform is
that the time localization of these frequencies will not be lost
However, the time localization will have a resolution that
depends on which level they appear
This procedure in effect offers a good time resolution at high
frequencies, and good frequency resolution at low frequencies
Most practical signals encountered are of this type
Discrete Wavelet Transform
One important property of the discrete wavelet
transform is the relationship between the impulse
responses of the highpass and lowpass filters. The
highpass and lowpass filters are not independent of
each other, and they are related by
where g[n] is the highpass, h[n] is the lowpass filter, and L is the
filter length (in number of points).
Discrete Wavelet Transform
Note that the two filters are odd index alternated
reversed versions of each other. Lowpass to
highpass conversion is provided by the (-1)n term.
Filters satisfying this condition are commonly used
in signal processing, and they are known as the
Quadrature Mirror Filters (QMF). The two filtering
and subsampling operations can be expressed by
Discrete Wavelet Transform
The reconstruction in this case is very easy since half band
filters form orthonormal bases
The procedure is followed in reverse order for the
reconstruction
The signals at every level are upsampled by two, passed
through the synthesis filters g’[n], and h’[n] (highpass and
lowpass, respectively), and then added
The interesting point here is that the analysis and synthesis
filters are identical to each other, except for a time reversal
Therefore, the reconstruction formula becomes (for each layer)
Discrete Wavelet Transform
However, if the filters are not ideal halfband, then
perfect reconstruction cannot be achieved.
Although it is not possible to realize ideal filters,
under certain conditions it is possible to find filters
that provide perfect reconstruction
The most famous ones are the ones developed by
Ingrid Daubechies, and they are known as
Daubechies’ wavelets
2D- Discrete Wavelet Transform
Significant lossy data reduction is possible using
DWT
How do we generalize these concepts to 2D?
2D functions images f(x,y) I[m,n]
intensity function
Reasons to take 2D-DWT of an image
Compression
Denoising
Feature extraction
Discrete Wavelet Transforms
2D-DWT of an image
We start by defining a two-dimensional scaling and
wavelet functions
s ( x, y ) ( x) ( y ) s ( x, y ) ( x) ( y )
“Subset” of scale and position based on power of two
rather than every “possible” set of scale and position in
continuous wavelet transform
Behaves like a filter bank: signal in, coefficients out
Discrete Wavelet Transform
2 D DWT for Image
Discrete Wavelet Transform
2 D DWT for Image
Discrete Wavelet Transform
2 D DWT for Image/ has applications in Image
Compression/ Image Recognition
DWT on Images
~
LL Ak 1
COLUMNS
H 1 2
ROWS ~ 2 1
H
COLUMNS
…… ~ (h)
G 1 2 LH Dk 1
ROWS
INPUT COLUMNS
……
IMAGE ~
H 1 2 HL Dk(v)1
~ 2 1
G
ROWS
~ (d)
G 1 2 D
HH k 1
COLUMNS
LLL LLH LLH
LL LH LH LH
INPUT LHL LHH LL
LHL LHH
IMAGE
HL HH HL HH HL HH
DWT on Images
Downsample columns along the rows: For each row, keep the
2 1 even indexed columns, discard the odd indexed columns
Downsample rows along the columns : For each column, keep
1 2 the even indexed rows, discard the odd indexed rows
Upsample columns along the rows: For each row, insert zeros at
2 1 between every other sample (column)
Upsample rows along the columns: For each column, insert zeros
1 2 at between every other sample (row)
DWT on Images
LL Ak 1 1 2 H
2 1 H
LH (h)
Dk 1 1 2 G
ORIGINAL
IMAGE
Dk(v)1
HL
1 2 H
2 1 G
HH D(d) 1 2 G
k 1
We perform the 2-D wavelet transform by applying 1-D wavelet
transform first on rows and then on columns.
Rows Columns
H 2
H 2 LL
G 2
f(m, n) LH
H 2
G 2
HL
G 2
HH
Integer based Wavelets
By using a so-called lifting scheme, integer-
based wavelets can be created.
Using the integer-based wavelet, one can
simplify the computation.
Integer-based wavelets are also easier to
implement by a VLSI chip than non-integer
wavelets.
Image Features using Wavelets
The wavelet transform provides an appropriate basis for image
handling because of its beneficial features
The characteristics of the wavelet transform are:
The ability to compact most of the signal’s energy into a few
transformation coefficients, which is called energy compaction
The ability to capture and represent effectively low frequency
components (such as image backgrounds) as well as high frequency
transients (such as image edges)
The variable resolution decomposition with almost uncorrelated
coefficients
The ability of a progressive transmission, which facilitates the
reception of an image at different qualities
Image Features using Wavelets
The wavelet transform of the images will lead to
computation four different types of coefficients,
namely,
Approximation Coefficients
Horizontal Coefficients
Vertical Coefficients
Diagonal coefficients
Energy Values of the transformed images at various
levels
Image Features using Wavelets
Different type of texture features can be extracted
at various levels of decomposition from wavelets,
some of the wavelets employed are,
Applications of Wavelets to Image Processing
Wavelets for Image Compression
The Discrete wavelet transform decomposes an image into a set of
successfully smaller orthogonal images. Often it is possible to coarsely
quantize or eliminate low valued coefficients without sacrificing the integrity
of the image
For a given image, you can compute the DWT of, say each row, and discard
all values in the DWT that are less then a certain threshold
We then save only those DWT coefficients that are above the threshold for
each row, and when we need to reconstruct the original image, we simply pad
each row with as many zeros as the number of discarded coefficients, and use
the inverse DWT to reconstruct each row of the original image
We can also analyze the image at different frequency bands, and reconstruct
the original image by using only the coefficients that are of a particular band
Applications of Wavelets to Image Processing
Wavelets for Image Enhancement
Since the DWT decomposes the image into components of different
size, position and orientation, you can alter the coefficients before
reconstruction so that you can attenuate certain characteristics of the
image
Image Fusion
Image fusion combines 2 or more registered images of the same
object into a single object, that in most cases is better than the
original. This is helpful in the medical imaging field where multiple
images from different machines are employed
The images are combined in the transform domain by taking the
highest amplitude coefficient then performing the inverse on the new
fused image
Word Level Script Identification of Text in Low Resolution Images of
Display Boards using Wavelet Features
Wavelet features of image for recognition
applications
Coefficients
Transforms of coefficients
Energy Coefficients etc
Word Level Script Identification of Text in Low Resolution Images of
Display Boards using Wavelet Features
Themethod distinguishes input word into five scripts namely;
Devanagari, Kannada, English, Malayalam and Tamil.
The method investigates use of
zone wise wavelet energy features
wavelet log mean deviation features and
newly obtained properties of wavelet coefficients
for script identification of text in low resolution images of
display boards.
54
City Block Distance Measure for Script Identification using Wavelet Features
Preprocessing and Feature Extraction
55
The preprocessing is done to binarize the image and generate
bounding box around it.
Zone Wise Wavelet Energy Features
The detailed coefficient Dj1 (Horizontal Energy Band) is
divided into three horizontal and four vertical zones at each
level j as shown in figure in next slide.
The three horizontal zones namely top zone, middle zone and
bottom zone covers 30%, 40% and 30% of the region of band
and all vertical zones are divided to have equal size.
Feature Extraction
56 Zone Wise Wavelet Energy Features
Three horizontal and four vertical zones of detailed coefficient Dj1
• Further, 3j horizontal and 4j vertical zone energy features are obtained, at
each level j.
•The method also computes 2 relational features as difference values
between top and middle zone, and middle and bottom again at each level j.
Feature Extraction
57
Then, the wavelet energy feature is computed from the detailed coefficient
Dj3 (Diagonal Energy Band) at every level j.
Hence, totally 10 wavelet energy features are obtained at each level j.
These features (10 at each level) are stored into feature vector X.
u1 N
Ej1h1 ( (D ( m, n )) /( MxN )
j1
m 1 n 1
u2 N
E j1h 2 ( (D
m u 1 1 n 1
( m , n )) /( MxN )
j1
M N
E j 1h 3 ( (D
m u 2 1 n 1
j1 ( m, n )) /( MxN )
Feature Extraction
58 Wavelet Log Mean Deviation Features
•The method computes totally 3j wavelet log mean deviation features at every
resolution level j, which are stored into feature vector X.
M N |Djp (m ,n )| 1)
log(
m 1n 1 Sj
LMDjp
MN
•In the current work, the value of ∂ = 0.001 is experimented
• During experiments it is observed that, the obtained values gives better representation of
texture.
•Further, 2 more additional features that model relation between detailed energy bands are
determined as stated in equations below.
•Hence, this step records 10 features (5 at each level) into feature vector X.
LMDj4 = LMDj1 - LMDj2;
LMDj5 = LMDj2 - LMDj3;
Feature Extraction
59 Wavelet Vertical Run Features
•A wavelet vertical run R(Ø,d) is defined as number of consecutive wavelet
coefficients that runs for a distance greater than or equal to a specified value
d, in a given direction Ø=90 degree (The value 90 is fixed for vertical
direction).
•And the wavelet vertical run feature WRFj2z is number of occurrences of
wavelet vertical runs in a given area or region.
•These statistical features are obtained from vertical detailed coefficient Dj2
halved into four equal sized vertical regions/zones leading to a dimension of
8 features at both decomposition levels (4 features for every level j), which
are further recorded into feature vector X.
Feature Extraction
60 Wavelet Vertical Run Features
WRFj21 WRFj22 WRFj23 WRFj24
Four vertical zones of detailed coefficient Dj2
k*zone_ size
WRFj 2 z R(Ø,d)
n1( k 1)*zone_ size
X = [Ej1h1 Ej1h2 Ej1h3 Ej1h4 Ej1h5 Ej1v1 Ej1v2 Ej1v3 Ej1v4 Ej3 LMDj1 LMDj2 LMDj3 LMDj4 LMDj5
WRFj1 WRFj2 WRFj3 WRFj4, j=1,2 ]
Knowledge Base Construction
61
•For the purpose of knowledge base construction,
the images were captured from display boards of
government offices in India. The image database
consists of 1450 Kannada, 1200 English, 900
Malyalam, 900 Tamil and 900 Devanagari script
word images of varying resolutions
•The images in the database are characterized by
variable number of characters, variable font size and
style, uneven thickness and spacing between
characters, minimal information context, small skew,
noise and other degradations.
Knowledge Bases Construction
62 •Then, 70% of the different samples from each
script are chosen to train the system. The stored
information in the knowledge base sufficiently
characterizes all variations in input and script class
separation.
•It is also noticed that, training system with more
samples will improve the performance of the system.
At the end of training four knowledge bases
WD_IMKB_KAN, WD_IMKB_ENG,
WD_IMKB_MAL, and WD_IMKB_TAM for
Kannada, English, Malyalam and Tamil Scripts are
generated.
Knowledge Bases Construction
63
•Testing is carried out for all word images of
database containing 70% trained and 30% test
samples
•The experimentation is also done to identify the
script of 14081 word images (13271 Kannada and
810 English words) of experimental data set 1 and
14252 word images (13442 Kannada and 810
English words) of data set 2
Script Class Identification
64
• Computational Strategy for Devanagari Script
Identification
•In this stage, horizontal run statistics of test word image are
used to determine whether the written word in display board
image belongs to Hindi or other scripts.
•Initially, the horizontal runs of length greater than 6 are
computed for every row of word image and are stored into a run
feature vector HRV. The vector records row number and run
length count of all runs for all rows.
•Then, the model uses linear discrimanant function D1 to classify
word image into two classes’ w1 and w2 based on run vector.
Where, w1 corresponds to Hindi script and w2 corresponds to
other scripts category.
Script Class Identification
65 City Block Distance Measure for Script Identification
•In this stage, test data instance is processed to obtain wavelet
features, and a feature vector Xt is constructed .
•Xt = [tEj1h1 tEj1h2 tEj1h3 tEj1h4 tEj1h5 tEj1v1 tEj1v2
tEj1v3 tEj1v4 tEj3 tLMDj1 tLMDj2 tLMDj3 tLMDj4 tLMDj5
tWRFj1 tWRFj2 tWRFj3 tWRFj4, j=1,2 ]
•Then, the smallest city block distance between test data
instance Xt and data set of each knowledge base is determined
to obtain distances d1, d2, d3, and d4.
Script Class Identification
66
The smallest distance between test word image and knowledge base is
used to identify the script class.
Results and Analysis
67
The effectiveness of proposed methodology for
script identification using wavelet features has
been evaluated for 33683 low resolution word
images. The images were captured from display
boards of government offices in India.
The proposed methodology has produced good
results for low resolution word images
containing text of different size, font, and
alignment with varying background.
Results and Analysis
68
The approach also identifies script of small skewed text regions.
Hence, the proposed method is robust and achieves an identification
accuracy of 92% for Kannada Script, 97.65% for English, 82.5%
for Malyalam and 87% for Tamil Script.
A closer examination of results revealed that misclassifications
arise due to minimal information content, noise and larger skew,
which affect the texture of region of text and performance of the
texture based approach.
It is also found that, if the knowledge bases are trained for all
variations and degradations, better performance can be obtained.
69
THANK YOU
Useful Link
Matlab wavelet tool using guide
http://www.wavelet.org
http://www.multires.caltech.edu/teaching/
http://www-dsp.rice.edu/software/RWT/
www.multires.caltech.edu/teaching/courses/
waveletcourse/sig95.course.pdf
http://www.amara.com/current/wavelet.html
