Chapter 1

Overview on Machine
Learning
Assoc. Prof. Dr. Duong Tuan Anh
Faculty of Computer Science and Engineering,
HCMC Univ. of Technology
1/2023

1
Outline
 Pattern Recognition
 Machine learning
 Related fields of pattern recognition
 Classification

2
1. Pattern recognition
 Humans are good at recognizing objects (or patterns).
 We find it difficult to write a computer program to recognize
objects.
 Ex: By analyzing sample images of faces, a program should
be able to capture the pattern specific to a face and identify it
as a face. This is pattern recognition.
 There may be several classes and we have to classify a

particular face into a certain category (or class). This is

classification.
 In pattern recognition, the term pattern is used to include all
objects that we want to classify.
 A class is a collection of objects that are similar, but not
necessarily identical, and which is distinguishable from other
classes.

3
Figure 1.1 illustrates the difference between classification
where the classes are known beforehand and classification
where classes are created after inspecting the objects.

Figure 1.1 Classification when the classes are (a) known and b)
unknown beforehand.

4
Applications of pattern
recognitions
Interest in pattern recognition has grown due to emerging
applications. These include:
 Data mining
 Biometrics
 personal identification based on physical attributes of the face, iris,
fingerprints, etc.
 Machine vision
 automatic visual inspection in an assembly line
 Character recognition

automatic mail sorting by zip code, automatic check scanners at at ATMs.
 Document recognition
 recognize whether an email is spam or not, based on the message
header and content.
 Speech recognition
 helping handicapped patients to control machines.

5
 Computer-added diagnosis
 helping doctors make diagnostic decisions based on interpreting
medical data such as ultrasound images, electrocardiograms
(ECGs) or electroencephalograms (EEGs).
 Medical imaging
 classifying cells as malignant or benign based on magnetic
resonance imaging (MRI) scans, or classify different emotional
and cognitive states from the images of brain activity in functional
MRI
 Bioinformatics
 DNA sequence analysis to detect genes related to particular
diseases.
 Remote sensing
 land use and crop yield
 Astronomy
 classifying galaxies based on their shapes

6
2. Machine learning
 What learning is?
 Learning from experience
 Remembering, adapting and generalization
 Reasoning and logical deduction
 Machine learning is about making computers modify or adapt
their actions so that their actions get more accurate, where
accuracy is measured by how well the chosen actions reflect
the correct ones.
 Through learning, computers can recognize patterns correctly.
 So machine learning “helps” pattern recognition. Without
machine learning, computers can not achieve pattern
recognition.

7
 Some paradigms of machine learning
 Supervised learning
 A training set of examples (patterns) with the correct responses
(class labels) are provided and based on this training set, the
algorithm generalizes to respond correctly to all possible
inputs. This is also called learning from examplars.
 Unsupervised learning
 Correct responses (class labels) are not provided, instead the
algorithm tries to identify similarities between the inputs so that
inputs that have something in common are categorized
together.
 The statistical approach to unsupervised learning is called

density estimation.
Note: pattern = example, instance, point, examplar

8
3. Related fields of pattern recognition

 The methods used for pattern recognition have been

developed in various fields, often independently.
 In statistics, going from particular observations to general
descriptions is called inference, learning (i.e., using training
data) is called estimating and classification is discriminant
analysis.
 In engineering, classification is called pattern recognition and
the approach is nonparametric and much more empirical.
 Other methods have their origins in machine learning, artificial
intelligence, artificial neural networks and data mining.
 We will incorporate techniques from these different fields to
give pattern recognition a more unified treatment.

9
Figure 1.2 Pattern recognition and
related fields

10
4. Classification
 Classification is often the final step in a general process. It
involves sorting objects into separate classes.
 In the case of an image, the acquired image is segmented
to isolate different objects from each other and from the
background, and the different objects are labeled.
 A typical pattern recognition system contains
 a sensor,
 a preprocessing mechanism (prior to segmentation),
 a feature extraction mechanism,
 a set of examples (training data) already classified and
 a classification algorithm.

11
 Figure 1.3 A general classification system
The feature extraction step reduces the data by measuring certain
characteristic properties or features (such as size, shape, and texture)
of the labeled objects.
These features (i.e. values of these features) are then passed to a
classifier that evaluates the evidence presented and makes a decision
regarding to class each object should be assigned, depending on
whether a value of its feature fall inside or outside the tolerance of that
class.
This process is used, e.g., in classifying lesions as benign or malignant .
12
 The quality of the acquired image depends on the resolution,
sensitivity, bandwidth and signal-to-noise ratio of the imaging
system. Pre-processing step such as image enhancement and
image restoration may be required prior to segmentation, which is
often a challenging process.
 The quality of the features is related to their ability to discriminate
examples from different classes. Examples from the same class
should have similar feature values, while examples from different
classes should have different feature values, i.e. good features
should have small intra-class variations and large inter-class
variations (Figure 1.4).

Figure 1.4 A good

feature, x, measured for
two different classes
(blue and red) should
have intra-class
variations and large
inter-class variations

13
  The features can be continuous or categorical or non-metric (i.e.
qualitative). Categorical features can either be nominal (i.e.
unordered) or ordinal.
 Humans are adept at recognize objects within an image, using size,
shape, color, and other visual clues. They can do this despite the
fact that the objects may appear from different viewpoints and under
different lightning conditions, have different sizes, be rotated or
when the images are partially obstructed from view.

Fig. 1.5 Face recognition needs to be able to handle different expressions,

lighting and occlusions 14
  The goal of the classifier is to classify new data (test data) to
one of the classes, characterized by a decision region. The
borders between decision regions are called decision
boundaries.

Figure 1.6 Classes mapped as decision regions, with decision boundaries

15
Datasets for pattern
recognition
 There are a wide variety of datasets available on the
Internet.
 One popular site is the machine learning repository at
Univ. of California, Irvine:
http://archive.ics.uci.edu/ml/datasets/

 Large datasets used for data mining tasks are available at

kdd.ics.uci.edu
www.kdnuggets.com/datasets/

kdd: knowledge discovery and data mining

16
References
 G. Dougerty, 2013, Pattern Recognition and
Classification – An Introduction, Springer
 S. Marshland, 2015, Machine Learning – An
Algorithmic Approach, Chapman & Hall/CRC.
 M. N. Murty and V. S. Devi, 2011, Pattern
Recognition – An Algorithmic Approach, Springer.

17
 Terminology
 Machine learning: học máy, pattern: mẫu, pattern recognition:
nhận dạng mẫu, feature: đặc trưng, feature extraction: rút trích
đặc trưng, classification: phân lớp, discriminant analysis: phân
tích phân biệt, training data: dữ liệu huấn luyện, test data: dữ liệu
thử, image segmentation: phân mảnh hình ảnh, decision
boundary: biên quyết định, decision region: vùng quyết định, pre-
processing: tiền xử lý, continuous feature: đặc trưng liên tục,
categorical feature: đặc trưng rời rạc, supervised learning: học có
giám sát, unsupervised learning: học không giám sát, intra-class
variation: độ sai biệt trong nội bộ lớp, inter-class variation: độ sai
biệt giữa các lớp, sensor: bộ cảm biến, artificial neural network:
mạng nơ ron nhân tạo, bioinformatics: sinh tin học, machine
vision: thị giác máy tính, biometrics: sinh trắc học, remote
sensing: viễn thám, density estimation: ước lượng mật độ,

18