INTELLIGENT CONTROL SYSTEM

PART 1:
Neural
Network
Instructor: Dr. Dang Xuan Ba
Email: [email protected]

Department of Automatic Control

 Chapter 1: Introduction of Neural
network

Chapter 2: Single-layer
Feedforward Neural Network

Chapter 3: Multi-layer Neural

Content Network

Chapter 4: RBF neural Network

Chapter 5: Several Applications.

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 2

 CHAPTER 1 – INTRODUCTION OF
NEURAL NETWORK

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 3

Chapter 1: Introduction of Neural Network
1. Biological neurons (Cấu trúc tế bào thần kinh sinh học)/Structure

Fig. 1.1 A biological neuron

Cell body (soma): thân tế bào Human brain consists of nearly 1011 biological neurons.
Axon: trục dẫn đầu ra
Dendrite: nhánh dẫn đầu vào
Synapse: khớp thần kinh
One neuron could be connected to up to 104 other
Excitatory: Khớp kích thích
neurons.
Inhibitory: Khớp ức chế

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 4

Chapter 1: Introduction of Neural Network
1. Biological neurons (Cấu trúc tế bào thần kinh sinh học)/Working principle

Cell body (soma): thân tế bào

Axon: trục dẫn đầu ra
Dendrite: nhánh dẫn đầu vào
Synapse: khớp thần kinh
Excitatory: Khớp kích thích
Inhibitory: Khớp ức chế

Fig. 1.1 A biological neuron

The cell body is the central unit processor.

If sum of all incoming signals is greater than its threshold level, the cell body will send an
impulse to the axon.

otherwise, no any action.

The biological neuron model is a foundation of artificial neuron.

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 5

Chapter 1: Introduction of Neural Network
1. Biological neurons (Cấu trúc tế bào thần kinh sinh học)/Characteristics

A neural network is characterized by three

following factors:
 Structure of neurons

 Interconnection of the whole network

 Learning method

Diverse personalities are made by various structure and different number of biological neurons.
Part I: Neural Network Presenter: Dr. Dang Xuan Ba 6
Chapter 1: Introduction of Neural Network
2. Artificial Neural Network (Mạng thần kinh nhân tạo)

The first artificial neuron was proposed by Warren McCulloch and Walter Pitts in 1943
in paper of “A Logical Calculus of Ideas Immanent in Nervous Activity.”

A neural network is characterized by three following factors:

 Structure of neurons

 Interconnection of the whole network

 Learning method

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 7

Chapter 1: Introduction of Neural Network
2. Artificial Neural Network (Mạng thần kinh nhân tạo)

The first artificial neuron was proposed by Warren McCulloch and Walter Pitts in 1943
in paper of “A Logical Calculus of Ideas Immanent in Nervous Activity.”

2.1 Structure of a neuron Input (dendrite)

x1
w1 Output (Axon)
…
wi y = a(net )
y
xi net = f ( x) − b

… wn
Soma (Cell body)
xn
b
Threshold
Fig. 2.2.1: An artificial neuron

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 8

Chapter 1: Introduction of Neural Network
2.1 Structure of a neuron
Input (dendrite)
Input signal (x) x1
w1 Output (Axon)
Weight matrix (w) …
(Bộ trọng số)
Output signal (y) wi y = a(net )
y
xi net = f ( x) − b

Bias or threshold (b) … wn

Soma (Cell body)
(Mức ngưỡng)
Integrated function (net) xn
(Hàm tích hợp) b
Activation function (a(net)) Threshold
(Hàm tác động) Fig. 2.2.1: An artificial neuron

x =  x1 , x2 ,..., xn 
T

w =  w1 , w2 ,..., wn 
T

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 9

Chapter 1: Introduction of Neural Network
2.1 Structure of a neuron/Integration function (Hàm tích hợp)

Linear function (Hàm tuyến tính)

 n   n +1 
T

 i =1



net = f ( w, x) − b = w x − b =  wi xi  − b =  wi xi 

 i =1



Quadratic function (Hàm toàn phương)
n n +1
net = f ( w, x) − b = ( 
j =1
wj x j ) − b =
2
j =1
wj x j2

Spherical function (Hàm cầu)

 n 
(w )  − b = ( w − x )T ( w − x ) − b = w − x 2 − b
2
net = f ( w, x) − b =   −2 − xj
 j
 2
 j =1 

…………………………………….

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 10

Chapter 1: Introduction of Neural Network
2.1 Structure of a neuron/Activation function (Hàm tác động)

Threshold activation functions (Hàm tác động ngưỡng):

Binary threshold function - Step function – hardlim function
(Hàm ngưỡng nhị phân - Hàm bước hoặc Hàm giới hạn cứng)
1 if net  0
a (net ) = 
0 if net  0

Bipolar threshold function - Signum function - symmetric hardlim

(Hàm ngưỡng lưỡng cực - Hàm dấu hoặc Hàm giới hạn cứng đối xứng)

 1 if net  0
a (net ) = 
−1 if net  0

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 11

Chapter 1: Introduction of Neural Network
2.1 Structure of a neuron/Activation function (Hàm tác động)

Linear functions (Hàm tác động tuyến tính):

Pure linear function - identity function
(Hàm tuyến tính, hàm thuần tuyến tính – Hàm dốc đơn vị)

a (net ) = net

Piecewise linear function – Linear threshold function

(Hàm tuyến tính từng phần - hoặc hàm ngưỡng tuyến tính
– Hàm tuyến tính bão hòa)

amax if net  amax


a (net ) = net
a
 min if net  amin

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 12

Chapter 1: Introduction of Neural Network
2.1 Structure of a neuron/Activation function (Hàm tác động)

Squashing functions – Sigmoidal function:

(Hàm tác động nén – S-shaped function):
Unipolar Sigmoidal function – logsig function
(Hàm Sigmoid đơn cực, hàm nén đơn cực, hoặc hàm logsig)

1
a (net ) =
1 + e −  net

Bipolar Sigmoidal function – Tangent hyperbolic function - Tansig function

(Hàm Sigmoid lưỡng cực, hàm nén lưỡng cực, hoặc hàm tansign)

2 1 − e −  net
a (net ) = −  net
−1 =
1+ e 1 + e −  net

…………………………………..

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 13

Chapter 1: Introduction of Neural Network

A neural network is characterized by three following factors:

 Structure of neurons

 Interconnection of the whole network

 Learning method

Diverse personalities are made by various structure and different number of biological neurons.
Part I: Neural Network Presenter: Dr. Dang Xuan Ba 14
Chapter 1: Introduction of Neural Network
2.2 Connection in a neural network (Kết nối trong mạng thần kinh nhân tạo)

x1 y1
x2 y2
x3
.. ... ..
. .y
n
xm
Input layer Hidden layer Output layer

Single-layer feedforward network Multiple-layer feedforward network

Single-layer feedback network Input layer Hidden layer Output layer

Multiple-layer recurrent network

………………………………………………..
Part I: Neural Network Presenter: Dr. Dang Xuan Ba 15
Chapter 1: Introduction of Neural Network

A neural network is characterized by three following factors:

 Structure of neurons

 Interconnection of the whole network

 Learning method

Diverse personalities are made by various structure and different number of biological neurons.
Part I: Neural Network Presenter: Dr. Dang Xuan Ba 16
Chapter 1: Introduction of Neural Network
2.3 Learning (training) method (Phương pháp học – hoặc huấn luyện mạng)

Parameter learning (học thông số) Structure learning (học cấu trúc)

Supervised learning (học có giám sát)

Reinforcement learning (học củng cố)

Unsupervised learning (Học không có giám sát)

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 17

Chapter 1: Introduction of Neural Network
2.3 Applications (Các ứng dụng)

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 18

Chapter 1: Introduction of Neural Network

How to choose a good neural network for a

particular problem?

How to choose the best neural network for

all problems?

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 19

 End of Chapter 1

Part I: Neural Network Presenter: Dr. Dang Xuan Ba 20