Chapter 5

Learning
 Outline
• Introduction to Learning
• Knowledge in Learning
• Learning from Observation
– Learning Agents
– Forms of learning
– Inductive Learning
• Neural Networks
– Biological Neurons
– Artificial Neural Networks (ANN)
 What is learning mean?
Learning − is the activity of gaining knowledge or skill by
 studying,

practicing, being taught, or experiencing something.

Learning enhances the awareness of the subjects of the study.
The ability of learning is possessed by humans, some animals, and
AI-enabled systems.

Learning is categorized as the following slide

What is learning mean?
What is learning mean?
 Machine Learning
• The question of how machines can be endowed
with the ability to acquire and robustly manipulate
commonsense knowledge is a fundamental
scientific problem.
• One of the mechanisms to represent a common
sense knowledge is through knowledge infusion.
• “knowledge infusion” is not always the best way
of providing an agent with knowledge
– impractical, tedious
– incomplete, imprecise, possibly incorrect
 Learning….
• The other mechanism is learning; learning is
important for agents to deal with unknown
environments and Changes
• in many cases, it is more efficient to train an
agent via examples, than to “manually” extract
knowledge from the examples, and “instill” it into
the agent (infusion).
• agents capable of learning can improve their
performance
 Learning….
• why learning is important for AI systems? Often, the
AI system designer does not have full knowledge of
the environment in which the agent must operate.
• In these cases the agent must discover (i.e. learn
about) the environment itself.
• For example, if we wanted to design an artificially
intelligent vehicle for exploring the surface of Mars,
we will clearly not have detailed knowledge of the
environment before the agent arrives.
• an agent tries to improve its behavior through
observation, reasoning, or reflection.
 Knowledge in Learning
• background knowledge available before a task
is attempted can increase performance or
decrease learning time considerably
• many learning schemes assume that no prior
knowledge is available
• in reality, some prior knowledge is almost
always available
– but often in a form that is not immediately usable
by the agent
 Learning from Observation

• Learning Agents
• Inductive Learning
 Learning Agents
• based on agent designs, such as reflexive, model-
based, goal-based agents
– those aspects of agents are encapsulated into the
performance element of a learning agent
• a learning agent has an additional learning
element
– usually used in combination with a critic and a
problem generator for better learning
• most agents learn from examples/observation
– Eg. inductive learning
Learning Agents
 Performance Element
• selects external actions
• collects percepts, decides on actions
• Performance element can be replaced with any
of the 4 types described in chapter 2.
 Learning Element
• responsible for making improvements
• uses knowledge about the agent and feedback
on its actions to improve performance
• The Learning Element is responsible for
suggesting improvements to any part of the
performance element.
• The input to the learning element comes from
the Critic.
– The critic analyses incoming percepts and decides
if the actions of the agent have been good or not
 Critic
• informs the learning element about the
performance of the action
• must use a fixed standard of performance
– should be from the outside
– an internal standard could be modified to improve
performance
• sometimes used by humans to justify or disguise low
performance (Reward/Punishment)
 Problem Generator
• The Problem Generator is responsible for
suggesting actions that will result in new
knowledge about the world being acquired.
• suggests actions that might lead to new
experiences
• may lead to some sub-optimal decisions in the
short run
 Forms of Learning
• what feedback is available to guide
the learning process? we can define
three types of learning process:
– supervised learning
– unsupervised learning
– reinforcement learning
 supervised learning
– an agent learns a function that matches examples
from a sample set
• each example provides an input together with the
correct output. If there is x, there will be f(x).
– a teacher provides feedback on the outcome
– An external teacher or “Critic” is used to determine
whether the network is functioning correctly.
– The important point to note is that the function
inputs and outputs are both known.
– For example, the taxi-driving agent may learn a function in
which the input is the instructor shouting “brake!” and the
output is a Boolean result (to brake or not to brake).
 unsupervised learning
• the agent learns patterns in the inputs, without actually
knowing what the correct output should be.
• For instance, it may be implicitly stated in the utility
function.
• unsupervised or self-organised learning does not require an
external teacher or critic.
• Does not require any correct answer associated with
each input pattern in the training data set.
• An example of unsupervised learning is that the taxi-
driving system may learn to recognise good traffic days
and bad traffic days. (ANN)
 Reinforcement learning
• Sometimes called as hybrid learning
• the agent does not know the exact output for an input,
but it receives feedback on the desirability of its
behavior
the feedback can come from an outside entity, the
environment, or the agent itself
the feedback is provided in the form of reward or
punishment
• For example, the taxi-driving system may
sometimes get tips from satisfied passengers. In
this case it might learn to recognise what
properties of the journeys generally lead to it
receiving tips (e.g. fast, safe, etc.).
 Inductive Learning
• The simplest type of learning: all inputs and outputs
to the function we want to learn are known, and that
we have no prior knowledge of the problem domain.
• tries to find a function h (the hypothesis) that
approximates a set of samples defining a function f
– the samples are usually provided as
input-output pairs (x, f(x))
• supervised learning method
• relies on inductive inference, or induction
– conclusions are drawn from specific instances to
more general statements
 Example Inductive Learning 1
f(x) • input-output pairs
displayed as points
in a plane
• the task is to find a
hypothesis
(functions) that
connects the points
– either all of
them, or most
of them
• various
performance
measures
– number of
x
points
connected
 Example Inductive Learning 2
f(x)
• hypothesis is a
function consisting
of linear segments
• fully incorporates
all sample pairs
• very easy to
calculate
• has discontinuities
at the joints of the
segments
• moderate predictive
performance

x
 Example Inductive Learning 3
f(x)
• hypothesis
expressed as a
polynomial
function
• incorporates all
samples
• more
complicated to
calculate than
linear segments
• no
discontinuities
x • better predictive
power
 Example Inductive Learning 4
f(x)
• hypothesis is a
linear
functions
• does not
incorporate
all samples
• extremely
easy to
compute
x • low predictive
power
Artificial Neural Network (ANN)
 Introduction to NN

• Development of Neural Networks date

back to the early 1940s.
• It experienced a rise in popularity in
the late 1980s.
• This was a result of the discovery of
new techniques and developments and
general advances in computer
hardware technology.
 Introduction to NN
• Most NNs are models of biological
neural networks.
• historically, much of the inspiration for
the field of NNs came from the desire to
produce artificial systems capable of
sophisticated, perhaps intelligent,
computations similar to those that the
human brain routinely performs, and
thereby possibly to enhance our
understanding of the human brain.
 Introduction to NN
• Most NNs have some sort of training rule.
In other words, NNs learn from examples
(as children learn to recognize dogs from
examples of dogs)

• NN are constructed and implemented to

model the human brain.

• These tasks are difficult for traditional

computers
 Some Applications of NNs
• classification
 in marketing: consumer spending pattern
classification
 In defence: radar and sonar image classification
 In agriculture & fishing: fruit and catch grading
 In medicine: ultrasound and electrocardiogram image
classification, EEGs, medical diagnosis
• recognition and identification
 In general computing and telecommunications:
speech, vision and handwriting recognition
 In finance: signature verification and bank note
verification
 Some Applications of NNs…
• assessment
– In engineering: product inspection monitoring and
control
– In defence: target tracking
– In security: motion detection, surveillance image
analysis and fingerprint matching
• forecasting and prediction
– In finance: foreign exchange rate and stock market
forecasting
– In agriculture: crop yield forecasting
– In marketing: sales forecasting
– In meteorology: weather prediction
 END