Select Collage

Computer Science
CHAPTER 2
Intelligent Agents

By Samuel M. DEC. 2022

 Outline
2

 Agents and environments

 Rationality
 PEAS (Performance measure, Environment,
Actuators, Sensors)
 Environment types
 Agent types

Artificial Intelligence a modern approach

 Architecture of IoT (1)
3

►IoT device can be explained as a network of things that consists of

hardware, software, network connectivity, and sensors.
►The architecture of IoT devices consists of four major components

1. Sensing layer
2. Network layer
3. Data processing layer and
4. Application layers.
Architecture of IoT
 Architecture of IoT
(2)
1. Sensing Layer− The main purpose of the sensing layer
is to identify any phenomena in the devices’ peripheral and
obtain data from the real world.
► This layer consists of several sensors.
►Using multiple sensors for applications is one of the primary
features of IoT devices.
►Sensors in IoT devices are usually integrated through
sensor hubs.
►A sensor hub is a common connection point for multiple sensors that
accumulate and forward sensor data to the processing unit of a
device.
 Architecture of IoT (3)

► Sensors in IoT devices classified into three categories:

A.Motion Sensors: Motion sensors measure the change in
motion as well as the orientation of the devices. linear and
angular motions.
B.Environmental Sensors: Sensors such as Light sensors,
Pressure sensors, etc. are embedded in IoT devices to sense
the change in environmental parameters in the device’s
peripheral. e.g. home automation systems, smart locks, smart lights.
C. Position sensors: Position sensors of IoT devices deal
with the physical position and location of the device. E.g.
magnetic sensors and Global Positioning System (GPS) sensors.
19
Architecture of IoT
(4)
2. Network Layer− The network layer acts as a communication
channel to transfer data, collected in the sensing layer, to other
connected devices.
►In IoT devices, the network layer is implemented by using
diverse communication technologies (e.g. Wi-Fi, Bluetooth,
Zigbee, Z-Wave, LoRa, cellular network, etc.) to allow data flow
between other devices within the same network.
3. Data Processing Layer− The data processing layer consists
of the main data processing unit of IoT devices.
►This layer may share the result of data processing with other
connected devices via the network layer.
 Architecture of IoT (5)

4. Application Layer− The application layer implements

and presents the results of the data processing layer to
accomplish disparate applications of IoT devices.
►The application layer is a user-centric layer that executes
various tasks for the users.
►There exist diverse IoT applications, which include smart
transportation, smart home, personal care, healthcare, etc.
 Agents
9

• An agent is anything that can be viewed as

perceiving its environment through sensors and
acting upon that environment through actuators

• Human agent:
– eyes, ears, and other organs for sensors;
– hands, legs, mouth, and other body parts for actuators

• Robotic agent:
– cameras and infrared range finders for sensors
– various motors for actuators

Artificial Intelligence a modern approach

 Agents and environments
10

• The agent function maps from percept histories to

actions:
[f: P*  A]

• The agent program runs on the physical architecture to

produce f
• agent = architecture + program
Artificial Intelligence a modern approach
 Agents and environments

 The agent view is really quite generic.

 In a sense, all areas of engineering can be seen as designing
artifacts that interact with the world. AI operates that end of the
spectrum, where the artifacts use significant computational
resources and the task and environment requires non-trivial
decision making.
 But, the definition of “agents” does technically also include, e.g.,
calculators or cars, artifacts with very limited to no intelligence.
(Self-driving cars come closer to what we view as intelligent
agents.)
 Next: definition of rational agents. Sufficiently complex rational
agents can be viewed as “intelligent agents.”
Agent / Robot E.g., vacuum-cleaner world

iRobot Corporation

Percepts: location and contents, Founder Rodney Brooks (MIT)

e.g., [A, Dirty]

Actions: Left, Right, Suck, NoOp

 Vacuum-cleaner world
13

 Percepts: location and contents, e.g., [A,Dirty]

 Actions: Left, Right, Suck, NoOp
 Agent’s function  look-up table
 For many agents this is a very large table

Artificial Intelligence a modern approach

 7

Rational agents
• An agent should strive to "do the right thing", based on what it can perceive and the
actions it can perform.

• The right action is the one that will cause the agent to be most successful

• Performance measure: An objective criterion for success of an agent's behavior

• E.g., performance measure of a vacuum-cleaner agent could be amount of dirt

cleaned up, amount of time taken, amount of electricity consumed, amount of noise
generated, etc.
• As a general rule, it is better to design performance measures according to what one
actually wants in the environment. Rather than according to how one thinks the
agent should behave (amount of dirt cleaned vs a clean floor)

• A more suitable measure would reward the agent for having a clean floor
 Rational Agent
• In summary what is rational at any given point depends on four
things.
I. Perception:
Everything that the agent has perceived so far concerning the
current scenario in the environment.
II. Prior Knowledge:
What an agent already knows about the environment
III. Action:
The actions that the agent can perform back to the
environment
IV. Performance measure:
The performance measure degrees of success of the agent.
Designing an intelligent agent, one has to remember
PEAS (Performance, Environment, Actuators, Sensors) framework.
 10

Back to Vacuum-Agent
• Percepts:
Location and status,
e.g., [A,Dirty]
• Actions:
Left, Right, Suck, NoOp

function Vacuum-Agent([location,status]) returns an action

• if status = Dirty then return Suck
• else if location = A then return Right
• else if location = B then return Left

• Is this agent rational?

– Depends on performance measure, environment properties
 12

Vacuum cleaner agent

• Same agent would be irrational under different circumstances

– once all dirt is cleaned up it will oscillate needlessly back and forth.
– If the performance measure includes a penalty of one point for each
movement left or right, the agent will fare poorly.
– A better agent for this case would do nothing once it is sure that all the
squares are clean.
– If the clean squares can become dirty again, the agent should occasionally
check and clean them if needed.
– If the geography of the environment is unknown the agent will need to
explore it rather than stick to squares A and B
 13

Rational agents

• Rationality is distinct from omniscience (all-knowing with

infinite knowledge)
• Rationality maximizes expected performance while
perfection maximizes actual performance
• Agents can perform actions in order to modify future
percepts so as to obtain useful information (information
gathering, exploration)
• An agent is autonomous if its behavior is determined by its
own experience (with ability to learn and adapt)
 14

Specifying the task environment (PEAS)

• PEAS:
– Performance measure,
– Environment,
– Actuators,
– Sensors
• In designing an agent, the first step must always
be to specify the task environment (PEAS) as fully
as possible
 15

Specifying the task environment

• PEAS: Performance measure, Environment, Actuators,
Sensors
• P: a function the agent is maximizing (or minimizing)
– Assumed given
– In practice, needs to be computed somewhere
• E: a formal representation for world states
– For concreteness, a tuple (var1=val1, var2=val2, … ,varn=valn)
• A: actions that change the state according to a transition
model
– Given a state and action, what is the successor state
(or distribution over successor states)?
• S: observations that allow the agent to infer the world state
– Often come in very different form than the state itself
– E.g., in tracking, observations may be pixels and state variables 3D
coordinates
 16

PEAS Example: Autonomous taxi

• Performance measure
– Safe, fast, legal, comfortable trip, maximize
profits
• Environment
– Roads, other traffic, pedestrians, customers
• Actuators
– Steering wheel, accelerator, brake, signal, horn
• Sensors
– Cameras, LIDAR, speedometer, GPS,
odometer, engine sensors, keyboard
 17

Another PEAS example: Spam filter

• Performance measure
– Minimizing false positives, false negatives
• Environment
– A user’s email account, email server
• Actuators
– Mark as spam, delete, etc.
• Sensors
– Incoming messages, other information about
user’s account
 19

PEAS for a satellite image analysis system

• Performance measure: correct image categorization

• Environment: downlink from orbiting satellite

• Actuators: display categorization of scene

• Sensors: color pixel arrays

 22

PEAS for Interactive English tutor

• Performance measure: Maximize student's score on test

• Environment: Set of students

• Actuators: Screen display (exercises, suggestions, corrections)

• Sensors: Keyboard
 Classes of Environments
 Actions are done by the agent on the environment.
 Environments provide percepts to an agent.
 Agent perceives and acts in an environment.
 Hence in order to design a successful agent , the
designer of the agent has to understand the type of the
environment it interacts with.

Properties of Environments:
Fully Observable vs. Partially Observable
Deterministic vs. Stochastic
Discrete vs. Continuous
Static vs. Dynamic
Single agent vs. multiagent:
 Fully Observable vs. Partially Observable
 Does the agent’s sensory see the complete state of the
environment?
 If an agent has access to the complete state of the environment,
then the environment is accessible or fully observable.

 An environment is effectively accessible if the sensors

detect all aspects that are relevant to the choice of
action.
 Taxi driving is partially observable
 Any example of fully observable?
 Example: Chess.
 Deterministic vs. Stochastic
 Is there a unique mapping from one state to another
state for a given action?
 The environment is deterministic if the next state of
the environment is completely determined by
 the current state of the environment and
 the actions selected by the agents.
 Non deterministic = the next state has some
uncertainty associated with it.
 Taxi driving is non-deterministic (i.e. stochastic)
 Any example of deterministic?

27
 Discrete vs. Continuous

• Are the distinct percepts & actions limited or unlimited?

– If there are a limited/finite number of distinct,
clearly defined percepts and actions, we say the
environment is discrete.
– time moves in fixed steps, usually with one
measurement per step
– Continuous - signal constantly coming into
sensors, actions continually changing
• Taxi driving is continuous whereas Game of chess is an example of
discrete
 Static vs. Dynamic
 Can the world change while the agent is thinking?
 If the environment can change while the agent is choosing an
action, the environment is dynamic otherwise it is static.

 Taxi driving is dynamic

 Expert system is static unless knowledge changes.

Single agent vs. multiagent:

 An agent operating by itself in an
environment is single agent
 Examples: Crossword is a single agent while
chess is two-agents
 37

Agent types

• Rather than a table how we can produce rational

behavior from a small amount of code

• Four basic types in order of increasing generality:

– Simple reflex agents
– Model-based reflex agents
– Goal-based agents
– Utility-based agents
 Simple Reflex Agents
 Works by finding a rule whose condition matches the current
situation (as defined by the percept) and then doing the
action associated with that rule.
 Choose actions based on the current percept, ignoring
the rest of percept history
 No memory
 Simple but limited intelligence.
 Will only work if the environment is fully observable
otherwise infinite loops may occur.
 Fails if the environment is partially observable.
 Implemented through condition-action rules.
 40

Simple reflex agents

function SIMPLE-REFLEX-AGENT(percept) returns an action

static: rules, a set if condition-action rules
state <-- INTERPRET_INPUT(percept)
rule <-- RULE_MATCH(state, rules)
action <-- RULE_ACTION[rule]
return action
 41

Model-based reflex agents

 The agent should keep track of the part of the world it can't see now
 The agent should maintain some sort of internal state that depends
on the percept history and reflects at least some of the unobserved
aspects of the current state
 Updating the internal state information as time goes by requires two
kinds of knowledge to be encoded in the agent program
 Information about how the world evolves independently of
the agent
 Information about how the agent's own actions affects the world

 Model of the world – model based agents

 42

Model-based reflex agents

function REFLEX-AGENT-WITH-STATE(percept) returns an action

static: state, a description of the current world state
rules, a set of condition-action rules
action, the most recent action, initially none
state <-- UPDATE_INPUT(state, action, percept)
rule <-- RULE_MATCH(state, rules)
action <-- RULE_ACTION[rule] return
action
 44

Goal-based agents
• Knowing about the current state of the environment is not
always enough to decide what to do (e.g. decision at a road
junction)
• The agent needs some sort of goal information that
describes situations that are desirable
• The agent program can combine this with information
about the results of possible actions in order to choose
actions that achieve the goal
• Usually requires search and planning
 45

Goal-based agents
 46

Goal-based agents vs reflex-based agents

 Although goal-based agents appears less efficient, it is

more flexible because the knowledge that supports its
decision is represented explicitly and can be modified
 On the other hand, for the reflex-agent, we would have
to rewrite many condition-action rules
 The goal based agent's behavior can easily be
changed
 The reflex agent's rules must be changed for a new
situation
 47

Utility-based agents

 Goals alone are not really enough to generate high quality

behavior in most environments – they just provide a
binary distinction between happy and unhappy states
 A more general performance measure should allow a
comparison of different world states according to exactly
how happy they would make the agent if they could be
achieved
 Happy – Utility (the quality of being useful)
 A utility function maps a state onto a real number which
describes the associated degree of happiness
 49

Learning agents

 Turing – instead of actually programming intelligent

machines by hand, which is too much work, build
learning machines and then teach them
 Learning also allows the agent to operate in initially
unknown environments and to become more competent
than its initial knowledge alone might allow.
 Learning element – responsible for making improvements
 Performance element – responsible for selecting external
actions (it is what we had defined as the entire agent
before)
 50

Learning agents
 Learning element uses feedback from the critic on how the agent is
doing and determines how the performance element should be
modified to do better in the future
 Problem generator is responsible for suggesting actions that will
lead to a new and informative experiences
 51

THANK YOU