8/5/2014
INTRODUCTION TO SOFT
COMPUTING
Harshali Patil
Introduction
Evolution of Computing
Soft computing constituents
From conventional AI to Computational Intelligence
Machine Learning basics
8/5/2014
Problem Solving
Reasoning
New
Problem
1) What kind
of similar
problems you
have solved?
4) Is it
necessary to
retain the
knowledge?
Solution
2) How to reuse
the previous
knowledge?
3) Proposed
solution is
valid or not?
Case Based Reasoning
8/5/2014
Artificial Intelligence
If intelligence can be induced in machines it is
called as artificial intelligence.
Soft computing is a part of artificial intelligent
techniques
Closed related to machine
intelligence/computational intelligence
Soft Computing
Neural Networks
Neuro+ Derivative- =
Fuzzy
Free
Computing
Optimization
Soft Computing
Fuzzy Inference
systems
8/5/2014
What is Soft Computing?
The idea behind soft computing is to model cognitive
behaviour of human mind.
Soft computing is foundation of conceptual
intelligence in machines.
Unlike hard computing , Soft computing is tolerant of
imprecision, uncertainty, partial truth, and
approximation.
Soft Computing
According to Prof.
Prof. Zadeh:
Zadeh:
"...in contrast to traditional hard
computing, soft computing exploits the
tolerance for imprecision, uncertainty, and
partial truth to achieve tractability,
robustness, low solution-cost, and better
rapport with reality
8/5/2014
Soft Vs Hard Computing
HARD COMPUTING
Precise Models
Symbolic
Logic
Reasoning
Traditional
Numerical
Modeling and
Search
SOFT COMPUTING
Approximate Models
Approximate
Reasoning
Soft Computing
Hard Computing
Conventional computing requires a Soft
precisely stated analytical model.
Functional
Approximation
and Randomized
Search
computing
is
tolerant
of
imprecision.
Often requires a lot of computation Can solve some real world problems in
time.
reasonably less time.
Not suited for real world problems for Suitable for real world problems.
which ideal model is not present.
It requires full truth
Can work with partial truth
It is precise and accurate
Imprecise.
High cost for solution
Low cost for solution
8/5/2014
Overview techniques of SC
Neural Networks
Fuzzy Logic
Genetic Algorithm
Hybrid Systems
Evolution of Computing
1940s
1947
Cybemetics
1943 McCulloach
Pitts neuron model
1950s
1956 AI
1957 Perceptron
1960s
1960 LISP
language
1960 Adaline
Masaline
1965 Fuzzy
sets
1970s
Mid 1970s
knowledge
engineering
(expert system)
1974 Birth of back
propogation
algorithm
1975 Cognitron
Neocognitron
1974 Fuzzy
controller
1970s
Genetic
algorithm
1980 self organizing
map
1982 hopfield net
1983 Boltzmann
machine
1986
Backpropogation
algorithm boom
1985 Fuzzy
modelling
Mid 1980s
Artificial life
Immune
modelling
1980s
1990s
1990s Neuro
1990 Genetic
fuzzy modelling programming
1991 ANFIS
8/5/2014
SC constituents (the first three
items) and conventional AI
Methodology
Strength
Neural network
Learning and adaptation
Fuzzy set theory
Knowledge representation via fuzzy
if-then rules
Genetic algorithm and
simulated annealing
Systematic random search
Conventional AI
Symbolic manipulation
Character recognizer
8/5/2014
Features of Conventional AI
Conventional
AI manipulates symbols on the
assumption that human intelligence behavior can
be stored in symbolically structured knowledge
bases: this is known as: The physical symbol
system hypothesis
The
knowledge-based system (or expert system) is
an example of the most successful conventional AI
product
What is expert system?
An expert system is software that uses a knowledge
base of human expertise for problem solving, or to
clarify uncertainties where normally one or more
human experts would need to be consulted
8/5/2014
Expert system
Building blocks of expert system
Knowledge base: factual knowledge and heuristic
knowledge
Knowledge representation: in the form of rules
Problem solving model: forward chaining or
backward chaining
Note: Knowledge
engineering:- building an expert system
Knowledge engineers:- practitioners.
8/5/2014
Definitions of AI
AI is the study of agents that exists in an
environment and perceive and act [S. Russel & P.
Norvig]
AI is the act of making computers do smart
things [Waldrop]
AI is a programming style, where programs
operate on data according to rules in order to
accomplish goals [W.A. Taylor]
Definitions of AI
AI is the activity of providing such machines as
computers with the ability to display behavior that
would be regarded as intelligent if it were
observed in humans [R. Mc Leod]
Expert system is a computer program using
expert knowledge to attain high levels of
performance in a narrow problem area [D.A.
Waterman]
10
8/5/2014
Definitions of AI
Expert system is a caricature of the human
expert, in the sense that it knows almost
everything about almost nothing [A.R. Mirzai]
AI is changing rapidly, these definitions are
already obsolete!
Applications of expert system
Diagnosis and Troubleshooting of Devices and
Systems of All Kinds
Planning and Scheduling
Configuration of Manufactured Objects from
Subassemblies
Financial Decision Making
Knowledge Publishing
Design and Manufacturing
11
8/5/2014
If the facts don't fit the
theory, change the facts.
- Albert Einstein
GEMS CBR for Remote Diagnostics
On-site Monitoring
Remote Data
Access
Phone, E-mail
FAX, Web
GE Regional
Service Team
Parts
Specialist
Diagnostics
Specialist
Technical
Answer Center
Information
System
Servers
Call Management/
Commitment Tracking
System
Monitoring &
Diagnostics
Problem/Solution DB
CBR using DB
Service
Engineer
12
8/5/2014
Machine Learning basics
Machine learning, a branch of artificial intelligence,
concerns the construction and study of systems that
can learn from data.
E.g. a machine learning system could be trained on email
messages to learn to distinguish between spam and nonspam messages. After learning, it can then be used to
classify new email messages into spam and non-spam
folders.
Research Areas
Intelligent Robots
Bayesian inference and design
technique for uncertain scene
recognition
Combination Image filtering and
Bayesian inference
Navigation technique research for
autonomous mobile robot
Evolving a mobile robot controller
13
8/5/2014
Research Areas
Intelligent Agent
Intelligent virtual secretary agent
Conversational agent
Intelligent assistants for smart
phone service
Research Areas
Bioinformatics
Bioinformatics: the collection, classification,
storage, and analysis of biochemical and biological
information using computers especially as applied
in molecular genetics and genomics
Classification techniques in Bioinformatics
Agent driven virtual cell modelling
14
8/5/2014
Research Areas
Ubiquitous
Developing an adaptation scheme in the context
of middleware and applications
Developing context-aware system for ubiquitous
systems
Developing basic theories and algorithms of the
advanced intelligent models for ubiquitous
environment
Research Areas
Intrusion Detection System (IDS)
HMM-based intrusion detection system
Generation of various intrusion patterns using
interactive genetic algorithm
Viterbi algorithm for intrusion type identification
Rule-based integration of multiple measure-models
15
8/5/2014
Research Areas
Biometrics
Analysis and evaluation techniques of fingerprint
recognition system
Development of classification and matching algorithm
for fingerprint recognition
Neuro Fuzzy and Soft
Computing Characteristics
With NF modeling as a backbone, SC can be
characterized as:
Human expertise (fuzzy if-then rules)
Biologically inspired computing models (NN)
New optimization techniques (GA, SA, RA)
Numerical computation (no symbolic AI so far, only
numerical)
New application domains: mostly computation
intensive like adaptive signal processing, adaptive
control, nonlinear system identification etc
16
8/5/2014
Neuro Fuzzy and Soft
Computing Characteristics
Model
free learning:-models are constructed based on
the target system only
Intensive computation: based more on computation
Fault tolerance: deletion of a neuron or a rule does not
destroy the system. The system performs with lesser
quality
Goal driven characteristics:- only the goal is
important and not the path.
Real world application:- large scale, uncertainties
Neural Network
DARPA Neural Network Study (1988, AFCEA
International Press, p. 60):
... a neural network is a system composed of many
simple processing elements operating in parallel
whose function is determined by network
structure, connection strengths, and the
processing performed at computing elements or
nodes.
17
8/5/2014
Definition of Neural Network
According to Haykin (1994), p. 2:
A neural network is a massively parallel distributed
processor that has a natural propensity for storing
experiential knowledge and making it available for
use. It resembles the brain in two respects:
Knowledge is acquired by the network through a learning
process.
Interneuron connection strengths known as synaptic
weights are used to store the knowledge
According to Nigrin (1993), p. 11:
A neural network is a circuit composed of a very
large number of simple processing elements that
are neurally based. Each element operates only on
local information.
Furthermore each element operates
asynchronously; thus there is no overall system
clock.
18
8/5/2014
According to Zurada (1992):
Artificial neural systems, or neural networks, are
physical cellular systems which can acquire, store
and utilize experiential knowledge.
Multi disciplinary view of
Neural Networks
19
8/5/2014
Fuzzy Logic
Origins: Multivalued Logic for treatment of imprecision
and vagueness
1930s: Post, Kleene, and Lukasiewicz attempted to represent
undetermined, unknown, and other possible intermediate
truth-values.
1937: Max Black suggested the use of a consistency profile to
represent vague (ambiguous) concepts.
1965: Zadeh proposed a complete theory of fuzzy sets (and its
isomorphic fuzzy logic), to represent and manipulate ill-defined
concepts.
FUZZY LOGIC LINGUISTIC
VARIABLES
Fuzzy logic gives us a language (with syntax and local
semantics) in which we can translate our qualitative
domain knowledge.
Linguistic variables to model dynamic systems
These variables take linguistic values that are
characterized by:
a label - a sentence generated from the syntax
a meaning - a membership function determined by a local
semantic procedure
20
8/5/2014
Linguistic variables
Linguistic variables associate a linguistic condition
with a crisp variable.
A crisp variable is the kind of variable that is used
in most computer programs: an absolute value.
A linguistic variable,
variable on the other hand, has a
proportional nature: in all of the software
implementations of linguistic variables, they are
represented by fractional values in the range of 0
to 1.
Linguistic variables
Linguistic variables in soup instructions
21
8/5/2014
An old friend comes into your shop asking to buy a few
widgets, and wants your best price. The onus is on you to
come up with a price given many parameters. Taking this
hypothetical case we need to account for:
Cost of the widgets
Normal markup
Shelf time of the product
Shelf life of the product
Length of the relationship
Customer payment history
Quantity of the sale
Repeat business potential
Parameters to consider pricing a widget
22
8/5/2014
Figure: Linguistic variable HOT
23
8/5/2014
Figure: F_OR operator (Fuzzy OR)
Figure: F_EQ (Fuzzy equal)
24
8/5/2014
FUZZY LOGIC REASONING
METHODS
The meaning of a linguistic variable may be interpreted as an
elastic constraint on its value.
These constraints are propagated by fuzzy inference operations,
based on the generalized modus-ponens.
An FL Controller (FLC) applies this reasoning system to a
Knowledge Base (KB) containing the problem domain heuristics.
The inference is the result of interpolating among the outputs of
all relevant rules.
The outcome is a membership distribution on the output space,
which is defuzzified to produce a crisp output.
There are two consistent logical argument constructions: modus
ponens ("the way that affirms by affirming") and modus tollens ("the
way that denies by denying"). Here are how they are constructed:
Modus Ponens: "If A is true, then B is true. A is true. Therefore, B is true."
Modus Tollens: "If A is true, then B is true. B is not true. Therefore, A is
not true.
There are two related incorrect and inconsist
constructions: affirming the consequent and denying the
antecedent.
Affirming the Consequent: "If A is true, then B is true. B is true.
Therefore, A is true."
Denying the Antecedent: "If A is true, then B is true. A is not true.
Therefore, B is not true."
25
8/5/2014
Example
Here is a sensible example, illustrating each of the
above:
"If it is a car, then it has wheels. It is a car.
Therefore, it has wheels." (Modus Ponens CORRECT)
"If it is a car, then it has wheels. It does not have
wheels. Therefore, it is not a car." (Modus Tollens CORRECT)
"If it is a car, then it has wheels. It has wheels.
Therefore, it is a car." (Affirming the Consequent INCORRECT.)
Comment: why is this incorrect? Well, the thing might
have wheels but that doesn't mean it has to be a car. It
might be a cart, or rollerblades, or a moped. It doesn't have
to be a car.
"If it is a car, then it has wheels. It is not a car.
Therefore, it does not have wheels." (Denying the
Antecedent - INCORRECT)
Comment: why is this incorrect? Consider
the argument for the "affirming the consequent" example.
Rollerblades are not cars, but they DO have wheels.
26
8/5/2014
Genetic algorithm
EVOLUTIONARY PROCESS
Steps involved in Genetic
Algorithm
The genetic algorithms follow the evolution process in the
nature to find the better solutions of some complicated
problems. Foundations of genetic algorithms are given in
Holland (1975) and Goldberg (1989) books.
Genetic algorithms consist the following steps:
Initialization
Selection
Reproduction with crossover and mutation
Selection and reproduction are repeated for each
generation until a solution is reached.
During this procedure a certain strings of symbols, known
as chromosomes, evaluate toward better solution.
27
8/5/2014
Hybrid Systems
Hybrid systems enables one to combine various
soft computing paradigms and result in a best
solution. The major three hybrid systems are as
follows:
Hybrid
Fuzzy Logic (FL) Systems
Hybrid
Neural Network (NN) Systems
Hybrid
Evolutionary Algorithm (EA) Systems
Applications of Soft Computing
Handwriting Recognition
Image Processing and Data Compression
Automotive Systems and Manufacturing
Soft Computing to Architecture
Decision-support Systems
Soft Computing to Power Systems
Neuro Fuzzy systems
Fuzzy Logic Control
Machine Learning Applications
Speech and Vision Recognition Systems
Process Control and So on
28
8/5/2014
29
