Unit-5

Syllabus
Genetic Algorithms: Introduction, Biological Background, Traditional Optimization
and Search Techniques, Genetic Algorithm and search space, Basic Terminology,
Simple GA, Operators in Genetic Algorithm, Stopping Condition for Generic
Algorithm Flow, Classification of Genetic Algorithms, Genetic Programming

Genetic Algorithms(GAs)
Genetic Algorithms(are adaptive heuristic search algorithms that belong to the
larger part of evolutionary algorithms. Genetic algorithms are based on the ideas of
natural selection and genetics. These are intelligent exploitation of random searches
provided with historical data to direct the search into the region of better
performance in solution space. They are commonly used to generate high-quality
solutions for optimization problems and search problems.
Genetic algorithms simulate the process of natural selection which means those
species that can adapt to changes in their environment can survive and reproduce
and go to the next generation. In simple words, they simulate “survival of the fittest”
among individuals of consecutive generations to solve a problem. Each generation
consists of a population of individuals and each individual represents a point in
search space and possible solution. Each individual is represented as a string of
character/integer/float/bits. This string is analogous to the Chromosome.

Foundation of Genetic Algorithms

Genetic algorithms are based on an analogy with the genetic structure and behavior
of chromosomes of the population. Following is the foundation of GAs based on this
analogy –
1. Individuals in the population compete for resources and mate
2. Those individuals who are successful (fittest) then mate to create more
offspring than others
3. Genes from the “fittest” parent propagate throughout the generation, that
is sometimes parents create offspring which is better than either parent.
 4. Thus each successive generation is more suited for their environment.

Search space

The population of individuals are maintained within search space. Each individual
represents a solution in search space for given problem. Each individual is coded as
a finite length vector (analogous to chromosome) of components. These variable
components are analogous to Genes. Thus a chromosome (individual) is composed
of several genes (variable components).

Fitness Score

A Fitness Score is given to each individual which shows the ability of an individual
to “compete”. The individual having optimal fitness score (or near optimal) are
sought.
The GAs maintains the population of n individuals (chromosome/solutions) along
with their fitness scores.The individuals having better fitness scores are given more
chance to reproduce than others. The individuals with better fitness scores are
selected who mate and produce better offspring by combining chromosomes of
parents. The population size is static so the room has to be created for new arrivals.
So, some individuals die and get replaced by new arrivals eventually creating new
generation when all the mating opportunity of the old population is exhausted. It is
hoped that over successive generations better solutions will arrive while least fit die.
Each new generation has on average more “better genes” than the individual
(solution) of previous generations. Thus each new generations have better “partial
solutions” than previous generations. Once the offspring produced having no
significant difference from offspring produced by previous populations, the
population is converged. The algorithm is said to be converged to a set of solutions
for the problem.

Operators of Genetic Algorithms

Once the initial generation is created, the algorithm evolves the generation using
following operators –
1) Selection Operator: The idea is to give preference to the individuals with good
fitness scores and allow them to pass their genes to successive generations.
2) Crossover Operator: This represents mating between individuals. Two
individuals are selected using selection operator and crossover sites are chosen
randomly. Then the genes at these crossover sites are exchanged thus creating a
completely new individual (offspring). For example –

Operators of Genetic Algorithms

Once the initial generation is created, the algorithm evolves the generation using
following operators –
1) Selection Operator: The idea is to give preference to the individuals with good
fitness scores and allow them to pass their genes to successive generations.
2) Crossover Operator: This represents mating between individuals. Two
individuals are selected using selection operator and crossover sites are chosen
randomly. Then the genes at these crossover sites are exchanged thus creating a
completely new individual (offspring). For example –
3) Mutation Operator: The key idea is to insert random genes in offspring to
maintain the diversity in the population to avoid premature convergence. For
example –

The whole algorithm can be summarized as –

1) Randomly initialize populations p
2) Determine fitness of population
3) Until convergence repeat:
a) Select parents from population
b) Crossover and generate new population
c) Perform mutation on new population
d) Calculate fitness for new population

Traditional optimization and search techniques in genetic algorithm

Genetic Algorithms (GAs) are optimization techniques inspired by natural selection

and genetics. Here's a comparison with traditional optimization and search
techniques:

Traditional Optimization Techniques:

1. Gradient-based methods: Use derivatives to find optimal solutions (e.g.,

gradient descent).

These methods use derivatives to find the optimal solution.

Examples include:
 a. Gradient Descent: An iterative method that uses the gradient of the
objective function to converge to the optimal solution.
b. Newton's Method: A method that uses the Hessian matrix (second
derivatives) to find the optimal solution.
c. Quasi-Newton Methods: Methods that approximate the Hessian matrix
to reduce computational complexity.
2. Linear Programming: Optimizes linear objective functions subject to linear
constraints.

A method used to optimize linear objective functions subject to linear

constraints. Examples include:

a. Simplex Method: A popular algorithm for solving linear programming

problems.
b. Interior Point Methods: Methods that solve linear programming
problems by iterating through the interior of the feasible region.
3. Dynamic Programming: Breaks down complex problems into smaller sub-
problems.

A method that breaks down complex problems into smaller sub-problems,

solving each sub-problem only once. Examples include:

a. Shortest Path Problems: Dynamic programming can be used to find the

shortest path in a graph.
b. Knapsack Problems: Dynamic programming can be used to solve 0/1
knapsack problems.

Traditional Search Techniques:

1. Exhaustive Search: Explores all possible solutions.

2. Heuristic Search: Uses problem-specific knowledge to guide the search.
3. Hill Climbing: Iteratively improves a single solution.

Genetic Algorithm Advantages:

1. Handling Non-Linearity: GAs can handle non-linear problems and multiple

local optima.
 2. Robustness: GAs are robust to noisy or changing environments.
3. Parallelism: GAs can be parallelized, making them suitable for large-scale
problems.

How GAs Work:

1. Initialization: A population of candidate solutions is generated.

2. Selection: Solutions are selected based on fitness.
3. Crossover: Selected solutions are combined to create new offspring.
4. Mutation: Offspring are randomly modified.
5. Replacement: The least fit solutions are replaced.

When to Use GAs:

1. Complex Problems: GAs are suitable for complex, non-linear problems.

2. Multiple Optima: GAs can handle multiple local optima.
3. Noisy Environments: GAs are robust to noisy or changing environments.

Basic Terminology

Population:
It is a subset of all the possible (encoded) solutions to the given problem. The
population for a GA is analogous to the population for human beings except that
instead of human beings, we have Candidate Solutions representing human beings.

Chromosomes:
A chromosome is one such solution to the given problem.

Gene:
A gene is one element position of a chromosome.

Allele:
It is the value a gene takes for a particular chromosome.

Genotype:
Genotype is the population in the computation space. In the computation space, the
solutions are represented in a way which can be easily understood and manipulated
using a computing system.

Phenotype:
Phenotype is the population in the actual real world solution space in which
solutions are represented in a way they are represented in real world situations.

Decoding and Encoding:

For simple problems, the phenotype and genotype spaces are the same. However,
in most of the cases, the phenotype and genotype spaces are different. Decoding is a
process of transforming a solution from the genotype to the phenotype space, while
encoding is a process of transforming from the phenotype to genotype space.
Decoding should be fast as it is carried out repeatedly in a GA during the fitness
value calculation.

For example, consider the 0/1 Knapsack Problem. The Phenotype space consists of
solutions which just contain the item numbers of the items to be picked.However, in
the genotype space it can be represented as a binary string of length n (where n is
the number of items). A 0 at position x represents that xth item is picked while a 1
represents the reverse. This is a case where genotype and phenotype spaces are
different.

simple Genetic Algorithm(GA)

simple Genetic Algorithm (SGA) is one of the three types of strategies followed in
Genetic algorithm.
1. SGA starts with the creation of an initial population of size N.
2. Then, we evaluate the goodness/fitness of each of the
solutions/individuals.
After that, the convergence criterion is checked, if it meets then we converge the
algorithm otherwise go for the next steps -
1. Select Np individuals from the previous population.
2. Create the mating pool randomly.
3. Perform Crossover.
4. Perform Mutation in offspring solutions.
5. Perform inversion in offspring solutions.
6. Replace the old solutions of the last generation with the newly created
solutions and go to step (2).
Important Parameters while solving problems in Simple Genetic Algorithm :
1. Initial Population N
2. Mating pool size Np
3. Convergence threshold
4. Crossover
5. Mutation
6. Inversion

Features :
● Computationally Expensive
● Biased towards more highly fit individuals
● Performs well when the initial population is large enough.

Applications :
● Learning robot behaviour using Simple Genetic Algorithm.
● In the finance industry.
● Neural networks.
● Soft computing Applications like Fuzzy logic, Neurocomputing.
● Optimization problems.
\
Operators in Genetic Algorithm

1. Selection
Selection operators choose individuals from the current population based on their
fitness, favoring better solutions to pass their genes to the next generation. Common
selection methods include:
● Roulette Wheel Selection: Probability proportional to fitness.
● Tournament Selection: Random groups compete, best selected.
● Rank Selection: Selection based on rank rather than raw fitness.
Selection ensures good solutions are preserved and can include elitism, where
the best individuals are guaranteed to survive unchanged to the next
generation

2. Crossover (Recombination)
Crossover combines genetic material from two parent solutions to create offspring,
promoting diversity by mixing good traits. Types of crossover include:
● One-Point Crossover: Single crossover point where tails are swapped.
● Multi-Point Crossover: Multiple crossover points swapping alternating
segments.
● Uniform Crossover: Each gene is independently chosen from either parent.
● Order-Based Crossovers (e.g., OX1): Used for permutation problems,
preserving relative order.
● Whole Arithmetic Recombination: Weighted averages of parents, often for
numeric genes.
Crossover is analogous to biological reproduction and is crucial for exploring
new solutions

3. Mutation
Mutation introduces random changes to individual genes in a chromosome,
maintaining genetic diversity and preventing premature convergence to local optima.
It operates on one chromosome at a time and helps explore new areas of the solution
space
Stopping Condition for Generic Algorithm Flow
The stopping condition for a generic genetic algorithm (GA) flow determines when
the algorithm should terminate its search process. Common stopping criteria include:

● Maximum number of generations: The algorithm stops after evolving a

predefined number of generations
● No improvement in fitness: The algorithm halts if the best fitness value in the
population does not improve for a specified number of consecutive
generations, also called stall generations
● Fitness threshold reached: The algorithm stops when the objective function
(fitness) value reaches or surpasses a predefined target or convergence value
● Elapsed time limit: The algorithm terminates after running for a specified
amount of time
● Stall time limit: The algorithm stops if no improvement in fitness occurs
within a given time interval
● Population convergence criteria: Variations include stopping when the best
individual meets criteria, when the worst individual reaches a threshold, or
when median or sum fitness values stabilize

Classification of Genetic Algorithms

Genetic Algorithms (GAs) are a key component of Soft Computing, which is a

collection of methodologies that work synergistically to provide flexible information
processing systems capable of handling uncertainty, imprecision, and partial truth.
GAs are inspired by the principles of natural evolution and genetics, and they are
widely used for optimization, search, and machine learning tasks.

Classification of Genetic Algorithms in Soft Computing

Genetic Algorithms can be classified based on various criteria, including their
structure, operators, and applications. Here’s a broad classification:

1. Based on Population Structure

● Steady-State GA
○ Only a few individuals are replaced in each generation.
○ Uses an overlapping population where parents and offspring coexist.
● Generational GA
○ The entire population is replaced in each iteration.
○ New offspring replace the old population entirely.
● Island (Distributed) GA
○ Multiple subpopulations evolve independently with occasional
migration.
○ Helps maintain diversity and avoid premature convergence.
● Cellular GA
○ Individuals are placed in a grid, and selection is restricted to neighbors.
○ Promotes local competition and slow diffusion of good solutions.

2. Based on Selection Mechanisms

● Roulette Wheel Selection

○ Probability of selection is proportional to fitness.
● Tournament Selection
○ A subset of individuals competes, and the best is selected.
● Rank-Based Selection
○ Individuals are selected based on their rank rather than raw fitness.
● Elitism
○ The best individuals are preserved across generations.

3. Based on Crossover Operators

● Single-Point Crossover
○ One crossover point is selected, and parts are swapped.
● Multi-Point Crossover
 ○ Multiple crossover points are used for recombination.
● Uniform Crossover
○ Each gene is independently swapped based on a probability.
● Arithmetic Crossover
○ Genes are combined using arithmetic operations (common in real-
coded GAs).

4. Based on Mutation Operators

● Bit-Flip Mutation (for Binary GAs)

○ Randomly flips bits in binary strings.
● Gaussian Mutation (for Real-Coded GAs)
○ Adds Gaussian noise to real-valued genes.
● Swap Mutation (for Permutation Problems)
○ Swaps two randomly selected genes.
● Insertion/Inversion Mutation
○ Inserts or reverses a segment of genes.

5. Based on Encoding Scheme

● Binary-Coded GA
○ Solutions are represented as binary strings.
● Real-Coded GA
○ Solutions use real-valued vectors.
● Integer/Discrete GA
○ Used for combinatorial optimization (e.g., TSP).
● Tree-Based GA (Genetic Programming)
○ Solutions are represented as trees (used in evolving programs).

6. Based on Hybridization

● Memetic Algorithms (GA + Local Search)

○ Combines GA with local optimization techniques.
● Neuro-Genetic Hybrids (GA + Neural Networks)
○ Optimizes neural network weights/architectures.
 ● Fuzzy-GA Hybrids
○ Uses GA to optimize fuzzy rule bases or membership functions.

7. Based on Application Domain

● Combinatorial Optimization (e.g., TSP, Knapsack)

● Continuous Optimization (e.g., Function Optimization)
● Machine Learning (Feature Selection, Hyperparameter Tuning)
● Evolutionary Robotics & Game AI
● Bioinformatics (Sequence Alignment, Protein Folding)

Genetic Programming in soft computing

Genetic Programming (GP) is an evolutionary computation technique within Soft

Computing that extends Genetic Algorithms (GAs) to automatically generate
computer programs, mathematical models, or complex structures that solve a given
problem. Unlike traditional GAs that optimize fixed-length strings (binary or real-
valued), GP evolves tree-structured programs or expressions, making it highly
flexible for symbolic regression, control systems, and artificial intelligence tasks.

Classification of Genetic Programming

GP can be classified based on representation, operators, and applications:

1. Based on Representation

(a) Tree-Based GP

● Programs are represented as syntax trees (e.g., LISP S-expressions).

● Nodes: Functions (operators), Leaves: Terminals (variables/constants).
● Example: Evolving mathematical expressions like (sin(x) + (y * 2)).

(b) Linear GP

● Programs are represented as linear sequences (e.g., machine code, bytecode).

 ● Used in evolving low-level code for efficiency.

(c) Grammatical Evolution (GE)

● Uses a grammar to map binary/real-valued strings into programs.

● Useful for constrained syntax (e.g., generating valid SQL queries).

(d) Cartesian GP (CGP)

● Programs are represented as directed graphs (grid-based).

● Allows more flexible structures than trees.

(e) Stack-Based GP

● Uses a stack-based representation (similar to Forth or PostScript).

● Efficient for certain mathematical operations.

2. Based on Genetic Operators

(a) Crossover in GP

● Subtree Crossover: Swaps subtrees between two parents.

● One-Point Crossover: Swaps branches at a randomly selected node.
● Uniform Crossover: Randomly swaps nodes between trees.

(b) Mutation in GP

● Subtree Mutation: Replaces a subtree with a new random subtree.

● Point Mutation: Changes a single node (e.g., + to -).
● Shrink Mutation: Reduces tree depth by replacing a subtree with a terminal.

(c) Selection Mechanisms

● Tournament Selection (common in GP).

 ● Fitness-Proportionate Selection (roulette wheel).
● Lexicographic Parsimony Pressure (favors smaller programs).

3. Based on Application Domains

(a) Symbolic Regression

● Evolves mathematical models from data (e.g., f(x) = x² + sin(x)).

● Alternative to neural networks for interpretable models.

(b) Control Systems

● Evolves rules for robotic control, game AI, or autonomous systems.

(c) Automated Programming

● Generates algorithms or code snippets automatically.

(d) Feature Selection & Model Optimization

● Used in machine learning to evolve optimal feature sets.

(e) Circuit Design & Hardware Optimization

● Evolves electronic circuits or FPGA configurations.

Advantages of GP in Soft Computing

● No Need for Fixed Representation: Can evolve complex structures.

● Interpretability: Unlike black-box models (e.g., deep learning), GP produces

human-readable solutions.
 ● Adaptability: Can solve problems where traditional methods fail.

● Hybridization: Can be combined with fuzzy logic, neural networks, and

reinforcement learning.

Challenges of GP

● Computationally Expensive: Large trees require significant resources.

● Bloat Problem: Programs grow unnecessarily complex without control.

● Premature Convergence: May get stuck in local optima.