Genetic Algorithms
Genetic Algorithms
• Genetic Algorithms (GAs) were invented by John Holland in early 1970's to mimic
some of the processes observed in natural evolution.
• GAs simulate natural evolution, a combination of selection, recombination and
mutation to evolve a solution to a problem.
• GAs simulate the survival of the fittest, among individuals over consecutive
generation for solving a problem. Each generation consists of a population of
character strings that are analogous to the chromosome in our DNA
(Deoxyribonucleic acid). DNA contains the genetic instructions used in the
development and functioning of all known living organisms.
What are Genetic Algorithms
• Genetic Algorithms (GAs) are adaptive heuristic search algorithm based on
the evolutionary ideas of natural selection and genetics.
• Genetic algorithms (GAs) are a part of evolutionary computing, a rapidly
growing area of artificial intelligence. GAs are inspired by Darwin's theory about
evolution - "survival of the fittest".
• GAs represent an intelligent exploitation of a random search used to solve
optimization problems.
• GAs, although randomized, exploit historical information to direct the search into
the region of better performance within the search space.
Mechanics of Biological Evolution
• Genetic Algorithms are a way of solving problems by mimicking processes
the nature uses - Selection, Crosses over, Mutation and Accepting to evolve a
solution to a problem.
1. Every organism has a set of rules, describing how that organism is built, and encoded in the genes of an
organism.
2. The genes are connected together into long strings called chromosomes.
3. Each gene represents a specific trait (feature) of the organism and has several different settings, e.g.
setting for a hair color gene may be black or brown.
4. The genes and their settings are referred as an organism's genotype.
5. When two organisms mate they share their genes. The resultantoffspring may end up having half the genes
from one parent and half from the other parent. This process is called crossover (recombination).
6. The newly created offspring can then be mutated. A gene may be mutated and expressed in the
organism as a completely newtrait. Mutation means, that the elements of DNA are a bit changed. This
change is mainly caused by errors in copying genes from parents.
7. The fitness of an organism is measured by success of the organismin its life.
Taxonomy of Evolution & Search Optimization
Classes
Comparison of Natural Evolution and Genetic
Algorithm Terminology
Natural Evolution Genetic Algorithm
Chromosome String
Gene Feature or Character
Allele Feature Value
Locus String Position
Genotype Structure or coded string
Phenotype Parameter set
General Genetic Algorithm
1. [Start] Generate random population of n chromosomes (i.e. suitablesolutions for the problem).
2. [Fitness] Evaluate the fitness f(x) of each chromosome x in thepopulation.
3. [New population] Create a new population by repeating followingsteps until the new population is complete.
(a) [Selection] Select two parent chromosomes from a population according to their fitness (better the fitness, bigger the chance to be
selected)
(b) [Crossover] With a crossover probability, cross over the parents to form new offspring (children). If no crossover was performed,
offspring is the exact copy of parents.
(c) [Mutation] With a mutation probability, mutate new offspring at each locus (position in chromosome).
(d) [Accepting] Place new offspring in the new population
4. [Replace] Use new generated population for a further run of thealgorithm
5. [Test] If the end condition is satisfied, stop, and return the bestsolution in current population
6. [Loop] Go to step 2
Flowchart
for Genetic
Algorithm
Operators of Genetic Algorithm
• Genetic operators are analogous to those which occur in the
natural world:
• Reproduction (or Selection) ;
• Crossover (or Recombination); and
• Mutation.
Reproduction, or Selection
• Reproduction is usually the first operator applied on population.
From the population, the chromosomes are selected to be
parents to crossover and produce offspring.
• The most commonly used methods of selecting chromosomes for
parents to crossover are :
1. Roulette wheel selection,
2. Rank selection
3. Boltzmann selection,
4. Tournament selection,
Roulette wheel selection (Fitness-Proportionate Selection)
• Roulette-wheel selection, also known as Fitness Proportionate
Selection, is a genetic operator, used for selecting potentially
useful solutions for recombination.
• In fitness-proportionate selection :
• the chance of an individual's being selected is proportional to its
fitness, greater or less than its competitors' fitness.
• conceptually, this can be thought as a game of Roulette.
The Roulette-wheel simulates 8 individuals with
fitness values Fi, marked at its circumference; e.g.,
the 5th individual has a higher fitness than others,
so the wheel would choose the 5th individual
more than other individuals .
the fitness of the individuals is
calculated as the wheel is spun n = 8 times,
each time selecting an instance, of the string,
chosen by the wheel pointer.
Boltzmann Selection
• Simulated annealing is a method used to minimize or maximize a function.
• This method simulates the process of slow cooling of molten metal to achieve the minimum
function value in a minimization problem.
• The cooling phenomena is simulated by controlling a temperature like parameter introduced
with the concept of Boltzmann probability distribution.
• The system in thermal equilibrium at a temperature T has its energy distribution based on the
probability defined by
P(E) = exp ( - E / kT ) were k is Boltzmann constant.
• This expression suggests that a system at a higher temperature has almost uniform
probability at any energy state, but at lower temperature it has a small probability of being at
a higher energy state.
• Thus, by controlling the temperature T and assuming that the search process follows
Boltzmann probability distribution, the convergence of the algorithm is controlled.
Crossover
• Crossover is a genetic operator that combines (mates) two
chromosomes (parents) to produce a new chromosome
(offspring). The idea behind crossover is that the new
chromosome may be better than both of the parents if it takes the
best characteristics from each of the parents. Crossover occurs
during evolution according to a user-definable crossover
probability. Crossover selects genes from parent chromosomes
and creates a new offspring.
The Crossover operators are mainly of 2 types.
• One-Point crossover.
• Two Point crossover.
One-Point Crossover
• One-Point crossover operator randomly selects one crossover
point and then copy everything before this point from the first
parent and then everything after the crossover point copy from the
second parent. The Crossover would then look as shown below.
• Consider the two parents selected for crossover.
• Parent 1 11011|00100110110
• Parent 2 11011|11000011110
• Interchanging the parents chromosomes after the crossover
points - The Offspring produced are :
• Offspring 1 1 1 0 1 1 | 1 1 0 0 0 0 1 1 1 1 0
• Offspring 2 1 1 0 1 1 | 0 0 1 0 0 1 1 0 1 1 0
Two-Point Crossover
• Two-Point crossover operator randomly selects two crossover
points within a chromosome then interchanges the two parent
chromosomes between these points to produce two new
offspring.
• Consider the two parents selected for crossover :
• Parent 1 11011|0010011|0110
• Parent 2 11011|1100001|1110
• Interchanging the parents chromosomes between the crossover
points - The Offspring produced are :
• Offspring 1 1 1 0 1 1 | 1 1 0 0 0 0 1 | 0 1 1 0
• Offspring 2 1 1 0 1 1 | 0 0 1 0 0 1 1 | 1 1 1 0
Mutation
• After a crossover is performed, mutation takes place.
• Mutation is a genetic operator used to maintain genetic diversity from
one generation of a population of chromosomes to the next.
• Mutation is an important part of the genetic search, helps to prevent
the population from stagnating at any local optima. Mutation is
intended to prevent the search falling into a local optimum of the state
space.
Flip Bit
• The mutation operator simply inverts the value of the chosen gene.
• i.e. 0 goes to 1 and 1 goes to 0.
• This mutation operator can only be used for binary genes.
Consider the two original off-springs selected for mutation.
Invert the value of the chosen gene as 0 to 1 and 1 to 0
The Mutated Off-spring produced are :
Example:Maximize the function f(x) = x2 over the range of integers from 0 . . . 31.
1. Devise a means to represent a solution to the problem :Assume, we represent x
with five-digit unsigned binary integers.
2. Devise a heuristic for evaluating the fitness of any particular solution : The function
f(x) is simple, so it is easy to use the f(x) value itself to rate the fitness of a solution;
else we might have considered a more simpler heuristic that would more or less serve
the same purpose.
3. Coding - Binary and the String length :
GAs often process binary representations of solutions. This works well, because
crossover and mutation can be clearly defined for binary solutions. A Binary string of
length 5 can represents 32 numbers (0 to 31).
4. Randomly generate a set of solutions :
Here, considered a population of four solutions. However, larger populations are used
in real applications to explore a larger part of the search. Assume, four randomly
generated solutions as : 01101, 11000, 01000, 10011. These are chromosomes or
genotypes.
5.Evaluate the fitness of each member of the population :
The calculated fitness values for each individual are -
(a) Decode the individual into an integer (called phenotypes),
01101-> 13; 11000 -> 24; 01000 -> 8; 10011 -> 19;
(b) Evaluate the fitness according to f(x) = x 2 ,
13 -> 169; 24 -> 576; 8 -> 64; 19 -> 361.
(c) Expected count = N * Prob i , where N is the number of individuals
in the population called population size, here N = 4.
Thus the evaluation of the initial population summarized in table below
String No i Initial X value Fitness f(x) Prob i Expected
Population (Pheno = x2 (fraction count N *
(chromosome) types) of total)
Prob I
1 01101 13 169 0.14 0.58
2 11000 24 1.97
576 0.49
3 01000 8 0.22
64 0.06
4 10011 19 361 1.23
0.31
Total (sum) 1170 1.00 4.00
Average 293 0.25 1.00
Max 576 0.49 1.97
Thus, the string no 2 has maximum chance of selection.
Strings Prob i Associated Bin
6. Produce a new generation of
solutions by picking from the
01101 0.14 0.0 ... 0.14 existing pool of solutions with a
11000 0.49 0.14 ... 0.63
preference for solutions which are
better suited than others:
01000 0.06 0.63 ... 0.69
We divide the range into four bins,
10011 0.31 0.69 ... 1.00
sized according to the relative
fitness of the solutions which they
represent.
• By generating 4 uniform (0, 1) random values and seeing which bin
they fall into we pick the four strings that will form the basis for the
next generation.
Random No Falls into bin Chosen string
0.08 0.0 . . . 0.14 01101
0.24 0.14 . . . 0.63 11000
0.52 0.14 . . . 0.63 11000
0.87 0.69 . . . 1.00 10011
7. Randomly pair the members of the new generation
Random number generator decides for us to mate the first two
strings together and the second two strings together.
8. Within each pair swap parts of the members solutions to create
offspring which are a mixture of the parents :
• For the first pair of strings: 0 1 1 0 1 ,11000
• We randomly select the crossover point to be after the fourth digit.
Crossing these two strings at that point yields:
• For the second pair of strings: 1 1 0 0 0 , 1 0 0 1 1
• We randomly select the crossover point to be after the
second digit. Crossing these two strings at that point yields
9. Randomly mutate a very small fraction of genes in the population : With a typical
mutation probability of per bit it happens that none of the bits in our population are
mutated.
10.Go back and re-evaluate fitness of the population (new generation) : This would be the
first step in generating a new generation of solutions. However it is also useful in showing
the way that a single iteration of the genetic algorithm has improved this sample.
