Genetic Algorithms
Tutorials
Asst. Prof.
Dr. Mohammed Najm Abdullah
Genetic Algorithm Process
2
Example: Travelling Salesman Problem
• Find a tour of a given set of cities so that 1) each city is visited only
once 2) the total distance traveled is minimised
• Representation is an ordered list of city numbers (known as order-based
GA):
1) London 3) Dunedin 5) Beijing 7) Tokyo
2) Venice 4) Singapore 6) Phoenix 8) Victoria
CityList1 (3 5 7 2 1 6 4 8)
CityList2 (2 5 7 6 8 1 3 4)
• Recombination uses i) crossover and ii) mutation by inversion:
Parent1 (3 5 7 2 1 6 4 8)
Parent2 (2 5 7 6 8 1 3 4) crossover
Child (2 5 7 2 1 6 3 4)
3
Example: Travelling Salesman Problem
• Mutation involves reordering of the list: eg flip elements 3 and 6
• * *
Before: (2 5 7 2 1 6 3 4) mutation
After : (2 5 6 2 1 7 3 4)
• Now consider 30 unnamed cities on a grid, each with (x,y) coordinates
4
Example: Travelling Salesman Problem
ICT219 5
Example: Travelling Salesman Problem
ICT219 6
Example: Travelling Salesman Problem
7
Pros and Cons of Genetic Algorithms
• GAs are a flexible, widely applicable optimization process
• Unlike NNs, GAs tend to avoid local minima, and find the global solution
(if you are not in a hurry) because search is not restricted to a single
part of the problem space
•
• Can optimize a lot of parallel measures simultaneously (multi-objective)
• Operators can be customized to take advantage of regularities or
constraints in a particular domain to improve speed or quality of
convergence
But...
• Abstractions about the problem itself such as mathematical
simplifications can’t be used
• Difficult to predict how long convergence will take – randomness in the
process means this might vary widely
• Because it requires representation and processing on a sizable
population of genotypes, it could be expensive in terms of memory and
computation – very complex problems could be infeasible
8
GAs can be used when...
• Alternative solutions are too slow or overly complicated
• Need an exploratory tool to examine new approaches
• Need an “anytime” algorithm – always a solution, only improves
• Want to make a hybrid with another system
• Problem is similar to one that has already been successfully solved
using a GA!
GAs are now used to optimise
the design parameters of complex
machines, such as jet engines
9
Case Study
Consider the determination of the nearest integer value
( in the range 0 to 15 ) of x that maximises the cost
function :
y( x) 0.2x 1.5x
2
Using a population of 4, chromosome length of 4 bits,
one point crossover, mutation probability of 25% and
weighted fitness via maximum integer method.
Case Study Solution - Step 1
Generate an initial random population and evaluate
fitness :
Chromosome Fitness ( Cost ) Weighted Fitness Copies
0011 (3) 2.7 2.08 3
0111 (7) 0.7 0.54 1
0001 (1) 1.3 1.00 2
1110 (14) ( -14.3 ) 0.5 0.39 1
5.2
Note ! Saturation at +0.5 minimum level to
keep fitness positive
Case Study Solution - Step 2
1st Reproduction Cycle ( 2nd Generation ) :
0 0 1 1 0 0 0 1 0 0 0 1
0 0 0 1 0 0 1 1 0 0 1 1
0 0 1 1 0 1 1 1 0 1 0 1
0 1 1 1 0 0 1 1 0 0 1 1
Crossover Mutation
Case Study Solution - Step 3
Evaluate the fitness of the 2nd generation :
Note ! Even though 2 copies of the 3rd
chromosome are requested none will
contribute parents for the next generation
Case Study Solution - Step 4
2nd Reproduction Cycle ( 3rd Generation ) :
0 0 1 1 0 1 0 1 0 1 0 0
0 1 0 1 0 0 1 1 0 0 1 1
0 0 1 1 0 0 1 1 0 0 1 1
0 0 1 1 0 0 1 1 0 0 1 1
Crossover Mutation
Note! Local Minima without mutation
Case Study Solution - Step 5
Evaluate the fitness of the 3rd generation :
Chromosome Fitness ( Cost ) Weighted Fitness Copies
0011 (3) 2.7 0.99 1
0100 (4) 2.8 1.03 2
0011 (3) 2.7 0.99 1
0011 (3) 2.7 0.99 1
10.9
Note ! Now that the ‘optimum’ solution has
been found then the chromosomes will
quickly converge onto it.
A GA EXAMPLE
Objective - to find a binary string of length 5 with 4
1’s.
Representation: binary string of length 5
Solution space: 5 feasible solutions among 25
solutions.
First step: randomly generate 5 candidates, and
evaluate their fitness using the number of 1ís in the
string as a criterion.
00010 (eval: 1)
10001 (eval: 2)
10000 (eval: 1)
01011 (eval: 3)
10010 (eval: 2)
Population evaluation average: 1.8
16
Second Step: generate new chromosomes
Modification methods:
(a) crossover during which two genes interchange their
chromosomes;
(b) inversion by flipping sub-string of the same gene; and
(c) mutation by randomly perturbation.
Selectionist distribution: Genes with higher fitness value
has higher probability to produce off-springs!
1 00010 (eval: 1) 2 10001 (eval: 2) 3 10001 (repeat)
4 10000 (eval: 1) 5 01011 (eval: 3) 6 01011 (repeat)
7 01011 (repeat) 8 10010 (eval: 2) 9 10010 (repeat)
Select pairs (indices from selectionist distribution): 1 & 4
@1, 4 & 5 @ 4, 9 & 7 @3, 8 & 6 @1, 7 & 5 @1
GENERATE NEW GENES
For example, crossover 1 (00010) and 4 (10000) at
position 1 yields
00000 which evaluates 0! Other results are:
4+5@4 = 10001 (eval: 2)
9+7@3 = 10011 (eval: 3)
8+6@1 = 11011 (eval: 4)
7+5@1 = 01011 (eval: 3)
New population evaluation average: 2.4
Since 8 + 6 produces a feasible solution, the iteration
terminates, and the GA algorithm successfully
found a solution.
18
Genetic Programming
A program in C
• int foo (int time)
{
int temp1, temp2;
if (time > 10)
temp1 = 3;
else
temp1 = 4;
temp2 = temp1 + 1 + 2;
return (temp2);
}
• Equivalent expression (similar to a
classification rule in data mining):
(+ 1 2 (IF (> TIME 10) 3 4))
19
Program tree
(+ 1 2 (IF (> TIME 10) 3 4))
20
Given data
Input: Independent variable X Output: Dependent variable Y
-1.00 1.00
-0.80 0.84
-0.60 0.76
-0.40 0.76
-0.20 0.84
0.00 1.00
0.20 1.24
0.40 1.56
0.60 1.96
0.80 2.44
1.00 3.00
21
Problem description
Objective: Find a computer program with one
input (independent variable X) whose
output Y equals the given data
1 Terminal set: T = {X, Random-Constants}
2 Function set: F = {+, -, *, /}
3 Initial population: Randomly created individuals from
elements in T and F.
4 Fitness: |y0’ – y0| + |y1’ – y1| + … where yi’ is
computed output and yi is given
output for xi in the range [-1,1]
5 Termination: An individual emerges whose sum of
absolute errors (the value of its fitness
function) is less than 0.1
22
Generation 0
Population of 4 randomly created individuals
x+1 x2 + 1 2 x
23
X Y X+1 |X+1- X2+1 |X2+1- 2 |2-Y| X |X-Y|
Y| Y|
-1.00 1.00 0 1 2 1 2 1 -1.00 2
-0.80 0.84 0.20 0.64 1.64 0.80 2 1.16 -0.80 1.64
-0.60 0.76 0.40 0.36 1.36 0.60 2 1.24 -0.60 1.36
-0.40 0.76 0.60 0.16 1.16 0.40 2 1.24 -0.40 1.16
-0.20 0.84 0.80 0.04 1.04 0.20 2 1.16 -0.20 1.04
0.00 1.00 1.00 0 1 0 2 1 0.00 1
0.20 1.24 1.20 0.04 1.04 0.20 2 0.76 0.20 1.04
0.40 1.56 1.40 0.16 1.16 0.40 2 0.44 0.40 1.16
0.60 1.96 1.60 0.36 1.36 0.60 2 0.04 0.60 1.36
0.80 2.44 1.80 0.64 1.64 0.80 2 0.44 0.80 1.64
1.00 3.00 2.00 1 2 1 2 1 1.00 2
Σ Σ Σ Σ
Fitness: 4.40 6.00 9.48 15.40
Best in Gen 0
Crossover
Crossover:
picking “+”
subtree and
leftmost “x” as
crossover points
25
Mutation
Mutation:
/
picking “2”
as mutation
point
26
Generation 1
Second offspring
First offspring of of crossover of
Mutant of (c) crossover of (a) (a) and (b)
and (b) picking “+” of
Copy of (a) picking “2” picking “+” of parent (a) and
as mutation parent (a) and
point left-most “x” of
left-most “x” of parent (b) as
parent (b) as crossover points
crossover points
27
X Y X+1 |X+1- 1 |1-Y| X |X-Y| X2+X |X2+X+
Y| +1 1-Y|
-1.00 1.00 0 1 1 0 -1.00 2 1 0
-0.80 0.84 0.20 0.64 1 0.16 -0.80 1.64 0.84 0
-0.60 0.76 0.40 0.36 1 0.24 -0.60 1.36 0.76 0
-0.40 0.76 0.60 0.16 1 0.24 -0.40 1.16 0.76 0
-0.20 0.84 0.80 0.04 1 0.16 -0.20 1.04 0.84 0
0.00 1.00 1.00 0 1 0 0.00 1 1 0
0.20 1.24 1.20 0.04 1 0.24 0.20 1.04 1.24 0
0.40 1.56 1.40 0.16 1 0.56 0.40 1.16 1.56 0
0.60 1.96 1.60 0.36 1 0.96 0.60 1.36 1.96 0
0.80 2.44 1.80 0.64 1 1.44 0.80 1.64 2.44 0
1.00 3.00 2.00 1 1 2 1.00 2 3 0
Σ Σ Σ Σ
Fitness: 4.40 6.00 15.40 0.00
Found!
