Department of Biochemical and Chemical Engineering
Process Dynamics and Operations Group (DYN)
Evolutionary Algorithms
Maren Urselmann
Prof. Dr.-Ing. S. Engell
Process Dynamics and Operations Group
TU Dortmund, Germany
Lecture | 14.07.2011
Process Dynamics
and Operations
Literature and Slides
Literature:
A.E. Eiben, J.E. Smith: Introduction to
Evolutionary Computation, Springer,
2007
Slides of this lecture are developed on
the basis of the proposed slides for an
evolutionary computation course by
A.E. Eiben and J.E. Smith
Lecture | 14.07.2011
Process Dynamics
and Operations
Evolutionary Algorithms
Evolutionary Algorithms (EA) are inspired by the
process of biological evolution
Biology delivered inspiration and terminology
EVOLUTION
PROBLEM SOLVING
Environment
Problem
Individual
Candidate Solution
Fitness
Quality (objective fct value)
Fitness chances for survival and reproduction
Quality chance for seeding new solutions
Lecture | 14.07.2011
Process Dynamics
and Operations
Darwinian Evolution
All environments have limited resources
Lifeforms have the basic instinct to reproduce
Some kind of selection is inevitable
Natural selection = survival of the fittest
Competition for those resources
Individuals that are better adapted to the environment
(fitter individuals) have increased chances to
survive/reproduce
Over time Natural selection causes a rise in the fitness
of the population
Lecture | 14.07.2011
Process Dynamics
and Operations
Natural Genetics
The information required to build a living organism is
coded in the DNA of that organism
Genotype (DNA inside) determines phenotype
Small changes in the genotype lead to small changes in
the organism (e.g., height, hair colour)
Sexual reproduction:
Recombination = Crossover
- Genes of offspring are build by parts of the maternal and by
parts of the paternal genes
Mutation
- Some of the genetic material changes very slightly
(replication error)
- The child might have genetic material information not
inherited from either parent
Lecture | 14.07.2011
Process Dynamics
and Operations
General Scheme of EAs
Lecture | 14.07.2011
Process Dynamics
and Operations
General Scheme of EAs II
(1) INITIALIZE population with random candidate solutions;
(2) EVALUATE each candidate;
(3) Repeat until (TERMINATION CONDITION is satisfied)
(1) SELECT parents;
(2) RECOMBINE pairs of parents;
VARIATION Operators
(3) MUTATE the resulting offspring;
(4) EVALUATE new candidates;
(5) SELECT individuals for the next generation;
Lecture | 14.07.2011
Process Dynamics
and Operations
Representation
Candidate solutions (individuals) exist in phenotype
space
They are encoded in chromosomes, which exist in
genotype space
Encoding : phenotype genotype (not necessarily one to one)
Decoding : genotype phenotype (must be one to one)
Chromosomes contain genes
In order to find the global optimum, every feasible solution must
be represented in genotype space
Lecture | 14.07.2011
Process Dynamics
and Operations
Evaluation (Fitness) Function
Represents the requirements that the population should
adapt to
a.k.a. quality function or objective function
Assigns a single real-valued fitness to each phenotype
which forms the basis for selection
Typically we talk about fitness being maximized
Some problems may be best posed as minimization
problems, but conversion is trivial
Lecture | 14.07.2011
10
Process Dynamics
and Operations
Population
Holds (representations of) possible solutions
Usually has a fixed size and is a multiset of genotypes
Selection operators usually take whole population into
account i.e., reproductive probabilities are relative to
current generation
Diversity of a population refers to the number of
different fitnesses / phenotypes / genotypes present
(note not the same thing)
Lecture | 14.07.2011
11
Process Dynamics
and Operations
Parent selection mechanism
Assigns variable probabilities of individuals acting as
parents depending on their fitnesses
Usually probabilistic
high quality solutions more likely to become parents than
low quality
but not guaranteed
even worst in current population usually has non-zero
probability of becoming a parent
This stochastic nature can aid to escape from local
optima
Lecture | 14.07.2011
12
Process Dynamics
and Operations
Variation Operators
Role is to generate new candidate solutions
Usually divided into two types according to their arity
(number of inputs):
Arity 1 : mutation operators
Arity >1 : Recombination operators
Arity = 2 typically called crossover
There has been much debate about relative importance
of recombination and mutation
Nowadays most EAs use both
Choice of particular variation operators is representation
dependant
Lecture | 14.07.2011
13
Process Dynamics
and Operations
Recombination
Merges information from parents into offspring
Choice of what information to merge is stochastic
Most offspring may be worse, or the same as the
parents
Hope is that some are better by combining elements of
genotypes that lead to good traits
Lecture | 14.07.2011
14
Process Dynamics
and Operations
Mutation
Acts on one genotype and delivers another
Element of randomness is essential and differentiates it
from other unary heuristic operators
Importance ascribed depends on representation and
dialect
May guarantee connectedness of search space and
hence convergence proofs
Lecture | 14.07.2011
15
Process Dynamics
and Operations
Survivor Selection
a.k.a. replacement
Most EAs use fixed population size so need a way of
going from (parents + offspring) to next generation
Often deterministic
Fitness based : e.g., rank parents+offspring and take best
Age based: make as many offspring as parents and
delete all parents
Sometimes do combination (elitism)
Lecture | 14.07.2011
16
Process Dynamics
and Operations
Initialization and Termination
Initialization usually done at random,
Need to ensure even spread and mixture of possible
genes
Can include existing solutions, or use problem-specific
heuristics, to seed the population
Termination condition checked every generation
Reaching some (known/hoped for) fitness
Reaching some maximum allowed number of generations
Reaching some minimum level of diversity
Reaching some specified number of generations without
fitness improvement
Lecture | 14.07.2011
17
Process Dynamics
and Operations
Example: The 8 Queens Problem
Place 8 queens on an 8x8 chessboard in such a way
that they cannot check each other
Lecture | 14.07.2011
18
Process Dynamics
and Operations
The 8 Queens Problem: Representation
Phenotype:
a board configuration
Genotype:
a permutation of
the numbers 1 - 8
Obvious mapping
1 3 5 2 6 4 7 8
Lecture | 14.07.2011
19
Process Dynamics
and Operations
The 8 Queens Problem: Fitness Evaluation
Penalty of one queen:
the number of queens she can check.
Penalty of a configuration:
the sum of the penalties of all queens.
Note: penalty is to be minimized
Fitness of a configuration:
inverse penalty to be maximized
Lecture | 14.07.2011
20
Process Dynamics
and Operations
The 8 Queens Problem: Mutation
Small variation in one permutation, e.g.:
swapping values of two randomly chosen positions
Offspring
Parent individual
1 3 5 2 6 4 7 8
1 3 7 2 6 4 5 8
Lecture | 14.07.2011
21
Process Dynamics
and Operations
The 8 Queens Problem: Recombination
Combining two permutations into two new permutations:
choose random crossover point
copy first parts into children
create second part by inserting values from other parent:
in the order they appear there
beginning after crossover point
skipping values already in child
1 3 5 2 6 4 7 8
8 7 6 5 4 3 2 1
Lecture | 14.07.2011
1 3 5 4 2 8 7 6
8 7 6 2 4 1 3 5
22
Process Dynamics
and Operations
The 8 Queens Problem: Selection
Parent selection:
Pick 5 parents and take best two to undergo crossover
Survivor selection (replacement)
When inserting a new child into the population, choose an
existing member to replace by:
Sorting the whole population by decreasing fitness
Enumerating this list from high to low
Replacing the individual with the lowest fitness by the
child
Lecture | 14.07.2011
23
Process Dynamics
and Operations
The 8 Queens Problem: Summary
20
Note that this is only one possible set of choices of
operators and parameters
Lecture | 14.07.2011
24
Process Dynamics
and Operations
The 8 Queens Problem: Results
Lecture | 14.07.2011
25
Process Dynamics
and Operations
Typical Behavior of an EA
Phases in optimizing on a 1-dimensional fitness landscape
Early phase:
quasi-random population distribution
Mid-phase:
population arranged around/on hills
Late phase:
population concentrated on high hills
Lecture | 14.07.2011
26
Process Dynamics
and Operations
Best fitness in population
Typical run: progression of fitness
Time (number of generations)
Typical run of an EA shows so-called anytime behavior
Lecture | 14.07.2011
27
Process Dynamics
and Operations
Evolutionary Algorithms
EAs fall in the category of generate & test algorithms
They are stochastic, population-based algorithms
Variation operators (recombination and mutation) create
the necessary diversity and thereby facilitate novelty
Selection reduces diversity and acts as a force pushing
quality
Lecture | 14.07.2011
28
Process Dynamics
and Operations
Different Types of EAs
Historically different flavours of EAs have been
associated with different representations
Binary strings : Genetic Algorithms
Real-valued vectors : Evolution Strategies
Finite state Machines: Evolutionary Programming
LISP trees: Genetic Programming
These differences are largely irrelevant, best strategy
choose representation to suit problem
choose variation operators to suit representation
Selection operators only use fitness and so are
independent of representation
Lecture | 14.07.2011
29
Process Dynamics
and Operations
Evolution Strategies
Lecture | 14.07.2011
30
Process Dynamics
and Operations
Evolution Strategies
Developed: Germany in the 1970s
Early names: I. Rechenberg, H.-P. Schwefel
Today: ES belong to the class of EAs
Typically applied to:
numerical optimization
Attributed features:
fast
good optimizer for real-valued optimization
relatively much theory
Special:
self-adaptation of (mutation) parameters
Lecture | 14.07.2011
31
Process Dynamics
and Operations
Introductory Example
Task: minimimize f : Rn R
Algorithm: ES with two members using
Vectors from R directly as chromosomes
Population size =1
No recombination
Always Mutation
One child per generation (=1)
Greedy selection ( (+) selection)
No self-adaptation of the step size
Lecture | 14.07.2011
32
Process Dynamics
and Operations
Introductory example: scheme
(1) INITIALIZE population with a random candidate
solution: t:=0; xt=<x1t, x2t, , xnt>
(2) EVALUATE the candidate f(xt);
(3) Repeat until (TERMINATION CONDITION is satisfied)
(1) zi = *N(0,1) , for i=1,,n;
(2) MUTATION: yit = xit + zi;
(3) EVALUATE the offspring f(yt);
(4) SELECT the individual with better fitness for the next
generation
(1) IF f(xt) < f(yt) THEN
(2) ELSE
xt+1 = xt;
xt+1 = yt;
(5) t=t+1;
Lecture | 14.07.2011
33
Process Dynamics
and Operations
Introductory example: mutation mechanism
z values drawn from normal distribution N(,)
Expected value is set to 0
Standard deviation is called mutation strength
The one
dimensional case
Illustration of the
normal distribution *N(0,1)
Lecture | 14.07.2011
34
Process Dynamics
and Operations
Self-adaptation
Pecularity of the ES is the self-adaptation
Representation of the individuals is extended by one or
more strategy parameters:
E.g.
x = <x1,,xn, >
The strategy parameters are used to control certain
statistical properties of the genetic operators (especially
the mutation)
They can evolve during the evolution process
Inclusion of the strategy parameters in the individuals
genome selection and inheritance together with the
individual
Lecture | 14.07.2011
35
Process Dynamics
and Operations
Self-adapted mutation
Idea:
is part of the chromosome x1,,xn,
is also mutated into
Order is important:
first
then x x = x + N(0,1)
Rationale: new x , is evaluated twice
Primary: x is good if f(x) is good
Secondary: is good if the x it created is good
Lecture | 14.07.2011
36
Process Dynamics
and Operations
Evolution strategies: properties
Prerequisits for self-adaptation:
> 1 to carry different strategies
> to generate offspring surplus, e.g., 7
Survivor selection with high selection pressure
Parent selection:
With uniform random distribution
Each individual has the same probability to be selected
Lecture | 14.07.2011
37
Process Dynamics
and Operations
Recombination
Creates one child
Acts per variable / position by either
Averaging parental values,
Intermediate recombination
Selecting one of the parental values
Discrete recombination
From two or more parents by either:
local
Using two selected parents to make a child
Selecting two parents for each position anew global
Lecture | 14.07.2011
38
Process Dynamics
and Operations
Survivor selection
Basis of selection is either:
The set of children only: (,)-selection
The set of parents and children: (+)-selection
The set of parents with an age < generations and
children: (,,)-selection
(+)-selection is an elitist strategy
(,)-selection can forget
Often (,)-selection is preferred for:
Better in leaving local optima
Using the + strategy bad values can survive in x, too
long if their host x is very fit
Selective pressure in ES is very high ( 7 is the
common setting)
Lecture | 14.07.2011
39
Process Dynamics
and Operations
Uncorrelated mutation with one
Chromosomes: x1,,xn,
Mutation:
= exp( N(0,1))
xi = xi + N(0,1)
Mutation of strategy parameter
Mutation of object parameters
Typically the learning rate 1/ n
boundary rule: < =
minimal mutation strength =
Lecture | 14.07.2011
40
Process Dynamics
and Operations
Mutants with equal likelihood
Parent individual
defines the radius
Circle: mutants having the same chance to be created
Lecture | 14.07.2011
41
Process Dynamics
and Operations
Uncorrelated mutation with n step sizes
Chromosomes: x1,,xn, 1,, n
Mutation:
i = i exp( N(0,1) + Ni (0,1))
xi = xi + i Ni (0,1)
Two learning rate parmeters:
overall learning rate
coordinate wise learning rate
1
2n
and
1
2 n
And i < i =
Lecture | 14.07.2011
42
Process Dynamics
and Operations
Mutants with equal likelihood
Parent individual
i define the axes
Ellipse: mutants having the same chance to be created
Lecture | 14.07.2011
43
Process Dynamics
and Operations
Correlated mutation (CMA-ES)
Parent individual
Rotation angles are needed
Rotated ellipse: mutants having the same chance to be created
Lecture | 14.07.2011
44
Process Dynamics
and Operations
Covariance Matrix Adaptation - ES
Chromosomes: x1,,xn, 1,, n ,1,, k
where k = n (n-1)/2
I.e. for each pair of coordinates (i,j), i,j {1,,n}, ij one
rotation angle exists
and the covariance matrix C is defined as:
cii = i2
cij = 0 if i and j are not correlated
cij = ( i2 - j2 ) tan(2 ij) if i and j are correlated
Lecture | 14.07.2011
45
Process Dynamics
and Operations
Correlated mutations
The mutation mechanism is then:
i = i exp( N(0,1) + Ni (0,1))
j = j + N (0,1)
x = x + N(0,C)
x stands for the vector x1,,xn
C is the covariance matrix C after mutation of the
values
1
2n
and
and 5
2 n
i < i = and
| j | > j = j - 2 sign(j)
Lecture | 14.07.2011
46
Process Dynamics
and Operations
Mutants with equal likelihood
Parent individual
i define the axes
k define the rotation angles
Rotated ellipse: mutants having the same chance to be created
Lecture | 14.07.2011
47
Process Dynamics
and Operations
ES technical summary tableau
Representation
Real-valued vectors
Recombination
Discrete or intermediary
Mutation
Gaussian perturbation
Parent selection
Uniform random
Survivor selection
(,) or (+) or (,,)
Specialty
Self-adaptation of mutation step
sizes
Lecture | 14.07.2011
48
Process Dynamics
and Operations
Performance of methods on problems
EAs as problem solvers:
Goldbergs 1989 view
Special, problem tailored method
Evolutionary algorithm
Random search
Scale of all problems
Lecture | 14.07.2011
49
Process Dynamics
and Operations
Michalewicz 1996 view
Performance of methods on problems
EA 4
EA 2
EA 3
EA 1
P
Scale of all problems
Lecture | 14.07.2011
50
Process Dynamics
and Operations
EC and Global Optimization
Global Optimization: search for finding best solution x*
out of some fixed set S
Deterministic approaches
e.g. box decomposition (branch and bound etc)
Guarantee to find x* , but may run in super-polynomial
time
Heuristic Approaches (generate and test)
rules for deciding which x S to generate next
no guarantees that best solutions found are globally
optimal
Lecture | 14.07.2011
51
Process Dynamics
and Operations
