Model Order Reduction Using Genetic Algorithm
Ahmed Adel
Alexandria University
Alexandria, Egypt
[email protected]
Khaled Salah
Mentor Graphics
Cairo, Egypt
[email protected]
ABSTRACT
Model order reduction has been one of the most challenging
topics in the past years. Conventional mathematical methods have
been used to obtain a reduced order model of high order complex
models. In this paper, genetic algorithm (GA) which is one of the
artificial intelligence algorithms is used to approximate high-order
transfer functions (TFs) as lower-order TFs. Genetic algorithm is
considered as one of the evolutionary techniques which are used
in optimization problems. In this approach, genetic algorithm is
applied to model order reduction putting in consideration
improving accuracy and preserving the properties of the original
model which are two important issues for improving the
performance of simulation and computation and maintaining the
behavior of the original complex models being reduced. The
proposed technique could be used in EDA tools.
Keywords
Model reduction, crossover, selection, mutation,
algorithm, optimization, MSE, transfer function.
genetic
1. INTRODUCTION
In nature, biological structures that are more successful in
grappling with their environment survive and reproduce at a
higher rate[2]. Genetic algorithm is an artificial intelligence
algorithm based on natural selection which has been used in
optimization problems in the past years. Chromosomes are the
basic elements of genetics. Chromosomes of parents combine
with each other producing new individuals with new
chromosomes which have modified properties different from
properties of parents chromosomes. Successive generations of
individuals create a great probability for improvement over time.
Genetic algorithms were first developed by John Holland, his
colleagues, and his students at the University of Michigan for the
purpose of creating adaptive algorithms of natural systems and
designing system software based on them [1]. Holland introduced
the concept of genetic algorithm as a principle of Charles
Darwinian theory of evolution to natural biology [2]. Genetic
algorithm was proven to be a powerful technique in optimization
problems and it proves its efficiency mostly in problems requiring
efficient and effective search. The main reason for being powerful
in this area is although the algorithm is computationally simple
978-1-5090-1496-5/16/$31.00 2016 IEEE
its so powerful in search for improvement and development.
Another advantage concerning genetic algorithm over the random
search methods is parallelization of search due to the presence of
variety of individuals in each generation depending on specified
number of individuals where the fittest ones are selected for the
next generations.
In Mathematics, the desire to approximate complex functions to a
simpler form appeared early and there were several approaches
towards model order reduction. The first step in model order
reduction was taken by Lanczos where he found a way to reduce a
matrix in tridiagonal form. The fundamental methods in the area
of model order reduction were published in the eighties and the
nineties of the last century. Firstly, the method of balancedtruncated realization was published[10]. In the next years, Hankelnorm reduction method and Proper Orthogonal Decomposition
method were published [11][12]. All these methods were
developed in the field of systems and control theory. In more
recent years research has been developed in the field of model
order reduction and more methods have been evolved and variety
of these methods has been available. Some of these methods are
dedicated for special purposes and the others are for general
purposes.
Conventional mathematical methods were used over the past years
in model order reduction. These methods proved to be efficient in
terms of time, but it didnt prove to have the same efficiency in
terms of accuracy and preserving the basic properties of the
original model.
1.1 Related work
Genetic algorithm has been used in different optimization
problems and it was applied to model order reduction. Simple
schemes of artificial genetic algorithm were used to reduce a
models transfer function from s specific order to a lower one. The
previous work compares genetic algorithm with other artificial
intelligence algorithms using ready schemes on a chosen transfer
function of the 4th order[3] and 8th order [4] proving the possibility
of using these algorithms in this area of optimization.
1.2 Contribution
In our work, we introduce a genetic algorithm model. The
proposed model is fully generic and able to be optimized. The
proposed model reduces the order of complex models of high
order transfer function to any desired order model. The model
targets high accuracy and preserving the basic properties of the
original model. The results of the proposed model simulations is
compared to powerful conventional mathematical methods like:
Pade Approximation which is based on moments matching, Routh
approximation, and Truncated balanced method in terms of
accuracy of step response, time required for reducing the model,
and frequency response. The proposed model proved efficiency in
terms of accuracy over some methods and in terms of frequency
response over other methods.
Moreover, the implemented model add some modifications to the
arithmetic crossover operator by extending the search domain for
the solution where the initial population is generated in a specific
range and the range extended with the generations increasing
possibility to be used in applications as there is no range can be
efficiently defined for search based on physical reasonability.
1.3 Organization of the paper
The rest of the paper is organized as follows: section (2) we will
review the necessary background for the genetic algorithm.
Section (3): in this section we will discuss the proposed model
and methodologies used.
Section (4): we will show the implementation of the model.
Section (5): we will show the testing results for implementation of
the proposed model and a comparison between the model and
mathematical methods.
The final conclusion is in Section 6.
2. Background
Genetic algorithm is based on iterative procedures of search for
an optimal solution for a problem which have multiple local
minima or maxima. The algorithm passes through steps of
recombination including crossover, and mutation then selection
which increase the probability of finding the most optimum
solution, which is the reduced model with the least error
compared to the original transfer model. The error is compared
with the original transfer function in terms of fitness function.
Before applying the genetic operators, a method of encoding
should be chosen to represent the data either in float form which is
the raw form of the data or binary representation or any other
representation.
The crossover operator is a process of creating new individuals
by selecting two or more parents and passing them through
crossover procedures producing one or two or more individuals.
Unlike real life there is no obligation to be abided by nature rules,
so the new individual can have more than two parents. There is
more than one method for crossover process like simple crossover
which includes exchange of genes between the two chromosomes
according to a specified crossover rate.
Arithmetic crossover occurs by choosing two or more
individuals randomly from the current generation and multiplying
them by one random in the case of the Inevitability of the
presence of a certain defined domain and more than one random if
there is no physical need for a defined search domain.
The second process of genetic operators is mutation which is a
process that occurs to prevent falling of all solutions of the
population into a local optimum of the problem. The third genetic
operator is selection which is the process of choosing a certain
number of individuals for the next generation to be recombined
generating new individuals aiming to find the most optimum
solution. There is more than one technique to select individuals
for the next generation.
The Elitism selection is simply selecting the fittest individuals
from the current population for the next population. This method
guarantees a high probability of getting closer to the most
optimum solution due to passing of the fittest chromosomes to the
crossover operator producing fitter individuals. Another technique
is Roulette wheel selection; in this kind of selection the
parenthood probability is directly proportional to the fitness of the
individuals where every individual is given a weight proportional
to its fitness having a higher probability to be chosen as a parent
for the next generation individuals and this technique is similar to
rank selection where all individuals in the current population are
ranked according to their fitness. Each individual is assigned a
weight inversely proportional to the rank.
The fitness function is the common evaluation factor among all
selection techniques. Fitness function is a function which
evaluates the fitness of each individual to be selected for the next
generation. The fitness function is always a measure of error for
the step response of the reduced order model and the original
model. An example for the error calculation fitness function is the
mean square error which can be calculated as follows:
=
(1)
The term fi is calculated as the square of difference between the
original step response and the reduced transfer function step
response.
3. Proposed Method
Consider we have a linear system of nth order of q inputs and r
outputs described using state space in time domain as follows:
=
( ) +
( )=
( )
( )
(2)
(3)
Where H is n dimensional state vector, u is q dimensional control
vector and Y is r dimensional output vector with q n and r n.
Also, A is n n system matrix, B is nq input matrix and C is
rn output matrix.
By converting the state space model from time domain to
frequency domain, the transfer function can be described in
equation (4):
( )=
( )
( )
(4)
Where N(s) is the numerator and D(s) is the denominator of the
higher order system. Also, Ai and ai are the coefficients of
numerator and denominator respectively.
The problem is to find a mth lower order model Rm(s), where m<n
in the following form
( )=
( )
( )
(5)
In addition, we must take into consideration that the reduced
model maintains the characteristics of the original transfer
function.
Original TF
Reduced TF
Iterations
MSE
GA MOR
Desired order
MODEL
Gen. Population
Figure. 1 The GA MOR model
Time elapsed
step/freq. response
The proposed model is a 4 inputs and 4 outputs model. The
model is able to reduce a transfer function of any order to a lower
selected order. The inputs for the model are the original transfer
function, number of iterations, desired reduction order, and
generation population. The outputs are the reduced model transfer
function, the mean square error, the time elapsed for reducing the
transfer function, and the step and frequency responses of the
models. Sometimes more time is needed for some transfer
functions than the others to be reduced within the given criteria.
The model contains a monitoring system for the step response and
frequency response. The final implemented model is shown in
Figure. 1.
Identi fy fitness function
(MS E)
Create initi al population
Apply arithmetic
cros sover using 2 random
numbers
4. Implementation
The model is implemented using genetic operators: crossover
and selection. The mutation operator wasnt used as it proves no
extra improvement for the model however it adds more processing
time. After generating an initial population from a selected
domain, crossover operator is applied. Arithmetic crossover was
used as an operator by generating two random numbers which add
extra randomness and extend the domain of search of the genetic
algorithm.
There was no physical need for a fixed domain of search for the
coefficients of the transfer function. Selection operator was based
on elitism where the fittest individuals in the current population
(with the least mean square error) are chosen and passed to the
next generation. This method guarantees that the best individuals
are passed to the next generation and at the same time it provides
a high degree of randomness unlike the other methods. Other
methods like roulette wheel selection and methods based on
giving weight to the individuals have a great disadvantage where
individuals with higher weight may only pass to the next
population which decrease randomness and decrease the
probability of generating individuals of different properties. The
chosen fitness function is the mean square error. The proposed
methodology is shown in Figure. 2.
A pseudo code for the implemented model that describes the
procedures of implementation is given as follows:
Listing 1
Pseudo code of GA
Input: problem, Population size, desired order.
Output: best_solution
Create initial population
Current population=initial population
For (i=1 to iterations)
For (j=1: to population size)
Choose 2 random parents (current population)
Do crossover producing 2 individuals
End for
Add best solution to best solutions set
Choose the best individuals of new generation
Compute fitness function
for new genera tion an d
select the fittes t for next
generation
No
Iterations=max
iterations?
Yes
Compare the fittes t of al l
generations and choos e
the most o ptimum
solution
Figure. 2 GA-based MOR methodology.
5. Testing
The simulations are carried on Matlab 2012b software, core i7
processor, 6 GB Ram memory. The reduced order model is
compared to the original one in terms of mean square error.
The testing procedures includes a plot of step response for the
reduced and the original TF, frequency response using Bode plot,
mean square error, and the time elapsed.
The time taken for most of the original transfer functions to be
reduced to lower order is nearly the same in all transfer functions
but, sometimes some functions need more time than the others.
The testing procedures was carried on more than 30 different
transfer functions. The transfer functions have different forms of
step responses and frequency responses where the step response
of some of them has an exponential form and other responses
were sinusoidal. The proposed model showed a great efficiency in
terms of accuracy and preserving the original models properties.
The first transfer function is a 10th order transfer function
expressed as follows:
.
G1(s)=
.
.
.
Current generation= best individuals
End for
If (iterations= max)
Solution=best (best solutions)
End if
.
.
(6)
The reduced 2nd order model is expressed as follows:
( )=
.
.
.
.
(7)
The results of reducing the original model to a 2nd order model is
shown in Table. 1 and the step and frequency response is shown
in Figure. 3 and Figure. 4 respectively.
( )=
(9)
The results of reducing the original model of the second transfer
function to a 2nd order model is shown in Table. 2 and the step and
frequency response are shown in figure. 5 and Figure. 6
respectively.
Step Response
1.2
original model
1
The reduced 2nd order model is expressed as follows:
reduced model
0.8
Step Response
2.5
original model
Amplitude
0.6
reduced model
2
0.4
0.2
Amplitude
1.5
-0.2
-0.4
10
0.5
15
Time (seconds)
Figure. 3 Step response for the original and the reduced
model of G1(s) with MSE 0.07 %.
Time (seconds)
Figure. 5 Step response for G2(s) with MSE 0.18 %.
Bode Diagram
20
20
-20
-40
Magnitude (dB)
Magnitude(dB)
Bode Diagram
original model
reduced model
-60
90
-40
-60
0
45
Phase(deg)
-20
original model
Phase (deg)
0
-45
-90
-135
-2
10
-45
-135
10
10
-180
Frequency (rad/s)
-1
R1(s)
10
Iterations
MSE (%)
100
The second transfer function is a
expressed as follows:
Time
0.07
10th
23 sec
order transfer function
G2(s)=
.
.
.
.
.
.
.
.
10
10
10
10
Frequency (rad/s)
Figure. 6 The frequency response of the original and reduced
transfer models for G2(s) with mag. MSE 0.50%.
Table. 1 The results of the reduced order model G1(s)
Population
10
Figure. 4 The frequency response of the original and reduced
transfer models for G1(s) with mag. MSE 0.77%.
Metric
reduced model
-90
(8)
Table. 2 The results of the reduced order model G2(s)
Metric
Population
Iterations
MSE
(%)
Time
R2(s)
10
100
0.18
23 sec
5.1 Comparison with Related Work
The first comparison was made between the results of the reduced
order model of G1(s) and the mathematical methods. The step
responses are shown in Figure. 7 while the frequency responses
are shown in figure 8. The results of the proposed model and the
mathematical methods are compared in Table. 3.
Bode Diagram
Magnitude (dB)
20
0
original
-20
balanced
Pade
Routh
-40
reduced
-60
540
360
Phase (deg)
A comparison has been made between the results of the proposed
model and the results of mathematical methods of Pade
approximation, Routh approximation, and truncated-balanced
method. The proposed model showed efficiency in terms of
accuracy over Routh approximation while the results were nearly
equal compared with Pade approximation, but the accuracy of
Pade approximation was bad and sometimes unacceptable when
testing transfer function of sinusoidal step response. Truncatedbalanced method showed efficiency concerning accuracy over the
proposed model, but the proposed model showed efficiency over
it concerning frequency response as the frequency response of the
truncated-balanced method was not accurate enough in most of
the test cases and sometimes completely different from the
original model. Ready schemes for the mathematical methods
used in the experiments [13]- [15].
180
0
-180
-360
-2
10
-1
10
10
10
10
10
Frequency (rad/s)
Figure. 8 Frequency responses for the first transfer function
Table. 3 A comparison of the results of all methods for the
first transfer function.
compared with the mathematical methods for G1(s).
Metric
GA
Pade
Routh
Balanced
Step response MSE
(%)
0.07
1760
5.28
0.05
The second comparison is carried on G2(s) and the step responses
are shown in figure 9 while the frequency responses are shown in
figure 10. The results of the proposed model and the mathematical
methods are compared in Table 4.
Freq. response mag.
MSE (%)
0.77
34
100
0.63
Table. 4 A comparison of the results for all methods for the
second transfer function.
Metric
GA
Pade
Routh
Balanced
Freq. response
phase MSE
200
5*104
1.492*
103
1.2*105
Step response MSE
(%)
0.07
1760
5.28
0.05
Time
23 sec
0.1 sec
0.1 sec
0.1 sec
Freq. response mag.
MSE (%)
0.50
0.33
23.8
0.19
Freq. response
phase MSE
34
291.86
1*103
3.7*103
Time
23 sec
0.1 sec
0.1 sec
0.1 sec
Although Pade approximation showed efficiency in terms of
accuracy nearly similar to the efficiency of the proposed model
but the step response was completely different from the step
response of the original model in the first test case. The step
response of Pade approximation is shown in figure 8.
Step Response
2.5
Step Response
1.2
2
Amplitude
0.8
Amplitude
0.6
1.5
0.4
original transf er f unction
0.2
Blanced truncated
0.5
blanced truncated
Routh approximation
-0.2
reduced model
-0.4
Pade approximation
Routh approximation
Original model
10
15
Time (seconds)
Figure. 7 The step response of the proposed model compared
with the mathematical methods for G1(s).
Reduced model
0
Time (seconds)
Figure. 9 Step responses for the second transfer function
compared with the mathematical methods.
Electrical,
Computer,
Energetic,
Electronic
Communication Engineering Vol:6, No:9, 2012.
Bode Diagram
20
M
agnitude(dB
)
[4]
S.N. Sivanandam, S. N. Deepa, A Comparative Study Using
Genetic Algorithm and Particle Swarm Optimization for
Lower Order System Modeling, Department of Computer
Science and Engineering, PSG College of Technology.
[5]
P.O. Gutman, C.F. Mannerfelt and P. Molander,
Contributions to the model reduction problem, IEEE
Trans. Auto. Control, Vol. 27, pp 454-455, 1982.
-20
original
-40
balance
-60
Pade
Routh
-80
reduced
-100
45
0
P
hase(deg)
and
[6] E.S. Gopi, Algorithm Collections for Digital Signal
Processing Applications Using Matlab, National Institute of
Technology, Tiruchi, India.
-45
-90
-135
-180
0
10
10
Frequency
10
(rad/s)
Figure. 10 frequency responses for the second transfer
function compared with the mathematical methods.
Compared to other related work that uses GA in MOR, the
proposed model introduced a generic model achieving success in
reducing any model of 10th order or less to any desired lower
order. The simulations were carried on more than 30 models of
different transfer function having exponential and sinusoidal
forms of step responses including transfer functions having
variety of complex poles while the previous trials were applied on
specific models of 4th order in [3] and 8th order in [4] using simple
schemes. Another advantage for the proposed model that it can be
used without defining a specific range of search for the solution
and without additional procedures of gain adjusting and scaling as
in [5].
The experiments compared the proposed model with powerful
mathematical methods showing the efficiency of the proposed
model over the mathematical model in terms of accuracy and
preserving the properties of the original model. GA-based MOR is
a promising solution in reducing the circuit-level simulation time.
6. Conclusion
Model order reduction using genetic algorithm proved to be a
promising solution that can reduce complex high order models.
The proposed methodology showed efficiency in terms of
accuracy and preserving the basic properties of the original
complex model over conventional mathematical methods
increasing the opportunity for enhancing simulations and
increasing the accuracy of modeling.
REFERENCES
[1] J. Holland (1975), Adaptation in Natural and Artificial
Systems, University of Michigan Press.
[2] John R. Koza, Genetic Programming On the Programming of
Computers by Means of Natural Selection, 1992
Massachusetts Institute of Technology.
[3] S. Panda, J. S. Yadav, N. P. Patidar and C. Ardil,
Evolutionary Techniques for Model Order Reduction of
Large Scale Linear Systems, World Academy of Science,
Engineering and Technology International Journal of
[7] Model Order Reduction: Theory, Research Aspects and
Applications (2000), Wilhelmus H.A. Schilders, Henk A. van
der Vorst, Joost Rommes.
[8] Model Reduction for Circuit Simulation, Peter Benner,
Michael Hinze, and E. Jan W. ter Maten.
[9]
Genetic algorithm tutorial, Darrell Whitley, computer
science department, university of Colorado.
[10] Tombs, M., Postlethweite, I.: Truncated balanced
realization of a stable non-minimal state-space system.
Internat. J. Control, 46, 13191330 (1987).
[11] K. Glover. Optimal Hankel-norm approximations of linear
multivariable systems and their error bounds. Int. J. Control,
39(6):115-193, 1984.
[12] Gal Berkooz, Philip Holmes, and John L. Lumley, Cornell
University, Ithaca, New York 14853 ,THE PROPER
ORTHOGONAL DECOMPOSITION IN THE ANALYSIS
OF TURBULANT FLOWS.
[13] http://www.mathworks.com/matlabcentral/fileexchange/2139
3-balanced-truncation.
[14] http://www.mathworks.com/matlabcentral/fileexchange/2146
6-pade-approximation.
[15] http://www.mathworks.com/matlabcentral/fileexchange/2145
7-routh-approximation.
