Theory in Model Predictive Control :"

Constraint Satisfaction and Stability!

Colin Jones, Melanie Zeilinger!

Automatic Control Laboratory, EPFL!

Example: Cessna Citation Aircraft!

Linearized continuous-time model:!
(at altitude of 5000m and a speed of 128.2 m/sec)!

!
1.2822
0
0.98
0
0.3

0
0
1
0
! x =
x + 0 u

5.4293
17
0
1.8366 0
!
128.2 128.2
0
0
0

!
0 1 0 0
y
=
x
!
0 0 0 1
!

Angle of attack!
V!
horizon!

Pitch angle!

Input: elevator angle!

States: x1: angle of attack, x2: pitch angle, x3: pitch rate, x4: altitude!
Outputs: pitch angle and altitude!
Constraints: elevator angle 0.262rad (15), elevator rate 0.524rad (60)!
!
!
! pitch angle 0.349 (30)!

Open-loop response is unstable (open-loop poles: 0, 0, -1.55942.2900i )!

[J. Maciejowski, Predictive Control with constraints, 2002]!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

2!

LQR and Linear MPC with quadratic cost!

Quadratic performance measure!
Linear system dynamics ! x + = Ax + Bu
!
y = Cx + Du
Linear constraints on inputs and states!
!
!!
!
LQR:!
MPC:!
!
N1

T
T
J (x) = min
xiT Qxi + uiT Rui
J (x) = min
xi Qxi + ui Rui
x,u
x,u
!
i=0
i=0
!

!
!
!
!
!

Z[ xi+1

x0

= Axi + Bui
=x

Z[ xi+1

x0

= Axi + Bui

=x

Cxi + Dui b
(ZZ\TL! Q = QT 0, R = RT 0

MPC problem can be translated into a quadratic program (QP)!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

3!

Linear MPC with linear costs!

Linear performance measure (e.g. economically motivated cost, errors)!
Linear system dynamics ! x + = Ax + Bu
y = Cx + Du
Linear constraints on states and inputs!
!

N1
Resulting MPC problem:!

J (x) = min
Qxi p + Rui p
!
x,u
i=0
!
Z[ xi+1
= Axi + Bui
!
Cxi + Dui b
!
x0
=x
!
!
Optimization problem can be translated into a linear program (LP) for p =1/!.

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

4!

Example: Cessna Citation Aircraft"

LQR with saturation!
Linear quadratic regulator with saturated inputs:!
!

100

100
200
0

1
2

2
10

Elevator angle u (rad)

Time (sec)
0.5

Pitch angle x2 (rad)

Altitude x4 (m)

At time t = 0 the plane is flying with a deviation of

10m of the desired altitude, i.e. x (0)=[0; 0; 0; 10]!
! 200
2

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10

! Closed-loop system is
unstable!
! Applying LQR control
and saturating the
controller can lead to
instability!

Input is saturated!

0.5
0

10

Time (sec)
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

5!

Example: Cessna Citation 500 aircraft"

MPC with bound constraints on inputs!

40

20

0.5

20
40
0

0.5
2

1
10

10

Elevator angle u (rad)

Time (sec)
0.5

0.5
0

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=10
Pitch angle x2 (rad)

Altitude x4 (m)

MPC controller with input constraints |ui |"0.262

The MPC controller uses

the knowledge that the
elevator will saturate, but
it does not consider the
rate constraints.!
!
! System does not
converge to desired
steady-state but to a
limit cycle!

Time (sec)
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

6!

Example: Cessna Citation Aircraft"

MPC with all input constraints!

20

0.2

10

0
10
0

0.2

0.4
10

10

Time (sec)

Elevator angle u (rad)

Pitch angle x2 (rad)

Altitude x4 (m)

MPC controller with input constraints |ui |"0.262

and rate constraints! |ui | 0.349
approximated by! |uk uk 1 | 0.349Ts , u 1 = upr ev

0.2
0.1

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=10

The MPC controller

considers all constraints
on the actuator!
!
! Closed-loop system is
stable!!
! Efficient use of the
control authority!

0
0.1
0.2
0

4
Time (sec)

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

7!

Example: Cessna Citation Aircraft"

Inclusion of state constraints!

Altitude x4 (m)

150

0.5

100

50

0.5

0
50
0

Pitch angle #-0.9, i.e. -50

4

1
10

10

Elevator angle u (rad)

Time (sec)
0.5

0.5
0

Pitch angle x2 (rad)

MPC controller with input constraints |ui |"0.262

and rate constraints! |ui | 0.349
approximated by! |uk uk 1 | 0.349Ts , u 1 = upr ev

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=10

Increase step:!
At time t = 0 the plane is
flying with a deviation of
100m of the desired
altitude, !
i.e. x (0)=[0; 0; 0; 100]!
!
! Pitch angle too large !
during transient!
!

Time (sec)
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

8!

Example: Cessna Citation Aircraft"

Inclusion of state constraints!

Altitude x4 (m)

150
100

Constraint on pitch angle active!

50

0.2
0

0
50
0

0.4

Pitch angle x2 (rad)

MPC controller with input constraints |ui |"0.262

and rate constraints! |ui | 0.349
approximated by! |uk uk 1 | 0.349Ts , u 1 = upr ev

0.2
2

0.4
10

10

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=10

Add state constraints for

passenger comfort:!
!

|x2| "0.349

Elevator angle u (rad)

Time (sec)
0.5

0.5
0

4
Time (sec)

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

9!

Example: Cessna Citation Aircraft"

Shorter horizon causes loss of stability!

Altitude x4 (m)

20

0.5

20
0

Pitch angle x2 (rad)

MPC controller with input constraints |ui |"0.262

and rate constraints! |ui | 0.349
approximated by! |uk uk 1 | 0.349Ts , u 1 = upr ev

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=4

Decrease in the prediction

horizon causes loss of the
stability properties

0.5
10

Elevator angle u (rad)

Time (sec)
0.5

This part of the workshop:!

How can constraint
satisfaction and stability in
MPC be guaranteed? !

0.5
0

10

Time (sec)
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

10!

Outline!
Motivating Example: Cessna Citation Aircraft!
!
Constraint satisfaction and stability in MPC!
Main idea !
Problem setup for the linear quadratic case!
!
How to prove constraint satisfaction and stability in MPC!
!
Implementation !
!
Theory extensions !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

11!

Loss of feasibility and stability guarantees!

What can go wrong with standard MPC approach?!
!

! No feasibility guarantee, i.e. the MPC problem

may not have a solution !
! No stability guarantee, i.e. trajectories may not
converge to the origin!
!

min
x,u

Z[

N1

xiT Qxi + uiT Rui

i=0

xi+1

= Axi + Bui

x0

=x

Cxi + Dui b

!
!
Definition: Feasible set!
! ;OL MLHZPISL ZL[ X PZ KLULK HZ [OL ZL[ VM PUP[PHS Z[H[LZ x MVY ^OPJO [OL 47*
N
! WYVISLT^P[OOVYPaVU N PZMLHZPISL PL
! X := {x | [u , . . . , u ] Z\JO[OH[ Cu + Dx b, i = 1, . . . , N}
N
0
N1
i
i
!
!
!
!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

12!

Example: Loss of feasibility !

1 1
0
=
x+
u
0 1
1

Consider the double integrator:! x +

!
subject to the input constraints! 0.5 u 0.5


!
5
5

and the state constraints!

!
!
! .!
5
5

!
1 0
, !R = 10 !. !
We choose N =! 3, Q = !
4
0 1
3.5
!
Time step 1: x (0)=[-4 ; 4] , u0*(x)=-0.5
Time step 3: x (2)=[ 3 ; 2] MPC problem !
!

!
!
!

infeasible!

3
2.5

x2

Time step 2: x (1)=[ 0 ; 3] , u0*(x)=-0.5

Set of initial
feasible states!
(feasible set)!

x(0)

x(1)

x(3)

1.5
1
0.5
0
5

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

0
x1

13!

Example: Loss of stability!

2 1
1
=
x+
u
0 0.5
0

Consider the unstable system:! x +

!
1 u 1
subject to the input constraints!

!
10
10

and the state constraints!

10
10
!

1 0
We choose Q =!
! !
0 1
!
and investigate the stability properties for different horizons N and weights R by
solving the finite horizon MPC problem in a receding horizon fashion !
!
!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

14!

Example: Loss of stability !

!

R=10, N=2
R=2, N=3
R=1 , N=4

x2

* Initial points leading to trajectories !

that converge to the origin!
" Intitial points that diverge!
!

0
x1

10

0
x1

10

10
5

x2

x2

10
10

10

0
5

5
10
10

10

0
x1

10

10
10

Parameters have complex effect on closed-loop trajectory!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

15!

Feasibility and stability in MPC Main Idea!

Main idea: !
Introduce terminal cost and constraints to explicitly ensure stability and
feasibility:!
!
N1

!
J (x) = min
xiT Qxi + uiT Rui + xNT P xN Terminal cost!
x,u
!
i=0
!
Z[ xi+1
= Axi + Bui
!
Cxi + Dui b
!
xN
Xf Terminal constraint!
!
Xf !
x0
=x
!
t!
xN
!
x0=x
! How to choose P and Xf ?
!
!
!"#$%!#&'('$)#*$+(
!
+!,$*!#$*-

16!

How to choose the terminal set and cost"

Main Idea!
Problems originate from the use of a short sight strategy!
! Finite horizon causes deviation between the open-loop prediction and
the closed-loop system:!

Closed-loop
trajectories!

Open-loop
predictions!

5
5

x2

!
!
!
!

x2

0
x1

Set of feasible
initial states for
open-loop
Set of initial
prediction!
states leading to
feasible closedloop trajectories!

5
5

0
x1

Ideally we would solve the MPC problem with an infinite horizon, but that is
computationally intractable!
!

! Design finite horizon problem such that it approximates the infinite horizon !
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

17!

How to choose the terminal cost!

We can split the infinite horizon problem into two subproblems:!
!
Up to time k =N, where the
For k >N, where there are no
!
constraints may be active!
constraints active!
J (x) = min
x,u

Z[

!
!

N1

xiT Qxi + uiT Rui

i=0

xi+1

= Axi + Bui

Cxi + Dui b
x0

=x

+ min
x,u

xiT Qxi + uiT Rui

i=N

Z[

+ xNT P xN
!
!

xi+1

= Axi + Bui ,

Unconstrained LQR !
!starting from state xN

Bound the tail of the infinite horizon cost from N to ! using the !
LQR control law u=KLQR x
xNT P xN is the corresponding infinite horizon cost!
P is the solution of the discrete-time algebraic Riccati equation!
!

Choice of N such that constraint satisfaction is guaranteed? !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

18!

How to choose the terminal set!

Terminal constraint provides a sufficient condition for constraint satisfaction :!
!
! J (x) = min
x,u

Z[

N1

xiT Qxi + uiT Rui + xNT P xN

i=0

xi+1

= Axi + Bui

Cxi + Dui b
xN

Xf

Infinite horizon cost !

starting from xN
!

Xf !
t!

x0=x

xN

!
x0
=x
!
All input and state constraints are satisfied for the closed-loop system using
the LQR control law for! x Xf
Terminal set is often defined by linear or quadratic constraints!
!

! The bound holds in the terminal set and is used as a terminal cost!
! The terminal set defines the terminal constraint!
!

In the following: Show that this problem setup provides feasibility and stability!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

19!

Outline!
Motivating Example: Cessna Citation Aircraft!
!
Constraint satisfaction and stability in MPC!
Main idea !
Problem setup for the linear quadratic case!
!
How to prove constraint satisfaction and stability in MPC!
!
Implementation !
!
Theory extensions !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

20!

Formalize goals: "

Definition of feasibility and stability!
Goal 1: Feasibility at all times!
!Definition: Recursive feasibility!
!The MPC problem is called recursively feasible, if for all feasible initial states
!feasibility is guaranteed at every state along the closed-loop trajectory. !
!
Goal 2: Stability!
!
Definition:
Lyapunov stability!
!;OLLX\PSPIYP\TWVPU[H[[OLVYPNPUVMZ`Z[LT x(k + 1) = Ax(k) + B(x(k)) =
!f (x(k)) PZ ZHPK [V IL 3`HW\UV] Z[HISL PU X PM MVY L]LY` > 0 [OLYL L_PZ[Z H
!() > 0 Z\JO[OH[ MVYL]LY` x(0) X !
x(0) () x(k) < k N .

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

x(0)

21!

Employed concept for the analysis of feasibility: "

Invariant sets!
! Definition: Invariant set!
! ( ZL[ O PZJHSSLK WVZP[P]LS`PU]HYPHU[ MVYZ`Z[LT x(k + 1) = f (x(k)) PM
!
x(k) O f (x(k)) O, k N .
!
The positively invariant set that contains every closed positively invariant set is
called the maximal positively invariant set O! .!
!
Infeasible after
one step!

Infeasible after
two steps!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

Invariant!
! Recursively!
feasible!
!

22!

Analysis of Lyapunov stability!

Lyapunov stability will be analyzed using Lyapunovs direct method:!
Definition: Lyapunov function!
3L[ X ILHWVZP[P]LS`PU]HYPHU[ZL[MVYZ`Z[LT x(k + 1) = f (x(k)) JVU[HPUPUNH
ULPNOIVYOVVK VM [OL VYPNPU PU P[Z PU[LYPVY ( M\UJ[PVU V : X R+ 1 PZ JHSSLK H
3`HW\UV]M\UJ[PVU PU X PMMVYHSS x X !

V (x) > 0 x = 0, V (0) = 0,

V (x(k + 1)) V (x(k)) 0

Theorem: (e.g. [Vidyasagar, 1993])!

0M H Z`Z[LT HKTP[Z H 3`HW\UV] M\UJ[PVU PU X  [OLU [OL LX\PSPIYP\T WVPU[ H[ [OL
VYPNPUPZ 3`HW\UV]Z[HISL PU X 
1 For

simplicity it is assumed that V(x) is continuous. This assumption can be relaxed by requiring an additional !
state dependent upper bound on V (x), see e.g. [Rawlings & Mayne, 2009] !
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

23!

How to prove feasibility and stability of MPC!

Main steps:!
Prove recursive feasibility by showing the existence of a feasible control
sequence at all time instants when starting from a feasible initial point!
! NOTE: Recursive feasibility does not imply stability of the closed-loop system!
!

Prove stability by showing that the optimal cost function is a Lyapunov

function!
!
!

We will discuss two main cases in the following:!

Terminal constraint at zero:
!
!xN = 0
Terminal constraint in some (convex) set: !xN & Xf!
!
For simplicity, we use the more general notation:!
!N 1
!
J (x) = minx,u
i=0 l(xi , ui ) + Vf (xN )
!

stage cost! terminal cost!

( In the quadratic case: l(xi!, ui ) = xiT! Qxi + uiT!Rui , Vf (x!N ) = xNT P! xN )!
!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

24!

Stability of MPC Zero terminal state constraint!

Terminal constraint xN= 0 :!

Assume feasibility of x and let [u0*, u1*, . . .,uN* -1 ] be the optimal

control sequence computed at x!

At x (k + 1) the control sequence [u1*, u2*, . . .,uN* -1 , 0] is feasible

(apply 0 control input to stay at the origin)!

! Recursive feasibility!

The associated cost function value is given by:!

x1
x2
x0 = x
x5 = 0
x3
x4

!
!

We obtain for the optimal solution!

!!

!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

25!

Stability of MPC Zero terminal state constraint!

Terminal constraint xN= 0 :!

Assume feasibility of x and let [u0*, u1*, . . .,uN* -1 ] be the optimal

control sequence computed at x!
At x + the control sequence [u1*, u2*, . . .,uN* -1 , 0] is feasible
(apply 0 control input to stay at the origin)!

! Recursive feasibility!

The associated cost function value is given by:!

x1 = x +
x2
x0 = x
x5 = 0
x3
x4

!
!

We obtain for the optimal solution!

!!

!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

26!

Stability of MPC Zero terminal state constraint!

Terminal constraint xN= 0 :!

Assume feasibility of x and let [u0*, u1*, . . .,uN* -1 ] be the optimal

control sequence computed at x!
At x + the control sequence [u1*, u2*, . . .,uN* -1 , 0] is feasible
(apply 0 control input to stay at the origin)!

! Recursive feasibility!

!
!

The associated cost function value is given by:!

+ ) = J (x) l(x, u0 ) + l(0, 0)
J(x

x2
x0 = x
x5 = 0
x3
x4

Subtract cost Add cost for

staying at 0!
at stage 0!

x1 = x +

We obtain for the optimal solution! J (x) ! J(x)

!
!! J (x + ) J (x) J(x
+ ) J (x) l(x, u0 ) 0

! J*(x) is a Lyapunov function ! (Lyapunov) Stability!

!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

27!

Extension to more general terminal sets!

Problem: The terminal constraint xN= 0 reduces the feasible set!

Goal: " Use convex set for Xf that contains the origin in its interior!
!

Example:!
3

x2

!
!

Feasible
set for xN & Xf !
!!
!
Feasible set for xN= 0 !
!
Xf!

3
6

0
x1

Double Integrator: !

1
1
0
x+ =
x+
u
0 1
1


5
5
x
0.5 u 0.5
5
5

1 0
N = 5, Q =
, R = 10
0 1

How can the proof be generalized to the constraint xN Xf ?!

!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

28!

Stability of MPC Main result!

Consider that the following standard assumptions hold: !
!

The stage cost is a positive definite function, i.e. it is strictly positive and
only zero at the origin!
!

The terminal set is invariant under the local control law 'f (x):
x + = Ax + Bf (x) Xf MVYHSS x Xf
!
!

All state and input constraints are satisfied in Xf :!

Xf X, f (x) U MVYHSS x Xf

The terminal cost is a continuous Lyapunov function in the terminal set Xf :!

!
Vf (x + ) Vf (x) l(x, f (x)) MVYHSS x Xf
!
! The closed-loop system under the MPC control law is stable in the feasible !
set XN . !
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

29!

Stability of MPC Outline of the proof!

Assume feasibility of x and let [u0*, u1*, . . .,uN* -1 ] be the optimal

control sequence computed at x!

x1

At x (k + 1) the control sequence [u1*, u2*, . . .,'f (xN*)] is feasible:

xN is in Xf ! 'f (xN*) is feasible and xN+=AxN*+B 'f (xN*) in Xf !

!

Xf

x5

! Terminal constraint provides recursive feasibility!

x2
x3

x4

x0 = x

The associated cost function value is given by:!

! Vf (x) is a Lyapunov function: " 0

We obtain for the optimal solution!

!!
! :!

!
!

! J*(x) is a Lyapunov function ! (Lyapunov) Stability !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

30!

Stability of MPC Outline of the proof!

Assume feasibility of x and let [u0*, u1*, . . .,uN* -1 ] be the optimal

control sequence computed at x!

x1 = x +

At x + the control sequence [u1*, u2*, . . .,'f (xN*)] is feasible:

xN is in Xf ! 'f (xN*) is feasible and xN+=AxN*+B 'f (xN*) in Xf !

!

x5

! Terminal constraint provides recursive feasibility!

x2
x3

x4

x0 = x
x6 Xf

The associated cost function value is given by:!

! Vf (x) is a Lyapunov function: " 0

We obtain for the optimal solution!

!!
! :!

!
!

! J*(x) is a Lyapunov function ! (Lyapunov) Stability !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

31!

Stability of MPC Outline of the proof!

Assume feasibility of x and let [u0*, u1*, . . .,uN* -1 ] be the optimal

control sequence computed at x!
At x + the control sequence [u1*, u2*, . . .,'f (xN*)] is feasible:

xN is in Xf ! 'f (xN*) is feasible and xN+=AxN*+B 'f (xN*) in Xf !

!

! Terminal constraint provides recursive feasibility!

!

!
!

x0 = x
x6 Xf
x5

x2
x3

x4

The associated cost function value is given by:!

= J (x) l(x, u0 ) + Vf (
xN+1 ) Vf (xN ) + l(xN , f (xN ))
J(x)

!
!

x1 = x +

Vf (x) is a Lyapunov function: " 0

!!

:!
We obtain for the optimal solution! J (x) !J(x)

+ ) J (x) l(x, u0 ) 0
J (x + ) J (x) J(x

! J*(x) is a Lyapunov function ! (Lyapunov) Stability !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

32!

Stability of MPC Remarks !

The terminal set Xf and the terminal cost ensure recursive feasibility and
stability of the closed-loop system. !
But: the terminal constraint usually reduces the region of attraction.!
!

If the open-loop system is stable, Xf can be chosen as the positively

invariant set of the system under zero control input, which is feasible.!
!

Often no terminal set Xf , but N is required to be sufficiently large to ensure

recursive feasibility!
! Applied in practice, makes MPC work without terminal constraint!
! Determination of a sufficiently long horizon difficult!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

33!

Proof of asymptotic stability!

! Definition: Asymptotic stability!
! .P]LUH70 ZL[ X PUJS\KPUN[OLVYPNPUHZHUPU[LYPVYWVPU[ [OLLX\PSPIYP\TWVPU[H[
! [OLVYPNPUVMZ`Z[LT x(k + 1) = f (x(k)) PZZHPK[VIL HZ`TW[V[PJHSS`Z[HISL PU X
PMP[PZ
!
%
!

3`HW\UV]Z[HISL

H[[YHJ[P]LPU X  PL limk x(k) = 0 MVYHSS x(0) X 

x(0)

Extension of Lyapunovs direct method: (see e.g. [Vidyasagar, 1993])!

If the continuous Lyapunov function additionally satisfies!
V (x(k + 1)) V (x(k)) < 0 x #= 0
!
then the closed loop system converges to the origin and is hence asymptotically stable.!
!

Recall: Decrease of the optimal MPC cost was given by!

!
J (x(k + 1)) J (x(k)) l(x(k), u0 )
where the stage cost was assumed to be positive and only 0 at 0.!
!

! The closed-loop system under the MPC control law is asymptotically stable.!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

34!

Extension to nonlinear MPC!

Consider the nonlinear system dynamics:! x + = f (x, u)
!

! Nonlinear MPC problem:!

J (x) = min
x,u

Z[

N1

i=0

xi+1

= f (xi , ui )

Cxi + Dui

xN
x0

l(xi , ui ) + Vf (xN )

Xf ,
=x ,

Presented assumptions on the terminal set and cost did not rely on linearity!
Lyapunov stability is a general framework to analyze stability of nonlinear
dynamic systems!
!

! Results can be directly extended to nonlinear systems. !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

35!

Outline!
Motivating Example: Cessna Citation Aircraft!
!
Constraint satisfaction and stability in MPC!
Main idea !
Problem setup for the linear quadratic case!
!
How to prove constraint satisfaction and stability in MPC!
!
Implementation !
!
Theory extensions !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

36!

Multi-Parametric Toolbox!

Modeling!

Optimal control!

Analysis!

Computational !
geometry!

MPT Toolbox: M. Kvasnica, P. Grieder and M. Baotic!

Real-Time Model Predictive Control via Multi-Parametric Programming: Theory and Tools!
Michal Kvasnica!
ISBN: 3639206444!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

37!

Implementation using Matlab and MPT!

Matlab!

1. Compute terminal weight P and LQR

control law K by solving the discrete time
Riccati equation!
!
[K,P]=dlqr(A,B,Q,R)
!
!

2. Compute terminal set Xf :

Ellipsoidal invariant set of the form!
!
Xf := {x | x T PE x 1}

Double Integrator: !

1
1
0
x+ =
x+
u
0 1
1


5
5
x
0.5 u 0.5
5
5

1 0
N = 5, Q =
, R = 10
0 1

! Can be written in the form of a !

! Linear Matrix inequality (LMI)!
!![Boyd et al., LMIs in System and Control Theory, 1994]!

Polytopic invariant set of the form!

Xf := {x | Hx K}
!
!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

hi
Xf

38!

Implementation using Matlab and MPT:"

Polytopic Invariant Terminal Set!
Linear system x+ = Ax + Bu
LQR controller u = K x
Input and state constraints !U = {u | umin u umax } X = {x | xmin x xmax }

M X = polytope([I;-I],[xmax; -xmin]) % State constraints

P
T! U = polytope([K;-K],[umax; -umin]) % Input constraints

Compute maximum invariant set that satisfies the constraints!

O = {xk | (A + BK)k xk O , k 0}
M P = X & U
P while 1
T!
Pprev = P;

P = P & inv(A+BK)*P;
if Pprev == P, break; end
end
!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

Iteration 1!
Iteration 3!
Iteration 2!

39!

Implementation using Matlab and MPT:"

Polytopic Invariant Terminal Set!
Linear system x+ = Ax + Bu
LQR controller u = K x
Input and state constraints !U = {u | umin u umax } X = {x | xmin x xmax }

M X = polytope([I;-I],[xmax; -xmin]) % State constraints

P
T! U = polytope([K;-K],[umax; -umin]) % Input constraints

Compute maximum invariant set that satisfies the constraints!

O = {xk | (A + BK)k xk O , k 0}
M P = X & U
P while 1
T!
Pprev = P;

P = P & inv(A+BK)*P;
if Pprev == P, break; end
end
!

Iteration 1!
Iteration 3!
Iteration 2!

Simpler procedure
M X = mpt_infset(A+BK, X&U, 100);
P
T!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

40!

Example: Cessna Citation Aircraft"

Revisited!

Altitude x4 (m)

20

0.5

20
0

Pitch angle x2 (rad)

MPC controller with input constraints |ui |"0.262

and rate constraints! |ui | 0.349
approximated by! |uk uk 1 | 0.349Ts , u 1 = upr ev

0.5
10

10

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=4

Decrease in the prediction

horizon causes loss of the
stability properties

Elevator angle u (rad)

Time (sec)
0.5

0.5
0

4
Time (sec)

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

41!

Example: Cessna Citation Aircraft"

Terminal cost and constraint provide stability guarantee!

20

0.2

10

0
10
0

Pitch angle x2 (rad)

Altitude x4 (m)

MPC controller with input constraints |ui |"0.262

and rate constraints! |ui | 0.349
approximated by! |uk uk 1 | 0.349Ts , u 1 = upr ev

0.2

0.4
10

10

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=4

! Inclusion of terminal
cost and constraint
provides stability

Elevator angle u (rad)

Time (sec)
0.2
0.1
0
0.1
0.2
0

4
Time (sec)

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

42!

Outline!
Motivating Example: Cessna Citation Aircraft!
!
Constraint satisfaction and stability in MPC!
Main idea !
Problem setup for the linear quadratic case!
!
How to prove constraint satisfaction and stability in MPC!
!
Implementation !
!
Theory extensions !
Reference Tracking
!!
Uncertain Systems!
Soft constrained MPC!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

43!

Extensions "
Reference Tracking!
!

Consider the system !x + = Ax + Bu

!
y = Cx
!

Regulation vs. Tracking:!

Regulation: !Reject disturbances around one desirable steady-state!
Tracking: !Make output follow a given reference r !
!

Steady-state computation:! !I A B " !x " !0"

ss
=
Compute steady-state that yields !
r
C
0
uss
the desired output!

Note: Many
solutions possible!

For a solution to exist the matrix must be full column rank and the steadystate must be feasible, i.e.! xss X, uss U

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

44!

Example: Cessna Citation Aircraft"

Reference tracking!

Altitude x4 (m)

0.5
200

200
0

Elevator angle u (rad)

Pitch angle x2 (rad)

Track given altitude profile:!

10
Time (sec)

15

0.5
20

10
Time (sec)

15

20

Problem parameters: !
Sampling time Ts=0.25sec,!
Q=I, R=10, N=10

0.2
0
0.2
0

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

45!

Extensions "
Reference Tracking!
MPC problem for reference tracking:!
Penalize the deviation from the desired steady-state !
!
N1

(xi xss )T Q(xi xss ) + (ui uss )T R(ui uss ) + (xN xss )T P (xN xss )
! J (x) = min
x,u
i=0
!
Z[ xi+1
= Axi + Bui
Here: Reference assumed constant
!
over horizon. !
Cxi + Dui
b
If reference trajectory is known this can
!
x0
=x
be included ! Preview !
!
!
Steady-state is computed at each time step!
Same problem structure as the standard MPC problem with additional
parameter r
!

Steady-state offset: If the model is not accurate the output will show an offset
from the desired reference!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

Note: Parameterized terminal set required for stability.!

46!

Extensions "
Offset-free reference tracking!
Goal: Zero steady-state tracking error, i.e. y (k) r (k) 0 MVY k
!

Consider the augmented model:! x + = Ax + Bu + Bw w

!
y = Cx + Cw w
with w & Rw.
!
"
A I Bw
Assume size of w =number of states , Cw=I and det
!
!
! 0 .!
"=
C
I
Then the augmented system is observable.!
!

! We can correct for the disturbance by choosing xss, uss such that!
"
!
"! " !
I A B xss
Bw w
=
r Cw w
C
0
uss
!

where w is an estimate of the uncertainty obtained from a state observer.

! See [Mder et al., Automatica 2009] for more details. !

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

47!

Extensions"
Uncertain systems !
In practice, the nominal system model will not be accurate due to model
mismatch or external disturbances that are acting on the system.!
!

Consider the uncertain system !

!
!x += Ax + Bu+Bww
where w &W is a bounded disturbance.!
!

Stability:!
In the presence of uncertainties, asymptotic stability of the origin can often
not be achieved. Instead, it can be shown that the trajectories converge to a
neighborhood of the origin.!
Stability can be analyzed in the framework of input-to-state stability.!
Idea: Effect of the uncertainty on the system is bounded and depends on
the size of the uncertainty.
!

Feasibility?!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

48!

Extensions"
Approaches for uncertain systems !
x
terminal
t!
set!

x+=Ax+Bu+Bww
Trajectories differ from what is expected using the
nominal system model!
! Uncertainties can lead to infeasibility of the
MPC problem!

Soft constrained MPC:!

Idea: Tolerate temporary violation of state constraints by constraint relaxation!
! Feasibility of the optimization problem despite disturbances!
!

Robust MPC:!
Idea: Design MPC problem for the worst-case disturbance by constraint
! tightening !

!! Constraint satisfaction and stability of the uncertain system!

!
!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

49!

Extensions"
Soft constrained MPC!
Idea: Input constraints are hard (e.g. actuator limitations),!
state constraints may (temporarily) be violated!
!

Introduce soft state and terminal constraints by means of slack variables !

Introduce penalties on the slack variables in the cost!
J (x) = min
x,u

Z[

xi+1

N1

l(xi , ui ) + l (i ) + Vf (xN ) + l (N ) Penalties on the amount of

i=0
constraint violation!
= Axi + Bui

Cxi + Dui b + i
GN xN

x0

fN + N

Slack variables!

=x

Standard soft constrained MPC setup does not provide stability guarantees!
!

! New soft constrained MPC setup with (robust) stability properties developed !
in [Zeilinger et al., CDC 2010]!
!"#$%!#&'('$)#*$+(
+!,$*!#$*-

50!

Further extensions!
Reference tracking with recursive feasibility guarantees!
[Gilbert et al., 1994; Bemporad 1998; Gilbert & Kolmanovsky 1999, 2002; Limon et al., 2008;
Borrelli et al., 2009; Ferramosca et al., 2009; ]!

Move blocking: Reduce the computational complexity by fixing the inputs or

its derivatives to be constant over several time steps!
[Li & Xi 2007, 2009; Cagienard et al. 2007; Gondhalekar & Imura, 2010; ]!

Stochastic MPC: Consider uncertainties that are unbounded and/or follow a

certain distribution!
Probabilistic constraints!
Expected value constraints!
Expected value cost!
[Lee et al. 1998; Couchman et al., 2005; Cannon et al., 2007; Grancharova et al. 2007;
Hokayem et al., 2009; Cinquemani et al., 2011; ]!

!"#$%!#&'('$)#*$+(
+!,$*!#$*-

51!