NPTEL Online Certification Courses Indian Institute

of Technology Kharagpur

Foundations of Cyber-Physical Systems

Assignment- Week 9
TYPE OF QUESTION: MCQ/MSQ/Numerical

Number of questions: 10 Total mark: 10 X 3 = 30

Question 1

Choose the correct property of the weight matrices used in LQR loss function.
a. Symmetric and positive-definite
b. Not necessarily symmetric
c. Symmetric and positive-semidefinite
d. Symmetric and indefinite

Correct Answer: c. Symmetric and positive-semidefinite

Detailed Explanation: The weight matrices must be symmetric i.e. the elements above the principal
diagonal of the cost matrix must be equal to the elements below because the cost of (𝑥1 , 𝑥2 ) must be
same as the cost of (𝑥2 , 𝑥1 ). Also, cost must be non-negative in real-world.

Question 2

The objective of LQR is to design an optimal control signal that minimizes the following loss function
𝑁ℎ
𝑥(𝑡)
𝐽 = 𝐸 [∫ [𝑥 𝑇 (𝑡) 𝑢𝑇 (𝑡)]𝑄𝑐 [ ]]
0
𝑢(𝑡)
For solving the optimization using dynamic programming, we introduce the following notation,
𝑁−1

V𝑘 = min ∑ (𝑥 𝑇 (𝑖)𝑄1 𝑥(𝑖) + 𝑢𝑇 (𝑖)𝑄2 𝑢(𝑖) + 2𝑥 𝑇 (𝑖)𝑄12 𝑢(𝑖)) + 𝑥 𝑇 (𝑁)𝑄0 𝑥(𝑁)

𝑢(𝑘),…,𝑢(𝑁−1)
𝑖=𝑘
Here V𝑘 is the minimum loss from k to N-th iteration. Then state whether the following is true or false
min 𝐽 = 𝑉0

a. True
b. False

Correct Answer: a. True

Detailed Explanation:

Over a window of N iterations, dynamic programming will solve this minimization problem in a
backward approach. Solving 𝑉𝑘 , we will get the control signals 𝑢(𝑘), 𝑢(𝑘 + 1), … , 𝑢(𝑁) for which loss at
𝑘 − 𝑡ℎ iteration is minimized. Similarly, solving 𝑣0 we will get 𝑢(0), 𝑢(1), … , 𝑢(𝑁) that will minimize the
loss function for the entire window of N iterations which is nothing but the loss function 𝐽.
 NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur

Question 3

A first-order continuous system is represented by the following differential equation:

𝑥̇ (𝑡) = 𝑥(𝑡) + 𝑢(𝑡)
We want to design a feedback controller such that 𝑢(𝑡) = −𝐾𝑥(𝑡) and desired equilibrium condition is:
𝑥(𝑡) = 0 as 𝑡 → ∞. What should be the value of 𝐾 for the above system to be stable?

a. 𝐾 ≥1
b. 𝐾 ≤1
c. 𝐾 <1
d. 𝐾 >1

Correct Answer: d. 𝐾 > 1

Detailed Explanation:

𝑑𝑥(𝑡)
𝑥̇ (𝑡) = = 𝑥(𝑡) + 𝑢(𝑡) = 𝑥(𝑡) − 𝐾𝑥(𝑡) = (1 − 𝐾)𝑥(𝑡)
𝑑𝑡
𝑑𝑥(𝑡)
==> = (1 − 𝐾)𝑑𝑡
𝑥(𝑡)
𝑡 𝑡
𝑑𝑥(𝑡)
==> ∫ = ∫ (1 − 𝐾)𝑑𝑡
0 𝑥(𝑡) 𝑜
𝑡
==> [log(𝑥(𝑡))]𝑜 = (1 − 𝐾)𝑡
𝑥(𝑡)
==> log ( ) = (1 − 𝐾)𝑡
𝑥(0)
==> 𝑥(𝑡) = 𝑥(0)𝑒 (1−𝐾)𝑡

Therefore, for the system to be stable (i.e. 𝑥(𝑡) = 0 as 𝑡 → ∞), 𝐾 > 1.

Question 4

Consider the same system and stability criteria given in Question 3. The performance of the system is
measured by the following loss function:
∞
𝐽 = ∫ [𝑥 2 (𝑡) + 𝑚𝑢2 (𝑡)]𝑑𝑡
0
𝑑𝐽
Given 𝑚 > 1, compute 𝐾 such that 𝐽 is minimum. [Hint: find out value of 𝐾 for which 𝑑𝐾 = 0].

√𝑚2 +𝑚
a. 𝐾 = 1 + 𝑚
√𝑚2 −𝑚
b. 𝐾 = 1 + 𝑚
√𝑚2 +𝑚
c. 𝐾 = 1 − 𝑚
√𝑚2 −𝑚
d. 𝐾 = 1 − 𝑚

√𝑚2 +𝑚
Correct Answer: a. 𝐾 = 1 + 𝑚
 NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur

Detailed Explanation:

Replacing 𝑥(𝑡) by 𝑥(0)𝑒 (1−𝐾)𝑡 in J gives

∞
𝐽 = ∫ [𝑥 2 (𝑡) + 𝑚𝑢2 (𝑡)]𝑑𝑡
0
∞ ∞
 = ∫0 [1 + 𝑚𝐾 2 ]𝑥 2 (𝑡)𝑑𝑡 = 𝑥 2 (0)(1 + 𝑚𝐾 2 ) ∫0 𝑒 (1−𝐾)2𝑡 𝑑𝑡
𝑥 2 (0)(1+𝑚𝐾2 ) ∞ 𝑥 2 (0)(1+𝑚𝐾2 )
 = 2(1−𝐾)
[𝑒 (1−𝐾)𝑡 ]0 = −2(1−𝐾)
(since 𝐾 > 1)

For 𝐽 to be minimized,
𝑑𝐽 𝑥 2 (0) 1 2𝑚𝑘 𝑚𝐾 2
= [− + − ]=0
𝑑𝐾 2 (𝐾 − 1)2 𝐾 − 1 (𝐾 − 1)2
𝑥 2(0) 𝑥 2(0) 𝑥 2(0)
 𝑚𝐾 2 2
− 2𝑚𝐾 2
− 2
=0
𝑚𝑥 2 (0)±√𝑚 2 𝑥 4 (0)+𝑚𝑥 4 (0) √𝑚2 +𝑚
 𝐾= 2
𝑚𝑥 (0)
= 1 ± 𝑚
√𝑚2 +𝑚
Since, 𝐾, 𝑚 >1➔𝐾 =1+ 𝑚

Question 5

A first-order continuous system is represented by the following differential equation:

𝑥̇ (𝑡) = 𝑥(𝑡) + 𝑢(𝑡)
We want to design a feedback controller such that 𝑢(𝑡) = −𝐾𝑥(𝑡) and desired equilibrium condition is:
𝑥(𝑡) = 0 as 𝑡 → ∞. The performance of the system is measured by the following loss function:
∞
𝐽 = ∫ [𝑥 2 (𝑡) + 2𝑢2 (𝑡)]𝑑𝑡
0
𝑑𝐽
Compute 𝐾 such that 𝐽 is minimum. [Hint: find out value of 𝐾 for which 𝑑𝐾 = 0].

a. 0.22
b. 0.75
c. 2.22
d. -0.75

Correct Answer: c. 2.22

Detailed Explanation:

√𝑚2 +𝑚
From explanation given in Question 4, we get 𝐾 = 1 + 𝑚
√6
Given 𝑚 = 2, we get 𝐾 = 1 + 2
≈ 2.22
 NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur

Question 6

Flow function of a dynamical system is globally Lipschitz continuous implies

_________ ≤ 𝑘|𝑥1 − 𝑥2 | ∀𝑥 ∈ 𝒟
for a real constant 𝑘 ≥ 0 in domain 𝒟
a. |𝑓(𝑥1 ) − 𝑓(𝑥2 )|
b. |𝑓(𝑥1 , 𝑥2 )|
c. |𝑙𝑜𝑔(𝑓(𝑥1 )/𝑓(𝑥2 ))|
d. |𝑓(𝑥1 ) + 𝑓(𝑥2 )|

Correct Answer: a. |𝑓(𝑥1 ) − 𝑓(𝑥2 )|

Detailed Explanation: See the definition of Lipschitz continuity.

Question 7

Which of the following is an invalid constraint for designing a CLF-CBF based optimal control strategy
that outputs the optimal control action 𝑢 for a dynamical system 𝑥̇ = 𝑓(𝑥) + 𝑔(𝑥)𝑢 such that 𝑢𝐻𝑢𝑇 +
𝑝δ2 is minimum for a positive definite and symmetric 𝐻 where 𝛿 is a positive slack variable?

∂𝐵 ∂𝐵
a. ∂𝑥
𝑓(𝑥) + ∂𝑥 𝑔(𝑥)𝑢 ≤ −𝜆𝐵(𝑥), 𝑤ℎ𝑒𝑟𝑒 𝜆 > 0
∂V ∂𝑉
b. ∂𝑥
𝑓(𝑥) + ∂𝑥 𝑔(𝑥)𝑢 ≤ −𝛾𝑉(𝑥), 𝑤ℎ𝑒𝑟𝑒 𝛾 > 0
∂V ∂𝑉
c. 𝑓(𝑥) + 𝑔(𝑥)𝑢 ≤ −𝛾𝑉(𝑥) + 𝛿, 𝑤ℎ𝑒𝑟𝑒 𝛾 > 0
∂𝑥 ∂𝑥
∂𝐵 ∂𝐵
d. ∂𝑥
𝑓(𝑥) + ∂𝑥 𝑔(𝑥)𝑢 ≤ −𝜆𝐵(𝑥) + 𝛿, 𝑤ℎ𝑒𝑟𝑒 𝜆 > 0

Correct Answer: d

Detailed Explanation: See the definition of CLF-CBF-based optimal control in slide 31 of week 9.
The relaxation on CLF is used since the stability of the system does not require invariance of the initial
set of states i.e. the system states might go beyond the initial set of states before it approaches the
equilibrium point. Also, without relaxation the CLF constraint is true but hard to solve. On the other
hand, the CBF constraint must not be relaxed since the system states must lie within the safe set for
any trajectory i.e. the safe set of states must be invariant. Therefore a, b, and c are correct and d is
wrong.

Question 8

Which of the following are true as we say a set 𝒮 is invariant for a dynamic system state vector 𝑥 ?
a. 𝑥(𝑡0 ) ∈ 𝒮 ⇒ 𝑥(𝑡) ∈ 𝒮, ∀𝑡 ≥ 𝑡0
b. 𝑥(𝑡0 ) ∈ 𝒮 ⇒ 𝑥(𝑡) ∈ 𝒮, ∃𝑡 > 𝑡0
c. 𝑥(𝑡) ∈ 𝒮, ∀𝑡 ≥ 0
d. 𝑥(𝑡0 ) = 0 ⇒ 𝑥(𝑡) ∈ 𝒮, ∀𝑡 ≥ 𝑡0

Correct Answer: a. 𝑥(𝑡0 ) ∈ 𝒮 ⇒ 𝑥(𝑡) ∈ 𝒮, ∀𝑡 ≥ 𝑡0

Detailed Explanation: An invariant set of any set of states of a dynamical system are mapped by the
dynamic system flow into the other states that are in the same set. Hence a. 𝑥(𝑡0 ) ∈ 𝒮 ⇒ 𝑥(𝑡) ∈ 𝒮, ∀𝑡 ≥
𝑡0
 NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur

Question 9

Match the followings:

A. safe control 1. 𝑉̇ ≤ −γ𝑉, γ > 0 for CLF 𝑉
B. asymptotically stable control 2. 𝐵̇ ≤ −λ𝐵, γ > 0 for CBF B
C. exponentially stable control 3. 𝑉̇ < 0, γ > 0 for CLF 𝑉

(i) A matches with __, (ii)B matches with ___, (iii)C matches with ___ (choose among 1,2,3)

Correct Answer: (i) 2, (ii) 3, (iii) 1

Detailed Explanation:
Safe control must be imposed by the CBF criteria i.e. 2.
Asymptotic stability must be ensured by the negative slope of a CLF, hence 3.
Exponential stability is however a stronger notion of stability that is guaranteed by a negative slope of
CLF that is proportional to the current state of the CLF, hence 1.

Question 10

Find out whether the following statements are true or false

A. All sublevel sets of CLF are invariant
a. T, b. F
B. Zero sub-level set of CBF is invariant
a. T, b. F
C. Zero level set of CBF is safety boundary of the system
a. T, b. F
Correct Answer: A.a. T, B.a. T, C.a. T

Detailed Explanation: See the definitions in week 9 slides 26, 30