NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur
Foundations of Cyber-Physical Systems
Assignment- Week 10
TYPE OF QUESTION: MCQ/MSQ/Numerical
Number of questions: 10 Total mark: 10 X 3 = 30
Question 1
Consider the following neural network:
𝑏11 𝑏12
1 2
𝑤1,1 1
𝑤1,1
𝑝1 𝑖1 𝑓 𝑜1 𝑞1
1
𝑤2,1 2
𝑤2,1
2
1
𝑤1,2
𝑤1,2
𝑝2 𝑖2 1 𝑓1 2 𝑜2 𝑞2
𝑤2,2 𝑤2,2
𝑏21 𝑏22
Input Hidden Output
layer layer layer
1 1 1 1
Consider initially 𝑤1,1 = 0.1, 𝑤2,1 = 0.2, 𝑤1,2 = 0.3, 𝑤2,2 = 0.4, 𝑏11 = 𝑏21 = 0.25, 𝑤1,1
2 2
= 0.5, 𝑤2,1 2
= 0.6, 𝑤1,2 =
2 2 2
0.7, 𝑤2,2 = 0.8, 𝑏1 = 𝑏2 = 0.35.
𝑝 0.6
Find out the values p and q in the weight matrix [ ] for the output layer
0.7 𝑞
0.5 0.7
Correct Answer: [ ]
0.6 0.8
2 2
𝑤1,1 𝑤1,2 0.5 0.6
Detailed Explanation: 𝒘𝟐 =: [ 2 2 ] = [0.7 0.8], p=0.5, q=0.8
𝑤2,1 𝑤2,2
NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur
Question 2
Which of the following are the expression of the logistic sigmoid function given 𝑎𝑚−1 as the input to the
function?
a. 𝑚𝑎𝑥(0, 𝑎𝑚−1 )
𝑚−1
b. 1/(1 + 𝑒 −𝑎 )
2
c. 𝑚−1 −1
(1+𝑒 −2𝑎 )
d. 𝑡𝑎𝑛ℎ(𝑎𝑚−1 )
Correct Answer: b
Detailed Explanation: See the definition in week 10 slide 10
Question 3
Considering logistic sigmoid function as the activation function for a controller designed using the NN in
question 1, find out the values passed to the output layer (𝑎^1 following the terminology used in week
0.1
10 slide) when the input state vector is [ ].
0.5
0.601
Correct Answer: [ ].
0.6154
1 1 1
𝑤1,1 𝑤2,1 𝑝1 𝑏1 1/(1 + 𝑒 −𝑎1 )
Detailed Explanation: 𝒏𝟏 = [ 1 ]
1 [𝑝2 ] + [ ] , 𝒂𝟏
= [ 1 ]
𝑤1,2 𝑤2,2 𝑏2 1/(1 + 𝑒 −𝑎2 )
0.601
Putting the given values, we get [ ]
0.6154
Question 4
Sensitivity of the squared error/loss function 𝐹 of an NN w.r.t. its net input of m-th layer 𝑛𝑚 can be
∂F ∂F ∂am ∂F ∂nm+1 ∂am
expressed by 𝑠 𝑚 = m = m × m = m+1 × × = 𝑠 𝑚+1 ×___× ____ . Fill the blanks
∂n ∂a ∂n ∂n ∂am ∂nm
appropriately.
𝑑𝑓𝑚+1
a. 𝑤 𝑚 ,
𝑑𝑛𝑚+1
𝑑𝑓𝑚+1
b. 𝑏 𝑚+1 , 𝑑𝑎𝑚+1
𝑑𝑓𝑚
c. 𝑤 𝑚+1 , 𝑑𝑛𝑚
𝑑𝑓𝑚+1
d. 𝑤 𝑚 , 𝑑𝑛𝑚+1
NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur
𝑑𝑓𝑚
Correct Answer: c. 𝑤 𝑚+1 , 𝑑𝑛𝑚
Detailed Explanation: Follow the derivation in week 10 slide 14
Question 5
2
For the NN given in Question 1 calculate the updated 𝑤1,1 in the first iteration for the learning rate α =
0.05 0.73492
0.3, target outputs [ ] and actual outputs [ ].
0.95 0.77955
2
2 (1) 2 𝜕𝐹 𝜕𝐹 𝜕𝐹 𝜕 𝑎1 𝜕 𝑛12 2 2
Hint: 𝑤1,1 = 𝑤1,1 (0) − 𝛼 × 2 , 2 = 2 × 2 × 2 , 𝐹 = (𝑞𝑡1 − 𝑞1 ) + (𝑞𝑡2 − 𝑞2 ) =
𝜕𝑤1,1 𝜕𝑤1,1 𝜕𝑎1 𝜕𝑛1 𝜕𝑤1,1
1
(𝑞𝑡1 − 𝑎13 )2 + (𝑞𝑡2 − 𝑎13 )2 , 𝑎13 = 3
3 , 𝑛1 =
2 2
𝑤1,1 𝑎1 2 2
+ 𝑤1,2 𝑎2 + 𝑏1 and use the value of 𝑎12 from
1+𝑒 −𝑛1
question 3.
A. 0.4
B. 0.3
C. 0.452
D. 0.1
Correct Answer: C. 0.452
Detailed Explanation:
𝛛𝑭 𝛛𝑭 𝛛 𝒂𝟑𝟏 𝛛 𝒏𝟑𝟏
= × ×
𝛛𝒘𝟐𝟏,𝟏 𝛛𝒂𝟑𝟏 𝛛𝒏𝟑𝟏 𝛛𝒘𝟐𝟏,𝟏
𝟐 𝟐 𝛛𝑭
𝑭 = (𝒒𝒕𝟏 − 𝒒𝟏 )𝟐 + (𝒒𝒕𝟐 − 𝒒𝟐 )𝟐 = (𝒒𝒕𝟏 − 𝒂𝟑𝟏 ) + (𝒒𝒕𝟐 − 𝒂𝟑𝟏 ) ⇒ = −𝟐(𝒒𝒕𝟏 − 𝒂𝟑𝟏 )
𝛛𝒂𝟑𝟏
𝟏 𝛛 𝒂𝟑𝟏
𝒂𝟑𝟏 = ⇒ = 𝒂𝟑𝟏 (𝟏 − 𝒂𝟑𝟏 )
𝟏 + 𝐞−𝒏𝟏
𝟑
𝛛𝒏𝟑𝟏
𝛛 𝒏𝟑
𝒏𝟑𝟏 = 𝒘𝟐𝟏,𝟏 𝒂𝟐𝟏 + 𝒘𝟐𝟏,𝟐 𝒂𝟐𝟐 + 𝒃𝟏 ⇒ 𝛛𝒘𝟐𝟏 = 𝒂𝟐𝟏 = 0.601 (from q 3)
𝟏,𝟏
𝛛𝑭 𝛛𝑭 𝛛 𝒂𝟑𝟏 𝛛 𝒏𝟑𝟏
= × × = −𝟐(𝒒𝒕𝟏 − 𝒂𝟑𝟏 ). 𝒂𝟑𝟏 (𝟏 − 𝒂𝟑𝟏 ). 𝒂𝟐𝟏 = 𝟐(𝟎. 𝟕𝟑𝟒𝟗𝟐 − 𝟎. 𝟎𝟓). 𝟎. 𝟕𝟑𝟒𝟗𝟐. (𝟏 −
𝛛𝒘𝟐𝟏,𝟏 𝛛𝒂𝟑𝟏 𝛛𝒏𝟑𝟏 𝛛𝒘𝟐𝟏,𝟏
𝟎. 𝟕𝟑𝟒𝟗𝟐).0.601 = 0.16
2 (1) 2
𝑤1,1 = 𝑤1,1 (0) − α × 0.16 = 0.452
NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur
Question 6
Match the followings:
A. Imitation Learning 1. requires desired sample input output pairs for training
B. Reinforcement learning 2. penalizes based on a reward function that expresses desired functionality
C. Supervised Learning 3. requires a trained teacher NN to train an apprentice NN
(i) A matches with __ (ii) B matches with ___ (iii) C matches with ____
Correct Answer: (i) 3, (ii) 2, (iii) 1
Detailed Explanation: Read slides 18-19 in week 10 materials
Question 7
Compensation
for noise
Tracking error
Which one is true for the above adaptive controller?
a. As tracking error increases the controller becomes accurate
b. As tracking error decreases the controller becomes accurate
c. Compensation for noise characterizes the error in the output caused by only the process noise
d. Compensation for noise characterizes the input required to reduce the error in the desired output
caused by the sensor and process noise
Correct Answer:
b. As tracking error decreases compensation for noise decreases
d. Compensation for noise characterizes the input required to reduce the error in the output caused by
the noise
Detailed Explanation: As the NN controller gets trained to track the noisy output of the closed loop
plant, it learns to inject a suitable control input to the plant to achieve similar/desired outputs under
noise. Evidently the tracking error decrease. (b)
The noise compensation is the output of the inverse plant model that takes the difference between the
desired and actual output as input and produce the excess input required to minimize the difference. (d)
NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur
Question 8
Which of the following should be true in order to design a safe controller using NN?
a. should penalize the agent for generating such control inputs that change the future state such that
control barrier function becomes negative inside the safe region
b. should reward the agent for generating such control inputs that change the future state such that
control barrier function becomes negative inside the safe region
c. should reward the agent for generating such control inputs that change the future state such that the
rate of change of control barrier function becomes positive inside the safe region
d. should penalize the agent for generating such control inputs that change the future state such that
the rate of change of control barrier function becomes non-negative inside the safe region
Correct Answer: b, d
Detailed Explanation: For a safe controller CBF must remain negative inside the initial state space
and the rate of change of CBF must remain negative inside the safety region. The NN control agent
must be penalized if these criteria are not fulfilled and rewarded if they are.
Question 9
For the following reward/penalty schemes which of the cases will help in designing a stable NN-based
controller?
a. should penalize the agent for generating such control inputs that change the future state such that
the eigenvalues of discrete-time closed loop state-transition matrix approach zero inside the state-
space
b. should reward the agent for generating such control inputs that change the future state such that the
eigenvalues of discrete-time closed loop state-transition matrix approach zero inside the state-space
c. should reward the agent for generating such control inputs that change the future state such that the
rate of change of CLF becomes negative around the equilibrium point
d. should penalize the agent for generating such control inputs that change the future state such that
the rate of change of CLF becomes negative around the equilibrium point
Correct Answer: b, c
Detailed Explanation: For stability the controller gain must be designed in a way such that the eigen
values of the state-transition matrix of a discrete-time closed loop system or the norm of the discrete-
time closed-loop transition function must be minimized/approach zero/within (-1,1). Moreover, the
negative slope of CLF w.r.t. time around an equilibrium point ensures that the state trajectory of the
NPTEL Online Certification Courses Indian Institute
of Technology Kharagpur
closed-loop system converges the equilibrium point. Hence an NN controller agent must be rewarded if
it eventually learns to satisfy these criteria.
Question 10
State whether the following statements are true or false:
(i) A trained NN-based safe controller can guarantee safety
a. T
b. F
(ii) A trained NN-based safe controller must go through safety verification in order to ensure safety
a. T
b. F
Correct Answer: (i) b (ii) a
Detailed Explanation: NN-based controllers take statistically safe actions most of the time as it learns.
But the safety is not guaranteed as there exists a very small probability for it to take unsafe action.
Hence it must be verified to ensure safety.
