1.

Chapter I: Matrices and Determinants

This chapter is an introduction to matrices and matrix operations. Determinants, Cramer’s
rule, and Gauss’s elimination method are introduced. Some definitions and examples are not
applicable to subsequent material presented in this text, but are included for subject continuity,
and reference to more advanced topics in matrix theory.

1.1. Matrix Definition

A matrix is a rectangular array of numbers such as those shown below.
1 3 1
2 3 7
[ ] or [−2 1 −5]
1 −1 5
4 −7 6
In general form, a matrix A is denoted as
𝑎11 ⋯ 𝑎1𝑛 (3.1)
𝐴= [ ⋮ ⋱ ⋮ ]
𝑎𝑚1 ⋯ 𝑎𝑚𝑛

The numbers 𝑎𝑖𝑗 are the elements of the matrix where the index 𝑖 indicates the row and 𝑗
indicates the column in which each element is positioned. Thus, 𝑎43 indicates the element
positioned in the fourth row and third column.
A matrix of m rows and n columns is said to be of 𝑚 × 𝑛 order matrix.
If 𝑚 = 𝑛, the matrix is said to be a square matrix of order 𝑚 (or 𝑛). Thus, if a matrix has five
rows and five columns, it is said to be a square matrix of order 5.
In a square matrix, the elements 𝑎11 , 𝑎22 , 𝑎33 , … , 𝑎𝑛𝑛 are called the main diagonal elements.
Alternately, we say that the matrix elements 𝑎11 , 𝑎22 , 𝑎33 , … , 𝑎𝑛𝑛 , are located on the main
diagonal

Note:
• The sum of the diagonal elements of a square matrix A is called the trace of A.
• A matrix in which every element is zero, is called a zero matrix.

1.2. Matrix Operations

Two matrices 𝐴 = [𝑎𝑖𝑗 ] and B= [𝑏𝑖𝑗 ] are equal, that is, 𝐴 = 𝐵 , if and only if

𝑎𝑖𝑗 = 𝑏𝑖𝑗 for 𝑖 = 1,2, … , 𝑚 and j= 1,2, … , 𝑛 (3.2)

Two matrices are said to be conformable for addition /subtraction, if they are of the same order
𝑚 × 𝑛. Hence, their sum /difference will be another matrix C with the same order as A and B,
where each element of C is the sum /difference of the corresponding elements of and, that is,

𝐶 = 𝐴 ∓ 𝐵 = [𝑎𝑖𝑗 ∓ 𝑏𝑖𝑗 ] (3.3)

Example 3.2.1
Compute 𝐴 + 𝐵 and 𝐴 – 𝐵 given that,
1 2 3 2 3 0
𝐴= [ ] and 𝐵 = [ ]
0 1 4 −1 2 5
Solution
1+2 2+3 3+0 3 5 3
𝐴+𝐵 = [ ]=[ ],
0 + (−1) 1+2 4+5 −1 3 9
and
1−2 2−3 3−0 −1 −1 3
𝐴−𝐵 = [ ]=[ ]
0 − (−1) 1−2 4−5 1 −1 −1
Check with MATLAB
Try it! In the command window
>> A= [1 2 3;0 1 4]; B= [2 3 0;-1 2 5]; % Define matrices A and B
>> A+B % Add A and B
ans =
3 5 3
-1 3 9
>> A-B
ans =
-1 -1 3
-1 -1 -1

Example 3.2.2
1 −2
Multiply the matrix 𝐴 = [ ] by
2 3
(a) 𝑘1 = 5 and (b) 𝑘2 = −3 + 2𝑖
Solution
1 −2 5 × 1 5 × (−2) 5 −10
(a) 𝑘1 . 𝐴 = 5 × [ ]=[ ]=[ ]
2 3 5×2 5×3 10 15

1 −2 (−3 + 2𝑖) × 1 (−3 + 2𝑖) × (−2)

(b) 𝑘2 . 𝐴 = (−3 + 2𝑖) × [ ]=[ ]=
2 3 (−3 + 2𝑖) × 2 (−3 + 2𝑖) × 3

−3 + 2𝑖 6 − 4𝑖
[ ]
−6 + 4𝑖 −9 + 6𝑖

1
 Check with MATLAB
Try it! In the command window
>> k1=5; k2= (-3+2*i); % Define the scalers k1 and k2
>> A=[1 -2; 2 3]; % Define matrix A
>> k1* A % Multiply matrix A by constant k1
ans =
5 10
10 15
>> k2* A % Multiply matrix A by constant k2
ans =
-3.0000 + 2.0000 i 6.0000 - 4.0000 i
-6.0000+ 4.0000 i 9.0000+ 6.0000 i

Two matrices 𝐴 and 𝐵 are said to be conformable for multiplication 𝐴. 𝐵 in that order, only when
the number of columns of matrix 𝐴 is equal to the number of rows of matrix 𝐵. That is, the
product 𝐴. 𝐵 (but not 𝐵. 𝐴) is conformable for multiplication only if 𝐴 is an 𝑚 × 𝑝 and matrix 𝐵
is an 𝑝 × 𝑛 matrix. The product will then be 𝐴. 𝐵 an 𝑚 × 𝑛 matrix. 𝐴 convenient way to
determine if two matrices are conformable for multiplication is to write the dimensions of the
two matrices side−by−side as shown below.

For matrix multiplication, the operation is row by column. Thus, to obtain the product 𝐴. 𝐵, we
multiply each element of a row of 𝐴 by the corresponding element of a column of 𝐵; then, we
add these products.

2
Example 3.2.3
1
Given that 𝐶 = [2 3 4] and 𝐷 = [−1], Compute the product 𝐶. 𝐷 and 𝐷. 𝐶 .
2
Solution
The dimensions of matrices 𝐶 and 𝐷 are respectively 1 × 3, 3 × 1; therefore, the product 𝐶. 𝐷 is
feasible, and will result in a 1 × 1 , that is,
1
𝐶. 𝐷 = [2 3 4] [−1] = [ (2 × 1) + (3 × (−1)) + (4 × 2)] = [7]
2
The dimensions for 𝐷 and 𝐶 are respectively 3 × 1, 1 × 3 and therefore, the product 𝐷. 𝐶 is also
feasible. Multiplication of these will produce a 3 × 3 matrix as follows.

1 (1 × 2) (1 × 3) (1 × 4) 2 3 4
𝐷. 𝐶 = [−1] [2 3 4] = [(−1 × 2) (−1 × 3) (−1 × 4)] = [−2 −3 −4]
2 (2 × 2) (2 × 3) (2 × 4) 4 6 8

Check with MATLAB

Try it! In the command window
>> C= [2 3 4]; D= [1; -1; 2]; % Define matrices C and D
>> C* D % Multiply C by D
ans =
7
>> D* C % Multiply D by C
ans =
2 3 4
-2 -3 -4
4 6 8

3
1.3. Special forms of Matrices
A square matrix is said to be upper triangular when all the elements below the diagonal are zero.
The matrix A below is an upper triangular matrix.

(3.3)

• A square matrix is said to be lower triangular, when all the elements above the diagonal
are zero. The matrix 𝐵 below is a lower triangular matrix.

(3.4)

• A square matrix is said to be diagonal, if all elements are zero, except those in the
diagonal. The matrix 𝐵 below is a diagonal matrix.

(3.5)

• A diagonal matrix is called a scalar matrix, if 𝑎11 = 𝑎22 = 𝑎33 = ⋯ = 𝑎𝑛𝑛 = 𝑘 where 𝑘
is a scalar. The matrix 𝐷 below is a scalar matrix with 𝑘 = 4.

(3.6)

• A scalar matrix with 𝑘 = 1, is called an identity matrix I. Shown below are 2 × 2, 3 × 3,

and 4 × 4 identity matrices.

4
 (3.7)

The MATLAB eye(n) function displays an 𝑛 × 𝑛 identity matrix. For example,

Try it! In the command window

>> eye (4) % displays 4 by 4 identity matrix
ans =
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1

The function eye(size(A)), produces an identity matrix whose size is the same as matrix A. For
example, let A be defined as

Try it! In the command window

>> A= [1 3 1; -2 1 -5; 4 -7 6] % Define matrix A
A=
1 3 1
-2 1 -5
4 -7 6
>> eye (size (A))
ans =
1 0 0
0 1 0
0 0 1

• The transpose of a matrix 𝐴 denoted as 𝐴𝑇 , is the matrix that is obtained when the rows
and columns of matrix 𝐴 are interchanged. For example, if

(3.8)

5
In MATLAB we use the apostrophe (′) symbol to denote and obtain the transpose of a matrix.
Thus, for the above example,

Try it! In command window

>> A= [1 2 3; 4 5 6] % Define Matrix A
ans =
1 2 3
4 5 6
>> A’ % Display the transpose of A
ans =
1 4
2 5
3 6

• A symmetric matrix 𝐴 is one such that 𝐴 = 𝐴𝑇 , that is, the transpose of a matrix 𝐴 is the
same as 𝐴. An example of a symmetric matrix is shown below.

(3.9)

• If a matrix 𝐴 has complex numbers as elements, the matrix obtained from 𝐴 by replacing
each element with its conjugate is called the conjugate of 𝐴, and it is denoted as 𝐴∗ .

(3.10)

• MATLAB has built−in functions that compute the complex conjugate of a number. The
conj(A) computes the conjugate of a matrix. Using MATLAB with matrix A defined as
above, we obtain

6
Try it! In the command window
>> A= [1+2i i; 3 2-3i] % Define and display matrix A
A=
1.0000 + 2.0000i 0 + 1.0000i
3.0000 2.0000 - 3.0000i
>> conj_A= conj (A) % Compute and display the conjugate of matrix A
conj_A =
1.0000 - 2.0000i 0 - 1.0000i
3.0000 2.0000 + 3.0000i

• A square matrix 𝐴 such that 𝐴𝑇 = −𝐴 is called skew−symmetric. For example,

(3.11)

• A square matrix 𝐴 such that 𝐴𝑇∗ = 𝐴, is called Hermitian. For example,

(3.12)

• A square matrix 𝐴 such that 𝐴𝑇∗ = −𝐴, is called skew−Hermitian. For example

(3.13)

1.4. Determinants
Let matrix 𝐴 be defined as the square matrix

(3.14)

then, the determinant of 𝐴, denoted as 𝑑𝑒𝑡𝐴, is defined as

(3.15)

7
The determinant of a square matrix of order n is referred to as determinant of order n.
Let 𝑑𝑒𝑡𝐴 be a determinant of order 2 for matrix 𝐴, that is,

(3.16)

Then

(3.17)

Example 3.4.1
1 2 2 −1
Given that 𝐴 = [ ] and 𝐵 = [ ], then compute the 𝑑𝑒𝑡𝐴 and 𝑑𝑒𝑡𝐵.
3 4 2 0
Solution
𝑑𝑒𝑡𝐴 = 1 × 4 − 3 × 2 = 4 − 6 = −2
𝑑𝑒𝑡𝐵 = 2 × 0 − 2 × (−1) = 0 − (−2) = 2
Check with MATLAB
Try it! In command window
>> A= [1 2; 3 4]; B= [2 -1; 2 0]; % Define matrices A and B
>> det (A) % compute determinant of A
ans =
-2
>> det (B) % compute determinant of B
ans =
2

Now let 𝐴 be a matrix of order 3, that is,

(3.18)

Then 𝑑𝑒𝑡𝐴, is found from

8
 (3.19)

A convenient method to evaluate the determinant of order 3, is to write the first two columns to
the right of the matrix of order 3 × 3, and add the products formed by the diagonals from upper
left to lower right; then subtract the products formed by the diagonals from lower left to the
upper right as shown on the diagram below. When this is done properly, we obtain (3.19) above.

This method works only with second and third order determinants. To evaluate higher order
determinants, we must first compute the cofactors; these will be defined shortly.

Example 3.4.2
Compute the 𝑑𝑒𝑡𝐴 and 𝑑𝑒𝑡𝐵 where 𝐴 and 𝐵 are given by
2 3 5 2 −3 −4
𝐴 = [1 0 1 ] and 𝐵 = [ 1 0 −2]
2 1 0 0 −5 −6
Solution

Or

Likewise,

Or

9
Check with MATLAB
Try it! In command window
>> A= [2 3 5; 1 0 -1; 2 1 0]; det (A) % Define matrix A and compute detA
ans =
9
>> B= [2 -3 -4; 1 0 -2; 0 -5 -6]; det (B) % Define matrix B and compute detB
ans =
-18

1.5. Minors and cofactors

Recall the matrix 𝐴 defined in equation (3.14).

If we remove the elements of its 𝑖 − 𝑡ℎ row, and 𝑗 − 𝑡ℎ column, the determinant of the
remaining 𝑛 − 1 square matrix is called the minor of determinant A, and it is denoted as [𝑀𝑖𝑗 ].
The signed minor (−1)𝑖+𝑗 . [𝑀𝑖𝑗 ] is called the cofactor of 𝑎𝑖𝑗 and it is denoted as 𝛼𝑖𝑗 .

Example 3.5.1
Given that

Compute minors [𝑀11 ], [𝑀12 ], [𝑀13 ] and cofactors 𝛼11 , 𝛼12 , 𝛼13 .
Solution

10
 The remaining minors [𝑀21 ], [𝑀22 ], [𝑀23 ], [𝑀31 ], [𝑀32 ], [𝑀33 ]
And cofactors 𝛼21 , 𝛼22 , 𝛼23 , 𝛼31 , 𝛼32 , 𝛼33 are defined similarly. (check that!)

Example 3.5.2
1 2 −3
Compute the cofactors of the following matrix, 𝐴 = [ 2 −4 2 ]
−1 2 −6
Solution

It is useful to remember that the signs of the cofactors follow the pattern

11
that is, the cofactors on the diagonals have the same sign as their minors.
Let 𝐴 be a square matrix of any size; the value of the determinant of 𝐴 is the sum of the products
obtained by multiplying each element of any row or any column by its cofactor.

Example 3.5.3
1 2 −3
Compute the determinant of A using the elements of the first row. 𝐴 = [ 2 −4 2 ]
−1 2 −6
Solution

Check with MATLAB

Try it! In command window
>> A= [1 2 -3; 2 -4 2; -1 2 -6]; det (A) % Define matrix A and compute detA
ans =
40

To determine the determinant of a 4 × 4 matrix, see the following:

The fourth-order determinant can first be expressed as the sum of the products of the elements of
its first row by its cofactor as shown below.
𝑎11 𝑎12 𝑎13 𝑎14
𝑎22 𝑎23 𝑎24 𝑎12 𝑎13 𝑎14
𝑎21 𝑎22 𝑎23 𝑎24
𝐴 = |𝑎 𝑎
| = 𝑎11 | 32 𝑎 33 𝑎 34 | − 𝑎21 | 32 𝑎33 𝑎34 |
𝑎
31 𝑎32 𝑎33 𝑎34
𝑎42 𝑎43 𝑎44 𝑎42 𝑎43 𝑎44
𝑎41 𝑎42 𝑎43 𝑎44
𝑎12 𝑎13 𝑎14 𝑎12 𝑎13 𝑎14
+ 𝑎31 |𝑎22 𝑎23 𝑎24 | − 𝑎41 |𝑎22 𝑎23 𝑎24 |
𝑎42 𝑎43 𝑎44 𝑎32 𝑎33 𝑎34

12
Determinants of order five or higher can be evaluated similarly.
Example 3.5.4
2 −1 0 −3
−1 1 0 −1
Compute the determinant of the following matrix, 𝐴 = [ ]
4 0 3 −2
−3 0 0 1
Solution

Next, using the procedure of Example 3.5.2 or example 3.5.3, we find

[𝑎] = 6, [𝑏] = −3, [𝑐] = 0, [𝑑] = −36
Thus, 𝑑𝑒𝑡𝐴 = [𝑎] + [𝑏] + [𝑐] + [𝑑] = 6 + (−3) + 0 + (−36) = −33
Check with MATLAB
Try it! In command window
>> A= [2 -1 0 -3; -1 1 0 -1; 4 0 3 -2; -3 0 0 1]; delta = det (A)
delta =
-33

13
To compute the determinant of a 𝑛 × 𝑛 matrix MATLAB users may use the following code.
Try it! In Editor window
1. % This file computes the determinant of a nxn matrix
2. % It must be saved as function (user defined) file
3. % detnxn.m in the current Work Directory. Make sure
4. % that his directory is added to MATLAB's search
5. % path accessed from the Editor Window as File>Set Path>
6. % Add Folder. It is highly recommended that this
7. % function file is created in MATLAB's Editor Window.
8. function y=detnxn(A);
9. % The following statement initializes y
10. y = 0;
11. % The following statement defines the size of the matrix A
12. [n, n] =size(A);
13. % MATLAB allows us to use the user-defined functions to be recursively
14. % called on themselves so we can call det2x2(A) for a 2x2 matrix,
15. % and det3x3(A) for a 3x3 matrix.
16. if n==2
17. y=det2x2(A);
18. return
19. end
20. %
21. if n==3
22. y=det3x3(A);
23. return
24. end
25. % For 4x4 or higher order matrices we use the following:
26. % (We can define n and matrix A in Command Window
27. for i=1: n
28. y=y+(-1) ^(i+1) *A (1, i) *detnxn (A (2: n, [1:(i-1) (i+1): n]));
29. end
30. % To run this program, define the nxn matrix in
31. % MATLAB's Command Window as A= [....] and then
32. % type detnxn(A) at the command prompt.
After all the aforementioned information about the determinants, the following properties need to
be considered:
Property 1:
• If all elements of one row or one column are zero, the determinant is zero. An
example of this is the determinant of the cofactor [c] above.

Property 2:
• If all the elements of one row or column are m times the corresponding elements of
another row or column, the determinant is zero. For example, if

14
 2 4 1 2 4 1 2 4
𝐴 = [3 6 1], then 𝐴 = |3 6 1| 3 6 = 12 + 4 + 6 − 6 − 4 + 12 = 0
1 2 1 1 2 1 1 2
Here, 𝑑𝑒𝑡𝐴 is zero because the second column in 𝐴 is 2 times the first column. Check with
MATLAB.
Try it! In command window
>> A = [2 4 1; 3 6 1; 1 2 1]; det(A)
Ans =
0

Property 3:
• If two rows or two columns of a matrix are identical, the determinant is zero. This
follows from Property 2 with 𝑚 = 1.

After absorbing the idea of using MATLAB to solve math problems, we proceed to deal with
some important concept that is, solving a system of linear equations. Regarding this subject, we
will discuss some basic numerical methods, which are Cramer's rule, the Gaussian elimination
method and simultaneous equations.

1.6. Cramer’s Rule

Consider the system of the three equations below
𝑎11 𝑥 + 𝑎12 𝑦 + 𝑎13 𝑧 = 𝐴
𝑎21 𝑥 + 𝑎22 𝑦 + 𝑎23 𝑧 = 𝐵 (3.20)
𝑎31 𝑥 + 𝑎32 𝑦 + 𝑎33 𝑧 = 𝐶

And let
𝑎11 𝑎12 𝑎13 𝐴 𝑎12 𝑎13 𝑎11 𝐴 𝑎13 𝑎11 𝑎12 𝐴
∆= |𝑎21 𝑎22 𝑎23 |, 𝐷1 = |𝐵 𝑎22 𝑎23 |, 𝐷2 = |𝑎21 𝐵 𝑎23 |, 𝐷3 = |𝑎21 𝑎22 𝐵|
𝑎31 𝑎32 𝑎33 𝐶 𝑎32 𝑎33 𝑎31 𝐶 𝑎33 𝑎31 𝑎32 𝐶
Cramer’s rule states that the unknowns 𝑥, 𝑦, and 𝑧 can be found from the relations
𝑫𝟏 𝑫𝟐 𝑫𝟑 (3.21)
𝒙= , 𝒚= , 𝒛=
∆ ∆ ∆

• provided that the determinant ∆ (delta) is not zero.

15
 • We observe that the numerators of equation (3.21) are formed from 𝛥 by substituting the
known values 𝐴, 𝐵, and 𝐶, for the coefficients of the desired unknown.
• Cramer’s rule applies to systems of two or more equations.
• If equation (3.20) is a homogeneous set of equations, if 𝐴 = 𝐵 + 𝐶, then 𝐷1, 𝐷2, 𝐷3 are
all zero as we found in Property 1 above. Then, 𝑥 = 𝑦 = 𝑧 = 0 also.
Example 3.6.1
Use Cramer’s rule to find 𝑣1 , 𝑣2 and 𝑣3 if
2𝑣1 − 5 −𝑣2 + 3𝑣3 = 0
−2𝑣3 −3𝑣2 −4𝑣1 = 8
𝑣2 + 3𝑣1 − 4 −𝑣3 = 0
Verify your answers with MATLAB.
Solution
Rearranging the unknowns 𝑣, and transferring known values to the right side, we obtain
2𝑣1 −𝑣2 + 3𝑣3 = 5
−4𝑣1 −3𝑣2 −2𝑣3 = 8
3𝑣1 + 𝑣2 −𝑣3 = 4
Now by Cramer’s rule, we have
2 −1 3 2 −1
∆= |−4 −3 −2| −4 −3 = 6 + 6 − 12 + 27 + 4 + 4 = 35
3 1 −1 3 1
5 −1 3 5 −1
𝐷1 = |8 −3 −2| 8 −3 = 15 + 8 + 24 + 36 + 10 − 8 = 85
4 1 −1 4 1
2 5 3 2 5
𝐷2 = |−4 8 −2| −4 8 = −16 − 30 − 48 − 72 + 16 − 20 = 170
3 4 −1 3 4
2 −1 5 2 −1
𝐷3 = |−4 −3 8| −4 −3 = −24 − 24 − 20 + 45 − 16 − 16 = −55
3 1 4 3 1
Using equation (3.21) we obtain
𝐷1 85 17 𝐷2 170 34 𝐷3 −55 −11
𝑣1 = = = , 𝑣2 = = = , 𝑣3 = = =
∆ 35 7 ∆ 35 7 ∆ 35 7

16
Now, verify with MATLAB as follows.
Try it! In Editor window
1. % The following script will compute and display the values of v1, v2 and v3.
2. format rat % Express answers in ratio form
3. B=[2 -1 3; -4 -3 -2; 3 1 -1]; % The elements of the determinant D
4. delta=det(B); % Compute the determinant D of B
5. d1=[5 -1 3; 8 -3 -2; 4 1 -1]; % The elements of D1
6. detd1=det(d1); % Compute the determinant of D1
7. d2=[2 5 3; -4 8 -2; 3 4 -1]; % The elements of D2
8. detd2=det(d2); % Compute the determinant of D2
9. d3=[2 -1 5; -4 -3 8; 3 1 4]; % The elements of D3
10. detd3=det(d3); % Compute he determinant of D3
11. v1=detd1/delta; % Compute the value of v1
12. v2=detd2/delta; % Compute the value of v2
13. v3=detd3/delta; % Compute the value of v3
14. disp('v1='); disp (v1); % Display the value of v1
15. disp('v2='); disp (v2); % Display the value of v2
16. disp('v3='); disp (v3); % Display the value of v3
In Command window you will see,
v1=
17/7
v2=
-34/17
v3=
-11/7

1.7. Gaussian Elimination Method

We can find the unknowns in a system of two or more equations also by the Gaussian
elimination method. With this method, the objective is to eliminate one unknown at a time. This
can be done by multiplying the terms of any of the equations of the system by a number such that
we can add (or subtract) this equation to another equation in the system so that one of the unknowns
will be eliminated. Then, by substitution to another equation with two unknowns, we can find the
second unknown. Subsequently, the substitution of the two values found can be made into an
equation with three unknowns from which we can find the value of the third unknown. This
procedure is repeated until all unknowns are found. This method is best illustrated with the
following example which consists of the same equations as the previous example.

17
Example 3.7.1
Use Gaussian elimination method to find 𝑣1 , 𝑣2 and 𝑣3 of
2𝑣1 −𝑣2 + 3𝑣3 = 5 (3.22)
−4𝑣1 −3𝑣2 −2𝑣3 = 8
3𝑣1 + 𝑣2 −𝑣3 = 4

Solution
As a first step, we add the first equation of (3.22) with the third to eliminate the unknown 𝑣2 ,
and we obtain the following equation.
5𝑣1 + 2𝑣3 = 9 (3.23)

Next, we multiply the third equation of (3.22) by 3, and we add it with the second to eliminate
𝑣2 . Then, we obtain the following equation

5𝑣1 − 5𝑣3 = 20 (3.24)

Subtraction of (3.24) from (3.23) yields

−11 (3.25)
7𝑣3 = −11 or 𝑣3 =
7

Now, we can find the unknown 𝑣1 from either (3.23) or (3.24). By substitution of (3.25) into
(3.23), we obtain
−11 17 (3.26)
5𝑣1 + 2 . ( ) = 9 or 𝑣1 =
7 7
Finally, we can find the last unknown 𝑣2 from any of the three equations of (3.22). By
substitution into the first equation we obtain
34 33 35 −34 (3.27)
𝑣2 = 2𝑣1 + 3𝑣3 − 5 = − − =
7 7 7 7

• These are the same values as those we found in Example 3.6.1

Before we discuss the simultaneous equations, we should know the following concepts.

18
1.8. The Adjoint of a Matrix
Let us assume that 𝐴 is an n square matrix and 𝛼𝑖𝑗 is the cofactor of 𝑎𝑖𝑗 . Then the adjoint of 𝐴,
denoted as 𝑎𝑑𝑗𝐴, is defined as the square matrix shown below,

(3.28)

We observe that the cofactors of the elements of the 𝑖 − 𝑡ℎ row (column) of 𝐴, are the elements
of the 𝑖 − 𝑡ℎ column (row) of 𝑎𝑑𝑗𝐴.

Example 3.8.1
Compute the 𝑎𝑑𝑗𝐴 given that,
1 2 3
𝐴 = [ 1 3 4]
1 4 3
Solution
3 4 2 3 2 3
| | −| | | |
4 3 4 3 3 4 −7 6 −1
1 4 1 3 2 3
𝑎𝑑𝑗𝐴 = − | | | | −| | =[ 1 0 −1]
1 3 1 3 3 4
1 3 1 2 1 2 1 −2 1
[ |1 4
| −|
1 4
| |
1 3
| ]

1.9. Singular and Non-Singular Matrices

An 𝑛 square matrix 𝐴 is called singular if 𝑑𝑒𝑡𝐴 = 0 ; if 𝑑𝑒𝑡𝐴 ≠ 0, A is called non−singular. If
an 𝑛 square matrix 𝐴 is nearly singular, that is, if the determinant of that matrix is very small, the
matrix is said to be ill−conditioned see [1].
Example 3.9.1
1 2 3
Given that 𝐴 = [2 3 4], determine whether this matrix is singular or non-singular.
3 5 7
Solution

19
 1 2 3 1 2
𝑑𝑒𝑡𝐴 = |2 3 4| 2 3 = 21 + 24 + 30 − 27 − 20 − 28 = 0
3 5 7 3 5
Therefore, matrix A is singular.

1.10. The Inverse of a Matrix

If 𝐴 and 𝐵 are square matrices such that 𝐴𝐵 = 𝐵𝐴 = 𝐼, where I is the identity matrix, 𝐵 is called
the inverse of 𝐴, denoted as 𝐵 = 𝐴−1 , and likewise, A is called the inverse of B, that is, 𝐴 = 𝐵 −1
If a matrix 𝐴 is non−singular, we can compute its inverse from the following relation

1 (3.29)
𝐴−1 = 𝑎𝑑𝑗𝐴
𝑑𝑒𝑡𝐴

Example 3.10.1
1 2 3
Given that, 𝐴 = [1 3 4], Compute the inverse of A.
1 4 3
Solution
Here, 𝑑𝑒𝑡𝐴 = 9 + 8 + 12 − 9 − 16 − 6 = −2, and since this is a non-zero value, it is possible
to compute the inverse of A using (3.29). From Example 3.8.1, we already now that,
−7 6 −1
𝑎𝑑𝑗𝐴 = [ 1 0 −1],
1 −2 1
Hence,

1 1 −7 6 −1 3.5 −3 0.5
𝐴−1 = 𝑎𝑑𝑗𝐴 = [1 0 −1] = [−0.5 0 0.5 ]
𝑑𝑒𝑡𝐴 −2
1 −2 1 −0.5 0 −0.5

20
Check with MATLAB
Try it! In command window
>> A= [1 2 3; 1 3 4; 1 4 3], invA=inv(A) % Define matrix A and compute its inverse
A=
1 2 3
1 3 4
1 4 3
inv A=
3. 5000 -3. 0000 0. 0000
-0. 5000 0 0. 5000
-0. 5000 1. 0000 -0. 5000
Multiplying a matrix 𝐴 by its inverse 𝐴−1 produces the identity matrix 𝐼, that is,

𝐴𝐴−1 = 𝐼 or 𝐴−1 𝐴 = 𝐼 (3.30)

Example 3.10.2
4 3
Prove the validity of equation (3.30) for following matrix 𝐴 = [ ]
2 2
Proof
2 −3
𝑑𝑒𝑡𝐴 = 8 − 6 = 2 and 𝑎𝑑𝑗𝐴 = [ ]
−2 4
1 1 2 −3 1 −3/2
𝐴−1 = 𝑎𝑑𝑗𝐴 = [ ]=[ ]
𝑑𝑒𝑡𝐴 2 −2 4 −1 2
And
4 3 1 −3/2 4 − 3 −6 + 6 1 0
𝐴𝐴−1 = [ ][ ]=[ ]=[ ]
2 2 −1 2 2 − 2 −3 + 4 0 1

21
1.11. Solution of Simultaneous equations with Matrices
Consider the relation

𝐴𝑋 = 𝐵 (3.31)

where 𝐴 and 𝐵 are matrices whose elements are known, and 𝑋 is a matrix (a column vector)
whose elements are the unknowns. We assume that 𝐴 and 𝑋 are conformable for multiplication.
Multiplication of both sides of (3.31) by yields:

𝐴−1 𝐴𝑋 = 𝐴−1 𝐵 = 𝐼𝑋 = 𝐴−1 𝐵 (3.31)

or

𝑋 = 𝐴−1 𝐵 (3.32)

Therefore, we can use (3.32) to solve any set of simultaneous equations that have solutions. We
will refer to this method as the inverse matrix method of solution of simultaneous equations.

Example 3.11.1
2𝑥1 +3𝑥2 +𝑥3 =9
Given the system of equations { 𝑥1 +2𝑥2 +3𝑥3 = 6}, compute the unknowns 𝑥1 , 𝑥2 , and
3𝑥1 +𝑥2 +2𝑥2 =8
𝑥3 using the inverse matrix method
Solution
In matrix form, the given set of equations is 𝐴𝑋 = 𝐵 where
2 3 1 𝑥1 9
𝐴 = [1 2 3], 𝑋 = [𝑥2 ], 𝐵 = [6 ]
3 1 2 𝑥3 8
Then
𝑋 = 𝐴−1 𝐵
Or
𝑥1 2 3 1 −1 9
[𝑥2 ] = [1 2 3] [6]
𝑥3 3 1 2 8
Next, we find the determinant 𝑑𝑒𝑡𝐴, and the 𝑎𝑑𝑗𝐴.
1 −5 7
𝑑𝑒𝑡𝐴 = 18 and 𝑎𝑑𝑗𝐴 = [ 7 1 −5]
−5 7 1
Thus,
22
 1 1 1 −5 7
𝐴−1 = 𝑎𝑑𝑗𝐴 = [7 1 −5]
𝑑𝑒𝑡𝐴 18
−5 7 1
and by equation (4.53) we obtain the solution as follows.

𝑥1 35/18
1 1 −5 7 9 1 35 1.94
𝑋 = [𝑥2 ] = [7 1 −5] [6] = [29] = [29/18] = [1.61]
𝑥3 18 18
−5 7 1 8 5 5/18 0.28
To verify our results, we could use the MATLAB inv(A) function, and multiply 𝐴−1 by 𝐵.
However, it is easier to use the matrix left division operation 𝑋 = 𝐴 \ 𝐵; this is MATLAB’s
solution of𝐴−1 𝐵 for the matrix equation 𝐴𝑋 = 𝐵, where matrix 𝑋 is the same size as matrix 𝐵,
see [1].

Check with MATLAB

Try it! In command window

>> A= [2 3 1; 1 2 3; 3 1 2]; B = [9; 6; 8]; X= A\ B % Observe that B is column vector

X=
1.9444
1.6111
0.2778

Example 3.11.2
Consider that electric circuit is shown in the following figure

Figure 3.11.1: Electric circuit for example 3.11.2

The mesh equations are given as

23
Use the inverse matrix method to compute the values of the currents 𝐼1 , 𝐼2 and 𝐼3 .
Solution
For this example, the matrix equation is 𝑅𝐼 = 𝑉 or 𝐼 = 𝑅 −1 𝑉, where
10 −9 0 100 𝐼1
𝑅 = [−9 20 −9] , 𝑉 = [ 0 ], and 𝐼 = [𝐼2 ]
0 −9 15 0 𝐼3
The next step is to find 𝑅 −1. This is found from the relation
1
𝑅 −1 = 𝑎𝑑𝑗𝑅
𝑑𝑒𝑡𝑅
Therefore, we find the determinant and the adjoint of 𝑅. For this example, we find that
219 135 81
𝑑𝑒𝑡𝑅 = 975, 𝑎𝑑𝑗𝑅 = [135 150 90 ], then
81 90 119
219 135 81
−1 1 1
𝑅 = 𝑎𝑑𝑗𝑅 = 975 [135 150 90 ], and hence,
𝑑𝑒𝑡𝑅
81 90 119
𝐼1 1 219 135 81 100 100 219 22.46
𝐼 = [𝐼2 ] = [135 150 90 ] [ 0 ] = [135] = [13.85]
𝐼3 975 975
81 90 119 0 81 8.31
Type equation here.

Check with MATLAB

Try it! In command window

>> R= [10 -9 0; -9 20 -9; 0 -9 15]; V = [100; 0; 0]; I= R\ V % Observe that B is column vector
I=
22.4615
13.8562
8.31077

24
1.12. Exercise
For Exercises 1 through 3 below, the matrices 𝐴,𝐵, and 𝐶 are defined as:
1 −1 −4 5 9 −3 4 6
1 −2 3
𝐴 = [5 7 −2], 𝐵 = [−2 8 2 ], 𝐶 = [−3 8 ], and 𝐷 = [ ]
−3 6 −4
3 −5 6 7 −4 6 5 −2
1. Perform the following computations, if possible. Verify your answers with MATLAB.
a) A+B b) A+C c) B+D d) C+D e) A-B f) A-C g) B-D h) C-D

2. Perform the following computations, if possible. Verify your answers with MATLAB.
a) A. B b) A.C c) B.D d) C.D e) B.A f) C.A g) D.A h) D.C

3. Perform the following computations, if possible. Verify your answers with MATLAB.
a) 𝑑𝑒𝑡𝐴 b) 𝑑𝑒𝑡𝐵 c) 𝑑𝑒𝑡𝐶 d) 𝑑𝑒𝑡𝐷 e) 𝑑𝑒𝑡(𝐴. 𝐵) f) 𝑑𝑒𝑡(𝐴. 𝐶)
4. Solve the following system of equations using Cramer’s rule. Verify your answers with
MATLAB.

𝑥1 −2𝑥2 +𝑥3 = −4
−2𝑥1 +3𝑥2 +𝑥3 = 9
3𝑥1 + 4𝑥2 −5𝑥3 = 0
5. Repeat Exercise 4 using the Gaussian elimination method.

6. Use the MATLAB det(A) function to find the unknowns of the system of equations
below.
−𝑥1 +2𝑥2 −3𝑥3 +5𝑥4 = 14
𝑥1 +3𝑥2 +2𝑥3 −𝑥4 =9
3𝑥1 −3𝑥2 +2𝑥3 +4𝑥4 = 19
4𝑥1 +2𝑥2 +5𝑥3 +𝑥4 = 27

25
Chapter II: Least Squares Regression
Motivation
Often in physics and engineering coursework, we are asked to determine the state of
a system given the parameters of the system. For example, the relationship between
the force exerted by a linear spring, F, and the displacement of the spring from its
natural length, x, is usually represented by the model,
𝐹 = 𝑘𝑥,
where k is the spring stiffness. We are then asked to compute the force for a given k
and x value. However, in practice, the stiffness and in general, most of the parameters
of a system, are not known a priori. Instead, we are usually presented with data
points about how the system has behaved in the past. For our spring example, we
may be given (x, F) data pairs that have been previously recorded from an
experiment. Ideally, all these data points would lie exactly on a line going through
the origin (since there is no force at zero displacement). We could then measure the
slope of this line and get our stiffness value for k. However, practical data usually
has some measurement noise because of sensor inaccuracy, measurement error, or a
variety of other reasons. Figure 2.1 shows an example of what data might look like
for a simple spring experiment. This chapter teaches methods of finding the “most
likely” model parameters given a set of data; for example, how to find the spring
stiffness in our mock experiment. By the end of this chapter you should understand
how these methods choose model parameters, the importance of choosing the correct
model, and how to implement these methods in MATLAB.

26
Fig2.1: Results from force-displacement experiment for spring (fictional). The theoretical linear
relationship between force and displacement in a linear spring is F = kx. What do you think k
should be given as the experimental data?

2.1 Least Squares Regression Problem Statement

Given a set of independent data points 𝑥𝑖 and dependent data points 𝑦𝑖 , 𝑖 =

1, … , 𝑚, we would like to find an estimation function, 𝑦̂(𝑥), that describes the
data as well as possible. Note that 𝑦̂ can be a function of several variables, but for
the sake of this discussion, we restrict the domain of 𝑦̂to be a single variable. In
least squares regression, the estimation function must be a linear combination of
basis functions, 𝑓𝑖 (𝑥). That is, the estimation function must be of the form
𝑛

𝑦̂(𝑥) = ∑ 𝛼𝑖 𝑓𝑖 (𝑥)
𝑖=1
The scalars 𝛼𝑖 are referred to as the parameters of the estimation function, and each
basis function must be linearly independent from the others. In other words, in the
proper “functional space” no basis function should be expressible as a linear
combination of the other functions. Note: In general, there are significantly more
data points, m, than basis functions, n (i.e., m >> n).

TRY IT! Create an estimation function for the force-displacement relationship of a

linear spring. Identify the basis function(s) and model parameters. The relationship
between the force, F, and the displacement, x, can be described by the function
𝐹 = 𝑘𝑥 where k is the spring stiffness. The only basis function is the function
𝑓𝑖 (𝑥) = 𝑥 and the model parameter to find is 𝛼1 = 𝑘.

The goal of least squares regression is to find the parameters of the estimation
function that minimize the total squared error, E, defined by
𝐸 = ∑𝑚 ̂ − 𝑦𝑖 )2 .
𝑖=1(𝑦
The individual errors or residuals are defined as
𝑒𝑖 = (𝑦̂ − 𝑦𝑖 ) .
If e is the vector containing all the individual errors, then we are also trying to
minimize
𝐸 = ‖𝑒‖22 ,
which is the 𝐿2 norm defined in the previous chapter. In the next two sections we
derive the least squares method of finding the desired parameters. The first
derivation comes from linear algebra, and the second derivation comes from
multivariable calculus. Although they are different derivations, they lead to the
27
same least squares formula. You are free to focus on the section with which you
are most comfortable.

2.2 Least Squares Regression Derivation (Linear Algebra)

First, we enumerate the estimation of the data at each data point 𝑥𝑖 .

𝑦̂(𝑥1 ) = 𝛼1 𝑓1 (𝑥1 ) + 𝛼2 𝑓2 (𝑥1 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥1 )

𝑦̂(𝑥2 ) = 𝛼1 𝑓1 (𝑥2 ) + 𝛼2 𝑓2 (𝑥2 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥2 )
…
𝑦̂(𝑥𝑚 ) = 𝛼1 𝑓1 (𝑥𝑚 ) + 𝛼2 𝑓2 (𝑥𝑚 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥𝑚 )

Let 𝑋 ∈ ℝ𝑛 be a column vector such that the i -th element of X contains the value
of the i -th x-data point, 𝑥1 , 𝑌̂ be a column vector with elements, 𝑌̂𝑖 = 𝑦̂(𝑥𝑖 ), β be
a column vector such that 𝛽𝑖 = 𝛼𝑖 , 𝐹𝑖 (𝑥) be a function that returns a column vector
of 𝑓𝑖 (𝑥) computed on every element of x, and A be an m × n matrix such that the
i -th column of A is 𝐹𝑖 (𝑥). Given this notation, the previous system of equations
becomes 𝑌̂ = 𝐴𝛽.
Now if Y is a column vector such that 𝑌𝑖 = 𝑦𝑖 , the total squared error is given by
2
𝐸 = ‖𝑌̂ − 𝑌‖2 .
You can verify this by substituting the definition of the 𝐿2 norm. Since we want to
make E as small as possible and norms are a measure of distance, this previous
expression is equivalent to saying that we want 𝑌̂ and Y to be a “close as possible.”
Note that in general Y will not be in the range of A and
therefore E > 0.
Consider the following simplified depiction of the range of A; see Figure 13.2. Note
this is not a plot of the data points (𝑥𝑖 , 𝑦𝑖 ). From observation, the vector in the range
of A, 𝑌̂, that is closest to Y is the one that can point perpendicularly to Y. Therefore,
we want a vector Y − 𝑌̂ that is perpendicular to the vector 𝑌̂ . Recall from the chapter
on Linear Algebra that two vectors are perpendicular, or orthogonal, if their dot
product is 0. Noting that the dot product between two vectors, v and w, can be written
as
𝑑𝑜𝑡(𝑣, 𝑤) = 𝑣 𝑇 𝑤, we can state that 𝑌̂and Y − 𝑌̂ are perpendicular if
𝑑𝑜𝑡(𝑌̂, 𝑌 − 𝑌̂) = 0;
therefore, 𝑌̂ 𝑇 (𝑌 − 𝑌̂) = 0, which is equivalent to (𝐴𝛽)𝑇 (𝑌 − 𝐴𝛽) = 0.

28
 Fig2.2: Illustration of the L2 projection of Y on the range of A.

Noting that for two matrices 𝐴 and 𝐵, (𝐴𝐵)𝑇 = 𝐵𝑇 𝐴𝑇 and using distributive
properties of vector multiplication, this is equivalent to
𝛽 𝑇 𝐴𝑇 𝑌 − 𝛽 𝑇 𝐴𝑇 𝐴𝛽 = 𝛽 𝑇 (𝐴𝑇 𝑌 − 𝐴𝑇 Aβ) = 0.
The solution, β = 0, is a trivial solution, so we use 𝐴𝑇 𝑌 − 𝐴𝑇 𝐴 𝛽 = 0 to find a
more interesting solution. Solving this equation for β gives the least squares
regression formula:
𝛽 = (𝐴𝑇 𝐴)−1 𝐴𝑇 𝑌

Note that (𝐴𝑇 𝐴)−1 𝐴𝑇 is called the pseudo-inverse of A and exists when m > n
and A has linearly independent columns.

2.3 Least Squares Regression Derivation (Multivariable Calculus)

Recall that the total error for m data points and n basis functions is:

𝑚 𝑚 𝑚 𝑛 2
2 2
𝐸=∑ 𝑒𝑖 = ∑ (𝑦̂(𝑥𝑖 ) − 𝑦𝑖 ) = ∑ (∑ 𝛼𝑗 𝑓𝑗 (𝑥𝑖 ) − 𝑦𝑖 )
𝑖=1 𝑖=1 𝑖=1 𝑗=1
which is an n-dimensional paraboloid in αk . From calculus, we know that the
minimum of a paraboloid is where all the partial derivatives equal zero. So, taking
partial derivative of E with respect to the variable αk (remember that in this case the
parameters are our variables), setting the system of equations equal to 0 and solving
for the αk ’s should give the correct results. The partial derivative with respect to αk
and setting equal to 0 yields:

29
 𝝏𝑬 𝑚 𝑛
= ∑ 2 ( ∑ 𝛼𝑗 𝑓𝑗 (𝑥𝑖 ) − 𝑦𝑖 ) 𝑓k (𝑥𝑖 ) = 0
𝝏𝜶𝒊 𝑖=1 𝑗=1

With some rearrangement, the previous expression can be manipulated to the

following:
𝑚 𝑛 𝑚
∑ ∑ 𝛼𝑗 𝑓𝑗 (𝑥𝑖 )𝑓k (𝑥𝑖 ) − ∑ 𝑦𝑖 𝑓k (𝑥𝑖 ) = 0
𝑖=1 𝑗=1 𝑖=1

and further rearrangement taking advantage of the fact that addition commutes result in:
𝑛 𝑚 𝑚
∑ 𝛼𝑗 ∑ 𝑓𝑗 (𝑥𝑖 )𝑓k (𝑥𝑖 ) = ∑ 𝑦𝑖 𝑓k (𝑥𝑖 )
𝑗=1 𝑖=1 𝑖=1

Now let X be a column vector such that the i -th element of X is 𝑥𝑖 and Y similarly
constructed, and let 𝐹𝑗 (𝑋) be a column vector such that the i -th element of 𝐹𝑗 (𝑋) is
𝑓𝑗 (𝑥𝑖 ). Using this notation, the previous expression can be rewritten in vector
notation as:

If we repeat this equation for every k, we get the following system of linear
equations in matrix form:

If we let 𝐴 = [𝐹1 (𝑋), 𝐹2 (𝑋), . . . , 𝐹𝑗 (𝑋), . . . , 𝐹𝑛 (𝑋)] and β be a column vector such
that j -th element of β is 𝛼𝑗 , then the previous system of equations becomes
𝐴𝑇 𝐴 𝛽 = 𝐴𝑇 𝑌 ,

30
and solving this matrix equation for β gives 𝛽 = (𝐴𝑇 𝐴)−1 𝐴𝑇 𝑌, which is exactly
the same formula as the previous derivation.

2.4 Least Squares Regression in MATLAB

Recall that if we enumerate the estimation of the data at each data point, 𝑥𝑖 , this
gives us the following system of equations:
𝑦̂(𝑥1 ) = 𝛼1 𝑓1 (𝑥1 ) + 𝛼2 𝑓2 (𝑥1 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥1 )
𝑦̂(𝑥2 ) = 𝛼1 𝑓1 (𝑥2 ) + 𝛼2 𝑓2 (𝑥2 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥2 )
…
𝑦̂(𝑥𝑚 ) = 𝛼1 𝑓1 (𝑥𝑚 ) + 𝛼2 𝑓2 (𝑥𝑚 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥𝑚 )

If the data was absolutely perfect (i.e., no noise), then the estimation function
would go through all the data points, resulting in the following system of
equations:

𝑦1 = 𝛼1 𝑓1 (𝑥1 ) + 𝛼2 𝑓2 (𝑥1 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥1 )

𝑦2 = 𝛼1 𝑓1 (𝑥2 ) + 𝛼2 𝑓2 (𝑥2 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥2 )
…
𝑦𝑚 = 𝛼1 𝑓1 (𝑥𝑚 ) + 𝛼2 𝑓2 (𝑥𝑚 ) + ⋯ + 𝛼𝑛 𝑓𝑛 (𝑥𝑚 )

If we take A to be as defined previously, this would result in the matrix equation

𝑦 = 𝐴𝑥
However, since the data is not perfect, there will not be an estimation function that
can go through all the data points, and this system will have no solution. However,
recall in the previous chapter that
x =A\y
would return a solution even if no solution existed. It turns out the solution
returned by MATLAB for this command is the least squares solution derived in the
previous two sections. In other words, if there is a solution, x = A\y will return one,
otherwise it will return the x that is the closest to being a solution to the matrix
equation. The pseudo-inverse for of A can be computed using the MATLAB
function pinv.

31
32
33
34
3. Chapter III: Differential Equations
This chapter is a review of ordinary differential equations.

3.1. Simple Differential equations

In this section we present two simple examples to show the importance of differential
equations in applications; especially, in Engineering fields.
Example 4.1.1
The current and voltage in a capacitor are related by
𝑑𝑣𝐶 (4.1)
𝑖𝐶 (𝑡) = 𝐶
𝑑𝑡
Where 𝑖𝐶 (𝑡) is the current through the capacitor, 𝑣𝐶 is the voltage across the capacitor, and the
constant 𝐶 is the capacitance in farads (𝐹). For this example, 𝐶 = 1 𝐹 and the capacitor is being
charged by a constant current 𝐼. Find the voltage 𝑣𝐶 across this capacitor as a function of time
given that the voltage at some reference time 𝑡 = 0 is 𝑣0 .

Solution
It is given that the current, as a function of time, is constant, that is,

𝑖𝐶 (𝑡) = 𝐼 = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 (4.2)

By substituting equation (4.2) in equation (4.1) we get

𝑑𝑣𝐶
=𝐼
𝑑𝑡
And by separation of the variables,

𝑑𝑣𝐶 = 𝐼𝑑𝑡 (4.3)

Integrating both sides of equation (4.4) we obtain

𝑣𝐶 (𝑡) = 𝐼𝑡 + 𝐾 (4.4)

Where 𝐾 represents the constants of integration of both sides.

We can find the value of the constant 𝐾 by making use of the initial condition, i.e., at 𝑡 = 0,
𝑣𝐶 = 𝑣0 and equation (4.4) then becomes
𝑣0 = 0 + 𝐾 (4.5)

35
Or 𝑘 = 𝑣0 and by substitution in equation (4.4) we get

𝑣𝐶 (𝑡) = 𝐼𝑡 + 𝑣0 (4.6)

This example shows that when a capacitor is charged with a constant current, a linear voltage is
produced across the terminals of the capacitor.

Example 4.1.2
Given that,
𝑑𝑖𝐿
= cos( 𝑡) (4.7)
𝑑𝑡
By separating the variables, we obtain

𝑑𝑖𝐿 = cos( 𝑡) 𝑑𝑡 (4.8)

and integrating both sides we obtain

𝑖𝐿 (𝑡) = sin( 𝑡) + 𝐾 (4.9)

where represents the constants of integration of both sides.

We find the value of the constant 𝐾 by making use of the initial condition. For this example, 𝜔 =
1 and thus at 𝜔𝑡 = 𝑡 = 𝜋⁄2, 𝑖𝐿 = 1. With these values, equation (4.9) becomes
𝜋 (4.10)
1 = sin ( ) + 𝐾
2
Or 𝑘 = 0, and by substitution into equation (4.9)

𝑖𝐿 (𝑡) = sin(𝑡) (4.11)

3.2. Classification of Differential equations

Differential equations are classified by:
1. Type - Ordinary or Partial.
2. Order - The highest order derivative which is included in the differential equation.
3. Degree - The exponent of the highest power of the highest order derivative after the
differential equation has been cleared of any fractions or radicals in the dependent
variable and its derivatives
For example, the differential equation
2 4 6
𝑑4𝑦 𝑑3𝑦 𝑑2𝑦 𝑑𝑦 8 𝑦2
( 4) + 5 ( 3) + 6 ( 2) + 3 ( ) + 3 = 𝑦𝑒 −2𝑥
𝑑𝑥 𝑑𝑥 𝑑𝑥 𝑑𝑥 𝑥 +1
is an ordinary differential equation of order 4 and degree 2

36
If the dependent variable 𝑦 is a function of only a single variable 𝑥, that is, if 𝑦 = 𝑓(𝑥), the
differential equation which relates 𝑦 and 𝑥 is said to be an ordinary differential equation and it is
abbreviated as ODE.
The differential equation
𝑑2 𝑦 𝑑𝑦
2
+3 + 2 = 5 cos(4𝑡)
𝑑𝑡 𝑑𝑡
is an ODE with constant coefficients.
The differential equation
𝑑2𝑦 𝑑𝑦
𝑥2 + 𝑥 + (𝑥2 − 𝑛2 ) = 0
𝑑𝑡 2 𝑑𝑡
is an ODE with variable coefficients.
If the dependent variable 𝑦 is a function of two or more variables such as 𝑦 = 𝑓(𝑥, 𝑡), where 𝑥
and 𝑡 are independent variables, the differential equation that relates 𝑦, 𝑥,and 𝑡 is said to be a
partial differential equation and it is abbreviated as PDE.
An example of a partial differential equation is the well-known one-dimensional wave equation
shown below.
𝜕 2𝑦 2 𝜕 2𝑦
=𝑎
𝑑𝑡 2 𝑑𝑥 2
Most engineering problems are solved with ordinary differential equations with constant
coefficients; however, partial differential equations provide often quick solutions to some
practical applications as illustrated with the following three examples, see [1].

Example 4.2.1
The equivalent resistance 𝑅𝑇 of three resistors 𝑅1 , 𝑅2 , and 𝑅3 in parallel is obtained from
1 1 1 1
= + +
𝑅𝑇 𝑅1 𝑅2 𝑅3
Given that initially 𝑅1 = 5 Ω, 𝑅2 = 20 Ω , and 𝑅3 = 4 Ω, compute the change in𝑅𝑇 if 𝑅2 is
increased by 10% and 𝑅3 is decreased 5% by while 𝑅1 does not change.
Solution
The initial value of the equivalent resistance is 𝑅𝑇 = 5 ‖20‖4 = 2 Ω.
Now, we treat 𝑅2 and 𝑅3 as constants and differentiating 𝑅𝑇 with respect to 𝑅1 we obtain

1 𝜕𝑅𝑇 1 𝜕𝑅𝑇 𝑅 2
− =− or = ( 𝑇)
𝑅𝑇 2 𝜕𝑅1 𝑅1 2 𝜕𝑅1 𝑅 1
Similarly,

𝜕𝑅𝑇 𝑅 2 𝜕𝑅𝑇 𝑅 2
= ( 𝑇) and = ( 𝑇)
𝜕𝑅2 𝑅 2 𝜕𝑅3 𝑅 3
and the total differential 𝑑𝑅𝑇 is
𝜕𝑅𝑇 𝜕𝑅 𝜕𝑅 𝑅 2 𝑅 2 𝑅 2
𝑑𝑅𝑇 = 𝑑𝑅1 + 𝜕𝑅𝑇 𝑑𝑅2 + 𝜕𝑅𝑇 𝑑𝑅3 = ( 𝑅𝑇 ) 𝑑𝑅1 + ( 𝑅𝑇 ) 𝑑𝑅2 + (𝑅𝑇 ) 𝑑𝑅3
𝜕𝑅1 2 3 1 2 3

37
By substitution of the given numerical values we obtain
2 2 2 2 2 2
𝑑𝑅𝑇 = (5) (0) + (20) (2) + (4) (−0.2) = 0.02 − 0.05 = −0.03
Therefore, the equivalent resistance decreases by 3%.
Example 4.2.2
In a series 𝑅𝐶 electric circuit that is excited by a sinusoidal voltage, the magnitude of the
impedance 𝑍 is computed from 𝑍 = √𝑅 2 + 𝑋𝐶2 . Initially, 𝑅 = 4 Ω and 𝑋𝐶 = 3 Ω. Find the
change in the impedance 𝑍 if the resistance 𝑅 is increased by 0.25 Ω (6.25%) and the capacitive
reactance 𝑋𝐶 is decreased by 0.125 Ω (−4.167%).

Solution
𝑑𝑍 𝑑𝑍
We will first find the partial derivatives and ; then we compute the change in impedance
𝑑𝑅 𝑑𝑋𝐶
from the total differential 𝑑𝑍. Thus,
𝜕𝑍 𝑅 𝜕𝑍 𝑋𝐶
= and =
𝜕𝑅 𝜕𝑋𝐶
√𝑅 2 +𝑋𝐶2 √𝑅 2+𝑋𝐶2

And hence,
𝜕𝑍 𝜕𝑍 𝑅 𝑑𝑅 + 𝑋𝐶 𝑑𝑋𝐶
𝑑𝑍 = 𝑑𝑅 + 𝑑𝑋𝐶 =
𝜕𝑅 𝜕𝑋𝐶 √𝑅 2 + 𝑋𝐶2
and by substitution of the given values
4 (0.25) + 3 (−0.125) 1 − 0.375
𝑑𝑅 = = = 0.125
√42 + 33 5
Therefore, if 𝑅 increases by 6.25% and 𝑋𝐶 decreases by 4.167%, the impedance 𝑍 increases by
4.167%.

38
 3.3. Solutions of Ordinary Differential Equations (ODE)

A function 𝑦 = 𝑓(𝑥) is a solution of a differential equation if the latter is satisfied when and its
derivatives are replaced throughout by 𝑓(𝑥) and its corresponding derivatives. Also, the initial
conditions must be satisfied.

𝑑2𝑦
For example, a solution of the differential equation + 𝑦 = 0 is
𝑑𝑥 2
𝑦 = 𝑘1 sin(𝑥) + 𝑘2 cos (𝑥)
Since 𝑦 and its second derivative satisfy the given differential equation.
Any linear, time-invariant system can be described by an ODE which has the form

(4.12)

If the excitation in (4.12) is not zero, that is, if 𝑥(𝑡) ≠ 0, the ODE is called a non-homogeneous
ODE. If 𝑥(𝑡) = 0, it reduces to:

(4.13)

The differential equation of (4.13) above is called a homogeneous ODE and has different linearly
independent solutions denoted as 𝑦1 (𝑡), 𝑦2 (𝑡), 𝑦3 (𝑡), … , 𝑦𝑛 (𝑡).
• It's not hard to prove that equation (4.13) has a general solution (see [1]) and presented as
𝑦𝐻 (𝑡) = 𝑘1 𝑦1 (𝑡) + 𝑘2 𝑦2 (𝑡) + 𝑘3 𝑦3 (𝑡) + ⋯ + 𝑘𝑛 𝑦𝑛 (𝑡) (4.14)

where the subscript 𝐻 on the left side is used to emphasize that this is the form of the solution of
the homogeneous ODE and 𝑘1 , 𝑘2 , 𝑘3 , … , 𝑘𝑛 are arbitrary constants.

In our subsequent discussion, the solution of the homogeneous ODE, i.e., the complementary
solution, will be referred to as the natural response, and will be denoted as 𝑦𝑁 (𝑡) or simply 𝑦𝑁 .
The particular solution of a non-homogeneous ODE will be referred to as the forced response,
and will be denoted as 𝑦𝐹 (𝑡) or simply 𝑦𝐹 .

39
Accordingly, we express the total solution of the non-homogeneous ODE of (4.12) as:

(4.15)

The natural response 𝑦𝑁 contains arbitrary constants and these can be evaluated from the given
initial conditions. The forced response 𝑦𝐹 , however, contains no arbitrary constants. It is
imperative to remember that the arbitrary constants of the natural response must be evaluated
from the total response.

3.4. Solution of the Homogeneous ODE

Let the solutions of the homogeneous ODE
𝑑𝑛 𝑦 𝑑 𝑛−1 𝑦 𝑑𝑦 (4.16)
𝑎𝑛 𝑛 + 𝑎𝑛−1 𝑛−1 + ⋯ + 𝑎1 + 𝑎0 𝑦 = 0
𝑑𝑡 𝑑𝑡 𝑑𝑡
Will be of the form
𝑦 = 𝑘𝑒 𝑠𝑡 (4.17)
Then by substituting equation (4.17) in eq. (1.16) we obtain
𝑎𝑛 𝑘𝑠 𝑛 𝑒 𝑠𝑡 + 𝑎𝑛−1 𝑘𝑠 𝑛−1 𝑒 𝑠𝑡 + ⋯ + 𝑎1 𝑘𝑠𝑒 𝑠𝑡 + 𝑎0 𝑘𝑒 𝑠𝑡 = 0

(𝑎𝑛 𝑠 𝑛 + 𝑎𝑛−1 𝑠 𝑛−1 + ⋯ + 𝑎1 𝑠 + 𝑎0 ) 𝑘𝑒 𝑠𝑡 = 0 (4.18)

We observe that (4.18) can be satisfied when
(𝑎𝑛 𝑠 𝑛 + 𝑎𝑛−1 𝑠 𝑛−1 + ⋯ + 𝑎1 𝑠 + 𝑎0 ) = 0 or 𝑘 = 0 or 𝑠 = −∞ (4.19)
but the only meaningful solution is the quantity enclosed in parentheses since the latter two yield
trivial (meaningless) solutions. We, therefore, accept the expression inside the parentheses as the
only meaningful solution and this is referred to as the characteristic (auxiliary) equation, that is,

(4.20)

Since the characteristic equation is an algebraic equation of an nth-power polynomial, its

solutions are 𝑠1 , 𝑠2 , 𝑠3 , … , 𝑠𝑛 , and thus the solutions of the homogeneous ODE are:
𝑦1 = 𝑘1 𝑒 𝑠1 𝑡 , 𝑦2 = 𝑘2 𝑒 𝑠2𝑡 , 𝑦3 = 𝑘3 𝑒 𝑠3𝑡 , … , 𝑦𝑛 = 𝑘𝑛 𝑒 𝑠𝑛𝑡 (4.21)

Case 1: Distinct Roots

If the roots of the characteristic equation are distinct (different from each another), the solutions
of (4.20) are independent and the most general solution is:

(4.22)

40
Case 2: Repeated Roots
If two or more roots of the characteristic equation are repeated (same roots), then some of the
terms of (4.21) are not independent and therefore (4.22) does not represent the most general
solution. If, for example, 𝑠1 = 𝑠2 , then,
𝑘1 𝑒 𝑠1 𝑡 + 𝑘2 𝑒 𝑠2𝑡 = 𝑘1 𝑒 𝑠1 𝑡 + 𝑘2 𝑒 𝑠1𝑡 = (𝑘1 + 𝑘2 )𝑒 𝑠1𝑡 = 𝑘3 𝑒 𝑠1𝑡
and we see that one term of (4.22) is lost. In this case, we express one of the terms of (4.22), say
𝑘2 𝑒 𝑠1𝑡 as 𝑘2 𝑡𝑒 𝑠1 𝑡 . These two represent two independent solutions and therefore the most general
solution has the form:
𝑦𝑁 = (𝑘1 + 𝑘2 )𝑒 𝑠1𝑡 + 𝑘3 𝑒 𝑠3 𝑡 + ⋯ + 𝑘𝑛 𝑒 𝑠𝑛𝑡 (4.23)
If there 𝑚 are equal roots the most general solution has the form:

(4.24)

Case 3: Complex Roots

If the characteristic equation contains complex roots, these occur as complex conjugate pairs.
Thus, if one root is 𝑠1 = 𝛼 + 𝛽𝑖 where 𝛼 and 𝛽 are real numbers, then another root is 𝑠1 = −𝛼 −
𝛽𝑖. Then,

(4.25)

If (4.25) is to be a real function of time, the constants 𝑘1 and 𝑘2 must be complex conjugates. The
other constants 𝑘3 , 𝑘4 , 𝑘5 , and the phase angle 𝜑 are real constants. The forced response can be
found by
• The Method of Undetermined Coefficients

3.5. Using the Method of Undetermined Coefficients for the

Forced Response
For simplicity, we will only consider ODEs of order 2.
Consider the non-homogeneous ODE
𝑑2𝑦 𝑑𝑦
𝑎 2 +𝑏 + 𝑐𝑦 = 𝑓(𝑥) (4.26)
𝑑𝑡 𝑑𝑡
where a, b, and c are real constants.
We have learned that the total (complete) solution consists of the summation of the natural and
forced responses.

41
For the natural response, if 𝑦1 and 𝑦2 are any two solutions of (4.26), the linear
combination, 𝑦3 = 𝑘1 𝑦1 + 𝑘2 𝑦2 where 𝑘1 and 𝑘2 are arbitrary constants, is also a solution, that is,
if we know the two solutions, we can obtain the most general solution by forming the linear
combination of 𝑦1 and 𝑦2 . To be certain that there exist no other solutions, we examine the
Wronskian Determinant defined below.

(4.27)

If (4.27) is true, we can be assured that all solutions of (4.26) are indeed the linear combination
of 𝑦1 and 𝑦2 .
The forced response is obtained by observation of the right side of the given ODE as it is
illustrated by the examples that follow.
Example 4.5.1
Find the total solution of the ODE
𝑑2𝑦 𝑑𝑦
+ 4 + 3𝑦 = 0
𝑑𝑡 2 𝑑𝑡 (4.28)
subject to the initial conditions 𝑦(0) = 3 and 𝑦 ′ (0) = 4 where 𝑦 ′ = 𝑑𝑦/𝑑𝑡.

Solution
This is a homogeneous ODE and its total solution is just the natural response found from the
characteristic equation 𝑠 2 + 4𝑠 + 3 = 0 whose roots are 𝑠1 = −1 and 𝑠2 = −3. The total
response is:
𝑦(𝑡) = 𝑦𝑁 (𝑡) = 𝑘1 𝑒 −𝑡 + 𝑘2 𝑒 −3𝑡 (4.29)
The constants 𝑘1 and 𝑘2 are evaluated from the given initial conditions. For this example,
𝑦(0) = 3 = 𝑘1 𝑒 0 + 𝑘2 𝑒 0 or
𝑘1 + 𝑘2 = 3 (4.30)
Also,
𝑑𝑦
𝑦 ′ (0) = 4 = | = 𝑘1 𝑒 −𝑡 − 3𝑘2 𝑒 −3𝑡 |𝑡=0
𝑑𝑡 𝑡=0
−𝑘1 − 3𝑘2 = 4 (4.31)
Simultaneous solution of (4.30) and (4.31) yields 𝑘1 = 6.5 and 𝑘2 = −3.5. By substitution into
(4.29), we obtain
𝑦(𝑡) = 𝑦𝑁 (𝑡) = 6.5 𝑒 −𝑡 − 3.5 𝑒 −3𝑡 (4.32)

42
Check with MATLAB:
Try it! In command window
>> y= dsolve ('D2y+4*Dy+3*y=0', 'y(0)=3', 'Dy(0)=4')
y=
(-7/2*exp(-3*t)*exp(t)+13/2)/exp(t)
>> pretty(y)
- 7/2 exp (-3 t) exp(t) + 13/2
------------------------------------
exp(t)

The function 𝑦 = 𝑓(𝑥), of relation (4.32), shown in Figure 4.5.1, was plotted with the use of the
MATLAB script
Try it! In Editor window
1. y=dsolve ('D2y+4*Dy+3*y=0', 'y(0)=3', 'Dy(0)=4');
2. ezplot (y, [0 5])
3. xlabel (‘t’)
4. title (‘13/2 exp(-t)-7/2 exp(-3t)’)

Figure 4.5.1: Plot pf the function 𝑦 = 𝑓(𝑥) for example 4.5.1

Example 4.5.2
Find the total solution of the ODE
𝑑2𝑦 𝑑𝑦 (4.33)
+ 4 + 3𝑦 = 3𝑒 −2𝑡
𝑑𝑡 2 𝑑𝑡
subject to the initial conditions 𝑦(0) =1 and 𝑦 ′ (0) = −1.
Solution

43
The left side of (4.33) is the same as that of Example 4.5.1; Therefore,
𝑦𝑁 (𝑡) = 𝑘1 𝑒 −𝑡 + 𝑘2 𝑒 −3𝑡 (4.34)
(We must remember that the constants 𝑘1 and 𝑘2 must be evaluated from the total response). To
find the forced response, we assume a solution of the form
𝑦𝐹 = 𝐴𝑒 −2𝑡 (4.35)
by substituting (4.35) into the given ODE of (4.33). Then,
4𝐴𝑒 −2𝑡 − 8𝐴𝑒 −2𝑡 + 3𝐴𝑒 −2𝑡 = 3𝑒 −2𝑡 (4.36)
from which 𝐴 = −3 and the total solution is
𝑦(𝑡) = 𝑦𝑁 + 𝑦𝐹 = 𝑘1 𝑒 −𝑡 + 𝑘2 𝑒 −3𝑡 − 3𝑒 −2𝑡 (4.37)
The constants and are evaluated from the given initial conditions
𝑦(0) = 𝑘1 𝑒 0 + 𝑘2 𝑒 0 − 3𝑒 0
Or
𝑘1 + 𝑘2 = 4 (4.38)
Also,
𝑑𝑦
𝑦 ′ (0) = −1 = | = 𝑘1 𝑒 −𝑡 −3𝑘2 𝑒 −3𝑡 |𝑡=0 + 6𝑒 −2𝑡 |𝑡=0
𝑑𝑡 𝑡=0
Or
−𝑘1 + 3𝑘2 = −7 (4.39)
Simultaneous solution of (4.38) and (4.39) yields 𝑘1 = 2.5 and 𝑘2 = 1.5. By substitution into
(4.37), we obtain
𝑦(𝑡) = 𝑦𝑁 + 𝑦𝐹 = 2.5 𝑒 −𝑡 + 1.5 𝑒 −3𝑡 − 3𝑒 −2𝑡 (4.40)
Check with MATLAB:

Try it! In command window

>> y=dsolve('D2y+4*Dy+3*y=3*exp(−2*t)', 'y(0)=1', 'Dy(0)=−1')
y=
(-3*exp(-2*t)*exp(t)+3/2*exp(-3*t)*exp(t)+5/2)/exp(t)
>> pretty(y)
-3 exp(-2 t) exp(t) + 3/2 exp(-3 t) exp(t) + 5/2
------------------------------------------------------------
exp(t)

The plot is shown in Figure 5.2 was produced with the MATLAB script
Try it! In Editor window
1. y=dsolve('D2y+4*Dy+3*y=3*exp(−2*t)', 'y(0)=1', 'Dy(0)=−1');
2. ezplot (y, [0 8])
3. title (‘5/2 exp(-t)-3/2 exp(-3t)-3 exp(-2t)’);
4. xlabel(‘t’);

44
 Figure 4.5.2: Plot pf the function 𝑦 = 𝑓(𝑥) for example 4.5.2

Example 4.5.3
Find the total solution of the ODE
𝑑2𝑦 𝑑𝑦 (4.41)
2
+6 + 9𝑦 = 0
𝑑𝑡 𝑑𝑡
subject to the initial conditions 𝑦(0) = -1 and 𝑦 ′ (0) = 1.
Solution
This is a homogeneous ODE and therefore its total solution is just the natural response found
from the characteristic equation 𝑠 2 + 6𝑠 + 9 = 0 whose roots are 𝑠1 = 𝑠2 = −3 (repeated roots).
Thus, the total response is
𝑦(𝑡) = 𝑦𝑁 (𝑡) = 𝑘1 𝑒 −3𝑡 + 𝑘2 𝑡𝑒 −3𝑡 (4.42)
Next, we evaluate the constants 𝑘1 and 𝑘2 from the given initial conditions.

𝑦(0) = −1 = 𝑘1 𝑒 0 + 𝑘2 (0) 𝑒 0
Or
𝑘1 = −1 (4.43)
Also,
𝑑𝑦
𝑦 ′ (0) = 1 = | = −3 𝑘1 𝑒 −3𝑡 −𝑘2 𝑒 −3𝑡 |𝑡=0 + 3𝑘2 𝑒 −3𝑡 |𝑡=0
𝑑𝑡 𝑡=0

Or
−3𝑘1 + 𝑘2 = 1 (4.44)
From (4.43) and (4.44) we obtain 𝑘1 = −1 and 𝑘2 = −2. By substitution into (4.42), we get
𝑦(𝑡) = −𝑒 −3𝑡 − 2𝑡𝑒 −3𝑡 (4.45)

45
Check with MATLAB
Try it! In command window
>> y=dsolve ('D2y+6*Dy+9*y=0', 'y(0)=−1', 'Dy(0)=1')
y =
-exp(-3*t)-2*exp(-3*t)*t

The plot shown in Figure 4.5.3 was produced with the following MATLAB script
Try it! In Editor window
1. y=dsolve ('D2y+6*Dy+9*y=0', 'y(0)=−1', 'Dy(0)=1');
2. ezplot(y,[0 3])
3. title(‘ ‘);
4. xlabel(‘ t’);

Figure 4.5.3: Plot pf the function 𝑦 = 𝑓(𝑥) for example 4.532

46
References
[1] Karris, S. T. (2007). Numerical analysis using MATLAB and Excel. Orchard
Publications.
[2] Siauw, T., & Bayen, A. (2014). An introduction to MATLAB® programming and
numerical methods for engineers. Academic Press.

47