MA1522: Linear Algebra for Computing

Chapter 2: Matrix Algebra

2.1 Definition and Special types of Matrices
Definition

Definition
A (real-valued) matrix is a rectangular array of (real) numbers
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
m n
A= . ..  = (aij )m×n = (aij )i=1 j=1 .,
 
. .. . .
 . . . . 
am1 am2 · · · amn

where aij ∈ R are real numbers. The size of the matrix is said to be m × n (read as m by n), where m is the number
of rows and n is the number of columns.

The numbers in the array are called entries. The (i, j)-entry, aij , i = 1, ..., m, j = 1, ..., n, is the number in the i-th
row j-th column.
Question

Let  
1 2 −1 5 10
A= 3 5 13 23 41
−7 2 0 0 11

1. What is the size of the matrix?

2. What is the (2, 3)-entry?

Remarks

▶ The size
 of a matrix is read as
 m by n. One should not multiply the numbers. For example,
1 2 −1 5 10
A= 3 5 13 23 41 is a 3 by 5 matrix, not a size 15 matrix.
−7 2 0 0 11

▶ There is a need to add a comma when labelling the (i, j)-entry, ai,j , if there is ambiguity. For example, a123 may
mean the (1, 23)-entry, in which case we label it is as a1,23 instead, or the (12, 3)-entry, in which case we label it
as a12,3 instead.
Example

We may define a matrix by some formula on its entry. For example,

A = (aij )2×3 , aij = i + j

 
2 3 4
A=
3 4 5
Example

B = (bij )3×2 , bij = (−1)i+j

 
1 −1
B = −1 1
1 −1
Special types of Matrices
Vectors

A n × 1 matrix is called a (column) vector, and a 1 × n matrix is called a (row) vector.

Remark
If it is not specified whether the vector is a column or a row vector, by default we will assume it is a column vector.
Zero matrices

All entries equal 0, denoted as 0m×n . Not necessarily a square matrix.

Example
 
  0
  0 0 0
0 0 0 0

02×3 = , 03×3 = 0
 0 0 , 04×1 =
0 ,
 01×1 = 0 .
0 0 0
0 0 0
0
Square matrices
Number of rows = number of columns
 
a11 a12 ··· a1n
a21 a22 ··· a2n 
A = (aij )n =  . ..  ,
 
.. ..
 .. . . . 
an1 an2 ··· ann

Example
 
  1 1 0 1
  1 2 3
2 −3 4
0 1 0 −1
, 5 6 ,  .
7 0 1 1 0 0
7 8 9
−1 −1 0 1

Definition
i. A size n × n matrix is a square matrix of order n.

ii. The entries aii , i = 1, 2, ..., n, (explicitly, a11 , a22 , ..., ann ) are called the diagonal entries of the (square) matrix.
Diagonal, Scalar, Identity matrices
 
d1 0 ··· 0
0 d2 ··· 0
̸ j. Denote as D = diag (d1 , d2 , ..., dn ) =  .
1. Diagonal matrix D = (aij )n , aij = 0 for i = .. .
 
.. ..
 .. . . .
0 0 ··· dn
 
c 0 ··· 0

c if i = j
0 c ··· 0
2. Scalar matrix C = (aij ), aij = , C = diag (c, c, ..., c) =  . .. .
 
.. ..
0 if i ̸= j  .. . . .
0 0 ··· c
 
1 0 ··· 0

1 if i = j
0 1 ··· 0
3. Identity matrix I = (aij ), aij = , In = diag (1, 1, ..., 1) =  . . .. .
 
0 if i ̸= j  .. .. ..
. .
0 0 ··· 1
A scalar matrix can also be denoted as C = cI, where I is the identity matrix. See later for definition of scalar
multiplication.
Question

Which of the following options are true? The 3 by 3 zero matrix 03×3 is a
(a) diagonal matrix.

(b) scalar matrix.

(c) identity matrix.

Triangular matrices
 
∗ ∗ ··· ∗
0 ∗ ··· ∗
Upper triangular A = (aij ), aij = 0 for all i > j:  .
 
.. . . .. 
 .. . . .
0 0 ··· ∗
 
0 ∗ ··· ∗
0 0 · · · ∗
Strictly upper triangular A = (aij ), aij = 0 for all i ≥ j:  .
 
.. .. .
 .. . . .. 
0 0 ··· 0
 
∗ 0 ··· 0
∗ ∗ · · · 0
Lower triangular A = (aij ), aij = 0 for all i < j:  . . .
 
 .. .. . . ... 

∗ ∗ ··· ∗
 
0 0 ··· 0
∗ 0 · · · 0
Strictly lower triangular A = (aij ), aij = 0 for all i ≤ j:  . . .
 
 .. .. . . ... 

∗ ∗ ··· 0
Question

Which of the following statements are true?

(i) Every square matrix is row equivalent to an upper triangular matrix.

(ii) Every square matrix is row equivalent to a strictly upper triangular matrix.
Symmetric matrices

A = (aij )n , aij = aji .

   
a11 a12 ··· a1n a11 a21 ··· an1
a21 a22 ··· a2n  a12 a22 ··· an2 
..  =  ..
   
 .. .. .. .. .. .. 
 . . . .   . . . . 
an1 an2 ··· ann a1n a2n ··· ann

Example
 
1 2 b
a 0 4  is symmetric ⇔ a = 2, b = 3, c = 4, d ∈ R.
3 c d
Question

A symmetric upper triangular matrix is a zero matrix. True or false?

2.2 Matrix Algebra
Equality

Two matrices are equal if they have the same size and their corresponding entries are equal;

A = (aij )n×m and B = (bij )k×l are equal if and only if n = k, m = l, and aij = bij for all i = 1, ..., n, j = 1, ..., m.

Example
   
2 1 a b c
1. ̸= for any a, b, c, d, e, f ∈ R since the matrices do not have the same sizes.
3 2 d e f

   
1 1 a b
2. = ⇔ a = b = 1, c = 3, d = 2.
3 2 c d
Matrix Addition and Scalar Multiplication
Definition    
a11 a12 ··· a1n ca11 ca12 ··· ca1n
 a21 a22 ··· a2n   ca21 ca22 ··· ca2n 
1. Scalar multiplication: c  . ..  =  .. .. .
   
.. .. .. ..
 .. . . .   . . . . 
am1 am2 ··· amn cam1 cam2 ··· camn
2. 
Matrix addition:     
a11 a12 · · · a1n b11 b12 ··· b1n a11 + b11 a12 + b12 ··· a1n + b1n
 a21 a22 · · · a2n   b21 b22 ··· b2n   a21 + b21 a22 + b22 ··· a2n + b2n 
..  +  .. ..  = 
     
 .. .. .. .. .. .. .. .. .. 
 . . . .   . . . .   . . . . 
am1 am2 · · · amn bm1 bm2 ··· bmn am1 + bm1 am2 + bm2 ··· amn + bmn

Example
   
6 1 −1 30 5 −5
1. 5 2 −4 3  = 10 −20 15 
4 9 −11 20 45 −55
     
1 2 −3 2 1 5 3 3 2
2. + =
4 5 6 1 −3 2 5 2 8
Remark

1. Matrix addition is only defined between matrices of the same size.

2. −A = (−1)A.

3. Matrix subtraction is defined to be the addition of a negative multiple of another matrix,

A − B = A + (−1)B.
Properties of Matrix Addition and Scalar Multiplication

Theorem
For matrices A = (aij )m×n , B = (bij )m×n , C = (cij )m×n , and real numbers a, b ∈ R,
(i) (Commutative) A + B = B + A,

(ii) (Associative) A + (B + C) = (A + B) + C,

(iii) (Additive identity) 0m×n + A = A,

(iv) (Additive inverse) A + (−A) = 0m×n ,

(v) (Distributive law) a(A + B) = aA + aB,

(vi) (Scalar addition) (a + b)A = aA + bA,

(vii) (Associative) (ab)A = a(bA),

(viii) If aA = 0m×n , then either a = 0 or A = 0.

Remarks

▶ The proof of the theorem can be found in the appendix.

▶ However, intuitively, the properties follow from the properties of addition and multiplication of real numbers,
since scalar multiplication and matrix addition is defined entries-wise.

▶ By the associativity of matrix addition, if A1 , A2 , ..., Ak are k matrices of the same size, we simply write the
sum as
A1 + A2 + · · · + Ak ,
without the parenthesis (brackets).
Matrix Multiplication

Pp
AB = (aij )m×p (bij )p×n = ( k=1 aik bkj )m×n

 
 b b12 b13 b14
a11 a12 a13  11

b21 b22 b23 b24 
a21 a22 a23
b31 b32 b33 b34
 
a11 b11 + a12 b21 + a13 b31 a11 b12 + a12 b22 + a13 b32 a11 b13 + a12 b23 + a13 b33 a11 b14 + a12 b24 + a13 b34
=  
a21 b11 + a22 b21 + a23 b31 a21 b12 + a22 b22 + a23 b32 a21 b13 + a22 b23 + a23 b33 a21 b14 + a22 b24 + a23 b34

That is, the (i, j)-entry of the product AB is the sum of the product of the entries in the i-th row of A with the j-th
column of B.
Example

   
  1 1 (1)(1) + (2)(2) + (3)(−1) (1)(1) + (2)(3) + (3)(−2)
1 2 3 
2 3 =  
4 5 6
−1 −2 (4)(1) + (5)(2) + (6)(−1) (4)(1) + (5)(3) + (6)(−2)
 
2 1
=
8 7
Question
▶ What is the (2, 3)-entry
  of the product
1 0 1 1  
−1 2 1 1 1 1 1
1 1
  1 2 −1 0 0
0 1 2 −1    ?

3 3 0 1 2 −1
1 0 1
0 0 0 1 1
2 −5 9 1

▶ What is the (2, 3)-entry of the product

  
1 2 3 1 0 5
?
−1 0 9 1 3 −2
Remark

1. For AB to be defined, the number of columns of A must agree with the number of rows of B. The resultant
matrix has the same number of rows as A, and the same number of columns as B.

(m × p)(p × n) = (m × n).

2. Matrix multiplication is not commutative,

     
0 1 1 2 1 2 0 1
̸=
1 0 3 4 3 4 1 0

3. The order of the product matter.

i. If we multiply A to the left of B, we are pre-multiplying A to B.

ii. If we multiply A to the right of B, we are post-multiplying A to B.

Properties of Matrix Multiplication

Theorem
(i) (Associative) (AB)C = A(BC).

(ii) (Left distributive law) A(B + C) = AB + AC.

(iii) (Right distributive law) (A + B)C = AC + BC.

(iv) (Commute with scalar multiplication) c(AB) = (cA)B = A(cB).

(v) (Multiplicative identity) For any m × n matrix A, Im A = A = AIn .

(vi) (Nonzero Zero divisor) There exists A ̸= 0m×p and B ̸= 0p×n such that AB = 0m×n .

(vii) (Zero matrix) For any m × n matrix A, A0n×p = 0m×p and 0p×m A = 0p×n
Remarks

▶ To not overcrowd the slide, we left out the sizes of the matrices in the theorem, assuming that the matrices
have the appropriate sizes for the operations to be well-defined. See the appendix for the detail statements.

▶ The proof of the theorem can be found in the appendix.

▶ By the associativity of matrix multiplication, if A1 , A2 , ..., Ak are k matrices of the right sizes such that their
product is well-defined, we write it as
A1 A2 · · · Ak ,
without the parenthesis (brackets).
Zero Divisors

Consider    
1 0 0 0
A= , B= .
0 0 0 1

Then     
1 0 0 0 0 0
AB = =
0 0 0 1 0 0
Powers of Square Matrices

Definition
Define the power of square matrices inductively as such.
(i) A0 = I,
(ii) An = AAn−1 , for n ≥ 1.

That is, An is A multiplied to itself n times, for n ≥ 2. It follows that An Am = An+m for positive integers m, n.
Example

 3         
1 1 1 1 1 1 1 1 2 2 1 1 4 4
= = =
1 1 1 1 1 1 1 1 2 2 1 1 4 4

Note that powers of a matrix can only be defined for square matrices.
Question

Is it true that for any square matrices A and B of order n, (A + B)2 = A2 + 2AB + B2 ?
Challenge

Show that for diagonal matrices A and B, (A + B)2 = A2 + 2AB + B2 .

Transpose

Let A = (aij ) be a m × n matrix. The transpose of A, denoted as AT , is the n × m matrix whose (i, j)-entry is the
(j, i)-entry of A, AT = (bij )n×m , bij = aji .
   
a11 a12 · · · a1n a11 a21 ··· am1
 a21 a22 · · · a2n  a12 a22 ··· am2 
A= . AT =  .
   
.. .. ..  .. .. .. 
 .. . . .   .. . . . 
am1 am2 · · · amn m×n a1n a2n ··· amn n×m
Examples

 
 T 1 4
1 2 3
1. = 2 5
4 5 5
3 5

 T
1

2. 1 = 1 1 0
0

 T  
1 2 1 2
3. =
2 0 2 0
Properties of Transpose

Theorem
(i) (AT )T = A.

(ii) (cA)T = cAT .

(iii) (A + B)T = AT + BT .

(iv) (AB)T = BT AT .

The proof is left as an exercise

The transpose provides an alternative definition of symmetric matrix. A square matrix A is symmetric if A = AT .
Example

T
Here is an example to demonstrate the last property (AB) = BT AT of the previous theorem.
  T  
  1 1  T     1 4
 1 2 3  2 1 2 8 1 2 −1 
2 3  = = = 2 5
4 5 6 8 7 1 7 1 3 −2
−1 −2 3 6
 T
1 1  T
2 1 2 3
= 3 
4 5 6
−1 −2

In fact, if A is a m × p matrix and B a p × n matrix, then AT is p × m and BT is n × p, and thus the product
AT BT is not well defined if m ̸= n.
Question

Is it true that if A and B are symmetric matrices of the same size, then so is A + B?
Challenge

Is it true that if A and B are symmetric matrices of the same order, then so is AB?
2.3 Linear System and Matrix Equation
Matrix Equation

A linear system in standard form



 a11 x1 + a12 x2 + ··· + a1n xn = b1
 a21 x1 + a22 x2 + ··· + a2n xn = b2

..


 .
am1 x1 + am2 x2 + ··· + amn xn = bm


Can be expressed as a matrix equation

    
a11 a12 ··· a1n x1 b1
 a21 a22 ··· a2n  x2   b2 
..   ..  =  .. , Ax = b
    
 .. ..
 . . .  .   . 
am1 am2 ··· amn xn bm

Here A = (aij )m×n is called the coefficient matrix, x = (xi )n×1 the variable vector, and b = (bi )m×1 the
constant vector.
Vector Equation

The linear system 


 a11 x1 + a12 x2 + ··· + a1n xn = b1
 a21 x1 + a22 x2 + ··· + a2n xn = b2

..


 .
am1 x1 + am2 x2 + ··· + amn xn = bm


can also be expressed as a vector equation:

       
a11 a12 a1n b1
 a21   a22   a2n   b2 
x1  .  + x2  .  + · · · + xn  .  =  . , x1 a1 + x2 a2 + · · · + xn an = b
       
 ..   ..   ..   .. 
am1 am2 amn bm

Here ai is called the coefficient vector for variable xi , for i = 1, ..., n.

Example

The linear system 

 3x + 2y − z = 1
x + 2y + z = 3
x + z = 2


can be written as a matrix equation     

3 2 −1 x 1
1 2 1  y  = 3
1 0 1 z 2
or a vector equation        
3 2 −1 1
x 1 + y 2 + z  1  = 3 .
1 0 1 2
Example

The
 (unique)
 solution to the system is x = 12 , y = 21 , z = 32 . We can express the solution as a 3 by 1 matrix,
1/2
1/2. Check that
3/2
    
3 2 −1 1/2 1
1 2 1  1/2 = 3
1 0 1 3/2 2
and        
3 2 −1 1
1  1  3   
1 + 2 + 1 = 3 .
2 2 2
1 0 1 2
Properties of Homogeneous Linear System

Recall that a linear system is homogeneous if it has the following corresponding matrix equation

Ax = 0,

for some m × n matrix A, a variable n-vector x, and the m × 1 zero matrix (or zero m-vector) 0 = 0m×1 .

Theorem
A homogeneous linear system is always consistent.

Proof.
Write the homogeneous linear system as Ax = 0 for some m × n matrix A. Then

A0 = 0,

that is, the m × 1 zero matrix is a solution to the system.

Properties of Homogeneous Linear System

Definition
The zero solution is called the trivial solution. If x ̸= 0 is a nonzero solution to the homogeneous system, it is called
a nontrivial solution.

Theorem
A homogeneous linear system has infinitely many solutions if and only if it has a nontrivial solution.

Proof.
(⇒) If the system has infinitely many solutions, it must surely have a nontrivial solution.

(⇐) Suppose now u ̸= 0 is a nontrivial solution to the homogeneous system Ax = 0, that is, Au = 0. Now for any
real number s ∈ R, since scalar multiplication commutes with matrix multiplication,

A(su) = s(Au) = s0 = 0,

that is, su is a solution to the homogeneous linear system too. Hence, the system has infinitely many solutions.
Example

Consider the following homogeneous linear system

    
0 1 x 0
=
0 1 y 0
 
1
See that is a nontrivial solution to the system,
0
    
0 1 1 0
= .
0 1 0 0
   
1 s
Now for any real number s ∈ R, s = is a solution to the system too,
0 0
    
0 1 s 0
= .
0 1 0 0
Question

Show that if the trivial solution is a solution to the linear system, then it must be a homogeneous linear system.
Question

Which of the following statements is/are true?

1. If the homogeneous system has a unique solution, it must be the trivial solution.

2. If the homogeneous system has the trivial solution, it must be the unique solution.
Solutions to Homogeneous and Non-homogeneous Linear System
Lemma
Let v be a particular solution Ax = b, and u be a particular solution to the homogeneous system Ax = 0 with the
same coefficient matrix A. Then v + u is also a solution to Ax = b.

Proof.
By hypothesis, Av = b and Au = 0. Hence, by the distribution properties of matrix multiplication,

A(v + u) = Av + Au = b + 0 = b.

Lemma
Suppose v1 and v2 are solutions to the linear system Ax = b. Then v1 − v2 is a solution to the homogeneous linear
system Ax = 0 with the same coefficient matrix.

Proof.
By hypothesis, Av1 = b, Av2 = b. Hence,

A(v1 − v2 ) = Av1 − Av2 = b − b = 0.

Challenge

Let v be a particular solution to a non-homogeneous system Ax = b, b ̸= 0.

Show that
v + s1 u1 + s2 u2 + · · · + sk uk , s1 , s2 , ..., sk ∈ R
is the general solution to Ax = b if and only if

s1 u1 + s2 u2 + · · · + sk uk , s1 , s2 , ..., sk ∈ R

is the general solution to the homogeneous linear system Ax = 0.

Introduction to Submatrices
Let A be an m × n matrix,  
a11 a12 ··· a1n
 a21 a22 ··· a2n 
A= . ..  .
 
.. ..
 .. . . . 
am1 am2 ··· amn
Rows of A:

r1 = a11 a12 ··· a1n

r2 = a21 a22 ··· a2n
..
.

rm = am1 am2 ··· amn
Columns of A:      
a11 a12 a1n
 a21   a22   a2n 
c1 = 
···
 c2 = 
 · · ·  ... cn =  · · · 
  

am1 am2 amn

The rows and columns of a matrix are examples of submatrices of a matrix.
Submatrix

Definition
A p × q submatrix of an m × n matrix A, p ≤ m, q ≤ n, is formed by taking a p × q block of the entries of the
matrix A.

Example
Consider the matrix  
1 3 5 7 1
A = 2 4 6 1 2 .
1 2 2 1 3

(i) Each row is a 1 by 5 submatrix, each column is a 3 by 1 submatrix.

 
2 4 6
(ii) is a 2 by 3 submatrix obtained by taking rows 2 and 3, and columns 1 to 3.
1 2 2
 
5 7
(iii) 6 1 is a 3 by 2 submatrix obtained by taking rows 1 to 3, and columns 3 and 4.
2 1
Block Multiplication

Theorem
Let A be an m × p matrix and B a p × n matrix. Let A1 be a (m2 − m1 + 1) × p submatrix of A obtained by taking
rows m1 to m2 , and b1 a p × (n2 − n1 + 1) submatrix of B obtained by taking columns n1 to n2 . Then the product
A1 B1 is a (m2 − m1 + 1) × (n2 − n1 + 1) submatrix of AB obtained by taking rows m1 to m2 and columns n1 to n2 .

The proof is left as an exercise. We call this block multiplication.

Example  
1 0 1  
1 1 3 5 7 1
2 2
 and B = 2
Let A = 
3 4 6 1 2. Then
2 1
1 2 2 1 3
4 4 2
   
1 0 1   2 5 7 8 4
1 1 3 5 7 1
2 2
 2
 7 15 21 11 11
AB = 3 4 6 1 2 = 
 8 19 29 24 10 .

2 1
1 2 2 1 3
4 4 2 14 32 48 34 18
 
  5 7
1 2 2
(i) A1 = is obtained from A by taking rows 2 and 3, and B1 = 6 1 is obtained from B by taking
3 1 2
  2 1
21 11
columns 3 and 4. Then A1 B1 = is a submatrix of AB by taking rows 2 and 3, columns 3 and 4.
29 24
 
  3
3 2 1
(ii) A1 = is obtained from A by taking rows 3 and 4, and B1 = 4 is column 2 of B. Then
4 4 2
2
 
19
A1 B1 = is a submatrix of AB by taking rows 3 and 4, and column 2.
32
Question
Given that    
1 0 1   2 5 7 8 4
1 1 3 5 7 1
2 2 2
7 15 21 11 11

3 4 6 1 2 =  .
2 1 8 19 29 24 10
1 2 2 1 3
4 4 2 14 32 48 34 18
 
1 0 1  
1 3 5
2 2
 4
1. What is 
3 6?
2 1
2 2
4 4 2

 

1 3 5 7 1
2. What is 4 4 2 2 4 6 1 2?
1 2 2 1 3
Block Multiplication

In particular, let bi be the j-th column of B. Then

AB = A b1 b2 ··· bn = Ab1 Ab2 ··· Abn .

That is, the j-th column of the product AB is the product of A with the j-th column of B.

Also, if ai is the i-th row of A, then 

  
a1 a1 B
 a2   a2 B 
AB =  .  B =  .  .
   
 ..   .. 
am am B
That is, the i-th row of the product AB is the product of the i-th row of A with B.
Solving Matrix Equations

 
3 2 −1
Let A = 5 −1 3 . Find a 3 × 3 matrix X such that
2 1 −1
    
3 2 −1 x1 x2 x3 1 2 1
AX = 5 −1 3  y1 y2 y3  = 2 1 1 .
2 1 −1 z1 z2 z3 3 1 0

By block multiplication, we are solving for the 3 linear systems

           
x1 1 x2 2 x3 1
A y1  = 2 , A y2  = 1 , A y3  = 1 .
z1 3 z2 1 z3 0
Example

Solve the first linear system,

     
3 2 −1 1 R2 − 53 R1
3 2 −1 1 1
R3 − 13 R2 − 13 R
3 2 −1 1
9 3
 5 −1 3 2  −−−−−→  0 −13/3 14/3 1/3  −−−−−→−−− −→  0 1 −14/13 −1/13 
R3 − 23 R1 3
− 13 R2
2 1 −1 3 0 −1/3 −1/3 7/3 0 0 1 −10/3
   
14
R2 + 13 R3
3 2 0 −7/3 1 1 0 0 5/3
R −2R2 3 R1
−−−−−→  0 1 0 −11/3  −−1−−−→ −−→  0 1 0 −11/3 
R1 +R3
0 0 1 −10/3 0 0 1 −10/3
   
x1 5
1 
So, y1  = 3
−11.
z1 10
Example

Solve the second linear system,

     
3 2 −1 2 R2 − 53 R1
3 2 −1 2 1
R3 − 13 R2 − 13 R
3 2 −1 2
9 3
 5 −1 3 1  −−−−−→  0 −13/3 14/3 −7/3  −−−−−→−−− −→  0 1 −14/13 7/13 
R3 − 23 R1 3
− 13 R2
2 1 −1 1 0 −1/3 −1/3 −1/3 0 0 1 2/9
   
R2 + 14 R
3 2 0 20/9 1
R
1 0 0 2/9
13 3 R −2R2 3 1
−−−− −→  0 1 0 7/9  −−1−−−→ −−→  0 1 0 7/9 
R1 +R3
0 0 1 2/9 0 0 1 2/9
   
x2 2
So, y2  = 91 7. The row operations used here in the reduction process is exactly those in solving the first linear
z2 2
system. This is because they both have the same coefficient matrix.

We can conclude likewise that the exact same row operations could be used to solve the third linear system. Hence,
we could instead solve all 3 linear systems simultaneously!
Combining Augmented Matrices

Solve the following 3 linear systems

       
3x + 2y − z = a a 1 2 1
5x − y + 3z = b , for b  = 2 , 1 , 1 .
2x + y − z = c c 3 1 0

Matrix equation:     
3 2 −1 x1 x2 x3 1 2 1
5 −1 3  y1 y2 y3  = 2 1 1
2 1 −1 z1 z2 z3 3 1 0
Augmented matrix:  
3 2 −1 1 2 1
 5 −1 3 2 1 1 
2 1 −1 3 1 0
Solve the 3 linear system simultaneously.
Example
   
3 2 −1 1 2 1 R2 − 53 R1
3 2 −1 1 2 1 1
R3 − 13 R2 − 13
9 R3
 5 −1 3 2 1 1  −−−−−→  0 −13/3 14/3 1/3 −7/3 −2/3  −− −−−→−−− −→
2 3
R3 − 3 R1 − 13 R2
2 1 −1 3 1 0 0 −1/3 −1/3 7/3 −1/3 −2/3
   
3 2 −1 1 2 1 R2 + 14 R
3 2 0 −7/3 20/9 17/9
13 3
 0 1 −14/13 −1/13 7/13 2/13  −−−− −→  0 1 0 −11/3 7/9 10/9 
R1 +R3
0 0 1 −10/3 2/9 8/9 0 0 1 −10/3 2/9 8/9
 
1
R
1 0 0 5/3 2/9 −1/9
R −2R2 3 1
−−1−−−→ −−→  0 1 0 −11/3 7/9 10/9 
0 0 1 −10/3 2/9 8/9
           
x1 5 x2 2 x3 −1
1 1
y1  = −11 , y2  = 7 , y3  =  10  . 1
3 9 9
z1 −10 z2 2 z3 8
That is,  
5/3 2/9 −1/9
X = −11/3 7/9 10/9 
−10/3 2/9 8/9
Combining Augmented Matrices

In general: p linear systems with the same coefficient matrix A = (aij )m×n , for k = 1, ..., p,

 a11 x1 + a12 x2 + · · · + a1n xn = b1k

 a21 x1 + a22 x2 + · · · + a2n xn = b2k

..


 .
am1 x1 + am2 x2 + · · · + amn xn = bmk


Combined augmented matrix:

 
a11 a12 ··· a1n b11 b12 b1p

 a21 a22 ··· a2n b21 b22 b2p 

 .. .. .. .. .. .. .. 
 . . . . . . ··· . 
am1 am2 ··· amn bm1 bm2 bmp
2.4 Inverse of Matrices
Introduction

1. Suppose ab = ac for some a, b, c ∈ R with a ̸= 0, then b = c.

1
2. This is because if a ̸= 0, a exists, and a 1a = 1.

3. Hence, multiplying both sides of ab = ac with 1a , we have b = 1 × b = 1a ab = 1a ac = 1 × c = c.

4. This is used to solve ax = b for some a, b ∈ R. We can conclude that x = ba .

5. Ideally, we want to apply the same idea to solve a linear system,

b
Ax = b ⇒ x= .
A

1
6. Idea: Try to define A.
Introduction

1 1
1. To define A, we need the matrix equivalent of 1, since a is defined such that 1a a = 1.

2. 1 is the multiplicative identity of the real numbers, 1 × a = a for any a ∈ R.

3. The matrix multiplicative identity is the identity matrix, In ,

IA = A = AI

for any matrix A.

1 1
4. So, define A as the matrix such that AA = I.
Problem 1

For matrices A, B, C with the appropriate sizes, if AB = AC and A ̸= 0 is not the zero matrix, can we conclude that
B = C?

No. For example,        

0 1 1 1 3 4 0 1 2 5
= = ,
0 2 3 4 6 8 0 2 3 4
   
1 1 2 5
but ̸= .
3 4 3 4

Here although A is not the zero matrix, we do not have the cancellation law.
Problem 2

1 1
Recall that matrix multiplication is not commutative, AB ̸= BA. Suppose a matrix A exists such that AA = I. Is it
true that A A1 = I?

No. For example,

     
  1 0   1 0   1 0 0
1 0 0  1 0 1 0 0
0 1 =
 = I, but 0 1 = 0 1 0 ̸= I
0 1 0 0 1 0 1 0
0 0 0 0 0 0 0
Conclusion

▶ So, it seems like our dream of solving a linear system

b
Ax = b ⇒ x=
A
is IMPOSSIBLE!

▶ Except, it is possible in some cases.

▶ Restrict our attention only to square matrices.

Inverse of Square Matrices

Definition
A n × n square matrix A is invertible if there exists a matrix B such that

AB = In = BA.

A matrix is said to be non-invertible otherwise.

Example

       
1 1 1 1 0 −1 1 0 0 1 0 −1 1 1 1
1 2 1 −1 1 0  = 0 1 0 = −1 1 0  1 2 1
0 1 1 1 −1 1 0 0 1 1 −1 1 0 1 1
Remarks

1. By definition, for a square matrix A to be invertible, there must be a B such that BA = In AND AB = In
simultaneously.

2. This necessitates B to also be a n × n square matrix.

3. Only square matrices are invertible. If A is a n × m matrix with n ̸= m, then for AB = In , B must be of size
m × n, and hence BA ̸= In .

4. Hence, all non-square matrices are non-invertible. In fact, we will see later that if A is a non-square matrix such
that BA = I (or AB = I, respectively) for some matrix B, then there exists no matrix C such that AC = I (or
CA = I, respectively).
Uniqueness of inverse

Theorem
If B and C are both inverses of a square matrix A, then B = C.

Proof.
Suppose B and C are such that AB = I = BA and AC = I = CA. Then by associativity of matrix multiplication,

B = BI = B(AC) = (BA)C = IC = C.

So since the inverse is unique, we can denote the inverse of an invertible matrix A by A−1 and call it the inverse of
A.

Question: Is the identity matrix I invertible? If it is, what is it’s inverse?

Non-Invertible Square Matrix

 
0 0
Not all matrices are invertible. For example, consider the matrix .
1 1
        
a b 0 0 a b 0 0 1 0
It is not invertible since for any order 2 square matrix , = ̸= .
c d 1 1 c d a+c b+d 0 1

Definition
A non-invertible square matrix is called a singular matrix.

Remark
Some textbook use the term singular interchangeably with non-invertible, that is, they do not insist that singular
matrices are square matrices.
Inverse of 2 by 2 Square Matrices

Theorem  
a b
A 2 × 2 square matrix A = is invertible if and only if ad − bc ̸= 0. In this case, the inverse is given by
c d
 
1 d −b
A−1 = .
ad − bc −c a

We will proof   A is invertible if and only if ad − bc ̸= 0 later. The verification that it’s inverse is
the claim that
d −b
A−1 = ad−bc1
is left as an exercise for the readers. The formula above is known as the adjoint formula
−c a
for inverse.
Cancellation law for matrices

Theorem
Let A be an invertible matrix of order n.
(i) (Left cancellation) If B and C are n × m matrices with AB = AC, then B = C.

(ii) (Right cancellation) If B and C are m × n matrices with BA = CA, then B = C.

Proof.
(i)
AB = AC ⇒ B = A−1 AB = A−1 AC = C.

(ii)
BA = CA ⇒ B = BAA−1 = CAA−1 = C.

Caution
If AB = CA, we cannot conclude that B = C.
Question

Suppose A is invertible. Can we conclude that the system Ax = b is consistent? If so, what is a solution, and is it
unique?
Invertibility and Linear System

Theorem
Suppose A is an n × n invertible square matrix. Then for any n × 1 vector b, Ax = b has a unique solution.

Proof.
There are two claims in the theorem; it claims that the system Ax = b is consistent, and that the solution is unique.
Firstly, we will check that u = A−1 b is a solution. Indeed,

Au = A(A−1 b) = b.

Now suppose v is also a solution, that is, Av = b. Then

Av = b = Au.

By the cancellation law, we have v = u = A−1 b. Hence, A−1 b is the unique solution to the system.
Invertibility and Linear System

Corollary
Suppose A is invertible. Then the trivial solution is the only solution to the homogeneous system Ax = 0.

Proof.
This follows immediately from the previous theorem by letting b = 0.
Example
Consider that linear system     
1 0 1 x1 2
1 1 0 x2  = 4 .
0 1 1 x3 6
   
1 0 1 0.5 0.5 −0.5
Let A = 1 1 0. Readers should check that A−1 = −0.5 0.5 0.5 . Solving the system, we have
0 1 1 0.5 −0.5 0.5
   
1 0 1 2 1 0 0 0
RREF
 1 1 0 4  −−−→  0 1 0 4 .
0 1 1 6 0 0 1 2
 
0
The unique solution is 4, which is exactly equal to A−1 b,
2
      
2 0.5 0.5 −0.5 2 0
A−1 4 = −0.5 0.5 0.5  4 = 4 .
6 0.5 −0.5 0.5 6 2
Exercise

Verify that the homogeneous system     

1 0 1 x1 0
1 1 0 x2  = 0
0 1 1 x3 0
has only the trivial solution.
Remark

1. The theorem and its corollary on invertibility and linear system are actually equivalent statements. That is,
(i) a square matrix A is invertible if and only if Ax = b has a unique solution for all b, and

(ii) a square matrix A is invertible if and only if the homogeneous system Ax = 0 has only the trivial solution.

2. In fact, one can observe that

Ax = b has

a unique solution if and only if the reduced row-echelon form of the
augmented matrix A | b is I | A−1 b .

3. This also hint towards the fact that A is invertible if and only if the reduced row-echelon form of A is the
identity matrix.

4. The proofs will be given in the next section.

Algorithm to Computing Inverse

Suppose A is an invertible n × n matrix. By uniqueness of the inverse, there must be a unique solution to

AX = I.

By block multiplication, we are solving the augmented matrix

RREF
A | I −−−→ I | A−1 .


Example
 
3 5
1. Find the inverse of .
1 2
    
3 5 x11 x12 1 0
Solve = .
1 2 x21 x22 0 1
     
3 5 1 0 R1 −3R2 R2 +2R1 0 −1 1 −3 −R1 R1 ↔R2 1 0 2 −5
−−−−−→−−−−→ −−→−−−−→ .
1 2 0 1 1 0 2 −5 0 1 −1 3
 −1  
3 5 2 −5
So, = . Indeed, this is exactly the answer obtained if we used
1 2 −1 3
   
1 d −b 1 2 −5
= .
ad − bc −c a (3)(2) − (5)(1) −1 3
Example

 
2 7 1
2. Find the inverse of 1 4 −1.
1 3 0
   
2 7 1 1 0 0 1 0 0 −3/2 −3/2 11/2
RREF
 1 4 −1 0 1 0  −−−→  0 1 0 1/2 1/2 −3/2  .
1 3 0 0 0 1 0 0 1 1/2 −1/2 −1/2
 −1  
2 7 1 −3 −3 11
1 
So, 1 4 −1 = 2
1 1 −3.
1 3 0 1 −1 −1
Question

 
1 1 1
Consider the matrix A = 1 0 0. Following the steps above, we have
0 1 1
   
1 1 1 1 0 0 1 0 0 0 1 0
RREF
 1 0 0 0 1 0  −−−→  0 1 1 0 0 1 .
0 1 1 0 0 1 0 0 0 1 −1 −1

Is A invertible? If it is, what is its inverse?

Remarks

1. In the algorithm to finding the inverse, we are solving to AX = I. Technically, we are solving for a right inverse.

2. Can we guarantee that the solution is also a left inverse? That is, if AB = I, can we be sure that BA = I too?
Properties of Inverses

Theorem
Let A be an invertible matrix of order n.
(i) (A−1 )−1 = A.

(ii) For any nonzero real number a ∈ R, (aA) is invertible with inverse (aA)−1 = 1a A−1 .

(iii) AT is invertible with inverse (AT )−1 = (A−1 )T .

(iv) If B is an invertible matrix of order n, then (AB) is invertible with inverse (AB)−1 = B−1 A−1 .

Partial Proof.
The proof of (i) to (iii) is left as a exercise. For (iv),
−1
(B−1 A−1 )(AB) = B−1 (A−1 A)B = B−1 B = I = AA−1 = A(BB )A−1 = (AB)(B−1 A−1 ).
Remark

1. The precise statement for (iii) is that as follows.

A square matrix A is invertible if and only if its transpose AT is. In this case, the inverse of its transpose is the
transpose of its inverse, (A−1 )T .

2. Caution: (AB)−1 = B−1 A−1 ̸= A−1 B−1 .

3. If A1 , A2 , ..., Ak are invertible, the product A1 As · · · Ak is invertible with (A1 A2 · · · Ak )−1 = A−1 −1 −1
k · · · A2 A1 if
Ai is an invertible matrix for i = 1, ..., k.

4. The negative power of an invertible matrix is defined to be

A−n = (A−1 )n

for any n > 0.

Question

Statement (iv) of the theorem on properties of inverse says that the product of invertible matrices is invertible. Is the
converse true? That is, if the product of two square matrices AB is invertible, can we conclude that both A and B
are invertible? Why?
Question

Is the inverse of an invertible symmetric matrix symmetric?

2.5 Elementary Matrices
Elementary Matrices

Definition
A square matrix E of order n is called an elementary matrix if it can be obtained from the identity matrix In by
performing a single elementary row operation
r
In →
− E,
where r is an elementary row operation. The elementary row operation is said to be the row operation corresponding
to the elementary matrix.

Example    
1 0 0 0 0 0 1 0
R2 +3R4 0 1 0 3 R1 ↔R3 0 1 0 0
1. I4 −−−−→ E = 0
 2. I4 −−−−→ E =  .
0 1 0 1 0 0 0
0 0 0 1 0 0 0 1
 
1 0 0 0
3R2 0 3 0 0
3. I4 −−→ E=0 0
.
1 0
0 0 0 1
Elementary Matrices and Elementary Row Operations

Let A be an n × m matrix and let E be the n × n elementary matrix corresponding to the elementary row operation
r . Then the product EA is the resultant of performing the row operation r on A,
r
A→
− EA.

That is, performing elementary row operations is equivalent to premultiplying by the corresponding elementary
matrix.
Example
   
1 1 1 0 1 1 1 0
R2 +2R1
1. 1
 2 3 −1 −−−−→ 3 4 5 −1
2 1 4 
2 2 1 4 2
 
1 0 0 1 1 1 0 1 1 1 0
2 1 0 1 2 3 −1 = 3 4 5 −1
0 0 1 2 1 4 2 2 1 4 2
Example
   
1 1 1 0 1 1 1 0
R2 ↔R3
2. 1 2 3 −1 −−−−→ 2 1 4 2 
2 1 4 
2 1 2 3 −1 
1 0 0 1 1 1 0 1 1 1 0
0 0 1 1 2 3 −1 = 2 1 4 2
0 1 0 2 1 4 2 1 2 3 −1

   
1 1 1 0 1 1 1 0
−2R2
3. 1 2 3 −1 −−−→ −2 −4 −6 2
2 1 4 2 2 1 4 2
    
1 0 0 1 1 1 0 1 1 1 0
0 −2 0 1 2 3 −1 = −2 −4 −6 2
0 0 1 2 1 4 2 2 1 4 2
Row Equivalent Matrices

Recall that matrices A and B are said to be row equivalent if B can be obtained from A by performing a series of
elementary row operations.

Suppose now B is row equivalent to A,

1r 2 r k r
A−
→ −
→ ··· −
→ B.
Let Ei be the elementary matrix corresponding to the row operation ri , for i = 1, 2, ..., k. Then

B = Ek · · · E2 E1 A.

This follows from the previous discussion,

1r 2 r 3 r k r
A−
→ E1 A −
→ E2 E1 A −
→ ··· −
→ Ek · · · E2 E1 A = B.
Example

   
1 1 1 0 1 1 1 0
R2 +2R1 R2 ↔R3
1 2 3 −1 −−−−→−−−−→ 2 1 4 2
2 1 4 2 3 4 5 −1

        
1 0 0 1 0 0 1 1 1 0 1 0 0 1 1 1 0 1 1 1 0
0 0 1 2 1 0 1 2 3 −1 = 0 0 1 3 4 5 −1 = 2 1 4 2
0 1 0 0 0 1 2 1 4 2 0 1 0 2 1 4 2 3 4 5 −1
Question

Consider the following    

1 0 0 1 0 0
R2 +3R1
I3 = 0 1 0 −− −−→ 3 1 0 = B
0 0 1 0 0 1
Give a single row operation that reduces B to the identity matrix I3 ,
   
1 0 0 1 0 0
???
B = 3 1 0 −−→ 0 1 0
0 0 1 0 0 1
Question

Consider the following    

1 0 0 1 0 0
R2 +3R1
I3 = 0 1 0 −− −−→ 3 1 0 = B
0 0 1 0 0 1
Give a single row operation that reduces B to the identity matrix I3 ,
   
1 0 0 1 0 0
R2 −3R1
B = 3 1 0 −−−−−→ 0 1 0
0 0 1 0 0 1
Inverse of Elementary Matrices

Recall that every elementary row operations has an inverse, and that performing a row operation is equivalent to
premultiplying by the corresponding elementary matrix.

Consider now a row operation r , with its corresponding elementary matrix, E. Let r ′ be the reverse of the row
r
operation r , and E′ its corresponding elementary matrix. By definition, I →
− E. Now if we apply r ′ to E, since it is
the reverse of r , we should get back the identity matrix,
r r′
I→
− E−
→ I.

Hence, we have
I = E′ EI = E′ E.
′
r r
→ E′ →
Similarly, I − − I tells us that
I = EE′ I = EE′ .
This shows that E is invertible with inverse E′ , which is also an elementary matrix.
Inverse of Elementary Matrices

Theorem
Every elementary matrices E are invertible. The inverse E−1 is the elementary matrix corresponding to the reverse of
the row operation corresponding to E.
(i)
Ri +cRj Ri −cRj
In −−−−→ E −−−−→ In ⇒ E : Ri + cRj , E−1 : Ri − cRj .
(ii)
Ri ↔Rj Ri ↔Rj
In −−−−→ E −−−−→ In ⇒ E : Ri ↔ Rj , E−1 : Ri ↔ Rj .
(iii)
cR
1
Ri 1
i
In −−→ E −c−→ In ⇒ E : cRi , E−1 : Ri .
c
Example
       −1  
1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0
0 1 0 3 0 1 0 −3 0 1 0 0 0 1 0 3 0 1 0 −3
    =   ⇒   = 
1. 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0
0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1
R +3R
2 4 R −3R
−− −−→ −−2−−−→
4

       −1  
0 0 1 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0
0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0
    =   ⇒   = 
2. 1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0
0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1
R ↔R
1 3 R ↔R
1 3
−− −−→ −− −−→
       −1  
1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0
0 2 0 0 0 1/2 0 0 0 1 0 0 0 2 0 0 = 0 1/2 0 0

    = 
  ⇒ 
 
3. 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0
0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1
1
2R2 R2
−−→ −2−→
Question

Find the inverse of this elementary matrix  

1 0 0 0
0 1 1/2 0
E=
0

0 1 0
0 0 0 1
2.6 Equivalent Statements for Invertibility
Introduction

This section will be mainly proving important equivalent statements of invertibility. That is, as long as any one of
these statements holds, we know that all the other statements will hold true too. The equivalent statements of
invertibility is like a junction, where knowing that any of the statements opens you up to all the other statements, of
which one of them might be useful in solving the problem you have at hand.

Before we get lost in a sea of theorems and proofs, we will be illustrating the statements with some examples.
Example  
1 −1 1
Consider the matrix A = −1 1 1. Reducing, we have
0 −1 1
   
1 −1 1 1
R
1 0 0
R −R3 R2 +R1 R3 +R2 2 3 R2 −R3
−1 1 1 −−1−−→ −−−−→−−−−→−−→−−−−→ 0 1 0 .
0 −1 1 0 0 1

Here we can conclude that the reduced row-echelon form of A is the identity matrix, and from the previous section,
       
1 0 0 1 0 0 1 0 0 1 0 0 1 0 −1 1 0 0
0 1 −1 0 1 0  0 1 0 1 1 0 0 1 0  A = 0 1 0 .
0 0 1 0 0 1/2 0 1 1 0 0 1 0 0 1 0 0 1

By taking the inverse of the elementary matrices, we get

     
1 0 1 1 0 0 1 0 0 1 0 0 1 0 0
A = 0 1 0 −1 1 0 0 1 0 0 1 0 0 1 1 .
0 0 1 0 0 1 0 −1 1 0 0 2 0 0 1

Observe that A is a product of elementary matrices!

Example

Next, let us evaluate the product of the elementary matrices in the reduction of A,
         
1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 −1 1 0 −1
0 1 0 = 0 1 −1 0 1 0  0 1 0 1 1 0 0 1 0  A = 1/2 1/2 −1 A.
0 0 1 0 0 1 0 0 1/2 0 1 1 0 0 1 0 0 1 1/2 1/2 0

Observe that if we multiply the matrix to the right of A, we do get the identity matrix too,
    
1 −1 1 1 0 −1 1 0 0
−1 1 1 1/2 1/2 −1 = 0 1 0 .
0 −1 1 1/2 1/2 0 0 0 1
 
1 0 −1
This shows that A−1 = 1/2 1/2 −1.
1/2 1/2 0
Example

 
b1
Now consider the linear system Ax = b, where b = b2 , for some b1 , b2 , b3 ∈ R. Let’s solve the equation by row
b3
reduction
   
1 −1 1 b1 1 1 0 0 b1 − b3
R1 −R3 R2 +R1 R3 +R2 2 R3 R2 −R3
 −1 1 1 b2  −−−−→−−−−→−−−−→−−→−−−−→  0 1 0 b1 /2 + b2 /2 − b3  .
0 −1 1 b3 0 0 1 b1 /2 + b2 /2

This shows that the system is not only consistent, but have a unique solution for every b. In fact,
    
1 0 −1 b1 b1 − b3
A−1 b = 1/2 1/2 −1 b2  = b1 /2 + b2 /2 − b3  .
1/2 1/2 0 b3 b1 /2 + b2 /2
Example
 
1 1 −1
Now consider A =  1 0 0 . Reducing, we have
−1 −1 1
   
1 1 −1 1 0 0
R3 +R1 R2 ↔R1 R2 −R1
1 0 0  −−−−→−−−−→−−−−→ 0 1 −1 .
−1 −1 1 0 0 0

The reduced row-echelon

 form of A has both a non-pivot column and a zero row. Now consider the linear system
b1
Ax = b, where b = b2 , for some b1 , b2 , b3 ∈ R. Let’s solve the equation by row reduction.
b3
   
1 1 −1 b1 1 0 0 b2
R3 +R1 R2 ↔R1 R2 −R1
 1 0 0 b2  −−−−→−−−−→−−−−→  0 1 −1 b1 − b2  .
−1 −1 1 b3 0 0 0 b1 + b3

If b1 + b3 ̸= 0, then the system is inconsistent. If b1 + b3 = 0, then the system has infinitely many solutions. Can A
be written as a product of elementary matrices? Can the reduced row-echelon form of A be the identity matrix? Is A
invertible?
Elementary Matrices and Inverse
Theorem
If A = Ek · · · E2 E1 is a product of elementary matrices, then A is invertible.

Proof.
This follows from the fact that the product of invertible matrices is invertible. Hence, if a square matrix A can be
written as a product of elementary matrices, then since elementary matrices are invertible, A invertible.

Corollary
If the reduce row-echelon form of A is the identity matrix, then A is invertible.

Proof.
By the hypothesis, A is row equivalent to I. This means that there are elementary matrices E1 , E2 , ..., Ek such that

Ek · · · E2 E1 A = I.

By premultiplying the inverse of the elementary matrices, we have

A = E−1 −1 −1 −1 −1 −1
1 E2 · · · Ek I = E1 E2 · · · Ek .

However, since the inverse of elementary matrices are elementary matrices, this shows that A is a product of
elementary matrices, and hence invertible.
Equivalent Statements for Invertibility

Theorem
A square matrix A is invertible if and only if the homogeneous system Ax = 0 has only the trivial solution.

Proof.
(⇒) We have shown that if A is invertible, then the homogeneous system Ax = 0 has only the trivial solution.

(⇐) Now suppose the homogeneous system Ax = 0 has only the trivial solution. This necessitates the reduced
row-echelon form of A to be the identity matrix. For otherwise, the reduced row-echelon form must have a non-pivot
column, which then implies that the homogeneous system Ax = 0 has infinitely many solutions, a contradiction.
Hence, by the previous corollary, A is invertible.
The proof also shows that if A is invertible, then its reduced row-echelon form is the identity matrix.
Theorem
A square matrix A is invertible if and only if it’s reduced row-echelon form is the identity matrix.

Theorem
A square matrix A is invertible if and only if it is a product of elementary matrices.
The proofs of theorems are left as exercises.
Left and Right Inverses

Definition
Let A be a n × m matrix.
(i) A m × n matrix B is said to be a left inverse of A if BA = Im , where Im is the m × m identity matrix.
(ii) A m × n matrix B is said to be a right inverse of A if AB = In , where In is the n × n identity matrix.

Observe that B is a left inverse of A if and only if A is a right inverse of B.

Example
     
  1 0     1 0 1 0
1 0 0  1 0 1 0 0
1. 0 1 =
 . is a left inverse of 0 1; 0 1 is a right inverse of
0 1 0 0 1 0 1 0
0 0 0 0 0 0
 
1 0 0
.
0 1 0

2. The inverse of a square matrix A is both a (the) left and right inverse, A−1 A = I = AA−1 .
Equivalent Statements for Invertibility

Theorem
A square matrix A is invertible if and only if it has a left inverse.

Proof.
(⇒) An inverse is a left inverse.

(⇐) Suppose now B is a left inverse of A, BA = I. Consider now the homogeneous system Ax = 0. Let u be a
solution, Au = 0. Then premultiplying by the left inverse B,

0 = B(0) = B(Au) = (BA)u = Iu = u

tells us that u = 0, that is, the homogeneous system has only the trivial solution. Hence, A is invertible.
Equivalent Statements for Invertibility

Theorem
A square matrix A is invertible if and only if it has a right inverse.

Proof.
Recall that A is invertible if and only if AT is. Let B be a right inverse of A, AB = I. Then by taking the transpose,
and observing that the identity matrix is symmetric, we have BT AT = I. This shows that BT is a left inverse of AT .
Therefore, by the previous theorem, AT is invertible, which thus proves that A is invertible.
Equivalent Statement for Invertibility

Theorem
A square matrix A is invertible if and only if Ax = b has a unique solution for all b.

Proof.
(⇒) We have shown that if A is invertible, then Ax = b has a unique solution for all b.

(⇐) Now suppose Ax = b is consistent for all b. In particular, Ax = ei is consistent, where ei is the i-th column of
the identity matrix, for
i = 1, ..., n, where n is the order of A. Let bi be a solution to Ax = ei . Let
B = b1 b2 · · · bn . Then, by block multiplication




AB = A b1 b2 ··· bn = Ab1 Ab2 ··· Abn = e1 e2 ··· en = I,

which shows that B is a right inverse of A. Thus, by the previous theorem, A is invertible.
Algorithm for Finding Inverse

Thus, we will now formally introduce the algorithm to testing if a matrix is invertible, and finding its inverse it it is
invertible.

Let A be a n × n matrix.


Step 1: Form the n × 2n (augmented) matrix A In .

Step 2: Reduce the matrix A I −→ R B to its REF or RREF.

Step 3: If RREF R ̸= I or REF has a zero row, then A is not invertible. If RREF R = I or REF has no zero row,
A is invertible with inverse A−1 = B.
Question

Let A and B be n × m matrices. Show that A and B are row equivalent if and only if B = PA for some invertible
n × n matrix P.
Equivalent Statements of Invertibility

Theorem (Equivalent statements of invertibility)

Let A be a square matrix of order n. The following statements are equivalent.
(i) A is invertible.

(ii) AT is invertible.

(iii) (left inverse) There is a matrix B such that BA = I.

(iv) (right inverse) There is a matrix B such that AB = I.

(v) The reduced row-echelon form of A is the identity matrix.

(vi) A can be expressed as a product of elementary matrices.

(vii) The homogeneous system Ax = 0 has only the trivial solution.

(viii) For any b, the system Ax = b has a unique solution.

2.7 LU Factorization
Example
 
1 2 1 −1 0
Consider the matrix A = 1 0 1 −1 0. Reducing to a row-echelon form,
3 1 0 0 3
 
1
R3 − 25 R2
2 1 −1 0
R −R1 R3 −3R1
A −−2−−→ −−−−−→−−−−−→ 0 −2 0 0 0 .
0 0 −3 3 3

This means that

     
1 0 0 1 0 0 1 0 0 1 2 1 −1 0
0 1 0  0 1 0 −1 1 0 A = 0 −2 0 0 0 ,
0 −5/2 1 −3 0 1 0 0 1 0 0 −3 3 3

and thus
       
1 0 0 1 0 0 1 0 0 1 2 1 −1 0 1 0 0 1 2 1 −1 0
A = 1 1 0 0 1 0 0 1 0  0 −2 0 0 0 = 1 1 0 0 −2 0 0 0 .
0 0 1 3 0 1 0 5/2 1 0 0 −3 3 3 3 5/2 1 0 0 −3 3 3
Example

In the example above, we may write A as a product of a lower triangular matrix and a row-echelon of A,
  
1 0 0 1 2 1 −1 0
A = 1 1 0 0 −2 0 0 0 .
3 5/2 1 0 0 −3 3 3

Observe furthermore that the diagonal entries of the lower triangular matrix are 1. Such matrices are known as
unit lower triangular matrices. We will write it as A = LU, where L is a unit lower triangular matrix, and U is a
row-echelon form of A.
Example
 
1
Consider now the linear system Ax = 2. Replacing A with LU, we have LUx = b. Let Ux = y, and we first solve
1
for Ly = b. But since L is a unit lower triangular matrix, this is easy. From
 
1 0 0 1
 1 1 0 2 
3 5/2 1 1

We can observe that y1 = 1, y2 = 1, y3 = −9/2 is the unique solution. Next, we solve for Ux = y. Now since U is in
row-echelon form, this is easy too,
   
1 2 1 −1 0 1 1
− R2
1 2 1 −1 0 1
 0 −2 0 0 0 1  −−2−→  0 1 0 0 0 −1/2 
− 13 R3
0 0 −3 3 3 −9/2 0 0 1 −1 −1 3/2
1
tells us that x1 = 2 − t, x2 = − 12 , x3 = 3
2 + s + t, x4 = s, x5 = t, s, t, ∈ R is the general solution.
LU Factorization

Definition
A square matrix L is a unit lower triangular matrix if L is a lower triangular matrix with 1 in the diagonal entries.

An LU factorization of a m × n matrix A is the decomposition

A = LU,

where L is a unit lower triangular matrix, and U is a row-echelon form of A.

If such LU factorization exits for A, we say that A is LU factorizable.

Unit Lower Triangular Matrices
Lemma
Let A and B be unit lower triangular matrices of the same size. Then AB is a unit lower triangular matrix too.

Proof.    
1 0 ··· 0 1 0 ··· 0
a21 1 ··· 0 b21 1 ··· 0
Write A =  . ..  and B =  .. .. . Then
   
.. .. .. ..
 .. . . .   . . . .
an1 an2 ··· 1 bn1 bn2 ··· 1
1 0 ··· 0 1 0 ··· 0
  
a21 1 ··· 0 b21 1 ··· 0
AB =  . .. ..   .. .. .. 
  
 .. .. ..
. . .  . . . .
an1 an2 ··· 1 bn1 bn2 ··· 1
1 0 ··· 0
 
 a21 + b21 1 ··· 0
= .. .. .. 
 
 ..
 . . . .
an1 + an2 b21 + · · · + bn1 an2 + an3 bb2 + · · · + bn2 ··· 1
Unit Lower Triangular Matrix

(i) Observe that the elementary matrix E corresponding to the operation Ri + cRj for i > j for some real number c
is a lower triangular matrix.

(ii) Also, since the inverse of an elementary matrix is an elementary matrix corresponding to an elementary row
operation same type, E−1 is also a unit lower triangular matrix.

(iii) Hence, by the previous lemma, if E1 , E2 , ..., Ek are series of elementary matrices corresponding to row operations
of the type Ri + cRj for i > j for some c, then E−1 −1 −1
1 E2 · · · Ek is a unit lower triangular matrix.
Algorithm to LU Factorization
r1 ,r2 ,...,rk
Suppose A −−−−−−→ U, where each row operation rl is of the form Ri + cRj for some i > j and real number c, and
U is an row-echelon form of A. Let Ei be the elementary matrix corresponding for ri , for i = 1, 2, ..., k. Then

Ek · · · E2 E1 A = U ⇒ A = E−1 −1 −1
1 E2 · · · Ek U = LU,

where L = E−1 −1 −1
1 E2 · · · Ek . Then
  
1 0 ··· 0 ∗ ··· ∗
∗ 1 ··· 0  0 ··· 0 ∗ ··· ∗ 
A = LU =  .
  
.. .. ..   .. .. 
 .. . . .  . . 
∗ ∗ ··· 1 0 ··· ··· ∗

is a LU factorization of A.

In this case, we could obtain L quickly without computing E−1 −1 −1

1 E2 · · · Ek . For each row operation rl = Ri + cl Rj for
some i > j and real number cl , we will put −cl in the (i, j)-entry of L.
Question

1. What if we use other row operations that are not of the type Ri + cRj for some i > j and real number c? Is L
still a unit lower triangular matrix?

2. Is it possible to reduce any matrix A to a row-echelon form with only the type of row operations mentioned
above?
Example
     
2 4 1 5 −2 2 4 1 5 −2 2 4 1 5 −2
−4 −5 3 −8 1 −3 2 −3
 −−−2−−−1−→ 0
R +2R 3 5 2 R +3R 2 0 3 5
 
A=  −−3−−−→ 
2 −5 −4 1 8  R3 − R1 0 −9 −5 −4 10  R4 −4R2 0 0 10 2 1
−6 0 7 −3 1 R4 + 3R1 0 12 10 12 −5 0 0 −10 4 7
 
2 4 1 5 −2
R4 +R3  0 3 5 2 −3
−−−−→  0

0 10 2 1
0 0 0 6 8
 
1 0 0 0
−2 1 0 0
So, L =   and thus
1 −3 1 0
−3 4 −1 1
  
1 0 0 0 2 4 1 5 −2
−2 1 0 0 0 3 5 2 −3
A=  .
1 −3 1 0 0 0 10 2 1
−3 4 −1 1 0 0 0 6 8
Example

Solve the system  

 x
−2  1 
  
2 4 1 5 6
−4 −5 3 −8 1  x2   1 
   x3  =   .
2 −5 −4 1 8   −4
 x4 
−6 0 7 −3 1 7
x5
     
1 0 0 0 6 y1 6
 −2 1 0 0 1  . y2  = 13 is the unique solution. Now solve for Ux = y,
   
First solve Ly = b, 

1 −3 1 0 −4  y3  29
−3 4 −1 1 7 y4 2
     
2 4 1 5 −2 6 x1 71 + 37s
 0
 3 5 2 −3 13 . The general solution is x2  = 1 −22 + 58s , s ∈ R.
    
 0 0 10 2 1 29  x3  36  102 + 6s 
0 0 0 6 8 2 x4 12 − 48s
Questions

Let A = LU be a LU factorization of A.
1. Show that the system Ly = b has a unique solution for any b.

2. Is every matrix LU factorizable? If not, provide a counter-example.

Remarks

1. The question on the characterization of whether a matrix is LU factorizable is beyond the scope of this course.

2. Any matrix admits a LU factorization with pivoting (LUP factorization), that is, any matrix A can be written as
A = PLU, where L is a unit lower triangular matrix, U is a row-echelon form of A, and P is a permutation
matrix (see below for definition).

Definition
A n × n matrix P is a permutation matrix if every rows and columns has a 1 in only one entry, and 0 everywhere else.
Equivalently, P is a permutation matrix if and only if P is the product of elementary matrices corresponding to row
swaps.
2.8 Determinant by Cofactor Expansion
Order 1 and 2 Square Matrices

We will define the determinant of A of order n, denoted as det(A), or |A|, by induction.

1. For n = 1, A = (a), det(A) = a.
 
a b
2. For n = 2, A = , det(A) = ad − bc.
c d

Example
 
1 2
(i) A = , det(A) = (1)(1) − (2)(3) = −5.
3 1

 
3 −1
(ii) A = , det(A) = (3)(1) − (−1)(5) = 8.
5 1
Inductive Step: Matrix Minor
Suppose we have defined the determinant of all square matrices of order ≤ n − 1. Let A be a square matrix of order
n.
Define Mij , called the (i, j) matrix minor of A, to be the matrix obtained from A be deleting the i-th row and j-th
column.
Example
 
5 1 2 −1
−1 −3 1 3
A=
 
3 8 2 1
2 0 1 11

       
−3 1 3 −1 1 3 5 1 −1 5 1 −1
M11 = 8 2 1 , M12 = 3 2 1 , M23 = 3 8 1 , M43 = −1 −3 3
0 1 11 2 1 11 2 0 11 3 8 1
Inductive Step: Cofactor

The (i, j)-cofactor of A, denoted as Aij , is the (real) number given by

Aij = (−1)i+j det(Mij ).

Take note of the sign of the (i, j)-entry, (−1)i+j . Here’s a visualization of the sign of the entries of the matrix
 
+ − + ···
− + − · · ·
+ − + · · · .
 
.. .. . .
 
. . .
Example
 
1 2 −1
−1 1 3
3 2 1

1 3 −1 3 −1 1
A11 = A12 = − A13 =
2 1 3 1 3 2
= (1)(1) − (2)(3) = −(−1)(1) + (3)(3) = (−1)(2) − (1)(3)
= −5 = 10 = −5

2 −1 1 −1 1 2
A21 = − A22 = A23 = −
2 1 3 1 3 2
= −(2)(1) + (−1)(2) = (1)(1) − (−1)(3) = −(1)(2) + (2)(3)
= −4 = 4 = 4

2 −1 1 −1 1 2
A31 = A32 = − A33 =
1 3 −1 3 −1 1
= (2)(3) − (−1)(1) = −(1)(3) + (−1)(−1) = (1)(1) − (2)(−1)
= 7 = −2 = 3
Determinant by Cofactor Expansion

The determinant of A is defined to be

n
X
det(A) = ai1 Ai1 + ai2 Ai2 + · · · + ain Ain = aik Aik (1)
k=1
Xn
= a1j A1j + a2j A2j + · · · + anj Anj = akj Akj (2)
k=1

row i (3)
This is called the cofactor expansion along .
column j (4)

Remark
The above is both a theorem and a definition. The theorem states that evaluating the cofactor expansion along any
row or column produces the same result. Hence, we may define the determinant to be the cofactor expansion along
any rows or columns. Readers may refer to the appendix for details.
Property of Determinant

Theorem (Determinant is invariant under tanspose)

Let A be a square matrix. Then the determinant of A is equal to the determinant of AT ,

det(A) = det(AT ).

Students can try to prove the theorem, the details can be found in the appendix. The idea is that taking the cofactor
expansion of A along column 1 is equal to the cofactor expansion of AT along row 1. But the first produces det(A),
while the latter det(AT ).
Order 3 Matrices

a b c
e f d f d e
d e f =a −b +c = aei − afh − bdi + bfg + cdh − ceg .
h i g i g h
g h i

a b c a b
d e f d e
g h i g h

Example
0 1 1
2 1 0 = (0)(1)(3) + (1)(0)(1) + (1)(2)(2) − (1)(1)(1) − (1)(2)(3) − (0)(0)(2) = −3
1 2 3
Example
1 3 1
1. 2 1 0 = (1)(1)(3) + (3)(0)(1) + (1)(2)(2) − (1)(1)(1) − (3)(2)(3) − (1)(0)(2) = −12
1 2 3
2 1 3 1
0 1 1 1 3 1
1 0 1 1
2. =2 2 1 0 − 2 1 0 = 2(−3) − (−12) = 6 (cofactor expansion along column 1)
0 2 1 0
1 2 3 1 2 3
0 1 2 3
1 5 1 2
2 6 3
0 2 6 3 1 2
3. = (1) 0 1 2 = (1)(2) = (1)(2)(1)(1) = 2. Observe that the determinant of a triangular
0 0 1 2 0 1
0 0 1
0 0 0 1
matrix is the product of the diagonal entries.
1 1 0 1
1 1 1
0 2 0 1 2 1
4. = 2 0 2 1 = (2)(1) = 2(4 − 2) = 4 (cofactor expansion along column 3, then along
1 1 2 3 2 2
0 2 2
0 2 0 2
column 1)
Question

What is the determinant of the following matrix?

 
1 1 3 0 5 −2
2 2 0 0 1 0
 
−1 7 2 0 0 9
 
0 1 0 0 1 2
 
1 1 0 0 2 1
1 −5 7 0 8 0
Determinant of Triangular Matrices
Theorem (Determinant of a triangular matrix is the product of diagonal entries)
If A = (aij )n is a triangular matrix, then
n
Y
det(A) = a11 a22 · · · ann = aii .
k=1

Sketch of proof.
Upper triangular matrix , continuously cofactor expand along first column,
a11 a12 a13 a14
a22 a23 a24
0 a22 a23 a24 a a34
= a11 0 a33 a34 = a11 a22 33 = a11 a22 a33 a44 .
0 0 a33 a34 0 a44
0 0 a44
0 0 0 a44

Lower triangular matrix, continuously cofactor expand along the first row,
a11 0 0 0
a22 0 0
a21 a22 0 0 a 0
= a11 a32 a33 0 a11 a22 33 = a11 a22 a33 a44 .
a31 a32 a33 0 a43 a44
a42 a43 a44
a41 a42 a43 a44
Example

1 5 1 2
0 2 6 3
= (1)(2)(1)(1) = 2.
0 0 1 2
0 0 0 1

Upshot: Computation of determinant of triangular matrices is easy!

But is it useful? Majority of the square matrices are not triangular. But, every matrix is related to some triangular
matrix!
Question

1. Fill in the blanks. Every square matrix is a triangular matrix.

2. Suppose A and B are row equivalent. How are the determinants of A and B related?
2.9 Determinant by Reduction
Determinant and Elementary Row Operations
Ri + aRj

1 5 1 2
0 2 6 3
det(B) = =2
0 0 1 2
0 0 0 1
   
1 5 1 2 1 5 1 2
0 2 6 3 R −3R 1 0
 2 6 3
A=  −−3−−−→ =B
3 15 4 8 0 0 1 2
0 0 0 1 0 0 0 1
1 5 1 2
1 5 1
0 2 6 3 2 6 5 1
det(A) = = 0 2 6 = +3 = 2.
3 15 4 8 15 4 2 6
3 15 4
0 0 0 1

det(B) = det(A).
Determinant and Elementary Row Operations
cRi

1 5 1 2
0 2 6 3
det(B) = =2
0 0 1 2
0 0 0 1
   
1 5 1 2 1 5 1 2
0 1
4 12 6 2 R2 0
 −−→  2 6 3
A=
0
=B
0 1 2 0 0 1 2
0 0 0 1 0 0 0 1
1 5 1 2
0 4 12 6
det(A) = = 4.
0 0 1 2
0 0 0 1

1
det(B) = 2 det(A).
Determinant and Elementary Row Operations
Ri ↔ Rj

1 5 1 2
0 2 6 3
det(B) = =2
0 0 1 2
0 0 0 1
   
1 5 1 2 1 5 1 2
0 0 0 1 R2 ↔R 4
0 2 6 3
A=  −−−−→  =B
0 0 1 2 0 0 1 2
0 2 6 3 0 0 0 1
1 5 1 2
0 0 1
0 0 0 1 0 1
det(A) = = 0 1 2 = = −2.
0 0 1 2 2 6
2 6 3
0 2 6 3
det(B) = − det(A).
Determinant and Elementary Row Operations
Theorem
Let A be a n × n square matrix. Suppose B is obtained from A via a single elementary row operation. Then the
determinant of B is obtained as such.
Ri +aRj
A −−−−→ B det(B) = det(A)
cRi
A −−→ B det(B) = c det(A)
Ri ↔Rj
A −−−−→ B det(B) = − det(A)

The proof can be found in the appendix.

Corollary
The determinant of an elementary matrix E is given as such.
(i) If E corresponds to Ri + aRj , then det(E) = 1.
(ii) If E corresponds to cRj , then det(E) = c.
(iii) If E corresponds to Ri ↔ Rj , then det(E) = −1.

Proof.
This follows immediately from the previous theorem by letting A = I.
Determinant of Row Equivalent Matrices

Theorem
Let A and R be square matrices such that
R = Ek · · · E2 E1 A
for some elementary matrices E1 , E2 , ..., Ek . Then

det(R) = det(Ek ) · · · det(E2 ) det(E1 ) det(A).

Proof.
This follows immediately from the previous theorem and corollary.

Remark
Note that the determinant of an elementary matrix is nonzero. This means that the determinant of R could be
computed from the determinant of A and vice versa.
Determinant of Row Equivalent Matrices
Corollary  
d1 ∗ ··· ∗
r1 r2 rk
0 d2 ··· ∗
Let A be a n × n square matrix. Suppose A −
→ −
→ ··· −
→ R=. .. , where R is the reduced
 
.. ..
 .. . . .
0 0 · · · dn
row-echelon form of A. Let Ei be the elementary matrix corresponding to the elementary row operation ri , for
i = 1, ..., k. Then
d1 d2 · · · dn
det(A) = .
det(Ek ) · · · det(E2 ) det(E1 )

Proof.
Using the previous theorem and the fact that the determinant of a triangular matrix is the product its diagonal
entries,
d1 d2 · · · dn = det(Ek ) · · · det(E2 ) det(E1 ) det(A).
Hence,
d1 d2 · · · dn
det(A) = .
det(Ek ) · · · det(E2 ) det(E1 )
Example

   
2 1 3 1 0 1 1 −1
1 0 1 1
A= − − − − −−→ 1 0 1
E1 :R1 −2R2  1− E3 :R4 −R1
−−−−−→
0 2 1 0 E2 :R3 −2R4 0 0 −3 −6 E4 :− 13 R3
0 1 2 3 0 1 2 3
     
0 1 1 −1 0 1 1 −1 1 0 1 1
1 0 1 1−E5 :R4 −R3  1 0 1 1 E6 :R1 ↔R2  0 1 1 −1

0 −−−−−→  −−−−−−→  
0 1 2   0 0 1 2   0 0 1 2
0 0 1 4 0 0 0 2 0 0 0 2

⇒ 2 =
det(E6 ) det(E5 ) det(E4 ) det(E3 ) det(E2 ) det(E1 ) det(A)
 
1
= (−1)(1) − (1)(1)(1) det(A)
3
⇒ det(A) = 6
2.10 Properties of Determinant
Determinant and Invertiblity

Theorem
A square matrix A is invertible if and only if det(A) ̸= 0.

Proof.
Suppose A is invertible. Then we can write A = E1 E2 · · · Ek as a product of elementary matrices. Hence,

det(A) = det(E1 ) det(E2 ) · · · det(Ek ),

which is nonzero since determinant of elementary matrices are nonzero.

Conversely, suppose A is singular. Then the last row of the reduce row-echelon form R of A is a zero row. Thus
det(R) = 0 by cofactor expanding along the last row. Write R = Ek · · · E2 E1 A. Then

det(R)
det(A) = = 0.
det(Ek ) · · · det(E2 ) det(E1 )

We will add this to the equivalent statement of invertibility.

Equivalent Statements for Invertibility
Let A be a square matrix of order n. The following statements are equivalent.
(i) A is invertible.

(ii) AT is invertible.

(iii) (left inverse) There is a matrix B such that BA = I.

(iv) (right inverse) There is a matrix B such that AB = I.

(v) The reduced row-echelon form of A is the identity matrix.

(vi) A can be expressed as a product of elementary matrices.

(vii) The homogeneous system Ax = 0 has only the trivial solution.

(viii) For any b, the system Ax = b has a unique solution.

(ix) The determinant of A is nonzero, det(A) ̸= 0.

Determinant of Product of Matrices

Theorem
Let A and B be square matrices of the same size. Then

det(AB) = det(A) det(B).

Students may try to prove the theorem by considering the cases where A is invertible or not. The proof can be found
in the appendix.

By induction, we get
det(A1 A2 · · · Ak ) = det(A1 ) det(A2 ) · · · det(Ak ).
Question

Let A be a LU factorizable square matrix with LU factorization

  
1 0 ··· 0 u11 ∗ ··· ∗
∗ 1 · · · 0  0 u22 ··· ∗ 
A = LU =  . . . .
  
 .. .. . . ...   .
  ..
..
.
..
.
..
. 
∗ ∗ ··· 1 0 0 ··· unn

What is the determinant of A?

Determinant of inverse

Theorem
If A is invertible, then
det(A−1 ) = det(A)−1 .

Proof.
Using the fact that det(I) = 1, where I is the identity matrix, and that the determinant of product is the product of
determinant, we have
1 = det(I) = det(A−1 A) = det(A−1 ) det(A).
Since A is invertible, its determinant is nonzero. Hence, we can divide both sides of the equation above by det(A) to
obtain the conclusion
1
det(A−1 ) = .
det(A)
Determinant of Scalar Multiplication

Theorem
For any square matrix A of order n and scalar c ∈ R,

det(cA) = c n det(A).

Proof.
Observe that scalar
 multiplication
 is equivalent to matrix multiplication by scalar matrix,
c ··· 0
cA = (cI)A =  ... . . . ...  A. Hence,
 

0 ··· c

c ··· 0
det(cA) = det((cI)A) = ... ..
.
.. det(A) = c n det(A).
.
0 ··· c
Example
   
1 5 1 2 1 0 1 1
0 2 6 3 0 1 1 −1
Let A = 
0
 and B =  .
0 1 2 0 0 1 2
0 0 1 1 0 0 0 3
 
1 5 1 2
R4 −R3  0 2 6 3
A −−−−→  0
.
0 1 2
0 0 0 −1

So det(A) = −2 and det(B) = 3.

1. det(3AT ) = 34 det(AT ) = 34 det(A) = −162

2. det(3AB−1 ) = 34 det(AB−1 ) = 34 det(A) det(B−1 ) = 34 det(A) det(B)

1
= −54

3. det((3B)−1 ) = 1
det(3B) = 1
34 det(B) = 1
35 = 1
243 .
Adjoint

Definition
Let A be a n × n square matrix. The adjoint of A, denoted as adj(A), is the n × n square matrix whose (i, j) entry is
the (j, i)-cofactor of A,
 T  
A11 A12 ··· A1n A11 A21 ··· An1
A21 A22 ··· A2n  A12 A22 ··· An2 
adj(A) =  . ..  =  .. ..  .
   
.. .. .. ..
 .. . . .   . . . . 
An1 An2 ··· Ann A1n A2n ··· Ann
Example

 
a b
1. Let A = . Then
c d
 
d −b
adj(A) = .
−c a
 
1 2 −1
2. Let A = −1 1 3  (see the example of computing the cofactors). Then
3 2 1
 
−5 −4 7
adj(A) =  10 4 −2 .
−5 4 3
Adjoint Formula

Theorem
Let A be a square matrix and adj(A) it adjoint. Then

A(adj(A)) = det(A)I,

where I is the identity matrix.

The proof can be found in the appendix.

Corollary (Adjoint Formula for Inverse)

Let A be an invertible matrix. Then the inverse of A is given by
1
A−1 = adj(A).
det(A)

The corollary follows immediately from the previous theorem, and the fact that det(A) ̸= 0. From the adjoint
 −1  
a b 1 d −b
formula for inverse, we have the inverse formula for 2 × 2 matrix = ad−bc
c d −c a
Examples
 
1 2 −1
1. Let A = −1 1 3  (see example in computing adjoint). Then
3 2 1
 
−5 −4 7
1 1 
A−1 = adj(A) = 10 4 −2 .
det(A) 20
−5 4 3
   
1 2 −1 1 0 0 1 0 0 −1/4 −1/5 7/20
RREF
Indeed,  −1 1 3 0 1 0  −−−→  0 1 0 1/2 1/5 −1/10 
3 2 1 0 0 1 0 0 1 −1/4 1/5 3/20
   
−1 2 0 4 −2 2
2. Let A = −2 4 1. Then adj(A) = 2 −1 1 (check it). Observe that A is singular. Then
0 0 1 0 0 0
    
−1 2 0 4 −2 2 0 0 0
A(adj(A)) = −2 4 1 2 −1 1 = 0 0 0 .
0 0 1 0 0 0 0 0 0
Question

1. Show that a square matrix A is singular if and only if A times its adjoint is the zero matrix, A(adj(A)) = 0.

2. Is it true that A is singular if and only if the adjoint of A times A is the zero matrix, (adj(A))A = 0?
Appendix
Properties of Matrix Addition and Scalar Multiplication

Theorem
For matrices A = (aij )m×n , B = (bij )m×n , C = (cij )m×n , and real numbers a, b ∈ R,
(i) (Commutative) A + B = B + A,

(ii) (Associative) A + (B + C) = (A + B) + C,

(iii) (Additive identity) 0m×n + A = A,

(iv) (Additive inverse) A + (−A) = 0m×n ,

(v) (Distributive law) a(A + B) = aA + aB,

(vi) (Scalar addition) (a + b)A = aA + bA,

(vii) (Associative) (ab)A = a(bA),

(viii) If aA = 0m×n , then either a = 0 or A = 0.

Proof of the Properties of Scalar Multiplication and Matrix Addition

Proof.
(i) A + B = (aij + bij )m×n = (bij + aij )m×n = B + A, since addition of real numbers is commutative.
(ii) A + (B + C) = (aij + (bij + cij ))m×n = ((aij + bij ) + cij )m×n = (A + B) + C, since addition of real numbers is
associative.
(iii) A + 0 = (aij + 0)m×n = (aij )m×n = A.
(iv) A + (−A) = (aij + (−aij ))m×m = (aij − aij )m×n = (0)m×n = 0m×n .
(v) a(A + B) = (a(aij + bij ))m×n = (aaij + abij )m×n = (aaij )m×n + (abij )m×n = a(aij )m×n + a(bij )m×n = aA + aB,
where the 2nd equality follows from the distributive property of real number addition and multiplication, and the
4th equality follows from the definition of scalar multiplication.
(vi) (a + b)A = ((a + b)aij )m×n = (aaij + baij )m×n = (aaij )m×n + (baij )m×n = a(aij )m×n + b(aij )m×n = aA + bA.
(vii) (ab)A = ((ab)aij )m×n = (abaij )m×n = (a(baij ))m×n = a(baij )m×n = a(bA).
(viii) Suppose aA = (aaij )m×n = (0)m×n . This means that aaij = 0 for all i, j. So, if a ̸= 0, then necessarily aij = 0
for all i, j, which means that A = 0.
Properties of Matrix Multiplication

Theorem
(i) (Associative) For matrices A = (aij )m×p , B = (bij )p×q , and C = (cij )q×n (AB)C = A(BC).

(ii) (Left distributive law) For matrices A = (aij )m×p , B = (bij )p×n , and C = (cij )p×n , A(B + C) = AB + AC.

(iii) (Right distributive law) For matrices A = (aij )m×p , B = (bij )m×p , and C = (cij )p×n , (A + B)C = AC + BC.

(iv) (Commute with scalar multiplication) For any real number c ∈ R, and matrices A = (aij )m×p , B = (bij )p×n ,
c(AB) = (cA)B = A(cB).

(v) (Multiplicative identity) For any m × n matrix A, Im A = A = AIn .

(vi) (Zero divisor) There exists A ̸= 0m×p and B ̸= 0p×n such that AB = 0m×n .

(vii) (Zero matrix) For any m × n matrix A, A0n×p = 0m×p and 0p×m A = 0p×n .
Proof of the Properties of Matrix Multiplication
Proof.
We will check that the corresponding entries on each side agrees. The check for the size of matrices agree is trivial
and is left to the reader.
(i) The (i, j)-entry of (AB)C is
q X
X p q X
X p
( aik bkl )clj = aik bkl clj .
l=1 k=1 l=1 k=1

The (i, j)-entry of A(BC) is

p
X Xq p X
X q
aik ( bkl clj ) = aik bkl clj .
k=1 l=1 k=1 l=1

Since both sums has finitely many terms, the sums commute and thus the (i, j)-entry of (AB)C is equal to the
(i, j)-entry of A(BC).
Pp Pp Pp Pp
(ii) The (i, j)-entry of A(B + C) is k=1 aik (bkj + ckj ) = k=1 (aik bkj + aik ckj ) = k=1 aik bkj + k=1 aik ckj , which
is the (i, j)-entry of AB + AC.

(iii) The proof is analogous to left distributive law.

Proof of the Properties of Matrix Multiplication

Continue.
Pp Pp Pp
(iv) The i, j entry of cAB is c( k=1 aik bkj ) = k=1 (caik )bkj = k=1 aik (cbkj ).

1 if i = j
(v) Note that I = (δij ), where δij = . So the (i, j)-entry of Im A is
0 ̸ j
if i =

δi1 a1j + · · · + δii aij + · · · + δim amj = 0a1j + · · · + 1aij + · · · + 0amj = aij .

The proof for A = AIn is analogous.

    
1 0 0 0 0 0
(vi) Consider for example = .
1 0 1 1 0 0

(vii) Left to reader, if you have read till this far, surely this proof is trivial to you.
Row Equivalent Augmented Matrices have the Same Solutions
Theorem
Let Ax = b and Bx = c be two linear system such that their augmented matrices (A | b) and (B | c) are row
equivalent. Then u is a solution to Ax = b if and only if it is a solution to Bx = c. That is, row equivalent
augmented matrices have the same set of solutions.

Proof.
By the hypothesis, there exists an invertible matrix P such that

(PA | Pb) = P(A | b).

This means that PA = B and Pb = c. Now suppuse u is a solution to Ax = b, that is, Au = b. Then premultiplying
the equation by P, we have
Bu = PAu = Pb = c,
which shows that u is a solution to Bx = c too. Conversely, suppose u is a solution to Bx = c, Bu = c.
Premultiplying both sides of the equation by P−1 , we have

Au = P−1 Bu = P−1 c = b,

which shows that u is a solution to Ax = b too.

Permutation Groups

Definition
The group of permutations of n objects, denoted as Sn , is called the n-permutation group. An element in the
permutation group is called a permutation. An transposition is a permutation that only exchanges 2 objects.

Example

S1 = {(1)}
S2 = {(1, 2), (2, 1)}
S3 = {(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)}

We may also interpret a member of the permutation group as a function. For example σ = (2, 1, 3) as the function
σ(1) = 2, σ(2) = 1, σ(3) = 3. (2, 1), (1, 3, 2), and (3, 2, 1) are examples of inversions.
Permutation Groups

Theorem
Every permutation is a composition of transposition.

Example
1. The permutation (2, 3, 1) is obtained by exchanging 1 and 2, then 3 and 2.
2. The permutation (3, 1, 4, 2) is obtained by exchanging 1 and 2, 1 and 3, and 3 and 4.

Definition
Let σ ∈ Sn be a permutation. The sign of σ, denoted by sgn(σ), is define to be

sgn(σ) = (−1)number of transposition in the decomposition of σ .

Leibniz Formula for Determinant
Definition
Let A = (aij )n×n be a n × n square matrix. Let Sn denote the n-permutation group. The determinant of A is defined
to be
X Yn
det(A) = sgn(σ) aiσ(i) .
σ∈Sn i=1

Example
 
a11 a12
1. For A = , det(A) = sgn((1, 2)))(a11 a22 ) + sgn((2, 1)))(a12 a21 ) = a11 a22 − a12 a21 .
a21 a22
 
a11 a12 a13
2. For A = a21 a22 a23 ,
a31 a32 a33

det(A) = sgn((1, 2, 3))a11 a22 a33 + sgn((1, 3, 2))a11 a23 a32 + sgn((2, 1, 3))a12 a21 a33
+sgn((2, 3, 1))a12 a23 a31 + sgn((3, 1, 2))a13 a21 a32 + sgn((3, 2, 1))a13 a22 a31
= a11 a22 a33 − a11 a23 a32 − a12 a21 a33 + a12 a23 a31 + a13 a21 a32 − a13 a22 a31
Equivalence in the Definition of Determinant

Theorem
Let det(A) denote the determinant of a n × n square matrix A computed using the Leibniz formula. Then
n
X
det(A) = ai1 Ai1 + ai2 Ai2 + · · · + ain Ain = aik Aik
k=1
Xn
= a1j A1j + a2j A2j + · · · + anj Anj = akj Akj
k=1

That is, the determinant is equal to the cofactor expansion along any row or column.

This proves the claim that the evaluation of the cofactor expansion along any row or column produces the same
result, and that the result is the determinant of a square matrix.
Proof that Determinant is Invariant under Transpose
Theorem (Determinant is invariant under transpose)
Let A be a square matrix. Then the determinant of A is equal to the determinant of AT ,

det(A) = det(AT ).

Proof.
We will prove by induction on the order of A. The theorem holds trivially for a 1 × 1 matrix since the transpose of a
real number is itself.
Now suppose det(A) = det(AT ) for all matrices of order k. Let A be a k + 1 × k + 1 matrix. Write
A = (aij )k+1×k+1 , for all i, j = 1, ..., n. Cofactor expand along the first row of A, we have
Qk+1
det(A) = i=1 a1i (−1)i+1 det M1i ). Cofactor expand along the first column of AT , and noting that the (i, j)-matrix
minor of AT is the transpose of the (j, i)-matrix minor of A we have
k+1
Y k+1
Y k+1
Y
det(AT ) = bi1 (−1)i+1 det MT
1i ) = a1i (−1)i+1 det MT
1i ) = a1i (−1)i+1 det M1i ) = det(A),
i=1 i=1 i=1

where for the second equality we use the fact that bij = aji , and the induction hypothesis in the third equality since
the matrix minor Mij is a k × k matrix.
Determinant of Elementary Matrices

Theorem
Let A be a n × n square matrix, and B the matrix obtained from A via exchanging the k-th row and the l-th row,
R ↔Rl
A −−k−−→ B. Then
det(B) = − det(A).

Proof.
We will proof by induction on the order n of A for n ≥ 2. The case for n = 2 is clear. Suppose now the statement is
true for all matrices of size n × n. Write A = (aij ) and B = (bij ). Fix m ̸= k, l. Observe that for each j = 1, ..., n1 ,
the (m, j) matrix minor of B can be obtained from the (m, j) matrix minor of A by exchanging the k-th row and the
l-th row. Hence, if we let Mmj be the (m, j) matrix minor of A, then by the induction hypothesis, the determinant of
(m, j) matrix minor of B is − det(Mmj ). Observe also that bmj = amj for all j = 1, ..., n + 1. Hence, cofactor expand
along the m-th row, we have
n+1
X n+1
X
m+j
det(B) = bmj (−1) (− det(Mmj )) = − amj (−1)m+j det(Mmj ) = − det(A).
i=1 i=1
Determinant of Elementary Matrices

Theorem
m cR
Let A be a square matrix, and B the matrix obtained from A via multiplying row m by c ̸= 0, A −−→ B. Then

det(B) = c det(A).

Proof.
Write A = (aij ) and B = (bij ). Then bij = aij for all i ̸= m and all j = 1, ..., n, and bmj = camj for all j = 1, ..., n.
Also observe that the (m, j) matrix minor of B is equal to the (m, j) matrix minor of A for any j = 1, ..., n. We will
denote the common (m, j) matrix minor as Mmj . Now cofactor expand along row m of B,
n
X n
X n
X
det(B) = bmj det(Mmj ) = camj det(Mmj ) = c amj det(Mmj ) = c det(A).
j=1 j=1 j=1
Determinant of Elementary Matrices
Lemma
The determinant of a square matrix with two identical rows is zero.

Proof.
We will proof by induction on the order n of A, for n ≥ 2. The case for n = 2 is clear. Suppose now the statement is
true for all matrices of size n × n. Let A be a n + 1 × n + 1 matrix such that the k-th row is equal to the l-th row.
Now compute the determinant of A by cofactor expansion along the m-th row, where m ̸= k, l,
n+1
X
det(A) = amj (−1)m+j det(Mmj ).
j=1

Since Mmj is obtained from A by deleting the m ̸= k, l-th row and j-th column, Mmj is a n × n square matrix with 2
identical rows. Thus, by the induction hypothesis, det(Mmj ) = 0 for all j = 1, ..., n + 1, and hence, det(A) = 0.

Lemma
The determinant of a square matrix with two identical columns is zero.

Proof.
Follows immediately from the previous theorem, and the fact that determinant is invariant under transpose.
Determinant of Elementary Matrices

Theorem
Let A be a square matrix, and B the matrix obtained from A by adding a times of row l to row m, for some real
R +aRl
number a, A −−m−−−→ B. Then
det(B) = det(A).

Proof.
Write A = (aij ). Then the entries of B are those of A except for the m-th row, where the (m, j)-entry of B is
amj + aalj . Now cofactor expanding along the m-th row of B,
 
Xn X n Xn Xn
det(B) = (amj + aalj )Amj = amj Amj + aalj Amj = det(A) + a  alj Amj  ,
j=1 j=1 j=1 j=1

Pn
where in the last equality, we note that j=1 amj Amj is the determinant of A computed by cofactor expanding along
the m-th row.
Determinant of Elementary Matrices

Comtinue.
Now consider the matrix C obtained from A by replacing the m-th row of A with the l-th row, that is, the m-th and
l-th row of C are the l-th row of A. Since C has 2 identical rows, det(C) = 0. Now since all the other rows of C are
identical to A except the m-th row, the (m, j)-cofactor of C is Amj , the (m, j)-cofactor of A. Hence, if we cofactor
expand along the m-th row of C, remember that the m-th row is the l-th row of A), we have
n
X
0 = det(C) = alj Amj .
j=1

Thus,  
Xn
det(B) = det(A) + a  alj Amj  = det(A) + a0 = det(A).
j=1
Determinant of Product of Matrices

Theorem
Let A and B be square matrices of the same size. Then

det(AB) = det(A) det(B).

Proof.
Suppose A is singular. Then the product is singular too. Hence,

det(AB) = 0 = 0 det(B) = det(A) det(B).

Suppose A is invertible. Write A = E1 E2 · · · Ek as a product of elementary matrices. Then AB = E1 E2 · · · Ek B and

thus

det(AB) = det(E1 ) det(E2 ) · · · det(Ek ) det(B) = det(A) det(B),

where in the last equality, we used the fact that det(A) = det(E1 ) det(E2 ) · · · det(Ek )
Proof of the Adjoint Formula

Theorem
Let A be a square matrix and adj(A) it adjoint. Then

A(adj(A)) = det(A)I,

where I is the identity matrix.

Proof.
Let (A(adj(A)))[i, j] denote the (i, j) entry of the product A(adj(A)). Suffice to show that

det(A) if i = j,
A(adj(A))[i, j] =
0 if i ̸= j.

Now write A = (aij ), then

n
X
A(adj(A))[i, j] = aik Ajk .
k=1

If i = j, then the cofactor expansion of A along the i-th row.

Proof of the Adjoint Formula

Comtinue.
For i ̸= j, consider the matrix C obtained from A by replacing the j-th row of A with the i-th row, that is the i-th
row and the j-th row of C is the i-th row of A. Since C has 2 identical rows, det(C) = 0. Also, since all the other
rows of C are identical to A except the j-th row, the (j, k)-cofactor of C is Ajk , the (j, k)-cofactor of A. Hence,
cofactor expanding along the j-th row of C, noting the the j-th row is the i-th row of A,
n
X
0 = det(C) = aik Ajk .
k=1

This completes the proof of the theorem.

Cramer’s Rule

Definition
Let A be a n × n square matrix and b a n × 1 vector. Construct a new matrix Ai (b) be replacing the i-th column of
A with b, for i = 1, ..., n.

Theorem (Cramer’s Rule)

Let A be an invertible n × n matrix. For any n × 1 vector b, the unique solution to the system Ax = b is
 
det(A1 (b))
1  det(A2 (b))

x= .. .
det(A) 

. 
det(An (b))
Cramer’s Rule

Proof.
Since A is invertible, by the adjoint formula for inverse, the unique solution is
1
x = A−1 b = adj(A)b.
det(A)

Now the i-th entry of adj(A)b is

n
X n
X
(adj(A)[i, k])bk = bk Aki
k=1 k=1

On the other hand, by cofactor expansion along the i-th column of Ai (b), we have
n
X
det(Ai (b)) = bk Aki
k=1

det(Ai (b))
which is exactly the i-th entry of adj(A)b. Hence, the i-th entry of A−1 b is det(A) as required.
Uniqueness of Reduced Row-Echelon Form

Theorem (Uniqueness of RREF)

Suppose R and S are two reduced row-echelon forms of a m × n matrix A. Then R = S.

Proof.
First note that there exists an invertible matrix P such that

PR = S. (3)

This is because A is row equivalent to R and S, and so there are invertible matrices P1 , P2 such that A = P1 R and
A = P2 S. Let P = P−1 2 P1 . We will prove by induction on the numbers of rows n of R and S.
Suppose n = 1. Then R, S are row matrices and P is a nonzero real number. Since the leading entries of R and S
must be 1, by the equation (3), P = 1. So R = S.


Now suppose n > 1. Write R = r1 r2 · · · rn and S = s1 s2 · · · sn , where rj and sj is the j-th column of
R and S, respectively. By equation (3), we have
Prj = sj , (4)
for j = 1, ..., n. Since P is invertible, R and S must have the same zero columns. By deleting the zero columns and
forming a new matrix, we may assume that R and S has no zero columns.
Uniqueness of Reduced Row-Echelon Form

Continue.
With this assumption, and the fact that R and S are in RREF, necessarily the first column of both R and S must
have 1 in the first entry and 0 everywhere else. By the equation (3), the first column of P also have 1 in the first
entry and zero everywhere else. So we write R, S, and P in is submatrices,

p′ r′ s′
     
1 1 1
 0   0   0 
P= . ′ , R =  ′ , and S = ′ ,
     
P  . R  . S 
 ..  ..  ..
 

0 0 0

where p′ , r′ , s′ are row matrices. By the equation (3) and block multiplication, we have P′ R′ = S′ . Note that P′ is
invertible. Since R and S are in RREF, R′ and S′ are in RREF too. Hence, by the induction hypothesis, R′ = S′ . We
are left to show that r′ = s′ . Since R′ = S′ , and both R and S are in RREF, R and S must have the same pivot
columns, say columns i1 , i2 , ..., ir . In these columns, the entries of r′ and s′ must be zero. For the nonzero entries, by
equation (4), and the fact that the entries of the columns agree from second row onward, the entries in the first row
of each column agrees too, that is r′ = s′ too. Thus the inductive step in complete, and the statement is proven.
Row Equivalent Matrices has the same Reduced Row-Echelon Form

Theorem
Two matrices are row equivalent if and only if they have the same reduced row-echelon form.

Proof.
Suppose A and B has the same RREF R. Then there are invertible matrices P and Q such that PA = R and
QB = R. Then
Q−1 PA = Q−1 R = B.
Since Q−1 P is invertible, it can be written as a product of elementary matrices, and so A is row equivalent to B.
Suppose now A is row equivalent to B. Let P be an invertible matrix such that PA = B. Let R be the RREF of A
and S be the RREF of B. Then R = UA and S = VB for some invertible matrices U and V. Then

VPU−1 R = VPA = VB = S,

which shows that R is row equivalent to S. By the uniqueness of RREF, R = S.