LECTURE 12 RECAP
1) A theorem on the usefulness of knowing the dimension
of a vector space.
2) Two more equivalent statements to "A is an invertible
square matrix of order n."
3) Transition matrices: how to find and how to use.
4) Transition matrices are invertible and their inverses
are also transition matrices.
LECTURE 13
ROW SPACES AND COLUMN SPACES
DEFINITION
Given any m n matrix A,
n
a11 a12 ... a1n r1 a11 a12 ... a1n
a
a ... a r2 a21 a22 ... a2 n
A 21 22 2n
a rm am1 am 2 ... amn
m1 m 2
a ... amn
n
The rows of A can be considered as vectors in .
span{r1 , r2 ,..., rm } is a subspace of n
,
This subspace is called the row space of A.
DEFINITION
Given any m n matrix A,
a11 a12 ... a1n The columns of A can be
a a ... a m
considered as vectors in .
A 21 22 2n
m
span{c1 , c2 ,..., cn } is
a
m1 m 2
a ... amn
a subspace of m
,
a11 a12 a1n
a a a This subspace is called
c1 21 c2 22 cn 2 n the column space of A.
a a a
m1 m2 mn
REMARK
1 0 0
0 1 0
1 0 1 1 4
A 0 1 4 2 1 A 1 4 2
T
1
0 0 2 0 1 2 0
4 1 1
The row space of A is the column space of AT
The column space of A is the row space of AT
EXAMPLE
2 1 0
1 1 3
A
5 1 0
1 0
1
3
The row space of A is a subspace of .
4
The column space of A is a subspace of .
Note that if A is not a square matrix, then the row space
and column space of A contains totally 'different type'
of vectors.
EXAMPLE
2 1 0 r1
1 1 3 r2 We write r1 (2, 1, 0) (as a vector)
A
5 1 0 r3 rather than a row matrix 2 1 0 .
1 0 1
r4
3
The row space of A is a subspace of .
span{ r1 , r2 , r3 , r4 }
{a(2,1, 0) b(1, 1, 3) c(5,1, 0) d (1, 0,1)| a , b, c, d }
{(2a b 5c d , a b c , 3b d )|a ,b , c , d }
EXAMPLE
2 1 0
1 1 3
A
5 1 0
1 0 1
c1 c2 c3
4
The column space of A is a subspace of .
span{c1 , c2 , c3 }
2 1 0 2a b
1 1 3 a b 3c
| a , b , c
a b c |a ,b, c
5 1 0 5a b
1 0 1 a c
NOTATION
n
We have observed that a vector in can be identified
as a row or matrix.
Henceforth, when we write Identified with:
(u1 , u2 ,..., un ) row vector u1 u2 ... un
u1
u
(u1 , u2 ,..., un )T column vector 2
:
u
n
EXAMPLE
1 0 1 1 4
A 0 1 4 2 1 How to find a basis for the
0 0 2 0 1
row space or column space of A?
5
The row space of A is a subspace of .
3
The column space of A is a subspace of .
EXAMPLE
1 0 1 1 4
A 0 1 4 2 1 How to find a basis for the
0 0 2 0 1
row space or column space of A?
row space of A span{(1, 0, 1,1, 4),(0,1, 4, 2,1),(0, 0, 2, 0,1)}
If (1, 0, 1,1, 4),(0,1, 4, 2,1),(0, 0, 2, 0,1) (that is, the rows of A)
are linearly independent, then obviously they will form
a basis for the row space of A.
EXAMPLE
1 0 1 1 4
A 0 1 4 2 1 How to find a basis for the
0 0 2 0 1
row space or column space of A?
row space of A span{(1, 0, 1,1, 4),(0,1, 4, 2,1),(0, 0, 2, 0,1)}
(0, 0, 0, 0, 0) a(1, 0, 1,1, 4) b(0,1, 4, 2,1) c(0, 0, 2, 0,1)
a 0 ,b 0 ,c 0
So the three rows of A are linearly independent and
thus form a basis for the row space of A.
EXAMPLE
1 0 1 1 4 Note that A is in row echelon form.
A 0 1 4 2 1
What if we want to find a basis
0 0 2 0 1
for the row space of a matrix R
that is in row echelon form?
* *
non zero
* *
rows
*
R 0
* *
0 ... ... ... 0 0 zero rows
(if any)
EXAMPLE
1 0 1 1 4 Note that A is in row echelon form.
A 0 1 4 2 1 The non zero rows of R are always
0 0 2 0 1
linearly independent and thus forms
a basis for the row space of R.
* *
non zero
* *
rows
*
R 0
* *
0 ... ... ... 0 0 zero rows
(if any)
EXAMPLE
1 0 1 1 4
A 0 1 4 2 1 How to find a basis for the
0 0 2 0 1
row space or column space of A?
1 0 1 1 4
column space of A span 0 , 1 , 4 , 2 , 1
0 0 2 0 1
column space of A is a subspace of 3 ,
the dimension of this subspace is at most 3
So if we can identify 3 linearly independent vectors
(out of the 5) from the set above...
EXAMPLE
1 0 1 1 4
A 0 1 4 2 1 How to find a basis for the
0 0 2 0 1
row space or column space of A?
1 0 1 1 4
column space of A span{ 0 , 1 , 4 , 2 , 1 }
0 0 2 0 1
3
linearly
column space of A is a subspace of ,
independent
the dimension of this subspace is at most 3
3
Column space of A is the entire .
DISCUSSION
1 0 1 1 4
A 0 1 4 2 1 How to find a basis for the
0 0 2 0 1
row space or column space of A?
That was based on Yes, you are right.
observations... A more systematic
approach is needed.
DISCUSSION
Let S be the set of all matrices of the same size (say m n).
Recall the definition of row equivalent.
Series of e.r.o A and B are
A B
row equivalent
Row equivalence is an equivalence relation on S .
(Reflexive) A is row equivalent to A
(Symmetric) If A B , then B A
(Transitive) If A B and B C then A C
DISCUSSION
Let S be the set of all matrices of the same size (say m n).
Recall the definition of row equivalent.
Series of e.r.o A and B are
A B
row equivalent
Two matrices A and B (of the same size) are row equivalent
if and only if they have a similar row-echelon form (or they
have the same unique reduced row-echelon form).
THEOREM
Let A and B be row equivalent matrices. Then the row
space of A and the row space of B are identical.
Series of e.r.o
A B
same row space
That is to say, performing elementary row operations
on A does not change its row space.
EXAMPLE
0 0 1 R R 12 1 2
0 2 4 1 3
0 2 4
span{(0, 0,1),(0, 2, 4),( 12 ,1, 2)}
1 1 2 0 0 1
2
span{(12 ,1, 2),(0, 2, 4),(0, 0,1)}
2R1
span{(1, 2, 4),(0, 2, 4),(0, 0,1)}
1 0 0 R1 R2 1 2 4
0 2 4 0 2 4 span{(1, 0, 0),(0, 2, 4),(0, 0,1)}
0 0 1 0 0 1
BACK TO THIS QUESTION
Question : How to find a basis for the row space of a matrix A?
* * The non zero rows of R are always
* *
linearly independent and thus forms
*
R 0 a basis for the row space of R.
* *
0 ... ... ... 0 0 Answer :
Find a row-echelon form R of A.
A basis for the row space of R is also a basis for the
row space of A.
Let A and B be row equivalent matrices. Then the row
space of A and the row space of B are identical.
EXAMPLE
Find a basis for the row space of the following matrix.
2 2 1 0 1 2 2 1 0 1
1 1 2 3 1 Gaussian 0 0 3 3 32
A 2
0 0 1 1 1 Elimination 0 0 0 3 0
1 1 2 0 1 0 0 0 0 0
Performing Gaussian Elimination on A :
A basis for the row space of A is
{(2, 2, 1, 0,1),(0, 0, 32 , 3, 32),(0, 0, 0, 3, 0)}
FINDING BASIS FOR COLUMN SPACES
1 0 1 1 4 1 0 0
0 1 0
A 0 1 4 2 1 column space of A
0 0 2 0 1
row space of A A 1 4 2
T
T
1 2 0
Ok, what about
4 1 1
column spaces?
Note the
relationship
between column
space of A and
row space of AT ...
FINDING BASIS FOR COLUMN SPACES
1 0 1 1 4 1 0 0
0 1 0
A 0 1 4 2 1 column space of A
0 0 2 0 1
row space of A A 1 4 2
T
T
1 2 0
4 1 1
So to find a basis for the column space of A, we can
use the previous method to find a basis for the row
space of AT .
In what follows, we will discuss another method.
IMPORTANT TO NOTE
Elementary row operations preserve the row space of a
matrix but NOT the column space.
1 0 R1 R2 0 0
A 1 0 B
0 0
column space of A column space of B
1 y 0
span{ } span{ }
0 1
x axis y axis
x
THEOREM
Let A and B be row equivalent matrices. Then the following
statements hold:
* * * * * * * * * *
* * * * * Series of * * * * *
A: : : : : e.r.o : : : : :B
: : : : : : : : : :
* * * *
* * * * * *
A given set of columns of A is linearly independent
if and only if the corresponding columns of B is linearly
independent.
THEOREM
Let A and B be row equivalent matrices. Then the following
statements hold:
* * * * * * * * * *
* * * * * Series of * * * * *
Remember: Column space of A Column space of B
A: : : : : e.r.o : : : : :B
: : : : : : : : : :
* * * *
* * * * * *
A given set of columns of A forms a basis for the column
space of A if and only if the corresponding columns of B
forms a basis for the column space of B.
OBSERVATION
If R is a matrix in row echelon form,
* *
the pivot columns of R always form * *
a basis for the column space of R. R 0 *
* *
Question : How to find a basis for the 0 ... ... ... 0 0
column space of a matrix A?
Let A and B be row equivalent matrices.
A given set of columns of A forms a basis for the column
space of A if and only if the corresponding columns of B
forms a basis for the column space of B.
FINDING A BASIS FOR COLUMN SPACE
If R is a matrix in row echelon form,
* *
the pivot columns of R always form * *
a basis for the column space of R. R 0 *
* *
Question : How to find a basis for the 0 ... ... ... 0 0
column space of a matrix A?
Answer
Let R be a row echelon Remember NOT to take the
form of A. columns of R as your answer!
A basis for the column space of A can be obtained by taking
the columns of A that correspond to the pivot columns in R.
EXAMPLE
Find a basis for the column space of the following matrix.
2 2 1 0 1 2 2 1 0 1
1 1 2 3 1 Gaussian 0 0 3 3 3
A 2 2
0 0 1 1 1 Elimination 0 0 0 3 0
1 1 2 0 1 0 0 0 0 0
Performing Gaussian Elimination on A :
2 1 0
1 2 3
A basis for the column space of A is , ,
0 1 1
1 2 0
APPLYING THIS KNOWLEDGE
Let u1 (1, 2, 0, 4), u2 (0,1, 5, 0), u3 (1, 3, 2, 4),
u4 (2,1, 0,8), u5 (3,1, 1,12) and V span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V . We can construct a
There is no matrix matrix ourselves!
in this question!
Let me show you
two methods...
METHOD 1
Let u1 (1, 2, 0, 4), u2 (0,1, 5, 0), u3 (1, 3, 2, 4),
u4 (2,1, 0,8), u5 (3,1, 1,12) and V span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
Put the vectors as
same as "Find a basis for the
rows. So V is the
row space of A".
row space of the
1 2 0 4 matrix A.
0 1 5 0
Let A 1 3 2 4 .
2 1 0 8
3 1 1 12
METHOD 1
Let u1 (1, 2, 0, 4), u2 (0,1, 5, 0), u3 (1, 3, 2, 4),
u4 (2,1, 0,8), u5 (3,1, 1,12) and V span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
{(1, 2, 0, 4),(0,1, 5, 0),(0, 0, 23, 0)}
same as "Find a basis for the
is a basis for V .
row space of A".
1 2 0 4 1 2 0 4
0 1 5 0 0 1 5 0
Gaussian
A 1 3 2 4 0 0 23 0 R
2 1 0 8
Elimination 0
0 0 0
3 1 1 12 0 0
0 0
METHOD 2
Let u1 (1, 2, 0, 4), u2 (0,1, 5, 0), u3 (1, 3, 2, 4),
u4 (2,1, 0,8), u5 (3,1, 1,12) and V span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V . Note that Put the vectors as
same as "Find a basis for the
B AT columns. So V is
column space of B ".
( A from the columns space
1 0 1 2 3 Method 1) of the matrix B.
2 1 3 1 1
Let B .
0 5 2 0 1
4 0 4 8 12
METHOD 2
Let u1 (1, 2, 0, 4), u2 (0,1, 5, 0), u3 (1, 3, 2, 4),
u4 (2,1, 0,8), u5 (3,1, 1,12) and V span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
{(1, 2, 0, 4),(0,1, 5, 0),(1, 3, 2, 4)}
same as "Find a basis for the
is a basis for V .
column space of B ".
1 0 1 2 3 1 0 1 2 3
2 1 3 1 1 Gaussian 0 1 5 3 5
B R
0 5 2 0 1 Elimination 0 0 23 15 24
4 0 4 8 12 0 0 0
0 0
METHOD 1 VS METHOD 2
Let u1 (1, 2, 0, 4), u2 (0,1, 5, 0), u3 (1, 3, 2, 4),
u4 (2,1, 0,8), u5 (3,1, 1,12) and V span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
Method 1 Method 2
{(1, 2, 0, 4),(0,1, 5, 0),(0, 0, 23, 0)} {(1, 2, 0, 4),(0,1, 5, 0),(1, 3, 2, 4)}
is a basis for V . is a basis for V .
{u1 , u2 , u3 } is a basis for V .
The basis found using Method 2 will always be a subset
of { u1 , u2 , u3 , u4 , u5 }.
EXAMPLE
S {(1, 4, 2, 5,1),(2, 9, 1,8, 2),(2, 9, 1, 9, 3)}.
1) Show that S is a linearly independent set.
5
2) Extend S to a basis for .
Here we mean add vectors to the set S so that the
5
resulting set becomes a basis for .
How many vectors do we need to add?
COLUMN SPACE AND LINEAR SYSTEMS
2x y 1
x y 3z 4
Ax b where
5 x y 2
2 1 0
x z 3 1 1 3
A
2 1 0 1 5 1 0
1 1 3 4 1 0
1
x y z
5 1 0 2 1
1 0 1 3 x 4
x y b
2
z
3
COLUMN SPACE AND LINEAR SYSTEMS
2x y 1
x y 3z 4
Ax b
5 x y 2
x Ax b is consistent means
z 3
x , y , z can be found
2 1 0 1 to satisfy (*)
1 1 3 4
x y z b is a linear combination
5 1 0 2 of the columns of A.
1 0 1 3
That is, b belongs to
(*)
the column space of A.
COLUMN SPACE AND LINEAR SYSTEMS
2x y 1
x y 3z 4
Ax b
5 x y 2
x Ax b is consistent
z 3
x , y , z can be found
2 1 0 1 to satisfy (*)
1 1 3 4
x y z b is a linear combination
5 1 0 2 of the columns of A.
1 0 1 3
b belongs to
(*)
the column space of A.
THEOREM
Let A be a m n matrix. Then the column space
of A is
u1
u
2
A u , u ,..., u Au u n
:
1 2 n
un
A system of linear equations Ax b is consistent if and
only if b lies in the column space of A.
END OF LECTURE 14
LECTURE 15: RANKS
NULLSPACE AND NULLITY(END OF CHAPTER 4)
