Fundamental Theorem of Linear Algebra
• Orthogonal Vectors
• Orthogonal and Orthonormal Set
• Orthogonal Complement of a Subspace W
• Column Space, Row Space and Null Space of a Matrix A
• The Fundamental Theorem of Linear Algebra
Orthogonality
Definition 1 (Orthogonal Vectors)
Two vectors ~u, ~
v are said to be orthogonal provided their dot product is zero:
u ·~
~ v = 0.
If both vectors are nonzero (not required in the definition), then the angle θ between the
two vectors is determined by
u ·~
~ v
cos θ = = 0,
k~
ukk~
vk
which implies θ = 90◦ . In short, orthogonal vectors form a right angle.
Orthogonal and Orthonormal Set
Definition 2 (Orthogonal Set of Vectors)
A given set of nonzero vectors ~
u1 , . . . , ~
uk that satisfies the orthogonality condition
ui · ~
~ uj = 0, i 6= j,
is called an orthogonal set.
Definition 3 (Orthonormal Set of Vectors)
A given set of unit vectors ~
u1, . . . , ~
uk that satisfies the orthogonality condition is called
an orthonormal set.
Orthogonal Complement W ⊥ of a Subspace W
Definition. Let W be a subspace of an inner product space V , inner product h~ vi. The
u, ~
⊥
orthogonal complement of W , denoted W , is the set of all vectors ~ v in V such that
h~ vi = 0 for all ~
u, ~ u in W . In set notation:
W ⊥ = {~
v : h~ vi = 0 for all ~
u, ~ u in W }
Example. If V = R3 and W = span{~ u2}, then W ⊥ is the span of the calcu-
u1 , ~
u2. The equation dim(W ) + dim(W ⊥) = 3 holds
u1 × ~
lus/physics cross product ~
(in general dim(W ) + dim(W ⊥ ) = dim(V )).
Theorem. If W is the span of the columns ~ un of m × n matrix A (the column
u1 , . . . , ~
space of A), then
u = ~0}.
W ⊥ = nullspace(AT ) = span{Strang’s Special Solutions for AT ~
Proof. Given W = span{~ un}, then
u1 , . . . , ~
W ⊥ = {~ v·w
v : ~ ~ ∈ W}
~ = 0, all w
= {~ uj · ~
v : ~ v = 0, j = 1, . . . , n}
v = ~0}.
v : AT ~
= {~
u = ~0. Therefore,
Strang’s Special solutions are a basis for the homogeneous problem AT ~
W ⊥ = nullspace(AT ) = span{Strang’s Special Solutions for AT ~ u = ~0}.
Column Space, Row Space and Null Space of a Matrix A
The column space, row space and null space of an m × n matrix A are sets in Rn or Rm ,
defined to be the span of a certain set of vectors. The span theorem implies that each of
these three sets are subspaces.
Definition. The Column Space of a matrix A is the span of the columns of A, a subspace
of Rm . The Pivot Theorem implies that
colspace(A) = span{pivot columns of A}.
Definition. The Row Space of a matrix A is the span of the rows of A, a subspace of
Rn. The definition implies two possible bases for this subspace, just one selected in an
application:
rowspace(A) = span{Nonzero rows of rref (A)} = span{pivot columns of AT }.
Definition. The Null Space of a matrix A is the set of all solutions ~
x to the homogeneous
problem A~ x = ~0, a subspace of Rn. Because solution ~ x of A~ x = ~0 is a linear
combination of Strang’s special solutions, then
x = ~0}.
nullspace(A) = span{Strang’s Special Solutions for A~
The Row space is orthogonal to the Null Space
Theorem. Each row vector ~r in matrix A satisfies ~r · ~
x = 0, where ~
x is a solution of the
homogeneous equation A~x = ~0. Therefore
rowspace(A) ⊥ nullspace(A).
The theorem is remembered from this diagram:
· · · 0 row 1 · ~
x 0
· · · ~
x=0 is equivalent to row 2 · ~
x=0
· · · 0 row 3 · ~
x 0
which says that the rows of A are orthogonal to solutions ~ x = ~0.
x of A~
Computing the Orthogonal Complement of a subspace W
Theorem. In case W is the subspace of R3 spanned by two independent vectors ~ u1 , ~
u2 ,
then the orthogonal complement of W is the line through the origin generated by the cross
u1 × ~
product vector ~ u2 :
W ⊥ = span{~ u2}⊥ = span{~
u1, ~ u1 × ~
u2 }
Theorem. In case W is a subspace of Rm spanned by all column vectors ~
u1, . . . , ~
un of
an m × n matrix A, then the orthogonal complement of W is the subspace
W⊥ = span{~ un}⊥
u1, . . . , ~
= {~ y ·~
y : ~ ui = 0 for all i = 1, . . . , n}
= nullspace(AT )
= span{Strang’s Special Solutions for AT ~u = ~0}
Method. To compute a basis for W ⊥ , find Strang’s special Solutions for the homogeneous
u = ~0. The basis size is k = number of free variables in AT ~
problem AT ~ u = ~0.
Applications may add an additional step to replace this basis by the Gram-Schmidt orthog-
onal basis ~ yk . Then W ⊥ = span{~
y1, . . . , ~ yk }.
y1, . . . , ~
Fundamental Theorem of Linear Algebra
The four fundamental subspaces are rowspace(A), colspace(A),
Definition.
nullspace(A) and nullspace(AT ).
The Fundamental Theorem of Linear Algebra has two parts:
(1) Dimension of the Four Fundamental Subspaces.
Assume matrix A is m × n with r pivots. Then
dim(rowspace(A)) = r , dim(colspace(A)) = r ,
dim(nullspace(A)) = n − r , dim(nullspace(AT )) = m − r
(2) Orthogonality of the Four Fundamental Subspaces.
rowspace(A) ⊥ nullspace(A)
colspace(A) ⊥ nullspace(AT )
Gilbert Strang’s textbook Linear Algebra has a cover illustration for the fundamental theo-
rem of linear algebra. The original article is The Fundamental Theorem of Linear Algebra,
http://www.jstor.org/stable/2324660. The free 1993 jstor PDF is available
via the Marriott library. Requires UofU 2-factor login.
