Linear Independence, span, basis, dimension

The span of a set of vectors is the subspace consisting of all linear combinations of the
vectors in the set.

Given a subspace we say a set S of vectors spans the subspace if the span of the set S is
the subspace.

A basis of a subspace is a set of vectors that spans the subspace and is linearly
independent.
If you have a basis of a subspace and you add any vector v to it, the resulting set is no
longer linearly independent.
(Reason: v is in the span of the basis, hence adding v to the basis renders it linearly
dependent)
If you have a basis of a subspace, and you remove any vector from that set, the
resulting set does not span the subspace.
(Reason: The vector you removed cannot be in the span of the remaining vectors
because the original basis is linearly independent)

The subspace consisting of solutions of Ax 0 is called the nullspace of the matrix A. It has
a basis consisting of the fundamental solutions of Ax 0 that we know how to calculate.

The span of a given set of vectors is a subspace. When we put these vectors in a matrix,
that subspace is called the column space of the matrix: to find a basis of the span, put the
vectors in a matrix A. The columns of A that wind up with leading entries in Gaussian
elimination form a basis of that subspace.

The dimension of a subspace U is the number of vectors in a basis of U. (There are many
choices for a basis, but the number of vectors is always the same.) There are many
possible choices of a basis for any vector space; different bases can have different useful
features.

1 2
2 2
Example: Find a basis for the space spanned by the vectors , ,
1 1
1 2
0 1 1
6 4 1
, ,
1 0 1
4 3 2
Solution: This method finds a basis from among the given vectors. We put them in a matrix

1
 1 2 0 1 1 1 0 2 1 0
2 2 6 4 1 0 1 1 1 0
and reduce the matrix: ~ . The leading entries
1 1 1 0 1 0 0 0 0 1
1 2 4 3 2 0 0 0 0 0
appear in columns 1,2,5 so the vectors in those columns in the original matrix are our basis.
The other columns can be generated from these, and, by themselves, columns 1,2,5 are
linearly independent. So here is our basis:
1 2 1
2 2 1
, , The dimesion of this subspace is 3. (Note that we need
1 1 1
1 2 2
only go far enough to determine where the leading entries will be - row echelon form is
sufficient for that)

Why is this result true: From the row reduced echelon form we see that column 3 is a linear
combination of cols 1 and 2 and the same is true of column 4. Columns 1,2,5 are linearly
independent and their span includes columns 2 and 3 and hence their span is the same as
the five original columns.

Row spaces: Note that the span of the rows of a matrix is also a subspace, called the row
space of the matrix, in this case a subspace of R n (if the matrix is m n ). Now every
nonzero row in the row echelon (or reduced row echelon) form of the matrix is in the row
space (they were obtained by a sequence of linear combinations from the original rows) and
also every row of the original matrix is in the span of the nonzero rows of the row echelon
(or reduced row echelon) form, because we can go backwards and recover the original
rows. Hence the row space is also the span of the nonzero rows of the reduced matrix. On
the other hand, the rows of the echelon (or reduced row echelon) form are linearly
independent - this is easy to see for the reduced row echelon form, and not hard even for
the row echelon form. So: The nonzero rows of the row echelon form (or reduced row
echelon form) of a matrix are a basis for the row space. This is a useful basis. Although
we have not chosen this basis from among the original rows (we could do that if we turned
the rows into columns and used the method above), this basis is useful in that it is easy to
compute the linear combination needed to generate any vector in the subspace from the
basis, or easy to check whether any given vector is in the subspace.
In the example above:
1 2 0 1 1
2 2 6 4 1
A . If we consider the span of the rows, a basis for this subspace
1 1 1 0 1
1 2 4 3 2
are the nonzero rows of the reduced form on the right:

2
 1 2 0 1 1 1 0 2 1 0
2 2 6 4 1 0 1 1 1 0
~ . The (nonzero) rows of the reduced matrix on
1 1 1 0 1 0 0 0 0 1
1 2 4 3 2 0 0 0 0 0
the right are our basis. There are three such vectors so the dimension of the row space of
this matrix is 3.

This gives rise to the following famous and important observation about subspaces
associated with matrices:
dimension of column space dimension of row space rank of the matrix
because all of these quantities are equal to the number of leading entries in the (reduced)
row echelon form of the matrix.