Solving Systems of Linear Equations

Systems of linear equations occur frequently in math and in applications. I’ll explain what they
are, and then how to use row reduction to solve them.

Systems of linear equations

If a1 , a2 , . . . , an , b are numbers and x1 , x2 , . . . , xn are variables, a linear equation is an equation of
the form
a1 x1 + a2 x2 + · · · + an xn = b.
The a’s are referred to as the coefficients and b is the constant term. If the number of variables is
small, we may use unsubscripted variables (like x, y, z).
For the moment I won’t be specific about what number system we’re using; if you want, you can think
of some commutative ring with identity. In a particular situation, the number system will be specified.
And as usual, any equation which is algebraically equivalent to the one above is considered a linear
equation; for example,
a1 x1 + a2 x2 + · · · + an xn − b = 0.
But I will almost always write linear equations in the first form, with the variable terms on one side
and the constant term on the other.
A system of linear equations is a collection of linear equations

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
....
..
am1 x1 + am2 x2 + · · · + amn xn = bm

All the a’s and b’s are assumed to come from some number system.
For example, here are some systems of linear equations with coefficients in R:
 
  
x+y−z = 3
  x+y = 13 
5x − 2y = 7
2x + y = 10
−2x + y = 3 −x − y + 5z = −7 x−y = 5
 

(The curly braces around each system are just here to make them stand out; if there’s no chance of
confusion, they aren’t necessary.)
A solution to a system of linear equations is a set of values for the variables which makes all the
equations true. For example,
 
5x − 2y = 7
x = 13, y = 29 is a solution to .
−2x + y = 3

You can check this by plugging in:

5 · 13 − 2 · 29 = 65 − 58 = 7
−2 · 13 + 29 = −26 + 29 = 3

Note that we consider the pair (x, y) = (13, 29) a single solution to the system. In fact, it’s the only
solution to this system of equations.
Likewise, you can check that (x, y, z) = (1, 3, −1) and (x, y, z) = (2, 0, −1) are solutions to

x+y−z = 3

.
−x − y + 5z = −7

1
 They aren’t the only solutions: In fact, this system has an infinite number of solutions.
This system has no solutions:  
 x + y = 13 
2x + y = 10
x−y = 5
 

Here’s one way to see this. Add the first and third equations to get 2x = 18, which means x = 9. Plug
x = 9 into the second equation 2x + y = 10 to get 18 + y = 10, or y = −8. But now if you plug x = 9,
y = −8 into the first equation x + y = 13, you get 9 + (−8) = 13, which is a contradiction.
What happened in these examples is typical of what happens in general: A system of linear equation
can have exactly one solution, many solutions, or no solutions.

Solving systems by row reduction

Row reduction can be used to solve systems of linear equations. In fact, row operations can be viewed
as ways of manipulating whole equations.
For example, here’s a system of linear equations over R:

x + 2y = −4
2x + 3y = 5

The row operation that swaps two rows of a matrix corresponds to swapping the equations. Here we
swap equations 1 and 2:
   
x + 2y = −4 → 2x + 3y = 5
2x + 3y = 5 r1 ↔ r2 x + 2y = −4

It may seems pointless to do this with equations, but we saw that swapping rows was sometimes needed
in reducing a matrix to row reduced echelon form. We’ll see the connection shortly.
The row operation that multiplies (or divides) a row by a number corresponds to multiplying (or
dividing) an equation by a number. For example, I can divide the second equation by 2:

x + 2y = −4
  ( )
x + 2y = −4 →
3 5
2x + 3y = 5 r2 → r2 /2 x + y =
2 2
The row operation that adds a multiple of one row to another corresponds to adding a multiple of one
equation to another. For example, I can add −2 times the first equation to the second equation (that is,
subtract 2 times the first equation from the second equation):
   
x + 2y = −4 → x + 2y = −4
2x + 3y = 5 r2 → r2 − 2r1 −y = 13

You can see that row operations certainly correspond to “legal” operations that you can perform on
systems of equations. You can even see in the last example how these operations might be used to solve a
system.
To make the connection with row reducing matrices, suppose we take our system and just write down
the coefficients:    
x + 2y = −4 1 2 −4
→
2x + 3y = 5 2 3 5
This matrix is called the augmented matrix or auxiliary matrix for the system.
Watch what happens when I row reduce the matrix to row-reduced echelon form!
       
1 2 −4 → 1 2 −4 → 1 2 −4 → 1 0 22
2 3 5 r2 → r2 − 2r1 0 −1 13 r2 → −1r2 0 1 −13 r1 → r1 − 2r2 0 1 −13

2
 Since row operations operate on rows, they will “preserve” the columns for the x-numbers, the y-
numbers, and the constants: The coefficients of x in the equations will always be in the first column,
regardless of what operations I perform, and so on for the y-coefficients and the constants. So at the end, I
can put the variables back to get equations, reversing what I did at the start:
x y constants
 
1 0 22 1·x + 0·y = 22
→
0 1 −13 0·x + 1·y = −13
That is, x = 22 and y = −13 — I solved the system!
Geometrically, the graphs of x + 2y = −4 and 2x + 3y = 5 are lines. Solving simultaneously amounts to
finding where the lines intersect — in this case, in the point (22, −13).

(I will often write solutions to systems in the form (x, y) = (22, −13), or even (22, −13). In the second
form, it’s understood that the variables occur in the same order in which they occurred in the equations.)
You can see several things from this example. First, we noted that row operations imitate the operations
you might perform on whole equations to solve a system. This explains why row operations are what they
are.
Second, you can see that having row reduced echelon form as the target of a row reduction corresponds
to producing equations with “all the variables solved for”. Think particularly of the requirements that
leading entries (the 1’s) must be the only nonzero elements in their columns, and the “stairstep” pattern of
the leading entries.
Third, by working with the coefficients of the variables and the constants, we save ourselves the trouble
of writing out the equations. We “get lazy” and only write exactly what we need to write to do the problem:
The matrix preserves the structure of the equations for us. (This idea of “detaching coefficients” is useful in
other areas of math.)
Let’s see some additional examples.
Example. Use row reduction to solve the following system of linear equations over R:
x + y + z = 5
x + 3y + z = 9
4x − y + z = −2

Write down the matrix for the system and row reduce it to row reduced echelon form:
     
1 1 1 5 1 1 1 5 1 1 1 5 →
1 3 1 9  → 1 3 → 1
1 9  0 2 0 4 
r3 → r3 − 4r1 r2 → r2 − r1 r2 → r2
4 −1 1 −2 0 −5 −3 −22 0 −5 −3 −22 2
     
1 1 1 5 1 0 1 3 1 0 1 3 →
0 1 → → 1
0 2  0 1 0 2  0 1 0 2 
r1 → r1 − r2 r3 → r3 + 5r2 r3 → − r3
0 −5 −3 −22 0 −5 −3 −22 0 0 −3 −12 3

3
    
1 0 1 3 1 0 0 −1
0 →
1 0 2 0 1 0 2 
r1 → r1 − r3
0 0 1 4 0 0 1 4
The last matrix represents the equations x = −1, y = 2, z = 4. So the solution is (−1, 2, 4).
In this case, the three equations represent planes in space, and the point (−1, 2, 4) is the intersection of
the three planes.

Everything works in similar fashion for systems of linear equations over a finite field. Here’s an example
over Z3 .
Example. Solve the following system of linear equations over Z3 :

x + 2y = 2
2x + y + z = 1
2x + z = 2

Remember that all the arithmetic is done in Z3 . So, for example,

1 + 2 = 0, 2 + 2 = 1, 2 · 2 = 1.

The last equation tells you that when you would think of dividing by 2, you multiply by 2 instead.
Write down the corresponding matrix and row reduce it to row reduced echelon form:
     
1 2 0 2 1 2 0 2 1 2 0 2
2 1 → → →
1 1 2 1 1 1 0 0 1 0
r3 → r3 + r1 r2 → r2 + r1 r2 ↔ r3
2 0 1 2 0 2 1 1 0 2 1 1
     
1 2 0 2 1 2 0 2 1 0 2 1
0 → → →
2 1 1 0 1 2 2 0 1 2 2
r2 → 2r2 r1 → r1 + r2 r1 → r1 + r3
0 0 1 0 0 0 1 0 0 0 1 0
   
1 0 0 1 1 0 0 1
0 1 2 →
2 0 1 0 2
r2 → r2 + r3
0 0 1 0 0 0 1 0
The last matrix represents the equations x = 1, y = 2, z = 0. Hence, the solution is (1, 2, 0).

It’s possible for a system of linear equations to have no solutions. Such a system is said to be incon-
sistent. You can tell a system of linear equations is inconsistent if at any point one of the equations gives
3
a contradiction, such as “0 = 13” or “0 = − ”.
2
4
Example. Solve the following system of equations over R:

3x + y + 3z = 2
x + 2z = −3
2x + y + z = 4

Form the coefficient matrix and row reduce:

   
3 1 3 2 1 0 2 0
1 0 2 −3  → 0 1 −3 0 
2 1 1 4 0 0 0 1

The corresponding equations are

x + 2z = 0
y − 3z = 0
0 = 1

The last equation says 0 = 1. Hence, the system has no solutions.

Note: If you’re row reducing to solve a system and you notice that a row of the current matrix would
give an equation that is a contradiction (like “0 = 1” or “0 = −5”), you can stop row reducing and conclude
that there is no solution; you don’t need to go all the way to row reduced echelon form.
Besides a single solution or no solutions, a system of linear equations can have many solutions. When
this happens, we’ll express the solutions in parametrized or parametric form. Here’s an example over
the real numbers.
Example. Solve the following system of linear equations over R:

a + 2b + d = 3
−a − 2b + c − 2e =
−2a − 4b + c − d − 2e = −5

Write down the corresponding matrix and row reduce:

   
1 2 0 1 0 3 1 2 0 1 0 3
 −1 −2 → →
1 0 −2 −2   −1 −2 1 0 −2 −2 
r3 → r3 + 2r1 r2 → r2 + r1
−2 −4 1 −1 −2 −5 0 0 1 1 −2 1
   
1 2 0 1 0 3 1 2 0 1 0 3
0 →
0 1 1 −2 1  0 0 1 1 −2 1 
r3 → r3 − r2
0 0 1 1 −2 1 0 0 0 0 0 0
The corresponding equations are

a + 2b + d = 3
c + d − 2e = 1

Notice that the variables are not solved for as particular numbers. This means that the system will have
more than one solution. In this case, we’ll write the solution in parametric form.
The variables a and c correspond to leading entries in the row reduced echelon matrix. b, d, and e are
called free variables. Solve for the leading entry variables a and c in terms of the free variables b, d, and e:

a = −2b − d + 3
c = −d + 2e + 1

5
 Next, assign parameters to each of the free variables b, d, and e. Often variables like s, t, u, and v are
used as parameters. Try to pick variables “on the other side of the alphabet” from the original variables to
avoid confusion. I’ll use
b = s, d = t, e = u.
Plugging these into the equations for a and c gives

a = −2s − t + 3, c = −t + 2u + 1.

All together, the solution is

a = −2s − t + 3
b = s
c = −t + 2u + 1
d = t
e = u

(I lined up the variables so you can see the structure of the solution. You don’t need to do this in
problems for now, but it will be useful when we discuss how to find the basis for the null space of a matrix.)
Each assignment of numbers to the parameters s, t, and u produces a solution. For example, if s = 1,
t = 0, and u = 2,
a = (−2) · 1 − 0 + 3 = 1, c = −0 + 2 · 2 + 1 = 5.
The solution (in this case) is (a, b, c, d, e) = (1, 1, 5, 0, 2). Since you can assign any real number to each
of s, t, and u, there are infinitely many solutions.
Geometrically, the 3 original equations represented sets in 5-dimensional space R5 . The solution to the
system represents the intersection of those 3 sets. Since the solution has 3 parameters, the intersection is
some kind of 3-dimensional set. Since the sets are in 5-dimensional space, it would not be easy to draw a
picture!
Example. Solve the following system over Z5 :

w + x + y + 2z = 1
2x + 2y + z = 0
2w + 2y + z = 1

Write down the matrix for the system and row reduce:
     
1 1 1 2 1 1 1 1 2 1 1 1 1 2 1
0 → → →
2 2 1 0 0 2 2 1 0 0 1 1 3 0
r3 → r3 − 2r1 r2 → 3r2 r1 → r1 − r2
2 0 2 1 1 0 3 0 2 4 0 3 0 2 4
     
1 0 0 4 1 1 0 0 4 1 1 0 0 4 1
0 → → →
1 1 3 0 0 1 1 3 0 0 1 1 3 0
r3 → r3 − 3r2 r3 → 3r3 r2 → r2 − r3
0 3 0 2 4 0 0 2 3 4 0 0 1 4 2
 
1 0 0 4 1
0 1 0 4 3
0 0 1 4 2
Putting the variables back, the corresponding equations are

w + 4z = 1, x + 4z = 3, y + 4z = 2.

Solve for the leading entry variables w, x, and y in terms of the free variable z. Note that 4z + z = 0,
so I can solve by adding z to each of the 3 equations:

w = z + 1, x = z + 3, y = z + 2.

6
 To write the solution in parametric form, assign a parameter to the “non-leading entry” variable z.
Thus, set z = t:
w = t + 1, x = t + 3, y = t + 2, z = t.
Remember that the number system is Z5 = {0, 1, 2, 3, 4}. So there are 5 possible values for the parameter
t, and each value of t gives a solution. For instance, if t = 1, I get

w = 2, x = 4, y = 3, z = 1.

It follows that this system of equations has 5 solutions.

Remark. A system of linear equations over the real numbers can have no solutions, a unique solution, or
infinitely many solutions. (That is, such a system can’t have exactly 3 solutions.)
On the other hand, if p is prime, a system of linear equations over Zp will have pk solutions, for k ≥ 0,
or no solutions. For example, a system over Z5 can have no solutions, one (that is, 50 ) solution, 5 solutions,
25 solutions, and so on.

Remember that if A is a square (n × n) matrix, the inverse A−1 of A is an n × n matrix such that

AA−1 = I and A−1 A = I.

Here I is the n × n identity matrix. Note that not every square matrix has an inverse.

Row reduction provides a systematic way to find the inverse of a matrix. The algorithm gives another
application of row reduction, so I’ll describe the algorithm and give an example. We’ll justify the algorithm
later.
To invert a matrix, adjoin a copy of the identity matrix to the original matrix. “Adjoin” means to place
a copy of the identity matrix of the same size as the original matrix on the right of the original matrix to
form a larger matrix (the augmented matrix).
Next, row reduce the augmented matrix. When the block corresponding the original matrix becomes
the identity, the block corresponding to the identity will have become the inverse.
Example. Invert the following matrix over R:
 
2 −1
.
5 −3

Place a copy of the 2 × 2 identity matrix on the right of the original matrix, then row reduce the
augmented matrix:

1 1
 
 
→
" 1 1 #
1 − 0
2 −1 1 0 1 1 − 0 → 2 2 →
2 2
 
5 −3 0 1 r1 → r1 r2 → r2 − 5r1
 1 5 
r2 → −2r2
2 5 −3 0 1 0 − − 1
2 2
" 1 1 #
→
 
1 − 0 1 1 0 3 −1
2 2 r1 → r1 + r2 0 1 5 −2
0 1 5 −2 2
You can see that the original matrix in the left-hand 2 × 2 block has turned into the 2 × 2 identity
matrix. The identity matrix that was in the right-hand 2 × 2 block has at the same time turned into the
inverse.
Therefore, the inverse is  
3 −1
5 −2

7
 You can check that
         
2 −1 3 −1 1 0 3 −1 2 −1 1 0
= and = .
5 −3 5 −2 0 1 5 −2 5 −3 0 1

Of course, there’s a formula for inverting a 2 × 2 matrix. But the procedure above works with (square)
matrices of any size. To explain why this algorithm works, I’ll need to examine the relationship between row
operations and inverses more closely.
You might have discussed lines and planes in 3 dimensions in a calculus course; if so, you probably
saw problems like the next example. The rule of thumb is: You can often find the intersection of things by
solving equations simultaneously.
Example. Find the line of intersection of the planes in R3 :

x + 2y − z = 4 and x − y − z = 3.

Think of the equations as a system of linear equations over R. Write down the coefficient matrix and
row reduce:
→
   
1 2 −1 4 → 1 2 −1 4 1
1 −1 −1 3 r2 → r2 − r1 0 −3 0 −1 r2 → − r2
3
10
 
1 2 −1 4 1 0 −1
" #
→ 3 
1

0 1 0 r1 → r1 − 2r2
 1 
3 0 1 0
3
The last matrix gives the equations
10 1
x−z = , y= .
3 3
10 1
Thus, x = z + and y = . Letting z = t, the parametric solution is
3 3
10 1
x= t+ , y= , z = t.
3 3
These are the parametric equations of the line of intersection.

The solution of simultaneous linear equations dates back 2000 years to the Jiuzhang Suanshu, a collection
of problems compiled in China. The systematic algebraic process of eliminating variables seems to be due to

8
Isaac Newton. Matrix notation developed in the second half of the 1800’s; what we call Gaussian elimination,
applied to matrices, was developed in the first half of the 1900’s by various mathematicians. Grcar [1] contains
a nice historical account.

[1] Joseph Grcar, Mathematics of Gaussian elimination, Notices of the American Mathematical Society,
58(6)(2011), 782–792.

c 2021 by Bruce Ikenaga 9