BASIC MATRIX ALGEBRA

Vector and matrix operations

Data Science Institute, Hasselt University

Contents
Theory . . . . . . . . . . . . . . . . . . . . 1

Examples . . . . . . . . . . . . . . . . . . 6

R-software . . . . . . . . . . . . . . . . . . 7

Exercises . . . . . . . . . . . . . . . . . . . 8

Advanced: Statistical Applications . . 9

Acknowledgment . . . . . . . . . . . . . 9

Solutions Exercises . . . . . . . . . . . . 10

Theory
Vectors and vector arithmetic

In physics, there are a lot of quantities that can not be represented by a single number. For
instance, wind is characterised by both a direction (such as north-west) and a magnitude (say,
50 kilometres per hour). In this respect, one makes use of the concept of vectors to denote both
this direction and magnitude.

A column vector is an ordered collection of numbers that are written in a column, such as
   
  7 0
    8.5
10 1    4 0
, , 5.6 , 3 , 0 .
  
4 2
3.1
1 0
The individual numbers in these columns are called the components of the vector. In the specific
examples above, one can see that the first two vectors have two components, the middle vector
has three components and the final two have four components. In general an n-component
column vector is denoted by  
u1
 u2 
u =  . .
 
 .. 
un
In full analogy, a row vector is an ordered collection of numbers, written in a row. Again here,
the numbers in the vector are referred
to as the components and a general n-component row
vector is denoted by v = v1 , v2 , . . . , vn .

1
Data Science Institute, Hasselt University

Definition
Two row vectors, or two column vectors, are said to be equal if and only if the corresponding
components in the vector are equal. Hence, for the vectors
 

2


a = 5, 2 , b = , c = 5, 2 , d = 2, 5 ,
5
it can be noted that a = c, but that a ̸= b and a ̸= d.

Different operations can be performed using vectors, as discussed in the following sections.

Vector addition
For two n-component column vectors, u and v, their sum u + v is defined by component-wise
addition as follows:      
u1 v1 u1 + v1
 u2   v2   u2 + v2 
u+v = . + . = .
     
. . ..
 .  .  . 
un vn un + vn
Hence, the result is another n-component column vector. In a similar fashion, if u and v are row
vectors, their sum is defined to be



u + v = u1 , u2 , . . . , un + v1 , v2 , . . . , vn


= u1 + v1 , u2 + v2 , . . . , un + vn .

It is important to note that we do not define the addition of vectors unless they are both column
or both row vectors, having the same number of components. Finally, since the order in which
ordinary scalars are added does not affect the result, is is also true that the order in which two
vectors are added does not matter (i.e. u + v = v + u).

Scalar multiplication
The multiplication of a number a with a vector v is defined by the component-wise multiplication
of a with the components of v, i.e.    
v1 av1
 v2   av2 
av = a  .  =  .  ,
   
 ..   .. 
vn avn

for columns vectors and

av = a v1 , v2 , . . . , vn = av1 , av2 , . . . , avn ,

for row vectors. The result av is referred to as a multiple of v.

Linear combinations
The concepts of vector addition and scalar multiplication can be combined to obtain linear
combinations of vectors. More specifically, consider k vectors, denoted by v1 , . . . , vk . Then,
every new vector of the form
a1 v1 + a2 v2 + . . . + ak vk ,
where the ai are real numbers, is called a linear combination of v1 , . . . , vk .

Vector multiplication
Let u be a row vector and v be a column vector, both with the same number of components n.
We define the product u ∗ v to be

u ∗ v = u1 ∗ v1 + u2 ∗ v2 + . . . + un ∗ vn .

2
Data Science Institute, Hasselt University

Matrices and matrix arithmetic

Up to this point, we have discussed row and column vectors, together with their most im-
portant operations. In fact, vectors are a special case of a more general concept: matrices. A
matrix is a rectangular array of numbers, written in the following format:
 
a11 a12 . . . a1n
 a21 a22 . . . a2n 
A= . ..  ,
 
..
 .. . ... . 
am1 am2 ... amn
where the entries aij denote real numbers and m and n are integers. As can be seen above,
m represents the number of rows, while n corresponds to the number of columns. Therefore,
we call it an m × n matrix. In the case that m = n, the matrix is called a square matrix.
With this notation, an n-component row vector is also referred to as a 1 × n matrix, while an
n-component column vector is also referred to as a n × 1 matrix.

Definition
Two matrices with the same shape (i.e. the same number of columns and the same number
of rows) are said to be equal if and only if the corresponding entries are equal.

Again here, different operations can be performed using matrices (see following sections).

Matrix addition
Two matrices with the same shape can be added together by adding the corresponding compo-
nents. More specifically,    
a11 a12 ... a1n b11 b12 ... b1n
 a21 a22 ... a2n   b21 b22 ... b2n 
A+B = . ..  +  ..
   
.. .. .. 
 .. . ... .   . . ... . 
am1 am2 . . . amn bm1 bm2 . . . bmn
 
a11 + b11 a12 + b12 ... a1n + b1n
 a21 + b21 a22 + b22 ... a2n + b2n 
=
 
.. .. .. 
 . . ... . 
am1 + bm1 am2 + bm2 . . . amn + bmn

Scalar multiplication
In full similarity to the multiplication of a vector with a scalar, we define the product of a matrix
A with a scalar k by:    
a11 a12 . . . a1n ka11 ka12 . . . ka1n
 a21 a22 . . . a2n   ka21 ka22 . . . ka2n 
kA = k  . ..  =  .. ..  .
   
.. ..
 .. . ... .   . . ... . 
am1 am2 . . . amn kam1 kam2 . . . kamn

Matrix multiplication
Let A be an m × k matrix and B be an k × n matrix, then the product matrix C = AB is an
m × n matrix whose components areobtained  as follows:
b1j
b2j 

 
cij = ai1 , ai2 , . . . , aik  .  = ai1 ∗ b1j + ai2 ∗ b2j + . . . + aik ∗ bkj .
 .. 
bkj
It is important to note that, in order to multiply two matrices, the number of columns of the
first matrix has to equal the number of rows of the second matrix.

3
Data Science Institute, Hasselt University

Operations on matrices

Matrix transposition
The transpose of a matrix A is a new matrix, denoted by AT , which is formed by turning all the
rows of A into columns for AT . Alternative notations are At and A′ . In a more formal notation,
we can say that the ith row and jth column element of AT is the jth row and ith column element
of A:
[A]Tij = [A]ji .
Of course, it is clear that in case A is an m × n matrix, then AT is an n × m matrix.

Matrix trace
The trace of a matrix, which is only defined for square matrices (i.e. matrices that have the
same number of rows and columns), is calculated by summing over the diagonal elements of the
matrix. More specifically, for an n × n square matrix A:
n
X
tr(A) = a11 + a22 + . . . + ann = aii .
i=1

Some specific properties related to the trace include:

• tr(A + B) = tr(A) + tr(B)
• tr(cA) = c tr(A)
• tr(AT ) = tr(A)
In the above, A and B are square matrices of the same dimension and c is a scalar. Another
nice property is that the trace of a product can be rewritten as the sum of entry-wise products
of elements:
X
tr(X T Y ) = tr(XY T ) = tr(Y T X) = tr(Y X T ) = Xij Yij .
i,j

Matrix determinant
Similar to the trace, the determinant of a matrix is a special number that can be calculated
for square matrices. The determinant of a matrix A is denoted by det(A) or |A| and can be
calculated as follows:
 
a11 a12
• For a 2 × 2 matrix A = , the determinant is given by
a21 a22

|A| = a11 a22 − a12 a21 .

Note that this corresponds to taking the product of the diagonal elements (a11 a22 ) minus
the product of the remaining elements (a12 a21 ).
 
a11 a12 a13
• For a 3 × 3 matrix A = a21 a22 a23 , calculations are given by:
a31 a32 a33

|A| = a11 (a22 a33 − a23 a32 ) − a12 (a21 a33 − a23 a31 ) + a13 (a21 a32 − a22 a31 ).

The formula above may look cumbersome, but there is a very simple pattern. More
specifically, we consider the first row of A. The first element in that row (a11 ) is multiplied
with the determinant of the 2 × 2 matrix that is not in the row or column of that first
element, i.e. containing the elements in the indicated box below
 
a11 a12 a13
 a
 21 a22 a23 

a31 a32 a33
.

4
Data Science Institute, Hasselt University

Similarly, the second element of the first row (a12 ) is multiplied with (-1) and the deter-
minant of the 2 × 2 matrix that is not in the row or column of that second element, i.e.
containing the elements in the indicated box below
 
a11 a12 a13
 a
 21 a22 a23 

a31 a32 a33
.
Finally, the same applies for the final element in the first row (a13 ), which has to be
multiplied by the determinant of the 2×2 matrix that contains the elements in the indicated
box below
 
a11 a12 a13
 a
 21 a22 a23 

a31 a32 a33
.
The three obtained products are then added together.
• For higher dimensional matrices, the same pattern applies. More specifically, we consider
the first row of elements. We multiply each of these elements with the determinant of the
matrix that remains after removing all elements in the row and column of the respective
element in the first row. The products that coincide with an element that is in an even
place in the first row receive a minus sign. Finally, all products are added together. Hence:
* plus a11 times the determinant of the matrix that is not in a11 ’s row or column,
* minus a12 times the determinant of the matrix that is not in a12 ’s row or column,
* plus a13 times the determinant of the matrix that is not in a13 ’s row or column,
* minus a14 times the determinant of the matrix that is not in a14 ’s row or column
* ...

Some interesting properties include:

• det(AT ) = det(A)
• det(cA) = cn det(A)
• det(AB) = det(A) det(B)

where A and B are square matrices of size n and c is a scalar.

Special matrices

One special type of matrices are the so-called symmetric matrices, which are square matrices
that are equal to their transpose. Formally, a matrix A is called symmetric if and only if A = AT .

Another type of matrices that deserves some special attention are the identity matrices, which
are square matrices with ones on the main diagonal and zeros elsewhere. With B being an m × n
matrix, it is a property of matrix multiplication that

Im B = BIn = B,

where Im and In denote m × m and n × n identity matrices, respectively.

5
Data Science Institute, Hasselt University

Examples
Vector addition        
8.5 1 8.5 + 1 9.5
5.6 + 3 = 5.6 + 3 = 8.6
3.1 5 3.1 + 5 8.1

Multiplication of a scalar and a vector

     
4 10 ∗ 4 40
10 ∗ 9 = 10 ∗ 9 = 90
3 10 ∗ 3 30

Linear combinations Below, we show a linear combination of three vectors:

         
5 1 0 5∗5+2∗1+3∗0 27
5 ∗ 8 + 2 ∗ 2 + 3 ∗ 1 = 5 ∗ 8 + 2 ∗ 2 + 3 ∗ 1 = 47
1 3 0 5∗1+2∗3+3∗0 11

Vector multiplication In the following example, we multiply a four-component row vector

with a four-component column vector:
 
7

4
3 = 2 ∗ 7 + 1 ∗ 4 + 4 ∗ 3 + 6 ∗ 1 = 36
2, 1, 4, 6 ∗  

Matrix multiplication In the following example, we will multiply a 2 × 3 matrix with a 3 × 2

matrix. As such, the result will be a 2 × 2 matrix.
 
  1 1    
1 2 3 1∗1+2∗2+3∗3 1∗1+2∗7+3∗4 14 27
∗ 2 7 = =
2 2 2 2∗1+2∗2+2∗3 2∗1+2∗7+2∗4 12 24
3 4

Matrix transpose, trace and determinant Consider the following square matrix
 
1 3 5
A = 2 2 4
8 0 6

We have that the transpose of A equals

 
1 2 8
AT = 3 2 0 ,
5 4 6

the trace of A is given by

tr(A) = 1 + 2 + 6 = 9
and the determinant of A corresponds to

2 4 2 4 2 2
|A| = 1 ∗ −3∗ +5∗ = 1 ∗ 12 − 3 ∗ (−20) + 5 ∗ (−16) = −8
0 6 8 6 8 0

The reader can verify that tr(A) = tr(AT ) and |A| = |AT |.

6
Data Science Institute, Hasselt University

R-software
R is a free software environment for statistical computing and graphics. It compiles and runs on
a wide variety of UNIX platforms, Windows and MacOS. You can download this software for
free from the website (https://www.r-project.org/). You can also use the basic functions of
R without installing the software using R-Fiddle (http://www.r-fiddle.org/). R-Fiddle is a
tool that provides you with a free and powerful environment to write and run R-code right inside
your browser.

Vectors can be defined in R using the concatenate-function: v = c(v1, v2, . . . , vn).

For matrices, a vector of elements can be put in a matrix of n rows and m columns using the
matrix-function: M = matrix(c(v1, v2, . . . , vn),nrow=n,ncol=m).

For vector and matrix addition or multiplication with a scalar, the standard operators “+”
and “∗” can be used. Vector and matrix multiplication are indicated by the operator “%∗%”.
Further interesting functions for matrix operations include dim(), t(), tr() (”psych” package)
and det() to get the dimensions, transpose, trace and determinant of a matrix.

> #### Vectors

>
> # Definition of vector
> v = c(1,4,7,5,8)
> v
[1] 1 4 7 5 8
>
> # Vector addition
> c(8.5,5.6,3.1) + c(1,3,5)
[1] 9.5 8.6 8.1
>
> # Multiplication of a scalar and a vector
> 10 * c(4,9,3)
[1] 40 90 30
>
> # Linear combinations
> 5 * c(5,8,1) + 2 * c(1,2,3) + 3 * c(0,1,0)
[1] 27 47 11
>
> # Vector multiplication
> c(2,1,4,6) %*% c(7,4,3,1)
[,1]
[1,] 36
>
>
> #### Matrices
>
> # Definition of a matrix
> A = matrix(c(1,2,8,3,2,0,5,4,6),nrow=3,ncol=3)
> A
[,1] [,2] [,3]
[1,] 1 3 5
[2,] 2 2 4
[3,] 8 0 6
>
> # Dimension of a matrix
> dim(A) # displays first number of rows, next number of columns
[1] 3 3
>

7
Data Science Institute, Hasselt University

> # Matrix multiplication

> M1 = matrix(c(1,2,2,2,3,2),nrow=2,ncol=3)
> M2 = matrix(c(1,2,3,1,7,4),nrow=3,ncol=2)
> M1%*%M2
[,1] [,2]
[1,] 14 27
[2,] 12 24
>
> # Matrix transpose
> t(A)
[,1] [,2] [,3]
[1,] 1 2 8
[2,] 3 2 0
[3,] 5 4 6
>
> # Matrix trace
> library(psych)
> tr(A)
[1] 9
>
> # Matrix determinant
> det(A)
[1] -8

Exercises
Exercises indicated with a ⋆ are preferable solved using R or R-Fiddle. Exercises indicated with a
⋆ between brackets can be solved using R or R-Fiddle, but a manual calculation is recommended.

1. Compute the quantities below for the following vectors. (⋆)

   
3 2


u = −2 , v = 4 , w = −7, 0, 3 , x = 2, 1, −1
5 1

a) u+v e) (w + x) ∗ (u − v)
b) 3w + 4x
f) (u − v) ∗ (w + x)
c) u∗x
d) x∗u g) u ∗ w + 3(v ∗ x)

2. Perform the calculations below using the following matrices. (⋆)

 
1 0  
1 1 −1
A = 0 1 , B=
1 3 −5
2 3

a) AB e) BA
T
b) (AB) f) (BA)T
c) tr(AB) g) tr(BA)
d) det(AB) h) det(BA)

8
Data Science Institute, Hasselt University

3. Given the matrix  

λ λ+1 λ+2
L = 2 4 6 
2 3 2
a) Calculate det(L).
b) For which value of λ is det(L) = 0?
 
1 0 −1
c) Calculate the product of 2 −1 0  and matrix L.
2 0 −1
 
a 0
4. Given the matrix A = calculate the values of a for which the following expression
0 2
2
holds: A − 3 · A + 2 · I = 0.
5. Let M be a 3x3 matrix such that mij = i + j. Compute det(M ).

Advanced: Statistical Applications

Vectors and matrices can be seen as one- and two-dimensional arrays of numbers and are often
used to represent data, e.g. the outcome of an experiment with multiple observations (given
as rows) for multiple variables (given as columns). Hence, vector and matrix arithmetics can
be needed to process the data prior to analysis. For example, an ecologist measures the pH of
ten Michigan lakes at two points in the summer. Vector addition and scalar multiplication are
needed to study the mean pH in each lake.

More advanced examples of applications of matrix operations can be found in:

• Linear regression: Matrix multiplication helps in calculating the optimal weights for
features, minimizing the difference between predicted and actual values.
• Principal Component Analysis (PCA): By performing matrix operations on the data
covariance matrix, PCA extracts important features and reduces dimensionality.
• Machine learning: Matrix multiplication plays a pivotal role in transforming data, ap-
plying weights to features, and calculating predictions. Moreover, neural networks, the
heart of deep learning, heavily rely on matrix operations to propagate information, update
weights, and make predictions.

Acknowledgment
This contribution was written by S. Abrams, M. Aerts, L. Garcia Barrado, N. Cosemans, S.
Jaspers & Y. Vandendijck.

Last update: January, 2025

Elaborate reading material includes

• Chapter “Matrix Arithmetic” in “Fundamentals of Matrix Algebra” by Hartman G. (3th
edition in 2011)
• Chapter “Operations on Matrices” in “Fundamentals of Matrix Algebra” by Hartman G.
(3th edition in 2011)

9
Data Science Institute, Hasselt University

Solutions Exercises
      
3 2 3+2 5
1. a) −2 + 4 = −2 + 4 = 2
5 1 5+1 6

b) 3 ∗ −7, 0, 3 + 4 ∗ 2, 1, −1 = −21, 0, 9 + 8, 4, −4 = −13, 4, 5

    
3
3∗2 3∗1 3 ∗ −1 6 3 −3
c) −2 ∗ 2, 1, −1 = −2 ∗ 2 −2 ∗ 1 −2 ∗ −1 = −4 −2 2
5 5∗2 5∗1 5 ∗ −1 10 5 −5

 

3
d) 2, 1, −1 ∗ −2 = 2 ∗ 3 + 1 ∗ (−2) + (−1) ∗ 5 = 6 − 2 − 5 = −1
5

     



3 2
1
e) −7, 0, 3 + 2, 1, −1 ∗ −2 − 4 = −5, 1, 2 ∗ −6 = −3
5 1 4

       
3 2


1
−5 1 2
f) −2 − 4∗ −7, 0, 3 + 2, 1, −1 = −6∗ −5, 1, 2 =  30 −6 −12
5 1 4 −20 4 8

   
3
2

g) −2 ∗ −7, 0, 3 + 3 ∗ 4 ∗ 2, 1, −1
5 1
     
−21 0 9 4 2 −2 −9 6 3
=  14 0 −6 + 3 ∗ 8 4 −4 =  38 12 −18
−35 0 15 2 1 −1 −29 3 12

   
1 0   1 1 −1
1 1 −1
2. a) 0 1 ∗
 = 1 3 −5 
1 3 −5
2 3 5 11 −17

 T  
1 1 −1 1 1 5
b) 1 3 −5  =  1 3 11 
5 11 −17 −1 −5 −17

 
1 1 −1
c) tr1 3 −5  = 1 + 3 − 17 = −13
5 11 −17

d)
 
1 1 −1
3 −5 1 −5 1 3
det 1 3 −5  = 1 ∗ −1∗ + (−1) ∗
11 −17 5 −17 5 11
5 11 −17
= 1 ∗ 4 − 1 ∗ 8 + (−1) ∗ (−4) = 0

10
Data Science Institute, Hasselt University

 
  1 0  
1 1 −1 −1 −2
e) ∗ 0 1 =

1 3 −5 −9 −12
2 3

 T  
−1 −2 −1 −9
f) =
−9 −12 −2 −12

 
−1 −2
g) tr = −1 − 12 = −13
−9 −12

 
−1 −2
h) det = (−1) ∗ (−12) − (−9) ∗ (−2) = −6
−9 −12

3. a)
 
λ λ+1 λ+2
4 6 2 6 2 4
det  2 4 6 =λ∗ − (λ + 1) ∗ + (λ + 2) ∗
3 2 2 2 2 3
2 3 2
= λ ∗ (−10) − (λ + 1) ∗ (−8) + (λ + 2) ∗ (−2)
= −4λ + 4

b) −4λ + 4 = 0 ⇔ λ = 1

     
1 0 −1 λ λ+1 λ+2 λ−2 λ−2 λ
c) 2 −1 0  ∗ 2 4 6  = 2λ − 2 2λ − 2 2λ − 2
2 0 −1 2 3 2 2λ − 2 2λ − 1 2λ + 2

         2 
a 0 a 0 a 0 1 0 a − 3a + 2 0
4. ∗ −3∗ +2 =
0 2 0 2 0 2 0 1 0 0

⇒ a2 − 3a + 2 = 0 ⇔ a = {1, 2}
 
2 3 4
5. A = 3 4 5 ⇒ det(A) = 2 ∗ (−1) − 3 ∗ (−2) + 4 ∗ (−1) = 0
4 5 6

11