Analysis

y of Variance and
Design of Experiments
Experiments--I
MODULE - I

LECTURE - 1
SOME RESULTS ON LINEAR
ALGEBRA, MATRIX THEORY
AND DISTRIBUTIONS
Dr. Shalabh
D
Department
t t off M
Mathematics
th ti and
d Statistics
St ti ti
Indian Institute of Technology Kanpur
 2

We need some basic knowledge to understand the topics in analysis of variance.

Vectors

A vector Y is an ordered n-tuple of real numbers. A vector can be expressed as row vector or a column vector as

⎛ y1 ⎞
⎜ ⎟
y
Y =⎜ 2⎟ is a column vector of order n × 1 .
⎜# ⎟
⎜ ⎟
⎝ yn ⎠
and

Y ' = ( y1 , y2 ,..., yn ) is a row vector of order 1× n.

If all yi = 0 for all i = 1,2,…,n then Y ' = (0, 0,..., 0) is called the null vector.

If ⎛ x1 ⎞ ⎛ y1 ⎞ ⎛ z1 ⎞
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
⎜ x2 ⎟ ⎜ y2 ⎟ z
X= , Y= , Z =⎜ 2⎟
⎜# ⎟ ⎜# ⎟ ⎜# ⎟
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
⎝ xn ⎠ ⎝ yn ⎠ ⎝ zn ⎠
then
⎛ x1 + y2 ⎞ ⎛ ky1 ⎞
⎜ ⎟ ⎜ ⎟
⎜ x2 + y2 ⎟ ⎜ k 2⎟
ky
X +Y = , kY =
⎜# ⎟ ⎜# ⎟
⎜ ⎟ ⎜ ⎟
⎝ xn + yn ⎠ ⎝ kyn ⎠
 3

X + (Y + Z ) = ( X + Y ) + Z
X '(Y + Z ) = X ' Y + X ' Z
k ( X ' Y ) = ( kX ) ' Y = X '( kY )
k ( X + Y ) = kX + kY
X ' Y = x1 y1 + x2 y2 + ... + xn yn

where k is a scalar.

Orthogonal
g vectors

Two vectors X and Y are said to be orthogonal if X ' Y = Y ' X = 0.

The null vector is orthogonal to every vector X and is the only such vector.

Linear combination

if x1 , x2 ,..., xm are m vectors of same order and k1 , k2 ,..., km are scalars, Then
m
t = ∑ ki xi
i =1

is called the linear combination of x1 , x2 ,..., xm .

 4

Linear independence

If X 1 , X 2 ,..., X m are m vectors then they are said to be linearly independent if there exist scalars k1 , k 2 ,..., k m
such that
m

∑k X
i =1
i i = 0 ⇒ ki = 0 for all i = 1,2,…,m.
m
If there exist k1 , k 2 ,..., k m with at least one k i to be nonzero, such that ∑kx
i =1
i i = 0 then x1 , x2 ,..., xm are said to

be linearly dependent
dependent.

ƒ Any set of vectors containing the null vector is linearly dependent.

ƒ Any set of non-null
non null pair
pair-wise
wise orthogonal vectors is linearly independent
independent.
ƒ If m > 1 vectors are linearly dependent, it is always possible to express at least one of them as a linear
combination of the others.
 5

Linear function

Let K = (k1 , k 2 ,..., k m ) ' be a m × 1 vector of scalars and X = ( x1 , x2 ,..., xm ) be a m × 1 vector of variables, then
m
K 'Y = ∑i =1
k i y i is called a linear function or linear form. The vector K is called the coefficient vector.

example mean of x1 , x2 ,..., xm can be expressed as

For example,

⎛ x1 ⎞
⎜ ⎟
1 m x
x = ∑ xi = (1,1,...,1) ⎜ 2 ⎟ = 1'm X
1 1
m i =1 m ⎜# ⎟ m
⎜ ⎟
⎝ xm ⎠
where 1m is a m ×1 vector of all elements unity.
'

Contrast
m m
The linear function K ' X = ∑k x
i =1
i i
is called a contrast in x1 , x2 ,..., xm if ∑k
i =1
i = 0.

For example, the linear functions

x1 x
x1 − x2 , 2 x1 − 3 x2 + x3 , − x2 + 3
2 3
are contrasts.
m
1 m
• A linear function K ' X is a contrast if and only if it is orthogonal to a linear function ∑ xi or to the linear function x =
i =1
∑ xi .
m i =1
• Contrasts x1 − x2 , x1 − x3 ,..., x1 − x j are linearly independent for all j = 2, 3,…, m.
i=

• Every contrast in x1 , x2 ,..., xn in can be written as a linear combination of (m - 1) contrasts x1 − x2 , x1 − x3 ,..., x1 − xm .

 6

Matrix

A matrix is a rectangular array of real numbers. For example,

⎛ a11 a12 ... a1n ⎞

⎜ ⎟
⎜ a21 a22 ... a2 n ⎟
⎜# # # ⎟
⎜ ⎟
⎝ am1 am 2 ... amn ⎠

is a matrix of order m × n with m rows and n columns.

• If m = n, then A is called a square matrix.

• If aij = 0, i ≠ j , m = n, then A is a diagonal matrix and is denoted as A = diag ( a11 , a22 ,..., ann ).
• If m = n, (square matrix) and aij = 0 for i > j , then A is called an upper triangular matrix. On the other hand if
m = n, and aij = 0 for i < j then A is called a lower triangular matrix.
• If A is a m × n matrix, then the matrix obtained by writing the rows of A and columns of A as columns of A and

rows of A respectively, is called the transpose of a matrix A and is denoted as A ' .

• If A = A ' then A is a symmetric matrix.
matrix
• If A = − A ' then A is skew symmetric matrix.
• A matrix whose all elements are equal to zero is called as null matrix.

• An identity matrix is a square matrix of order p whose diagonal elements are unity (ones) and all the off diagonal
elements are zero. It is denotes as I p.
 7

• If A and B are matrices of order m × n then

( A + B ) ' = A '+ B '.'

• If A and B are the matrices of order m x n and n x p respectively and k is any scalar, then
( AB ) ' = B ' A '

( kA) B = A( kB ) = k ( AB ) = kAB.

• If the orders of matrices A is m x n, B is n x p and C is n x p then

A( B + C ) = AB + AC.

• If the orders of matrices A is m x n, B is n x p and C is p x q then

( AB )C = A( BC ).

• If A is the matrix of order m x n then

I m A = AI n = A.
 8

Trace of a matrix

The trace of n x n matrix A, denoted as tr(A) or trace(A) is defined to be the sum of all the diagonal elements of A,
n
i.e., tr ( A) = ∑a .
i =1
ii

ƒ If A is of order m × n and B is of order n × m , then

. tr ( AB ) = tr ( BA).

ƒ If A is n x n matrix and P is any nonsingular n x n matrix then

tr ( A) = tr ( P −1 AP ).
)

If P is an orthogonal matrix than tr ( A) = tr ( P ' AP ).

• If A and B are n x n matrices, a and b are scalars then

. tr ( aA + bB ) = a tr ( A) + b tr ( B )

ƒ If A is a m x n matrix, then
n n
tr ( A ' A) = tr ( AA ') = ∑∑ aij2
j =1 i =1
and

tr ( A ' A) = tr ( AA ') = 0 if and only if A = 0.

ƒ If A is n x n matrix then
tr ( A ') = trA
.
 9

Rank
a o of a matrix
at

The rank of a matrix A of m × n is the number of linearly independent rows in A.

Let B be another matrix of order n × q.

• A square matrix of order m is called non-singular if it has a full rank.

• rank ( AB) ≤ min ( rank ( A), rank ( B)).
• rank ( A + B ) ≤ rank ( A) + rank ( B )).

• Rank of A is equal to the maximum order of all nonsingular square sub-matrices of A.

• rank ( AA ') = rank ( A ' A) = rank ( A) = rank ( A ').
• A is of full row rank if rank(A) = m < n.
• A is of full column rank if rank(A) = n < m.
 10

Inverse of matrix

The inverse of a square matrix A of order m, is a square matrix of order m, denoted as A−1, such that A−1 A = AA−1 = I m .
The inverse of A exists if and only if A is non singular.
−1 −1
• ( A ) = A.
• If A is non singular, then ( A ') −1 = ( A−1 ) '.
• If A and B are non-singular matrices of same order, then their product, if defined, is also nonsingular and

( AB) −1 = B −1 A−1.

Idempotent matrix

A square matrix A is called idempotent if A2 = AA = A.

If A is an n × n idempotent matrix with rank ( A) = r ≤ n. Then

• the eigenvalues of A are 1 or 0.

• trace ( A ) = rank ( A ) = r .
• If A is of full rank n, then A = I n .
• If A and B are idempotent and AB = BA, then AB is also idempotent.
• If A is idempotent then (I – A) is also idempotent and A(I - A) = (I - A) A = 0.