Chapter 3.

Analytic Geometry

0
Equivalent to 2 slots. Lecturers can use extra materials for other slots in the chater
FPT University Chapter 3. Analytic Geometry 1 / 25
 Chapter 3. Analytic Geometry

1 3.1 Norm
2 3.2 Inner Products
3 3.3 Lengths and Distances
4 3.4 Angles and Orthogonality
5 3.5 Orthonormal Basis
6 3.6 Orthogonal Complement
7 3.8 Orthogonal Projections
FPT University Chapter 3. Analytic Geometry 2 / 25
3.1 Norm

Let V be a vector space.

Definition
A norm on V is a non-negative function

k·k : V → R

which assigns each vector x its length kxk ∈ R,

FPT University Chapter 3. Analytic Geometry 3 / 25

3.1 Norm

Let V be a vector space.

Definition
A norm on V is a non-negative function

k·k : V → R

which assigns each vector x its length kxk ∈ R, such that for all k ∈ R and
x,y ∈ V the following hold:

FPT University Chapter 3. Analytic Geometry 3 / 25

3.1 Norm

Let V be a vector space.

Definition
A norm on V is a non-negative function

k·k : V → R

which assigns each vector x its length kxk ∈ R, such that for all k ∈ R and
x,y ∈ V the following hold:
Absolute homogeneous: kkxk = |k| kxk

FPT University Chapter 3. Analytic Geometry 3 / 25

3.1 Norm

Let V be a vector space.

Definition
A norm on V is a non-negative function

k·k : V → R

which assigns each vector x its length kxk ∈ R, such that for all k ∈ R and
x,y ∈ V the following hold:
Absolute homogeneous: kkxk = |k| kxk
Triangle inequality : kx + yk ≤ kxk + kyk

FPT University Chapter 3. Analytic Geometry 3 / 25

3.1 Norm

Let V be a vector space.

Definition
A norm on V is a non-negative function

k·k : V → R

which assigns each vector x its length kxk ∈ R, such that for all k ∈ R and
x,y ∈ V the following hold:
Absolute homogeneous: kkxk = |k| kxk
Triangle inequality : kx + yk ≤ kxk + kyk
→
−
Positive definite: kxk ≥ 0 and kxk = 0 if and only if x = 0 .

FPT University Chapter 3. Analytic Geometry 3 / 25

3.1 Norm

Let V be a vector space.

Definition
A norm on V is a non-negative function

k·k : V → R

which assigns each vector x its length kxk ∈ R, such that for all k ∈ R and
x,y ∈ V the following hold:
Absolute homogeneous: kkxk = |k| kxk
Triangle inequality : kx + yk ≤ kxk + kyk
→
−
Positive definite: kxk ≥ 0 and kxk = 0 if and only if x = 0 .

FPT University Chapter 3. Analytic Geometry 3 / 25

Example
Let p be a positive number. The lp -norm k·kp in Rn :
v
u n
uX T
∈ Rn .

kxkp = t
p
|xi |p for any x = x1 · · · xn
i=1

n
P
For p = 1: The Manhattan norm l1 : kxk1 = |xi |.
i=1

FPT University Chapter 3. Analytic Geometry 4 / 25

Example
Let p be a positive number. The lp -norm k·kp in Rn :
v
u n
uX T
∈ Rn .

kxkp = t
p
|xi |p for any x = x1 · · · xn
i=1

Pn
For p = 1: The Manhattan norm l1 : kxk1 = |xi |.
i=1
s
n
xi2 .
P
For p = 2: The Euclidean norm l2 : kxk2 =
i=1

FPT University Chapter 3. Analytic Geometry 4 / 25

Example
Let p be a positive number. The lp -norm k·kp in Rn :
v
u n
uX T
∈ Rn .

kxkp = t
p
|xi |p for any x = x1 · · · xn
i=1

Pn
For p = 1: The Manhattan norm l1 : kxk1 = |xi |.
i=1
s
n
xi2 .
P
For p = 2: The Euclidean norm l2 : kxk2 =
i=1

Example
The l∞ -norm or maximum norm in Rn is the limit of lp -norm when
p → ∞:

FPT University Chapter 3. Analytic Geometry 4 / 25

Example
Let p be a positive number. The lp -norm k·kp in Rn :
v
u n
uX T
∈ Rn .

kxkp = t
p
|xi |p for any x = x1 · · · xn
i=1

Pn
For p = 1: The Manhattan norm l1 : kxk1 = |xi |.
i=1
s
n
xi2 .
P
For p = 2: The Euclidean norm l2 : kxk2 =
i=1

Example
The l∞ -norm or maximum norm in Rn is the limit of lp -norm when
p → ∞:

kxk∞ = lim kxkp = max{|x1 |, · · · , |xn |}.

p→∞

FPT University Chapter 3. Analytic Geometry 4 / 25

Example
Let p be a positive number. The lp -norm k·kp in Rn :
v
u n
uX T
∈ Rn .

kxkp = t
p
|xi |p for any x = x1 · · · xn
i=1

Pn
For p = 1: The Manhattan norm l1 : kxk1 = |xi |.
i=1
s
n
xi2 .
P
For p = 2: The Euclidean norm l2 : kxk2 =
i=1

Example
The l∞ -norm or maximum norm in Rn is the limit of lp -norm when
p → ∞:

kxk∞ = lim kxkp = max{|x1 |, · · · , |xn |}.

p→∞

FPT University Chapter 3. Analytic Geometry 4 / 25

Example
Denote by C [a, b] the vector space of all continuous functions defined on
[a, b]. The maximum norm in C [a, b]:

kf k = max |f (x)|.
a≤x≤b

FPT University Chapter 3. Analytic Geometry 5 / 25

Example
Denote by C [a, b] the vector space of all continuous functions defined on
[a, b]. The maximum norm in C [a, b]:

kf k = max |f (x)|.
a≤x≤b

FPT University Chapter 3. Analytic Geometry 5 / 25

Example
Denote by Rm×n the vector space of all matrices with the matrix addition
and matrix scalar multiplication.
The Euclidean norm on Rm×n :
 1/2
Xm Xn
aij2  , for any A = aij ∈ Rm×n .
 
kAk = 
i=1 j=1

FPT University Chapter 3. Analytic Geometry 6 / 25

Example
Denote by Rm×n the vector space of all matrices with the matrix addition
and matrix scalar multiplication.
The Euclidean norm on Rm×n :
 1/2
Xm Xn
aij2  , for any A = aij ∈ Rm×n .
 
kAk = 
i=1 j=1

The spectral norm on Rm×n (see Definition 4.23):

kAxk2
kAk2 = max−
→
= max kAxk2 .
Rn 3x6= 0 kxk2 Rn 3x,kxk2 =1

FPT University Chapter 3. Analytic Geometry 6 / 25

Example
Denote by Rm×n the vector space of all matrices with the matrix addition
and matrix scalar multiplication.
The Euclidean norm on Rm×n :
 1/2
Xm Xn
aij2  , for any A = aij ∈ Rm×n .
 
kAk = 
i=1 j=1

The spectral norm on Rm×n (see Definition 4.23):

kAxk2
kAk2 = max−
→
= max kAxk2 .
Rn 3x6= 0 kxk2 Rn 3x,kxk2 =1

FPT University Chapter 3. Analytic Geometry 6 / 25

3.2 Inner Products

Definition
The dot product or scalar product on Rn is given by
n
X
xT y = xi yi .
i=1

Definition
Let V be a vector space. A bilinear mapping or bilinear form B on V is a
function B : V × V → R that is linear in each argument separately:

FPT University Chapter 3. Analytic Geometry 7 / 25

3.2 Inner Products

Definition
The dot product or scalar product on Rn is given by
n
X
xT y = xi yi .
i=1

Definition
Let V be a vector space. A bilinear mapping or bilinear form B on V is a
function B : V × V → R that is linear in each argument separately:
B(kx + hy, z) = kB(x, z) + hB(y, z)

FPT University Chapter 3. Analytic Geometry 7 / 25

3.2 Inner Products

Definition
The dot product or scalar product on Rn is given by
n
X
xT y = xi yi .
i=1

Definition
Let V be a vector space. A bilinear mapping or bilinear form B on V is a
function B : V × V → R that is linear in each argument separately:
B(kx + hy, z) = kB(x, z) + hB(y, z)
B(x, ky + hz) = kB(x, y) + hB(x, z)

FPT University Chapter 3. Analytic Geometry 7 / 25

3.2 Inner Products

Definition
The dot product or scalar product on Rn is given by
n
X
xT y = xi yi .
i=1

Definition
Let V be a vector space. A bilinear mapping or bilinear form B on V is a
function B : V × V → R that is linear in each argument separately:
B(kx + hy, z) = kB(x, z) + hB(y, z)
B(x, ky + hz) = kB(x, y) + hB(x, z)
for any x, y, z ∈ V and any k, h ∈ R.

FPT University Chapter 3. Analytic Geometry 7 / 25

3.2 Inner Products

Definition
The dot product or scalar product on Rn is given by
n
X
xT y = xi yi .
i=1

Definition
Let V be a vector space. A bilinear mapping or bilinear form B on V is a
function B : V × V → R that is linear in each argument separately:
B(kx + hy, z) = kB(x, z) + hB(y, z)
B(x, ky + hz) = kB(x, y) + hB(x, z)
for any x, y, z ∈ V and any k, h ∈ R.

FPT University Chapter 3. Analytic Geometry 7 / 25

Example
The dot product is a bilinear mapping on Rn .

Example
Show that the mapping B : R2 × R2 → R defined by

B(x, y) = x1 y1 − 2x1 y2 + 3x2 y2

is a bilinear mapping on R2 .

FPT University Chapter 3. Analytic Geometry 8 / 25

Example
The dot product is a bilinear mapping on Rn .

Example
Show that the mapping B : R2 × R2 → R defined by

B(x, y) = x1 y1 − 2x1 y2 + 3x2 y2

is a bilinear mapping on R2 .

Example
Show that B : C [a, b] × C [a, b] → R defined by
Z b
B(f , g ) = f (x)g (x)dx
a

is a bilinear mapping on C [a, b].

FPT University Chapter 3. Analytic Geometry 8 / 25

Example
The dot product is a bilinear mapping on Rn .

Example
Show that the mapping B : R2 × R2 → R defined by

B(x, y) = x1 y1 − 2x1 y2 + 3x2 y2

is a bilinear mapping on R2 .

Example
Show that B : C [a, b] × C [a, b] → R defined by
Z b
B(f , g ) = f (x)g (x)dx
a

is a bilinear mapping on C [a, b].

FPT University Chapter 3. Analytic Geometry 8 / 25

Definition
A bilinear mapping B : V × V → R is called
symmetric if B(x, y) = B(y, x) for any x, y ∈ V .

FPT University Chapter 3. Analytic Geometry 9 / 25

Definition
A bilinear mapping B : V × V → R is called
symmetric if B(x, y) = B(y, x) for any x, y ∈ V .
→
−
positive definite if B(x, x) > 0 for any V 3 x 6= 0 .

FPT University Chapter 3. Analytic Geometry 9 / 25

Definition
A bilinear mapping B : V × V → R is called
symmetric if B(x, y) = B(y, x) for any x, y ∈ V .
→
−
positive definite if B(x, x) > 0 for any V 3 x 6= 0 .

Example
The dot product on Rn is both symmetric and positive definite.

FPT University Chapter 3. Analytic Geometry 9 / 25

Definition
A bilinear mapping B : V × V → R is called
symmetric if B(x, y) = B(y, x) for any x, y ∈ V .
→
−
positive definite if B(x, x) > 0 for any V 3 x 6= 0 .

Example
The dot product on Rn is both symmetric and positive definite.

Example
The bilinear mapping B : R2 × R2 → R defined by:

B(x, y) = x1 y1 − 2x2 y2

FPT University Chapter 3. Analytic Geometry 9 / 25

Definition
A bilinear mapping B : V × V → R is called
symmetric if B(x, y) = B(y, x) for any x, y ∈ V .
→
−
positive definite if B(x, x) > 0 for any V 3 x 6= 0 .

Example
The dot product on Rn is both symmetric and positive definite.

Example
The bilinear mapping B : R2 × R2 → R defined by:

B(x, y) = x1 y1 − 2x2 y2

is symmetric but not positive definite.

FPT University Chapter 3. Analytic Geometry 9 / 25

Definition
A bilinear mapping B : V × V → R is called
symmetric if B(x, y) = B(y, x) for any x, y ∈ V .
→
−
positive definite if B(x, x) > 0 for any V 3 x 6= 0 .

Example
The dot product on Rn is both symmetric and positive definite.

Example
The bilinear mapping B : R2 × R2 → R defined by:

B(x, y) = x1 y1 − 2x2 y2

is symmetric but not positive definite.

FPT University Chapter 3. Analytic Geometry 9 / 25

Definition
A positive definite, symmetric bilinear mapping B : V × V → R is
called an inner product on V . We typically write hx, yi instead of
B(x, y).

FPT University Chapter 3. Analytic Geometry 10 / 25

Definition
A positive definite, symmetric bilinear mapping B : V × V → R is
called an inner product on V . We typically write hx, yi instead of
B(x, y).
The pair (V , h·, ·i) is called an inner product space or (real) vector
space with inner product.

FPT University Chapter 3. Analytic Geometry 10 / 25

Definition
A positive definite, symmetric bilinear mapping B : V × V → R is
called an inner product on V . We typically write hx, yi instead of
B(x, y).
The pair (V , h·, ·i) is called an inner product space or (real) vector
space with inner product.If we use the dot product, we call (Rn , h·, ·i)
a Euclidean vector space.

FPT University Chapter 3. Analytic Geometry 10 / 25

Definition
A positive definite, symmetric bilinear mapping B : V × V → R is
called an inner product on V . We typically write hx, yi instead of
B(x, y).
The pair (V , h·, ·i) is called an inner product space or (real) vector
space with inner product.If we use the dot product, we call (Rn , h·, ·i)
a Euclidean vector space.

FPT University Chapter 3. Analytic Geometry 10 / 25

Example
Consider V = R2 . Define

hx, yi = x1 y1 − x1 y2 + x2 y1 + x2 y2 .

Show that h·, ·i is an inner product in R2 .

Example
If A, B are matrices in Rm×n . Define

hA, Bi = tr(AB T ),

where tr is the trace operator (see Chapter 4).

FPT University Chapter 3. Analytic Geometry 11 / 25

Example
Consider V = R2 . Define

hx, yi = x1 y1 − x1 y2 + x2 y1 + x2 y2 .

Show that h·, ·i is an inner product in R2 .

Example
If A, B are matrices in Rm×n . Define

hA, Bi = tr(AB T ),

where tr is the trace operator (see Chapter 4).Show that h·, ·i is an inner
product in Rm×n .

FPT University Chapter 3. Analytic Geometry 11 / 25

Example
Consider V = R2 . Define

hx, yi = x1 y1 − x1 y2 + x2 y1 + x2 y2 .

Show that h·, ·i is an inner product in R2 .

Example
If A, B are matrices in Rm×n . Define

hA, Bi = tr(AB T ),

where tr is the trace operator (see Chapter 4).Show that h·, ·i is an inner
product in Rm×n .

FPT University Chapter 3. Analytic Geometry 11 / 25

Definition
A symmetric matrix A ∈ Rn×n is called
positive definite if

xT Ax > 0, for all non-zero vectors x ∈ Rn .

FPT University Chapter 3. Analytic Geometry 12 / 25

Definition
A symmetric matrix A ∈ Rn×n is called
positive definite if

xT Ax > 0, for all non-zero vectors x ∈ Rn .

positive semidefinite if

xT Ax ≥ 0, for all vectors x ∈ Rn .

FPT University Chapter 3. Analytic Geometry 12 / 25

Definition
A symmetric matrix A ∈ Rn×n is called
positive definite if

xT Ax > 0, for all non-zero vectors x ∈ Rn .

positive semidefinite if

xT Ax ≥ 0, for all vectors x ∈ Rn .

Example
Consider the matrices
   
2 1 1 2
A= ,B =
1 3 2 3

FPT University Chapter 3. Analytic Geometry 12 / 25

Definition
A symmetric matrix A ∈ Rn×n is called
positive definite if

xT Ax > 0, for all non-zero vectors x ∈ Rn .

positive semidefinite if

xT Ax ≥ 0, for all vectors x ∈ Rn .

Example
Consider the matrices
   
2 1 1 2
A= ,B =
1 3 2 3

Show that A is positive definite but B is not.

FPT University Chapter 3. Analytic Geometry 12 / 25

Definition
A symmetric matrix A ∈ Rn×n is called
positive definite if

xT Ax > 0, for all non-zero vectors x ∈ Rn .

positive semidefinite if

xT Ax ≥ 0, for all vectors x ∈ Rn .

Example
Consider the matrices
   
2 1 1 2
A= ,B =
1 3 2 3

Show that A is positive definite but B is not.

FPT University Chapter 3. Analytic Geometry 12 / 25

Consider an n-dimensional vector space V with an inner product
h·, ·i : V × V → R and an ordered basis B = {b1 , · · · , bn } of V . Any
vectors x, y ∈ V can be uniquely written as
n
X n
X
x= xi bi , y= yj bj
i=1 j=1

Hence
n X
X n
hx, yi = xi hbi , bj iyj = x̂T Aŷ,
i=1 j=1

FPT University Chapter 3. Analytic Geometry 13 / 25

Consider an n-dimensional vector space V with an inner product
h·, ·i : V × V → R and an ordered basis B = {b1 , · · · , bn } of V . Any
vectors x, y ∈ V can be uniquely written as
n
X n
X
x= xi bi , y= yj bj
i=1 j=1

Hence
n X
X n
hx, yi = xi hbi , bj iyj = x̂T Aŷ,
i=1 j=1
   
where A = Aij = hbi , bj i and x̂, ŷ are the coordinates of x and y w.r.t
the basis B.

FPT University Chapter 3. Analytic Geometry 13 / 25

Consider an n-dimensional vector space V with an inner product
h·, ·i : V × V → R and an ordered basis B = {b1 , · · · , bn } of V . Any
vectors x, y ∈ V can be uniquely written as
n
X n
X
x= xi bi , y= yj bj
i=1 j=1

Hence
n X
X n
hx, yi = xi hbi , bj iyj = x̂T Aŷ,
i=1 j=1
   
where A = Aij = hbi , bj i and x̂, ŷ are the coordinates of x and y w.r.t
the basis B.
We can see A is symmetric, positive definite.

FPT University Chapter 3. Analytic Geometry 13 / 25

Consider an n-dimensional vector space V with an inner product
h·, ·i : V × V → R and an ordered basis B = {b1 , · · · , bn } of V . Any
vectors x, y ∈ V can be uniquely written as
n
X n
X
x= xi bi , y= yj bj
i=1 j=1

Hence
n X
X n
hx, yi = xi hbi , bj iyj = x̂T Aŷ,
i=1 j=1
   
where A = Aij = hbi , bj i and x̂, ŷ are the coordinates of x and y w.r.t
the basis B.
We can see A is symmetric, positive definite.

FPT University Chapter 3. Analytic Geometry 13 / 25

Theorem
For an n-dimensional vector space V and an ordered basis B of V , it
holds that h·, ·i : V × V → R is an inner product if and only if there exists
a symmetric, positive definite matrix A ∈ Rn×n with

hx, yi = x̂T Aŷ.

FPT University Chapter 3. Analytic Geometry 14 / 25

3.3 Lengths and Distances

Definition
Consider an inner product space (V , h·, ·i). Then
p
The norm induced by the inner product kxk := hx, xi

FPT University Chapter 3. Analytic Geometry 15 / 25

3.3 Lengths and Distances

Definition
Consider an inner product space (V , h·, ·i). Then
p
The norm induced by the inner product kxk := hx, xi
p
d(x, y) := kx − yk = hx − y, x − yi is called the distance between x
and y in V .

FPT University Chapter 3. Analytic Geometry 15 / 25

3.3 Lengths and Distances

Definition
Consider an inner product space (V , h·, ·i). Then
p
The norm induced by the inner product kxk := hx, xi
p
d(x, y) := kx − yk = hx − y, x − yi is called the distance between x
and y in V .

Theorem
For an inner product vector space (V , h·, ·i) the induced norm k·k satisfies
the Cauchy-Schwarz inequality:

FPT University Chapter 3. Analytic Geometry 15 / 25

3.3 Lengths and Distances

Definition
Consider an inner product space (V , h·, ·i). Then
p
The norm induced by the inner product kxk := hx, xi
p
d(x, y) := kx − yk = hx − y, x − yi is called the distance between x
and y in V .

Theorem
For an inner product vector space (V , h·, ·i) the induced norm k·k satisfies
the Cauchy-Schwarz inequality:

|hx, y| ≤ kxk kyk .

FPT University Chapter 3. Analytic Geometry 15 / 25

3.3 Lengths and Distances

Definition
Consider an inner product space (V , h·, ·i). Then
p
The norm induced by the inner product kxk := hx, xi
p
d(x, y) := kx − yk = hx − y, x − yi is called the distance between x
and y in V .

Theorem
For an inner product vector space (V , h·, ·i) the induced norm k·k satisfies
the Cauchy-Schwarz inequality:

|hx, y| ≤ kxk kyk .

FPT University Chapter 3. Analytic Geometry 15 / 25

Example
Let the inner product h·, ·i be defined on R2 by

hx, yi = x1 y1 − x1 y2 − x2 y1 + 3x2 y2 .
 T  T
Find the distance between two vectors u = 1 1 and v = −1 2 .

FPT University Chapter 3. Analytic Geometry 16 / 25

Example
Let the inner product h·, ·i be defined on R2 by

hx, yi = x1 y1 − x1 y2 − x2 y1 + 3x2 y2 .
 T  T
Find the distance between two vectors u = 1 1 and v = −1 2 .

FPT University Chapter 3. Analytic Geometry 16 / 25

3.4 Angles and Orthogonality

Consider an inner product space (V , h·, ·i).

Definition
The angle between the vectors x and y is the number ω ∈ [0, π] defined by

hx, yi
cos ω =
kxk kyk

FPT University Chapter 3. Analytic Geometry 17 / 25

3.4 Angles and Orthogonality

Consider an inner product space (V , h·, ·i).

Definition
The angle between the vectors x and y is the number ω ∈ [0, π] defined by

hx, yi
cos ω =
kxk kyk

Definition
Two vectors x and y are orthogonal if and only if hx, yi = 0, and we write
x ⊥ y.

FPT University Chapter 3. Analytic Geometry 17 / 25

3.4 Angles and Orthogonality

Consider an inner product space (V , h·, ·i).

Definition
The angle between the vectors x and y is the number ω ∈ [0, π] defined by

hx, yi
cos ω =
kxk kyk

Definition
Two vectors x and y are orthogonal if and only if hx, yi = 0, and we write
x ⊥ y. Additionally, if kxk = kyk = 1, i.e., the vectors are unit vectors,
then x and y are orthonormal.

FPT University Chapter 3. Analytic Geometry 17 / 25

3.4 Angles and Orthogonality

Consider an inner product space (V , h·, ·i).

Definition
The angle between the vectors x and y is the number ω ∈ [0, π] defined by

hx, yi
cos ω =
kxk kyk

Definition
Two vectors x and y are orthogonal if and only if hx, yi = 0, and we write
x ⊥ y. Additionally, if kxk = kyk = 1, i.e., the vectors are unit vectors,
then x and y are orthonormal.

FPT University Chapter 3. Analytic Geometry 17 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

Theorem
If A ∈ Rn×n is orthogonal then

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

Theorem
If A ∈ Rn×n is orthogonal then
A−1 = AT .

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

Theorem
If A ∈ Rn×n is orthogonal then
A−1 = AT .
A preserve the length of any vector x ∈ Rn .

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

Theorem
If A ∈ Rn×n is orthogonal then
A−1 = AT .
A preserve the length of any vector x ∈ Rn .
A preserve the angle between two vectors x and y in Rn .

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

Theorem
If A ∈ Rn×n is orthogonal then
A−1 = AT .
A preserve the length of any vector x ∈ Rn .
A preserve the angle between two vectors x and y in Rn .

FPT University Chapter 3. Analytic Geometry 18 / 25

Definition
A square matrix A ∈ Rn×n is an orthogonal matrix if and only if its
columns are orthonormal

AAT = AT A = In

Example
" #
√1 − √12
Show that A = 2 is orthogonal.
√1 √1
2 2

Theorem
If A ∈ Rn×n is orthogonal then
A−1 = AT .
A preserve the length of any vector x ∈ Rn .
A preserve the angle between two vectors x and y in Rn .

FPT University Chapter 3. Analytic Geometry 18 / 25

3.5 Orthonormal Basis

Consider an n-dimensional inner product vector space (V , h·, ·i) and a

basis B = {b1 , · · · , bn } of V .

Definition
B is called an orthonormal basis (ONB) if
(
1 if i = j,
hbi , bj i = δij =
0 otherwise

FPT University Chapter 3. Analytic Geometry 19 / 25

3.5 Orthonormal Basis

Consider an n-dimensional inner product vector space (V , h·, ·i) and a

basis B = {b1 , · · · , bn } of V .

Definition
B is called an orthonormal basis (ONB) if
(
1 if i = j,
hbi , bj i = δij =
0 otherwise

Example
The canonical basis for an Euclidean vector space Rn is an orthonormal
basis.

FPT University Chapter 3. Analytic Geometry 19 / 25

3.5 Orthonormal Basis

Consider an n-dimensional inner product vector space (V , h·, ·i) and a

basis B = {b1 , · · · , bn } of V .

Definition
B is called an orthonormal basis (ONB) if
(
1 if i = j,
hbi , bj i = δij =
0 otherwise

Example
The canonical basis for an Euclidean vector space Rn is an orthonormal
basis.

FPT University Chapter 3. Analytic Geometry 19 / 25

3.5 Orthonormal Basis

Consider an n-dimensional inner product vector space (V , h·, ·i) and a

basis B = {b1 , · · · , bn } of V .

Definition
B is called an orthonormal basis (ONB) if
(
1 if i = j,
hbi , bj i = δij =
0 otherwise

Example
The canonical basis for an Euclidean vector space Rn is an orthonormal
basis.

FPT University Chapter 3. Analytic Geometry 19 / 25

Example
T  T
basis of R2 with inner

Show that { 1 −1 , 1 0 } is an orthonormal
product
 
1 1
hx, yi = xT Ay, where A = .
1 2

Theorem
Let {b1 , · · · , bn } be an orthonormal basis of an inner product space V . If
x is any vector in V , then

FPT University Chapter 3. Analytic Geometry 20 / 25

Example
T  T
basis of R2 with inner

Show that { 1 −1 , 1 0 } is an orthonormal
product
 
1 1
hx, yi = xT Ay, where A = .
1 2

Theorem
Let {b1 , · · · , bn } be an orthonormal basis of an inner product space V . If
x is any vector in V , then

x = hx, b1 ib1 + · · · + hx, bn ibn

FPT University Chapter 3. Analytic Geometry 20 / 25

Example
T  T
basis of R2 with inner

Show that { 1 −1 , 1 0 } is an orthonormal
product
 
1 1
hx, yi = xT Ay, where A = .
1 2

Theorem
Let {b1 , · · · , bn } be an orthonormal basis of an inner product space V . If
x is any vector in V , then

x = hx, b1 ib1 + · · · + hx, bn ibn

FPT University Chapter 3. Analytic Geometry 20 / 25

Theorem (Gram-Schmidt process)
Let V be an inner product space and let {v1 , v2 , . . . , vn } be any basis of
V . Define vectors b1 , b2 , . . . , bn in V successively as follows:

b1 = v1
hv2 , b1 i
b2 = v2 − b1
kb1 k2
···
hvk , b1 i hvk , b2 i hvk , bk−1 i
bk = vk − 2
b1 − 2
b2 − · · · − bk−1
kb1 k kb2 k kbk−1 k2

for each k = 2, . . . , n. Then

FPT University Chapter 3. Analytic Geometry 21 / 25

Theorem (Gram-Schmidt process)
Let V be an inner product space and let {v1 , v2 , . . . , vn } be any basis of
V . Define vectors b1 , b2 , . . . , bn in V successively as follows:

b1 = v1
hv2 , b1 i
b2 = v2 − b1
kb1 k2
···
hvk , b1 i hvk , b2 i hvk , bk−1 i
bk = vk − 2
b1 − 2
b2 − · · · − bk−1
kb1 k kb2 k kbk−1 k2

for each k = 2, . . . , n. Then

(i) {b1 , b2 , . . . , bn } is an orthogonal basis of V .

FPT University Chapter 3. Analytic Geometry 21 / 25

Theorem (Gram-Schmidt process)
Let V be an inner product space and let {v1 , v2 , . . . , vn } be any basis of
V . Define vectors b1 , b2 , . . . , bn in V successively as follows:

b1 = v1
hv2 , b1 i
b2 = v2 − b1
kb1 k2
···
hvk , b1 i hvk , b2 i hvk , bk−1 i
bk = vk − 2
b1 − 2
b2 − · · · − bk−1
kb1 k kb2 k kbk−1 k2

for each k = 2, . . . , n. Then

(i) {b1 , b2 , . . . , bn } is an orthogonal basis of V .
(ii) span{b1 , . . . , bk } = span{v1 , . . . , vk } holds for each k = 1, . . . , n.

FPT University Chapter 3. Analytic Geometry 21 / 25

Theorem (Gram-Schmidt process)
Let V be an inner product space and let {v1 , v2 , . . . , vn } be any basis of
V . Define vectors b1 , b2 , . . . , bn in V successively as follows:

b1 = v1
hv2 , b1 i
b2 = v2 − b1
kb1 k2
···
hvk , b1 i hvk , b2 i hvk , bk−1 i
bk = vk − 2
b1 − 2
b2 − · · · − bk−1
kb1 k kb2 k kbk−1 k2

for each k = 2, . . . , n. Then

(i) {b1 , b2 , . . . , bn } is an orthogonal basis of V .
(ii) span{b1 , . . . , bk } = span{v1 , . . . , vk } holds for each k = 1, . . . , n.
(iii) {b1 / kb1 k , b2 / kb2 k , . . . , bn / kbn k} is an orthonormal basis of V .

FPT University Chapter 3. Analytic Geometry 21 / 25

Theorem (Gram-Schmidt process)
Let V be an inner product space and let {v1 , v2 , . . . , vn } be any basis of
V . Define vectors b1 , b2 , . . . , bn in V successively as follows:

b1 = v1
hv2 , b1 i
b2 = v2 − b1
kb1 k2
···
hvk , b1 i hvk , b2 i hvk , bk−1 i
bk = vk − 2
b1 − 2
b2 − · · · − bk−1
kb1 k kb2 k kbk−1 k2

for each k = 2, . . . , n. Then

(i) {b1 , b2 , . . . , bn } is an orthogonal basis of V .
(ii) span{b1 , . . . , bk } = span{v1 , . . . , vk } holds for each k = 1, . . . , n.
(iii) {b1 / kb1 k , b2 / kb2 k , . . . , bn / kbn k} is an orthonormal basis of V .

FPT University Chapter 3. Analytic Geometry 21 / 25

Theorem (Gram-Schmidt process)
Let V be an inner product space and let {v1 , v2 , . . . , vn } be any basis of
V . Define vectors b1 , b2 , . . . , bn in V successively as follows:

b1 = v1
hv2 , b1 i
b2 = v2 − b1
kb1 k2
···
hvk , b1 i hvk , b2 i hvk , bk−1 i
bk = vk − 2
b1 − 2
b2 − · · · − bk−1
kb1 k kb2 k kbk−1 k2

for each k = 2, . . . , n. Then

(i) {b1 , b2 , . . . , bn } is an orthogonal basis of V .
(ii) span{b1 , . . . , bk } = span{v1 , . . . , vk } holds for each k = 1, . . . , n.
(iii) {b1 / kb1 k , b2 / kb2 k , . . . , bn / kbn k} is an orthonormal basis of V .

FPT University Chapter 3. Analytic Geometry 21 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .
→
−
(i) U ∩ U ⊥ = { 0 }.

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .
→
−
(i) U ∩ U ⊥ = { 0 }.
(ii) U ⊥ is a subspace of V and V = U ⊕ U ⊥ (each vector v in V can be
uniquely written as the sum of two vectors u ∈ U and w ∈ U ⊥ ).

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .
→
−
(i) U ∩ U ⊥ = { 0 }.
(ii) U ⊥ is a subspace of V and V = U ⊕ U ⊥ (each vector v in V can be
uniquely written as the sum of two vectors u ∈ U and w ∈ U ⊥ ).
(iii) dim(U ⊥ ) = dim(V ) − dim(U ⊥ ).

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .
→
−
(i) U ∩ U ⊥ = { 0 }.
(ii) U ⊥ is a subspace of V and V = U ⊕ U ⊥ (each vector v in V can be
uniquely written as the sum of two vectors u ∈ U and w ∈ U ⊥ ).
(iii) dim(U ⊥ ) = dim(V ) − dim(U ⊥ ).
(iv) (U ⊥ )⊥ = U.

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .
→
−
(i) U ∩ U ⊥ = { 0 }.
(ii) U ⊥ is a subspace of V and V = U ⊕ U ⊥ (each vector v in V can be
uniquely written as the sum of two vectors u ∈ U and w ∈ U ⊥ ).
(iii) dim(U ⊥ ) = dim(V ) − dim(U ⊥ ).
(iv) (U ⊥ )⊥ = U.

FPT University Chapter 3. Analytic Geometry 22 / 25

3.6 Orthogonal Complement

Let U be a subspace of an inner product space V .

Definition
The orthogonal complement U ⊥ of U in V is defined by

U ⊥ = {v ∈ V | hv, ui = 0 for all u ∈ U}

Theorem
Let U be a finite dimensional subspace of an inner product space V .
→
−
(i) U ∩ U ⊥ = { 0 }.
(ii) U ⊥ is a subspace of V and V = U ⊕ U ⊥ (each vector v in V can be
uniquely written as the sum of two vectors u ∈ U and w ∈ U ⊥ ).
(iii) dim(U ⊥ ) = dim(V ) − dim(U ⊥ ).
(iv) (U ⊥ )⊥ = U.

FPT University Chapter 3. Analytic Geometry 22 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection if

FPT University Chapter 3. Analytic Geometry 23 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

FPT University Chapter 3. Analytic Geometry 23 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

Definition
The projection P on U with ker(P) = U ⊥ is called the orthogonal
projection on U and is denoted by projU : V → U.

FPT University Chapter 3. Analytic Geometry 23 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

Definition
The projection P on U with ker(P) = U ⊥ is called the orthogonal
projection on U and is denoted by projU : V → U.

Theorem (Projection Theorem)

Let U be a finite dimensional subspace of an inner product space V and
let v be a vector in V .

FPT University Chapter 3. Analytic Geometry 23 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

Definition
The projection P on U with ker(P) = U ⊥ is called the orthogonal
projection on U and is denoted by projU : V → U.

Theorem (Projection Theorem)

Let U be a finite dimensional subspace of an inner product space V and
let v be a vector in V .
(i) projU (v) ∈ U and v − projU (v) ∈ U ⊥ .

FPT University Chapter 3. Analytic Geometry 23 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

Definition
The projection P on U with ker(P) = U ⊥ is called the orthogonal
projection on U and is denoted by projU : V → U.

Theorem (Projection Theorem)

Let U be a finite dimensional subspace of an inner product space V and
let v be a vector in V .
(i) projU (v) ∈ U and v − projU (v) ∈ U ⊥ .
(ii) If {b1 , . . . , bm } is an orthonormal basis of U then

FPT University Chapter 3. Analytic Geometry 23 / 25

3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

Definition
The projection P on U with ker(P) = U ⊥ is called the orthogonal
projection on U and is denoted by projU : V → U.

Theorem (Projection Theorem)

Let U be a finite dimensional subspace of an inner product space V and
let v be a vector in V .
(i) projU (v) ∈ U and v − projU (v) ∈ U ⊥ .
(ii) If {b1 , . . . , bm } is an orthonormal basis of U then

projU (v) = hv , b1 ib1 + · · · + hv , bm ibm

FPT University Chapter 3. Analytic Geometry 23 / 25
3.8 Orthogonal Projections
Let V be a vector space and U ⊆ V a subspace of V .
Definition
A linear mapping P : V → U is called a projection ifP 2 = P ◦ P = P.

Definition
The projection P on U with ker(P) = U ⊥ is called the orthogonal
projection on U and is denoted by projU : V → U.

Theorem (Projection Theorem)

Let U be a finite dimensional subspace of an inner product space V and
let v be a vector in V .
(i) projU (v) ∈ U and v − projU (v) ∈ U ⊥ .
(ii) If {b1 , . . . , bm } is an orthonormal basis of U then

projU (v) = hv , b1 ib1 + · · · + hv , bm ibm

FPT University Chapter 3. Analytic Geometry 23 / 25
Theorem (Approximation Theorem)
Let U be a finite dimensional subspace of an inner product space V . If v is
any vector in V , then projU (v) is the vector in U that is closest to v.Here
closest means that
kv − projU (v)k < kv − uk
for all u in U, u 6= projU (v).

FPT University Chapter 3. Analytic Geometry 24 / 25

Theorem (Approximation Theorem)
Let U be a finite dimensional subspace of an inner product space V . If v is
any vector in V , then projU (v) is the vector in U that is closest to v.Here
closest means that
kv − projU (v)k < kv − uk
for all u in U, u 6= projU (v).

FPT University Chapter 3. Analytic Geometry 24 / 25

Now we consider orthogonal projections of vectors x ∈ Rn onto
lower-dimensional subspaces U ⊆ Rn with a basis B = {b1 , . . . , bm }. It is
easy to find projU (x) after bringing B into a orthonormal basis by
Gram-Schmidt process, then apply Projection Theorem. The following we
are using a different approach.
m
X
projU (x) = λi bi = Bλ,
i=1
   T
B = b1 · · · bm , λ = λ1 · · · λm
bT T
1 (x − projU (x)) = · · · = bm (x − projU (x)) = 0
⇒ bT T
1 (x − Bλ) = · · · = bm (x − Bλ) = 0
⇒ B T (x − Bλ) = 0
⇒ B T Bλ = B T x
⇒ λ = (B T B)−1 B T x
⇒ projU (x) = B(B T B)−1 B T x

FPT University Chapter 3. Analytic Geometry 25 / 25

Now we consider orthogonal projections of vectors x ∈ Rn onto
lower-dimensional subspaces U ⊆ Rn with a basis B = {b1 , . . . , bm }. It is
easy to find projU (x) after bringing B into a orthonormal basis by
Gram-Schmidt process, then apply Projection Theorem. The following we
are using a different approach.
m
X
projU (x) = λi bi = Bλ,
i=1
   T
B = b1 · · · bm , λ = λ1 · · · λm
bT T
1 (x − projU (x)) = · · · = bm (x − projU (x)) = 0
⇒ bT T
1 (x − Bλ) = · · · = bm (x − Bλ) = 0
⇒ B T (x − Bλ) = 0
⇒ B T Bλ = B T x
⇒ λ = (B T B)−1 B T x
⇒ projU (x) = B(B T B)−1 B T x

FPT University Chapter 3. Analytic Geometry 25 / 25