Functional Analysis Basics

c 2007 Ulrich G. Schuster SVN Revision: 2109, December 11, 2007

Finite-Dimensional Normed Spaces. complete, then A⊥ is complete.
Vector Space Concepts • Every finite-dimensional linear sub- • If A ⊂ B, then B ⊥ ⊂ A⊥ .
space S of a normed space U is complete; • A ⊂ (A⊥ )⊥ .
incomplete complete in particular, every finite-dimensional • If g ∈ A ∩ A⊥ , then g = 0.
linear spaces normed space is complete. • If A ⊂ G, then A⊥ = ((A⊥ )⊥ )⊥ .
• Every finite-dimensional linear subspace • {0}⊥ = G and G ⊥ = {0}.
of a normed space U is closed in U and • If A is a dense subset of G, then A⊥ =
normed spaces separable. {0}.
• On a finite-dimensional linear space, all • If {An } is a sequence of sub-
Banach space norms are equivalent.
⊥
• In a finite-dimensional normed space U, spaces, then span{An } = ∩n A⊥ n,
any subset S ⊂ U is compact iff S is and (∩n An )⊥ = span{A⊥ n }.
inner product spaces closed and bounded. An orthonormal set O in an inner product
Inner Product Space [1, §3]. Let (G, F) space G that is total in G is called a total
be a linear space. An inner product is a orthonormal set, or sometimes a maximal or
Hilbert space mapping h·, ·i : G × G → F that satisfies the complete orthonormal set.
following properties for all x, y, z ∈ V and • Let O ⊂ G be a subset of an inner product
scalars α ∈ F: space G. Then, if O is total in G, there
1. hx + y, zi = hx, zi + hy, zi. does not exist a nonzero vector g ∈ G
2. hαx, yi = αhx, yi. that is orthogonal to every element of O.
3. hx, yi = hy, xi∗ . • If G is complete, i.e., a Hilbert space, the
4. hx, xi ≥ 0 with equality iff x = 0. above condition is sufficient for O to be
A linear space G on which an inner prod- total in G.
uct h·, ·i is defined is called an inner product Hilbert Space [1, §3]. A complete inner
space (G, h·, ·i). product space (H, h·, ·i) is called a Hilbert
• An
p inner product defines a norm kxk := space. Thus, a Hilbert space is a Banach
hx, xi
p and a metric d(x, y) := ky − space on which an inner product is defined.
xk = hy − x, y − xi on G. Hence, in- • For any inner product space G there ex-
metric spaces ner product spaces are normed spaces. ists a Hilbert space H and an isomor-
• The inner product is called sesquilin- phism from G onto a dense linear sub-
ear, because it is linear in the first space D ⊂ H. The space H is unique
term and conjugate linear in the second except for isomorphisms. Thus, every
Scalar Field. For all linear (vector) spaces • If a normed space U contains a se- term: hx, αyi = α∗ hx, yi. inner product space can be completed.
in the following, the scalar field will be ei- quence {en } with the property that for • The inner product satisfies the Schwarz • Let {hn } be a sequence in a Hilbert
ther the field of real numbers, F = R, or the every u ∈ U there is a unique sequence of inequality: |hx, yi| ≤ kxkkyk. w
space H. Then, hn − → h iff hhn , zi →
complex field, F = C. scalars {αn } such that ku−(α1 e1 +α2 e2 + • The induced norm satisfies the triangle in- hh, zi for all z ∈ H.
· · ·+αN eN k → 0 as N → ∞, then {en } is equality: kx + yk ≤ kxk + kyk with equal- • In every Hilbert space H =
Normed Space [1, 2, §2]. A norm k·k on 6 {0}, there
called a Schauder basis for U. A Schauder ity iff y = cx for some positive scalar c.
a linear space (U, F) is a mapping k·k : U → exists a total orthonormal set.
basis is different from a Hamel basis in • The induced norm satisfies the parallel- • An orthonormal set O in a Hilbert
[0, ∞) that satisfies, for all u, v ∈ U, α ∈ F, that a countably infinite number of ba-
1. kuk = 0 ⇐⇒ u = 0. ogram equality: kx + yk2 + kx − yk2 = space H is total in H iff for all h ∈ H the
sis vectors and scalar coefficients may be 2(kxk2 + kyk2 ).
2. kαuk = |α| kuk. Parseval relation holds:
needed to uniquely represent a given vec- • Continuity: if in an inner product space G
3. Triangle inequality: ku + vk ≤ kuk +
X 2
tor. |hh, ei| = khk2 .
kvk. xn → x and yn → y, then hxn , yn i →
e∈O
A norm defines a metric d(u, v) := ku − vk Convergence [1, §4.8]. Let {un } be a hx, yi, where {xn }, x, {yn }, y ∈ G.
on U. A normed (linear) space (U, k·k) is a sequence of vectors in a normed space U. • If hx1 , yi = hx2 , yi for all y in an inner • A total orthonormal sequence, i.e., a
product space, then x1 = x2 . countable total orthonormal set, in a
linear space U with a norm k·k defined on • The sequence {un } is said to be strongly
Two inner product spaces G and V are called Hilbert space H is called an orthonormal
it. convergent, or convergent in norm, if
unitarily equivalent if there is an isomor- basis for H.
• The norm is a continuous mapping of U there is a u ∈ U, called the strong limit • If a Hilbert space H is separable, every
into R+ . of {un }, such that limn→∞ kun − uk = 0. phism U : G → V of G onto V that preserves
inner products, i.e., hUu1 , Uu2 i = hu1 , u2 i total orthonormal set is countable, i.e., ev-
• A norm k·k on a linear space U is said Strong convergence is written un → u ery total orthonormal set is an orthonor-
to be equivalent to a norm k·k0 on U if and often referred to simply as conver- for all u1 , u2 ∈ G. The mapping U is called
a unitary operator. mal basis. Conversely, if H contains an
there are positive numbers a and b such gence. orthonormal sequence that is total in H,
that akuk0 ≤ kuk ≤ bkuk0 for all u ∈ U. • The sequence {un } is said to be weakly Orthogonality [2, 1]. An element x of an then H is separable. Thus, there exists
Equivalent norms define the same topol- convergent if there is a u ∈ U, inner product space G is said to be orthog- an orthonormal basis for H iff H is sepa-
ogy on U. called the weak limit of {un }, such onal to an element y ∈ G, denoted x ⊥ y, rable.
• The metric d induced by a norm is trans- that limn→∞ f (un ) = f (u) for every if hx, yi = 0. Similarly, for A, B ⊂ G, • All total orthonormal sets in a given
lation invariant, i.e., it satisfies bounded linear functional f on U, i.e., x ⊥ A means that x ⊥ a for all a ∈ A, Hilbert space have the same cardinality,
0
◦ d(u + x, v + x) = d(u, v), for every f in the dual space U . Weak and A ⊥ B means that a ⊥ b for all a ∈ A called the Hilbert dimension of H.
◦ d(αu + αv) = |α| d(u, v) convergence is written un −
w
→ u. and all b ∈ B. • Two Hilbert spaces H and H̃, both over
for all u, v, x ∈ U and α ∈ F. • An orthogonal set O in an inner product the same scalar field, are isomorphic iff
◦ The weak limit u is unique.
• Riesz’s Lemma: Let Y and Z be linear space G is a subset O ⊂ G whose elements they have the same Hilbert dimension.
◦ Every subsequence of {un } converges
subspaces of a normed space U and let Y are pairwise orthogonal. An orthonormal • Let Y be any closed linear subspace of
weakly to u.
be a closed proper subset of U. Then, for set is an orthogonal set whose elements a Hilbert space H. Then, H = Y ⊕ Z,
◦ The sequence {kun k} is bounded.
every θ ∈ (0, 1), there is a z ∈ Z such have unit norm. A countable orthogonal where Z = Y ⊥ is the orthogonal comple-
that kz − yk ≥ θ for kzk = 1 and for • Strong convergence implies weak conver- (orthonormal) set is called an orthogonal ment of Y. Each h ∈ H can be uniquely
all y ∈ Y. gence to the same limit.
(orthonormal) sequence. represented as h = y + z, where y ∈ Y
• A subset T of a normed space U is said • If dim U < ∞, then weak convergence • An orthogonal set is linearly independent. and z ∈ Z = Y ⊥ , and khk = kyk + kzk.
to be total in U if span T is dense in U. implies strong convergence.
• Let {eα } be an orthonormal set in an • Let S ⊂ H be a linear subspace of H;
• Let S be a linear subspace of a normed • The (infinite) series u1 + u2 + . . . is said inner product space G, and let g be any then, (S ⊥ )⊥ = S. If S is closed,
space U. If S is open as a subset in U, to converge (strongly) if the sequence of
partial sums sn := u1 + u2 + · · · + un point in G. Then hg, eα i is nonzero for at then (S ⊥ )⊥ = S.
then S = U. most a countable number of vectors eα . • For any nonempty susbspace S of a
converges, i.e., if sn → s for some s ∈ U.
Basis and Dimension [2, 1]. • The above series is said to be absolutely • Let G be an inner product space and C a Hilbert space H, span S is dense in H
• Let V be a linear space. A linearly in- convergent if the infinite series ku1 k + nonempty convex subset of G that is com- iff S ⊥ = {0}. If S is closed and S ⊥ =
dependent subset S ⊂ V that spans V is ku2 k + . . . converges. plete in the metric induced by the inner {0}, then S = H.
called a Hamel basis for V. • A series is said to be unconditionally con- product. Then, for every g ∈ G there ex- Let Y and Z be two subspaces of H. The
◦ Every linear space has a Hamel basis, vergent if (i) it is convergent for each pos- ists a unique c0 ∈ C such that inf c∈C kg − canonical correlation ρ(Y, Z) between these
so that every nonzero v ∈ V has a sible rearrangement of terms, and (ii) if ck = kg − c0 k. If C is a complete linear two subspaces is defined as
unique representation as a linear com- each rearrangement converges to the same subspace of G, then (g − c0 ) ⊥ C. 
limit. • Bessel inequality: Let {en } be an or- ρ(Y, Z) := sup |hy, zi| : y ∈ Y,
bination of finitely many elements of S
with nonzero scalar coefficients. thonormal sequence in an inner product z ∈ Z, kyk = kzk = 1

Banach Space [1, §2]. A Banach space G. Then, for every g ∈ G,
◦ If B1 and B2 are Hamel basis for a lin- and the angle θ(Y, Z) between these sub-
space (B, k·k) is a complete normed space, ∞
ear space V, then they have the same X 2 spaces as θ(Y, Z) = cos ρ(Y, Z).
cardinality.
complete in the metric induced by its |hg, en i| ≤ kgk2 .
norm k·k. n=1 • Let y ∈ Y, z ∈ Z. Then, the following
• The dimension dim V of a linear space V conditions are equivalent:
is defined as the cardinality of any Hamel • A linear subspace S of a Banach space B Orthogonal Complement [2, 1, 3].
basis of V. is a Banach space, i.e., it is complete, iff S ◦ ρ(Y, Z) < 1, i.e., θ(Y, Z) > 0.
is closed in B. Let A and B be nonempty subsets in an
◦ If dim V is finite, V is called a finite- inner product space G. The set A⊥ := ◦ inf {ky − zk : kyk = kzk = 1} > 0.
• For a series on a Banach space, absolute ◦ There is a constant c such that kyk ≤
dimensional linear space.
convergence implies strong convergence {g ∈ G : g ⊥ A} is called the orthogonal
◦ A linear space V is finite dimensional complement of A in G. cky + zk for all y, z.
and unconditional convergence. ◦ The direct sum Y ⊕ Z is a closed sub-
iff there is a positive integer N such
• Let (U, k·k) be a normed space. Then • The orthogonal complement A⊥ of A in G space of H.
that V contains a linearly indepen- is a closed linear subspace of G. If G is
there is a Banach space B and an isome-
dent set of N vectors whereas any set
try f from B onto a linear subspace S ⊂ B
of N + 1 vectors of V is linearly depen-
that is dense in B. The space B is
dent.
unique except for isometries. Thus, every
◦ If V1 and V2 are linear spaces over the
normed space can be completed.
same scalar field, then they are isomor-
phic iff dim V1 = dim V2 .
Fourier Series [2]. Riesz-Fischer Theo- • An element b ∈ B is called invertible if b Hardy Space [3]. Let D := with norm
rem: Let {en } be an orthonormal sequence has an inverse in B. The invertible ele- {z ∈ C : |z| < 1} be the open disk in the Z 1/p
f (reiλ )p dλ
 
in a Hilbert space H, and let {αn } be a ments of B form a group with respect to complex plane. For 0 < p < ∞, the space kf kp := sup
sequence of scalars. Then, the series vector multiplication. n 0≤r<1

X∞ • Let S ⊂ B denote the set of all invertible Hp := f : f analytic in D, is a Banach space, called the Hardy space.
h= αn en elements of B. If b ∈ B and kbk < 1, Z o Reproducing Kernel Hilbert Spaces.
f (reiλ )p dλ < ∞
 
n=1 then, sup to write
P∞ 2
converges in norm iff n=1 |αn | < ∞. In ◦ 1 + b ∈ S, P 0≤r<1
∞
this case, the coefficients αn are called the ◦ (1 + b)−1 = n=0 (−1)n bn ,
Fourier coefficients of h, and they are given ◦ k(1 + b)−1 − 1 + bk ≤ kbk2 /(1 − kbk). Linear Operators and Linear Functionals
as αn = hh, en i. Conversely, the above se- ◦ The set S is open, and the map-
ries always converges to h if the αn are the ping b → b−1 is a homeomorphism Linear Operator [1, 2]. A linear opera- disjoint subspaces of X such that X =
Fourier coefficients of any h ∈ H. of S onto S. tor T is a mapping of a linear space V into R(P) + N (P) = R(P) ⊕ N (P), i.e., R(P)
• The above series is convergent iff it con- • The spectrum S(b) of an element b ∈ B a linear space Z such that and N (P) are algebraic complements of
verges unconditionally. is defined as the set of all complex num- 1. The domain D(T) is a linear space V, one another.
bers λ such that b − λ1 is not invertible. and the range R(T) lies in a linear • If P is a projection, so is I−P, and R(P) =
Let {en } be an orthonormal set in a Hilbert space Z over the same scalar field F. N (I − P) and N (P) = R(I − P).
space H, then the following statements are • Let f be a bounded linear functional on B.
Then, for any fixed b ∈ B, the func- 2. For all v, u ∈ V and scalars α, • Let S ⊂ X be a subspace of X . Then
equivalent: there exists a projection P : X → X such
tion g(λ) := f ((b − λ1)−1 ), λ ∈ / S(b), is T(v + u) = Tv + Tu,
• The set {en } is an orthonormal basis holomorphic in the complement of S(b), that R(P) = S.
for H. T(αv) = αTv. • Given two disjoint subspaces V and U
and g(λ) → 0 as λ → ∞.
• For any h ∈ H, the Fourier series P ex- • For every b ∈ B, the spectrum S(b) is The null space N (T) of T is the set of with X = U ⊕ V, there is a unique projec-
pansion of h is given as h = n αn en , compact and not empty. all v ∈ D(T) such that Tv = 0. The null tion P such that R(P) = U and N (P) =
where αn = hh, en i. • If each nonzero element of B is invert- space is a linear space. V.
• Parseval equality: For any x, y ∈ H, ible, then the complex Banach algebra B • The range space R(T) of a linear operator
X
is a linear space. Finite-Dimensional Spaces [1, §2.9,
hx, yi = hx, en ihy, en i∗ . is isometrically isomorphic to the com- §7.1]. Let X and Y be finite-
plex field C. This also implies that B is • Two linear operators T and S are said to
n
be equal if they have the same domain dimensional linear spaces over the same
• For any h ∈ H, commutative. field F, with dim X = N, dim Y = K.
• For any b ∈ B, the spectral ra- and if Tv = Sv for all v ∈ D(T) = D(S).
2
X 2
• D(T) ∞ R(T) ≤ Let E := {e1 , . . . , eN } be a basis for X , and
khk = |hh, en i| . dius rb of b is defined as rb := dim = N < =⇒ dim
N. let B := {b1 , . . . , bK } be a basis for Y.
n sup {|λ| : λ ∈ S(b)}; it can be computed
• Let M be any linear subspace of H that as r= limn→∞ kbn k1/n . • The dimensions of the null space N (T),, • Any linear operator T : X → Y is
contains {en }; then M is dense in H. the range space R(T) and the space X it- uniquely determined by the K images
A complex-valued homomorphism f on a self are related as dim N (T)+dim R(T) = of the N basis vectors yk = Ten .
Banach Algebra [4, 5]. Strictly speak- Banach algebra B is a linear functional that dim X . • Any linear operator T on a finite-
ing, a Banach Algebra is an algebra B over preserve vector multiplication, i.e., a func- • Let X1 , X2 , Y1 , Y2 be linear spaces over dimensional linear space can be repre-
a scalar field F, where B is also a Banach tional f for which f (αx + βy) = αf (x) + the same scalar field so that X1 and X2 sented by a matrix T with [T]k,n = tk,n ,
space under a norm k·k that satisfies the βf (y) and f (xy) = f (x)f (y) for all x, y ∈ are isomorphic and Y1 and Y2 are isomor- where T depends on the bases E and B.
multiplicative inequality kxyk ≤ kxkkyk B and α, β ∈ F. Furthermore, f is not phic. The linear operators T1 : X1 → Y1 Hence, the image of any vector x ∈ X
for all x, y ∈ B. identical to 0. Let M denote the set of and T2 : X2 → Y2 are said to be isomor- can be obtained as
In the following, though, an associative unit all complex-valued homomorphisms f of B. phically equivalent if there exists isomor- XK X N
complex Banach algebra, i.e., a Banach al- Then, phisms U : X1 → X2 and W : Y1 → Y2 y = Tx = (tk,n ξn )bk
gebra over the complex field C that is asso- • λ ∈ S(b) iff f (b) = λ for some f ∈ M. such that T1 = W−1 T2 U and T2 = k=1 n=1
ciative and contains an identity element 1 • The vector b is invertible in B iff f (b) 6= 0 WT1 U−1 . PN
where x = n=1 ξn en .
with respect to vector multiplication such for every f ∈ M. • Let X1 and X2 be isomorphic linear • For given bases E and B, the matrix T is
that k1k = 1 is simply referred to as a com- • f (b) ∈ S(b) for every b ∈ B and f ∈ M. spaces. The linear operators T1 : X1 → uniquely determined by T.
plex Banach algebra. • |f (b)| ≤ rb ≤ kbk for every b ∈ B X1 and T2 : X2 → X2 are said to be sim- • Conversely, any K × N matrix T defines
and f ∈ M. ilar if there exists an isomorphism U : a linear operator with respect to given
X1 → X2 such that T1 = U−1 T2 U bases forX and Y.
Some Important Linear Spaces and T2 = UT1 U−1 . • Two matrices that represent a linear oper-
• Let T : V → V be a linear operator ator on a finite-dimensional normed space
Euclidean Space [6]. The N -dimensional Lebesgue Space [3, 7]. Let X be an arbi- and M ⊂ V a linear subspace of V such relative to two different bases are similar.
complex Euclidean space trary set, F the σ-algebra of subsets of X , that T(M) ⊂ M; then M is called invari-
and µ a nonnegative measure on F . The ant under T . In this case, the restriction Linear Functionals [1, 2]. A linear func-
n o
CN := x : x = x0 x1 · · · xN −1 T, xn ∈ C Lebesgue space
 
of T to M is a mapping of M into itself. tional is a linear operator f : V → F, defined
with inner product n • Let T : H → H be a linear operator on on some linear space V, whose range is in
N −1 Lp (X , F , µ) := f : X → C a Hilbert space H. If some closed lin- the scalar field F of the linear space.
ear subspace M ⊂ H and its orthogonal • Hahn-Banach Theorem: Let V be a real
X
∗
hx, yi := xn yn
Z o
measurable, |f |
p
dµ < ∞ complement M⊥ are invariant under T, or complex linear space, and let g be a
n=0
then M is said to reduce T. real-valued functional on V that is sub-
and corresponding induced norm is a finite- with norm • Any operator that maps a Banach space additive, i.e., g(u + v) ≤ g(u) + g(v) for
dimensional Hilbert space. Z 1/p onto another Banach space is an open all u, v ∈ V, and that satisfies g(αu) =
p
Sequence Space [3]. The sequence space kf kp := |f | dµ mapping. |α| g(v) for every scalar α. Let f be
a linear functional, defined on a sub-
Inverse Operator. Let T : V → Z be
n
lp := x : x = {xn }∞ is a Banach space for 1 ≤ p < ∞. space Z of V, that satisfies |f (z)| ≤ g(z)
n=0 ,
• For p = ∞, the space L∞ (X , F , µ) with a linear −1
operator. Then, the inverse oper- for all z ∈ Z. Then, f has a linear exten-
X∞
p
o µ-essential supremum norm kf k∞ := ator T : R(T) → D(T) exists iff Tv = 0 sion f˜ from Z to V that satisfies |f (v)| ˜ ≤
xn ∈ C, |xn | < ∞ essµ supt |f (t)| is also a Banach space. implies that v = 0.
−1 g(v) for all v ∈ V.
n=0 • If µ is finite and X = (a, b], the spaces • If T exists, it is a linear operator. −1 • The codimension of N (f ) is 1.
with norm L (µ) := L ((a, b], F , µ) are nested: • If dim D(T) = N < ∞ and T exists, • If A is any subspace of V with N (f ) ⊂ A
p p
∞
! 1/p Lp (µ) ⊆ Lq (µ) for p ≥ q. then dim R(T) = dim D(T).
X p • An invertible linear operator is a homeo- and N (f ) 6= 0, then A = V.
kxkp := |xn | • The spaceR L (X , F µ) with inner product
2
• For some linear functional f and some
hf, gi := X f g ∗ dµ and induced norm is a morphism.
n=0
• Let T : X → Y and S : Y → Z be scalar α, the set {v ∈ V : f (v) = α} is
is a Banach space for 1 ≤ p ≤ ∞. Hilbert space, called the Hilbert function bijective linear operators, where X , Y, called the hyperplane in V determined
• For p = ∞, the norm is the supremum space. The elements of L2 (X , F , µ) are and Z are linear spaces. Then, the in- by f and α.
norm: kxk∞ := supn |xn |. equivalence classes of functions that differ −1
verse (ST) : Z → X of the composition Algebraic Dual [1]. The set V ? of all lin-
• An important subspace of l∞ is the space on null sets. (also called product) ST := S ◦ T exists ear functionals defined on a linear space V
whose elements are sequences that decay • The space L1 (R) ∩ L2 (R) is a Hilbert −1
and (ST) = T−1 S−1 . is itself a linear space, called the algebraic
to zero, i.e., xn → 0 as n → ∞. space. It is a dense subspace of L2 (R). • A bounded bijective operator T : X → Y dual space of V. Its vector sum is defined
2
• For p = 2, the space l with inner product • For X = {0, 1, . . . , N − 1} or X = Z+
∞
between two Banach spaces X and Y has as s(v) = (f1 + f2 )(v) := f1 (v) + f2 (v) for
X and µ the counting measure on the col- a bounded inverse. all v ∈ V, and the product of a scalar α and
hx, yi := xn yn ∗
lection F of all subsets of X , the space • Von Neumann Theorem: Let T : B → B a vector, i.e., a functional f ∈ V ? , is defined
n=0 L (X , F µ) reduces to C or l , respec-
2 N 2
be a bounded operator on a Banach for all v ∈ V ? as p(v) = (αf )(v) := αf (v).
and norm kxk2 := hx, xi 1/2
is an infinite- tively. space B that satisfies kI − P Tk < 1. Then, • Let V be an N -dimensional linear space,
dimensional Hilbert space, called the • When µ is a probability measure, i.e., T is invertible, and T −1
=
∞ n
n=0 (I − T) . and let E = {e1 , . . . , eN } be a basis
Hilbert sequence space. µ(X ) = 1 for arbitrary X , then −1
L (X , F , µ) is the space of all random
2 Furthermore, kT k ≤ 1/(1 − kI − Tk). for V. Define the set of linear function-
Space of Continuous Functions. variables with finite second moment. als B := {f1 , . . . , fN } with fk (en ) = δkn .
Projections [2]. A linear operator P : Then B is a basis for the algebraic dual
Let C[a, b] denote the space of all complex- X → X that satisfies P2 = P is called a
valued continuous functions f : [a, b] → C Schwarz Space [7]. The Schwarz space S projection. space V ? of V, and dim E = dim B; B is
with pointwise addition and scalar multipli- is the space of all infinitely differentiable, called the dual basis of E.
cation. rapidly decaying functions of a real parame- • Range R(P) and null space N (P) are
• C ∞ [a, b], endowed with the supremum ter t:
norm kf k∞ := supa≤t≤b |f (t)|, is a Ba- n dn f (t) Linear Functionals on Normed Spaces
S := f : R → C : lim tm =0
nach space. t→∞ dtn
• Endowed with the inner product hf, gi := Linear Functionals [1, 2]. Let f : Theorem implies that every bounded lin-
Rb ∀m, n ∈ N U → F be a linear functional on a normed ear functional f on a subspace S ⊂ U has
a
f (t)g ∗ (t)dt and the induced norm, this space U. a linear extension f˜ on U that has the
space is an inner product space but not Paley-Wiener Space. to write
• The norm kf k of a linear functional f same norm, 
a Hilbert space. Sobolev Space. to write is the usual operator norm: kf k = ˜ 

supu∈U ,u6=0 |f (u)|. sup  f (u)  = sup |f (s)| .
u∈U ,kuk=1 s∈S,ksk=1
• A bounded linear functional is a linear
• Let U be a normed space and let u ∈
function f that satisfies kf k ≤ a for
U. Then, there exists a bounded lin-
some a ∈ R.
ear functional f on U such that kf k = 1
• On a normed space U, the Hahn-Banach
and f (u) = kuk.
Sesquilinear Form [1, §3.8]. Let V • For every u in a normed space U, Compact Linear Operators [1, 2]. for any u ∈ U.
and Z be liner spaces over the same scalar |f (u)| Let U and Z be normed spaces. A linear • Let {un } be a weakly convergent se-
field F. A sesquilinear form, or sesquilin- kuk = sup . operator T : U → Z is called compact or quence in U with un −
w
→ u. Then {Tun }
0 kf k
ear function f on V × Z is a mapping f : f ∈U completely continuous if for every bounded is strongly convergent in Z and has the
f 6=0
V × Z → F such that for all v, v1 , v2 ∈ V subset S ⊂ U, the image T(S) is relatively strong limit z = Tu.
and z, z1 , z2 ∈ Z and all scalars α and β • Given a linearly independent set compact, i.e., the closure T(S) is (sequen- • If T is compact, so is its adjoint opera-
0
◦ f (v1 + v2 , z) = f (v1 , z) + f (v2 , z), {f1 , . . . , fN } ∈ U , there are elements tially) compact. 0
tor T× : Z → U .
0

◦ f (v, z1 + z2 ) = f (v, z1 ) + f (v, z2 ), u1 , . . . , uN in U such that fj (uk ) = δjk . • Every compact linear operator T is • Let {Tn } be a sequence of compact linear
◦ f (αv, z) = αf (v, z), Convergence [1, §4.9]. For linear func- bounded and, therefore, continuous. operators from a normed space U into
◦ f (v, βz) = β ∗ f (v, z). tionals, strong and weak convergence are • If dim U = ∞, the identity operator I, a Banach space B. If {Tn } is uniformly
Dual Space [1]. Let U be a normed equivalent, so that a sequence {fn } of which is continuous, is not compact. operator convergent, i.e., kTn − Tk → 0,
space. Then the set of all bounded lin- bounded linear functionals on a normed • A linear operator T : U → Z is compact then the limit operator T is compact.
ear functionals on U constitutes a normed space U is said to be iff it maps every bounded sequence {un } • A compact linear operator T : U → B
space under the usual operator norm kf k = • strongly convergent if there is an f ∈ in U onto a sequence {Tun } in Z that from a normed space U into a Banach
supu∈U ,kuk=1 |f (u)|. This space is called the
0
U , called the strong limit of {fn }, such has a convergent subsequence. space B has a compact linear extension T̃ :
0
that kfn −f k → 0; this is written as fn → • If T is bounded and dim R(T) < ∞, Ũ → B, where Ũ is the completion of U.
dual space U of U. then T is compact.
0 f; • Let T : B → A and S : B → A be com-
• The dual space U of a normed space U 0 • If U is a finite-dimensional normed linear pact linear operators, where B and A are
• weak∗ convergent if there is an f ∈ U , space, every linear operator defined on U
is a Banach space, whether or not U is ∗ Banach spaces. Then, T + S is compact.
called the weak limit of {fn }, such is compact.
complete. • Let T : U → U be a compact linear op-
that fn (u) → f (u) for all u ∈ U; this • Given  > 0, there exists a finite-
w∗ erator and S : U → U a bounded linear
is written as fn −−→ f . dimensional subspace M ⊂ R(T) such operator on a normed space U. Then TS
that and ST are compact.
inf kTu − mk < kuk
Linear Operators on Normed and Banach Spaces m∈M

Continuity [2, 1]. Let X and Y be normed addition (T1 +T2 )u := T1 u+T2 u, for all u ∈
spaces, and let T : X → Y be a linear oper- U, and scalar multiplication (αT)u := αTu Spectral Theory of Linear Operators
ator. with α ∈ F. Resolvent, Spectrum [2, 1]. Let B
• The operator T is continuous iff • The linear space G(U, Z) is a normed be a complex Banach space, and let T : Tλ one-to-one?
∞ ∞ RRR
space, whose norm is the usual operator D(T) → R(T) be a linear operator RRR no
RRR
X  X
T αn xn = αn T(xn ) norm kTk for all T ∈ G(U, Z). with D(T), R(T) ⊂ B. yes RRR
n=1 n=1 • Let B ba a Banach space; then, G(U, B) • Associated with T is the the opera-  R(
P∞ R(T ) dense in X ? λ ∈ Sp (T)
for every convergent series n=1 αn xn is a Banach space. tor Tλ := T − λI, where λ ∈ C and I λ
QQQ
in X . • Let H is a Hilbert space, then G(H, H) is denotes the identity operator. QQQ no
QQQ
• If T is continuous at a single point, it is a Banach algebra. • If Tλ has an inverse defined on its range, yes
QQQ
 Q(
continuous. Convergence [1, §4.9]. Let U and Z be it is called the resolvent of T and denoted
−1 −1 R (T) continuous? λ ∈ Sr (T)
• The linear operator T is continuous iff it normed spaces. A sequence {Tn } of opera- as Rλ (T) := Tλ = (T − λI) on R(Tλ ). λ
QQQ
QQQ no
is bounded. tors Tn ∈ G(U, Z) is said to be The resolvent set Q(T) of T is defined as the QQQ
• If a linear operator T is continuous, it is
yes
QQQ
• uniformly operator convergent if {Tn } set of all complex numbers λ such that the  Q(
uniformly continuous. range of Tλ is dense in B and that Tλ has λ ∈ Q(T) λ ∈ Sc (T)
converges in the operator norm
• If X is finite dimensional, then T is con- a continuous inverse defined on its range.
on G(U, Z), i.e., kTn − Tk → 0;
tinuous. The numbers λ ∈ Q(T) are called regular • The four sets are pairwise disjoint
• strongly operator convergent if {Tn u} con-
Operator Norm [1, §2.7]. Let T : U → Z verges strongly in Z for every u ∈ U, values. The set S(T) := Q(T)c is called the and C = Q(T) ∪ Sp (T) ∪ Sc (T) ∪ Sr (T);
be a linear operator that maps a normed i.e., kTn u − Tuk → 0 for all u ∈ U; spectrum of T; a λ ∈ S(T) is called a spec- some of the sets may be empty.
space U into a normed space Z. The opera- • weakly operator convergent if {Tn u} con- tral value of T. The spectrum S(T) can be • If Rλ (T) exists, it is a linear operator.
tor norm is defined as verges weakly in Z for every u ∈ U, partitioned into three disjoint sets: • Let B be a complex Banach space, T :
kTuk i.e., |f (Tn u) − f (Tu)| → 0 for all u ∈ U ◦ The point spectrum Sp (T) is the set such B → B a linear operator, and λ ∈ Q(T).
kTk := sup and all bounded linear functionals f on U, that Tλ is not one-to-one. A λ ∈ Sp (T) If T is closed or bounded, then, Rλ (T)
u∈U kuk 0
u6=0 that is, for all f in the dual space U of U. is called an eigenvalue of T. is defined on the whole space B and is
where the norms on the RHS are vec- Uniform convergence implies strong conver- ◦ The continuous spectrum Sc (T) is the bounded.
tor norms in Z and U. If D(T) = {0}, gence, which in turn implies weak conver- set such that Tλ is one-to-one, has its
Eigenvalues [1, §7]. Let U be a normed
then kTk := 0. gence, all with the same limit. range dense set in B, but Rλ (T), defined
space over the complex field and T : D(T) →
• The operator norm kTk of T is equivalent • Let Tn ∈ G(B, U), where B is a Banach on R(Tλ ), is not continuous and, there-
U a linear operator with domain D(T) ⊂ U.
to fore, unbounded.
space and U a normed space. If {Tn } is ◦ The residual spectrum Sr (T) is the set • The resolvent Rλ (T) exists iff Tu = 0
kTk = sup kTuk. strongly operator convergent with limit T, implies u = 0, i.e., the null space N (T)
u∈U such that T is one-to-one, but R(Tλ ) is
kuk=1 then T ∈ G(B, U). not dense in B. is {0}.
• A sequence {Tn } of operators in G(B, Z), • If Tλ u = 0 for some u 6= 0, then λ ∈
Bounded Linear Operators [1, §2.7]. where B and Z are Banach spaces, is In summary: Sp (T). The vector u is then called an
The linear operator T : U → Z that maps a strongly operator convergent iff (i) the eigenvector of T with eigenvalue λ.
normed space U into a normed space Z is sequence {kTn k} is bounded, and (ii) the • The subspace of D(T) that consists of 0
said to be bounded if there is a real number a sequence {Tn b} is Cauchy in Z for ev- and all eigenvectors of T with eigen-
such that kTk ≤ a. ery b in a total subset of B. value λ is called the eigenspace of T cor-
• A linear operator T is bounded iff it is responding to that eigenvalue.
continuous. Adjoint Operator [1, §4.5]. Let X and Y
• Eigenvectors with different eigenvalues
• If a normed space U is finite dimen- be normed spaces and let T : X → Y be constitute a linearly independent set.
sional, then every linear operator on U is a bounded linear operator. Then, 0
for any
bounded. bounded linear functionals f ∈ X and g∈
0
• T = 0 iff hTu, zi = 0 for all u ∈ U Y , the adjoint operator T× : Y → X of T
0 0
Spectral Properties of Operators on Normed Spaces
and z ∈ Z. is defined by f (x) = (T× g)(x) = g(Tx) for Bounded Linear Operators on a Com- every point λ0 of the resolvent set Q(T).
• The null space N (T) of T is closed. all x ∈ X . plex Banach Space [1, §7.3]. Let B be a Hence, it is locally holomorphic on Q(T).
• If {un } a sequence in D(T), then un → u • The adjoint operator T× is linear and complex Banach space, and let T ∈ G(B, B) • The spectral radius of T is defined as rT :=
implies Tun → Tu. bounded, and kT× k = kTk. be a bounded linear operator. supλ∈S(T) |λ|.
• For bounded linear operators T1 : X → • If T is represented by a matrix T, then
• The resolvent set Q(T) is not empty. • The spectral radius is given as rT =
Y and T2 : Y → Z on normed the adjoint operator T× is represented • The spectrum S(T) is not empty.
spaces X , Y, and Z, it follows that by TT . limn→∞ kTn k1/n .
kT1 T2 k ≤ kT1 kkT2 k, and for T : X → X • The resolvent set Q(T) is open; hence, • The spectrum S(T) is compact and lies
• Let S : X → Y be another bounded linear the spectrum S(T) is closed.
that kTn k ≤ kTkn . operator. Then in a disk with spectral radius rT ≤ kTk.
• Uniform Boundedness Theorem: • If kTk < 1, then (I − T)−1 exists, • Let λ, µ ∈ Rλ (T). Then,
◦ (S + T)× = S× + T× . is a bounded linear operator on the ◦ The resolvent Rλ (T) satisfies the
Let {Tn } be a sequence of linear op- ◦ (αT)× = αT× , α ∈ F.
erators Tn : B → U from a Banach whole space B, and has the following se- Hilbert relation, also called resolvent
◦ (ST)× = T× S× . ries expansion, convergent in the norm identity:
space B into a normed space U such
that kTn bk ≤ cb < ∞ for every b ∈ B • If T
−1
exists and T−1 ∈ B(X , Y), on G(B, B): Rµ − Rλ = (µ − λ)Rµ Rλ ;
× −1 ∞
and every n = 1, 2, . . . . Then, the se- then (T ) also exists, (T× )−1 ∈ X ◦ Rλ (T) commutes with any S ∈ G(B, B)
quence of norms {kTn k} is bounded, i.e., 0 0
B(X , Y ), and (T× )−1 = (T−1 )× . (I − T)−1 = Tn = I + T + T2 + . . . that commutes with T;
there exists a c such that kTn k ≤ c for n=0 ◦ Rλ Rµ = Rµ R λ .
all n = 1, 2, . . . . Closed Linear Operators [1, §4.13]. and k(I − T)−1 k ≤ (1 − kTk)−1 . • Spectral mapping: Let p(λ) := αn λn +
• A bounded linear operator T from a Ba- Let U and Z be normed spaces and let T : • For every λ0 ∈ Q(T), the resolvent Rλ (T) αn−1 λn−1 + · · · + α0 λ0 with αn 6= 0.
nach space B onto a Banach space Z has D(T) → Z be a linear operator with do- has the representation Then, S(p(T)) = p(S(T)). That is, the
the property that the image T(B1 (0)) of main D(T) ⊂ U. Then, T is called a ∞
spectrum of the operator p(T) = αn Tn +
the open unit ball around the origin con- closed linear operator if its graph G(T) :=
X
Rλ (T) = (λ − λ0 )n Rn+1
λ0 . αn−1 Tn−1 + · · · + α0 I consists of all those
tains an open ball around 0 ∈ Z. {(u, z) : u ∈ D(T), z = Tu} is closed in the
n=0 values that the polynomial p assumes on
• Open mapping theorem: A bounded linear normed space U × Z. • The resolvent Rλ (T) is holomorphic at the spectrum S(T) of T.
operator T from a Banach space onto an- • Closed graph theorem: Let T be a closed
other Banach space is an open mapping. operator. If D(T) is closed in V, the op-
Hence, if T is bijective, T−1 is continuous erator T is bounded.
and thus bounded. • Let T : D(T) → Z be a linear operator,
where D(T) ⊂ U and U, Z are normed
Operator Topologies [1, 2]. Let G(U, Z) spaces. Then, T is closed iff it has the fol-
denote the set of all bounded linear opera- lowing property: If un → u for un ∈
tors from a normed space U into a normed D(T), and Tun → z, then u ∈ D(T)
space Z over the same scalar field. The and Tu = z.
set G(U, Z) is a linear space under operator
Compact Linear Operators [1, §8]. • There exists a smallest integer n = r, Normal Operators [2]. Let T : H → H uct ST is nonnegative.
Let T : U → U be a compact operator on a depending on λ, such that be a normal operator of a Hilbert space H • If T is bounded and self adjoint, then T2
normed space U, and let Tλ := T − λI. r r+1 r+2
N (Tλ ) = N (Tλ ) = N (Tλ ) . . . into itself. is nonnegative.
• Every spectral value λ ∈ S(T), λ 6= 0, if and • Let en be an eigenvector associated with A monotone sequence {Tn } of bounded,
it exists, is an eigenvalue of T. r r+1 r+2 the eigenvalue λn . Then, the vector en is self-adjoint, linear operators is a sequence
• The set of eigenvalues Sp (T) is at most Tλ (U) = Tλ (U) = Tλ (U) . . .
also an eigenvector of the Hilbert adjoint that is either monotonically increasing, i.e.,
countable, and its only possible limit If r > 0, the inclusions operator T? of T and associated with the T1  T2  T3  . . . , or monotonically de-
point is λ = 0. N (T0λ ) ⊂ N (T1λ ) ⊂ · · · ⊂ N (Trλ ) eigenvalue λ∗n . creasing, T1  T2  T3  . . . .
• If λ = 0 ∈ Q(T), then T is finite dimen- and • The null space satisfies N (T − λI) = • Let {Tn } be a monotonically increasing
sional. T0λ (U) ⊃ T1λ (U) ⊃ · · · ⊃ Trλ (U) N (T? − λ∗ I). sequence of bounded, self-adjoint, linear
• For every λ 6= 0 and every n = 1, 2, . . . , • For any µ 6= λ, the null spaces N (T − λI) operators such that T1  T2  . . . 
the null space N (Tnλ ) is finite dimensional are proper. Furthermore, the space U can and N (T − µI) are orthogonal to one an-
r
be represented as U = N (Tλ ) ⊕ Tλ (U). r Tn  . . .  S, where S is also bounded
and the range R(Tnλ ) is closed. other. and self adjoint. Suppose that all ele-
• For each complex number λ, the closed ments of the sequence commute pairwise
Linear Operators and Functionals on Hilbert Space linear subspace N (Tλ − λI) reduces T. and also commute with S. Then, {Tn } is
• kT2 k = kTk2 . strongly operator convergent, Tn h → Th
• A bounded linear operator T on a Hilbert for all h ∈ H, and the limit operator T
space H is normal iff kT? hk = kThk for is linear, bounded, self adjoint, and satis-
every h ∈ H. fies T  S.
• The residual spectrum Sr (T) of a normal A bounded, self-adjoint linear operator S
normal operators operator is empty. is called a square root of another bounded,
TT = T T
! ! • A complex number λ is in S(T) iff self-adjoint, linear operator T if S2 = T.
there exists a sequence {hn } with hn ∈ If, in addition, S  O, then S is called a
H, khn k = 1 for all n, such that k(T − nonnegative square root of T and is denoted
λI)hn k → 0 as n → ∞; in other words, by S = T1/2 .
continuous self-adjoint operators the operator T − λI is not bounded below.
linear • Every nonnegative, bounded, self-adjoint,
T=T ! • Let a bounded linear operator H on
operators I a Hilbert space H have the Cartesian linear operator T : H → H on a complex
Hilbert space H has a nonnegative square
T:H→H unitary
operators
decomposition H = T + iS, where T
root S that is unique.
and S are self-adjoint. Then, H is nor-
orthogonal mal iff T and S commute. In that • The square-root operator S of T com-
projections U : U = U
! −1
case, max kTk2 , kSk2 ≤ kHk2 ≤ kTk2 + mutes with every bounded linear operator
P on H that commutes with T.
O kSk2 .
• Let T and S be normal operators on Compact Normal Operators [2].
trace-class operators Let T : H → H be a normal operator on a
a Hilbert space H such that one com-
mutes with the adjoint of the other, i.e., nontrivial Hilbert space H, and let T have
Hilbert-Schmidt operators TS? = S? T and T? S = ST? , or such the Cartesian decomposition T = A + iB.
that the two operators commute, i.e., • The operator T is compact iff both A
TS = ST; then, T + S, TS, and ST are and B are compact.
compact operators normal. • The operator T is compact iff T? is com-
Bounded Self-Adjoint Linear Opera- pact.
tors [1, 2]. Let T : H → H be a bounded • If T is compact, it has an eigenvalue λ
self-adjoint linear operator on a complex with max{kAk, kBk} ≤ |λ|. If T is
Representation of Functionals [1, A bounded linear operator T : H → H on a Hilbert space H, let Tλ := T − λI, and self-adjoint, then it has an eigenvalue λ
§3.8]. Hilbert space H is said to be let h ∈ H. with λ = kTk.
• If T is compact and has no eigenvalues,
• Riesz Theorem: Every bounded linear ◦ normal, if TT? = T? T, • The set of all self-adjoint linear operators then H = {0}.
functional f on a Hilbert space H can be ◦ unitary, if T is bijective and if T? = T−1 , on H is a closed set in G(H, H). • If H is not separable, then λ = 0 is neces-
represented by an inner product f (h) = ◦ self adjoint or Hermitian, if T? = T. • The set of all self-adjoint linear operators sarily an eigenvalue of any compact nor-
hh, zi, where h ∈ H, and where z ∈ H If T is self adjoint or unitary, it is normal. on H forms a real normed linear space mal operator on H.
is uniquely determined by f and has under the operator norm.
norm kzk = kf k. Unitary Operators. Let the opera- • A bounded linear operator T on a com- Hilbert-Schmidt Operators [9, 2, 8].
• Riesz representation: Let H1 and H2 be tors U, V : H → H be unitary, H a Hilbert plex Hilbert space H is self adjoint Let {xn } be an orthonormal basis for a
Hilbert spaces, let h1 ∈ H1 , h2 ∈ H2 , space. Then, iff hTh, hi = hh, Thi is real for all h ∈ H. Hilbert space H. A bounded linear op-
and g : H1 × H2 → F a bounded • U is isometric, i.e., kUhk = khk for If H is a real Hilbert space, the direct erator T : H → H isPcalled a Hilbert-
∞
sesquilinear form. Then g has a represen- all h ∈ H, part holds but the converse is no longer Schmidt(HS) operator if n=1 kTxn k2 < ∞.
tation g(h1 , h2 ) = hSh1 , h2 i, where S : • kUk = 1, true. The number
H1 → H2 is a bounded linear operator ∞
• U−1 is unitary, • The spectrum S(T) of T lies in the closed X 1/2
that is uniquely determined by g and has • UV is unitary. intervall [mT , MT ] ∈ R, where kTkHS := kTxn k2
norm kSk = kgk. • A bounded linear operator on a Hilbert mT = inf hTh, hi, MT = suphTh, hi. n=1

Hilbert Adjoint Operator [1, 2]. Let T : space over the complex field is unitary iff khk khk is called the Hilbert-Schmidt norm of T.
H → Z be a bounded linear operator it is isometric and onto. Both mT and MT are spectral values of T. • The HS norm does not depend on the
that maps the Hilbert space H into the Polar Decomposition [8, §30]. Let T : • The operator norm of T is given by choice of orthonormal basis for H.
Hilbert space Z. The Hilbert adjoint op- H → H be a compact linear operator on a kTk = max(|mT | , |MT |) = sup |hTh, hi| . • The HS norm of a matrix is also called
erator T? of T is the operator T? : Z → H separable complex Hilbert space H; let T? khk the Frobenius norm.
such that hTh, zi = hh, T? zi for all h ∈ denote the Hilbert adjoint of T. • Eigenvectors that correspond to numeri- • If T is HS, then T? is HS, and kTk ≤
H and z ∈ Z. This operator exists, is • The operator T can be factored as T = cally different eigenvalues of T are orthog- kTkHS , as well as kTkHS = kT? kHS .
unique, and is a bounded linear operator onal • Every HS operator is compact; hence it
UA, where A is a positive Hermitian op- is bounded and continuous.
with norm kT? k = kTk. erator and U? U = I on the range of A. • A number λ belongs to the resolvent
Let S : H → Z be another bounded lin- set Rλ (T) iff there exists a c > 0 such • Every HS operator is the limit in HS-
The above factorization is called the po- norm of a sequence of operators with
ear operator, and let α be any scalar. The lar decomposition of T; the operator A is that kTλ hk ≥ ckhk for every h ∈ H.
Hilbert adjoint operator has the following • The product of two self adjoint linear op- finite-dimensional range.
called the absolute value of T. The polar •A P compact linear operator is HS
properties: decomposition exists even if T is bounded erators on a Hilbert space is self adjoint
• I? = I. only if the operators commute. iff n σn2 (T) < ∞.
instead of compact. • For a representation of a given Hilbert
• hT? h, zi = hz, Thi. • The absolute value A can be taken • Every bounded linear operator T : H →
• (S + T)? = S? + T? . H has a so-called Cartesian decomposi- space as L2 (H, M , µ) with positive mea-
as A := (T? T)1/2 , the unique positive sure µ and the corresponding collec-
• (αT)? = α∗ T? . square root of T? T; the operator U satis- tion: T = A + iB, where A and B are
• (T? )? = T. self-adjoint. tion M of measurable subsets, HS op-
fies U : Ah → Th for all h ∈ H. erators are those operators T that have a
• kT? Tk = kTT? k = kTk2 . • If T is compact, then its absolute value A ◦ The Cartesian decomposition is
• T? T = 0 ⇐⇒ T = 0. representation in
Z the form
is compact. unique.
• (ST)? = T? S? . ◦ A = 1/2(T + T? ). (Tf )(t) = k(t, s)f (s)dµ(s),
• If T can be represented by a matrix T, Singular Values [8, §30]. Let T : H → H ◦ B = 1/(2i)(T − T? ). H
then T? can be represented by TH . be a compact linear operator on a separable
? ◦ If λ is an eigenvalue of T, then λ = where f = f (t) ∈ L2 (H, M , µ), and the
Let T : H → H a bounded linear operator complex Hilbert space H, and let T be its α + iβ, where α is an eigenvalue of A integral kernel k(t, s) satisfies
that maps a Hilbert space H into itself. Hilbert adjoint. Furthermore, let T = UA be and β is an eigenvalue of B. ZZ
? the polar decomposition of T, and let {σn } 2
|k(t, s)| dµ(s)dµ(t) < ∞.
• The ranges and null spaces of T and T denote the set of nonzero eigenvalues of A; Nonnegative Self-Adjoint Linear Op-
are related as follows: H
they are all positive, as A is Hermitian. Let erators [1, §9]. Consider the set of • If T ∈ S(H), and f is a single-valued an-
◦ R(T) = N ⊥ (T? ). the σn be indexed in decreasing order. The all bounded, self-adjoint, linear operators alytic function on S(T) that vanishes at
◦ N ⊥ (T) = R(T? ). numbers σn are called the singular values on a complex Hilbert space H. A reflex- zero, then f (T) is a HS operator, and the
• Let T be continuous, and let M be a of T, denoted also as σn (T). ive partial ordering  on this set is de- mapping T → f (T) of S(H) into itsef is
closed linear subspace of H. Then, M • The singular values of T form an at most fined by T1  T2 iff hT1 h, hi ≤ hT2 h, hi continuous.
is invariant under T iff M⊥ is invariant countable sequence whose only possible for h ∈ H. A bounded, self-adjoint, lin-
under T? . ear operator T is said to be nonnegative The set S(H) of all HS operators on a
limit point is 0. Hilbert space H, together with the HS norm,
• A closed linear subspace M ⊂ H re- • Let the nonzero eigenvalues of T (although not strictly correct, sometimes
duces T iff M is invariant under both T also called positive) and denoted T  O, is a Banach algebra with kTSkHS ≤ kTkHS ·
be λ1 (T), λ2 (T), . . . , arranged in decreas- kSkHS for every T, S ∈ S(H). It contains
and T? . ing order of their absolute value, includ- if hTh, hi ≥ 0 for all h ∈ H.
• The Hilbert adjoint operator T? : Z → H operators of finite range as a dense subset.
0 0
ing multiplicities. Then, for any N ∈ Z+ • T1  T2 ⇐⇒ O  T2 − T1 . The set of HS operators is a self-adjoint
and the adjoint operator T× : Z → H N N • If two bounded, self-adjoint, linear oper- ideal in G(H, H), the Banach algebra of all
are related as T? = A1 T× A−1 ators T and S are nonnegative and com-
Y Y
2 , where A 1 : |λ n (T)| ≤ σn (T). bounded linear operators in Hilbert space.
0 0
H → H and A2 : Z → Z are bijective, n=1 n=1
mute, i.e., TS = ST, then their prod-
isometric, conjugate linear operators that
are uniquely defined by Riesz’s theorem.
Trace Class Operators [8, §30]. A com- as the product of two HS operators. Bounded Self-Adjoint Operators [1, fine f (T) as the limit p(T) of the polyno-
pact linear operator T : H → H on a Hilbert Let {xn } be any orthonormal basis of H. §9]. Let T : H → H be a bounded, self- mial Tn := pn (T) := αn Tn + αn−1 Tn−1 +
space H is said to be in trace class if For a trace class operator T, the trace is adjoint, linear operator on a Hilbert space H, · · · + α0 I for n → ∞, where pn (λ) is
X∞ defined as the limitXof the series let Tλ := T − λI, and define the positive part such that it converges uniformly to f (λ)
σn (T) < ∞. tr T := hTxn , xn i. of Tλ as T+ 2 1/2
λ := ((Tλ ) + Tλ )/2. Further- on [mT , MT ]. Then, the operator f (T)
n=1
n more, let Yλ := N (T+ ) denote the null space has the spectral representation
λ
The above sum defines the trace norm kTktr : This series converges absolutely. For trace +
of Tλ .
Z MT
∞ f (T) = f (λ)dEλ ,
X class operators
P T • Let Eλ with λ ∈ R be the projection of H mT −0
kTktr := σn (T). • tr T = n λn (T), where λn (T) are the +
onto the null space Yλ of Tλ . Then, the
n=1
and for all x, y ∈ H,
eigenvalues of T. collection E (T) := {Eλ : λ ∈ R} is the Z MT
For T in trace class and any bounded oper- • If T is a trace class operator that has no unique spectral family associated with T hf (T)x, yi = f (λ)dw(λ).
ator B : H → H eigenvalues except λ = 0. Then, tr T = 0. on the interval [mT , MT ] ∈ R. mT −0
• kTk ≤ kTktr , • |tr T| ≤ kTktr . • For λ < µ, the projection operator Eµ −
• kTktr = kT? ktr , ◦ The operator f (T) is self adjoint.
• tr T is a linear mapping of T. Eλ satisfies λ(Eµ − Eλ )  T(Eµ − Eλ ) 
• kBTktr ≤ kBktr · kTktr , ◦ If f (λ) = f1 (λ)f2 (λ), then f (T) =
• tr T? = (tr T)∗ . µ(Eµ − Eλ ).
• kTBktr ≤ kBktr · kTktr . f1 (T)f2 (T).
• For any bounded operator B, tr(TB) = • The mapping λ → Eλ has a discontinuity
• For any pair of trace class operators T ◦ If f (λ) ≥ 0 for all λ ∈ [mT , MT ], then
tr(BT). at λ0 , i.e., Eλ0 6= Eλ+ , iff λ0 is an eigen-
and S, T+S is trace class, and kT+Sktr ≤ Let T be a trace class operators, and 0 f (T)  O.
kTktr + kSktr . value of T. In this case, the eigenspace ◦ If f1 (λ) ≤ f2 (λ) for all λ ∈ [mT , MT ],
let {Tn } be a sequence of degenerate op- that corresponds to the eigenvalue λ0 then f1 (T)  f2 (T).
• The trace class is a two-sided ideal in the erators, i.e., operators with finite range that
algebra of all bounded linear operators converge to T in trace norm. Then, the is N (T − λ0 I) = (Eλ0 − Eλ+ )(H). ◦ kf (T)k ≤ maxλ∈[mT ,MT ] |f (λ)|.
0
on a complex Hilbert space. • A real λ0 belongs to the resolvent ◦ If a bounded linear operator commutes
determinant det(I + Tn ) of the matrix rep- set Rλ (T) iff there is an  > 0 such with T, it also commutes with f (T).
• Trace class operators form a Banach resentation of I + T , I + T , tends to a
space with respect to the trace norm.
n n that E (T) is constant on the interval [λ0 −
limit that is independent of the choice of Unitary Operators [1, §10.5]. Let U :
• Every trace class operator is HS. , λ0 + ].
the sequence {Tn }. This limit is called the H → H be a unitary operator on a complex
• The product of two HS operators T and S determinant of I + T: • A real λ0 belongs to the continuous spec-
Hilbert space H.
is in trace class, and kSTktr ≤ kSkHS · trum Sc (T) iff the mapping λ → Eλ is
kTk .
HS
det(I + T) := lim det(I + Tn ).
n→∞ continuous at λ0 (thus Eλ0 = Eλ+ ) and is • The spectrum S(U) is a closed subset of
0 the unit circle. Consequently, |λ| = 1 for
• Every trace class operator can be written not constant in any neighborhood of λ0 . every λ ∈ S(U).
A bounded, self-adjoint, linear operator T on • There exists a spectral family E = {Eλ }
The Spectral Theorem a complex Hilbert space H has the spectral on [−π, π] such that
representation Z π
Orthogonal Projection [2, 1]. A pro- ◦ The sequence {Pn } is strongly operator Z MT Z MT U= eiλ dEλ .
jection P : H → H on a Hilbert space H is convergent, say Pn h → Ph for all h ∈ H, T= λdEλ = mT EmT + λdEλ , −π
called an orthogonal projection if its range and the limit operator P is a projection mT −0 mT • for every continuous function f on the
and null space are orthogonal: R(P) ⊥ on H. where E = {Eλ } is the spectral family as- unit circle,
N (P). ◦ The limit operator P projects H onto sociated with T, and the integral is to be
Z π
f eiλ dEλ ,


• A bounded linear operator P : H → H on ∞ understood in the sense of uniform operator f (U) =
−π
[
a Hilbert space H is an orthogonal pro- P(H) = Pn (H). convergence in the norm on B(H, H).
jection if P is self adjoint and idempotent, where the integral is to be understood
n=1 • For x, y ∈ H,
i.e., P2 = P. in the sense of uniform operator conver-
◦ The limit operator P has the null space Z MT
gence.
• An orthogonal projection is continuous ∞ hTx, yi = λdw(λ),
(even if H is not complete).
\
mT −0
• For all x, y ∈ H,
N (P) = N (Pn ). Z π
• A continuous projection on a Hilbert n=1 where w(λ) := hEλ x, yi, and the integral hf (U)x, yi = f eiλ dw(λ),


space is orthogonal iff it is self-adjoint. is of Riemann-Stieltjes type. −π
• P  O; Spectral Family [1, 2]. A real spectral • Let f (λ) : [mT , MT ] → R be a continuous,
family, is a collection E := {Eλ : λ ∈ R} where w(λ) := hEλ x, yi, and the integral
• kPk ≤ 1 with equality if P(H) 6= {0}. real-valued function on [mT , MT ]. De- is an ordinary Riemann-Stieltjes integral.
• N (P) = R(P)⊥ and R(P) = N (P)⊥ . of projection operators Eλ on a Hilbert
• For any orthogonal projection P on a space H of any dimension that satisfy the
Hilbert space H and for any h ∈ H, following properties for any h ∈ H: Linear Operator Equations
hPh, hi = kPhk2 . ◦ Eλ ≤ Eµ for (λ ≤ µ); hence, Eλ Eµ =
• Each h ∈ H can be written uniquely Eµ Eλ = Eλ . Fredholm Alternative [1, §8.7]. A Let T : U → U be a compact linear operator
0 0
as r + n, where r ∈ R(P) and n ∈ N (P); ◦ limλ→−∞ Eλ h = 0. bounded linear operator S : U → U on a on a normed space U and T× : U → U its
◦ limλ→∞ Eλ h = h. normed space is said to satisfy the Fredholm adjoint operator. For x, y ∈ U, f, g ∈ U ,
0
furthermore, kxk2 = krk2 + knk2 . alternative if either one of the following con-
• Let M be any closed subspace of a Hilbert ◦ Eλ+ h := limµ↓λ Eµ h = Eλ h. and λ 6= 0, consider the set of linear operator
Special cases: ditions holds equations
space H. Then there is exactly one or-
• The nonhomogeneous equations Tx − λx = y (y ∈ X given) (OE1)
thogonal projection P with R(P) = M. • A countable resolution of the identity is
Let {en } be a countable orthonormal set a sequence {Pn } of orthogonal projection Sx = y, S× f = g
Tx − λx = 0 (OE2)
in H such that M = span{en }; then, the operators with P Pn Pm = O for n 6= m have unique solutions x and f , respec- 0
mapping P : H → H defined by so that I = n Pn , where the sum is
tively, for every given y ∈ U and g ∈
0
T× f − λf = g (g ∈ X given) (OE3)
strongly operator convergent. The se- U , and the corresponding homogeneous
T× f − λf = 0.
X
Ph := hh, en ien (OE4)
quence {Pn } defines a spectral family E equations
n with Then,
Sx = 0, S× f = o
for any h ∈ H is the orthogonal projec-
X
Eλ := Pn . • Equation (OE1) has a solution x
tion of H onto M. have only the trivial solutions x = 0iff f (y) = 0 for all solutions f of (OE4).
n≤λ and f = 0, respectively.
• Let M be a closed subspace of H, let h ∈ • A spectral family on an interval [a, b] ∈ R Hence, if f = 0 is the only solution
H, and let P be the orthogonal projection • The homogeneous equations of (OE4), then (OE1) is solvable for ev-
is a spectral family E that satisfies Eλ = Sx = 0, S× T = 0
on H with R(P) = M. Then, kh−Phk = O for λ < a and Eλ = I for λ ≥ b. ery y.
inf m∈M kh − mk. have the same number of linearly • Equation (OE3) has a solution f
Let P1 and P2 be orthogonal projections Compact Normal Operators [2]. independent solutions x1 , x2 , . . . , xN
iff g(x) = 0 for all solutions x of (OE2).
on a Hilbert space H, and let Y1 := P1 (H) Let T : H → H be a compact normal and f1 , f2 , . . . , fN , respectively, and the
Hence, if x = 0 is the only solution
and Y2 := P2 (H). operator on a Hilbert space H. corresponding nonhomogeneous equa- of (OE2), then (OE3) is solvable for ev-
• The composite operator P := P1 P2 is a • There is a countable resolution of the tions ery g.
projection on H iff P1 and P2 commute. identity {Pn } and a sequence of complex Sx = y, S× f = g
• Equation (OE1) has a solution x for ev-
In this case, P projects H onto Y = numbers {µn } such that ery y ∈ U iff x = 0 is the only solution
are not solvable for all y and f , respec-
Y1 ∩ Y2 . Conversely, the projection
X tively. They have a solution iff y and g
of (OE2).
T= µn Pn , • Equation (OE3) has a solution f for ev-
are such that fn (y) = 0 and g(xn ) = 0
onto span{Y1 , Y2 } is P1 + P2 − P1 P2 . n for all n = 1, 2, . . . , N . 0
• The sum P := P1 + P2 is a projection ery g ∈ U iff f = 0 is the only solution
where convergence is uniform in the op- For a compact linear operator T on a normed
of (OE4).
operator iff Y1 ⊥ Y2 . In this case, P erator norm. space U, the operator Tλ := T − λI, for λ 6=
• Equations (OE2) and (OE4) have the
projects H onto Y1 ⊕ Y2 . • There exists an orthonormal basis {en } 0, satisfies the Fredholm alternative. same number of linearly independent so-
• The difference P := P2 − P1 is a projec- of eigenvectors and a corresponding se-
tion on H iff Y1 ⊂ Y2 . In this case, P Linear Operator Equations [1, §8.5]. lutions.
quence ofP eigenvalues {λn } such that,
projects H onto the orthogonal comple- if h = n hh, en ien is the Fourier ex-
ment of Y1 in Y2 . pansion for h ∈ H, then
• The following conditions are equivalent X
Th = λn hh, en ien .
◦ P 2 P1 = P1 P2 = P1 ,
n
◦ Y1 ⊂ Y 2 , P
◦ N (P1 ) ⊃ N (Y2 ), • A weighted sum of projections n λn Pn ,
◦ kP1 hk ≤ kP2 hk for all h ∈ H, where {Pn } is a resolution of the iden-
◦ P 1  P2 . tity, and {λn } is a sequence of com-
Let {Pn } be a monotonically increasing plex numbers, is compact if (i) for every
sequence of projection operators Pn on a nonzero λn , the range of Pn is finite di-
Hilbert space H. Then, mensional, and (ii) for every real α > 0,
the number of λn with |λn | ≥ α is finite.
References [5] ——, Functional Analysis, 2nd ed. New York, NY, U.S.A.: McGraw-Hill, 1991.

[1] E. Kreyszig, Introduction to Functional Analysis with Applications. New York, NY, [6] N. Dunford and J. T. Schwarz, Linear Operators, New York, NY, U.S.A., 1958, vol. 1.
U.S.A.: Wiley, 1989.
[7] R. L. Allen and D. Mills, Signal Analysis: Time, Frequency, Scale, and Structure.
[2] A. W. Naylor and G. R. Sell, Linear Operator Theory in Engineering and Science. Piscataway, NJ, U.S.A.: IEEE Press, 2004.
New York, NY, U.S.A.: Springer, 1982.
[8] P. D. Lax, Functional Analysis. New York, NY, U.S.A.: Wiley, 2002.
[3] M. Pourahmadi, Foundations of Time Series Analysis and Prediction Theory. New
York, NY, U.S.A.: Wiley, 2001. [9] N. Dunford and J. T. Schwarz, Linear Operators. New York, NY, U.S.A.: Wiley,
1963, vol. 2.
[4] W. Rudin, Real and Complex Analysis, 3rd ed. New York, NY, U.S.A: McGraw-Hill,
1987.