MAT335 Notes

⃗
u

⃗ α = αu
w ⃗ + (1 − α)v⃗

⃗v

Jason Siefken
 Dynamical Systems

Dynamical System
A pair (T, X ) is called a dynamical system if X is a set and T : X → X is a function. In the
DEF

context of a dynamical system, T is often called a transformation.

This definition is very general, and most things you encounter could be considered a dynamical
system. For example, if X = {air molecules and their positions on earth} and T : X → X is the
result of the wind blowing for one second, then (T, X ) is a dynamical system. Alternatively, we
could take the state of your computer’s memory (RAM) to be a set and your processor executing
a single instruction to be a transformation.
It’s hard to say much about general dynamical systems. However, throughout the course, we
will find ways to classify dynamical systems. Once we “narrow the field”, we’ll be able to say
lot’s of interesting things.

Newton’s Method
Newton’s method is a way of using tangent-line approximations to functions to estimate their
roots. It is an iterative procedure.1

1 Let f : R → R be a differentiable function and let T f : {guesses} → {guesses} be a single

application of Newton’s method.
1.1 Find a general formula for T f .
1.2 Let f (x) = x(x − 2)(x − 3). Compute T fn (4) for n = 0, 1, 2, 3.
1.3 Do you think
lim T fn (4)
n→∞

converges? If so, what does it converge to? Can you prove your answer?

Fixed Point
DEF

Let (T, X ) be a dynamical system. A point a ∈ X is called a fixed point if T (a) = a.

Basin of Attraction
Let (T, X ) be a dynamical system and let x ∈ X . The basin of attraction of x is the set
DEF

A x = { y ∈ X : lim T n y = x}.
n→∞

Eventually, we will talk about more general basins of attraction, but for now we will limit
ourselves to that of a single point.

2 Let f (x) = x(x − 2)(x − 3) and let T f be the function that applies a single iteration of Newton’s
method (as before).
2.1 Is [3, 4) ⊆ A3 for T f ? What about [100, 1000]? (2, 3)?
2.2 Describe A3 .
2.3 Is A3 connected?

1
Whenever something is iterative, you should think dynamics!
1 ⃝
c Jason Siefken, 2015–2019
 Inverse Image
Let f : A → B be a function and let X ⊆ B. The inverse image of X under f , denoted
DEF

f −1 (X ), is
f −1 (X ) = {x ∈ A : f (x) ∈ X }.

Note: a function need not be invertible to have inverse images. In fact, the idea of inverse
images applies to every function.

3 3.1 Let g : R → R be defined by g(x) = x 2 . Find g −1 ({1}), g −1 ({4}), g −1 ({0}), g −1 ({−1}), and
g −1 ([3, 4]).
3.2 Let f and T f be as before. (Recall, f (x) = x(x − 2)(x − 3)). Find T f−1 ([3, 4)).
3.3 Define ⋃︂
Q= T f−n ([3, 3.1))
n≥0

where T f0 is the identity function.

Is Q = A3 ? Why or why not?

2 ⃝
c Jason Siefken, 2015–2019
Using a computer, we can graph A0 , A2 , and A3 .

3 ⃝
c Jason Siefken, 2015–2019
Zooming in around x ≈ 2.4588:

We’ve just seen our first fractal! For now, we will define a fractal as a set with repeated patterns
at all scales.

4 ⃝
c Jason Siefken, 2015–2019
 Fractals
Let’s construct some famous fractals.

4 Let K0 be an equilateral triangle with sides of length 1. Let K1 be the result of applying
→ to each side of K0 . Repeat this process to get K2 from K1 , etc. and define

K∞ = lim Kn .
n→∞

4.1 Draw K0 , K1 , and K2 .

4.2 Find the perimeter of K0 , K1 , and K2 . Find a general formula for the perimeter of Kn .
4.3 What is the perimeter of K∞ ? What is the area enclosed by K∞ ?
K∞ is called the Koch Snowflake.

5 Let T0 be a filled-in equilateral triangle. To get T1 , T2 , etc., apply the substitution rule
→ to each (sub)triangle of T0 , T1 , etc.. Define T∞ to be the limit of this process.
5.1 Draw T0 , T1 , and T2 .
5.2 Find a formula for the area of Tn .
5.3 Compute the area of T∞ .
5.4 Is T∞ the empty set? Why or why not?
T∞ is called Sierpinski’s Triangle.

6 Let C0 = [0, 1] be the unit interval. Recursively define Ci by the substitution rule → ,
which removes the middle 1/3 of every interval. Define C∞ to be the limit of this process.
6.1 Compute the length of Cn .
6.2 Compute the length of C∞ .

C∞ is called the Cantor set.

Our normal sense of measurement fails when it comes to these fractals. We need a new idea:
similarity dimension.

5 ⃝
c Jason Siefken, 2015–2019
 Dimension

Dimension can be thought of as a relationship between scale and content.2

7 7.1 Let ℓ = [0, 1), 2ℓ = [0, 2), 3ℓ = [0, 3), etc.. How many disjoint copies of ℓ does it take to cover
nℓ?
7.2 Let S = [0, 1)2 , 2S = [0, 2)2 , etc.. How many disjoint copies of S does it take to cover nS?
7.3 Let C = [0, 1)3 , 2C = [0, 2)3 , etc.. How many disjoint copies of C does it take to cover nC?
7.4 Based on the patterns you see, describe an algorithm that can be used to find the dimensions
of ℓ, S, and C.
7.5 Let T be the filled-in equilateral triangle. Apply your algorithm to 2T .
7.6 Let T∞ be the Sierpinski triangle. Apply your algorithm to 2T∞ . Does the number you get
make sense?
Similarity Dimension
A set Q ⊆ Rn has similarity dimension d if there exists a c ∈ Z and s ∈ R≥1 satisfying
DEFINITION

d = logs (c)

and sQ (Q scaled up by a factor of s) is covered by c copies of Q (at most overlapping on

their boundaries).

8 8.1 Compute the similarity dimension of a (a) a line segment, (b) the Cantor set, and (c) Sierpinski’s
triangle.
8.2 Compute the similarity dimension of the Koch snowflake.

What about sets that aren’t self-similar?

9 ′
Let K∞ be the “Koch snowflake” obtained with the substitution rule → .
′
9.1 Find the perimeter and dimension of K∞ .
9.2 Let Kstrange be the “Koch snowflake” obtained by the rule → or chosen randomly
at each stage. What should the dimension of Kstrange be? Can you compute it’s similarity
dimension?

2
Here “content” refers to the “stuff inside” of an object.
6 ⃝
c Jason Siefken, 2015–2019
 We need a way to define dimension for shapes that aren’t self-similar. Let’s again work from
sets whose dimension we know: cubes.
Box Covering
A d-dimensional box covering of X ⊆ Rk is a collection C = {Bi } of d-dimensional cubes
which satisfy
DEFINITION

1. if i ̸= j, Bi and B j intersect at most on their boundaries;

2. Bi ∩ X ̸= {} for all i;
⋃︁
3. X ⊆ i Bi .

Outer Measure
The d-dimensional outer measure of X ⊆ Rk is
DEFINITION

lim inf volume(Cn )

n→∞ Cn

where Cn is a d-dimensional box covering of X with cubes of side-length 1/n.

You should think of infCn volume(Cn ) as the “smallest possible box covering of size 1/n that
still covers the set”.

10 ⃗ to (1, 0, 0).
Let ℓ ⊆ R3 be the line segment from 0
10.1 Find the 1, 2, and 3-dimensional outer measures of ℓ.
10.2 Does ℓ have a 0-dimensional outer measure?
10.3 Let T ⊆ R3 be the filled in triangle with vertices (0, 0, 0), (1, 0, 0), and (0, 1, 0). Find the 1, 2,
and 3-dimensional outer measures of T .
10.4 Find the 1-dimensional outer measure of the Cantor set.

7 ⃝
c Jason Siefken, 2015–2019
 What would it mean to have a fractional-dimensional outer measure? Let B be a d-dimensional
box with side lengths k. Its volume is k d . Divide the box in half along every dimension and
each sub-box has volume (k/2)d = (1/2)d k d , and so there must be 2d sub-boxes.
What if there were fewer “sub-boxes”?

11 Let Cα be the Cantor-like set obtained by removing the middle α of each subinterval. (I.e., the
standard Cantor set is C1/3 .)
11.1 Find the number of boxes in a 1-dimensional box-covering of C0 (the interval) and C1/3 where
the width of each box is 1/3, 1/9, 1/27, etc..
11.2 Based on what you know about how many width-k boxes it takes to fill d-dimensional space,
find a formula relating d, the number of boxes, and the width of the boxes.
11.3 Use your formula to estimate d for C1/3 . How does this compare to the similarity-dimension of
C1/3 ?

Box-counting Dimension
Let X ⊆ Rm and let B ⊆ Rm and let B be a minimal-dimensional, minimally-sized box such
that X ⊆ B. The box-counting dimension of X is
DEFINITION

log(# of sub-boxes of Bn that intersect X )

d = lim ,
n→∞ log n

where Bn is B “cut” along each axis into n equally-spaced slices.

11.4 Find the box-counting dimension of C1/3 .

⃗ ∈ R2 : ⃗v = αe⃗1 +
11.5 Find the box-counting dimension of the unit simplex in R2 . I.e. {v
β ⃗e2 for some α, β ≥ 0 satisfying α + β ≤ 1}
11.6 Intuitively, what should limα→0 dim(Cα ) be? Find the box-counting and similarity dimension of
Cα and verify.
Box-counting dimension is difficult to compute exactly, but it’s useful for approximations.
Computers are pretty good at counting boxes!

8 ⃝
c Jason Siefken, 2015–2019
 Markov Chains

Transition Matrix
Let G be a directed graph with vertices {1, . . . , n}. A transition matrix for G is an n × n
DEFINITION

matrix A = [ai j ] where

¨
1 if there is an edge from j to i
ai j = .
0 otherwise

12 The map shows the direct, one-way flights offered by the Pacific Rim air shipping company.

Ottawa London

Los Angeles Tokyo

Singapore

We can think of their flight map as a graph (in the graph-theory sense).
12.1 Write down a transition matrix A for the above graph.

(a) What do the diagonal entries tell you about the available flights?
(b) Should ai j = a ji ? Explain.

12.2 Write down a matrix B = [bi j ] where the entry bi j indicates the number of ways to take exactly
two flights from city i to city j.

(a) Compute A2 and compare with B.

(b) What information does the 1st row of A give you about flights?
(c) What information does the 2nd column of A give you about flights?
(d) Based upon your last two answers what does the 1, 2 entry of A2 tell you about flights?

12.3 Compute A3 . What does it tell you about shipping routes?

12.4 A package with a lost tracking number is getting kicked around from route to route! Each time
it lands, it is randomly (and with equal probability) put on another flight. After several weeks
(and 100s of flights), the package is finally noticed. Where is it most likely to be?

9 ⃝
c Jason Siefken, 2015–2019
 Markov Chain
Given a graph G, a stationary Markov chain on G, is a random process, denoted X 1 , X 2 , . . .
DEF
where X i indicates the location on the graph at the ith step and the probability distribution
of X i is completely determined by the value of X i−1 .

Some people call stationary Markov chains “memoryless” processes because what happened
more than one step prior has no affect on what happens in the next step.

13 Consider the following directed graph with transition probabilities labeled.

1/2
1/2
a b
1

13.1 Find a transition matrix, T , for the graph.

13.2 Find a matrix P1 whose i, j entry represents the probability of transitioning from state j to state
i in exactly one step.
13.3 Find a matrix P2 whose i, j entry represents the probability of transitioning from state j to state
i in exactly two steps.
13.4 How do P2 and P12 relate?
13.5 Does limn→∞ P1n exist? If so, what is it?

Stochastic Matrix
A vector ⃗v ∈ Rn is called a probability vector if its entries are non-negative and sum to one.
DEF

A matrix is called a stochastic matrix if its columns are probability vectors.

In the context of Markov chains, we also refer to stochastic matrices as transition matrices.3

14 You’re a picky eater. You have meals of healthy food and dessert, but you never have healthy
food twice in a row. Each time you eat dessert, you flip a weighted coin to decide what meal to
eat next. Let p represent the weight of the coin.
14.1 Are your eating habits described by a Markov chain? Why or why not?
14.2 Draw a graph representing this situation.
14.3 Your friend visits you for New Years and sees your having a healthy dinner. You lose touch after
that, but bump into each other at a restaurant six years later. What type of meal are you most
likely to be eating? Does this depend on p?

15 15.1 Is a Markov chain a dynamical system? Why or why not?

• ˜
1/2 1
15.2 Consider the Markov chain with states {a, b} given by the transition matrix . Given
1/2 0
that you start in state a, give probability vectors indicating the probability of being in a particular
state after 0, 1, 2, and 3 steps along the Markov chain.
15.3 Can this Markov chain be modeled by a dynamical system? If so, describe such a model.

3
Yes, this word is doing double duty!
10 ⃝
c Jason Siefken, 2015–2019
 Iterated Function Systems (IFS)

Realization of a Markov Chain

Given a Markov Chain M = (X 1 , X 2 , . . .) with state space S, a realization of M is a sequence
DEFINITION

of states whose transitions are allowed by the Markov chain. I.e., an allowed element of S N .
A realization ⃗r ∈ S N is called generic for M if the transition probabilities for M can be
recovered from ⃗r by averaging.

16 Consider the Markov chain M with state space [0, 1] and the transition rule
¨
X i /3 with probability 1/2
X i+1 =
X i /3 + 2/3 with probability 1/2

16.1 Using a random number generator, write down the initial segment (up to 4 transitions) of two
different realizations of M.
16.2 Let ⃗r = (r0 , r1 , . . .) be a realization of M. Is it possible that limi→∞ ri exists? Why or why not?
16.3 Let ⃗s = (s0 , s1 , . . .) be a realization for M that is generic. Is it possible that limi→∞ si exists?
Why or why not?
16.4 Can M be modeled by a dynamical system? If so, describe the model.
16.5 Suppose we start off with a uniform distribution on [0, 1]. Draw the resulting distribution after
1, 2, and 3 steps along M.

11 ⃝
c Jason Siefken, 2015–2019
 Iterated Function System (IFS)
An iterated function system with functions ( f1 , . . . , f n ) and transition probabilities (p1 , . . . , pn )
is a stationary Markov chain where transitions are given by the rule
DEFINITION

⎨ f1 (X i ) with probability p1
⎧
⎪
X i+1 = f2 (X i ) with probability p2
⎪ ..
.
⎩

Invariant Set
Let I be an iterated function system with functions ( f1 , . . . , f n ) and non-zero transition
DEFINITION

probabilities (p1 , . . . , pn ). The set X is called an invariant set for I if

⋃︂
X= f i (X ).
i

17 Let I be the iterated function system with transition probabilities (1/2, 1/2) and functions
( f1 : [0, 1] → [0, 1], f2 : [0, 1] → [0, 1]) given by f1 (x) = x/3 and f2 (x) = x/3 + 2/3.
17.1 Find an invariant set for I .
17.2 Is the Cantor set an invariant set for I ?
17.3 Is there a larger invariant set for I than the Cantor set? Why or why not?
17.4 If the probabilities for f1 and f2 change, will that affect the invariant sets?

18 Define f a⃗ : [0, 1]2 → [0, 1]2 by f a⃗ (x⃗ ) = ⃗x /2 + a⃗ . Let F be the iterated function system with
functions ( f0⃗ , f(1/2,0) , f(0,1/2) ) and equal probabilities.
18.1 Draw a maximal invariant set for F .
18.2 Can you create an iterated function system so that the Sierpinski triangle is an invariant set? If
so, do it!

12 ⃝
c Jason Siefken, 2015–2019
 Continuous-time Dynamical Systems
The dynamical systems we’ve seen so far involve taking one “step” at a time. However, as we
experience life, it seems like time advances continuously, not discretely. Can we capture that in
a dynamical system?
Continuous-time Dynamical System
Let X be a set and let (T a : X → X )a∈R be a family of functions satisfying T a ◦ T b = T a+b .
DEF

Then, (T a , X ) is called a continuous time dynamical system.

19 19.1 Let S a : R → R be defined by S a (x) = a x. Is (S a , R) a continuous time dynamical system?

19.2 Let T a : R → R be defined by T a (x) = 2a x. Is (T a , R) a continuous time dynamical system?
19.3 Let F = {functions from R to R} and define E t : F → F by E t ( f (x)) = f (x + t). Is (E t , F ) a
continuous time dynamical system?

13 ⃝
c Jason Siefken, 2015–2019
20 20.1 Consider the following garden paths.

Let P represent the set of points on the paths and define W t : P → P to be the result of walking
at unit speed counter-clockwise around your path for t seconds.
Is (W t , P) a continuous time dynamical system?
20.2 You’re watching the surface of a lake and carefully map out the velocity of the water at each
point. You produce the following picture of velocities.

Let S be the points on the surface of the lake and let Φ t : S → S be the result of “flowing” along
the surface for t seconds. Is (Φ t , S) a continuous time dynamical system?
Systems like (Φ t , S) come up often because they are described by local rules. That is, recorded
in the picture is information about where you flow after tiny time increments—finding where
you end up after bigger time increments takes work.

14 ⃝
c Jason Siefken, 2015–2019
 Vector Field
Given a set X , a n-dimensional vector field on X is a function F : X → Rn which assigns a
DEF
vector to each point in X .

Vector fields show up a lot in physics because velocities and accelerations of particles in a fluid
naturally produce vector fields.

21 Let V : R2 → R2 be a vector field and let Φ t : R2 → R2 be the function which “flows” points
along the vector field with velocity at a given point equal to the vector at that point.
21.1 Find a V so that all points flow vertically at unit speed.
21.2 Find a V so that all points flow radially outward and increase in speed.
21.3 Find a V so that all points flow around in a circle.
21.4 Find a V so that all points flow around in a circle at the same speed.
21.5 Find a V so that all points flow around in a circle with the same period (that is, it takes the
same amount of time for any given point to go around the circle).
Flow & Orbit
Let (W t , X ) be a continuous time dynamical system and let x ∈ X . The flow of W t starting
at x is the function f : R → X defined by
DEFINITION

f (t) = W t (x).

The orbit of x under W t is the set { f (t) : t ∈ R}, where f is the flow of W t starting at x.
We call the set { f (t) : t ∈ R+ } the forward orbit of x.

We will often notate the flow of W t starting at x by φ t (x). As a flow, we think of t as the
variable.

22 Let V : R2 → R2 be a vector field and let Φ t : R2 → R2 be the function that flows points along
the vector field.
22.1 If possible, find a V so that some orbits are infinitely long.
22.2 If possible, find a V so that some orbits are not infinitely long.
22.3 If possible, find a V so that some orbits are infinitely long and others are not.
22.4 If possible, find a V so that all orbits are finite line segments.
22.5 If possible, find a V so that the forward orbit of every point ends up at ⃗e1 .
⃗ or ⃗e1 (and at least one
22.6 If possible, find a V so that the forward orbit of every point ends up at 0
⃗ and one towards ⃗e1 ).
orbit heads towards 0

15 ⃝
c Jason Siefken, 2015–2019
 Stable & Unstable Points
Let (W t , Rn ) be a continuous time dynamical system and let φ t (w ⃗ ) represent the flow of
W t starting at w ⃗ ∈ Rn . We call the point ⃗x ∈ Rn stable if for all ϵ > 0, there exists a δ > 0
DEFINITION

so that for all ⃗y ∈ Rn ,

∥x⃗ − ⃗y ∥ < δ implies ∥φ t (x⃗ ) − φ t ( ⃗y )∥ < ϵ for all t > 0.

Otherwise, ⃗x is called unstable.

23 Given a 2 × 2 matrix M , define a vector field VM : R2 → R2 by VM (x⃗ ) = M ⃗x . Consider the

following matrices and their associated vector fields:
• ˜ • ˜ • ˜ • ˜ • ˜ • ˜
1 0 −1 0 0 −1 1 −1 −1 −1 0 1
I= A= B= C= D= E=
0 1 0 −1 1 0 1 1 1 −1 1 0

23.1 Explain in plain language what it means for x to be a stable point of a continuous dynamical
system.
23.2 Use an online plotter (for example https://www.desmos.com/calculator/eijhparfmd or
https://anvaka.github.io/fieldplay) to plot VX for X ∈ {I, A, B, C, D, E} and determine
which points are stable/unstable for each.
23.3 Compute the eigenvalues for each of the matrices. Can you relate the eigenvalues to stable/un-
stable points?

16 ⃝
c Jason Siefken, 2015–2019
 Differential Equations
To really get a handle on what’s going on, let’s think about some differential equations!

24 24.1 Let a be a constant. Find a non-trivial solution to the differential equation f ′ (t) = a f (t).
24.2 Consider the (boring) vector field V : R1 → R1 defined by V (t) = at. Find a non-trivial flow
for the corresponding dynamical system. Is V given by matrix multiplication? If so, what’s the
matrix?
• ˜
a 0
24.3 Let D = and define the vector field W : R2 → R2 by W (x⃗ ) = Dx⃗ . Find a non-trivial
0 b
flow for the corresponding dynamical system.
⃗ is stable/unstable with respect to the corresponding
Can you give conditions on a and b so that 0
dynamical system? If a and b are complex numbers, does that change your answer?
24.4 Suppose (W t , Rn ) is a continuous dynamical system. Define what the derivative of W t with
respect to time should mean. How does this relate to vector fields and flows?
24.5 Prove that if f : R → Rn is a function and M is a matrix, then (M f )′ = M ( f ′ ).
24.6 Suppose A is a matrix and A = P DP −1 where D is diagonal. Further, suppose f is a solution to
f ′ (t) = D f (t). Find a solution to the differential equation Φ′ (t) = AΦ(t).
24.7 Let M be a matrix and consider the continuous dynamical system coming from the vector field
⃗ based on the eigenvalues of M .
⃗x ↦→ M ⃗x . Classify the behaviour near 0

25 Consider the continuous dynamical systems (W1t , R2 ), (W2t , R2 ), (W3t , R2 ), and (W4t , R2 ) given
by flows along the vector fields

x + y2 −x + y 2
• ˜ • ˜ • ˜ • ˜ • ˜ • ˜ • ˜ • ˜
x −y x x −y x
V1 = V2 = V3 = V4 =
y x y −y y x + y2 y −y

25.1 Classify (0, 0) as stable or unstable for each system.

25.2 Classify (0.01, 0.01) as stable or unstable for each system.
25.3 Conjecture: under what conditions will a first-order approximation about at a fixed point tell
you about stability near that fixed point.
Structural Stability
DEFINITION

A continuous dynamical system (W t , X ) is called structurally stable at ⃗x ∈ X with re-

spect to fixed points/stable points/periodic points/etc. if the topology of the set of fixed
points/stable points/periodic points/etc. near ⃗x remains invariant with respect to small
perturbations of W t .

26 26.1 What should a small perturbation of (W t , Rn ) mean?

26.2 What should it mean for the topology of a set to be invariant?
⃗?
26.3 Out of the examples in 25, which are structurally stable at 0
26.4 Revisit your conjecture from 25.3. Can you rephrase it in terms of structural stability?

17 ⃝
c Jason Siefken, 2015–2019
 Time-1 Map
Let (W t , X ) be a continuous dynamical system. The time-1 map associated with (W t , X ) is
DEFINITION

the dynamical system (T, X ) where

T (x) = W 1 (x)

for all x ∈ X .

27 Let S be a circle of circumference 1 and let (Wkt , S) be the continuous dynamical system that
flows points counter-clockwise along S at speed k.
27.1 For which k is the time-1 map associated with (Wkt , S) trivial?
27.2 For which k does the time-1 map associated with (Wkt , S) have periodic points?

18 ⃝
c Jason Siefken, 2015–2019
 Circle Maps
Let S be a circle of circumference 1. We can associate S with the unit interval [0, 1] provided
we “glue” the endpoints together. We write [0, 1]/0 ∼ 1 to notate the set [0, 1] where 0 and 1
are considered the same point. That is, if you move to the right starting at 0.99, you’ll wrap
around and end up at 0.01.
Given a function f : [0, 1) → R, we can create a function g : [0, 1) → [0, 1) via the formula

g(x) = f (x) mod 1.

That is, compute the value f (x) with “wrap around”, and that’s what g(x) is.
Symbolic Coding
Let (T, X ) be a discrete dynamical system and let P = {Pa , Pb , . . .} be a partition of X . A
symbolic coding of x ∈ X relative to the partition P is the sequence
DEFINITION

C(x) = (c0 , c1 , . . .)

where ⎧
⎨a
⎪ if T i x ∈ Pa
ci = b if T i x ∈ Pb
⎩ ..
⎪
.

28 Let (Wkt , S) be as in 27. We can describe this dynamical system by

Wkt (x) = x + kt mod 1.

Let Pa = [0, 1/2) and Pb = [1/2, 1) be a partition of [0, 1]/0 ∼ 1, and let (Tk , S) be the time-1
map for (Wkt , S).
28.1 Let k = 0.25. Find C(0), C(1/5), and C(1/3) for Tk , {Pa , Pb }.
28.2 Suppose C(x) = (a, a, b, b, a, a, b, b, . . .). What can you say about x and k?
28.3 Is (a, b, a, a, b, b, a, a, a, b, b, b, . . .) the symbolic coding of any point? Why or why not?
28.4 For which k can x be exactly recovered from C(x)?
28.5 Let {a, b}N be the set of sequences of a’s and b’s. We can think of C : S → {a, b}N as a function.
For which k is C one-to-one? Onto?

19 ⃝
c Jason Siefken, 2015–2019
 The Doubling Map

Orbit
Let (T, X ) be a discrete dynamical system. The orbit of a point x ∈ X under T is the set
DEFINITION

O(x) = {T i x : i ∈ Z}

if T is invertible and 
O(x) = T i x : i ∈ N ∪ {0}
if T is non-invertible.

The doubling map is the function T : [0, 1) → [0, 1) defined by x ↦→ 2x mod 1.

29 Let P = {P0 , P1 } = {[0, 1/2), [1/2, 1)} be a partition of [0, 1) And let C : [0, 1) → {0, 1}N be
the coding function arising from the doubling map and the partition P .
29.1 Is the map Q : R → R defined by x ↦→ 2x invertible? Is the doubling map invertible? Explain.
29.2 Find O(1/3), O(4/5), and O(0) under the doubling map.
29.3 Find C(1/3), C(4/5), and C(0).
29.4 Suppose O(x) is a finite set. What can you say about C(x)?
29.5 Suppose x ̸= y. Is it possible that C(x) = C( y)?
29.6 Is C invertible?
29.7 Define E : {0, 1}N → [0, 1) by ∑︂ a
i
(a1 , a2 , a3 , . . .) ↦→ .
2i
Compute E(C(1/2)), E(C(1/3)), and E(C(4/5)).
29.8 What is E ◦ C? How about C ◦ E?
Base-n Expansion
A number x ∈ [0, 1) has a base-n expansion 0.d1 d2 d3 · · · if
DEF

∑︂ d
i
x= .
ni

30 30.1 Explain how base-2 representations relate to dynamical systems.

30.2 Find a dynamical system and a partition so that C produces base-10 representations.
30.3 Is the map from a number to its base-n representation invertible? Explain.

20 ⃝
c Jason Siefken, 2015–2019
 The Shift Space

Full Shift
Let Ω = {0, 1, . . . , n − 1}N and define σ : Ω → Ω by
DEFINITION

σ(a0 , a1 , a2 . . .) = (a1 , a2 , . . .).

The dynamical system (σ, Ω) is called the full shift on n symbols.

The dynamical system (σ, Ω) is also called a shift space and is equipped with a topology coming
from the standard metric on Ω. This metric determines which sequences converge.
Standard Metric on Ω
The standard metric on Ω is the function d : Ω × Ω → [0, ∞) defined by
DEFINITION

¨
0 if x = y
d(x, y) =
2 −i
where i = min j { j : x j ̸= y j }

where x = (x 0 , x 1 , . . .) and y = ( y0 , y1 , . . .) are points in Ω.

31 Let (σ, Ω) be the full shift on two symbols.

31.1 Let x = (1, 0, 1, 0, 1, 0, . . .) and y = (0, 0, 0, 0, 1, 0, 1, 0, . . .). Find d(x, y), d(σ(x), σ( y)),
d(σ2 (x), σ2 ( y)), . . . .
⃗ = (1, 1, 1, . . .). Find points a, b ∈ Ω so that
31.2 Let 1

⃗) ↗ 1
d(σ n (a), 1 and ⃗) ↘ 0.
d(σ n (a), 1

31.3 Let (T, [0, 1)) be the doubling map and let C be the coding function for the partition {[0, 1/2), [1/2, 1)}.
You can view C as a function from [0, 1) → Ω.
For t = 1/3, compute C(t), C ◦ T (t), and σ ◦ C(t). What do you notice? Will this always
happen?
31.4 Is C([0, 1)) ⊆ Ω closed? (I.e., does it include all its limit points?)
31.5 Let t, t ′ ∈ [0, 1) and let a = C(t) and a′ = C(t ′ ).

(a) If d(a, a′ ) is small, what can you say about |t − t ′ |?

(b) If |t − t ′ | is small, what can you say about d(a, a′ )?

21 ⃝
c Jason Siefken, 2015–2019
 The full 2-shift isn’t a perfect representation of the doubling map, but it’s close! And, it turns
out to be much easier to study.
Subshift
DEFINITION

Let (σ, Ω) be the full shift on n symbols. The system (σ, X ) is called a subshift provided
1. X ⊆ Ω is a closed set (i.e., it contains all its limit points); and

2. X is σ-invariant (i.e., σ(X ) = X ).

32 Consider the circle rotation with k = 1/4. Let O(0) be the orbit of 0 under this map, and define
X = C(O(0)) with the usual partition.
32.1 Write down X .
32.2 Is (σ, X ) a subshift?
32.3 For each x ∈ X , find a corresponding x ′ ∈ [0, 1) so that x = C T (x ′ ) where C T is the coding
with respect to the doubling map and the standard partition.
32.4 Can you model a circle rotation with the doubling map/subshift?

22 ⃝
c Jason Siefken, 2015–2019
 • ˜
33 .5 1
Let M be the Markov chain with transition matrix and let M′ be the Markov chain
.5 0
• ˜
.5 .9
with transition matrix .
.5 .1
Let X = {realizations of M} and let X ′ = {realizations of M′ }.
33.1 Is (σ, X ) a subshift? Why or why not? What does σ mean in terms of the original Markov
chain?
33.2 Is (σ, X ′ ) a subshift? What does it “look” like?
33.3 Let Y be the set of generic realizations of M. Is (σ, Y ) a subshift?

23 ⃝
c Jason Siefken, 2015–2019
 Word
Fix Ω = {0, . . . , n − 1}N . The finite list w = (w0 , . . . , w k−1 ) ∈ {0, . . . , n − 1}k is called a word
DEFINITION

of length k in Ω.
If x = (x 0 , x 1 , . . .) ∈ Ω, we say that x contains the word w if for some j, w i = x j+i for all
i = 0, . . . , k − 1.
If X ⊆ Ω, we say X contains the word w if there exists x ∈ X where x contains the word w.

Normalized Topological Entropy

DEFINITION

Let (σ, X ) be a subshift of the full shift (σ, Ω). The normalized topological entropy of X is

log(# words of length k in X )

H(X ) = lim .
k→∞ log(# words of length k in Ω)

⎡ ⎤
34 1 1 1
Let C be the 3-state Markov chain with transition matrix ⎣0 0 0⎦ and let X ⊆ {0, 1, 2}N = Ω
1 1 1
be the set of realizations of C starting at 0 or 2.
34.1 Draw the graph associated with C .
34.2 Find the (normalized topological) entropy of (σ, X ).
34.3 Find the (normalized topological) entropy of (σ, Ω).
34.4 Draw E(X ) as a subset of [0, 1]. What is its fractal dimension?

24 ⃝
c Jason Siefken, 2015–2019
 Entropy gives a way of classifying subshifts based on how “big” they are. This can be interpreted
as how densely a subshift can store information.

35 Let Ω = {0, 1}N and let X ⊆ Ω be the set of points with no two 1’s in a row.
35.1 How many words of length 1, 2, 3, 4 are there in Ω? How about in X ?
35.2 Is (σ, X ) a subshift? Why or why not?
35.3 Can (σ, X ) be modeled by a Markov chain? If so, write down its transition matrix (not the
stochastic one, the other one).
35.4 How do A, A2 , A3 ,. . . relate to the number of words in X ?
35.5 Find the entropy of (σ, X ).
35.6 If you wanted to transmit n bits of information but you were only allowed to send subwords of
X , how long of a subword would you need?

25 ⃝
c Jason Siefken, 2015–2019
 Entropy relates to how much information you can get about a point by asking a series of
questions.

36 Let C ⊆ [0, 1] be the standard Cantor set. Every “piece” of C has a left and a right. You are
trying to locate a point x ∈ C by asking a series of questions like:

• Is x in the left or right half of C?

• Is x in the left or right half of the left/right half of C?

..
.

36.1 What can you say about x if you know

(a) it’s in the left half of C?

(b) it’s in the left half of the left half of C?
(c) it’s in the right half of the left half of the left half of C?

36.2 Let An = (a1 , . . . , an ) ∈ {left, right}n be the answers to the first n questions you ask.
Given An , what is the maximum error you could have in guessing x?
36.3 Playing the same game trying to find the point y ∈ [0, 1], you ask a series of left-right questions
and get a sequence of answers A′n . Given A′n what is the maximum error you could have in
guessing y?
36.4 Let En be the maximum error you have in guessing x given the answers An and let En′ be the
maximum error you have in guessing y given the answers A′n . Compute

log(En )
lim .
n→∞ log(En′ )

36.5 If you want to locate a point in [0, 1] with error less than ϵ, how many left-right questions do
you need to ask?
36.6 If you want to locate a point in C with error less than ϵ, how many left-right questions do you
need to ask?

26 ⃝
c Jason Siefken, 2015–2019
37 Let X ⊆ Ω = {0, 1, 2}N be the set of all sequences without 1’s.
37.1 Given that w is a length-n word in X , how many yes-no questions do you need to ask to
determine w?
37.2 Given that w is a length-n word in Ω, how many yes-no questions do you need to ask to
determine w?
37.3 How does entropy relate to the number of yes-no questions needed to determine a word?

27 ⃝
c Jason Siefken, 2015–2019
 Expansive
A dynamical system (T, X ) is expansive at the point x ∈ X if there exists ϵ > 0 such that for
DEFINITION

all y ̸= x,
d(T n x, T n y) > ϵ
for some n ≥ 0. The system (T, X ) is called expansive if it is expansive at every point in X .

38 38.1 What’s the difference between being expansive and not being stable?
38.2 Let (T, S) be a circle rotation by 1/4. Is (T, S) expansive?
38.3 Let (σ, Ω) be the full two-shift. Is (σ, Ω) expansive?
38.4 Let (T f , R) be the Newton’s method map from the first day of class. Is (T f , R) expansive?
38.5 Suppose (T, X ) is an expansive dynamical system.

(a) Can (T, X ) have fixed points?

(b) Can (T, X ) have a periodic point?
(c) Can (T, X ) have an attracting point?

28 ⃝
c Jason Siefken, 2015–2019
 Transitivity
Let (T, X ) be a dynamical system. The point x ∈ X is called transitive if O(x) = X . The
DEF

system (T, X ) is called transitive if there exists a transitive point in X .

Here, O(x) = X denotes the closure of the orbit of x. That is, all points that are limits of points
in O(x).

39 39.1 Let (T1/4 , S) be a circle rotation with angle 1/4. Find O(0). Is (T1/4 , S) transitive?
⎷
39.2 Let (T⎷2 , S) be a circle rotation with angle 2. Find O(0). Is (T⎷2 , S) transitive?
39.3 Let (σ, Ω) be the full two-shift and let x = 10100100010000100 · · · . What is O(x).
39.4 Is (σ, Ω) transitive?

29 ⃝
c Jason Siefken, 2015–2019
 Chaos
DEF
A dynamical system (T, X ) is called chaotic if it is both expansive and transitive.

40 40.1 Let (Tθ , S) be a circle rotation with angle θ . Is (Tθ , S) chaotic?

40.2 Let (σ, Ω) be the full two-shift. Is (σ, Ω) chaotic?
40.3 Let (σ, X ) be a subshift of the full two-shift. Is (σ, X ) chaotic?

30 ⃝
c Jason Siefken, 2015–2019
 Chaos is a battle between order and disorder. Expansivity says close points eventually get far,
so you cannot “approximate” a chaotic system into the future. Transitivity says that points (and
open neighborhoods in particular) get smeared all over the space.
We see systems like this in nature, and have a theory of probability that we use to describe
unpredictable events. Can we do something similar to “predict” chaotic systems?
Indicator Function
DEFINITION

Given a set A ⊆ X , the indicator of A is the function IA : X → {0, 1} defined by

¨
1 if x ∈ A
IA(x) = .
0 otherwise

Empirical Measure
Let (T, X ) be a dynamical system and let x ∈ X . The empirical measure generated by x is
the function µ x : {open or closed subsets of X } → [0, 1] given by
DEFINITION

1 ∑︂
µ x (A) = lim IA(T i x) = % of time O(x) is in A,
n→∞ n
0≤i<n

provided this limit exists (for all open/closed A ⊆ X ).

An empirical measure can be extended to more than just open subsets. Typically, the domain of
an empirical measure is extended to the Borel subset, which you’d learn about in an analysis
class.

41 Let (T, [0, 1)) be the doubling map.

41.1 Find µ x ([0, 1/2)) and µ x ([1/2, 1)) for x = 0.
41.2 Find µ x ([0, 1/2)) and µ x ([1/2, 1)) for x = 1/5.
⎷
41.3 Find µ x ([0, 1/2)) and µ x ([1/2, 1)) for x = 2/2.
41.4 Find µ x ([0, 1/2)) and µ x ([1/2, 1)) when x ∈ [0, 1] is such that the binary expansion of x has
0’s 2/3rds of the time.
41.5 Find an x such that µ x doesn’t exist.

31 ⃝
c Jason Siefken, 2015–2019
Chaotic systems can often be “coupled” to probability spaces. When this coupling is possible,
almost every empirical measure is the same and is equal to the underlying probability measure.
This means, despite being deterministic, a chaotic system can be thought of as interchangeable
with a probability space.

32 ⃝
c Jason Siefken, 2015–2019
 To really understand this situation, we need to know some things about probability theory. A
central object of study in probability theory are random variables.
Random Variable
DEF

Let Q be a set. A random variable with state space Q is a function X : [0, 1] → Q.

We think of a random variable X : [0, 1] → Q as a “to be determined quantity”. Of course, if

ω ∈ [0, 1], then X (ω) ∈ Q is an actual element of the state space. It would be incorrect to write
“X ∈ Q”.
Probability Function
The probability function on [0, 1], is a function
DEFINITION

P : {measureable subsets of [0, 1]} → [0, 1]

that assigns every measurable subset of [0, 1] a number corresponding to the “percentage”
of [0, 1] it occupies.

We often call elements of the domain of P events instead of calling them sets.

42 Let X and Y be random variables with state space {H, T } defined by

¨ ¨
H if ω > 1/2 H if ω > 2/3
X (ω) = Y (ω) =
T otherwise T otherwise

42.1 By writing X = H, we mean the event (set) X −1 (H) = {ω ∈ [0, 1] : X (ω) = H}.
What is P(X = H) and P(X = T )? If you were given a fair coin, what would the probability of
getting heads or tails be?

33 ⃝
c Jason Siefken, 2015–2019