Turing Machines

Standard Turing Machine

1
Topics

• The Standard Turing Machine

• Combining Turing Machines for Complicated Tasks
• Turing's Thesis
• Other Models of Turing Machines
• A Universal Turing Machine

2
Alan Turing
(1912~1954)

• British computer scientist, logician, cryptanalyst.

• He built a foundation of theoretical computer science.
• Imitation Game: a.k.a Turing Test.
• It is about how to verify whether machines can think (or imitate human) or
not.

• Halting Problem: Proved the existence of undecidable problem.

• Mostly known to public about his contribution in WWII, decrypting German
cryptography.
3
The Standard Turing Machine

• Turing machine consists of a

control unit and an infinite tape.

• Tape: an infinite tape, each cell

contains one symbol.

• Read-write Head: read the

current symbol, or write a symbol
to the current cell.
Figure 9.1

• Control unit: a unit defines

internal states and transition
function.
4
The Standard Turing Machine
Definition 9.1

• A Turing Machine M is defined by M = (Q, Σ, Γ, δ, q0, □ , F),

• where
• Q, Σ, q0 and F as usual,
• Γ is a finite set of symbols called the tape alphabet, Q

• □ ∈ Γ is a special symbol called the blank. 1 0 a b 1

• Assume Σ ⊆ Γ − { □ }.
• δ : Q × Γ → Q × Γ × {L, R}.
5
The Standard Turing Machine
Example 9.1

• Figure 9.1 shows the situation before and after the move
• δ(q0, a) = (q1, d, R).

Figure 9.2

6
The Standard Turing Machine

• We can consider a Turing machine as a simple computer.

• It has a processing unit w/ a finite memory, and its tape can be considered
as a secondary storage of unlimited capacity.

• The machine can only rewrite the current symbol, change the state of the
control, and to move the read-write head.

• Still, it's quite powerful in principle, even with such limited instructions.
• The transition function defines how the machine acts, which is similar to a
"program".

7
The Standard Turing Machine
Example 9.2

• Consider the Turing machine defined by

• Q = {q0, q1}, Σ = {a, b}, Γ = {a, b, □ }, F = {q1},
Figure 9.4
• and
• δ(q0, a) = (q0, b, R),
• δ(q0, b) = (q0, b, R),
• δ(q0, □ ) = (q1, □ , L).
Figure 9.3 A sequence of moves.

• A Turing machine is said to be in a halt state, whenever it reaches a configuration where δ

is not defined.
8
The Standard Turing Machine
Example 9.3

• Suppose that the tape initially contains ab…,

• with the head on the a. Figure 9.5

• It moves between q0 and q1, while not changing the content on the tape.
• No matter what contents on the tape, this machine will run forever.
• This is an instance of a Turing machine that does not halt.
• In programming, we can say that this machine is in an infinite loop.

9
The Standard Turing Machine
Main features

1. The Turing machine has a tape that is unbounded in both directions, allowing any
number of left and right moves.

2. The Turing machine is deterministic in the sense that δ defines at most one move
for each configuration.

3. There is no special input file.

• We assume that at the initial time the tape has some specified content, which may
be considered as input.

• Similarly, there is no special output device.

• Whenever the machine halts, contents of the tape may be viewed as output.
10
The Standard Turing Machine
Instantaneous Description

• We will use the notation in which

• x1qx2 or a1a2⋯ak−1qakak+1⋯an Figure 9.6

• is the instantaneous description of a machine in state q with the tape depicted in

Figure 9.6.

• The unspecified part of the tape is assumed to contain all blanks.

• Usually these blanks are irrelevant.
• However, if the blank symbol is relevant, it must be appeared in the instantaneous
description.
11
The Standard Turing Machine
Another Visualization
Control Unit

• We can view a Turing machine as b c e f

a machine with the moving tape. … # # # # # # # …

• The head is fixed inside the Head

control unit,

• and tape is moving to make

the head to go left or right. current state symbol operations final state

b # P0, R c

• The machine reads and writes a c

e
#
#
S
P1, R
e
f
symbol on the current location of f # S b
the tape pointed by the head.

12
The Standard Turing Machine
Example 9.4

• The pictures drawn in Figure 9.3 correspond to the sequence of

instantaneous descriptions

• q0aa, bq0a, bbq0 □ , bq1b.

Figure 9.3 A sequence of moves.

13
The Standard Turing Machine
Example 9.5

• A move from one configuration to another will be denoted by ⊢.

• If δ(q1, c) = (q2, e, R), then abq1cd ⊢ abeq2d
• is made whenever the internal state is q1, the tape contains abcd, and the head is on the
c.
• The action of the Turing machine in Figure 9.3 can be represented by
• q0aa ⊢ bq0a ⊢ bbq0 □ ⊢ bq1b
• or
• q0aa ⊢* bq1b. Figure 9.3 A sequence of moves.

14
The Standard Turing Machine
Definition 9.2

• Let M = (Q, Σ, Γ, δ, q0, □ , F) be a Turing machine.

• Then any string a1⋯ak−1q1akak+1⋯an, with ai ∈ Γ and q1 ∈ Q, is an instantaneous description of M.
• A move a1⋯ak−1q1akak+1⋯an ⊢ a1⋯ak−1bq2ak+1⋯an is possible iff.
• δ(q1, ak) = (q2, b, R).
• M is said to halt starting from some initial configuration x1qi x2 if
• x1qi x2 ⊢* y1qjay2 for any qj and a, for which δ(qj, a) is undefined.
• The sequence of configurations leading to a halt state will be called a computation.
• A Turing machine which never halts can be represented as x1qx2 ⊢* ∞.
15
Turing Machines as Language Accepters
Definition 9.3

• Let M = (Q, Σ, Γ, δ, q0, □ , F) be a Turing machine.

• Then the language accepted by M is
+
• L(M) = {w ∈ Σ : q0w ⊢* x1qf x2 for some qf ∈ F, x1, x2 ∈ Γ*}.
• The definition indicates that the input is written on the tape with blanks on either
side.

• This convention restricts the area on the tape which the machine should consider.
• When w ∉ L(M), either the machine can halt in a non-final state, or it can enter an
infinite loop.
16
Turing Machines as Language Accepters
Example 9.6

• For Σ = {0,1}, design a Turing machine that accepts the language denoted
by the regular expression 00*.

• Strings only have 0's.

• We can easily check whether symbols on the tape are all 0's.
• δ(q0,0) = (q0,0,R), δ(q0, □ ) = (q1, □ , R).
• Note that we don't consider λ due to Definition 9.3.

17
Turing Machines as Language Accepters
Example 9.7

n n
• For Σ = {a, b}, design a Turing machine that accepts L = {a b : n ≥ 1}.
• What we have to do is matching a's and b's.
• Intuitively, we can try this method.
1. Replace the left most a with x.
□ a a b b □
2. Find left most b and replace it with y.

3. Repeat 1, 2 until no a's and b's are left.

18
Turing Machines as Language Accepters
Example 9.7

• We can break down transitions into several parts.

• Matching a pair of a and b and replace them with x and y respectively.
• δ(q0, a) = (q1, x, R), δ(q1, a) = (q1, a, R),
• δ(q1, y) = (q1, y, R), δ(q1, b) = (q2, y, L). □ a a b b □

• Reset the head position for another match.

• δ(q2, y) = (q2, y, L), δ(q2, a) = (q2, a, L), δ(q2, x) = (q0, x, R).

19
Turing Machines as Language Accepters
Example 9.7

• Finally, we need transitions to terminate the process.

• δ(q0, y) = (q3, y, R), δ(q3, y) = (q3, y, R), δ(q3, □ ) = (q4, □ , R).
• Based on these transitions, we can come up with the following for the
complete Turing machine definition.

• Q = {q0, q1, q2, q3, q4}, F = {q4}, Σ = {a, b}, Γ = {a, b, x, y, □ }.

• Does it refuse a string not in the language?
• How about aab?
20
Turing Machines as Language Accepters
Example 9.7

• δ(q0, a) = (q1, x, R), δ(q1, a) = (q1, a, R),

• δ(q1, y) = (q1, y, R), δ(q1, b) = (q2, y, L),
• δ(q2, y) = (q2, y, L), δ(q2, a) = (q2, a, L), δ(q2, x) = (q0, x, R),
• δ(q0, y) = (q3, y, R), δ(q3, y) = (q3, y, R), δ(q3, □ ) = (q4, □ , R).
• How about aab?

21
Turing Machines as Language Accepters

• This way of thinking is particularly useful to understand many algorithms, if you get
familiar with the view.

• Insertion of an element in a Red-Black Tree.

• Minimum Spanning Tree Algorithm
• Adding minimum cost vertex or edge to the MST.
• KMP algorithm for string pattern matching.
• Using pre-processing.
• Tells you which position to go back when the current matching is failed.
22
Turing Machines as Language Accepters
Example 9.8

n n n
• Design a Turing machine that accepts L = {a b c : n ≥ 1}.
• We can try a method similar to the previous example.
• We can replace matched a, b and c with x, y and z respectively.
• Although we only need to match one more symbol, we need more states and
transitions to complete the task.

• Try it by yourself to get familiar with the concept.

n n n n n
• Note that {a b } and {a b c } can be accepted by Turing machines of similar
structures, although the former is context-free and the latter is not.
23
Turing Machines as Transducers

• Unlike other automata we have been discussed, Turing machines can be also
used as transducers.

• It is an abstract model of a computer which transforms inputs to outputs.

• A Turing machine transducer M as an implementation of a function f defined
by

• ŵ = f(w) provided by that

• q0w ⊢*M qf ŵ , for some final state qf.

24
Turing Machines as Transducers
Definition 9.4

• A function f with domain D is said to be Turing-computable or just

computable,

• if there exists some Turing machine M = (Q, Σ, Γ, δ, q0, □ , F) s.t.

• q0w ⊢*M qf f(w), qf ∈ F,
• for all w ∈ D.
• In short, f is Turing computable if it can be simulated by a Turing machine.
• All the common mathematical functions are Turing computable.
25
Turing Machines as Transducers
Example 9.9

• Given two positive integers x and y, design a Turing machine that computes x + y.
• We will use unary notation to represent positive integers for simplicity.
+
• w(x) ∈ {1} , | w(x) | = x.
• e.g.) w(3) = 111, w(5) = 11111, etc.
• We also need to decide how x, y and x + y are placed on the tape.
• We will separate x and y with a single 0, and after the computation, w(x + y) will be on
the tape.

• e.g.) 1+2 will be appeared as 1011, and after computation, we will have 1110.
26
Turing Machines as Transducers
Example 9.9

• We want to design a Turing machine for performing the computation

• q0w(x)0w(y) ⊢* qf w(x + y)0.
• Eventually, what we have to do is moving 0 to the right end of w(y).
• δ(q0,1) = (q0,1,R), δ(q0,0) = (q1,1,R),
• δ(q1,1) = (q1,1,R), δ(q1, □ ) = (q2, □ , L),
• δ(q2,1) = (q3,0,L), δ(q3,1) = (q3,1,L),
• δ(q3, □ ) = (q4, □ , R).
• You can check the sequence of instantaneous descriptions for adding 111 to 11 in the textbook.
27
Turing Machines as Transducers
Example 9.10

• Design a Turing machine that copies strings of 1's.

• More precisely, find a machine that performs the computation
+
• q0w ⊢* qf ww, for any w ∈ {1} .
• To solve the problem, we implement the following process.
1. Replace every 1 by an x.

2. Find the rightmost x and replace it with 1.

Figure 9.7

3. Travel to the right end of the current non-blank region and create a 1 there.

4. Repeat Steps 2 and 3 until there are no more x's.

28
Turing Machines as Transducers
Example 9.11

• Let x and y be two positive integers represented in unary notation.

• Construct a Turing machine that will halt in a final state qy if x ≥ y, and that will
halt in a non-final state qn if x < y.

• q0w(x)0w(y) ⊢* qyw(x)0w(y) if x ≥ y,
• q0w(x)0w(y) ⊢* qnw(x)0w(y) if x < y.
• We use the idea of matching a's and b's, but this time, we match 1's on the
left and right side of 0.
29
Turing Machines as Transducers
Example 9.11

• At the end of the matching, we will have symbols on the tape either
• xx⋯110xx⋯x □ or
• xx⋯xx0xx⋯x11 □ ,
• depending on whether x > y or y > x.
• This example shows that a Turing machine can be programmed to make decisions
based on arithmetic comparisons.

• Based on the given computation, we may need to revert x's into 1's again to restore
the tape content, once it enters state qy or qn.

30
Combining Turing Machines for
Complicated Tasks

31
Combining Turing Machines for Complicated Tasks
Example 9.12

• Design a Turing machine that computes the function

f(x, y) = x + y if x ≥ y,
• =0 if x < y . Figure 9.8

• The value 0 will be represented by 0, with the rest of the tape blank.
• We combine multiple Turing machines for the computation.
• How can we connect the machines?

32
Combining Turing Machines for Complicated Tasks
Example 9.12

• We use states indexed with the initial of each machine, C, A, and E.

• For C,
qC,0w(x)0w(y) ⊢* qA,0w(x)0w(y) if x ≥ y,
• qC,0w(x)0w(y) ⊢* qE,0w(x)0w(y) if x < y .
• For A,
• qA,0w(x)0w(y) ⊢* qA,f w(x + y)0. Figure 9.8

• For E,
• qE,0w(x)0w(y) ⊢* qE,f 0.
33
Combining Turing Machines for Complicated Tasks
Example 9.13

• We can use macroinstruction to represent Turing machine more conveniently.

• Consider the macroinstruction
• if a then qj else qk,
• with the following interpretation.
• If the Turing machine reads an a, then regardless of its current state, it is to go into
state qj without changing the tape content or moving the head.

• If the symbol read is not an a, the machine is to go into state qk without changing
anything.

34
Combining Turing Machines for Complicated Tasks
Example 9.13

• if a then qj else qk,

• To implement this macroinstruction, we need several steps.
• δ(qi, a) = (qj0, a, R) for all qi ∈ Q,
• δ(qi, b) = (qk0, b, R) for all qi ∈ Q and all b ∈ Γ − {a},
• δ(qj0, c) = (qj, c, L) for all c ∈ Γ,
• δ(qk0, c) = (qk, c, L) for all c ∈ Γ.
• We can employ Stay operation, to avoid the burden of moving head.
• δ(qi, a) = (qj, a, S).
35
Combining Turing Machines for Complicated Tasks
Using Turing machines as Subprograms

• As shown in the previous example, it is useful to consider a Turing machine as

a subprogram.

• We can call a subprogram, do some tasks, and return to the original process.
• To support this, we need a new feature to store the current configuration of a
Turing machine.
Figure 9.9

36
Combining Turing Machines for Complicated Tasks
Example 9.14

• Design a Turing machine that multiplies two positive integers in unary

notation.

• We can construct the machine using adding and copying.

1. Repeat the following steps until x contain no more 1's.

• Find a 1 in x and replace it with another symbol a.

• Replace the leftmost 0 by 0y.
2. Replace all a's with 1's. Figure 9.10

37
Turing's Thesis

38
Turing's Thesis

• What we want to know is that, in some sense,

• Turing machines are equal in power to a typical digital computer?
• This is a question which is very difficult to answer.
• What is a "typical digital computer" in the first place?
• However, although lack of a formal proof, we have a statement which
indicates that

• Turing machines are in principle powerful as any computer.

39
Turing's Thesis

• Any computation that can be carried out by mechanical means can be

performed by some Turing machine.

• It is more of a hypothesis rather than a proved theorem. We can consider it as a

basic law of computer science.

• Alternatively, anything that can be done on any existing digital computer can also be
done by a Turing machine.

• No one has yet been able to suggest a problem, which is solvable by what we
intuitively consider an algorithm, but a Turing machine program cannot be written.

• Alternative models have been proposed for mechanical computation, but none of
them is more powerful than the Turing machine model.
40
An Algorithm
Definition 9.5

• An algorithm for a function f : D → R is a Turing machine M,

• which given as input any d ∈ D on its tape,
• eventually halts with the correct answer f(d) ∈ R on its tape.
• Specifically, we can require that
• q0d ⊢*M qf f(d), qf ∈ F, for all d ∈ D.
• Constructing an explicit algorithm for a specific task is often a very lengthy, tedious process.
• We can now claim that, anything we can do on any computer can also be done on a Turing machine.
• This will greatly simplifies future discussions.

41
Other Models of Turing Machines

42
Other Turing Machines

• The content is from Chapter 10 of the textbook.

• This lecture is about Section 10.4 only.
• The other three sections are about variations of Turing machine.
• It is not that difficult to understand, if you already know well about the
standard Turing machine.

43
A Universal Turing Machine

• An argument against Turing's thesis:

• A Turing machine (Definition 9.1) is a special-purpose computer.
• Once δ is defined, the machine is restricted to carrying out one type of
computation.

• Digital computers have no such restriction (we have programs run on them).
• A universal Turing machine is a reprogrammable Turing machine, which can
rebut the argument.

44
A Universal Turing Machine

• A universal Turing machine Mu is an automaton that,

• given as input the description of any Turing machine M and a string w,
• can simulate the computation of M on w.
Figure 10.16

• Can we obtain the description of M in a string?

45
Countable / Uncountable

• For infinite sets, we distinguish between sets that are countable and sets that
are uncountable.

• A set is said to be countable, if its elements can be put into a one-to-one

correspondence with the positive integers.

• It means that the elements of the set can be written in some order such as
x1, x2, ⋯. Figure 10.17

• e.g.) the set of all quotients of the form p/q,

• where p, q are positive integers.
46
Enumeration Procedure
Definition 10.4

• We can prove that a set is countable if we can provide method by which its elements can be written in
some sequence.

• We call such a method an enumeration procedure.

• Let S be a set of strings on some alphabet Σ.
• Then an enumeration procedure for S is a Turing machine that can carry out the sequence of steps
• q0 □ ⊢* qs x1 # s1 ⊢* qs x2 # s2⋯, with xi ∈ Γ* − {#}, si ∈ S,
• in such a way that any s ∈ S is produced in a finite number of steps.
• The state qs is a state signifying membership in S.
• Whenever qs is entered, the string following # must be in S.
47
Proper order
Example 10.3

• Let Σ = {a, b, c}.

+
• We can show that the S = Σ is countable if we can find an enumeration
procedure that produces its elements in some order.

• Maybe we use the order used in dictionaries.

• However, since we have an infinite number of words, listing a words first then
b next strategy will not work.

48
Proper order
Example 10.3

• In Definition 10.4, any s ∈ S is produced in a finite number of steps.

• The dictionary order doesn't satisfy this condition.
• Instead, we take the length of the string as the first criterion, followed by an
alphabetic ordering of all equal-length strings.

• This is an enumeration procedure that gives the sequence

• a, b, c, aa, ab, ac, ba, bb, bc, ca, cb, cc, aaa, ⋯.
• We will call it the proper order.
49
Turing Machine and Countable
Theorem 10.3

• The set of all Turing machines, although infinite, is countable.

• Proof: We can encode each Turing machine using 0 and 1.
• For a transition, we have five different elements which should be represented.
• For each element, we can convert its index into a unary representation.
• e.g.) Q = {q1, q2, …, qn}, Γ = {a1, a2, …, am}.
• For δ(q1, a2) = (q2, a3, L), we have 10110110111010,
• where L = 1 and R = 11.
50
Turing Machine and Countable
Theorem 10.3

• With this encoding, we then construct the following enumeration procedure.

+
1. Generate the next string in {0,1} in proper order.

2. Check the generated string to see if it defines a Turing machine.

• If so, write it on the tape in the form required by Definition 10.4.

• If not, ignore the string.
3. Return to Step 1.

• Since every Turing machine has a finite description, any specific machine will
eventually be generated.
51
Summary

• After this lecture,

• You should remember the name of the important figure.
• You need to understand the standard Turing machines and how it works.
• You have to get familiar with notations regarding Turing machines.
• You need to understand the details about how Turing machines can be combined for a
complicated task.

• You need to know what is Turing's thesis, what is the claim and indications.
• You also need to remember some definitions introduced in the lectures since they will be
used in later lectures.

52