Formal Languages and Automata

Theory

1
Objectives

• Introduce concepts in automata theory and theory of

computation
• Identify different formal language classes and their
relationships
• Design grammars and recognizers for different formal
languages
• Prove or disprove theorems in automata theory using its
properties
• Determine the decidability and intractability of
computational problems

2
Course Organization

Broadly, the course will contain three parts:

• Part I) Regular languages

• Part II) Context-free languages
• Part III) Turing machines & decidability

3
Pre-requisites

• Data Structures
• Discrete Mathematics

4
Required Textbook

• Introduction to Automata Theory, Languages and

Computation
• By J.E. Hopcroft, R. Motwani, J.D. Ullman
• 3rd Edition
• Pearson

5
Grading

• Homework and Quiz (20%)

• Mid Term (30%)
• End Term (40%)
• Surprise Test (10%)

6
Homework Policy

• Hardcopy to be submitted in class on the due date

• Early submissions allowed
• No late submissions
• Homework will be posted on UMS or email
• All homework must be done individually
• Cheating:
• Helping others, getting help, looking up website for
solutions, etc.
• Students caught cheating will be awarded ZERO

7
Exam Policy
• Mid Term and End Term
• Closed book, closed notes, comprehensive

8
Lecture basics
• Classes will involve both Slides + Board
• Lecture slides will be available
• Use of laptops and mobile phones not allowed in classroom.

9
What is Automata Theory?

• Study of abstract computing devices, or “machines”

• Automaton = an abstract computing device
• Note: A “device” need not even be a physical hardware!
• A fundamental question in computer science:
• Find out what different models of machines can do and
cannot do
• The theory of computation
• Computability vs. Complexity

10
 (A pioneer of automata theory)
Alan Turing (1912-1954)

Father of Modern Computer Science

English mathematician

Studied abstract machines called Turing

machines even before computers existed

Heard of the Turing test?

11
Theory of Computation: A Historical
Perspective

1930s • Alan Turing studies Turing machines

• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages

1969 Cook introduces “intractable” problems

or “NP-Hard” problems

1970- Modern computer science: compilers,

computational & complexity theory evolve
12
 Languages & Grammars

• Languages: “A language is a
Or “words” collection of sentences of finite
length all constructed from a
finite alphabet of symbols”
• Grammars: “A grammar can be
regarded as a device that
enumerates the sentences of a
language” - nothing more,
nothing less

• N. Chomsky, Information and

Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002 13

The Chomsky Hierachy

• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) free Context-
Recursively-
(PDA) sensitive
enumerable
(LBA)
(TM)

14
The Central Concepts of Automata
Theory

15
Alphabet

An alphabet is a finite, non-empty set of symbols

• We use the symbol ∑ (sigma) to denote an alphabet
• Examples:
• Binary: ∑ = {0,1}
• All lower case letters: ∑ = {a,b,c,..z}
• Alphanumeric: ∑ = {a-z, A-Z, 0-9}
• DNA molecule letters: ∑ = {a,c,g,t}
•…

16
Strings

A string or word is a finite sequence of symbols

chosen from ∑
• Empty string is  (or “epsilon”)

• Length of a string w, denoted by “|w|”, is equal to

the number of (non- ) characters in the string
• E.g., x = 010100 |x| = 6
• x = 01  0  1  00  |x| = ?

• xy = concatentation of two strings x and y

17
Powers of an alphabet

Let ∑ be an alphabet.

• ∑k = the set of all strings of length k

• ∑* = ∑0 U ∑1 U ∑2 U …

• ∑+ = ∑1 U ∑2 U ∑3 U …

18
Languages

L is a said to be a language over alphabet ∑, only if L  ∑*

➔ this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by
n 1’s:
L = {, 01, 0011, 000111,…}
2. Let L be the language of all strings of with equal number of 0’s
and 1’s:
L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}

Definition: Ø denotes the Empty language

• Let L = {}; Is L=Ø?
NO
19
The Membership Problem

Given a string w ∑*and a language L over ∑, decide whether or not w

L.

Example:
Let w = 100011
Q) Is w  the language of strings with equal number of 0s and 1s?

20
Finite Automata

• Some Applications
• Software for designing and checking the behavior of
digital circuits
• Lexical analyzer of a typical compiler
• Software for scanning large bodies of text (e.g., web
pages) for pattern finding
• Software for verifying systems of all types that have a
finite number of states (e.g., stock market transaction,
communication/network protocol)

21
Finite Automata : Examples action

state
• On/Off switch

• Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state

22
Proof Methods

• Proving p→q
• Direct proof: Assume p is true, and prove q.
• Indirect proof: Assume q, and prove p.
• Trivial proof: Prove q true.
• Vacuous proof: Prove p is true.
• Proving p
• Proof by contradiction: Prove p→ (r  r)
(r  r is a contradiction); therefore p must be false.
• Prove (a  b) → p
• Proof by cases: prove (a→ p) and (b→ p).

23
Direct Proof Example

• Definition: An integer n is called odd iff n=2k+1 for

some integer k; n is even iff n=2k for some k.
• Axiom: Every integer is either odd or even.
• Theorem: (For all numbers n) If n is an odd integer,
then n2 is an odd integer.
• Proof:
• If n is odd, then n = 2k+1 for some integer k.
• Thus, n2 = (2k+1)2 = 4k2 + 4k + 1 = 2(2k2 + 2k) + 1.
• Therefore, n2 is of the form 2j + 1 (with j the integer 2k2 +
2k),
• thus n2 is odd. □

24
Direct Proof Example

• Example: Give a direct proof that if m and n are

both perfect squares, then n*m is also a perfect
square

25
 Direct Proof Example
The sum of two even numbers is even.
Proof x = 2m, y = 2n
x+y = 2m+2n
= 2(m+n)
The product of two odd numbers is odd.
Proof x = 2m+1, y = 2n+1
xy = (2m+1)(2n+1)
= 4mn + 2m + 2n + 1
= 2(2mn+m+n) + 1.
Another Example

• Definition: A real number r is rational if there exist

integers p and q ≠ 0, such that r=p/q. A real number
that is not rational is called irrational.

• Theorem: Prove that the sum of two rational

numbers is rational.

27
Indirect Proof

• Proving p→q
• Indirect proof: Assume q, and prove p.

28
Indirect Proof Example

• Theorem: (For all integers n)

If 3n+2 is odd, then n is odd.
• Proof:
• Suppose that the conclusion is false, i.e., that n is even.
• Then n=2k for some integer k.
• Then 3n+2 = 3(2k)+2 = 6k+2 = 2(3k+1).
• Thus 3n+2 is even, because it equals 2j for integer j = 3k+1.
• So 3n+2 is not odd.
• We have shown that ¬(n is odd)→¬(3n+2 is odd),
• thus its contra-positive (3n+2 is odd) → (n is odd) is also
true. □
29
Another Example

• Theorem: Prove that if n is an integer and n2 is odd, then n is odd.

30
Trivial Proof

• Proving p→q
• Trivial proof: Prove q true.

31
Trivial Proof Example

• Theorem: (For integers n) If n is the sum of two prime numbers, then

either n is odd or n is even.
• Proof: Any integer n is either odd or even. So the conclusion of the
implication is true regardless of the truth of the hypothesis. Thus
the implication is true trivially. □

32
Vacuous Proof

• Proving p→q
• Vacuous proof: Prove p is true.

33
Vacuous Proof Example

• Theorem: (For all n) If n is both odd and even, then n2 = n + n.

• Proof: The statement “n is both odd and even” is necessarily false,
since no number can be both odd and even. So, the theorem is
vacuously true. □

34
Proof by Contradiction

• Proving p
• Assume p, and prove that p→ (r  r)
• (r  r) is a trivial contradiction, equal to F
• Thus p→F is true only if p=F

35
Contradiction Proof Example

• Theorem: Prove that 2 is irrational.

36
Another Example

• Prove that the sum of a rational number and an irrational number is

always irrational.

37
Prove by Mathematical Induction

• Example: n4 – 4n2 is divisible by 3, for n ≥ 0

38