Terminology
Statement ( 敘述 )
» declaration, assignment containing expression ( 運算式 )
Grammar ( 文法 )
» a set of rules specify the form of legal statements
Syntax ( 語法 ) vs. Semantics ( 語意 )
» Example: assuming J,K:integer and X,Y:float
» I:=J+K vs I:=X+Y
Compilation: 編譯
» matching statements written by the programmer to structures d
efined by the grammar and generating the appropriate object c
ode
1
System Software
Assembler
Loader and Linker
Macro Processor
Compiler
Operating System
Other System Software
» RDBS
» Text Editors
» Interactive Debugging System
2
Basic Compiler
Lexical analysis -- scanner
» scanning the source statement, recognizing and classifying
the various tokens
Syntactic analysis -- parser
» recognizing the statement as some language construct
Code generation --
3
Scanner
PROGRAM
STATS
SUM VAR
:= SUM
0 ,
; SUMSQ
SUMSQ ,
:= I
READ
(
VALUE
)
;
4
Parser
Grammar: a set of rules
» Backus-Naur Form (BNF)
» Ex: Figure 5.2
Terminology
» Define symbol ::=
» Nonterminal symbols <>
» Alternative symbols |
» Terminal symbols
5
Simplified Pascal Grammar
6
Parser
READ(VALUE) <read> ::= READ (<id-list>)
<id-list>::= id | <id-list>,id
SUM := 0 <assign>::= id := <exp>
<exp> ::= <term> |
SUM := SUM + VALUE <exp>+<term> |
<exp>-<term>
<term>::=<factor> |
MEAN := SUM DIV 100
<term>*<factor> | <term> DIV <f
actor>
<factor>::= id | int | <exp>
7
Syntax Tree
8
Syntax Tree for Program 5.1
9
Lexical Analysis
Function
» scanning the program to be compiled and recognizing the to
kens that make up the source statements
Tokens
» Tokens can be keywords, operators, identifiers, integers, flo
ating-point numbers, character strings, etc.
» Each token is usually represented by some fixed-length cod
e, such as an integer, rather than as a variable-length chara
cter string (see Figure 5.5)
» Token type, Token specifier (value) (see Figure 5.6)
10
Scanner Output
Token specifier
» identifier name, integer value
Token coding scheme
» Figure 5.5
11
Example - Figure 5.6
Statement Token type Token specifier
PROGRAM STATS 1
22 ^STATS
VAR 2
SUM,SUMSQ,I, …, : INTEGER 22 ^SUM
14
22 ^SUMSQ
14
…..
14
22 ^VARIANCE
13
6
BEGIN 3
SUM:=0; 22 ^SUM
15
23 #0
12
Token Recognizer
By grammar
» <ident> ::= <letter> | <ident> <letter>| <ident><digit>
» <letter> ::= A | B | C | D | … | Z
» <digit> ::= 0 | 1 | 2 | 3 | … | 9
By scanner - modeling as finite automata
» Figure 5.8(a)
13
Recognizing Identifier
Identifiers allowing underscore (_)
» Figure 5.8(b)
State A-Z 0-9 _
1 2
2 2 2 3
3 2 2
A -Z
0 -9
1 A -Z 2 - 3
A -Z
0 -9
14
Recognizing Integer
Allowing leading zeroes
» Figure 5.8(c)
0 -9
1 0 -9 2
Disallowing leading zeroes
» Figure 5.8(d)
0 -9
1 1 -9 2
3 space 4
15
Scanner -- Implementation
Figure 5.10 (a)
» Algorithmic code for identifer recognition
Tabular representation of finite automaton for Figure 5.9
State A-Z 0-9 ;,+-*() : = .
1 2 4 5 6
2 2 2 3
3
4 4
5
6 7
7
16
Syntactic Analysis
Recognize source statements as language constructs
or build the parse tree for the statements
» bottom-up: operator-precedence parsing
» top-down:: recursive-descent parsing
17
Operator-Precedence Parsing
Operator
» any terminal symbol (or any token)
Precedence
» * »+
» +«*
Operator-precedence
» precedence relations between operators
18
Precedence Matrix for the Fig. 5.2
Operator-Precedence Parse Example
BEGIN READ ( VALUE ) ;
20
(i) … id1 := id2 DIV
(ii) … id1 := <N1> DIV int -
(iii) … id1 := <N1> DIV <N2> -
(iv) … id1 := <N3> - id3 *
(v) … id1 := <N3> - <N4> * id4 ;
(vi) … id1 := <N3> - <N4> * <N5> ;
(vi) … id1 := <N3> - <N6> ;
(vii) … id1 := <N7> ;
Operator-Precedence Parsing
Bottom-up parsing
Generating precedence matrix
» Aho et al. (1988)
23
Shift-reduce Parsing with Stack
Figure 5.14
24
Recursive-Descent Parsing
Each nonterminal symbol in the grammar is associate
d with a procedure
<read> ::= READ (<id-list>)
<stmt> ::= <assign> | <read> | <write> | <for>
Left recursion
» <dec-list> ::= <dec> | <dec-list>;<dec>
Modification
» <dec-list> ::= <dec> {;<dec>}
25
26
Recursive-Descent Parse of READ
27
Simplified Pascal Grammar for Recursive-
Descent Parser
28
29
30
31
32
33
34
35
36
37
38
39
