cpsc 388 – compiler design and construction parsers – context free grammars
TRANSCRIPT
CPSC 388 – Compiler Design and Construction
Parsers – Context Free Grammars
Announcements HW3 due via Sakai
Solution to HW3
Homework: HW4 assigned today, due next Friday, via Sakai
Reading: 4.1-4.4
Progress Report Grades due Sept 28th
Compilers
Lexical Analyzer(Scanner)
Syntax Analyzer(Parser)
Symantic Analyzer
Intermediate CodeGenerator
Optimizer
Code Generator
SourceCode
TokenStream
AbstractSyntaxTree
Scanner
Source Code:
position = initial + rate * 60 ;
Corresponding Tokens:
IDENT(position)ASSIGNIDENT(position)PLUSIDENT(rate)TIMESINT-LIT(60)SEMI-COLON
Example ParseSource Code:
position = initial + rate * 60 ;
Abstract-Syntax Tree: =
position +
initial *
rate 60
•Interior nodes are operators. •A node's children are operands. •Each subtree forms "logical unit"
e.g., the subtree with * at its root shows thatbecause multiplication has higher precedencethan addition, this operation must be performedas a unit (not initial+rate).
Limitations of RE and FSA
Regular Expressions and Finite State Automata cannot express all languages
For Example the language that consists of all balanced parenthesis: () and ((())) and (((((())))))
Parsers can recognize more types of languages than RE or FSA
Parsers Input: Sequence of Tokens
Output: a representation of program Often AST, but could be other things
Also find syntax errors
CFGs used to define Parser(Context Free Grammar)
Context Free Grammars
if ( expr ) stmt else stmtstmt →
terminals non-terminalsRule or Production
Context Free Grammar
CFGs consist of:
Σ – Set of Terminals (use tokens from scanner)
N – Set of Non-Terminals (variables)
P – Set of Productions (also called rules)
S – the Start Non-Terminal (one on left of first rule if not specified)
Productions
if ( expr ) stmt else stmtstmt →
Single non-terminal
Sequence of zero or more terminals and non-terminals
Example with Boolean Expressions
“true” and “false” are boolean expressions
If exp1 and exp2 are boolean expressions, then so are: exp1 || exp2
exp1 && exp2
! exp1
( exp1 )
Corresponding CFG
bexp → TRUEbexp → FALSEbexp → bexp OR bexpbexp → bexp AND bexpbexp → NOT bexpbexp → LPAREN bexp RPAREN
UPPERCASE represent tokens (thus terminals)lowercase represent non-terminals
CFG for Assignments
Here is CFG for simple assignment statements(Can only assign boolean expressions to identifiers)
stmt → ID ASSIGN bexp SEMICOLON
CFG for simple IF statements
Combine these CFGs and add 2 more rules for simple IF statements:1. stmt → IF LPAREN bexp RPAREN stmt2. stmt → IF LPAREN bexp RPAREN stmt ELSE stmt3. stmt → ID ASSIGN bexp SEMICOLON4. bexp → TRUE5. bexp → FALSE6. bexp → bexp OR bexp7. bexp → bexp AND bexp8. bexp → NOT bexp9. bexp → LPAREN bexp RPAREN
You Try It
Write a context-free grammar for the language of very simple while loops (in which the loop body only contains one statement) by adding a new production with nonterminal stmt on the left-hand side.
CFG Languages The language defined by a context-free
grammar is the set of strings (sequences of terminals) that can be derived from the start nonterminal.
Think of productions as rewriting rulesSet cur_seq = starting non-terminalWhile (non-terminal, X, exists in cur_seq):
Select production with X on left of “→”Replace X with right portion of selected
production Try it with given CFG
What Strings are in Language
1. stmt → IF LPAREN bexp RPAREN stmt2. stmt → IF LPAREN bexp RPAREN stmt ELSE stmt3. stmt → ID ASSIGN bexp SEMICOLON4. bexp → TRUE5. bexp → FALSE6. bexp → bexp OR bexp7. bexp → bexp AND bexp8. bexp → NOT bexp9. bexp → LPAREN bexp RPAREN
Set cur_seq = starting non-terminalWhile (non-terminal, X, exists in cur_seq):
Select production with X on left of “→”Replace X with right portion of selected production
Try It Again
exp → exp PLUS termexp → exp MINUS termexp → termterm → term TIMES factorterm → term DIVIDE factorterm → factorfactor → LPAREN exp RPARENfactor → ID
What is the language?
Leftmost and Rightmost Derivations
A derivation is a leftmost derivation if it is always the leftmost nonterminal that is chosen to be replaced.
It is a rightmost derivation if it is always the rightmost one.
Derivation Notation
E => a E =>* a E =>+ a
Parse Trees
Start with the start nonterminal.
Repeat: choose a leaf nonterminal X
choose a production X --> alpha
the symbols in alpha become the children of X in the tree
until there are no more leaf nonterminals left.
The derived string is formed by reading the leaf nodes from left to right.
Ambiguous Grammars Consider the grammar
exp → exp PLUS expexp → exp MINUS expexp → exp TIMES expexp → exp DIVIDE expexp → INT_LIT
Construct Parse tree for 3-4/2 Are there more than one parse trees? If there is more than one parse tree for a string then
the grammar is ambiguous Ambiguity causes problems with parsing (what is the
correct structure)?
Precedence in Grammars
To write a grammar whose parse trees express precedence correctly, use a different nonterminal for each precedence level. Start by writing a rule for the operator(s) with the lowest precedence ("-" in our case), then write a rule for the operator(s) with the next lowest precedence, etc:
Precedence in Grammars
exp → exp MINUS exp | term
term → term DIVIDE term | factor
factor → INT_LIT | LPAREN exp RPAREN
Now try constructing multiple parse trees for 3-4/2
Grammar is still ambiguous. Look at associativity. Construct 2 parses tree for 5-3-2.
Recursion on CFGs A grammar is recursive in nonterminal X
if: X derives a sequence of symbols that includes an X.
A grammar is left recursive in X if: X derives a sequence of symbols that starts with an X.
A grammar is right recursive in X if: X derives a sequence of symbols that ends with an X.
For left associativity, use left recursion. For right associativity, use right
recursion.
Ambiguity Removed in CFG
exp → exp MINUS term | term
term → term DIVIDE factor | factor
factor → INT_LIT | LPAREN exp RPAREN
One level for each order of operation Left recursion for left assiciativity Now try constructing 2 parse trees for
5-3-2.
You Try it Construct a grammar for arithmetic
expressions with addition, multiplication, exponentiation (right assoc.), subtraction, division, and unary negative.exp → exp PLUS exp |
exp MINUS exp |exp TIMES exp |exp DIVIDE exp |exp POW exp |MINUS exp |LPAREN exp RPAREN |INT_LIT
Solutionexp → exp PLUS term |
exp MINUS term |term
term → term TIMES factor |term DIVIDE factor |factor
factor → exponent POW factor |exponent
exponent → MINUS exponent |final
final → INT_LIT |LPAREN exp RPAREN
You Try It Write a grammar for the language of boolean
expressions, with two possible operands: true false, and three possible operators: and or not. Add nonterminals so that or has lowest precedence, then and, then not. Finally, change the grammar to reflect the fact that both and and or are left associative.bexp → TRUEbexp → FALSEbexp → bexp OR bexpbexp → bexp AND bexpbexp → NOT bexpbexp → LPAREN bexp RPAREN
List Grammars Several types of lists can be created using
CFGs One or more x's (without any separator or
terminator):1. xList → X | xList xList2. xList → X | xList X3. xList → X | X xList
One or more x's, separated by commas:1. xList → X | xList COMMA xList2. xList → X | xList COMMA X3. xList → X | X COMMA xList
List Grammars One or more x's, each x terminated by a
semi-colon: You Try It1. xList → X SEMICOLON | xList xList2. xList → X SEMICOLON | xList X SEMICOLON3. xList → X SEMICOLON | X SEMICOLON xList
Zero or more x's (without any separator or terminator):
1. xList → ε | X | xList xList2. xList → ε | X | xList X3. xList → ε | X | X xList
List Grammars Zero or more x's, each terminated by a
semi-colon:1. xList → ε | X SEMICOLON | xList xList
2. xList → ε | X SEMICOLON | xList X SEMICOLON
3. xList → ε | X SEMICOLON | X SEMICOLON xList Zero or more x's, separated by commas:
You Try It
1. xList → ε | nonEmptyXListnonEmptyXList → X | X COMMA nonEmptyXList
CFGs for Whole Languages
To write a grammar for a whole programming language, break down the problem into pieces. For example, think about a Java program: a program consists of one or more classes:
program → classlist
classlist → class | class classlist
CFGs for Whole Languages
A class is the word "class", optionally preceded by the word "public", followed by an identifier, followed by an open curly brace, followed by the class body, followed by a closing curly brace:
class → PUBLIC CLASS ID LCURLY classbody RCURLY |
CLASS ID LCURLY classbody RCURLY
CFGs for Whole Languages
A class body is a list of zero or more field and/or method definitions: classbody → ε | deflist
deflist → def | def deflist
And So On…