lecture 10 yacc – yet another compiler compiler introduction to yacc and bison topics yacc/bison...

Lecture 10 YACC – Yet Another Compiler Compiler

Introduction to YACC and Bison

Lecture 10 YACC – Yet Another Compiler Compiler

Introduction to YACC and Bison

Topics Topics Yacc/Bison Introduction

Readings:Readings:

February 13, 2006

CSCE 531 Compiler Construction

– 2 – CSCE 531 Spring 2006

OverviewOverviewLast TimeLast Time

Test 1 Post Mortem Lexical Analysis

Hash tableToken / Lexeme / Token Code

Review of LR Parsing

Today’s Lecture Today’s Lecture YACC introduction

References: References:

Homework: write a YACC specification for the language Homework: write a YACC specification for the language “core”“core”

– 3 – CSCE 531 Spring 2006

YACC Generated LALR(1) ParsersYACC Generated LALR(1) Parsers% flex lang.l% flex lang.l // lex.yy.c// lex.yy.c

% bison lang.y % bison lang.y // lang.c// lang.c

% gcc lex.yy.c lang.c –o parse% gcc lex.yy.c lang.c –o parse

% parse input% parse input

lang.y

lang.l FLEXlex.yy.cyylex()

lang.cyyparse()

BISON

Input source program

Executable Program

– 4 – CSCE 531 Spring 2006

YACC Format and UsageYACC Format and Usage

Yet Another Compiler Compiler Stephen Johnson 1976Yet Another Compiler Compiler Stephen Johnson 1976

Takes grammar specification and generates the Takes grammar specification and generates the Action and GOTO tablesAction and GOTO tables

Bison = new and improved YACCBison = new and improved YACC

YACC FormatYACC Format

Definitions sectionDefinitions section

%%%%

productions / semantic actions sectionproductions / semantic actions section

%%%%

routinesroutines

– 5 – CSCE 531 Spring 2006

Simple0.y in web/Examples/SimpleYaccSimple0.y in web/Examples/SimpleYacc

%token DIGIT%token DIGIT

%%%%

line : expr '\n' line : expr '\n'

;;

expr : expr '+' term expr : expr '+' term

| term| term

;;

term : term '*' factor term : term '*' factor

| factor| factor

;;

factor : '(' expr ')' factor : '(' expr ')'

| DIGIT| DIGIT

;;

%%%%

Grammar Grammar Bison Specification Bison Specification

expr expr expr '+' term | term expr '+' term | term

term term term '*' factor | factor term '*' factor | factor

factor factor '(' expr ')' | DIGIT '(' expr ')' | DIGIT

We first augment withWe first augment with

line line expr ‘\n’ expr ‘\n’

– 7 – CSCE 531 Spring 2006

Bison Reporting ConflictsBison Reporting Conflicts

deneb> bison simple1.ydeneb> bison simple1.y

simple1.y contains 4 shift/reduce conflicts.simple1.y contains 4 shift/reduce conflicts.

bison -v simple1.ybison -v simple1.y


So what are they?So what are they?

The “.output file” contains a description of the state The “.output file” contains a description of the state machine and actions.machine and actions.

– 8 – CSCE 531 Spring 2006

.output.output

deneb> more simple1.output deneb> more simple1.output

State 8 contains 2 shift/reduce conflicts.State 8 contains 2 shift/reduce conflicts.


GrammarGrammar

Number, Line, RuleNumber, Line, Rule

1 5 expr -> expr '+' expr1 5 expr -> expr '+' expr

2 6 expr -> expr '*' expr2 6 expr -> expr '*' expr

3 7 expr -> '(' expr ')'3 7 expr -> '(' expr ')'

4 8 expr -> DIGIT4 8 expr -> DIGIT

– 9 – CSCE 531 Spring 2006

Terminals, with rules where they appearTerminals, with rules where they appear

$ $ (-1)(-1)

'(' '(' (40) 3(40) 3

')' ')' (41) 3(41) 3

'*' '*' (42) 2(42) 2

'+' '+' (43) 1(43) 1

error error (256)(256)

DIGIT DIGIT (257) (257) 44

Nonterminals, with rules where they appearNonterminals, with rules where they appear

expr (8)expr (8)

on left: 1 2 3 4, on right: 1 2 3on left: 1 2 3 4, on right: 1 2 3

– 10 – CSCE 531 Spring 2006

state 0state 0

DIGIT shift, and go to state 1DIGIT shift, and go to state 1

'(' shift, and go to state 2'(' shift, and go to state 2

expr go to state 3expr go to state 3

state 1state 1

expr -> DIGIT . (rule 4)expr -> DIGIT . (rule 4)

$default reduce using rule 4 (expr)$default reduce using rule 4 (expr)

– 11 – CSCE 531 Spring 2006

state 2state 2

expr -> '(' . expr ')' (rule 3)expr -> '(' . expr ')' (rule 3)



expr go to stateexpr go to state

– 12 – CSCE 531 Spring 2006

state 3state 3

expr -> expr . '+' expr (rule 1)expr -> expr . '+' expr (rule 1)

expr -> expr . '*' expr (rule 2)expr -> expr . '*' expr (rule 2)

$ go to state 10$ go to state 10

'+' shift, and go to state 5'+' shift, and go to state 5

'*' shift, and go to state 6'*' shift, and go to state 6

… … for states 4 through 7for states 4 through 7

– 13 – CSCE 531 Spring 2006

ConflictsConflictsstate 8state 8


expr -> expr '+' expr . (rule 1)expr -> expr '+' expr . (rule 1)




'+' [reduce using rule 1 (expr)]'+' [reduce using rule 1 (expr)]

'*' [reduce using rule 1 (expr)]'*' [reduce using rule 1 (expr)]


– 14 – CSCE 531 Spring 2006

state 10state 10


state 11state 11

$default accept$default accept

– 15 – CSCE 531 Spring 2006

Main and yyerrorMain and yyerror

– 16 – CSCE 531 Spring 2006

bison simple0.ybison simple0.y

deneb> deneb>

deneb> ls -lrtdeneb> ls -lrt

……

--rw-r--r-- 1 matthews faculty 28499 Jun 30 12:04 simple0.tab.crw-r--r-- 1 matthews faculty 28499 Jun 30 12:04 simple0.tab.c

deneb> wc simple0.tab.c deneb> wc simple0.tab.c

1084 4111 28499 simple0.tab.c1084 4111 28499 simple0.tab.c

– 17 – CSCE 531 Spring 2006

gcc simple0.tab.c -lygcc simple0.tab.c -ly

Undefined first referencedUndefined first referenced

symbol in filesymbol in file

yylex /var/tmp//ccW88jE5.oyylex /var/tmp//ccW88jE5.o

ld: fatal: Symbol referencing errors. No output written to ld: fatal: Symbol referencing errors. No output written to a.outa.out

collect2: ld returned 1 exit statuscollect2: ld returned 1 exit status

– 19 – CSCE 531 Spring 2006

deneb> bison simple1.ydeneb> bison simple1.y


bison -v simple1.ybison -v simple1.y


deneb> ls -lrtdeneb> ls -lrt

……

--rw-r--r-- 1 matthews faculty 2311 Jun 30 12:10 simple1.outputrw-r--r-- 1 matthews faculty 2311 Jun 30 12:10 simple1.output

– 20 – CSCE 531 Spring 2006

.output.output

deneb> more simple1.output deneb> more simple1.output



GrammarGrammar

Number, Line, RuleNumber, Line, Rule

1 5 expr -> expr '+' expr1 5 expr -> expr '+' expr

2 6 expr -> expr '*' expr2 6 expr -> expr '*' expr

3 7 expr -> '(' expr ')'3 7 expr -> '(' expr ')'

4 8 expr -> DIGIT4 8 expr -> DIGIT

– 21 – CSCE 531 Spring 2006

Terminals, with rules where they appearTerminals, with rules where they appear

$ (-1)$ (-1)

'(' (40) 3'(' (40) 3

')' (41) 3')' (41) 3

'*' (42) 2'*' (42) 2

'+' (43) 1'+' (43) 1

error (256)error (256)

DIGIT (257) 4DIGIT (257) 4

Nonterminals, with rules where they appearNonterminals, with rules where they appear

expr (8)expr (8)

on left: 1 2 3 4, on right: 1 2 3on left: 1 2 3 4, on right: 1 2 3

– 22 – CSCE 531 Spring 2006

state 0state 0



expr go to state 3expr go to state 3

state 1state 1

expr -> DIGIT . (rule 4)expr -> DIGIT . (rule 4)


44

– 23 – CSCE 531 Spring 2006

state 2state 2

expr -> '(' . expr ')' (rule 3)expr -> '(' . expr ')' (rule 3)



expr go to stateexpr go to state

– 24 – CSCE 531 Spring 2006

state 3state 3






… … for states 4 through 7for states 4 through 7

– 25 – CSCE 531 Spring 2006

ConflictsConflicts

state 8state 8


expr -> expr '+' expr . (rule 1)expr -> expr '+' expr . (rule 1)







– 26 – CSCE 531 Spring 2006

state 9state 9



expr -> expr '*' expr . (rule 2)expr -> expr '*' expr . (rule 2)






– 27 – CSCE 531 Spring 2006

state 10state 10


state 11state 11

$default accept$default accept

– 28 – CSCE 531 Spring 2006

YACC Generated LALR(1) ParsersYACC Generated LALR(1) Parsers% flex lang.l% flex lang.l // lex.yy.c// lex.yy.c

% bison lang.y % bison lang.y // lang.c// lang.c

% gcc lex.yy.c lang.c –o parse% gcc lex.yy.c lang.c –o parse

% parse input% parse input

lang.y

lang.l FLEXlex.yy.cyylex()

lang.cyyparse()

BISON

Input source program

Executable Program

– 29 – CSCE 531 Spring 2006

man yaccman yaccNAMENAME

yacc - yet another compiler-compileryacc - yet another compiler-compiler

SYNOPSISSYNOPSIS

yacc [-dltVv] [-b file_prefix] [ -Q [y | n] ] [-P parser] [-p sym_prefix] fileyacc [-dltVv] [-b file_prefix] [ -Q [y | n] ] [-P parser] [-p sym_prefix] file

DESCRIPTIONDESCRIPTION

The yacc command converts a context-free grammar into a set of The yacc command converts a context-free grammar into a set of tables for a simple automaton that executes an LALR(1) parsing tables for a simple automaton that executes an LALR(1) parsing algorithm. The grammar may be ambiguous. Specified precedence algorithm. The grammar may be ambiguous. Specified precedence rules are used to break ambiguities.rules are used to break ambiguities.

The output file, y.tab.c, must be compiled by the C compiler to The output file, y.tab.c, must be compiled by the C compiler to produce a function yyparse(). This program must be loaded with the produce a function yyparse(). This program must be loaded with the lexical analyzer program, yylex(), as well as main() and yyerror(), lexical analyzer program, yylex(), as well as main() and yyerror(), an error handling routine. These routines must be supplied by the an error handling routine. These routines must be supplied by the user. user.

– 30 – CSCE 531 Spring 2006

Bison Arguments: Bison –h Bison Arguments: Bison –h deneb> bison -hdeneb> bison -h

GNU bison generates parsers for LALR(1) grammars.GNU bison generates parsers for LALR(1) grammars.

Usage: bison [OPTION]... FILEUsage: bison [OPTION]... FILE

If a long option shows an argument as mandatory, then it is If a long option shows an argument as mandatory, then it is mandatorymandatory

for the equivalent short option also. Similarly for optional for the equivalent short option also. Similarly for optional arguments.arguments.

Operation modes:Operation modes:

-h, --help display this help and exit-h, --help display this help and exit

-V, --version output version information and exit-V, --version output version information and exit

-y, --yacc emulate POSIX yacc-y, --yacc emulate POSIX yacc

– 31 – CSCE 531 Spring 2006

Bison Arguments continuedBison Arguments continued

Output:Output:

-d, --defines also produce a header file-d, --defines also produce a header file

-v, --verbose -v, --verbose also produce an explanation of the automatonalso produce an explanation of the automaton

-b, --file-prefix=PREFIX specify a PREFIX for output files-b, --file-prefix=PREFIX specify a PREFIX for output files

-o, --output=FILE leave output to FILE-o, --output=FILE leave output to FILE

-g, --graph -g, --graph also produce a VCG description of the also produce a VCG description of the automatonautomaton

Report bugs to <[email protected]>.Report bugs to <[email protected]>.

deneb> deneb>

– 32 – CSCE 531 Spring 2006

Bison Arguments continuedBison Arguments continued

Parser:Parser:

-S, --skeleton=FILE specify the skeleton to use-S, --skeleton=FILE specify the skeleton to use

-t, --debug instrument the parser for -t, --debug instrument the parser for debuggingdebugging

--locations enable locations computation--locations enable locations computation

-p, --name-prefix=PREFIX prepend PREFIX to the -p, --name-prefix=PREFIX prepend PREFIX to the external symbolsexternal symbols

-l, --no-lines don't generate `#line' directives-l, --no-lines don't generate `#line' directives

-n, --no-parser generate the tables only-n, --no-parser generate the tables only

-k, --token-table include a table of token names-k, --token-table include a table of token names

– 33 – CSCE 531 Spring 2006

MakefilesMakefiles

A makefile is a file that specifies how to build A makefile is a file that specifies how to build something usually a piece of software something usually a piece of software

The makefile is a collection of targets with each target The makefile is a collection of targets with each target having a series of commands to build the targethaving a series of commands to build the target

To build the system one merely types “make”To build the system one merely types “make”

Make maintains dependencies and recompiles only Make maintains dependencies and recompiles only those files that need to be recompiledthose files that need to be recompiled

Make documents how to build a system and allows a Make documents how to build a system and allows a novice to install a piece of softwarenovice to install a piece of software

– 34 – CSCE 531 Spring 2006

MakefilesMakefiles

The makefile is a collection of targets with each target The makefile is a collection of targets with each target having a series of commands to build the targethaving a series of commands to build the target

The target specification has the formThe target specification has the form

targ: files that targ depends ontarg: files that targ depends on

<TAB>first unix command in a sequence to build target<TAB>first unix command in a sequence to build target

<TAB>next command in sequence to build target<TAB>next command in sequence to build target

<TAB> …<TAB> …

The Makefile specifies a forest of targets depending on The Makefile specifies a forest of targets depending on other targetsother targets

– 35 – CSCE 531 Spring 2006

Example MakefileExample Makefile

prog: prog.o routines.o prog: prog.o routines.o

prog.o: prog.c prog.hprog.o: prog.c prog.h

gcc –c prog.cgcc –c prog.c

routines.o: routines.c prog.hroutines.o: routines.c prog.h

gcc –c routines.cgcc –c routines.c

– 36 – CSCE 531 Spring 2006

Make Builtin RulesMake Builtin Rules

MacrosMacros

Name=valueName=value

$(Name)$(Name) //invocation of macro; replace //invocation of macro; replace macro macro with its value with its value

Suffixes: .c, .f, .y, .l …Suffixes: .c, .f, .y, .l …

Rules Rules

prog.o: prog.cprog.o: prog.c

$(CC) $(CFLAGS) –c prog.c$(CC) $(CFLAGS) –c prog.c

Actually the real rules are generalizations of the above.Actually the real rules are generalizations of the above.

Use “make –p” to see all the rules.Use “make –p” to see all the rules.

– 37 – CSCE 531 Spring 2006

/class/csce531-001/web/Examples/SimpleYacc/Makefile/class/csce531-001/web/Examples/SimpleYacc/Makefile

# Makefile for Simple Yacc and Lex examples # Makefile for Simple Yacc and Lex examples

CC=gccCC=gcc

CFLAGS=CFLAGS=

CFLAGS=-DYYDEBUGCFLAGS=-DYYDEBUG

LEX=flexLEX=flex

YACC=bisonYACC=bison

YACCFLAGS=YACCFLAGS=

YACCFLAGS=-t YACCFLAGS=-t

simple3: simple3.tab.h simple3.tab.o lex.yy.osimple3: simple3.tab.h simple3.tab.o lex.yy.o

$(CC) $(CFLAGS) simple3.tab.o lex.yy.o -ly -o simple3$(CC) $(CFLAGS) simple3.tab.o lex.yy.o -ly -o simple3

simple3.tab.h: simple3.ysimple3.tab.h: simple3.y

bison -d $(YACCFLAGS) simple3.ybison -d $(YACCFLAGS) simple3.y

simple3.tab.c: simple3.ysimple3.tab.c: simple3.y

bison $(YACCFLAGS) simple3.ybison $(YACCFLAGS) simple3.y

lex.yy.c: simple3.llex.yy.c: simple3.l

flex simple3.l flex simple3.l

– 38 – CSCE 531 Spring 2006

Lex Specification simple3.lLex Specification simple3.l%{%{

#include "simple0.tab.h"#include "simple0.tab.h"

%}%}

digit [0-9]+digit [0-9]+

%%%%

\n \n return('\n');return('\n');

\+ \+ return(PLUS);return(PLUS);

\* \* return(TIMES);return(TIMES);

$ \( return(LPAREN);return(LPAREN);

$ \) return(RPAREN);return(RPAREN);

{digit} return(DIGIT);{digit} return(DIGIT);

%%%%

yywrap(){yywrap(){

}}

– 39 – CSCE 531 Spring 2006

Example Yacc Spec file simple3.yExample Yacc Spec file simple3.y%token DIGIT TIMES PLUS LPAREN RPAREN%token DIGIT TIMES PLUS LPAREN RPAREN

%%%%

line : expr '\n' {printf("recognized an expr\n");line : expr '\n' {printf("recognized an expr\n");

exit(0);exit(0);

}}

;;

expr : expr PLUS term expr : expr PLUS term

| term| term

;;

term : term TIMES factor term : term TIMES factor

| factor| factor

;;

factor : LPAREN expr RPAREN factor : LPAREN expr RPAREN

| DIGIT| DIGIT

;;

%%%%

– 40 – CSCE 531 Spring 2006

Simple4.lSimple4.l%{%{

#include "simple4.tab.h"#include "simple4.tab.h"

%}%}

digit [0-9]+digit [0-9]+

%%%%

\n return('\n');\n return('\n');

\+ return(PLUS);\+ return(PLUS);

\* return(TIMES);\* return(TIMES);

$ return(LPAREN);\( return(LPAREN);

$ return(RPAREN);\) return(RPAREN);

{digit} { yylval= atoi(yytext); return(INTEGER);}{digit} { yylval= atoi(yytext); return(INTEGER);}

%%%%

yywrap(){ }yywrap(){ }

– 41 – CSCE 531 Spring 2006

Simple4.ySimple4.y%{%{

#include <ctype.h>#include <ctype.h>

int yylineno = 1;int yylineno = 1;

%}%}

%token INTEGER PLUS TIMES LPAREN RPAREN%token INTEGER PLUS TIMES LPAREN RPAREN

%%%%

line : expr '\n' {printf("%d\n", $1);}line : expr '\n' {printf("%d\n", $1);}

| line expr '\n' {printf("%d\n", $2);}| line expr '\n' {printf("%d\n", $2);}

;;

expr : expr PLUS term {$$ = $1 + $3;}expr : expr PLUS term {$$ = $1 + $3;}

| term| term

;;

term : term TIMES factor {$$ = $1 * $3;}term : term TIMES factor {$$ = $1 * $3;}

| factor| factor

;;

factor : LPAREN expr RPAREN {$$ = $2;}factor : LPAREN expr RPAREN {$$ = $2;}

| INTEGER {$$ = $1;} /* the default action */| INTEGER {$$ = $1;} /* the default action */

;;

%%%%

lecture 10 yacc – yet another compiler compiler introduction to yacc and bison topics yacc/bison...

Documents

expr expr expr

expr expr expr

expr expr expr digit

state expr

expr n expr

expr slide

expr rule

expr digit slide