1 ast

7/29/2019 1 Ast

1/109

Abstract SyntaxCMSC

CS431

7/29/2019 1 Ast

2/109

UMBC

2

Abstract Syntax Trees

So far a parser traces the derivation of asequence of tokens

The rest of the compiler needs a structural

representation of the program Abstract syntax trees

Like parse trees but ignore some details

Abbreviated as AST

7/29/2019 1 Ast

3/109

UMBC

3

Abstract Syntax Tree. (Cont.)

Consider the grammarE int | ( E ) | E + E

And the string

5 + (2 + 3) After lexical analysis (a list of tokens)

int5+ ( int2+ int3)

During parsing we build a parse tree

7/29/2019 1 Ast

4/109

UMBC

4

Evaluation of semantic rules

Parse-tree methods (compile time) Build a parse tree for each input

Build a dependency graph from the parse tree

Obtain evaluation order from a topological order of the

dependency graph Rule-based methods (compiler-construction time)

Predetermine the order of attribute evaluation for eachproduction

Oblivious methods (compiler-construction time) Evaluation order is independent of semantic rules

Evaluation order forced by parsing methods

Restrictive in acceptable attribute definitions

7/29/2019 1 Ast

5/109

UMBC

5

Compile-time semantic evaluation

SourceProgram

Lexical Analyzer

Syntax Analyzer

Semantic Analyzer

Intermediate CodeGenerator

Code Optimizer

Code Generator

Target

Program

Tokens

Parse tree /

Abstract syntax tree

Attributed AST

Results

Program input

compilersinterpreters

7/29/2019 1 Ast

6/109

UMBC

6

Abstract Syntaxvs. Concrete Syntax

Concrete syntax: the syntax programmers write Example: different notations of expressions

Prefix + 5 * 15 20 Infix 5 + 15 * 20 Postfix 5 15 20 * +

Abstract syntax: the syntax recognized by compilers Identifies only the meaningful components

The operation The components of the operation

e

ee

e

5

*

+

1520

eParse Tree for5+15*20

+

20

5

15

*

Abstract Syntax Tree for 5 + 15 * 20

7/29/2019 1 Ast

7/109

UMBC

7

Abstract syntax trees

Condensed form of parse tree for representinglanguage constructs Operators and keywords do not appear as leaves

They define the meaning of the interior (parent) node

Chains of single productions may be collapsed

If-then-else

B S1 S2

S

IF B THEN S1 ELSE S2

E

E + T

5T

3

+

3 5

7/29/2019 1 Ast

8/109

UMBC

8

Constructing AST

Use syntax-directed definitions Problem: construct an AST for each expression Attribute grammar approach

Associate each non-terminal with an AST Each AST: a pointer to a node in AST

E.nptr T.nptr

Definitions: how to compute attribute? Bottom-up: synthesized attribute

if we know the AST of each child, how to compute the AST ofthe parent?

E ::= E + T | E T | TT ::= (E) | id | num

Grammar:

7/29/2019 1 Ast

9/109

UMBC

9

Constructing ASTfor expressions (by Hand)

Associate each non-terminal with an AST E.nptr, T.nptr: a pointer to ASTtree

Synthesized attribute definition: If we know the AST of each child, how to compute the AST of

the parent?

Production Semantic rules

E ::= E1 + T E.nptr=mknode_plus(E1.nptr,T.nptr)

E ::= E1 T E.nptr=mknode_minus(E1.nptr,T.nptr)

E ::= T E.nptr=T.nptrT ::= (E) T.nptr=E.nptr

T ::= id T.nptr=mkleaf_id(id.entry)

T ::= num T.nptr=mkleaf_num(num.val)

7/29/2019 1 Ast

10/109

UMBC

10

Example: constructing AST

1. reduce 5 to T1 using T::=num:T1.nptr = leaf(5)

2. reduce T1 to E1 using E::=T:E1.nptr = T1.nptr = leaf(5)

3. reduce 15 to T2 using T::=num:T2.nptr=leaf(15)4. reduce T2 to E2 using E::=T:

E2.nptr=T2.nptr = leaf(15)5. reduce b to T3 using T::=num:

T3.nptr=leaf(b)6. reduce E2-T3 to E3 using E::=E-T:

E3.nptr=node(-,leaf(15),leaf(b))7. reduce (E3) to T4 using T::=(E):

T4.nptr=node(-,leaf(15),leaf(b))8. reduce E1+T4 to E5 using E::=E+T:

E5.nptr=node(+,leaf(5),node(-,leaf(15),leaf(b)))

Parse tree for 5+(15-b)

E5

E1 + T4

( E3 )

E2- T3

bT2

15

T1

5

Bottom-up parsing: evaluate attribute at each reduction

7/29/2019 1 Ast

11/109

UMBC

11

Implementing AST in C

Define different kinds of AST nodes typedef enum {PLUS, MINUS, ID, NUM} ASTNodeTag;

Define AST nodetypedef struct ASTnode {

AstNodeTag kind;union { symbol_table_entry* id_entry;

int num_value;struct ASTnode* opds[2];

} description;};

Define AST node construction routines ASTnode* mkleaf_id(symbol_table_entry* e); ASTnode* mkleaf_num(int n); ASTnode* mknode_plus(struct ASTnode* opd1, struct ASTNode* opd2); ASTnode* mknode_minus(struct ASTnode* opd1, struct ASTNode* opd2);

E ::= E + T | E T | TT ::= (E) | id | num

Grammar:

7/29/2019 1 Ast

12/109

UMBC

12

Implementing AST in Java

Define different kinds of AST nodes typedef enum {PLUS, MINUS, ID, NUM} ASTNodeTag;

Define AST node

class ASTexpression {public ASTNodeTag kind();};class ASTidentifier inherit ASTexpression { private symbol_table_entry* id_entry; }class ASTvalue inherit ASTexpression { private int num_value; }class ASTplus inherit ASTexpression { private ASTnode* opds[2]; }Class ASTminus inherit ASTexpression { private ASTnode* opds[2]; ... }

Define AST node construction routines ASTexpression* mkleaf_id(symbol_table_entry* e); ASTexpression* mkleaf_num(int n); ASTexpression* mknode_plus(struct ASTnode* opd1, struct ASTNode* opd2); ASTexpression* mknode_minus(struct ASTnode* opd1, struct ASTNode* opd2);

E ::= E + T | E T | TT ::= (E) | id | num

Grammar:

7/29/2019 1 Ast

13/109

UMBC

13

More ASTs

S::= if-else E S S | while E S | E | E::= var | num | true | false | E bop E | uop Ebop ::= < | | >= | && | = | + | * | .uop ::= - | * | & |

Abstract syntax:

if-else

T E E-> +T E E -> - T E E->T -> F T T -> * F T T -> / F T T ->

F -> id F -> num F -> ( E )

Needs type of non-terminals and tokens

7/29/2019 1 Ast

22/109

UMBC

22

Recursive-descent interpreter

int T() { switch (tok.kind) {

case ID: case NUM: case LPAREN

return Tprime( F() );

default:print(expected ID, NUM, or left-paren);skipto(T_follow); return 0; }}

int Tprime(int a) {switch (tok.kind) {

case TIMES: eat(TIMES); return Tprime(a*F());

case DIVIDE: eat(DIVIDE); return Tprime(a/F());

case PLUS: case MINUS: case RPAREN: case EOF:return a;

default: /* error handling */ }}

7/29/2019 1 Ast

23/109

7/29/2019 1 Ast

24/109

UMBC

24

JavaCC version

void Start() :

{ int i; }

{ i=Exp() {System.out.println(i); }

}

int Exp() :

{ int a, i; }

{ a=Term() ( + i=Term() { a=a+i; }

| - i=Term() { a=a+i; } )*

{ return a; }

}

Int Factor() :

{ Token t; int i; }

{ t = {return lookup(t.image); }

| t= {return Integer.parseInt(t.image);}

| ( i=Exp() ) {return i; }

}

7/29/2019 1 Ast

25/109

UMBC

25

Semantic Actions Reduce andShift

We can now illustrate how semantic actionsare implemented for LR parsing

Keep attributes on the stack

On shift a, push attribute for a on stack

On reduce X

pop attributes for compute attribute for X

and push it on the stack

7/29/2019 1 Ast

26/109

UMBC

26

Performing Semantic Actions.Example

Recall the example from previous lecture

E T + E1 { E.val = T.val + E1.val }

| T { E.val = T.val }T int * T1 { T.val = int.val * T1.val }

| int { T.val = int.val }

Consider the parsing of the string 3 * 5 + 8

7/29/2019 1 Ast

28/109

7/29/2019 1 Ast

29/109

UMBC

29

A Line Calculator

Each line contains an expression

E int | E + E

Each line is terminated with the = sign

L E = | + E = In second form the value of previous line is

used as starting value

A program is a sequence of linesP e | P L

7/29/2019 1 Ast

30/109

UMBC

30

Attributes for theLine Calculator

Each E has a synthesized attribute val Calculated as before

Each L has a synthesized attribute val

L E = { L.val = E.val }| + E = { L.val = E.val + L.prev }

We need the value of the previous line

We use an inherited attributeL.prev

7/29/2019 1 Ast

31/109

UMBC

31

Attributes for theLine Calculator (Cont.)

Each P has a synthesized attribute val The value of its last line

P e { P.val = 0 }

| P1 L { P.val = L.val;L.prev = P1.val }

Each L has an inherited attribute prev

L.prev is inherited from sibling P1.val

Example

7/29/2019 1 Ast

32/109

UMBC

32

Example of Inherited Attributes

val synthesized

prev inherited

All can be

computed indepth-firstorder

P

e

L

+ E3=

E4+

int2

E5

int3

+

+

2

0

3

P

7/29/2019 1 Ast

33/109

UMBC

33

Semantic Actions: Notes (Cont.)

Semantic actions can be used to build ASTs

And many other things as well

Also used for type checking, code generation,

Process is called syntax-directed translation

Substantial generalization over CFGs

7/29/2019 1 Ast

34/109

UMBC

34

Constructing An AST

We first define the AST data type Supplied by us for the project

Consider an abstract tree type with two

constructors:mkleaf(n)

mkplus(

T1

) =,

T2

=

PLUS

T1 T2

n

7/29/2019 1 Ast

35/109

UMBC

35

Constructing a Parse Tree

We define a synthesized attribute ast Values of ast values are ASTs

We assume that int.lexval is the value of the

integer lexeme Computed using semantic actions

E int E.ast = mkleaf(int.lexval)

| E1 + E2 E.ast = mkplus(E1.ast, E2.ast)| ( E1 ) E.ast = E1.ast

7/29/2019 1 Ast

36/109

UMBC

36

Parse Tree Example

Consider the string int5+ ( int2+ int3) A bottom-up evaluation of the ast attribute:

E.ast = mkplus(mkleaf(5),

mkplus(mkleaf(2), mkleaf(3))

PLUS

PLUS

25 3

7/29/2019 1 Ast

37/109

7/29/2019 1 Ast

38/109

UMBC

38

Abstract Syntax

E -> E + E

E -> E EE -> E * E

E -> E / E

E -> idE -> num

Abtract Parse Trees : ExpressionGrammar

7/29/2019 1 Ast

39/109

UMBC

39

AST : Node types

public abstract class Exp {public abstract int eval():}public class PlusExp extends Exp {private Exp e1, e2;public PlusExp(Exp a1, Exp a2) { e1=a1; d2=a2; }public int eval() {

return e1.eval()+e2.eval():}}public class Identifier extends Exp {

private String f0;public Indenfifier(String n0) { f0 = n0; }public int eval() {

return lookup(f0);

}}public class IntegerLiteral extends Exp {

private String f0;public IntegerLiteral(String n0) { f0 = n0; }public int eval() {

return Integer.parseInt(f0);}

}

7/29/2019 1 Ast

40/109

UMBC

40

JavaCC Example for ASTconstruction

Exp Start() :{ Exp e; }

{ e=Exp() { return e; }}

Exp Exp() :

{ Exp e1, e2; }

{ e1=Term() ( + e2=Term() { e1=new PlusExp(e1,e2); }

| - e2=Term() { e1=new MinusExp(e1,e2); } )*{ return a; }

}

Exp Factor() :

{ Token t; Exp e; }

{ t = {return new Identifier(t.image); }

| t=

{return new IntegerLiteral(t.image);}

| ( e=Exp() ) {return e; }

}

7/29/2019 1 Ast

41/109

UMBC

41

Positions

Must remember the position in the source file Lexical analysis, parsing and semantic analysis are

not done simultaneously. Necessary for error reporting

AST must keep the pos fields, which indicatethe position within the original source file.

Lexer must pass the information to theparser.

Ast node constructors must be augmented toinit the pos fields.

7/29/2019 1 Ast

42/109

UMBC

42

JavaCC : Class Token

Each Token object has the following fields: int kind;

int beginLine, beginColumn, endLine, endColumn;

String image; Token next;

Token specialToken;

static final Token newToken(int ofKind);

Unfortunately, .

7/29/2019 1 Ast

43/109

UMBC

43

Visitors

syntax separate from interpretation style ofprogramming Vs. object-oriented style of programming

Visitor pattern Visitor implements an interpretation.

Visitor object contains a visit method for eachsyntax-tree class.

Syntax-tree classes contain accept methods. Visitor calls accept(what is your class?). Then

accept calls the visit of the visitor.

7/29/2019 1 Ast

44/109

UMBC

44

Example :Expression Classes

public abstract class Exp {public abstract int accept(Visitor v):}public class PlusExp extends Exp {

private Exp e1, e2;public PlusExp(Exp a1, Exp a2) { e1=a1; d2=a2; }

public int accept(Visitor v) { return v.visit(this) ; }}public class Identifier extends Exp {

private String f0;public Indenfifier(String n0) { f0 = n0; }public int accept(Visitor v) { return v.visit(this) ; }

}

public class IntegerLiteral extends Exp {private String f0;public IntegerLiteral(String n0) { f0 = n0; }public int accept(Visitor v) { return v.visit(this) ; }}

7/29/2019 1 Ast

45/109

7/29/2019 1 Ast

46/109

UMBC

46

Abstract Syntax for MiniJava (I)

Package syntaxtree;

Program(MainClass m, ClassDecList c1)MainClass(Identifier i1, Identifier i2, Statement s)----------------------------abstract class ClassDecl

ClassDeclSimple(Identifier i, VarDeclList vl,methodDeclList m1)

ClassDeclExtends(Identifier i, Identifier j,VarDecList vl, MethodDeclList ml)

-----------------------------

VarDecl(Type t, Identifier i)MethodDecl(Type t, Identifier I, FormalList fl,VariableDeclList vl, StatementList sl, Exp e)

Formal(Type t, Identifier i)

7/29/2019 1 Ast

47/109

UMBC

47

Abstract Syntax for MiniJava (II)

abstract class type

IntArrayType()

BooleanType()

IntegerType()

IndentifierType(String s)

---------------------------abstract class Statement

Block(StatementList sl)

If(Exp e, Statement s1, Statement s2)

While(Exp e, Statement s)

Print(Exp e)Assign(Identifier i, Exp e)

ArrayAssign(Identifier i, Exp e1, Exp e2)

-------------------------------------------

Abst t S t f

7/29/2019 1 Ast

48/109

UMBC

48

Abstract Syntax forMiniJava (III)

abstract class Exp

And(Exp e1, Exp e2) LessThan(Exp e1, Exp e2)Plus(Exp e1, Exp e2) Minus(Exp e1, Exp e2)Times(Exp e1, Exp e2) Not(Exp e)

ArrayLookup(Exp e1, Exp e2) ArrayLength(Exp e)Call(Exp e, Identifier i, ExpList el)IntergerLiteral(int i)True() False()IdentifierExp(String s)This()

NewArray(Exp e) NewObject(Identifier i)-------------------------------------------------Identifier(Sting s)--list classes-------------------------ClassDecList() ExpList() FormalList() MethodDeclList()StatementLIst() VarDeclList()

7/29/2019 1 Ast

49/109

C

7/29/2019 1 Ast

50/109

UMBC

50

ClassDecl.java

package syntaxtree;

import visitor.Visitor;

import visitor.TypeVisitor;

public abstract class ClassDecl {

public abstract void accept(Visitor v);

public abstract Type accept(TypeVisitor v);

}

7/29/2019 1 Ast

51/109

UMBC

7/29/2019 1 Ast

52/109

UMBC

52

StatementList.java

package syntaxtree;import java.util.Vector;

public class StatementList {private Vector list;

public StatementList() {list = new Vector();}public void addElement(Statement n) {

list.addElement(n);}public Statement elementAt(int i) {

return (Statement)list.elementAt(i);}public int size() {

return list.size();}

}

UMBC

7/29/2019 1 Ast

53/109

UMBC

53

Package Visitor/visitor.java

package visitor;import syntaxtree.*;

public interface Visitor {public void visit(Program n); public void visit(MainClass n);public void visit(ClassDeclSimple n); public void visit(ClassDeclExtends n);public void visit(VarDecl n); public void visit(MethodDecl n);

public void visit(Formal n); public void visit(IntArrayType n);public void visit(BooleanType n); public void visit(IntegerType n);public void visit(IdentifierType n); public void visit(Block n);public void visit(If n); public void visit(While n);public void visit(Print n); public void visit(Assign n);public void visit(ArrayAssign n); public void visit(And n);public void visit(LessThan n); public void visit(Plus n);public void visit(Minus n); public void visit(Times n);

public void visit(ArrayLookup n); public void visit(ArrayLength n);public void visit(Call n); public void visit(IntegerLiteral n);public void visit(True n); public void visit(False n);public void visit(IdentifierExp n); public void visit(This n);public void visit(NewArray n); public void visit(NewObject n);public void visit(Not n); public void visit(Identifier n);}

UMBC

7/29/2019 1 Ast

54/109

UMBC

54

X = y.m(1,4+5)

Statement -> AssignmentStatementAssignmentStatement -> Identfier1= Expression

Identifier1 ->

Expression -> Expression1. Identifier2( ( ExpList)? )

Expression1 -> IdentifierExpIdentifierExp ->

Identifier2 ->

ExpList -> Expression2( , Expression3 )*

Expression2

->

Expression3 -> PlusExp -> Expression + Expression

-> ,

7/29/2019 1 Ast

55/109

UMBC

7/29/2019 1 Ast

56/109

UMBC

56

MiniJava : Grammar(I)

Program -> MainClassClassDecl*

Program(MainClass, ClassDeclList)

Program Goal() :{ MainClass m; ClassDeclList cl = new ClassDeclList();

ClassDecl c;

}

{ m = MainClass() (c = ClassDecl() {cl.addElement(c);})*

{return new Program(m,cl)}

UMBC

7/29/2019 1 Ast

57/109

UMBC

57

MiniJava : Grammar(II)

MainClass-> classid { publicstaticvoidmain ( String [] id){ Statement} }

MainClass(Identifier, VarDeclList)

ClassDecl -> classid { VarDecl * MethodDecl * }->classid extendsid { VarDecl* MethodDecl * }

ClassDeclSimple(), ClassDecExtends()

VarDecl ->Type id;VarDecl(Type, Identifier)

MethodDecl -> publicType id ( FormalList )

{ VarDecl * Statement* return Exp; }MethodDecl(Type,Identifier,FormalList,VarDeclList

StaementList, Exp)

UMBC

7/29/2019 1 Ast

58/109

UMBC

58

MiniJava : Grammar(III)

FormalList -> Type id FormalRest*->

FormalRest -> , Type id

Type -> int []

-> boolean

-> int

-> id

UMBC

7/29/2019 1 Ast

59/109

UMBC

59

MiniJava : Grammar(IV)

Statement -> { Statement * }-> if ( Exp) Statement else Statement

-> while ( Exp) Statement

-> System.out.println ( Exp);

-> id = Exp ;-> id [ Exp]= Exp;

ExpList -> Exp ExpRest*

->

ExpRest -> , Exp

UMBC

7/29/2019 1 Ast

60/109

UMBC

60

MiniJava : Grammar(V)

Exp -> Exp op Exp-> Exp [ Exp]

-> Exp . length

-> Exp . Id ( ExpList )

-> INTEGER_LITERAL-> true

-> false

-> id

-> this

-> newint [ Exp ]

-> newid ( )

-> !Exp-> ( Exp)

7/29/2019 1 Ast

61/109

UMBC

7/29/2019 1 Ast

62/109

UMBC

62

References

Andrew W. Appel, Modern CompilerImplementation in Java (2nd Edition),Cambridge University Press, 2002

http://compiler.kaist.ac.kr/courses/cs420/classtps/Chapter05.pps

Modern Compiler Design, Scott Galles, ScottJones

http://www.cs.utsa.edu/~qingyi/cs4713/handouts/AbstractSyntaxTree.ppt
http://compiler.kaist.ac.kr/courses/cs420/classtps/Chapter05.ppshttp://compiler.kaist.ac.kr/courses/cs420/classtps/Chapter05.ppshttp://www.cs.utsa.edu/~qingyi/cs4713/handouts/AbstractSyntaxTree.ppthttp://www.cs.utsa.edu/~qingyi/cs4713/handouts/AbstractSyntaxTree.ppthttp://www.cs.utsa.edu/~qingyi/cs4713/handouts/AbstractSyntaxTree.ppthttp://www.cs.utsa.edu/~qingyi/cs4713/handouts/AbstractSyntaxTree.ppthttp://compiler.kaist.ac.kr/courses/cs420/classtps/Chapter05.ppshttp://compiler.kaist.ac.kr/courses/cs420/classtps/Chapter05.pps

7/29/2019 1 Ast

63/109

Using Java Tree Builder

CMSC 431

Shon Vick

UMBC

7/29/2019 1 Ast

64/109

UMBC

64

Lecture Outline

Introduction Syntax Directed Translation

Java Virtual Machine

Examples Administration

7/29/2019 1 Ast

65/109

7/29/2019 1 Ast

66/109

7/29/2019 1 Ast

67/109

UMBC First Attempt:

7/29/2019 1 Ast

68/109

UMBC

68

First Attempt:Instanceof and Type Casts

List l;

// The List-object we are working on.

int sum = 0;// Contains the sum after the loop.

boolean proceed = true;while (proceed) {

if (l instanceof Nil)proceed = false;else if (l instanceof Cons) {

sum = sum + ((Cons) l).head; // Type cast!l = ((Cons) l).tail; // Type cast!

}

}

What are theproblems here?

UMBC

7/29/2019 1 Ast

69/109

UMBC

69

What are the problems here?

Type Casts? We want static (compile time) type checking

Flexible?

Probably not well illustrated with this example

UMBC Second Attempt:

7/29/2019 1 Ast

70/109

UMBC

70

Second Attempt:Dedicated Methods

interface List {

int sum();

}

class Nil implements List {

public int sum() { return 0; }

}

class Cons implements List {

int head;

List tail;

public int sum() {return head + tail.sum();

}

}

UMBC

7/29/2019 1 Ast

71/109

UMBC

71

Tradeoffs

Can compute the sum of all components of a givenList-object l by writing l.sum().

Advantage: type casts and instanceof operationshave disappeared, and that the code can be

written in a systematic way. Disadvantage: Every time we want to perform a

new operation on List-objects, say, compute theproduct of all integer parts, then new dedicatedmethods have to be written for all the classes, andthe classes must be recompiled

UMBC

7/29/2019 1 Ast

72/109

UMBC

72

Third Attempt: The Visitor Pattern.

interface List {void accept(Visitor v);

}

class Nil implements List {

public void accept(Visitor v) { v.visitNil(this); }}

class Cons implements List {

int head;

List tail;

public void accept(Visitor v) {v.visitCons(this);

} }

UMBC

7/29/2019 1 Ast

73/109

UMBC

73

Second Part of Visitor Idea

interface Visitor {void visitNil(Nil x);void visitCons(Cons x);

}

class SumVisitor implements Visitor {int sum = 0;

public void visitNil(Nil x) {}public void visitCons(Cons x){

sum = sum + x.head;

x.tail.accept(this); } }

UMBC

7/29/2019 1 Ast

74/109

U C

74

Summary

Each accept method takes a visitor as argument.

The interface Visitor has a header for each of thebasic classes.

We can now compute and print the sum of allcomponents of a given List-object l by writingSumVisitor sv = new SumVisitor();

l.accept(sv);

System.out.println(sv.sum);

UMBC

7/29/2019 1 Ast

75/109

75

Summary Continued

The advantage is that one can write code thatmanipulates objects of existing classes withoutrecompiling those classes.

The price is that all objects must have an accept

method. In summary, the Visitor pattern combines the

advantages of the two other approaches

7/29/2019 1 Ast

76/109

UMBC

f G d F l

7/29/2019 1 Ast

77/109

77

Overview of Generated Files

To begin using JTB, simply run it using yourgrammar file as an argument Run JTB without any argumentsfor list.

This will generate an augmented grammarfile, as well as the needed classes

UMBC

7/29/2019 1 Ast

78/109

78

Details

jtb.out.jj, the original grammar file, now with syntax treebuilding actions inserted

The subdirectory/package syntaxtree which contains a javaclass for each production in the grammar

The subdirectory/package visitor which contains

Visitor.java, the default visitor interface, also DepthFirstVisitor.java, a default implementation which visits

each node of the tree in depth-first order.

ObjectVisitor.java, another default visitor interface thatsupports return value and argument. ObjectDepthFirst.java is a defualt implemetation of

ObjectVisitor.java.

UMBC

7/29/2019 1 Ast

79/109

79

General Instructions

To generate your parser, simply run JavaCCusing jtb.out.jj as the grammar file.

Let's take a look at all the files and

directories JTB generates.

UMBC The grammar file

7/29/2019 1 Ast

80/109

80

The grammar file

Named jtb.out.jj This file is the same as the input grammar file except

that it now contains code for building the syntax treeduring parse.

Typically, this file can be left alone aftergeneration. The only thing that needs to be done to it is to run it

through JavaCC to generate your parser

7/29/2019 1 Ast

81/109

7/29/2019 1 Ast

82/109

UMBC What gets produced?

7/29/2019 1 Ast

83/109

83

What gets produced?Part 1

// Generated by JTB 1.1.2 //package syntaxtree;

/**

* Grammar production:

* f0 -> "import"

* f1 -> Name()* f2 -> [ "." "*" ]

* f3 -> ";"

*/

public class ImportDeclaration implements Node {public NodeToken f0;

public Name f1;

public NodeOptional f2;

public NodeToken f3;

All parts of a production

are represented in the tree,including tokens.

UMBC

7/29/2019 1 Ast

84/109

84

The Syntax Tree Classes

Notice the package "syntaxtree". The purpose of separating the generated tree

node classes into their own package is that itgreatly simplifies file organization, particularly

when the grammar contains a large number ofproductions. Its often not necessary to pay the syntax classes

any more attention. All of the work is to done tothe visitor classes.

Note that this class implements an interfacenamed Node.

UMBC

Automatically-Generated Tree Node Interface

7/29/2019 1 Ast

85/109

85

Automatically Generated Tree Node Interface

and Classes

Node All tree nodes implement this

Nodelistinterface List interface that NodeList, NodeListOptional, andNodeSeqeunce implement

Nodechoice Represents ( A | B )

Nodelist Represents ( A ) +

NodeListOptional Represents ( A ) *

NodeOptional Represents[ A ] or ( A )?

NodeSequence Represents nexted sequence like[ "extends" Name() ]

NodeToken Represents a token string
http://www.cs.purdue.edu/jtb/docs.htmlhttp://www.cs.purdue.edu/jtb/docs.htmlhttp://www.cs.purdue.edu/jtb/docs.htmlhttp://www.cs.purdue.edu/jtb/docs.htmlhttp://www.cs.purdue.edu/jtb/docs.htmlhttp://www.cs.purdue.edu/jtb/docs.html

7/29/2019 1 Ast

86/109

7/29/2019 1 Ast

87/109

7/29/2019 1 Ast

88/109


7/29/2019 1 Ast

89/109

89

What gets produced?Part 1

// Generated by JTB 1.1.2 //package syntaxtree;

/**

* Grammar production:

* f0 -> "import"

* f1 -> Name()* f2 -> [ "." "*" ]

* f3 -> ";"

*/

public class ImportDeclaration implements Node {public NodeToken f0;

public Name f1;

public NodeOptional f2;

public NodeToken f3;

All parts of a productionare represented in the tree,

including tokens.

UMBC

7/29/2019 1 Ast

90/109

90

NodeListInterface

The interface NodeListInterface is implementedby NodeList, NodeListOptional, andNodeSequence. NodeListInterface looks like this:

public interface NodeListInterface extends Node {public void addNode(Node n);public Node elementAt(int i);public java.util.Enumeration elements();public int size();

}

7/29/2019 1 Ast

91/109

7/29/2019 1 Ast

92/109

UMBC

D t il

7/29/2019 1 Ast

93/109

93

Details

The type stored by this NodeChoice would not bedetermined until the file was actually parsed.

The node stored by a NodeChoice would then beaccessible through the choice field.

Since the choice is of type Node, typecasts aresometimes necessary to access the fields of the nodestored in a NodeChoice.

7/29/2019 1 Ast

94/109

7/29/2019 1 Ast

95/109

7/29/2019 1 Ast

96/109

7/29/2019 1 Ast

97/109

7/29/2019 1 Ast

98/109

UMBC

N deT ken

7/29/2019 1 Ast

99/109

99

NodeToken

This class is used by JTB to store all tokensinto the tree, including JavaCC "specialtokens" (if the -tk command-line option isused).

In addition, each NodeToken containsinformation about each token, including itsstarting and ending column and line numbers.

7/29/2019 1 Ast

100/109

UMBC

C ntin d

7/29/2019 1 Ast

101/109

101

Continued

public class NodeToken implements Node {// .

public String tokenImage;public int beginLine, beginColumn, endLine, endColumn;// -1 if not available.// Equal to the JavaCC token "kind" integer.

public int kind;// Special Token methods below

public NodeToken getSpecialAt(int i);public int numSpecials();

public void addSpecial(NodeToken s);public void trimSpecials();public String withSpecials();public Vector specialTokens;

}

7/29/2019 1 Ast

102/109

7/29/2019 1 Ast

103/109


7/29/2019 1 Ast

104/109

104

g pPart 2

public ImportDeclaration(NodeToken n0,

Name n1, NodeOptional n2, NodeToken n3)

{

f0 = n0;

f1 = n1;

f2 = n2;f3 = n3;

}public ImportDeclaration(Name n0, NodeOptional n1) {

f0 = new NodeToken("import");

f1 = n0;f2 = n1;

f3 = new NodeToken(";");

}

UMBC

Constructors

7/29/2019 1 Ast

105/109

105

Constructors

The next portion of the generated class is thestandard constructor. It is called from the tree-building actions in the annotated grammar so you willprobably not need to use it.

Following the first constructor is a convenienceconstructor with the constant tokens of theproduction already filled-in by the appropriateNodeToken. This constructor's purpose is to help inmanual construction of syntax trees.


7/29/2019 1 Ast

106/109

106

g pPart 3

public void accept(visitor.Visitor v)

{ v.visit(this);

}

public Object

accept(visitor.ObjectVisitorv, Object argu) {

return v.visit(this,argu);

}

}

7/29/2019 1 Ast

107/109

7/29/2019 1 Ast

108/109

7/29/2019 1 Ast

109/109

1 ast

Documents