LING 388Language and Computers

Lecture Lecture 1111

10/710/7/03/03

Sandiway FONGSandiway FONG

Administrivia

Computer LabComputer Lab Double Class:Double Class:

Thursday 9th and Tuesday 14thThursday 9th and Tuesday 14th Location:Location:

SBS 224SBS 224 TA Office HoursTA Office Hours

Change in time and locationChange in time and location Now Tuesdays after class 12:15 pm - 1:15 pmNow Tuesdays after class 12:15 pm - 1:15 pm SBS 224SBS 224

Review Chomsky Hierarchy:Chomsky Hierarchy:

Type-0 General rewrite rulesType-0 General rewrite rules Type-1 Context-sensitive rulesType-1 Context-sensitive rules aannbbnnccnn

ImplementationImplementation: : type-2 rules + countertype-2 rules + counter• Type-2 Context-free rulesType-2 Context-free rules• aannbbnn

• ImplementationImplementation: : type-2 rules or type-3 rules + countertype-2 rules or type-3 rules + counter– Type-3 Regular grammar rulesType-3 Regular grammar rules– aa++bb++

– ImplementationImplementation: : type-3 type-3 rulesrules

Review

Production rule formats:Production rule formats: Type-1:Type-1:

Next slide…Next slide…Type-2: Type-2:

• A -> A -> • Type-3: Type-3:

– A -> Bc A -> Bc A -> c A -> c or or – A -> cB A -> cB A -> cA -> c

where …where … A A V VNN, c , c V VTT and and (V (VNN u V u VTT))**

Context-sensitive Grammars Type-2 and 3 grammars may only have a single non-terminal on Type-2 and 3 grammars may only have a single non-terminal on

the left.the left. Type-1 (context-sensitive) grammars extend what’s allowed on Type-1 (context-sensitive) grammars extend what’s allowed on

the left.the left. Production rules have the format:Production rules have the format:

-> -> such that |such that || >= || >= || | where …where …

(V(VNN u V u VTT))++ Note: Note: should not be comprised of just terminal symbols should not be comprised of just terminal symbols

(V(VNN u V u VTT))**

NotesNotes:: Length constraint means a rule like A -> Length constraint means a rule like A -> is not permitted is not permitted

Context-sensitive Grammars (Almost equivalent) alternative definition:(Almost equivalent) alternative definition:

Production rules have the format:Production rules have the format: -> ->

where …where … VVNN

(V(VNN u V u VTT))**

NotesNotes:: and and are “copied” over from the left to the right side unchanged are “copied” over from the left to the right side unchanged constitutes the (left and right) contexts for non-terminal Aconstitutes the (left and right) contexts for non-terminal A

i.e. A -> i.e. A -> in context in context However,However,

… … will be more convenient (for our purposes) to use the first definition will be more convenient (for our purposes) to use the first definition

Context-sensitive Grammars Equivalence (informal)Equivalence (informal)

Can transform a context-sensitive rule of form:Can transform a context-sensitive rule of form: AB -> CD AB -> CD

• where A,B,C,D are all different non-terminalswhere A,B,C,D are all different non-terminalsinto rules respecting the form:into rules respecting the form:

-> -> Invent non-terminals A’ and B’Invent non-terminals A’ and B’ Conversion:Conversion:

AABB -> A’ -> A’BB (left context (left context empty, right context empty, right context = B) = B) A’A’B -> B -> A’A’B’B’ (left context (left context = A’, right context = A’, right context empty) empty) A’A’B’B’ -> C -> CB’B’ (left context (left context empty, right context empty, right context = B’) = B’) CCB’ -> B’ -> CCDD (left context (left context = C, right context = C, right context empty) empty) Note:Note:

All four rules respect All four rules respect -> -> yet clearly AB =>yet clearly AB =>++ CD CD

Context-sensitive Grammars ExampleExample: :

GGabcabc is a type-1 grammar such that L(G is a type-1 grammar such that L(Gabcabc) = {a) = {annbbnnccnn | n >=1 } | n >=1 } GGabcabc has 4 production rules: has 4 production rules:

S -> aSBcS -> aSBc S -> abcS -> abc cB -> BccB -> Bc bB -> bbbB -> bb

NotesNotes:: 1st two rules context-free1st two rules context-free 3rd/4th rules context-sensitive3rd/4th rules context-sensitive

• c (resp. b) left context for non-terminal B in cB -> Bc (bB -> bb)c (resp. b) left context for non-terminal B in cB -> Bc (bB -> bb) Length restriction is respectedLength restriction is respected

Context-sensitive Grammars

Compare GCompare Gabcabc to DCG (based on type-2 rules) to DCG (based on type-2 rules) shown in Lecture 10:shown in Lecture 10:

s --> [a],t(1),[c].s --> [a],t(1),[c]. t(N) --> [a],{M is N+1},t(M),[c].t(N) --> [a],{M is N+1},t(M),[c]. t(N) --> u(N).t(N) --> u(N).u(N) --> {N > 1}, [b],{M is N-1},u(M).u(N) --> {N > 1}, [b],{M is N-1},u(M).u(1) --> [b].u(1) --> [b].

Context-sensitive Grammars

How does GHow does Gabcabc work? work? Basic Idea: Basic Idea:

GGabcabc has two stages: has two stages:

1.1. Build an equal number of Build an equal number of aas, s, bbs and s and ccss2.2. Re-arrange them into the correct linear orderRe-arrange them into the correct linear order

We have 4 production rules:We have 4 production rules: S -> aSBcS -> aSBc Stage 1: BuildStage 1: Build S -> abcS -> abc cB -> BccB -> Bc Stage 2: Re-arrangeStage 2: Re-arrange bB -> bbbB -> bb

Context-sensitive Grammars Build Stage:Build Stage:

S -> aSBcS -> aSBc S -> abcS -> abc

Sentential Forms:Sentential Forms: SS aaSSBcBc Using 1st rule Using 1st rule aaaaSSBcBcBcBc Using 1st ruleUsing 1st rule aaabcBcBcaaabcBcBc Using 2nd ruleUsing 2nd rule

Notes:Notes: same number of same number of aas, s, bbs (counting b and B together) ands (counting b and B together) and c cs at s at

each sentential formeach sentential form

Context-sensitive Grammars Re-arrangement Stage:Re-arrangement Stage:

cB -> BccB -> Bc bB -> bbbB -> bb

Example Sentential Form:Example Sentential Form: aaabcBcBcaaabcBcBc

QuestionQuestion: What can we do?: What can we do? AnswerAnswer::

Basic Idea 1: Re-arrange the order of Basic Idea 1: Re-arrange the order of BBs and s and ccss want to move the want to move the BBs to the lefts to the left want to move the want to move the ccs to the rights to the right

QuestionQuestion: How do we know when to stop re-arranging? : How do we know when to stop re-arranging? AnswerAnswer::

Basic Idea 2: Stop when a Basic Idea 2: Stop when a BB comes into contact with a comes into contact with a bb

Context-sensitive Grammars Re-arrangement Stage:Re-arrangement Stage:

cB -> BccB -> Bc re-arrange re-arrange cc and and BB bB -> bbbB -> bb stopping conditionstopping condition

Example Sentential Form:Example Sentential Form: aaabcBcBcaaabcBcBc

Basic Idea 1: Re-arrange the order of Basic Idea 1: Re-arrange the order of BBs and s and ccss cB -> BccB -> Bc ““If a If a cc precedes a precedes a BB, the order is wrong, let’s flip them”, the order is wrong, let’s flip them”

Example of Derivation:Example of Derivation: aaabaaabcBcBcBccBc aaabBcaaabBccBcBcc aaabBaaabBcBcBcccc aaabBBcccaaabBBccc

Context-sensitive Grammars Re-arrangement Stage:Re-arrangement Stage:

cB -> BccB -> Bc re-arrange re-arrange cc and and BB bB -> bbbB -> bb stopping conditionstopping condition

Example Sentential Form:Example Sentential Form: aaabBBcccaaabBBccc

We still have non-terminals in the sentential string We still have non-terminals in the sentential string so we’re not done yetso we’re not done yet

Apply stopping condition:Apply stopping condition: bB -> bbbB -> bb

Example Derivation:Example Derivation: aaaaaabBbBBcccBccc aaabaaabbBbBcccccc aaabbbcccaaabbbccc (a(a33bb33cc33))

Final sentential form contains no non-terminals, so we’re done!Final sentential form contains no non-terminals, so we’re done!

Grammar Rule Implementations Chomsky Hierarchy:Chomsky Hierarchy:

Type-0 General rewrite rulesType-0 General rewrite rules Type-1 Context-sensitive rulesType-1 Context-sensitive rules aannbbnnccnn

ImplementationImplementation: : type-2 rules + counter or type-1 rulestype-2 rules + counter or type-1 rules• Type-2 Context-free rulesType-2 Context-free rules• aannbbnn

• ImplementationImplementation: : type-2 rules or type-3 rules + countertype-2 rules or type-3 rules + counter– Type-3 Regular grammar rulesType-3 Regular grammar rules– aa++bb++

– ImplementationImplementation: : type-3 type-3 rulesrules

Type-0 Grammars

General rewrite rule systemGeneral rewrite rule system Rule format:Rule format:

-> -> where where

(V(VNN u V u VTT))++

(V(VNN u V u VTT))**

Equivalent Automata Implementations

Chomsky Hierarchy:Chomsky Hierarchy: Type-0 General rewrite rulesType-0 General rewrite rules ImplementationImplementation: : Turing Machine (TM)Turing Machine (TM)

Type-1 Context-sensitive rulesType-1 Context-sensitive rules aannbbnnccnn

ImplementationImplementation: : Linear Bounded Automata (LBA)Linear Bounded Automata (LBA)• Type-2 Context-free rulesType-2 Context-free rules• aannbbnn

• ImplementationImplementation: : Non-deterministic Push-Down Automata Non-deterministic Push-Down Automata (NPDA)(NPDA)

– Type-3 Regular grammar rulesType-3 Regular grammar rules– aa++bb++

– ImplementationImplementation: : Finite State Finite State Automata (FSA)Automata (FSA)

Equivalent Automata Implementations

Machine Characteristics:Machine Characteristics: All the machine types have a finite state coreAll the machine types have a finite state core However, they differ with respect to working memoryHowever, they differ with respect to working memory

The difference in expressive power can be traced to The difference in expressive power can be traced to limitations on the working memory…limitations on the working memory…

Equivalent Automata Implementations

GrammarGrammar Working Working MemoryMemory

NotesNotes

Type-3Type-3 NoneNone

Type-2Type-2 StackStack Push/Pop operationsPush/Pop operations

No limit on size of stackNo limit on size of stack

Type-1Type-1 TapeTape Write/(Erase)/Read Move (read head) Write/(Erase)/Read Move (read head) left/rightleft/right

Length limited to a linear function of Length limited to a linear function of the input lengththe input length

Type-0Type-0 TapeTape Write/(Erase)/Read Move (read head) Write/(Erase)/Read Move (read head) left/rightleft/right

Unlimited lengthUnlimited length

DCG and Context-sensitive Grammars

DCG formalism is based on type-2 (context-free) DCG formalism is based on type-2 (context-free) grammarsgrammars

It is powerful enough to encode type-1 (context-It is powerful enough to encode type-1 (context-sensitive) grammarssensitive) grammars

and beyond…and beyond… Example:Example:

in Lecture 10, we exhibited a DCG for ain Lecture 10, we exhibited a DCG for annbbnnccn n

• … … based on type-2 rules plus a counterbased on type-2 rules plus a counter

DCG and Chomsky Hierarchy

Regular Grammars=

Type-3

FSA Regular Expressions

DCG = Type-0

Type-2Type-1

DCG and Context-sensitive Grammars

However, we cannot write DCG rules for context-sensitive However, we cannot write DCG rules for context-sensitive rules directlyrules directly Example:Example:

can’t write [c],b --> b, [c].can’t write [c],b --> b, [c]. for cB -> Bcfor cB -> Bc

NoteNote:: we can write Prolog code that takes a context-sensitive we can write Prolog code that takes a context-sensitive

grammar and interprets itgrammar and interprets it Possible term programming project?Possible term programming project?

This concludes our tour of the grammar This concludes our tour of the grammar hierarchy for this coursehierarchy for this course

But before we leave …But before we leave …

Extra Credit Homework Question

We now know We now know {a{annbbnnccnn | n >=1 } is a context-sensitive (not context-free) | n >=1 } is a context-sensitive (not context-free)

language language {a{annbbnn | n >=1 } is a context-free (not regular) language | n >=1 } is a context-free (not regular) language

How about?How about? LLabcdabcd = {a = {annbbnnccnnddnn | n >=1 } | n >=1 }

Write a grammar for LWrite a grammar for Labcdabcd

NotesNotes:: Not a computer lab homeworkNot a computer lab homework Extra credit question is entirely optionalExtra credit question is entirely optional A chance to make up some ground if you lost points on Homeworks 1 or 2A chance to make up some ground if you lost points on Homeworks 1 or 2 Hand in your solution at the same time as Homework 3Hand in your solution at the same time as Homework 3