Logic Programming for Linguist(ic)s+Logic Grammars with Constraints≈ an easy way to Semantics

Tutorial at GRLMC, Tarragona, 20 November 2009

Henning Christiansenprofessor of Computer Science

Research group PLIS: Programming, Logic and Intelligent Systems

Department of Communication, Business and Information Technologies

Roskilde University

P.O.Box 260, DK-4000 Roskilde, DENMARK

http://www.ruc.dk/~henning

2

Speaker’s background

• Computer Science; 1980ies and onwards programming languages, compilers and syntax (“extensible syntax”)

• Current interests: Logic Programming, Databases, AI, Implementation of advanced reasoning, applications for NLP, Bioinformatics, Probabilistic Models, etc.

• Background in linguistics = Ø, but inspiration from– working with people who know more about language than I do– the reception of my LP work by such people

• Projects– The CONTROL research project (2004–8)

• “CONstrainTs for RObust Language processing”

– The LoSt research project (2007-2012)• "Logic-Statistical analysis of biological sequence data"• 3 PhD students, 2 postdocs, 2 companies, partners in Denmark, Tokyo, ...

contacts to GB, Norway...

3

Motivation & intuition

Logic programming in linguistics / for linguistic students:

• You can program without being a programmer • Logic programming in Prolog is simple, intuitive and well-

suited for phrase-structure grammars

Involving constraints in logic grammars:• Provides an easy way to semantics/pragmatics• while keeping the qualities of Prolog programming

4

Outline of this tutorial1. Crash course on Prolog programming

– with references if you want to continue yourselves

2. Definite Clause Grammars, DCGs (simple add-on to Prolog)

3. Introducing Constraint Handling Rules, CHR (another add-on to Prolog)

4. Relation to Abductive Logic Programming

5. Using CHR for to add meanings to DCGs

6. A practical application

7. Conclusion

• Approach won’t solve all problems, but good for illustrative purposes• and in teaching• (prototype) applications• and (not least) fun to play with!

1. A crash course on PrologProlog differs from Java, C, etc.:• based on logic

– focuses on "what" rather that "how"– programs are concise and compact– programs can be tested and built incrementally– rule-based

• symbols and structure (rather that numbers)• and very easy to learn!

History• A.Colmerauer & co. (Marseille), ca. 1970: "Prolog"• D.H.D. Warren: Efficient compiler, 1975, • Language made known by R.Kowalski "Logic for Problem

solving", 1979, ....• Since then: several available systems...

5

Program is a description of data

parent( tom, bob).

parent( tom, liz).

parent( bob, ann).

parent( bob, pat).

parent( pat, jim).

parent( pam, bob). % Pam is a parent of Bob

pam tom

bob liz

ann pat

jim

Basic notions:

• predicates: parent– describes a relation– defined by facts, rules, collectively called clauses

• constants: tom, bob, x, y• variables: X, Y, Tom• simple goals: parent(A,a)• Queries....

Queries

Atomic queries

?- parent(X,Y).

... give me values of X and Y so parent(X,Y) logically follows from program

Compound query

?- parent(pam, X), parent(X, Y).

... give me X and Y, so that...

Procedural semantics

parent( pam, bob).

parent( tom, bob).

parent( tom, liz).

parent( bob, ann).

parent( bob, pat).

parent( pat, jim).

?- parent(pam, X), parent(X, Y).

X=bob

?- parent(bob, Y).

Y=ann

Success!Other solutions?

Y=pat Success!Other Solutions?

No more possiblesolutions at all :(No more possiblesolutions here :(

•Unification term=term?

•from left to right

•from start to end

•backtracking

≈ undo and try new choices

Rules

grandparent(X,Z):- parent(X, Y), parent(Y, Z).

Procedural semantics

as before + rewrite subgoal using rules

Declarative semantics ≈ logical consequence in 1st order logic

The nice property:

procedural ≈ declarative

Other features

• structures and lists (will see later)

• control mechanisms (advanced; not needed for today)

11

2. Definite Clause Grammars

Syntactic sugar for Prolog programs that perform top-down, backtracking parsing.

In Prolog since 1970ies [Colmerauer; Pereira, Warren; ...]

Features ≈ arguments and terms as in Prolog predicates

Example:

Popular, easy to work with, fun to play with, etc.

?- [simpleDCG].

12

Adding contexts ≈ knowledge bases

Usual approach: more and complex features.

Discourse analysis with semantics

story(C0,M1+M2) --> [] ; s(C0,C1,M1), ['.'], story(C1,M2).

We try another something else: use a global resource to store context and meanings.

Thus stay with the rule

story --> [] ; s, ['.'], story.

But “global resource” in Prolog??= constraint store and meanings etc. as constraints...

13

3. Constraint Handling Rules, CHR

A recent addition to Prolog; available in several major Prologs [Frühwirth, 1992, 1998]

A “glass box” approach to constraint solving:• defining constraints and their behaviour in a

declarative language

Prolog: Backward chaining

CHR: Forward chaining

Prolog+CHR by an example that anticipates applications for language analysis...

?- [happy1]. ?- [happy2]. ?- [happy3].

14

3. Theoretical interlude:

You have just seen an implementation of Abductive Logic Programming using Prolog+CHR (Peirce, 18-something); (Conan Doyle, 19-something - Sherlock Holmes); Kakas, Kowalski, etc. 1980ies, ......)

Def: Given Rules and Observations, abducible predicates and Integrity Constraints, a set of abducible facts A is an abductive answer wheneverRules U A |= ObservationsRules U A |= Integrity Constraints

?- [RulesAbdPredsICs].?- Observations.abductive-answer ? ;

Relationship ALP <-> CHR:

[S.Abdennadher, HC, 2000; HC, V.Dahl, 2004, etc.]

H. Christiansen. Executable specifications for hypothesis-based reasoning with Prolog and Constraint Handling Rules, Journal of Applied Logic, vol 7, 2009. pp. 341-362

15

3. Back to CHR

Three kinds of rules:

Propagatec, c, ..., c ==> Guard | ... c ...

adds constraints

Simplify c, c, ..., c <=> Guard | ... c ...

replaces constraints

Simpagate c, ... \ c, ... <=> Guard | ... c ...

remove some, keep some and add others

———————————————————————

Declarative semantics: as indicated by the arrow shapes

Obs: Prolog+CHR sum up to a language with declarative semantics

Theorem: Implementation of abduction is correct :)

16

5. Back to discourse analysis

First version, just collecting facts

Second version, adding rules about semantic/pragmatic universe

—————————————————————————————Reflections:• An instance of “Interpretation of abduction”, [Hobbs & al, 1993]• Notice that distinction semantics/pragmatics disappears, i.e.,

– i.e., no “objective and context independent” meaning before mapping to “real world objects”,

• [Christiansen, Dahl, Context05] formalize “Meaning in Context” with possible-worlds semantics and relates to abduction and CHR

?- [discourse1].

?- [discourse2].

17

A taste of “Meaning in Context” [Christiansen, Dahl, Context05]

Meaning of

The tall, red-haired man carrying a laptop won a brand new Ferrari.

is

won(X,Z)

in the context

tall(X), red_haired(X), carries(X,Y), laptop(Y),

won(X,Z), ferrari(Z), brand_new(Z)

Might be the same as the meaning of

He won it

18

Context in disambiguation

• version 1, static context

• version 2, non-monotonic change of context

Theoretical interlude• Version 2 goes beyond the indicated declarative

semantics• A programmer’s mind uses procedural knowledge

to implement a sort of non-monotonic logic.• [Betz, Frühwirth, CP05] has defined a linear-logic

semantics for CHR which fits such usages

?- [contextLex1].

?- [contextLex2].

19

Assumptions in logic grammars• [Dahl & al., 1997; Christiansen, Dahl, 2004, ...]• Similar to abduction but with explicit creation and

application + simple scoping• Can be implemented in CHR more or less the same way• Included in the Hyprolog system

Examples...

+A Assert linear assumption A for subsequent proof steps.

Linear means “can be used once”.

*A Assert intuitionistic assumption A for subsequent proof steps.

Intuitionistic means “can be used any number of times”.

-A Expectation: consume/apply existing intuitionistic assumption in the state which unifies with A.

=+A, =*A, =-A Timeless versions of the above, meaning that order of assertion of assumptions and their application or consumption can be arbitrary.

20

Assumptions in HyprologAn example: Pronoun resolution

assumptions acting/1.

abducibles fact/1.

sentence --> np(A,_), verb(V), np(B,_), {fact(A,V,B)}.

sentences --> [] ; sentence(S1),sentences(S2).

np(X,Gender) --> name(X,Gender), {*acting(X,Gender)}.

name(peter,masc) --> [peter].

...

np(X,Gender) --> {-acting(X,Gender)}, pronoun(Gender).

pronoun(fem) --> [her].

...

verb(like) --> [likes].

“Peter likes Mary. She likes him”

*acting(peter,masc) *acting(mary,fem)-acting(X,fem) X=mary-acting(X, masc) X=peter

fact(peter,like,mary) fact(mary, like,peter)

21

Assumptions in HyprologAnother example: Coordination

assumptions ..., verb_used/1.

sentence --> np(A,_), verb(V), np(B,_),

{*verb_used(V), fact(A,V,B)}.

...

verb(like) --> [likes].

verb(V) --> [],{-verb_used(V)}.

“Peter likes Mary, and she him”...*verb_used(like)-verb_used(X) X=like

fact(peter,like,mary) fact(mary,like,peter)

22

Side remark

What we have shown until now, may be seen as logic programming dressed up by a nice syntax.

It has been described in a formal way:

Henning Christiansen, Verónica Dahl

Abductive Logic Grammars.

Logic, Language, Information and Computation. 16th International Workshop, WoLLIC 2009. Tokyo, Japan, June 21-24, 2009. Lecture Notes in Computer Science 5514, pp. 170-181, Springer 2009.

23

6. An practical application in more detail

Mapping use case text (a la OOP/OOD) into UML diagrams

Example of discourse analysis, building a knowledge base = context

Features:• non-trivial properties of semantic/pragmatic universe• resolution of pronouns based on well-defined heuristics

[Christiansen, Have, Tveitane, RANLP2007, CSLP2007]

The setting...

• Use Cases– Textual description of what a “system” does without

specifying how.– Used in the inception phase of the object oriented software

development process.

• Reasoning about Use Cases– Parsing use cases to uncover programmatic semantics – Result: Diagrammatic representation (UML)

Language for Use Cases

• Restricted natural language (English)• Natural flow, style and expressiveness• Unambiguous parsing• Why avoid ambiguity

Programmatic Semantics

• Classes and objects• Methods• Properties• Inheritance• Instantiation• Temporality (not shown here)

Programmatic semantics, methods

• The professor teaches.

Generates one class:

With one method

Programmatic semantics, methods

• The professor teaches. • A student reads, writes

projects and takes exams.

Programmatic Semantics, properties

• A professor has an office.

Programmatic Semantics, properties

• A professor has an office.

• The university has five study lines.

Programmatic Semantics, inheritance

• Students and professors are persons.

System Architecture

Architecture = Prolog + DCG + CHR Diagrams are generated by GraphViz

Input grammar

sentence --> fc_noun_phrase(Number,_ , subj, IdSub), subord_verb(Number,_ ), fc_noun_phrase(_ , Number, obj, IdObj), {extends(IdSub,IdObj)}.

The use case text is parsed using a Definite Clause Grammar (DCG)

'extends' is not ausual Prolog predicate,

but declared as a constraint

There are sentence grammar rules like this for different sentence types Each rule captures the semantic features of a particular type of sentence

fc_ “cats and dogs” cat+dogindiv_ “her, Peter and Paul” mary+peter+paulrc_ “Mary and the boys” woman+boyq_ “a tail and some legs” tail:1+legs:n

Reasoning with Constraints, I

extends(A+B, C) <=> extends(A,C), extends(B,C).

extends(A, B+C) <=> extends(A, B), extends(A, C).

Example:Professors and students are persons.

extends(professor+student, person)

extends(professor, person)extends(student, person)

extends(professor, teacher+researcher)

Example:A professor is a teacher and a researcher.

extends(professor, teacher)extends(professor, researcher)

Constraint Handling rule: Constraint Handling rule:

Constraint store: Constraint store:

Reasoning with Constraints, II

property(C,P:N), property(C,P:M) <=> q_count(N), q_count(M), q_less_eq(N,M) | property(C,P:(N..M)).

Example: Paul has a dog and Peter has five dogs. (Peter and Paul are men).

...property(man, dog:1)property(man, dog:5)

Constraint Handling rule:

property(man, dog:(1..5))

Reasoning with Constraints, III

Ranges can be merged:

property(C,P:(N1..M1)),property(C,P:(N2..M2)) <=> q_min(N1,N2,N), q_max(M1,M2,M), property(C,P:(N..M)).

property(man, dog:(0..2))property(man, dog:(1..n))

property(man, dog:(0..n))

Constraint Handling rule:

Reasoning with Constraints, IVAnaphora resolution - Individuals as prototypes.

Example: John is a student. John studies.

class(student)object(john, student)method(john, study)

class(student)object(john, student)method(student, study)

object(Id, Cl) \ method(Id, M) <=> method(Cl, M)

Constraint Handling Rule:

Reasoning with Constraints, VAnaphora resolution - Pronouns.Example: Jack and John are teachers. Jack teaches music. John teaches computer science. Mary is a student. He has many students.

He refers to John.

...referent(sing,masc,1,jack)referent(sing,masc,1,john)referent(sing,masc,2,jack)referent(sing,masc,3,john)referent(sing,fem,4,mary)

expect_referent(sing,masc,X)

Constraint store:

Referents are labelled with the sentence number in which they occur.

Heuristic method resolves the entity matching number and gender in the most recent sentence.

Ambiguous pronouns are rejected.

sentence_no(Now), referent(No,G,Id,T) \ expect_referent(No,G,X) <=>

T < Now, there is no other relevant referent with Timestamp > T

|

if there is another relevant referent with Timestamp = T then X = errorcode(ambiguous) else X = Id.

Constraint Handling Rule:

39

Conclusion of the example

Shows an application of discourse analysis with• restricted language• “concise” semantics• a desire of an unambiguous interpretation

Demonstrates that Prolog+DCG+CHR applied for “interpretation as abduction” in this case provides

• flexibility and ease of modeling• concise, executable specifications

– logic embedded in a full programming language, i.e., you can hack the logic if you feel like it

40

Other approaches to languages analysis using CHRCHR grammars [Christiansen, 2002-3, TPLP2005]Grammar notation for bottom-up parsers entirely with CHR; features• Context-sensitive rules

a -\ b,c /- ..., d ::> e compiled into

a(_,N1), b(N1,N2), c(N2,N3), d(N4,_) ==> N3=<N4 | e(N1,N3)

• Ambiguity and partial parsing handled for free• Includes abduction and assumption• Disadvantage: Alternative interpretations all mixed up• a version exists, which handles this correctly; never documented or released

(awkward implementation in CHR; low-level impl. considered)

Applications• Blache & Dahl: Property grammars (repr. phrase as set of sat/unsat constraints)

• Dahl and her students: Biomedical text; sequence analysis; etc.• My students: Hieroglyph inscriptions; use cases (a la OOP);

sign language (strong potential; but, sic!, we did not get the grant)

41

Conclusions

• Constraints adds a state component to logic grammars• The state can be first-order formulas or other other

kinds of knowledge representation• Interpretation-as-abduction for discourse analysis fits in

naturally• Nonmonotonic updates can be modelled, e.g., reflecting

a time dimension over a changing world• Prolog + CHR provides

– a flexibility for a programmer’s mind to hack the logic (and perhaps formalize afterwards;-)

– an effective environment for prototyping / executable specifications / implementation

42

Recent resultsProbabilistic & prioritized abduction in CHR, best first search• H.Christiansen. Implementing Probabilistic Abductive Logic Programming with

Constraint Handling Rules, 35 pp., Constraint Handling Rules, Current Research Topics. T. Schrijvers, T. Frühwirth (eds.). Lecture Notes in Artificial Intelligence 5388, Springer 2008. pp. 85-118

• H. Christiansen. Prioritized Abduction with CHR, Proc. Fifth Workshop on Constraint Handling Rules, Hagenberg, Austria, July 14, 2008.

Probabilistic DCGs• C.T.Have. Stochastic Definite Clause Grammars. Proc. RANLP 2009, Recent

Advances in Natural Language Processing, Bulgaria, September 14-17, 2009.• C.T.Have., Stochastic Definite Clause Grammars; Master thesis, IT-University of

Copenhagen, 2008

Work in progressGeneral probabilistic-logic models for bioinformatics, e.g.• H. Christiansen. C.T. Have, O.T. Lassen, M. Petit. A Constraint Model for

Constrained Hidden Markov Models: a first Biological Application WCB09, Workshop on Constraint Based Methods for Bioinformatics, Associated to CP 2009. Lisbon, Portugal, September 20, 2009.

ReferencesTexts• Intro to Prolog with CHR and AI applications

H.Christiansen: Introduction to Prolog as a framework for Knowledge Representation and Artificial Intelligence, (2006)

http://www.ruc.dk/~henning/KIIS08/courseMaterial/CourseNote.pdf

• Intro to Prolog, focus on DCGP.Blackburn, J.Bos, and K.Striegnitz: Learn Prolog Now! (2001)http://www.learnprolognow.org

• Overview of Prolog+CHR for logical reasoningHenning Christiansen. Executable specifications for hypothesis-based reasoning with Prolog and

Constraint Handling Rules, Journal of Applied Logic, vol 7, 2009. pp. 341-362Preliminary version: http://www.ruc.dk/~henning/publications/HypoReason2008_HC.pdf

Good Prolog systems with CHR• SICStus Prolog (costs money, institution license?, 30 days test version)

http://www.sics.se/isl/sicstuswww/site/index.html

• SWI Prolog, freehttp://www.swi-prolog.org/

Software by the speaker:• HYPROLOG: http://www.ruc.dk/~henning/hyprolog/

• CHR Grammars: http://www.ruc.dk/~henning/chrg/

43

44

The end