Reading to LearnQ1 review (7/6/05)

Peter Clark Michael Glass

Phil Harrison Tom Jenkins

John Thompson Rick Wojcik

Boeing Phantom Works

Agenda

Introduction

1. Textual knowledge to CPL

2. CPL to logic: How CPL is interpretedProcessing some CPL (demo)

3. Knowledge Integration

Extracting knowledge from text

LbR Framework

Corpus

Using what youknow to get moreKnowledge

Repository“Worldview”

Knowledge Acquisition Loop

RobustReasoning

Tasks

Introspection

Solidification Loop

KnowledgeIntegration

Corpus

Using what youknow to get more

KnowledgeIntegration

KnowledgeRepository

“Worldview”

Knowledge Acquisition Loop

RobustReasoning

Tasks

Introspection

Solidification Loop

CPLor

Logic

LbR Framework

Overview

What does a person do?a) Start aready knowing something about a domain + general

knowledge

b) Read; existing knowledge helps him/her understand the new material

c) integrate the new knowledge into pre-existing knowledge

d) can now perform new tasks

– In its full and unrestricted form, is too difficult to be implemented

– BUT: significant, partial approaches are feasible

Reduced Version

Select a domain where KB exists (chemistry) Manually reformulate a section of text into controlled

English (CPL) Automatically process that controlled English to

generate new knowledge Integrate the new knowledge into the KB

Rationale: Two key problems in reading to learn:

full natural language processing knowledge integration

This approach separates them and can focus on (2)

Corpus

Using what youknow to get more

KnowledgeIntegration

KnowledgeRepository

“Worldview”

Knowledge Acquisition Loop

RobustReasoning

Tasks

Introspection

Solidification Loop

CPLor

Logic

Step 1: Unrestricted NL to CPL

Step 1: Possible methods

Texts(unrestricted

English)

Textin restricted

English

Manual reformulation

Machine “translation”

Locate subset in simple English

Knowledge extraction from corpora

Step 1: Possible methods

Texts(unrestricted

English)

Textin restricted

English

Manual reformulation

Machine “translation”

Locate subset in simple English

Knowledge extraction from corpora

Reformulating Chemistry Text into CPL

John A. Thompson

Selecting the important text

Textbook: “Acids have a sour taste (for example, citric acid in lemon

juice) and cause certain dyes to change color (for example, litmus turns red on contact with acids). Indeed, the word acid comes from the Latin word acidus, meaning sour or tart.”

Selected and reworded as CPL: “Acids have a sour taste.” “Acids cause some dyes to change color.”

Judgment calls regarding “what might be on the test” Simplified rewordings – no pronouns or complex

sentences, see CPL User Guide

Example 2: Text to CPL

Textbook: “Sodium hydroxide is an Arrhenius base. Because NaOH is

an ionic compound, it dissociates into Na+ and HO- ions when it dissolves in water, thereby releasing OH- ions into the solution.”

CPL: “Sodium hydroxide is an Arrhenius base.” “NaOH is sodium hydroxide.” [inserted background knowledge] “NaOH is an ionic compound.” “NaOH dissolves in water.” “NaOH dissociates in water.” “The dissociating produces Na-plus ions and OH-minus ions.”

Example 3: Text to CPL

Textbook: “Some substances can act as an acid in one reaction and as

a base in another. For example, H2O is a Bronsted-Lowry base in its reaction with HCl and a Bronsted-Lowry acid in its reaction with NH3. A substance that is capable of acting as either an acid or a base is called amphoteric.”

CPL: “Some substances sometimes act as a Bronsted-Lowry acid and

sometimes act as a Bronsted-Lowry base.” [required “and”] “These substances are called amphoteric substances.” [vocab] “H2O acts as a Bronsted-Lowry base in a reaction with HCl.” “H20 acts as a Bronsted-Lowry acid in a reaction with NH3.” “Therefore, H2O is an amphoteric substance.” [deduced]

Rewording generics

“Acids have a sour taste” is a generic sentence about a class of things – the textbook is full of generics!

One interpretation: “Every instance of an acid has a sour taste” But is it “every” instance, or just “typically”?

Another interpretation: “If a person is tasting an acid, then the person is experiencing

a sour taste” We are planning to do some automatic interpretation

of generic sentences in the future Our short-term strategy is to reword every generic

sentence to another form, especially to an if-then rule

Rewording generics - 2

Another example: “Acids cause some dyes to change color”

One interpretation: “There are some dyes that change color when in contact with

any acid” Another interpretation:

“For each acid there is some dye that changes color when in contact with the acid”

Possible CPL rewording as an if-then rule: “If an acid-sensitive dye is in contact with an acid, then the

acid is causing the dye to change color” Bottom line: Generics are a major issue with textbook

knowledge

CPL now includes if-then rules

Rewriting generic sentences required if-then rules to be added to CPL

Examples of if-then rules in CPL: “If HCl is immersed in water, then the HCl is dissolving in the

water.” “If HCl is dissolving water, then each molecule of the HCL is

reacting with a molecule of the water.” “If an HCl molecule is reacting with a water molecule, then an

H-plus ion is transferring from the HCl molecule to the water molecule.”

Connecting verb and noun forms

Example: reacts vs. reaction “Hydrogen chloride gas in water reacts with the water” “The reaction produces H-plus ions and Cl-minus ions

The CPL writer assumes that the interpreter will make the connection from the verb to the related noun in the next sentence

We are building a large set of verb-noun relations to handle these cases

Gross vs. molecular events

Chemistry text switches between: gross-level events involving substances reactions between two molecules

Example: “When HCl dissolves in water, we find that the HCl molecule

transfers an H+ ion (a proton) to a water molecule” In CPL, we interpret “HCl” as the gross-level substance,

and “HCl molecule” as the molecular-level individual If-then rules carry the logic from the gross level to the

molecular level: “If HCl is dissolving water, then each molecule of the HCL is

reacting with a molecule of the water.”

Issues with fuzzy information

Examples: “An H-plus ion sometimes reacts with an H2O molecule” “A molecule of a Bronsted-Lowry acid can donate a proton to

another substance” “The NH4 is mostly solid particles” “Some substances containing hydrogen are not acids” “An ion of a Bronsted-Lowry acid must have a hydrogen atom.”

These are easy to state in CPL, but what logical representation should be produced?

This is a universal problem in any kind of language interpretation

We are developing solutions as we progress

The textbook teaches by example

Example: “A molecule of a Bronsted-Lowry acid can donate a proton to

another substance” “An HCl molecule in water donates a proton to an H2O

molecule” “Therefore, the HCl molecule acts as a Bronsted-Lowry acid”

Text is using an example to teach the student how to make similar deductions

Unclear how to capture this in CPL – how to specify where the deductive chain begins?

Perhaps we should skip the examples and only enter the general principles being taught?

Teaching by hypothetical situations

Example: “[Assume that] H2O is a stronger base than X-minus in Equation 16.9” “[Assume that] X-minus is the conjugate base of HX in Equation 16.9” “H2O extracts the proton from HX in the reaction in Equation 16.9” “The reaction produces H3O-plus and X-minus” Therefore, the equilibrium is on the right side of Equation 16.9”

How can CPL make it clear that this is a hypothetical? Note that X stands for a variety of atoms So the hypothetical teaches the student a useful pattern How should we represent and reason about this?

Real world vs. written equations

The textbook switches between the real world and the syntax of written chemical equations: “If H2O (the base in the forward reaction) is a stronger base

than X- (the conjugate base of HX), then H2O will abstract the proton from HX to produce H30+ and X-. As a result, the equilibrium will lie to the right.”

The CPL author must be careful to mention the equation number when referring to its syntax: “In Equation 16.9 H2O is the base in the forward reaction.” “The equilibrium is on the right side of Equation 16.9”

Textbook figures and tables

A textbook is not just text! Some figures and diagrams can be re-expressed in

text, but others cannot be Tables (such as 16.4, a listing of the relative strengths

of some conjugate acid-base pairs) can be converted to many lines of CPL text

We are omitting sample & practice exercises from the CPL

Textbook to CPL: Conclusions

Authors can learn to restate the essential parts of a textbook in simple CPL sentences

Most textbook sentences contain ambiguous “generics” about classes of things

For now, we can disambiguate generics by rewriting them as if-then rules in CPL

For most CPL sentences, we can process them to get an adequate logical representation (with some user assistance)

Special problems involving fuzzy statements and hypothetical examples require further work

Corpus

Using what youknow to get more

KnowledgeIntegration

KnowledgeRepository

“Worldview”

Knowledge Acquisition Loop

RobustReasoning

Tasks

Introspection

Solidification Loop

CPLor

Logic

Step 2: CPL Interpretation

“HCl is immersed in water”

Parser & LF Generator

Word sense disambiguator

Relational disambiguator

Coreference identifier

Structural reorganizer

HCl Water

Immerse

object is-inside-of

(_HCl13320 instance_of HCl-Substance)(_Water13321 instance_of Water)(_Immerse13319 instance_of Move-Into)(_Immerse13319 object _HCl13320)(_Immerse13319 is-inside-of _Water13321)

WorldKnowledge

LinguisticKnowledge

Step 2: CPL Interpretation (overview)

CPL Processing Spots coreferences across sentences Handles nominalizations (“reacts”, “the reaction”) Use of WordNet to coerce text to KB’s ontology Set of heuristics for identifying semantic relations Interprets rules, as well as ground facts

Rules are in KM (inference-capable)

Can be used in interactive or automatic mode

Integrating Knowledge from Reading into a Knowledge Base

Michael Glass

Boeing Phantom Works

Corpus

Using what youknow to get more

KnowledgeIntegration

KnowledgeRepository

“Worldview”

Knowledge Acquisition Loop

RobustReasoning

Tasks

Introspection

Solidification Loop

CPLor

Logic

Step 3: Knowledge Integration

The Task

After knowledge is input through CPL it must integrated with existing knowledge in the KB.

This task carries with it three main problems. Missing Concepts: The new knowledge may

contain concepts the KB does not have. Conflicting Concepts: The new knowledge may

use concepts in the KB in an incompatible way. Elaboration Requires Updates: The knowledge

base may be resistant to new knowledge.

1. Missing Concepts

Consider: If HCl is immersed in water, then the HCl is dissolving in the water.

The knowledge base does not know the concept “immerse” or “dissolve”. Even if they were added in a superficial way, the knowledge

base would have no axioms allowing it to reason about immersion or dissolving.

This is the simplest type of mismatch. It is relatively easy to detect. It simply requires adding the concepts to the KB or

the user may restrict himself to concepts the KB already has.

1. Missing Concepts: Solutions

Add the concepts to the KB. The concepts could be added selectively as needed

and (partially) axiomatized. The concepts could be added in bulk in a vacuous

way. Defining new concepts through CPL.

Restrict the user to the set of concepts in the KB. In many cases a similar concept in the knowledge

base may serve just as well.

2. Conflicting Concepts

Two significant classes of conflicting concepts:Genuine Conflicting Concepts: A concept in

the KB may simply not match what is intended.

Naïve Encodings: The KB may expect knowledge structured in a specific way.

2. Conflicting Concepts

Consider: If HCl is dissolving water, then each molecule of the HCl is reacting with a molecule of the water.

The knowledge base has a concept of a Reaction, but the inputs are required to be Chemicals (aggregates of molecules). NOT Molecules.

The reaction concept is well axiomatized, but some of these axioms are not applicable to molecules.

2. Conflicting Concepts

This might be considered another case of missing concepts. The KB has concepts for reasoning about reactions

at the macroscopic level, but not at the molecular level.

2. Conflicting Concepts

Consider: If an HCl molecule is reacting with a water molecule, then an H-plus ion is transferring from the HCl molecule to the water molecule.

Transferring is in the KB, but it is axiomatized in a way consistent with a person transferring an object to another person. The object transferred must be in the “possession”

of the donor.

2. Naïve Encodings, A Special Case of Conflicting Concepts

There can be a mismatch in the form in which a new piece of knowledge is stated and the form in which it is expected.

A chemistry textbook might refer to the equilibrium constant of a reaction rather than the equilibrium constant of a equilibrium reaction.

The Chemistry knowledge base expects only equilibrium reactions to have equilibrium constants.

2. Naïve Encodings

Also the CPL parser may produce KM that is not quite right:Produced by CPL: (a Chemical with (color

(*red)))Expected by KM: (a Chemical with (color ((a

Color-Value with (value (*red))))))The difference is purely structural.

2. Conflicting Concepts: Solutions

First the conflict must be detected Constraint violations: Certainly if a reaction

requires Chemical raw-materials and it is given Molecules instead, there must be a conflict.

Resemblance check: If the knowledge looks totally novel, such as a Transfer from one Molecule to another, it may be a conceptual conflict.

To fix a naïve encoding an automatic solution may be used to coerce the concepts into an acceptable form. (James Fan’s Loosespeak)

3. Elaboration Requiring Updates

Two significant ways a KB may require updates to add knowledge:Closed World Assumption: The KB may

implicitly or explicitly assume it already knows everything about a given topic.

Unstated Assumptions: The KB designers may have built the KB with some assumptions about the circumstances it will reason about.

3. Updates and the Closed World Assumption

Some rules in the KB close off the KB to further elaboration.

The else part of an if…then…else may give a default value that prevents other components from concluding anything else.

3. Unstated Assumptions

Often a knowledge base will be built with a set of unstated assumptions about how it will be used.

These assumptions may lead to assertions in the knowledge base stated as always true, but really true only under certain circumstances.

This can result in knowledge that is difficult to extend.

3. Unstated Assumptions: Example

Consider a knowledge base meant to deal with chemical reactions at room temperature and standard pressure.

This knowledge base might include the assertion that water is a liquid. (every Water has (state (*liquid)))

However, if the scope of the knowledge base is extended to consider a range of temperatures, the knowledge base is not simply incomplete, but wrong.

3. Elaboration Requiring Updates: Solutions

First the conflict must be detected.The knowledge base could forward chain for

a while on concepts related to the new knowledge.

Resolving the conflict is difficult in general. It may entail a case of the attribution

problem.

Conclusion and Future Directions

Missing ConceptsAllow additions to ontology through CPL

Naïve EncodingsLoosespeak interpreter

Genuine Conflicting Concepts and Elaboration Requiring UpdatesDetection through constraint checks,

resemblance check and forward chaining.Correction assisted by feedback.

Acquiring Knowledge from Reading

Phil Harrison

Boeing Phantom Works

Corpus

Using what youknow to get more

KnowledgeIntegration

KnowledgeRepository

“Worldview”

Knowledge Acquisition Loop

RobustReasoning

Tasks

Introspection

Solidification Loop

CPLor

Logic

Step 1: Unrestricted NL to CPL - an additional approach

The Problem

Uncontrolled (non CPL) text is difficult for computers to analyze.

Advances are needed in all areas of NLP: grammar, WSD, semantic representation, and discourse processing.

A method is needed for extracting as much knowledge as possible from text.

Tuple extraction

The Parse trees are a source of “head-complement” or “head-modifier” relations: From “The heavy man bought an expensive book”

(S “man” “buy” “book”) (AN “heavy” “man”) (AN “expensive” “book”)

“Books can be bought” “Men can be heavy” “Books can be expensive”

Even incorrect parses can generate some valid tuples.

Examples from chemistry

Acids can be strong or weak.Acids can be vitamins.Acids can release ions.Bases can turn litmus.Carbonates can form CO2.Chlorides can become ions.Concentrations can be measured.

Uses of tuples

Extracted tuples can guide parsing.Human review of tuples is necessary.Iteration over a corpus improves

accuracy and allows more knowledge extraction.

The generic sentences derived from tuples can be integrated into a knowledge base.

Interpretation of generic sentences

Generics are characterized by the use of bare plurals, mass terms, or adverbs of quantification.

Examples: “Birds fly” “Ice is cold” “John usually drinks a beer”

Interpretation of generics is problematic, but a substantial amount of work has been done.

Summary

This is not a method for full and general reading.

But, can potentially acquire certain types of knowledge very quickly

Illustrated in the domain of chemistry

Overall Future Directions

CPL extensionsCPL processing of generics

Appropriate structure of KB for effective knowledge integration

Degree of automation possiblenow, 5 years from now

Evaluation