building a system that can learn by reading
DESCRIPTION
Building a System that can Learn by Reading. Kevin Livingston PhD Candidate Cognitive Systems Division EECS Department Northwestern University Presented at University of Dayton November 3, 2006 As part of the Computer Science Research Colloquium Series. Text. Understanding Systems. Text - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/1.jpg)
1
Building a System that canLearn by Reading
Kevin LivingstonPhD Candidate
Cognitive Systems DivisionEECS Department
Northwestern University
Presented at University of Dayton November 3, 2006As part of the Computer Science Research Colloquium Series
![Page 2: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/2.jpg)
2
Understanding Systems
Knowledge Base
Text
Text Understanding
System QuestionAnswering
Explanation
Reasoning
![Page 3: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/3.jpg)
3
Examples of Intelligent Systems
• Digital Assistants
• Conversational Agents– Imagine a video game character that actually
talked to you
• Intelligent Games
• Search
• Question Answering
![Page 4: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/4.jpg)
4
Modeling and Using Knowledge
• Frames (Minsky)
• Scripts (Shank)
• Qualitative Reasoning (Forbus)
![Page 5: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/5.jpg)
5
Semantic Memory
• Ontology– Doctors are people– Bombings are events
• Semantics– Doctors have patients– Bombings have targets
![Page 6: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/6.jpg)
6
Episodic Memory
• Memory Instances– Madrid is a city– Madrid is in Spain– The Madrid Bombings is a terrorist attack– The Madrid Bombings occurred on March 11,
2004– Al Qaida is a terrorist organization
![Page 7: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/7.jpg)
7
Modeling Reasoning
• Logic Based Systems– Cyc (Cycorp; Austin, TX)– Fire (Qualitative Reasoning Group (QRG);
Northwestern)
• Statistical• Bayesian• Markovian• Neural Nets
![Page 8: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/8.jpg)
8
Knowledge Bases
Knowledge Base
Text
Text Understanding
System QuestionAnswering
Explanation
Reasoning
![Page 9: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/9.jpg)
9
Available Knowledge Bases• Information Retrieval (IR) techniques
– mine information from Internet (MUC and TREC)
• Open Mind Common Sense (OMCS)– Sentences collected from Internet contributors– Mined for knowledge
• Knowledge Machine (KM)– Frame based
• ResearchCyc– Latest release ~3,000,000 assertions– Predicate Logic in CycL
• First order and some second order constructs
![Page 10: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/10.jpg)
10
ResearchCyc
• Bombings are Attacks and Attacks are Events(genls Bombing AttackOnObject)
(genls AttackOnObject Event)
• The Madrid terrorist attack was a bombing(isa
TerroristAttack-September-8-2003-Madrid
Bombing)
• Al Qaida is an Islamist Terrorist Group(isa AlQaida TerroristGroup-Islamist)
![Page 11: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/11.jpg)
11
ResearchCyc (cont.)
• Location of an event
(eventOccursAt
TerroristAttack-September-8-2003-Madrid
CityOfMadridSpain) • Perpetrator of an event(perpetrator
TerroristAttack-September-8-2003-Madrid
AlQaida)
![Page 12: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/12.jpg)
12
ResearchCyc (cont.)
• Person being killed(organismKilled SpaceShuttleChallengerDisaster ChristaMcAuliffe)
• Deaths caused by an attack(deathToll
TerroristAttack-September-8-2003-Madrid
Person190)
![Page 13: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/13.jpg)
13
Building Knowledge Bases
• Slow and Tedious– years to grow Cyc from 1.2M to 3M assertions
• Requires Training– measured in weeks+, for GUI tools (SHAKEN)
• Expensive– Project Halo estimates $10,000 per page!
of AP level Chemistry content
![Page 14: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/14.jpg)
14
Available Information
• Encyclopedias
• Newspapers
• Online sources
![Page 15: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/15.jpg)
15
A Better Way?Teach the Computer to Read
![Page 16: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/16.jpg)
16
What we want to Read
• Episodic Knowledge– New people– New events
• General Knowledge– “the heart is a pump”
![Page 17: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/17.jpg)
17
Standard Model forNatural Language Processing
Dictionary
Text
POSTagging
Grammar
SyntacticParser
SemanticInterpreter
Semantics
Tagged Text
![Page 18: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/18.jpg)
18
“Time flies like an arrow.”
![Page 19: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/19.jpg)
19
“Time flies like an arrow.”• Time moves quickly just like an arrow does
![Page 20: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/20.jpg)
20
“Time flies like an arrow.”• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
![Page 21: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/21.jpg)
21
“Time flies like an arrow.”• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
• Time flies in the same way that an arrow would (time them)
![Page 22: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/22.jpg)
22
“Time flies like an arrow.”• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
• Time flies in the same way that an arrow would (time them)
• Time those flies that are like arrows;
![Page 23: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/23.jpg)
23
“Time flies like an arrow.”• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
• Time flies in the same way that an arrow would (time them)
• Time those flies that are like arrows;
• A type of flying insect, "time-flies," enjoy arrows (compare Fruit flies like a banana.)
![Page 24: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/24.jpg)
24
Picture ofGeorge Burns
AndGrace Allen
![Page 25: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/25.jpg)
25
Grace,Those are beautiful flowers.
Picture ofGeorge Burns
AndGrace Allen
![Page 26: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/26.jpg)
26
Grace,Those are beautiful flowers.
Where did they come
from?
Picture ofGeorge Burns
AndGrace Allen
![Page 27: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/27.jpg)
27
Don’t you remember, George?
Picture ofGeorge Burns
AndGrace Allen
![Page 28: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/28.jpg)
28
Picture ofGeorge Burns
AndGrace Allen
You said that if I went to visit Clara Bagley
in the hospital I should be sure to take her flowers.
So when she wasn’t looking,
I did.
![Page 29: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/29.jpg)
29
Picture ofGeorge Burns
AndGrace Allen
![Page 30: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/30.jpg)
30
Common Sense
• “You said that if I went to visit Clara Bagley in the hospital I should be sure to take her flowers. So, when she wasn't looking, I did.”
– take flowers from her– take flowers to her
![Page 31: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/31.jpg)
31
Picture ofElevatorOperator
![Page 32: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/32.jpg)
32
Down?
Picture ofElevatorOperator
![Page 33: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/33.jpg)
33
ThatWay!
Picture ofElevatorOperator
![Page 34: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/34.jpg)
34
Context in the Environment
“Down?”
What does the question mean?– Which way is down?– Are you going down?
![Page 35: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/35.jpg)
35
Language UnderstandingGoals include understanding:
– The meaning of the text– How it fits into what is known– The purpose of being told
Requires:– Common sense knowledge– Awareness of context
Our model is to get to knowledge as quickly as possible, with as few intermediate steps as possible.
![Page 36: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/36.jpg)
36
![Page 37: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/37.jpg)
37
Reader ExampleAn attack occurred in MadridThe bombing killed 190 peopleThe bombing was perpetrated by Al-Qaida
(eventOccursAt TerroristAttack-September-8-2003-Madrid CityOfMadridSpain)(perpetrator TerroristAttack-September-8-2003-Madrid AlQaida)(deathToll TerroristAttack-September-8-2003-Madrid Person 190)(isa TerroristAttack-September-8-2003-Madrid Bombing)
![Page 38: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/38.jpg)
38
Lexical Processing
“An attack occurred in Madrid.”• “attack”(singular Attack-TheWord “attack”)(denotation Attack-TheWord CountNoun 0
AttackOnObject)(isa ?x AttackOnObject)
• “Madrid”(placeName-Standard CityOfMadridSpain
"Madrid")(isa CityOfMadridSpain City)
![Page 39: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/39.jpg)
39
Rule Pattern
• Pattern:(isa ?event Event)
Occur-TheWord
In-TheWord
(isa ?location GeographicLocation)
• Results:(eventOccursAt ?event ?
location)
![Page 40: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/40.jpg)
40
Pattern Matching• Pattern:
(isa ?event Event) Occur-TheWordIn-TheWord (isa ?location GeographicLocation)
• Input:– “An”– “attack” (isa ?x AttackOnObject)
– “occurred” Occur-TheWord
– “in” In-TheWord– “Madrid” (isa CityOfMadridSpain City)
![Page 41: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/41.jpg)
41
Rule Completion“An attack occurred in Madrid.”• Pattern:
(isa ?event Event) Occur-TheWordIn-TheWord (isa ?location GeographicLocation)
• Results:(eventOccursAt ?event ?location)
• Constraints:(isa ?event AttackOnObject)
• Bindings from Reading:((?location . CityOfMadridSpain))
![Page 42: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/42.jpg)
42
Remindings“An attack occurred in Madrid.”• Pattern: (isa ?event Event)
Occur-TheWordIn-TheWord (isa ?location GeographicLocation)
• Results:(eventOccursAt ?event ?location)
• Constraints:(isa ?event AttackOnObject)
• Bindings from Reading:((?location . CityOfMadridSpain))
• Remindings from Memory:((?event . TerrorAttack-Sept8-2003-Madrid))
![Page 43: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/43.jpg)
43
Coreference ResolutionAn attack occurred in Madrid. The bombing was
perpetrated by Al-Qaida.
![Page 44: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/44.jpg)
44
Coreference ResolutionAn attack occurred in Madrid
Results: (eventOccursAt?event?location)
Constraints:(isa ?event AttackOnObject)
Bindings from Reading:((?location .
CityOfMadridSpain))Remindings from Memory:((?event . TerrorAttack-
Sept8-2003-Madrid))
The bombing was perpetrated by Al-Qaida
Results: (perpetrator?action
?agent)
Constraints:(isa ?action Bombing)
Bindings from Reading:((?agent . AlQaida))
![Page 45: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/45.jpg)
45
Coreference ResolutionAn attack occurred in Madrid
Results: (eventOccursAt?event?location)
Constraints:(isa ?event AttackOnObject)
Bindings from Reading:((?location .
CityOfMadridSpain))Remindings from Memory:((?event . TerrorAttack-
Sept8-2003-Madrid))
The bombing was perpetrated by Al-Qaida
Results: (perpetrator?action
?agent)
Constraints:(isa ?action Bombing)
Bindings from Reading:((?agent . AlQaida))
![Page 46: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/46.jpg)
46
Coreference ResolutionAn attack occurred in Madrid
Results: (eventOccursAt?event?location)
Constraints:(isa ?event AttackOnObject)
Bindings from Reading:((?location .
CityOfMadridSpain))Remindings from Memory:((?event . TerrorAttack-
Sept8-2003-Madrid))
The bombing was perpetrated by Al-Qaida
Results: (perpetrator?action
?agent)
Constraints:(isa ?action Bombing)
Bindings from Reading:((?agent . AlQaida))
![Page 47: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/47.jpg)
47
Coreference ResolutionAn attack occurred in Madrid
Results: (eventOccursAt?event?location)
Constraints:(isa ?event AttackOnObject)
Bindings from Reading:((?location .
CityOfMadridSpain))Remindings from Memory:((?event . TerrorAttack-
Sept8-2003-Madrid))
The bombing was perpetrated by Al-Qaida
Results: (perpetrator?action
?agent)
Constraints:(isa ?action Bombing)
Bindings from Reading:((?agent . AlQaida))
![Page 48: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/48.jpg)
48
Coreference ResolutionAn attack occurred in Madrid
Results: (eventOccursAt?event?location)
Constraints:(isa ?event AttackOnObject)
Bindings from Reading:((?location .
CityOfMadridSpain))Remindings from Memory:((?event . TerrorAttack-
Sept8-2003-Madrid))
The bombing was perpetrated by Al-Qaida
Results: (perpetrator?action
?agent)
Constraints:(isa ?action Bombing)
Bindings from Reading:((?agent . AlQaida))
![Page 49: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/49.jpg)
49
Coreference
• Refer to a more general or specific type– “bombing” and “attack”
• Consistent with being the same,
have a known shared instance in memory– “doctor” and “father of four”
![Page 50: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/50.jpg)
50
Tracking Ambiguity
• Words are ambiguous– “Bush”
• A shrubbery?• A president? Which one?
• Sentences and Phrases– “take her flowers”– “Iraq borders Iran on the North.”
• Intension– “Down?”– “Where is Baghdad?”
![Page 51: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/51.jpg)
51
% of sentences processed under time
0
10
20
30
40
50
60
70
80
90
100
1 10 100 1000 10000 100000 1E+06 1E+07
time (ms)
% o
f co
rpu
s se
nte
nce
s p
roce
ssed
RDE
SLE
WLE
![Page 52: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/52.jpg)
52
What Learning Reader can Read
• Information about existing instances– “the 2003 Madrid bombing”
• Information extending existing instances– “the attack killed 190 people”
• Information representing new instances– “An attack occurred in Al Anbar.”
![Page 53: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/53.jpg)
53
What Learning Reader can’t Read
• Larger patterns– Scripts: “A bomb went off.”
“6 people were arrested.”
• Generalizations“All countries have political leaders.”
“The heart is a pump.”
![Page 54: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/54.jpg)
54
Dr. Christopher K. RiesbeckDr. Ken ForbusDr. Larry BirnbaumAbhishek Sharma
Dr. Jennifer SeitzerDr. Saverio Perugini
grant HR0011-04-1-0051
Acknowledgements
![Page 55: Building a System that can Learn by Reading](https://reader036.vdocuments.mx/reader036/viewer/2022062803/5681478c550346895db4bb40/html5/thumbnails/55.jpg)
55
For more Information
http://cs.northwestern.edu/~livingston/http://cs.northwestern.edu/~livingston/talks.html
Kevin LivingstonPhD Candidate
Cognitive Systems DivisionEECS Department
Northwestern University