MEANING: a Roadmap to MEANING: a Roadmap to Knowledge TechnologiesKnowledge Technologies

German Rigau. TALP Research Center. UPC. Barcelona. rigau@lsi.upc.esGerman Rigau. TALP Research Center. UPC. Barcelona. rigau@lsi.upc.esBernardo Magnini. ITC-IRST. Povo-Trento. magnini@itc.itBernardo Magnini. ITC-IRST. Povo-Trento. magnini@itc.it

Eneko Agirre. IXA group. EHU. Donostia. eneko@si.ehu.esEneko Agirre. IXA group. EHU. Donostia. eneko@si.ehu.es Piek Vossen. Irion Technologies. Delft. Piek.Vossen@irion.nlPiek Vossen. Irion Technologies. Delft. Piek.Vossen@irion.nl

John Carroll. COGS. U. Sussex. Brighton. johnca@cogs.susx.ac.ukJohn Carroll. COGS. U. Sussex. Brighton. johnca@cogs.susx.ac.uk

http://www.lsi.upc.es/~nlp/meaning/meaning.htmlhttp://www.lsi.upc.es/~nlp/meaning/meaning.html

MeaningMeaning

MeaningMeaningIntroductionIntroduction

Knowledge technologiesKnowledge technologies (semantic (semantic web): web): make sense of petabytes of informationmake sense of petabytes of information

Range of techniques to Range of techniques to automate automate knowledge lifecycleknowledge lifecycle– Lexical KB (ontologies)Lexical KB (ontologies)– Text understanding (IE or other)Text understanding (IE or other)extract high-level meaningextract high-level meaningrepresent and manage in a KBrepresent and manage in a KB

HLTHLT to enable knowledge technologies to enable knowledge technologies

MeaningMeaningIntroductionIntroduction

Building large and rich KBBuilding large and rich KB by hand: by hand:– ExpensiveExpensive

E.g. CYC, WordNet (EuroWordNet)E.g. CYC, WordNet (EuroWordNet)– Introspection fails to reflect reality in texts, Introspection fails to reflect reality in texts,

domainsdomainsIs a “saint” an animate being? not always, image.Is a “saint” an animate being? not always, image.

– ContradictionsContradictions Hamper applications of HLT and KTHamper applications of HLT and KT

Richer KBs (ontologies)Richer KBs (ontologies)– Domain knowledgeDomain knowledge– Contradictory subsetsContradictory subsets Semi-automatic meansSemi-automatic means

MeaningMeaningIntroductionIntroduction

Crucial intermediate tasksCrucial intermediate tasks Word Sense DisambiguationWord Sense Disambiguation

From words to concepts From words to concepts (word sense (word sense≈concept in KB)≈concept in KB)

Large scale enrichment of Large scale enrichment of (multilingual) Lexical KB(multilingual) Lexical KB Enable semantic processing Enable semantic processing

GoalGoalLarge-scale extraction of shallow meaning: Large-scale extraction of shallow meaning:

relations among conceptsrelations among concepts

MeaningMeaningShallow semanticsShallow semantics

((Chirac) (invita) (al Dalai_Lama) (a un almuerzo oficial)Chirac) (invita) (al Dalai_Lama) (a un almuerzo oficial)

Invite Invite s456s456 objectobjectsourcesource destinatiodestinatio

nn

actact

s378s378 s412s412 s933s933

((Chirac) (invites) (the Dalai_Lama) (to an official lunch)Chirac) (invites) (the Dalai_Lama) (to an official lunch)

MeaningMeaningIntroductionIntroduction

Crucial intermediate tasksCrucial intermediate tasks Word Sense Disambiguation Word Sense Disambiguation Large scale enrichment of (multilingual) Lexical KBLarge scale enrichment of (multilingual) Lexical KB

Problems (research goals)Problems (research goals):: Enriching LKBs, acquisition of linguistic knowledge:Enriching LKBs, acquisition of linguistic knowledge:

– Corpora need to be accurately tagged with conceptsCorpora need to be accurately tagged with concepts Accurate WSD needs: Accurate WSD needs:

– Hand-tagged data OR richer LKBHand-tagged data OR richer LKB Multilinguality:Multilinguality:

– Words in several languages linked to common concepts Words in several languages linked to common concepts

MeaningMeaningOutlineOutline

Major research goalsMajor research goals– Knowledge acquisition into LKBsKnowledge acquisition into LKBs– WSD into LKB conceptsWSD into LKB concepts– MultilingualismMultilingualism

Meaning roadmapMeaning roadmap Overview of the projectOverview of the project

MeaningMeaningKnowledge acquisition into LKBsKnowledge acquisition into LKBs

Semi-automatic acquisition of linguistic Semi-automatic acquisition of linguistic knowledge from corpora is workingknowledge from corpora is working– Subcategorization informationSubcategorization information– Selectional preferencesSelectional preferences– Thematic role assignmentsThematic role assignments– Diathesis alternationsDiathesis alternations– Domain informationDomain information– Topic signaturesTopic signatures– Rich lexico-semantic relations between words Rich lexico-semantic relations between words

(dictionaries)(dictionaries)– ……

Large bodies of text with (fast) shallow processorsLarge bodies of text with (fast) shallow processors

MeaningMeaningKnowledge acquisition into LKBsKnowledge acquisition into LKBs

Knowledge for words is not enough: Knowledge for words is not enough: – Verb senses have different selectional Verb senses have different selectional

preferences for e.g. the subjectpreferences for e.g. the subject The car ate all the petrol (WN)The car ate all the petrol (WN)

– Verb senses may have different subcat. framesVerb senses may have different subcat. frames– ……

Better to key into word senses: Better to key into word senses: source corpora should be tagged source corpora should be tagged – Better reflect linguistic phenomenaBetter reflect linguistic phenomena– Detect new sensesDetect new senses– Clustering sensesClustering senses– Integrate easily into the multilingual LKBIntegrate easily into the multilingual LKB

MeaningMeaningWSD into LKB conceptsWSD into LKB concepts

Senseval-2 uses word senses (concepts) from WN Senseval-2 uses word senses (concepts) from WN 1.71.7

No large-scale broad-coverage WSD system is No large-scale broad-coverage WSD system is availableavailable

Accuracy around 60%-70% (V/A/N) when hand-Accuracy around 60%-70% (V/A/N) when hand-tagged data availabletagged data available– Use hand-tagged data to train ML systemsUse hand-tagged data to train ML systems

Ng’s estimate: 16 persons/year (short)Ng’s estimate: 16 persons/year (short) Promising research linesPromising research lines

– Automatically create training corpus using semantic Automatically create training corpus using semantic relations in the LKB (WN)relations in the LKB (WN)

– Use untagged data to improve performanceUse untagged data to improve performance– Higher precision if more knowledgeable features are used Higher precision if more knowledgeable features are used

(subcat, sel. preferences, domains)(subcat, sel. preferences, domains)– Coarse grained: Domain tagging / Clusters of sensesCoarse grained: Domain tagging / Clusters of senses

MeaningMeaningExploiting EWN Semantic Relations Exploiting EWN Semantic Relations WSDWSD

<evento social><evento social>

<competición, concurso><competición, concurso>

<evento><evento>

<<partido_1partido_1>>

<semifinal><semifinal><cuartos_de_final><cuartos_de_final>

<grupo_social><grupo_social>

<organización><organización>

<agrupación grupo colectivo><agrupación grupo colectivo>

<<partido_2partido_2, , partido_político>partido_político>

<partido_laborista><partido_laborista>

MeaningMeaningExploiting EWN Semantic RelationsExploiting EWN Semantic Relations

partido 1partido 1

Pero España puso al Pero España puso al partidopartido intensidad, ritmo y coraje. intensidad, ritmo y coraje.El seleccionador cree que el El seleccionador cree que el partidopartido de hoy contra Italia dará de hoy contra Italia dará la medida de Españala medida de EspañaEl Racing no gana en su campo desde hace seis El Racing no gana en su campo desde hace seis partidospartidos..

partido 2partido 2

Todos los Todos los partidospartidos piden reformas legales para TV3. piden reformas legales para TV3.La derecha planea agruparse en un La derecha planea agruparse en un partidopartido..El diputado reiteró que ni él ni UDC, “como El diputado reiteró que ni él ni UDC, “como partidopartido”, han ”, han recibido dinero de Pellerols.recibido dinero de Pellerols.

MeaningMeaningExploiting EWN Semantic RelationsExploiting EWN Semantic Relations

partido 1partido 1

Rivera pide el soporte de la afición para encarrilar las Rivera pide el soporte de la afición para encarrilar las semifinalessemifinales..Sólo el equipo de Valero Ribera puede sentenciar una Sólo el equipo de Valero Ribera puede sentenciar una semifinalsemifinal como lo hizo ayer en un Palau Blaugrana como lo hizo ayer en un Palau Blaugrana completamente entregado.completamente entregado.El Racing ganó los El Racing ganó los cuartos de finalcuartos de final en su campo. en su campo.

partido 2partido 2

No negociaremos nunca com un No negociaremos nunca com un partido políticopartido político que sea que sea partidario de la independencia de Taiwan.partidario de la independencia de Taiwan.Una vez más es noticia la desviación de fondos destinados a Una vez más es noticia la desviación de fondos destinados a la formación ocupacional hacia la financiación de un la formación ocupacional hacia la financiación de un partido partido políticopolítico..Estas lleyes fueron votadas gracias a un consenso general de Estas lleyes fueron votadas gracias a un consenso general de los los partidos políticospartidos políticos..

MeaningMeaningMultilingualismMultilingualism

Language diversity is a barrierLanguage diversity is a barrier Language diversity is helpfulLanguage diversity is helpful

– Languages realize meaning in different waysLanguages realize meaning in different ways Use EuroWN multilingual architecture: Interlingual Use EuroWN multilingual architecture: Interlingual

Index (ILI) links translation equivalents via Index (ILI) links translation equivalents via interlingual concepts:interlingual concepts:– head ---------- s984574 --------- cabezahead ---------- s984574 --------- cabeza– -------- s984557 --------- jefe-------- s984557 --------- jefe

Research on how linguistic knowledge behaves Research on how linguistic knowledge behaves when ported to other language (e.g.subcat when ported to other language (e.g.subcat information)information)– Very important for resource-poor languagesVery important for resource-poor languages

MeaningMeaningMultilingualismMultilingualism

Selectional preference for the object of the Selectional preference for the object of the first sense of know:first sense of know:

sense 1: know, cognize -- (be cognizant or aware of a fact or a specific piece of sense 1: know, cognize -- (be cognizant or aware of a fact or a specific piece of information; possess knowledge or information about;information; possess knowledge or information about;

0,11280,1128 <communication><communication>0,06150,0615 <measure quantity amount quantum><measure quantity amount quantum>0,05350,0535 <attribute><attribute>0,03890,0389 <object physical_object><object physical_object>0,03070,0307 <cognition knowledge><cognition knowledge>

In EuroWordNet (In EuroWordNet (http://ixa.si.ehu.eshttp://ixa.si.ehu.es) ) – antzeman_1, jakin_2 and ezagutu_1 in Basque. antzeman_1, jakin_2 and ezagutu_1 in Basque. – conocer_1 and saber_1 in Spanish conocer_1 and saber_1 in Spanish – conèixer_1 and saber_1 in Catalanconèixer_1 and saber_1 in Catalan

MeaningMeaningMEANING roadmapMEANING roadmap Solutions have been tried with relative success in Solutions have been tried with relative success in

isolationisolation Combination for significant advances (which?)Combination for significant advances (which?) Web as corpus: BNC (100 Mw) small for many Web as corpus: BNC (100 Mw) small for many

phenomenaphenomena Incremental design:Incremental design:

a)a) WSD using whatever knowledge available at the time for WSD using whatever knowledge available at the time for bootstrappingbootstrapping

b)b) Acquisition of linguistic knowledge using WSD available at Acquisition of linguistic knowledge using WSD available at the time (may discard low accuracy examples)the time (may discard low accuracy examples)

c)c) Integrating acquired knowledge in the Multilingual Central Integrating acquired knowledge in the Multilingual Central Repository and porting knowledge from one language to the Repository and porting knowledge from one language to the otherother

Series of cycles: Series of cycles: WSD0, WSD1, WSD2, ACQ0, ACQ1, ACQ2, PORT0, PORT1, PORT2WSD0, WSD1, WSD2, ACQ0, ACQ1, ACQ2, PORT0, PORT1, PORT2

MultilingualMultilingualCentral RepositoryCentral Repository

ItalianItalianEWNEWN

BasqueBasqueEWNEWN

SpanishSpanishEWNEWN

EnglishEnglishEWNEWN

BasqueWeb Corpus

ItalianWeb Corpus

EnglishWeb Corpus

CatalanCatalanEWNEWN

SpanishWeb Corpus

CatalanWeb Corpus

ACQACQ

ACQACQACQACQ

ACQACQ

UPLOADUPLOADUPLOADUPLOAD

UPLOADUPLOADUPLOADUPLOAD

PORTPORT

PORTPORT

PORTPORT

PORTPORT

WSDWSD

WSDWSD

WSDWSD

WSDWSD

MeaningMeaningArchitectureArchitecture

3 years 3 years researchresearch project (started march project (started march 2002)2002)

1.610 M Euro1.610 M Euro– 2 contracted people per site2 contracted people per site

ConsortiumConsortium– TALP, UPC (German Rigau)TALP, UPC (German Rigau)– ITC-IRST (Bernardo Magnini)ITC-IRST (Bernardo Magnini)– IXA, UPV/EHU (Eneko Agirre)IXA, UPV/EHU (Eneko Agirre)– University of Sussex (John Carroll)University of Sussex (John Carroll)– Irion Technologies (Piek Vossen)Irion Technologies (Piek Vossen)

MeaningMeaningProject overviewProject overview

MeaningMeaning Project resultsProject results

A Tool Set that using the semantic knowledge of EWN will obtain A Tool Set that using the semantic knowledge of EWN will obtain automatically from the web large collections of examples for each automatically from the web large collections of examples for each particular word sense. particular word sense.

A Tool Set for enriching EWN using the knowledge acquired A Tool Set for enriching EWN using the knowledge acquired automatically from the Web. automatically from the Web.

A Tool Set for selecting accurately the senses of the open-class A Tool Set for selecting accurately the senses of the open-class words for the languages involved in the project.words for the languages involved in the project.

Multilingual Central Repository to maintain compatibility between Multilingual Central Repository to maintain compatibility between WordNets of different languages and versions, past and new. WordNets of different languages and versions, past and new.

A semantically annotated corpus for each WordNet word sense, A semantically annotated corpus for each WordNet word sense, that is, a multilingual web corpus with semantically annotated that is, a multilingual web corpus with semantically annotated corporcorporaa

Demonstration: CLIR, Q/A system.Demonstration: CLIR, Q/A system.

The results of MEANING will be public and free for research.

MeaningMeaningWhy now?Why now? Huge amounts of data: throw out non Huge amounts of data: throw out non

reliable reliable Syntactic dependencies with high enough Syntactic dependencies with high enough

accuracyaccuracy Supervised WSD with high enough accuracySupervised WSD with high enough accuracy

– Coarser grains, sense domain taggingCoarser grains, sense domain tagging– BootstrappingBootstrapping

Success coping with multilingualism:Success coping with multilingualism:– Porting linguistic knowledge from one Porting linguistic knowledge from one

language to other using MT / comparable language to other using MT / comparable corporacorpora

– CLIR as good as monolingual IRCLIR as good as monolingual IR

MEANING: a Roadmap to MEANING: a Roadmap to Knowledge TechnologiesKnowledge Technologies

German Rigau. TALP Research Center. UPC. Barcelona. rigau@lsi.upc.esGerman Rigau. TALP Research Center. UPC. Barcelona. rigau@lsi.upc.esBernardo Magnini. ITC-IRST. Povo-Trento. magnini@itc.itBernardo Magnini. ITC-IRST. Povo-Trento. magnini@itc.it

Eneko Agirre. IXA group. EHU. Donostia. eneko@si.ehu.esEneko Agirre. IXA group. EHU. Donostia. eneko@si.ehu.es Piek Vossen. Irion Technologies. Delft. Piek.Vossen@irion.nlPiek Vossen. Irion Technologies. Delft. Piek.Vossen@irion.nl

John Carroll. COGS. U. Sussex. Brighton. johnca@cogs.susx.ac.ukJohn Carroll. COGS. U. Sussex. Brighton. johnca@cogs.susx.ac.uk

http://www.lsi.upc.es/~nlp/meaning/meaning.htmlhttp://www.lsi.upc.es/~nlp/meaning/meaning.html

MeaningMeaning