research article improving causality induction with...

Research ArticleImproving Causality Induction with Category Learning

Yi Guo12 Zhihong Wang1 and Zhiqing Shao1

1 Department of Computer Science and Engineering East China University of Science and TechnologyPO Box 408 Shanghai 200237 China

2 School of Information Science and Technology Shihezi University Shihezi 832003 China

Correspondence should be addressed to Yi Guo yguo71625foxmailcom

Received 7 April 2014 Accepted 14 April 2014 Published 30 April 2014

Academic Editor Shan Zhao

Copyright copy 2014 Yi Guo et al This is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Causal relations are of fundamental importance for human perception and reasoning According to the nature of causality causalityhas explicit and implicit forms In the case of explicit form causal-effect relations exist at either clausal or discourse levels Theimplicit causal-effect relations heavily rely on empirical analysis and evidence accumulationThis paper proposes a comprehensivecausality extraction system (CL-CIS) integrated with the means of category-learning CL-CIS considers cause-effect relations inboth explicit and implicit forms and especially practices the relation between category and causality in computation In elaboratelydesigned experiments CL-CIS is evaluated together with general causality analysis system (GCAS) and general causality analysissystemwith learning (GCAS-L) and it testified to its own capability and performance in construction of cause-effect relationsThispaper confirms the expectation that the precision and coverage of causality induction can be remarkably improved by means ofcausal and category learning

1 Introduction

A general philosophical definition of causality states thatthe philosophical concept of causality refers to the set of allparticular causal or cause-and-effect relations [1] Causal rela-tions are of fundamental importance for human perceptionand reasoning Since ignoring causal relationships may havefatal consequences their knowledge plays a crucial role indaily life to ensure survival in an ever changing environment

In many research works the causality generally refersto the existence of causality in mathematics and physicsCausalities are often investigated in situations influencedby uncertainty involving several variables Thus causalitieswhich can be presented in terms of flows among processesor events are generally expressed in mathematical languagesand analyzed in a mathematical manner Therefore statisticsand probability theories seem to be the most popular math-ematical languages for modeling causality in most scientificdisciplines There is an extensive range of literature oncausality modeling applying and combining mathematicallogic graph theoryMarkovmodels Bayesian probability andso forth [2] But they seem not to be able to predominate allrelevant issues or questions

In recent years clarification and extraction of cause-effect relationships among texts (eg objects or events)causality extraction is elevated to a prominent researchtopic in text mining knowledge engineering and knowledgemanagement

A variety of research works testify that causalities intexts can be extracted at three technology levels The firstis the clausal level (CL) which includes cue phrases andlexical clues [3] and semantic similarity [4]The second is thediscourse level (DL) which implements connective markers[5] and constructs discourse relations [6] The third is themode level (ML) which extracts causal relations from a QAsystem [7 8] applies commonsense rules [9] associativememory [10] and Chain Event Graphs [11]

According to the nature of causality in texts causalityhas explicit and implicit forms In the case of explicit formcausal-effect relations exist at either clausal or discourselevels The implicit causal-effect relations heavily rely onempirical analysis and evidence accumulation

Moreover Waldmann and Hagmayer [12] in theirresearch work of cognitive psychology state that ldquocategoriesthat have been acquired in previous learning contexts mayinfluence subsequent causal learningrdquo which indicate that

Hindawi Publishing Corporatione Scientific World JournalVolume 2014 Article ID 650147 8 pageshttpdxdoiorg1011552014650147

2 The Scientific World Journal

(1) the category information about objects or events in textsis a necessary supplement for causality extraction and (2)the category information has impact on subsequent learning-based cause-effect identification

Based on the facts the task of causality extraction eveninduction in texts could not be accomplished in an arbitrarymanner This paper proposes a comprehensive causalityextraction system (CL-CIS) integrated with the means ofcategory-learning CL-CIS considers cause-effect relations inboth explicit and implicit forms and especially practices therelation between category and causality in computation

The rest of this paper is organized as follows Section 2states the causality category information and the relation-ships between them in texts and constructs the theoreticalfoundation of our research work Section 3 expatiates themethodology and the technical details of system structureand kernel algorithms Section 4 focuses on experimentillustration and result analysis of the experimental resultsSection 5 concludes this paper and provides future researchworks

2 Causality and Category

Traditionally research about the representation of causalrelations and research about the representation of categorieswere separatedThis research strategy rests on the assumptionthat categories summarize objects or events on the basis oftheir similarity structure whereas causality refers to relationsbetween causal objects or events Literature [12] proves thatthe relationship between causality and categorization is moredynamic than previously thought

21 Causality The standard view guiding research on causal-ity presupposes the existence of objective networks of causesand effects that cognitive systems try to mirror Regardlessof whether causal learning is viewed as the attempt to inducecausality on the basis of statistical information or on the basisof mechanism information it is generally assumed that thegoal of causal learning is to form adequate representations ofthe texture of the causal world

22 Causality Rests on Fixed Categories Studies on causallearning typically investigate trial-by-trial learning taskswhich involve learning the contingencies between causes andeffects In a large number of studies which focus on causalcontingency learning A characteristic feature of these tasksis that they present categorized events representing causesand effects which are statistically related Cause and effectcategories are viewed as fixed entities that are already presentprior to the learning task The goal of learning is to estimatecausal strength of individual causal links or to induce causalmodels on the basis of observed covariations The role ofcause and effect categories in the learning process is not thefocus of interest in these approaches they are simply viewedas given

A similar approach underlies research on the relationshipbetween categories and causality According to the view thatcategorization is theory-based traditional similarity-based

accounts of categorization are deficient because they ignorethe fact that many categories are grounded in knowledgeabout causal structures [13] In natural concepts featuresoften represent causes or effects with the category labelreferring to a complex causalmodel For example disease cat-egories frequently refer to common-cause models of diseaseswith the category features representing causes (eg virus) andeffects (eg symptoms) within this causal model A numberof studies using these and similar materials have shown thatthe type of causalmodel connecting otherwise identical causeand effect features influences learning typicality judgmentsor generalization [14ndash17] The main goal of these studieswas to investigate the effect of different causal relationsconnecting the causal features As in contingency learningstudies the cause and effect features within the causal modelswere treated as fixed categorized entities which alreadyexisted prior to the learning context

23 Categories Shape Causality It is certainly true that manyinteresting insights can be gained from investigating howpeople learn about causal models on the basis of preexistingcause and effect categories However there is also a linkbetween categories and causality in the opposite directionThe categories that have been acquired in previous learningcontexts may have a crucial influence on subsequent causallearning

The basis of the potential influence of categories on causalinduction lies in the fact that the acquisition and use of causalknowledge is based on categorized events Causal laws suchas the fact that smoking causes heart disease can only benoticed on the basis of events that are categorized (eg eventsof smoking and cases of heart disease)

Without such categories causal laws neither could bedetected nor could causal knowledge be applied to newcases Thus causal knowledge not only affects the creation ofcategories it also presupposes already existing categories forthe description of causes and effects The potential influenceof categories is due to the fact that one of the most importantcues to causality is statistical covariation between causes andeffects

To study the relation between categories and causalinduction [12] have developed a new paradigm that consistsof three phases the category learning phase the causallearning phase and the third test phase The main goal isto answer the question that under what condition learnerswill tend to activate the categorical information (Figure 1)from the earlier category learning phase when learning aboutcausal contingencies in the later phase This effect is entailedby the fact that the alternative categories form differentreference classes

In category learning phase causes will be classified intothe distinct categories (upper left arrow in Figure 1) Incausal learning phase causes in collection are paired withthe presence or absence of an effect (lower arrow) In thesubsequent test phase the test causes are rated with thelikelihood of the effect The crucial question is when peoplewould go through the upper route in Figure 1 and assign thetest causes to the categories in the category learning phase or

The Scientific World Journal 3

Categories

EffectsCauses

Figure 1 Possible routes of category learning between causes andeffects

Categories

EffectsCausesCauses

Category learning

learning

Figure 2 Enhanced strategy of category learning between causesand effects

whether they would stick to the lower route and induce newcategories within causal learning phase

If the learning strategy opted for the lower route onepossible solution may be to induce new categories that aremaximally predictive of the effects This solution wouldobviously generate maximally predictive categories Lien andCheng [18] reported a research that is consistent with themaximal-contrast hypothesis Their experiments show thatthe substances are categorized according to the feature and tothe hierarchical level that were maximally predictive for theeffect Thus the induced substance category was determinedby its suitability for predicting the effect Lien and Cheng[18] interpreted this as evidence for their maximal-contrasthypothesis In another word people tend to induce categoriesthat maximize their causal predictability

In sum our research addresses the question of whichroute learners will go Will they routinely go through theupper road and activate category knowledge when learningabout novel effects A simple connectionist one-layer net-work that may be used to understand the task we are going toexplore in our experiments or will they go the lower road andlearn a new set of categories on the basis of causal informationin causal learning phase that is maximally predictive withinthis learning phase Thus Figure 1 is enhanced with twolearning phases as shown in Figure 2

3 Research Methodology andSystem Structure

31 Research Methodology Causality is a fundamental con-cept in human thinking and reasoning It is not surprisingthat most if not all languages in the world have a range of

Table 1 Statistics for backward causality connectives

Connectives For objectivereason

For subjectivereason

Total percentage inall connectives

Because 83 17 50For 29 71 18As 60 40 13Since 14 86 6While 14 86 6Misc 0 100 7Total 56 44 100

lexical expressions specifically designed for communicatingcausal relations

This paper focuses on explicit causality markers andimplicit causal relations in text The explicit causality mark-ers include two grammatically different types of causalitymarkers in English We investigate the semantic contrastsexpressed by different causal auxiliary verbs marking causalrelations expressed within one clause and those expressedby different causal connectives marking causal relationsbetween clauses

311 Causality in Verbs Some instances of causality in verbsare listed below These verbs include but are not limitedto ldquomakerdquo ldquoletrdquo ldquohaverdquo ldquocauserdquo and their synonyms fromWordNet [19] which is generally referred to as an onlinelexical database

[The extreme cold] cause madecaused even [the rivers(to) freeze] effect[She] cause madehad [her son empty his plate] effectdespite his complaints

312 Causality in Discourse Connectives For a complexsentence the predicative or relative clauses of the nounsynonyms of ldquocauserdquo are labeled as ldquopotential causerdquo ofantecedent sentence or main clause Additionally each clauseinducted with ldquosincerdquo ldquoasrdquo or ldquobecauserdquo is also labeled asldquopotential causerdquo of its main clause which is labeled asldquopotential effectrdquo

Table 1 lists the statistics for above backward causalitycollected upon Reuters-21578 currently the most widelyused test collection for text processing research and BBC-News2000 collected by a self-developed Web Crawler fromhttpwwwbbccouk The forward causality where in pre-sentation order the cause precedes the effect is the mostfrequently used ones for example ldquothereforerdquo ldquobecause ofthatrdquo ldquothat is whyrdquo and ldquosordquo As the forward causality con-nectives are one hundred percent strong causality indicatorsthe backward causality connectives in Table 1 need morespecific notation For example in all ldquobecauserdquo connectivesin collections 83 of them indicate objective reasons whilethe other 17 indicate subjective reasons Meanwhile theldquobecauserdquo connectives persist 50 of all the connectivesincluding ldquobecauserdquo ldquoforrdquo ldquoasrdquo ldquosincerdquo ldquowhilerdquo and othermiscellaneous connectivesThe ldquoforrdquo ldquoasrdquo ldquosincerdquo ldquowhilerdquo and


Table 2 Types of implicit causality sentences

Types Subtypes Exemplar sentences

Compoundsentences

Cause-effect sentencesconnected with ldquoandrdquo

Cause-effect(1)This was the first time I made an international call and my heart was beating fast(2) The filter is much more efficient than the primary filter and it removes allremaining solid particles from the fuel

Effect-cause(3) The crops had failed and there had not been any rain for months(4) Aluminum is used as engineering material for planes and spaceships and it islight and tough

Cause-effect sentenceswithout connectives

Effect-cause(5) My heart sank Some of them did not weigh more than 135 pounds and theothers looked too young to be in trouble(6) but the Voyagerrsquos captain and three crewmen had stayed in board They hadhoped the storm would die out and they could save the shipCause-effect(7) The red distress flare shot up from the sinking ship the Voyager Everymanaboard our Coast Guard cutter knew that time had run out

Relative clauses SV SVO SVC SVOC SVOOSVA SVOA

(8) To make an atom we have to use uranium in which the atoms are available forfission(9) We know that a cat whose eyes can take in more rays of light than our eyes cansee clearly in the night

If clauses

(10) This system of subsidies must be maintained if the farmer will sufferconsiderable losses if it is abolished(11) If the water will rise above this level we must warn everybody in theneighborhood

That clauses (12) The act was even the bolder that he stood utterly alone(13) The crying was all the more racking that she was not hysterical but hopeless

SVO-SVOC(14) Her falling ill spoiled everything(15) Timidity and shamefacedness caused her to stand back and looked indifferentlyaway

other miscellaneous connectives correspondingly hold 1813 6 6 and 7 of all the connectives in collections

313 Implicit Causality in Texts Causal effect relations aregeneral connections in the objective world and the causalitysentences exist in languages in a pervasive manner Theexplicit causality sentences are those conducted by backwardand forward causality connectives listed in Section 312while the implicit causality sentences are those connectedwith other connectives even without any one Our researchworks have testified that there are five types of implicitcausality sentences shown in Table 2

In the practical English texts there exist a few inter-personal verbs like ldquopraiserdquo and ldquoapologizerdquo thus supplyinginformation about whose behavior or state is the more likelyimmediate cause of the event at hand Because it is conveyedimplicitly as part of themeaning of the verb this probabilisticcue is usually referred to as implicit causality

Such exemplar implicit causality verbs [20] adopted inthis paper are listed in Table 3 together with their biasindicating the probabilities as causal cues for example 100is the causal baseline the higher the bias value is the morelikely is the cause

32 System Description Our causality induction system withassistance of category learning named CL-CIS is composedwith the following functional modules (shown in Figure 3)category learning classify exemplars into categories categoryand causal mapping causal learning building causal-effectrelations and the final testing module CL-CIS also includesthree reference libraries (databases) causality in verbscausality in discourse connectives and implicit causalityfor query and assistance Table 4 compares the compositiondifference among general causality analysis system (GCAS)general causality analysis system with learning (GCAS-L)and CL-CIS As the construction of three libraries is elabo-rated in Section 31 Methodology this section concentrateson the six functional modules

321 Category Learning (CL) The category learning modulebuilds up different distinct and exhaustive categories accord-ing to semantic contents (eg noun phrases verb phrases)of sentences in our text collections This module is a basicpreprocessing step and the target is to construct a collectionof distinct and exhaustive categories with the text learningmethods


Table 3 Exemplar implicit causality verbs

NP1 verbs Bias NP2 verbs BiasAmazed 119 Admire 200Annoy 119 Adore 186Apologize 100 Appreciate 200Be in the way 108 Comfort 191Beg 117 Compliment on something 196Bore 105 Congratulate 195Call 119 Criticize 200Confess 103 Envy 200Disappoint 103 Fear 200Disturb 114 Fire 196Fascinate 100 Hate 196Hurt 113 Hold in contempt 200Inspire 123 Hold responsible 191Intimidate 123 Loathe 186Irritate 122 Love 186Lie to 122 Praise 196Mislead 122 Press charges against 191Swindle 113 Punish 195Win 119 Respect 195Worry 119 Thank 182

Table 4 System composition comparison of GCAS GCAS-L andCL-CIS

System composition GCAS GCAS-L CL-CISModules

Category learning radic

Classify exemplars intocategories radic

Category and causal mapping radic

Causal learning radic radic

Building causal-effectrelations radic radic radic

Testing causal-effectrelations radic radic radic

LibrariesCausality in verbs radic radic radic

Causality in discourseconnectives Optional radic radic

Implicit causality Optional radic

322 Classify Exemplars into Categories (CEC) In the clas-sify exemplars into categories module each sentence istreated as an individual independent event and parsed intophrases The classification is based on the comparison ofa sentence with features of each category so as to set upthe category background knowledge for each sentence Forexample the concept bank can be clarified as a financeorganization or a river body boundary with correspondingcategory background knowledge of its sentence and contexts

Category learning

Classify exemplars into

categories

Category and causal

mapping

Causal learning

Building causal-effect

relations

Testing causal-effect

relations

Causality in verbs

Causality in discourse

connectivesImplicit causality

Figure 3 System framework of CL-CIS

Inputlayer (IL)

Recurrentlayer (RL)

Outputlayer (OL)

Contextlayer (CL)

Time delay

OU1 OU2

RU1RU2

WOR

IU1 IU2

WRI W

RC

CU1 CU2

middot middot middot


middot middot middot middot middot middot

OU|O|

RU|R|

IU|I| CU|C|

Figure 4 Architecture of SRNN

323 Category and Causal Mapping (CCM) This moduleconstructs the category-causal-effect mapping relations forexample a virus from a predefined category causes specificdisease-related symptoms such as a swelling of the spleen(splenomegaly) The construction mechanism to a greatextent is based on collection storage and indexing ofmassive cases

324 Causal Learning (CauL) As analyzed before a sim-ple connectionist one-layer network that may be used tounderstand the task of when to activate category knowledgewhen learning about novel effects This module adopts asimple recurrent neural network (SRNN) to simulate theconnectionist model

Symbols in Figure 4 are defined in Table 5 the first orderweight matrices WRI and WOR fully connect the units ofthe input layer (IL) the recurrent layer (RL) and the outputlayer (OL) respectively as in the feed forward multilayerperceptron (MLP) The current activities of recurrent unitsRU(119905) are fed back through time delay connections to thecontext layer which is presented as CU(119905+1) = RU(119905)

Therefore each unit in recurrent layer is fed by activitiesof all recurrent units from previous time step through


Table 5 Definition of SRNN Symbols

Symbols DefinitionIU A unit of input layerRU A unit of recurrent layerCU A unit of context layerOU A unit of output layer|119868| The number of units in IL|119877| The number of units in RL|119862| The number of units in CL|119874| The number of units in OLWRI The weight vector from IL to RLWRC The weight vector from CL to RLWOR The weight vector from RL to OL

recurrent weight matrix WRC The context layer which iscomposed of activities of recurrent units from previous timestep can be viewed as an extension of input layer to therecurrent layer Above working procedure represents thememory of the network via holding contextual informationfrom previous time steps

The weight matrices 119882RI 119882RC and 119882OR are presentedas follows

119882RI= [(119908

RI1 )119879 (119908

RI2 )119879 (119908

RI|119877|)119879]

=

[[[[[

[

119908ri11 119908

ri12 sdot sdot sdot 119908

ri1|119877|

119908ri21 119908ri22 sdot sdot sdot 119908

ri1|119877|

d

119908ri|119868|1 119908

ri|119868|2 sdot sdot sdot 119908

ri|119868||119877|

]]]]]

]

119882RC= [(119908

RC1 )119879 (119908

RC2 )119879 (119908

RC|119877| )119879]

=

[[[[

[

119908rc11 119908rc12 sdot sdot sdot 119908

rc1|119877|


rc1|119877|

d

119908rc|119862|1 119908

rc|119862|2 sdot sdot sdot 119908

rc|119862||119877|

]]]]

]

119882OR= [(119908

OR1 )119879 (119908

OR2 )119879 (119908

OR|119874| )119879]

=

[[[[

[

119908or11 119908or12 sdot sdot sdot 119908

or1|119874|


or1|119874|

d

119908or|119877|1 119908

or|119877|2 sdot sdot sdot 119908

or|119877||119874|

]]]]

]

(1)

In the above formulations (119908RI119896 )119879 is the transpose of 119908RI119896

for the instance of119882RI where 119908RI119896 is a row vector and (119908RI119896 )119879

is the column vector of the same elements The vector 119908RI119896 =(119908ri1119896 119908

ri2119896 119908

ri|119868|119896) represents the weights from all the input

layer units to the recurrent (hidden) layer unit RU119896The sameconclusion applies with119882RC and119882OR

Given an input pattern in time t IU(t) = (IU(119905)1 IU(119905)2

IU(119905)|119868|) and recurrent activities RU(t) = (RU(119905)1 RU

(119905)2

RU(119905)|119877|) for the 119894th recurrent unit the net input RU1015840(119905)

119894and

output activity RU(119905)119894

are calculated as follows

RU1015840(119905)119894 = IU(t)sdot (119908

RI119894 )119879+ RU(tminus1) sdot (119908RC119894 )

119879

=

|119868|

sum119895=1

IU(119905)119895 119908ri119895119894 +

|119877|

sum119895=1

RU(119905minus1)119895 119908rc119895119894

RU(119905)119894 = 119891 (RU1015840(119905)

119894 )

(2)

For the kth output unit its net inputOU1015840(119905)119896

and output activityOU(119905)119896

are calculated as (3) Consider the following

OU1015840(119905)119896= RU(t) sdot (119908OR119896 )

119879=

|119877|

sum119895=1

RU(119905)119895 119908or119895119896

OU(119905)119896= 119891 (OU1015840(119905)

119896)

(3)

Here the activation function 119891 applies the logistic sigmoidfunction (4) in this paper Consider the following

119891 (119909) =1

1 + 119890minus119909=119890119909

1 + 119890119909 (4)

325 Building Causal-Effect Relations (BCER) This modulebuilds up the ldquocause-effectrdquo pairs and connections with theinputs of SRNN and corresponding outputs The causalitydetection results could include four possible types (1) singlecause-effect pairs in which any two pairs are independentfrom each other (2) cause-effect chains which are formedwithmore than one cause-effect pairs connected together (aneffect is a cause of another effect) (3) one-cause-multiple-effect pairs (4) multiple-cause-one-effect pairs All of thecausality connections are archived in a database Both CauLandBCERmodules exploit three libraries (databases) causal-ity in verbs causality in discourse connectives and implicitcausality for reference

326 Testing Causal-Effect Relations (TCER) This moduletests and verifies a new processed causal-effect relation withour existing and expanding collection ofmassive causal-effectrelations The concrete technology includes comparison oftriples ⟨CategoryCauseEffect⟩

4 Experiments and Results

Our experiments test general causality analysis system(GCAS) general causality analysis system with learning(GCAS-L) and CL-CIS together in order to examine andreveal the assertion that category information is a necessarysupplement for causality extraction and has impact on subse-quent learning-based cause-effect identification

In our experiments we have used two text collections (1)Reuters-21578 currently the most widely used test collectionfor text processing research (2) BBC-News2000 collected bya self-developed Web Crawler from httpwwwbbccouk


00

01

02

03

04

05

06

07

08

09

10

Reut2-000sgm Reut2-001sgm Both (reut2) BBC-News2000

0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS

(c) F-measure (balanced)

Figure 5 Evaluation results of GCAS GCAS-L and CL-CIS

BBC-News2000 collected 2000 news articles which havebeen pruned off unnecessary Web page elements such ashtml tags images URLs and so forth BBC-News2000 is notcategorized and is treated as a whole hybrid

30 judges are involved to manually search cause-effectpairs and construct standard causality references (SCR) SCR-Reuters for Reuters-21578 and SCR-BBC for BBC-News2000Due to limited human resources in current stage SCR-Reuters only covers 2000 documents newid ranges from 1 to2000 contained in files ldquoreut2-000sgmrdquo and ldquoreut2-001sgmrdquoThe rest of the files of Reuters-21578 are still involved in thetraining phase

The performances of GCAS GCAS-L and CL-CIS areevaluated with precision recall and F-measure [21] thetraditional measures that have been widely applied by mostinformation retrieval systems to analyze and evaluate theirperformance The F-measure is a harmonic combination ofthe precision and recall values used in information retrievalAs shown in Tables 6 7 and 8 the experimental results statethat (1) GCAS scores from 0681 to 0732 on precision andfrom 0581 to 0706 on recall (2) GCAS-L scores from 0719to 0793 on precision and from 0609 to 0734 on recall (3)CL-CIS scores from 0776 to 0897 on precision and from0661 to 0805 on recall Figure 5 explicitly states that (1)

Table 6 Experimental results of GCAS

Experiment files Precision Recall 119865-measurereut2-000sgm (newid 1ndash1000) 0681 0593 0634reut2-001sgm (newid 1001ndash2000) 0706 0581 0637Both (reut2) (newid 1ndash2000) 0698 0602 0646BBC-News2000 0732 0706 0719

Table 7 Experimental results of GCAS-L


the general causality analysis system with learning (GCAS-L) performances better than the general causality analysissystem (GCAS) on all evaluation measures (2) the causalityanalysis system strengthened with causal and category learn-ing (CL-CIS) exceeds GCAS-L in the meantime


Table 8 Experimental results of CL-CIS


5 Concluding Remarks

In recent years detection and clarification of cause-effectrelationships among texts events or objects has been elevatedto a prominent research topic of natural and social sciencesover the human knowledge development history

This paper demonstrates a novel comprehensive causalityextraction system (CL-CIS) integrated with the means ofcategory-learning CL-CIS considers cause-effect relations inboth explicit and implicit forms and especially practices therelation between category and causality in computation

CL-CIS is inspired with cognitive philosophy in categoryand causality In causality extraction and induction tasks CL-CIS implements a simple recurrent neural network (SRNN)to simulate human associative memory which has the abilityto associate different types of inputs when processing infor-mation

In elaborately designed experimental tasks CL-CIS hasbeen examined in full with two text collections Reuters-21578and BBC-News2000 The experimental results have testifiedthe capability and performance of CL-CIS in constructionof cause-effect relations and also confirmed the expectationthat the means of causal and category learning will improvethe precision and coverage of causality induction in a notablemanner

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgment

This work is financially supported by the National NaturalScience Foundation of China (Grant no 61003126)

References

[1] httpenwikipediaorgwikiCausality[2] J Pearl Causality Models Reasoning and Inference Cambridge

University Press New York NY USA 2000[3] D-S Chang and K-S Choi ldquoIncremental cue phrase learning

and bootstrapping method for causality extraction using cuephrase and word pair probabilitiesrdquo Information Processing andManagement vol 42 no 3 pp 662ndash678 2006

[4] S Kim R H Bracewell and K M Wallace ldquoA frameworkfor automatic causality extraction using semantic similarityrdquoin Proceedings of the ASME International Design EngineeringTechnical Conferences and Computers and Information in Engi-neering Conference (IDETCCIE rsquo07) pp 831ndash840 Las VegasNev USA September 2007

[5] T Inui K Inui andYMatsumoto ldquoAcquiring causal knowledgefrom text using the connective markersrdquo Transactions of theInformation Processing Society of Japan vol 45 no 3 pp 919ndash932 2004

[6] D Marcu and A Echihabi ldquoAn unsupervised approach torecognizing discourse relationsrdquo in Proceedings of the 40thAnnual Meeting of the Association for Computational LinguisticsConference (ACL rsquo02) pp 368ndash375 University of PennsylvaniaPhiladelphia Pa USA 2002

[7] C Pechsiri and A Kawtraku ldquoMining causality from texts forquestion answering systemrdquo IEICE Transactions on Informationand Systems vol 90 no 10 pp 1523ndash1533 2007

[8] R Girju ldquoAutomatic detection of causal relations for questionansweringrdquo in Proceedings of the 41st Annual Meeting of theAssociation for Computational Linguistics 2003

[9] K Torisawa ldquoAutomatic extraction of commonsense inferencerules from corporardquo in Proceedings of the 9th Annual Meeting ofthe Association for Natural Language Processing 2003

[10] Y Guo N Hua and Z Shao ldquoCognitive causality detectionwith associative memory in textual eventsrdquo in Proceedings ofthe IEEE International Symposium on Information Engineeringand Electronic Commerce (IEEC rsquo09) pp 140ndash144 TernopilUkraine May 2009

[11] P Thwaites J Q Smith and E Riccomagno ldquoCausal analysiswith Chain Event GraphsrdquoArtificial Intelligence vol 174 no 12-13 pp 889ndash909 2010

[12] M R Waldmann and Y Hagmayer ldquoCategories and causalitythe neglected directionrdquo Cognitive Psychology vol 53 no 1 pp27ndash58 2006

[13] G L MurphyTheBig Book of Concepts MIT Press CambridgeMass USA 2002

[14] B Rehder ldquoA causal-model theory of conceptual representationand categorizationrdquo Journal of Experimental Psychology Learn-ing Memory and Cognition vol 29 no 6 pp 1141ndash1159 2003

[15] B Rehder ldquoCategorization as casual reasoningrdquo Cognitive Sci-ence vol 27 no 5 pp 709ndash748 2003

[16] B Rehder and R Hastie ldquoCausal knowledge and categoriesthe effects of causal beliefs on categorization induction andsimilarityrdquo Journal of Experimental Psychology General vol 130no 3 pp 323ndash360 2001

[17] B Rehder and R Hastie ldquoCategory coherence and category-based property inductionrdquo Cognition vol 91 no 2 pp 113ndash1532004

[18] Y Lien and PWCheng ldquoDistinguishing genuine from spuriouscauses a coherence hypothesisrdquo Cognitive Psychology vol 40pp 87ndash137 2000

[19] G A Miller R Beckwith C Fellbaum D Gross and K JMiller ldquoIntroduction to WordNet an on-line lexical databaserdquoInternational Journal of Lexicography vol 3 no 4 pp 235ndash3121990

[20] A W Koornneef and J J A Van Berkum ldquoOn the use of verb-based implicit causality in sentence comprehension evidencefrom self-paced reading and eye trackingrdquo Journal of Memoryand Language vol 54 no 4 pp 445ndash465 2006

[21] C D Manning and H Schutze Foundations of StatisticalNatural Language Processing TheMIT Press Cambridge MassUSA 4th edition 2001

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Distributed Sensor Networks


Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014


ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014


Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications


Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia


Biomedical Imaging


ArtificialNeural Systems

Advances in


RoboticsJournal of



Computational Intelligence and Neuroscience

Industrial EngineeringJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in



(1) the category information about objects or events in textsis a necessary supplement for causality extraction and (2)the category information has impact on subsequent learning-based cause-effect identification

Based on the facts the task of causality extraction eveninduction in texts could not be accomplished in an arbitrarymanner This paper proposes a comprehensive causalityextraction system (CL-CIS) integrated with the means ofcategory-learning CL-CIS considers cause-effect relations inboth explicit and implicit forms and especially practices therelation between category and causality in computation

The rest of this paper is organized as follows Section 2states the causality category information and the relation-ships between them in texts and constructs the theoreticalfoundation of our research work Section 3 expatiates themethodology and the technical details of system structureand kernel algorithms Section 4 focuses on experimentillustration and result analysis of the experimental resultsSection 5 concludes this paper and provides future researchworks

2 Causality and Category

Traditionally research about the representation of causalrelations and research about the representation of categorieswere separatedThis research strategy rests on the assumptionthat categories summarize objects or events on the basis oftheir similarity structure whereas causality refers to relationsbetween causal objects or events Literature [12] proves thatthe relationship between causality and categorization is moredynamic than previously thought

21 Causality The standard view guiding research on causal-ity presupposes the existence of objective networks of causesand effects that cognitive systems try to mirror Regardlessof whether causal learning is viewed as the attempt to inducecausality on the basis of statistical information or on the basisof mechanism information it is generally assumed that thegoal of causal learning is to form adequate representations ofthe texture of the causal world

22 Causality Rests on Fixed Categories Studies on causallearning typically investigate trial-by-trial learning taskswhich involve learning the contingencies between causes andeffects In a large number of studies which focus on causalcontingency learning A characteristic feature of these tasksis that they present categorized events representing causesand effects which are statistically related Cause and effectcategories are viewed as fixed entities that are already presentprior to the learning task The goal of learning is to estimatecausal strength of individual causal links or to induce causalmodels on the basis of observed covariations The role ofcause and effect categories in the learning process is not thefocus of interest in these approaches they are simply viewedas given

A similar approach underlies research on the relationshipbetween categories and causality According to the view thatcategorization is theory-based traditional similarity-based

accounts of categorization are deficient because they ignorethe fact that many categories are grounded in knowledgeabout causal structures [13] In natural concepts featuresoften represent causes or effects with the category labelreferring to a complex causalmodel For example disease cat-egories frequently refer to common-cause models of diseaseswith the category features representing causes (eg virus) andeffects (eg symptoms) within this causal model A numberof studies using these and similar materials have shown thatthe type of causalmodel connecting otherwise identical causeand effect features influences learning typicality judgmentsor generalization [14ndash17] The main goal of these studieswas to investigate the effect of different causal relationsconnecting the causal features As in contingency learningstudies the cause and effect features within the causal modelswere treated as fixed categorized entities which alreadyexisted prior to the learning context

23 Categories Shape Causality It is certainly true that manyinteresting insights can be gained from investigating howpeople learn about causal models on the basis of preexistingcause and effect categories However there is also a linkbetween categories and causality in the opposite directionThe categories that have been acquired in previous learningcontexts may have a crucial influence on subsequent causallearning

The basis of the potential influence of categories on causalinduction lies in the fact that the acquisition and use of causalknowledge is based on categorized events Causal laws suchas the fact that smoking causes heart disease can only benoticed on the basis of events that are categorized (eg eventsof smoking and cases of heart disease)

Without such categories causal laws neither could bedetected nor could causal knowledge be applied to newcases Thus causal knowledge not only affects the creation ofcategories it also presupposes already existing categories forthe description of causes and effects The potential influenceof categories is due to the fact that one of the most importantcues to causality is statistical covariation between causes andeffects

To study the relation between categories and causalinduction [12] have developed a new paradigm that consistsof three phases the category learning phase the causallearning phase and the third test phase The main goal isto answer the question that under what condition learnerswill tend to activate the categorical information (Figure 1)from the earlier category learning phase when learning aboutcausal contingencies in the later phase This effect is entailedby the fact that the alternative categories form differentreference classes

In category learning phase causes will be classified intothe distinct categories (upper left arrow in Figure 1) Incausal learning phase causes in collection are paired withthe presence or absence of an effect (lower arrow) In thesubsequent test phase the test causes are rated with thelikelihood of the effect The crucial question is when peoplewould go through the upper route in Figure 1 and assign thetest causes to the categories in the category learning phase or


Categories

EffectsCauses


Categories

EffectsCausesCauses

Category learning

learning





















Compoundsentences








If clauses

























Category learning


categories

Category and causal

mapping

Causal learning


relations


relations

Causality in verbs




Inputlayer (IL)

Recurrentlayer (RL)

Outputlayer (OL)

Contextlayer (CL)

Time delay

OU1 OU2

RU1RU2

WOR

IU1 IU2

WRI W

RC

CU1 CU2




OU|O|

RU|R|

IU|I| CU|C|











119882RI= [(119908

RI1 )119879 (119908

RI2 )119879 (119908

RI|119877|)119879]

=

[[[[[

[

119908ri11 119908


ri1|119877|


ri1|119877|

d

119908ri|119868|1 119908


ri|119868||119877|

]]]]]

]

119882RC= [(119908

RC1 )119879 (119908

RC2 )119879 (119908

RC|119877| )119879]

=

[[[[

[


rc1|119877|


rc1|119877|

d

119908rc|119862|1 119908


rc|119862||119877|

]]]]

]

119882OR= [(119908

OR1 )119879 (119908

OR2 )119879 (119908

OR|119874| )119879]

=

[[[[

[


or1|119874|


or1|119874|

d

119908or|119877|1 119908


or|119877||119874|

]]]]

]

(1)




ri2119896 119908





(119905)2


119894and



RU1015840(119905)119894 = IU(t)sdot (119908


119879

=

|119868|

sum119895=1

IU(119905)119895 119908ri119895119894 +

|119877|

sum119895=1

RU(119905minus1)119895 119908rc119895119894

RU(119905)119894 = 119891 (RU1015840(119905)

119894 )

(2)




OU1015840(119905)119896= RU(t) sdot (119908OR119896 )

119879=

|119877|

sum119895=1

RU(119905)119895 119908or119895119896

OU(119905)119896= 119891 (OU1015840(119905)

119896)

(3)


119891 (119909) =1

1 + 119890minus119909=119890119909

1 + 119890119909 (4)







00

01

02

03

04

05

06

07

08

09

10


0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS





















Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




Categories

EffectsCauses


Categories

EffectsCausesCauses

Category learning

learning





















Compoundsentences








If clauses

























Category learning


categories

Category and causal

mapping

Causal learning


relations


relations

Causality in verbs




Inputlayer (IL)

Recurrentlayer (RL)

Outputlayer (OL)

Contextlayer (CL)

Time delay

OU1 OU2

RU1RU2

WOR

IU1 IU2

WRI W

RC

CU1 CU2




OU|O|

RU|R|

IU|I| CU|C|











119882RI= [(119908

RI1 )119879 (119908

RI2 )119879 (119908

RI|119877|)119879]

=

[[[[[

[

119908ri11 119908


ri1|119877|


ri1|119877|

d

119908ri|119868|1 119908


ri|119868||119877|

]]]]]

]

119882RC= [(119908

RC1 )119879 (119908

RC2 )119879 (119908

RC|119877| )119879]

=

[[[[

[


rc1|119877|


rc1|119877|

d

119908rc|119862|1 119908


rc|119862||119877|

]]]]

]

119882OR= [(119908

OR1 )119879 (119908

OR2 )119879 (119908

OR|119874| )119879]

=

[[[[

[


or1|119874|


or1|119874|

d

119908or|119877|1 119908


or|119877||119874|

]]]]

]

(1)




ri2119896 119908





(119905)2


119894and



RU1015840(119905)119894 = IU(t)sdot (119908


119879

=

|119868|

sum119895=1

IU(119905)119895 119908ri119895119894 +

|119877|

sum119895=1

RU(119905minus1)119895 119908rc119895119894

RU(119905)119894 = 119891 (RU1015840(119905)

119894 )

(2)




OU1015840(119905)119896= RU(t) sdot (119908OR119896 )

119879=

|119877|

sum119895=1

RU(119905)119895 119908or119895119896

OU(119905)119896= 119891 (OU1015840(119905)

119896)

(3)


119891 (119909) =1

1 + 119890minus119909=119890119909

1 + 119890119909 (4)







00

01

02

03

04

05

06

07

08

09

10


0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS





















Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in






Compoundsentences








If clauses

























Category learning


categories

Category and causal

mapping

Causal learning


relations


relations

Causality in verbs




Inputlayer (IL)

Recurrentlayer (RL)

Outputlayer (OL)

Contextlayer (CL)

Time delay

OU1 OU2

RU1RU2

WOR

IU1 IU2

WRI W

RC

CU1 CU2




OU|O|

RU|R|

IU|I| CU|C|











119882RI= [(119908

RI1 )119879 (119908

RI2 )119879 (119908

RI|119877|)119879]

=

[[[[[

[

119908ri11 119908


ri1|119877|


ri1|119877|

d

119908ri|119868|1 119908


ri|119868||119877|

]]]]]

]

119882RC= [(119908

RC1 )119879 (119908

RC2 )119879 (119908

RC|119877| )119879]

=

[[[[

[


rc1|119877|


rc1|119877|

d

119908rc|119862|1 119908


rc|119862||119877|

]]]]

]

119882OR= [(119908

OR1 )119879 (119908

OR2 )119879 (119908

OR|119874| )119879]

=

[[[[

[


or1|119874|


or1|119874|

d

119908or|119877|1 119908


or|119877||119874|

]]]]

]

(1)




ri2119896 119908





(119905)2


119894and



RU1015840(119905)119894 = IU(t)sdot (119908


119879

=

|119868|

sum119895=1

IU(119905)119895 119908ri119895119894 +

|119877|

sum119895=1

RU(119905minus1)119895 119908rc119895119894

RU(119905)119894 = 119891 (RU1015840(119905)

119894 )

(2)




OU1015840(119905)119896= RU(t) sdot (119908OR119896 )

119879=

|119877|

sum119895=1

RU(119905)119895 119908or119895119896

OU(119905)119896= 119891 (OU1015840(119905)

119896)

(3)


119891 (119909) =1

1 + 119890minus119909=119890119909

1 + 119890119909 (4)







00

01

02

03

04

05

06

07

08

09

10


0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS





















Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in


















Category learning


categories

Category and causal

mapping

Causal learning


relations


relations

Causality in verbs




Inputlayer (IL)

Recurrentlayer (RL)

Outputlayer (OL)

Contextlayer (CL)

Time delay

OU1 OU2

RU1RU2

WOR

IU1 IU2

WRI W

RC

CU1 CU2




OU|O|

RU|R|

IU|I| CU|C|











119882RI= [(119908

RI1 )119879 (119908

RI2 )119879 (119908

RI|119877|)119879]

=

[[[[[

[

119908ri11 119908


ri1|119877|


ri1|119877|

d

119908ri|119868|1 119908


ri|119868||119877|

]]]]]

]

119882RC= [(119908

RC1 )119879 (119908

RC2 )119879 (119908

RC|119877| )119879]

=

[[[[

[


rc1|119877|


rc1|119877|

d

119908rc|119862|1 119908


rc|119862||119877|

]]]]

]

119882OR= [(119908

OR1 )119879 (119908

OR2 )119879 (119908

OR|119874| )119879]

=

[[[[

[


or1|119874|


or1|119874|

d

119908or|119877|1 119908


or|119877||119874|

]]]]

]

(1)




ri2119896 119908





(119905)2


119894and



RU1015840(119905)119894 = IU(t)sdot (119908


119879

=

|119868|

sum119895=1

IU(119905)119895 119908ri119895119894 +

|119877|

sum119895=1

RU(119905minus1)119895 119908rc119895119894

RU(119905)119894 = 119891 (RU1015840(119905)

119894 )

(2)




OU1015840(119905)119896= RU(t) sdot (119908OR119896 )

119879=

|119877|

sum119895=1

RU(119905)119895 119908or119895119896

OU(119905)119896= 119891 (OU1015840(119905)

119896)

(3)


119891 (119909) =1

1 + 119890minus119909=119890119909

1 + 119890119909 (4)







00

01

02

03

04

05

06

07

08

09

10


0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS





















Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in








119882RI= [(119908

RI1 )119879 (119908

RI2 )119879 (119908

RI|119877|)119879]

=

[[[[[

[

119908ri11 119908


ri1|119877|


ri1|119877|

d

119908ri|119868|1 119908


ri|119868||119877|

]]]]]

]

119882RC= [(119908

RC1 )119879 (119908

RC2 )119879 (119908

RC|119877| )119879]

=

[[[[

[


rc1|119877|


rc1|119877|

d

119908rc|119862|1 119908


rc|119862||119877|

]]]]

]

119882OR= [(119908

OR1 )119879 (119908

OR2 )119879 (119908

OR|119874| )119879]

=

[[[[

[


or1|119874|


or1|119874|

d

119908or|119877|1 119908


or|119877||119874|

]]]]

]

(1)




ri2119896 119908





(119905)2


119894and



RU1015840(119905)119894 = IU(t)sdot (119908


119879

=

|119868|

sum119895=1

IU(119905)119895 119908ri119895119894 +

|119877|

sum119895=1

RU(119905minus1)119895 119908rc119895119894

RU(119905)119894 = 119891 (RU1015840(119905)

119894 )

(2)




OU1015840(119905)119896= RU(t) sdot (119908OR119896 )

119879=

|119877|

sum119895=1

RU(119905)119895 119908or119895119896

OU(119905)119896= 119891 (OU1015840(119905)

119896)

(3)


119891 (119909) =1

1 + 119890minus119909=119890119909

1 + 119890119909 (4)







00

01

02

03

04

05

06

07

08

09

10


0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS





















Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




00

01

02

03

04

05

06

07

08

09

10


0681 0706 069807320719

0755 074207930776

0815 0809

0897

(a) Precision

00

01

02

03

04

05

06

07

08

09

10

0593 0581 0602

0706

0621 0609 0616

0734

0685 0661 0683

0805


(b) Recall

00

01

02

03

04

05

06

07

08

09

10

0634 0637 0646 0719

0666 0674 0673 0762

0728 0730 0741

0849


GCASGCAS-L

CL-CIS





















Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in













Acknowledgment


References





























Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in










Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in



research article improving causality induction with...

Documents