Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8078–8088November 7–11, 2021. c©2021 Association for Computational Linguistics

8078

Honey or Poison? Solving the Trigger Curse in Few-shot Event Detectionvia Causal Intervention

Jiawei Chen1,3, Hongyu Lin1,∗, Xianpei Han1,2,∗∗, Le Sun1,2,1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science

Institute of Software, Chinese Academy of Sciences, Beijing, China3University of Chinese Academy of Sciences, Beijing, China

{jiawei2020}@iscas.ac.cn{hongyu,xianpei,sunle}@iscas.ac.cn

Abstract

Event detection has long been troubled by thetrigger curse: overfitting the trigger will harmthe generalization ability while underfitting itwill hurt the detection performance. This prob-lem is even more severe in few-shot scenario.In this paper, we identify and solve the trig-ger curse problem in few-shot event detection(FSED) from a causal view. By formulatingFSED with a structural causal model (SCM),we found that the trigger is a confounder ofthe context and the result, which makes previ-ous FSED methods much easier to overfit trig-gers. To resolve this problem, we propose tointervene on the context via backdoor adjust-ment during training. Experiments show thatour method significantly improves the FSEDon ACE05, MAVEN and KBP17 datasets.

1 Introduction

Event detection (ED) aims to identify and clas-sify event triggers in a sentence, e.g., detecting anAttack event triggered by fire in “They killedby hostile fire in Iraqi”. Recently, supervised EDapproaches have achieved promising performance(Chen et al., 2015; Nguyen and Grishman, 2015;Nguyen et al., 2016; Lin et al., 2018, 2019b,a; Duand Cardie, 2020; Liu et al., 2020a; Lu et al., 2021),but when adapting to new event types and domains,a large number of manually annotated event datais required which is expensive. By contrast, few-shot event detection (FSED) aims to build effectiveevent detectors that are able to detect new eventsfrom instances (query) with a few labeled instances(support set). Due to their ability to classify noveltypes, many few-shot algorithms have been used inFSED, e.g., metric-based methods like PrototypicalNetwork (Lai et al., 2020; Deng et al., 2020; Conget al., 2021).

Unfortunately, there has long been a “triggercurse” which troubles the learning of event detec-

∗Corresponding authors.

An Attackevent in Iraqi.

fire

They were killed byhostile [MASK] in Iraqi.

They were killed byhostile fire in Iraqi. 1 or 0

Query

ET

C S Y

E

C

T

S Y

Q

(a) Structural Causal Model for the data distribution ofFSED

An Attackevent in Iraqi.

forces, fire,attack, ...

They were killed byhostile [MASK] in Iraqi.

They were killed byhostile [T] in Iraqi. 1 or 0

QueryE T

C S Y

E

C

T

S Y

Q

(b) The data distribution of FSED after causal intervention

Figure 1: Illustration of the causal intervention strategyproposed in this paper. The graph includes the event E,the trigger set T , the context setC, the support instanceS, the prediction Y and the query instance Q.

tion models, especially in few-shot scenario (Bron-stein et al., 2015; Liu et al., 2017; Chen et al., 2018;Liu et al., 2019; Ji et al., 2019). For many eventtypes, their triggers are dominated by several pop-ular words, e.g., the Attack event type is domi-nated by war, attack, fight, fire, bomb in ACE05.And we found the top 5 triggers of each event typecover 78% of event occurrences in ACE05. Due tothe trigger curse, event detection models nearly de-generate to a trigger matcher, ignore the majority ofcontextual information and mainly rely on whetherthe candidate word matches the dominant triggers.This problem is more severe in FSED: since thegiven support instances are very sparse and lackdiversity, it is much easier to overfit the trigger ofthe support instances. An intuitive solution for thetrigger curse is to erase the trigger information ininstances and forces the model to focus more onthe context. Unfortunately, due to the decisive roleof triggers, directly wiping out the trigger infor-mation commonly hurts the performance (Lu et al.,2019; Liu et al., 2020b). Some previous approachestry to tackle this problem by introducing more di-

8079

versified context information like event argumentinformation (Liu et al., 2017, 2019; Ji et al., 2019)and document-level information (Ji and Grishman,2008; Liao and Grishman, 2010; Duan et al., 2017;Chen et al., 2018). However, rich context infor-mation is commonly not available for FSED, andtherefore these methods can not be directly applied.

In this paper, we revisit the trigger curse in FSEDfrom a causal view. Specifically, we formulate thedata distribution of FSED using a trigger-centricstructural causal model (SCM) (Pearl et al., 2016)shown in Figure 1(a). Such trigger-centric formu-lation is based on the fact that, given the eventtype, contexts have a much lower impact on trig-gers, compared with the impact of triggers on con-texts. This results in the decisive role of triggers inevent extraction, and therefore conventional eventextraction approaches commonly follow the trigger-centric procedure (i.e., identifying triggers first andthen using triggers as an indicator to find argumentsin contexts). Furthermore, the case grammar theoryin linguistics (Fillmore, 1967) also formulate thelanguage using such trigger/predicate-centric as-sumption, and have been widely exploited in manyNLP tasks like semantic role labeling (Gildea andJurafsky, 2002) and abstract meaning representa-tion (Banarescu et al., 2013).

From the SCM, we found that T (trigger set)is a confounder of the C(context set) and theY (result), and therefore there exists a backdoorpath C ← T → Y . The backdoor path explainswhy previous FSED models disregard contextualinformation: it misleads the conventional learningprocedure to mistakenly regard effects of triggersas the effects of contexts. Consequently, the learn-ing criteria of conventional FSED methods are opti-mized towards spurious correlation, rather than cap-turing causality between C and Y . To address thisissue, we propose to intervene on context to blockthe information from trigger to context. Specifi-cally, we apply backdoor adjustment to estimatethe interventional distribution that is used for opti-mizing causality. Furthermore, because backdooradjustment relies on the unknown prior confounder(trigger) distribution, we also propose to estimateit based on contextualized word prediction.

We conducted experiments on ACE051,MAVEN2 and KBP173 datasets. Experiments

1https://catalog.ldc.upenn.edu/LDC2006T062https://github.com/THU-KEG/MAVEN-dataset3https://tac.nist.gov/2017/KBP/data.html

show that causal intervention can significantlyalleviate trigger curse, and therefore the proposedmethod significantly outperforms previous FSEDmethods.

2 Structural Causal Model for FSED

This section describes the structural causal model(SCM) for FSED, illustrated in Figure 1(a). Notethat, we omit the causal structure of the query forsimplicity since it is the same as the support set.Concretely, the SCM formulates the data distribu-tion of FSED: 1) Starting from an event E we wantto describe (in Figure 1(a) is an Attack in Iraqi).2) The path E → T indicates the trigger decisionprocess, i.e., selecting words or phrases (in Fig-ure 1(a) is fire) which can almost clearly expressthe event occurrence (Doddington et al., 2004). 3)The path E → C ← T indicates that a set of con-texts are generated depending on both the event andthe trigger, which provides background informa-tion and organizes this information depending onthe trigger. For instance, the context “They killedby hostile [fire] in Iraqi” provides the place, therole and the consequences of the event, and thisinformation is organized following the structure de-termined by fire. 4) an event instance is generatedby combining one of the contexts in C and one ofthe triggers in T via the path C → S ← T . 5)Finally, a matching between query and support setis generated through S → Y ← Q.

Conventional learning criteria for FSED di-rectly optimize towards the conditional distributionP (Y |S,Q). However, from the SCM, we foundthat the backdoor path C ← T → Y pass on asso-ciations (Pearl et al., 2016) and mislead the learningwith spurious correlation. Consequently, the learn-ing procedure towards P (Y |S,Q) will mistakenlyregard the effects of triggers as the effects of con-texts, and therefore overfit the trigger information.

3 Causal Intervention for Trigger Curse

Based on the SCM, this section describes how toresolve the trigger curse via causal intervention.Context Intervention. To block the backdoorpath, we intervene on the context C and the newcontext-intervened SCM is shown in Figure 1(b).Given support set s, event set e of s, context set Cof s and query instance q, we optimize the interven-tional distribution P (Y |do(C = C), E = e,Q =q) rather than P (Y |S = s,Q = q), where do(·)denotes causal intervention operation. By interven-

8080

ing, the learning objective of models changes fromoptimizing correlation to optimizing causality.Backdoor Adjustment. Backdoor adjustment isused to estimate the interventional distribution4:

P (Y |do(C=C), E=e,Q=q)

=∑t∈T

∑s∈S

P (Y |s, q)P (s|C, t)P (t|e), (1)

where P (s|C, t) denotes the generation of s fromthe trigger and contexts. P (s|C, t) = 1/|C| if andonly if the context of s in C and the trigger of s ist. P (Y |s, q) ∝ φ(s, q; θ) is the matching modelbetween q and s parametrized by θ.Estimating P (t|e) via Contextualized Predic-tion. The confounder distribution P (t|e) is un-known because E is a hidden variable. Since theevent argument information is contained in C, weargue that P (t|e) ∝ M(t|C) where M(·|C) indi-cates a masked token prediction task (Taylor, 1953)which is constructed by masking triggers in thesupport set. In this paper, we use masked lan-guage model to calculate P (t|e) by first generat-ing a set of candidate triggers through the context:Tc = {ti|i = 1, 2, . . .} ∪ {t0}, where ti is the i-thpredicted token and t0 is the original trigger of thesupport set instance, then P (t|e) is estimated byaveraging logit obtained from the MLM:

P (ti|e) =

λ i = 0

(1− λ) exp(li)∑j exp(lj)

i 6= 0(2)

where li is the logit for the ith token. To reduce thenoise introduced by MLM, we assign an additionalhyperparameter λ ∈ (0, 1) to t0.Optimizing via Representation Learning. Giventhe interventional distribution, FSED model can belearned by minimizing the loss function on it:

L(θ) = −∑q∈Q

f(P (Y |do(C), e, q; θ))

= −∑q∈Q

f(∑t∈T

∑s∈S

P (Y |s, q; θ)P (s|C, t)P (t|e))

(3)

where Q is training queries and f is a strictmonotonically increasing function. However, theoptimization of L(θ) needs to calculate everyP (Y |s, q; θ), which is quite time-consuming. Tothis end, we propose a surrogate learning criteriaLSG(θ) to optimize the causal relation based onrepresentation learning:

4The proof is shown in Appendix

LSG(θ) = −∑q∈Q

g(R(q; θ),

∑t∈T

∑s∈S

P (s|C, t)P (t|e)R(s; θ))(4)

Here R is a representation model which inputs sor q and outputs a dense representation. g(·, ·) is adistance metric measuring the similarity betweentwo representations. Such loss function is widelyused in many metric-based methods (e.g., Proto-typical Networks and Relation Networks). In theAppendix, we prove LSG(θ) is equivalent to L(θ).

4 Experiments

4.1 Experimental SettingsDatasets.5 We conducted experiments onACE05, MAVEN (Wang et al., 2020c) and KBP17datasets. We split train/dev/test sets accordingto event types and we use event types with moreinstances for training, the other for dev/test. Toconduct 5-shot experiments, we filter event typesless than 6 instances. Finally, for ACE05, itstrain/dev/test set contains 3598/140/149 instancesand 20/10/10 types respectively, for MAVEN, thoseare 34651/1494/1505 instances and 120/45/45types, for KBP17, those are 15785/768/792instances and 25/13/13 types.Task Settings. Different from episode evaluationin Lai et al. (2020) and Cong et al. (2021), weemploy a more practical event detection settinginspired by Yang and Katiyar (2020) in Few-shotNER. We randomly sample few instances as sup-port set and all other instances in the test set areused as queries. A support set corresponds to anevent type and all types will be evaluated by travers-ing each event type. Models need to detection thespan and type of triggers in a sentence. We alsocompared the results across settings in Section 4.3.We evaluate all methods using macro-F1 and micro-F1 scores, and micro-F1 is taken as the primarymeasure.Baselines. We conduct experiments on twometric-based methods: Prototypical Network(Snell et al., 2017) and Relation Network (Sunget al., 2018), which are referred as FS-Base. Basedon these models, we compare our causal interven-tion method (FS-Casual) with 1) FS-LexFree (Luet al., 2019), which address overfit triggers via ad-versarial learning, we use their lexical-free encoder;

5Our source codes are openly available athttps://github.com/chen700564/causalFSED

8081

ModelACE05 MAVEN KBP17

Macro Micro Macro Micro Macro Micro

Finetuing-based

Finetune 51.0±1.4 58.2±1.6 30.7±1.5 31.6±2.3 59.4±1.9 62.7±1.8Finetune* 39.9±1.1 45.5±0.7 20.8±1.0 20.6±0.8 45.0±0.7 47.3±0.6Pretrain+Finetune 22.9±6.0 20.3±4.3 20.9±4.6 16.9±5.2 35.1±5.9 30.1±5.5Pretrain+Finetune* 14.6±3.3 15.6±3.4 12.5±3.8 14.9±4.0 23.4±6.8 25.8±6.3

Prototypical Net

FS-Base 63.8±2.8 67.3±2.7 44.7±1.4 44.5±2.0 65.5±2.7 67.3±3.1FS-LexFree 52.7±2.9 53.9±3.2 25.6±1.0 21.8±1.4 60.7±2.5 61.4±2.8FS-ClusterLoss 64.9±1.5 69.4±2.0 44.2±1.2 44.0±1.2 65.5±2.3 67.1±2.4FS-Causal (Ours) 73.0±2.2 76.9±1.4 52.1±0.2 55.0±0.4 70.9±0.6 73.2±0.9

Relation Net

FS-Base 65.7±3.7 68.7±4.5 52.4±1.4 56.0±1.4 67.2±1.5 71.2±1.4FS-LexFree 59.3±3.5 60.1±3.9 43.8±1.9 45.9±2.4 61.9±2.4 65.4±2.8FS-ClusterLoss 57.6±2.3 60.2±3.2 46.3±1.1 51.8±1.4 56.8±3.0 62.1±2.5FS-Causal (Ours) 67.2±1.4 71.8±1.9 53.0±0.5 57.0±0.9 66.4±0.4 72.0±0.6

Table 1: F1 score of 5-shot FSED on test set. * means fixing the parameters of encoder when finetuning. ± is thestandard deviation of 5 random training rounds.

Episode Ambiguity Ours55

60

65

70

75

80FS-Base FS-Cluster FS-Causal

Figure 2: Micro F1 of prototypical network with differ-ent settings on ACE05 test set.

2) FS-ClusterLoss (Lai et al., 2020), which addtwo auxiliary loss functions when training. Further-more, we compare our method with models fine-tuned with support set (Finetune) and pretrainedusing the training set (Pretrain). BERTbase (un-cased) is used as the encoder for all models andMLM for trigger collection.

4.2 Experimental Results

The performance of our method and all baselinesis shown in Table 1. We can see that:

1) By intervening on the context in SCMand using backdoor adjustment during train-ing, our method can effectively learn FSEDmodels. Compared with the original metric-basedmodels, our method achieves 8.7% and 1.6% micro-F1 (average) improvement in prototypical networkand relation network respectively.

2) The causal theory is a promising techniquefor resolving the trigger cruse problem. Noticethat FS-LexFree cannot achieve the competitiveperformance with the original FS models, which in-dicates that trigger information is import and under-fitting triggers will hurt the detection performance.This verifies that trigger curse is very challengingand causal intervention can effectively resolve it.

3) Our method can achieve state-of-the-artFSED performance. Compared with best scorein baselines, our method gains 7.5%, 1.0%, and2.0% micro-F1 improvements on ACE05, MAVENand KBP17 datasets respectively.

4.3 Effect on Different SettingsTo further demonstrate the effectiveness of the pro-posed method, we also conduct experiments underdifferent FSED settings: 1) The primal episode-based settings (Episode), which is the 5+1-way5-shot settings in Lai et al. (2020). 2) Episode +ambiguous instances (Ambiguity), which samplessome additional negative query instances that in-clude words same as triggers in support set to verifywhether models overfit the triggers.

The performance of different models with dif-ferent settings is shown in Figure 2. We can seethat: 1) Generally speaking, all models can achievebetter performance on Episode because correctlyrecognize high-frequent triggers can achieve goodperformance in this setting. Consequently, the per-formance under this setting can not well representhow FSED is influenced by trigger overfitting. 2)The performance of all models dropped on Am-biguity setting, which suggests that trigger over-fitting has a significant impact on FSED. 3) Ourmethod still maintains good performance on Ambi-guity, which indicates that our method can alleviatethe trigger curse problem by optimizing towardsthe underlying causality.

4.4 Case StudyWe select ambiguous cases (in Table 2) to better il-lustrate the effectiveness of our method. For Query1, FS-Base wrongly detects the word run to be atrigger word. In Support set 1, run means nomi-

8082

Support set 1 I mean , I ’d like to - – I ’d like to see the Greens [Nominate]run[/Nominate] David Cobb again.Query 1 Release a known terrorist to run the PLO and that will bring about peaceFS-Base Release a known terrorist to [Nominate]run[/Nominate] the PLO and that will bring about peaceFS-Causal Release a known terrorist to run the PLO and that will bring about peaceSupport set 2 They were [Suspicion]suspected[/Suspicion] of having facilitated the suicide bomber.Query 2 A fourth suspect, Osman Hussein, was arrested in Rome, Italy, and later extradited to the UK.FS-Base A fourth [Suspicion]suspect[/Suspicion], Osman Hussein, was arrested in Rome, Italy, and later extradited to the UK.FS-Causal A fourth suspect, Osman Hussein, was arrested in Rome, Italy, and later extradited to the UK.

Table 2: Ambiguous cases from ACE05 and MAVEN test set. The results are based on prototypical network andSupport set means one instance in the support set.

nating while run means managing in Query 1. FS-Base fails to recognize such different sense of wordunder context. For Query 2, FS-Base makes mis-take again on the ambiguous word suspect. Eventhough suspect is the noun form of suspected inSupport set 2, it does not trigger a Suspicionevent in Query 2. In contract to FS-Base, our ap-proach is able to handle both cases correctly, illus-trating its effectiveness.

5 Related Work

Causal Inference. Causal inference aims to makereliable predictions using the causal effect betweenvariables (Pearl, 2009). Many studies have usedcausal theory to improve model robustness (Wanget al., 2020a,b; Qi et al., 2020; Tang et al., 2020b;Zeng et al., 2020). Recently, backdoor adjustmenthas been used to remove the spurious associationbrought by the confounder (Tang et al., 2020a;Zhang et al., 2020; Yue et al., 2020; Liu et al.,2021; Zhang et al., 2021).Few-shot Event Detection. Few-shot event detec-tion has been studied in many different settings.Bronstein et al. (2015) collect some seed triggers,then detect unseen event with feature-based method.Deng et al. (2020) decompose FSED into two sub-tasks: trigger identification and few-shot classifi-cation. Feng et al. (2020) adopt a sentence-levelfew-shot classification without triggers. Lai et al.(2020) and Cong et al. (2021) adopt N+1-way few-shot setting that is closest to our setting.

6 Conclusions

This paper proposes to revisit the trigger curse inFSED from a causal view. Specifically, we iden-tify the cause of the trigger curse problem from astructural causal model, and then solve the prob-lem through casual intervention via backdoor ad-justment. Experimental results demonstrate theeffectiveness and robustness of our methods.

Acknowledgments

We thank the reviewers for their insightful com-ments and helpful suggestions. This research workis supported by National Key R&D Program ofChina under Grant 2018YFB1005100, the Na-tional Natural Science Foundation of China underGrants no. 62106251 and 62076233, and in partby the Youth Innovation Promotion AssociationCAS(2018141).

ReferencesLaura Banarescu, Claire Bonial, Shu Cai, Madalina

Georgescu, Kira Griffitt, Ulf Hermjakob, KevinKnight, Philipp Koehn, Martha Palmer, and NathanSchneider. 2013. Abstract Meaning Representationfor sembanking. In Proceedings of the 7th Linguis-tic Annotation Workshop and Interoperability withDiscourse, pages 178–186, Sofia, Bulgaria. Associa-tion for Computational Linguistics.

Ofer Bronstein, Ido Dagan, Qi Li, Heng Ji, and AnetteFrank. 2015. Seed-based event trigger labeling:How far can event descriptions get us? In Proceed-ings of the 53rd Annual Meeting of the Associationfor Computational Linguistics and the 7th Interna-tional Joint Conference on Natural Language Pro-cessing (Volume 2: Short Papers), pages 372–376,Beijing, China. Association for Computational Lin-guistics.

Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, andJun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceed-ings of the 53rd Annual Meeting of the Associationfor Computational Linguistics and the 7th Interna-tional Joint Conference on Natural Language Pro-cessing (Volume 1: Long Papers), pages 167–176,Beijing, China. Association for Computational Lin-guistics.

Yubo Chen, Hang Yang, Kang Liu, Jun Zhao, andYantao Jia. 2018. Collective event detection via ahierarchical and bias tagging networks with gatedmulti-level attention mechanisms. In Proceedings ofthe 2018 Conference on Empirical Methods in Nat-ural Language Processing, pages 1267–1276, Brus-sels, Belgium. Association for Computational Lin-guistics.

8083

Xin Cong, Shiyao Cui, Bowen Yu, Tingwen Liu, WangYubin, and Bin Wang. 2021. Few-Shot Event Detec-tion with Prototypical Amortized Conditional Ran-dom Field. In Findings of the Association for Com-putational Linguistics: ACL-IJCNLP 2021, pages28–40, Online. Association for Computational Lin-guistics.

Shumin Deng, Ningyu Zhang, Jiaojian Kang, YichiZhang, Wei Zhang, and Huajun Chen. 2020. Meta-learning with dynamic-memory-based prototypicalnetwork for few-shot event detection. In Proceed-ings of the 13th International Conference on WebSearch and Data Mining, pages 151–159.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. BERT: Pre-training ofdeep bidirectional transformers for language under-standing. In Proceedings of the 2019 Conferenceof the North American Chapter of the Associationfor Computational Linguistics: Human LanguageTechnologies, Volume 1 (Long and Short Papers),pages 4171–4186, Minneapolis, Minnesota. Associ-ation for Computational Linguistics.

George R Doddington, Alexis Mitchell, Mark A Przy-bocki, Lance A Ramshaw, Stephanie M Strassel, andRalph M Weischedel. 2004. The automatic contentextraction (ace) program-tasks, data, and evaluation.In Lrec, volume 2, pages 837–840. Lisbon.

Xinya Du and Claire Cardie. 2020. Event extrac-tion by answering (almost) natural questions. InProceedings of the 2020 Conference on EmpiricalMethods in Natural Language Processing (EMNLP),pages 671–683, Online. Association for Computa-tional Linguistics.

Shaoyang Duan, Ruifang He, and Wenli Zhao. 2017.Exploiting document level information to improveevent detection via recurrent neural networks. InProceedings of the Eighth International Joint Con-ference on Natural Language Processing (Volume 1:Long Papers), pages 352–361.

Rui Feng, Jie Yuan, and Chao Zhang. 2020. Probingand fine-tuning reading comprehension models forfew-shot event extraction. CoRR, abs/2010.11325.

Charles Fillmore. 1967. The case for case.

Matt Gardner, Joel Grus, Mark Neumann, OyvindTafjord, Pradeep Dasigi, Nelson F. Liu, Matthew E.Peters, Michael Schmitz, and Luke Zettlemoyer.2018. Allennlp: A deep semantic natural languageprocessing platform. CoRR, abs/1803.07640.

Daniel Gildea and Daniel Jurafsky. 2002. Automatic la-beling of semantic roles. Computational linguistics,28(3):245–288.

Heng Ji and Ralph Grishman. 2008. Refining event ex-traction through cross-document inference. In Pro-ceedings of ACL-08: HLT, pages 254–262, Colum-bus, Ohio. Association for Computational Linguis-tics.

Yuze Ji, Youfang Lin, Jianwei Gao, and Huaiyu Wan.2019. Exploiting the entity type sequence to benefitevent detection. In Proceedings of the 23rd Confer-ence on Computational Natural Language Learning(CoNLL), pages 613–623, Hong Kong, China. Asso-ciation for Computational Linguistics.

Viet Dac Lai, Thien Huu Nguyen, and Franck Der-noncourt. 2020. Extensively matching for few-shotlearning event detection. In Proceedings of the FirstJoint Workshop on Narrative Understanding, Story-lines, and Events, pages 38–45, Online. Associationfor Computational Linguistics.

Shasha Liao and Ralph Grishman. 2010. Using doc-ument level cross-event inference to improve eventextraction. In Proceedings of the 48th Annual Meet-ing of the Association for Computational Linguistics,pages 789–797, Uppsala, Sweden. Association forComputational Linguistics.

Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun.2018. Nugget proposal networks for Chinese eventdetection. In Proceedings of the 56th Annual Meet-ing of the Association for Computational Linguistics(Volume 1: Long Papers), pages 1565–1574, Mel-bourne, Australia. Association for ComputationalLinguistics.

Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun.2019a. Cost-sensitive regularization for labelconfusion-aware event detection. In Proceedings ofthe 57th Annual Meeting of the Association for Com-putational Linguistics, pages 5278–5283, Florence,Italy. Association for Computational Linguistics.

Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun.2019b. Sequence-to-nuggets: Nested entity mentiondetection via anchor-region networks. In Proceed-ings of the 57th Annual Meeting of the Associationfor Computational Linguistics, pages 5182–5192,Florence, Italy. Association for Computational Lin-guistics.

Fangchao Liu, Lingyong Yan, Hongyu Lin, XianpeiHan, and Le Sun. 2021. Element intervention foropen relation extraction. In Proceedings of the 59thAnnual Meeting of the Association for Computa-tional Linguistics and the 11th International JointConference on Natural Language Processing (Vol-ume 1: Long Papers), pages 4683–4693, Online. As-sociation for Computational Linguistics.

Jian Liu, Yubo Chen, and Kang Liu. 2019. Exploit-ing the ground-truth: An adversarial imitation basedknowledge distillation approach for event detection.In Proceedings of the AAAI Conference on ArtificialIntelligence, volume 33, pages 6754–6761.

Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and XiaojiangLiu. 2020a. Event extraction as machine readingcomprehension. In Proceedings of the 2020 Con-ference on Empirical Methods in Natural LanguageProcessing (EMNLP), pages 1641–1651, Online. As-sociation for Computational Linguistics.

8084

Jian Liu, Yubo Chen, Kang Liu, Yantao Jia, andZhicheng Sheng. 2020b. How does context matter?on the robustness of event detection with context-selective mask generalization. In Findings of theAssociation for Computational Linguistics: EMNLP2020, pages 2523–2532, Online. Association forComputational Linguistics.

Shulin Liu, Yubo Chen, Kang Liu, and Jun Zhao.2017. Exploiting argument information to improveevent detection via supervised attention mechanisms.In Proceedings of the 55th Annual Meeting of theAssociation for Computational Linguistics (Volume1: Long Papers), pages 1789–1798, Vancouver,Canada. Association for Computational Linguistics.

Yaojie Lu, Hongyu Lin, Xianpei Han, and Le Sun.2019. Distilling discrimination and generaliza-tion knowledge for event detection via delta-representation learning. In Proceedings of the 57thAnnual Meeting of the Association for Computa-tional Linguistics, pages 4366–4376, Florence, Italy.Association for Computational Linguistics.

Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, JialongTang, Annan Li, Le Sun, Meng Liao, and ShaoyiChen. 2021. Text2Event: Controllable sequence-to-structure generation for end-to-end event extraction.In Proceedings of the 59th Annual Meeting of theAssociation for Computational Linguistics and the11th International Joint Conference on Natural Lan-guage Processing (Volume 1: Long Papers), pages2795–2806, Online. Association for ComputationalLinguistics.

Thien Huu Nguyen, Kyunghyun Cho, and Ralph Gr-ishman. 2016. Joint event extraction via recurrentneural networks. In Proceedings of the 2016 Con-ference of the North American Chapter of the As-sociation for Computational Linguistics: HumanLanguage Technologies, pages 300–309, San Diego,California. Association for Computational Linguis-tics.

Thien Huu Nguyen and Ralph Grishman. 2015. Eventdetection and domain adaptation with convolutionalneural networks. In Proceedings of the 53rd AnnualMeeting of the Association for Computational Lin-guistics and the 7th International Joint Conferenceon Natural Language Processing (Volume 2: ShortPapers), pages 365–371, Beijing, China. Associa-tion for Computational Linguistics.

Judea Pearl. 1995. Causal diagrams for empirical re-search. Biometrika, 82(4):669–688.

Judea Pearl. 2009. Causality. Cambridge universitypress.

Judea Pearl. 2014. Probabilistic reasoning in intelli-gent systems: networks of plausible inference. Else-vier.

Judea Pearl, Madelyn Glymour, and Nicholas P Jewell.2016. Causal inference in statistics: A primer. JohnWiley & Sons.

Jiaxin Qi, Yulei Niu, Jianqiang Huang, and HanwangZhang. 2020. Two causal principles for improvingvisual dialog. In Proceedings of the IEEE/CVF Con-ference on Computer Vision and Pattern Recogni-tion, pages 10860–10869.

Jake Snell, Kevin Swersky, and Richard Zemel. 2017.Prototypical networks for few-shot learning. In Ad-vances in neural information processing systems,pages 4077–4087.

Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang,Philip HS Torr, and Timothy M Hospedales. 2018.Learning to compare: Relation network for few-shotlearning. In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition, pages1199–1208.

Kaihua Tang, Jianqiang Huang, and Hanwang Zhang.2020a. Long-tailed classification by keeping thegood and removing the bad momentum causal effect.In Advances in Neural Information Processing Sys-tems 33: Annual Conference on Neural InformationProcessing Systems 2020, NeurIPS 2020, December6-12, 2020, virtual.

Kaihua Tang, Yulei Niu, Jianqiang Huang, Jiaxin Shi,and Hanwang Zhang. 2020b. Unbiased scene graphgeneration from biased training. In Proceedings ofthe IEEE/CVF Conference on Computer Vision andPattern Recognition, pages 3716–3725.

Wilson L Taylor. 1953. “cloze procedure”: A newtool for measuring readability. Journalism quarterly,30(4):415–433.

Tan Wang, Jianqiang Huang, Hanwang Zhang, andQianru Sun. 2020a. Visual commonsense r-cnn. InProceedings of the IEEE/CVF Conference on Com-puter Vision and Pattern Recognition, pages 10760–10770.

Tan Wang, Jianqiang Huang, Hanwang Zhang, andQianru Sun. 2020b. Visual commonsense represen-tation learning via causal inference. In Proceed-ings of the IEEE/CVF Conference on Computer Vi-sion and Pattern Recognition Workshops, pages 378–379.

Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang,Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, YankaiLin, and Jie Zhou. 2020c. MAVEN: A Massive Gen-eral Domain Event Detection Dataset. In Proceed-ings of the 2020 Conference on Empirical Methodsin Natural Language Processing (EMNLP), pages1652–1671, Online. Association for ComputationalLinguistics.

Thomas Wolf, Lysandre Debut, Victor Sanh, JulienChaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, and Jamie Brew. 2019. Huggingface’s trans-formers: State-of-the-art natural language process-ing. CoRR, abs/1910.03771.

8085

Yi Yang and Arzoo Katiyar. 2020. Simple and effectivefew-shot named entity recognition with structurednearest neighbor learning. In Proceedings of the2020 Conference on Empirical Methods in NaturalLanguage Processing (EMNLP), pages 6365–6375,Online. Association for Computational Linguistics.

Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua. 2020. Interventional few-shot learning.In Advances in Neural Information Processing Sys-tems 33: Annual Conference on Neural InformationProcessing Systems 2020, NeurIPS 2020, December6-12, 2020, virtual.

Xiangji Zeng, Yunliang Li, Yuchen Zhai, and YinZhang. 2020. Counterfactual generator: A weakly-supervised method for named entity recognition. InProceedings of the 2020 Conference on EmpiricalMethods in Natural Language Processing (EMNLP),pages 7270–7280, Online. Association for Computa-tional Linguistics.

Dong Zhang, Hanwang Zhang, Jinhui Tang, Xian-Sheng Hua, and Qianru Sun. 2020. Causal interven-tion for weakly-supervised semantic segmentation.In Advances in Neural Information Processing Sys-tems 33: Annual Conference on Neural InformationProcessing Systems 2020, NeurIPS 2020, December6-12, 2020, virtual.

Wenkai Zhang, Hongyu Lin, Xianpei Han, and Le Sun.2021. De-biasing distantly supervised named en-tity recognition via causal intervention. In Proceed-ings of the 59th Annual Meeting of the Associationfor Computational Linguistics and the 11th Interna-tional Joint Conference on Natural Language Pro-cessing (Volume 1: Long Papers), pages 4803–4813,Online. Association for Computational Linguistics.

A Proof of Backdoor Adjustment

We prove the backdoor adjustment for our SCMusing the rules of do-calculus (Pearl, 1995).

For a causal graph G, let GX denote the graphwhere all of the incoming edges to Node X areremoved. let GX denote the graph where all of theoutgoing edges from Node X are removed. ⊥⊥ G

denotes d-separation in G.D-separation (Pearl, 2014): Two (sets of) nodes

X and Y are d-separation by a set of nodes Z (i.e.X ⊥⊥ GY |Z) if all of the paths between (any nodein) X and (any node in) Y are blocked by Z.

The rules of do-calculus are:Rule 1

P (y|do(t), z, w) = P (y|do(t), w)if Y ⊥⊥ GT

Z|T,W

Rule 2

P (y|do(t), do(z), w) = P (y|do(t), z, w)if Y ⊥⊥ GTZ

Z|T,W

Rule 3

P (y|do(t), do(z), w) = P (y|do(t), w)if Y ⊥⊥ GTZ(W )

Z|T,W(5)

where Z(W ) denotes the set of nodes of Z thataren’t ancestors of any node of W in GT .

We can prove our interventional distributionP (Y |do(C = C), E = e):

Step 1 Using the law of total probability:

P (Y |do(C = C), E = e,Q = q)

=∑t∈T

∑s∈S

[P (Y |do(C), e, t, s, q)×

P (s, t|do(C), e, q)]

Step 2 Using the law of conditional probability:

P (Y |do(C = C), E = e,Q = q)

=∑t∈T

∑s∈S

[P (Y |do(C), e, t, s, q)×

P (s|do(C), e, t, q)P (t|do(C), e, q)]

Step 3 Using the Rule 3:

P (Y |do(C = C), E = e,Q = q)

=∑t∈T

∑s∈S

[P (Y |e, t, s, q)×

P (s|do(C), e, t, q)P (t|e, q)]

8086

Step 4 Using the Rule 1:

P (Y |do(C = C), E = e,Q = q)

=∑t∈T

∑s∈S

P (Y |s, q)P (s|do(C), t)P (t|e)

Step 5 Using the Rule 2:

P (Y |do(C = C), E = e,Q = q)

=∑t∈T

∑s∈S

P (Y |s, q)P (s|C, t)P (t|e)

B Detailed Task Settings

One-way K-Shot Settings. We adopt One-wayK-shot setting in our experiments, in which thesupport set in an episode contains one event type(called concerned event) and the query can containany event type. The model aims to detect triggersof the concerned event in query and all types willbe evaluated by traversing each event type. Thesupport set and query in an episode can be formu-lated as follows:

S = {(S1, E, Y 1), . . . , (SK , E, Y K)}

where S is the support set, E is the concernedevent, Si = {si1, si2, . . . , sini

} is the i-th sentencein support, sij is the j-th token in Si, Y i =

{yi1, yi2, . . . , yin} is the labels of tokens in Si andyij = 1 only if ti is the trigger (or part of trigger)of concerned event, otherwise yij = 0.

Q = {Q1, Q2, . . . , QM}

where Q is the set of query and Qi ={qi1, qi2, . . . , qimi

} is the i-th query sentence and qijis the j-th token in Qi

The model is expected to output the concernedevent in Q:

OQ = {(Q1, E, T 11 ), . . . , (Q

1, E, T 1n1),

(Q2, E, T 21 ), . . . , (Q

2, E, T 2n2),

. . . ,

(QM , E, TM1 ), . . . , (QM , E, TM

nM)}

where OQ is the set of triggers of concerned eventdetected in Q, T i

k is the k-th trigger of concernedevent in sentence Qi and ni ≥ 0 means the numberof triggers of concerned event in Qi.

Evaluation We improve the traditional episodeevaluation setting by evaluating the full test set. Foreach event type in test set, we randomly sample Kinstances as support set and all other instances areused as query. Following previous event detectionworks (Chen et al., 2015), the predicted trigger iscorrect if its event type and offsets match those of agold trigger. We evaluate all methods using macro-F1 and micro-F1 scores, and micro-F1 is taken asthe primary measure.

C Few-shot Event Detection Baselines

We use two metric-base methods in our experi-ments: Prototypical network (Snell et al., 2017)and Relation network (Sung et al., 2018), whichcontain an encoder component and a classifier com-ponent.

Encoder We use BERT (Devlin et al., 2019) toencoder the support set and the query. Given asentece X = {x1, x2, . . . , xn}, BERT encodes thesequence and output the represent of each token inX: R = {r1, r2, . . . , rn}. After obtaining the fea-ture representation of the support set, we calculatethe prototype of the categories (concerned eventand other):

pi =1

|Ri|∑r∈Ri

r, i = 0, 1

where pi is the prototype of category i, Ri is theset of feature representation of tokens that labeledwith y = i in support set.

Classifier The models classify each token inquery based on its similarity to the prototype.

We first calculate the similarity between proto-type and token in query.

si,j,k = g(pk,qij), k = 0, 1 (6)

where g(x, y) measures the similarity between xand y, qi

j is the represent of j-th token in i-th querysentence.

Then we calculate the probability distribution oftoken qij :

P (Y |qij ,S) = Softmax(si,j,0, si,j,1) (7)

During training, we use the Cross-Entropy losson each token of query. And the support set andthe query are randomly sampled from the trainingset.

8087

When evaluating, we treat the labels as IO tag-ging schemes, and adjacent I are considered to bethe same trigger so that we can handle a triggerwith multiple tokens.

Similarity Functions For prototypical network,the similarity in Equation 6 is Euclidean distance.For relation network, we calculate similarity us-ing neural networks. Unlike the original paper, wefind the following calculation to be more efficient:g(pk,q

ji ) = F (pk ⊕ qj

i ⊕ |pk − qji |) where ⊕

means concatenation vectors and F is two-layerfeed-forward neural networks with a ReLU func-tion on the first layer.

D Proof of Loss Function

We prove LSG(θ) is equivalent to L(θ), whichindicates that minimizing LSG(θ) is equivalentto minimizing L(θ). At first , we define a func-tion φ(s, q) ∝ P (Y |s, q; θ) and then we need toprove that g(

∑t∈T

∑s∈S P (t|e)p(s|C, t)rs,q) =

f(∑

t∈T∑

s∈S P (t|e)P (s|C, t)φ(s, q)).From Appendix-A, we can obtain:∑

t∈T

∑s∈S

P (t|e)p(s|C, t)

=∑t∈T

∑s∈S

P (s, t|do(C), e, q)

=1

D.1 Prototypical NetworkFor prototypical network, g(r,q) = (r− q)2. Letφ(s, q) = |rs − q| and f(x) = x2, x > 0.

g(∑t∈T

∑s∈S

P (t|e)p(s|C, t)rs,q)

=[∑t∈T

∑s∈S

P (t|e)p(s|C, t)rs − q]2

=[∑t∈T

∑s∈S

P (t|e)p(s|C, t)rs

−∑t∈T

∑s∈S

P (t|e)p(s|C, t)q]2

=[∑t∈T

∑s∈S

P (t|e)P (s|C, t)|rs − q|]2

=f [∑t∈T

∑s∈S

P (t|e)P (s|C, t)φ(s, q)]

∝f(P (Y |do(C = C), E = e,Q = q))

D.2 Relation NetworkLet g(r,q) = F [r ⊕ q ⊕ |r − q|]. Let φ(s, q) =g(rs,q) and f(x) = x

g(∑t∈T

∑s∈S

P (t|e)P (s|C, t)rs,q)

=F [∑t∈T

∑s∈S

P (t|e)P (s|C, t)rs ⊕ q

⊕ |∑t∈T

∑s∈S

P (t|e)P (s|C, t)rs − q|]

=F [∑t∈T

∑s∈S

P (t|e)P (s|C, t)rs

⊕∑t∈T

∑s∈S

P (t|e)P (s|C, t)q

⊕ |∑t∈T

∑s∈S

P (t|e)P (s|C, t)rs

−∑t∈T

∑s∈S

P (t|e)P (s|C, t)q|]

≈F [∑t∈T

∑s∈S

P (t|e)P (s|C, t)rs

⊕∑t∈T

∑s∈S

P (t|e)P (s|C, t)q

⊕∑t∈T

∑s∈S

P (t|e)P (s|C, t)|rs − q|]

=∑t∈T

∑s∈S

P (t|e)P (s|C, t)g(r,q)

=f [∑t∈T

∑s∈S

P (t|e)P (s|C, t)φ(s, q)]

∝f(P (Y |do(C = C), E = e,Q = q))

Here, we assume that the feature representationsof the same event type in support are close to eachother so that |

∑s psrs−

∑s psq| ≈

∑s ps|rs−q|.

E Implementation Details

All of our experiments are implemented on oneNvidia TITAN RTX. Our implementation is basedon HuggingFace’s Transformers (Wolf et al., 2019)and Allennlp (Gardner et al., 2018). We tune thehyperparameters based on the dev performance.We train each model 5 times with different randomseed, and when evaluating, we sample 4 differentsupport sets.

Metric-based Methods The hyperparameter isshown in Table 5. During training, the supportset and the query is sampled in training set, thequery contains 2 positive instances and 10 negativeinstances (5 times of positive instances). Duringvalidating, the support set and the query is sampledin dev set, the query contains 10 positive instancesand 100 negative instances (10 times of positive

8088

ModelACE05 MAVEN KBP17

Macro Micro Macro Micro Macro Micro

Prototypical Network

FS-Base(Snell et al., 2017) 66.2±3.8 63.8±4.1 44.1±1.6 44.0±2.3 67.1±1.7 68.0±1.6FS-Lexfree (Lu et al., 2019) 50.4±2.8 50.7±3.6 24.9±1.1 20.5±1.4 60.8±2.7 60.4±3.7FS-Cluster (Lai et al., 2020) 69.9±1.9 67.3±2.2 43.8±1.4 43.6±1.5 67.6±2.4 68.7±2.6FS-Causal (Ours) 76.8±0.6 76.3±0.7 51.8±0.5 55.1±0.4 72.6±0.9 74.9±0.9

Relation Network

FS-Base(Sung et al., 2018) 65.7±3.9 66.9±2.1 51.0±1.1 55.6±1.6 66.8±2.2 71.1±2.0FS-Lexfree (Lu et al., 2019) 59.1 ±6.4 59.6±3.6 42.9±1.0 45.4±2.1 63.5±1.9 65.7±2.0FS-Cluster (Lai et al., 2020) 54.4±2.9 57.2±3.2 45.8±2.3 51.4±1.4 57.5±4.2 62.0±3.2FS-Causal (Ours) 65.0±2.1 69.3±1.5 53.6±0.7 57.9±1.0 66.9±0.1 72.7±0.9

Table 3: F1 score of 5-shot FSED on dev set. ± is the standard deviation of 5 random training rounds.

ACE MAVEN KBPProto (support) 76.25 55.11 74.80Proto (query) 65.75 37.89 69.25Proto (support + query) 74.03 51.19 74.93Relation (support) 68.40 54.34 69.09Relation (query) 66.31 57.85 71.59Relation (support + query) 69.31 56.93 72.65

Table 4: Micro F1 score of 5-shot FSED on dev set.(support) means the backdoor adjustment is used in thesupport set, (query) means the backdoor adjustment isused in the query.

ACE MAVEN KBPOptimizer AdamW AdamW AdamWLearning rate 2e-5 2e-5 2e-5Warmup step 40 240 50Batch size 1 1 1patience 15 15 15max epoch num 80 80 80batches per epoch 40 240 50λ for P (T |C) 0.5 0.5 0.5FS-Causal (Prototypical) S S S+QFS-Causal (Relation) S+Q Q S+Q

Table 5: Hyperparameters of metric-based methods.For FS-Causal, S means the backdoor adjustment isused in the support set, Q means the backdoor adjust-ment is used in the query.

ACE MAVEN KBPOptimizer AdamW AdamW AdamWLearning rate (Pretrain) 1e-5 1e-5 1e-5batches per epoch (Pretrain) 50 200 200Warmup step (Pretrain) 50 200 200Batch size (Pretrain) 128 128 128patience (Pretrain) 10 10 10max epoch num (Pretrain) 50 50 50Learning rate (Finetune) 2e-5 2e-5 2e-5Learning rate (Finetune*) 1e-3 1e-3 1e-3Finetuning Step (Finetune) 20 20 20Finetuning Step (Pretrain + Finetune) 10 10 10

Table 6: Hyperparameters of finetuning-based meth-ods.

instances). The results of dev set are shown inTable 3. For FS-Causal, we found that there is animpact on whether backdoor adjustment is appliedseparately to the support set and query, as shownin Table 4. Based on the best results of the dev set,we evaluate it on the test set.

Finetuning-based Methods The hyperparame-ter is shown in Table 6. For pretraining, we train asupervised event detection model using the trainingset. For finetuning, we use the support set to fine-tune the parameters of the event detection modeland then detect the event in query.