(lrec) – coling – acl 2018 · coling 2018 - tutorials • nlp for conversations: sentiment,...

29
(LREC) – COLING – ACL 2018 Udo Hahn

Upload: others

Post on 03-Oct-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

(LREC) – COLING – ACL 2018

UdoHahn

Page 2: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

Statistics

LREC2018 COLING2018 ACL2018

Submissions 1102 1.017(129withdrawn!) 1.621(L:1045|S:576)

Acceptedpapers 718 331 L:256/1018|S:125/526=381

Acceptanceratio 65.0%(NC:„inclusive“) 37,3% 24.7%

#reviewers 1.273 1.029 1.610

#participants ~1200 ~700 ~1500

Ranking B A–[àB+] A+

Ourcontributions Buechel&Hahn(main) Buechel&Hahn(main) ECONLPWorkshop(org)

Lohretal.(main) Hellrichetal.(demo) ECONLPWorkshop:Buecheletal.(short)

Page 3: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – Santa Fé, New Mexico, USA 84.000Einwohner,2200mHöhe(!)

Page 4: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 - Tutorials

• NLPforConversations:Sentiment,Summarization,andGroupDynamics–GabrielMurray,GiuseppeCareniniandShafiqJoty• DeepBayesianLearningandUnderstanding–Jen-TzungChien• DeepLearningforDialogueSystems–Yun-NungChen,AsliCelikyilmazandDilekHakkani-Tur• PracticalParsingforDownstreamApplications–DanielDakotaandSandraKübler•  FrameSemanticsacrossLanguages:TowardsaMultilingualFrameNet–CollinBaker,MichaelEllsworth,MiriamPetruckandSwabhaSwayamdipta• Data-DrivenTextSimplification–SanjaŠtajnerandHoracioSaggion

Page 5: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – Major Themes

•  Sentiment+EmotionAnalysis(I-IV),Humor/Sarcasm/Rumor(I-II)•  Embeddings(I-III),DistributionalSemantics• MachineLearning(I-II)• Parsing(I-II),Generation(I-II)• Coreference,Discourse(I-II)•  Low-ResourceLanguages•  Ethics• HistoricalLinguistics(L.change)

• NamedEntityRecognition(I-II),InformationExtraction(I-II)• DialogSystems(I-II),QA(I-II)• MachineTranslation(I-III)•  Summarization• Multimodality

Page 6: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 - Keynotes •  JamesPustejovsky:VisualizingMeaning–ModelingCommunicationthroughMultimodalSimulations•  Human-computer/humanrobotinteraction•  Contextual/situatedgrounding(niceVoxWorldvideo,blocksworld:„graspacup“)•  VoxML:modelinglanguageforconstructingcontextualized3Dvirtualrealization•  Multiplelogicscombined

•  HannahRohde:Whyareyoutellingmethis?Relevanceandinformativityinlanguageprocessing•  Inferenceofrelevanceandprocessingof(un)informativeinformation(fromahuman/cognitive)perspective

•  Discoursecoherencerelations:implicit(inferable)vs.explicitrelations–psycholinguisticexperiments(insertionofappropriateadverbialsintoblanktext,givendiscoursecontext);self-pacedreadingtimeexperimentstodetermineinferenceforrelevanceandredundancyfor(lackof)informativeness,e.g.,colorrelativetopiecesofclothingandfruits

•  Min-YenKan:Research–fastandslow•  Accelerationofscience:arXiv,GitHub/JupyterNotebook,SharedTasks•  Researchfast:worksbutwedon’tknowwhy;researchfast+slow:itworksandwethinkweknowwhyandwe’lladvocateforit

•  FabiolaHenri:Investigatingadiscriminativeapproachtocreolization•  Standardclaim:creolelanguage(simplificationsofEuropeanlanguages)donothavemorphology–yet,theyhaveone(thoughsimpler)[25/47slidesshown]

Page 7: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – Interesting Stuff & Observations

• AndreyKutuzov,LiljaØvrelid,TerrenceSzymanski,ErikVelldal:Diachronicwordembeddingsandsemanticshifts:asurvey• AndrewMooreandPaulRayson:BringingreplicationandreproductiontogetherwithgeneralisabilityinNLP:ThreereproductionstudiesforTargetDependentSentimentAnalysis•  Hugedifferencesbetweenoriginalpublicationandreplicationstudieswhendomainsandgenresarechanged

•  INCEptionannotationtool(TUDarmstadt)

Page 8: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – Interesting Stuff & Observations

•  ZiedBouraoui,ShoaibJameel,StevenSchockaert:RelationInductioninWordEmbeddingsRevisited•  Cosine-basedwordsimilaritygivescounter-intuitiveresults:(horses,horse)vs.(berlin,germany)•  SolutioncombinesBayesianlinearregressionsimilaritybetweensandtandhighprobabilityofs-t,sandtbeinginasemanticrelation

•  AlanAkbik,DuncanBlythe,RolandVollgraf:ContextualStringEmbeddingsforSequenceLabeling•  ZalandoWEframework(flair)increaseseffectiveness(WEandsystemarebeingdistributed)•  Problems:Wordambiguitysolvedbycontextualizingembeddings;fixedvocabulary->meaningfulembeddingsforanytypeofwordàproposecontextualstringembeddings(character-based)!

•  Contextualizationisrealizedbycombiningembeddingsfromleft2rightandright2left:outperformsword-levelembeddings;butconcatenatestringembeddingswithstandardwordembeddings!VerygoodresultsforNEtasks(English,German)–movefromword-leveltochar-levellanguagemodelling

•  JieYang,ShuailongLiang,YueZhang:DesignChallengesandMisconceptionsinNeuralSequenceLabeling•  Nicesurveyofproblemsofreproducibilityofsequencelabelingtasks(POS,NER,chunking)àcommonframeworkforcomparison(hyperparameters,datasets,etc.):NCRF++(GitHub)

Page 9: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – Workshop on Trolling, Aggression and Cyberbullying

•  KeynoteTalk–RadaMihalcea:WhatHidesBehindOnlineIdentity(andAnonymity)•  Cyber-selfoftendifferentfromrealself:live24-7:addictivebehavior,fearofmissingout,perfect(eatingdisorders,lowself-esteem),partofunnaturallylargegroups(isolation,negativerelations),differentself(bullying,prediation)àidentitydeception(pretendingtobesomeoneyouarenot)

•  Linguisticstyledifferentwhenengaginginidentitydeception:age,binarygender•  Newidentitydeceptiondataset(AMT):openendedquestions(600individuals)relatedtoportrayoneof4fakeidentities(18/65aged:m/f)usingrealandfakeidentity(imagineyouare18/65:f/m)–canwerecognizeinstancesofidentitydeception?(e.g.,femaleclaimingtobemale):result:85.8(usingSVM)onallidentities

•  Findinggenderdeceivers(predictinggenderdeception):86%menlying(beingfemales)arehardertoidentifythanwomenpretendingtobemen

•  Findingagedeceivers(predictingagedeception):82.7%(oldpeoplearebettertodeceivethanyoungones)

•  Howtospotaliar(relatedtogenderandage)?LIWC,n-grams,wordembeddings•  Lyingaboutothers(fakenews)notaboutthemselves:celebrityandpoliticaldomains(ngrams,punctuation,LIWC,readability,syntaxasfeatures)–ngrams,readabilitybutparticularlyLIWCareperformingwell

•  Humanvs.automaticfakenewsdetection:74-76%(system,70-80%(humans)•  Morethanjustlanguage…videoclipsfrom121realtrialsàgestureannotation

Page 10: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – Workshop on Trolling, Aggression and Cyberbullying • RiTUAL-UHatTRAC2018SharedTask:AggressionIdentification:NiloofarSafiSamghabadi,DeepthiMave,SudiptaKarandThamarSolorio•  130registered,30submitted,25papers•  Annotateddatasetof2languages:Hindi(1200tweets)&English(1269tweets):3classes–overlyaggressive,covertlyaggressive,non-aggressive,postsfromFacebookpages(politicalgroupsandnews)

•  Methods:LSTM,RNN(2ndbestusedSVMwithcharandword-leveln-grams)–dangerofoverfittingwhensurprisedatasethadtobeanalyzed(particularlyforHindi)

•  Summary:NNapproachesnotnecessarilybetterthanstandardML;dataaugmentation(translation,pseudolabelling)usingotherresourcesiseffective;carefulpreprocessingpaysoff

•  Semeval2019:task6–HateEvalandOffensEval(startednow),trainingatendof2018,testingbeginof2019

Page 11: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – LaTech Workshop

• KeynoteTalkbyTedUnderwood:MeasurementandHumanPerspective•  Perspectivalknowledge–crucialforqualitativevs.quantitativeapproachesto(digital)humanities

•  Bayesianapproachescantakethisintoaccount

Page 12: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – German Impact •  TUDarmstadt[6]

•  IliaKuznetsovandIrynaGurevych-FromTexttoLexicon:BridgingtheGapbetweenWordEmbeddingsandLexicalResources•  SteffenEger,JohannesDaxenberger,ChristianStabandIrynaGurevych–Cross-lingualArgumentationMining:MachineTranslation(andabitofProjection)isAll

YouNeed!•  Erik-LânDoDinh,SteffenEgerandIrynaGurevych–KillingFourBirdswithTwoStones:Multi-TaskLearningforNon-LiteralLanguageDetection•  AndreasHanselowski,AvineshPVS,BenjaminSchiller,FelixCaspelherr,DebanjanChaudhuri,ChristianM.MeyerandIrynaGurevych–ARetrospectiveAnalysisof

theFakeNewsChallengeStance-DetectionTask•  LisaBeinborn,TeresaBotschenandIrynaGurevych-MultimodalGroundingforLanguageProcessing•  DaniilSorokinandIrynaGurevych–ModelingSemanticswithGatedGraphNeuralNetworksforKnowledgeBaseQuestionAnswering•  Jan-ChristophKlie,MichaelBugert,BetoBoullosa,RichardEckartdeCastilho&IrynaGurevych–TheINCEpTIONPlatform:Machine-AssistedandKnowledge-

OrientedInteractiveAnnotation[demo]

•  UStuttgart[4]•  JeremyBarnes,RomanKlingerandSabineSchulteimWalde–ProjectingEmbeddingsforDomainAdaption:JointModelingofSentimentAnalysisinDiverse

Domains•  EvgenyKimandRomanKlinger–WhoFeelsWhatandWhy?AnnotationofaLiteratureCorpuswithSemanticRolesofEmotions•  LauraAnaMariaBostanandRomanKlinger-AnAnalysisofAnnotatedCorporaforEmotionClassificationinText•  InaRoesiger,ArndtRiesterandJonasKuhn–Bridgingresolution:Taskdefinition,corpusresourcesandrule-basedexperiments•  MarkusGärtner,SvenMayer,ValentinSchwind,EricHämmerle,EmineTurcan,FlorinRheinwald,GustavMurawski,LarsLischke&JonasKuhn–NLATool:an

ApplicationforEnhancedDeepTextUnderstanding[demo]

•  LeibnizScienceCampus,Heidelberg/Mannheim[2,5]•  InesRehbeinandJosefRuppenhofer–Sprucingupthetrees–Errordetectionintreebanks•  JosefRuppenhofer,MichaelWiegand,RebeccaWilmandKatjaMarkert–Distinguishingaffixoidformationsfromcompounds•  MarcSchulder,MichaelWiegandandJosefRuppenhofer–AutomaticallyCreatingaLexiconofVerbalPolarityShifters:Mono-andCrosslingualMethodsfor

German

Page 13: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – German Impact •  UTübingen[2]

•  ZarahWeißandDetmarMeurers–ModelingtheReadabilityofGermanTargetingAdultsandChildren:Anempiricallybroadanalysisanditscross-corpusvalidation

•  JohannesDellert–CombiningInformation-WeightedSequenceAlignmentandSoundCorrespondenceModelsforImprovedCognateDetection

•  UHamburg[2]•  SeidMuhieYimamandChrisBiemann–Par4Sim:AdaptiveParaphrasingforTextSimplification•  ArneKöhn–IncrementalNaturalLanguageProcessing:Challenges,Strategies,andEvaluation

•  UPassau[2]•  MatthiasCetto,ChristinaNiklaus,AndréFreitasandSiegfriedHandschuh–Graphene:Semantically-LinkedPropositionsinOpen

InformationExtraction•  ChristinaNiklaus,MatthiasCetto,AndréFreitasandSiegfriedHandschuh–ASurveyonOpenInformationExtraction•  MatthiasCetto,ChristinaNiklaus,AndréFreitas&SiegfriedHandschuh–Graphene:aContext-PreservingOpenInformationExtraction

System[demo]

•  BUWeimar[2]•  MartinPotthast,TimGollub,KristofKomlossy,SebastianSchuster,MattiWiegmann,ErikaPatriciaGarcesFernandez,MatthiasHagen

andBennoStein–CrowdsourcingaLargeCorpusofClickbaitonTwitter•  HenningWachsmuth,ManfredStede,RoxanneElBaff,KhalidAlKhatib,MariaSkeppstedtandBennoStein–ArgumentationSynthesis

followingRhetoricalStrategies

•  FSUJena[1,5]•  SvenBuechelandUdoHahn–EmotionRepresentationMappingforAutomaticLexiconConstruction(Mostly)PerformsonHumanLevel•  JohannesHellrich,SvenBuechel&UdoHahn–JeSemE:InterleavingSemanticsandEmotionsinaWebServicefortheExplorationof

LanguageChangePhenomena[demo]

Page 14: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

COLING 2018 – German Impact •  UDuisburg-Essen

•  SebastianDungs,AhmetAker,NorbertFuhrandKalinaBontcheva–CanRumourStanceAlonePredictVeracity?

•  UMünchen•  WenpengYin,YadollahYaghoobzadehandHinrichSchütze–RecurrentOne-HopPredictionsforReasoningoverKnowledgeGraphs

•  UdesSaarlandes,SpokenLanguageSystems•  MarcSchulder,MichaelWiegandandJosefRuppenhofer–AutomaticallyCreatingaLexiconofVerbalPolarityShifters:Mono-andCrosslingualMethodsforGerman

•  UniversityofMannheim•  SanjaˇStajnerandIoanaHulpus–AutomaticAssessmentofConceptualTextComplexityUsingKnowledgeGraphs

•  TUDresden•  KilianGebhardt–Genericrefinementofexpressivegrammarformalismswithanapplicationtodiscontinuousconstituentparsing

•  FUBerlin,DepartmentofLiteraryStudies•  TimoBaumann,HusseinHusseinandBurkhardMeyer-Sickendiek–StyleDetectionforFreeVersePoetryfromTextandSpeech

•  [Fach-]HochschuleHannover•  JeanCharbonnierandChristianWartena–UsingWordEmbeddingsforUnsupervisedAcronymDisambiguation

•  ZalandoResearch,Berlin•  AlanAkbik,DuncanBlytheandRolandVollgraf–ContextualStringEmbeddingsforSequenceLabeling

•  UDÜsseldorf•  AndreasvanCranenburgh–ActiveDOP:anActiveLearningConstituencyTreebankAnnotationTool[demo]

•  HITSHeidelberg•  Mark-ChristophMüller|MichaelStrube–Transparent,Efficient,andRobustWordEmbeddingAccesswithWOMBAT[demo]

•  KITKarlsruhe•  FlorianDessloch,Thanh-LeHa,MarkusMüller,JanNiehues,ThaiSonNguyen,Ngoc-QuanPham,ElizabethSalesky,MatthiasSperber,SebastianStüker,ThomasZenkel&Alexander

Waibel–KITLectureTranslator:MultilingualSpeechTranslationwithOne-ShotLearning[demo]

•  JvGUFrankfurt/Main•  DanielBaumartz|TolgaUslu|AlexanderMehler–LTV:LabeledTopicVector[demo]

Page 15: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – Melbourne, Australia 5MEinwohner(LargerM.,31mHöhe)

Page 16: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018

• Hugegrowthinmembership(3000++)• Asian&PacificChapteroftheACLfounded(AACL)• CallforpapersforaspecialissueofComputationalLinguisticson“Computationalapproachesinhistoricallinguisticsafterthequantitativeturn”–typological(phylogenetic)approachessoughtafter(MPI)

Page 17: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 - Tutorials

•  100ThingsYouAlwaysWantedtoKnowaboutSemantics&PragmaticsButWereAfraidtoAsk–EmilyM.Bender

•  NeuralApproachestoConversationalAI–JianfengGao,MichelGalley,andLihongLi•  VariationalInferenceandDeepGenerativeModels–WilkerAzizandPhilipSchulz•  ConnectingLanguageandVisiontoActions–PeterAnderson,AbhishekDas,andQiWu

•  BeyondMultiwordExpressions:ProcessingIdiomsandMetaphors–ValiaKordoni•  NeuralSemanticParsing–LukeZettlemoyer,MattGardner,PradeepDasigi,SrinivasanIyer,andAlaneSuhr

•  DeepReinforcementLearningforNLP–WilliamYangWang,JiweiLi,andXiaodongHe•  Multi-lingualEntityDiscoveryandLinking–AvirupSil,HengJi,DanRoth,andSilviu-PetruCucerzan

Page 18: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – Major Themes

• MachineLearning(I-III)•  Semantics(I-II),WordSemantics(I-II)•  Parsing(I-II),SemanticParsing(I-III),Generation(I-II)•  Discourse(I-II)•  Inference&Reasoning•  Resources,Annotation•  Language&DocumentModels

• MachineTranslation(I-II),Multilinguality(I-II)•  InformationExtraction(I-III),TextMining,ArgumentMining• QuestionAnswering(I-III),DialogSystems&Discourse(I-III)•  Sentiment(I-II),SocialMedia•  Summarization(I-II)•  Evaluation•  Vision• Multimodality•  InformationRetrieval•  Psycholinguistics&CognitiveModeling

Page 19: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 - Keynotes • AntonvandenHengel:DeepNeuralNetwork,andsomethingsthey’renotverygoodat•  110MLpersons@AustralianInstituteforMachineLearning•  Vision:benchmarkdrivenDL–whatifproblem,dataorinformationchanges•  VisualQuestionAnswering(VQA)–givenpictures,askquestionsabouttheircontents,getanswers(trainingdata:images,questions,answers)requiresexplicitlycodedknowledge(e.g.,DBPedia)

•  Solvingthelearningproblem:NeuralTuringMachine(computesreasonsforanswers)bycouplingimagerecognitionandNL–attentionasreasoning(attention=associativechaining?)–nofixedontology(VisualQuestionAnswering-CVVP17)

•  LifetimeAchievementAward:MarkSteedman•  HistoryofCombinatoryCategorialGrammar(CCG)–whyislanguagecombinatory?–fromLambdacalculustostatisticalCCG;CCGintheAgeofDL

•  Semanticsmattersmore!whichsemanticrepresentation–unsolvedproblem?

Page 20: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 - Keynotes • CarolynPensteinRosé:WhoistheBridgeBetweentheWhatandtheHow?•  Distinguishthewhatoflanguage,namelyitspropositionalcontent,andthehowoflanguage,oritsform,style,orframing.

• Whatbridgesbetweentheserealmsaresocialprocessesthatmotivatethelinguisticchoicesthatresultinspecificrealizationsofpropositionalcontentsituatedwithinsocialinteractions,designedtoachievesocialgoals.

•  thistalkprobesintoaspecificqualityofdiscussionreferredtoasTransactivity,theextenttowhichacontributionarticulatesthereasoningofthespeaker,thatofaninterlocutor,andtherelationbetweenthem(associatedwithsolidarity,influence,expertisetransfer,andlearning)

Page 21: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – Interesting Stuff & Observations •  Significancetesting&NLP

•  TheHitchhiker’sGuidetoTestingStatisticalSignificanceinNaturalLanguageProcessing:Dror,Baumer,Shlomov,andReichart•  Only1/5ofNLPpapersusecorrectstatisticaltesting–surveyoftestingproblems•  Decisionprocedureforselectionfpropertests:If(distributionisknown):selectparametrictest;IF

datasetissmall(..)…•  ReplicabilityAnalysisforNaturalLanguageProcessing:TestingSignificancewithMultipleDatasets:Dror,Baumer,Bogomolov,andReichart[TACL]•  LookattypeI/IIerrorsandoveralltypeerror•  Ratherthanshowinghugetablesbetterspecifyk(FisherorBonferroni)andgivep-valuesforthesek

cases

•  Smalltohugedatasetgeneralizations•  Predictingaccuracyonlargedatasetsfromsmallerpilotdata:Johnson,Anderson,Dras,andSteedman

•  Moredata=betteraccuracy,higherqualityproducesbetteraccuracy,buthowmuchdataareneeded(engineerquestion)àstatisticalpoweranalysis(Cohen,1992):howlargemustadatasetbetoguaranteestatisticalsignificance

•  Idea:extrapolateperformancefromsmallpilotdatatopredictperformanceonmuchlargerdata–provide9extrapolationmethods(Powerlaw,inversesquare-root,biasedpowerlaw,etc.)on8textcorporausingFastTextclassifier;hyper-parameterestimationforeachtestconfiguration!

Page 22: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – Interesting Stuff & Observations •  Scalabilityofnamedentitytypes

•  10ktypes•  LearningNEtypeshierarchies•  Murty,Verga,Vilnis,Radovanovic,andMcCallum:HierarchicalLossesandNewResourcesforFine-grainedEntityTypingandLinking

•  Choi,Levy,Choi,andZettlemoyer:Ultra-FineEntityTyping[medicalappl.]•  Subjectiveembeddings

•  SearchingfortheX-Factor:Exploringcorpussubjectivityforwordembeddings:Tkachenko,Chia,andLauw

•  Wikipedia(neutral,nobias)vs.Amazonreviews(subjective,opinionated)àobjectivevs.subjectiveembeddings;

•  3classificationstasks:sentiment,subjectivityandtopicclassification–similarperformanceofobjective/subjectiveembeddingsonobjectivetasksbutsubjectiveembeddingsbetterforsubjectivegenresàmoreinformationinsubjectiveembeddingsàsentimentvectors(SentiVec)=Word2Vec+LexicalResource

Page 23: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – Interesting Stuff & Observations • Readingcomprehension(machinereading)•  TheNarrativeQAReadingComprehensionChallenge:Koˇciský,Schwarz,Blunsom,Dyer,Hermann,Melis,andGrefenstette

•  Narrative–constructanswersthatcannotbereadofffromthetext(requiresreasoningandcontext),takenfrombooksandmoviescripts

• Why,how,listquestionsanswered

•  Languageunderstandinginthelarge• Whodunnit?CrimeDramaasaCaseforNaturalLanguageUnderstanding:Frermann,Cohen,andLapata

• Whodunnit–CSI(CrimeSceneInvestigation)ranfor15seasons(337episodesàlotsofdata),eachepisode40-64minutes.Video+audio+textualdata

• WhodunitphrasedasasequencelabelingproblemàLSTMdetective

Page 24: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – BioNLP Workshop •  55participants•  Newresources:

•  AllenNLP@github(AllenAI)•  CLEW–ClinicalLanguageEngineeringWorkbench(JonGriffith)@github

•  Sub-wordinformationinpre-trainedbiomedicalwordrepresentations:evaluationandhyper-parameteroptimization:DieterGalea,IvanLaponogov,andKirillVeselkov•  Word2vecvs.fastTextcomparison;optimizinghyper-parameterscangetperformanceimprovementscomparabletolatestarchitectures–lookatcharacters!

•  InvitedPresentation:“ACorpuswithMulti-LevelAnnotationsofPatients,InterventionsandOutcomestoSupportLanguageProcessingforMedicalLiterature”–BenNye•  Supportforevidence-basedmedicine,improveaccessibilityofmedicalliterature:5kabstractsfromPubMed,annotationbycrowdworkers(filteringveryimportant!);questionsdecomposedintoPICOcomponents(Participant/Problem,Intervention,Comparator,Outcomes)àincreasinglymoredetailedinfoisannotated

Page 25: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – EcoNLP Workshop •  ~40participants• Adobe,Zalando,Frenchinvestmentcompany• CausalityAnalysisofTwitterSentimentsandStockMarketReturns–NargesTabari,PiyushaBiswas,BhanuPraneeth,ArminSeyeditabari,MirsadHadzikadic,andWlodekZadrozny•  CausalityanalysisofTwitterbasedonGrangercausality•  Crowdworkersjudgesentimentoftweets(-2to+2,neg/pos)relatedtocompaniesà2000negs/8000postweets,9000neutrals

•  usedpos/negdictionary(Loughran)•  RandomForestandSVMclassifiers:80%F•  Testwhethersenti(score)causesstockmarketreturnorstockreturncausessentiment(score)

Page 26: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – German Impact •  Co-Chair:IrynaGurevych,TUDarmstadt,Germany•  UdesSaarlandes(Cl,MPI)

•  JonasGroschwitz,MatthiasLindemann,MeaghanFowlie,MarkJohnson&AlexanderKoller–AMRdependencyparsingwithatypedsemanticalgebra[L]

•  DominicSeyler,TatianaDembelova,LucianoDelCorro,JohannesHoffart&GerhardWeikum–AStudyoftheImportanceofExternalKnowledgeintheNamedEntityRecognitionTask[S]

•  StefanGrünewald,SophieHenning&AlexanderKoller–GeneralizedchartconstraintsforefficientPCFGandTAGparsing[S]

•  PrabalAgarwal,JannikStrötgen,LucianoDelCorro,JohannesHoffart&GerhardWeikum–diaNED:Time-AwareNamedEntityDisambiguationforDiachronicCorpora[S]

•  LMUMünchen•  NinaPoerner,HinrichSchütze&BenjaminRoth–Evaluatingneuralnetworkexplanationmethodsusinghybriddocuments

andmorphosyntacticagreement[L]•  ViktorHangya,FabienneBraune,AlexanderFraser&HinrichSchütze–TwoMethodsforDomainAdaptationofBilingual

Tasks:DelightfullySimpleandBroadlyApplicable[L]•  PhilippDufter,MengjieZhao,MartinSchmitt,AlexanderFraser&HinrichSchütze–EmbeddingLearningThrough

MultilingualConceptInduction[L]•  WenpengYin,DanRoth&HinrichSchütze–End-TaskOrientedTextualEntailmentviaDeepExplorationsofInter-Sentence

Interactions[S]•  UHeidelberg

•  TodorMihaylov&AnetteFrank–KnowledgeableReader:EnhancingCloze-StyleReadingComprehensionwithExternalCommonsenseKnowledge[L]

•  JuliaKreutzer,JoshuaUyheng&StefanRiezler–ReliabilityandLearnabilityofHumanBanditFeedbackforSequence-to-SequenceReinforcementLearning[L]

•  CarolinLawrence&StefanRiezler–ImprovingaNeuralSemanticParserbyCounterfactualLearningfromHumanBanditFeedback[L]

Page 27: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

ACL 2018 – German Impact •  UStuttgart

•  JeremyBarnes|RomanKlinger|SabineSchulteimWalde–BilingualSentimentEmbeddings:JointProjectionofSentimentAcrossLanguages[L]

•  MartinRiedl&SebastianPadó–ANamedEntityRecognitionShootoutforGerman[S]•  BUWeimar

•  HenningWachsmuth,ShahbazSyed&BennoStein–RetrievaloftheBestCounterargumentwithoutPriorTopicKnowledge[L]

•  KhalidAlKhatib,HenningWachsmuth,KevinLang,JakobHerpel,MatthiasHagen&BennoStein–ModelingDeliberativeArgumentationStrategiesonWikipedia[L]

•  RWTHAachen•  WeiyueWang,DeruiZhu,TamerAlkhouli,ZixuanGan&HermannNey–NeuralHiddenMarkovModelforMachine

Translation[S]•  AlbertZeyer,TamerAlkhouli&HermannNey–RETURNNasaGenericFlexibleNeuralToolkitwithApplicationto

TranslationandSpeechRecognition[demo]•  UMannheim+UHamburg

•  DmitryUstalov,AlexanderPanchenko,AndreyKutuzov,ChrisBiemann&SimonePaoloPonzetto–UnsupervisedSemanticFrameInductionusingTriclustering[S]

•  FacebookResearch•  HolgerSchwenk–FilteringandMiningParallelDatainaJointMultilingualSpace[S]

•  UBielefeld•  MatthiasHartung,HendrikterHorst,FrankGrimm,TimDiekmann,RomanKlinger&PhilippCimiano–SANTO:AWeb-

basedAnnotationToolforOntology-drivenSlotFilling[demo]

Page 28: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

2018 Germany‘s Top Performers ACL2018 COLING2018 Σ

Ud.Saarlandes 4 1 5

LMUMünchen 3,5 1 4,5

UStuttgart 2 4,5 6,5

TUDarmstadt – 6,5 6,5

BUWeimar 2 2 4

UHeidelberg 3 – 3

RWTHAachen 1,5 – 1,5

UHamburg 0,5 2 2,5

LeibnizCampusMA/HD – 2,5 2,5

UPassau – 2,5 2,5

UTübingen – 2 2

UMannheim 0,5 1 1,5

FSUJena – 1,5 1,5

ACL2018 COLING2018 Σ

UBielefeld 1 – 1

FacebookRes. 1 – 1

UDuisburg-Essen – 1 1

TUDresden – 1 1

FUBerlin – 1 1

FHHannover – 1 1

Zalando – 1 1

UDüsseldorf – 0,5 0,5

HITSHeidelberg – 0,5 0,5

KITKarlsruhe – 0,5 0,5

JXvGFrankfurt/Main – 0,5 0,5

Page 29: (LREC) – COLING – ACL 2018 · COLING 2018 - Tutorials • NLP for Conversations: Sentiment, Summarization, and Group Dynamics – Gabriel Murray, Giuseppe Carenini and Shafiq

Over-all Summary

•  Languageunderstandingisbackonthescene(aftermorethan30years!)–thisincludesreasoningtasks,e.g.,formachinereading,vision-basedQA• NeuralNetworks•  Variousmodelsareinvestigated(GAN,attention-basedàcontext)•  Incorporatingnon-textialresources(lexicons/terminologies,knowledgegraphs)•  Transferlearning,multi-tasklearning•  interpretability

• Methodologyunderlyingstatisticalanalysisiscriticiallyreflected