Generative Adversarial Nets for Information Retrieval:Fundamentals and Advances

Weinan ZhangShanghai Jiao Tong University

wnzhang@sjtu.edu.cn

ABSTRACTGenerative adversarial nets (GANs) have been widely studied dur-ing the recent development of deep learning and unsupervisedlearning. With an adversarial training mechanism, GAN managesto train a generative model to fit the underlying unknown realdata distribution under the guidance of the discriminative modelestimating whether a data instance is real or generated. Such aframework is originally proposed for fitting continuous data distri-bution such as images, thus it is not straightforward to be directlyapplied to information retrieval scenarios where the data is mostlydiscrete, such as IDs, text and graphs. In this tutorial, we focus ondiscussing the GAN techniques and the variants on discrete datafitting in various information retrieval scenarios. (i) We introducethe fundamentals of GAN framework and its theoretic properties;(ii) we carefully study the promising solutions to extend GAN ontodiscrete data generation; (iii) we introduce IRGAN, the fundamentalGAN framework of fitting single ID data distribution and the directapplication on information retrieval; (iv) we further discuss thetask of sequential discrete data generation tasks, e.g., text gener-ation, and the corresponding GAN solutions; (v) we present themost recent work on graph/network data fitting with node em-bedding techniques by GANs. Meanwhile, we also introduce therelevant open-source platforms such as IRGAN and Texygen to helpaudience conduct research experiments on GANs in informationretrieval. Finally, we conclude this tutorial with a comprehensivesummarization and a prospect of further research directions forGANs in information retrieval.

KEYWORDSGenerative Adversarial Nets, Information Retrieval, Discrete Data,Text Generation, Network Embedding, Reinforcement Learning,Deep LearningACM Reference Format:Weinan Zhang. 2018. Generative Adversarial Nets for Information Retrieval:Fundamentals and Advances. In SIGIR ’18: The 41st International ACM SIGIRConference on Research and Development in Information Retrieval, July 8–12, 2018, Ann Arbor, MI, USA. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3209978.3210184

1 BACKGROUND AND MOTIVATIONSInformation retrieval (IR) refers to the task of returning a list ofinformation items (e.g., documents, products or answers) given a

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).SIGIR ’18, July 8–12, 2018, Ann Arbor, MI, USA© 2018 Copyright held by the owner/author(s).ACM ISBN 978-1-4503-5657-2/18/07.https://doi.org/10.1145/3209978.3210184

particular information need (e.g., represented by query keywords,user profile or question sentence) [24]. For the methodologies ofIR, in general there are two schools of thinking, namely (i) thegenerative modeling methods, which assume a generative processfrom the query to the document (or the inverse one) and buildgenerative models to maximize the observation likelihood [20, 28],and (ii) the discriminative modeling methods, which directly buildmodel to score each candidate pair of information need and itemand then train the model in a regression or ranking framework[3, 16]. For both directions, machine learning techniques based onbig data have been playing an important role.

On the other hand, the recent development and success of deeplearning [14] have made it applied to various fields including in-formation retrieval (IR). In fact, during the recent three years deeplearning has to-some-extent been revolutionizing the IR area, in-volving the IR applications of web search [9, 21], recommendersystems [1, 6], sentiment analysis [25] and question answering[19] etc. The Neural Information Retrieval (NeuIR) workshop1 hasbecome one of the most popular workshops in SIGIR 2016 and 2017.

However, most above deep learning work is merely focused onreplacing the IR scoring functions with some carefully designedneural network architectures, i.e., the majority of the work on deeplearning for IR is on discriminative models [9, 11, 21]. Therefore,one would think whether deep generative models could benefit IRsolutions and, furthermore, whether the merge of generative anddiscriminative models could help each other towards a better finalsolution.

Generative adversarial nets (GANs) [8] are a novel deep unsuper-vised learning framework which has attracted much attention sinceits birth in 2014 [7]. In GAN framework, a generative model triesto map a randomly sampled noise vector to a data instance (e.g., animage) while a discriminative model is trained to judge whethera data instance is real or generated. With the interplay betweenthe generative and discriminative models, there theoretically existsa Nash equilibrium where the generative model perfectly fits theunderlying real data distribution while the discriminative model istotally fooled by the generated data.

With the inspiration from GANs, there has been some prelim-inary work trying applying GANs to solve information retrievaltasks since 2017 [24]. Regarding the user’s preference distributionover the candidate documents as the underlying data distributionto fit, it is straightforward to build a GAN framework for IR tasks.The generative retrieval model p(d |q, r ) captures the conditionalprobability of the user (represented by her query q) selecting a can-didate document d given the relevance criteria r . The discriminativeretrieval model p(r |q,d) estimates whether the user likes the can-didate document d . Such an interplay between the generative and

1https://neu-ir.weebly.com/

arX

iv:1

806.

0357

7v1

[cs

.IR

] 1

0 Ju

n 20

18

discriminative retrieval models potentially unifies the two schoolsof thinking in IR [24].

From the data perspective, the major data format in IR is dis-crete tokens, e.g., IDs, text, or linked nodes in a network. Directlyapplying the original version of GANs onto such discrete data isinfeasible because one cannot direct take gradient over the discretedata instance itself [8]. Thus, several branches of solutions havebeen proposed, including end-to-end Gumbel-softmax [12] andpolicy gradient based reinforcement learning [22].

In this tutorial, we introduce the very recent development ofGAN techniques on information retrieval. We start from introduc-ing the original GAN framework designed for continuous datageneration and show its failure to be directly applied to discretedata generation. Then we introduce the several methodologies tobypassing the problem of taking gradient over discrete data andderive the corresponding models. Based on these fundamental tech-niques, we discuss the GAN models working on three differentdiscrete data format in IR tasks, namely• IDs (tokens), for the most basic IR task and collaborative fil-tering for recommender systems;

• Text (sequences of tokens), for sentence, poem, article gen-eration and question answering;

• Networks (graphs of tokens), for node embedding, node clas-sification, link prediction and community detection etc;

where the fundamental thinking and different solutions will bediscussed. Finally, we conclude this tutorial by summarizing theexisting work on GANs for IR and discuss some promising futuredirections, e.g., the recently proposed Cooperative Training (CoT)[17].

2 OBJECTIVESAs amerge of information retrieval andmachine learning, the targetaudience of this tutorial is sufficiently broad. And the main objec-tive of this tutorial is to encourage the technical communicationsbetween informmation retrieval and machine learning, betweenapplication-oriented and methodology-oriented research, and be-tween engineering and theory.• For the researchers, engineers and students working on informa-tion retrieval such as text mining, natural language processing,graph pattern mining, network analysis, recommender systemsetc., this tutorial offers a series of new methodologies which arepotential to improve the model performance in their studiedscenarios.

• For machine learning or artificial intelligence researchers, thistutorial helps them broad the scope by providing various appli-cation scenarios for generative adversarial nets and reinforce-ment learning.With this tutorial, we hope the community would develop new

research problems, novel methodologies and even new solutionparadigms on information retrieval.

3 FORMAT AND DETAILED SCHEDULE3.1 Tutorial FormatThe tutorial is expected to be presented as a 3-hour lecture withslides and hangouts to the audience. It is free for the audience to

raise questions during the lecture and there will be a half-hour QAsession at the end of the tutorial.

3.2 Detailed ScheduleThe planned 3-hour schedule of the tutorial is listed as follows.

Introduction to GANs (35 mins). We introduce the backgroundand fundamentals of the original version of generative adversar-ial nets which are designed for continuous data generation. Thetheoretic Nash equilibrium of GAN will be discussed to help the au-dience understand the rationale behind such a minimax game. Alsothe recent advanced applications of GANs on image and speechgeneration scenarios will be presented to help the audience knowabout the wide applications of GANs. Finally, we reveal the problemof applying the original version of GANs to discrete data in infor-mation retrieval tasks, which lies in taking gradient over discretetokens. Based on such a problem, we introduce several promisingsolutions including Gumbel-softmax and reinforcement learning.

Reinforcement Learning (25 mins). In order to extend GANframework onto discrete data generation, we perform a brief re-view of reinforcement learning (RL). The basic idea of Markoviandecision processes (MDPs), value function, policy gradient, RE-INFORCE algorithm [26], likelihood ratio trick and Monte Carlosearch will be presented. A simple example will be provided to helpthe audience get familiar with the presented RL techniques.

GANs for IR Tasks (35 mins). With the previous fundamentaldelivered to the audience, we start to introduce the basic GANmodels to the simplest IR tasks, where the information items arejust represented by IDs (e.g., document ID or product ID with-out detailed information like text or multi-field features) and eachquery-document pair has been represented by a set of categoricaland continuous features. Such a simple setting is illustrated in Fig-ure 1. The IRGAN framework [24] is the central content of this part.We will also present more details beyond the IRGAN paper itself,e.g., the reward function design and importance sampling tricks.Furthermore, we will come to the ad hoc text retrieval task and linkthe GAN mechanism to (pseudo) relevance feedback and presentthe most recent work on this direction.

Figure 1: An illustration of GAN for simplest IR tasks whereeach document is represented just by IDs. For each step, thegenerative model selects one candidate document for thequery and the discriminative model makes a relevance judg-ment over the query-document pair which is used to guidethe generative model for better modeling the user’s prefer-ence over the candidate documents.

GANs for Text Generation (35mins). Extending from the classicdocument retrieval tasks, we investigate the potential of leveragingGANs to generate sequences of discrete tokens, where text genera-tion is the most typical scenario [18]. We first reveal the problemof exposure bias of the traditional maximum likelihood (MLE) es-timation methods and some previous solutions such as scheduledsampling [2], then we discuss the advantages of GAN techniquesfor text generation tasks. Many recently proposed techniques willbe introduced and compared in this part, including SeqGAN [27],RankGAN [15], LeakGAN [10], MaliGAN [5], GSGAN [13] etc.,where SeqGAN is considered as the most representative frame-work, as illustrated in Figure 2. We will also make an empiricalcomparison between these models over various evaluation metrics.

Reward

Nextaction

State

MCsearchG D

Generate

True data

Train

G

Real WorldD

Reward

Reward

Reward

Policy Gradient

Figure 2: An illustration of SeqGAN for text generation [27].Compared to one-step generation of discrete token as shownin Figure 1, generating sequence of discrete tokens needs theconsideration of subsequent states, which is nationally for-mulated as a reinforcement learning problem. Here MonteCarlo search is leveraged tomake an unbiased estimation ofstate-action value following the current generation policy.

GANs for Graph/Network Learning (15 mins). Graph or net-work is a complicated discrete data format, based on which thereare various information retrieval related tasks like item recommen-dation, social network analysis, fraud detection, knowledge graphsetc. In this part, we will first briefly introduce network embeddingtechniques and then introduce the recent work on applying GANsto network embedding and the subsequent graph mining tasks likeGraphGAN [23] and KBGAN [4]. The key difference of applyingGANs on graph data and sequence data lies in the token (node) sam-pling strategies as illustrated in Figure 3. The solutions on graphdata make use of the graph neighborhood information to higherthe sampling efficiency.

Future Prospective and Summarization (20 mins). At the endof the tutorial, we raise several unsolved problems and promisingfuture research directions of GANs for IR tasks, e.g., cooperativetraining (CoT) [17] and generation evaluation methods [29], arediscussed. We finally summarize this tutorial by highlighting thekey factors and taking home messages of this lecture.

Q&A Session (15 mins).We communicate with the audience byquestion answering, where we could also share some experienceof conducting experiments of GANs and reinforcement learningtechniques for IR tasks.

Figure 3: An illustration of GraphGAN for neighbor nodegeneration (selection) [23]. Given a target node vc , for eachstep the generativemodel selects another node which is sup-posed to be linked to vc and the discriminative model istrained to distinguish the real node pairs and the generatedones.

4 SUPPORTING MATERIALS4.1 Online MeterialsA website (http://wnzhang.net/tutorials/sigir2018/) for this tutorialwill be made available online right before the lecture is presented.All the relevant materials will be made available on this website,including the talk information, presentation slides, referred papers,speaker information and related open source projects etc.

4.2 Offline MaterialsWe may prepare paper handouts for each audience in the lecture.This depends on the cost of printing in Ann Arbor.

4.3 Open Sourced PlatformsSome open sourced platforms will be introduced to the audience toencourage them to run simple code of GANs for IR tasks.• IRGAN [24] (https://github.com/geek-ai/irgan), a code frame-work of GANs for discrete data generation based on REIN-FORCE algorithms to support web search, item recommenda-tion and question answering tasks.

• Texygen [29] (https://github.com/geek-ai/Texygen), a bench-marking platform, to support research on open-domain textgeneration models. Texygen has not only implemented a major-ity of text generation models, but also covered a set of metricsthat evaluate the diversity, the quality and the consistency ofthe generated texts.

• GraphGAN [23] (https://github.com/hwwang55/GraphGAN),a repository of the recent solutions for learning node embed-ding, link prediction and node classification based on GANtechniques.

5 RELEVANCE TO THE INFORMATIONRETRIEVAL COMMUNITY

Apparently, this tutorial is highly relevant to SIGIR 2018 referredtopics and the information retrieval community. As described inSection 3.2, the core studied problem is the deep generative anddiscriminative models in information retrieval, which is highly rel-evant to Artificial Intelligence, Semantics, and Dialog track in SIGIR

2018; the discussed application scenarios on which the machinelearning models are based including web search, recommendersystems, question answering, text composing, social network anal-ysis etc., which are the cared topics in Search and Ranking trackand Content Recommendation, Analysis and Classification track; theoverall adversarial learning framework itself is a novelty, whichwould be insightful to the audience from Foundations and FutureDirections track.

Past relevant tutorials in top-tier conferences include:• Hang Li, Zhengdong Lu. Deep Learning for Information Re-trieval. SIGIR 2016.

• Tom Kenter, Alexey Borisov, Christophe van Gysel, MostafaDehghani, Maarten de Rijke, Bhaskar Mitra. Neural Networksfor Information Retrieval (NN4IR). SIGIR 2017.

• Bhaskar Mitra, Nick Craswell. Neural Text Embeddings forInformation Retrieval. WSDM 2017.

Although these tutorials cover deep learning techniques and theirapplications on IR, they focus on discriminative models or singlegenerative models rather than adversarial training mechanismsfor IR. To our knowledge, this is the first tutorial focused on GANtechniques for IR.

6 SPEAKER INFORMATION ANDQUALIFICATION

The tutorial speaker, Dr. Weinan Zhang, is current a tenure-trackassistant professor in Shanghai Jiao Tong University. His researchinterests include machine learning and big data mining, partic-ularly, deep learning and reinforcement learning techniques forreal-world data mining scenarios, such as computational advertis-ing, recommender systems, text mining, web search and knowledgegraphs. He was selected as one of the 20 rising stars of KDD re-search community in 2016. His work IRGAN won SIGIR 2017 BestPaper Honorable Mention Award. He and his teammates won the3rd place in KDD-CUP 2011 for Yahoo! Music recommendationchallenge and the final champion in 2013 global RTB advertisingbidding algorithm competition. Weinan earned his Ph.D. from Uni-versity College London in 2016 and Bachelor from ACM Class ofShanghai Jiao Tong University in 2011. He was an intern at Google,Microsoft Research and DERI.

As for presenting such a tutorial on GANs for IR, Weinan Zhangwould be qualified because (i) he is one of the major authors ofIRGAN [24], which is the first fundamental research work on solveIR problems with a GAN solution; (ii) he leads the research projectsof SeqGAN [27], LeakGAN [10] and Texygen [29], which are amongthe most advanced researches for text generation; (iii) he is alsoan author of GraphGAN [23], which is the first work on nodeembedding learning based on graph data.

Besides, Weinan Zhang has led or participated several tutorials:• Weinan Zhang, Jian Xu. Learning, Prediction and Optimisationin RTB Display Advertising. CIKM 2016.

• JunWang, Shuai Yuan,Weinan Zhang. Real-Time Bidding basedDisplay Advertising: Mechanisms and Algorithms. ECIR 2016.

Acknowledgment. The work is sponsored by National NaturalScience Foundation of China (61632017, 61702327, 61772333), Shang-hai Sailing Program (17YF1428200), MSRA Collaborative Researchand DIDI Collaborative Research Funds.

REFERENCES[1] Bahriye Akay, Oğuz Kaynar, and Ferhan Demirkoparan. 2017. Deep learning

based recommender systems. In UBMK. IEEE, 645–648.[2] Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled

sampling for sequence prediction with recurrent neural networks. In Advancesin Neural Information Processing Systems. 1171–1179.

[3] Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton,and Greg Hullender. 2005. Learning to rank using gradient descent. In Proceedingsof the 22nd international conference on Machine learning. ACM, 89–96.

[4] Liwei Cai and William Yang Wang. 2017. KBGAN: Adversarial Learning forKnowledge Graph Embeddings. arXiv preprint arXiv:1711.04071 (2017).

[5] Tong Che, Yanran Li, Ruixiang Zhang, R Devon Hjelm, Wenjie Li, Yangqiu Song,and Yoshua Bengio. 2017. Maximum-Likelihood Augmented Discrete GenerativeAdversarial Networks. arXiv preprint arXiv:1702.07983 (2017).

[6] Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networksfor youtube recommendations. In Proceedings of the 10th ACM Conference onRecommender Systems. ACM, 191–198.

[7] Ian Goodfellow. 2016. NIPS 2016 tutorial: Generative adversarial networks. arXivpreprint arXiv:1701.00160 (2016).

[8] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley,Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarialnets. In Advances in neural information processing systems. 2672–2680.

[9] Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016. A deep relevancematching model for ad-hoc retrieval. In Proceedings of the 25th ACM Internationalon Conference on Information and Knowledge Management. ACM, 55–64.

[10] Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Jun Wang, and Yong Yu. 2017.Long Text Generation via Adversarial Training with Leaked Information. arXivpreprint arXiv:1709.08624.

[11] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-SengChua. 2017. Neural collaborative filtering. In Proceedings of the 26th InternationalConference on World Wide Web. International World Wide Web ConferencesSteering Committee, 173–182.

[12] Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterizationwith gumbel-softmax. ICLR (2017).

[13] Matt J Kusner and José Miguel Hernández-Lobato. 2016. Gans for sequencesof discrete elements with the gumbel-softmax distribution. arXiv preprintarXiv:1611.04051 (2016).

[14] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature521, 7553 (2015), 436.

[15] Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming-Ting Sun. 2017.Adversarial Ranking for Language Generation. arXiv preprint arXiv:1705.11001(2017).

[16] Tie-Yan Liu et al. 2009. Learning to rank for information retrieval. Foundationsand Trends® in Information Retrieval 3, 3 (2009), 225–331.

[17] Sidi Lu, Lantao Yu, Weinan Zhang, and Yong Yu. 2018. CoT: Cooperative Trainingfor Generative Modeling. arXiv preprint arXiv:1804.03782 (2018).

[18] Sidi Lu, Yaoming Zhu, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Neural TextGeneration: Past, Present and Beyond. arXiv preprint arXiv:1803.07133 (2018).

[19] Lin Ma, Zhengdong Lu, and Hang Li. 2016. Learning to Answer Questions fromImage Using Convolutional Neural Network.. In AAAI, Vol. 3. 16.

[20] Jay M Ponte and W Bruce Croft. 1998. A language modeling approach to in-formation retrieval. In Proceedings of the 21st annual international ACM SIGIRconference on Research and development in information retrieval. ACM, 275–281.

[21] Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. 2014.Learning semantic representations using convolutional neural networks for websearch. In Proceedings of the 23rd International Conference on World Wide Web.ACM, 373–374.

[22] Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et al.1999. Policy Gradient Methods for Reinforcement Learning with Function Ap-proximation.. In NIPS. 1057–1063.

[23] Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, FuzhengZhang, Xing Xie, and Minyi Guo. 2017. GraphGAN: Graph RepresentationLearning with Generative Adversarial Nets. AAAI (2017).

[24] Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, PengZhang, and Dell Zhang. 2017. Irgan: A minimax game for unifying generativeand discriminative information retrieval models. In SIGIR. ACM, 515–524.

[25] Ying Wen, Weinan Zhang, Rui Luo, and Jun Wang. 2016. Learning text represen-tation using recurrent convolutional neural network with highway layers. NeuIRWorkshop (2016).

[26] Ronald J Williams. 1992. Simple statistical gradient-following algorithms forconnectionist reinforcement learning. Machine learning 8, 3-4 (1992), 229–256.

[27] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. SeqGAN: SequenceGenerative Adversarial Nets with Policy Gradient.. In AAAI. 2852–2858.

[28] Chengxiang Zhai and John Lafferty. 2004. A study of smoothing methods for lan-guage models applied to information retrieval. ACM Transactions on InformationSystems (TOIS) 22, 2 (2004), 179–214.

[29] Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, andYong Yu. 2018. Texygen: A Benchmarking Platform for Text Generation Models.In SIGIR.