QeCSO: Design of hybrid Cuckoo Search based Query expansion model for efficient information retrievalQeCSO: Design of hybrid Cuckoo Search based Query expansion model for efficient information retrieval
J FELICIA LILIAN* , K SUNDARAKANTHAM and S MERCY SHALINIE
Department of Computer Science and Engineering, Thiagarajar College of Engineering, Madurai, India
e-mail: jfelicia02@gmail.com
MS received 1 February 2020; revised 20 January 2021; accepted 17 July 2021
Abstract. The web contains lots of information that gets updated every second. Searching for a relevant
document from the web needs an efficient scrutinization. As the user’s need varies based on location, intention
and purpose the retrieval of an efficient response is a challenge. To address this challenge an information
retrieval technique has been put forth along with the advent of the machine learning and deep learning models.
We have proposed a QeCSO algorithm to perform an efficient retrieval of relevant response. The Attention-
based Bi-directional LSTM (ATT-BLSTM) helps to improve the retrieval of the relevant document based on its
feature that correlates the semantics between the query and the content. On further expanding the query we can
observe a steep improvement in retrieving the response. To perform this, the output from ATT-BLSTM is given
as an input to the meta-heuristic algorithm called cuckoo search. It helps us to retrieve the exact term to expand
the query and make our search to move closer to an optimal solution. The performance of our approach is
compared to those of other models based on the evaluation metrics such as accuracy and F-measure. It is
evaluated by applying the model over the SQuAD 1.1 dataset. By analyzing the results it is verified that our
proposed algorithm achieves an accuracy of 95.8% for an efficient information retrieval of relevant response
with an increase in F-measure.
Keywords. Information retrieval; query optimization; bi-directional long short-term memory; cuckoo search.
1. Introduction
found to be time-consuming [1]. In general, the query given
by maximum number of users will be of 1–3 word length.
Figure 1 shows, in the report generated by ‘‘Keyword
(2019)’’ [2], the number of terms in the query given by the
users of various languages and from different countries.
There were a lot of research in information retrieval (IR) to
converge the search to an optimal solution for the raised
query [3]. The development of meta-heuristic algorithms
and evolutionary algorithms has taken the research on IR a
step forward [4]. There are two strategies for dealing with
meta-heuristic algorithms. They are exploration and
exploitation. To retrieve the exact response to the user is the
challenge to all IR researchers [5]. Hence we try to incor-
porate the meta-heuristic algorithms to make the search
optimal. Searching using appropriate query terms is what
makes us retrieve the user’s exact need. To make the search
easier and faster, ‘‘query expansion’’ as an approach has
been adopted by various researchers [6].
The terminology query expansion (QE) is the reforma-
tion done on the query to intensify the effectiveness of IR
[6]. QE is categorized into (1) manual, (2) interactive and
(3) automatic [7]. The manual QE is done by the user,
interactive QE is a joint action of both the user and the
system and automatic query expansion (AQE) is done by
the system alone.
methodologies. In accomplishment with AQE the retrieval
of documents is done through soft computing algorithms
[8], such as machine learning (ML), fuzzy logic (FL),
evolutionary computation (EC) and probabilistic reasoning
(PR) [9]. These soft computing methods along with
approximation methods give us an optimal solution to real-
world problems as they are meta-heuristic-based approa-
ches. The meta-heuristic approach is problem independent
and an iterative process that helps the heuristic to delve into
the search space efficiently. The use of meta-heuristic-
based evolutionary computing methods yields an added
advantage for AQE. It takes a local move and a global
move and converges to the best optimal solution. There are
various evolutionary algorithms for QE: to name a few, ant
colony optimization [10], cuckoo search optimization [11]
and particle swarm optimization [12]. In comparison with
the optimization methods, we have considered cuckoo
search optimization for its simplicity [13]; it holds fewer
parameters and thus it has been proved that it outperforms*For correspondence
Sådhanå (2021) 46:181 Indian Academy of Sciences
https://doi.org/10.1007/s12046-021-01706-0Sadhana(0123456789().,-volV)FT3](0123456789().,-volV)
the particle swarm optimization and ant colony optimiza-
tion techniques by Yang and Deb [14], it speeds up to
converge [14], it can handle multi-criteria optimization
problems [15] and it can be combined with another swarm-
intelligence-based optimization algorithm [13]. On looking
into these advantages, we have considered the cuckoo
search optimization to perform AQE.
To the best of our knowledge, there are few works that
specifically address the retrieval based on user’s intention.
We thus focus our research towards the improvement in the
retrieval of response based on context with the user’s
intention. To develop such a system we have adopted soft
computing techniques along with the deep learning algo-
rithm. This tries to overlook a huge dataset and thus makes
the response relevant.
The organization of the paper is as follows: In section 2,
the key contributions of the paper are listed; section 3
discusses a few terminologies related to AQE; section 4
describes the proposed work in handling AQE along with
soft computing techniques; section 5 shows the experi-
mental results and finally in section 6, the conclusion and
future work are given.
1. We have used the Glove Word embedding and have
learned the context information of the sentence.
2. We have retrieved the candidate set (relevant docu-
ments) using similarity function and generated the
unique terms from the candidate set through the pre-
processing stages.
expanding the query.
3. Related work
tion technique that is used to analyze and represent the
text in a natural way [16]. When we start dealing with
texts, the main idea is the language for which we build
the system [17]. Language depends on the place and
people, which in turn deals with syntax and semantic
representation of text in NLP. The basic pyramid to
understand the language is Morphology, Syntax, Seman-
tics and Pragmatics [18].
The research on NLP started with hand-coding with a set
of rules framed by statistical models [19]. During the 1980s
and 1990s, NLP took its shoe with the ML paradigm [20].
To analyze and understand the text we need a huge amount
of corpus as an input feature. Hence, there were develop-
ments raised in building the corpus with more challenges by
comparing with human performance. The datasets based on
various applications on NLP are Project Gutenberg [21],
Brown University Standard Corpus [22] and Google one
Billion word corpus [23] for language modelling applica-
tion, Reuters Newswire Topic Classification [24] for
Classification of Text applications, IMDB Movie Review
[25] for Sentiment Analysis, SQuAD 1.1 & 2.0 [26, 27],
CoQA [28], TriviaQA [29], DuoRC [30], Deepmind QA
[31] for Question Answering (QA), LDC [32], Europarl
[33], HindEnCorp [34] for Machine Translation and so on.
On increase in the neural network community, embedding
Figure 1. Number of terms used for searching the web—a report from ‘‘https://www.keyworddiscovery.com/keyword-stats.html’’.
181 Page 2 of 11 Sådhanå (2021) 46:181
intelligence into NLP tasks helps the machines to behave
like humans.
However the working of QA system is based on query
and response. The machine mimics features of humans to
respond as humans; this is becoming a booming area for
researchers, industries and academicians [35]. The inven-
tion of chatbots from the 1950s to date is shown in figure 2.
This is by the advancements laid in the ML environment
with Artificial Neural Networks (ANN) [36]. To understand
the context between the textual data, Word embeddings
were developed. The Word embeddings such as Continuous
Bag of Words (CBoW), Skip Gram (SG), an architecture of
Word2Vec embedding [37], Glove embedding [38], Elmo
embedding [39] and Infersent Sentence embedding [40] are
presented in the recent works with neural network. These
embeddings will let us understand the language and the
words.
The research is focused more onto the QA system. There
are various models built with various datasets in QA. The
deep learning models such as Recurrent Neural Networks
[41], Long Short-Term Memory [42, 43]/Gated Recurrent
Unit (RNN) [44] and Attention-Mechanism-based QA
models [45, 46] have produced good results in QA system.
The inclusion of the attention mechanism has brought great
insight into understanding the content with query [47, 48].
This was first brought into research for QA in Bi-Direc-
tional Attention Flow (BiDAF) [49] model for the SQuAD
dataset.
To reduce the retrieval of the candidate set, i.e. number
of relevant documents to the query, QE methodology is
enforced using optimization algorithms [50]. The QE
enables us to expand the query based on relevant terms
from the documents retrieved [51]. There were various
methodologies handled for QE; one such model was tried
by [51] using ANN with Word embedding models. When
considering huge millions of data, deep learning algorithms
perform well when compared with ANN as discussed by
[52]. [53] uses term frequency for Arabic QE along with a
cuckoo search optimization algorithm. The number of
terms is limited and they form a relationship. The opti-
mization algorithm helps to apply an effective weighting
system and helps to move closer to the relevant document.
4. Problem definition
addresses the problem of understanding user’s intention
over a huge dataset is the main part of research in QA
system as shown in figure 3. The SQuAD 1.1 dataset is
considered for our research, which contains 1? million QA
pairs and 500? articles. We have incorporated the use of
Glove embedding, which has been trained over billion of
words globally. This will enhance the contextual under-
standing between the words. The use of attention mecha-
nisms along with the Bi-directional Long Short Term
Memory (ATT-BLSTM) model helps to learn the context
based on understanding. We have also achieved an optimal
response retrieval with the help of using cuckoo search
algorithm. It optimizes the query and retrieves the relevant
response based on user’s intention.
5. Proposed work on QE
Figure 5 shows the step-by-step process of the work flow.
The main motivation behind the development of this hybrid
model is for the reading comprehension task in QA system.
In this approach, the query (Q) and content (C) are given as
input to the model. The model looks into the context of the
text to retrieve the answer. The methodology flow of the
work starts with Word embedding using Glove vector that
generates a meaningful vector for each word.
Then we try to understand the context of the words using
the Attention-based Bi-directional Long Short-Term
Memory (ATT-BLSTM). The output values from ATT-
BLSTM for the content and query are considered for gen-
erating the candidate set (Cs) using cosine similarity. The
generated candidate set (Cs) is split into individual terms
Figure 2. Evolution of conversational agents. Figure 3. Overview of the proposed work.
Sådhanå (2021) 46:181 Page 3 of 11 181
using the text preprocessing stages like stemming,
lemmatization and removal of stop words. These individual
terms are considered as cuckoo eggs and are given to the
proposed hybrid cuckoo search algorithm.
5.1 Embedding of text
At the initial stage, the words are embedded into its vector
form using the Glove model [38]. The Glove model gen-
erates the vector of over 300 dimension for each word
based on the word to word co-occurrence matrix (W). This
matrix is generated based on the probability of the given
word (wi) to the context word (wj). Figure 4 shows the co-
occurrence probability Probij where it is calculated as
Prob(i/j) = wij=wi. It is the probability of the word wj that
appears in the context of the word wi.
The word vector by Glove is based on the syntactic and
semantic forms. The resultant vector is trained over 6 bil-
lion words. In comparison with the other embedding
models, the Glove model outperforms on the similarity
checking of words, analogy words and for tasks on Named
Entity Recognition (NER). This model is used for our work
to generate the embedding for words.
The input to the system is the Question (Q) and the
Content (C) where the entire sentence is decoupled into
individual terms:
Q ¼wq1 ;wq2 ;wq3 ; :::;wqm
C ¼sc1 ; sc2 ; sc3 ; :::; scn
sc1 ¼ws11 ;ws12 ;ws13 ; :::;ws1a
and so on as given in figure 5. The total number of words in
query ’m’ and the words in content ’n’ are added up as N, i.e. ðmþ nÞ, to give the total number of words for
embedding. From these N words we retrieve the vocabulary
size Nvoc, which is unique. The total number of N words are
embedded into a 300-dimension-size vector
Wword Rdim sizejNvocj. Here the dimension size is set as a
hyperparameter. Now, each individual word is transformed
into the embedded vector using Wword ðNvocÞ.
5.2 Learning the context words using ATT-BLSTM
The word vectors are pushed into the ATT-BLSTM as
shown in figure 5 to learn the context of the content and the
query. In general, LSTM holds four main components [42].
They are input gate (ipt), forget gate (fgt), output gate (opt) and the cell state (cst). Each component holds weights and
bias as the hyperparameters such as Wxip , Whfip , bip for input
gate, Wxop ;Whfop ; bop for output gate and Wxfg ;Whffg ; bfg for
forget gate.
The input xt is the embedded value of the word, based on
the input dimension as the weight matrix (W) is considered.
The LSTM as shown in figure 6 is for forward direction
(hft) and the reverse will be in backward direction (hbt). In the forward direction it takes t 1 value, i.e. the previous
state as given in equations (1)–(5), whereas in the backward
direction it takes t þ 1 value, i.e. in the reverse direction.
ipt ¼rðWxipxt þWhfiphft1 þ bipÞ ð1ÞFigure 4. Co-occurrence probability as discussed by Pennington
et al [38].
the search.
181 Page 4 of 11 Sådhanå (2021) 46:181
fgt ¼rðWxfgxt þWhffghft1 þ bfgÞ ð2Þ
opt ¼rðWxopxt þWhfophft1 þ bopÞ ð3Þ
gt ¼tanhðWxgxt þWhfghft1 þ bgÞ ð4Þ
cst ¼ipt gt þ cst1 fgt ð5Þ The output of the LSTM hft is dependent on the cell state
ðcstÞ and it gets removed using the tanh function that drives
the values between –1 and ?1 as given in equation (6):
hft ¼ opt tanhðcstÞ ð6Þ The same is followed for the backward direction and hence
its output will be hbt. The context-based value for each
word from ATT-BLSTM is ht as given in equation (7). The
words in the content are given as hct and for the query, it is
hqt. These values decide the importance of each word in the
content and hence the highly preferred words are retained
with positive to high positive values and the less important
words will hold negative values. These are taken forward
for query optimization.
The output from the ATT-BLSTM generates the impor-
tance of each sentence w.r.t. the query (Q). The candidate
set ðCsÞ is generated by comparing the query and the
content and finding similarity. The top-ranked sentences
ðsiÞ are retrieved using cosine similarity given in equation
(8):
ð8Þ
In all, 50% of the top number of sentences that are ranked
will be retrieved. For example if we have n ¼ 10 sentences
in the content then the 50%, that is 5 top ranking sentences,
is considered as the candidate set ðCsÞ. It will be retrieved
for further processing of QE. When retrieving the Cs we
also consider the cosine value that is calculated between the
sentences. This value should be more than 40; hence we
consider that the sentence could hold the possible answer
for the query.
In general, for achieving efficiency in IR the cuckoo search
optimization algorithm is used. It works as follows:
1. The cuckoo’s eggs (new solution) are laid in the host bird’s nest (solution).
Figure 6. Long Short-Term Memory (LSTM) – forward
direction.
Figure 8. Similarity score for host solution and new solution.
Table 1. Parameters for training the Glove model.
Glove model
Parameter Value
Dimension 300
Sådhanå (2021) 46:181 Page 5 of 11 181
In our proposed work, the number of host nests is the
number of individual terms (l) in the candidate set ðCswi Þ
where i ¼ 1; 2; 3; :::; l. These individual terms are
retrieved through the pre-processing stages such as
tokenization, removal of stop words, lemmatization and
stemming as shown in figure 7.
The retrieved individual terms are used as the host bird’s
solution. Here, l is the number of host nests. The cuckoo
egg is the term in the query (Q), which when appended
with the host bird egg refines the candidate set and
generates an updated set (solution) as given in equation (9)
where Stþ1 i represents the new solution for the ith word and
t represents the updation of solutionw.r.t. iteration; a is the step size and this value is dependent on the problem:
Stþ1 i ¼ Sti þ a LevyðF; kÞ ð9Þ
To figure out the host bird’s nest it takes a Levy flight,
shown as LevyðF; kÞ in equation (10):
LevyðF; kÞ ¼ Fk ð1\k 3Þ ð10Þ where F is the size of the step that is taken from Levy’s
distribution. Levy’s distribution takes values that are
continuous, strictly non-negative and change over a short
duration.
2. If the new solution yields good results then it is replaced in the host nest, else if it is a not-so-good solution then it is thrown off. Here the initial solution in the host nest is the possible
words that can be appended with the original query for
optimization. To these solutions the cuckoo’s egg gets
appended, where it contains the original query. The
generation of a new solution is based on two factors: first is
the original query term (Q) given as the cuckoo egg and the next factor is the new word ðPiÞwhere i ¼ 1; 2; :::; l that is available in the host nest. By appending both factors we
generate a new solution as given in equation (11) and find
out whether the new solution is good or not.
jQþ Pij ¼ ðwq1 ;wq2 ; :::;wqn þ wPi Þ .. .
jQþ Plj ¼ ðwq1 ;wq2 ; :::;wqn þ wPl Þ
ð11Þ
selection of nest. The new solution is updated when the
host is selected at random. This solution is updated into
an index file to obtain all the solutions of the candidate
pool terms as shown in figure 8. Along with the new
solution, the similarity score is also appended using the
equation given in equation (8).
3. Cuckoo lays one egg at a time; it randomly chooses the host nest and places its egg in it. The random selection of the host bird to figure out the
cuckoo egg takes a probability Pr that lies between 0,1.
4. The best solution with good quality is carried over to the next iteration. The best solution is based on the similarity score
ðsimscoreÞ that is calculated. This is considered as the
fitness function, F(Q). The local best fitness function that we calculate for our work is given in eq. (12). The global
best solution is found out by repeating a similar task until
convergence of Pr and a.
FðQiÞ ¼maxðsimscoreðwpi ;CsÞ; simscoreððQþ wpiÞ;CsÞÞ
ð12Þ
5.5 Proposed hybrid cuckoo search algorithm (QeCSO)
All the basic variables of the cuckoo search help us to
retrieve an optimal solution for our work. The cuckoo
search performance is dependent on the parameters such as
Figure 9. An example query and the content from SQuAD1.1 dataset to retrieve the top relevant sentences using cosine similarity.
Table 2. Accuracy comparison with other state-of-art models for
SQuAD1.1 dataset.
number of host nests, maximum iteration, stop criteria and
fitness function. The cuckoo search algorithm as proposed
in [14] is retrieved for our purpose and the parameter
calculations are performed as given in the previous steps.
The pseudo-code of the proposed QeCSO algorithm 1 is as
follows:
Sådhanå (2021) 46:181 Page 7 of 11 181
The variables that play the major role in algorithm 1 are
the Pr and the Levy flight value. Based on these we obtain
the local best and the global best solutions that will help us
to figure out the best words to append with the query. These
variables also help us to converge the algorithm.
6. Implementation and results
The dataset that we have used is SQuAD 1.1 [26] to per-
form the QA experiments. From this we have randomly
selected 1000 questions and answers from 52 articles to
examine the performance of the QA system along with the
AQE approach; 70% of the dataset is set for training and
30% for testing. As an initial work, we have concentrated to
obtain embedding of the words using the Glove model with
a dimension of 300. We retrieve the most similar words
following the word from the given content. The parameters
that are defined are given in Table 1.
In order to evaluate the performance of the proposed
algorithm, we have set up the environment with a hardware
that contains a Nvidia Tesla P100 12 GB GPU that supports
CUDA Cores - 3584, Memory clock - 715 MHz/1430 Mbps
and Bus - PCI Express 3.0 16; this helps in processing
huge datasets; the software is python2.7, which helps in
using the inbuilt deep learning functions.
Based on the afore-mentioned parameter tuning, the
Glove model is trained for the given corpus and the
embedding is performed. The embedded content is given to
the ATT-BLSTM model with the activation function as
softmax, the optimizer as rmsprop and the loss function as
categorical_crossentropy. The hyperparameters used for
training are learning rate of 0.1, minibatch size of 30 and
the dropout performs well at 0.3 and 0.5; the other
parameters are tuned as and when needed. The comparison
between the query and the content is done based on the
context and the top-ranking sentences are retrieved for QE.
The cosine similarity function is used to retrieve the top
similar sentences to query as given in figure 9. This shows
the sentences that are relevant to the query through con-
textual learning.
The ATT-BLSTM model is compared to the other state-
of-art models for handling the question and answering task
over the SQuAD 1.1 dataset. The models such as RNN [54],
LSTM [55], Shortest Dependency Paths LSTM (SDP-
LSTM) [56], Bidirectional LSTM (BLSTM) [57] and the
proposed ATT-BLSTM obtain an accuracy that is tabulated
in Table 2.
The training and testing accuracy of the models for 50
epochs is shown in figure 10. From this, we can figure out
that ATT-BLSTM performs better than the other models.
The output from ATT-BLSTM is taken and given to the
cuckoo search to figure out the best term for expanding the
query. As for the parameters that we consider for the
Levy’s flight, a is set as 0–1.5 and k is 1–3. Using these
values, we have obtained the F-measure for the top 40
documents and we compare the results with the F-measure
for the original query with the F-measure of the proposed
algorithm as shown in figure 11.
From these graphs, we have obtained the results with an
increase of 8% in F-measure value using the proposed
Figure 11. Comparison of F-measure for top 40 documents.
181 Page 8 of 11 Sådhanå (2021) 46:181
approach and this will help us to retrieve the correct answer
to the user query. This approach is significantly different
from all other approaches for AQE. The contribution of the
paper as mentioned in section 2 has been implemented. The
learning of context-based data using ATT-BLSTM is ana-
lyzed based on accuracy; the top 40 documents are con-
sidered to retrieve the relevant query term for expanding
through the hybrid cuckoo search algorithm.
The proposed algorithm is applied primarily for SQuAD
1.1 benchmarked dataset; for further showcasing with
increase in complexity of question level, the various other
QA datasets such as SQuAD2.0 [27], CoQA [28], Triv-
iaQA-Web [29] and DuoRC [30] have been compared. The
comparison is done over the type of corpus, corpus feature,
F1 score and the Exact Match (EM) score shown in
Table 3.
In this paper, an improved IR technique using hybrid deep
learning and the meta-heuristic algorithm is proposed. It has
been proved that the use of the Attention-based Bi-direc-
tional LSTMmodel using the Glove embedding outperforms
the other models in performing the QA task. On processing
the output from ATT-BLSTM into the proposed hybrid
cuckoo search algorithm the number of relevant documents
gets narrowed down. It then focuses on retrieving the exact
term for QE and brings out the best suitable query. The
proposed work verifies the accuracy of the algorithm that is
trained and tested over the SQuAD 1.1 dataset and also
compares with the other bench-marked datasets. The work
has been evaluated based on the accuracy and F-measure.
The work can be extended further over huge document col-
lection with multimodal datasets and other evolutionary
algorithms to enhance the retrieval of documents.
References
[1] Boushaki S I, Kamel N and Bendjeghaba O (2015) Improved
Cuckoo search algorithm for document clustering. In: Amine
A, Bellatreche L, Elberrichi Z, Neuhold E, Wrembel R (Eds).
Computer Science and Its Applications, CIIA 2015, IFIP
Advances in Information and Communication Technology.
vol. 456. Cham: Springer
https://www.keyworddiscovery.com/keyword-stats.html
[3] J Wang, B Zhou, S Zhou 2016. An improved cuckoo search
optimization algorithm for the problem of chaotic systems
parameter estimation. Comput. Intell. Neurosci. 2959370, 8 [4] Wang Z, Li X, Zhang D and Wu F 2006 A PSO-based web
document query optimization algorithm. In: Mizoguchi R,
Shi Z and Giunchiglia F (Eds.) The Semantic Web, – ASWC 2006, Lecture Notes in Computer Science, vol. 4185. Berlin– Heidelberg: Springer
[5] Carpineto C and Romano G 2012 A survey of automatic
query expansion in information retrieval. ACM Comput. Surv. (CSUR) 44(1): 1
[6] Azad H K and Deepak A 2019 Query expansion techniques
for information retrieval: a survey. Inf. Process. Manag. 56(5): 1698–1735
[7] Sharma D K, Pamula R and Chauhan D S 2019 A hybrid
evolutionary algorithm based automatic query expansion for
enhancing document retrieval system. J. Ambient Intell. Hum. Comput.https://doi.org/10.1007/s12652-019-01247-9
[8] Sharma D K, Pamula R and Chauhan D S 2019 Soft
computing techniques based automatic query expansion
approach for improving document retrieval. In: Proceedings of the 2019 Amity International Conference on Artificial Intelligence (AICAI) February, IEEE, pp. 972–976
[9] Zadeh L A 1994 Fuzzy logic, neural networks, and soft
computing. Commun. ACM 37(3): 77–85
[10] Veningston K and Shanmugalakshmi R 2014 Efficient
implementation of web search query reformulation using
ant colony optimization. In: Proceedings of the International Conference on Big Data Analytics. Cham: Springer,
pp. 80–94
[11] Fister I, Yang X S and Fister D 2014. Cuckoo search: a brief
literature review. In: Cuckoo Search and Firefly Algorithm. Cham: Springer, pp. 49–62
[12] Khennak I and Drias H 2017 An accelerated PSO for query
expansion in web information retrieval: application to
medical dataset. Appl. Intell. 47(3): 793–808 [13] Yang X S and Deb S 2014 Cuckoo search: recent advances
and applications. Neural Comput. Appl. 24(1): 169–174 [14] Yang X S and Deb S 2009 Cuckoo search via Levy flights. In:
Proceedings of the 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC), December, IEEE,
pp. 210–214
[15] Kawam A A and Mansour N 2012 Metaheuristic optimiza-
tion algorithms for training artificial neural networks. Int. J. Comput. Inf. Technol. 1(2): 156–161
Table 3. Results by the proposed model QeCSO over various Machine Reading Comprehension (MRC) datasets are compared.
Dataset Corpus type Year Feature F1 score EM
SQuAD1.1 Textual 2016 Contains unanswerable questions 94.4 95.7
TriviaQA - Web Textual 2017 Complex reasoning 77.9 74.6
SQuAD2.0 Textual 2018 Complex reasoning and contains unanswerable questions 85.8 82.6
DuoRC Textual 2018 Paraphrased paragraph, requires commonsense (world knowledge),
complex reasoning and has unanswerable questions
65.2 63.1
CoQA Textual 2019 Conversational RC and contains unanswerable questions 75.6 73.1
Sådhanå (2021) 46:181 Page 9 of 11 181
[16] Liddy E D 2001 Natural language processing. In: Encyclo- pedia of Library and Information Science, 2nd ed. NY:
Marcel Decker, Inc.
[17] Hirschberg J and Manning C D 2015 Advances in natural
language processing. Science 349(6245): 261–266
[18] Rau L F, Jacobs P S and Zernik U 1989 Information
extraction and text summarization using linguistic knowl-
edge acquisition. Inf. Process. Manag. 25(4): 419–428 [19] Winograd T 1971 Procedures as a representation for data in
a computer program for understanding natural language (No. MAC-TR-84). Massachusetts Institute of Technology,
Cambridge, Project MAC
(computational) linguistics. In: Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous? Association for Computational Linguistics, pp. 3–11
[21] Reck R P and Reck R A 2007 Generating and rendering
readability scores for Project Gutenberg texts. In: Proceed- ings of the Corpus Linguistics Conference, Brimingham, UK
[22] Francis W N and Kucera H 1964 Brown corpus. Department
of Linguistics, Brown University, Providence, Rhode Island
[23] Chelba C, Mikolov T, Schuster M, Ge Q, Brants T, Koehn P
and Robinson T 2013 One billion word benchmark for measuring progress in statistical language modeling. Pre- print arXiv:1312.3005
[24] Dewdney N, VanEss-Dykema C and MacMillan R 2001 The
form is the substance: classification of genres in text. In:
Proceedings of the Workshop on Human Language Tech- nology and Knowledge Management, Association for Com-
putational Linguistics, July 7
[25] Oghina A, Breuss M, Tsagkias M and De Rijke M 2012
Predicting IMDB movie ratings using social media. In:
Proceedings of the European Conference on Information Retrieval, April. Berlin–Heidelberg: Springer, pp. 503–507
[26] Rajpurkar P, Zhang J, Lopyrev K and Liang P 2016 SQuAD: 100,000? questions for machine comprehension of text. Preprint arXiv:1606.05250
[27] Rajpurkar P, Jia R and Liang P 2018 Know what you don’t know: unanswerable questions for SQuAD. Preprint arXiv: 1806.03822
[28] Reddy S, Chen D and Manning C D 2019 CoQA: a
conversational question answering challenge. Trans. Assoc. Comput. Linguistics 7: 249–266
[29] Joshi M, Choi E, Weld D S and Zettlemoyer L 2017
TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. Preprint arXiv:cs.CL/
1705.03551
[30] Saha A, Aralikatte R, Khapra M M and Sankaranarayanan K
2018 Duorc: towards complex language understanding with paraphrased reading comprehension. Preprint arXiv:1804.
07927
[31] Powles J and Hodson H 2017. Google DeepMind and
healthcare in an age of algorithms. Health Technol. 7(4): 351–367
[32] Ma X and Cieri C 2006 Corpus support for machine translation
at LDC. In: Proceedings of LREC, May, pp. 859–864
[33] Koehn P 2005 Europarl: A parallel corpus for statistical
machine translation. In: Proceedings of MT Summit, September, vol. 5, pp. 79–86
[34] Bojar O, Diatka V, Rychly P, Stranak P, Suchomel V,
Tamchyna A and Zeman D 2014 HindEnCorp-Hindi–
English and Hindi-only corpus for machine translation. In:
Proceedings of LREC, May, pp. 3550–3555
[35] Usbeck R, Roder M, Hoffmann M, Conrads F, Huthmann J,
Ngonga-Ngomo A C and Unger C 2019 Benchmarking question answering systems. Semantic Web, (Preprint),
pp. 1–12
[36] Dale R, Moisl H and Somers H (Eds.) 2000 Handbook of natural language processing. CRC Press
[37] Mikolov T, Chen K, Corrado G and Dean J 2013 Efficient estimation of word representations in vector space. Preprint arXiv:1301.3781
[38] Pennington J, Socher R and Manning C 2014 Glove: global
vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543
[39] Peters M E, Neumann M, Iyyer M, Gardner M, Clark C, Lee
K and Zettlemoyer L 2018 Deep contextualized word representations. Preprint arXiv:1802.05365
[40] Conneau A, Kiela D, Schwenk H, Barrault L and Bordes A
2017 Supervised learning of universal sentence representa- tions from natural language inference data. Preprint arXiv: 1705.02364
[41] Xu D and Li W J 2016 Full-time supervision based bidirectional RNN for factoid question answering. Preprint arXiv:1606.05854
[42] Olah C 2015 Understanding LSTM networks. Blog [43] Tan M, Santos C D, Xiang B and Zhou B 2015 LSTM-based
deep learning models for non-factoid answer selection.
Preprint arXiv:1511.04108
[44] Chen S, Wen J and Zhang R 2016 GRU-RNN based question
answering over Knowledge Base. In: Proceedings of the China Conference on Knowledge Graph and Semantic Computing, September, Singapore, pp. 80–91
[45] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L,
Gomez A N, Kaiser L and Polosukhin I 2017 Attention is all
you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008
[46] Lilian J F, Sundarakantham K and Shalinie S M 2021 Anti-
negation method for handling negation words in question
answering system. J. Supercomput. 77(5): 4244–4266 [47] Singh A and Kaur M 2019. Intelligent content-based
cybercrime detection in online social networks using cuckoo
search metaheuristic approach. J. Supercomput. 76:
5402–5424
[48] Dua R D, Madaan D M, Mukherjee P M and Lall B L 2019
Real time attention based bidirectional long short-term
memory networks for air pollution forecasting. In: Proceed- ings of the 2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService), IEEE, April, pp. 151–158
[49] Seo M, Kembhavi A, Farhadi A and Hajishirzi H 2016
Bidirectional attention flow for machine comprehension. Preprint arXiv:1611.01603
[50] Dillon J V and Collins-Thompson K 2010 A unified
optimization framework for robust pseudo-relevance feed-
back algorithms. In: Proceedings of the 19th ACM Interna- tional Conference on Information and Knowledge Management, ACM, October, pp. 1069–1078
181 Page 10 of 11 Sådhanå (2021) 46:181
[51] Imani A, Vakili A, Montazer A and Shakery A 2019 Deep
neural networks for query expansion using word embed-
dings. In: Proceedings of the European Conference on Information Retrieval, April. Cham: Springer, pp. 203–210
[52] Wang X, Macdonald C and Ounis I 2020 Deep reinforced query reformulation for information retrieval. Preprint
arXiv:2007.07987
[53] ALMarwi H, Ghurab M and Al-Baltah I 2020 A hybrid
semantic query expansion approach for Arabic information
retrieval. J. Big Data 7(1): 1–19
[54] Ture F and Jojic O 2016 No need to pay attention: simple recurrent neural networks work! (for answering ‘‘simple’’ questions). Preprint arXiv:1606.05029
[55] Sundermeyer M, Schluter R and Ney H 2012 LSTM neural
networks for language modeling. In: Proceedings of the
Thirteenth Annual Conference of the International Speech Communication Association
[56] Xu Y, Mou L, Li G, Chen Y, Peng H and Jin Z 2015
Classifying relations via long short term memory networks
along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, September, pp. 1785–1794
[57] Graves A, Fernandez S and Schmidhuber J 2005 Bidirec-
tional LSTM networks for improved phoneme classification
and recognition. In: Proceedings of the International Con- ference on Artificial Neural Networks, September. Berlin–
Heidelberg: Springer, pp. 799–804
[58] Ma J, Ting T O, Man K L, Zhang N, Guan S U and Wong P
W 2013 Parameter estimation of photovoltaic models via
cuckoo search. J. Appl. Math. 362619
Sådhanå (2021) 46:181 Page 11 of 11 181