adaptive relevance feedback in information retrieval
TRANSCRIPT
![Page 1: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/1.jpg)
{
1
Adaptive Relevance Feedback
in Information RetrievalYuanhua Lv and ChengXiang Zhai(CIKM ‘09)
Date: 2010/10/12Advisor: Dr. Koh, Jia-Ling
Speaker: Lin, Yi-Jhen
![Page 2: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/2.jpg)
2
Introduction Problem Formulation A Learning Approach To Adaptive Relevance
Feedback Experiments Conclusions
Outline
![Page 3: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/3.jpg)
3
Relevance Feedback helps to improve the retrieval performance.
The balance between the original query and feedback information is usually set to a fixed value
This balance parameter should be optimized for each query and each set of feedback documents.
Introduction
![Page 4: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/4.jpg)
4
Three cases to set a larger feedback coefficient: The query is discriminative The feedback documents are discriminative Divergence between a query and its feedback
document is large We assume there is a function B, that can map a
query Q and the corresponding feedback documents J to the optimal feedback coefficient( i.e., = B(Q,J) )
We explore the problem of adaptive relevance feedback in KL-divergence retrieval model and mixture-model feedback method
Problem Formulation
![Page 5: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/5.jpg)
5
Heuristics and Features Discrimination of Query Discrimination of Feedback documents Divergence between Query and Feedback
Documents Learning Algorithm
A Learning Approach To Adaptive Relevance Feedback
![Page 6: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/6.jpg)
6
Query Length Q = “apple ipad case”, |Q| = 3
Entropy of Query Based on top-N result documents F’ QEnt_A = ) =
Clarity of Query Kullback-Leibler divergence of the query
model from the collection model QEnt_R1 = QEnt_R2 =
) = (1-) ) +) , 0.7 QEnt_R3 = QEnt_R4 =
Discrimination of Query
Top-2 result documents F’ ={ }
: : ) 0
: : ) )1
: : , ) )1
![Page 7: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/7.jpg)
7
, , , : , ,
) + ) )
Feedback Length D = {} , |F| = 3
Feedback Radius to measure if feedback documents are
concentrated on similar topics Entropy of Feedback Documents
FBEnt_A = ) =
Clarity of Feedback Documents FBEnt_R1 =
) = (1-) ) +) , 0.7 FBEnt_R2 = FBEnt_R3 = Discrimination of Feedback
documents( judged relevant by the user for feedback )
𝑑1 ,𝑑2,𝑑4
![Page 8: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/8.jpg)
8
Absolute Divergence QFBDiv_A = ) = ,
Relative Divergence QFBDiv_R = : the rank of document d ) : precision of top documents K : a constant
Divergence between Query and Feedback Documents
K=10 { } = 0.3
= 0.21
![Page 9: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/9.jpg)
9
Logistic regression model Its function form: z = We learn these weights from training
data (e.g., past queries) once the weights has been derived for a
particular data set, the equation can be used to predict feedback coefficients for new data sets (i.e., future queries)
Learning Algorithm
feature vector
( Query Length, Entropy of Query, Clarity of Query, Feedback Length, … , )
![Page 10: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/10.jpg)
10
TREC Data set Assume top-10 results were judged by
users for relevance feedback KL-Divergence retrieval model with the
mixture model feedback to get the optimal feedback coefficients for training queries; through trying different feedback coefficient { 0.0, 0.1,…, 1.0 }
ExperimentsExperiment Design
![Page 11: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/11.jpg)
11
ExperimentsSensitivity of Feedback Coefficient
![Page 12: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/12.jpg)
12
ExperimentsFeature Analysis and Selection
![Page 13: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/13.jpg)
13
an example: Weights derived from Terabyte04&05
data
Given a new query, we can predict its feedback coefficient using the formula:
ExperimentsFeature Analysis and Selection
![Page 14: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/14.jpg)
14
Evaluate in three variant cases : Ideal: the training set and the testing set
are in the same domain Toughest: which is dominated by the data
not in the same domain Have sufficient training data in the same
domain, but it is mixed with “noisy” data
ExperimentsPerformance of Adaptive Relevance Feedback
![Page 15: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/15.jpg)
15
ExperimentsPerformance of Adaptive Relevance Feedback
Ideal:
![Page 16: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/16.jpg)
16
ExperimentsPerformance of Adaptive Relevance Feedback
Toughest:
![Page 17: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/17.jpg)
17
ExperimentsPerformance of Adaptive Relevance Feedback
noisy:
![Page 18: Adaptive relevance feedback in information retrieval](https://reader036.vdocuments.mx/reader036/viewer/2022081604/58a839ef1a28ab30658b4dfd/html5/thumbnails/18.jpg)
18
Contributions Propose an adaptive relevance feedback algorithm to
dynamically handle the balance between query and feedback documents
Propose three heuristics to characterize the balance between original query and feedback information
Future work Rely on explicit user feedback for training, how to
adaptively exploit pseudo and implicit feedback Apply on other feedback approach, e.g., Rocchio
feedback, to examine its performance Study more effective and robust features Incorporate negative feedback into the proposed
adaptive relevance feedback method
Conclusions