relevance feedback based on parameter estimation of target distribution k. c. sia and irwin king...
Post on 20-Dec-2015
215 views
TRANSCRIPT
Relevance Feedback Relevance Feedback based on Parameter based on Parameter Estimation of Target Estimation of Target
DistributionDistribution
K. C. SiaK. C. Sia and Irwin King and Irwin KingDepartment of Computer Science & EngineeringDepartment of Computer Science & Engineering
The Chinese University of Hong KongThe Chinese University of Hong Kong
15 May15 May
IJCNN 2002IJCNN 2002
Relevance Feedback Based on Parameter Estimation of Target Distribution
AgendaAgenda
Introduction to content based image Introduction to content based image retrieval (CBIR) and relevance retrieval (CBIR) and relevance feedback (RF)feedback (RF)
Former approachesFormer approaches Tackling the problemTackling the problem
Parameter estimation of target Parameter estimation of target distributiondistribution
ExperimentsExperiments Future works and conclusionFuture works and conclusion
Relevance Feedback Based on Parameter Estimation of Target Distribution
Content Based Image Content Based Image RetrievalRetrieval
How to represent an image?How to represent an image? Feature extractionFeature extraction
Colour histogram Colour histogram (RGB)(RGB) Co-occurrence matrix texture analysisCo-occurrence matrix texture analysis Shape representationShape representation
Feature vectorFeature vector Map images to points in hyper-spaceMap images to points in hyper-space Similarity is based on distance Similarity is based on distance
measuremeasure
Relevance Feedback Based on Parameter Estimation of Target Distribution
Feature Extraction ModelFeature Extraction Model
R
B
G
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance FeedbackRelevance Feedback
Relevance feedbackRelevance feedback Architecture to capture user’s target Architecture to capture user’s target
of searchof search Learning processLearning process
Two stepsTwo steps FeedbackFeedback – how to learn from the – how to learn from the
user’s relevance feedbackuser’s relevance feedback DisplayDisplay – how to select the next set of – how to select the next set of
documents and present to userdocuments and present to user
Relevance Feedback Based on Parameter Estimation of Target Distribution
1st iteration
UserFeedback
Display
2nd iteration
Display
UserFeedback
Estimation &Display selection
Feedbackto system
Relevance Feedback Based on Parameter Estimation of Target Distribution
Former ApproachesFormer Approaches
Multimedia Analysis and Retrieval System Multimedia Analysis and Retrieval System (MARS)(MARS) Yong Rui et al. Relevance feedback: A powerful tool for Yong Rui et al. Relevance feedback: A powerful tool for
interactive content-based image retrieval. - 1998interactive content-based image retrieval. - 1998 Using weight to capture user’s preferenceUsing weight to capture user’s preference
Pic-HunterPic-Hunter Ingemar J. Cox et al. The Bayesian image retrieval Ingemar J. Cox et al. The Bayesian image retrieval
system, pichunter, theory, implementation, and system, pichunter, theory, implementation, and psychophysical experiments. - 2000psychophysical experiments. - 2000
Images are associated with a probability Images are associated with a probability being the user’s targetbeing the user’s target
Bayesian learningBayesian learning
Relevance Feedback Based on Parameter Estimation of Target Distribution
ComparisonComparison
MARSMARS Pic-HunterPic-Hunter Our approachOur approach
Capturing Capturing user’s target user’s target of searchof search
Weight on Weight on different feature different feature and dimensionand dimension
Probability Probability associated associated with imageswith images
Estimated Estimated parameter of parameter of target cluster target cluster
UpdatingUpdating Counting and Counting and variancevariance
Bayes’ ruleBayes’ rule EM algorithmEM algorithm
DisplayDisplay Most likelyMost likely Maximum Maximum Entropy Entropy PrinciplePrinciple
Maximum Maximum Entropy Entropy PrinciplePrinciple
Relevance Feedback Based on Parameter Estimation of Target Distribution
The ModelThe Model Feature ExtractionFeature Extraction
II - raw image data - raw image data - set of feature extraction method- set of feature extraction method ff - feature extraction operation - feature extraction operation
Images Images data point in hyper-space data point in hyper-space (R(Rdd)) Problem scope is narrowed down to a particular Problem scope is narrowed down to a particular
featurefeature
Relevance Feedback Based on Parameter Estimation of Target Distribution
Inconsistence in FeedbackInconsistence in Feedback
User tells liesUser tells lies
Too many false positive or false Too many false positive or false negativenegative
Conflict of feedback in each Conflict of feedback in each iteration by careless mistakeiteration by careless mistake
Relevance Feedback Based on Parameter Estimation of Target Distribution
Resolving ConflictsResolving Conflicts
How to deal with inconsistent user How to deal with inconsistent user feedback?feedback? Maintain a relevance measure for Maintain a relevance measure for
each data pointseach data points Relevance measure > 0 counted as Relevance measure > 0 counted as
relevant and use in estimationrelevant and use in estimation
Relevance Feedback Based on Parameter Estimation of Target Distribution
Estimating Target Estimating Target DistributionDistribution
User’s target is a cluster User’s target is a cluster Assume it follows a Gaussian Assume it follows a Gaussian
distributiondistribution Model a distribution that fits Model a distribution that fits
the relevant data pointsthe relevant data points Based on the parameterBased on the parameter
of distribution, systemof distribution, systemlearns what user wantslearns what user wants
Data points selected as relevant
Red
Relevance Feedback Based on Parameter Estimation of Target Distribution
Expectation MaximizationExpectation Maximization
Fitting a Gaussian distribution function using Fitting a Gaussian distribution function using feedback data pointsfeedback data points By expectation maximizationBy expectation maximization
Distribution represent user’s targetDistribution represent user’s target Expectation function match the display modelExpectation function match the display model
Relevance Feedback Based on Parameter Estimation of Target Distribution
Updating ParametersUpdating Parameters
Estimated mean is the averageEstimated mean is the average Estimated variance by Estimated variance by
differentiationdifferentiation Iterative approachIterative approach
Relevance Feedback Based on Parameter Estimation of Target Distribution
Maximum Entropy DisplayMaximum Entropy Display
Why maximum entropy display?Why maximum entropy display?
ReasonReason: fully utilize information : fully utilize information contained in user feedback to reduce contained in user feedback to reduce number of feedback iterationnumber of feedback iteration
ResultResult: near boundary images will be : near boundary images will be selected to fine tune parametersselected to fine tune parameters
Relevance Feedback Based on Parameter Estimation of Target Distribution
Maximum Entropy DisplayMaximum Entropy Display
How to simulate maximumHow to simulate maximumentropy display in ourentropy display in ourmodel?model? Data points 1.18 Data points 1.18 away away
from from are selected are selected Why 1.18?Why 1.18?
2P(2P(+1.18+1.18)=P()=P())
Querytargetclustercenter
Selectedby knnsearch
Selectedby Max.Entropy
Relevance Feedback Based on Parameter Estimation of Target Distribution
ExperimentExperiment
Synthetic data generated by MatlabSynthetic data generated by Matlab
Mixture of GaussiansMixture of Gaussians Class label of data points shown for Class label of data points shown for
reference to give feedbackreference to give feedback Dose it works and works better?Dose it works and works better?
Relevance Feedback Based on Parameter Estimation of Target Distribution
ConvergenceConvergence
Is the estimated parameter (mean Is the estimated parameter (mean and variance) converge to the and variance) converge to the actual parameter of target actual parameter of target distribution?distribution?
Is the maximum entropy display Is the maximum entropy display correctly done?correctly done?
Relevance Feedback Based on Parameter Estimation of Target Distribution
Estimated mean RMS error along each iteration
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 6 7 8 9 10
iteration
RM
S e
rro
r o
f es
tim
ated
mea
n 4 dimension
6 dimension
8 dimension
Relevance Feedback Based on Parameter Estimation of Target Distribution
Estimated standard deviation RMS error along iteration
0
0.05
0.1
0.15
0.2
0.25
1 2 3 4 5 6 7 8 9 10
iteration
RM
S e
rro
r o
f es
tim
ated
sta
nd
ard
dev
iati
on
4 dimension
6 dimension
8 dimension
Relevance Feedback Based on Parameter Estimation of Target Distribution
No. of feedback given along each iteration
0
2
4
6
8
10
12
14
16
18
1 2 3 4 5 6 7 8 9
iteration
No
. of
feed
bac
k g
iven
4 dimension
6 dimension
8 dimension
Relevance Feedback Based on Parameter Estimation of Target Distribution
PerformancePerformance
Compares to Rui’s intra-weight Compares to Rui’s intra-weight updating modelupdating model Nearest neighbour search performed Nearest neighbour search performed
after several feedbacks (6-7 after several feedbacks (6-7 iterations)iterations)
Data points outside 2 Data points outside 2 are discarded are discarded in our algorithmin our algorithm
Precision-Recall graphPrecision-Recall graph
Relevance Feedback Based on Parameter Estimation of Target Distribution
Precision vs Recall
0
0.2
0.4
0.6
0.8
1
1.2
0 0.05 0.1 0.15 0.2 0.25 0.3
Recall
Pre
cis
ion
Expectation Maximization
Rui's weight updating
Relevance Feedback Based on Parameter Estimation of Target Distribution
Precision vs Recall
0
0.2
0.4
0.6
0.8
1
1.2
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
Recall
Pre
cis
ion
Expectation Maximization
Rui's weight updating
Relevance Feedback Based on Parameter Estimation of Target Distribution
Precision vs Recall
0
0.2
0.4
0.6
0.8
1
1.2
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Recall
Prec
isio
n
Expectation Maximization
Rui's weight updating
Relevance Feedback Based on Parameter Estimation of Target Distribution
Precision vs Recall
0
0.2
0.4
0.6
0.8
1
1.2
0 0.05 0.1 0.15 0.2 0.25
Recall
Pre
cis
ion
Expectation Maximization
Rui's weight updating
Relevance Feedback Based on Parameter Estimation of Target Distribution
Precision vs Recall
0
0.2
0.4
0.6
0.8
1
1.2
0 0.05 0.1 0.15 0.2 0.25
Recall
Pre
cis
ion
Expectation Maximization
Rui's weight updating
Relevance Feedback Based on Parameter Estimation of Target Distribution
Precision vs Recall
0
0.2
0.4
0.6
0.8
1
1.2
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
Recall
Pre
cis
ion
Expectation Maximization
Rui's weight updating
Relevance Feedback Based on Parameter Estimation of Target Distribution
Future WorksFuture Works
Modification to learn from Modification to learn from information contained in non-information contained in non-relevant setrelevant set
To capture correlation in different To capture correlation in different featuresfeatures
Apply in CBIR system for Apply in CBIR system for performance measurementperformance measurement
Relevance Feedback Based on Parameter Estimation of Target Distribution
ConclusionConclusion
Proposed an approach to interpret Proposed an approach to interpret the feedback information from the feedback information from user and learn his target of searchuser and learn his target of search
Compares our approach with Rui’s Compares our approach with Rui’s intra-weight updating methodintra-weight updating method