quantifying and transferring contextual information in object detection
DESCRIPTION
Quantifying and Transferring Contextual Information in Object Detection. Professor: S. J. Wang Student : Y. S. Wang. Outline. Background Goal Difficulties in Usage of Contextual Information Provided solutions Another method: TAS Experimental Results and Discussion - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/1.jpg)
1
Quantifying and Transferring Contextual Information
in Object Detection
Professor: S. J. WangStudent : Y. S. Wang
![Page 2: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/2.jpg)
2
OutlineBackgroundGoalDifficulties in Usage of Contextual
InformationProvided solutionsAnother method: TASExperimental Results and
DiscussionConclusion and Future Direction
![Page 3: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/3.jpg)
3
Background (I)Only the properties of target
object used in the detection task in the past.◦Problem: Intolerable number of false
positive
![Page 4: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/4.jpg)
4
Background (I)Only the properties of target
object used in the detection task in the past.◦Problem: Intolerable number of false
positive
![Page 5: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/5.jpg)
5
Background (II)What else??? Contextual
information!
![Page 6: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/6.jpg)
6
GoalEstablish a model to efficiently
utilize the contextual information to boost the performance of detection accuracy.
![Page 7: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/7.jpg)
7
Difficulties (I)Diversity of Contextual Information
◦There are may different types of context often co-existing with different degrees of relevance to the detection for the target object(s) in different images.
◦Terminology: Things (e.g. cars and people) Stuffs (e.g. roads and sky) Scene (e.g. what happen in the image)
◦Thing-Thing, Thing-Stuff, Stuff-Stuffand Scene-Thing
![Page 8: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/8.jpg)
8
Difficulties (II)Ambiguity of Contextual
Information◦Contextual information can be
ambiguous and unreliable, thus may not always have a positive effect on object detection.
◦Ex: Crowded Scene with constant movement and occlusion among multiple objects.
![Page 9: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/9.jpg)
9
Difficulties (III)Lack of Data for Context Learning
◦Not enough training data : Over-fitting problem Wrong degree of relevance
◦Ex: The contextual information of people on top of sofa can be more useful than people on top of grass.
![Page 10: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/10.jpg)
10
Training Data Preparation & Notation Representation
Base Detector(HOG)
Training Image Candidate windows
Positive sample: Red
Negative sample: Green
![Page 11: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/11.jpg)
11
Provided SolutionsA polar geometric descriptor for
contextual representation.A maximum margin context model
(MMC) for quantifying context.A context transfer learning model for
context learning with limited data.
![Page 12: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/12.jpg)
12
Polar Geometric Descriptor
Instead of traditional annotation based descriptor, here we use polar geometric descriptor to describe two kind of contextual information (Thing-Thing, Thing-Stuff).
r :orientationb+1 :radial binsr*b+1 :patches0.5σ, σ and 2σ :bin lengthFeature :HOGPatch representation:Bag of Words method using K-means with K = 100
![Page 13: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/13.jpg)
13
Provided SolutionsA polar geometric descriptor for
contextual representation.A maximum margin context model
(MMC) for quantifying context.A context transfer learning model for
context learning with limited data.
![Page 14: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/14.jpg)
14
Quantifying Context (I)Risk function:
![Page 15: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/15.jpg)
15
Quantifying Context (II)Goal = Minimize the Risk function
Minimize L equal to fulfill the following constraint
Hard to be solved, could be replaced by
![Page 16: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/16.jpg)
16
Quantifying Context (III)Maximum Margin Context Model
Add some extra variables and constraints
![Page 17: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/17.jpg)
17
Provided SolutionsA polar geometric descriptor for
contextual representation.A maximum margin context model
(MMC) for quantifying context.A context transfer learning model
for context learning with limited data.
![Page 18: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/18.jpg)
18
Context Transfer LearningTwo Cases:
◦Similar contextual information Ex: Cars and motorbikes
◦Little in common in both appearance and context, but similar level of assistance provided by contextual information. Ex: People and bikes
![Page 19: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/19.jpg)
19
TMMC-I: Transferring Discriminant Contextual Information
Similar context provide the assistance on the learning of w.
![Page 20: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/20.jpg)
20
TMMC-I: Transferring Discriminant Contextual Information
New Constraint:
Modified optimization function:
![Page 21: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/21.jpg)
21
TMMC-II: Transferring the Weight of Prior Detection Score
Similar level of assistance, same weight
![Page 22: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/22.jpg)
22
TMMC-II: Transferring the Weight of Prior Detection Score
New Constraint:
Modified optimization function:
![Page 23: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/23.jpg)
23
Another Method: TAS
![Page 24: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/24.jpg)
24
Another Method: TAS (I)Steps:1. Segmenting image
into regions.2. Use base-detector
to get the candidate patches.
3. Establish the relationships between candidate patches and regions.
4. Use the relationships to judge there is a target object in the patch or not.
![Page 25: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/25.jpg)
25
Another Method: TAS (II)Region clusters:
![Page 26: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/26.jpg)
26
Another Method: TAS (III)Examples of experiment:
![Page 27: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/27.jpg)
27
Experimental Result and Discussion
Use four data sets for testing:◦VOC 2005◦VOC 2007◦I-LIDS◦FORECOURT
![Page 28: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/28.jpg)
28
Experimental Result and Discussion
![Page 29: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/29.jpg)
29
Experimental Result and Discussion
![Page 30: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/30.jpg)
30
Experimental Result and DiscussionContext Transfer Learning
Models:
![Page 31: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/31.jpg)
31
Experimental Result and DiscussionContext Transfer Learning
Models:
![Page 32: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/32.jpg)
32
Conclusion and Future Direction
In this paper, the author proposes a contextual information model to quantify and select useful context information to boost the detection performance.
What can we do next?◦HOG feature not suits for stuff (e.g. sky, road)◦Automatic selection between TMMC-I, TMMC-
II◦Automatic selection between target object
category and source category
![Page 33: Quantifying and Transferring Contextual Information in Object Detection](https://reader035.vdocuments.mx/reader035/viewer/2022062812/568162a2550346895dd31d49/html5/thumbnails/33.jpg)
33
ReferenceWei-Shi Zheng, Member, IEEE,
Shaogang Gong, and Tao Xiang, ”Quantifying and Transferring Contextual Information in Object Detection ”, PAMI accepted.
Geremy Heitz, Daphne Koller, “ Learning Spatial Context: Using Stuff to Find Things”, ECCV 2008.
Youtube Search “Hard-Margin SVM”