describing images using attributes. describing images farhadi et.al. cvpr 2009
TRANSCRIPT
![Page 1: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/1.jpg)
Describing Images Using Attributes
![Page 2: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/2.jpg)
Describing Images
Farhadi et.al. CVPR 2009
![Page 3: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/3.jpg)
No examples from these object categories were seen during training
Describing Objects by their Attributes
Farhadi et.al. CVPR 2009
![Page 4: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/4.jpg)
Absence of typical attributes
752 reports
68% are correct Farhadi et.al. CVPR 2009
![Page 5: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/5.jpg)
Presence of atypical attributes
951 reports47% are correct Farhadi et.al. CVPR 2009
![Page 6: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/6.jpg)
NormalitySaleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
![Page 7: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/7.jpg)
Abnormal Object DatasetSaleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
![Page 8: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/8.jpg)
Abnormality Prediction and RankingMethod AUC
One class SVM 0.5980
Two class SVM 0.8657
Graphical Model 0.8703
Our Model with surprise score
0.9105
Less Abnormal High Abnormal
• Based on Abnormality Score, we can classify an object as Normal vs. Abnormal.
• Also, using this score we are able to rank images based on how strange they look like.
Saleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
![Page 9: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/9.jpg)
Reasoning about Abnormality via Attributes
Saleh et. al. Object Centric Anomalty Detection by Attribute-Based Reasoning, CVPR13
![Page 10: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/10.jpg)
Describing Objects
• Detector input– Strongest category response with good overlap– Strongest part response within each spatial bin
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
![Page 11: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/11.jpg)
Describing Objects
• Learn spatial correlations and co-occurrence
Detector Responses
True Value for Categories and Spatial
Parts
Has PartHas Function
Pose/Viewpoint
Latent “Root”
Learned by EM in training
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
![Page 12: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/12.jpg)
animal
function: can bitefunction: can fly
part: eyepart: footpart: headpart: legpart: mouthpart: tailpart: wing
Pose: objects_front
Animalblc: eaglefunction: can bitefunction: can flyfunction: is predatorfunction: is carnivorouspart: eyepart: footpart: headpart: legpart: mouthpart: wingPose: extended_wingsPose: objects_front
Describing Familiar Objects
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
![Page 13: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/13.jpg)
Using Localized Attributes
Vehicle
Wheel
Animal
Leg
HeadFour-leggedMammal
Can runCan JumpIs HerbivorousFacing right
Moves on roadFacing right
Farhadi et. al, Attribute-Centric Recognition for Cross-Category Generalization, CVPR10
![Page 14: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/14.jpg)
Relative (ours):
More natural than insidecity Less natural than highway
More open than street Less open than coast
Has more perspective than highway Has less perspective than insidecity
Binary (existing):
Not natural
Not open
Has perspective
Using Relative Attributes
14Parikh, Grauman, Relative Attributes, ICCV 2011
![Page 15: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/15.jpg)
Relative (ours):
More natural than tallbuilding Less natural than forest
More open than tallbuilding Less open than coast
Has more perspective than tallbuilding
Binary (existing):
Not natural
Not open
Has perspective
Using Relative Attributes
15Parikh, Grauman, Relative Attributes, ICCV 2011
![Page 16: Describing Images Using Attributes. Describing Images Farhadi et.al. CVPR 2009](https://reader035.vdocuments.mx/reader035/viewer/2022081505/551b6fb3550346d6338b4db0/html5/thumbnails/16.jpg)
Relative (ours):
More Young than CliveOwenLess Young than ScarlettJohansson
More BushyEyebrows than ZacEfron Less BushyEyebrows than AlexRodriguez
More RoundFace than CliveOwenLess RoundFace than ZacEfron
Binary (existing):
Not Young
BushyEyebrows
RoundFace
Using Relative Attributes
16
(Viggo)
Parikh, Grauman, Relative Attributes, ICCV 2011