relative attributes

Download Relative attributes

Post on 21-Jun-2015

4.240 views

Category:

Documents

4 download

Embed Size (px)

TRANSCRIPT

  • 1. 2011/11/27 CVICCV2011 takmin

2. Relative Attributes Devi Parikh (Toyota Technological Institute Chicago) Kristen Grauman (University of Texas at Austin) 3. /A B CD E F 4. /A B CD E F 5. BCNaturalBANatural EFSmilingEDSmiling A BC D EF 6. Relative Attributes 7. Relative Attributes Zero-Shot Learning / 8. Relative Attributes mOm : ,mSm : , 9. Relative Attributes mwmrm (xi ) w x T m i i (i, j ) Om : wT xi wT x j m m(i, j ) Sm : wT xi wT x j m m 10. Relative Attributes mwmrm (xi ) w xTm i i (i, j ) Om : wT xi wT x j 1 ij m m ij 0 (i, j ) Sm : wT xi wT x j ij m m ij 0 11. Relative Attributes (SVM)*m 1 w arg min w m C22ij 2 ij wm 2(i, j ) Om : wT (xi x j ) 1 ij m ij 0(i, j ) Sm : wT (xi x j ) ij m ij 0 12. Relative Attributes (SVM)Binary Attributes Relative Attributes 13. Zero-Shot Learning From Relationships N S U S(Seen) mAttributes ex. U(Unseen) SAttributesAttribute mc (s)(u )c (s)cc(u ) q (s)c r c (ps ) cqu )(pq rUqSpUqS UqSSr p p 14. Zero-Shot Learning From RelationshipsSeenUnseen 15. Zero-Shot Learning From Relationships (Seen)1. SeenRelative Attributesrm (xi ) w x T m i 16. Zero-Shot Learning From Relationships (Seen)1. SeenRelative Attributesrm (xi ) w x T m i 2. Seen~ r (x ),, r (x ),, r (x )Txi1imiMi 17. Zero-Shot Learning From Relationships (Seen)1. SeenRelative Attributes rm (xi ) w x Tm i2. Seen~ r (x ),, r (x ),, r (x )Txi1imiMi3. Seenc (ps ) N ( (ps ) , (ps ) ) c (s ) ~i p xc (s ) ~i p x 18. Zero-Shot Learning From Relationships (Unseen)4. Unseen c (ps ) cqu ) cr( s ) (1 (s) (u )qm ( pm rm) ) (s2 c (ps ) cqu ) ( qm) (pm d m(us)1 S (s) (ju ) i cqu ) cr( s ) (S i 1 qm) rm) d m(u (s 1 (u ) qm rm (xi ) S iS 19. Zero-Shot Learning From Relationships (Unseen)4. Unseen c (ps ) cqu ) cr( s ) ( 1 (s) (u ) qm ( pm rm) )(s 2mpr 20. Zero-Shot Learning From Relationships (Unseen)4. Unseen c (ps ) cqu ) cr( s ) ( 1 (s) (u ) qm ( pm rm) )(s 2 c (ps ) cqu ) ( qm) (pm d m(us)1 d m rm (xi ) (pms)S iSmSeenp 21. Zero-Shot Learning From Relationships (Unseen)4. Unseen c (ps ) cqu ) cr( s ) ( 1 (s) (u ) qm ( pm rm) )(s 2 c (ps ) cqu ) ( qm) (pm d m(us) cqu ) cr( s ) ( qm) rm) d m(u (s 22. Zero-Shot Learning From Relationships (Unseen)4. Unseen c (ps ) cqu ) cr( s ) (1 (s) (u )qm ( pm rm) ) (s2 c (ps ) cqu ) ( qm) (pm d m(us) cqu ) cr( s ) ( qm) rm) d m(u (s 1 (u ) qm rm (xi ) mSeen S iS 23. Zero-Shot Learning From Relationships (Unseen)4. Unseen c (ps ) cqu ) cr( s ) (1 (s) (u )qm ( pm rm) ) (s2 c (ps ) cqu ) ( qm) (pm d m(us)1 S (s) (ju ) i cqu ) cr( s ) (S i 1 qm) rm) d m(u (s Seen 1 (u ) qm rm (xi ) S iS 24. Zero-Shot Learning From Relationships 1. rm (xi ) wT xim~ r (x ),, r (x ),, r (x )Txi1imiMi2. c* arg max P(~i | p , p ) xp1,, N 25. Zero-Shot Learning From Relationships x 26. Describing Images in Relative Terms Relative AttributesAmBCABCABAC1/8 27. Describing Images in Relative Terms 28. 1. Outdoor Scene Recognition (OSR) Dataset 2688 / 512gistSpatial Envelope2. Public Figure Face Database (PubFig) 800 / 8 gist45Lab 29. AttributesT: tall-building I : inside-city S: streetH: highwayC: coastO: open-countryM: mountainF: forestA: Alex RodriguezC: Clive OwenH: Hugh Laurie J : Jared LetoM: Miley CyrusS : Scarlett JohanssonV: Viggo MortensenZ: Zac Efron 30. Attributes SVMBinary AttributesRelative Attributes/ BinaryRelativeAttributesAttributesOSR 80% 89%PubFig67% 82% 31. Zero-Shot Learning Direct Attribute Prediction (DAP) model C. H. Lampert et al., Learning To Detect Unseed Object Class byBetween-Class Attribute Transfer, CVPR2009 Score-based Relative Attributes (SRA)wT m x i SeenUnseen Seen UnseenSeen 32. Zero-Shot Learning Unseen Seen + Unseen = 8Unseen 33. Zero-Shot Learning Seen 34. Zero-Shot Learning Unseen DAP 35. Zero-Shot LearningUnseenSeen 36. Describing Images 37. Describing Images 18 20PubFig10OSR 38. 39. Describing Images 40. Relative AttributesOSR Binary Image Relative descriptions descriptionsmore natural than tallbuilding, less natural than forest not natural more open than tallbuilding, less open than coastnot openmore perspective than tallbuilding perspective not naturalmore natural than insidecity, less natural than highwaynot open more open than street, less open than coast perspectivemore perspective than highway, less perspective thaninsidecity more natural than tallbuilding, less natural than natural mountainopen more open than mountain perspectiveless perspective than opencountry 41. Relative AttributesPubFig Binary Image Relative descriptionsdescriptions more White than AlexRodriguezWhitemore Smiling than JaredLeto, less Smiling than not SmilingZacEfron VisibleForeheadmore VisibleForehead than JaredLeto, lessVisibleForehead than MileyCyrusmore White than AlexRodriguez, less WhiteWhite than MileyCyrus not Smilingless Smiling than HughLaurienot more VisibleForehead than ZacEfron, less VisibleForeheadVisibleForehead than MileyCyrus more Young than CliveOwen, less Young than ScarlettJohansson not Youngmore BushyEyebrows than ZacEfron, less BushyEyebrows BushyEyebrows than AlexRodriguez RoundFace more RoundFace than CliveOwen, less RoundFace than ZacEfron 42. Relative Attributes Zero-Shot Learning Zero-Shot Learning 43. Q&A