Relative Attributes

Speaker DengLei

At I-VisionGroup

i - VisionGroup

Devi Parikh & Kristen Grauman

ICCV 2011 Marr Prize

i - VisionGroup

Publication—Devi Parikh

i - VisionGroup

… last 3 years: ICCV 3(one only her) ECCV 1 CVPR 9 IJCV 1 NIPS 1

i - VisionGroup

Outline

Introduction

Algorithms

Experiments

i - VisionGroup

Introduction—Backgrounds

Visual attributes Benefit various recognition tasks Restrict on categorical label Binaries are unnatural

Motivation How to describe middle image Relative description —  one image’s attribute strength with

respect to others E.g. less natural than left, more nature than right Richer mode of communication Allow more detailed human supervision (maybe higher

recognition accuracy) More informative descriptions of novels

i - VisionGroup

Proposal

Steps Training – learn ranking function per attribute Testing – predict the relative strength per attribute on novel

image

New Tasks Build generative model over joint space of ranking output Zero-shot learning relates unseen to seen

E.g. 'bears are furrier than giraffes‘ Enable richer textual description for new images

More precise Tested on faces and natural scenes compared with binaries

i - VisionGroup

Outline

Introduction

Algorithms

Experiments

i - VisionGroup

Algorithms— learning relative attrs

i - VisionGroup

wide-margin {ranking VS binary} classifier

i - VisionGroup

Novel zero-shot learning

Setup N total categories: S seen, U unseen (no images available) S: described relative to each other via attrs (no need all pairs) U: described relative to seen in (subset of ) attrs

Gaussian Test by Max-likehood

i - VisionGroup

Auto gen relative textual desc of images

Img -> Img Img -> Class More info

than bin

i - VisionGroup

Outline

Introduction

Algorithms

Experiments

i - VisionGroup

Experiments

Setup Outdoor Scene Recognition (OSR)

I: 2668, C: 8, Coast, forest, highway, inside-city,

mountain, open-country, street, tall-building Gist

Public Figures Face (PubFig) I: 772, C: 8 Alex, Clive, Hugh, Jared, Miley, Scarlett, Viggo, Zac Concatenated gist and color feature

i - VisionGroup

Database — relative attrs

Marked

By a

colleague

i - VisionGroup

zero-shot learning

i - VisionGroup

i - VisionGroup

i - VisionGroup

Conclusion

Idea to learn relative visual attrs. Two new tasks

Zero-shot learning Img description Based on relative description

i - VisionGroup

Thanks