context-aware saliency detection

52
Context-Aware Saliency Detection

Upload: kamala

Post on 23-Feb-2016

94 views

Category:

Documents


0 download

DESCRIPTION

Context-Aware Saliency Detection. Introduction Principles of context – aware saliency Detection of context – aware saliency Result Application Conclusion. Introduction. How to describe a figure / picture ? Description: What most people think is Important or Salient . Algorithms. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Context-Aware Saliency Detection

Context-AwareSaliency Detection

Page 2: Context-Aware Saliency Detection

Introduction Principles of context – aware saliency Detection of context – aware saliency Result Application Conclusion

Page 3: Context-Aware Saliency Detection

Introduction How to describe a figure / picture ?

Description:

What most people think is Important or Salient.

Page 4: Context-Aware Saliency Detection

Algorithms First glance: human attention.

EX: auto focusing. Dominant object

EX: object recognition/segmentation. context of the dominant objects:image classification, summarization of a photo collection, thumb nailing, and retargeting.

Page 5: Context-Aware Saliency Detection
Page 6: Context-Aware Saliency Detection

Context-Aware Saliency Detection

Salient regions are distinctive with respect to both their local and global surroundings.

Prioritize regions close to the foci of attention. -> Maintains the background texture.

(Gestalt Law)

Page 7: Context-Aware Saliency Detection

Application Retargeting

Summarization

Page 8: Context-Aware Saliency Detection

Contribution Principles for context-aware saliency

Algorithm

Applicability

Page 9: Context-Aware Saliency Detection

Principles of context-aware saliency

1. Local low-level considerations, including factors such as contrast and color.

2. Global considerations, which suppress frequently occurring features, while maintaining features that deviate from the norm.

Page 10: Context-Aware Saliency Detection

Principles of context-aware saliency

3. Visual organization rules, which state that visual forms may possess one or several centers of gravity about which the form is organized.

4. High-level factors, such as human faces.

Page 11: Context-Aware Saliency Detection
Page 12: Context-Aware Saliency Detection

D.Walther and C. Koch.Modeling attention to salient protoobjects. [2006]

X. Hou and L. Zhang.Saliency detection: A spectral residual approach. [2007]

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum.Learning to Detect A Salient Object. [2007]

Page 13: Context-Aware Saliency Detection
Page 14: Context-Aware Saliency Detection
Page 15: Context-Aware Saliency Detection

Detection of context-aware saliency

Areas that have distinctive colors or patterns should obtain high saliency. -- P1

frequently-occurring features should be suppressed. --P2

The salient pixels should be grouped together, and not spread all over the image. --P3

Page 16: Context-Aware Saliency Detection

Local-global single-scale saliency

Consider a single patch of scale r at each pixel. Thus, a pixel i is considered salient if the appearance of the patch pi centered at pixel i is distinctive with respect to all other image patches.

Page 17: Context-Aware Saliency Detection

Patch image

Page 18: Context-Aware Saliency Detection

𝑫𝒄𝒐𝒍𝒐𝒓 (𝒑𝒊 ,𝒑 𝒋) Euclidean distance between the vectorized

patches and in CIE L*a*b color space.

Page 19: Context-Aware Saliency Detection

𝑫𝒑𝒐𝒔𝒊𝒕𝒊𝒐𝒏(𝒑𝒊 ,𝒑 𝒋) Euclidean distance between the positions of

patches and

Page 20: Context-Aware Saliency Detection
Page 21: Context-Aware Saliency Detection

Dissimilarity This dissimilarity measure is proportional to the difference in appearance and inverse proportional to the positional distance.

Page 22: Context-Aware Saliency Detection

For every patch pi, we search for the K most similar patches in the image.

Page 23: Context-Aware Saliency Detection
Page 24: Context-Aware Saliency Detection

Multi-scale saliency enhancement

Background pixels (patches) are likely to have similar patches at multiple scales.

Incorporating multiple scales to further decrease the saliency of background pixels, improving the contrast between salient and non-salient regions.

Page 25: Context-Aware Saliency Detection

For a patch pi of scale r, we consider as candidate neighbors all the patches in the image.

We choose the K most similar patchs to compute saliency.

Page 26: Context-Aware Saliency Detection

The saliency at pixel i is taken as the mean of its saliency at different scales:

Page 27: Context-Aware Saliency Detection
Page 28: Context-Aware Saliency Detection

Including the immediate context

Gestalt laws:

visual forms may possess one or several centers of gravity about which the form is organized.

Page 29: Context-Aware Saliency Detection

Simulate the visual contextual effect

1. A pixel is considered attended if its saliency value exceeds a certain threshold.

2. each pixel outside the attended areas is weighted according to its Euclidean distance to the closest attended pixel.

Page 30: Context-Aware Saliency Detection
Page 31: Context-Aware Saliency Detection

High-level factors Enhancement factors:

1.Recognized objects2.Face detection

Page 32: Context-Aware Saliency Detection

Results 3 cases: Images show a single salient object over an uninteresting background. Images where the immediate surroundings of the salient object shed light

on the story the image tells. Images of complex scenes.

Compare method: D.Walther and C. Koch. Modeling attention to salient protoobjects. [2006] X. Hou and L. Zhang. Saliency detection: A spectral residual Approach. [2007]

Page 33: Context-Aware Saliency Detection
Page 34: Context-Aware Saliency Detection
Page 35: Context-Aware Saliency Detection
Page 36: Context-Aware Saliency Detection
Page 37: Context-Aware Saliency Detection
Page 38: Context-Aware Saliency Detection
Page 39: Context-Aware Saliency Detection
Page 40: Context-Aware Saliency Detection

Applications Image retargeting

Summarization through collage creation

Page 41: Context-Aware Saliency Detection

Image retargeting Resizing an image by expanding or shrinking

the non-informative regions.

Seam carvingM. Rubinstein, A. Shamir, and S. Avidan.Improved seam carving for video retargeting. [2008]

Context-aware saliency

Page 42: Context-Aware Saliency Detection
Page 43: Context-Aware Saliency Detection
Page 44: Context-Aware Saliency Detection
Page 45: Context-Aware Saliency Detection

Summarization through collage creation

The salient objects as well as informative pieces of the background should be maintained in summaries.

Page 46: Context-Aware Saliency Detection

Automating collage creation

S. Goferman, A. Tal, and L. Zelnik-Manor.Puzzle-like collage. [2010]

3 stages:

Compute the saliency maps for images. Extracts regions-of-interest by considering both saliency and

image edge information. Assemble non-rectangular ROIs.

Page 47: Context-Aware Saliency Detection
Page 48: Context-Aware Saliency Detection
Page 49: Context-Aware Saliency Detection

Conclusion Propose a new type of saliency:

context-aware saliency

Evaluate in 2 applications:retargeting、 summarization

Page 50: Context-Aware Saliency Detection

EXTRA ?

Page 51: Context-Aware Saliency Detection
Page 52: Context-Aware Saliency Detection

Thanks for attention!