discovering pictorial brand associations from large-scale online image data

1

Discovering Pictorial Brand Associations from Large-Scale

Online Image Data

Gunhee Kim* Eric P. Xing

School of Computer Science, Carnegie Mellon University

*: (now with Disney Research Pittsburgh)

December 7, 2013

2

• Problem Statement • Dataset • Algorithm• Visualization – Brand Association Maps• Results• Conclusion

Outline

3

A set of values or assets linked to a brand's name and symbol

Brand Equity

• Bearing competitive advantage over its competitors• Boosting efficiency and effectiveness of marketing programs• Attaining price premium due to increased customer satisfaction and loyalty

Interbrand(2013)

4

• [Keller 1993]

• [Aaker 1991]

Brand Knowledge

Brand Awareness Brand Image

Brand recall

Brand Recognition

Types of brand

association

Favorability of brand

association

Strength of brand

association

Uniqueness of brand

association

Brand Equity

Name awareness

Brand loyalty

Perceived quality

Brand association

Other properties (ex. trademarks, patens)

A set of values or assets linked to a brand's name and symbol

Brand Equity

• Bearing competitive advantage over its competitors• Boosting efficiency and effectiveness of marketing programs• Attaining price premium due to increased customer satisfaction and loyalty

5

• [Keller 1993]

• [Aaker 1991]

Brand Knowledge

Brand Awareness Brand Image

Brand recall

Brand Recognition

Types of brand

association

Favorability of brand

association

Strength of brand

association

Uniqueness of brand

association

Brand Equity

Name awareness

Brand loyalty

Perceived quality

Brand association

Other properties (ex. trademarks, patens)

A set of values or assets linked to a brand's name and symbol

Brand Equity

• Bearing competitive advantage over its competitors• Boosting efficiency and effectiveness of marketing programs• Attaining price premium due to increased customer satisfaction and loyalty

6

Brand Associations(What Comes to Mind When You Think of …)

soccer, national teams, …

basketball, golf, …

swimming, diving, beach, …

A set of associations that consumers perceive with the brand• Top-of-mind attitudes or feelings toward the brand

7

Brand Associations(What Comes to Mind When You Think of …)

A set of associations that consumers perceive with the brand• Top-of-mind attitudes or feelings toward the brand

• Consumer-driven brand equity

I need a men’s golf shirts… What would I buy?

I associate with

How can we find out people’s brand associations?

8

A Traditional Way to Brand Associations

Collect direct consumer responses to carefully-designed questionnaires

In-N-Out [Dane et al. 2010] Reese’s [Till et al. 2011]

• Expensive and time-consuming

• Sampling bias /common methods bias

Can Web data save us?

Cheap and immediate

Candid impression from a large crowd of potential customers

9

• Built from consumer-generated text data on online social media (eg. Weblogs, Boards, Wiki)

‘ ’s Brand Association Map

The most important concepts and themes that consumers have

10

Our Idea: Photo-based Brand Associations

Take advantage of large-scale online photo collections• No previous attempts so far to leverage the pictures

Our underlying rational

• Take a picture and tag as burger+king• Pictorial opinion on burger+king• Crawl millions of online images tagged as burger+king• We can read people’s candid pictorial opinions on burger+king

Pictures cannot replace texts. They are complementary. ex) Nike shoes are too expensive. Their design is tacky!

11

2. Take or bookmark pictures of the products you wish to buy.

1. Brands are recognized by images.

Why are online images helpful to discover brand associations?

Our Idea: Photo-based Brand Associations

Take advantage of large-scale online photo collections• No previous attempts so far to leverage the pictures

12

Why are online images helpful to discover brand associations?

Our Idea: Photo-based Brand Associations

Take advantage of large-scale online photo collections• No previous attempts so far to leverage the pictures

3. Capture interactions between users and products

in natural social context

13

• Identifying a small number of exemplars / image clusters

• Project the clusters into a circular layout

Objective of This Paper

1. (Image-level) Detect and visualize key pictorial concepts associated with brands

Two visualization tasks as a first technical step

14

• A basic function to reveal interactions between users and products

• Potential benefits: content-based retrieval and online multimedia ad

Objective of This Paper

2. (Subimage-level) Localize the regions of brand in images

Two visualization tasks as a first technical step

15

• Problem Statement • Dataset• Algorithm• Visualization – Brand Association Maps• Results• Conclusion

Outline

16

Image Dataset from Five Websites

4,720,724 images of 48 brands of four categories

Luxury Sports Beer Fastfood

Two largest and most popular photo sharing Web communities

Pictures bookmarked by users

Various forms of artwork by users

Photo hosted for twitter

17

• Problem Statement • Dataset • Algorithm• Visualization – Brand Association Maps• Results• Conclusion

Outline

18

Approach – KNN Graph Generation

Input: a set of images of a brand (e.g. louis+vuitton): Feature extraction

• Dense feature extraction of Color SIFT and HOG• Histogram intersection for similarity measure

KNN Similarity graph• Approximate method (Multiple random divide-and-conquer using meta-data) [Wang et al. 2012] in O(NlogN)

19

Approach – Exemplar Detection/Clustering

Detecting L number of exemplars• A small set of representative images• Diversity ranking algorithm (temperature maximization) [Kim & Xing 2011]

• Solving submodular optimization on

20

Approach – Exemplar Detection/Clustering

• Each image I is associated with its closest exemplar• Random walk model (What exemplar is most likely reachable from each I?)

Clustering

21

Approach – Brand Localization via Cosegmentation

Find the regions that are most relevant to the brand• Use MFC algorithm [Kim&Xing. 2012] to each cluster of coherent images • Make parallelization easy and prevent cosegmenting images of no commonality• Optionally, generate bounding boxes

22

Approach – Closing a Loop

Go back to graph generation

(Our argument) even imperfect segmentation helps obtain better image similarity measurement.

• Not robust against the location and scale change

• Image similarity as the mean of best matched segment similarity

23

• Problem Statement • Dataset• Algorithm• Visualization – Brand Association Maps• Results• Conclusion

Outline

24

Embedding on a Circular Layout

Goal: Compute two coordinates of key clusters

• Compute stationary distribution of nodes

Radial distance

• Radial distance: a larger cluster closer to the center• Angular distance: the smaller, the higher correlation

Radial distances

Angular distance• Sum of stationary distributions of nodes of each cluster the probability that a random walker stays

(Based on Nielson’s BAM)

25

Embedding on a Circular Layout

Angular distance

Radial distances

Angular distance

• Using spherical Laplacian eigenmap [Carter et al. 2009.]

• Pairwise similarity btw clusters S using the random walk with restart [Sun et al. 2005]

Similar clusters should havesmaller angle difference

Penalty to avoid a trivial solution

Goal: Compute two coordinates of key clusters• Radial distance: a larger cluster closer to the center• Angular distance: the smaller, the higher correlation

(Based on Nielson’s BAM)

26

• Problem Statement • Dataset• Algorithm• Visualization – Brand Association Maps• Results

Examples of Brand Association Maps Quantitative results for exemplar detection/clustering Quantitative results for brand localization Correlations btw Web image data and actual sales data

• Conclusion

Outline

27

Experiments – Brand Association Maps

Brands' characteristic visual themes

Events sponsored by brands

Weddings

28

Sub-M: Multiple runs of our clustering + cosegmentation Sub: Our clustering without cosegmentation Kmean/Spect: K-mean clustering / Spectral clusteringLP: Label propagation [Raghavan et al. 2007] AP: Affinity propagation [Frey & Dueck 2008]

Experiments – Exemplar Detection/Clustering

Groundtruth for clustering accuracy

• Randomly select 100 sets of three (i,j,k) images per class

• Accuracy is measured by how many sets are correctly clustered

• Manually select which of (j,k) is closer to i :

Wilcoxon-Mann-Whitney statistics

Similarity btw the clusters of image i and j

100

29

Experiments – Brand Localization

Task: Foreground detection

MFC-M: Multiple runs of our clustering + cosegmentationMFC-S: Single run of our clustering + cosegmentationMFC: Random partition + our cosegmentationCOS: Submodular optimization [Kim et al. ICCV11] LDA: LDA-based localization [Russell et al. CVPR06]

• Manually annotate 50 images per class

• Accuracy is measured by intersection-over-union

30

Experiments – Brand Localization

Task: Foreground detection

• Manually annotate 50 images per class

• Accuracy is measured by intersection-over-union

Examples

31

Experiments – Correlation with Sales Data

Photo volumes vs. Market share• Nike’s market share is 57.6% in sports brands. How’s about image volumes?

• Based on brands’ annual reports

Observations• Ranking are roughly similar, but the proportions do not agree.• Guinness : premium beer (more photo data) Taco+bell : cheap fastfood brand (fewer photo data)

32

• Problem Statement • Dataset• Algorithm• Visualization – Brand Association Maps• Results• Conclusion

Outline

33

• Visualizing core pictorial concepts associated with brands

Study of brand associations from millions of Web images

• Experiments on 5 Millions images of 48 brands from five popular websites

Conclusion

• Complementary views on the brand associations beyond texts

Jointly achieving two levels of visualization tasks

• Localizing the regions of brand in images

Various potential applications

• Online multimedia contextual advertisement, competitor mining

34

Press Release

Thank You!

Extended version will be presented in

http://www.cs.cmu.edu/news/carnegie-mellon-researchers-create-brand-associations-mining-millions-images-social-media-sites