interpreting advertiser intent in sponsored search bhanu c vattikonda, santhosh kodipaka, hongyan...
TRANSCRIPT
1
Interpreting Advertiser Intent in Sponsored SearchBHANU C VATTIKONDA, SANTHOSH KODIPAKA, HONGYAN ZHOU, VACHA DAVE, SAIKAT GUHA, ALEX C SNOEREN
2
Organic results
Sponsoredresults
Query
3
women’s shoes
Ad Keyword
shoesQuery
4
Huge and growing industry
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140
5
10
15
20
25
30
Search Revenues ($ billions)
20%
5
What is today’sdate in Japan
Query
Date JapanAd Keyword
6
Virgin River Utah
Query
Virgin MediaAd Keyword
7
Virgin River Utah
Query
Virgin MediaAd Keyword
We use organic search results for ad keyword
8
Capturing user and advertiser intents
OverviewMismatching advertiser and user intent
Organic results are accurate
Ad keywords capture advertiser intent
Ad keyword is very brief
Use organic results for ad keyword to interpret advertiser intent
9
10
Relevance in Sponsored Search
11
Ad selection pipeline
Ad Corpus(M’s of ads)
UserQuery
• Quick look up • “virgin media”
matched to “virgin river Utah”
• Choose ads that provide most revenue
• Measure the relevance of each ad to user query
• Virgin media ad seen as relevant to “virgin river Utah”
Ad retrieval Relevance Auction
12
Supervised machine learning
Training
Training set(query, ad, judgement)
Trained Ranker
Compute numerical featuresE.g., no. of words common in query and ad keyword
(features, judgement)
Ranker can compute relevance of (query, ad) pair
13
Signal sources
Virgin River UtahQuery
Virgin MediaAd Keyword
14
Signal sources
Virgin River UtahQuery
Virgin MediaAd Keyword
Ad Creative
15
Signal sources
Virgin River UtahQuery
Virgin MediaAd Keyword
Ad Creative
Landing page
16
Features from signal sourcesQuery and ad creative
Query: “virgin river Utah”, ad title: “virgin media | virginmedia.com”
Word bigram overlap: 0
Query and landing page
Query: “virgin river utah”, landing page title: “Virgin Media – Cable Broadband,
Digital TV”
Ordered word bigram overlap: 0
Query and ad keyword
Query: “virgin river Utah”, ad keyword: “virgin media”
Word unigram overlap: 0.5
17
Interpreting user and advertiser intent Query and ad keyword are very short -- 2.5 words on average
Hard to determine user and advertiser intent
Query and ad keyword may mean same without overlap or vice versa
Sneakers vs shoes
Virgin river Utah vs virgin media
Use organic search results to boost query and ad keyword
18
Matching user and advertiser intents
50% overlap
25% overlap3% overlap
19
Improving relevance rankerIntroduce features capturing overlap between user and advertiser intents
User intent captured using search results for user query
Advertiser intent captured using search results for ad keyword
75 Features introduced:
Overlap between elements of search results
Overlap between ad creative and search results for ad keyword
20
Evaluation
21
Measuring performance of ranker
Trained Ranker
Validation set(query, ad, judgement)
(query, ad, relevance)
Compare ranker result against human judgement
Compute numerical features
(query, ad, judgement)
22
Evaluating performance of ranker Did the ranker identify most of the relevant results?
How accurately did the ranker identify relevant ads?
23
Precision-RecallLow precision results in irrelevant ads being shown to users
Bad user experience
Wasteful spend for advertisers
Low recall would lead to missed chances
Lost revenue opportunities for search engine
Lost targeting for advertisers
Lost desirable ads for users
24
Dataset and approachData from a large search engine
1.28M (query, ad) pairs of training data
320k (query, ad) pairs of validation data
Several hundred existing features
The ranker is trained on a combination of the features we introduce and existing production system features
Compare new ranker to current ranker
25
Query and ad matching Query and ad matched using different match types chosen by advertiser
Exact match
Ad keyword: “shoes” and query: “shoes”
Broad match
Ad keyword: “shoes” and query: “sneakers”
Features we introduce capture similarity between query and ad keyword
26
Significant improvement in broad match
2.7% improvement in area under precision-recall curve
27
SummaryMismatched user and advertiser intents leads to errors in sponsored ads
Interpreting ad keywords poses a challenge due to their brevity
Organic search results for the ad keyword capture advertiser intent
2.7% gain in area under precision-recall curve over production
28
Thanks
29
Thanks
30
Backup
Organic results vs ad results Organic and ad result goals are different
Ads “related” to user query
Example:
User query: “prom dresses”
Ad for Limousine service targeting “prom dresses”
Good ad but bad organic result
8
Ad keywords are for targeting