adauctions
TRANSCRIPT
Expense constrained bidder optimization in repeated auctions
Ramki Gummadi Stanford University
(Based on joint work with P. Key and A. Proutiere)
Overview
• Introduction/Motivation
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Three Aspects of Sponsored Search
1. Sequential setting.
2. Micro-transactions per auction.
3. The long tail of advertisers is expense constrained.
Modeling Expense ConstraintsFixed budget over finite horizon => any balance at time is worthless.
Balance
timeT0
B
Modeling Expense ConstraintsStochastic fluctuations could cause spend rate different from target.
Balance
timeT0
B
Modeling Expense Constraints
“…the nature of what this budget limit means for the bidders themselves is somewhat of a mystery. There seems to be some risk control element to it, some purely administrative element to it, some bounded-rationality element to it, and more…”
-- “Theory research at google”, SIGACT News, 2008.
Modeling Expense ConstraintsAdd a fixed income, per unit time to the balance and relax time horizon.
Balance
time0
B
Responsibility for expense constraints Auctioneer Bidder
Bids fixed -- Auction entry throttled.
Bids adjusted dynamically.
Online bipartite matching between queries and bidders.
Online knapsack type problems.
Expense constraints = fixed budget.
Possible to model more general expense constraints.
Bid optimization
Preview
Sequential X-auction with true value v
Static X-auction with virtual value: shade* v
X can be SP, GSP, FP, etc. (any quasi linear utility)
Shade(remaining balance B) =
will be characterized explicitly.
1
1 '( )V B
Preview: Optimal Shading factors
Overview
• Introduction
• Budgeted Second Price auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Model: Budgeted Second Price
• Discrete time, indexed • Balance: • Constant income per time slot - • I.I.D. environment sampled from– Private valuation (observable) – Competing bid (not observable)
• Decision variable is bid at time – Can depend on and , but not
Model: Budgeted Second Price
Constraint: a.s.
• Utility:
• Objective function:
The Value Function
• : max utility starting with balance
• Can use dynamic programming (“one step look ahead”) to write out a functional fixed point relation.
The Value Function
1 2,
( ) max E 1{ } 1{ }v pu b
v b u p T u p T
( )v p e v b a p
( )e v b a
But boundary conditions can not be inferred from the DP argument.
Currentauction
Loss
Win1T
2T
Future opportunity cost
Characterization of value function
“Effective price” for nominal at balance :
Theorem: Optimal bid is *:i.e: Buy all auctions with “effective price” is a functional fixed point to:
( , )*u b v
,
( ) ( ) ( , )v p
v b e v b a v p b
( , ) ( ( ) ( ))p b p e v b a v b a p
1
,( ) ( ) ( , )i i i
v pv b e v b a v p b
Value Iteration:
𝛽=0.1
Each auction has miniscule utility compared to overall utility:
Value Iteration:
𝛽=0.01
1
,( ) ( ) ( , )i i i
v pv b e v b a v p b
Numerical estimation when is small:• State space quantization errors propagate due
to lack of boundary value.• Need longer iterations over larger state space.
will be studied under scaling:
( ) ( ) ( / )V B v b v B
Limiting case: micro-value auctions
Overview
• Introduction
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
General Online Budgeting ModelDecision Maker Environment
, i.i.d
Unobservable
Observable
Balance:
Utility:
Action
Payment:Income
𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒∑𝑡=0
∞
𝑒−𝛽 𝑡𝔼 [𝑔(𝑢 (𝑡 ) , 𝜉 (𝑡 )) ]
Ex1: Second Price Auction
(Random environment) (Observable part) is the bid (Action) (Utility function)
(Payment function)
Ex2: GSP Auction
Random environment: Observable part: is the bidUtility function:Payment function: 1
1
( , ) 1{ } ( )L
l l l ll
g u p u p v p
11
( , ) 1{ }L
l l l ll
c u p u p p
Click events for L slots
Overview
• Introduction
• Budgeted Second Price Auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Limiting Regime:
( ) ( ) ( / )V B v b v B
Notation:
(( ]) [ , )E g ug u
(( ]) [ , )E c uc u
is an inverseand
is the minimum of:
Theorem
*( ), (0) ,dV
f V VdB
( )x
is the solution to:
*
*( ') , (0) ,V V V
0
( ) sup( ( ) ( ) )u
x ax g u c u x
F
* 0min ( )x x
𝑥
𝑉
𝐵
𝑉 (𝐵)
Theorem
( )x
*
) =
Application to Second Price Auctions𝐸 [𝟏𝑢>𝑝 (𝑣−𝑝 )]
p]
Second Price Auction ExampleOpponents bid p
Private Valuation
𝜙(𝑥 )
Value functions
Optimal bid
0 ( )
( )
sup( ( ) ( ) '( )) sup 1{ }( (1 '( ))
sup 1{ }1 '( )
u u v
u v
g u c u V B uE p v p V B
vu p p
VE
B
F
i.e., Static SP with shaded valuation: 1 '( )
v
V B
* at balance B solves:
Optimal Scaling factor
Optimal Bid: GSP
0
1( ) 1
sup( ( ) ( ) '( ))
sup 1{ }1 '( )
u
L
l l l lu v l
g u c u V B
vp u p p
VE
B
F
Static GSP with “virtual valuation”: 1 '( )
v
V B
Proof Overview
• Variant: Retire with payoff when .
• Value function of variant converges to ODE with initial value .
• But what is the right boundary condition ?To prove:
Because exit payoff optional Next 2 slides
Goal: Exhibit a sequence of policies parametrized by which can achieve a scaled payoff as
Lemma: For any ε > 0, there is a policy * such that ε AND
If could be played continuously, we can get arbitrarily close to ! But every now and then balance is exhausted, so we need a variant of u* that still manages to achieve nearly as much payoff
𝜂∗≤ lim inf 𝑉 𝛽(0)
time
B(t)
B
Play U*
𝜂∗≤ lim inf 𝑉 𝛽(0)
Show that fraction of time spent in green phase by the random walk gets arbitrarily close to 1 as ->0
Overview
• Introduction
• MDP for budgeted SP auctions
• A General Online Budgeting Framework
• Optimal Bids for Micro-Value Auctions
• Conclusion
Conclusion
• A two parameter model for expense constraints in online budgeting problems.
• Optimal bid can be mapped to static auction with a shaded virtual valuation.
• Paper has more contents: MFE analysis and a finite horizon model.