online procurement auctions for resource pooling in client-assisted cloud storage systems jian zhao,...
TRANSCRIPT
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Online Procurement Auctions for Resource Pooling
in Client-Assisted Cloud Storage Systems
Jian Zhao, Xiaowen Chu, Hai Liu, Yiu-Wing LeungDepartment of Computer ScienceHong Kong Baptist University
Zongpeng LiDepartment of Computer ScienceUniversity of Calgary 1
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Outline
Motivation: why client-assisted?
Background: auctions
Model formulation
Some technical details (without equations)
Conclusions 2
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Cloud Storage Services
3
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
4Durability: Our data will be there without error forever.
Most of the time, CSPs are
Availability: Data can be accessed anywhere, anytime, from any device.
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
5
But occasionally, they become
Cloud Outages!
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Cloud OutagesCloud services could become unavailable because of
6
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Evidence of Cloud Outages
• “The worst cloud outagesof 201X “ by J. R. Raphael
Inforworld.com has been
tracing cloud outages since
2011
• Amazon• Google• Microsoft• Apple• Dropbox• Facebook• Adobe• …
Big names on the list
7
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
What can we do?Plan A: Cloud Federation
8
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Alternatively
9
Data Center A
Data Center B
Data Center C
storage pooling
storage retail
storage pooling
storage retail
high speed network
Plan B: Client-Assisted Cloud Storage
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Client-Assisted Examples in Academia
• Leverage peer bandwidth to mitigate server bandwidth cost
• Improve availability and downloading performance
FS2You
• Data is kept at a central cloud and replicated among distributed peersAmazingStore
• Peer-assisted architecture with a focus on data consistencyTriton
10
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
A Counter Example:
Wuala abandoned the “client-assisted” design in 2012
Hybrid architecture is very complicated
Bandwidth cost is dropping
Contribution from peers are marginal
More focus on business customers
Wuala was designed to be “client-assisted”.Servers in datacenters
Client users• casual peers without contribution• storage peers trading local storage for increased
storage space
11
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Another Story: Symform
Observation•“cloud providers use extremely inefficient centralized infrastructure to store stuff.”
•“most users had tons of excess local storage just going to waste”
Goal•“Creating the World’s Largest Datacenter”
•“Without Building a Datacenter”
Method•“Users that contribute get 1 GB free for every 2 GB contributed”
12
Acquired by Quantum in 2014
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Incentive makes a difference
Public BitTorrent
• Tit-for-Tat only• Free-rider problem• Stop seeding after
downloading• Limit the uploading
bandwidth
Private BitTorrent (or Darknet)
• Sharing Ratio Enforcement• Users fight to contribute
as much as they can to survive
• Everyone in the community can get high downloading performance
• People are willing to pay to get into the community
13
Total upload = Total download
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Auctions are economical approaches for allocating resources or trading commodities
Participants: auctioneer + bidders
Auction mechanism: a set of institutions for the buying/selling of goods or services, such as allocation rules and pricing rules
Auctions
14
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Auctions as Incentive
No need to predict users’ demand: users reveal their true information through bids
Balance among supply and demand
Allocate available resources efficiently
Obtain higher revenue 15
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Research Problem
How to design the auction mechanism for resource pooling from clients?
• For clients: get reasonable monetary return by selling resources (storage space & bandwidth)
• For CSPs: save cost (and overcome cloud outages)• Assumption: the cost of providing service by
datacenter herself is expensive than procuring resources from clients 16
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Our Approach:Online Procurement Auctions
Online auctions
• Different bidders arrive at different times
• Auctioneer makes decision about each bid as it is received• Different from the
traditional case that the auctioneer receives all the bids before determining the allocation
• In line with asynchronous arrivals of user bids and requests
Procurement auctions
• Ordinary auctions (or forward auctions): one seller, multiple buyers• Buyers compete
• Procurement auctions: one buyer, multiple sellers• Sellers compete• reverse auctions
17
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Our Auction Model (I)
Storage & bandwidth are unified as conceptual “resource”• Needs further study
Clients (i.e., bidders) want to sell resources through bids• Bid: (starting time, ending time, amount, unit price)• Valuation: the “true” value of the resource, private to the
bidder• Utility = the received payment - the cost of offering the sold
resource• Target: maximize utility
18
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Our Auction Model (II)
CSP (i.e., auctioneer ) wants to buy resources from some clients• Set a target S for time period [0, T]• For each incoming bid, determine
• how much to procure (allocation rule)• how much to pay (payment rule)
19
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Design Objectives
• If ( truthful bidding always maximizes utility )
• Then ( rational bidders will report their true valuations )
Truthful
• The CSP’s total cost is bounded by γ times of the total cost of offline optimal auction
Competitive20
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Our Results
Truthfulness• We derive price-based allocation rule and
payment rule that result in truthful auction mechanism.
Competitiveness• We find a solution to guarantee a target
competitive ratio against offline optimal auctions.
21
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Our Methodology
Start from Myerson’s Principles of Truthfulness (1981)•Gives two conditions to satisfy the truthfulness property
•For one single indivisible good
Extend to the case of online procurement auction for divisible goods•We first study the conditions for truthfulness of an online auction
•We then derive allocation and payment equations such that the auction mechanism is truthful.
Design marginal pricing function for CSP to achieve a desired competitive ratio
22
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Conclusions
We consider “client-assisted” a promising approach to addressing cloud outages.
We argue that “auctions” can be a good incentive mechanism.
We propose an online procurement auction mechanism, and prove its truthfulness and competitiveness. 23
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
24
Aand
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Myerson’s Principle of Truthfulness
Consider the auction of a single indivisible good
The auction mechanism is truthful if and only if• The probability of a bidder winning an auction
is monotonically non-decreasing in its bid• The payment charged to a bidder is
independent of its bid25
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Extension to Online Procurement Auctions
We first define “allocation monotonicity” for online procurement auctions.• Better bids get more allocation
If allocation rule is monotone, we can find a payment rule that results in truthful online procurement auction.
We design a monotone allocation rule and the corresponding payment rule.
26
IEEE
INFO
COM
201
5, H
ON
G K
ON
G
Competitive Analysis
Our auction mechanism assumes a non-increasing marginal pricing function for the CSP to procure resources from clients.• The marginal pricing function is a variable
How to minimize the total cost by adjusting the marginal pricing function?
• An online algorithm design problem
We find a solution of setting the marginal pricing function to achieve a target competitive ratio. 27