icec11slides uno
Post on 20-Aug-2015
98 Views
Preview:
TRANSCRIPT
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
A Multi-Agent Prediction Market based onPartially Observable Stochastic Game
Janyl Jumadinova, Raj Dasgupta
C-MANTIC Research GroupComputer Science Department
University of Nebraska at Omaha, USA
ICEC 2011
1 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Outline
Problem: Traders’ behavior in a prediction market andits impact on the prediction market’s behavior
Solution: A multi-agent system that formalizes thestrategic behavior and decision making by market’sparticipants based on a partially observable stochasticgame
Experimental validation: Comparison with othertrading approaches
1 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
A Prediction market is
a market-based mechanism used to
- combine the opinions on a future event from differentpeople and
- forecast the possible outcome of the event based on theaggregated opinion
2 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
3 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
4 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
5 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
6 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
7 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
8 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
9 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
10 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
11 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
12 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction Market
13 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Prediction MarketMain Features
A prediction market is run for a real-life unknown event
Each event has a finite duration
Each event’s outcome has a security associated with it
Traders buy and sell the securities based on their beliefsabout the outcome of the event
Traders’ beliefs are expressed as probabilities
Market maker aggregates the probabilities from all thetraders into a single probability, market price
Market price of a security represents the probability ofthe outcome of an event associated with that securityhappening
Traders get paid according to their reported beliefs
14 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
A Multi-Agent Prediction Market
Software trading agents perform calculations and tradeon behalf of human tradersProvides testbed for modeling different strategicbehaviors of traders through simulations with tradingagents
15 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Do Prediction Markets Work?Yes, evidence from real markets, laboratory experiments, and theory
I.E.M. beat political polls 451/596 [Forsythe 1999, Berg2001, Pennock 2002]
HP market beat sales forecast 6/8 [Plott 2000]
Sports betting markets provide accurate forecasts ofgame outcomes [Debnath 2003, Schmidt 2002]
Market games work [Pennock 2001]
Laboratory experiments confirm informationaggregation [Forsythe 1990, Plott 1997, Chen 2001]
Theory of Rational Expectations [Lucas 1972, Grossman1981]
and more...
16 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Research Problem Addressed
Develop a formal, game-theoretic model of the tradingagent behavior in prediction markets including
- impact of information from external sources on tradingagent decisions/behavior,
- a solution concept for calculating the equilibriumstrategies of the trading agents
Research Questions:
How does different traders’ behaviors affect marketprices?
What trading strategies give the highest utilities to thetraders?
How can prediction markets incentivize traders toparticipate and report their beliefs truthfully in order toachieve a higher prediction accuracy?
17 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Our Solution
A partially observable stochastic game with information(POSGI)-based model of the trading agent behavior
A correlated equilibrium (CE)-based solution todetermine equilibrium strategy in the POSGIrepresentation
18 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Partially Observable Stochastic Game withInformation (POSGI)
- Logarithmic Market Scoring Rule (LMSR) [Hanson2007] gives formula to calculate aggregate market pricefrom the outstanding quantity of a security
19 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Partially Observable Stochastic Game withInformation (POSGI)
- Logarithmic Market Scoring Rule (LMSR) [Hanson2007] gives formula to calculate aggregate market pricefrom the outstanding quantity of a security
19 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Partially Observable Stochastic Game withInformation (POSGI)
- Belief state is updated using a Bayesian model of pastbeliefs, past actions and current observation
20 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Partially Observable Stochastic Game withInformation (POSGI)
- Information signal can be {−1, 0, 1} representingpositive, neutral, or negative information
21 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Partially Observable Stochastic Game withInformation (POSGI)
22 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Two Agent Trading Scenario with 1 securityExample
Actions = to buy, sell, or hold one security
Market state is denoted by q
C(q) is the cost function calculated by LMSR, thatreflects the total money collected by the market maker
Can construct a normal form game to capture thedecision problem for each agent
23 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Correlated Equilibrium (CE)Set up
Let N be the set of trading agents
Let Ai be the set of actions for agent i ∈ NLet Φi be the set of mixed strategy profiles defined overAi
Joint strategy space is then Φ = ×|N |i=1Φi
Let φ ∈ Φ be a strategy profile and φi denote agent i’scomponent in φ
Let ui(φ) be the utility of agent i from joint strategy φ
24 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Correlated Equilibrium (CE)
Correlated Equilibrium is a probability distribution p onΦ such that
for all agents i and all strategies φi, φ′i
if all agents follow a strategy profile φ,agent i has no incentive to play another strategy φ′iinstead
The Chicken-Dare GameAssume there is a trusted third party that draws astrategy for each player and announces it to each playerseparatelyPlayers agree to follow the strategy suggested by thethird partyPlayers get higher payoffs when use CE than NE
25 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Correlated Equilibrium (CE) Algorithm
CE algorithm applied to our POSGI model is based onthe method proposed by Papadimitriou [2008]
CE algorithm of the POSGI gives the action (buy orsell; quantity) for each agent at each time step
Time complexity of our CE algorithm is O(|N | × |Φi|2)
26 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Correlated Equilibrium (CE)with Trading Agents’ Risk Preferences
The beliefs and risk preferences of traders are directlycorrelated [Kadane 1988, Dimitrov 2008]
Use constant relative risk averse (CRRA) utility function
To find CE in the market with risk-averse tradingagents:
- Find the set of all Pareto optimal strategy profiles- Check whether a Pareto optimal strategy profile satisfies
CE constraints
27 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Correlated Equilibrium (CE)with Trading Agents’ Risk Preferences
A strategy profile φP is Pareto optimal if there does notexist another strategy profile φ′ such thatui(φ
′) ≥ ui(φP )∀i ∈ N with at least one inequalitystrict
Pareto optimal strategy is found by maximizingweighted utilities and solving the maximization problemusing Lagrangian method
Proposition
If p is a correlated equilibrium and φP is a Pareto optimalstrategy profile calculated by p in a prediction market withrisk averse agents, then the strategy profile φP is incentivecompatible, that is each agent is best off reporting truthfully
28 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsSet up
Default values for market related parameters came fromIowa Electronic Marketplace (IEM) movie market
Assume there is one event with two outcomes
Consider only disjoint events
Report utilities and market price for the securitycorresponding to the outcome of the event being 1
29 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsSet up
We use the following strategies for comparison of the tradingagents’ and market’s behavior
1 ZI (Zero Intelligence) - each agent submits randomlycalculated quantity to buy or sell
2 ZIP (Zero Intelligence Plus) - each agent selects a quantityto buy or sell that satisfies a particular level of profit
3 CP (by Preist and Tol) - each agent adjusts its quantity tobuy or sell based on past prices and tries to choose thatquantity so that it is competitive among other agents
4 GD (by Gjerstad and Dickhaut) - each agent maintains ahistory of past transactions and chooses the quantity to buyor sell that maximizes its expected utility
5 DP (Dynamic Programming solution for POSG game) - each
agent uses dynamic programming solution to find the best
quantity to buy or sell [Hansen 2004]30 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(a) DP strategy has the highest percentage of adoption ofsome CE
(b) Prediction market with trading agents using CE strategyresult in more accurate market prices
(c) Trading agents using CE strategy obtain 38% moreutility than the agents following the next bestperforming strategy DP
31 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(a) DP strategy has the highest percentage of adoption ofsome CE
(b) Prediction market with trading agents using CE strategyresult in more accurate market prices
(c) Trading agents using CE strategy obtain 38% moreutility than the agents following the next bestperforming strategy DP
31 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(a) DP strategy has the highest percentage of adoption ofsome CE
(b) Prediction market with trading agents using CE strategyresult in more accurate market prices
(c) Trading agents using CE strategy obtain 38% moreutility than the agents following the next bestperforming strategy DP
31 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(a) Trading agents using CE strategy obtain more utilitythan trading agents using ZIP strategy
(b) Trading agents using CE strategy obtain more utilitythan trading agents using CP strategy
32 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(a) Trading agents using CE strategy obtain more utilitythan trading agents using ZIP strategy
(b) Trading agents using CE strategy obtain more utilitythan trading agents using CP strategy
32 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(c) Trading agents using CE strategy obtain more utilitythan trading agents using GD strategy
(d) Trading agents using CE strategy obtain more utilitythan trading agents using DP strategy
33 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-neutral Agents
(c) Trading agents using CE strategy obtain more utilitythan trading agents using GD strategy
(d) Trading agents using CE strategy obtain more utilitythan trading agents using DP strategy
33 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-averse Agents
(a) DP strategy has the highest percentage of adoption ofsome CE
(b) Prediction market with risk-averse trading agents usingCE strategy result in more accurate market prices
(c) Risk-averse trading agents using CE strategy obtain41% more utility than the agents following the nextbest performing strategy DP
34 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-averse Agents
(a) DP strategy has the highest percentage of adoption ofsome CE
(b) Prediction market with risk-averse trading agents usingCE strategy result in more accurate market prices
(c) Risk-averse trading agents using CE strategy obtain41% more utility than the agents following the nextbest performing strategy DP
34 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Experimental ResultsRisk-averse Agents
(a) DP strategy has the highest percentage of adoption ofsome CE
(b) Prediction market with risk-averse trading agents usingCE strategy result in more accurate market prices
(c) Risk-averse trading agents using CE strategy obtain41% more utility than the agents following the nextbest performing strategy DP
34 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Conclusions and Future Work
In this work we have:
- described a novel multi-agent based representation ofthe prediction market using POSG
- developed a CE solution for solving POSG
- empirically compared different agent behavior strategiesin the prediction market
- showed how CE can be obtained in the predictionmarket with risk averse agents
In the future we plan to:
Conduct experiments in an n-agent scenario using richercommercial data sets
Investigate the dynamics evolving from multipleprediction markets that interact with each other
35 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Conclusions and Future Work
In this work we have:
- described a novel multi-agent based representation ofthe prediction market using POSG
- developed a CE solution for solving POSG
- empirically compared different agent behavior strategiesin the prediction market
- showed how CE can be obtained in the predictionmarket with risk averse agents
In the future we plan to:
Conduct experiments in an n-agent scenario using richercommercial data sets
Investigate the dynamics evolving from multipleprediction markets that interact with each other
35 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
References
1 Y. Chen, D. Pennock. Utility Framework forBounded-Loss Market Maker. Proc. of the 23rdConference on Uncertainty in Artificial Intelligence (UAI2007), pages 49-56, 2007.
2 E. Hansen, D. Bernstein, S. Zilberstein. Dynamicprogramming for partially observable stochastic games.In Proceedings of the 19th National Conference onArtificial Intelligence, pages 709-715, 2004.
3 R. Hanson. Logarithmic Market scoring rules forModular Combinatorial Information Aggregation.Journal of Prediction Markets, 1(1):3-15, 2007.
4 Iowa Electronic Marketplace. URL:www.biz.uiowa.edu/iem/
5 C. Papadimitriou, T. Roughgarden. Computingcorrelated equilibria in multi-player games. Journal ofACM, 55(3):1-29, 2008.36 / 37
A Multi-AgentPrediction Marketbased on Partially
ObservableStochastic Game
Janyl Jumadinova,Raj Dasgupta
Outline
Introduction
Research Problem
POSGI
Trading Agents’Strategy
ExperimentalResults
Future Work
Thank You!
Questions?
jjumadinova@unomaha.edu
C-MANTIC Research Group
http://cmantic.unomaha.edu/
37 / 37
top related