11

Who Works Together in Agent Who Works Together in Agent Coalition Formation?Coalition Formation?

Vicki Allan – Utah State UniversityVicki Allan – Utah State University

Kevin Westwood – Utah State UniversityKevin Westwood – Utah State University

CIA 2007CIA 2007

22

OverviewOverview

Tasks: Various skills and numbersTasks: Various skills and numbers

Agents form coalitionsAgents form coalitions

Agent types - Differing policiesAgent types - Differing policies

How do policies interact?How do policies interact?

33

Multi-Agent CoalitionsMulti-Agent Coalitions

““A coalition is a set of agents that work together A coalition is a set of agents that work together to achieve a mutually beneficial goal” (Klusch to achieve a mutually beneficial goal” (Klusch and Shehory, 1996)and Shehory, 1996)

Reasons agent would join CoalitionReasons agent would join Coalition Cannot complete task aloneCannot complete task alone Complete task more quicklyComplete task more quickly

44

Skilled Request For Proposal Skilled Request For Proposal (SRFP) Environment(SRFP) Environment

Inspired by RFP (Kraus, Shehory, and Taase 2003)Inspired by RFP (Kraus, Shehory, and Taase 2003)

Provide set of tasks T = {TProvide set of tasks T = {T11…T…Tii…T…Tnn} } Divided into multiple subtasksDivided into multiple subtasks In our model, task requires skill/levelIn our model, task requires skill/level Has a payment value V(THas a payment value V(Tii) )

Service Agents, A = {AService Agents, A = {A11…A…Akk…A…App}} Associated cost fAssociated cost fk k of providing serviceof providing service In the original model, ability do a task is In the original model, ability do a task is determined probabilistically – no two agents alike.determined probabilistically – no two agents alike. In our model, skill/levelIn our model, skill/level Higher skill is more flexible (can do any task with lower level skill)Higher skill is more flexible (can do any task with lower level skill)

55

Why this model?Why this model?Enough realism to be interestingEnough realism to be interesting An agent with specific skills has realistic An agent with specific skills has realistic

properties.properties. More skilled More skilled can work on more tasks, (more can work on more tasks, (more

expensive) is also realisticexpensive) is also realistic

Not too much realism to harm analysisNot too much realism to harm analysis Can’t work on several tasks at once Can’t work on several tasks at once Can’t alter its costCan’t alter its cost

66

Auctioning ProtocolAuctioning ProtocolVariation of a reverse auctionVariation of a reverse auction

Agents compete for opportunity to perform servicesAgents compete for opportunity to perform services Efficient way of matching goods to servicesEfficient way of matching goods to services

Central Manager (ease of programming)Central Manager (ease of programming)1)1) Randomly orders AgentsRandomly orders Agents

2)2) Each agent gets a turnEach agent gets a turnProposes or Accepts previous offer Proposes or Accepts previous offer

3)3) Coalitions are awarded taskCoalitions are awarded task

Multiple Rounds {0,…,rMultiple Rounds {0,…,rzz}}

77

Agent Costs by LevelAgent Costs by Level

0

10

20

30

40

50

60

0 1 2 3 4 5 6 7 8 9 10

Skill Level

Co

sts

General upward trend

88

05

10152025

3035

Agent 3 Agent 2 Agent 5

Costs

Agent Cost Profit

0

5

10

15

20

25

30

35

Agent 11 Agent 2 Agent 5

Costs

Agent Cost Profit

Agent costAgent costBase cost derived from skill and skill levelBase cost derived from skill and skill level Agent costs deviate from base cost Agent costs deviate from base cost

Agent paymentAgent paymentcost + proportional portion of net gaincost + proportional portion of net gain

Only Change in coalition

99

How do I decide what to propose?

1010

DecisionsDecisions

If I make an offer…If I make an offer…What task should I propose doing?What task should I propose doing?What other agents should I What other agents should I recruit?recruit?

If others have made me an offer…If others have made me an offer…How do I decide whether to How do I decide whether to accept?accept?

1111

Coalition Calculation AlgorithmsCoalition Calculation AlgorithmsCalculating all possible coalitionsCalculating all possible coalitions Requires exponential timeRequires exponential time Not feasible in most problems in which Not feasible in most problems in which

tasks/agents are entering/leaving the systemtasks/agents are entering/leaving the system

Divide into two stepsDivide into two steps1) Task Selection 1) Task Selection

2) Other Agents Selected for Team2) Other Agents Selected for Team polynomial time algorithmspolynomial time algorithms

1212

Task Selection- 4 Agent TypesTask Selection- 4 Agent Types1.1. Individual Profit – obvious, greedy Individual Profit – obvious, greedy

approachapproach

Competitive: best for meCompetitive: best for me

Why not always be greedy?Why not always be greedy?Others may not accept – your membership is Others may not accept – your membership is questionedquestioned

Individual profit may not be your goalIndividual profit may not be your goal

2.2. Global ProfitGlobal Profit

3.3. Best Fit Best Fit

4.4. Co-opetitiveCo-opetitive

1313

Task Selection- 4 Agent TypesTask Selection- 4 Agent Types

1.1. Individual Profit Individual Profit

2.2. Global Profit – somebody should do this taskGlobal Profit – somebody should do this task

I’ll sacrificeI’ll sacrificeWouldn’t this always be a noble thing to do?Wouldn’t this always be a noble thing to do?

Task might be better done by othersTask might be better done by others

I might be more profitable elsewhereI might be more profitable elsewhere

3.3. Best Fit – uses my skills wiselyBest Fit – uses my skills wisely

4.4. Co-opetitiveCo-opetitive

1414

Task Selection- 4 Agent TypesTask Selection- 4 Agent Types1.1. Individual Profit Individual Profit

2.2. Global Profit Global Profit

3.3. Best Fit – Best Fit – Cooperative: uses skills Cooperative: uses skills wiselywisely

Perhaps no one else can do itPerhaps no one else can do it

Maybe it shouldn’t be doneMaybe it shouldn’t be done

4.4. Co-opetitiveCo-opetitive

1515

44thth type: Co-opetitive Agent type: Co-opetitive Agent

Co-opetitionCo-opetition Phrase coined by business professors Phrase coined by business professors

Brandenburger and Nalebuff (1996),Brandenburger and Nalebuff (1996), to to emphasize the need to consider both emphasize the need to consider both competitive and cooperative strategies.competitive and cooperative strategies.

Co-opetitive Task SelectionCo-opetitive Task Selection Select the best fit task if profit is within P% of Select the best fit task if profit is within P% of

the maximum profit availablethe maximum profit available

1616

What about accepting offers?What about accepting offers?Melting – same deal gone laterMelting – same deal gone later

Compare to what you could Compare to what you could achieve with a proposalachieve with a proposal

Compare best proposal with Compare best proposal with best offerbest offer

Use utility based on agent typeUse utility based on agent type

1717

Some amount of compromise is necessary…

We term the fraction of the total possible you demand – the compromising ratio

1818

Resources ShrinkResources Shrink

Even in a task rich environment the Even in a task rich environment the number of tasks an agent has to choose number of tasks an agent has to choose from shrinksfrom shrinks Tasks get takenTasks get taken

Number of agents shrinks as others are Number of agents shrinks as others are assignedassigned

1919

Resources Task Rich

010

2030

4050

6070

80

1 2 3 4 5 6 7 8 9 10 11 12

Rounds

Availab

le R

eso

urc

es

My Tasks

Total Tasks

Total Agents

My tasks parallel total tasks

Task Rich: 2 tasks for every agent

2020

Scenario 1 – Bargain BuyScenario 1 – Bargain Buy

Store “Bargain Buy” Store “Bargain Buy” advertises a great priceadvertises a great price

300 people show up300 people show up

5 in stock5 in stock

Everyone sees the Everyone sees the advertised price, but it just advertised price, but it just isn’t possible for all to isn’t possible for all to achieve itachieve it

2121

Scenario 2 – selecting a spouseScenario 2 – selecting a spouse

Bob knows all the Bob knows all the characteristics of the characteristics of the perfect wifeperfect wife

Bob seeks out such a Bob seeks out such a wifewife

Why would the perfect Why would the perfect woman want Bob?woman want Bob?

2222

Scenario 3 – hiring a new PhDScenario 3 – hiring a new PhD

Universities ranked 1,2,3Universities ranked 1,2,3

Students ranked a,b,cStudents ranked a,b,c

Dilemma for second tier Dilemma for second tier universityuniversity

offer to “a” studentoffer to “a” student

likely rejectedlikely rejected

delay for acceptancedelay for acceptance

““b” students are goneb” students are gone

2323

Affect of Compromising RatioAffect of Compromising Ratio

equal distribution of each agent typeequal distribution of each agent type

Vary compromising ratio of only one type Vary compromising ratio of only one type (local profit agent)(local profit agent)

Shows profit ratio = profit achieved/ideal Shows profit ratio = profit achieved/ideal profit (given best possible task and profit (given best possible task and partners)partners)

2424

Local Profit Agents (Task Rich)

0

0.1

0.2

0.3

0.4

0.5

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95

Compromising Ratio of Local Profit

Pro

fit/

Idea

l Agent/Task .5

Agent/Task 1

Agent/Task 2

Agent/Task 3

Achieved/theoretical bestNote how profit is affect by load

2525

Profit only of scheduled agentsProfit only of scheduled agents

Scheduled Agent Profit (Task Rich)

0.440.46

0.480.5

0.520.54

0.560.58

0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95

Compromising Ratio of Local Profit

Pro

fit/

Idea

l LocalProf

GlobProf

Coopet

BestFit

Only Local Profit agentschange compromising ratio

Yet others slightly increase too

2626

NoteNote

Demanding local profit agents reject the Demanding local profit agents reject the proposals of others.proposals of others.

They are blind about whether they belong They are blind about whether they belong in a coalition.in a coalition.

They are NOT blind to attributes of others.They are NOT blind to attributes of others.

Proposals are fairly goodProposals are fairly good

2727

Balanced Proposers

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

LocalProf GlobProf Coopet BestFit

Rat

io t

o E

xpec

ted

LocalProf

GlobProf

Coopet

BestFit

For every agent type, the most likely proposer For every agent type, the most likely proposer was a Local Profit agent.was a Local Profit agent.

2828

Balanced Proposers

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

LocalProf GlobProf Coopet BestFit

Rat

io t

o E

xpec

ted

LocalProf

GlobProf

Coopet

BestFit

No reciprocity: Coopetitive eager to accept Local Profit proposals, No reciprocity: Coopetitive eager to accept Local Profit proposals, but Local Profit agent doesn’t acceptbut Local Profit agent doesn’t acceptCoopetitive proposals especially wellCoopetitive proposals especially well

2929

Balanced Proposers

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

LocalProf GlobProf Coopet BestFit

Rat

io t

o E

xpec

ted

LocalProf

GlobProf

Coopet

BestFit

For every agent type,For every agent type,Best Fit is a strong acceptor.Best Fit is a strong acceptor.

Perhaps because it isn’t accepted well as a proposerPerhaps because it isn’t accepted well as a proposer

3030

Coopetitive agents function better as Coopetitive agents function better as proposers to Local Profit agents in proposers to Local Profit agents in balanced or task rich environment.balanced or task rich environment. When they have more choices, they tend to When they have more choices, they tend to

propose coalitions local profit agents likepropose coalitions local profit agents like More tasks give a Coopetitive agent a better More tasks give a Coopetitive agent a better

sense of its own profit-potentialsense of its own profit-potential

Load balance seems to affect rolesLoad balance seems to affect roles

Coopetitive Agents look Coopetitive Agents look at fit as long as it isn’t too bad at fit as long as it isn’t too bad

compared to profit.compared to profit.

3131

Agent rich: 3 agents/taskAgent rich: 3 agents/task

Agent Rich Proposers

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

LocalProf GlobProf Coopet BestFit

LocalProf

GlobProf

Coopet

BestFit

Coopetitive accepts most proposals Coopetitive accepts most proposals from agents like itselffrom agents like itself

in agent rich environmentsin agent rich environments

3232

Do agents generally want to work with Do agents generally want to work with agents of the same type? agents of the same type? Would seem logical as agents of the same Would seem logical as agents of the same

type value the same things – utility functions type value the same things – utility functions are similar.are similar.

Coopetitive and Best Fit agents’ proposal Coopetitive and Best Fit agents’ proposal success is stable with increasing percentages success is stable with increasing percentages of their own type and negatively correlated to of their own type and negatively correlated to increasing percentages of agents of other increasing percentages of agents of other types. types.

3333

Look at function with increasing Look at function with increasing numbers of one other type.numbers of one other type.

Local Profit, Task Rich

0123456

Proposers

Joiners

3434

What happens as we change What happens as we change relative percents of each agent?relative percents of each agent?

Interesting correlation with profit Interesting correlation with profit ratio. ratio.

Some agents do better and Some agents do better and better as their dominance better as their dominance increases. Others do worse.increases. Others do worse.

3535

Individual Profit Ratio (std dev 5)

0.58

0.59

0.6

0.61

0.62

0.63

0.64

0% 20% 40% 60% 80% 100%

Agent %

Ind

ivid

ual

Pro

fit

Rat

io

Individual Profit

Global Profit

Best Fit

Co-opetitive

shows relationship if all shows relationship if all equal percentequal percent

Best fit Best fit does better does better and better and better as more as more

dominant in dominant in setset

Best fit Best fit does better does better and better and better as more as more

dominant in dominant in setset

Local ProfitLocal Profitdoes better whendoes better whenit isn’t dominantit isn’t dominant

3636

So who joins and who proposes?So who joins and who proposes?

Agents with a wider range of acceptable Agents with a wider range of acceptable coalitions make better joiners.coalitions make better joiners.

Fussier agents make better proposers.Fussier agents make better proposers.

However, the joiner/proposer roles are However, the joiner/proposer roles are affected by the ratio of agents to work.affected by the ratio of agents to work.

3737

ConclusionsConclusions

Some agent types are very good in Some agent types are very good in selecting between many tasks, but not as selecting between many tasks, but not as impressive when there are only a few impressive when there are only a few choices. choices.

In any environment, choices diminish In any environment, choices diminish rapidly over time.rapidly over time.

Agents naturally fall into role of proposer Agents naturally fall into role of proposer or joiner.or joiner.

3838

Future WorkFuture Work

Lots of experiments are possibleLots of experiments are possible

All agents are similar in what they value. All agents are similar in what they value. What would happen if agents deliberately What would happen if agents deliberately proposed bad coalitions?proposed bad coalitions?