Broadly, distributed problem solving is a subfield withinmultiagent systems, where the focus is to enable multipleagents to work together to solve a problem. These agents

are often assumed to be cooperative, that is, they are part of ateam or they are self-interested but incentives or disincentiveshave been applied such that the individual agent rewards arealigned with the team reward.

We illustrate the motivations for distributed problem solvingwith an example. Imagine a decentralized channel-allocationproblem in a wireless local area network (WLAN), where eachaccess point (agent) in the WLAN needs to allocate itself a chan-nel to broadcast such that no two access points with overlap-ping broadcast regions (neighboring agents) are allocated thesame channel to avoid interference. Figure 1 shows examplemobile WLAN access points, where each access point is a Createrobot fitted with a wireless CenGen radio card. Figure 2a showsan illustration of such a problem with three access points in aWLAN, where each oval ring represents the broadcast region ofan access point.

This problem can, in principle, be solved with a centralizedapproach by having each and every agent transmit all the rele-vant information, that is, the set of possible channels that theagent can allocate itself and its set of neighboring agents, to acentralized server. However, this centralized approach mayincur unnecessary communication cost compared to a distrib-

Articles

FALL 2012 53Copyright © 2012, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Distributed Problem Solving

William Yeoh, Makoto Yokoo

n Distributed problem solving is a subfieldwithin multiagent systems, where agents areassumed to be part of a team and collaboratewith each other to reach a common goal. In thisarticle, we illustrate the motivations for distrib-uted problem solving and provide an overviewof two distributed problem-solving models,namely distributed constraint-satisfaction prob-lems (DCSPs) and distributed constraint-opti-mization problems (DCOPs), and some of theiralgorithms.

uted approach. For example, agents in the central-ized approach need to send information to a cen-tralized server, which can be many hops away inthe WLAN. On the other hand, agents in a distrib-uted approach need only send information to theirneighboring agents, which are one hop away.Additionally, a distributed approach can also takeadvantage of parallelism to solve the problemfaster. For example, if the WLAN is composed oftwo disjoint networks, then the two problems canbe solved in parallel in a distributed approach butmust be solved in sequence in a centralizedapproach. A distributed approach also removes sin-gle points of failure, such as the centralized server,which increases robustness.

Although there are many distributed problem-solving models, we focus our scope in this articleon distributed constraint-reasoning (DCR) modelssuch as distributed constraint-satisfaction prob-lems (DCSPs)1 and distributed constraint-opti-mization problems (DCOPs). The DCR modelshave a rich history and have been used to model awide variety of distributed problems including thedistributed scheduling of jobs in a job shop (Sycaraet al. 1991), the distributed scheduling of meetings(Maheswaran et al. 2004; Zivan 2008), the distrib-uted allocation of targets to sensors in a network(Zhang et al. 2003; Zivan, Glinton, and Sycara2009), the distributed allocation of resources indisaster evacuation scenarios (Lass et al. 2008), the

distributed management of power distribution net-works (Kumar, Faltings, and Petcu 2009), the dis-tributed generation of coalition structures (Ueda,Iwasaki, and Yokoo 2010), and the distributedcoordination of logistics operations (Léauté andFaltings 2011).

A DCSP or DCOP can be visualized as a graph,where nodes are agents and edges are constraintsthat represent interactions between neighboringagents. If we model the decentralized channel allo-cation problem as a DCSP, then a constraint isunsatisfied if the two agents sharing that con-straint choose the same channel. The constraint issatisfied otherwise. The goal in a DCSP is to find anallocation of channels to all agents such that allconstraints are satisfied. If we model the problemas a DCOP, then a constraint incurs a cost of infin-ity if the two agents sharing that constraint choosethe same channel. The constraint incurs a finitecost otherwise. Each pair of nonconflicting chan-nels typically has a different cost to reflect thechannel preferences. The channel preferencesmight arise due to interference from externalsources (Clifford and Leith 2007). The goal in aDCOP is to find an allocation of channels to allagents such that the sum of the costs of all con-straints is minimized.

Our emphasis on DCR models in this articlecomes at the unfortunate cost of neglecting manyother topics within the distributed problem-solv-

Articles

54 AI MAGAZINE

Figure 1. Mobile WLAN Access Points.

ing subfield. For example, in this article, we assumeeach agent already has its own subtask. In ourdecentralized channel allocation problem, the sub-task of each agent is to assign its own channel.However, in general, there can be a situation wherea team of agents has a global task to accomplish,but how that task can be divided into subtasks andwhich agent should be responsible for each subtaskare not predetermined. Thus, the agents mustdecide how to do so. The research topic related tosuch a situation is called task sharing. A classic,representative approach in task sharing is the con-tract net protocol (Davis and Smith 1983). Also,when the subtask of each agent is complex andinterdependent with other subtasks, rather thansolving its subtask independently, agents shouldprovide each other with the information of partialsolutions as they work toward a solution. Such anapproach is called result sharing, which canimprove the accuracy of the obtained solutionsince agents can cross-check their partial solutionsand obtain a better overall view (Lesser and Corkill1981). In this article, we first describe centralizedconstraint-reasoning models before describingtheir distributed counterparts. We then describeother distributed problem-solving models and con-clude this article.

Centralized Constraint Reasoning There are two types of centralized constraint-rea-soning models, namely constraint-satisfactionproblems (CSPs) and constraint-optimizationproblems (COPs).2 Both CSPs and COPs are definedby a tuple 〈V, D, F〉, where V = {vi, ... } is a finite setof variables; D = {Di, ... } is a finite set of domains,where Di is the finite set of possible values for vari-able vi; and F = ⋃ Fi is a finite set of constraints,where Fn is the set of n-ary constraints. We assumethat all constraints in this article are binary con-

straints, that is, they involve only two variables, forsimplicity. Thus, F = F2 = {fij ... }, where fij : Di × Djis a binary constraint between variables vi and vj.Each constraint in a CSP returns a boolean value ofsatisfied or unsatisfied and each constraint in aCOP returns a numeric cost. A solution for theproblem is an assignment of values to variables. Anoptimal solution is an assignment where all con-straints are satisfied if the problem is a CSP or thesum of the costs of all constraints is minimized ifthe problem is a COP. Finding optimal solutions inboth problems is NP-hard.

Both problems can be visualized as a constraintgraph, where nodes are variables and edges areconstraints between variables. Figure 2b shows thegraph for our example decentralized channel allo-cation problem and figure 2d shows the cost ofeach pair of variables for a COP. If the problem is aCSP, we assume that the constraint is satisfiedwhen its cost is a finite amount. We will use thisexample throughout this article.

CSP Algorithms There are two classes of CSP algorithms, namelyinference algorithms and search algorithms. Infer-ence algorithms repeatedly simplify the CSP intoan equivalent CSP that is typically easier to solvewhile search algorithms enumerate the space ofpossible value assignments. We will describe com-plete and incomplete algorithms that have DCSPcounterparts for each class. Table 1 shows the algo-rithms described in this article.

Complete CSP Inference AlgorithmsWe describe bucket elimination (BE) (Dechter1999) as a representative algorithm of this class. Itfirst constructs a pseudotree, which is a spanningtree of the constraint graph with the property thatedges in the constraint graph connect a vertexwith one of its ancestors or descendants in the con-

Articles

FALL 2012 55

a c

v1 v2 v1 v3 v2 v3Costa a ∞a b 0b a 5b b ∞c a 10c b 10

f12 f13 1023

Costa b 0a c 0b b ∞b c 5c b 10c c ∞

Costa b 0a c 0b b ∞b c 0

d

v1

v2

v3

v2

b

v1

v3

Figure 2. Example WLAN Problem.

(a) Illustration (b) Constraint Graph (c) Pseudotree.

straint tree (Freuder and Quinn 1985, Bayardo andMiranker 1995). An edge of the constraint graphthat is not part of the pseudotree is called abackedge. Figure 2c shows a pseudotree for ourexample problem, where the dotted line is abackedge. The algorithm then starts the variableelimination phase by repeatedly eliminating a vari-able upwards from the leaves of the pseudotree. Iteliminates a variable by substituting the con-straints involving the eliminated variable withnew constraints that do not involve that variable.Tables 2a and 2b show the two steps of the algo-rithm eliminating variable v3. It first joins the con-straints that involve variable v3, namely con-straints f13 and f23, by taking all combinations ofpairs of values with the same value for variable v3,resulting in constraint f13 ⋈ f23. For example, thepairs of values (v1 = a, v3 = b) from constraint f13and (v2 = a, v3 = b) from constraint f23 are combinedinto the tuple (v1 = a, v2 = a, v3 = b) in constraint f13⋈ f23. The algorithm then projects that constraintdown on variable v3 by removing the values ofvariable v3 from the constraint, resulting in con-straint (f13 ⋈ f23) ⇓ v3.

This phase continues until every variable exceptthe root of the pseudotree is eliminated, at whichpoint the algorithm starts the variable assignment

phase by repeatedly assigning values to variablesdownwards from the root. The algorithm assignsthe root a value from its set of values that do notviolate any constraint and breaks ties randomly. Inour example, variable v1 is assigned the value a,which is shown in boldface in table 2d. The childof the root is then assigned a value from its set ofvalues given the value assignment of the root. Inour example, variable v2 is assigned the value b,which is shown in boldface in table 2c. This phasecontinues until all the leaves are assigned valuesgiven the values of all their ancestors in thepseudotree, at which point the CSP is solved.

Incomplete CSP Inference AlgorithmsWe describe arc-consistency algorithm #3 (AC-3)(Mackworth 1977), which simplifies a CSP into anarc-consistent CSP, as a representative algorithm ofthis class. A CSP is arc consistent if every value inthe domain of every variable is viable. A value of avariable is viable iff there exists a support for thatvalue with respect to each binary constraintinvolving that variable. Pairs of values are supportsfor each other with respect to the constraintbetween their respective variables if they satisfythat constraint. Therefore, arc-consistent CSPs typ-ically have variables with smaller domain sizes,which makes them easier to solve. If the domain of

Articles

56 AI MAGAZINE

Constraint reasoning models CSP COP DCSP DCOP

Complete inference algorithms BE BE DPOP DPOP

Incomplete inference algorithms AC-3 W-AC3 DisAC-9 Distributed SAC

Complete search algorithms BT AOBB ABT ADOPT

Incomplete search algorithms BA BA DBA DBA

v1 v2 v3 v1 v2 v1 v2 v1

a a b a a a b a

a a c a b b a b

a b c b a c a c

b a c b b

b b c c a

c a b

a b c d

Table 1. Centralized and Distributed Constraint-Reasoning Algorithms.

Table 2. CSP Variable Elimination Phase.

a. f13 ⋈ f23. b. (f13 ⋈ f23) ⇓v3. c. f12 ⋈ (f13 ⋈ f23) ⇓3. d. (f12 ⋈ (f13 ⋈ f23) ⇓ v3) ⇓v2

one variable in an arc-consistent CSP is empty,then the CSP does not have an optimal solution. Ifthe domains of all variables contain exactly onevalue, then an assignment of those values to theirrespective variables is a unique optimal solutionfor the CSP. Otherwise, there are multiple optimalsolutions for the CSP and a complete algorithm isneeded to find one.

Complete CSP Search AlgorithmsWe describe a depth-first branch-and-bound algo-rithm called Backtracking (BT) as a representativealgorithm of this class. Like BE, it also operates ona pseudotree. It sequentially assigns values to vari-ables downwards from the root of the pseudotreeuntil no value can be assigned to a variable with-out violating a constraint. The algorithm thenbacktracks, that is, changes the value of the mostrecently assigned variable. This process continuesuntil it has assigned a value to all variables, inwhich case it has found an optimal solution, or ithas tried all possible combination of values, inwhich case it reports that there are no optimalsolutions.

Incomplete CSP Search AlgorithmsWe describe a hill-climbing algorithm called theBreakout Algorithm (BA) (Morris 1993) as a repre-sentative algorithm of this class. It starts with aninitial assignment of values to all variables and aninitial weight of one to all constraints. It uses thesum of the weights of all violated constraints in asolution as the evaluation of the solution. If theinitial assignment violates some constraint, thenthe algorithm chooses a variable and changes itsvalue such that the evaluation of the solution isreduced. This process continues until the evalua-tion of the solution is zero, in which case it hasfound an optimal solution, or the time limit of thealgorithm is reached, in which case it returns thebest solution found. When the algorithm detectsthat it is in a local minima, it increases the weightof all violated constraints by one so that the eval-uation of the current solution is larger than its

neighboring solutions.

COP Algorithms Like CSP algorithms, there are also two classes ofCOP algorithms, namely inference algorithms andsearch algorithms. We will describe complete andincomplete algorithms that have DCOP counter-parts for each class.

Complete COP Inference AlgorithmsWe describe an extension of BE (Dechter 1999),which we described earlier, as a representative algo-rithm of this class. Like BE, its extension also elim-inates variables upwards from the leaves of thepseudotree in the variable elimination phase andassigns values to variables downwards from theroot of the pseudotree in the variable assignmentphase.

The difference in the variable elimination phaseis that each new constraint that substitutes theconstraints involving the eliminated variable nowhas costs. Tables 3a and 3b show the two steps ofthe algorithm eliminating variable v3. It first joinsthe constraints that involve variable v3, namelyconstraints f13 and f23, resulting in constraint f13 ⋈f23. The cost in each row is the sum of the corre-sponding costs in constraints f13 and f23. For exam-ple, the cost of (v1 = c, v2 = a, v3 = b) is the sum ofthe cost of (v1 = c, v3 = b) in constraint f13 and thecost of (v2 = a, v3 = b) in constraint f23. It then proj-ects that constraint down on variable v3 by remov-ing the values of variable v3 from the constraint,resulting in constraint (f13 ⋈ f23) ⇓ v3. The cost ineach row is the minimum over all correspondingcosts in the preprojected constraint. For example,the cost of (v1 = a, v2 = a) is the minimum over thecosts of (v1 = a, v2 = a, v3 = b) and (v1 = a, v2 = a, v3= c) in the unprojected constraint.

The difference in the variable assignment phaseis that each variable is now assigned a value suchthat the cost is minimized given the values of all itsancestors. In our example, variable v1 is assignedthe value a, which has the minimal cost, as shown

Articles

FALL 2012 57

v1 v2 v3 Cost v1 v2 Cost v1 v2 Cost v1 Cost a a b 0 a a 0 a b 0 a 0 a a c 0 a b 0 b a 10 b 10 a b c 0 b a 5 c a 20 c 20 b a c 5 b b 5 b b c 5 c a 10 c a b 10

a b c d

Table 3. COP Variable Elimination Process.

a. f13 ⋈ f23. b. (f13 ⋈ f23) ⇓v3. c. f12 ⋈ (f13 ⋈ f23) ⇓v3. d. (f12 ⋈ (f13 ⋈ f23) ⇓v3) ⇓v2

in table 3d. This phase continues until all theleaves are assigned values given the values of alltheir ancestors in the pseudotree, at which pointthe COP is solved.

Incomplete COP Inference AlgorithmsWe describe an extension of AC-3, which wedescribed earlier, called weighted AC-3 (W-AC3)(Larrosa and Schiex 2004), as a representative algo-rithm of this class. Like AC-3, W-AC3 also simpli-fies a COP into a soft arc-consistent COP. The defi-nition of soft arc consistency is similar to thedefinition of arc consistency with the exception ofthe definition of supports. In COPs, members in apair of values are supports for each other withrespect to the constraint between their respectivevariables if the cost of that pair of values in theconstraint is finite (Cooper and Schiex 2004, Lar-rosa and Schiex 2004).

Complete COP Search AlgorithmsWe describe an extension of BT, which wedescribed earlier, called AND/OR Branch-and-Bound (AOBB) (Marinescu and Dechter 2009), as arepresentative algorithm of this class. Like BT,AOBB also assigns values to variables in a depth-first manner. The difference is that it now main-tains lower and upper bounds and uses them tobacktrack. The lower bound is the sum of costs ofall constraints involving variables with assignedvalues, and the upper bound is cost of the bestsolution found so far. The algorithm backtrackswhen the lower bound is no smaller than theupper bound.

Incomplete COP Search AlgorithmsWe describe a simple extension of BA (Morris1993), which we described earlier, as a representa-tive algorithm of this class. Like BA, its extensionalso starts with an initial assignment of values toall variables and repeatedly changes the value of avariable such that the evaluation of the solution isreduced. The difference is that it now uses the sumof constraint costs in a solution as the evaluationof the solution.

Distributed Constraint Reasoning There are two types of distributed constraint-rea-soning models, namely distributed CSPs (DCSPs)and distributed COPs (DCOPs), which extend CSPsand COPs, respectively. Both DCSPs and DCOPs aredefined by a tuple 〈A, V, D, F, �〉, where V, D, and Fare sets of variables, domains, and constraints,respectively, like in CSPs and COPs. A = {ai, ... } is afinite set of agents and �: V → A is a function thatmaps each variable to an agent that owns it. In thisarticle, we assume that each agent owns exactly onevariable. Thus, we will use the terms agents and vari-ables interchangeably. Each agent is responsible forassigning values to its variable based only on its

knowledge of the constraints involving its variableand messages that it can exchange with otheragents. These messages can be delayed by a finiteamount of time but are never lost.

DCSP Algorithms Like CSP algorithms, there are also two classes ofDCSP algorithms, namely inference and searchalgorithms. We will describe one complete and oneincomplete algorithm for each class.

Complete DCSP Inference AlgorithmsWe describe an extension of BE, which wedescribed earlier, called Distributed PseudotreeOptimization Procedure (DPOP) (Petcu and Falt-ings 2005b), as a representative algorithm of thisclass. DPOP was actually designed to solve DCOPs,but it can be used to solve DCSPs as well. Like BE,DPOP also operates on a pseudotree. Thus, it firstcalls existing distributed pseudotree constructionalgorithm like Distributed DFS (Hamadi, Bessière,and Quinqueton 1998) to construct a pseudotree.

At a high level, the key ideas of DPOP are as fol-lows: When BE eliminates a variable, it knows theprojected constraint because it is a centralizedalgorithm. Unfortunately, each agent in DPOP isonly aware of the constraints in the original prob-lem that it is involved in. Thus, each agent sendsUTIL messages containing its projected constraintafter it eliminates its variable. When BE assigns avalue to a variable, it also knows the good valuesfor the current unassigned variable, that is, the val-ues that do not violate any constraints given thevalue assignment of all its ancestors. Unfortunate-ly, each agent in DPOP is only aware of its own val-ue assignment and it thus does not know its goodvalues. Thus, each agent sends VALUE messagescontaining its value after it assigns itself.

We now describe DPOP in more detail. There aretwo phases in the operation of DPOP. The firstphase, called the UTIL phase, is similar to the CSPvariable-elimination phase. The difference now isthat each agent eliminates its own variable andsends a UTIL message containing the projectedconstraint up to its parent agent. In our example,agent v3 performs the join and projection opera-tions as shown in tables 2a and 2b and sends aUTIL message containing the projected constraintto its parent agent v2. This phase continues untilthe root agent receives a projected constraint fromeach of its child agents in the pseudotree.

At the end of the variable elimination phase,DPOP starts the second phase, called the VALUEphase, which is similar to the CSP variable assign-ment phase. The difference now is that each agentassigns itself a value and sends a VALUE messagecontaining its value down to each of its descen-dant agents that it shares a constraint with. Theroot agent starts this phase by assigning itself a val-ue from its set of values that do not violate any

Articles

58 AI MAGAZINE

constraint. In our example, agent v1 assigns itselfthe value a as shown in table 2d and sends a VAL-UE message containing its value to its child agentv2. Upon receipt of the message, agent v2 assignsitself a value from its set of values given that agentv1 assigned itself the value in the VALUE message.In our example, agent v2 assigns itself the value bgiven that agent v1 assigned itself the value a. Thisphase continues until the leaf agents assign them-selves values given the values of all their ancestoragents in the pseudotree, at which point the DCSPis solved.

The number of messages sent between agents inDPOP is linear in the number of agents. However,the size of the messages and the memory require-ment of each agent are exponential in the inducedwidth of the DCOP.

Incomplete DCSP Inference AlgorithmsThere exist distributed arc-consistency-based algo-rithms, such as distributed arc-consistency algo-rithm #9 (DisAC-9) (Hamadi 2002). Like AC-3, Dis-AC-9 also simplifies a DCSP into an arc-consistentDCSP. The difference is that this simplification isnow done in a distributed manner; each agentknows only about the constraints involving itsvariable and must thus communicate with neigh-boring agents to exchange information.

Complete DCSP Search AlgorithmsA classic complete search algorithm for DCSP is anextension of BT, which we described earlier, calledasynchronous backtracking (ABT) (Yokoo et al.1992, 1998). Like BT, ABT also assigns values tovariables in a depth-first manner and backtrackswhen a constraint is violated. The difference is thatthey are done in a distributed and asynchronousmanner.

At a high level, the key ideas of ABT are as fol-lows: When BT assigns a value to a variable, itknows the current value assignment of all variablesbecause it is a centralized algorithm and it thusknows the good values for the current unassignedvariable, that is, the values that do not violate anyconstraints given the value assignment of all itsancestors. Unfortunately, when ABT assigns a val-ue to a variable, each agent is only aware of its ownvalue assignment and it thus does not know itsgood values. Thus, each agent sends OK? messagescontaining its value after it assigns itself. When BTbacktracks, it can also infer the cause of the con-straint violations because it knows the current val-ue assignment of all variables. Unfortunately,when ABT backtracks, the agent to which ABTbacktracks does not know the cause of the con-straint violations since it does not know theassumed value assignments of the ancestor agentsof the backtracking agent. Thus, each agent sendsNOGOOD messages containing its assumed valueassignments of its ancestor agents when it back-tracks.

We now describe ABT in more detail. Each agentmaintains an agent view, which is a set of agent-value pairs that represents the agent’s assumptionof its ancestor agents’ assigned values. The algo-rithm starts by having the root agent assign itself avalue and sends an OK? message containing its val-ue to each of its descendant agents that it shares aconstraint with. Upon receipt of an OK? message,the receiving agent updates its agent view to reflectthe latest value of the sending agent. If there existsa value in its domain that does not violate any con-straint given that its ancestors are assigned valuesaccording to its agent view, it assigns itself that val-ue and sends an OK? message to each of its descen-dant agents that it shares a constraint with. If nosuch value exists, then it backtracks by sending aNOGOOD message containing its agent view to anancestor agent. Upon receipt of a NOGOOD mes-sage, if they are compatible, that is, they agree onall common agent-value pairs, then the receivingagent changes its value to a different value thatdoes not violate any constraint given the assump-tion that agents are assigned values according totheir own agent view and the agent view in theNOGOOD message. If they are incompatible, thenthe receiving agent backtracks. This process con-tinues until either all agents are assigned valuesand no NOGOOD messages are sent, at whichpoint the algorithm has found an optimal solu-tion, or the root agent receives a NOGOOD mes-sage for each of its values, at which point the algo-rithm can conclude that there is no optimalsolution for the problem.

The number of messages sent between agents inABT is exponential in the number of agents. How-ever, the size of the messages and the memoryrequirement of each agent are linear in the numberof agents. Researchers have developed variousextensions of this algorithm including extensionswith dynamic ordering of agents in the pseudotree(Yokoo 1995; Zivan and Meisels 2005).

Incomplete DCSP Search AlgorithmsA classic incomplete DCSP search algorithm is anextension of BA, which we described earlier, calledthe distributed breakout algorithm (DBA) (Hiraya-ma and Yokoo 2005). Like BA, DBA also starts withan initial assignment of values to all variables andan initial weight of one to all constraints. The dif-ference is that this assignment is now done in adistributed manner; each agent assigns itself a val-ue randomly, sends that value in OK? messages toits neighboring agents and assigns a weight of oneto each constraint that it is involved in. Addition-ally, each agent now sums up the weights of vio-lated constraints that it is involved in only anduses that as the evaluation of its value. Each agentalso calculates its gain, that is, the possibleimprovement in the evaluation if it changes its val-ue, and sends an IMPROVE message containing

Articles

FALL 2012 59

that gain to each of its neighboring agents. Toguarantee that the (global) solution improves, onlythe agent with the highest gain amongst its neigh-boring agents can change its value. However, notethat two nonneighboring agents can change theirvalues concurrently. Researchers have developedvarious extensions of this algorithm includingextensions where any agent with a positive gaincan change its value with a given probability (Fitz-patrick and Meertens 2003; Zhang and Wittenburg2002).

DCOP Algorithms Like COP algorithms, there are also two classes ofDCOP algorithms, namely inference and searchalgorithms. We will describe one complete and oneincomplete algorithm for each class.

Complete DCOP Inference AlgorithmsAs BE can be used to solve both CSPs and COPs,DPOP, which we described earlier for solvingDCSPs, can also be used to solve DCOPs. There arethe same two phases in the operation of the ver-sion of DPOP that solves DCOPs, where the UTILphase is similar to the COP variable-eliminationphase and the VALUE phase is similar to the COPvariable-assignment phase.

The number and size of messages and the mem-ory requirement of each agent in this version ofDPOP are the same as those in the version thatsolves DCSPs. Researchers have developed variousextensions of this algorithm including extensionsthat trade between memory requirement and com-putation time (Petcu and Faltings 2007a; Petcu,Faltings, and Mailler 2007) or solution optimality(Petcu and Faltings 2005a) and an extension thatspeeds up DPOP by using function filtering (Britoand Meseguer 2010).

Incomplete DCOP Inference AlgorithmsThere exist distributed versions of arc-consistency-based algorithms called distributed soft arc-consis-tency (SAC) algorithms (Matsui et al. 2009; Gutier-rez and Meseguer 2010). Like W-AC3, thesealgorithms also simplify a DCOP into a soft arc-consistent DCOP. The difference is that this sim-plification is now done in a distributed manner;each agent knows only about the constraintsinvolving its variable and must thus communicatewith neighboring agents to exchange information.

Complete DCOP Search AlgorithmsA classic complete DCOP search algorithm is calledAsynchronous Distributed Optimization (ADOPT)(Modi et al. 2005). ADOPT can be considered as avariation of AOBB, but it was developed before it.Like AOBB, ADOPT also uses lower and upperbounds to backtrack. The difference is that it isnow done in a distributed and asynchronous man-ner.

At a high level, the key ideas of ADOPT are sim-ilar to those of ABT: Each agent sends VALUE mes-

sages (instead of OK? messages) containing its val-ue after it assigns itself and it sends COST messages(instead of NOGOOD messages) containing itsassumed value assignments of its ancestor agentswhen it backtracks. The difference is that COSTmessages also include the lower and upper boundsin addition to the assumed value assignments ofthe ancestor agents.

We now describe ADOPT in more detail. Eachagent vi maintains its current value di; its currentcontext Xi, which is a set of agent-value pairs thatrepresents the agent’s assumption of the currentvalues of its ancestor agents; the lower and upperbounds LBi and UBi, which are bounds on the opti-mal cost OPTi given that its ancestors take on theirrespective values in Xi; the lower and upperbounds LBi (d) and UBi(d) for all values d � Di,which are bounds on the optimal costs OPTi(d) giv-en that agent vi takes on the value d and its ances-tor agents take on their respective values in Xi; andthe lower and upper bounds lbi

c(d) and ubic(d) for

all values d � Di and child agents vc, which are itsassumption on the bounds LBc and UBc of its childagents vc with context Xi ∪ (vi, d). The optimalcosts are calculated using:

for all values d � Di, where Ci is the set of childagents of agent vi and �i(d) is the sum of the costsof all cost constraints between agent vi and itsancestor agents given that agent vi takes on the val-ue d and the ancestors take on their respective val-ues in Xi.

At the start, each agent vi initializes its currentcontext Xi to ∅, lower and upper bounds lbi

c(d) andubi

c(d) to 0 and ∞, respectively. For all values d � Diand all child agents vc, agent vi calculates theremaining lower and upper bounds and takes onits best value using:

Agent vi sends a VALUE message containing itsvalue di to each of its descendant agents that itshares a constraint with. It also sends a COST mes-sage containing its context Xi and its bounds LBiand UBi to its parent agent. Upon receipt of a VAL-UE message, if its current context Xi is compatible

OPTi(d)= !i(d)+ OPTcvc"Ci

#OPTi =mind"Di

OPTi(d)

!i(d)= fij(vj ,dj )"Xi

# (d,dj )

LBi(d)= !i(d)+ lvc"Ci

# bic(d)

UBi(d)= !i(d)+ uvc"Ci

# bic(d)

LBi =mind"DiLBi(d){ }

UBi =mind"DiUBi(d){ }

di = argmind"DiLBi(d){ }

Articles

60 AI MAGAZINE

with the value in the VALUE message, it updates itscontext to reflect the new value of its ancestoragent and reinitializes its lower and upper boundslbi

c(d) and ubic(d). Upon receipt of a COST message

from child agent vc, if its current context Xi is com-patible with the context in the message, then itupdates its lower and upper bounds lbi

c(d) andubi

c(d) to the lower and upper bounds in the mes-sage, respectively. Otherwise, the COST message isdiscarded. After processing either message, it recal-culates the remaining lower and upper bounds andtakes on its best value using the above equationsand sends VALUE and COST messages. This processrepeats until the root agent vr reaches the termina-tion condition LBr = UBr, which means that it hasfound the optimal cost.

The number and size of messages and the mem-ory requirement of each agent in ADOPT are thesame as those in ABT. Researchers have developedvarious extensions of this algorithm including anextension that incorporates elements of ABT likeNOGOOD messages (Silaghi and Yokoo 2009), anextension that uses depth-first search instead ofbest-first search (Yeoh, Felner, and Koenig 2010),an extension that trades between computationtime and solution optimality (Yeoh, Sun, andKoenig 2009), and an extension that tradesbetween memory requirement and computationtime (Matsui, Matsuo, and Iwata 2005; Yeoh,Varakantham, and Koenig 2009).

Incomplete DCOP Search AlgorithmsAs BA can be used to solve both CSPs and COPs,DBA, which we described earlier for solving DCSPs,can also be used to solve DCOPs. Like the versionthat solves DCSPs, each agent in this version alsoassigns itself a value randomly and sends an OK?message containing that value to each of its neigh-boring agents, calculates its gain from changingvalues, sends an IMPROVE message containingthat gain to each of its neighboring agents, andchanges its value if it has the highest gain amongstits neighboring agents. The difference is that eachagent now uses the cost of a constraint as theweight of that constraint in the evaluation of itscurrent value.

Recent Extensions Researchers have extended DCR models and algo-rithms in various directions to more accuratelymodel and solve real-world problems. We nowdescribe some of these extensions.

Privacy-Preserving AlgorithmsOne of the motivations for modeling problems likethe distributed scheduling of meetings with theDCR model is the preservation of privacy. Forexample, two users that do not need to schedule ameeting between them should not have access toeach other’s meeting time preferences. Unfortu-nately, privacy loss is often unavoidable because

the agents exchange messages that include aggre-gated constraint cost information. The exceptionis if the agents use obfuscation or encryptionmethods (Yokoo, Suzuki, and Hirayama 2005; Falt-ings, Léauté, and Petcu 2008; Léauté and Faltings2009, 2011). Researchers have thus introduced sev-eral metrics for measuring privacy loss in DCRalgorithms (Greenstadt, Pearce, and Tambe 2006)as well as extensions of DCR algorithms that pre-serve more privacy (Greenstadt, Grosz, and Smith2007; Greenstadt 2009).

Dynamic DCR modelsMany multiagent coordination problems like thedistributed management of power distribution net-works occur in dynamically changing environ-ments. For example, power lines might fail or pow-er requirements of a district might change.Researchers have thus extended DCR models todynamic DCR models like dynamic DCSPs (Mailler2005; Omomowo, Arana, and Ahriz 2008) anddynamic DCOPs (Petcu and Faltings 2005c; Sul-tanik, Lass, and Regli 2009; Yeoh et al. 2011). Atypical model of a dynamic DCR problem is asequence of (static) DCR problems with changesfrom one DCR problem to the next one in thesequence. The advantage of this approach is thatsolving a dynamic DCR problem is no harder thansolving multiple (static) DCR problems. Otherrelated extensions include a continuous-time mod-el where agents have deadlines to choose their val-ues (Petcu and Faltings 2007b) and a model whereagents can have imperfect knowledge about theirenvironment (Lass, Sultanik, and Regli 2008).

Multiobjective DCR ModelsMany multiagent coordination problems like thedistributed planning of truck routes can have mul-tiple objectives that need to be optimized. Forexample, two possible conflicting objectives mightbe the length of the route and the financial cost ofthe route (due to tolls); short routes are expensiveand long routes are cheap. Researchers have thusextended DCOPs to multiobjective DCOPs, whereeach pair of values in a constraint has multiplecosts, one for each objective (Delle Fave et al.2011). The goal in such a problem is to find a Pare-to-optimal solution. Other related extensionsinclude resource-constrained DCOPs, where eachpair of values in a constraint has an additionalresource cost (Bowring, Tambe, and Yokoo 2006;Matsui et al. 2008). The goal in such a problem isto find an optimal solution among the set of solu-tions whose sum of resource costs is within anupper bound.

Quantified DCR ModelsMany multiagent coordination problems like thedistributed tracking of targets in a sensor networkcan include adversarial agents. For example, thetarget might try to avoid detection and is thusadversarial. Researchers have thus extended DCR

Articles

FALL 2012 61

models to quantified DCR models (Baba et al.2010, Matsui et al. 2010). Quantified DCR modelsallow some agents in the problem to be adversari-al, that is, they can assign themselves any valueregardless whether the value will increase ordecrease the overall solution cost. The goal in sucha problem is to find an optimal solution given thatall the adversarial agents will assign themselvesthe worst possible values, which increase the over-all solution cost the most. Other related exten-sions include an algorithm that explicitly modelsthe deception of adversarial agents (Lisy et al.2010).

Other Models Aside from distributed constraint-reasoning mod-els, researchers have developed other distributedproblem-solving models to capture different char-acteristics of common distributed problems. Wenow describe some of these models.

DEC-POMDPsPartially observable Markov decision processes(POMDPs) (Smallwood and Sondik 1973) havebeen shown to be popular models for modelingcentralized sequential decision making underuncertainty problems. As a result, decentralizedPOMDPs (DEC-POMDPs) (Bernstein et al. 2002)have emerged as a natural extension for modelingdecentralized sequential decision making underuncertainty problems. An example application isthe cooperative multirobot navigation problem,where the robots (agents) need to cooperatively getto their respective goal locations. They have actu-ators for locomotion, but their movements dependon factors like wheel alignment and slippage. Theyalso have sensors for sensing their environment,but their sensory data depend on factors like sen-sor noise. Despite the uncertainty, they need tofind a sequence of actions that will result in thelargest likelihood of reaching their respective goallocations.

DCR models are ill-suited for modeling thisproblem as they do not take uncertainty intoaccount. Thus, a DEC-POMDP is a better-suitedmodel in such a problem. Unfortunately, theincreased richness of DEC-POMDPs compared toDCR models comes at a price of higher complexity— finding optimal DEC-POMDP solutions isNEXP-hard (Bernstein et al. 2002). Researchershave thus proposed specialized models that lever-age sparse agent interactions to improve the scala-bility of DEC-POMDP algorithms (Nair et al. 2005;Kumar and Zilberstein 2009; Oliehoek et al. 2008;Velagapudi et al. 2011).

AuctionsAuction-based approaches are also popular for

modeling decentralized coordination problems(Koenig, Keskinocak, and Tovey 2010). In an auc-tion-based algorithm, the auctioneer calls for bidsfor tasks/resources and the agents bid accordingto their valuation of the tasks and their capabili-ties of performing those tasks. Generally, eachagent maximizes its own payoff, which is definedas its income (from performing its task) minus itscost (of performing its task). Thus, the primarydifference between auctions and the DCR modelsis that auction-based algorithms expect that coor-dinated behaviors of agents emerge through theindividual optimization of each agent. However,unlike game-theoretic approaches, we usuallyassume that the agents always bid truthfully inde-pendent of the game mechanism. Auction-basedapproaches have been applied to solve the dis-tributed processing of streaming data (An,Douglis, and Ye 2008), the distributed allocationof resources in cloud computing centers (An et al.2010), and the distributed allocation of roles inRoboCup soccer (Frías-Martínez, Sklar, and Par-sons 2004).

Conclusions In this article, we gave a brief overview of distrib-uted problem solving. Due to space limitations, wefocused on distributed constraint-reasoning (DCR)models, which are problems where each agent hasa fixed, finite set of possible actions, and theagents try to find a combination of individualactions that satisfies or optimizes some global cri-teria. Distributed problem solving has a longresearch history and the topics covered in this arti-cle are very limited. We encourage readers to referto other articles that discuss other issues in dis-tributed problem solving (Lesser and Corkill 1981;Davis and Smith 1983; Durfee 1999). Researchershave developed several testbeds (Ezzahir et al.2007; Sultanik, Lass, and Regli 2007; Léauté,Ottens, and Szymanek 2009) for running DCRalgorithms including DisChoco3, DCOPolis4, andFRODO5. These testbeds include implementationsfor a variety of popular DCR algorithms (includ-ing most of the algorithms described in this arti-cle), evaluation domains and evaluation metrics.We encourage readers who are interested in dis-tributed constraint reasoning to download and tryout these testbeds.

Notes 1. Researchers have also used DisCSPs to refer to distrib-uted CSPs and DCSPs to refer to dynamic CSPs.

2. COPs are also known as weighted CSPs (Schiex, Fargi-er, and Verfaillie 1995; Bistarelli et al. 1999).

3. www.lirmm.fr/coconut/dischoco.

4. www.dcopolis.org.

5. sourceforge.net/projects/frodo2.

Articles

62 AI MAGAZINE

References An, B.; Douglis, F.; and Ye, F. 2008. Heuristics for Negoti-ation Schedules in Multiplan Optimization. In Proceed-ings of the Seventh International Joint Conference onAutonomous Agents and Multiagent Systems, 551–558. Rich-land, SC: International Foundation for AutonomousAgents and Multiagent Systems.

An, B.; Lesser, V.; Irwin, D.; and Zink, M. 2010. Automat-ed Negotiation with Decommitment for DynamicResource Allocation in Cloud Computing. In Proceedingsof the Ninth International Joint Conference on AutonomousAgents and Multiagent Systems, 981–988. Richland, SC:International Foundation for Autonomous Agents andMultiagent Systems.

Baba, S.; Iwasaki, A.; Yokoo, M.; Silaghi, M.; Hirayama, K.;and Matsui, T. 2010. Cooperative Problem SolvingAgainst Adversary: Quantified Distributed Constraint Sat-isfaction Problem. In Proceedings of the Ninth Internation-al Joint Conference on Autonomous Agents and MultiagentSystems, 781–788. Richland, SC: International Founda-tion for Autonomous Agents and Multiagent Systems.

Bayardo, R., and Miranker, D. 1995. On the Space-TimeTrade-Off in Solving Constraint Satisfaction Problems. InProceedings of the 14th International Joint Conference on Arti-ficial Intelligence, 558–562. San Francisco: Morgan Kauf-mann Publishers.

Bernstein, D.; Givan, R.; Immerman, N.; and Zilberstein,S. 2002. The Complexity of Decentralized Control ofMarkov Decision Processes. Mathematics of OperationsResearch 27(4): 819–840.

Bistarelli, S.; Montanari, U.; Rossi, F.; Schiex, T.; Verfaillie,G.; and Fargier, H. 1999. Semiring-Based CSPs and ValuedCSPs: Basic Properties and Comparison. Constraints 4(3):199–240.

Bowring, E.; Tambe, M.; and Yokoo, M. 2006. Multiply-Constrained Distributed Constraint Optimization. In Pro-ceedings of the Fifth International Joint Conference onAutonomous Agents and Multiagent Systems, 1413–1420.New York: Association for Computing Machinery.

Brito, I., and Meseguer, P. 2010. Improving DPOP withFunction Filtering. In Proceedings of the Ninth Internation-al Joint Conference on Autonomous Agents and MultiagentSystems, 141–158. Richland, SC: International Founda-tion for Autonomous Agents and Multiagent Systems.

Clifford, P., and Leith, D. 2007. Channel DependentInterference and Decentralized Colouring. In Proceedingsof the International Conference on Network Control and Opti-mization, Lecture Notes in Computer Science Volume4465, 95–104. Berlin: Springer.

Cooper, M., and Schiex, T. 2004. Arc Consistency for SoftConstraints. Artificial Intelligence 154(1-2): 199–227.

Davis, R., and Smith, R. 1983. Negotiation as a Metaphorfor Distributed Problem Solving. Artificial Intelligence20(1): 63–109.

Dechter, R. 1999. Bucket Elimination: A Unifying Frame-work for Reasoning. Artificial Intelligence 113(1–2): 41–85.

Delle Fave, F.; Stranders, R.; Rogers, A.; and Jennings, N.2011. Bounded Decentralised Coordination over Multi-ple Objectives. In Proceedings of the Tenth InternationalJoint Conference on Autonomous Agents and Multiagent Sys-tems, 371–378. Richland, SC: International Foundationfor Autonomous Agents and Multiagent Systems.

Durfee, E. 1999. Distributed Problem Solving and Plan-ning. In Multiagent Systems: A Modern Approach to Distrib-uted Artificial Intelligence, ed. G. Weiss. Cambridge, MA:The MIT Press. 121–164.

Ezzahir, R.; Bessière, C.; Belaissaoui, M.; and Bouyakhf, E.H. 2007. DisChoco: A Platform for Distributed ConstraintProgramming. Paper presented at the 8th InternationalWorkshop on Distributed Constraint Reasoning. Hyder-abad, India, 8 January.

Faltings, B.; Léauté, T.; and Petcu, A. 2008. Privacy Guar-antees Through Distributed Constraint Satisfaction. InProceedings of the 2008 IEEE/WIC/ACM International Con-ference on Intelligent Agent Technology, 350–358. Los Alami-tos, CA: IEEE Computer Society.

Fitzpatrick, S., and Meertens, L. 2003. Distributed Coor-dination Through Anarchic Optimization. In DistributedSensor Networks: A Multiagent Perspective, ed. V. Lesser, C.Ortiz, and M. Tambe, 257–295. Dordrecht, The Nether-lands: Kluwer.

Freuder, E., and Quinn, M. 1985. Taking Advantage ofStable Sets of Variables in Constraint Satisfaction Prob-lems. In Proceedings of the 9th International Joint Conferenceon Artificial Intelligence, 1076–1078. Los Altos, CA:William Kaufmann, Inc.

Frías-Martínez, V.; Sklar, E.; and Parsons, S. 2004. Explor-ing Auction Mechanisms for Role Assignment in Teamsof Autonomous Robots. In RoboCup 2004: Robot SoccerWorld Cup VIII. Lecture Notes in Computer Science 3276,532–539. Berlin: Springer.

Greenstadt, R. 2009. An Overview of Privacy Improve-ments to K-Optimal DCOP Algorithms (ExtendedAbstract). In Proceedings of the Eighth International JointConference on Autonomous Agents and Multiagent Systems,1279–1280. Richland, SC: International Foundation forAutonomous Agents and Multiagent Systems.

Greenstadt, R.; Grosz, B.; and Smith, M. 2007. SSDPOP:Improving the Privacy of DCOP with Secret Sharing. InProceedings of the Sixth International Joint Conference onAutonomous Agents and Multiagent Systems, 1098–1100.Richland, SC: International Foundation for AutonomousAgents and Multiagent Systems.

Greenstadt, R.; Pearce, J.; and Tambe, M. 2006. Analysisof Privacy Loss in DCOP Algorithms. In Proceedings of the21st National Conference on Artificial Intelligence, 647–653.Menlo Park, CA: AAAI Press.

Gutierrez, P., and Meseguer, P. 2010. BnB-ADOPT+ withSeveral Soft Arc Consistency Levels. In Proceedings of the29th European Conference on Artificial Intelligence, 67–72.Amsterdam: IOS Press.

Hamadi, Y. 2002. Optimal Distributed Arc-Consistency.Constraints 7(3-4): 367–385.

Hamadi, Y.; Bessière, C.; and Quinqueton, J. 1998. Dis-tributed Intelligent Backtracking. In Proceedings of the13th European Conference on Artificial Intelligence, 219–223.Chichester, UK: John Wiley and Sons.

Hirayama, K., and Yokoo, M. 2005. The DistributedBreakout Algorithms. Artificial Intelligence 161(1-2): 89–115.

Koenig, S.; Keskinocak, P.; and Tovey, C. 2010. Progresson Agent Coordination with Cooperative Auctions. InProceedings of the 24th AAAI Conference on Artificial Intelli-gence, 1713–1717. Menlo Park, CA: AAAI Press.

Kumar, A., and Zilberstein, S. 2009. Constraint-Based

Articles

FALL 2012 63

Articles

64 AI MAGAZINE

Dynamic Programming for Decentralized POMDPs withStructured Interactions. In Proceedings of the Eighth Inter-national Joint Conference on Autonomous Agents and Multi-agent Systems, 561–568. Richland, SC: International Foun-dation for Autonomous Agents and Multiagent Systems.

Kumar, A.; Faltings, B.; and Petcu, A. 2009. DistributedConstraint Optimization with Structured Resource Con-straints. In Proceedings of the Eighth International Joint Con-ference on Autonomous Agents and Multiagent Systems, 923–930. Richland, SC: International Foundation forAutonomous Agents and Multiagent Systems.

Larrosa, J., and Schiex, T. 2004. Solving Weighted CSP byMaintaining Arc Consistency. Artificial Intelligence 159(1-2): 1–26.

Lass, R.; Kopena, J.; Sultanik, E.; Nguyen, D.; Dugan, C.;Modi, P.; and Regli, W. 2008. Coordination of FirstResponders Under Communication and Resource Con-straints (Short Paper). In Proceedings of the Seventh Inter-national Joint Conference on Autonomous Agents and Multi-agent Systems, 1409–1413. Richland, SC: InternationalFoundation for Autonomous Agents and Multiagent Sys-tems.

Lass, R.; Sultanik, E.; and Regli, W. 2008. Dynamic Dis-tributed Constraint Reasoning. In Proceedings of the 23rdAAAI Conference on Artificial Intelligence, 1466–1469. Men-lo Park, CA: AAAI Press.

Léauté, T., and Faltings, B. 2009. Privacy-Preserving Mul-tiagent Constraint Satisfaction. In Proceedings of the 2009IEEE International Conference on Privacy, Security, Risk andTrust, 17–25. Piscataway, NJ: Institute of Electrical andElectronics Engineers.

Léauté, T., and Faltings, B. 2011. Coordinating LogisticsOperations with Privacy Guarantees. In Proceedings of the22nd International Joint Conference on Artificial Intelligence,2482–2487. Menlo Park, CA: AAAI Press.

Léauté, T.; Ottens, B.; and Szymanek, R. 2009. FRODO2.0: An Open-Source Framework for Distributed Con-straint Optimization. Paper presented at the IJCAI-09 Dis-tributed Constraint Reasoning Workshop, Pasadena, CA,13 July.

Lesser, V., and Corkill, D. 1981. Functionally-Accurate,Cooperative Distributed Systems. IEEE Transactions onSystems, Man and Cybernetics SMC11(1): 81–96.

Lisy, V.; Zivan, R.; Sycara, K.; and Péchoucek, M. 2010.Deception in Networks of Mobile Sensing Agents. In Pro-ceedings of the Ninth International Joint Conference onAutonomous Agents and Multiagent Systems, 1031–1038.Richland, SC: International Foundation for AutonomousAgents and Multiagent Systems.

Mackworth, A. 1977. Consistency in Networks of Rela-tions. Artificial Intelligence 8(1): 99–118.

Maheswaran, R.; Tambe, M.; Bowring, E.; Pearce, J.; andVarakantham, P. 2004. Taking DCOP to the Real World:Efficient Complete Solutions for Distributed Event Sched-uling. In Proceedings of the Third International Joint Confer-ence on Autonomous Agents and Multiagent Systems, 310–317. Piscataway, NJ: Institute of Electrical and ElectronicEngineers.

Mailler, R. 2005. Comparing Two Approaches to Dynam-ic, Distributed Constraint Satisfaction. In Proceedings ofthe Fourth International Joint Conference on AutonomousAgents and Multiagent Systems, 1049–1056. New York:Association for Computing Machinery.

Marinescu, R., and Dechter, R. 2009. AND/OR Branch-and-Bound Search for Combinatorial Optimization inGraphical Models. Artificial Intelligence 173(16–17): 1457–1491.

Matsui, T.; Matsuo, H.; Silaghi, M.; Hirayama, K.; Yokoo,M.; and Baba, S. 2010. A Quantified Distributed Con-straint Optimization Problem. In Proceedings of the NinthInternational Joint Conference on Autonomous Agents andMultiagent Systems, 1023–1030. Richland, SC: Interna-tional Foundation for Autonomous Agents and Multia-gent Systems.

Matsui, T.; Matsuo, H.; and Iwata, A. 2005. EfficientMethods for Asynchronous Distributed Constraint Opti-mization Algorithm. In Proceedings of the IASTED Interna-tional Conference on Artificial Intelligence and Applications,727–732. Calgary, AB, Canada: International Associationof Science and Technology for Development.

Matsui, T.; Silaghi, M.; Hirayama, K.; Yokoo, M.; and Mat-suo, H. 2008. Resource Constrained Distributed Con-straint Optimization with Virtual Variables. In Proceedingsof the 23rd AAA Conference on Artificial Intelligence, 120–125. Menlo Park, CA: AAAI Press.

Matsui, T.; Silaghi, M.; Hirayama, K.; Yokoo, M.; and Mat-suo, H. 2009. Directed Soft Arc Consistency in PseudoTrees. In Proceedings of the Eighth International Joint Con-ference on Autonomous Agents and Multiagent Systems,1065–1072. Richland, SC: International Foundation forAutonomous Agents and Multiagent Systems.

Modi, P.; Shen, W.-M.; Tambe, M.; and Yokoo, M. 2005.ADOPT: Asynchronous Distributed Constraint Optimiza-tion with Quality Guarantees. Artificial Intelligence 161(1-2): 149–180.

Morris, P. 1993. The Breakout Method for Escaping fromLocal Minima. In Proceedings of the 11th National Confer-ence on Artificial Intelligence, 40–45. Menlo Park, CA: AAAIPress.

Nair, R.; Varakantham, P.; Tambe, M.; and Yokoo, M.2005. Networked Distributed POMDPs: A Synthesis ofDistributed Constraint Optimization and POMDPs. InProceedings of the 20th National Conference on ArtificialIntelligence, 133–139. Menlo Park, CA: AAAI Press.

Oliehoek, F.; Spaan, M.; Whiteson, S.; and Vlassis, N.2008. Exploiting Locality of Interaction in Factored Dec-POMDPs. In Proceedings of the Seventh International JointConference on Autonomous Agents and Multiagent Systems,517–524. Richland, SC: International Foundation forAutonomous Agents and Multiagent Systems.

Omomowo, B.; Arana, I.; and Ahriz, H. 2008. DynABT:Dynamic Asynchronous Backtracking for Dynamic DisC-SPs. In Proceedings of the 13th International Conference onArtificial Intelligence: Methodology, Systems, and Applica-tions, Lecture Notes in Computer Science 5253, 285–296.Berlin: Springer.

Petcu, A., and Faltings, B. 2005a. Approximations in Dis-tributed Optimization. In Proceedings of the 11th Interna-tional Conference on Principles and Practice of Constraint Pro-gramming, Lecture Notes in Computer Science 3709,802–806. Berlin: Springer.

Petcu, A., and Faltings, B. 2005b. A Scalable Method forMultiagent Constraint Optimization. In Proceedings of the19th International Joint Conference on Artificial Intelligence,1413–1420. San Francisco: Morgan Kaufmann Publishers.

Petcu, A., and Faltings, B. 2005c. Superstabilizing, Fault-

Containing Multiagent Combinatorial Opti-mization. In Proceedings of the 20th NationalConference on Artificial Intelligence, 449–454.Menlo Park, CA: AAAI Press.

Petcu, A., and Faltings, B. 2007a. MB-DPOP:A New Memory-Bounded Algorithm for Dis-tributed Optimization. In Proceedings of the20th International Joint Conference on Artifi-cial Intelligence, 1452–1457. Menlo Park, CA:AAAI Press.

Petcu, A., and Faltings, B. 2007b. OptimalSolution Stability in Dynamic, DistributedConstraint Optimization. In Proceedings ofthe 2007 IEEE/WIC/ACM International Con-ference on Intelligent Agent Technology, 321–327. Los Alamitos, CA: IEEE Computer Soci-ety.

Petcu, A.; Faltings, B.; and Mailler, R. 2007.PC-DPOP: A New Partial CentralizationAlgorithm for Distributed Optimization. InProceedings of the 20th International Joint Con-ference on Artificial Intelligence, 167–172.Menlo Park, CA: AAAI Press.

Schiex, T.; Fargier, H.; and Verfaillie, G.1995. Valued Constraint Satisfaction Prob-lems: Hard and Easy Problems. In Proceed-ings of the 14th International Joint Conferenceon Artificial Intelligence, 631–637. San Fran-cisco: Morgan Kaufmann Publishers.

Silaghi, M., and Yokoo, M. 2009. ADOPT-ing: Unifying Asynchronous DistributedOptimization with Asynchronous Back-tracking. Autonomous Agents and MultiAgentSystems 19(2): 89–123.

Smallwood, R., and Sondik, E. 1973. TheOptimal Control of Partially ObservableMarkov Processes over a Finite Horizon.Operations Research 21(5): 1071–1088.

Sultanik, E.; Lass, R.; and Regli, W. 2007.DCOPolis: A Framework for Simulating andDeploying Distributed Constraint Reason-ing Algorithms. Paper presented at theEighth International Workshop on Distrib-uted Constraint Reasoning. Hyderabad,India, 8 January.

Sultanik, E.; Lass, R.; and Regli, W. 2009.Dynamic Configuration of Agent Organiza-tions. In Proceedings of the 21st InternationalJoint Conference on Artificial Intelligence, 305–311. Menlo Park, CA: AAAI Press.

Sycara, K.; Roth, S.; Sadeh, N.; and Fox, M.1991. Distributed Constrained HeuristicSearch. IEEE Transactions on Systems, Manand Cybernetics 6(21): 1446–1461.

Ueda, S.; Iwasaki, A.; and Yokoo, M. 2010.Coalition Structure Generation Based onDistributed Constraint Optimization. InProceedings of the 24th AAAI Conference onArtificial Intelligence, 197–203. Menlo Park,CA: AAAI Press.

Velagapudi, P.; Varakantham, P.; Scerri, P.;and Sycara, K. 2011. Distributed ModelShaping for Scaling to Decentralized

POMDPs with Hundreds of Agents. In Pro-ceedings of the Tenth International Joint Con-ference on Autonomous Agents and MultiagentSystems, 955–962. Richland, SC: Interna-tional Foundation for Autonomous Agentsand Multiagent Systems.

Yeoh, W.; Felner, A.; and Koenig, S. 2010.BnB-ADOPT: An Asynchronous Branch-and-Bound DCOP Algorithm. Journal of ArtificialIntelligence Research 38: 85–133.

Yeoh, W.; Sun, X.; and Koenig, S. 2009. Trad-ing Off Solution Quality for Faster Compu-tation in DCOP Search Algorithms. In Pro-ceedings of the 21st International JointConference on Artificial Intelligence, 354–360.Menlo Park, CA: AAAI Press.

Yeoh, W.; Varakantham, P.; and Koenig, S.2009. Caching Schemes for DCOP SearchAlgorithms. In Proceedings of the Eighth Inter-national Joint Conference on AutonomousAgents and Multiagent Systems, 609–616.Richland, SC: International Foundation forAutonomous Agents and Multiagent Sys-tems.

Yeoh, W.; Varakantham, P.; Sun, X.; andKoenig, S. 2011. Incremental DCOP SearchAlgorithms for Solving Dynamic DCOPs(Extended Abstract). In Proceedings of theTenth International Joint Conference onAutonomous Agents and Multiagent Systems,1069–1070. Richland, SC: InternationalFoundation for Autonomous Agents andMultiagent Systems.

Yokoo, M. 1995. Asynchronous Weak-Com-mitment Search for Solving DistributedConstraint Satisfaction Problems. In Pro-ceedings of the First International Conferenceon Principles and Practice of Constraint Pro-gramming, Lecture Notes in Computer Sci-ence 976, 88–102. Berlin: Springer.

Yokoo, M.; Durfee, E.; Ishida, T.; andKuwabara, K. 1992. Distributed ConstraintSatisfaction for Formalizing DistributedProblem Solving. In Proceedings of the 12thInternational Conference on Distributed Com-puting Systems, 614–621. Los Alamitos, CA:IEEE Computer Society.

Yokoo, M.; Durfee, E.; Ishida, T.; andKuwabara, K. 1998. The Distributed Con-straint Satisfaction Problem: Formalizationand Algorithms. IEEE Transactions on Knowl-edge and Data Engineering 10(5): 673–685.

Yokoo, M.; Suzuki, K.; and Hirayama, K.2005. Secure Distributed Constraint Satis-faction: Reaching Agreement WithoutRevealing Private Information. ArtificialIntelligence 161(1–2): 229–245.

Zhang, W., and Wittenburg, L. 2002. Dis-tributed Breakout Revisited. In Proceedings ofthe 18th National Conference on ArtificialIntelligence, 352–357. Menlo Park, CA: AAAIPress.

Zhang, W.; Xing, Z.; Wang, G.; and Witten-

Articles

FALL 2012 65

burg, L. 2003. An Analysis and Applicationof Distributed Constraint Satisfaction andOptimization Algorithms in Sensor Net-works. In Proceedings of the Second Interna-tional Joint Conference on Autonomous Agentsand Multiagent Systems, 185–192. New York:Association for Computing Machinery.

Zivan, R. 2008. Anytime Local Search forDistributed Constraint Optimization. InProceedings of the 23rd AAAI Conference onArtificial Intelligence, 393–398. Menlo Park,CA: AAAI Press.

Zivan, R., and Meisels, A. 2005. Dynamicordering for asynchronous backtracking ondisCSPs. In Proceedings of the 11th Interna-tional Conference on Principles and Practice ofConstraint Programming, Lecture Notes inComputer Science 3709, 32–46. Berlin:Springer.

Zivan, R.; Glinton, R.; and Sycara, K. 2009.Distributed Constraint Optimization forLarge Teams of Mobile Sensing Agents. InProceedings of the 2009 IEEE/WIC/ACM Inter-national Conference on Intelligent Agent Tech-nology, 347–354. Los Alamitos, CA: IEEEComputer Society.

William Yeoh is an assistant professor ofcomputer science at New Mexico State Uni-versity. He received his Ph.D. in computerscience at the University of Southern Cali-fornia. His research interests include multi-agent systems, distributed constraint rea-soning, heuristic search, and planning withuncertainty. He was a coorganizer of theInternational Workshop on DistributedConstraint Reasoning in 2008 and the AAAISymposium on Multiagent Coordinationunder Uncertainty in 2011.

Makoto Yokoo is a professor of informationscience and electrical engineering at KyushuUniversity, Japan. He received a Ph.D, ininformation and communication from theUniversity of Tokyo in 1995. His researchinterests include multiagent systems, con-straint reasoning, and game theory. He iscurrently the president of InternationalFoundation for Autonomous Agents andMultiagent Systems (IFAAMAS) and an AAAIfellow.