distributed constraint optimization for the...
TRANSCRIPT
Distributed Constraint Optimization for theInternet-of-Things
Pierre Rust Gauthier Picard
Orange LabsMINES Saint-Étienne
LaHC UMR CNRS 5516
UMR • CNRS • 5516 • SAINT-ETIENNE
Internet-of-Things (IoT) and its Control
Huge (marketing ?) trend today
25 billion of connected objects in 2020 ? (Gartner)
Hardware and communication is cheaper and cheaperContrained devices
I limited cpu and memory resourcesI limited communication capabilities
Connected things’ actions should becoordinatedCurrent approach: centralizingdecisions
I CommunicationsI ResilienceI ScalabilityI Cost
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 2
Distributed Coordination and Decision MakingAutonomous and spontaneous
Coordinating objects to achieveobjectives
CoordinationI DecentralizedI SpontaneousI Autonomous
No central point
Self-adaption to environmentalchanges
Self-repair in case of one componentfailure
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 3
About decisions
xi ?
s.t. “I’m happy with xi”
xj ?s.t. “agent i is fine with xj”
How can agents autonomously make their decisionsin a coordinated way, without external control ?
⇒ Decentralized decision making
Agents have to coordinate to perform best actions
Agents form a team→ best actions for the team
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 4
About decisions
xi ?s.t. “I’m happy with xi”
xj ?s.t. “agent i is fine with xj”
How can agents autonomously make their decisionsin a coordinated way, without external control ?
⇒ Decentralized decision making
Agents have to coordinate to perform best actions
Agents form a team→ best actions for the team
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 4
About decisions
xi ?s.t. “I’m happy with xi”
xj ?s.t. “agent i is fine with xj”
How can agents autonomously make their decisionsin a coordinated way, without external control ?
⇒ Decentralized decision making
Agents have to coordinate to perform best actions
Agents form a team→ best actions for the team
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 4
About decisions
xi ?s.t. “I’m happy with xi”
xj ?s.t. “agent i is fine with xj”
How can agents autonomously make their decisionsin a coordinated way, without external control ?
⇒ Decentralized decision making
Agents have to coordinate to perform best actions
Agents form a team→ best actions for the team
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 4
About decisions
xi ?s.t. “I’m happy with xi”
xj ?s.t. “agent i is fine with xj”
How can agents autonomously make their decisionsin a coordinated way, without external control ?
⇒ Decentralized decision making
Agents have to coordinate to perform best actions
Agents form a team→ best actions for the team
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 4
About decisions
xi ?s.t. “I’m happy with xi”
xj ?s.t. “agent i is fine with xj”
How can agents autonomously make their decisionsin a coordinated way, without external control ?
⇒ Decentralized decision making
Agents have to coordinate to perform best actions
Agents form a team→ best actions for the team
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 4
Application Domains
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 5
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 6
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 7
DCOP1
Distributed Constraints Optimization Problem
Definition (DCOP)A DCOP is a tuple 〈A,X ,D, C, µ〉, where:
A = {a1, . . . , a|A|} is a set of agents
X = {x1, . . . , xn} are variables
D = {Dx1 , . . . ,Dxn} is a set of finite domains, for the xi variables
C = {f1, . . . , fm} is a set of soft constraints, where each ci defines a cost∈ R ∪ {∞} for each combination of assignments to a subset of variables
µ is a function mapping variables to their associated agent
Definition (Solution)A solution to the DCOP is an assignment A to all variables that minimizes∑i fi
1Some contents taken from OPTMAS 2011 and OPTMAS-DCR 2014Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 8
DCOPExample and Graphical Representation
a1
a2 a3
a4
x1
x2 x3
x4
(a)
a1
a2
a3a4
x4
x2
x3
x1
(b)
a1
a2a3
a4
x1
f123x2 x3
x4f24
(c)
Objective Function
F (A) =∑
xi,xj∈Xfij where fij = (xi + xj + 1)mod3
In figure (a):F ({(x1, 0), (x2, 0), (x3, 0), (x4, 0)}) = 4
F ({(x1, 1), (x2, 1), (x3, 1), (x4, 1)}) = 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 9
DCOP Algorithms
Complete
Fully Decentralized
Partially Decentralized
Search Inference
Synchronous Asynchronous
Search Inference Search
Incomplete
Fully Decentralized
Synchronous Asynchronous
Search
OPTApo PC-DPOP SyncBB DPOP andvariants
AFB; ADOPT and variants
Region OptimalDSA; MGM
D-Gibbs
Sampling Inference
Max-Sum andvariants
Synchronous
Figure 5: Classical DCOP Algorithm Taxonomy.
3.3. Algorithms
The field of classical DCOPs is mature and a number of different algorithms have been proposed. DCOP algo-rithms can be classified as being either complete or incomplete, based on whether they can guarantee the optimalsolution or they trade optimality for smaller execution times, producing approximated solutions. In addition, each ofthese classes can be categorized into several groups, such as: (1) partially or fully decentralized, depending on the de-gree of locality exploited by the algorithms; and (2) synchronous or asynchronous, based on the way local informationis updated. Finally, the resolution process adopted by each algorithm can be classified in three categories [16]:• Search-based methods, which are based on the use of search techniques to explore the space of possible solutions.
These techniques are often derived from corresponding search techniques developed for centralized AI searchproblems, such as best-first search and depth-first search.• Inference-based methods, which are inspired from dynamic programming and belief propagation techniques.
These techniques allow agents to exploit the structure of the constraint graph to aggregate rewards from theirneighbors, effectively reducing the problem size at each step of the algorithm.• Sampling-based methods, which are incomplete approaches that sample the search space to approximate a func-
tion (usually a probability distribution) as a product of statistical inference.Figure 5 illustrates a taxonomy of classical DCOP algorithms. In the following subsections, we briefly describe
some representative complete and incomplete algorithms of each of the classes introduced above. A detailed descrip-tion of the DCOP algorithms is out of the scope of this document. We refer the interested readers to the originalarticles that introduce each algorithm.
Throughout this document, we will often refer to the following notation when discussing the complexity of thealgorithms: the size of the largest domain is denoted by d = maxDi2D |Di|, and w⇤ refers to the induced width of thepseudo-tree.
COMPLETE ALGORITHMS
SynchBB [17]. Synchronous Branch-and-Bound (SynchBB) is a complete, synchronous, search-based algorithmthat can be considered as a distributed version of a branch-and-bound algorithm. It uses a complete ordering ofthe agents in order to extend a Current Partial Assignment (CPA) via a synchronous communication process. TheCPA holds the assignments of all the variables controlled by all the visited agents, and, in addition, functions as amechanism to propagate bound information. The algorithm prunes those parts of the search space whose solutionquality is sub-optimal, by exploiting the bounds that are updated at each step of the algorithm. SynchBB agentsspace requirement and maximum size of message are in O(n), while they require, in the worst case, to perform O(dm)number of operations. The network load is also in O(dm).
AFB [18]. Asynchronous Forward Bounding (AFB) is a complete, asynchronous, search-based algorithm that can beconsidered as the asynchronous version of SynchBB. In this algorithm, agents communicate their reward estimates,
7
[FIORETTO et al., 2018]
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 10
Menu
DCOP Framework
Focus on Some Solution MethodsDPOPMax-SumDSAMGM
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 11
Distributed Pseudotree Optimization Procedure (DPOP)[PETCU and FALTINGS, 2005]
3-phase distributed algorithm
PHASES MESSAGES
1. DFS Tree construction token passing
2. Utility phase: from leaves toroot
util (child→ parent, constrainttable [-child])
3. Value phase: from root toleaves
value (parent → children∪ pseudochildren, parentvalue)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 12
DFS Tree Phase
Distributed DFS graph traversal: token, ID, neighbors(X)1. X owns the token: adds its own ID and sends it in turn to each of its
neighbors, which become children2. Y receives the token from X: it marks X as visited. First time Y receives the
token then parent(Y ) = X. Other IDs in token which are also neighbors(Y )are pseudoparent. If Y receives token from neighbor W to which it wasnever sent, W is pseudochild.
3. When all neighbors(X) visited, X removes its ID from token and sends it toparent(X).
A node is selected as root, which starts
When all neighbors of root are visited, the DFS traversal ends
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 13
DFS Tree Phase: Example
x1
x2 x3
x4
root
[x1] x1 parent of x2
x1
x2 x3
x4
[x1, x2]
x2 parent of x3x1 pseudoparent of x3
x1
x2 x3
x4
[x1, x2, x3]x3 parent of x4x3 pseudoparent of x1
x1
x2
x3 x4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 14
DFS Tree Phase: Example
x1
x2 x3
x4
root
[x1] x1 parent of x2
x1
x2 x3
x4
[x1, x2]
x2 parent of x3x1 pseudoparent of x3
x1
x2 x3
x4
[x1, x2, x3]x3 parent of x4x3 pseudoparent of x1
x1
x2
x3 x4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 14
DFS Tree Phase: Example
x1
x2 x3
x4
root
[x1] x1 parent of x2
x1
x2 x3
x4
[x1, x2]
x2 parent of x3x1 pseudoparent of x3
x1
x2 x3
x4
[x1, x2, x3]x3 parent of x4x3 pseudoparent of x1
x1
x2
x3 x4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 14
DFS Tree Phase: Example
x1
x2 x3
x4
root
[x1] x1 parent of x2
x1
x2 x3
x4
[x1, x2]
x2 parent of x3x1 pseudoparent of x3
x1
x2 x3
x4
[x1, x2, x3]x3 parent of x4x3 pseudoparent of x1
x1
x2
x3 x4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 14
Util Phase
Agent X:
receives from each child Yi a cost function: C(Yi)
combines (adds, joins) all these cost functions with the cost functionswith parent(X) and pseudoparents(X)
projects X out of the resulting cost function, and sends it to parent(X)
From the leaves to the root
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 15
Util Phase: Example
X
X T
a a 1a b 2b a 2b b 0
X Y
a a 1a b 2b a 2b b 0
X Z
a a 1a b 2b a 2b b 0
parent
children
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
add
All value combinationsCosts are the sum of ap-plicable costs
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
Remove XRemove duplicatesKeep the min cost
Project out X
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 16
Util Phase: Example
X
X T
a a 1a b 2b a 2b b 0
X Y
a a 1a b 2b a 2b b 0
X Z
a a 1a b 2b a 2b b 0
parent
children
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
add
All value combinationsCosts are the sum of ap-plicable costs
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
Remove XRemove duplicatesKeep the min cost
Project out X
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 16
Util Phase: Example
X
X T
a a 1a b 2b a 2b b 0
X Y
a a 1a b 2b a 2b b 0
X Z
a a 1a b 2b a 2b b 0
parent
children
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
add
All value combinationsCosts are the sum of ap-plicable costs
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
Remove XRemove duplicatesKeep the min cost
Project out X
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 16
Util Phase: Example
X
X T
a a 1a b 2b a 2b b 0
X Y
a a 1a b 2b a 2b b 0
X Z
a a 1a b 2b a 2b b 0
parent
children
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
add
All value combinationsCosts are the sum of ap-plicable costs
X Y Z T
a a a a 3a a a b 4a a b a 4a a b b 5a b a a 4a b a b 5a b b a 5a b b b 6b a a a 6b a a b 4b a b a 4b a b b 2b b a a 4b b a b 2b b b a 2b b b b 0
Remove XRemove duplicatesKeep the min cost
Project out X
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 16
Value Phase
1. The root finds the value that minimizes the received cost function in theutil phase, and informs its descendants (children ∪ pseudochildren)
2. Each agent waits to receive the value of its parent / pseudoparents
3. Keeping fixed the value of parent/pseudoparents, finds the value thatminimizes the received cost function in the Util phase
4. Informs of this value to its children/pseudochildren
This process starts at the root and ends at the leaves
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 17
DTREE : DPOP for DCOPs without backedges
X
Y
Z W
Y Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DTREE : DPOP for DCOPs without backedges
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
Ya b1 0
Ya b1 0
Xa b2 0
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size: linear
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 18
DPOP for any DCOP
X
Y
Z W
Y Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
DPOP for any DCOP
X
Y
Z WY Za a 1a b 2b a 2b b 0
Y Wa a 1a b 2b a 2b b 0
X Ya a 1a b 2b a 2b b 0
X Za a 1a b 2b a 2b b 0
Ya b
Xa 2 2b 2 0
Ya b1 0
Xa b2 0
X ← b
X ← b
Y ← b
Z ← b W ← b
Optimal solution:
linear number ofmessages
message size:exponential
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 19
GDL-based approaches
Generalized Distributive Law [AJI and MCELIECE, 2000]I Unifying framework for inference in Graphical modelsI Builds on basic mathematical properties of semi-ringsI Widely used in Info theory, Statistical physics, Probabilistic models
Max-sumI DCOP settings: maximise social welfare
AJI AND MCELIECE: THE GENERALIZED DISTRIBUTIVE LAW 327
TABLE ISOME COMMUTATIVE SEMIRINGS. HERE
DENOTES AN ARBITRARY COMMUTATIVE RING, IS AN ARBITRARY FINITESET, AND DENOTES AN ARBITRARY DISTRIBUTIVE LATTICE
For example, consider the min-sum semiring in Table I(entry 7). Here is the set of real numbers, plus the specialsymbol “ .” The operation “ ” is defined as the operation oftaking the minimum, with the symbol playing the role of thecorresponding identity element, i.e., we definefor all . The operation “ ” is defined to be ordinaryaddition [sic], with the real number playing the role ofthe identity, and for all . Oddly enough, thiscombination forms a semiring, because the distributive law isequivalent to
which is easily seen to be true. We shall get a glimpse of theimportance of this semiring in Examples 2.3 and 4.3, below.(In fact, semirings 5–8 are all isomorphic to each other; for ex-ample, 5 becomes 6 via the mapping , and 6 becomes7 under the mapping .)Having briefly discussed commutative semirings, we now de-
scribe the “marginalize a product function” problem, which isa general computational problem solved by the GDL. At theend of the section we will give several examples of the MPFproblem, which demonstrate how it can occur in a surprisinglywide variety of settings.Let be variables taking values in the finite
sets , with for . Ifis a subset of , we denote the
product by , the variable listby , and the cardinality of , i.e., , by . We denotethe product simply by , and the variable list
simply by .Now let be subsets of .
Suppose that for each , there is a function, where is a commutative semiring. The
variable lists are called the local domains and the functionsare called the local kernels. We define the global kernel
as follows:
(2.1)
With this setup, the MPF problem is this: For one or more ofthe indices , compute a table of the values of the
-marginalization of the global kernel , which is the function, defined by
(2.2)
In (2.2), denotes the complement of the set relative to the“universe” . For example, if , and if
, then
We will call the function defined in (2.2) the th objec-tive function, or the objective function at . We note that thecomputation of the th objective function in the obvious wayrequires additions and multipli-cations, for a total of arithmetic operations, wheredenotes the size of the set . We shall see below (Section V)
that the algorithm we call the “generalized distributive law” canoften reduce this figure dramatically.We conclude this section with some illustrative examples of
the MPF problem.
Example 2.1: Let , , , and be variables takingvalues in the finite sets , , , and . Suppose
and are given functions of these vari-ables, and that it is desired to compute tables of the functions
and defined by
ptThis is an instance of the MPF problem, if we define localdomains and kernels as follows:
local domain local kernel
The desired function is the objective function at localdomain , and is the objective function at local domain. This is just a slightly altered version of Example 1.1, andwe shall see in Section IV that when the GDL is applied, the“algorithm” of Example 1.1 results.
Example 2.2: Let , , , , , and be six vari-ables, each assuming values in the binary set , and let
be a real-valued function of the variables ,, and . Now consider the MPF problem (the commutative
semiring being the set of real numbers with ordinary additionand multiplication) with the following local domains andkernels:
local domain local kernel
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 20
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2
m2→1
m2→3m3→2
m3→4
m4→3
m4→1 m1→4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2
m4→1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2
m4→1
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2
m4→1
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2
m4→1
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
All incoming messages except x2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
All incoming messages except x2
z1(x1)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
All incoming messages except x2
z1(x1)
z1(x1) = m4→1(x1) +m2→1(x1)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
All incoming messages except x2
z1(x1)
z1(x1) = m4→1(x1) +m2→1(x1)
m4→1
m2→1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
All incoming messages except x2
z1(x1)
z1(x1) = m4→1(x1) +m2→1(x1)
m4→1
m2→1
All incoming messages
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum[FARINELLI et al., 2008]
Agents iteratively computes local functions that depend only on thevariable they control
x1 x2
x3x4
m1→2(x2) = maxx1 (F12(x1, x2) +m4→1(x1))
Shared constraint
All incoming messages except x2
z1(x1)
z1(x1) = m4→1(x1) +m2→1(x1)
m4→1
m2→1
All incoming messages
Chooseargmax
x1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 21
Max-Sum on acyclic graphs
Max-sum Optimal onacyclic graphs
I Different branches areindependent
I Each agent can build acorrect estimation of itscontribution to theglobal problem (zfunctions)
Message equations verysimilar to Util messages inDPOP
I Sum messages fromchildren and sharedconstraint
I Maximize out agentvariable
I GDL generalizes DPOP[VINYALS et al., 2011]
x1
x2
x3
x4
m1→2(x2) = maxx1(F12(x1, x2) +m4→1(x1))
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 22
Max-Sum Performance
Good performance on loopy networks [FARINELLI et al., 2008]I When it converges very good results
I Interesting results when only one cycle [WEISS, 2000]I We could remove cycle but pay an exponential price (see DPOP)
Max-sumPerformance• Goodperformanceonloopynetworks[Farinellietal.08]
– Whenitconvergesverygoodresults• Interes1ngresultswhenonlyonecycle[Weiss00]
– Wecouldremovecyclebutpayanexponen1alprice(seeDPOP)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 23
Max-Sum for low power devices
Low overheadI Msgs number/size
Asynchronous computationI Agents take decisions whenever new messages arrive
Robust to message loss
Max-Sumforlowpowerdevices• Lowoverhead
– Msgsnumber/size• Asynchronouscomputa1on
– Agentstakedecisionswhenevernewmessagesarrive• Robusttomessageloss
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 24
Local Greedy Approaches
Greedy local searchI Start from random solutionI Do local changes if global solution improvesI Local: change the value of a subset of variables, usually one
−4−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 25
Local Greedy Approaches
Greedy local searchI Start from random solutionI Do local changes if global solution improvesI Local: change the value of a subset of variables, usually one
−4−1 −1 −1
−1
−2 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 25
Local Greedy Approaches
Greedy local searchI Start from random solutionI Do local changes if global solution improvesI Local: change the value of a subset of variables, usually one
−1 −1 −1
−1
−2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 25
Local Greedy Approaches
Greedy local searchI Start from random solutionI Do local changes if global solution improvesI Local: change the value of a subset of variables, usually one
−1 −1 −1
−1
−2
−2 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 25
Local Greedy Approaches
Greedy local searchI Start from random solutionI Do local changes if global solution improvesI Local: change the value of a subset of variables, usually one
−1 −1 −1
−1
−0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 25
Local Greedy Approaches
ProblemsI Local minimaI Standard solutions: Random Walk, Simulated Annealing
−2−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 26
Local Greedy Approaches
ProblemsI Local minimaI Standard solutions: Random Walk, Simulated Annealing
−2−1 −1 −1
−1
−1 −1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 26
Local Greedy Approaches
ProblemsI Local minimaI Standard solutions: Random Walk, Simulated Annealing
−2−1 −1 −1
−1
−1 −1
−1 −1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 26
Local Greedy Approaches
ProblemsI Local minimaI Standard solutions: Random Walk, Simulated Annealing
−2−1 −1 −1
−1
−1 −1
−1 −1
−1 −1
−1 −1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 26
Distributed Local Greedy approaches
Local knowledgeParallel execution
I A greedy local move might be harmful/uselessI Need coordination
−4−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 27
Distributed Local Greedy approaches
Local knowledgeParallel execution
I A greedy local move might be harmful/uselessI Need coordination
−4−1 −1 −1
−1
0 −2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 27
Distributed Local Greedy approaches
Local knowledgeParallel execution
I A greedy local move might be harmful/uselessI Need coordination
−4−1 −1 −1
−1
0 −2
0 −2
0 −2
0 −2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 27
Distributed Local Greedy approaches
Local knowledgeParallel execution
I A greedy local move might be harmful/uselessI Need coordination
−4−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 27
Distributed Stochastic Search Algorithm (DSA)[ZHANG et al., 2005]
Greedy local search with activation probability to mitigate issues withparallel executions
DSA-1: change value of one variable at time
Initialize agents with a random assignment and communicate valuesto neighborsEach agent:
I Generates a random number and execute only if rnd less than activationprobability
I When executing changes value maximizing local gainI Communicate possible variable change to neighbors
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 28
DSA-1: Execution Example
−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 29
DSA-1: Execution Example
−1 −1 −1
−1
rnd?> 1
4rnd
?> 1
4rnd
?> 1
4rnd
?> 1
4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 29
DSA-1: Execution Example
−1 −1 −1
−1
−2 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 29
DSA-1: Execution Example
−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 29
DSA-1: Execution Example
−1 −1 −1
−1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 29
DSA-1: Discussion
Extremely “cheap” (computation/communication)Good performance in various domains
I e.g. target tracking [FITZPATRICK and MEERTENS, 2003; ZHANG et al., 2003]I Shows an anytime property (not guaranteed)I Benchmarking technique for coordination
ProblemsI Activation probablity must be tuned [ZHANG et al., 2003]I No general rule, hard to characterise results across domains
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 30
Maximum Gain Message (MGM-1)[MAHESWARAN et al., 2004]
Coordinate to decide who is going to moveI Compute and exchange possible gainsI Agent with maximum (positive) gain executes
AnalysisI Empirically, similar to DSAI More communication (but still linear)I No Threshold to setI Guaranteed to be monotonic (Anytime behavior)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 31
MGM-1: Example
−1 −1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
−1 −1
G = 2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
−1 −1
G = 2
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
−1 −1
G = 2
−1 −1
G = 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
−1 −1
G = 2
−1 −1
G = 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
−2 0
G = −2
−1 −1
G = 0
−1 −1
G = 2
−1 −1
G = 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
MGM-1: Example
−1 −1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 32
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 33
Hands on PyDCOP I
Install VirtualBoxImport the pyDCOP Virtual Machine (https://bit.ly/2JUrrKP)
I It’s a Debian image with everything preinstalled:I python3, pyDCOP, matplotlib, glpk, etc.
Alternatively, followhttps://pydcop.readthedocs.io/en/latest/installation.html
1. https://pydcop.readthedocs.io/en/latest/tutorials/getting_started.html
2. https://pydcop.readthedocs.io/en/latest/tutorials/analysing_results.html
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 34
Hands on PyDCOP IDCOP - Graph Coloring
v2
v1 v3
(a) constraints graph
p2 v2
d1,2
v1
d2,3
v3p1 p3
(b) factor graph
Objective: minimize
Domain: 2 colors R and B
Variables: V1, V2, V3
Constraints: neighbors must have different colors + preferences
Agents: 3 agents
Yaml representation
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 35
Hands on PyDCOP ISolving the Graph Coloring DCOP
Command:
$ pydcop solve --algo dpop graph_coloring.yaml
Output:
..."assignment": {"v1": "R","v2": "G","v3": "R"},"cost": -0.1,...
With other algorithms:
$ pydcop --timeout 2 solve --algo dsa graph_coloring.yaml$ pydcop solve --algo mgm --algo_params stop_cycle:20 \
graph_coloring.yaml
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 36
Hands on PyDCOP IResults
Full results :
{"agt_metrics": {...},"assignment": {"v1": "R","v2": "G","v3": "R"},"cost": -0.1,"cycle": 20,"msg_count": 158,"msg_size": 158,"status": "FINISHED","time": 0.03201029699994251,"violation": 0}
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 37
Hands on PyDCOP ILogs
Simple:use -v 0..3
$ pydcop -v 2 solve --algo dpop graph_coloring.yaml
Precise :use -log <log.conf>
$ pydcop --log log.conf -t 1 solve --algo dsa graph_coloring.yaml
Now, look at algo.log
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 38
Hands on PyDCOP IRun-time metrics
periodic: "--collect_on period --period <p>"
$ pydcop --log log.conf -t 10 solve \--collect_on period --period <p>--algo dsa graph_coloring.yaml
cycle: "--collect_on cycle_change"Only supported with synchronous algorithms !
$ pydcop solve --algo mgm --algo_params stop_cycle:20 \--collect_on cycle_change --run_metric ./metrics.csv \graph_coloring_50.yaml
value: "--collect_on value_change"
$ pydcop -t 5 solve --algo mgm --collect_on value_change \--run_metric ./metrics_on_value.csv \graph_coloring_50.yaml
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 39
Hands on PyDCOP IRun-time metrics
With a bigger graph coloring problem
$ pydcop solve --algo mgm --algo_params stop_cycle:20 \--collect_on cycle_change \--run_metric ./metrics.csv \graph_coloring_50.yaml
Plotting with matplotlib
$ python3 plot_cost.py ./metrics.csv
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 40
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 41
SECP modelSmart Environment Configuration Problem [RUST et al., 2016]
Example of applying DCOPs to a"real" problem
Coordinate objects in thebuildingModel
I objectsI relations between objects and
environmentI user objectives and
requirements
Formulate the problem as anoptimization problem
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 42
SECP modelSmart Environment Configuration Problem [RUST et al., 2016]
Focus on smart lighting use cases
Objects: anything that can produce light: light bulbs, windows withrolling shutter, etc.
User preferences: having a predefined luminosity level in a room,under some conditions
Energy efficiencyLinking objects and user preferences:
How to model the luminosity in a room ? variableHow to model the dependency between the light sources and theluminosity ? function / constraint
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 43
SECP modelExample application to ambient intelligence scenario
ActuatorsI Connected light bulbs, TV, Rolling shutters, ...
SensorsI Presence detector, Luminosity Sensor, etc.
Physical Dependency ModelsI E.g. Living-room light model
User PreferencesI Expressed as rules :
IF presence_living_room = 1
AND light_sensor_living_room < 60
THEN light_level_living_room ← 60
AND shutter_living_room ← 0
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 44
SECP modelExample application to ambient intelligence scenario
ActuatorsI Decision variable xi, domain DxiI Cost function ci : Dxi → R
SensorsI Read-only variable sl, domain Dsl
Physical Dependency Models 〈yj , φj〉I Give the expected state of the environment from a
set of actuator-variables influencing this modelI Variable yj representing the expected state of the
environmentI Function φj :
∏ς∈σ(φj)Dς → Dyj
User PreferencesI Utility function ukI Distance from the current expected state to the
target state of the environment
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 44
Formulating SECP as a DCOP
Multi-objective optimization problem
minxi∈ν(A)
∑i∈A
ci and maxxi∈ν(A)
yj∈ν(Φ)
∑k∈R
uk
s.t. φj(x1j , . . . , x
φj
j ) = yj ∀yj ∈ ν(Φ)
Then mono-objective DCOP formulation
maxxi∈ν(A)
yj∈ν(Φ)
ωu∑k∈R
uk − ωc∑i∈A
ci +∑ϕj∈Φ
ϕj
with reformulation of hard constraints φj into soft ones:
ϕj(x1j , . . . , x
|σ(φj)|j , yj) =
{0 if φj(x
1j , . . . , x
|σ(φj)|j ) = yj
−∞ otherwise
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 45
Formulating SECP as a DCOPRepresenting a DCOP as a factor graph
x1
x2
x3
c1
c2
c3
ϕ1 y1 u1
s1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 46
SECP Factor Graphin a house (without rules)
Desk
LivingRoom
TV
Kitchen
Entrance
Stairs
ld1
ld2llv3
llv2
llv1ltv1
ltv2 ltv3
lk1 lk2
lk3
le1
ls1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 47
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 48
Distribution of computationsAllocating computations to agents
DCOP: 〈A,X ,D, C, µ〉µ : function mapping variables to their associated agent
Why is distribution needed ?
Common assumptions:
computation ≡ variable
each agent controls exactlyone variable (bijection)
binary constraints
Real distributed problems:
agents must be hosted on realdevices
the set of devices might begiven by the problem
for some variables the relationwith an agent is obvious, butnot always
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 49
Distribution of computationsAllocating computations to agents
Distributing computationsI computations depends on the graph modelI variables and / or factors
Distribution impacts the system characteristics :I speed,I communication load,I hosting costs, etc.
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 50
Distribution of computationsAllocating computations to agents
Simple heuristicI No computation on sleepy devices (sensors)I Computation should be close the the impacted variablesI Spread the computation load amongst agents
How good is it ?
c1
x1
N1
x2 c2
ϕ1 y1
N2
u1 x3
c3
N3
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 51
Distribution of computationsOptimal definition
Optimal distribution ?Problem dependentOptimization problem :find the best distribution, for your problem’s criteriaOptimal distribution ≡ graph partitioning,NP-hard in general [BOULLE, 2004]
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 52
Distribution of computationsBetter definition
SECP distribution problem
Devices have limited memory
Communication is expensive and has limited bandwidth
Variable related to an actuator are hosted by it
Objective : minimize overall communication between agentsOptimization problem : define an ILP for it !
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 53
Binary ILP for computation distribution
xki , binary variables that map computations to agents andαmnij = xmi · fnj
∀xi ∈ X,∑am∈A
xmi = 1 (1)
Message’s size between variable xi and factor fj : msg(i, j)
minimizexmi
∑(i,j)∈D
∑(m,n)∈A2
msg(i, j) · αmnij (2)
Memory footprint of a computation: weight(e) , and memorycapacity for a device: cap(ak)
∀am ∈ A,∑xi∈D
weight(xi) · xmi ≤ cap(am) (3)
and a few linearization constraints
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 54
Binary ILP for computation distribution
More generic case:
Add route cost: com(i, j,m, n)
∀xi, xj ∈ X, ∀am, an ∈ A,
com(i, j,m, n) =
{msg(i, j) · route(m,n) if (i, j) ∈ D,m 6= n0 otherwise (4)
minimizexmi
∑(i,j)∈D
∑(m,n)∈A2
com(i, j,m, n) · αmnij (5)
Add hosting costs : host(am, xi)
minimizexmi
∑(xi,am)∈X×A
xmi · host(am, xi) (6)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 55
Binary ILP for computation distribution
minimizexmi
ωcom ·∑
(i,j)∈D
∑(m,n)∈A2
com(i, j,m, n) · αmnij
+ ωhost ·∑
(xi,am)∈X×A
xmi · host(am, xi) (7)
subject to
∀am ∈ A,∑xi∈D
weight(xi) · xmi ≤ cap(am) (8)
∀xi ∈ X,∑am∈A
xmi = 1 (9)
∀xi ∈ X, αmnij ≤ xmi (10)
∀xj ∈ X, αmnij ≤ xmj (11)
∀xi, xj ∈ X, am ∈ A, αmnij ≥ xmi + xnj − 1 (12)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 56
Solving the ILP for computation deployment
NP-hard, but can be solved with branch-and-cutLP solvers are very good at thisYet, only possible for small instancesGives us a reference for optimality: benchmarkingWhen not solvable, still gives us a metrics to compare heuristics
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 57
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 58
Hands on PyDCOP II
1. https://pydcop.readthedocs.io/en/latest/tutorials/deploying_on_machines.html
2. https://pydcop.readthedocs.io/en/latest/usage/cli/run.html
3. https://pydcop.readthedocs.io/en/latest/usage/cli/distribute.html
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 59
Hands on PyDCOP IISECP
A Very simple SECP: single room
3 light bulbs, 1 model and 3 rules
/tutorials/hands-on_2/single_room.yaml
Solve with
pydcop --log log.conf -t 10 solve--algo maxsum --algo_params damping:0.8--dist adhoc single_room.yaml
Result : "cost": 702.3000000000004, ...I not that good ...I Look at the yaml definitionI the rules contradict each other !
Change the yaml definitionI comment out rules to keep only one activeI could be done with ’read-only’ variablesI solve it again
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 60
Hands on PyDCOP IISECP - Running on several machines
We used solveI great for testingI everything run locally, in the same process
Launching several agents:I One agent for each light bulb a1, a2 and a3 (change port for each agent)
pydcop -v3 agent -n a1 -p 9001--orchestrator 127.0.0.1:9000
I an orchestrator
pydcop --log log.conf -t 10 orchestrator \--algo maxsum --algo_params damping:0.8 \--dist adhoc single_room.yaml
I run the agents on different Virtual machines, different computers
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 61
Hands on PyDCOP IIA bigger SECP
in /tutorials/hands-on_2/SimpleHouse.yml13 light bulbs, 6 models
Desk
LivingRoom
TV
Kitchen
Entrance
Stairs
ld1
ld2llv3
llv2
llv1ltv1
ltv2 ltv3
lk1 lk2
lk3
le1
ls1
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 62
Hands on PyDCOP IIDistributing a SECP
$ pydcop --output dist_house_fg_ilp.yaml distribute -d ilp_compref \-a maxsum SimpleHouse.yml
Need to specify the algorithm
computation graph
weight of computation’s
size of computations’ messages
On such a small system, we can compute the optimal distribution !
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 63
Hands on PyDCOP IIDistributing a SECP
cost: 8725.0distribution:a_d1: [mv_desk, mc_desk, l_d1, r_work, mc_livingroom, mv_livingroom]a_d2: [l_d2]a_e1: [mv_entry, r_entry, mv_stairs, l_e1, mc_entry, mc_stairs]a_e2: [l_e2]a_k1: [l_k1]a_k2: [l_k2]a_k3: [l_k3]a_lv1: [l_lv1]a_lv2: [mc_kitchen, l_lv2]a_lv3: [l_lv3]a_tv1: [l_tv1]a_tv2: [l_tv2]a_tv3: [r_lunch, l_tv3, mv_tv, r_cooking, r_homecinema, mc_tv, mv_kitchen]inputs:algo: maxsumdcop: [SimpleHouse.yml]dist_algo: ilp_comprefgraph: factor_graph
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 64
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 65
SECP and DCOP
So far we have:
Designed a model for SECP
Formulated this model as a DCOP
Distributed the computation of the DCOP on devices / agents(bootstrap)
Run our system to get an auto-configuration of the devices
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 66
But what happensin dynamic environments ?
if objects appear and disappear?
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 67
SECP is a dynamic problem
Dynamics in the infrastructure
Devices can disappear
New devices can be added tothe system
At runtime..
No powerful device available tosolve the ILP
The deployment must berepaired: self-adaptation
Only consider a portion of thefactor graph: the neighborhood
c1 x1 u1
a1
c2 x2 ϕ1 y1
a2
c3 x3 ϕ2
a3
c4 x4 y2 u2
a4
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 68
k-resilienceDynamics in the infrastruture
Definition (k-resiliency)k-resiliency is the capacity for a system to repair itself and operatecorrectly even in the case of the disappearance of up to k agents
Two parts:I Do not loose the definition of the computations: replicationI Migrate the orphaned computations to another agent: selection /
activation
Apply to any graph of computations, not only DCOP
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 69
Replication of computations
Replica distribution
For each computation, place k replica on k other agentsreplica equiv definition of the computation
Must be distributed !
Optimal replication ? impact the set of available agents whenrepairingwhich criteria ? too hard (quadratic multiple knapsack problem)...
Distributed Replica Placement Method (DRPM)
Heuristic : place replica on agents close (network) the activecomputation, while respecting capacity
Distributed version of iterative lengthening (aka uniform cost searchbased on path costs)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 70
Replication of computationsiterative lengthening on route and hosting costs
a1
a2 a3
a4
a2 a3
a4
route(a1, a2) = 1route(a2, a3) = 3
route(a2, a4) = 1
route(a1, a4) = 1
host(a2, xi) = 1 host(a3, xi) = 1
host(a4, xi) = 5
Figure: A sample route+host–graph with 4 agents (in gray), where a1 search forhosting computation xi. For k = 2, DRMP places a replica on a2 (cost of 1 + 1 = 2)and another on a3 (cost of 1 + 3 + 1 = 5) if enough capacity on these two agents,since the minimum cost path to host on a4 is higher (1 + 5 = 6).
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 71
Migrating computationsSelecting an agent
Migrating a set of xi computations Xc
set of candidate agents Acmigrating the computation must not exceed agent’s capacity
for each computation, select the agent that minimize hosting andcommunication cost
Same optimization problem than for initial distribution, but on a subset ofthe of the graph
Distributed process !
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 72
Migrating computationsSelecting an agent
Distributed optimization problem⇒ let’s use a DCOP!A is the set of candidate agents AcX are the binary decision variables xmiC are the constraints ensuring that all computations are hosted,agent’s capacities are respected and hosting and communicationcosts are minimized
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 73
Migrating computationsSelecting an agent
∑am∈Ai
c
xmi = 1 (13)
∑xi∈Xm
c
weight(xi) · xmi +∑
xj∈µ−1(am)\Xc
weight(xj) ≤ cap(am) (14)
∑xi∈Xm
c
host(am, xi) · xmi (15)
∑(xi,xj)∈Xm
c ×Ni\Xc
xmi · com(i, j,m, µ−1(xj))
+∑
(xi,xj)∈Xmc ×Ni∩Xc
xmi ·∑
an∈Ajc
xnj · com(i, j,m, n)) (16)
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 74
Decentralized reparation
When agents are removed:
computation to migrate = computation that where hosted on theseagents
candidate agents = remaining agents that posses a replica of theseorphaned computation
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 75
Solving the migration DCOPWhich algorithm should we use ?
Criteria:
lightweight
fast (even if not optimal !)
monotonic : mix of hard and soft constraints
MGM-2 : like MGM, with 2-coordination
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 76
Experimental results
0 30 60 90 120 150 180 210 240time (s)
5000
6000
7000
8000
9000
cost
average cost with perturbationaverage cost without perturbation
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 77
Menu
DCOP Framework
Focus on Some Solution Methods
Hands on PyDCOP I
Focus on Smart Environment Configuration Problems
Distributing Computations
Hands on PyDCOP II
Dynamic DCOPs
Conclusion
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 78
To sum up
What we’ve seen today:
Some generic conceptsI How to model coordination problems using DCOP formalismI Some solution methods (complete and incomplete) to solve DCOP
Some specificities of IoT-based appsI How to model a specific smart environment configuration problem as a
DCOPI How to use PyDCOP to model, run, solve, and distribute DCOPI How to equip a system with resilience using replication and DCOP-based
reparation
Want to go deeper into DCOPs→ OPTMAS-DCR workshop series(AAMAS/IJCAI), other tutorials at AAMAS/IJCAI
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 79
The End
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 80
References
AJI, S.M. and R.J. MCELIECE (2000). “The generalized distributive law”. In: Information Theory, IEEETransactions on 46.2, pp. 325–343. ISSN: 0018-9448. DOI: 10.1109/18.825794.
BOULLE, M. (2004). “Compact Mathematical Formulation for Graph Partitioning”. In: Optimization andEngineering 5.3, pp. 315–333. ISSN: 1573-2924. DOI: 10.1023/B:OPTE.0000038889.84284.c7. URL:http://dx.doi.org/10.1023/B:OPTE.0000038889.84284.c7.
BÜRGER, M., G. NOTARSTEFANO, F. BULLO, and F. ALLGÖWER (2012). “A distributed simplex algorithm fordegenerate linear programs and multi-agent assignments”. In: Automatica 48.9, pp. 2298 –2304. ISSN:0005-1098. DOI: http://dx.doi.org/10.1016/j.automatica.2012.06.040. URL://www.sciencedirect.com/science/article/pii/S0005109812002956.
FARINELLI, A., A. ROGERS, A. PETCU, and N. R. JENNINGS (2008). “Decentralised Coordination of Low-powerEmbedded Devices Using the Max-sum Algorithm”. In: Proceedings of the 7th International JointConference on Autonomous Agents and Multiagent Systems - Volume 2. AAMAS ’08. InternationalFoundation for Autonomous Agents and Multiagent Systems, pp. 639–646. ISBN: 978-0-9817381-1-6.
FIORETTO, F., E. PONTELLI, and W. YEOH (2018). “Distributed Constraint Optimization Problems andApplications: A Survey”. In: Journal of Artificial Intelligence Research 61, pp. 623–698.
FITZPATRICK, Stephen and Lambert MEERTENS (2003). “Distributed Coordination through AnarchicOptimization”. In: Distributed Sensor Networks: A Multiagent Perspective. Ed. by Victor LESSER,Charles L. ORTIZ, and Milind TAMBE. Boston, MA: Springer US, pp. 257–295. ISBN: 978-1-4615-0363-7.
MAHESWARAN, R.T., J.P. PEARCE, and M. TAMBE (2004). “Distributed Algorithms for DCOP: AGraphical-Game-Based Approach”. In: Proceedings of the 17th International Conference on Paralleland Distributed Computing Systems (PDCS), San Francisco, CA, pp. 432–439.
PETCU, Adrian and Boi FALTINGS (2005). “A scalable method for multiagent constraint optimization”. In:IJCAI International Joint Conference on Artificial Intelligence, pp. 266–271. ISBN: 1045-0823.
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 81
References (cont.)RUST, P., G. PICARD, and F. RAMPARANY (2016). “Using Message-passing DCOP Algorithms to SolveEnergy-efficient Smart Environment Configuration Problems”. In: International Joint Conference onArtificial Intelligence (IJCAI). AAAI Press.
— (2017). “On the Deployment of Factor Graph Elements to Operate Max-Sum in DynamicAmbient Environments”. In: Autonomous Agents and Multiagent Systems – AAMAS 2017 Workshops, BestPapers, Sao Paulo, Brazil, May 8-12, 2017, Revised Selected Papers. Ed. by G. SUKTHANKAR andJ.A. RODRIGUEZ-AGUILAR. Vol. 10642. Lecture Notes in Artificial Intelligence (LNAI). Extended Version.Springer, pp. 116–137. DOI: 10.1007/978-3-319-71682-4_8.
VINYALS, Meritxell, Juan A. RODRÍGUEZ-AGUILAR, and Jesus CERQUIDES (2011). “Constructing a unifyingtheory of dynamic programming DCOP algorithms via the generalized distributive law”. In: AutonomousAgents and Multi-Agent Systems 3.22, pp. 439–464. ISSN: 1387-2532. DOI: 10.1007/s10458-010-9132-7.
WEISS, Yair (2000). “Correctness of Local Probability Propagation in Graphical Models with Loops”. In:Neural Comput. 12.1, pp. 1–41. ISSN: 0899-7667. DOI: 10.1162/089976600300015880. URL:http://dx.doi.org/10.1162/089976600300015880.
ZHANG, W., G. WANG, Z. XING, and L. WITTENBURG (2005). “Distributed stochastic search and distributedbreakout: properties, comparison and applications to constraint optimization problems in sensornetworks.”. In: Journal of Artificial Intelligence Research (JAIR) 161.1-2, pp. 55–87.
ZHANG, Weixiong, Guandong WANG, Zhao XING, and Lars WITTENBURG (2003). “A Comparative Study ofDistributed Constraint Algorithms”. In: Distributed Sensor Networks: A Multiagent Perspective. Ed. byVictor LESSER, Charles L. ORTIZ, and Milind TAMBE. Boston, MA: Springer US, pp. 319–338.
Pierre Rust, Gauthier Picard Distributed Constraint Optimization for the Internet-of-Things 82