it is unlikely that there are efficient approximation algorithms with a very good worst case...
Post on 19-Dec-2015
222 views
TRANSCRIPT
![Page 1: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/1.jpg)
1
It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for
MAXSAT, MIN NODE COVER, MAX INDEPENDENT SET, MAX CLIQUE, MIN SET COVER, TSP, ….
But we have to find good solutions to these problems anyway – what do we do?
![Page 2: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/2.jpg)
2
• Simple approximation heuristics
including generic schemes such as LP-relaxation and rounding may find better solutions that the analysis guarantees on relevant concrete instances.
• We can improve the solutions using Local Search.
![Page 3: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/3.jpg)
3
Local Search
LocalSearch(ProblemInstance x)
y := feasible solution to x;
while 9 z ∊N(y): v(z)<v(y) do
y := z;
od;
return y;
![Page 4: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/4.jpg)
4
Examples of algorithms using local search
• Ford-Fulkerson algorithm for Max Flow• Klein’s algorithm for Min Cost Flow• Simplex Algorithm
![Page 5: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/5.jpg)
5
To do list
• How do we find the first feasible solution?• Neighborhood design?• Which neighbor to choose?• Partial correctness? • Termination? • Complexity?
Never Mind!
Stop when tired! (but optimize the time of each iteration).
![Page 6: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/6.jpg)
6
TSP
• Johnson and McGeoch. The traveling salesman problem: A case study (from Local Search in Combinatorial Optimization).
• Covers plain local search as well as concrete instantiations of popular metaheuristics such as tabu search, simulated annealing and evolutionary algorithms.
• An example of good experimental methodology!
![Page 7: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/7.jpg)
7
TSP
• Branch-and-cut method gives a practical way of solving TSP instances of up to ~ 1000 cities.
• Instances considered by Johnson and McGeoch: Random Euclidean instances and random distance matrix instances of several thousands cities.
![Page 8: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/8.jpg)
8
Local search design tasks
• Finding an initial solution
• Neighborhood structure
![Page 9: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/9.jpg)
9
The initial tour
• Christofides
• Greedy heuristic
• Nearest neighbor heuristic
• Clarke-Wright
![Page 10: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/10.jpg)
10
![Page 11: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/11.jpg)
11
Held-Karp lower bound
• Value of certain LP-relaxation of the TSP-problem.
• Guaranteed to be at least 2/3 of the true value for metric instances.
• Empirically usually within 0.01% (!)
![Page 12: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/12.jpg)
12
![Page 13: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/13.jpg)
13
2 2.5 3 3.5 4 4.50
1000
2000
3000
4000
5000
6000
GR
NN
CW
Random distance matrices
![Page 14: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/14.jpg)
14
Neighborhood design
Natural neighborhood structures:
2-opt, 3-opt, 4-opt,…
![Page 15: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/15.jpg)
15
2-opt neighborhood
![Page 16: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/16.jpg)
16
2-opt neighborhood
![Page 17: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/17.jpg)
17
2-opt neighborhood
![Page 18: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/18.jpg)
18
2-opt neighborhood
![Page 19: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/19.jpg)
19
2-optimal solution
![Page 20: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/20.jpg)
20
3-opt neighborhood
![Page 21: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/21.jpg)
21
3-opt neighborhood
![Page 22: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/22.jpg)
22
3-opt neighborhood
![Page 23: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/23.jpg)
23
Neighborhood Properties
• Size of k-opt neighborhood: O( )
• k ¸ 4 is rarely considered….
kn
![Page 24: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/24.jpg)
24
Discussion Exercise
Suggest local search based approximation heuristics for the following problems:
• MAXSAT• MAXCUT• MAX BISECTION• MAX INDEPENDENT SET
![Page 25: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/25.jpg)
25
![Page 26: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/26.jpg)
26
![Page 27: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/27.jpg)
27
![Page 28: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/28.jpg)
28
• One 3OPT move takes time O(n3). How is it possible to do local optimization on instances of size 106 ?????
![Page 29: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/29.jpg)
29
2-opt neighborhood
t1
t4
t3
t2
![Page 30: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/30.jpg)
30
A 2-opt move
• If d(t1, t2) · d(t2, t3) and d(t3,t4) · d(t4,t1), the move is not improving.
• Thus we can restrict searches for tuples where either d(t1, t2) > d(t2, t3) or d(t3, t4) > d(t4, t1).
• WLOG, d(t1,t2) > d(t2, t3).
![Page 31: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/31.jpg)
31
Neighbor lists
• For each city, keep a static list of cities in order of increasing distance.
• When looking for a 2-opt move, for each candidate for t1 with t2 being the next city, look in the neighbor list of t2 for t3 candidate, searching “inwards” from t1.
• For random Euclidean instance, expected time to for finding 2-opt move is linear.
![Page 32: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/32.jpg)
32
Problem
• Neighbor lists are very big.
• It is very rare that one looks at an item at position > 20.
• Solution: Prune lists to 20 elements.
![Page 33: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/33.jpg)
33
• Still not fast enough……
![Page 34: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/34.jpg)
34
Don’t-look bits.
• If a candidate for t1 was unsuccessful in previous iteration, and its successor and predecessor has not changed, ignore the candidate in current iteration.
![Page 35: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/35.jpg)
35
Variant for 3opt
• WLOG look for t1, t2, t3, t4,t5,t6 so that d(t1,t2) > d(t2, t3) and d(t1,t2)+d(t3,t4) > d(t2,t3)+d(t4, t5).
![Page 36: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/36.jpg)
36
Boosting local search
• Theme: How to escape local optima– Taboo search, Lin-Kernighan– Simulated annealing– Evolutionary algorithms
![Page 37: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/37.jpg)
37
Taboo search
• When the local search reaches a local minimum, keep searching.
![Page 38: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/38.jpg)
38
Local Search
LocalSearch(ProblemInstance x)
y := feasible solution to x;
while 9 z ∊N(y): v(z)<v(y) do
y := z;
od;
return y;
![Page 39: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/39.jpg)
39
Taboo search, attempt 1
LocalSearch(ProblemInstance x)
y := feasible solution to x;
while not tired do
y := best neighbor of y;
od;
return best solution seen;
![Page 40: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/40.jpg)
40
Serious Problem
• The modified local search will typically enter a cycle of length 2.
• As soon as we leave a local optimum, the next move will typically bring us back there.
![Page 41: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/41.jpg)
41
Attempt at avoiding cycling
• Keep a list of already seen solutions.
• Make it illegal (“taboo”) to enter any of them.
• Not very practical – list becomes long. Also, search tend to circle around local optima.
![Page 42: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/42.jpg)
42
Taboo search
• After a certain “move” has been made, it is declared taboo and may not be used for a while.
• “Move” should be defined so that it becomes taboo to go right back to the local optimum just seen.
![Page 43: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/43.jpg)
43
Discussion Exercise
• Suggest taboo search heuristics for the following problems:
• MAXSAT• MAXCUT• MAX BISECTION• MAX INDEPENDENT SET
![Page 44: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/44.jpg)
44
MAXSAT
• Given a formula f in CNF, find an assignment a to the variables of f, satisfying as many clauses as possible.
![Page 45: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/45.jpg)
45
Solving MAXSAT using GSAT
• Plain local search method: GSAT.
• GSAT Neighborhood structure: Flip the value of one of the variables.
• Do steepest descent.
![Page 46: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/46.jpg)
46
Taboo search for MAXSAT
• As in GSAT, flip the value of one of the variables and choose the steepest descent.
• When a certain variable has been flipped, it cannot be flipped for, say, n/4 iterations.We say the variable is taboo. When in a local optimum, make the “least bad” move.
![Page 47: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/47.jpg)
47
For each variable x not in T, compute the number of clauses satisfied bythe assignment obtained from a by flipping the value of x. Let x be the best choice and let a’ be the corresponding assignment.
TruthAssignment TabooGSAT(CNFformula f) t := 0; T :=Ø; a,best := some truth assignment; repeat Remove all variables from T with time stamp < t-n/4;. a = a’; Put x in T with time stamp t; if a is better than best then best = a; t := t +1 until tiredreturn best;
![Page 48: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/48.jpg)
48
TSP
• No variant of “pure” taboo search works very well for TSP.
• Johnson og McGeoch: Running time 12000 as slow as 3opt on instances of size 1000 with no significant improvements.
• General remark: Heuristics should be compared on a time-equalized basis.
![Page 49: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/49.jpg)
49
Lin-Kernighan
• Very successful classical heuristic for TSP.
• Similar to Taboo search: Boost 3-opt by sometimes considering “uphill” (2-opt) moves.
• When and how these moves are considered is more “planned” and “structured” than in taboo search, but also involves a “taboo criterion”.
• Often misrepresented in the literature!
![Page 50: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/50.jpg)
50
Looking for 3opt moves
• WLOG look for t1, t2, t3, t4,t5,t6 so that d(t1,t2) > d(t2, t3) and d(t1,t2) + d(t3,t4) > d(t2,t3)+d(t4, t5).
• The weight of (b) smaller than length of original tour.
![Page 51: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/51.jpg)
51
Lin-Kernighan move
![Page 52: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/52.jpg)
52
Lin-Kernighan moves
• A 2opt move can be viewed as LK-move.
• A 3opt move can be viewed as two LK-moves.
• The inequalities that can be assumed WLOG for legal 3-opt (2-opt) moves state than the “one-tree”s involved are shorter than the length of the original tour.
![Page 53: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/53.jpg)
53
Lin-Kernighan search• 3opt search with “intensification”.
• Whenever a 3opt move is being made, we view it as two LK-moves and see if we in addition can perform a number of LK-moves (an LK-search) that gives an even better improvement.
• During the LK-search, we never delete an edge we have added by an LK-move, so we consider at most n-2 additional LK-moves (“taboo criterion”). We keep track of the · n solutions and take the best one.
• During the LK-search, the next move we consider is the best LK-move we can make. It could be an uphill move.
• We only allow one-trees lighter than the current tour. Thus, we can use neighbor lists to speed up finding the next move.
![Page 54: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/54.jpg)
54
![Page 55: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/55.jpg)
55
What if we have more CPU time?
• We could repeat the search, with different starting point.
• Seems better not to throw away result of previous search.
![Page 56: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/56.jpg)
56
Iterated Lin-Kernighan
• After having completed a Lin-Kernighan run (i.e., 3opt, boosted with LK-searches), make a random 4-opt move and do a new Lin-Kernighan run.
• Repeat for as long as you have time. Keep track of the best solution seen.
• The 4-opt moves are restricted to double bridge moves (turning A1 A2 A3 A4 into A2 A1 A4 A3.)
![Page 57: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/57.jpg)
57
![Page 58: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/58.jpg)
58
Boosting local search
• Simulated annealing (inspired by physics)
• Evolutionary algorithms (inspired by biology)
![Page 59: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/59.jpg)
59
Metropolis algorithm and simulated annealing
• Inspired by physical systems (described by statistical physics).
• Escape local minima by allowing move to worse solution with a certain probability.
• The probability is regulated by a parameter, the temperature of the system.
• High temperature means high probability of allowing move to worse solution.
![Page 60: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/60.jpg)
60
Metropolis Minimization
FeasibleSolution Metropolis(ProblemInstance x, Real T)
y := feasible solution to x;
repeat
Pick a random member z of N(y);
with probability min(e(v(y)-v(z))/T, 1) let y:=z;
until tired;
return the best y found;
![Page 61: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/61.jpg)
61
Why ?
• Improving moves are always accepted, bad moves are accepted with probability decreasing with badness but increasing with temperature.
• Theory of Markov chains: As number of moves goes to infinity, the probability that y is some value a becomes proportional to exp(-v(a)/T)
• This convergence is in general slow (an exponential number of moves must be made). Thus, in practice, one should feel free to use other expressions.
)1,min( /))()(( Tzvyve
![Page 62: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/62.jpg)
62
What should T be?
Intuition:T large: Convergence towards limit distribution fast, but
limit distribution does not favor good solutions very much (if T is infinity, the search is random).
T close to 0 : Limit distribution favor good solution, but convergence slow.
T = 0: Plain local search.
One should pick “optimal” T.
![Page 63: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/63.jpg)
63
Simulated annealing
• As Metropolis, but T is changed during the execution of the algorithm.
• T starts out high, but is gradually lowered.
• Hope: T stays at near-optimal value sufficiently long.
• Analogous to models of crystal formation.
![Page 64: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/64.jpg)
64
Simulated Annealing
FeasibleSolution Metropolis(ProblemInstance x)
y := feasible solution to x; T:=big;
repeat
T := 0.99 T ;
Pick a random member z of N(y);
with probability min(e(v(y)-v(z))/T, 1) let y:=z
until tired;
return the best y found;
![Page 65: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/65.jpg)
65
Simulated annealing
• THM: If T is lowered sufficiently slowly (exponentially many moves must be made), the final solution will with high probability be optimal!
• In practice T must be lowered faster.
![Page 66: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/66.jpg)
66
TSP
• Johnson and McGeoch: Simulated annealing with 2opt neightborhood is promising but neighborhood must be pruned to make it efficient.
• Still, not competitive with LK or ILK on a time-equalized basis (for any amount of time).
![Page 67: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/67.jpg)
67
![Page 68: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/68.jpg)
68
Local Search – interpreted biologically
FeasibleSolution LocalSearch(ProblemInstance x) y := feasible solution to x; while Improve(y) != y and !tired do y := Improve(y); od; return y;
Improve(y) is an offspring of y. The fitter of the two will survive
Maybe y should be allowed to have other children?Maybe the “genetic material” of y should be combined with
the “genetic material” of others?
![Page 69: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/69.jpg)
69
Evolutionary/Genetic algorithms
• Inspired by biological systems (evolution and adaptation)
• Maintain a population of solution
• Mutate solutions, obtaining new ones.
• Recombine solutions, obtaining new ones.
• Kill solutions randomly, with better (more fit) solutions having lower probability of dying.
![Page 70: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/70.jpg)
70
Evolutionary Algorithm
FeasibleSolution EvolSearch(ProblemInstance x)
P := initial population of size m of feasible solutions to x;
while !tired do
Expand(P);
Selection(P)
od;
return best solution obtained at some point;
![Page 71: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/71.jpg)
71
Expansion of Population
Expand(Population P)
for i:=1 to m do
with probability p do
ExpandByMutation(P)
else (i.e., with probability 1-p)
ExpandByCombination(P)
htiw
od
![Page 72: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/72.jpg)
72
Expand Population by Mutation
ExpandByMutation(Population P)
Pick random x in P;
Pick random y in N(x);
P := P U {y};
![Page 73: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/73.jpg)
73
Expand Population by Combination
ExpandByCombination(Population P)
Pick random x in P;
Pick random y in P;
z := Combine(x,y);
P := P U {z};
![Page 74: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/74.jpg)
74
Selection
Selection(Population P)
while |P| > m do
Randomly select a member x of P but
select each particular x with probability
monotonically increasing with v(x);
P := P – {x};
od
![Page 75: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/75.jpg)
75
How to combine?
• Problem specific decision.
• There is a “Generic way”: Base it on the way biological recombination of genetic material is done.
![Page 76: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/76.jpg)
76
Biological Recombination
• Each feasible solution (the phenotype) is represented by a string over a finite alphabet (the genotype).
• String x is combined with string y by splitting x in x1x2 and y in y1y2 with |x1|=|y1| and |x2|=|y2| and returning x1y2.
![Page 77: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/77.jpg)
77
Evolutionary algorithms
• Many additional biological features can be incorporated.
• Dozens of decisions to make and knobs to turn.
• One option: Let the decisions be decided by evolution as well!
![Page 78: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/78.jpg)
78
Conclusions of McGeoch and Johnson
Best known heuristics for TSP:
• Small CPU time: Lin-Kernighan.
• Medium CPU time: Iterated Lin-Kernighan (Lin-Kernighan + Random 4opt moves).
• Very Large CPU time: An evolutionary algorithm.
![Page 79: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/79.jpg)
79
Combine operation in winning approach for large CPU time
x:
![Page 80: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/80.jpg)
80
y:
Combine operation in winning approach for large CPU time
![Page 81: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/81.jpg)
81
Take union of x and y
x+y:
![Page 82: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/82.jpg)
82
Solve to optimality, using only edges from x + y
x+y:
![Page 83: It is unlikely that there are efficient approximation algorithms with a very good worst case approximation ratio for MAXSAT, MIN NODE COVER, MAX INDEPENDENT](https://reader030.vdocuments.mx/reader030/viewer/2022032703/56649d2e5503460f94a059f6/html5/thumbnails/83.jpg)
83
Combine(x,y)
• Combine(x,y): Take the graph consisting of edges of x and y. Find the optimal TSP solution using only edges from that graph.
• Finding the optimal TSP tour in a graph which is the union of two Hamiltonian paths can be done efficiently in practice.
• More “obvious” versions of combine (like the generic combine) yield evolutionary algorithms which are not competitive with simpler methods.