1. TEMPLATE: ADRIAN WILKEAnt Colony OptimizationUniversity of PaderbornBio-Inspired Networking SeminarJanuary 18, 2011Adrian Wilke

2. Ant Colony OptimizationTalk focus: RoutingTEMPLATE: ADRIAN WILKEAnt Colony Optimization - Adrian Wilke January 18, 2011 2 3. History: Double bridge experiments[Goss et al., 1989]TEMPLATE: ADRIAN WILKE r = ratio of long to short branchAnt Colony Optimization - Adrian Wilke January 18, 2011 3 4. History: Double bridge experiments[Deneubourg et al., 1990]TEMPLATE: ADRIAN WILKE Analogy Students on a campus: The more a point is passed, the more the vegetation is trampled.Ant Colony Optimization - Adrian Wilke January 18, 2011 4 5. From realbehaviour to articial systemsTEMPLATE: ADRIAN WILKE Stigmergy: Indirect Possible elds?communication throughOptimization problems,modications in environmentrouting[Caro and Dorigo, 1998] Routing: Direct data ow Where to store pheromone from source to destinationin distributed systems?(Shortest path) Evaporation? (Pheromone Congestion?decreases over time) (Too much load)Ant Colony Optimization - Adrian Wilke January 18, 2011 5 6. Introductory examplePositive feedback TEMPLATE: ADRIAN WILKE (a) (b) (c)(d) Exploring Pheromone Additional Shortest trail node pathAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 6 7. Ant System[Dorigo et al., 1996] TEMPLATE: ADRIAN WILKEA new general-purpose heuristic algorithm [...] to solve different optimization problemsCovered topics Path selection: Choice of the next node Updating pheromone Ant System algorithmAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 7 8. Ant System: Transition probabilityPath selection TEMPLATE: ADRIAN WILKE ??Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 8 9. Ant System: Transition probabilityChoice of the next node TEMPLATE: ADRIAN WILKE [ij (t)] [ij ] if j alloweda apij (t) = [iy (t)] [iy ](1) y alloweda 0otherwiseAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 9 10. Ant System: Transition probabilityChoice of the next node TEMPLATE: ADRIAN WILKE [ij (t)] [ij ] if j alloweda apij (t) = [iy (t)] [iy ](1) y alloweda 0otherwiseij (t) is the intensity of trail on edge (i, j) at time t.Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 9 11. Ant System: Transition probabilityChoice of the next nodeTEMPLATE: ADRIAN WILKE [ij (t)] [ij ] if j allowedaa pij (t) = [iy (t)] [iy ](1) y alloweda0otherwise ij (t) is the intensity of trail on edge (i, j) at time t.dij is the euclidean distance i j between two nodes i and j.1ij =dijis called the visibility. Smaller distance higher visibility.Ant Colony Optimization - Adrian Wilke January 18, 2011 9 12. Ant System: Transition probabilityChoice of the next nodeTEMPLATE: ADRIAN WILKE [ij (t)] [ij ] if j allowedaa pij (t) = [iy (t)] [iy ](1) y alloweda0otherwise ij (t) is the intensity of trail on edge (i, j) at time t.dij is the euclidean distance i j between two nodes i and j.1ij =dijis called the visibility. Smaller distance higher visibility. is the weight of the trail (Simulation: = 1). is the weight of the visibility (Simulation: = 5).Ant Colony Optimization - Adrian Wilke January 18, 2011 9 13. Ant System: Transition probabilityChoice of the next node TEMPLATE: ADRIAN WILKE [ij (t)] [ij ] if j allowedaa pij (t) = [iy (t)] [iy ] (1) y alloweda0 otherwise ij (t) is the intensity of trail on edge (i, j) at time t.dij is the euclidean distance i j between two nodes i and j.1ij =dijis called the visibility. Smaller distance higher visibility. is the weight of the trail (Simulation: = 1). is the weight of the visibility (Simulation: = 5). aDenominator of fraction ensures thatpij (t) = 1.jNAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 9 14. Ant System: Transition probabilityChoice of the next node TEMPLATE: ADRIAN WILKE [ij (t)] [ij ] if j allowedaa pij (t) = [iy (t)] [iy ] (1) y alloweda0 otherwise ij (t) is the intensity of trail on edge (i, j) at time t.dij is the euclidean distance i j between two nodes i and j.1ij =dijis called the visibility. Smaller distance higher visibility. is the weight of the trail (Simulation: = 1). is the weight of the visibility (Simulation: = 5). aDenominator of fraction ensures thatpij (t) = 1.jNtabua is the tabu list of nodes, which were alredy visited by the ath ant.alloweda N tabua is the list of nodes, which were not visited by the ath ant.Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 9 15. Ant System: Transition probabilityChoice of the next nodeTEMPLATE: ADRIAN WILKE [ij (t)] [ij ]... = =1 [iy (t)] [iy ] y alloweda=5Greedy: ??Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 10 16. Ant System: Transition probabilityChoice of the next nodeTEMPLATE: ADRIAN WILKE [ij (t)] [ij ]... = =1 [iy (t)] [iy ] y alloweda =5Greedy: ??(Image source: [Dorigo et al., 1996])Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 10 17. Ant SystemUpdating pheromone TEMPLATE: ADRIAN WILKE S S ? ? S D D DAnt Colony Optimization - Adrian Wilke January 18, 2011 11 18. Ant SystemUpdating pheromoneTEMPLATE: ADRIAN WILKE QaLa if ath ant uses edge (i, j) in its tour ij =(2) 0otherwiseQis a constant, its inuence was found to be negligible.La is the length of the tour of the ath ant.Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 12 19. Ant SystemUpdating pheromoneTEMPLATE: ADRIAN WILKE QaLa if ath ant uses edge (i, j) in its tour ij =(2) 0otherwiseQis a constant, its inuence was found to be negligible.La is the length of the tour of the ath ant.m a ij = ij (3)a=1Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 12 20. Ant SystemUpdating pheromoneTEMPLATE: ADRIAN WILKE QaLa if ath ant uses edge (i, j) in its tour ij =(2) 0otherwiseQis a constant, its inuence was found to be negligible.La is the length of the tour of the ath ant. maij = ij(3) a=1 ij (t + n) = ij (t) + ij(4) < 1 is a constant , which represents the evaporation of trail.Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 12 21. Ant SystemAlgorithm: All together TEMPLATE: ADRIAN WILKERequire: NCMAX NC 0 and t 0 and s 1 ij 0 ij (t) c shortestTour for all ants a = 1 to m do tabua (s) = choose start nodeend forAnt Colony Optimization - Adrian Wilke January 18, 2011 13 22. Ant SystemAlgorithm: All together TEMPLATE: ADRIAN WILKERequire: NCMAX NC 0 and t 0 and s 1 ij 0 ij (t) c shortestTour for all ants a = 1 to m do tabua (s) = choose start nodeend forrepeat { // Each cyclerepeat s s +1 for all ants a = 1 to m doa move a to next node j by pij (t) insert node j in tabua (s) end foruntil s = nAnt Colony Optimization - Adrian Wilke January 18, 2011 13 23. Ant SystemAlgorithm: All togetherTEMPLATE: ADRIAN WILKERequire: NCMAX for all ants a = 1 to m do NC 0 and t 0 and s 1 compute length La of tabua ij 0 shortestTour update(tabua , La ) ij (t) c end for shortestTour for all edges (i, j) and all ants a = 1 to m dofor all ants a = 1 to m docompute ija tabua (s) = choose start nodecompute ijend forend forrepeat { // Each cycle for all edges (i, j) docompute ij (t + n)repeatij 0 s s +1 end for for all ants a = 1 to m doa move a to next node j by pij (t) insert node j in tabua (s) end foruntil s = nAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 13 24. Ant SystemAlgorithm: All togetherTEMPLATE: ADRIAN WILKERequire: NCMAX for all ants a = 1 to m do NC 0 and t 0 and s 1 compute length La of tabua ij 0 shortestTour update(tabua , La ) ij (t) c end for shortestTour for all edges (i, j) and all ants a = 1 to m dofor all ants a = 1 to m docompute ija tabua (s) = choose start nodecompute ijend forend forrepeat { // Each cycle for all edges (i, j) docompute ij (t + n)repeatij 0 s s +1 end for for all ants a = 1 to m doa move a to next node j by pij (t)NC NC + 1 and t t + n insert node j in tabua (s)move a to tabua (1) and clear tabua (2 . . . n) end foruntil s = n} until NC < NCMAX and no stagnation return shortestTourAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 13 25. Ant SystemSome results TEMPLATE: ADRIAN WILKE Oliver30 problem(30 nodes TSP) Columns: basicheuristic andimprovements Entries: Solutions Ant System outperformed 2-optBut: Longer computational time Best-known solution in less than 400 cycles,100 cycles for values under 430 As effective or better than some general-purpose heuristicsAnt Colony Optimization - Adrian Wilke January 18, 2011 14 26. From AntSystem to AntNetTEMPLATE: ADRIAN WILKEAnt SystemAntNet Focus: General-purpose Focus: Internet Protocolheuristicwith irregular topology Global data structure Routing tableAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 15 27. AntNet: Data structures[Caro and Dorigo, 1998]TEMPLATE: ADRIAN WILKE Routing tableTraffic modelQueuesdestinations destinationslow priority mean neighboursvariance - data packetstime - forward antshigh priorityprobabilistictraffic distribution desirabilitybackward antsnode identifiers s k xelapsed time 0 5 8xAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 16 28. AntNetAlgorithm in shortTEMPLATE: ADRIAN WILKE At time intervals: Forward ant is launched to a destinationChoice of destination: Data ow to possible targets Next node on a tour is chosen bythe routing table entries and queue lengths of neighbours At nodes on path, forward ant memorizesthe current node ID and the time from source node At a destination the forward ant is destroyed and abackward ant is launched Backward ants use the known paths back andupdate the routing table entries andthe trafc model on path nodesAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 17 29. From AntNet to Mobile ad hoc networks TEMPLATE: ADRIAN WILKEAntNet Introduced: Routing tables Approach works for IP What about dynamic networkswith leaving nodesAnt Colony Optimization - Adrian Wilke January 18, 2011 18 30. From AntNet to Mobile ad hoc networksTEMPLATE: ADRIAN WILKEAntNet MANETs Introduced: Routing tables Mobile ad hoc networks Approach works for IP Restriction: Limited bandwidth No central component, no xed infractructure Dynamic structure: Nodes are joining and leaving Paths can become useless What about dynamic networkswith leaving nodesAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 18 31. AntHocNet[Di Caro et al., 2005] TEMPLATE: ADRIAN WILKEAn approach for MANETs Hybrid approach: Reactive path setup phase Proactive maintenance Also of interest: Pheromone updatesAnt Colony Optimization - Adrian Wilke January 18, 2011 19 32. AntHocNetReactive path setupTEMPLATE: ADRIAN WILKE Node s wants to reach d Same generation: only best Unicast, if pheromone First hop different:available. Else: broadcast Acceptance factor 0.9Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 20 33. AntHocNetPheromone updatesTEMPLATE: ADRIAN WILKE Pheromone calculated by backward antsusing a known Path PMathematicalInformali iiTmac = Tmac + (1 ) tmacTime to send a packet form anode i (Running average)i iiTi+1 = (Qmac + 1) TmacTime for packets in queue (+1) n1 iTP =i=1 Ti+1Time for whole path T i +hT id = d 2 hopConsider unloaded conditions i iTnd = Tnd +i(1 ) dFinal routing table entry(Running average)Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 21 34. AntHocNetProactive probing & exploration TEMPLATE: ADRIAN WILKE No broadcast: Probe of path Broadcast: Improve paths Path variationsAnt Colony Optimization - Adrian Wilke January 18, 2011 22 35. AntHocNetProactive maintenance TEMPLATE: ADRIAN WILKEHelloMessagesLink failures Every 1 sec: HelloMessage Neighbors which lost best(Just senders address) path: Also broadcast Received HelloMessageLinkFailureNotication Add in routing table Failed transmissionLink failures Broadcast PathRepairAnt 2 HelloMessages missing: (like reactive forward ant) Remove from routing table But: Max. 2 broadcasts BroadcastLinkFailureNotication LinkFailureNotication: Listof destinations to which bestpath lost & new best delayAnt Colony Optimization - Adrian Wilke January 18, 2011 23 36. AntHocNetProactive maintenance TEMPLATE: ADRIAN WILKEHelloMessagesLink failures Every 1 sec: HelloMessage Neighbors which lost best(Just senders address) path: Also broadcast Received HelloMessageLinkFailureNotication Add in routing table Failed transmissionLink failures Broadcast PathRepairAnt 2 HelloMessages missing: (like reactive forward ant) Remove from routing table But: Max. 2 broadcasts BroadcastLinkFailureNotication Task for you: LinkFailureNotication: List Count the broadcastsof destinations to which beston this page!path lost & new best delayAnt Colony Optimization - Adrian Wilke January 18, 2011 23 37. From AntHocNet to HOPNET TEMPLATE: ADRIAN WILKEAntHocNetHOPNET Reactive path setup phase Hybrid approach Proactive maintenance Avoid overhead and Results: Better than AODVa achieve scalabilityexcept for overhead Idea: Use Zones Inside zones:Proactively maintained Between zones:Reactive part, on demand a AODV: Ad-hoc on-demand distance vector routing,reference algorithm in areaAnt Colony Optimization - Adrian Wilke January 18, 2011 24 38. HOPNET: Routing tables[Wang et al., 2009] TEMPLATE: ADRIAN WILKEIntrazone routing tableInterzone routing table Inside zones Between zones Proactively maintained Reactive part, on demand Size: Nodes in zone Used, if destination not inNode degree (neighbours) Intrazone routing tablePheromoneDestinationVisited timesSequence numberHops Between node andPath nodes in zone ExpireSequence number Incremented each time f./b.-ants are generatedAnt Colony Optimization - Adrian Wilke January 18, 2011 25 39. HOPNET example TEMPLATE: ADRIAN WILKES wants to send data to T. (Reworked example from paper)Ant Colony Optimization - Adrian Wilke January 18, 2011 26 40. HOPNET example TEMPLATE: ADRIAN WILKE T not in zone. S sends external forward ant to pheripheral nodes.Ant Colony Optimization - Adrian Wilke January 18, 2011 27 41. HOPNET exampleTEMPLATE: ADRIAN WILKE D sends external forward ant to K and M,the other ants are destroyed.Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 28 42. HOPNET exampleTEMPLATE: ADRIAN WILKEM sends external forward ant to pheripheral nodes, except of S and A.Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 29 43. HOPNET example TEMPLATE: ADRIAN WILKE T is in intrazone routing tables.Two paths were found.A backward ant is launched.Ant Colony Optimization - Adrian Wilke January 18, 2011 30 44. HOPNET: Pheromone updates ISource update algorithm TEMPLATE: ADRIAN WILKE Internal forward ants update pheromone during movementvs Vs Vi Vjvi vl (vi , vs ) = (vi , vs ) + T (vs , vi ) + w (vi , vj )constantPheromone(neighbor , source)+ =time(vs , vi ) + timeOnLink (vl , vs ) = (vl , vs ) (1 E ), l = iAnt Colony Optimization - Adrian Wilke January 18, 2011 31 45. HOPNET: Pheromone updates IIUpdates by backward ants TEMPLATE: ADRIAN WILKE Backward ants update pheromone vD vb VSVk VbVD vl (vb , vD ) = (vb , vD ) + T (vS , vD ) T (vS , vk ) (vl , vD ) = (vl , vD ) (1 E ), l = bAnt Colony Optimization - Adrian Wilke January 18, 2011 32 46. HOPNETResults: Overhead and Scalability TEMPLATE: ADRIAN WILKEAnt Colony Optimization - Adrian Wilke January 18, 2011 33 47. Ant Colony OptimizationSummaryTEMPLATE: ADRIAN WILKEOverview History: Double bridge experiments Ant System: First approach AntNet: Wired IP networks AntHocNet: First MANET approach HOPNET: MANET with zones To take away Stigmergy: Indirect communication throughmodications in environment Routing table for pheromone Proactive and reactive partsAnt Colony Optimization - Adrian WilkeJanuary 18, 2011 34 48. Thank you for your attentionTEMPLATE: ADRIAN WILKE11Source: http://xkcd.com/638/, Found: http://www.idsia.ch/gianni/Ant Colony Optimization - Adrian WilkeJanuary 18, 2011 35 49. References I TEMPLATE: ADRIAN WILKE[Bonabeau et al., 1999] Bonabeau, E., Dorigo, M., and Theraulz, G. (1999). Swarm Intelligence. Oxford University Prress.[Caro and Dorigo, 1998] Caro, G. D. and Dorigo, M. (1998).AntNet: Distributed Stigmergetic Control for Communications Networks.Journal of Articial Intelligence Research, 9:317365.[Deneubourg et al., 1990] Deneubourg, J.-L., Aron, S., Goss, S., and Pasteels, J. M. (1990).The Self-Organizing Exploratory Pattern of the Argentine Ant.Journal of lnsect Behavior, 3(2).[Di Caro et al., 2005] Di Caro, G., Ducatelle, F., and Gambardella, L. M. (2005). AntHocNet: an adaptive nature-inspired algorithm for routing in mobile ad hoc networks. European Transactions on Telecommunications, 16(5):443455.[Dorigo et al., 1996] Dorigo, M., Maniezzo, V., and Colorni, A. (1996).Ant system: optimization by a colony of cooperating agents.Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 26(1):2941.[Dorigo and Stutzle, 2004] Dorigo, M. and Stutzle, T. (2004). Ant Colony Optimization.MIT press.Ant Colony Optimization - Adrian Wilke January 18, 2011 36 50. References II TEMPLATE: ADRIAN WILKE[Dressler and Akan, 2010] Dressler, F. and Akan, O. B. (2010). A survey on bio-inspired networking. Computer Networks, 54(6):881900.[Goss et al., 1989] Goss, S., Aron, S., Deneubourg, J. L., and Pasteels, J. M. (1989).Self-organized Shortcuts in the Argentine Ant.Naturwissenschaften, 76:579581.[Wang et al., 2009] Wang, J., Osagie, E., Thulasiraman, P., and Thulasiram, R. K. (2009).HOPNET: A hybrid ant colony optimization routing algorithm for mobile ad hoc network.Ad Hoc Networks, 7(4):690705.Ant Colony Optimization - Adrian Wilke January 18, 2011 37