Artificial Intelligence

Local search algorithmsIncomputer science,local searchis metaheuristicmethod for solving computationally hardoptimizationproblems. Local search can be used on problems that can be formulated as finding a solution maximizing a criterion among a number ofcandidate solutions. Local search algorithms move from solution to solution in the space of candidate solutions (thesearch space) by applying local changes, until a solution deemed optimal is found or a time bound is elapsed.1In many optimization problems, the state space is the space of all possible complete solutionsWe have an objective function that tells us how good a given state is, and we want to find the solution (goal) by minimizing or maximizing the value of this function

Local search algorithmsExample: n-queens problemPut n queens on an n n board with no two queens on the same row, column, or diagonalState space: all possible n-queen configurationsWhats the objective function?Number of pair-wise conflicts

3Example: Traveling salesman problemFind the shortest tour connecting a given set of citiesState space: all possible toursObjective function: length of tour

4Local search algorithmsIn many optimization problems, the state space is the space of all possible complete solutionsWe have an objective function that tells us how good a given state is, and we want to find the solution (goal) by minimizing or maximizing the value of this functionThe start state may not be specifiedThe path to the goal doesnt matter

In such cases, we can that keep a single current state anduse local search algorithms gradually try to improve it5Example: n-queens problemPut n queens on an n n board with no two queens on the same row, column, or diagonalState space: all possible n-queen configurationsObjective function: number of pairwise conflictsWhats a possible local improvement strategy?Move one queen within its column to reduce conflicts

6Example: Traveling Salesman ProblemFind the shortest tour connecting n citiesState space: all possible toursObjective function: length of tourWhats a possible local improvement strategy?Start with any complete tour, perform pairwise exchanges

7Hill-climbing searchInitialize current to starting stateLoop:Let next = highest-valued successor of currentIf value(next) < value(current) return currentElse let current = next

Variants: choose first better successor, randomly choose among better successorsLike climbing mount Everest in thick fog with amnesia8The state space landscape

9Iterative Improvement and Hill Climbing

The main problem that hill climbing can encounter is that of local maxima.

This occurs when the algorithm stops making progress towards an optimal solution; mainly due to the lack of immediate improvement in adjacent states.

Local maxima can be avoided by a variety of methods: Simulated annealing tackles this issue by allowing some steps to be taken which decrease the immediate optimality of the current state.

Random-Restart Hill-ClimbingAnother way of solving the local maxima problem involves repeated explorations of the problem space.

Random-restart hill-climbing conducts a series of hill-climbing searches from randomly generated initial states, running each until it halts or makes no distinct progress.Simulated annealing searchIdea: escape local maxima by allowing some "bad" moves but gradually decrease their frequencyProbability of taking downhill move decreases with number of iterations, steepness of downhill moveControlled by annealing scheduleInspired by tempering of glass, metal

12Simulated annealing searchInitialize current to starting stateFor i = 1 to If T(i) = 0 return currentLet next = random successor of currentLet = value(next) value(current)If > 0 then let current = nextElse let current = next with probability exp(/T(i))

13Effect of temperature

exp(/T)14Simulated annealing searchOne can prove: If temperature decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching oneHowever:This usually takes impractically longThe more downhill steps you need to escape a local optimum, the less likely you are to make all of them in a rowMore modern techniques: general family of Markov Chain Monte Carlo (MCMC) algorithms for exploring complicated state spaces15Local beam searchStart with k randomly generated statesAt each iteration, all the successors of all k states are generatedIf any one is a goal state, stop; else select the k best successors from the complete list and repeat

Is this the same as running k greedy searches in parallel?

Greedy searchBeam search16Genetic algorithmsVariant of local beam search with sexual recombination.

Genetic algorithmsVariant of local beam search with sexual recombination.

19AI 1Genetic algorithmfunction GENETIC_ALGORITHM( population, FITNESS-FN) return an individualinput: population, a set of individualsFITNESS-FN, a function which determines the quality of the individualrepeatnew_population empty setloop for i from 1 to SIZE(population) dox RANDOM_SELECTION(population, FITNESS_FN)y RANDOM_SELECTION(population, FITNESS_FN)child REPRODUCE(x,y)if (small random probability) then child MUTATE(child )add child to new_populationpopulation new_populationuntil some individual is fit enough or enough time has elapsedreturn the best individual20AI 1Exploration problemsUntil now all algorithms were offline.Offline= solution is determined before executing it.Online = interleaving computation and actionOnline search is necessary for dynamic and semi-dynamic environmentsIt is impossible to take into account all possible contingencies.Used for exploration problems:Unknown states and actions.e.g. any robot in a new environment, a newborn baby,Online search problemsAgent knowledge:ACTION(s): list of allowed actions in state sC(s,a,s): step-cost function (! After s is determined)GOAL-TEST(s)An agent can recognize previous states.Actions are deterministic.Access to admissible heuristic h(s)e.g. manhattan distance

Online search problemsObjective: reach goal with minimal costCost = total cost of travelled pathCompetitive ratio=comparison of cost with cost of the solution path if search space is known. Can be infinite in case of the agentaccidentally reaches dead ends

The adversary argumentAssume an adversary who can construct the state space while the agent explores itVisited states S and A. What next?Fails in one of the state spacesNo algorithm can avoid dead ends in all state spaces.

Online search agentsThe agent maintains a map of the environment.Updated based on percept input.This map is used to decide next action.

Note difference with e.g. A*An online version can only expand the node it is physically in (local order)Online DF-searchfunction ONLINE_DFS-AGENT(s) return an actioninput: s, a percept identifying current statestatic: result, a table indexed by action and state, initially emptyunexplored, a table that lists for each visited state, the action not yet triedunbacktracked, a table that lists for each visited state, the backtrack not yet trieds,a, the previous state and action, initially null

if GOAL-TEST(s) then return stopif s is a new state then unexplored[s] ACTIONS(s)if s is not null then doresult[a,s] sadd s to the front of unbackedtracked[s]if unexplored[s] is empty thenif unbacktracked[s] is empty then return stopelse a an action b such that result[b, s]=POP(unbacktracked[s])else a POP(unexplored[s])s sreturn aOnline DF-search, exampleAssume maze problem on 3x3 grid.s = (1,1) is initial stateResult, unexplored (UX), unbacktracked (UB), are emptyS,a are also empty

AI 1Online DF-search, exampleGOAL-TEST((,1,1))?S not = G thus false(1,1) a new state? TrueACTION((1,1)) -> UX[(1,1)]{RIGHT,UP}s is null?True (initially)UX[(1,1)] empty? FalsePOP(UX[(1,1)])->aA=UPs = (1,1)Return a

S=(1,1)Online DF-search, exampleGOAL-TEST((2,1))?S not = G thus false(2,1) a new state? TrueACTION((2,1)) -> UX[(2,1)]{DOWN}s is null?false (s=(1,1))result[UP,(1,1)]