Advanced
Artificial Intelligence
Tabu Search
Prof Salwani Abdullah
[email protected]
Tabu Search
Proposed independently by Glover (1986) and
Hansen (1986)
“a meta-heuristic superimposed on another
heuristic. The overall approach is to avoid
entrapment in cycles by forbidding or penalizing
moves which take the solution, in the next
iteration, to points in the solution space
previously visited (hence tabu).”
Tabu Search
Accepts non-improving solutions deterministically
in order to escape from local optima (where all the
neighbouring solutions are non- improving) by
guiding a steepest descent local search (or steepest
ascent hill climbing) algorithm.
Tabu Search
After evaluating a number of neighbourhoods, we
accept the best one, even if it is low quality on cost
function.
-accept worse move
Uses of memory in two ways:
-prevent the search from revisiting previously
visited solutions;
-explore the unvisited areas of the solution
space; for example,
Tabu Search
Use past experiences to improve current decision
making.
By using memory (a “tabu list”) to prohibit certain
moves - makes tabu search a global optimizer
rather than a local optimizer.
Tabu Search
After evaluating a number of neighbourhoods, we
accept the best one, even if it is low quality on cost
function.
- Accept worse move
Uses of memory in two ways:
-prevent the search from revisiting previously visited
solutions;
-explore the unvisited areas of the solution space; for
example,
Tabu Search
Use past experiences to improve current decision
making.
By using memory (a “tabu list”) to prohibit certain
moves - makes tabu search a global optimizer rather
than a local optimizer.
Tabu Search vs SA
Accept worse move
Selection of neighbourhoods
Use of memory
Uses of memory during the
search?
Intelligence needs memory!
Information on characteristics of good solutions
(or bad solutions!)
Uses of memory during the
search?
Tabu move – what does it mean?
• Not allowed to re-visit exact the same state
that we’ve been before
• Discouraging some patterns in solution: e.g. in
TSP, tabu a state that has the towns listed in
the same order that we’ve seen before.
Uses of memory during the
search?
Tabu move – what does it mean?
• Not allowed to return to the state that the
search has just come from.
– just one solution remembered
– smaller data structure in tabu list
Uses of memory during the
search?
Tabu move – what does it mean?
• Tabu a small part of the state
-In TSP, tabu the two cities just been
considered in the last move – search is forced to
consider other cities.
Tabu Search Cycle
Neighbour solution
Current solution based on the defined
neighbourhood
Select the best non- Evaluate objective
tabu neighbour and function
update the tabu list
Tabu Search Algorithm
Function TABU_SEARCH(Problem)
returns a solution state
Inputs: Problem, a problem
Local Variables:
Current, a state
Next, a state
BestSolutionSeen, a state
H, a history of visited states
Tabu Search Algorithm
Current = MAKE-NODE(INITIAL-STATE[Problem])
While not terminate
Next = a highest-valued successor of Current
If (not Move_Tabu(H,Next) or Aspiration(Next)) then
Current = Next
Update BestSolutionSeen
H = Recency(H + Current)
Endif
End-While
Return BestSolutionSeen
Elements of Tabu Search
Memory related - recency (How recent the
solution has been reached)
• Tabu List (short term memory): to record a
limited number of attributes of solutions
(moves, selections, assignments, etc) to be
discouraged in order to prevent revisiting a
visited solution;
• Tabu tenure (length of tabu list): number of
iterations a tabu move is considered to remain
tabu;
Elements of Tabu Search
Memory related – recency (How recent the
solution has been reached
• Tabu tenure
-List of moves does not grow forever
restrict the search too much
-Restrict the size of list
-FIFO
-Other ways: dynamic
Elements of Tabu Search
Memory related – frequency
• Long term memory: to record attributes of
elite solutions to be used in:
– Diversification: Discouraging attributes of elite
solutions in selection functions in order to
diversify the search to other areas of solution
space.
– Intensification: giving priority to attributes of a set
of elite solutions (usually in weighted probability
manner)
Elements of Tabu Search
• If a move is good, but it’s tabu-ed, do we still
reject it?
• Aspiration criteria: accepting an improving
solution even if generated by a tabu move
– Similar to SA in always accepting improving
solutions, but accepting non-improving ones
when there is no improving solution in the
neighbourhood;
Example TSP using Tabu Search
Find the list of towns to be visited so that the
travelling salesman will have the shortest route
• Short term memory:
– Maintain a list of t towns and prevent them from
being selected for consideration of moves for a
number of iterations;
– After a number of iterations, release those towns
by FIFO
Example TSP using Tabu Search
• Long term memory:
– Maintain a list of t towns which have been
considered in the last k best (worst) solution
– encourage (or discourage) their selections in
future solutions
– using their frequency of appearance in the set of
elite solutions and the quality of solutions which
they have appeared in our selection function
Example TSP using Tabu Search
• Aspiration:
– If the next moves consider those moves in the
tabu list but generate better solution than the
current one
– Accept that solution anyway
– Put it into tabu list
Tabu Search (Pros & Cons)
Pros
• Generated generally good solutions for
optimisation problems compared with other
AI methods
Cons
• Tabu list construction is problem specific
• No guarantee of global optimal solutions
Practical considerations
Tabu tenure: the length of time t for which a move is
forbiden
t too small - risk of cycling
t too large - may restrict the search too much
If a tabu move is smaller than the aspiration level then
we accept the move
7
Basic Concepts
Tabu-lists contains moves which have been made in the recent past but are
forbidden for a certain number of iterations.
Algorithm
Step 1. k=1
Select an initial solution S1 using some heuristic and set Sbest = S1
Step 2.
Select ScÎN(Sk)
If the move Sk ® Sc is prohibited by a move on the tabu-list then go to Step 2
3
If the move Sk ® Sc is not prohibited by a move on the tabu-list
then
Sk+1 = Sc
Enter reverse move at the top of the tabu-list
Push all other entries in the tabu-list one position down
Delete the entry at the bottom of the tabu-list
If F(Sc) < F(Sbest) then Sbest = Sc
Go to Step 3.
Step 3.
k = k+1 ;
If stopping condition = true then STOP else go to Step 2
Example.
TSP
Neighbourhood: all paths that can be obtained through adjacent pairwise
interchanges.
Tabu-list: pairs of cities (j, k) that were swapped within the last two moves
S1 = 2, 1, 4, 3
First iteration:
F(S1) = 500 = F(Sbest)
F(1, 2, 4, 3) = 480
F(2, 4, 1, 3) = 436 = F(Sbest)
F(2, 1, 3, 4) = 652
Tabu-list: { (1, 4) }
Second iteration:
S2 = 2, 4, 1, 3 F(S2) = 436
F(4, 2, 1, 3) = 460
F(2, 1, 4, 3) (= 500) tabu!
F(2, 4, 3, 1) = 608
Tabu-list: { (2, 4), (1, 4) }
Third iteration
S3 = 4, 2, 1, 3 F(S3) = 460
F(2, 4, 1, 3) (= 436) tabu!
F(4, 1, 2, 3) = 440
F(4, 2, 3, 1) = 632
Tabu-list: { (2, 1), (2, 4) }
Fourth iteration:
S4 = 4, 1, 2, 3 F(S4) = 440
F(1, 4, 2, 3) = 408 = F(Sbest)
F(4, 2, 1, 3) (= 460) tabu!
F(4, 1, 3, 2) = 586
Tabu-list: { (4, 1), (2, 1) }
F(Sbest)= 408
Tabu Search Algorithm
600
500
400
300
200
100
0
0 1 2 3 4
Sol Best
References
• Glover, F. 1989. Tabu Search – Part I. ORSA Journal on
Computing, Vol. 1, No. 3, pp 190-206.
• Glover, F. 1990. Tabu Search – Part II. ORSA Journal on
Computing, Vol. 2, No. 1, pp 4-32.
• Glover, F., Laguna, M. 1998. Tabu Search. Kluwer Academic
Publishers
• Rayward-Smith, V.J., Osman, I.H., Reeves, C.R., Smith, G.D.
1996. Modern Heuristic Search Methods, John Wiley & Sons.
• Russell, S., Norvig, P. 1995. Artificial Intelligence A Modern
Approach. Prentice-Hall
End of Tabu Search
