In which we examine the problems that arise when we Iry 10, plan ahead ina w where other agents are planning against us. i = a dow wie _In multiagent environments, fh agent needs to consider the actions of 0 how they affect its own welfageJ The unpredictability of these other agents contingencies into'the agent’s problem-solving process. In this chapter we cove! scvironments, in which the agents’ goals are in conflict giving rise to adv owe problems—often known as games. : Mathematical/game theory, 2 branch of economics, views any multiags as a game, provided that the impact of each agent on,thé others is “signifi of whether the agents are ive or competitixe.f In Al, the most c of a rather specialized kind—what game theorists call deterministic, tur-takt zero-sum games of perfect information (such 2 chess). In our term deterministic, fully observable environmehts in ‘which two agents act a the utility values atthe end of the game are always. equal and 9p1 player wins a game of chess, the other player necessarily los the agents’ utility functions that makes the situation advers: ‘Games have engaged the intellectual faculties of humnans—someti degree—for as long as civilization has existed. For AI researchers, the games makes them an appealing subject for study. The state of a game is and agents are usually restricted to a small number of actions whose outcon precise rules. Physical games, such as croquet and ice hockey, have much descriptions, a much larger range of possible actions, and ral x We aezal of actions, With the exception of robot soccer, these phy: such interest in the Al community. i than ae ZO SM GAMES parect ‘NATION 1 Bovis % Environments with very many agents are often viewed as economies rather: 5 of Anif 2010 ia Ineligence: A Moder Approach, Third Eaton: Shit Russell and lucation, Inc, Published by Prentice Hall. All rights reserved.Adversarial Search Games, unlike many toy problems that you may have studied, are interesting because they are too hard to solve. For example, chess has an average branching factor of about 35, or 10!54 and games often go to 50 moves by each player, so the search tree has about 35100 nodes (although the search graph has “only” about 10" distinct nodes). Games, like the real world, therefore require the ability to make some decision even when calculating the optimal decision is infeasible. Games also penalize inefficiency severely. Whereas an implementation of A* search that is half as effjcient will simply take twice as long to run to completion, a chess program that is half as effcient in using its available time probably will be beaten into the ground, other things being equal. Game-playing research has therefore spawned a num- ber of interesting ideas on how to make the best possible use of time. We begin with a definition of the optimal move and an algorithm for finding it. We then look at techniques for choosing a good move when time is limited. Prontag allows us to ignore portions of the search tree that make no difference to the figal choice, and heuristic n functions allow us t esa fie uly ofa sate wind don Ximate the true utility of a state without doing a com- plete search} Section 5 discusses games such as backgammon that include an element of GHaMerWE'also discuss bridge, which includes elements of imperfect information because not all cards are visible to each player. Finally, we look at how state-of-the-art game-playing programs fare against human opposition and at directions for future developments. [We first consider games with two players, whom we call MAX and MIN for reasons that will soon become obvious. MAX moves first, and then they take tums moving until the game is qver. At the end of the game, points are awarded to the winning player and penalties are iven fo the loser. A game’can be formally defined as a kind of search problem with the following elements: * ‘© PLAYER(s): Defines which player has the move in a state. +b, 0, ‘y-AcTIONS(s): Retums the set of legal moves in a state. ® AESULT(s,a): The transition model, which defines the result of a move, ScrermiNaL-Test(s): A terminal test, which is true when the game is over and false otherwise, States where the game has ended are called terminal states. ~ : <«/Ur1Ty (6,7): 8 ulilty funetion (also called 20 objective function or payoff function), “defines the final numeric value for a game that ends in terminal stafe s for a player p. In chess, the outcome is awin, loss, or draw, with values +1.0, oF 2\Some games have a “les variety of possible ouicomes; the payofis in backgammon range from 0 10 +192. [A zero-sum game is (confusingly) defined as one where the total payoff to all players is the same for . Chess is zero-sum because every game has payoff of either 0 +1, 1+ 0 or 4 +}. }:Constant-sum” would have been a better term, bur zero-sum is traditional and rtiakes sense if you imagine each player is charged an s entry fee of 3. a (The initial state, ACTIONS function, and Resupt function define the game tree for the ame a tee where the nodes are game states and the ellges arS-mOveS: Fue hos i se (noughis . From tlfe initial state, MAX has art of tree for tic-tac-toc (noughis and crosses). : M nine pe Play alternates between MAX’s placing an X and MIN’s placing an o 165 of A ek «So; The initial state, which specifies how the game is set up at the start. at 2 6SEARONTREE Adversarial Search ‘until we reach leaf nodes corresponding to terminal states such that one player a row or all the squares are filled. The number on each leaf node indicates the of the terminal state from the point of view of MAX; high values are assumed to MAX and bad for MIN (which is how the players get their names). { For tic-tac-toe the game tree is relatively small—fewer than 9! = 362, nodes. But for chess there are over 10! nodes, so the game tree is best f theoretical construct that we cannot realize in the physical world. But regardle of the game tree, it is MAX’s job to search fora good move. We use the term seat r tree that is superimposed on the full game tree, and examines enough nodes to to determine what move to make. ‘ MAX @) | MIN(o) [% fs tae ; xor] Po] MAX (x) i] XOX] POL] fol | MIN (0) Xx xX xox] (XJORx) [xolx TERMINAL [-Jolx] [olobx) [1x fol} [xlxlo} [xlolo Utility -1 00° 41 Figure 1 A (partial) game tree for the game of tic-tac-toe. The top state, and MAX moves first, placing an x in an empty square. We show part of: alternating moves by MIN (0) and MAX (X), until we eventually reach i can be assigned utilities according to the rules of the game. _OPTIMAL DECISIONS IN GAMES i : In a normal search problem| the optimal splution would be a sequence of goal state—a terminal state that is a win) In adversarial search, MIN about it. MAX therefore must find a contingent strategy, which specifi the initial state, then MAX’s moyes in the states resulting from every 166 iAdversarial Search“ Figure 2 A two-ply, iurn to move, and the V nodes are “MIN nodes.” The terminal nodes show the utility values for MAX; the other ‘nodes are labeled with their minimax values. MAX’s best move at the root is a1, because it leads to the state with the highest minimax value, and MIN’s best reply is bs, because it leads to the state with the lowest imax value. \uts, then MAX’s moves in the states resulting from every possible response by MIN to those moves, and so on. This is exactly analogous to the AND-OR search algorithm with MAX slaying the role of OR and MIN equivalent to AND. Roughly speaking, an optimal strategy eads to outcomes at least as good as any other strategy when one is playing an infallible opponent. We begin by showing how to find this optimal strategy. Even a simple game like tic-tac-toe is too complex for us to draw the entire game tree con one page, so we will switch to the trivial game in Figure 2. The possible moves for MAX at the root node are labeled a1, ag, and a3. The possible replies to a1 for MIN are bi, b2, and so on. This particular game ends after one move each by MAX and MIN. (In game parlance, we say that this tree is one move deep, consisting of two half-moves, each of which : 's called a ply.) The utilities of the terminal states in this game range from 2 to 14. - [Given a game tree, the optimal strategy can be determined from the minimax value of each node, which we write as MiNJMAX(n). The minimax value of a node is the uulity (for MAX) of being in the corresponding state, assuming that both players play optimally from there to the end of the game, Obviously, the minimax value of a terminal state is just rthe ven a choice, MAX prefers to move to a state of maximum value, its wility, Furthermore, gi | whereas MIN prefers a state of minimum value. So we have the following: MINIMAX(s)= UTILITY(s) if TERMINAL-TEST(s) { maxa¢ Actiona(a) MINIMAX(RESULT(s,a)) if PLAYER(s) =MAX - mine Actiona(s) MINIMAX' (REsULT(s,a)) if PLAYER(s) = MIN Let us apply these definitions to the game tree in Figure 2. The terminal nodes on the bottom level get their utility values from the game's UTILITY function. The first MIN node, labeled | 8 has three successor states with values 3, 12, and 8, so its minimax value is 3. Similarly, the other two MIN nodes have minimax value 2. The root node is & MAX node; its successor states have minimax values 3, 2, and 2; so it has a minimax valiie of 3, We can also identify 3 167 ~ >,Adversarial Search the minimax decision at the ro ion a; is the optimal choice for MAX becat the state with the highest minimax value. This definition of optimal play for MAX assumes that MIN also plays maximizes the worst-caye outcome for MAX. What if MIN does not play optimall easy to show (Exercise 7) that MAX will do even better. Other strategies against ‘opponents may do better than the minimax strategy, but these strategies necess against optimal opponents. 2.1 The minimax algorithm ‘wemuxaconmy The minimax algorithm (Figure 3) computes the minimax decision from the et “Tcuses a simple recursive computation of the minimax values of each successor sta implementing the defining equations. The recursion proceeds all the way down of the tree, and then the minimax values are backed up through the tree as th unwinds. For example, in Figure 2, the algorithm first recurses down to the th left nodes and uses the UTILITY function on them to discover that their values an 8, respectively. Then it takes the minimum of these values, 3, and returns it as up value of node B. A similar process gives the backed-up values of 2 for @ Finally, we take the maximum of 3, 2, and 2 to get the backed-up value of 3 fort The minimax algorithm performs a complete depth-first exploration of th If the maximum depth of the tree is m and there are b legal moves at each ume complexity of the minimax algorithm is O(b”), The space complexity is algorithm that generates all actions at once, or O(m) for an algorithm that gene ee ime. For real games, of course, the time cost is totally impractical, but dl sepés as the basis for the mathematical analysis of games and for more prac /2.2 Optimal decisions in multiplayer games Many popular games allow more than two pivedd Let us examine how to ext idea to multiplayer games. This is straightforward from the technical vi some interesting new conceptual issues, First, we need to replace the single value for each node with a example inathree-player game with players A, B, and Ca vector (v4, 0B, Uo with each node. For terminal states, this vector gives the utility of the state fron Viewpoint. (In two-player, zero-sum games, the two-element vector can be value because the values are alwa: tree shown in Figure 4. In that state, player C' chooses what to do, The fo terminal states with utility yectors (va=1,up= Since 6 is bigger than 3, C' should choose Subsequent play will Jead to a terminal state with utilities (v4 =1,up =2 "the backed-up value of X is this vector. The backed-up value of a node n 168