See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/312869797

Behavior Trees for Computer Games

Article in International Journal of Artificial Intelligence Tools · January 2017

DOI: 10.1142/S0218213017300010

CITATIONS READS

32 9,444

1 author:

Yoones A. Sekhavat
Tabriz Islamic Arts University
65 PUBLICATIONS 700 CITATIONS

SEE PROFILE

All content following this page was uploaded by Yoones A. Sekhavat on 28 June 2021.

The user has requested enhancement of the downloaded file.

October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

International Journal on Artificial Intelligence Tools

c World Scientific Publishing Company

Behavior Trees for Computer Games

Yoones A. sekhavat∗
Faculty of Multimedia, Tabriz Islamic Art University, Hakim Nezami Square, Azadi Blvd,
Tabriz, 51647-36931, Iran
[email protected]

Received (Day Month Year)

Revised (Day Month Year)
Accepted (Day Month Year)

Although a Finite State Machine (FSM) is easy to implement the behaviors of None-
Player Characters (NPC) in computer games, it is difficult to maintain and control the
behaviors with increasing the number of states. Alternatively, Behavior Tree (BT), which
is a tree of hierarchical nodes to control the flow of decision making, is widely used in
computer games to address the scalability issues. This paper reviews the structure and
semantics of BTs in computer games. Different techniques to automatically learn and
build BTs as well as strengths and weaknesses of these techniques are discussed. This
paper provides a taxonomy of BT features and shows to what extent these features are
taken into account in computer games. Finally, the paper shows how BTs are used in
practice in the gaming industry.

Keywords: Behavior tree; computer games; decision making; non-player character

1. Introduction
Activities associated with Artificial Intelligence (AI) in computer games are de-
signed for behaviors of Non-Player Characters (NPC). Defining a logic for various
entity characters in computer games is an important issue in the design of a game
that specifies how these characters behave and react against different conditions.
Providing an easy to use AI component that is general enough to provide a game
designer the ability to specify complex behaviors is crucial.
Rule-based systems are the simplest and fairly limited to control the behavior, in
which the behavior of a NPC is a simple function of present conditions. Generally, a
rule-based system consists of a knowledge-base and a set of if-then rules available to
an AI agent. Rules are examined against knowledge-base to check if conditions are
met. Rules are triggered and executed when the conditions are met.1 For example,

∗ Yoones A. Sekhavat is an assistant professor in faculty of Multimedia, Tabriz Islamic Art Univer-
sity. He holds a PhD in Computer Science from Memorial University of Newfoundland, Canada. He
also holds a post-doc from University of Alberta in Canada. He is the head of cognitive augmented
reality lab (www.carlab.ir).

1
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

2 Yoones A. Sekhavat

the motion of a ghost in Pac-Mana game can be modeled by a rule-based system, in

which the agent goes ahead whenever possible. When the way is blocked, the agent
selects actions turning right, turning left, and reversing. In backward chaining rule-
based systems, the agent starts with a goal in the knowledge-base and tries to fire
a set of rules such that these rules lead to that goal. Although this system is easy
to understand and implement, it is almost impossible to model complex behaviors
using rule-based systems.
Finite State Machines (FSM) by adding a notion of state make it possible to
handle the complexity of behaviors. FSMs are generally used to represent the be-
haviors of a NPC in terms of a set of states and transitions between states. A FSM
is a directed graph in which each node represents a state. A NPC can only be in one
of the states. An edge in this graph corresponds to an event that triggers a change
from one state to another state. Unlike rule-based systems, the behavior of an NPC
is a function of the current state of the NPC as well as the present conditions. Scal-
ability is the main problem of FSM. Today’s games are complex games that require
a large number of states. Although this problem can be partially mitigated using
a hierarchical FSM (a.k.a., HFSM), it is still difficult to manage a large number of
states.2 Efforts to use sophisticated decision making techniques such as fuzzy logic
and neural networks have not resulted in considerable successes as these techniques
are hard to get them working right.1
Behavior Tree (BT) is a practical scalable solution to implement decision mak-
ing, which is a hierarchical structure of behaviors. An agent running a behavior
tree performs a depth-first search on the tree to find and run the lowest level leaf
node. Scalability, expressiveness and extensibility are the main advantages of BT
in comparison to FSM.3 Unlike explicit transitions from one state to another (like
FSM), each node of a tree indicates how to run its children. In a FSM, every state
must know the transition criteria for every other state, while BTs are stateless. As a
result, there is no need to know previously running nodes to decide what behaviors
should be executed next. The expressiveness of BTs come from the use of different
levels of abstraction, implicit transitions and arbitrarily complex control structures
for composite nodes. In FSM, once a switch from one state to another state is con-
ducted, since the source state is not saved, it is a one-way control transfer. On the
other hand, in BTs, transitions between states are achieved through calls and return
values passed between tree nodes, which is known as a two-way control transfer.
BTs are extensible as it is possible to start from a base algorithm and add extra
functionalities. Making changes to a behavior trees is much easier than FSM. First,
since creating and editing behavior trees are performed using a visual editor, main-
taining a behavior tree is simply conducted by drag and drop of nodes. Such visual
tools make BTs much more accessible and understandable to non-programmers.
However, adding a new node or changing a state in a FSM requires changing the
transitions between states, that is not easy with growing the number of states.

a http://pacman.com
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 3

Second, unlike FSM that requires considering all states and transitions to make a
change in the state of an agent, in a BT, this is performed just by modifying the
tasks around or adding a new parent. Moreover, BT is more flexible than FSM.
The only way to run two different states with FSM is creating two separate FSMs.
However, simultaneous execution of two behaviors can be easily handled in behav-
ior trees using parallel nodes. An AI designer may also want to prevent to run two
tasks at the same time. For example, suppose you have two different tasks that
play a sound effect and they could potentially play the sound effect at the same
time. There is no way in FSM to prevent the simultaneous running of these two
sound effects. However, adding guard nodes to behavior trees (a semaphore task
that allows only one active instance) makes it possible to play only one sound effect
at a time.
This paper reviews various issues regarding BTs in computer games. As shown
in Table 1, this paper classifies the study of BTs in four basics categories includ-
ing Basics of BTs, Forming BTs, BT Extension and BT Application. In the basics
of BTs, formal definitions and semantics of behavior trees in computer games from
different perspectives are discussed. In the context of forming behavior trees, auto-
matic and manual techniques to facilitate the forming of behavior trees are reviewed
for NPCs in computer games. In terms of BT extension, techniques to modify the
structure of behavior trees as well as methods to augment behavior trees with ad-
ditional knowledge to increase the expressiveness of behavior trees are discussed. A
list of features are proposed by which the strengths and weaknesses of these tech-
niques are compared. Finally, This paper shows show how BTs are used in practice
to implement decision making in research prototypes and commercial games.
Algorithms designed for sequential decision making problems with multiple goals
are discussed in several surveys4 including probabilistic inference techniques for
scalable multi-agent decision making,5 and decision making with dynamic uncertain
events6 . However, none of these techniques takes into account the specific aspects
of decision making for NPCs in computer games.

2. Behavior Tree Basics

BTs make it possible to control characters in computer games and to define hi-
erarchies of decisions and actions.7 Behavior trees are also crucial to implement
believable agents8 and to control groups of characters in an interactive way.9 This
section discusses the structure and the semantics of behavior trees. Then, a list of
features is propped to compare different BTs.

2.1. Behavior Tree Structure

As a classic definition10 , a Behavior Tree (BT) is a directed tree including a set of
nodes and edges. The root of a BT is a node without parents. On the other hand,
nodes without children are the leaves of this tree. In a basic BT, a non-leaf node
can be a selector node or a sequence node. Selectors are used when we aim to find
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

4 Yoones A. Sekhavat

Table 1: Classification of research issues for behavior trees in computer games in-
cluding supporting papers in each category.
Basics of BTs Formal Representation Behavior Tree Semantics
of BTs
[1], [2], [3], [4], [7], [8], [9], [13], [14], [15], [16], [17], [18],
[10], [11] [19]
Forming BTs Automatic Techniques Manual Techniques
Machine Learning: [20], [21], Facilitating tools: [36], [37]
[22], [23], [24], [25], [26]
Case-Based Reasoning: [27],
[28], [29]
BT Extension Modification in the Augmenting BTs with
structure of BTs Additional Knowledge
[38], [39], [40], [41], [42], [43], [47], [48], [49], [58], [51], [57],
[54], [55], [56], [44], [45], [50], [59], [60], [61], [28], [78], [79]
[52], [53], [80], [3], [46]
BT Application [29], [62, [63], [3], [64], [65], [66], [68], [69], [21], [70], [59],
[71], [10], [72], [73], [74], [75], [14], [76], [77]

and execute the first possible child that can run without failure. A selector node
succeeds once any child is performed successfully. In a sequence node, all nodes are
evaluated sequentially. A sequence node succeeds only if all children are performed
successfully. On the other hand, a leaf node can be an action or a condition. Action
nodes include playing an animation, changing the state of a character, or any activity
that changes the state of the game. On the other hand, a condition node is generally
used to test some values. Tests for proximity, testing the state of a character and
testing the line of sight are examples of condition nodes. A condition returns success
if the condition is met; otherwise, returns failure.
An example of a behavior tree including a selector node (i.e., root represented by
?), two sequence node (i.e., two children of the root represented by →), a condition
node (Is door open?) and 4 action nodes (move into room, move to the door, open
door and move into room) are shown in Fig. 1. Using the depth-first search to run
the behavior tree in this figure, the first node selected to run is “is door open?” if
the door is open, it returns success. Then, the sequence task moves on to its next
child, which is “move into room”. If this action succeeds, the whole tree returns
a success and execution is done. In the case if the door is closed, or “move into
room” returns failure, the second sequence node is tried staring from the “move
into door”. The tree will return success if all children of the second sequence node
return success.
In addition to the core nodes, behavior trees are also extended with decorator
and parallel nodes. Adapted from the “Decorator” design pattern in software en-
gineering, a decorator node in behavior tree is a node that has a single child task.
A decorator node can be used for filtering which makes a decision to allow a child
behavior to run “until fail”. A decorator node may also be used to limit the number
of runs. On the other hand, parallel nodes provide some sort of concurrency in BTs.
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 5

→ →
Is door Move into Move into Open Move into
Open? room door door room

Fig. 1. A behavior tree including selector, sequence, condition and action nodes

A parallel node is used when there are actions that must be run concurrently with
other actions (e.g., playing a move animation while actually moving from one place
to another). Some commercial products such as RAINb are also extended with Pri-
ority node (that allows setting priority both in run time and start time) and Custom
Action node (that allows defining a decision specific to a game).11 A node in a be-
havior tree can be as simple as checking the value of a variable, or as complex as
executing an animation. Arbitrary conditions and actions can be combined together
to form a behavior tree.
Sharing data between tree nodes is an important feature of BTs that makes
it possible to handle complex behaviors. In the case of the lack of data sharing
between nodes, we may form a big behavior tree with separate branches for each
option, which can be wasteful. The best approach to implement this option is by
decoupling the tasks from the data that behaviors need. This can be achieved by
employing an external repository to save all data that a behavior tree needs, which
is called “blackboard”. A blackboard can store any data that can be queried by any
task node.
Once a behavior tree is executed, the root of the tree is ticked in each time step of
the control that results in propagating down the tree. When this tick reaches a leaf
node, some states or variables of the BT are affected. Once a node representing a
task is executed, the return value is success, fail or running. The root node branches
down to some nodes until leaf nodes are reached.

2.2. Formal Model of Behavior Trees

Formally, a BT is defined as a triple Ti = {fi , ri , ∆t}, in which i the index of the
tree, fi :Rn → Rn is the right hand side of an ordinary difference equation, ∆t is a
time step and ri : Rn → {Running, Success, F ailure} is the return status.3 The
execution of a behavior tree Ti is defined as a standard difference equation:

b http://rivaltheory.com
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

6 Yoones A. Sekhavat

Algorithm 1: An implementation of Selector node

for i = 1 to N do
state = Tick(child(i));
if state == running then
return running
if state == success then
return success
end
return failure;

Algorithm 2: An implementation of Sequence node

for i = 1 to N do
state = Tick(child(i));
if state == running then
return running
if state == failure then
return failure
end
return success;

xk+t (tk+1 ) = fi (xk (tk )), where tk+1 =tk + ∆t.

In this formula, a condition is a node that does not return running state. An
action behavior in this formula is a behavior tree Ti that has no subtree. A sequence
operator allows combining two or more BTs in order to form a complex BT, which
is formally defined as T0 = Sequence(T1 ; T2 ). To run T0 , the system first tries to run
T1 . The second child runs if T1 returns success. The sequence T0 returns Success if
all children are succeeded. A selector operator, denoted T0 = Selector(T1 ; T2 ), is a
composition of two BTs such that once T0 is executed, the system first tries to run
child T1 as long as it returns running or success. T2 is executed only if T1 fails. T0
returns failure if T1 and T2 are failed. The node types Sequence, Selector, Action
and Condition behave according to Algorithm 1, Algorithm 2, Algorithm 3, and
Algorithm 4, respectively, where T ick(child(i)) represent triggering the algorithm,
and N is the number of children of that node.
In original sequence node of behavior tree algorithms, for an arbitrary child
Action Aj that is ticked at time tk , it is necessary that {X1 (tk ) ∈ S1 ∧ · · · ∧
Xj−1 (tk )Sj−1 }. Accordingly, in a selector node, for an arbitrary child Action Aj
that is ticked at time tk , it is necessary that {X1 (tk ) ∈ F1 ∧ · · · ∧ Xj−1 (tk ) ∈
Fj−1 }. Although this may be useful in many cases, there are situations in which
we need to remember which nodes have already returned success or failure. This
is used to not check them (and tick them accordingly) again on the next iteration.
Instead of storing success or failure statuses of nodes, a variable representing the
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 7

Algorithm 3: An implementation of Action node

if Xn (t) ∈ Success then
return Success
if Xn (t) ∈ Failure then
return Failure
if Xn (t) ∈ Running then
Un (t) = γn(Xn (t))
return Running

Algorithm 4: An implementation of Condition node

if Xn (t) ∈ Success then
return Success
if Xn (t) ∈ Failure then
return Failure

child that has most recently returned running can be employed.12 Every time the
selector or sequence returns success or failure, this variable is initialized. Algorithm
5 and Algorithm 6 show different implementations of sequence and selector nodes,
respectively, using this modification.

Algorithm 5: A modified implementation of Selector

for i = runIndex to N do
state = Tick(child(i));
if state == running then
runIndex = i;
return running
if state == success then
runIndex = 1;
return success
end
runIndex = 1;
return failure;

Behavior trees allow abstraction, where at design time, an AI designer creates

basic behavior trees representing simple behaviors. Such basic behaviors can be spe-
cializations of more general behaviors developed for one or more NPCs. To provide
re-usability in the design of a tree, behaviors can be parametrized (hard-coded at
design time or dynamically at run time). For example, in the “Take” behavior, the
taken object can be specified as a parameter to provide re-usability.60 Since behav-
ior trees represent the behavior of NPCs, an execution context is created for trees at
run time. This context includes a set of variables consisting attributes of the game
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

8 Yoones A. Sekhavat

Algorithm 6: A modified implementation of Sequence

for i = runIndex to N do
state = Tick(child(i));
if state == running then
runIndex = i; return running
if state == failure then
runIndex = 1;
return failure
end
runIndex = 1; return success;

state that can be accessed by NPCs. Different actions of an agent can change these
variables, representing the need for having a separate context for each tree assigned
to an agent.

2.3. Behavior Tree Semantics

Different semantics and definitions have been used for behavior trees in computer
games including the semantics of behavior trees, in which a tree is translated
into Communicating Sequential Processes (CSP).13 However, in the case of com-
plex states, it becomes very complicated. In formal semantics for behavior trees,14
CSP 15,16 is used as a formal notation to represent interaction between concurrent
processes. This new language, which is called CSPσ is extended with state-based
constructs and a message passing facility similar to publish/subscribe models of
communication.17 The semantics in this formal notation is used to develop an au-
tomated analysis of system behavior, simulation and model checking. In a formal
definition for translating complex requirements to behavior trees,14 a formal seman-
tics for BTs using CSP is proposed, in which state-based constructs are employed.
This technique improves the precision of behavior tree models and results in a better
understanding of modeling decisions, while resolving ambiguities and inconsisten-
cies. This technique can also be used in the automated analysis of system behaviors.
In a technique to transform a basic behavior tree model to UML state
machines,18 a framework is proposed that makes it possible to use a rich variety of
relations that capture non-functional aspects of requirements as extra annotations to
nodes in a behavior tree. This technique provides a tool to go from natural-language
system requirements to model driven engineering. In A behavior tree notation to
improve the modeling and understanding requirements18 , rules are stated in natural
language. In a process algebra for capturing the constructs of behavior trees14 , due
to the complexity of operational semantics, desired properties such as the compo-
sition of parallel behavior trees are not considered. In order to provide a meaning
to behavior trees, a language called Behavior Tree Process Algebra (BTPA)19 is
proposed that provides operational semantics, while allowing to define a mechan-
ical translation of behavior trees into BTPA. This allows providing a meaning for
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 9

synchronization and message passing. However, this architecture provides a small

set of primitives from which complex behaviors can be simulated.

2.4. Feature Comparison

In eleven features for behavior trees12 , only part of them are supported in different
behavior trees. These features can be used to evaluate different behavior trees.12
Although various behavior tree models are proposed and implemented in the con-
text of computer games, different features of these trees are not generalized to be
exploited in other research works. NodeCanvas compared the features of major be-
havior trees based on 22 featuresc . These features are mostly from implementation
view and user interface design. Based on the structure of behavior trees and con-
sidering the behavior tree features12 and NodeCanvas, this paper came up with a
comprehensive list of BT features that can be used to evaluate and compare existing
works. A summarized list of BT features as well as supporting works are shown in
Table 2.
As shown in Table 2, features such as automatic generation of BTs, reusability,
runtime support of BTs, flexible variables and multi-agent behavior trees are widely
supported features in computer games. These are the basic features that must be
supported and taken into account to develop new behavior tree components. On
the other hand, supporting persistent states, behavior tagging, hinted execution
BTs, supporting social and emotional factors, visual editing and debugging, and
supporting component reasoner are features that are rarely considered in behavior
trees for computer games. The rest of features including supporting utility-based
AI, enhanced states, flexible nodes and parametrization are complementary features
that enhance the application of behavior trees. This classification not only provides
a benchmark to compare behavior tree components, but also provides a road map to
develop new behavior tree components for future research prototypes and computer
game industry.

3. Learning and Forming Behavior Trees

This section reviews techniques that have been proposed to form behavior trees for
computer games. A taxonomy of techniques to form behavior trees including the
main features, strengths and the weaknesses of them are shown in Table 3.

3.1. Automatic Generation of BTs

An efficient design of behavior trees is an important issue in computer games, which
is generally performed by experienced experts. There is little research into auto-
mated manipulation or improvement of an initial behavior tree implementation.
This section reviews automatic techniques to generate BTs.

c http://nodecanvas.com/comparison
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

10 Yoones A. Sekhavat

Table 2: A summarized list of features of behavior trees extracted in this paper as

well as supporting works.

Feature Details Supporting works

Automatic generation Automatic techniques [20], [21], [22], [23], [24], [25],
[26], [27], [28], [29]
Support utility based AI Utility based BTs [38], [39], [40], [41], [42], [43],
[54], [55], [56]
Enhanced states Supporting internal states [47], [48], [49], [58]
Persistent states [58]
Behavior Tagging Behavior Tagging [57]
Hinted Execution BTs [59]
Social and Emotional Social Territoriality [46]
Emotional Factor [51]
Re-usability Modularity [54],[58]
Planning integration [57], [6], [16]
Bt library [40], [41], [42], [15], [16], [64],
[52], [52] [57], [6]
Runtime support Non-blocking actions [44],[72], [55], [37], [16], [30],
[4], [6]
Dynamic tree [7], [16], [35], [21], [65], [4],
[15], [6], [30]
Infinite execution [44], [6], [72], [16], [30]
Live editing [7], [20], [21], [16], [22], [26],
[44], [45]
Dynamic variables [5], [6], [7]
Flexible Nodes Node with several parents [44],[6]
Component reasoner [47], [19]
Timed behavior nodes [37]
Flexible Variables Global variables [57], [15], [44], [6], [72], [37],
[74], [16], [30], [4], [6]
Multiple Agents Multiple agents [57], [6], [72], [55], [37], [74],
[30], [6]
Enhanced Editor Visual Editor [29], [30]
Visual debugging [2], [23]
Parametrization Agent-centric parametriza- [7], [2], [44], [51]
tion

Genetic Algorithms. There have been efforts to automatically design behavior

trees using evolutionary algorithms. Behavior trees of DEFCON game20 are gen-
erated based on an initial population of trees, where genetic operators are used to
produce improved trees. Starting with some manually created trees, this technique
first encodes possible behaviors for different parts. Then, different genetic program-
mings run for each part, that consequently result in generating new behaviors from
the original set. In this technique, genetic algorithms is used to evolve BTs in an
exploratory process, in which a fitness function is used for evaluating evolved be-
havior trees. In using genetic programming to dynamically evolve behavior trees21
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 11

Table 3: A taxonomy of techniques to form behavior trees including the main fea-
tures, strengths (+) and the weaknesses (-) of these techniques.
Genetic Algorithms + Automatically evolving BTs
+ Different genetic algorithm can be used to each part
- Initial set of BTs are required
- Genetic Algorithms does not necessarily result in better
behavior tree in all cases
CBR + Allows to extract behaviors from a knowledgebase
+ Nodes are enhanced with querying functionality
+ behaviors are considered as patterns that can be reused
- Performance of reasoning is the main problem of this tech-
nique
Q-Learning + Makes it possible to decide when is the right time to exe-
cute AI logic
+ allows debugging BTs
+ Allows to optimize behavior trees
+ This technique cannot be used to make plans
- This technique suffers from performance issues
Learning Examples + Quickly reacts to changes at runtime
+ The generated BT summarizes a large amount of expert
knowledge
+ Employs hierarchical Machin Learning to model player
behavior
+ The tree can make nondeterministic decisions based on
samples
- The initial set of behaviors significantly changes the final
tree
BT Generation - This is still the game programmer who should implement
programming classes
- The scalability as well as the complexity of communications
between the game world and the behavior tree authoring tool
are unanswered questions

(which is used in Super Mario), the syntax of possible solution is determined by

a context-free grammar.21 In a modified version of this approach,22 an AND-OR
tree structure is used to different layers of selector and sequence nodes. In learning
domain-specific planners from example plans,23 domain-specific planners are used to
solve specific planning problems. In spite of promising results, Genetic Algorithms
do not necessarily result in better behavior tree in all cases.21,22,23
In Grammatical Evolution,24 which is a grammar-based form of genetic pro-
gramming, the syntax of a possible solution is employed to map binary strings to
syntactically correct solutions. In the genotype-to-phenotype mapping in grammat-
ical evolution, variable-length strings are evolved using genetic algorithms that are
used to select production rules from a grammar. After evaluating the program, its
fitness is used by the evolutionary algorithm. An example of an Illustrative grammar
for a simple approach to a generic shooting game21 is shown in Fig. 2.
Meta-heuristic evolutionary learning algorithms are better than grammatical
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

12 Yoones A. Sekhavat

<BT> ::= <BT> <Node> | <Node>

<Node> ::= <Condition> | <Action>
<Condition>::= if(obstacleAhead) then <Action>;
| if(enemyAhead) then <Action>;
<Action> ::= moveLeft; | moveRight; | jump; | shoot;

Fig. 2. An Illustrative grammar for simple approach to a generic shooting game21 .

evolution since they provide a natural way of manipulating BTs and applying ge-
netic operators.25 They tested their technique on Mario AI benchmark to simulated
autonomous behavior of the characters. BT crafting algorithm can be initialized
with a BT that is created manually26 , where behaviors can be used to control flying
robots, while the BT is evolved in every experiment.
As an example of a mapping process given the grammar in Fig. 2, and the
string (4, 5, 3, 6, 8, 5, 9, 1), the final program will be moveRight; if (enemyAhead),
then shoot. More specifically, the first value (4) is used to select from the start
symbol hBT i based on the formula 4%2 = 0. Thus, the first expression, which is
hBT i hN odei is selected. Accordingly, using 5%2 = 1, hBT i is replaced by hN odei,
that consequently results in hN odeihN odei. In the same way, hActionihN odei is
generated from 3%2 = 1, and moveRight; hN odei is generated from 5%2 = 1.
Case Based Reasoning. In Case-Based Reasoning (CBR) technique,27 prior ex-
perience is employed to solve new problems. For example, CBR28 is used to extract
behaviors from a knowledge base. In this technique, behavior nodes are enhanced
with querying functionality. Being in a state allows the node to query the knowl-
edge base for cases of similar states that are visited in the past, and consequently,
selecting the best behavior. In the manual editing of behavior trees for Non-Player
Characters (NPCs)29 , the main idea is that behaviors typically occur in patterns
that can be reused in future designs. In this technique, CBR is used to retrieve and
reuse behaviors that are already represented as behavior trees. In this technique,
the behavior of a NPC is built at run time using CBR.
CBR can be used in BTs to extract behaviors from a knowledge base21 . In this
technique, CBR is known as solving new problems based on prior experience.27
In this method, an NPC in a particular state will cause the node to query the
knowledge base for similar states that are already processed in order to load an
appropriate behavior.
Q-learning. In Q-learning behavior trees2 , AI designer is guided by indicating
when is the right time to execute branches of AI logic. This technique also facil-
itates the AI design procedure by debugging, analyzing and optimizing behavior
trees. Originally, in the Q-Learning30 algorithm, a table of values are generated
and stored representing the utility of taking an action in a state. In this algorithm,
there are associations between different states and rewards. Rewards are feedback
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 13

to state-action pairs, where utility estimates are improved. In the training phase,
this algorithm saves Q-values for visited states. Updates are conducted based on
this formula:
Q(s, a) = (1 − α)Q(s, a) + α(r + γ maxa (Q(s0 , a0 )))
where Q(s, a) and Q(s0 , a0 ) are the q value of the current state and the successor
state-action pairs, respectfully. α is the learning rate parameter representing how
much the new data affects the previous data in q value. r represents the rewards
for the successor state. In this formula, γ is the discount factor specifying the im-
portance of future rewards. Determining which action is selected for an agent is
performed using a greedy policy, in which the action which is estimated as the
best action is selected.2,31,32 Although learning probabilistic behavior models,31,32
require little domain knowledge, they cannot be used to build plans. Deepest se-
quence nodes from an initial behavior tree can be extracted, where such nodes are
employed in an off-line Q-learning phase to generate a Q-value table.31 Then, sub-
tables from this table are generated. The highest valued states for the action are
extracted into Q-Condition nodes. Then, condition nodes are replaced with the Q-
Condition nodes. The final step is reconstructing of the organization of BT through
sorting children nodes based on maximum Q-value.
Learning from Examples. Domain knowledge can be learned from examples of
behaviors,33 which can quickly react to changes at runtime. The knowledge learned
is shown in the form of a behavior tree. Formally, given a set of examples of a
high-level task, {E1 , E2 , . . . , En }, in which an example is a sequence of cases Ei =
(Ci1 , Ci2 , . . . , Cim ), where a case is an observation and action pair Cij = (Oij , Aij ),
a similarity metric between pairs of observations and pairs of actions, M (Oij , Okl ) ∈
[0, 1], and M (Aij , Akl ) ∈ [0, 1]. This technique aims to extract policies that predict
the next action given previous cases and the current observations. This way, the
behavior tree can show and summarize a large amount of expert knowledge from this
behavior. The procedure starts with identifying areas of commonality within action
sequences, as they show common sub-behaviors. To achieve this goal, a maximally
specific BT is generated using samples of case sequences. Then, the BT is pruned
and reduced by merging common patterns of actions. This continues until no new
pattern is found.
Automatically modifying behavior trees designed by AI designer can be taken
into account in order to display characteristics of human players.34 This starts
with a deterministic behavior tree in a multi-player online role-playing game. This
technique learns behavior trees that employ human player traces as well as a human-
designed tree to cover the variations. In particular, hierarchical machine learning
approaches are used to model the behaviors of player. In this technique, a given
deterministic behavior tree is adapted to cover a range of observed human behaviors.
To this end, an agent controlled by the behavior tree in the game is synchronized
with observations from a human player traces. The agent is continuously updated by
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

14 Yoones A. Sekhavat

→
Generate At
Path
Loop Target

Set Learn on
wayPoint Success

Move

Fig. 3. Refining the behavior tree using the learning technique [35].

observations. The generated tree makes it possible to produce most of the observed
behaviors, while storing contextual observations representing the conditions for each
variation. This way, the tree can make non-deterministic decisions based on the
samples.
Machine learning can be used to refine a single behavior within the context
of other behaviors.35 This is implemented using a decorator node (a node used to
control and modify return values). Given the behavior tree in Fig. 3, when the Move
node returns Success (representing that the agent has traversed the wayPoint), a
learning routine is called on a modified Waypoint behavior. In the training phase,
an agent is initialized with this tree and data about traverses of players. Quadruple
(L, E, F, M ) shows the learning method, where L is the set of possible locations,
E is the error function (between current location set by the behavior l1 and the
current location actually traveled l2 ), F returns a set of environmental features and
M is a learning model. When the model is trained, the set of wayPoint behaviors
is perturbed based on the probabilities learned by M . This way, more human-like
behavior is created.

3.2. Manual BT Generation

Behavior Tree Authoring. Authoring high-quality behaviors for NPCs is an im-

portant research issue in computer games in which games benefit from the adoption
of AI techniques.36 However, since authoring behaviors for NPC requires related
skills in design and programming, the duty of behavior generation process is re-
stricted to individuals who have a certain expertise, which limits exploiting the
ideas of talented people who are not familiar with these expertises. This problem
has resulted in techniques that allow individuals without technical skills to author
and design NPC behaviors in an easy way. In creation of behavior trees by novice
users,36 a digital authoring tool is proposed by which novice users can employ to
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 15

Multiple Goals
Probabilistic
‐ A tree is designed such that Behavior Tree
follows more than one goal ‐ Increase in the expressiveness of
simultaneously the behaviors
‐ Introduces meta behaviors are ‐ Transition form one node to
able to change the state of other another node is annotated with
Integrating BTs with behaviors at runtime probabilities
Utility‐based AI
‐ Utility Score indicates when to
move from one node to another
‐ Decisions are measured based Modification in the
on probabilities that are structure of behavior tree
calculated
‐ Among multiple choices, the one Augmenting BTs with
with highest utility is selected Additional Knowledge
Timed Behaviors ‐Multiple branches can integrate and run at
the same time
‐ Different clocks are added to the ‐ Makes it possible to follow conflicting goals
behavior tree ‐ Allows to assign different priorities to
‐ Delay time to each node can be nodes
assigned

Fig. 4. A taxonomy of variations in the structure of BTs

design the behaviors of NPCs.

In AIPaint37 , which is a tool to design and implement game independent behav-
ior trees, users can create and edit behavior trees using natural sketching interface
overlaid on the game world. As a proof of concept, AIPaint is used to recreate the
behaviors of classic Pac-Man ghosts as well as a computer soccer game. In AIPaint,
game independence is achieved by using Java interface classes to communicate with
the game world. However, this is the game programmer who should implement these
Java interface classes. On the other hand, the game must provide AIPiant the world
state information using feature-value pairs. The scalability of this approach as well
as the complexity of communications between the game world and the behavior tree
authoring tool are the biggest unanswered questions in AIPaint.

4. Extensions to Behavior Trees

Behavior trees are widely extended from different perspectives to address the various
requirements of decision-making in computer games. This section reviews different
variations of behavior trees in computer games literature. A taxonomy of variations
in the structure of BTs as well as strengths and weaknesses of them are shown in
Fig. 4

4.1. Modification in the structure of BTs

BTs and utility-based AI. In a technique to combine behavior tree and utility-
based AI,38 an architecture is proposed to design realistic bots for military sim-
ulators. The most important requirement of a military simulator is designing an
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

16 Yoones A. Sekhavat

enemy acting like a human enemy. In this framework, each level of a behavior tree
corresponds to a different state, where utility-based AI39,41,40 is used to decide how
the state goes in from one transition in a level to another. In addition to the selector
node in the original behavior tree that chooses one event at a time, reasoned nodes
are used to calculate the probability of many events at a time.38 Then, on the basis
of their values, one event is chosen. Reasoner is a complex composite that dynam-
ically calculates probabilities, which on this basis the choices are made.42 In this
setting, instead of a single correct choice, there may exist multiple choices, where
one of which is selected based on external factors. The probabilities of options are
given based on f (x) = e−x , where x = α ∗ threat + β ∗ health + γ ∗ ammo, and α,
β and γ are normalizing factors. The response is aggressive as this function is not
a linear function. In the case of having more health and ammunition, much more
aggressive actions are taken. As an example, suppose once an enemy is detected,
the AI goes into the attack mode including fire (f(x)) and go to cover (1-f(x)).
Increasing the threat results in increasing the probability of going to go to cover
option. Computing the probabilities of each option is based on computing utility
functions. Given ui denoting the option i, the probability of option i is calculated
Pn
as P = ui / i=1 (ui ).
Behavior trees are extended with utility-based AI,43 in which scores (ranging
from 0 to 100) are assigned to each action using utility-based AI. There is a threshold
that results in interrupting the current action execution when the score of an action
exceeds this threshold. This results in suspending an action to make it possible for
the execution of urgent actions.
Component Reasoner54 , which is a modular and hierarchical decision-making
technique, has some similarities with the structure of Behavior Tree (BT). In addi-
tion to supporting many decision-making approaches, Component Reasoner relies
primarily on Weight-Based Reasoners such as a utility-based technique. This tech-
nique is used in video games such as Zoo Tycoon 2 franchised , Iron Mane , and Red
Dead Redemptionf . Originally, behavior trees use boolean discriminators to run se-
lectors. Although this is simple to develop, examining a complex situation may not
be possible before making a decision.55 There are cases in which a more complex ap-
proach to decision-making is required, while making it possible to keep simplicity in
simple decision-making situations. This was a motivation for a technique54 to pro-
pose a framework that keeps the hierarchy and modularity of behaviors trees, where
it is possible to use complex decision makers when it is appropriate. Reasoners54
instead of selectors can be used to make such decisions.
While selectors employ simple logics such as following the first valid option
or fixed probabilities (that are computed at design time) for decision-making, a
reasoner allows using any arbitrarily complex logic. This way, highly deterministic

d https://www.bluefangsolutions.com/
e https://www.gameloft.com
f http://www.rockstargames.com
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 17

behaviors can still use a BT-style selector, while in the cases it is required to compute
relative advantages of several possibilities, Component Reasoner is used. In the cases
when AI agents need to learn from past results, Genetics-Based Machine Learning56
techniques can be employed.
Timed Behaviors. BTs can be extended by a time notation44 (based on the
concepts used in timed automata45 ). This is conducted in a way that different clocks
are added to behavior trees such that they work simultaneously. The clocks include
a constraint indicating how long the process can delay after taking an action. The
authors also provided a formal semantics for timed behavior trees. In a timed BT,
standard BT nodes (e.g., selection and sequence) are augmented with clock reset
(for state realization), clock guard (that restricts the timing of a transition from
one location to the next), and invariant (a constraint on how long the process can
delay after taking an action).
Probabilistic Behavior Trees. Current syntaxes for behavior trees do not have
an option to represent probabilistic behaviors.50 . In a technique to extend BTs,50
probabilistic behavior is expressed, which results in increasing the expressiveness
of behaviors. This technique, which is called probabilistic timed Behavior Trees
(ptBTs), makes it possible for AI designers to model timed as well as probabilistic
behaviors. In this technique, transitions are annotated with a probability indicating
the probability of taking place of that transition. In particular, each node is associ-
ated with an optional probability (between 0 to 1). Probabilistic branching is used
in addition to concurrent branching. The sum of probabilities in the child nodes is
less than or equal to 1. In this setting, given p as the sum of probabilities of a node,
no transition is taken with the probability of 1 − P .
Multiple Goals. Because behavior trees are designed to control the behav-
iors of a single unit, using them to simultaneously processing of multiple goals is
complicated.52 In a game agent (EISBot)52 , ABL (A Behavior Language)53 is used
as a reactive planning language that includes a set of agents, where each agent must
deal with a set of goals to achieve (e.g., Fig. 5). In order to achieve goals, agents
choose and execute a behavior from behaviors collection, where a behavior is a set
of preconditions specifying if this behavior can be run in the current conditions.
This technique supports meta behaviors, which are known as behaviors that can
change the state of other behaviors at runtime. This makes it possible for an agent
to deal with multiple goals simultaneously, while allows communications between
behaviors. At runtime, all of the goals of an agent are stored in an active behav-
ior tree, where in each execution, one of the open leaf nodes (which is a behavior
that follows a goal and includes component steps) are executed (i.e., scripts to run
actions or to compute).
In an architecture to control the behaviors of AI agents80 , responsive, collabora-
tive, interruptible and resumable behaviors based on behavior queues are supported.
In this technique, behaviors are assigned to roles in order to encapsulate behaviors
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

18 Yoones A. Sekhavat

Root

Goal 1 Goal 2

→→
→ →→

Is door Move into Move into Animate Detect

Open? room door Move Enemy

Fig. 5. A tree with multiple goals. Goal 1 and Goal 2 are continued with sequential and parallel
behaviors, respectively.

into components that evolve dynamically, depending on environmental conditions.

Stochastic Behavior Trees. In a technique to compute performance measures of
plans using BTs,3 the concept of Stochastic Behavior Trees is introduced to interpret
the interaction of a BT node with its children in terms of a Discrete Time Markov
Chain. This way, computations for performances are propagated from one level to
the next level in the BT. Finally, the reliability of a BT (that is a composition of
primitive actions) is computed. This is reported in terms of the success probability
and time to success.
Integration with Social Territoriality. Social territoriality can be integrated
with behavior trees, in which multiple branches can integrate and run at the same
time. 46 In this architecture, different branches of a node may follow goals that are
conflicting. In this architecture, tree nodes have a priority, where nodes with higher
priority can subsume lower priority nodes, which makes it possible to immediately
respond critical events. The subsumption allows handling complex behavior trees
in terms of horizontal layers of goals at different levels of abstraction. In computer
games, this technique results in the responsiveness and the continuity of motions
with controlling where to look, stand, and move, while playing a right animation.

4.2. Augmenting BTs with Additional Knowledge

A summary of the techniques to enhance BTs by augmenting them with additional
knowledge is shown in Table 4. Main features of these techniques as well as strengths
and weaknesses of them are shown in this figure.
Augmenting with Internal States. There are behavioral cases that cannot be
performed using standard behavior tree implementations.47 They attribute this
problem to the lack of internal states in standard BTs, that do not allow initializa-
tion and termination of tasks. To address this problem a technique47 is proposed
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 19

Table 4: A summary of the techniques to enhance BTs by augmenting them with

additional knowledge.
Genetic Algorithms
+ Integration of FSM and BTs
+ Adding parameter to parent behavior to store the state of the behaviors
- Requires additional time manager to mange long-term missions
Emotional Factor
+ BT designer can indicate which behaviors can be affected by emotional factors
+ Emotional selector node indicates which child node to run based on emotion values
- Extra processing time
Behavior Tagging
+ Employs the concept of behavior tag to maintain specific factors
+ Employs the concept of behavior message that re used to inform other nodes
regarding the changes in the behaviors
Hinted Execution Behavior Trees
+ makes it possible to create and test new features in a plugin fashion
+ Hint is a piece of information that tell AI what to do
Dynamic Behavior Trees
+ Some nodes hold queries instead of actual behaviors, where at runtime, query nodes
are replaced with actual behavior nodes
+ Allows to make abstract behavior trees
Self Validating Behavior Trees
+ Allows to detect faulty BTs at design time
+ Adding parameter to parent behavior to store the state of the behaviors
- Not applicable on existing BT design tools

that integrates state machines within behavior trees. This technique is compatible
to existing behavior tree libraries48,49 . In this technique, state machines are inte-
grated into arbitrary behavior trees. This way, the full power of state machines are
available for AI designer in addition to standard features of BTs. In this technique,
state machine are clocked, where there are delays in the control flow. Time events
are used in this technique for long term mission statements.
Classic behavior trees suffer from the problem that all states in a behavior are
lost once the behavior is removed.58 One solution would be adding a separate per-
manent component to an NPC, where the behaviors can access to this component.
One alternative general solution is allowing to store persistent data class specific
to the behavior. However, the hard-coded structure of the behavior trees do not
provide modularity options such as programming. To address the problem of two
NPCs of the same type that behave differently, a parameter can be added to the
parent behavior to store the states of behaviors.58
Emotional Factor. Emotion is an important factor for human in decision-making,
that allows choosing among alternative solutions.51 Although emotions are integral
parts of decision-making, current behavior trees are not integrated with emotions.
Behaving in an emotional way is key to have a natural human behavior. In a tech-
nique to extend behavior trees with emotions,51 an AI designer indicates which
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

20 Yoones A. Sekhavat

behaviors can be affected by emotional factors. They argue that time discounting,
risk perception, and planning are affected by emotions. They proposed a new selec-
tor composite called emotional selector, in which emotions are used to change the
priorities of child nodes at runtime. In the emotional selector, the values for time,
risk, and planning are computed and attached to each child of the emotional selec-
tor. The overall weight of the child node i of emotional selector node is computed
as: Wi = α.Wrisk,i + β.Wtime,i + γ.Wplan,i , where α, β and γ are importance factors
of the parameters. After sorting child nodes of the emotional selector node based
on increasing weights Wi , a probability probi to every child node i is assigned such
that: probi = a(1 − a)i − 1), where 0.5 ≤ a < 1.
Behavior Tagging. In order to handle complex variations of behaviors, a behav-
ior organizing structure similar to behavior trees is proposed.57 In this technique,
behavior tags and behavior messages are used to manage behaviors of virtual char-
acters. The idea for behavior tag is to maintain specific factors in the behavior in
terms of tags, which are ignored when a tree is customized. On the other hand,
behavior messages are used to inform the changes regarding the behaviors. Instead
of representing this in a static behavior tree structure, messages are dynamically
issued to a specific behavior node during the execution of the behavior tree.
Hinted Execution Behavior Trees. Hinted Execution Behavior Tree (HeBTs)59
is proposed as an extension to BT that provides more controls for developers to
create and test new features in a plug-in fashion. In this technique, a hint is a piece
of information tells the AI what to do. However, the AI itself decides if it should
accept the suggestion. HeBTs allow generating low-risk prototypes of behavior trees
in a way that allows teams to try new ideas at final stages of development without
hard changes in the code.
Dynamic Behavior Trees. A technique to extend behavior trees is proposed
called Dynamic Behavior Trees (DBT)28 , in which some nodes hold queries instead
of actual behaviors. A DBT is generated at runtime through substituting query
nodes with actual behaviors. This allows an AI designer to specify high level be-
haviors. This way, properties of the desired behavior for a given state is specified
without behavior implementation. In this technique, new basic behaviors are auto-
matically selected such that they best fit to designers specifications.
Self-Validating Behavior Trees. In an architecture for a BT that employs
modules with reflection capabilities,60 it is possible to detect faulty BTs during the
design of a behavior tree. This provides an extra check to design BTs. A similar
method is also used at runtime such that the reflective components is used before
BT execution. In the application of parameterization in behavior trees,61 , BT tasks
are functions with parameters in comparison to classic behavior trees, where tasks
are nonparametric. In this technique, a subtree is encapsulated with an exposed
parameter interface by a lookup node.
A model reducing technique for behavior trees78 can be applied before model
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 21

checking for the purpose of reducing time and memory resources at runtime. This
model reducing technique is developed based on the slicing79 technique, which is
originally used to help understanding and debugging programs.

5. Behavior Tree In Practice

BTs are widely used in the game industry29,62,63 since their introduction for the
video game industry. Major published games as well as major game engines such
as Unity, Unreal Engine, and CryEngine provide facilities to define and use behav-
ior trees. Behavior trees make it possible to design intelligent behaviors for NPCs
in computer games.3 They are useful to control complex multi mission agents,63
while allow to formally verify mission plans of NPCs.64 They also allow to compute
execution times activities.65
RAINg and Behavh are two important commercial behavior trees developed for
Unity game engine. UNREALKISMET, that is integrated in the Unreal Develop-
ment Kit game editori , and flow-graph editor in CryENGINEj are two examples
of visual scripting tools to model the behaviors in computer games through some
variation of data flow diagrams without programming.
Wide variety of games including Halo 266 and Spore68 , and GTA69 use behavior
trees. In Halo 2, which is a first-person shooter game in which the player fights
groups of aliens, behavior tree is used to control behaviors including search, com-
bat, flight, self-preservation, idle, charging, fighting, and guarding. This tree has a
maximum depth of four, including leaf nodes that execute concrete behaviors. Re-
active behaviors are enabled using stimulus behaviors. In this game, events received
from the game react by inserting stimulus nodes into the behavior tree at runtime.
In the next execution, the new stimulus behavior is executed. Although this makes
it possible to react to rare events without a need to check conditions, the behavior
tree becomes hard to understand. Memory usage in Halo is limited because of using
a shared static structure for the behavior tree. This prevents each character to have
its own AI.
In an evolutionary technique for behavior trees20 , a technique is proposed to
develop a competitive player that outperforms DEFCON game’s original player in
a majority of cases. Grammatical Evolution21 was employed to evolve behavior trees
for the Mario Game. In Darmok 70 , a technique to build a repository of behaviors is
employed from which behaviors are selected. In the decision cycle during updates,
finished steps are checked, and then the world state is updated. Finally, the next
step for the next action is selected. In an adaptive behavior tree 59 to modify and
update the behavior tree based on knowledge gained from the player during playing,
BTs are employed to generate levels tailored to the player. This makes it possible

g http://angryant.com/behave/
h http://rivaltheory.com/wiki/behaviortrees/behaviortreeeditor
i https://www.unrealengine.com/
j http://cryengine.com/
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

22 Yoones A. Sekhavat

to decrease the difficulty of the game to accommodate casual players.

Since the opponents in computer games often move in some kind of continu-
ous space, it can be represented by a Hybrid Dynamical System (HDS).71 In the
formalized behavior trees in computer games10 , relations between BTs and HDS
are defined to use in Unmanned Aerial Vehicle guidance (UAV). In classic HDS,
transitions are one-way transfers of control, (similar to the goto-statement in the
programming), while the implicit and two-way transfer of control in behavior trees
are more similar to function calls. Behavior trees are used to improve Unmanned
Aerial Vehicle guidance.10 In this technique, BTs are used as a Hybrid Dynami-
cal Systems (HDS), in which state transitions are represented in a tree structure.
This architecture is used in the control architecture of many automated in-game
opponents. As shown in this paper, any HDS can be written in terms of a BT.
In a behavior tree based control to control the decision-making processes for
robot soccer,72 a hierarchical behavior tree is proposed to simplify the decision-
making process for soccer robots. This is an example of uncertain environment,
where a priori knowledge of environment and movement of other robots are absent
or partial. They also proposed a hierarchical behavior tree based algorithm to en-
hance path and improve navigation of soccer robots. In the BTs proposed in this
work, complex states are easily defined and designed in a modular way such that
incremental development of behaviors are possible, while makes it easy to main-
tain and extend the control system. BTs notation and model checking were used to
evaluate system safety.73 They argue that since verifying the logic of system safety
designs is not trivial for humans (due to large details and scenarios), using this
notation makes it possible to automate such analysis.
Behavior trees are used to address the problem of representing requirements of a
system to be developed such that it is readable by clients and expert modelers.74,75
In this technique, BT is a graphical representation that allows representing a range
of constructs, states, synchronization, message passing, concurrency, choice and it-
eration control structures. A requirement can be represented using a behavior tree,
where each node is annotated with a requirement, allowing traceability back to the
original informal requirements.14 In this technique inconsistencies, redundancies,
incompleteness, and ambiguities are identifiable. Such tree can be considered as
a common language that is used to validate the process. In this framework, be-
havior tree is used as a systematically structured representation of the system. A
technique76 is proposed that exploits behavior tree notation to allow the integration
of requirements and describing the integration steps.
Behavior trees can be used to show functional requirements of software as an
easy to understand structure.77 In this method, each requirement is translated to
a behavior tree, where behavior trees can be integrated into a single behavior tree,
which is known as design behavior trees (DBTs). This representation makes it pos-
sible to extract component-based design of the system. When a system is evolved,
each version is shown using different behavior trees. Merging algorithms can be used
to compare different DBTs that results in generation of an evolutionary design be-
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 23

havior tree (EvDBT). An EvDBT includes data regarding all the compared DBTs,
where the evolution information is represented.
BRAINSk is an open-source behavior tree component employed in different types
of games including action, tactical, and stealth games. This component exploits
UDK AI and Navigation, Unreal script for Behavior Tree Hierarchies, and Kismet
as a Visual Editor. This BT is currently used in SanctityLost. Think is an AI be-
havior tree editor that can be configured and used for different types of projects.
AI designers can create their own custom behavior tree nodes with custom proper-
ties. Such user defined nodes are integrated to create behavior trees. This makes it
possible to reuse tree nodes for future projects. The behavior tree can be exported
as XML, text, C++ code and Unreal Script code.
BSTl (Behavior Selection Tree) is a tool to create and manage behavior trees in
CryENGINE3. AI agents have their own BST, which is used to choose a behavior
based on game conditions. Brainiacm is a visual behavior tree design tool that
makes it possible to build behavior trees employing drag and drop of nodes. This
tool allows exporting the behavior trees in XML format. The game designer writes
her own code to generate logic for the trees.
ComBEn is a behavior tree editor that allows an AI designer to simultaneously
edit the trees using textual and graphical items. RAIN’s behavior treeo supports
simple to complex decision-making process. This component provides a graphical
editor to design behaviors and add different actions, animations, and sensing fea-
tures. This behavior tree component is widely used in Unity game engine.
The pervasiveness of mobile devices as well as the growth in the technologies of
application development is driving games market faster than before. Revenue from
games is now surpassing revenue from movie sales 81,82,83 . Entering to this market
requires employing powerful AI tools to make advanced games. Behavior trees by
providing facilities to define complex behaviors make it possible to design believable
non-player characters.

6. Conclusion
Behavior trees have become essential to design, control and monitor of NPCs in
computer games. The paper presents a survey of literature for techniques to au-
tomatically and manually generate behavior trees. This paper also reviews various
techniques to extend and enhance the structure of behavior trees with the aim of
increasing the modularity, reusability, dynamic control of trees, and flexibility in
the design of behaviors. This paper presents a list of features to compare BTs and
classifies BTs based on these features. This classification provides a benchmark to

k https://www.epicgames.com/
l http://www.cryengine.com
m http://www.brainiac.codeplex.com/
n http://www.oskarvanrest.github.io/ComBE/
o http://www.rivaltheory.com/
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

24 Yoones A. Sekhavat

compare behavior tree research prototypes as well as commercial products. The

paper also surveys how and to what extend BTs are used in practice in computer
games.

References
1. Millington, I. (2009). Artificial Intelligence for Games. J. Funge (Ed.). Elsevier Science
and Technology.
2. Dey, R., and Child, C. (2013, August). Ql-bt: Enhancing behaviour tree design and
implementation with q-learning. In Computational Intelligence in Games (CIG), 2013
IEEE Conference on (pp. 1-8). IEEE.
3. Colledanchise, M., and Ogren, P. (2014, September). How Behavior Trees modularize
robustness and safety in hybrid systems. In Intelligent Robots and Systems (IROS
2014), 2014 IEEE/RSJ International Conference on (pp. 1482-1488). IEEE.
4. Roijers, D. M., Vamplew, P., Whiteson, S., and Dazeley, R. (2013). A survey of multi-
objective sequential decision-making. Journal of Artificial Intelligence Research.
5. Kumar, A., Zilberstein, S., and Toussaint, M. (2015). Probabilistic inference techniques
for scalable multiagent decision making. Journal of Artificial Intelligence Research,
53(1), 223-270.
6. Kalech, M., and Reches, S. (2015). Decision making with dynamic uncertain events.
Journal of Artificial Intelligence Research, 54(1), 233-275.
7. Robertson, G., and Watson, I. (2014). A review of real-time strategy game AI. AI
Magazine, 35(4), 75-104.
8. Rizzo, P. (1999). Goal-based personalities and social behaviors in believable agents.
Applied artificial intelligence, 13(3), 239-271.
9. Rodrigues, R. A., de Lima Bicho, A., Paravisi, M., Jung, C. R., Magalhaes, L. P., and
Musse, S. R. (2010). An interactive model for steering behaviors of groups of characters.
Applied Artificial Intelligence, 24(6), 594-616.
10. Ogren, P. (2012, August). Increasing Modularity of UAV Control Systems using Com-
puter Game Behavior Trees. In AIAA Guidance, Navigation and Control Conference,
Minneapolis, MN.
11. Bennett, C., and Sagmiller, D. V. (2014). Unity AI Programming Essentials. Packt
Publishing Ltd.
12. Marzinotto, A., Colledanchise, M., Smith, C., and Ogren, P. (2014, May). Towards
a unified behavior trees framework for robot control. In Robotics and Automation
(ICRA), 2014 IEEE International Conference on (pp. 5420-5427). IEEE.
13. Winter, K. (2004, April). Formalising behaviour trees with CSP. In IFM (pp. 148-167).
14. Colvin, R. J., and Hayes, I. J. (2011). A semantics for Behavior Trees using CSP with
specification commands. Science of Computer Programming, 76(10), 891-914.
15. Hoare, C. A. R. (1978). Communicating sequential processes. Communications of the
ACM, 21(8), 666-677.
16. Roscoe, B. (1998). The theory and practice of concurrency.
17. Eugster, P. T., Felber, P. A., Guerraoui, R., Kermarrec, A. M. (2003). The many faces
of publish/subscribe. ACM Computing Surveys (CSUR), 35(2), 114-131.
18. Kim, S. K., Myers, T., Wendland, M. F., and Lindsay, P. A. (2012). Execution of
natural language requirements using State Machines synthesised from Behavior Trees.
Journal of Systems and Software, 85(11), 2652-2664.
19. Colvin, R., and Hayes, I. J. (2010). A semantics for Behavior Trees (No. SSE-2010-03,
pp. 1-26).
20. Lim, C. U., Baumgarten, R., and Colton, S. (2010). Evolving behaviour trees for the
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 25

commercial game DEFCON. In Applications of evolutionary computation (pp. 100-

110). Springer Berlin Heidelberg.
21. Perez, D., Nicolau, M., ONeill, M., and Brabazon, A. (2011). Evolving behaviour
trees for the Mario AI competition using grammatical evolution. In Applications of
Evolutionary Computation (pp. 123-132). Springer Berlin Heidelberg.
22. Champanard, A. (2012). Understanding the second-generation of behavior trees.
http://aigamedev. com/insider/tutorial/second-generation-bt.[Hmtad: 14.04. 06].
23. E. Winner and M. Veloso, Distill: Learning domain-specific planners by example, in
Proceedings of the International Conference on Machine Learning, 2003, pp. 800807.
24. O’Neill, M., and Ryan, C. (2012). Grammatical evolution: evolutionary automatic
programming in an arbitrary language (Vol. 4). Springer Science and Business Media.
25. Colledanchise, M., Parasuraman, R., and gren, P. (2015). Learning of Behavior Trees
for Autonomous Agents. arXiv preprint arXiv:1504.05811.
26. Scheper, K. Y., Tijmons, S., de Visser, C. C., and de Croon, G. C. (2016). Behavior
Trees for Evolutionary Robotics. Artificial life.
27. J.L. Kolodner, An introduction to case-based reasoning, Artificial Intelligence Review
6, 1992, pp. 334.
28. Flrez-Puga, G., Gomez-Martin, M., Diaz-Agudo, B., and Gonzalez-Calero, P. (2008).
Dynamic expansion of behaviour trees. Proceedings of AIIDE, 36-41.
29. Flrez-Puga, G., Gmez-Martn, M. A., Gmez-Martn, P. P., Daz-Agudo, B., and Gonzlez-
Calero, P. A. (2009). Query-enabled behavior trees. Computational Intelligence and AI
in Games, IEEE Transactions on, 1(4), 298-308.
30. Watkins, C. J. C. H. (1989). Learning from delayed rewards (Doctoral dissertation,
University of Cambridge).
31. Dereszynski, E. W., Hostetler, J., Fern, A., Dietterich, T. G., Hoang, T. T., and
Udarbe, M. (2011, September). Learning Probabilistic Behavior Models in Real-Time
Strategy Games. In AIIDE.
32. Synnaeve, G., and Bessiere, P. (2011). A Bayesian model for plan recognition in RTS
games applied to StarCraft. arXiv preprint arXiv:1111.3735.
33. Robertson, G., and Watson, I. (2015, September). Building behavior trees from ob-
servations in real-time strategy games. In Innovations in Intelligent SysTems and Ap-
plications (INISTA), 2015 International Symposium on (pp. 1-7). IEEE.
34. Tomai, E., and Flores, R. (2014). Adapting in-game agent behavior by observation of
players using learning behavior trees. In Proceedings of the 9th International Conference
on the Foundations of Digital Games (FDG 2014).
35. Tomai, E., Salazar, R., and Flores, R. (2013, July). Simulating aggregate player be-
havior with learning behavior trees. In Proceedings of the 22nd Annual Conference on
Behavior Representation in Modeling and Simulation.
36. Mehta, M., and Corradini, A. (2014, June). An Approach to Behavior Authoring for
Non-Playing Characters in Digital Games. In Proceedings of the 2014 Mulitmedia, In-
teraction, Design and Innovation International Conference on Multimedia, Interaction,
Design and Innovation (pp. 1-7). ACM.
37. Becroft, D., Bassett, J., Meja, A., Rich, C., and Sidner, C. L. (2011, September).
AIPaint: A Sketch-Based Behavior Tree Authoring Tool. In AIIDE.
38. Jadon, S., Singhal, A., and Dawn, S. (2014). Military Simulator-A Case Study of
Behaviour Tree and Unity based architecture. arXiv preprint arXiv:1405.7944. Chicago
39. Mark, D., Dill, K., and Engineer, A. I. (2010). Improving ai decision modeling through
utility theory. In Game Developers Conference.
40. Laird, J., and VanLent, M. (2001). Human-level AI’s killer application: Interactive
computer games. AI magazine, 22(2), 15.
October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

26 Yoones A. Sekhavat

41. Dill, K., and Mark, D. (2012). Embracing the Dark Art of Mathematical Modeling in
AI. In Game Developers Conference.
42. van Lent, M., and Laird, J. (1998). Developing an artificial intelligence engine. Ann
Arbor, 1001, 48109-2110.
43. Othman, M. N. M., and Haron, H. (2014, September). Implementing game artificial
intelligence to decision making of agents in emergency egress. In Software Engineering
Conference (MySEC), 2014 8th Malaysian (pp. 316-320). IEEE.
44. Colvin, R., Grunske, L., and Winter, K. (2008). Timed behavior trees for failure mode
and effects analysis of time-critical systems. Journal of Systems and Software, 81(12),
2163-2182.
45. Bengtsson, J., and Yi, W. (2004). Timed automata: Semantics, algorithms and tools.
In Lectures on Concurrency and Petri Nets (pp. 87-124). Springer Berlin Heidelberg.
46. Pedica, C., and Vilhjlmsson, H. H. (2012, August). Lifelike interactive characters with
behavior trees for social territorial intelligence. In ACM SIGGRAPH 2012 Posters (p.
32). ACM.
47. Klckner, A. (2015). Behavior Trees with Stateful Tasks. In Advances in Aerospace
Guidance, Navigation and Control (pp. 509-519). Springer International Publishing.
48. Elmqvist, H., Gaucher, F., Mattsson, S. E., and Dupont, F. (2012, September). State
machines in modelica. In Proceedings of 9th International Modelica Conference, Mu-
nich, Germany, September (pp. 3-5).
49. Klckner, A., van der Linden, F., and Zimmer, D. (2014). The Modelica BehaviorTrees
Library: Mission planning in continuous-time for unmanned aircraft. In Proceedings of
the 10th International Modelica Conference (No. 96, pp. 727-736).
50. Colvin, R., Grunske, L., and Winter, K. (2007, January). Probabilistic timed behavior
trees. In Integrated Formal Methods (pp. 156-175). Springer Berlin Heidelberg.
51. Johansson, A., and Dell’Acqua, P. (2012, September). Emotional behavior trees. In
Computational Intelligence and Games, 2012 IEEE Conference on (pp. 355-362). IEEE.
52. Weber, B. G., Mawhorter, P., Mateas, M., and Jhala, A. (2010, August). Reactive
planning idioms for multi-scale game AI. In Computational Intelligence and Games
(CIG), 2010 IEEE Symposium on (pp. 115-122). IEEE.
53. Mateas, M. (2002). Interactive drama, art and artificial intelligence.
54. Dill, K., and Martin, L. (2011). A game AI approach to autonomous control of virtual
characters. In Proceedings of the 2011 Interservice/Industry Training, Simulation, and
Education Conference.
55. Dill, K. (2008). Embracing declarative AI with a goal-based approach. AI Game Pro-
gramming Wisdom, 4, 229-238.
56. Harrison, G. A., and Worden, E. W. (2007, July). Genetically programmed learning
classifier system description and results. In Proceedings of the 9th annual conference
companion on Genetic and evolutionary computation (pp. 2729-2736). ACM.
57. Li, L., Liu, G., Zhang, M., Pan, Z., and Song, E. (2010). BAAP: a behavioral ani-
mation authoring platform for emotion driven 3D virtual characters. In Entertainment
Computing-ICEC 2010 (pp. 350-357). Springer Berlin Heidelberg.
58. Alt, G. (2004). The suffering: A game AI case study. In Challenges in Game AI
workshop, Nineteenth national conference on Artificial Intelligence (pp. 134-138).
59. Ocio, S. (2012). Adapting AI Behaviors To Players in Driver San Francisco Hinted.
60. Llans, D., Gmez-Martn, M. A., and Gonzlez-Calero, P. A. (2009). Self-Validated Be-
haviour Trees through Reflective Components. In AIIDE.
61. Shoulson, A., Garcia, F. M., Jones, M., Mead, R., and Badler, N. I. (2011). Parame-
terizing behavior trees. In Motion in Games (pp. 144-155). Springer Berlin Heidelberg.
62. Champandard, A. (2008). Getting started with decision making and control systems.
 October 31, 2016 13:46 WSPC/INSTRUCTION FILE IJAIT-D-16-00092

Behavior Trees for Computer Games 27

AI Game Programming Wisdom, 4, 257-264.

63. Palma, R., Gonzlez-Calero, P. A., Gmez-Martn, M. A., and Gmez-Martn, P. P. (2011,
March). Extending Case-Based Planning with Behavior Trees. In FLAIRS Conference.
64. Fikes, R. E., and Nilsson, N. J. (1972). STRIPS: A new approach to the application
of theorem proving to problem solving. Artificial intelligence, 2(3), 189-208.
65. Wang, X. (1995, July). Learning by observation and practice: An incremental approach
for planning operator acquisition. In ICML (pp. 549-557).
66. Isla, D. (2005, March). Handling complexity in the Halo 2 AI. In Game Developers
Conference (Vol. 12).
67. Isla, D. (2008). Building a better battle. In Game Developers Conference, San Fran-
cisco.
68. Hecker, C. (2009). My liner notes for spore/spore behavior tree docs.
www.chrishecker.com, 2007
69. Champandard, A. (2007, December). Behavior trees for next-gen game AI. In Game
Developers Conference, Audio Lecture.
70. Ontann, S., Mishra, K., Sugandh, N., and Ram, A. (2010). ON-LINE CASE-BASED
PLANNING. Computational Intelligence, 26(1), 84-119.
71. Branicky, M. S. (2005). Introduction to hybrid systems. In Handbook of Networked
and Embedded Control Systems (pp. 91-116). Birkhuser Boston.
72. Abiyev, R. H., Akkaya, N., and Aytac, E. (2013, June). Control of soccer robots using
behaviour trees. In Control Conference (ASCC), 2013 9th Asian (pp. 1-6). IEEE.
73. Lindsay, P., Winter, K., and Yatapanage, N. (2010, September). Safety assessment
using Behavior Trees and model checking. In Software Engineering and Formal Methods
(SEFM), 2010 8th IEEE International Conference on (pp. 181-190). IEEE.
74. Liu, Z., and Jifeng, H. (2006). Mathematical Frameworks for Component Software:
Models for Analysis and Synthesis (Series on Component-Based Software Develop-
ment). World Scientific Publishing Co., Inc..
75. Smith, C., Winter, K., Hayes, I., Dromey, G., Lindsay, P., and Carrington, D. (2004,
September). An environment for building a system out of its requirements. In Proceed-
ings of the 19th IEEE international conference on Automated software engineering (pp.
398-399). IEEE Computer Society.
76. Winter, K., Hayes, I. J., and Colvin, R. (2010, September). Integrating requirements:
the Behavior Tree philosophy. In Software Engineering and Formal Methods (SEFM),
2010 8th IEEE International Conference on (pp. 41-50). IEEE.
77. Wen, L., Kirk, D., and Dromey, R. G. (2007, September). A tool to visualize behav-
ior and design evolution. In Ninth international workshop on Principles of software
evolution: in conjunction with the 6th ESEC/FSE joint meeting (pp. 114-115). ACM.
78. Yatapanage, N., Winter, K., Zafar, S. (2010). Slicing behavior tree models for verifi-
cation. In Theoretical Computer Science (pp. 125-139). Springer Berlin Heidelberg.
79. Weiser, M. (1981, March). Program slicing. In Proceedings of the 5th international
conference on Software engineering (pp. 439-449). IEEE Press.
80. Cutumisu, M., and Szafron, D. (2009). An Architecture for Game Behavior AI: Be-
havior Multi-Queues. In AIIDE.
81. Sekhavat, Y. A. (2016). Nowcasting Mobile Games Ranking Using Web Search Query
Data. International Journal of Computer Games Technology, 2016.
82. Sekhavat, Y. A., & Abdollahi, P. Can Google Nowcast the Market Trend of Iranian
Mobile Games?.
83. Sekhavat, Y. A. (2016). KioskAR: An Augmented Reality Game as a New Business
Model to Present Artworks. International Journal of Computer Games Technology,
2016.

View publication stats