CSC384: Introduction to Artificial Intelligence

Constraint Satisfaction Problems

(Backtracking Search)

• Chapter 6
– 6.1: Formalism
– 6.2: Constraint Propagation
– 6.3: Backtracking Search for CSP
– 6.4 is about local search which is a very
useful idea but we won’t cover it in class.
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 1
Acknowledgements

• Much of the material in the lecture slides comes from

Fahiem Bacchus, Sheila McIlraith, and Craig Boutilier.

• Some slides come from a tutorial by Andrew Moore via

Sonya Allin.

• Some slides are modified or unmodified slides provided

by Russell and Norvig.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 2
Constraint Satisfaction Problems (CSP)

• The search algorithms we discussed so far had no

knowledge of the states representation (black box).
– For each problem we had to design a new state
representation (and embed in it the sub-routines we pass
to the search algorithms).
• Instead we can have a general state representation
that works well for many different problems.
• We can then build specialized search algorithms that
operate efficiently on this general state representation.
• We call the class of problems that can be represented
with this specialized representation:
CSPs – Constraint Satisfaction Problems.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 3
Constraint Satisfaction Problems (CSP)

• The idea: represent states as a vector of feature

values.
– k-features (or variables)
– Each feature takes a value. Each variable has a
domain of possible values:
• height = {short, average, tall},
• weight = {light, average, heavy}

• In CSPs, the problem is to search for a set of values

for the features (variables) so that the values satisfy
some conditions (constraints).
– i.e., a goal state specified as conditions on the
vector of feature values.
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 4
Example: Sudoku

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 5
Example: Sudoku

• 81 variables, each representing the value of a cell.

• Values: a fixed value for those cells that are

already filled in, the values {1-9} for those cells that
are empty.

• Solution: a value for each cell satisfying the

constraints:
– No cell in the same column can have the same value.
– No cell in the same row can have the same value.
– No cell in the same sub-square can have the same value.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 6
Formalization of a CSP

•More formally, a CSP consists of

– A set of variables V1, …, Vn
– For each variable a domain of possible values
Dom[Vi].
– A set of constraints C1,…, Cm.

– A solution to a CSP is an assignment of a value

to all of the variables such that every constraint
is satisfied.
– A CSP is not satisfiable, if no solution exists.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 7
Formalization of a CSP
• Each variable can be assigned any value from its
domain.
• Vi = d where d ∈ Dom[Vi]

• Each constraint C
– Has a set of variables it is over, called its scope;
• e.g., C(V1,V2,V4) ranges over V1, V2, V4
– Has a restriction on the values of the variables in the scope;
• e.g. C(V1,V2,V3) = ‹(V1,V2,V3), V1≠ V2 ^ V1 ≠ V4 ^ V2 ≠ V4 ›
or (shorter) C(V1,V2,V3): V1≠ V2, V1 ≠ V4, V2 ≠ V4
– Is a Boolean function that maps assignments to the variables in
its scope to true/false.
• e.g. C(V1=a,V2=b,V4=c) = True
– this set of assignments satisfies the constraint.
• e.g. C(V1=b,V2=c,V4=c) = False
– this set of assignments falsifies the constraint.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 8
Formalization of a CSP
• Unary Constraints (over one variable)
– e.g. C(X):X=2; C(Y): Y>5
• Binary Constraints (over two variables)
– e.g. C(X,Y): X+Y<6
– Can be represented by Constraint Graph
• Nodes are variables, arcs show constraints.
• e.g. 4-Queens:

• Higher-order constraints: over 3 or more variables

– We can convert any constraint into a set of binary
constraints (may need some auxiliary variables).
• Look at the exercise in the book.
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 9
Example: Sudoku
• Variables: V11, V12, …, V21, V22, …, V91, …, V99
• Domains:
– Dom[Vij] = {1-9} for empty cells
– Dom[Vij] = {k} a fixed value k for filled cells.
• Constraints:
– Row constraints:
• CR1(V11, V12, V13, …, V19)
• CR2(V21, V22, V23, …, V29)
• ...., CR9(V91, V92, …, V99)
– Column Constraints:
• CC1(V11, V21, V31, …, V91)
• CC2(V21, V22, V13, …, V92)
• ...., CC9(V19, V29, …, V99)
– Sub-Square Constraints:
• CSS1(V11, V12, V13, V21, V22, V23, V31, V32, V33)
• CSS1(V14, V15, V16,…, V34, V35, V36)
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 10
Example: Sudoku
• Each of these constraints is over 9 variables, and they
are all the same constraint:
– Any assignment to these 9 variables such that each variable
has a unique value satisfies the constraint.
– Any assignment where two or more variables have the same
value falsifies the constraint.

• Special kind of constraints called ALL-DIFF constraints.

– An ALL-DIFF constraint over k variables can be equivalently
represented by (k choose 2) “not-equal constraints” (NEQ)
over each pair of these variables.
– e.g. CSS1(V11, V12, V13, V21, V22, V23, V31, V32, V33) = NEQ(V11,V12),
NEQ(V11,V13), NEQ(V11,V21) …, NEQ(V32,V33)
– Remember: all higher-order constraints can be converted into
a set of binary constraints

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 11
Example: Sudoku

•Thus Sudoku has 3x9 ALL-DIFF constraints, one

over each set of variables in the same row, one
over each set of variables in the same column,
and one over each set of variables in the same
sub-square.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 12
Solving CSPs
• CSPs can be solved by a specialized
version of depth-first search.
– Actually depth-limited search. Why?

• Key intuitions:
– We can build up to a solution by searching through the
space of partial assignments.
– Order in which we assign the variables does not matter –
eventually they all have to be assigned. We can decide
on a suitable value for one variable at a time!

This is the key idea of backtracking search.
– If during the process of building up a solution we falsify a
constraint, we can immediately reject all possible ways of
extending the current partial assignment.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 13
CSP as a Search Problem

• Initial state: empty assignment

• Successor function: a value is assigned to
any unassigned variable, which does not
conflict with the currently assigned variables
• Goal test: the assignment is complete
• Path cost: irrelevant

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 14
Solving CSPs – Backtracking Search
• Bad news: 3SAT is a finite CSP and known to be
NP-complete, so we cannot expect to do better in
the worst case

• Backtracking Search: DFS with single-variable

assignments for a CSP
– Basic uninformed search for solving CSPs
– Gets rid of unnecessary permutations in search tree and
significantly reduces search space:
• Time complexity: reduction from O(dn!) to O(dn)
d … max. number of values of some variable (braching factor)
n … number of variables (depth)
• Sudoku example: order of filling a square does not matter
– […, (2,3)=7, (3,3)=8, …] = […, (3,3)=8, (2,3)=7, …]
– 981 states instead of 981! states

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 15
 Backtracking Search: The Algorithm BT
• These ideas lead to the backtracking search algorithm
BT(Level)
If all variables assigned
PRINT Value of each Variable
RETURN or EXIT (RETURN for more solutions)
(EXIT for only one solution)
V := PickUnassignedVariable()
Variable[Level] := V
Assigned[V] := TRUE
for d := each member of Domain(V) (the domain values of V)
Value[V] := d
for each constraint C such that V is a variable of C
and all other variables of C are assigned:
IF C is not satisfied by the set of current
assignments: BREAK;
ELSE BT(Level+1)
return
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 16
 Backtracking Search

• The algorithm searches a tree of partial assignments.

Children of a node are The root has the empty

all possible values of set of assignments
some (any) unassigned
variable Root {}

Vi=a Vi=b Vi=c

Search stops
Vj=1 Vj=2 descending if the
assignments on
path to the node
Subtree
violate a constraint
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 17
Backtracking Search
•Heuristics are used to determine
– the order in which variables are assigned:
PickUnassignedVariable()
– the order of values tried for each variable.

•The choice of the next variable can vary from

branch to branch, e.g.,
– under the assignment V1=a we might choose to
assign V4 next, while under V1=b we might choose
to assign V5 next.

•This “dynamically” chosen variable ordering has

a tremendous impact on performance.
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 18
Example: N-Queens

• Place N Queens on an N X N chess board so that no

Queen can attack any other Queen.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 19
Example: N-Queens

• Problem formulation:
– N variables (N queens)
– N2 values for each variable representing the
positions on the chessboard

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 20
Example: N-Queens

• Q1 = 1, Q2 = 15, Q3 = 21, Q4 = 32,

Q5 = 34, Q6 = 44, Q7 = 54, Q8 = 59

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 21
Example: N-Queens

• This representation has (N²)N states

(different possible assignments in the
search space)
– For 8-Queens: 648 = 281,474,976,710,656

• Is there a better way to represent the N-

queens problem?
– We know we cannot place two queens in a
single row  we can exploit this fact in the
choice of the CSP representation already

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 22
Example: N-Queens

• Better Modeling:
– N variables Qi, one per row.
– Value of Qi is the column the Queen in row i
is placed; possible values {1, …, N}.

• This representation has NN states:

– For 8-Queens: 88 = 16,777,216

• The choice of a representation can decided

whether or not we can solve a problem!
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 23
Example: N-Queens

• Q1 = 1, Q2 = 7, Q3 = 5, Q4 = 8,
Q5 = 2, Q6 = 4, Q7 = 6, Q8 = 3

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 24
Example: N-Queens

• Constraints:
– Can’t put two Queens in same column
Qi ≠ Qj for all i ≠ j
– Diagonal constraints
|Qi-Qj| ≠ i-j
•i.e., the difference in the values assigned
to Qi and Qj can’t be equal to the
difference between i and j.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 25
Example: N-Queens

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 26
Example: N-Queens

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 27
Example: N-Queens

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 28
Example: N-Queens

Solution!

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 29
Problems with Plain Backtracking
Sudoku: The 3,3 cell has no possible value.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 30
Problems with Plain Backtracking
• In the backtracking search we won’t detect that the
(3,3) cell has no possible value until all variables of the
row/column (involving row or column 3) or the sub-
square constraint (first sub-square) are assigned.
So we have the following situation:

Variable has no possible

value, but we don’t
detect this. Until we try
to assign it a value

• Leads to the idea of constraint propagation

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 31
Constraint Propagation

• Constraint propagation refers to the technique

of “looking ahead” at the yet unassigned
variables in the search .

• Try to detect obvious failures: “Obvious” means

things we can test/detect efficiently.

• Even if we don’t detect an obvious failure we

might be able to eliminate some possible part
of the future search.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 32
Constraint Propagation
• Propagation has to be applied during the
search; potentially at every node of the search
tree.
• Propagation itself is an inference step which
needs some resources (in particular time)
– If propagation is slow, this can slow the search down
to the point where using propagation actually slows
search down!
– There is always a tradeoff between searching fewer
nodes in the search, and having a higher
nodes/second processing rate.

• We will look at two main types of propagation.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 33
Constraint Propagation: Forward Checking

• Forward checking is an extension of

backtracking search that employs a “modest”
amount of propagation (look ahead).

• When a variable is instantiated we check all

constraints that have only one uninstantiated
variable remaining.

• For that uninstantiated variable, we check all

of its values, pruning those values that violate
the constraint.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 34
Forward Checking Algorithm

•For a single constraint C:

FCCheck(C,x)
// C is a constraint with all its variables already
// assigned, except for variable x.
for d := each member of CurDom[x]
IF making x = d together with previous assignments
to variables in scope C falsifies C
THEN remove d from CurDom[V]
IF CurDom[V] = {} then return DWO (Domain Wipe Out)
return ok

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 35
Forward Checking Algorithm
FC(Level) /*Forward Checking Algorithm */
If all variables are assigned
PRINT Value of each Variable
RETURN or EXIT (RETURN for more solutions) (EXIT for only one solution)
V := PickAnUnassignedVariable()
Variable[Level] := V
Assigned[V] := TRUE
for d := each member of CurDom(V)
Value[V] := d
DWOoccured:= False
for each constraint C over V that has one unassigned variable
in its scope (say X).
if(FCCheck(C,X) == DWO) /* X domain becomes empty*/
DWOoccurred:= True /* no point to continue*/
break
if(not DWOoccured) /*all constraints were ok*/
FC(Level+1)
RestoreAllValuesPrunedByFCCheck()
return;
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 36
4-Queens Problem
• Encoding with Q1, …, Q4 denoting a queen per column
– cannot put two queens in same row (instead of same column)

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 37
4-Queens Problem
• Forward checking reduced the domains of all variables that are
involved in a constraint with one uninstantiated variable:
– Here all of Q2, Q3, Q4

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 38
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 39
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}
DWO

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 40
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 41
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 42
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 43
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}
DWO

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 44
4-Queens Problem

• Exhausted the subtree with Q1=1; try now Q1=2

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 45
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 46
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 47
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 48
4-Queens Problem

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 49
4-Queens Problem
• We have now find a solution: an assignment of all variables to
values of their domain so that all constraints are satisfied

Q1 Q2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
Q3 Q4
{1,2,3,4} {1,2,3,4}

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 50
FC: Restoring Values

• After we backtrack from the current

assignment (in the for loop) we must restore
the values that were pruned as a result of that
assignment.

• Some bookkeeping needs to be done, as we

must remember which values were pruned by
which assignment (FCCheck is called at every
recursive invocation of FC).

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 51
FC: Minimum Remaining Values Heuristics (MRV)
• FC also gives us for free a very powerful
heuristic to guide us which variables to try next:
– Always branch on a variable with the smallest
remaining values (smallest CurDom).
– If a variable has only one value left, that value is
forced, so we should propagate its consequences
immediately.
– This heuristic tends to produce skinny trees at the
top. This means that more variables can be
instantiated with fewer nodes searched, and thus
more constraint propagation/DWO failures occur
with less work.
– We can find a inconsistency such as in the Sudoku
example much faster.
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 52
 MRV Heuristic: Human Analogy
• What variables would you try first?
Domain of each variable:
{1, …, 9}

(1, 5): impossible values:

Row: {1, 4, 5, 6, 8}
Column: {1, 3, 4, 5, 7, 9}
Subsquare: {5, 7, 9}
 Domain = {2}

(9, 5): impossible values:

Row: {1, 5, 7, 8, 9}
Column: {1, 3, 4, 5, 7, 9}
Subsquare: {1, 5, 7, 9}
Domain = {2, 6}

After assigning value 2 to

Most restricted variables! = MRV cell (1,5): Domain = {6}
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 53
Example – Map Colouring

• Color the following map using red, green, and

blue such that adjacent regions have different
colors.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 54
Example – Map Colouring

• Modeling
– Variables: WA, NT, Q, NSW, V, SA, T
– Domains: Di={red, green, blue}
– Constraints: adjacent regions must have
different colors.
• E.g. WA ≠ NT

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 55
Example – Map Colouring
• Forward checking idea: keep track of remaining legal
values for unassigned variables.
• Terminate search when any variable has no legal
values.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 56
Example – Map Colouring
• Assign {WA=red}

• Effects on other variables connected by constraints to

WA
– NT can no longer be red
– SA can no longer be red

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 57
Example – Map Colouring
• Assign {Q=green}
• Effects on other variables connected by constraints with Q
– NT can no longer be green
– NSW can no longer be green
– SA can no longer be green
• MRV heuristic would automatically select NT or SA next

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 58
Example – Map Colouring
• Assign {V=blue}
• Effects on other variables connected by constraints with V
– NSW can no longer be blue
– SA is empty

• FC has detected that partial assignment is inconsistent with the

constraints and backtracking can occur.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 59
Empirically

• FC often is about 100 times faster than BT

• FC with MRV (minimal remaining values) often
10000 times faster.
• But on some problems the speed up can be
much greater
– Converts problems that are not solvable to problems
that are solvable.
• Other more powerful forms of consistency are
commonly used in practice.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 60
Constraint Propagation: Arc Consistency
•Another form of propagation:
make each arc consistent
– C(X,Y) is consistent iff for every value of X there
is some value of Y that satisfies C.
– Idea: ensure that every binary constraint is
satisfiable (2-consistency)
• Binary constraints = arcs in the constraint graph
• Remember: All higher-order constraints can be
expressed as a set of binary constraints

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 61
Constraint Propagation: Arc Consistency
• Can remove values from the domain of variables:
– e.g. C(X,Y): X>Y Dom(X)={1,5,11} Dom(Y)={3,8,15}
• For X=1 there is no value of Y s.t. 1>Y => remove 1 from domain X
• For Y=15 there is no value of X s.t. X>15, so remove 15 from domain Y
• We obtain more restricted domains Dom(X)={5,11} and Dom(Y)={3,8}
– Have to try much fewer values in the search tree.

• Removing a value from a domain may trigger further

inconsistency, so we have to repeat the procedure until
everything is consistent.
– For efficient implementation, we keep track of inconsistent arcs by
putting them in a Queue (See AC3 algorithm in the book).

• This is stronger than forward checking. why?

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 62
Arc Consistency – Map Colouring Example

• Since NSW loses a value, we need to recheck all

constraints involving NSW: other neighbours are Q, V
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 63
Arc Consistency – Map Colouring Example

ok

• Since V loses a value, we need to recheck all

constraints involving V: other neighbours are SA
•Recheck all constraints involving SA

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 64
Arc Consistency – Map Colouring Example

ok

• (SA, NT) is not satisfiable any longer – we

detected an unavoidable failure in the assignment
{WA=red, Q=green}
– Forward checking would have detected it as well. Why?

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 66
 Arc-Consistency – Example

• CSP with 3 variables X, Y, Z

– Domains:
• Dom(X) = {1, · · · , 10}
• Dom(Y) = {5, · · · , 15}
• Dom(Z) = {5, · · · , 20}

– Constraints:
• C(X,Y): X > Y
• C(Y,Z): Y + Z = 12
• C(X,Z): X + Z = 16

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 67
Arc Consistency – Example

• Draw the constraint graph. Dom(X) = {1, · · · , 10}

Dom(Y) = {5, · · · , 15}
Dom(Z) = {5, · · · , 20}

X Y C(X,Y): X > Y
C(Y,Z): Y + Z = 12
C(X,Z): X + Z = 16
Z

• Are the constraints arc consistent?

C(X,Y) is consistent iff for every value of X

there is some value of Y that satisfies C.

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 68
Arc Consistency – Example

• Apply arc consistency method repeatedly

so they become arc consistent.
– For X=1,2,3,4,5 there is no value of Y s.t. X>Y
=> remove 1,2,3,4,5 from domain X
– For Y>7 there is no value of Z s.t. Y+Z=12
=> remove 8,…,15 from domain Y
– For Z>7 there is no value of Y s.t. Y+Z=12
=> remove 8,…,20 from domain Z
– For X=6,7,8 there is no value of Z s.t.
X+Z=16 => remove 6,7,8 from domain X Dom(X) = {9{6,
{1,,·10}
· · , 10}
– For Z=5 there is no value of X s.t. Dom(Y) = {5, ·6}
6,· 7}
· , 15}
Dom(Z) = {6
{5,,·6,
7}
· 7}
· , 20}
X+Z=16 => remove 5 from domain Z
– For Y=7 there is no value of Z s.t. C(X,Y): X > Y
Y+Z=12 => remove 7 from domain Y C(Y,Z): Y + Z = 12
C(X,Z): X + Z = 16
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 69
Other forms of Consistency
• Generalized arc consistency: for an n-ary constraint, for
each value of the domain of a variable, there exists a tuple
of values for the remaining n-1 variables that satisfy the
constraint
• Path consistency: a pair (V1, V2) is path-consistent with
respect to a third variable V3 if for every assignment
{V1=a, V2=b} consistent with all binary constraints of
{V1, V2}, there is an assignment to V3 that satisfies all
binary constraints on {V1, V3} and {V2, V3}.
– Think of V3 being on a path of length 2 from V1 to V2

• Strong k-consistency
– k-consistent, (k-1)-consistent, etc.
– Very expensive: any algorithm establishing k-consistency requires
exponential time and space in k
– In practical solvers: 2-consistency, sometimes 3-consistency

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 70
Many real-world applications of CSP

• Assignment problems
– who teaches what class
• Timetabling problems
– exam schedule
• Transportation scheduling
• Floor planning
• Factory scheduling
• Hardware configuration
– a set of compatible components
Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 71
CSP Solvers
• Much work on various heuristics for variable and value selection
• Fourth CSP Solver Competition Results 2009,
Category: Only binary constraints

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 72
 CSP Solvers

Time to solve an instance Time to solve an instance

(SAT answers, category 2-ARY-EXT) (UNSAT answers, category 2-ARY-EXT)
1800 1800

1600 1600

1400 1400

1200 1200
CPU time (s)

CPU time (s)

1000 1000

800 800

600 600

400 400

200 200

0 0
0 50 100 150 200 250 300 350 0 50 100 150 200 250
number of solved instances number of solved instances

Abscon 112v4 AC Conquer 2009-07-10 Abscon 112v4 AC Conquer 2009-07-10

Abscon 112v4 ESAC Mistral 1.545 Abscon 112v4 ESAC Mistral 1.545
bpsolver 09 pcs 0.3.2 bpsolver 09 pcs 0.3.2
Choco2.1.1 2009-06-10 pcs-restart 0.3.2 Choco2.1.1 2009-06-10 pcs-restart 0.3.2
Choco2.1.1b 2009-07-16 SAT4J CSP 2.1.1 Choco2.1.1b 2009-07-16 SAT4J CSP 2.1.1
Concrete 2009-07-14 Sugar v1.14.6+minisat Concrete 2009-07-14 Sugar v1.14.6+minisat
Concrete DC 2009-07-14 Sugar v1.14.6+picosat Concrete DC 2009-07-14 Sugar v1.14.6+picosat

Torsten Hahmann, CSC384 Introduction to Artificial Intelligence, University of Toronto, Fall 2011 73