Constraint Satisfaction

Problems (CSP)

Artificial Intelligence
Dr. Manel Mrabet
References
• Stuart Russell and Peter Norvig; "Artificial Intelligence: A Modern
Approach", Prentice Hall, Latest Edition
• Constraint satisfaction problems, University of Berkeley

Dr. Manel Mrabet

 Constraint Satisfaction Problems
(CSPs)
• Standard search problem:
• state is a "black box“ – any data structure that supports successor function,
heuristic function, and goal test

• CSP:
• state is defined by variables Xi with values from domain Di
• goal test is a set of constraints specifying allowable combinations of values
for subsets of variables

Dr. Manel Mrabet

 Constraint satisfaction problem
• A CSP is defined by:
• a set of variables Xi
• a domain of possible values for each variable Di
• a set of constraints C between variables

• The aim is to find an assignment that does not violate any

constraints (called a consistent or legal CONSISTENT
assignment).

• A complete assignment is one in which every variable is

mentioned.

• A solution to a CSP is
• A complete assignment that satisfies all the constraints
Dr. Manel Mrabet
 Example: Map-Coloring

WA

• Variables WA, NT, Q, NSW, V, SA, T

• Domains Di = {red, green, blue}
• Constraints: adjacent regions must have different colors
• e.g., WA ≠ NT

Dr. Manel Mrabet

 Example: Map-Coloring

• Solutions are complete and consistent assignments, e.g.,

WA = red, NT = green, Q = red, NSW = green, V = red, SA = blue, T = green

• A state may be incomplete e.g., just WA=red

Dr. Manel Mrabet
 Example: n-Queens Puzzle

 Put n queens on an n-by-n chess board so that no two

queens are attacking each other.
 For 8-Queens there are 16,777,216 configurations.
Dr. Manel Mrabet
 Example: Cryptarithmetic

 Variables: Constraints:
DEMNORSY M≠0, S ≠0 (unary constaraints)
Y=D+E or Y=D+E-10 , etc
D ≠E, D ≠M, D ≠N
 Domains:
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

Dr. Manel Mrabet

 Standard search formulation

Let’s try the standard search formulation.

We need:
 Initial state: none of the variables has a value
(color)
 Successor function: assign a value to an
unassigned variable that does not conflict with
previously assigned variables.
 fail if no legal assignments (not fixable!)
 Goal: all variables have a value and none of the
constraints is violated.
Dr. Manel Mrabet
 Constraint graph

• It is helpful to visualize a CSP as a constraint graph

• Binary CSP: each constraint relates two variables
• Constraint graph: nodes are variables, arcs are constraints

NT Q

WA
NSW

SA

T
Dr. Manel Mrabet
 Varieties of CSPs
Discrete variables
• finite domains:
• n variables, domain size d  O(dn) complete assignments
• e.g., Boolean CSPs, incl. Boolean satisfiability (NP-complete)
• infinite domains:
• integers, strings, etc.
• e.g., job scheduling, variables are start/end days for each job
• need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3

Continuous variables
• e.g., start/end times for Hubble Space Telescope observations
• linear constraints solvable in polynomial time by linear
programming

Dr. Manel Mrabet

 Varieties of constraints

• Unary constraints involve a single variable,

e.g., SA ≠ green
• Binary constraints involve pairs of variables,
e.g., SA ≠ WA

• Higher-order constraints involve 3 or more

variables,
e.g. Y = D+E or Y = D+E −10
• Inequality constraints on Continuous variables
EndJob1 + 5 ≤ StartJob3

Dr. Manel Mrabet

Constraint Satisfaction
problem
Backtracking Search

Dr.Walid Aydi
 Backtracking search

• Every solution appears at depth n with n variables

 use depth-first search

• Depth-first search for CSPs with single-variable

assignments is called backtracking search

• Backtracking search is the basic uninformed

algorithm for CSPs

• Can solve n-queens for n ≈ 25

Dr. Manel Mrabet
 Backtracking (Depth-First) search
• Special property of CSPs: They are commutative:
 [WA = red then NT = green] same as [NT = green then WA =
red]
This means: the order in which we assign variables
does not matter.
• Better search tree: First order variables, then assign them
values one-by-one.

Dr. Manel Mrabet

Backtracking search algorithm

Dr. Manel Mrabet

Backtracking
Example

Dr.Walid Aydi
Example of a csp

A B

E
C

F G

Dr. Manel Mrabet

Example of a csp

{violet, brown, red} {violet, brown, red}

A B
{violet, brown, red}

E
C {violet, brown,
red}

D {violet, brown,
red}

H
{violet, brown, red}
F G {violet, brown, red}
{violet, brown, red}

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

Dead end → backtrack

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Example of a csp

A B

E
C

H Solution !!!!

F G

1 = red
2 = violet
3 = BROWN

Dr. Manel Mrabet

Improving backtracking efficiency
• General-purpose heuristics can give huge gains in
speed:

• Which variable should be assigned next?

• In what order should its values be tried?
• Can we detect inevitable failure early?

Dr. Manel Mrabet

 Minimum Remaining Values (MRV)
For variable selection

• Also known as: Most Constrained Variable heuristic

• Heuristic Rule: choose variable with fewest legal moves

• I.e., with the fewest number of legal values in its domain
• E.g., will immediately detect failure if X has no legal values

Reduce the search branching factor now!!

The variable with the minimum remaining values in its
domain has the minimum search branching factor now.
Dr.Walid Aydi
Detailed Example of MRV

Dr.Walid Aydi
 Detailed Example of MRV

•Initially, all regions have MRV = 3

•Choose one randomly (e.g., WA=red)
•Do Forward Checking (next topic)

MRV=3 MRV=2 MRV=3

MRV=3

MRV=3 MRV=2
MRV=3 MRV=3 MRV=3

MRV=3 MRV=3

MRV=3 MRV=3
Dr. Manel Mrabet
Detailed Example of MRV

Dr.Walid Aydi
 Detailed Example of MRV

• Next, NT and SA both have MRV = 2

• Choose one randomly (e.g., NT=green)
• Do Forward Checking (next topic)

MRV=2 MRV=3 MRV=2

MRV=2 MRV=1
MRV=3 MRV=3

MRV=3 MRV=3

MRV=3 MRV=3
Dr. Manel Mrabet
Detailed Example of MRV

Dr.Walid Aydi
 Detailed Example of MRV

• Next, SA has MRV = 1 (must be blue)

• Choose it and assign it (i.e., SA=blue)
• Do Forward Checking (next topic)

MRV=2 MRV=1

MRV=1
MRV=3 MRV=2

MRV=3 MRV=2

MRV=3 MRV=3
Dr. Manel Mrabet
Detailed Example of MRV

Dr.Walid Aydi
 Degree Heuristic (DH)
(Caveat for tests, quizzes, and exams)
• This example works DH as a tie-breaker for MRV,
which is its usual role in practice.

• On tests, you may be asked to work DH ignoring

MRV, to prove that you know it.

• DH seeks the maximum degree in the constraint

graph to unassigned nodes.
• Whether or not it is a tie-breaker for MRV.

Dr. Manel Mrabet

 Degree heuristic (DH)
For variable selection (often a tie-breaker after MRV)

• Heuristic Rule: select variable that is involved in the largest number of

constraints with other unassigned variables.

• Degree heuristic very often used as a tie-breaker among variables that

have equal Minimum Remaining Values.

Dr.Walid Aydi
 Detailed Example of DH
(As a tie-breaker after MRV)

Dr.Walid Aydi
 Detailed Example of DH
(As a tie-breaker after MRV)
• Initially, all regions have MRV = 3
• SA has DH=5 (e.g., SA=blue)
• Do Forward Checking (next topic)

MRV=3 MRV=2
DH=3 MRV=3 DH=2 MRV=2
DH=3 DH=2
MRV=3
MRV=3 MRV=3 MRV=2 MRV=2
DH=5
DH=2 DH=3 DH=1 DH=2
MRV=3 MRV=2
DH=2 DH=1

MRV=3 Dr. Manel Mrabet

MRV=3
DH=0 DH=0
 Detailed Example of DH
(As a tie-breaker after MRV)

Dr.Walid Aydi
 Detailed Example of DH
(As a tie-breaker after MRV)

• NT, Q, and NSW have MRV = 2, DH = 2

• Choose one randomly (e.g., NT=green)
• Do Forward Checking (next topic)

MRV=2
DH=2 MRV=2 MRV=1
DH=2 DH=1
MRV=2 MRV=2 MRV=1 MRV=2
DH=1 DH=2 DH=0 DH=2
MRV=3 MRV=2
DH=2 DH=1

MRV=3 Dr. Manel Mrabet

MRV=3
DH=0 DH=0
 Detailed Example of DH
(As a tie-breaker after MRV)

Dr.Walid Aydi
 Detailed Example of DH
(As a tie-breaker after MRV)
• WA and Q have MRV = 1, Q has DH = 1
• Choose Q and assign it (i.e., Q=red)
• Do Forward Checking (next topic)

MRV=1
DH=1
MRV=1 MRV=2 MRV=1 MRV=1
DH=0 DH=2 DH=0 DH=1
MRV=3 MRV=2
DH=2 DH=1

MRV=3 Dr. Manel Mrabet

MRV=3
DH=0 DH=0
 Detailed Example of DH
(As a tie-breaker after MRV)

Dr.Walid Aydi
 least constraining value heuristic

• Given a variable, choose the least constraining value:

• the one that rules out the fewest values in the remaining variables

• Leaves maximal flexibility for a solution.

• Combining these heuristics makes 1000 queens feasible

Dr. Manel Mrabet

 Exercise

2) Consider the constraint graph on the right.

The domain for every variable is [1,2,3,4].
There are 2 unary constraints:
- variable “a” cannot take values 3 and 4.
- variable “b” cannot take value 4.
There are 8 binary constraints stating that variables
connected by an edge cannot have the same value.
b
a) [5pts] Find a solution for this CSP by using the following
heuristics: minimum value heuristic, degree heuristic,
forward checking. Explain each step of your answer.
a c e

Dr. Manel Mrabet

2) Consider the constraint graph on the right.
The domain for every variable is [1,2,3,4].
There are 2 unary constraints:
- variable “a” cannot take values 3 and 4.
- variable “b” cannot take value 4.
There are 8 binary constraints stating that variables
connected by an edge cannot have the same value.
b
a) [5pts] Find a solution for this CSP by using the following
heuristics: minimum value heuristic, degree heuristic,
forward checking. Explain each step of your answer.
a c e
MVH a=1 (for example)
FC+MVH b=2
FC+MVH c=3 d
FC+MVH d=4
FC e=1
Dr. Manel Mrabet
Summary
• CSPs are a special kind of problem:
• states defined by values of a fixed set of variables
• goal test defined by constraints on variable values

• Backtracking = depth-first search with one variable assigned per node

• Variable ordering and value selection heuristics help significantly

• Forward checking prevents assignments that guarantee later failure

• Constraint propagation (e.g., arc consistency) does additional work to

constrain values and detect inconsistencies

• Iterative min-conflicts is usually effective in practice

Dr. Manel Mrabet