Constraint Satisfaction Problems
Dr. Shahnawaz Qureshi
Standard Search Problems
Previous search problems:
Problems can be solved by searching in a space
of states
state is a “black box” – any data structure that
supports successor function, heuristic function,
and goal test – problem-specific
Constraint satisfaction problem
states and goal test conform to a standard,
structured and simple representation
general-purpose heuristic
2
Constraint Satisfaction Problems (CSP)
CSP is defined by 3 components (X, D, C):
state: a set of variables X, each Xi , with values from
domain Di
goal test: a set of constraints C, each Ci involves some
subset of the variables and specifies the allowable
combinations of values for that subset
Each constraint Ci consists of a pair <scope, rel>, where
scope is a tuple of variables and rel is the relation, either
represented explicitly or abstractly
X1 and X2 both have the domain {A, B}
Constraints:
<(X1, X2), [(A, B), (B, A)]>, or
<(X1, X2), X1 ≠ X2>
3
Solution
Each state in a CSP is defined by an assignment of
values to some or all of the variables
An assignment that does not violate any constraints
is called a consistent or legal assignment
A complete assignment is one in which every
variable is assigned
A solution to a CSP is consistent and complete
assignment
Allows useful general-purpose algorithms with more
power than standard search algorithms
4
Example: Map Coloring
Variables: X = {WA, NT, Q, NSW, V, SA, T }
Domains: Di = {red, green, blue}
Constraints: adjacent regions must have different colors
Solution?
5
Solution: Complete and Consistent Assignment
Variables: X = {WA, NT, Q, NSW, V, SA, T }
Domains: Di = {red, green, blue}
Constraints: adjacent regions must have different colors
Solution? {WA = red, NT = green, Q = red, NSW = green, V = red,
SA = blue, T = red}.
6
Constraint Graph
Constraint graph: nodes are variables, arcs are
constraints
Binary CSP: each constraint relates two variables
CSP conforms to a standard pattern
a set of variables with assigned values
generic successor function and goal test
generic heuristics
reduce complexity
7
CSP as a Search Problem
Initial state:
{} – all variables are unassigned
Successor function:
a value is assigned to one of the unassigned variables with
no conflict
Goal test:
a complete assignment
Path cost:
a constant cost for each step
Solution appears at depth n if there are n variables
Depth-first or local search methods work well
8
CSP Solvers Can be Faster
CSP solver can quickly eliminate large part of
search space
If {SA = blue}
Then 35 assignments can be reduced to 25
assignments, a reduction of 87%
In a CSP, if a partial assignment is not a solution,
we can immediately discard further refinements of it
9
Exercise (1)
Consider the problem of
crossword puzzle: fitting words
into a rectangular grid. Assume
that a list of words is provided
and that the task is to fill in the
blank squares using any subset of
the list. Formulate this problem
precisely in two ways:
As a general search problem. Choose
an appropriate search algorithm.
As a constraint satisfaction problem.
Word vs. letters
10
Exercise (2)
Problem formulation as CSP:
Class scheduling: There is a fixed number of
professors and classrooms, a list of classes to be
offered, and a list of possible time slots for
classes. Each professor has a set of classes that
he or she can teach.
11
Types of Variables
Discrete variables
finite domains:
n variables, domain size d O (d n) complete assignments
e.g., Boolean CSPs, such as 3-SAT (NP-complete)
Worst case, can’t solve finite-domain CSPs in less than
exponential time
infinite domains:
integers, strings, etc.
e.g., job scheduling, variables are start/end days for each job
need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3
Continuous variables
e.g., start/end times for Hubble Space Telescope
observations
linear constraints solvable in polynomial time by linear
programming
12
Types of Constraints
Unary constraints involve a single variable,
e.g., SA ≠ green
Binary constraints involve pairs of variables,
e.g., SA ≠ WA
Higher-order constraints involve 3 or more
variables
e.g., cryptarithmetic column constraints
13
Real-World CSPs
Assignment problems
e.g., who teaches what class
Timetabling problems
e.g., which class is offered when and where?
Transportation scheduling
Factory scheduling
14
What Search Algorithm to Use?
Since we can formulate CSP problems as standard
search problems, we can apply search algorithms
from chapter 3 & 4
If breadth-first search were applied
branching factor? nd
tree size? nd * (n-1)d * … * d = n! * dn leaves
complete assignments? dn
A crucial property to all CSPs: commutativity
the order of application of any given set of actions has no
effect on the outcome
Variable assignments are commutative, i.e., [ WA = red
then NT = green ] same as [ NT = green then WA = red ]
15
Backtracking Search
Only need to consider assignments to a single
variable at each node b = d and there are d n
leaves
Backtracking search is used for a depth-first search
that chooses values for one variable at a time and
backtracks when a variable has no legal values left
to assign
Backtracking search is the basic uninformed
algorithm for CSPs
16
Backtracking Search Algorithm
17
Backtracking Example
18
Backtracking Example
19
Backtracking Example
20
Backtracking Example
21
Improving Backtracking Efficiency
We can solve CSPs efficiently without
domain-specific knowledge, addressing the
following questions:
in what order should variables be assigned,
values be tried?
what are the implications of the current variable
assignments for the other unassigned variables?
when a path fails, can the search avoid repeating
this failure?
22
Variable and Value Ordering 1/3
Minimum remaining values (MRV)
choose the variable with the fewest “legal”
values
also called most constrained variable or
fail-first heuristic
does it help in choosing the first variable?
23
Variable and Value Ordering 2/3
Most constraining variable:
selecting the variable that has the largest
number of constraints on other
unassigned variables
also called degree heuristics
Tie-breaker among MRV
24
Variable and Value Ordering 3/3
Given a variable, choose the least
constraining value:
the one that rules out the fewest values in the
remaining variables
Combining these heuristics makes 1000
queens feasible
25
Propagating Information through Constraints
Forward checking:
Keep track of remaining legal values for unassigned
variables
Terminate search when any variable has no legal values
26
Propagating Information through Constraints
Forward checking:
Keep track of remaining legal values for unassigned
variables
Terminate search when any variable has no legal values
27
Propagating Information through Constraints
Forward checking:
Keep track of remaining legal values for unassigned
variables
Terminate search when any variable has no legal values
28
Propagating Information through Constraints
Forward checking:
Keep track of remaining legal values for unassigned
variables
Terminate search when any variable has no legal values
29
Propagating Information through Constraints
Forward checking propagates information from assigned to
unassigned variables, but doesn't provide early detection for
all failures:
NT and SA cannot both be blue!
Constraint propagation repeatedly enforces constraints locally
by propagating implications of a constraint of one variable
onto other variables
30
Node Consistency
A single variable is node-consistent if all the
values in the variable’s domain satisfy the
variable’s unary constraints
For example, SA dislikes green
A network is node-consistent if every
variable in the network is node-consistent
31
Arc Consistency
Simplest form of propagation makes each arc consistent
X Y is consistent iff
for every value x of X there is some allowed y
32
Arc Consistency
Simplest form of propagation makes each arc consistent
X Y is consistent iff
for every value x of X there is some allowed y
33
Arc Consistency
Simplest form of propagation makes each arc consistent
X Y is consistent iff
for every value x of X there is some allowed y
If X loses a value, neighbors of X need to be rechecked
34
Arc Consistency
Simplest form of propagation makes each arc consistent
X Y is consistent iff
for every value x of X there is some allowed y
If X loses a value, neighbors of X need to be rechecked
Arc consistency detects failure earlier than forward checking
Can be run as a preprocessor or after each assignment
35
AC-3 Algorithm
Time complexity O (n 2d 3)
36
Example
Use the AC-3 algorithm to show that arc consistency is
able to detect the inconsistency of the partial
assignment {WA = red, V = blue} in the following map
coloring problem.
37
One Possible Trace
One possible trace of the algorithm:
remove SA-WA, delete R from SA
remove SA-V, delete B from SA, leaving only G
remove NT-WA, delete R from NT
remove NT-SA, delete G from NT, leaving only B
remove NSW-SA, delete G from NSW
remove NSW-V, delete B from NSW, leaving only R
remove Q-NT, delete B from Q
remove Q-SA, delete G from Q
remove Q-NSW, delete R from Q, leaving no proper assignment
for Q
38
Path Consistency
Consider map-coloring with only two colors
Every arc is consistent initially
Check the set {WA, SA} path consistenty
with respect to NT
More generally, k-consistency
1-consistency = node consistency
2-consistency = arc consistency
3-consistency = path consistency
39
Local Search for CSPs
Hill-climbing, simulated annealing typically work
with "complete" states, i.e., all variables assigned
To apply to CSPs:
allow states with unsatisfied constraints
operators reassign variable values
Variable selection: randomly select any conflicted
variable
Value selection by min-conflicts heuristic:
choose value that violates the fewest constraints
i.e., hill-climb with h(n) = total number of violated
constraints
40
Example: 4-Queens
States: 4 queens in 4 columns (44 = 256 states)
Actions: move queen in column
Goal test: no attacks
Evaluation: h(n) = number of attacks
41
8-Queen Example
Min-conflicts is quite effective for many CSPs.
Given a random initial state, can solve n-queens in
almost constant time for arbitrary n with high
probability (e.g., n = 10,000,000)
42
Min-Conflicts Algorithms for CSP
43
Summary
CSPs are a special kind of problem:
states defined by values of a fixed set of variables
goal test defined by constraints on variable values
Backtracking = depth-first search with one variable assigned
per node
Variable ordering and value selection heuristics help
significantly
Forward checking prevents assignments that guarantee later
failure
Constraint propagation (e.g., arc consistency) does additional
work to constrain values and detect inconsistencies
Iterative min-conflicts is usually effective in practice
44
