CONSTRAINT SATISFACTION

PROBLEM
14S3205 - Artificial Intelligence
Guntur Petrus Boy Knight

Teknik Elektro
Fakultas Informatika dan Teknik Elektro
Institut Teknologi Del
 OUTLINE
• Constraint satisfaction problems
• CSP as a search problem
• Backtracking
• Forward Checking
• Constraint propagation in CSPs
• Arc-consistency
 INTRODUCTION
• Previous chapters explored the idea that problems can be solved by
searching in a space of states.
• These states can be evaluated by domain-specific heuristics and tested to
see whether they are goal states
• From the point of view of the search algorithm, however, each state is
atomic, or indivisible—a black box with no internal structure
• We can solve a wide variety of problems more efficiently, using factored
representation for each state: a set of variables, each of which has a value
• A problem is solved when each variable has a value that satisfies all the
constraints on the variable
• A problem described this way is called a constraint satisfaction problem, or
CSP
 MAIN IDEA
• Represent the constraints that solutions must satisfy in a uniform declarative
language
• Find solutions by GENERAL PURPOSE search algorithms with no changes from
problem to problem
• If we specify the problem in a formal declarative language, a general-
purpose algorithm will do everything else
 CONSTRAINT SATISFACTION
PROBLEMS
• A CSP consists of:
• 𝐹𝑖𝑛𝑖𝑡𝑒 𝑠𝑒𝑡 𝑜𝑓 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒𝑠 𝑋1 , 𝑋2 , … , 𝑋𝑛
• Nonempty domain of possible values for each variable
𝐷1 , 𝐷2 , … , 𝐷𝑛 𝑤ℎ𝑒𝑟𝑒 𝐷𝑖 = 𝑣1 , 𝑣2 , … , 𝑣𝑘
• 𝐹𝑖𝑛𝑖𝑡𝑒 𝑠𝑒𝑡 𝑜𝑓 𝑐𝑜𝑛𝑠𝑡𝑟𝑎𝑖𝑛𝑡𝑠 𝐶1 , 𝐶2 , … , 𝐶𝑚
• A consistent assignment does not violate the constraints.
• Example problem: Sudoku
 CONSTRAINT IN SUDOKU
In 1 row every
cell must have
different
number

In 1 square (sub-
region) every cell
must have different
number

In 1 colomn
every cell must
have different
number
 CONSTRAINT SATISFACTION
PROBLEMS
• An assignment is complete when every variable is assigned a value.
• An assignment that does not violate any constraints is called a consistent or
legal assignment
• A partial assignment is one that assigns values to only some of the variables.
• A solution to a CSP is a complete, consistent assignment.
• Solutions to CSPs can be found by a completely general-purpose algorithm,
given only the formal specification of the CSP.
 APPLICATIONS
• Map coloring
• Scheduling problems
• Class scheduling in ITDel
• e.g., which class is offered when and where?
• Scheduling the Hubble Space Telescope
• Bill Gates’ project using computer
• Sudoku
• Circuit layout
• …
 EXAMPLE OF CSP: MAP
COLORING
• Variables: WA, NT, Q, NSW, V, SA, T
• Domains: 𝐷𝑖 = {red, green, blue}
• Constraints: adjacent regions must have
different colors
• e.g., WA ≠ NT
• So (WA,NT) must be in {(red, green),(red, blue),
(green, red), …}
 EXAMPLE OF CSP: MAP
COLORING
• Solutions : Complete and consistent
assignments
• E.g. WA = red, NT = green,Q = red, NSW = green,
V = red, SA = blue, T = green
 EXAMPLE OF CSP: N-QUEENS
• Variables: 𝑄𝑘
• Domains: 𝐷𝑖 = {1, 2, …, N}
• Constraints: each queen must not at a
vulnerable position relative to other queens
• Implicit: ∀𝑖, 𝑗 𝑛𝑜𝑛 − 𝑡ℎ𝑟𝑒𝑎𝑡𝑒𝑛𝑖𝑔 (𝑄𝑖 , 𝑄𝑗 )
• Explicit: 𝑄𝑖 , 𝑄𝑗 ∈ { 1,3 , 1,4 , … }
…
 EXAMPLE OF CSP: N-QUEENS
• Variables: 𝐸𝑎𝑐ℎ (𝑜𝑝𝑒𝑛) 𝑠𝑞𝑢𝑎𝑟𝑒
• Domains: 𝐷𝑖 = {1, 2, …, 9}
• Constraints:
• 9-way alldiff for each column
• 9-way alldiff for each row
• 9-way alldiff for each square
 BENEFITS OF CSP
• Clean specification of many problems, generic goal, successor function &
heuristics
• CSP “knows” which variables violate a constraint and hence where to focus
the search
• CSPs: Automatically prune off all branches that violate constraints
• State space search could do this only by hand-building constraints into the
successor function
 CSP REPRESENTATIONS
• Constraint graph:
• nodes are variables
• arcs are (binary) constraints
• Standard representation pattern:
• variables with values
• Constraint graph simplifies search
• e.g. Tasmania is an independent
subproblem.
• Map coloring problem is an example of
binary CSP
• each constraint relates two variables
 VARIETIES OF CSPS
• Discrete variables
• finite domains:
• n variables, domain size 𝑑 → 𝑂(𝑑 𝑛 ) complete assignments
• e.g., Boolean CSPs, includes Boolean satisfiability (NP-complete)
• infinite domains:
• integers, strings, etc.
• e.g., job scheduling, variables are start/end days for each job
• need a constraint language,
• e.g., StartJob1 + 5 ≤ StartJob3

• Continuous variables
• e.g., start/end times for Hubble Space Telescope observations
 VARIETIES OF CONSTRAINTS
• Unary constraints involve a single variable
• e.g., SA ≠ green
• Binary constraints involve pairs of variables
• e.g., SA ≠ WA
• Higher-order constraints involve 3 or more variables
• e.g., sudoku
• Preference (soft constraints) e.g. red is better than green can be represented
by a cost for each variable assignment
 CSP AS A SEARCH PROBLEM
• A CSP can easily be expressed as a search problem
• Initial State: the empty assignment {}.
• Successor function: Assign value to any unassigned variable provided that there
is not a constraint conflict.
• Goal test: the current assignment is complete.
• Path cost: a constant cost for every step.
• Solution is always found at depth n, for n variables
• Hence Depth First Search can be used
 SEARCH AND BRANCHING
FACTOR

• n variables of domain size d

• Branching factor at the root is n*d
• Branching factor at next level is (n-1)*d
• Tree has n!*𝒅𝒏 leaves
 SEARCH AND BRANCHING
FACTOR
• The variable assignments are
commutative
• Eg [ step 1: WA = red; step 2: NT =
green ]
• equivalent to [ step 1: NT = green; step
2: WA = red ]
• Therefore, a tree search, not a graph
search
• Only need to consider assignments to
a single variable at each node
• b = d and there are 𝒅𝒏 leaves (n
variables, domain size d )
 SEARCH AND BACKTRACKING
• Depth-first search for CSPs with single-variable assignments is called
backtracking search
• The term backtracking search is used for a depth-first search that chooses
values for one variable at a time and backtracks when a variable has no
legal values left to assign.
• Backtracking search is the basic uninformed algorithm for CSPs
BACKTRACKING EXAMPLE
 IMPROVING BACKTRACKING
EFFICIENCY
• General-purpose methods & general-purpose heuristics can give huge gains
in speed, on average
• Heuristics:
• Q: Which variable should be assigned next?
1. Most constrained variable
2. (if ties:) Most constraining variable
• Q: In what order should that variable’s values be tried?
3. Least constraining value
• Q: Can we detect inevitable failure early?
4. Forward checking
 HEURESTIC 1:
MOST CONSTRAINED VARIABLE

• Choose a variable with the fewest legal values

• a.k.a. minimum remaining values (MRV) heuristic

 HEURESTIC 2:
MOST CONSTRAINING VARIABLE
• Tie-breaker among most constrained variables
• Choose the variable with the most constraints on remaining variables

These two heuristics together

lead to immediate solution of
our example problem
 HEURESTIC 3:
LEAST CONSTRAINING VALUE
• Given a variable, choose the least constraining value:
• the one that rules out the fewest values in the remaining variables

Note: demonstrated here

independent of the
other heuristics
 HEURISTIC 4: FORWARD
CHECKING
Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any unassigned variable has no remaining legal
values

New data (A first step towards Arc Consistency & AC-3)

structure
 FORWARD CHECKING
Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any unassigned variable has no remaining legal
values
 FORWARD CHECKING
Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any unassigned variable has no remaining legal
values
 FORWARD CHECKING
Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any unassigned variable has no remaining legal
values

Terminate! No possible value for SA

 EXAMPLE: 4-QUEENS PROBLEM
• How to assign 4 queens to a 4x4 chess board

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} {1,2,3,4}
2

3 Q3 Q4
{1,2,3,4} {1,2,3,4}
4
 EXAMPLE: 4-QUEENS PROBLEM
• Assign value to unassigned variable

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} {1,2,3,4}
2

3 Q3 Q4
{1,2,3,4} {1 ,2,3, 4}
4
 EXAMPLE: 4-QUEENS PROBLEM
• Forward check

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , ,3,4}
2

3 Q3 Q4
{ ,2, ,4} { ,2,3, }
4
 EXAMPLE: 4-QUEENS PROBLEM
• Assign value to unassigned variable

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , ,3,4}
2

3 Q3 Q4
{ ,2, ,4} { ,2,3, }
4
 EXAMPLE: 4-QUEENS PROBLEM
• Forward check

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , ,3,4}
2

3 Q3 Q4
{ , , , } { ,2, , }
4
 EXAMPLE: 4-QUEENS PROBLEM
• Backtrack

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , ,3,4}
2

3 Q3 Q4
{ , , , } { ,2, , }
4
 EXAMPLE: 4-QUEENS PROBLEM
• Picking up a little later after two steps of backtracking
• Assign value to unassigned variable

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} {1,2,3,4}
2

3 Q3 Q4
{1,2,3,4} {1,2,3,4}
4
 EXAMPLE: 4-QUEENS PROBLEM
• Forward Check

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , , ,4}
2

3 Q3 Q4
{1, ,3, } {1, ,3,4}
4
 EXAMPLE: 4-QUEENS PROBLEM
• Assign value to unassigned variable

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , , ,4}
2

3 Q3 Q4
{1, ,3, } {1, ,3,4}
4
 EXAMPLE: 4-QUEENS PROBLEM
• Forward Check

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , , ,4}
2

3 Q3 Q4
{1, , , } {1, ,3, }
4
 EXAMPLE: 4-QUEENS PROBLEM
• Assign value to unassigned variable

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , , ,4}
2

3 Q3 Q4
{1, , , } { , ,3, }
4
 EXAMPLE: 4-QUEENS PROBLEM
• Forward Check

Q1 Q2 Q3 Q4
Q1 Q2
1
{1,2,3,4} { , , ,4}
2

3 Q3 Q4
{1, , , } { , ,3, }
4
 TOWARDS CONSTRAINT
PROPAGATION
• Forward checking propagates information from assigned to unassigned
variables, but doesn't provide early detection for all failures:

• NT and SA cannot both be blue!

• Constraint propagation goes beyond forward checking & repeatedly
enforces constraints locally
 CONSTRAINT PROPAGATION
(INFERENCE) IN CSPS
• CSP solvers combine search and inference
• Search
• assigning a value to a variable
• Constraint propagation (inference)
• Eliminates possible values for a variable if the value would violate local consistency
• Can do inference first, or intertwine it with search
• Local consistency
• Node consistency: satisfies unary constraints
• Arc consistency: satisfies binary constraints
• 𝑋𝑖 is arch-consistent with regards to 𝑋𝑗 if for every value v in 𝑫𝒊 there is some
value w in 𝑫𝒋 that satisfies the binary constraint on the arc between 𝑋𝑖 and 𝑋𝑗
 CSP REPRESENTATIONS
• Constraint graph:
• nodes are variables
• edges are constraints
 EDGES TO ARCS:
FROM CONSTRAINT GRAPH TO
DIRECTED GRAPH
• Given a pair of nodes 𝑋𝑖 and 𝑋𝑗 connected by a constraint edge, we
represent this not by a single undirected edge, but a pair of directed arcs.
• For a connected pair of nodes 𝑋𝑖 and 𝑋𝑗 , there are two arcs that connect
them: 𝒊, 𝒋 𝑎𝑛𝑑 (𝒋, 𝒊)
 ARC CONSISTENCY
• Simplest form of propagation makes each arc consistent
• 𝑋𝑖 → 𝑋𝑗 is consistent if and only if and
For every value of for every value v in 𝑫𝒊 there is some value w in 𝑫𝒋
 ARC CONSISTENCY
• Simplest form of propagation makes each arc consistent
• 𝑋𝑖 → 𝑋𝑗 is consistent if and only if and
For every value of for every value v in 𝑫𝒊 there is some value w in 𝑫𝒋
 ARC CONSISTENCY
• Simplest form of propagation makes each arc consistent
• 𝑋𝑖 → 𝑋𝑗 is consistent if and only if and
For every value of for every value v in 𝑫𝒊 there is some value w in 𝑫𝒋

• If 𝑋𝑖 loses value, recheck neighbours of 𝑋𝑖

 ARC CONSISTENCY
• Simplest form of propagation makes each arc consistent
• 𝑋𝑖 → 𝑋𝑗 is consistent if and only if and
For every value of for every value v in 𝑫𝒊 there is some value w in 𝑫𝒋

• If 𝑋𝑖 loses value, recheck neighbours of 𝑋𝑖

• Detects failure earlier than forward checking
• Can be run as a preprocessor or after each assignment
 ARC CONSISTENCY
An arc 𝒊, 𝒋 is arc consistent if and only if every value v on 𝑋𝑖 is consistent with
some label on 𝑋𝑗

To make an arc 𝑖, 𝑗 arc consistent,

for each value v on 𝑋𝑖 ,
if there is no label on 𝑋𝑗 consistent with v
then remove v from 𝑋𝑖

Given d values, checking arc 𝒊, 𝒋 takes 𝑂(𝑑 2 ) time worst case

AC-3 ALGORITHM
 LIMITATIONS OF ARC
CONSISTENCY
• After enforcing arc consistency:
• Can have one solution left
• Can have multiple solutions left
• Can have no solutions left (and not know it)
• Arc consistency still runs inside a backtracking search!
 K-CONSISTENCY
• Increasing degrees of consistency
• 1-Consistency (Node Consistency): Each single node’s domain has a value
which meets that node’s unary constraints
• 2-Consistency (Arc Consistency): For each pair of nodes, any consistent
assignment to one can be extended to the other
• K-Consistency: For each k nodes, any consistent assignment to k-1 can be
extended to the kth node.
• Higher k more expensive to compute
 STRONG K-CONSISTENCY
• Strong k-consistency: also k-1, k-2, … 1 consistent
• Claim: strong k-consistency means we can solve without backtracking!
• Why?
• Choose any assignment to any variable
• Choose a new variable
• By 2-consistency, there is a choice consistent with the first
• Choose a new variable
• By 3-consistency, there is a choice consistent with the first 2
• …
• Lots of middle ground between arc consistency and n-consistency! (e.g.
k=3, called path consistency)
 REFERENCES
• [2010] Artificial Intelligence- A Modern Approach, 3rd Ed, Stuart Russell, Peter
Norvig
• Introduction to Artificial Intelligence slide presentation University of California
Berkeley (http://ai.berkeley.edu)
• Artificial Intelligence slide presentation University of Pennsylvania (CIS 391 fall
2015)