The Vehicle Routing Problem

with Time Windows

Dr Philip Kilby | Team Leader, Optimisation Applications and Platforms
June 2017

www.data61.csiro.au
Outline

• Problem Description
• Solving the VRP
• Construction
• Local Search
• Meta-heuristics
• Variable Neighbourhood Search
• Including Large Neighbourhood Search
• CP101
• A CP model for the VRP
Vehicle routing problem
Given a set of customers, and a fleet of vehicles to make deliveries,
find a set of routes that services all customers at minimum cost
Travelling Salesman Problem
• Find the tour of minimum cost that visits
all cities
 Why study the VRP?
• It’s hard: it exhibits all the difficulties of comb. opt.
• It’s useful:
o The logistics task is 9% of economic activity in Australia
o Logistics accounts for 10% of the selling price of goods
Why study the VRP in Robotics?
Appears as a sub-problem

Task allocation to agents

• Multiple agents with multiple tasks
• Best allocation minimizes cost

Scheduling with setup costs

• Can be modelled as a VRPTW

6 | Presentation title | Presenter name

Vehicle Routing Problem
For each customer, we know
• Quantity required
• The cost to travel to every other customer

For the vehicle fleet, we know

• The number of vehicles
• The capacity

We must determine which customers each vehicle serves,

and in what order, to minimise cost
Vehicle Routing Problem
Objective function
• In academic studies, usually a combination:
o First, minimise number of routes
o Then minimise total distance or total time

• In real world
o A combination of time and distance
o Must include vehicle- and staff-dependent costs
o Usually vehicle numbers are fixed
o Includes “preferences” – like pretty routes
Time window constraints
Vehicle routing with Constraints
• Time Window constraints
– A window during which service can start
– E.g. only accept delivery 7:30am to 11:00am

– Additional input data required

• Duration of each customer visit
• Time between each pair of customers

• (Travel time can be vehicle-dependent or time-dependent)

– Makes the route harder to visualise
Time Window constraints
Pickup and Delivery problems
• Most routing considers delivery to/from a depot (depots)
• Pickup and Delivery problems consider FedEx style problem:
• pickup at location A, deliver to location B

• Load profile:
Other variants

Profitable tour problem

• Not all visits need to be
completed
• Known profit for each visit
• Choose a subset that gives
maximum profit =
(revenue from visits) –
(routing cost)

Orienteering Problem
• Maximum revenue in
limited time
VRP meets the real world
Many groups now looking at real-world constraints
Rich Vehicle Routing Problem
• Attempt to model constraints common to many real-life enterprises
– Multiple Time windows
– Multiple Commodities
– Multiple Depots
– Heterogeneous vehicles
– Compatibility constraints
• Goods for customer A {must | can’t} travel with goods from
customer B
• Goods for customer A {must | can’t} travel on vehicle C
VRP as an instance

VRP is a Combinatorial Optimization problem

• Others include
o Scheduling
o Assignment
o Bin Packing

14 | Presentation title | Presenter name

Solving VRPs
Solution Methods
Exact:
• Integer Programming or Mixed Integer Programming
• Constraint Programming

Heuristic:
• Construct
• Improve
• Local Search
• Meta-heuristics
Exact Methods
VRP:
• MIP: Can only solve problems with 100-150 customers
• CP: Similar size
 ILP
minimise : c ij  xijk xijk = 1 if vehicle k travels
i, j k directly from i to j
subject to
 x
i k
ijk  1 j Exactly one vehicle in

 x ijk  1 i Exactly one vehicle out

j k

 x   x hjk  0  k, h It’s the same vehicle

ihk
j k j k

Capacity constraint
q x
i
i
j
ijk  Qk k

x ijk  S  1 S  P( N ),0  S Subtour elimination

xijk  {0,1}
 ILP
minimise : c ij  xijk Advantages
i, j k • Can find optimal solution
Disadvantages
subject to • Only works for small problems
• One extra constraint  back to the
 x
i k
ijk  1 j drawing board
• S is huge
 x
j k
ijk  1 i

 x
j k
ihk   x hjk
j k
 0  k, h

q x
i
i
j
ijk  Qk k

x ijk  S  1 S  P( N ),0  S
xijk  {0,1}
Depot
ILP – Column Generation
Columns
represent routes

Column/route cost ck 89 76 99 45 32

1 1 1 1 0 0 …
Rows represent
customers 2 0 1 1 0 1 …

3 0 0 0 0 0 …
Array entry aik = 1 iff
customer i is 4 1 0 1 1 0 …
covered by route k
5 1 0 0 0 0 …
Column Generation
• Decision var xk: Use column k?
• Column only appears if feasible
ordering is possible
• Cost of best ordering is ck
• Best order stored separately min c xk k
• Master problem at right
subject to
i a
k
ik xk  1 i

xk  {0,1}
 Heuristics for the VRP

22 | Presentation title | Presenter name

Heuristics:

Often variants of
• Construct
• Improve

23 | Presentation title | Presenter name

Heuristics for the VRP
Construction by Insertion
• Start with an empty solution
• Repeat
• Choose which customer to insert
• Choose where to insert it

E.g. (Greedy)
• Choose the customer that increases the cost by the least
• Insert it in the position that increases the cost by the least
Solving the VRP the easy way

Insert methods

Order is important:
Regret
Regret
Regret
Regret
Regret
Regret
Regret = C(insert in 2nd-best route) – C(insert in best route)
= f (2,i) – f (1,i)

K-Regret = ∑k=1,K (f (k,i) – f (1,i) )

Insert customer with maximum regret

Insertion with Regret
Seeds
Initialise each route with one (or more) customer(s)
• Indicates the general area where a vehicle will be
• May indicate time it will be there
• Depends on time window width

Distance-based seeding
• Find the customer (s1) most distant from the depot
• Find the customer (s2) most distant from s1
• Find the customer (s3) mist distance from s1, s2
• …
• Continue until all vehicles have a seed
Implementation
• Heart of algorithm is deciding which customer to insert next, and
where
• Data structure of “Insert Positions”
o legal positions to insert a customer
o Must calculate cost of insert
o Must ensure feasibility of insert
• After each modification (customer insert)
o Add new insert positions
o Update cost of affected insert positions
o Check legality of all insert positions
o O(1) check important for efficiency

34 | Presentation title | Presenter name

Local Search
Improvement Methods
Local Search
• Often defined using an “operator”
Improvement Methods
Local Search
• Often defined using an “operator”
• e.g. 1-move
Improvement Methods
Local Search
• Often defined using an “operator”
• e.g. 1-move
Improvement Methods
Local Search
• Often defined using an “operator”
• e.g. 1-move
Improvement Methods
Local Search
• Often defined using an “operator”
• e.g. 1-move
Improvement Methods
Local Search
• Often defined using an “operator”
• e.g. 1-move
Improvement Methods
Local Search
• Often defined using an “operator”
• e.g. 1-move
• Solutions that can be reached
using the operator termed the
neighbourhood
• Local Search explores
the neighbourhood of
the current solution
Local Search
Other Neighbourhoods for VRP:
• Swap 1-1
Local Search
Other Neighbourhoods for VRP:
• Swap 1-1
Local Search
Other Neighbourhoods for VRP:
• Swap 2-1
Local Search
Other Neighbourhoods for VRP:
• Swap 2-1
Local Search
Other Neighbourhoods for VRP:
• Swap tails
Local Search
Other Neighbourhoods for VRP:
• Swap tails
Local Search
Other Neighbourhoods for VRP:
• Swap tails
Improvement Methods
2-opt (3-opt, 4-opt…)
• Remove 2 arcs
• Replace with 2 others
Improvement Methods
Or-opt
• Consider chains of length k
• k takes value 1 .. n / 2
• Remove the chain from its current position
• Consider placing in each other possible position
• in forward orientation
• and reverse orientation

• Very effective
Local Search
• Local minima

Objective value

Current solution
Local Search
Escaping local minima
Meta-heuristics
• Heuristic way of combining heuristics
• Designed to escape local minima
Local Search
Escaping local minima
• Define more (larger) neighbourhoods
– 1-move (move 1 visit to another position)
– 1-1 swap (swap visits in 2 routes)
– 2-2 swap (swap 2 visits between 2 routes)
– Tail exchange (swap final portion of routes)
– 2-opt
– Or-opt (all sizes 2 .. n/2)
– 3-opt
Local Search
Variable Neighbourhood Search
• Consider multiple neighbourhoods
– 1-move (move 1 visit to another position)
– 1-1 swap (swap visits in 2 routes)
– 2-2 swap (swap 2 visits between 2 routes)
– 2-opt
– Or-opt
– Tail exchange (swap final portion of routes
– 3-opt
– Explore one neighbourhood completely
– If no improvement found, advance to next neighbourhood
– When an improvement is found, return to level 1
Local Search
Variable Neighbourhood Search

• For new constraints/new problems, add new neighbourhoods

• E.g. Orienteering problem

o New neighbourhoods:
– Unassign 1 customer (i.e. do not visit)
– Unassign clusters of customer (e.g. sequences of customers)
– Insert clusters of unassigned customers

56 | Presentation title | Presenter name

Local Search
Many Meta-heuristics have been tried
• Simulated Annealing
• Tabu Search
• Genetic Algorithms
• Ants
• Bees
• Particle Swarms

• Large Neighbourhood Search

Large Neighbourhood Search
• Originally developed by Paul Shaw (1997)
• This version Ropke & Pisinger (2007)1
• A.k.a “Record-to-record” search

• Destroy part of the solution

– Remove visits from the solution
• Re-create solution
– Use favourite construct method to re-insert customers
• If the solution is better, keep it
• Repeat

1: S Ropke and D Pisinger, An Adaptive Large Neighborhood Search Heuristic for the Pickup and
Delivery Problem with Time Windows, Transportation Science 40(4), pp 455-472, 2006
Large Neighbourhood Search
Destroy part of the solution (Select method)

• Remove some visits

• Move them to the “unassigned” lists
Large Neighbourhood Search
Destroy part of the solution (Select method)
Examples
• Remove a sequence of visits
Large Neighbourhood Search
Destroy part of the solution (Select method)
Examples
• Choose longest (worst) arc in solution
– Remove visits at each end
– Remove nearby visits
• Actually, choose rth worst
• r = n * (uniform(0,1))y
• y~6
– 0.56 = 0.016
– 0.96 = 0.531
Large Neighbourhood Search
Destroy part of the solution (Select method)
Examples
• Dump visits from k routes (k = 1, 2, 3)
– Prefer routes that are close,
– Better yet, overlapping
Large Neighbourhood Search
Destroy part of the solution (Select method)
Examples
• Choose first visit randomly
• Then, remove “related” visits
– Based on distance, time compatibility, load

Rij   Cij 
 (| ai  a j | ) 
 (| qi  q j |)
Large Neighbourhood Search
Destroy part of the solution (Select method)
Examples
• Dump visits from the smallest route
– Good if saving vehicles
– Sometimes fewer vehicles = reduced travel
Large Neighbourhood Search
Destroy part of the solution (Select method)

• Parameter: Max to dump

– As a % of n?
– As a fixed number e.g. 100 for large problems

• Actual number is uniform rand (5, max)

Large Neighbourhood Search
Re-create solution
– Systematic search
– Smaller problem, easier to solve
– Can be very effective

– E.g.: CP Backtracking search

– Constraint: objective must be less than current
– (Implicitly) Look at all reconstructions

– Backtrack as soon as a better sol is found

– Backtrack anyway after too many failures
Large Neighbourhood Search
Re-create solution
• Use your favourite insert method

• Better still, use a portfolio

– Ropke: Select amongst
– Minimum Insert Cost
– Regret
– 3-regret
– 4-regret
Large Neighbourhood Search
Large Neighbourhood Search
• If the solution is better, keep it
Large Neighbourhood Search
• If the solution is better, keep it

70/58
Large Neighbourhood Search
• If the solution is better, keep it
• Can use Hill-climbing
• Can use Simulated Annealing
• Can use Threshold Annealing
• …
Large Neighbourhood Search


T
P( accept increase )  e
Large Neighbourhood Search
Large Neighbourhood Search
Large Neighbourhood Search
Adaptive
– Ropke adapts choice based on prior performance
– “Good” methods are chosen more often

75/58
Large Neighbourhood Search
Adapting Select method

76/58
Large Neighbourhood Search
Ropke & Pisinger (with additions) can solve a variety of problems
• VRP
• VRP + Time Windows
• Pickup and Delivery
• Multiple Depots
• Multiple Commodities
• Heterogeneous Fleet
• Compatibility Constraints
Solution Methods
Summary so far:

• Introduced several successive insertion construction methods

o Various ways to choose the next visit to insert
o Various ways to choose where to insert

• Described two successful metaheuristics

o Variable Neighbourhood Search
o Large Neighbourhood Search

78 | Presentation title | Presenter name

Solution Methods
What’s wrong with that?

• New constraint  new code

– Often right in the core

• New constraints interact

– e.g. Multiple time windows mess up duration calculation

• Code is hard to understand, hard to maintain

Solution Methods

An alternative:
Constraint Programming
Constraint Programming
CP offers a language for representing problems
• Decision variables
• Constraints on variables

Also offers techniques for solving the problems

• Systematic search
• Heuristic Search
CP101
CP101
CP101
Eg Mutual Exclusion constraint in VRP
• (If any visit from the set D is assigned, then no others can be)

• Uses ‘isAssigned’ var

• Domain [0,1]
• Attach propagator to the ‘isAssigned’ variable for each of the visits
• Propagator wakes when ‘isAssigned’ is bound to 1 for any visit
• Propagates by binding isAssigned to ‘0’ for remaining visits.
CP101
• Typical execution:
• Establish choice point (store all current domains)
• Choose variable to instantiate
• Choose value to assign, and assign it
• Propagations fire until a fixed point is achieved, or an
inconsistency is proved (empty domain)
• If inconsistent,
• Backtrack (restore to choicepoint)
• Remove offending value from the variable’s domain
• Repeat until all variables are bound (assigned)
• For complete search, store sol, then act like inconsistent
CP101
‘Choose a variable to assign, choose value to assign’
• Very good fit for constructive route creation
• After each insert, propagators fire
• New variable domains give look-ahead to feasible future
insertions
• Constraints guide insertion process

Step-to-new-solution does not work as well

• Local move operators can only use CP as a rule checker
• Do not leverage full power of CP
Expressive Language (e.g. Minizinc)
string: Name; % Decision variables
var int: obj;
% Customers array[Visits] of var Visits:
int: NumCusts; routeOf;
set of int: Customers = array[Visits] of var Visits:
1..NumCusts; succ;
array[Visits] of var [0,1]:
% Locations isAssigned;
int: NumLocs = NumCusts + 1;
set of int: Locations = constraint alldifferent (succ);
1..NumLocs; constraint circuit (succ);

% Vehicles constraint
int: NumVehicles; obj = sum (i in Visits)
int: NumRoutes = NumVehicles; (dist[Loc[i],Loc[succ[i]]]);
set of int: Vehicles =
1..NumVehicles; constraint
sum (i in Visits,j = routeOf[i])
% Location data (demand[i]) < j)
array[Locations] of float: locX; for j in Vehicles;
array[Locations] of float: locY;
array [Locations, Locations] of
int: dist;
Constraint Programming for the VRP
Constraint Programming
Advantages:
• Expressive language for formulating constraints
• Each constraint encapsulated
• Constraints interact naturally
• Constraints guide construction

Disadvantages:
• Can be slow
• No fine control of solving
• (unless you use a low-level library like gecode )
Constraint Programming
Two ways to use constraint programming
• Rule Checker
• Properly

Rule Checker:
• Use favourite method to create/improve a solution
• Check it with CP
– Very inefficient.
A CP Model for the VRP
Vocabulary
• A solution is made up of routes
(one for each vehicle)
• A route is made up of a sequence of visits
• Some visits serve a customer (customer visit)

(Some tricks)
• Each route has a “start visit” and an “end visit”
• Start visit is first visit on a route – location is depot
• End visit is last visit on a route – location is depot
• Also have an additional route – the unassigned route
– Where visits live that cannot be assigned
Model
A (rich) vehicle routing problem
• n customers (fixed in this model)
• v vehicles (fixed in this model)
• m = v+1 one route per vehicle plus “unassigned” route
• fixed locations
– where things happen
– one for each customer + one for (each?) depot
• c commodities (e.g. weight, volume, pallets)
– Know demand from each customer for each commodity
• Know time between each location pair
• Know cost between each location pair
– Both obey triangle inequality
Referencing
Sets
• N = {1 .. n} – customers
• V = {1 .. v} – vehicles/real routes
• R = {1 .. m} - routes include ‘unassigned’ route
• S = {n+1 .. n+m} – start visits
• E = {n+m+1 .. n+2m} – end visits
• V = N  S  E – all visits
• VS= N  S – visits that have a sensible successor
• VE= N  E – visits that have a sensible predecessor
Referencing
Customers
• Each customer has an index in N = {1..n}
• Customers are ‘named’ in CP by their index
Routes
• Each route has an index in R = {1..m}
• Unassigned route has index m
• Routes are ‘named’ in CP by their index
Visits
• Customer visit index same as customer index
• Start visit for route k has index n + k ; aka startk
• End visit for route k has index n + m + k ; aka endk
Data
We know (note uppercase)
• Vi The ‘value’ of customer i
• Dik Demand by customer i for commodity k
• Ei Earliest time to start service at i
• Li Latest time to start service at i
• Qjk Capacity of vehicle j for commodity k
• Tij Travel time from visit i to visit j
• Cij Cost (w.r.t. objective) of travel from i to j
Basic Variables
Successor variables: si
• si gives direct successor of i, i.e. the index of the next visit on
the route that visits i
• si  VE for i in VS si = 0 for i in E
Predecessor variables pi
• pi gives the index of the previous visit in the route
• pi  VS for i in VE pi = 0 for i in S
• Redundant – but empirical evidence for its use
• Route variables ri
• ri gives the index of the route (vehicle) that visits i
• ri  R
Example
i si pi ri 1

1 4 2 2 Route 2
4
2 1 7 2
3 8 5 1 Route 1
2
4 9 1 2 End
3 visits
5 3 6 1
6 5 0 1
7 2 0 2 7 9
5
8
8 0 3 1
6
9 0 4 2 Start visits
Other variables
Accumulation Variables
• qik Quantity of commodity k after visit i
• ci Objective cost getting to i

• For problems with time constraints

• ai Arrival time at i
• ti Start time at i (time service starts)
• di Departure time at i

• Actually, only ti is required, but others allow for expressive

constraints
What can we model?
• Basic VRP
• VRP with time windows
• Multi-depot
• Heterogeneous fleet
• Open VRP (vehicle not required to return to base)
– Requires anywhere location
– Route end visits located at anywhere
– distance i  anywhere = 0
• Compatibility
– Customers on different / same vehicle
– Customers on/not on given vehicle
• Pickup and Delivery problems
What can we model?
• Variable load/unload times
– by changing departure time relative to start time
• Dispatch time constraints
– e.g. limited docks
– si for i in S is load-start time
• Depot close time
– Time window on end visits
• Fleet size and mix
– Add lots of vehicles
– Need to introduce a ‘fixed cost’ for a vehicle
– Cij is increased by fixed cost for all i  S, all j  N
What can’t we model
• Can’t handle dynamic problems
– Fixed domain for s, p, r vars
• Can’t introduce new visits post-hoc
– E.g. optional driver break must be allowed at start
• Can’t handle multiple visits to same customer
– ‘Larger than truck-load’ problems
– If qty is fixed, can have multiple visits / cust
– Heterogeneous fleet is a pain
• Can handle time- or vehicle-dependent travel times/costs with
mods
• Can handle Soft Constraints with mods
Objective

Want to minimize
• sum of objective (cij) over used arcs, plus
• value of unassigned visits

minimize

c v
iE
i
i ri  0
i
Basic constraints
Path ( S, E, { si | i  V } )
AllDifferent ( { pi | i  VE } )

Accumulate obj. csi  ci  Ci , si i  V S

Accumulate time asi  d i  Ti , si i  V S

Time windows ti  ai i  V
ti  Li i  V
ti  Ei i  V
ti  0 i  S
Constraints
• Load qsi k  qik  Qsi k i  V S , k  C
qik  Qri k i  V , k  C
qik  0 i  V , k  C
qik  0 i  S , k  C
• Consistency s pi  i i  V S
p si  i i  V E
S
ri  rsi i  V
rn  k  k k  M
rn  m  k  k k  M
Subtour elimination

• Most CP libraries have built-ins

– MiniZinc: ‘circuit’
– Comet: ‘circuit’
– ILOG: Path constraint
Propagation – Cycles
‘Path’ constraint
• Propagates subtour elimination
• Also propagates cost

• path (S, E, succ, P, z)

– succ array implies path
– ensures path from nodes in S to nodes in T through nodes in
P
– variable z bounds cost of path
– cost propagated incrementally based on shortest / longest
paths
Large Neighbourhood Search
revisited
Large Neighbourhood Search
Destroy & Re-create
• Destroy part of the solution
– Remove visits from the solution
• Re-create solution
– Use insert method to re-insert customers
– Different insert methods lead to new (better?) solutions
• If the solution is better, keep it
• Repeat
Large Neighbourhood Search
Destroy part of the solution (Select method)

In CP terms, this means:

• Relax some variable assignments

In CP-VRP terms, this means

• Relax some routeOf and successor assignments
Large Neighbourhood Search
Re-create solution

• Use insert methods

• Uses full power of CP propagations
A MiniZinc VRP model

111 | Presentation title | Presenter name

Advanced techniques – Recreate
Adaptive Decomposition

Decompose problem
• Only consider 2-3 routes
• Smaller problem is much easier to solve
Advanced techniques – Recreate
Adaptive Decomposition

Decompose problem
• Only consider 2-3 routes
• Smaller problem is much easier to solve

Adaptive
• Decompose in different ways
• Use problem features to determine decomposition
Conclusions
• Now you know
o How to construct a solution to a VRP by successive insertion
o How to improve the solution using
– Variable Neighbourhood Search
– Large Neighbourhood Search

• Argued that CP is “natural” for solving vehicle routing problems

– Real problems often have unique constraints
– Easy to change CP model to include new constraints
– New constraints don’t change core solve method
– Infrastructure for complete (completish) search in subproblems

• LNS is “natural” for CP

– Insertion leverages propagation