MONTE CARLO

METHODS
AND APPLICATIONS

Carnegie Mellon University | 21-387 / 15-327 / 15-627 / 15-860 | Fall 2023

 LECTURE 12
PDES & STOCHASTIC PROCESSES

MONTE CARLO METHODS

AND APPLICATIONS
Carnegie Mellon University | 21-387 | 15-327 | 15-627 | 15-860 | Fall 2023
Recap—SDEs & PDEs
• Ordinary & Stochastic Differential Equations (last lecture)
– how do we describe systems evolving over time? (ODEs)
– how do we incorporate randomness? (SDEs)
– how do we simulate motion numerically?

• Partial Differential Equations (this lecture)

analogy: trajectory of rock (+wind)
– how do we describe systems evolving over time & space? (PDEs)
– how do we simulate these systems numerically?

• SDE ⟷ PDE connection

– Somewhat surprising perspective: can use stochastic ODEs to
understand—and simulate—deterministic PDEs
– …and vice-versa! analogy: ripples on pond
Goal: Connect “microscopic” & “macroscopic”
Understand statements of two major concepts
and see how they can be used for computation.

Feynman-Kac formula Fokker-Planck equation

use random walks to solve PDEs solve PDEs to model random walks

Today: will develop numerical algorithms to actually do this…

Motivating Example—Basic Langevin Equation
• Consider our famous “jiggling pollen grains”

• How do we write an equation for the velocity

vt of a single particle as a function of time t?
• In vector field picture, drift term always
points toward origin (here the components
are components of a d-dimensional velocity,
rather than position & velocity)

• Hence, velocity tends toward zero (due to

drag force), but then hovers around the origin
(due to Langevin force) drag force Langevin
force
• Numerically integrating SDE many times due to
d o w e g e t a m ore viscosity/
gives rough sense of how velocity is How
precise solution? “friction”
due to thermal
distributed over space & time, i.e., P(vt = w) fluctuations of liquid
Overview—PDEs & Stochastic Processes
• Partial Differential Equations (PDEs)
– describe function implicitly via relative rates of change in space & time
• Numerical Methods for PDEs
• Feynman-Kac formula
– for a given PDE, can (sometimes) write down its solution in terms of a
stochastic process Xt
– can simulate this SDE numerically to compute solution to PDE
• Fokker-Planck equation
– for a given stochastic process Xt, we can (sometimes) write down a PDE
describing the evolution of its probability distribution over time
– can solve this PDE numerically to evaluate average properties of Xt
Partial Differential Equations
Ordinary Differential Equation (Review)
Recall general formula for an ordinary differential equation:

where F is any function of the (unknown) function x(t) and its first n
derivatives in time.

We said this ODE is:

– nth order in time (or simply nth order)
– linear (or nonlinear) if F is a linear (or nonlinear) function of its inputs
 Partial Differential Equation
A partial differential equation is similar, but adds spatial dependence:
linear if F and φ are
governing equation linear; nonlinear
otherwise
ODEs, nonlinear
boundary conditions Like
generally harder

e r i n s p ace,
“nth o r d dary condit
h o rd e r i n time” space & time un ion
mt o s
derivatives

b
of any order initial conditions

unknown function could take other

time kinds of values

time
space
domain

solution
could be, e.g., +∞
boundary
 Example—1D Heat Equation
Example. Heat equation on the real line: (no boundary conditions:
domain has no boundary)

n c a v e / c o n v e x
Intuition: co ed up/down
p s g e t p u s h
bum

Solution. “heat kernel”

Differential Operators
• In dimension d > 1, becomes ugly to write out PDE in terms of
all d spatial directions
• Can instead express in terms of differential operators
– divergence
– gradient heat equation (component-wise)

– curl
– Laplacian
– …
• (Also helps to drop arguments…) heat equation (differential operator)
Review: Divergence, Gradient, and Curl
f X Y

grad f div X curl Y

Review—Vector Derivatives in Coordinates
How do we express grad, div, and curl in coordinates? Consider:
• a scalar function
• a vector field X = (u, v, w)
– functions u, v, w: giving components of the vector field at each point

grad div curl

Review: Laplacian
(twice differentiable)

Coordinates.

e.g., in 2D:

Differential operators.
Laplacian Gives Deviation from Local Average
More intuitively, can think of u(x) u(x0)
the Laplacian of a function u as
difference between value at a
point x0, and the average value
over a small sphere (or ball)
around x0.

sphere integral over value at

area sphere center
 Elliptic / Parabolic / Hyperbolic PDEs In general: che
ck
of differential o symbol
perator

Three basic types of behavior for PDEs, encapsulated by three model equations:

Laplace equation heat equation wave equation

(elliptic) (parabolic) (hyperbolic)

dent” “time-dependent”
“time-indepen

EASIER MORE DIFFICULT MOST DIFFICULT

Initial & Boundary Conditions
• Initial & boundary conditions essentially define
which instance of the problem we're talking about
• Initial conditions (time-dependent problems only)
– E.g., what does the weather look like today, so
we can predict what it will look like tomorrow?
dary conditio
• Boundary conditions (domains with boundary only!) ou
n ns

b
– E.g., if we can measure the temperature only on
the exterior of a body, what might we guess the

time
temperature distribution looks like inside? domain

boundary
Dirichlet vs. Neumann Boundary Conditions
Dirichlet Neumann

interpolate given values interpolate given derivatives

Dirichlet vs. Neumann Boundary Conditions
Dirichlet Neumann
Mixed

interpolate given fixed

valuesvalues or derivatives
interpolate given derivatives
Dirichlet Boundary Conditions
On interval [0,1], many possible functions w/ values u0, u1 at endpoints:

Key idea: “Dirichlet” just means boundary values are fixed.

Dirichlet Boundary Conditions
Likewise many possible functions w/ slope v0, v1 at endpoints:

Key idea: “Neumann” just means boundary derivatives are fixed.

Mixed Dirichlet & Neumann
Can also prescribe some values, some derivatives:

But what if we also have conditions on the interior?

Laplace w/ Dirichlet Boundary Conditions (1D)
For a 1D Laplace equation, can we always satisfy Dirichlet conditions?

1D Laplace:

Solutions:

Yes: a line can interpolate any two points.

Laplace w/ Neumann Boundary Conditions (1D)
What about Neumann—can we prescribe the derivative at both ends?

1D Laplace:

Solutions:

No! A line can have only one slope.

 Laplace w/ Dirichlet Boundary Conditions (2D)
• Let’s now consider a Laplace equation in 2D
• Can we always satisfy Dirichlet boundary
conditions?
• Yes*: Laplace is steady-state solution to heat
flow—just let it run for a long time…
– Dirichlet data is “heat” along boundary

*Subject to very mild/reasonable conditions on boundary geometry, boundary data

Laplace w/ Neumann Boundary Conditions (2D)
• Suppose instead we prescribe the normal
derivative along the boundary
• Can we always find a solution to the
Laplace equation?
• Well, consider the divergence theorem—
“what goes in, must come out!”

• Can only solve if Neumann data h

integrates to zero over the boundary

Important: in general, a PDE may not have solutions

for given boundary conditions (“ill-posed”)
Convection-Diffusion Equation

TEMPERATURE

The convection-diffusion equation provides link

between PDEs and diffusion processes (SDEs).
Convection-Diffusion Equation

BOUNDARY VALUE

Intuition: temperature along boundary is fixed.

Convection-Diffusion Equation
SOURCE

Intuition: adds additional “background temperature.”

Convection-Diffusion Equation
ABSORPTION

Intuition: “cooling” due to absorption into background medium.

Convection-Diffusion Equation
DRIFT

Intuition: heat is dragged along with a flowing river.

Convection-Diffusion Equation
DIFFUSION

Intuition: how fast does heat “spread out”?

Convection-Diffusion Equation—Coefficients

In general, coefficient functions α, ω, λ can vary over space:

ABSORPTION SOURCE

DIFFUSION DRIFT

BOUNDARY
Convection-Diffusion Equation—Coefficients

In general, coefficient functions α, ω, λ can vary over space:

ABSORPTION SOURCE

DIFFUSION DRIFT

BOUNDARY

Notation: will often drop arguments for brevity.

Numerical Methods for PDEs
Numerical Methods for PDEs—Overview
• Numerical methods for ODEs were reasonably straightforward: start with some initial
conditions and "march through time"; some choice of time stepping scheme
• Much bigger "zoo" of approaches for PDEs:
– finite difference methods
– finite element methods
– finite volume methods
– boundary integral methods r e : w il l u s e s to c h astic/
Later lectu to s o lv e PDEs
lo p ic tu re
Monte Car e n s io n al
– spectral methods withou t fi n it e -d im
a ti o n ! (w a lk o n s p heres)
approxim
– …
• Common thread: all use finite-dimensional approximation
– e.g., finite system of equations, discrete set of unknowns
Finite Differences
• Finite difference method approximates sample points
differential equations via “pointwise” perspective
– Imagine I have values of a function f : →
only at a discrete set of sample points Taylor series
– How can I approximate derivatives?
• Example. Approximate 1st-order derivative via
first couple terms of Taylor series.

f(x)

“first-order
accurate”
forward difference
h
x
x0 x1
Finite Differences—Second Order
Example. Take “difference of differences” to approximate 2nd-order derivative:

Q: Is this a reasonable
approximation?
Feels like we’re piling approxim
ation
on top of approximation…
Finite Differences—Polynomial Fitting
• Similar perspective: fit polynomial to sample sample points sample values
points; derivatives of polynomial then
approximate derivatives of original function. –h 0 +h
• Example. Fit quadratic polynomial p(x) to quadratic fit
three points. Then p’(x) gives centered
difference approximation for 1st derivative of
linear system for a, b, c
f (x) at x0; p’’(x) gives finite difference formula
for 2nd derivative.
f(x) Q: How would we
recover 1st-order
forward difference?
A: Fit a linear O(h 2)
polynomial 1st derivative error
p(x) (centered)
p(x) = ax+b

x 2nd derivative
x0 x1 x2
Finite Differences - Laplacian Notice: approximates
deviation from local average!

• To approximate differential operators in multiple dimensions, can just

apply 1D finite difference approximations along each coordinate direction
• Example. Laplacian in 2D
Finite Differences - Gradient (Centered)
• One way to define gradient: take
centered differences along each axis
Finite Differences - Gradient (Centered)
• One way to define gradient: take
centered differences along each axis
• Unfortunately doesn’t play well with
subsequent operations
– e.g., suppose we again use centered
difference to approximate
divergence of gradient (Laplacian)
– resulting approximation has large
“checkerboard” stencil, ignores local
information
– leads to bizarre numerical behavior
(e.g., functions that don’t belong in
null space of Laplacian)
 Finite Differences—Staggered Grid
scalar
• Instead, quite common to store quantities on a
staggered grid*
– scalar functions (e.g., u) stored at cell centers
– vector fields (e.g., ∇u) stored at cell faces
◦ e.g., in 2D store x-component on vertical faces,
y-component on horizontal faces vector

◦ use half-index notation

– Finite difference approximations are otherwise
identical (in each dimension)

*Arises naturally from several perspectives: finite volumes, discrete exterior calculus, …
Gradient—Staggered Grid
• Example. To approximate the
gradient on a 2D staggered grid take
a “horizontal” difference across each
vertical edge, and “vertical”
difference across each horizontal
edge.
– Note: there is no complete vector
defined at any grid node!
– If necessary (e.g., for visualization),
can still (bi-)linearly interpolate
nodal values to get continuous
vector field in space
Finite Difference Solver
• Store values on a regular grid
• Use finite differences to approximate
spatial derivatives
• Use same strategies as with ODEs to

update rule
approximate temporal derivatives
• Now “march forward in time” by
repeatedly solving for uk+1 (update rule)
• Notation:
time
– subscript for space
– superscript for time space
Example — Heat Equation
Update rule: forward Euler, finite difference Laplacian 1
1 -4 1
Initial conditions: all values given at initial time step (k=0) 1
Boundary conditions: values given at boundary nodes for all steps k
function at
previous time

time
function at next step node spacing
moment in time

initial conditions u0 boundary conditions g solution u(t)

Heat Equation—Stability
• Q: How do we pick maximum stable time forward Euler eigenvectors/
step τ for solving heat equation? (heat equation) eigenvalues
(Laplacian)
• A: Same basic analysis as for ODEs, since a
discretized PDE is just a system of ODEs!

0.99τstable 1.01τstable

As domain geometry becomes more Many ways to Rule of thumb:

complicated (h → 0), cost grows 4 do better…
quadratically in space and time (O(h
)) τstable ≈ 1 h 2
4
Example — Heat Equation with Source
Easy enough to add other terms, like a source term f :

initial conditions u0 source term f boundary conditions g solution u(t)

 Example — Convection-Diffusion Equation
• Convective term not as straightforward
– central differences + forward Euler unstable for reasonable
time steps
– finite volume approach (integrate over cell) doesn’t really
lead anywhere…
• Principled approach: upwind differencing
– 1D: forward difference when velocity negative & vice-versa
– nD: apply 1D scheme in each dimension

forward difference backward difference

upwind (1D)
see: Osher & Fedkiw, “Level
Set Methods and Dynamic
Implicit Surfaces” §3.2
Numerical Methods for PDEs—Summary
• Only scratched the surface
• Many ways to think about approximating solution, space &
time derivatives
• Many criteria for “good” solver
– stability: how big of a time step can I take?
– accuracy: how fast do I converge under refinement?
• Just comparing stability & accuracy is shallow way of
judging method—in practice have to deal with:
– complex geometry
– complex boundary conditions
– singular or discontinuous functions
– …
• No universally “best” method—pick the right tool for the job!
PDEs and Stochastic Processes
Feynman-Kac Formula — Overview
• Main question: what’s the connection between PDEs and random walks?
• One direction: solution to PDE given by average behavior of many
random walks
– encapsulated by Feynman-Kac formula
• Another direction: behavior of stochastic process (e.g., probability
distribution) can be analyzed by solving a PDE
– encapsulated by Fokker-Planck equation
• Prelude: connection between Brownian motion & heat equation
Review: Donsker’s Theorem
• Consider a sequence of i.i.d. random variables X1, …, Xn
• Can associate these discrete steps with a time-continuous function
2
Ratio τ / h
is a fixed
constant.
spatial step time step
size h size τ = 1/n

As n → ∞, W n̂ (t) converges to a standard

• Donsker’s theorem.

Brownian motion Wt over t ∈ [0,1]

Review: Discrete Heat Equation
function at h
previous time 1
1 -4 1
time 1
solution value at step node spacing
next time
τstable = 1 h 2
4

initial conditions u0 boundary conditions g solution (0.99τstable) solution (1.01τstable)

 Prelude: Heat Equation & Random Walks
(i,j+1)
Discrete Random Walk Discrete Heat Equation
Consider a random walk on ℤ2
(i–1,j) (i,j) (i+1,j)
One step of 2D heat equation, via forward Euler:
(i,j–1)

Assume node spacing h = 1. Then

If P is the transition matrix and the row vector μ

encodes a probability distribution, then
1
Also let τ = h 2 to ensure stability. Then
4

I.e., just repeatedly average the four neighbors. I.e., just repeatedly average the four neighbors.

Discrete heat equation models probability distribution of discrete random walk.

 Prelude: Heat Equation & Random Walks

keeping fixed ratio

1
τ/h 2 = 4

discrete heat equation converges discrete random walk

to continuous solution converges to Brownian motion
(Taylor’s theorem) (Donsker’s theorem)

Heat equation models probability distribution of random walk.

Feynman-Kac Formuła
Feynman-Kac Formula — Overview
PARTIAL DIFFERENTIAL EQUATION (CONVECTION-DIFFUSION)

STOCHASTIC REPRESENTATION OF SOLUTION (FEYNMAN-KAC)

Boundary Term
KAKUTANI’S PRINCIPLE

boundary Brownian process

differential equation stochastic representation

(Laplace)

Key idea: track

distribution of where
random walkers first hit
domain boundary ∂Ω
(“harmonic measure”).
x
Source Term
SOURCE TIME TO REACH BOUNDARY

differential equation stochastic representation

(Poisson)

Intuition: accumulate
total “heat” felt by
random walker along
its trajectory.
Absorption Term
ABSORPTION EXPONENTIAL DECAY

source term boundary term

differential equation stochastic representation
(screened Poisson)

λ=0 small λ large λ

Key idea: absorption of
random walkers reduces
contribution of “heat”
from source & boundary

some walkers get “absorbed”

before reaching boundary
Drift Term
DIFFUSION PROCESS WITH DRIFT
DRIFT

differential equation
(convection-diffusion) stochastic representation

Key idea: adding drift to

random walk will
change distribution of
where we feel heat from
the source & boundary
distribution of
walks & exit points
Feynman-Kac Formula
Summary: we can express solution to any convection-diffusion equation (PDE)
via the average behavior of a diffusion process (SDE).

PARTIAL DIFFERENTIAL EQUATION (CONVECTION-DIFFUSION)

diffusion drift absorption source

boundary

STOCHASTIC REPRESENTATION OF SOLUTION (FEYNMAN-KAC FORMULA)

Feynman-Kac Formula
Summary: we can express solution to any convection-diffusion equation (PDE)
via the average behavior of a diffusion process (SDE).

PARTIAL DIFFERENTIAL EQUATION (CONVECTION-DIFFUSION)

STOCHASTIC REPRESENTATION OF SOLUTION (FEYNMAN-KAC FORMULA)

drift diffusion

absorption absorption boundary

source
Feynman-Kac—Numerical Approximation
• Q: How could you numerically approximate u at some point x?
“Tour de force” of
• A: Expected value is an integral, so let’s use Monte Carlo integration! numerical methods
we’ve learned so fa
r…
• Q: Ok, but how do you evaluate samples Yi?
• A: We have an SDE, so let’s use an SDE integrator! (e.g., Euler-Maruyama)

① Simulate many random ② For each walk, evaluate 1D time

walks, using SDE integrator integrals with deterministic quadrature

③ To get final value, just take

an average (Monte Carlo)
Feynman-Kac—Numerical Example (Laplace)
Example. Basic Laplace problem: u
+1

PDE Ω
e
Q: What’s th
solution?
A: Just a linear
function: y
u(x,y) = x

x -1
e
Q: What’s th E? Solution
e s p o n d i n g SD
cor r Q: What’s the Feynman-Kac
A: Just plain representation of the solution?
Brownian motion A: Just the expected boundary
value at the exit time T. So, no quadrature in time
SDE to worry about (for now).
Feynman-Kac—Numerical Example (Laplace)

u ̂ ← 0 initialize estimate
for i = 1,…, N:
x ← x0 start at evaluation point
① Simulate until ∥x∥ > 1: while still in the disk ③ Overall estimate is
random walk average of boundary
using Euler- ξ ∼ 𝒩2D(0,1) sample direction values at exit points
Maruyama x ← x + τξ take a step
u ̂ ← u ̂ + g(x) add boundary value
̂ return the average
return u /N

② Nothing to
integrate in time
 Feynman-Kac—Numerical Example (Laplace)
N=1 N = 16 N = 1024
τ = 1/10

u
+1

Can visualize solution by,

e.g., evaluating at each
point of a regular grid

grid reference
solution -1
Feynman-Kac—Numerical Example (Poisson)
Example. Let’s add a source term: u
+1

PDE For testing purposes, Ω

let’s define f such that
we already know the
solution:

x -1
e S DE now?
Q: W h a t ’ s t h Solution
Q: How does the Feynman-
A: Still just Brownian Kac representation change?
motion—we didn’t change
the differential operator (∆) A: We now have to integrate the
source term along each walk

SDE
 Integrating the Source Term
Q: Given that we’ve already generated a sequence of points
2
x0, x1, x2…, xK ∈ ℝ at equispaced time intervals 0,τ,2τ, …, Kτ, what
is a reasonable way to numerically approximate the source integral?

?
 Integrating the Source Term
Recall: a function of stochastic process is a stochastic process
So, we just n
approximate eed to
th
the curve” on e “area under
the right…
 Integrating the Source Term
Q: Given that we’ve already generated a sequence of points
2
x0, x1, x2…, xK ∈ ℝ at equispaced time intervals 0,τ,2τ, …, Kτ, what
is a reasonable way to numerically approximate the source integral?

A: Can just use midpoint rule.

 Feynman-Kac—Numerical Example (Poisson)
N=1 N = 16 N = 1024
τ = 1/10

u
+1

Converging… but
not clearly to the
correct solution!

grid reference
solution -1
What Can Go Wrong?
• Q: Are there any sources of numerical error in our algorithm?
• A: Yes, many!
– number of samples N (Monte Carlo estimate)
Ω
– step size τ (Euler Maruyama)
◦ crude approximation of time integrals (even if N is large!)
◦ final step may be far from boundary
◦ could even “jump” from one part of the domain to another
–…
Later: can nicely address some of these problems with walk on spheres.
Fokker-Planck Equation
Fokker-Planck Equation — Overview
• Main question: what’s the connection between PDEs and random walks?
• One direction: solution to PDE given by average behavior of many random walks
– encapsulated by Feynman-Kac formula
• Another direction: behavior of many random walks can be described by PDE
– encapsulated by Fokker-Planck equation (a.k.a. Kolmogorov forward equation)
• Not one single equation! Rather, for any SDE, can ask for its Fokker-Planck
equation
– often yields a familiar PDE (heat equation, convection-diffusion equation, etc.)
– solution is probability distribution p(x, t) describing location of Xt in space & time
Translating SDE to Fokker-Planck Equation
• Suppose we have an SDE, and want corresponding Fokker-Planck PDE
• We already know how to do this for many SDEs…
– just opposite direction of what we did for Feynman-Kac!
• Q: For instance, what's the PDE for pure Brownian motion?
– A: Heat equation
• Q: What's the PDE for a general diffusion process?
– A: (Screened) convection-diffusion equation
• For general SDE, there may or may not be an established PDE…
Example—Langevin Equation More generally: have to solve
numerically! (e.g., finite differences)

• Let’s return to Langevin equation from beginning (a.k.a. Ornstein-Uhlenbeck process)

• Rare case: simple enough to solve for probability p(v,t) in closed form!

SDE
time evolution damping Langevin just like linear ODE… approaches α2/2γ
of velocity force force

v0
PDE
diffusion convection

starts at v0
Using the Solution of the Fokker-Planck Equation
• Suppose we've solved for the distribution p(x)
• Can now compute, e.g., the expected value of any other quantity a(x)
by integration
– e.g., suppose we want the average temperature of one of our “pollen
grains” at some stopping time T

Will see a lot more of this kind of thing as we develop MCMC algorithms!
 Thanks!

MONTE CARLO METHODS

AND APPLICATIONS
Carnegie Mellon University | 21-387 / 15-327 / 15-627 / 15-860 | Fall 2023