San José State University

Math 263: Stochastic Processes

Poisson processes

Dr. Guangliang Chen

This lecture is based on the following textbook sections:
• Section 5.3 (5.3.1 - 5.3.4)

• Section 5.4 (5.4.1 - 5.4.2)

Outline of the presentation

• Counting processes

• Poisson processes

• Generalizations of Poisson processes

HW6: To be assigned in Canvas

 Math 263, Poisson processes

Def 0.1. A stochastic process {N (t ), t ≥ 0} is called a counting process

if N (t ) represents the total number of events that occur by time t .

N (t) b
|
|

b bc
|

b bc
|

b bc
|
|

b bc

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 3/35
 Math 263, Poisson processes

Remark. Any counting process N (t ) must satisfy:

• N (t ) ≥ 0;

• N (t ) is integer valued;

• If s < t , then N (s) ≤ N (t );

• For any s < t , N (t ) − N (s) equals the number of events that occur
in the interval (s, t ].

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 4/35
 Math 263, Poisson processes

Def 0.2. Let {N (t ), t ≥ 0} be a counting process.

• It is said to have independent increments, if the numbers of events

that occur in disjoint time intervals are independent;

• It is said to have stationary increments, if the distribution of the

number of events that occur in any interval of time depends only on
the length of the time interval.

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 5/35
 Math 263, Poisson processes

Below is the definition of the a Poisson process (An equivalent alternative

definition using o(h) is given in the book).

Def 0.3. The counting process {N (t ), t ≥ 0} is called a Poisson process

with rate λ, if

• N (0) = 0;

• The process has independent (and stationary) increments;

• The number of events in any interval of length t is Poisson distributed

with mean λt . That is, for any s, t ≥ 0:
(λt )n
P (N (t + s) − N (s) = n) = e −λt , n = 0, 1, 2, . . .
n!

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 6/35
 Math 263, Poisson processes

Consider a Poisson process:

• Denote the time of the first event by T1 .

• For any n > 1, let Tn denote the elapsed time between the (n − 1)st
and the n th event.

The sequence {Tn , n = 1, 2, . . .} is called the sequence of interarrival times.

T1 T2 Tn
b b b

| × × × × time
0

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 7/35
 Math 263, Poisson processes

Theorem 0.1. {Tn , n = 1, 2, . . .} are independent identically distributed ex-

ponential random variables with parameter λ.

Proof. The assumption of stationary and independent increments is ba-

sically equivalent to asserting that, at any point in time, the process
probabilistically restarts itself.

Therefore, Tn are independently and identically distributed, and it is enough

to determine the distribution of T1 :

P (T1 > t ) = P (N (t ) = 0) = e −λt −→ T1 ∼ Exp(λ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 8/35
 Math 263, Poisson processes

The following result then follows immediately (we proved this result at the
beginning of the semester).
Corollary 0.2. The total waiting time for n occurrences of the event has a
Gamma distribution (with parameters n, λ), i.e.,

S n = T1 + · · · + Tn ∼ Gamma(n, λ)

This implies that

n n
E(S n ) = , Var(S n ) = .
λ λ2

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 9/35
 Math 263, Poisson processes

Example 0.1. Suppose that people immigrate into a territory at a Poisson

rate λ = 10 per week.

(a) What is the expected time until the 100th immigrant arrives?

(b) What is the probability that the elapsed time between the 100th and
the 101st arrival exceeds one day?

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 10/35
 Math 263, Poisson processes

It is also possible to define a Poisson process from a sequence of iid

exponential random variables {Tn , n = 1, 2, . . .} with rate λ.
Theorem 0.3. Let

N (t ) = max{n ≥ 0 : T1 + · · · + Tn ≤ t }

Then {N (t ), t ≥ 0} is a Poisson process with rate λ.

Proof. Fix an integer n ≥ 0. Then S n = T1 + · · · + Tn ∼ Gamma(n, λ) and it

is independent of Tn+1 .

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 11/35
 Math 263, Poisson processes

By definition of N (t ),

P (N (t ) = n) = P (S n ≤ t , S n + Tn+1 > t )
Z tZ ∞
= f S n (s) f Tn+1 (x) dx ds
0 t −s
Z t
= P (Tn+1 > t − s) f S n (s) ds
0
t λ(λs)n−1 e −λs
Z
= e −λ(t −s) ds
0 (n − 1)!
(λt )n e −λt
= .
n!
This shows that N (t ) ∼ Pois(λt ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 12/35
 Math 263, Poisson processes

Consider a Poisson process {N (t ), t ≥ 0} with rate λ, and suppose that

each time the event occurs, it is classified as either a type I or a type II
event, which occurs with probability p or 1 − p respectively, independently
of all other events.

| u
× u
× × u
× |
time
0 t

Let N1 (t ) and N2 (t ) denote respectively the number of type I and type II

events occurring in [0, t ]. Note that N (t ) = N1 (t ) + N2 (t ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 13/35
 Math 263, Poisson processes

Theorem 0.4. {N1 (t ), t ≥ 0} and {N2 (t ), t ≥ 0} are both Poisson processes

having respective rates λp and λ(1 − p). Furthermore, the two processes
are independent.

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 14/35
 Math 263, Poisson processes

Proof. For fixed t > 0,

∞
X
P (N1 (t ) = k) = P (N1 (t ) = k | N (t ) = n)P (N (t ) = n)
n=k
à !
∞ n (λt )n −λt
p k (1 − p)n−k ·
X
= e
n=k k n!
∞ p k (1 − p)n−k (λt )n
e −λt
X
=
n=k k!(n − k)!
p k (λt )k −(λp)t X
∞ (1 − p)m (λt )m
= e e −λ(1−p)t
k! m=0 m!
((λp)t )k −(λp)t
= e , k = 0, 1, 2, . . .
k!
This shows that N1 (t ) ∼ Pois(λp) and similarly, N2 (t ) ∼ Pois(λ(1 − p)).
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 15/35
 Math 263, Poisson processes

To prove that the two processes are independent, consider for any k, j ≥ 0:
P (N1 (t ) = k, N2 (t ) = j ) = P (N1 (t ) = k, N (t ) = k + j )
= P (N1 (t ) = k | N (t ) = k + j ) · P (N (t ) = k + j )
à !
k+j k (λt )k+ j −λt
= p (1 − p) j · e
k (k + j )!
p k (1 − p) j (λt )k+ j −λt
= e
k! j !
(λpt )k −λpt (λ(1 − p)t ) j −λ(1−p)t
= e · e
k! j!
= P (N1 (t ) = k) · P (N2 (t ) = j ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 16/35
 Math 263, Poisson processes

Example 0.2 (Cont’d). If each immigrant is of certain descent with

probability 15 , then what is the probability that no people of that descent
will emigrate to the territory during the next two weeks?

1
Answer. P (N1 (t ) = 0) = e −2·(10· 5 ) = .0183

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 17/35
 Math 263, Poisson processes

Example 0.3 (The Coupon Collecting Problem). There are m different

types of coupons. Each time a person collects a coupon (independently
of ones previously obtained), it is a type j coupon with probability p j
(p j > 0, p j = 1). Let N denote the number of coupons one needs to
P

collect in order to have a complete collection of at least one of each type.

Find E[N ].

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 18/35
 Math 263, Poisson processes

Solution. Suppose that coupons are collected at times chosen according

to a Poisson process with rate λ = 1. Let N j (t ) denote the number of
type j coupons collected by time t . Then {N j (t ), t ≥ 0}, j = 1, . . . , m are
independent Poisson processes with respective rates λp j = p j .

Let X j denote the time of the first event of the j th process. Then

X = max X j
1≤ j ≤m

is the time at which a complete collection is obtained.

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 19/35
 Math 263, Poisson processes

Since the X j are independent exponential random variables with respective

rates p j , it follows that
m
(1 − e −p j t ).
Y
P (X < t ) = P (X 1 < t , . . . , X m < t ) =
j =1

Therefore,
Z ∞ Z ∞ m
(1 − e −p j t ) dt .
Y
E(X ) = P (X > t ) dt = 1−
0 0 j =1

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 20/35
 Math 263, Poisson processes

It remains to relate it to E(N ), the expected number of coupons it takes.

To compute it, let Ti denote i th interarrival time of the Poisson process
N (t ) = N1 (t ) + · · · + Nm (t ). It is easy to see that

N
X
X= Ti
i =1

from which we obtain that

E(X ) = E(N ) · E(T1 ) = E(N ).

Therefore, Z ∞ m
(1 − e −p j t ) dt .
Y
E(N ) = 1−
0 j =1

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 21/35
 Math 263, Poisson processes

Example 0.4 (The Coupon Collecting Problem, cont’d). What is the

expected number of coupon types that appear only once in the complete
collection?

Solution. Let I i be the indicator variable on whether there is only a single

type i coupon in the final set, and N1 = m i =1 I i . Then
P

m
X m
X
E(N1 ) = E(I i ) = P (I i = 1).
i =1 i =1

Note that there will be a single type i coupon in the final set if any other
coupon type has appeared before the second coupon of type i is obtained.

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 22/35
 Math 263, Poisson processes

Let S i ∼ Gamma(2, p i ) denote the time at which the second type i coupon
is obtained. Then
à !
\
P (I i = 1) = P {X j < S i }
j 6=i
à !
Z ∞
{X j < S i } | S i = x p i2 xe −p i x dx
\
= P
0 j 6=i
à !
Z ∞
S i = x p i2 xe −p i x dx
{X j < x} | ///////
\
= P
0 j 6=i
Z ∞
1 − e −p j x p i2 xe −p i x dx
Y¡ ¢
=
0 j 6=i

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 23/35
 Math 263, Poisson processes

It follows that
Z m
∞X
1 − e −p j x p i2 xe −p i x dx
Y¡ ¢
E(N1 ) =
0 i =1 j 6=i

Remark. In the coupon collector problem when m = 2:

1
E(N ) = −1
p1 p2
E(N1 ) = 2 − p 12 − p 22

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 24/35
 Math 263, Poisson processes

The next probability calculation related to Poisson processes is the prob-

ability that n events occur in one Poisson process before m events have
occurred in a second and independent Poisson process.

More formally, let {N1 (t ), t ≥ 0} and {N2 (t ), t ≥ 0} be two independent

Poisson processes having respective rates λ1 and λ2 .

Also, let S n(1) denote the time of the n th event of the first process, and
(2)
Sm the time of the m th event of the second process.

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 25/35
 Math 263, Poisson processes

Theorem 0.5.
à !µ ¶n µ ¶k−n
¡ (1) (2)
¢ n+m−1
X k − 1 λ1 λ2
P Sn < Sm =
k=n n − 1 λ1 + λ2 λ1 + λ2

Proof. In the special case of n = m = 1, where S 1(i ) ∼ Exp(λi ), i = 1, 2 are

independent, the formula reduces to
³ ´ λ1
P S 1(1) < S 1(2) = .
λ1 + λ2

This has been proved at the beginning of the semester.

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 26/35
 Math 263, Poisson processes

To prove the general result, observe that each event that occurs is going
to be

• an event of the N1 (t ) process with probability p = λ1λ+λ

1
2
, or

• an event of the N2 (t ) process with probability 1 − p = λ1λ+λ

2
2
,

independently of all that have previously occurred.

This question is thus equivalent to getting n heads before m tails when

repeatedly flipping a coin with probability of heads p .

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 27/35
 Math 263, Poisson processes

Suppose we are told that exactly one event of a Poisson process has taken
place by time t , and we are asked to determine the distribution of the time
at which the event occurred.
Theorem 0.6.
T1 | N (t ) = 1 ∼ Unif(0, t ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 28/35
 Math 263, Poisson processes

Proof. For s < t ,

P (T1 < s, N (t ) = 1)
P (T1 < s | N (t ) = 1) =
P (N (t ) = 1)
P (N (s) = 1, N (t ) − N (s) = 0)
=
P (N (t ) = 1)
P (N (s) = 1)P (N (t ) − N (s) = 0)
=
P (N (t ) = 1)
λse −λs e −λ(t −s)
=
λt e −λt
s
= .
t

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 29/35
 Math 263, Poisson processes

Another interesting result is the joint distribution of the cumulative arrival

times S 1 ≤ · · · ≤ S n when given N (t ) = n .

Theorem 0.7. Given that N (t ) = n , S 1 , . . . , S n have the same distribution as

the order statistics corresponding to n independent uniformly distributed
random variables on (0, t ):
n!
f (s 1 , . . . , s n | N (t ) = n) = , 0 < s1 < · · · < sn < t
tn

(The proof of the theorem as well as its application is in Section 5.3.4)

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 30/35
 Math 263, Poisson processes

Def 0.4. A stochastic process {X (t ), t ≥ 0} is said to be a compound

Poisson process if it has the form
NX
(t )
X (t ) = Yi
i =1

where

• {N (t ), t ≥ 0} is a Poisson process (with rate λ), and

• {Yi } are iid random variables that are independent of N (t ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 31/35
 Math 263, Poisson processes

Theorem 0.8. In a compound Poisson process,

E(X (t )) = λt E(Y1 ), Var(X (t )) = λt E(Y12 )

Proof. Let µ, σ2 be the expectation and variance of each Yi . By direct

calculation:

E(X (t )) = E(N (t ))µ = λt E(Y1 ),

Var(X (t )) = µ2 Var(N (t )) + σ2 E(N (t ))
= λt (µ2 + σ2 )
= λt E(Y12 ).

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 32/35
 Math 263, Poisson processes

Example 0.5. Suppose that families migrate to an area at a Poisson

rate λ = 2 per week. If the number of people in each family is inde-
pendent and takes on the values 1, 2, 3, 4, 5 with respective probabilities
1/4, 1/4, 1/3, 1/12, 1/12, then what is the expected value and variance of
the number of individuals migrating to this area during a fixed six-week
period?

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 33/35
 Math 263, Poisson processes

Answer. Let N (t ) be the number of families that migrate to the area over
t weeks, and Yi the size of each family. Then the number of individuals
migrating to this area over t weeks is X (t ) = iN=1
P (t )
Yi .

Since
5 1 41 23
E(Y1 ) = and E(Y12 ) = +1+3+ = ,
2 4 12 3
we have
5 23
E(X (6)) = 2 · 6 · = 30, Var(X (6)) = 2 · 6 · = 92
2 3

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 34/35
 Math 263, Poisson processes

Def 0.5. The counting process {N (t ), t ≥ 0} is called a nonhomogeneous

Poisson process with intensity function λ(t ), t ≥ 0, if

• N (0) = 0

• The process has independent increments

• For any s, t ≥ 0:

P (N (s + t ) − N (s) = n) = e −R(s,t ) R(s, t )n /n!, n = 0, 1, 2, . . .

where
Z s+t
R(s, t ) = λ(y) dy (= λt for constant function λ(y) = λ).
s

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 35/35