Chapter 3

PAIRS OF RANDOM VARIABLES

NGUYỄN THỊ THU THỦY(1)

FACULTY OF MATHEMATICS AND INFORMATICS

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

HANOI – 2024

(1)
Email: [email protected]
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 1/148
HANOI – 2024 1 / 148
CHAPTER OUTLINE

Chapter outline
1 TWO RANDOM VARIABLES
2 COVARIANCE AND CORRELATION
3 BIVARIATE NORMAL DISTRIBUTION
4 CENTER LIMIT THEOREM

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 2/148

HANOI – 2024 2 / 148
LEARNING OBJECTIVES

✍ After careful study of this chapter you should be able to do the following:
1 Use joint probability mass functions and joint probability density functions to
calculate probabilities
2 Calculate marginal and conditional probability distributions from joint probability
distributions
3 Interpret and calculate covariances and correlations between random variables
4 Understand properties of a bivariate normal distribution and be able to draw
contour plots for the probability density function
5 Calculate means and variances for linear combinations of random variables and
calculate probabilities for linear combinations of normally distributed random
variables
6 Understand the central limit theorem
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 3/148
HANOI – 2024 3 / 148
INTRODUCTION

✍ This chapter analyzes experiments that produce two random variables, X and Y .
1 FX,Y (x, y), the joint cumulative distribution function of two random variables.
2 pX,Y (x, y), the joint probability mass function for two discrete random variables.
3 fX,Y (x, y), the joint probability density function of two continuous random variables.
4 Functions of two random variables.
5 Conditional probability and independence.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 4/148

HANOI – 2024 4 / 148
 3.1 TWO RANDOM VARIABLES

CONTENT
1 3.1 TWO RANDOM VARIABLES
3.1.1 Joint Probability Distributions
3.1.2 Marginal Probability Distributions
3.1.3 Conditional Probability Distributions
3.1.4 Independence
Exercises for Section 3.1
2 3.2 COVARIANCE AND CORRELATION
3.2.1 Covariance. Covariance Matrix
3.2.2 Correlation Coefficient
Exercises for Section 3.2
3 3.3 BIVARIATE NORMAL DISTRIBUTION
3.3.1 Joint Probability Distributions
3.3.2 Marginal Probability Distributions
3.3.3 Conditional Probability Distributions
3.3.4 Correlation
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 5/148
HANOI – 2024 5 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

■ If X and Y are discrete random variables, the joint probability distribution of X and
Y is a description of the set of points (x, y) in the range of (X, Y ) along with the
probability of each point.
■ The joint probability distribution of two random variables is sometimes referred to as
the bivariate probability distribution or bivariate distribution of the random variables.
■ One way to describe the joint probability distribution of two discrete random
variables is through a joint probability mass function.
■ Also, P (X = x and Y = y) is usually written as P (X = x, Y = y).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 6/148

HANOI – 2024 6 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

Definition 1 (Joint Probability Mass Function)

The joint probability mass function of the discrete random variables X and Y ,
denoted as pX,Y (x, y), satisfies
■ p
X,Y (x, y) ≥ 0.
P P
y∈SY pX,Y (x, y) = 1.
■
x∈SX
■ pX,Y (x, y) = P (X = x, Y = y).

✍ Corresponding to SX , the range of a single discrete random variable, we use the

notation SX,Y to denote the set of possible values of the pair (X, Y ). That is,

SX,Y = {(x, y) | pX,Y (x, y) > 0}.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 7/148

HANOI – 2024 7 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

✍ Joint Probability Distribution
HH Y P
HH y1 ... yj ... yn
X HH j
x1 p11 ... p1j ... p1n P (X = x1 )
.. .. .. .. .. .. ..
. . . . . . .
xi pi1 ... pij ... pin P (X = xi )
.. .. .. .. .. .. ..
. . . . . . .
xP m pm1 ... pmj ... pmn P
P(XP= xm )
P (Y = y1 ) ... P (Y = yj ) . . . P (Y = yn ) i j = 1
i

where
Pm P pij = P (X = xi , Y = yj ) for all i = 1, . . . , m, j = 1, . . . , n. Note that,
n
i=1 j=1 pij = 1.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 8/148
HANOI – 2024 8 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

Theorem 1
For discrete random variables X and Y and any set B in the (X, Y ) plane, the
probability of the event {(X, Y ) ∈ B} is
X
P (B) = pX,Y (x, y) (1)
(x,y)∈B

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 9/148

HANOI – 2024 9 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

✍ There are various ways to represent a joint PMF. We use three of them in the
following example: a list, a matrix, and a graph.
Example 1
Three ballpoint pens are selected at random from a box that contains 5 blue pens, 4
red pens, and 3 green pens. If X is the number of blue pens selected and Y is the
number of red pens selected, find
(a) The joint probability function PX,Y (x, y).
(b) P [(X, Y ) ∈ A], where A is the region {(x, y) | x + y ≤ 1}.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 10/148

HANOI – 2024 10 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

Solution.
(a) The possible pairs of values (x, y) are (0, 0); (0, 1); (0, 2); (0, 3); (1, 0); (1, 1);
(1, 2); (2, 0); (2, 1); and (3, 0). Now, for x = 0, 1, 2, 3; y = 0, 1, 2, 3, and x + y ≤ 3,
(C5x )(C4y )(C33−x−y )
P (X = x, Y = y) = 3
.
C12
A representation of pX,Y (x, y) is the matrix:
pX,Y (x, y) y = 0 y = 1 y = 2 y = 3
x=0 1/220 3/55 9/110 1/55
x=1 3/44 3/11 3/22 0
x=2 3/22 2/11 0 0
x=3 1/22 0 0 0
Table 1: Joint probability distribution in Example 1(a)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 11/148

HANOI – 2024 11 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

(b) The probability that (X, Y ) fall in the region A is

1 3 3 7
P [(X, Y ) ∈ A] = pX,Y (0, 0) + pX,Y (0, 1) + pX,Y (1, 0) = + + = .
220 55 44 55

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 12/148

HANOI – 2024 12 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

Example 2
Test two integrated circuits one after the other. On each test, the possible outcomes
are A (accept) and R (reject). Assume that all circuits are acceptable with
probability 0.9 and that the outcomes of successive tests are independent. Count the
number of acceptable circuits X and count the number of successful tests Y before
you observe the first reject. (If both tests are successful, let Y = 2.)
(a) Find the joint PMF of X and Y .
(b) Find the probability of the event B that X, the number of acceptable circuits,
equals Y , the number of tests before observing the first failure.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 13/148

HANOI – 2024 13 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

Solution.
(a) The sample space of the experiment is SX,Y = {AA, AR, RA, RR}. We compute

P (AA) = 0.81, P (AR) = P (RA) = 0.09, P (RR) = 0.01.

■ Each outcome specifies a pair of values X and Y . Let g(s) be the function that
transforms each outcome s in the sample space S into the pair of random variables
(X, Y ). Then

g(AA) = (2, 2), g(AR) = (1, 1), g(RA) = (1, 0), g(RR) = (0, 0).
■ For each pair of values (x, y), pX,Y (x, y) is the sum of the probabilities of the
outcomes for which X = x and Y = y. For example, pX,Y (1, 1) = P (AR).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 14/148

HANOI – 2024 14 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

■ The joint PMF can be given as a set of labeled points in the (x, y) plane where
each point is a possible value (probability > 0) of the pair (x, y), or as a simple list:


 0.81, x = 2, y = 2,
0.09, x = 1, y = 1,



PX,Y (x, y) = 0.09, x = 1, y = 0,




 0.01, x = 0, y = 0,
0, otherwise.


(b) Mathematically, B is the event (X = Y ). The elements of B with nonzero

probability are B ∩ SX,Y = {(0, 0), (1, 1), (2, 2)}. Therefore,

P (B) = pX,Y (0, 0) + pX,Y (1, 1) + pX,Y (2, 2) = 0.01 + 0.09 + 0.81 = 0.91.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 15/148

HANOI – 2024 15 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(a) Joint Probability Mass Function

■ A second representation of pX,Y (x, y) is the matrix:

pX,Y (x, y) y = 0 y = 1 y = 2
x=0 0.01 0 0
x=1 0.09 0.09 0
x=2 0 0 0.81

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 16/148

HANOI – 2024 16 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(b) Joint Cumulative Distribution Function

Definition 2 (Joint Cumulative Distribution Function)

The joint cumulative distribution function of random variables X and Y is

FX,Y (x, y) = P (X < x, Y < y) (2)

✍ The joint cumulative distribution function of discrete random variables X and Y is

XX
FX,Y (x, y) = pij (3)
xi <x yj <y

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 17/148

HANOI – 2024 17 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(b) Joint Cumulative Distribution Function

Theorem 2
For any pair of random variables, X and Y ,
1 0 ≤ FX,Y (x, y) ≤ 1.
2 FX,Y (−∞, y) = FX,Y (x, −∞) = 0.
3 If x < x1 and y < y1 , then FX,Y (x, y) ≤ FX,Y (x1 , y1 ).
4 FX,Y (+∞, +∞) = 1.
5 If x1 < x2 , y1 < y2 , then

P (x1 ≤ X < x2 , y1 ≤ Y < y2 ) = FX,Y (x2 , y2 ) − FX,Y (x2 , y1 ) − FX,Y (x1 , y2 )

+ FX,Y (x1 , y1 ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 18/148

HANOI – 2024 18 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

Definition 3 (Joint Probability Density Function)

The joint probability density function of the continuous random variables X and Y
is a function fX,Y (x, y) with the property

Zx Zy
FX,Y (x, y) = fX,Y (u, v)dvdu (4)
−∞ −∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 19/148

HANOI – 2024 19 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

✍ Given FX,Y (x, y), Definition 3 implies that fX,Y (x, y) is a derivative of the CDF.
Theorem 3

∂ 2 FX,Y (x, y)
fX,Y (x, y) = .
∂x∂y

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 20/148

HANOI – 2024 20 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

✍ Properties (c) and (d) of Theorem 2 for the CDF FX,Y (x, y) imply corresponding
properties for the PDF.
Theorem 4
A joint PDF fX,Y (x, y) has the following properties:
1 fX,Y (x, y) ≥ 0 for all (x, y).
Z +∞ Z +∞
2 fX,Y (x, y)dxdy = 1.
−∞ −∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 21/148

HANOI – 2024 21 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

Theorem 5
For any region A of two-dimensional space,
ZZ
P [(X, Y ) ∈ A] = fX,Y (x, y)dxdy.
A

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 22/148

HANOI – 2024 22 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

Example 3
Random variables X and Y have joint PDF
(
k, 0 ≤ x ≤ 6, 0 ≤ y ≤ 3,
fX,Y (x, y) =
0, otherwise.

(a) Find the constant k.

(b) Find P (A) = P (2 ≤ X < 4, 2 ≤ Y < 3)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 23/148

HANOI – 2024 23 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

y the area of nonzero probability.

A Theorem 4 states that the integral of
3
the joint PDF over this rectangle is 1:
2
Z+∞ Z+∞
0 x 1= fX,Y (x, y)dxdy
2 4 6
−∞ −∞
Figure 1: Domain 0 ≤ x ≤ 6, 0 ≤ y ≤ 3 in Z6 Z3
Example 3 = dx kdy = 18k.
0 0
Solution.
(a) The large rectangle in the diagram is Therefore, k = 1/18.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 24/148

HANOI – 2024 24 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(c) Joint Probability Density Function

(b) The small cyan rectangle in the diagram is the event A = (2 ≤ X < 4, 2 ≤ Y < 3).
P (A) is the integral of the PDF over this rectangle, which is

P [(X, Y ) ∈ A] = P (2 ≤ X < 4; 2 ≤ Y < 3)

Z4 Z3
1 1
= dx dy = .
18 9
2 2

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 25/148

HANOI – 2024 25 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(d) Probability Distribution of a Function of Two Random Variables

✍ We perform an experiment and observe sample values of two random variables X and
Y . Based on our knowledge of the experiment, we have a probability model for X and Y
embodied in the joint PMF pX,Y (x, y) or a joint PDF fX,Y (x, y). After performing the
experiment, we calculate a sample value of the random variable W = g(X, Y ). The
mathematical problem is to derive a probability model for W .
■ When X and Y are discrete random variables, S
W , the range of W , is a countable
set corresponding to all possible values of g(X, Y ). Therefore, W is a discrete
random variable and has a PMF PW (w).
■ When X and Y are continuous random variables and g(x, y) is a continuous

function, W = g(X, Y ) is a continuous random variable. To find the PDF, fW (w),

it is usually helpful to first find the CDF FW (w) = P (W < w) and then calculate
the derivative.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 26/148
HANOI – 2024 26 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(d) Function of Two Random Variables

Example 4
In Example 3, let W = max{X, Y }. Find the PDF fW (w) of W .

Solution.
■ We have

1
ZZ
FW (w) = P [max(X, Y ) < w] = P (X < w, Y < w) = dxdy,
18
A

where, A = {(x, y) ∈ R2 | (x < w, y < w)}.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 27/148

HANOI – 2024 27 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(d) Function of Two Random Variables

■ If w ≤ 0, FW (w) = 0.
Z w Z w
1 w2
■ If 0 < w ≤ 3, FW (w) = dx 18
dy = 18
.
Z0 w Z0 3
1 w
■ If 3 < w ≤ 6, FW (w) = dx 18
dy = 6
.
0 0
■ If w > 6, FW (w) = 1.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 28/148

HANOI – 2024 28 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(d) Function of Two Random Variables

y y
3 3
w

0 x 0 x
w 3 6 3 w 6
(a) (b)

Figure 2: (a) Domain (0 < w ≤ 3); (b) Domain (3 < w ≤ 6)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 29/148

HANOI – 2024 29 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(d) Function of Two Random Variables

■ Hence, 

 0 if w ≤ 0,
 w2

18
if 0 < w ≤ 3,
FW (w) = w
 if 3 < w ≤ 6,
6


1 if w > 6.
■ Therefore, 
w

9 if 0 < w ≤ 3,
1
fW (w) = if 3 < w ≤ 6,
6
0 if w ≤ 0, w > 6.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 30/148

HANOI – 2024 30 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(e) Expected Values of a function of two random variables

✍ There are many situations in which we are interested only in the expected value of a
derived random variable W = g(X, Y ), not the entire probability model. In these
situations, we can obtain the expected value directly from pX,Y (x, y) or fX,Y (x, y)
without taking the trouble to compute pW (w) or fW (w).
Theorem 6

X X
E(W ) = E[g(X, Y )] = g(x, y)pX,Y (x, y) (W : discrete) (5)
x∈SX y∈SY
Z∞ Z∞
E(W ) = E[g(X, Y )] = g(x, y)fX,Y (x, y)dxdy (W : continuous). (6)
−∞ −∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 31/148

HANOI – 2024 31 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(e) Expected Values of a function of two random variables

Example 5
For X and Y in Example 1, using (5),

E(XY ) = (1)(1)(3/11) + (1)(2)(3/2) + (2)(1)(2/11) = 40/11.

Example 6
For X and Y in Example 3, using (6),
Z6  Z3 Z6
1  1 36
E(XY ) = xy dy dx = xdx = = 4.5.
18 4 8
0 0 0

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 32/148

HANOI – 2024 32 / 148
 3.1 TWO RANDOM VARIABLES 3.1.1 Joint Probability Distributions

(e) Expected Values of a function of two random variables

Theorem 7

E[g1 (X, Y ) + . . . + gn (X, Y )] = E[g1 (X, Y )] + . . . + E[gn (X, Y )].

Theorem 8
For any two random variables X and Y , E(X + Y ) = E(X) + E(Y ).

Theorem 9
The variance of the sum of two random variables is

Var(X + Y ) = Var(X) + Var(Y ) + 2E{[X − E(X)][Y − E(Y )]}.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 33/148
HANOI – 2024 33 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

Theorem 10
For discrete random variables X and Y with joint PMF pX,Y (x, y),
X X
pX (x) = pX,Y (x, y), pY (y) = pX,Y (x, y) (7)
y∈SY x∈SX

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 34/148

HANOI – 2024 34 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

✍
(a) Marginal probability distribution of X is
X x1 x2 ... xm
P (X = xi ) P (X = x1 ) P (X = x2 ) . . . P (X = xm )
(b) Marginal probability distribution of Y is
Y y1 y2 ... yn
P (Y = yj ) P (Y = y1 ) P (Y = y2 ) . . . P (Y = yn )

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 35/148

HANOI – 2024 35 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

Example 7
In Example 2, we found the joint PMF of X and Y to be

pX,Y (x, y) y=0 y=1 y=2

x=0 0.01 0 0
x=1 0.09 0.09 0
x=2 0 0 0.81

Find the marginal PMFs for the random variables X and Y .

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 36/148

HANOI – 2024 36 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

Solution.
■ To find p (x), we note that both X and Y have range {0, 1, 2}. Theorem 1 gives
X

2
X 2
X
pX (0) = pX,Y (0, y) = 0.01, pX (1) = pX,Y (1, y) = 0.18
y=0 y=0
2
X
pX (2) = pX,Y (2, y) = 0.81, pX (x) = 0, x ̸= 0, 1, 2.
y=0

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 37/148

HANOI – 2024 37 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

■ For the PMF of Y , we obtain
2
X 2
X
pY (0) = pX,Y (x, 0) = 0.10, pY (1) = PX,Y (x, 1) = 0.09
x=0 x=0
X2
pY (2) = pX,Y (x, 2) = 0.81, pY (y) = 0, y ̸= 0, 1, 2.
x=0
■ Hence,
 

0.01, x = 0, 
 0.1, y = 0,

0.18, 
x = 1, 0.09, y = 1,
pX (x) = pY (y) =

0.81, x = 2, 
 0.81, y = 2,
 
0, otherwise. 0, otherwise.
 

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 38/148

HANOI – 2024 38 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

Example 8
Referring to the matrix representation of pX,Y (x, y) in Example 2, we observe that
each value of pX (x) is the result of adding all the entries in one row of the matrix.
Each value of pY (y) is a column sum. We display pX (x) and pY (y) by rewriting the
matrix in Example 2 and placing the row sums and column sums in the margins.
pX,Y (x, y) y = 0 y=1 y=2 y=3 pX (x)
x=0 1/220 3/55 9/110 1/55 35/220
x=1 3/44 3/11 3/22 0 105/220
x=2 3/22 2/11 0 0 70/220
x=3 1/22 0 0 0 10/220
pY (y) 56/220 112/220 48/220 4/220 1

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 39/148

HANOI – 2024 39 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

Note that the sum of all the entries in the bottom margin is 1 and so is the sum of all
the entries in the right margin. The complete marginal PMF, pY (y), appears in the
bottom row of the table, and pX (x) appears in the last column of the table. From the
matrix of pX,Y (x, y), we have marginal probability distribution of X is

X 0 1 2 3
P (X = xi ) 7/44 21/44 7/22 1/22

and marginal probability distribution of Y is

Y 0 1 2 3
P (Y = yj ) 14/55 28/55 12/55 1/55

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 40/148

HANOI – 2024 40 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(a) Marginal Probability Mass Functions

Remark 1
The expected value E(X) and variance Var(X) can be obtained by first calculating
the marginal probability distribution of X and then determining E(X) and Var(X)
by the usual method.

Example 9
Find the expected value E(X) and variance Var(X) of random variable X in
Example 2.
Solution. Using the result of Example 7,
E(X) = (0)(0.01) + (1)(0.18) + (2)(0.81) = 1.8,
Var(X) = (02 )(0.01) + (12 )(0.18) + (22 )(0.81) − (1.8)2 = 0.18.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 41/148
HANOI – 2024 41 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(b) Marginal Cumulative Distribution Functions

The marginal CDF has properties that are direct consequences of the definition. For
example, we note that the event (X < x) suggests that Y can have any value so long as
the condition on X is met. This corresponds to the joint event (X < x, Y < ∞).
Therefore,

FX (x) = P (X < x) = P (X < x, Y < ∞) = lim FX,Y (x, y) = FX,Y (x, ∞) (8)
y→∞

Hence, the marginal cumulative distribution functions are

FX (x) = FX,Y (x, ∞) and FY (y) = FX,Y (∞, y). (9)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 42/148

HANOI – 2024 42 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

Theorem 11
If the joint probability density function of random variables X and Y is fX,Y (x, y),
the marginal probability density functions of X and Y are

Z+∞ Z+∞
fX (x) = fX,Y (x, y)dy, fY (y) = fX,Y (x, y)dx (10)
−∞ −∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 43/148

HANOI – 2024 43 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

Example 10
The joint PDF of X and Y is

 5y
, −1 ≤ x ≤ 1, x2 ≤ y ≤ 1
fX,Y (x, y) = 4
0, otherwise.

Find the marginal PDFs fX (x) and fY (y).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 44/148

HANOI – 2024 44 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

Solution. Set D = {(x, y) ∈ R2 | − 1 ≤ x ≤ 1, x2 ≤ y ≤ 1}. We use Theorem 11 to
find the marginal PDF fX (x).
When x < −1 or when
y
x > 1, fX,Y (x, y) = 0, and therefore fX (x) = 0.
For −1 ≤ x ≤ 1,
1 Z1
5y 5(1 − x4 )
D fX (x) = dy = .
4 8
x2

−1
The complete
O 1 x
expression for the marginal PDF of X is

 5(1 − x4 )
, −1 ≤ x ≤ 1,
fX (x) = 8
0, otherwise.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 45/148
HANOI – 2024 45 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

For the marginal PDF of Y ,
■ We note that for y < 0 or y > 1, f (y) = 0.
Y
■ For 0 ≤ y ≤ 1, we integrate over the horizontal bar marked Y = y. The boundaries
√ √
of the bar are x = − y and x = y. Therefore, for 0 ≤ y ≤ 1,
√
Zy √
5y 5y x= y 5y 3/2
fY (y) = dx = x √ = .
√
4 4 x=− y 2
− y

The complete marginal PDF of Y is

 3/2
 5y
, 0 ≤ y ≤ 1,
fY (y) =
0, 2 otherwise.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 46/148
HANOI – 2024 46 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

Example 11
The joint PDF of X and Y is
(
kx, if 0 < y < x < 1,
fX,Y (x, y) =
0, otherwise.

(a) Find constant k.

(b) Find the marginal PDFs fX (x) and fY (y).
(c) Find the expected value and variance of X.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 47/148

HANOI – 2024 47 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

Solution. Domain D = {(x, y) | 0 < y < x < 1}.
(a) From Theorem 4, k ≥ 0 and
y
Z+∞ Z+∞ ZZ
1 1= fX,Y (x, y)dxdy = kxdxdy
−∞ −∞ D
y=x Z1 Zx Z1
k
D = dx kxdy = k x2 dx = .
3
0 0 0
0 x
1 Hence k = 3 and
(
3x, if (x, y) ∈ D,
f (x, y) =
0, otherwise.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 48/148
HANOI – 2024 48 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

(b) From Theorem 5,

 x
Z
Z+∞  (
3xdy, 0 < x < 1, 3x2 , 0 < x < 1,


fX (x) = fX,Y (x, y)dy = =

 0 0, otherwise.
−∞ 
0, otherwise,
 1
Z 
Z+∞  3xdx, 0 < y < 1,  3 3 2


− y , 0 < y < 1,
fY (y) = fX,Y (x, y)dx = = 2 2

 y  0, otherwise.
−∞
0, otherwise


Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 49/148

HANOI – 2024 49 / 148
 3.1 TWO RANDOM VARIABLES 3.1.2 Marginal Probability Distributions

(c) Marginal Probability Density Functions

(c) We have
Z1
3
E(X) = x × 3x2 dx = and Var(X) = E(X 2 ) − [E(X)]2
4
0
Z1  3 2
2 2 3
= x × 3x dx − = .
4 80
0

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 50/148

HANOI – 2024 50 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

Conditional Probability Distributions

✍
■ When two random variables are defined in a random experiment, knowledge of one
can change the probabilities that we associate with the values of the other.
■ From the notation for conditional probability in Chapter 1, we can write conditional
probabilities such as P (Y = 1|X = 3) or P (Y = 3|X < 5).
■ Consequently, the random variables X and Y are expected to be dependent.
Knowledge of the value obtained for X changes the probabilities associated with the
values of Y .
■ Recall that the definition of conditional probability for events A and B is
P (B|A) = P P(A∩B)
(A)
. This definition can be applied with event A defined to be
X = x and event B defined to be Y = y.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 51/148

HANOI – 2024 51 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Definition 4 (Conditional Joint Probability Mass Function)

For discrete random variables X and Y and an event, B with P (B) > 0, the
conditional joint PMF of X and Y given B is

pX,Y |B (x, y) = P [(X = x, Y = y)|B].

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 52/148

HANOI – 2024 52 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Theorem 12
For any event B, a region of the X, Y plane with P (B) > 0,

 pX,Y (x, y)

(x, y) ∈ B,
pX,Y |B (x, y) = P (B) (11)
0, otherwise


Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 53/148

HANOI – 2024 53 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Definition 5 (Conditional Joint Probability Density Function)

Given an event B with P (B) > 0, the conditional joint probability density function
of X and Y is

 fX,Y (x, y) ,

(x, y) ∈ B,
fX,Y |B (x, y) = P (B) (12)
0, otherwise.


Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 54/148

HANOI – 2024 54 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Example 12
X and Y are random variables with joint PDF

1
, 0 ≤ x ≤ 5, 0 ≤ y ≤ 3,
fX,Y (x, y) = 15
0, otherwise.

Find the conditional PDF of X and Y given the event B = {X + Y ≥ 4}.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 55/148

HANOI – 2024 55 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Solution.
■ We calculate P (B) by integrating f
X,Y (x, y) over the region B.

Z3 Z5 Z3
1 1 1
P (B) = dxdy = (1 + y)dy = .
15 15 2
0 4−y 0

■ Definition 5 leads to the conditional joint PDF


2
, 0 ≤ x ≤ 5, 0 ≤ y ≤ 3, x + y ≥ 4,
fX,Y |B (x, y) = 15
0, otherwise.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 56/148

HANOI – 2024 56 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Theorem 13 (Conditional Expected Value)

For random variables X and Y and an event B of nonzero probability, the
conditional expected value of W = g(X, Y ) given B is
X X
E(W |B) = g(x, y)PX,Y |B (x, y) (13)
x∈SX y∈SY

if W is a discrete random variable and if W is a continuous random variable,

Z+∞ Z+∞
E(W |B) = g(x, y)fX,Y |B (x, y)dxdy (14)
−∞ −∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 57/148

HANOI – 2024 57 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Example 13
Continuing Example 12, find the conditional expected value of W = XY given the
event B = {X + Y ≥ 4}.

Solution. From Theorem 13,

Z3 Z5 Z3 Z3
2 1 5 1 123
E(XY |B) = xydxdy = x2 ydy = (9y + 8y 2 − y 3 )dy = .
15 15 4−y 15 20
0 4−y 0 0

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 58/148

HANOI – 2024 58 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(a) Conditioning by an Event

Definition 6 (Conditional Variance)

The conditional variance of the random variable W = g(X, Y ) is

Var(W |B) = E[(W − E(W |B))2 |B].

Theorem 14

Var(W |B) = E(W 2 |B) − (E(W |B))2 .

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 59/148

HANOI – 2024 59 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Definition 7 (Conditional PMF)

For any event Y = y such that PY (y) > 0, the conditional PMF of X given Y = y is

pX|y := pX|(Y =y) (x) = P [(X = x)|(Y = y)] (15)

where
P (X = x, Y = y)
P [(X = x)|(Y = y)] = .
P (Y = y)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 60/148

HANOI – 2024 60 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

✍ The following theorem contains the relationship between the joint PMF of X and Y
and the two conditional PMFs, pX|y (x) and pY |x (y).
Theorem 15
For random variables X and Y with joint PMF pX,Y (x, y), and x and y such that
pX (x) > 0 and pY (y) > 0,

pX,Y (x, y) = pX|(Y =y) (x)pY (y) = pY |(X=x) (y)pX (x) (16)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 61/148

HANOI – 2024 61 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Example 14
In Example 1,
P (X = 0, Y = 1) 3
p(X = 0|Y = 1) = = ,
P (Y = 1) 28
P (X = 1, Y = 1) 15
p(X = 1|Y = 1) = = ,
P (Y = 1) 28
P (X = 2, Y = 1) 10
p(X = 2|Y = 1) = = ,
P (Y = 1) 28
P (X = 3, Y = 1)
p(X = 3|Y = 1) = = 0.
P (Y = 1)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 62/148

HANOI – 2024 62 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Therefore, the conditional probability distribution of X given Y = 1 is

X 0 1 2 3
P (X|Y = 1) 3/28 15/28 5/14 0

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 63/148

HANOI – 2024 63 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Theorem 16 (Conditional Expected Value of a Function)

Let X and Y are discrete random variables. For any y ∈ SY , the conditional expected
value of g(X, Y ) given (Y = y) is
X
E[g(X, Y )|Y = y] = g(x, y)pX|y (x). (17)
x∈SX

The conditional expected value of X given Y = y is a special case of Theorem 16:

X
E([X|Y = y) = xpX|y (x). (18)
x∈SX

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 64/148

HANOI – 2024 64 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Example 15
Returning to Example 14,
3 15 5
E(X|(Y = 1)) = 0 × +1× +2× + 3 × 0 = 1.25.
28 28 14

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 65/148

HANOI – 2024 65 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

✍ Now we consider the case in which X and Y are continuous random variables. We
observe (Y = y) and define the PDF of X given (Y = y). We cannot use B = (Y = y)
in Definition 4 because P (Y = y) = 0. Instead, we define a conditional probability
density function, denoted as fX|y (x) : fX|(Y =y) (x).
Definition 8 (Conditional PDF)
For y such that fY (y) > 0, the conditional PDF of X given (Y = y) is

fX,Y (x, y)
fX|y (x) := fX|(Y =y) (x) = . (19)
fY (y)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 66/148

HANOI – 2024 66 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

✍ Definition 8 implies
fX,Y (x, y)
fY |x (y) := fY |(X=x) (y) = , fX (x) > 0. (20)
fX (x)

Remark 2
Because the conditional probability density function fY |(X=x) (y) is a probability
density function for all y, the following properties are satisfied:
(a) fY |x (y) ≥ 0.
Z +∞
(b) fY |x (y)dy = 1.
−∞
Z
(c) P (Y ∈ B|(X = x)) = fY |x (y)dy for any set B in the range of Y .
B
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 67/148
HANOI – 2024 67 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Example 16
The joint probability density function of random variables X and Y is
(
2, 0 < y < x < 1,
fX,Y (x, y) =
0, otherwise.

For 0 < x < 1, find the conditional PDF fY |x (y). For 0 < y < 1, find the conditional
PDF fX|y (x).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 68/148

HANOI – 2024 68 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Solution.
■ For 0 < x < 1, Theorem 6 implies

Z+∞ Zx
fX (x) = fX,Y (x, y)dy = 2dy = 2x.
−∞ 0

The conditional PDF of Y given X is

(
fX,Y (x, y) 1/x, 0 < y < x,
fY |x (y) = =
fX (x) 0, otherwise.

■ Given X = x, we see that Y is the uniform (0, x) random variable.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 69/148

HANOI – 2024 69 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

■ For 0 < y < 1, Theorem 6 implies

Z+∞ Z1
fY (y) = fX,Y (x, y)dx = 2dx = 2(1 − y).
−∞ y

■ Furthermore, Equation (20) implies

(
fX,Y (x, y) 1/(1 − y), y < x < 1,
fX|y (x) = =
fY (y) 0, otherwise.

■ Conditioned on Y = y, we see that X is the uniform (y, 1) random variable.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 70/148

HANOI – 2024 70 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Example 17
The joint probability density function of random variables X and Y is
(
1
, if 0 < y < x < 1,
fX,Y (x, y) = x
0, otherwise.

(a) Find the PDFs fX (x), fY (y). (b) Find the conditional PDFs fX|y (x), fY |x (y).

Solution.
(a) The PDFs of X and Y are:
Z x
Z+∞ 1
(

x
dy, 0 < x < 1, 1, 0 < x < 1,
fX (x) = f (x, y)dy = 0 =

0, otherwise, 0, otherwise.
−∞
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 71/148
HANOI – 2024 71 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Z 1
Z+∞ 1
(

x
dx, 0 < y < 1, − ln y, 0 < y < 1,
fY (y) = f (x, y)dx = y =
0, otherwise.
0, otherwise,

−∞

(b) The conditional PDFs are

1

fX,Y (x, y) − , 0 < y < x < 1,
fX|y (x) = = x ln y
fY (y)  0, otherwise.

1
fX,Y (x, y)  , 0 < y < x < 1,
fY |x (y) = = x
fX (x) 0, otherwise.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 72/148

HANOI – 2024 72 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Theorem 17

fX,Y (x, y) = fY |X=x (y)fX (x) = fX|Y =y (x)fY (y).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 73/148

HANOI – 2024 73 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Definition 9 (Conditional Expected Value of a Function)

For continuous random variables X and Y and any y such that fY (y) > 0, the
conditional expected value of g(X, Y ) given Y = y is

Z+∞
E[g(X, Y )|Y = y] = g(x, y)fX|y (x)dx. (21)
−∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 74/148

HANOI – 2024 74 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

✍ The conditional expected value of X given (Y = y) is a special case of Definition 9:

Z+∞
E(X|Y = y) = xfX|y (x)dx. (22)
−∞

Definition 10 (Conditional Expected Value)

The conditional expected value E(X|Y ) is a function of random variable Y such
that if Y = y then E(X|Y ) = E(X|Y = y).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 75/148

HANOI – 2024 75 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Example 18
For random variables X and Y in Example 3, we found in Example 16 that the
conditional PDF of X given Y is
(
fX,Y (x, y) 1/(1 − y), y < x < 1,
fX|y (x) = =
fY (y) 0, otherwise.

Find the conditional expected values E(X|Y = y) and E(X|Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 76/148

HANOI – 2024 76 / 148
 3.1 TWO RANDOM VARIABLES 3.1.3 Conditional Probability Distributions

(b) Conditioning by a Random Variable

Solution.
■ Given the conditional PDF f
X|y (x), we perform the integration

Z+∞
E[X|Y = y] = xfX|y (x)dx
−∞
Z1
1 x2 x=1 1+y
= xdx = = .
1−y 2(1 − y) x=y 2
y

■ Since E[X|Y = y] = (1 + y)/2, E[X|Y ] = (1 + Y )/2.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 77/148

HANOI – 2024 77 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

Definition 11 (Independent Random Variables)

Discrete random variables X and Y are independent if and only if

pX,Y (x, y) = pX (x)pY (y). (23)

Continuous random variables X and Y are independent if and only if

fX,Y (x, y) = fX (x)fY (y). (24)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 78/148

HANOI – 2024 78 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

✍ Theorem 15 implies that if X and Y are independent discrete random variables, then

pX|Y =y (x) = pX (x), pY |X=x (y) = pY (y) (25)

Theorem 17 implies that if X and Y are independent continuous random variables,

then

fX|Y =y (x) = fX (x), fY |X=x (y) = fY (y) (26)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 79/148

HANOI – 2024 79 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

Example 19
Random variables X and Y have joint PDF
(
4xy, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
fX,Y (x, y) =
0, otherwise.

Are X and Y independent?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 80/148

HANOI – 2024 80 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

Solution.
■ The marginal PDFs of X and Y are

( (
2x, 0 ≤ x ≤ 1, 2y, 0 ≤ y ≤ 1,
fX (x) = , fY (y) =
0, otherwise 0, otherwise.

■ It is easily verified that fX,Y (x, y) = fX (x)fY (y) for all pairs (x, y) and so we
conclude that X and Y are independent.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 81/148

HANOI – 2024 81 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

Theorem 18
For independent random variables X and Y ,
■ E[g(X)h(Y )] = E[g(X)]E[h(Y )],

■ E(XY ) = E(X)E(Y ),

■ Var(X + Y ) = Var(X) + Var(Y ),

■ E[X|Y = y] = E[X] for all y ∈ S ,

Y
■ E[Y |X = x] = E[Y ] for all x ∈ S .
X
■ P (X ∈ A, Y ∈ B) = P (X ∈ A)P (Y ∈ B) for any sets A and B in the range of

X and Y , respectively.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 82/148

HANOI – 2024 82 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

Example 20
Let the random variables X and Y denote the lengths of two dimensions of a
machined part, respectively. Assume that X and Y are independent random
variables, and further assume that the distribution of X is normal with mean 10.5
millimeters and variance 0.0025 (millimeter)2 and that the distribution of Y is
normal with mean 3.2 millimeters and variance 0.0036 (millimeter)2 . Determine the
probability that 10.4 < X < 10.6 and 3.15 < Y < 3.25.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 83/148

HANOI – 2024 83 / 148
 3.1 TWO RANDOM VARIABLES 3.1.4 Independence

Independence

Solution. Because X and Y are independent,

P (10.4 < X < 10.6, 3.15 < Y < 3.25) = P (10.4 < X < 10.6)P (3.15 < Y < 3.25)
h  10.6 − 10.5   10.4 − 10.5 i h  3.25 − 3.2   3.15 − 3.2 i
= Φ −Φ × Φ −Φ
 0.05  0.05  0.06 0.06
= Φ(2) − Φ(−2) Φ(0.83) − Φ(−0.83)
= (2 × 0.97725 − 1)(2 × 0.79673 − 1) = 0.5665.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 84/148

HANOI – 2024 84 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(a) Discrete Random Variables

Exercise 1
Random variables X and Y have the joint PMF
(
cxy, x = 1, 2, 4, y = 1, 3,
PX,Y (x, y) =
0, otherwise.

(a) What is the value of the constant c? Sol. 1/28.

(b) What is P (Y < X)?
(c) What is P (Y > X)?
(d) What is P (Y = X)?
(e) What is P (Y = 3)?
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 85/148
HANOI – 2024 85 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(a) Discrete Random Variables

Exercise 2
Random variables X and Y have the joint PMF
(
c|x + y|, x = −2, 0, 2, y = −1, 0, 1,
PX,Y (x, y) =
0, otherwise.

(a) What is the value of the constant c? Sol. 1/14.

(b) What is P (Y < X)?
(c) What is P (Y > X)?
(d) What is P (Y = X)?
(e) What is P (X < 1)?
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 86/148
HANOI – 2024 86 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(a) Discrete Random Variables

Exercise 3
Given the random variables X and Y in Problem 1, find
(a) The marginal PMFs pX (x) and pY (y).
(b) The expected values E(X) and E(Y ). Sol. 3 and 5/2.
(c) The standard deviations σX and σY .

Exercise 4
Given the random variables X and Y in Problem 2, find
(a) The marginal PMFs pX (x) and pY (y),
(b) The expected values E(X) and E(Y ),
(c) The standard deviations σX and σY .
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 87/148
HANOI – 2024 87 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(a) Discrete Random Variables

Exercise 5
Random variables X and Y have the joint probability distribution
H
HH Y
1 2 3
X H
HH
1 0.12 0.15 0.03
2 0.28 0.35 0.07

(a) Are X and Y independent?

(b) Find the marginal probability distribution of X and Y .
(c) Find the probability distribution of Z, where Z = XY .
(d) Find E(Z). Proof E(Z) = E(X)E(Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 88/148

HANOI – 2024 88 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(a) Discrete Random Variables

Exercise 6
Given random variables X and Y in Problem 2 and the function W = X + 2Y , find
(a) The probability mass function PW (w).
(b) The expected value E(W ). Sol. 0.
(c) P (W > 0). Sol. 3/7.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 89/148

HANOI – 2024 89 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 7
Random variables X and Y have the joint PDF
(
c, x + y ≤ 1, x ≥ 0, y ≥ 0,
fX,Y (x, y) =
0, otherwise.

(a) What is the value of the constant c? Sol. 2.

(b) What is P (X ≤ Y )?
(c) What is P (X + Y ≤ 1/2)?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 90/148

HANOI – 2024 90 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 8
Random variables X and Y have joint PDF
(
cxy 2 , 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
fX,Y (x, y) =
0, otherwise

(a) Find the constant c.

(b) Find P (X > Y ) and P (Y < X 2 ).
(c) Find P [min(X, Y ) ≤ 1/2]. Sol. 11/32.
(d) Find P [max(X, Y ) ≤ 3/4]. Sol. 0.237.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 91/148

HANOI – 2024 91 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 9
X and Y are random variables with the joint PDF
(
2, x + y ≤ 1, x ≥ 0, y ≥ 0,
fX,Y (x, y) =
0, otherwise.

(a) What is the marginal PDF fX (x)?

(b) What is the marginal PDF fY (y)?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 92/148

HANOI – 2024 92 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 10
Random variables X and Y have the joint PDF
(
cy, 0 ≤ y ≤ x ≤ 1,
fX,Y (x, y) =
0, otherwise.

(a) Draw the region of nonzero probability.

(b) What is the value of the constant c?
(c) What is FX (x)?
(d) What is FY (y)?
(e) What is P (Y ≤ X/2)? Sol. 1/4.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 93/148
HANOI – 2024 93 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 11
Random variables X and Y have joint PDF
(
x + y, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
fX,Y (x, y) =
0, otherwise.

Let W = max(X, Y ).
(a) What is SW , the range of W ?
(b) Find FW (w) and fW (w).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 94/148

HANOI – 2024 94 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 12
Random variables X and Y have joint PDF
(
5x2 /2, −1 ≤ x ≤ 1, 0 ≤ y ≤ x2 ,
fX,Y (x, y) =
0, otherwise.

Let A = {Y ≤ 1/4}.
(a) What is the conditional PDF fX,Y |A (x, y)?
(b) What is fY |A (y)?
(c) What is E(Y |A)?
(d) What is fX|A (x)?
(e) What is E(X|A)?
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 95/148
HANOI – 2024 95 / 148
 3.1 TWO RANDOM VARIABLES Exercises for Section 3.1

(b) Continuous Random Variables

Exercise 13
X and Y have joint PDF
(
(4x + 2y)/3, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
fX,Y (x, y) =
0, otherwise.

(a) For which values of y is fX|Y =y (x) defined? What is fX|Y =y (x)?
(b) For which values of x is fY |X=x (y) defined? What is fY |X=x (y)?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 96/148

HANOI – 2024 96 / 148
 3.2 COVARIANCE AND CORRELATION

CONTENT
1 3.1 TWO RANDOM VARIABLES
3.1.1 Joint Probability Distributions
3.1.2 Marginal Probability Distributions
3.1.3 Conditional Probability Distributions
3.1.4 Independence
Exercises for Section 3.1
2 3.2 COVARIANCE AND CORRELATION
3.2.1 Covariance. Covariance Matrix
3.2.2 Correlation Coefficient
Exercises for Section 3.2
3 3.3 BIVARIATE NORMAL DISTRIBUTION
3.3.1 Joint Probability Distributions
3.3.2 Marginal Probability Distributions
3.3.3 Conditional Probability Distributions
3.3.4 Correlation
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 97/148
HANOI – 2024 97 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Definition 12 (Covariance)
The covariance of two random variables X and Y is

Cov(X, Y ) = E{[X − E(X)][Y − E(Y )]}.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 98/148

HANOI – 2024 98 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance
✍ If the points in the joint probability distribution of X and Y that receive positive
probability tend to fall along a line of positive (or negative) slope, Cov(X, Y ) is positive
(or negative).
y y

x x
(a) (b)

Figure 3: Joint probability distributions and the sign of covariance between X and Y (a) Positive
covariance; (b) Negative covariance
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 99/148
HANOI – 2024 99 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

y
y

x
(a) (b)

Figure 4: Joint probability distributions and the sign of covariance between X and Y : Zero covariance

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 100/148

HANOI – 2024 100 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Example 21
In Example 1, the random variables X and Y are the number of blue pens and red
pens selected, respectively. Is the covariance between X and Y positive or negative?
Solution. As the number of blue pens increase, the number of red pens decreases.
Therefore, X and Y have a negative covariance. This can be verified from the joint
probability distribution in Table 1.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 101/148

HANOI – 2024 101 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

✍ The equality of the two expressions for covariance in Equation (??) is shown.

Cov(X, Y ) = E(XY ) − E(X)E(Y ) (27)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 102/148

HANOI – 2024 102 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Example 22
For the integrated circuits tests in Example 2, we found in Example 7 that the
probability model for X and Y is given by the following matrix.

PX,Y (x, y) y = 0 y = 1 y = 2 PX (x)

x=0 0.01 0 0 0.01
x=1 0.09 0.09 0 0.18
x=2 0 0 0.81 0.81
PY (y) 0.10 0.09 0.81

Find Cov(X, Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 103/148

HANOI – 2024 103 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance
Solution.
■ By (5),

2 X
X 2
E(XY ) = xyPX,Y (x, y) = 1 × 1 × 0.09 + 2 × 2 × 0.81 = 3.33.
x=0 y=0

■ To use Theorem 10 to find the covariance, we find

E(X) = 1 × 0.18 + 2 × 0.81 = 1.80, E(Y ) = 1 × 0.09 + 2 × 0.81 = 1.71.

■ Therefore, by Theorem 10,

Cov(X, Y ) = 3.33 − 1.80 × 1.71 = 0.252.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 104/148

HANOI – 2024 104 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

Covariance

Theorem 19
(a) Cov(X, Y ) = Cov(Y, X).
(b) Var(X) = Cov(X, X) and Var(Y ) = Cov(Y, Y ).
(c) If X and Y are independen random variables, cov(X, Y ) = 0. However, if the
covariance between two random variables is zero, we cannot immediately
conclude that the random variables are independent.
(d) Cov(aX, Y ) = aCov(X, Y ).
(e) Cov(X + Z, Y ) = Cov(X, Y ) + Cov(Z, Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 105/148

HANOI – 2024 105 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Example 23
The joint probability distribution of X and Y is
HH Y
H −1 0 1
X HHH
−1 4/15 1/15 4/15

0 1/15 2/15 1/15

1 0 2/15 0

(a) Find Cov(X, Y ).

(b) Find the marginal probability distributions of X and Y .
(c) Are X and Y independent?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 106/148

HANOI – 2024 106 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Solution.
(a) We have
9 4 2 7
E(X) = (−1) × +0× +1× =− .
15 15 15 15
5 5 5
E(Y ) = (−1) × +0× +1× = 0.
15 15 15
4 4
E(XY ) = (−1) × (−1) × + (−1) × (1) × + 1 × (−1) × 0 + 1 × 1 × 0 = 0.
15 15
Hence Cov(X, Y ) = E(XY ) − E(X) × E(Y ) = 0.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 107/148

HANOI – 2024 107 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

(b) The marginal probability distributions of X and Y are

X −1 0 1 Y −1 0 1
P 9/15 4/15 5/15 P 5/15 5/15 5/15

(c) Since P (X = −1, Y = −1) ̸= P (X = −1)P (Y = −1), X and Y are not

independent.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 108/148

HANOI – 2024 108 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Definition 13 (Correlation)
The correlation of X and Y is rX,Y = E(XY ).

Definition 14 (Orthogonal Random Variables)

Random variables X and Y are orthogonal if rX,Y = 0.

Definition 15 (Uncorrelated Random Variables)

Random variables X and Y are uncorrelated if Cov(X, Y ) = 0.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 109/148

HANOI – 2024 109 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Theorem 20
Let a, b, and c are real numbers.

Var(aX + bY + c) = a2 Var(X) + b2 Var(Y ) + 2abCov(X, Y ). (28)

✍
■ If a = 1, b = 1, and c = 0, Var(X + Y ) = Var(X) + Var(Y ) + 2Cov(X, Y ).
■ If a = 1, b = −1, and c = 0, Var(X − Y ) = Var(X) + Var(Y ) − 2Cov(X, Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 110/148

HANOI – 2024 110 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(a) Covariance

Corollary 1
If X and Y are independent,

Var(aX + bY ) = a2 Var(X) + b2 Var(Y ),

Var(aX − bY ) = a2 Var(X) + b2 Var(Y ),

and
Var(X ± Y ) = Var(X) + Var(Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 111/148

HANOI – 2024 111 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.1 Covariance. Covariance Matrix

(b) Covariance Matrix

Definition 16 (Covariance Matrix)

The covariance matrix of two random variables X and Y is
   
Cov(X, X) Cov(X, Y ) Var(X) Cov(X, Y )
Γ= =
Cov(Y, X) Cov(Y, Y ) Cov(X, Y ) Var(Y )

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 112/148

HANOI – 2024 112 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation Coefficient

✍ There is another measure of the relationship between two random variables that is
often easier to interpret than the covariance.
Definition 17 (Correlation Coefficient)
The correlation coefficient of two random variables X and Y is
Cov(X, Y ) Cov(X, Y )
ρX,Y = p = . (29)
Var(X)Var(Y ) σX σY

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 113/148

HANOI – 2024 113 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation Coefficient

Example 24
The joint PDF of two continuous random variables X and Y is
(
1
xy, 0 ≤ x ≤ 2, 0 ≤ y ≤ 4,
fX,Y (x, y) = 16
0, otherwise.

Find ρX,Y .

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 114/148

HANOI – 2024 114 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation Coefficient
Solution.
■ We have

Z2  Z4 Z2  Z4
1 2
 4 2
 8
E(X) = x ydy dx = , E(Y ) = xy dy dx = ,
16 3 3
0 0 0 0
Z2 Z4

2 2
 32
E(XY ) = x y dy dx =
9
0 0

■ Hence,
32 4 8
Cov(X, Y ) = − × = 0.
9 3 3
■ Therefore, ρX,Y = 0.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 115/148
HANOI – 2024 115 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation Coefficient

✍ Because σX > 0 and σY > 0, if the covariance between X and Y is positive,

negative, or zero, the correlation between X and Y is positive, negative, or zero,
respectively. The following result can be shown.
Theorem 21

−1 ≤ ρX,Y ≤ 1.

If ρXY equals +1 or −1, it can be shown that the points in the joint probability
distribution that receive positive probability fall exactly along a straight line. Two random
variables with nonzero correlation are said to be correlated. Similar to covariance, the
correlation is a measure of the linear relationship between random variables.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 116/148
HANOI – 2024 116 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation

Example 25
Suppose that the random variable X has the following distribution:

0, 2,


if x = 1,
0, 6, if x = 2,
PX (x) =

 0, 2, if x = 3,

0, otherwise.




 0, 2, if y = 7,

0, 6, if y = 9,
Let Y = 2X + 5. That is, PY (y) =

 0, 2, if y = 11,

0, otherwise.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 117/148
HANOI – 2024 117 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation

11 0, 2
9 0, 6
7 0, 2 ρ=1
x
1 2 3

Figure 5: Joint distribution for Example 25

Refer to Fig. 5. Because X and Y are linearly related, ρXY = 1. This can be verified by
direct calculations (29)!
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 118/148
HANOI – 2024 118 / 148
 3.2 COVARIANCE AND CORRELATION 3.2.2 Correlation Coefficient

Correlation Coefficient

Theorem 22
If X and Y are random variables such that Y = aX + b

−1,
 a < 0,
ρX,Y = 0, a = 0,

1, a > 0.

Theorem 23
If X and Y are independent,

ρX,Y = 0. (30)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 119/148

HANOI – 2024 119 / 148
 3.2 COVARIANCE AND CORRELATION Exercises for Section 3.2

Exercise for Section 3.2

Exercise 14
For the random variables X and Y in Problem 1, find
Y
(a) The expected value of W = X .
(b) The correlation, E(XY ).
(c) The covariance, Cov(X, Y ).
(d) The correlation coefficient, ρX,Y .
(e) The variance of X + Y , Var(X + Y ).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 120/148

HANOI – 2024 120 / 148
 3.2 COVARIANCE AND CORRELATION Exercises for Section 3.2

Exercise for Section 3.2

Exercise 15
Random variables X and Y have the joint probability distribution
HH Y
H 1 2 3
X HH
H
1 0.17 0.13 0.25
2 0.10 0.30 0.05

(a) Find the marginal probability distributions of X and Y .

(b) Find the covariance matrix of X and Y .
(c) Find the correlation coefficient of two random variables X and Y .
(d) Are X and Y independent?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 121/148

HANOI – 2024 121 / 148
 3.2 COVARIANCE AND CORRELATION Exercises for Section 3.2

Exercise for Section 3.2

Exercise 16
Random variables X and Y have joint PDF
(
(x + y)/3, 0 ≤ x ≤ 1, 0 ≤ y ≤ 2,
fX,Y (x, y) =
0, otherwise.

(a) What are E(X) and Var(X)?

(b) What are E(Y ) and Var(Y )?
(c) What is Cov(X, Y )?
(d) What is E(X + Y )?
(e) What is Var(X + Y )?
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 122/148
HANOI – 2024 122 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION

CONTENT
1 3.1 TWO RANDOM VARIABLES
3.1.1 Joint Probability Distributions
3.1.2 Marginal Probability Distributions
3.1.3 Conditional Probability Distributions
3.1.4 Independence
Exercises for Section 3.1
2 3.2 COVARIANCE AND CORRELATION
3.2.1 Covariance. Covariance Matrix
3.2.2 Correlation Coefficient
Exercises for Section 3.2
3 3.3 BIVARIATE NORMAL DISTRIBUTION
3.3.1 Joint Probability Distributions
3.3.2 Marginal Probability Distributions
3.3.3 Conditional Probability Distributions
3.3.4 Correlation
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 123/148
HANOI – 2024 123 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION 3.3.1 Joint Probability Distributions

Bivariate Normal Probability Density Function

Definition 18 (Bivariate Normal Probability Density Function)

The probability density function of a bivariate normal distribution is

1 n 1 h (x − µ )2
X
fX,Y (x, y) = p exp − 2 2
2πσX σY 1 − ρ2 2(1 − ρ ) σX
(31)
2ρ(x − µX )(y − µY ) (y − µY )2 io
− +
σX σY σY2

where −∞ < x, y < +∞, σX , σY > 0, −∞ < µX , µY < +∞, and −1 < ρ < 1.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 124/148

HANOI – 2024 124 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION 3.3.1 Joint Probability Distributions

Bivariate Normal Probability Density Function

✍ If µX = µY = 0, σX = σY = 1, and ρ = 0,

1 − x2 +y2
fX,Y (x, y) = e 2 for all x, y (32)
2π

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 125/148

HANOI – 2024 125 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION 3.3.2 Marginal Probability Distributions

Marginal Distributions of Bivariate Normal Random Variables

Theorem 24
If X and Y have a bivariate normal distribution with joint probability density
fX,Y (x, y), the marginal probability distributions of X and Y are normal with
means µX and µY and standard deviations σX and σY , respectively, and
(x−µX )2 (y−µY )2
1 −
2σ 2
1 −
2σ 2
fX (x) = √ e X and fY (y) = √ e Y .
σX 2π σY 2π

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 126/148

HANOI – 2024 126 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION 3.3.3 Conditional Probability Distributions

(c) Conditional Distribution of Bivariate Normal Random Variables

Theorem 25
If X and Y have a bivariate normal distribution with joint probability density
fX,Y (x, y), the conditional probability distribution of Y given X = x is normal with
mean
σY
µY |x = µY + ρ (x − µX ) (33)
σX

and variance

σY2 |x = σY2 (1 − ρ2 ) (34)

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 127/148

HANOI – 2024 127 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION 3.3.4 Correlation

Correlation of Bivariate Normal Random Variables

Theorem 26 (Correlation of Bivariate Normal Random Variables)

If X and Y have a bivariate normal distribution with joint probability density
function fX,Y (x, y), the correlation between X and Y is ρ.

Theorem 27 (For Bivariate Normal Random Variables Zero Correlation Implies

Independence)
If X and Y have a bivariate normal distribution with ρ = 0, X and Y are
independent.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 128/148

HANOI – 2024 128 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION 3.3.4 Correlation

Exercises for Section 3.3

Exercise 17
Suppose X and Y have a bivariate normal distribution with σX = 50.04,
σY = 50.08, µX = 53.00, µY = 57.70, and ρ = 0. Determine the following:
(a) P (2.95 < X < 3.05).
(b) P (7.60 < Y < 7.80).
(c) P (2.95 < X < 3.05, 7.60 < Y < 7.80).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 129/148

HANOI – 2024 129 / 148
 3.3 BIVARIATE NORMAL DISTRIBUTION Exercises for Section 3.3

Exercises for Section 3.3

Exercise 18
In an acid-base titration, a base or acid is gradually added to the other until they
have completely neutralized each other. Let X and Y denote the milliliters of acid
and base needed for equivalence, respectively. Assume X and Y have abivariate
normal distribution with σX = 5 mL, σY = 2 mL, µX = 120 mL, µY = 100 mL, and
ρ = 0.6. Determine the following:
(a) Covariance between X and Y .
(b) Marginal probability distribution of X.
(c) P (X < 16).
(d) Conditional probability distribution of X given that Y = 102.
(e) P (X < 116|Y = 102).

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 130/148

HANOI – 2024 130 / 148
 3.4 CENTERAL LIMIT THEOREM

CONTENT
1 3.1 TWO RANDOM VARIABLES
3.1.1 Joint Probability Distributions
3.1.2 Marginal Probability Distributions
3.1.3 Conditional Probability Distributions
3.1.4 Independence
Exercises for Section 3.1
2 3.2 COVARIANCE AND CORRELATION
3.2.1 Covariance. Covariance Matrix
3.2.2 Correlation Coefficient
Exercises for Section 3.2
3 3.3 BIVARIATE NORMAL DISTRIBUTION
3.3.1 Joint Probability Distributions
3.3.2 Marginal Probability Distributions
3.3.3 Conditional Probability Distributions
3.3.4 Correlation
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 131/148
HANOI – 2024 131 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.1. Central Limit Theorem

Central Limit Theorem

Theorem 28 (Central Limit Theorem)

Given {Xn }∞ Pniid random variables with expected value µ and finite
n=1 a sequence of
2 1
variance σ , and if X n = n i=1 Xi , the limiting form of the distribution of
Xn − µ
Zn = √ as n → ∞, is the standard normal distribution N (0, 1), that is
σ/ n

lim FZn (z) = Φ(z) (35)

n→∞

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 132/148

HANOI – 2024 132 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.1. Central Limit Theorem

Central Limit Theorem

✍ The normal approximation for depends on the sample size n. The following figures
show the distributions of average scores obtained when tossing one, two, three, and five
dice, respectively.

Figure 6: Distributions of average scores from throwing dice

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 133/148

HANOI – 2024 133 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.1. Central Limit Theorem

Central Limit Theorem

■ To use the central limit theorem, we observe that we can express the iid sum
Wn = X1 + X2 + · · · + Xn as
√
Wn = σ nZn + nµ (36)

■ The CDF of Wn can be expressed in terms of the CDF of Zn as

√  w − nµ 
FWn (w) = P (σ nZn + nµ < w) = FZn √ (37)
σ n

For large n, the central limit theorem says that FZn (z) ≈ Φ(z). This approximation
is the basis for practical applications of the central limit theorem.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 134/148
HANOI – 2024 134 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.1. Central Limit Theorem

Central Limit Theorem

✍ The PDF ofW n, the sum of n uniform U [0, 1] random variables, and the
corresponding central limit theorem approximation for n = 1, 2, 3, 4.

Figure 7: n = 1 Figure 8: n = 2

Figure 9: n = 3 Figure 10: n = 4

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 135/148
HANOI – 2024 135 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.1. Central Limit Theorem

Central Limit Theorem

Definition 19 (Central Limit Theorem Approximation)

Let Wn = X1 + · · · + Xn be the sum of n iid random variables, each with E(Xn ) = µ
and V (Xn ) = σ 2 . The central limit theorem approximation to the CDF of Wn is
 w − nµ 
FWn (w) ≈ Φ √ (38)
σ n

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 136/148

HANOI – 2024 136 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

✍ In addition to helping us understand why we observe bell-shaped curves in so many

situations, the central limit theorem makes it possible to perform quick, accurate
calculations that would otherwise be extremely complex and time consuming. In these
calculations, the random variable of interest is a sum of other random variables, and we
calculate the probabilities of events by referring to the corresponding normal random
variable.
Example 26
A modem transmits one million bits. Each bit is 0 or 1 independently with equal
probability. (a) Estimate the probability of at least 502,000 ones. (b) Transmit one
million bits. Let A denote the event that there are at least 499,000 ones but no
more than 501,000 ones. What is P (A)?

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 137/148

HANOI – 2024 137 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

Solution. Let Xi be the value of bit i (0 or 1). Because Xi is a Bernoulli random

variable with E(Xi ) = 0.5 and V (Xi ) = 0.25 for all i. The number of ones in one
P106
million bits is W = i=1 Xi . W is B(n, p) with n = 106 and p = 0.5.
(a) By the central limit theorem approximation,
 502000 − 106 × 0.5 
P (W ≥ 502000) = 1 − P (X < 502000) ≈ 1 − Φ √
0.5 106
= 1 − Φ(4) = 1 − 0.999968 = 3.2 × 10−5 .

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 138/148

HANOI – 2024 138 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

(b) By the central limit theorem approximation,

P (A) = P (499000 < X ≤ 501000) = P (X ≤ 502000) − P (X < 499000)

 501000 − 106 × 0, 5   499000 − 106 × 0, 5 
≈Φ √ −Φ √
0, 5 106 0, 5 106
= Φ(2) − Φ(−2) = 2 × 0.97725 − 1 = 0.9545.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 139/148

HANOI – 2024 139 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

Definition 20 (De Moivre–Laplace Formula)

For a binomial B(n, p) random variable K,
 k + 0.5 − np   k − 0.5 − np 
2 1
P (k1 ≤ K ≤ k2 ) ≈ Φ p −Φ p (39)
np(1 − p) np(1 − p)

✍ To appreciate why the ±0.5 terms increase the accuracy of approximation, consider
the following simple but dramatic example in which k1 = k2 .

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 140/148

HANOI – 2024 140 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

Example 27
Let K be a binomial (n = 20, p = 0.4) random variable. What is P (K = 8)?
Solution.
■ Since E(K) = np = 8 and V (K) = np(1 − p) = 4.8, the central limit theorem

approximation to K is a normal random variable X with E(X) = 8 and

V (X) = 4.8. Because X is a continuous random variable, P (X = 8) = 0, a useless
approximation to P (K = 8).
■ On the other hand, the De Moivre–Laplace formula produces

 0.5   −0.5 
P (8 ≤ K ≤ 8) ≈ P (7.5 ≤ X ≤ 8.5) = Φ √ −Φ √ = 2Φ(0.23) − 1
4.8 4.8
= 0.1819.
8
■ The exact value is P (X = 8) = C20 (0.4)8 (0.6)12 ≈ 0.1797.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 141/148
HANOI – 2024 141 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

Example 28
K is the number of heads in 100 flips of a fair coin. What is P (50 ≤ K ≤ 51)?

Solution.
■ Since K is a binomial (n = 100, p = 1/2) random variable,

50
P (50 ≤ K ≤ 51) = P (K = 50)+P (K = 51) = C100 (0, 5)100 +C100
51
(0, 5)100 ≈ 0.1576.
■ Since E(K) = 50 and σK = 5, the ordinary central limit theorem approximation
produces
 51 − 50   50 − 50 
P (50 ≤ K ≤ 51) ≈ Φ √ −Φ √ = Φ(0.33) − Φ(0)
25 25
= 0.62930 − 0.5 = 0.1293.
Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 142/148
HANOI – 2024 142 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

■ This approximation error of roughly 50% occurs because the ordinary central limit
theorem approximation ignores the fact that the discrete random variable K has
two probability masses in an interval of length 1.
■ As we see next, the De Moivre–Laplace approximation is far more accurate.
 51 + 0.5 − 50   50 − 0.5 − 50 
P (50 ≤ K ≤ 51) ≈ Φ √ −Φ √ = Φ(0.3) − Φ(0.1)
25 25
= 0.61791 + 0.53983 − 1 = 0.1577.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 143/148

HANOI – 2024 143 / 148
 3.4 CENTERAL LIMIT THEOREM 3.4.2 Applications of the Central Limit Theorem

Applications of the Central Limit Theorem

✍
■ Although the central limit theorem approximation provides a useful means of
calculating events related to complicated probability models, it has to be used with
caution. When the events of interest are confined to outcomes at the edge of the
range of a random variable, the central limit theorem approximation can be quite
inaccurate.
■ In these applications, it is necessary to resort to more complicated methods than a
central limit theorem approximation to obtain useful results. In particular, it is often
desirable to provide guarantees in the form of an upper bound rather than the
approximation offered by the central limit theorem.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 144/148

HANOI – 2024 144 / 148
 3.4 CENTERAL LIMIT THEOREM Exercises for Section 3.4

Exercises for Section 3.4

Exercise 19
Let X1 , X2 , . . . be an iid sequence of Poisson random variables, each with expected
value E(X) = 1. Let Wn = X1 + · · · + Xn . Use the improved central limit theorem
approximation to estimate P (Wn = n). For n = 4, 25, 64, compare the
approximation to the exact value of P (Wn = n).

✍ A sum of iid Poisson random variables is a Poisson random variable. Hence,

Wn ∼ P(λ), λ = n.
Sol. n = 4, exact 0.1954; approximate 0.1974; n = 64, exact 0.0498; approximate 0.0498

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 145/148

HANOI – 2024 145 / 148
 3.4 CENTERAL LIMIT THEOREM Exercises for Section 3.4

Exercises for Section 3.4

Exercise 20
Integrated circuits from a certain factory pass a certain quality test with probability
0.8. The outcomes of all tests are mutually independent.
(a) What is the expected number of tests necessaryto find 500 acceptable circuits?
Sol. 625
(b) Use the central limit theorem to estimate the probability of finding 500
acceptable circuits in a batch of 600 circuits.
(c) Use the central limit theorem to calculate theminimum batch size for finding
500 acceptable circuits with probability 0.9 or greater.

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 146/148

HANOI – 2024 146 / 148
 3.4 CENTERAL LIMIT THEOREM Exercises for Section 3.4
z
1
Z
−t2
Φ(z) = √ e 2 dt
2π −∞

z 0 1 2 3 4 5 6 7 8 9
0.0 0.50000 50399 50798 51197 51595 51994 52392 52790 53188 53586
0.1 53983 54380 54776 55172 55567 55962 56356 56749 57142 57535
0.2 57926 58317 58706 59095 59483 59871 60257 60642 61026 61409
0.3 61791 62172 62556 62930 63307 63683 64058 64431 64803 65173
0.4 65542 65910 66276 66640 67003 67364 67724 68082 68439 68739
0.5 69146 69447 69847 70194 70544 70884 71226 71566 71904 72240
0.6 72575 72907 73237 73565 73891 74215 74537 74857 75175 75490
0.7 75804 76115 76424 76730 77035 77337 77637 77935 78230 78524
0.8 78814 79103 79389 79673 79955 80234 80511 80785 81057 81327
0.9 81594 81859 82121 82381 82639 82894 83147 83398 83646 83891
1.0 84134 84375 84614 84850 85083 85314 85543 85769 85993 86214
1.1 86433 86650 86864 87076 87286 87493 87698 87900 88100 88298
1.2 88493 88686 88877 89065 89251 89435 89617 89796 89973 90147
1.3 90320 90490 90658 90824 90988 91149 91309 91466 91621 91774
1.4 91924 92073 92220 92364 92507 92647 92786 92922 93056 93189
1.5 93319 93448 93574 93699 93822 93943 94062 94179 94295 94408
1.6 94520 94630 94738 94845 94950 95053 95154 95254 95352 95449
1.7 95543 95637 95728 95818 95907 95994 96080 96164 96246 96327
1.8 96407 96485 96562 96638 96712 96784 96856 96926 96995 97062
1.9 97128 97193 97257 97320 97381 97441 97500 97558 97615 97670

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 147/148

HANOI – 2024 147 / 148
 3.4 CENTERAL LIMIT THEOREM Exercises for Section 3.4
z
1
Z
−t2
Φ(z) = √ e 2 dt
2π −∞

x 0 1 2 3 4 5 6 7 8 9
2.0 97725 97778 97831 97882 97932 97982 98030 98077 98124 98169
2.1 98214 98257 98300 98341 98382 98422 99461 98500 98537 98574
2.2 98610 98645 98679 98713 98745 98778 98809 98840 98870 98899
2.3 98928 98956 98983 99010 99036 99061 99086 99111 99134 99158
2.4 99180 99202 99224 99245 99266 99285 99305 99324 99343 99361
2.5 99379 99396 99413 99430 99446 99261 99477 99492 99506 99520
2.6 99534 99547 99560 99573 99585 99598 99609 99621 99632 99643
2.7 99653 99664 99674 99683 99693 99702 99711 99720 99728 99763
2.8 99744 99752 99760 99767 99774 99781 99788 99795 99801 99807
2.9 99813 99819 99825 99831 99836 99841 99846 99851 99856 99861
3.0 0,99865 3,1 99903 3,2 99931 3,3 99952 3,4 99966
3.5 99977 3,6 99984 3,7 99989 3,8 99993 3,9 99995
4.0 999968
4.5 999997
5.0 99999997

Nguyễn Thị Thu Thủy (FaMI-HUST) ProSta-CHAP3 148/148

HANOI – 2024 148 / 148