2.

Discrete Random Variables

Broadband Communication Lab, NCTU, Taiwan

Outline
2.1 Basic Concepts
2.2 Probability Mass Functions

2.3 Functions of Random Variables

2.4 Expectation, Mean, and Variance
2.5 Joint PMFs of Multiple Random

Variables
2.6 Conditioning
2.7 Independence

Broadband Communication Lab, NCTU, Taiwan

2.1 Visualization of a Random

Variable

Broadband Communication Lab, NCTU, Taiwan

Main Concepts Related to Random

Variables
A random variable is a real-valued function of

the outcome of the experiment.

A function of a random variable defines
another random variable.
We can associate with each random variable
certain averages of interest, such as the
mean and the variance.
A random variable can be conditioned on an
event or on another random variables.
There is a notion of independence of a random
variable from an event or from another
random variable.
Broadband Communication Lab, NCTU, Taiwan

Concepts Related to Discrete

Random Variables
A discrete random variable is a real-valued

function of the outcome of the experiment

that can take a finite or countably infinite
number of values.
A discrete random variable has an associated
probability mass function (PMF), which gives
the probability of each numerical value that
the random variable can take.
A function of a discrete random variable
defines another discrete random variable,
whose PMF can be obtained from the PMF of
the original random variable.

Broadband Communication Lab, NCTU, Taiwan

2.2 Probability Mass Functions

X: Discrete random variable.

pX: PMF.
x: Any possible value of X.
pX (x): Probability mass of x.
(
)
Note that :
For any set S of possible values of X, we
have :

Broadband Communication Lab, NCTU, Taiwan

Calculation of the PMF of a Random

Variable X
For each possible value x of X:
Collect all the possible outcomes that give
rise to the event {x = X}.
Add their probabilities to obtain pX (x).
Example:
Let X be the number of heads obtained in
two independent tosses of a fair coin.

The probability of at least one head is

Broadband Communication Lab, NCTU, Taiwan

The Bernoulli Random Variable

Toss a coin: p(head) = p, p(tail) = 1 p.
The Bernoulli random variable X is

Its PMF is

Broadband Communication Lab, NCTU, Taiwan

The Binomial Random Variable

Toss a coin n times. Tosses are indep..
p(a head) = p, p(a tail) = 1 p.
X: number of heads in the n-toss sequence.
X is a binomial random variable with
parameters n and p.
PMF:

Normalization property:

Broadband Communication Lab, NCTU, Taiwan

The Binomial Random Variable

Broadband Communication Lab, NCTU, Taiwan

10

The Geometric Random Variable

Repeatedly and independently toss a

coin with p(a head) = p, 0 < p < 1.

Geometric random variable X: Number
of tosses need for a head to come up
for the first time.
PMF:

Broadband Communication Lab, NCTU, Taiwan

11

The Geometric Random Variable

It is a legitimate PMF because

k 1

k 1

k 1
p
(
k
)

(1

p
)
p
x

p (1 p ) k
k 0

1
p
1 (1 p)
1

Broadband Communication Lab, NCTU, Taiwan

12

The Poisson Random Variable

PMF:

is a positive parameter characterizing

the PMF.

Broadband Communication Lab, NCTU, Taiwan

13

The Poisson Random Variable

Binomial RV with small p and large n

Poisson RV with = np.

Justification: See Problem 12.

Example of Poisson RV:

Number of typos in a book.
Number of cars involved in accidents in a
city on a given day.

Broadband Communication Lab, NCTU, Taiwan

14

2.3 Functions of Random Variables

If Y = g(X) is a function of a RV X, then

Y is also a RV.
The PMF of Y is

Broadband Communication Lab, NCTU, Taiwan

15

Example 2.1 (1/2)

Example 2.1: The PMF of X is

Find the PMF of Y=|X| and Z=X2.

Sol:

Broadband Communication Lab, NCTU, Taiwan

16

Example 2.1 (2/2)

Possible values of y = 0, 1, 2, 3, 4.

pY(0) = pX(0) = 1/9.

pY(1) = pX(-1) + pX(1) = 2/9.
Similar for y = 2, 3, 4.
PMF of Y is

PMF of Z is

Broadband Communication Lab, NCTU, Taiwan

17

2.4 Expectation
We define the expected value (also

called the expectation or the mean) of a

random variable X, with PMF pX, by

Physical meaning: Center of gravity of

the PMF.

Broadband Communication Lab, NCTU, Taiwan

18

Example 2.2
Example 2.2 : Consider two independent coin

tosses, each with a 3/4 probability of a head,

and let X be the number of heads obtained.
This is a binomial random variable, its PMF is :

So the mean is :

Broadband Communication Lab, NCTU, Taiwan

19

Moments, Variance, and Standard

Deviation
nth moment = E[Xn].
The 1st moment is just the mean.

Variance: var(X) = E[(X - E[X])2].

The variance provides a measure of
dispersion of X around its mean.
Another measure of dispersion is the

standard deviation of X:

X has the same unit as X.

Broadband Communication Lab, NCTU, Taiwan

20

Example 2.3
Recall Example 2.1.

Broadband Communication Lab, NCTU, Taiwan

21

Expected Value Rule for Functions of

Random Variables
Let X be a random variable with PMF pX . Then,

the expected value of the random variable g(X)

is given by :

Proof:

Broadband Communication Lab, NCTU, Taiwan

22

Example 2.3 (continued)

The variance of X can also be calculated as :

Broadband Communication Lab, NCTU, Taiwan

23

Mean and Variance of a Linear

Function of a Random Variable
Let X be a random variable and let Y = aX + b,

where a and b are given scalars. Then,

Proof : E[Y ] (ax b) p X ( x)

x

a xp X ( x) b p X ( x)
x

aE[ x] b

Broadband Communication Lab, NCTU, Taiwan

24

Variance in Terms of Moments

Expression

Proof:

Broadband Communication Lab, NCTU, Taiwan

25

Example 2.4 (1/2)

If the weather is good (which happens with

probability 0.6), Alice walks the 2 miles to

class at a speed of V = 5 miles per hour, and
otherwise drives her motorcycle at a speed of
V = 30 miles per hour. What is the mean of
the time T to get to class?
Sol:

Broadband Communication Lab, NCTU, Taiwan

26

Example 2.4 (2/2)

However, it is wrong to calculate the mean of

the speed V .

To summarize, in this example we have

Broadband Communication Lab, NCTU, Taiwan

27

Example 2.5
Mean and Variance of the Bernoulli.
PMF:

Broadband Communication Lab, NCTU, Taiwan

28

Discrete Uniformly Distributed

Random Variable.
Discrete Uniformly Distributed Random

Variable.
PMF:

Broadband Communication Lab, NCTU, Taiwan

29

Discrete Uniformly Distributed

Random Variable.
Consider the case where a = 1 and b = n.

General case :

Broadband Communication Lab, NCTU, Taiwan

30

Example 2.7
The Mean of the Poisson.
PMF:

var(X) = . (See Problem 24.)

Broadband Communication Lab, NCTU, Taiwan

31

Example 2.8 (1/2)

Consider a quiz game where a person is given

two questions and must decide which question

to answer first. Question 1 will be answered
correctly with probability 0.8, and the person
will then receive as prize $100, while question
2 will be answered correctly with probability
0.5, and the person will then receive as prize
$200. If the first question attempted is
answered incorrectly, the quiz terminates, i.e.,
the person is not allowed to attempt the
second question. If the first question is
answered correctly, the person is allowed to
attempt the second question. Which question
should be answered first to maximize the
expected value of the total prize money
received?
Broadband Communication Lab, NCTU, Taiwan

32

Example 2.8 (2/2)

Sol:

Broadband Communication Lab, NCTU, Taiwan

33

2.5 Joint PMF of Two Random

Variables
X, Y: Two discrete random variables.

pX,Y : Joint PMF of X and Y.

(x, y): A pair of possible values of X and Y.
pX,Y (x, y) = P(X = x, Y = y).
Marginal PMFs:

Verify:

Broadband Communication Lab, NCTU, Taiwan

34

Example 2.9
The marginal PMF of X or Y at a given value is

obtained by adding the table entries along a

corresponding column or row.

Broadband Communication Lab, NCTU, Taiwan

35

Functions of Multiple Random

Variables
Z = g(X, Y) defines another random

variables.
PMF:

Expected value:

When g(X, Y) = aX + bY + c,

Broadband Communication Lab, NCTU, Taiwan

36

Example 2.9 (continued)

Now, a new random variable Z is defined by:

Z X 2Y
Please find E[z]=?
E[Z ] E[ X ] 2E[Y ]
3
6
8
3 51
2 3 4
20
20
20
20 20
3
7
7
3 50
E[Y ] 1 2 3 4
20
20
20
20 20
51
50
E[ Z ]
2 7.55
20
20
E[ X ] 1

Alternatively, we can find E[z] from the PFM of

variable Z. (see textbook p.95)

Broadband Communication Lab, NCTU, Taiwan

37

More than Two Random Variables

PMF of three random variables:

Marginal PMFs:

Expected value:

g(X1, X2,, Xn) = a1X1 + a2X2 + +anXn

Broadband Communication Lab, NCTU, Taiwan

38

Example 2.10
Mean of the Binomial : Your probability class

has 300 students and each student has

probability 1/3 of getting an A, independently
of any other student. What is the mean of X,
the number of students that get an A?

Sol:

1
2 1
E[ X i ] 1 0
3
3 3

Broadband Communication Lab, NCTU, Taiwan

39

Example 2.11
The Hat Problem : Suppose that n people

throw their hats in a box and then each picks

one hat at random. What is the expected
value of X, the number of people that get back
their own hat?

Sol: We introduce a random variable Xi that takes

the value 1 if the ith person selects his/her own

hat, and takes the value 0 otherwise.
;

Broadband Communication Lab, NCTU, Taiwan

40

2.6 Conditioning a Random Variable

on an Event
The conditional PMF of a random variable X,

conditioned on a particular event A with

P(A) > 0, is defined by

Note that {X = x} A are disjoint for different x,

therefore

Combining above, we can see that

so pX|A is a legitimate PMF.

Broadband Communication Lab, NCTU, Taiwan

41

Visualization of pX|A(x)

Broadband Communication Lab, NCTU, Taiwan

42

Example 2.12
Roll a Die. X: The roll of a fair six-sided die.

A: Event that the roll is an even number.

Find pX|A(x).
Sol:

Broadband Communication Lab, NCTU, Taiwan

43

Example 2.13
A student will take a certain test repeatedly,

up to a maximum of n times, each time with a

probability p of passing, independently of the
number of pervious attempts. What is the PMF
of the number of attempts, given that the
student passes the test?
Sol : Let A be the event that the student
passes the test.
n

P( A) (1 p) m1 p
m 1

(1 p) k 1 p
, if k 1,..., n.
n

m 1
PX | A (k ) (1 p) p
m1
0,
otherwise
Broadband Communication Lab, NCTU, Taiwan

44

Conditioning one Random Variable

on Another
The conditional PMF pX|Y of X given Y is

Normalization property:

The conditional PMF is often convenient for the

calculation of the joint PMF,

Broadband Communication Lab, NCTU, Taiwan

45

Visualization of pX|Y(x|y)

Broadband Communication Lab, NCTU, Taiwan

46

Example 2.14 (1/2)

Professor May B. Right often has her facts

wrong, and answers each of her students

questions incorrectly with probability 1/4,
independently of other questions. In each
lecture May is asked 0, 1, or 2 questions with
equal probability 1/3. Let X and Y be the
number of questions May is asked and the
number of questions she answers wrong in a
given lecture, respectively. Please construct
the joint PMF pX,Y (x, y).

Broadband Communication Lab, NCTU, Taiwan

47

Example 2.14 (2/2)

Sol:

Broadband Communication Lab, NCTU, Taiwan

48

Example 2.15 (1/2)

Consider a transmitter that is sending

messages over a computer network. Define

the following two random variables:
X : the travel time of a given message,
Y : the length of the given message.

Assume that the travel time X of the message

depends on its length Y. The travel time is
104Y secs with probability 1/2, 103Y secs
with probability 1/3, and 102Y secs with
probability 1/6. Please find the PMF of X.
Broadband Communication Lab, NCTU, Taiwan

49

Example 2.15 (2/2)

Sol: We can know:

and
So:

Broadband Communication Lab, NCTU, Taiwan

50

Conditional Expectations
The conditional expectation of X given

an event A with P(A) > 0, is defined by

For a function g(X), we have

The conditional expectation of X given a

value y of Y is defined by

Broadband Communication Lab, NCTU, Taiwan

51

Total Expectation Theorem

Total expectation theorem:

Let A1, , An form a partition of the

sample space and P(Ai) > 0 for all i, then

Let A1, , An form a partition of an

event B and

P(Ai B) > 0

for all i, then

Broadband Communication Lab, NCTU, Taiwan

52

Verify the Total Expectation

Theorem
Verify

Broadband Communication Lab, NCTU, Taiwan

53

Example 2.16
Messages Transmission :

P(Boston to New York) = 0.5.

P(Boston to Chicago) = 0.3.
P(Boston to San Francisco) = 0.2.
X: transit time of a message (random).
E[X | New York] = 0.05 s.
E[X | Chicago] = 0.1 s.
E[X | San Francisco] = 0.3 s.
Please find E[X].
Sol:

Broadband Communication Lab, NCTU, Taiwan

54

Example 2.17 (1/2)

Mean and Variance of the Geometric Random

Variable.

Alternative way to fine E[X] and var(X).

Define A1 = {X = 1} = {first try is a success},

A2 = {X > 1} = {first try is a failure}.
(1)

Broadband Communication Lab, NCTU, Taiwan

55

Example 2.17 (2/2)

So
(2)

Broadband Communication Lab, NCTU, Taiwan

56

2.7 Independence of a Random

Variable from an Event
We say the random variable X is

independent of the event A if

From the definition of the conditional

PMF, we have

If P(A) > 0, independence is the same

as the condition

Broadband Communication Lab, NCTU, Taiwan

57

Example 2.19
Consider two independent tosses of a fair coin.

Let X be the number of heads and let A be the

event that the number of heads is even. The
(unconditional) PMF of X is : ( P(A) = )

pX|A(x) = P(X = x and A)/P(A):

Broadband Communication Lab, NCTU, Taiwan

58

Independence of Random Variables

We say that two random variables X

and Y are independent if

or, equivalently,

X and Y are said to be conditionally

independent, given P(A) > 0, if

Once more, this is equivalent to

Broadband Communication Lab, NCTU, Taiwan

59

Facts About Independent Random

Variables
If X and Y are independent random

variables, then

E[XY] = E[X] E[Y].

For any functions g and h, g(X) and h(Y)
are independent.
E[g(X) h(Y)] = E[g(X)] E[h(Y)].
var(X + Y) = var(X) + var(Y).

Broadband Communication Lab, NCTU, Taiwan

60

Facts About Independent Random

Variables
Conditional independence may not imply

unconditional independence.
Example :

Broadband Communication Lab, NCTU, Taiwan

61

Independence of Several Random

Variables
Three random variables X, Y, and Z are

said to be independent if

If X, Y, and Z are independent random

variables, then

f(X), g(Y), and h(Z) are also independent.

g(X, Y) and h(Z) are independent.
g(X, Y) and h(Y, Z) are usually not
independent.

Broadband Communication Lab, NCTU, Taiwan

62

Variance of the Sum of Independent

Random Variables
If X1, X2, , Xn are independent random

variables, then

Example 2.20 :

Xi ~ Bernoulli(p).
X = X1 + + Xn ~ Binomial(n, p).
{Xi} are independent.
Sol:

Broadband Communication Lab, NCTU, Taiwan

63

Example 2.21 (1/2)

We wish to estimate the approval rating of a president,

to be called C. To this end, we ask n persons drawn at

random from the voter population, and we let Xi be a
random variable that encodes the response of the ith
person:

We model X1,X2, . . . , Xn as independent Bernoulli

random variables with common mean p and variance
p(1 p). Naturally, we view p as the true approval
rating of C. We average the responses and compute
the sample mean Sn, defined as

Please find E[Sn] and var[Sn].

Broadband Communication Lab, NCTU, Taiwan

64

Example 2.21 (2/2)

Sol:

Broadband Communication Lab, NCTU, Taiwan

65

Example 2.22 (1/2)

In many practical situations, the analytical

calculation of the probability of some event of

interest is very difficult. However, if we have a
physical or computer model that can generate
outcomes of a given experiment in accordance
with their true probabilities, we can use simulation
to calculate with high accuracy the probability of
any given event A. In particular, we independently
generate with our model n outcomes, we record
the number m that belong to the event A of
interest, and we approximate P(A) by m/n. For
example, to calculate the probability p = P(Heads)
of a biased coin, we flip the coin n times, and we
approximate p with the ratio (number of heads
recorded)/n.

Broadband Communication Lab, NCTU, Taiwan

66

Example 2.22 (2/2)

To see how accurate this process is, consider n

independent Bernoulli random variables X1, . . . , Xn,

each with PMF

In a simulation context, Xi corresponds to the ith

outcome, and takes the value 1 if the ith outcome
belongs to the event A. The value of the random
variable

is the estimate of P(A) provided by the simulation.

According to Example 2.21, X has mean P(A) and
variance P(A)(1P(A))/n, so that for large n, it provides
an accurate estimate of P(A).

Broadband Communication Lab, NCTU, Taiwan

67