Ambo University

Institute of Technology
Department of Electrical & Computer Engineering

Probability and Rando Process

Chapter 2: Random Variables

 Random Variables
Outline
▪ Introduction
▪ The Cumulative Distribution Function
▪ Probability Density and Mass Functions
▪ Expected Value, Variance and Moments
▪ Some Special Distributions
▪ Functions of One Random Variable

2
Introduction
▪ A random variable X is a function that assigns a real number
X(ω) to each outcome ω in the sample space Ω of a random
experiment.
▪ The sample space Ω is the domain of the random variable and the
set RX of all values taken on by X is the range of the random
variable.
▪ Thus, RX is the subset of all real numbers.


A
X ( ) = x
Re al Line
x B

3
Introduction Cont’d……
▪ If X is a random variable, then {ω: X(ω)≤ x}={X≤ x} is an event
for every X in RX.
Example: Consider a random experiment of tossing a fair coin three
times. The sequence of heads and tails is noted and the sample
space Ω is given by:
 = {HHH , HHT , HTH , THH , THT , HTT , TTH , TTT }

Let X be the number of heads in three coin tosses. X assigns

each possible outcome ω in the sample space Ω a number from
the set RX={0, 1, 2, 3}.
 : HHH HHT HTH THH THT HTT TTH TTT
X ( ) : 3 2 2 2 1 1 1 0
4
The Cumulative Distribution Function

▪ The cumulative distribution function (cdf) of a random variable

X is defined as the probability of the event {X≤ x}.
FX ( x) = P( X  x)
Properties of the cdf, FX(x):
▪ The cdf has the following properties.

i. FX ( x) is a non - negative function, i.e.,

0  FX ( x)  1
ii. lim FX ( x) = 1
x →

iii. lim FX ( x) = 0
x →−
5
The Cumulative Distribution Function Cont’d…..

iv. FX ( x) is a non - decreasing function of X , i.e.,

If x1  x2 , then FX ( x1 )  FX ( x2 )

v. P( x1  X  x2 ) = FX ( x2 ) − FX ( x1 )
vi. P( X  x) = 1 − FX ( x)

Example:
Find the cdf of the random variable X which is defined as the
number of heads in three tosses of a fair coin.

6
 The Cumulative Distribution Function
Solution:
✓ We know that X takes on only the values 0, 1, 2 and 3 with
probabilities 1/8, 3/8, 3/8 and 1/8 respectively.
✓ Thus, FX(x) is simply the sum of the probabilities of the
outcomes from the set {0, 1, 2, 3} that are less than or equal to x.
0, x  0
1 / 8, 0  x  1

 FX ( x) = 1 / 2, 1  x  2
7 / 8, 2  x  3

1, x  3
 : HHH HHT HTH THH THT HTT TTH TTT
7
X ( ) : 3 2 2 2 1 1 1 0
Types of Random Variables
▪ There are two basic types of random variables.
i. Continuous Random Variable
✓ A continuous random variable is defined as a random variable
whose cdf, FX(x), is continuous every where and can be written as
an integral of some non-negative function f(x), i.e.,

FX ( x) =  f (u )du
−

ii. Discrete Random Variable

✓ A discrete random variable is defined as a random variable whose
cdf, FX(x), is a right continuous, staircase function of X with
jumps at a countable set of points x0, x1, x2,……

8
The Probability Density Function
▪ The probability density function (pdf) of a continuous random
variable X is defined as the derivative of the cdf, FX(x), i.e.,
dFX ( x)
f X ( x) =
dx
Properties of the pdf, fX(x):

i. For all values of X , f X ( x)  0


ii. 
−
f X ( x)dx = 1
x2
iii. P ( x1  X  x2 ) =  f X ( x)dx
x1

9
The Probability Mass Function
▪ The probability mass function (pmf) of a discrete random
variable X is defined as:

PX ( X = xi ) = PX ( xi ) = FX ( xi ) − FX ( xi −1 )

Properties of the pmf, PX (xi ):

i. 0  PX ( xk )  1, k = 1, 2, .....
ii. PX ( x) = 0, if x  xk , k = 1, 2, .....
iii. P
k
X ( xk ) = 1

10
Calculating the Cumulative Distribution Function

▪ The cdf of a continuous random variable X can be obtained by

integrating the pdf, i.e.,
x
FX ( x) =  f X (u )du
−

▪ Similarly, the cdf of a discrete random variable X can be obtained by

using the formula:

FX ( x) = P
xk  x
X ( xk )

11
Expected Value, Variance and Moments
i. Expected Value (Mean)
▪ The expected value (mean) of a continuous random variable X,
denoted by μX or E(X), is defined as:


 X = E ( X ) =  xf X ( x)dx
−

▪ Similarly, the expected value of a discrete random variable X is

given by:

 X = E ( X ) =  xk PX ( xk )
k

▪ Mean represents the average value of the random variable in a

very large number of trials.
12
Expected Value, Variance and Moments Cont’d…..
ii. Variance
▪ The variance of a continuous random variable X, denoted by
σ2X or VAR(X), is defined as:

 2 X = Var ( X ) = E[( X −  X ) 2 ]

 2
X = Var ( X ) =  ( x −  X ) 2 f X ( x)dx
−

▪ Expanding (x-μX )2 in the above equation and simplifying the

resulting equation, we will get:

 2
X = Var ( X ) = E ( X ) − [ E ( X )]
2 2

13
 Expected Value, Variance and Moments Cont’d…..
▪ The variance of a discrete random variable X is given by:

 2
X = Var ( X ) =  ( xk −  X ) PX ( xk )
2

▪ The standard deviation of a random variable X, denoted by σX, is

simply the square root of the variance, i.e.,

 X = E ( X −  X ) 2 = Var ( X )

iii.Moments
▪ The nth moment of a continuous random variable X is defined as:

E ( X ) =  x n f X ( x)dx ,
n
n 1
−
14
Expected Value, Variance and Moments Cont’d…..

▪ Similarly, the nth moment of a discrete random variable X is

given by:

E ( X ) =  xk PX ( xk ) , n 1
n

▪ Mean of X is the first moment of the random variable X.

15
Some Special Distributions

i. Continuous Probability Distributions

1. Normal (Gaussian) Distribution
▪ The random variable X is said to be normal or Gaussian random
variable if its pdf is given by:
1 −( x −  ) 2 / 2 2
f X ( x) = e .
2 2
▪ The corresponding distribution function is given by:
x 1  x− 
FX ( x) =  e −( y −  ) 2 / 2 2
dy = G 
−
2 2
  
x 1 −y /2
where G ( x) =  e dy
2

−
2
16
Some Special Distributions Cont’d……

▪ The normal or Gaussian distribution is the most common

continuous probability distribution.
f X (x)

x

Fig. Normal or Gaussian Distribution

17
 Some Special Distributions Cont’d……

2. Uniform Distribution f X (x)

1
 1 b−a
 , a xb
f X ( x) =  b − a a b
x

 0, otherwise. Fig. Uniform Distribution

3. Exponential Distribution
f X (x)
 1 −x / 
 e , x  0,
f X ( x) =   x

 0, otherwise. Fig. Exponential Distribution

18
 Some Special Distributions Cont’d……

4. Gamma Distribution

 x −1 −x / 
 e , x  0,
f X ( x ) =  ( )  


 0, otherwise.
5. Beta Distribution
 1
 x a −1 (1 − x) b −1 , 0  x  1,
f X ( x ) =   ( a, b)

 0, otherwise.
where
1
 ( a , b) = 0
u a −1 (1 − u )b−1 du.

19
 Some Special Distributions Cont’d……

6. Rayleigh Distribution

 x − x 2 / 2 2
 2e , x  0,
f X ( x ) = 

 0, otherwise.

7. Cauchy Distribution
 /
f X ( x) = , −   x  +.
 + (x − )
2 2

8. Laplace Distribution
1 −|x|/ 
f X ( x) = e , −   x  +.
2
20
Some Special Distributions Cont’d….

i. Discrete Probability Distributions

1. Bernoulli Distribution
P ( X = 0) = q, P ( X = 1) = p.
2. Binomial Distribution
 n  k n −k
P( X = k ) = 
k p q , k = 0,1,2,, n.
 
3. Poisson Distribution
k
P( X = k ) = e − , k = 0,1,2, , .
k!

21
Some Special Distributions Cont’d….

4. Hypergeometric Distribution
m  N −m 
   
k   n −k 
P( X = k ) =   
N 

, max(0, m + n − N )  k  min( m, n )
 
n 
 

5. Geometric Distribution

P( X = k ) = pqk , k = 0,1,2,, , q = 1 − p.

6. Negative Binomial Distribution

 k − 1 r k − r
P( X = k ) =   p q , k = r, r + 1, .
 r −1
22
Random Variable Examples

Example-1:
The pdf of a continuous random variable is given by:
kx , 0  x 1
f X ( x) = 
0 , otherwise

where k is a constant.
a. Determine the value of k .
b. Find the correspond ing cdf of X .
c. Find P (1 / 4  X  1)
d . Evaluate the mean and variance of X .
23
Random Variable Examples Cont’d……
Solution:
 1
a. −
f X ( x) dx = 1  
0
kxdx = 1
 x2 1
 k   = 1
 2 0
k
 =1
2
k = 2
2 x, 0  x 1
 f X ( x) = 
0, otherwise
24
Random Variable Examples Cont’d……
Solution:
b. The cdf of X is given by :
x
FX ( x ) =  −
f X (u ) du
Case 1 : for x  0
FX ( x ) = 0, since f X ( x ) = 0, for x  0
Case 2 : for 0  x  1
x x x
FX ( x) =  f X (u ) du =  2udu = u = x2
2
0 0 0

25
Random Variable Examples Cont’d……
Solution:

Case 3 : for x  1
1 1 1
FX ( x) =  f X (u ) du =  2udu = u =1
2
0 0 0
 The cdf is given by
0, x0
 2
FX ( x) =  x , 0  x 1
1, x 1


26
Random Variable Examples Cont’d……
Solution:
c. P(1 / 4  X  1)
i. Using the pdf
1 1
P(1 / 4  X  1) =  f X ( x)dx =  2 xdx
1/ 4 1/ 4

1
 P(1 / 4  X  1) = x = 15 / 16
2

1/ 4
 P(1 / 4  X  1) = 15 / 16
ii. Using the cdf
P(1 / 4  X  1) = FX (1) − FX (1 / 4)
 P(1 / 4  X  1) = 1 − (1 / 4) 2 = 15 / 16
 P(1 / 4  X  1) = 15 / 16
27
Random Variable Examples Cont’d……
Solution:
d. Mean and Variance
i. Mean
1 1
 X = E ( X ) =  xf X ( x) dx =  2 x 2 dx
0 0

2 x3 1
 X = = 2/3
3 0
ii. Variance
 X 2 = Var ( X ) = E ( X 2 ) − [ E ( X )]2
1 1
E ( X ) =  x f X ( x) dx =  2 x 3 dx = 1 / 2
2 2
0 0

  X = Var ( x ) = 1 / 2 − ( 2 / 3) 2 = 1 / 18
2

28
Random Variable Examples Cont’d……..

Example-2:
Consider a discrete random variable X whose pmf is given by:

1 / 3 , xk = −1, 0, 1

PX ( xk ) = 

0 , otherwise

Find the mean and variance of X .

29
Random Variable Examples Cont’d……
Solution:
i. Mean
1
 X = E( X ) = x
k = −1
k PX ( xk ) = 1 / 3(−1 + 0 + 1) = 0

ii. Variance
 X 2 = Var ( X ) = E ( X 2 ) − [ E ( X )]2
1
E( X ) =  k X k = − + + ] = 2/3
2 2 2 2 2
x P ( x ) 1 / 3[( 1) ( 0) (1)
k = −1

  X = Var ( x) = 2 / 3 − (0) 2 = 2 / 3
2

30
 Functions of One Random Variable
▪ Let X be a continuous random variable with pdf fX(x) and suppose
g(x) is a function of the random variable X defined as:
Y = g( X )

▪ We can determine the cdf and pdf of Y in terms of that of X.

▪ Consider some of the following functions.
aX + b
sin X X2

1 Y = g( X ) |X |
X
X
log X
eX | X | U ( x)

31
 Functions of a Random Variable Cont’d…..
▪ Steps to determine fY(y) from fX(x):
Method I:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. Determine the cdf of Y using the following basic approach.

FY ( y) = P( g ( X )  y) = P(Y  y)

3. Obtain fY(y) from FY(y) by using direct differentiation, i.e.,

dFY ( y )
fY ( y ) =
dy

32
 Functions of a Random Variable Cont’d…..
Method II:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. If Y=g(X) is one to one function and has an inverse
transformation x=g-1(y)=h(y), then the pdf of Y is given by:
dx dh( y)
fY ( y ) = f X ( x) = f X [h( y )]
dy dy
3. If Y=g(x) is not one-to-one function, then the pdf of Y can be
obtained as follows.
i. Find the real roots of the function Y=g(x) and denote them by xi

33
 Functions of a Random Variable Cont’d…..

ii. Determine the derivative of function g(xi ) at every real root xi ,

i.e. ,
dxi
g ( xi ) =
dy
iii. Find the pdf of Y by using the following formula.

dxi
f Y ( y) =  f X ( xi ) =  g ( xi ) f X ( xi )
i dy i

34
Examples on Functions of One Random Variable

Examples:
a. Let Y = aX + b. Find f Y ( y ).

b. Let Y = X 2 . Find f Y ( y ).

1
c. Let Y = . Find f Y ( y ).
X
 
d . The random variable X is uniform in the interval [− , ].
2 2
If Y = tan X , determine the pdf of Y .

35
Examples on Functions of One Random Variable…..
Solutions:

a. Y = aX + b
i. Using Method − I
Suppose that a  0

 y −b
Fy ( y ) = P(Y  y ) = P(aX + b  y ) = P X  
 a 

 y −b
FY ( y ) = FX  
 a 
dFY ( y ) 1  y −b
 f Y ( y) = = fX   (i )
dy a  a 

36
Examples on Functions of One Random Variable…..
Solutions:

a. Y = aX + b
i. Using Method − I
On the other if a  0, then

 y −b
Fy ( y ) = P(Y  y ) = P(aX + b  y ) = P X  
 a 

 y −b
FY ( y ) = 1 − FX  
 a 
dFY ( y ) 1  y −b
 f Y ( y) = = − fX   (ii )
dy a  a 
37
Examples on Functions of One Random Variable…..
Solutions:

a. Y = aX + b

i. Using Method − I

From equations (i ) and (ii ) , we obtain :

1  y −b
f Y ( y) = fX  , for all a
a  a 

38
 Examples on Functions of One Random Variable…..
Solutions:

a. Y = aX + b
ii. Using Method − II
The function Y = aX + b is one - to - one and the range
space of Y is R
y −b
For any y, x = = h( y ) is the principal solution
a
dx dh( y ) 1 dx 1
= =  =
dy dy a dy a
dh( y )  y −b
f X (h( y ) )  fY ( y ) =
dx 1
fY ( y ) = f X ( x) = fX  
dy dy a  a 

39
Examples on Functions of One Random Variable…..
Solutions:

b. The function Y = X 2 is not one - to - one and the range

space of Y is y  0
For each y  0, there are two solutions given by
x1 = − y and x 2 = y

40
Examples on Functions of One Random Variable…..
Solutions:
dx1 1 dx1 1
b. =−  = and
dy 2 y dy 2 y

dx2 1 dx2 1
=  =
dy 2 y dy 2 y

dxi dx1 dx2

f Y ( y) =  f X ( xi )  f Y ( y ) = f X ( x1 ) + f X ( x2 )
i dy dy dy

 1
2 y f X( ( y ) + f (− y )),
X y0
 f Y ( y) = 

 0, otherwise
41
Examples on Functions of One Random Variable…..
Solutions:
1
c. The function Y = is one - to - one and the range
X
space of Y is IR /0
1
For any y, x = = h( y ) is the principal solution
y
dx dh( y ) 1
= =− 2
dy dy y

dh( y ) 1
f X (h( y ) )  f Y ( y ) = 2 f X  
dx 1
f Y ( y) = f X ( x) =
dy dy y  y
1
IR /0
1
 f Y ( y) = 2
f X 
  ,
y  y
42
Examples on Functions of One Random Variable…..
Solutions:
d . The function Y = tan X is one - to - one and the range

space of Y is (−, )

For any y, x = tan −1 y = h( y ) is the principal solution

dx dh( y ) 1
= =
dy dy 1+ y2

dh( y ) 1/ 
f X (h( y ) )  f Y ( y ) =
dx
f Y ( y) = f X ( x) =
dy dy 1+ y2

1
 f Y ( y) = , −  y  
 (1 + y )
2

43
Examples on Functions of One Random Variable…..
Solutions:

44