Review of
Random Variables
for Communications
EN 2072
Semester 4 May 2011
Prof. Dileeka Dias
Department of Electronic & Telecommunication
Engineering
University of Moratuwa
Contents
Introduction
Random Variables
Discrete
Continuous
Joint Random Variables
Discrete
Continuous
Functions of Random Variables
Single Random Variable
Mean
Variance
Moments
Two Random variables
Correlation and Covariance
Random Variables: Definition
Outcome of a random experiment may be
A numerical value (1, 2, 3, 4, 5, 6)
Described by a phrase (Heads, Tails)
From a mathematical point of view it is preferable to have a
numerical value (real number) assigned to each sample
point according to some rule.
If there are m sample points
, we assign a real
number x( ) to each sample point.
X( ) is the function that maps sample points into real
numbers x1, x2, .. xm
X is a random variable which takes on values
x1, x2, .. xm.
Random Variables: Definition
Random Variables: Definition
Example 1
Heads 1
Tails 0
Example 2
1 10
2 20
3 30
4 40
5 50
6 60
X = 1 or 0 , each with probability 1/2
1, i " Heads"
xi
0, i "Tails "
X = 10, 20, 30 .60, each with
probability 1/6
xi 10 i
Random Variables: Discrete
A discrete random variable maps events to
values of a countable set (e.g., the integers),
with each value in the range having
probability greater than or equal to zero.
A discrete random variable is described by a
discrete proability density function (probablity
mass function)
PX ( xi ), i 1, 2, .......m
X ( xi ) 1
where
Random Variables: Discrete
Discrete Probability Density Function (PDF) or
Probability Mass Function (PMF) and Discrete
Cumulative Distribution Function (CDF)
Discrete CDF:
Discrete PDF:
1
6 , x 1
3
PX ( xi ) , x 2
6
2 , x 4
6
FX ( x) P( X x)
x 1
0,
1
, 1 x 2
6
4
, 2 x4
6
1,
x4
Random Variables: Discrete
Example: The Poisson random variable
A random variable X is said to have a Poisson
e k
distribution if P( X k )
, k 0, 1, 2, 3 ......
k!
Where is called average or the expected value
PDF
CDF
Random Variables: Discrete
Example:
Telephone calls arriving at a
switch, page view requests to a
website are examples of
Poisson processes
The probability that there are k
incoming calls during the time t
and t+ is given by
e( ) k
PN (t ) N (t ) k
k!
Where is the average call
arrival rate
Random Variables: Continuous
A random variable is called continuous if it can
assume all possible values in the possible
range of the random variable.
The continuous random variable X is described
by the (continuous) probability density
function.
Random Variables: Continuous
It is denoted by f X (x) , the probability that the
random variable X takes the value between
x and x+ x where is a very small change
in X. f X ( x) P( x X x x)
Random Variables: Continuous
If there are two points a and b then the
probability that the random variable will take
the value between a and b is given by
f X ( x)dx 1
Random Variables: Continuous
The CDF of a continuous random variable is
x
given by:
FX ( x)
fU (u )du
f X ( x) d FX ( x)
dx
Random Variables: Continuous
The Gaussian Random Variable
f X ( x)
( x ) 2
1
2
2 2 ,
N (0,1) Standard Normal Distribution
N ( , 2 )
Mean:
Variance: 2
Source: Wikipedia
Random Variables: Continuous
The Gaussian Random Variable
x
FX ( x)
X (u ) du
x (u ) 2
2 2 du
2 2
0 (u ) 2
2 2 du
2 2
u
2
1
1
2 2 du
2 2 0
x /
x (u ) 2
2 2
t 2
e
0
1 1
x
erf
2 2
2
2
dt
erf x
t 2
dt
Random Variables: Continuous
The Gaussian Random Variable
1 1
x
FX ( x) erf
2 2
2
2
x
1 Q
erf x
t 2
dt
x
1
Qx 1 erf
2
2
Random Variables: Continuous
Example:
Transmitted pulses
1 p(t)
0 -p(t)
Received signal
The received signal is
Sampled at the peak point
And compared with a threshold
of 0
The sample consists of a contribution
from the signal and a contribution from
the AWGN that corrupts the channel.
The sample value can be
Ap + n
-Ap + n
n is a sample value of the random variable with zero mean
Random Variables: Continuous
Probability of error
P(e|0) = P(-AP + n) > 0
= P(n > Ap )
P(e|1) = P(AP + n) < 0
= (n< - AP)
P(n< - AP)
P(n > Ap )
Random Variables: Continuous
For a Gaussian Random variable
x
F ( x) 1 Q
P( X x)
X
n
n
P( X n) 1 1 Q
Q
A
P(e | 0) P(n AP ) Q P
A
P(e | 1) P(n AP ) Q P
Two (Joint) Random Variables: Discrete
If X and Y are two discrete random variables, the
conditional probability of xi and yj is given by
PX |Y ( xi | y j )
X |Y ( xi | y j )
Conditional
Probability
Y | X ( y j | xi ) 1
PXY ( xi , y j ) PX |Y ( xi | y j ) PY ( y j ) PY | X ( y j | xi ) PX ( xi )
Joint Probability
Using
Joint Random Variables : Discrete
PXY ( xi , y j ) PX |Y ( xi | y j ) PY ( y j ) PY | X ( y j | xi ) PX ( xi )
XY ( xi , y j )
X |Y ( xi | y j ) PY ( y j )
PY ( y j )
X |Y ( xi | y j )
PY ( y j )
Similarly
XY ( xi , y j ) PX ( xi )
Marginal Probabilities
Joint Random Variables : Discrete
PXY ( xi , y j ) PX |Y ( xi | y j ) PY ( y j ) PY | X ( y j | xi ) PX ( x j )
X and Y are independent if
PX |Y ( xi | y j ) PX ( xi ) or
PY | X ( y j | xi ) PY ( y j )
Hence,
PXY ( xi , y j ) PX ( xi ) PY ( y j )
Joint Random Variables : Discrete
Example: A BSC has error probability p. The
probability of transmitting a 1 is a and the
probability of transmitting a 0 is 1-a.
Determine the probabilities of receiving a 1
and a 0 at the receiver.
X: RV indicating the input
Y: RV indicating the output
x1 = 1, x2 = 0
PX (1 ) = a, PX (0 ) = 1-a
Joint
Random
Variables
:
Discrete
y =0
x 0
2
2=
PY (0 ) = ??
PX (0 ) = 1-a
x1 = 1
PX (1 ) = a
y1 = 1
PY (1 ) = ??
Probability of error
PY | X (0 | 1) PY | X (1 | 0) p
PY | X (0 | 0) PY | X (1 | 1) 1 p
PY ( y j )
XY ( xi , y j )
Probability of
correct reception
Y | X ( y j | xi ) PX ( xi )
for j 1
PY (1) PY | X (1 | 1) P(1) PY | X (1 | 0) P(0) (1 p)a p (1 a)
( p a) 2 pa
for j 2
PY (0) PY | X (0 | 1) P(1) PY | X (0 | 0) P(0) pa (1 p)(1 a)
1 ( p a) 2 pa 1 PY (1)
Joint Random Variables: Continuous
Joint CDF of continuous random variables
FXY ( x, y) P( X x and Y y)
Joint PDF of continuous random variables
2
f XY ( x, y )
FXY ( x, y)
x y
x2 y 2
P( x1 X x2 , y1 Y y2 )
x1 y1
f XY ( x, y )dxdy 1
f XY ( x, y )dxdy
Joint Random Variables : Continuous
Conditional Densities for continuous random
variables
Joint
PDFs
f XY ( x, y ) f X |Y ( x | y ) fY ( y )
f XY ( x, y ) fY | X ( y | x) f X ( x)
Conditional PDFs
Hence,
f X |Y ( x | y )
fY | X ( y | x ) f X ( x )
fY ( y )
Bayes Rule for
Continuous RVs
X and Y are independent if
f X |Y ( x | y ) f X ( x) or
Which means that
fY | X ( y | x) fY ( y )
f XY ( x, y) f X ( x) fY ( y)
Functions of a Random Variable
The Mean (Expected Value)
Example: for a Gaussian RV,
E[ X ]
( x ) 2
xe
2 2 dx
( y) 2
( y )e
xf
X ( x)dx
or
x P( x )
i
( y ) 2
2 2 dy
( y) 2
2 2 dy e
ye
2
Odd function
of y
DC value of a
signal
E[ X ] X
2 2
Let x y
E[ X ]
__
2 2 dy
Functions of a Random Variable
The Mean (Expected Value)
Let Y = g(X)
__
E[Y ] Y
A function of a RV
g ( x) f
X ( x) dx
or
g ( x ) P( x )
i
Therefore:
Mean Square Value E[ X 2 ] x 2 f X ( x)dx
Functions of a Random Variable
The Mean (Expected Value)
Let Y = g(X)
__
E[Y ] Y
A function of a RV
g ( x) f
X ( x) dx
or
g ( x ) P( x )
i
Therefore:
E[ X 2 ]
x 2 f X ( x)dx
E[ X 2 ]
Mean Square Value
of a signal
Root Mean Square
(RMS) value of a signal
Functions of a Random Variable
Example
A sinusoidal signal is given by A cost. This is
sampled at random time instants. The sampled
output is a random variable X.
Find E[X] and E[X2]
The sampling instant is a random variable T.
Let be another rv.
~ U 0,2
P(q
Uniform Distribution
X A cos
1/2
Functions of a Random Variable
Example contd.
X A cos
2
E[ X ]
1
dq 0
2
A cosq
0
2
2
E[ X ]
1
A cos q
dq
2
2
A2 A2
2
2
Functions of a Random Variable
Variance
2
2
__
_
2
Variance( X ) X
E X X x X f X ( x)dx
__
Std. Deviation ( X ) E X X
Average AC power
of a signal
Functions of a Random Variable
Variance
__
X2 E X X
__ __ 2
EX 2 2X X X
__ 2
2
E[ X ] 2 E[ X ] X E X
__
__ 2
E[ X 2 ] X
E[ X 2 ] E[ X ]2
Functions of a Random Variable
Moments
The nth moment of X is given by:
__
n
E[ X n ] X
x n f X ( x)dx
The nth central moment of X is given by:
__
n
E[( X X ) ] x x f X ( x)dx
Functions of two Random Variables
Correlation and Covariance
Estimate the nature of dependence between two
random variables
Correlation
R XY E[ XY ]
xyf
XY ( x, y )dxdy
If X and Y are independent:
R XY
xf
X ( x)dx
yf ( y)dy E[ X ]E[Y ]
Y
X and Y are said to be orthogonal if
RXY E[ XY ] 0
X Y
Functions of two Random Variables
Correlation and Covariance
Covariance
__
__
__
__
C XY XY E X X Y Y x X y Y f XY ( x, y )dxdy
__
__
C XY XY E X X Y Y
__
__
__ __
E[ XY ] X E[Y ] Y E[ X ] X Y
__ __
E[ XY ] E[ X ]E[Y ] R XY X Y
Correlation can be
Positive
Zero or
Negative
For Independent RVs
__ __
R XY E[ X ]E[Y ] X Y
C XY 0
Functions of two Random Variables
Correlation and Covariance
Covariance
__ __
C XY RXY X Y
Correlation can be
Positive
Zero or
Negative
For Independent RVs
__ __
R XY E[ X ]E[Y ] X Y
C XY 0
X and Y are uncorrelated
Functions of two Random Variables
Correlation and Covariance
X and Y are positively correlated
Scatter
Plots
X and Z are negatively correlated
X and W are uncorrelated
Functions of two Random Variables
Relationship between Independence and
Correlatedness
Correlation Coefficient
XY
C XY
XY
1 XY 1
