Random Processes
Random processes represent the formal mathematical
model of these random signals.
Random processes have the following properties:
1. Random processes are functions of time.
2. Random processes are random in the sense that it is not
possible to predict exactly what waveform will be
observed in the future.
MITE-Moodabidri 3
Random Processes
MITE-Moodabidri 4
Random Processes
MITE-Moodabidri 5
Conditional Probability
◊ Example 1: Binary Symmetric Channel
◊ This channel is said to be discrete in that it is designed to handle
discrete messages.
◊ The channel is memoryless in the sense that the channel output at
any time depends only on the channel input at that time.
◊ The channel is symmetric, which means that the probability of
receiving symbol 1 when 0 is sent is the same as the probability
of receiving symbol 0 when symbol 1 is sent.
1
Conditional Probability
◊ Example 1 Binary Symmetric Channel (continued)
◊ The a priori probabilities of sending binary symbols 0 and 1:
P A0 p0 P A1 p1
◊ The conditional probabilities of error:
◊ The probability of receiving symbol 0 is given by:
◊ The probability of receiving symbol 1 is given by:
2
Conditional Probability
◊ Example 1 Binary Symmetric Channel (continued)
◊ The a posteriori probabilities P[A0|B0] and P[A1|B1]:
3
(a)
4
5
6
7
8
F Y OF Y IS 1 BY
B-A AS X IS
UNIFORML Y
DISTRIBUTE D
1/11
9
AREA
UNDER
DENSITY FN
10
11
12
Gaussian Process
N
◊ A random variable Y is defined by: Y ai Xi
i1
Y gt Xtdt
T
ai are constants
0
Xi are randomvariables
◊ We refer to Y as a linear functional of X(t). Y is a linear function of the Xi .
◊ If the weighting function g(t) is such that the mean-square value of
the random variable Y is finite, and if the random variable Y is a
Gaussian-distributed random variable for every g(t) in this class of
functions, then the process X(t) is said to be a Gaussian process.
◊ In other words, the process X(t) is a Gaussian process if every
linear functional of X(t) is a Gaussian random variable.
◊ The Gaussian process has many properties that make analytic
results possible.
◊ The random processes produced by physical phenomena are often
such that a Gaussian model is appropriate.
13
Gaussian Process
◊ The random variable Y has a Gaussian distribution if its probability
density function has the form
Y
2πσ
2σ2
μY : the mean of the random variable Y
σY2 : the variance of the random variable Y
◊ If the Gaussian random variable Y is normalized to have a mean of
zero and a variance of one, such a normalized Gaussian
distribution is commonly written as N(0,1).
14
Gaussian Process
◊ Central Limit Theorem
◊ Let Xi, i = 1, 2, …, N, be a set of random variables that satisfies
the following requirements:
◊ The Xi are statistically independent.
◊ The Xi have the same probability distribution with mean μX and
variance σ2X.
◊ The Xi so described are said to constitute a set of independent
and identically distributed (i.i.d.) random variables.
The central limit theorem states that the probability distribution
of VN approaches a normalized Gaussian distribution N(0,1) in
the limit as N approaches infinity.
15
Gaussian Process
◊ Property 1: If a Gaussian process X(t) is applied to a stable linear
filter, then the output of Y(t) is also Gaussian.
◊ Property 2: Consider the set of random variables or samples X(t1),
X(t2), …X(tn), obtained by observing a random process X(t) at time t1,
t2, …, tn. If the process X(t) is Gaussian, then this set of random
variables is jointly Gaussian for any n, with their n-fold joint
probability density function being completely determined by
specifying the set of means:
i 1, 2,…, n
and the set of autocovariance functions:
k, i 1, 2, ..., n
◊ Consider the composite set of random variables X(t1), X(t2),…, X(tn),
Y(u1), Y(u2),…, Y(um). We say that the processes X(t) and Y(t) are
jointly Gaussian if this composite set of random variables are jointly
Gaussian for any n and m.
16
Gaussian Process
◊ Property 3: If a Gaussian process is wide-sense stationary, then
the process is also stationary in the strict sense.
◊ Property 4: If the random variables X(t1), X(t2), …X(tn), are
uncorrelated, that is
then these random variables are statistically independent.
◊ The implication of this property is that the joint probability
density function of the set of random variables X(t1), X(t2),…,
X(tn) can be expressed as the product of the probability density
functions of the individual random variables in the set.
17
18
