PROBABILITY THEORY AND

STOCHASTIC PROCESSES

LECTURE NOTES

B. TECH
(II YEAR – I SEM)
(2024-25)

Prepared by:
Mrs. N. Saritha, Assistant Professor

Department of Electronics and Communication Engineering

MALLA REDDY COLLEGE

OF ENGINEERING & TECHNOLOGY
(Autonomous Institution – UGC, Govt. of India)
Recognized under 2(f) and 12 (B) of UGC ACT 1956
(Affiliated to JNTUH, Hyderabad, Approved by AICTE - Accredited by NBA & NAAC – ‘A’ Grade - ISO 9001:2015 Certified)
Maisammaguda, Dhulapally (Post Via. Kompally), Secunderabad – 500100, Telangana State, India
 MALLA REDDY COLLEGE OF
ENGINEERING AND TECHNOLOGY
(AUTONOMOUS INSTITUTION: UGC, GOVT. OF INDIA)

ELECTRONICS AND COMMUNICATION ENGINEERING

II ECE I SEM

PROBABILITY THEORY
AND STOCHASTIC
PROCESSES
 CONTENTS

 SYLLABUS

 UNIT-I-PROBABILITY AND RANDOM VARIABLE

 UNIT-II- DISTRIBUTION AND DENSITY FUNCTIONS AND OPERATIONS ON

ONE RANDOM VARIABLE

 UNIT-III-MULTIPLE RANDOM VARIABLES AND OPERATIONS

 UNIT-IV-STOCHASTIC PROCESSES-TEMPORAL CHARACTERISTICS

 UNIT-V- STOCHASTIC PROCESSES-SPECTRAL CHARACTERISTICS

 MALLA REDDY COLLEGE OF ENGINEERING AND
TECHNOLOGY
II Year B.Tech. ECE- I Sem L/T/P/C
3/-/-/3
(R22A0404) PROBABILITY THEORY AND STOCHASTIC PROCESSES
COURSE OBJECTIVES:
1) To expose the students to the basics of probability theory and random processes essential for
their subsequent study of analog and digital communication.
2) To understand the basic concepts of probability, single and multiple random variables and to
introduce some standard distributions applicable to engineering which can describe real life
phenomenon
3) To understand the basic concepts of random processes.
4) To understand the concept of correlation and spectral densities.
5) To understand the significance of linear systems with random inputs.

UNIT I:
PROBABILITY AND RANDOM VARIABLE
Probability: Set theory, Experiments and Sample Spaces, Discrete and Continuous Sample Spaces,
Events, Probability Definitions and Axioms, Joint Probability, Conditional Probability, Total
Probability, Bayes’ Theorem, and Independent Events, Bernoulli’s trials.
The Random Variable: Definition of a Random Variable, Conditions for a Function to be a Random
Variable, Discrete and Continuous.

UNIT II:
DISTRIBUTION AND DENSITY FUNCTIONS AND OPERATIONS ON ONE RANDOM
VARIABLE
Distribution and density functions: Distribution and Density functions, Properties, Binomial,
Uniform, Exponential, Gaussian and Conditional Distribution and Conditional Density function and
its properties, problems.
Operation on One Random Variable: Expected value of a random variable, function of a random
variable, moments about the origin, central moments, variance, characteristic function, moment
generating function.

UNIT III:
MULTIPLE RANDOM VARIABLES AND OPERATIONS ON MULTIPLE RANDOM
VARIABLES
Multiple Random Variables: Joint Distribution Function and Properties, Joint density Function and
Properties, Marginal Distribution and density Functions, conditional Distribution and density
Functions, Statistical Independence.
Operations on Multiple Random Variables: Expected Value of a Function of Random
Variables, Joint Moments about the Origin, Joint Central Moments.

UNIT IV:
Random Processes-Temporal Characteristics: The Random process Concept, Classification of
Processes, Deterministic and Nondeterministic Processes, Distribution and Density Functions,
Statistical Independence and concept of Stationarity: First-Order Stationary Processes, Second- Order
and Wide-Sense Stationarity, Nth-Order and Strict-Sense Stationarity, Autocorrelation Function and Its
Properties, Cross-Correlation Function and Its Properties, Covariance Functions and its properties.
 Linear system Response: Mean and Mean-squared value, Autocorrelation, Cross-Correlation
Functions.

UNIT V:
Random Processes-Spectral Characteristics: The Power Spectrum and its Properties, Relationship
between Power Spectrum and Autocorrelation Function, the Cross-Power Density Spectrum and
Properties, Relationship between Cross-Power Spectrum and Cross- Correlation Function.
Spectral characteristics of system response: Power Density Spectrum of response of linear
system, Cross Power Spectral Density of input and output of a Linear System.

TEXT BOOKS:
1) Probability, Random Variables & Random Signal Principles -Peyton Z. Peebles, TMH, 4th
Edition, 2001.
2) Probability and Random Processes-Scott Miller, Donald Childers,2Ed,Elsevier,2012

REFERENCE BOOKS:
1) Theory of probability and Stochastic Processes-Pradip Kumar Gosh, University Press
2) Probability and Random Processes with Application to Signal Processing - Henry Stark and
John W. Woods, Pearson Education, 3rd Edition.
3) Probability Methods of Signal and System Analysis- George R. Cooper, Clave D. MC
Gillem, Oxford, 3rd Edition, 1999.
4) Statistical Theory of Communication -S.P. Eugene Xavier, New Age Publications 2003
5) Probability, Random Variables and Stochastic Processes Athanasios Papoulis and
S.Unnikrishna Pillai, PHI, 4th Edition, 2002.

COURSE OUTCOMES
1) Exposed to the basics of probability theory and random processes essential for their
subsequent study of analog and digital communication.
2) Understand the axiomatic formulation of modern Probability Theory and think of
random variables as an intrinsic need for the analysis of random phenomena.
3) Characterize probability models and function of random variables based on single &
multiples random variables.
4) Evaluate and apply moments & characteristic functions and understand the concept of
inequalities and probabilistic limits.
5) Understand the concept of random processes and determine covariance and spectral
density of stationary random processes.
 UNIT I
Probability and Random Variable
Probability:

 Set theory
 Experiments
 Sample Spaces, Discrete and Continuous Sample Spaces
 Events
 Probability Definitions and Axioms
 Mathematical Model of Experiments
 Joint Probability
 Conditional Probability
 Total Probability
 Bayes’ Theorem
 Independent Events
 Bernoulli’s trials

Random Variable:

 Definition of a Random Variable

 Conditions for a Function to be a Random Variable
 Discrete and Continuous, Mixed Random Variable
 UNIT – I
PROBABILITY AND RANDOM VARIABLE

PROBABILITY

Introduction

It is remarkable that a science which began with the consideration of games of chance
should have become the most important object of human knowledge.

A brief history

Probability has an amazing history. A practical gambling problem faced by the French nobleman
Chevalier de Méré sparked the idea of probability in the mind of Blaise Pascal (1623-1662), the
famous French mathematician. Pascal's correspondence with Pierre de Fermat (1601-1665), another
French Mathematician in the form of seven letters in 1654 is regarded as the genesis of probability.
Early mathematicians like Jacob Bernoulli (1654-1705), Abraham de Moivre (1667-1754), Thomas
Bayes (1702-1761) and Pierre Simon De Laplace (1749-1827) contributed to the development of
probability. Laplace's Theory Analytique des Probabilities gave comprehensive tools to calculate
probabilities based on the principles of permutations and combinations. Laplace also said,
"Probability theory is nothing but common sense reduced to calculation."

Later mathematicians like Chebyshev (1821-1894), Markov (1856-1922), von Mises (1883-
1953), Norbert Wiener (1894-1964) and Kolmogorov (1903-1987) contributed to new
developments. Over the last four centuries and a half, probability has grown to be one of the most
essential mathematical tools applied in diverse fields like economics, commerce, physical
sciences, biological sciences and engineering. It is particularly important for solving practical
electrical-engineering problems in communication, signal processing and computers.
Notwithstanding the above developments, a precise definition of probability eluded the
mathematicians for centuries. Kolmogorov in 1933 gave the axiomatic definition of probability
and resolved the problem.

Randomness arises because of

 random nature of the generation mechanism

 Limited understanding of the signal dynamics inherent imprecision in measurement,
observation, etc.
For example, thermal noise appearing in an electronic device is generated due to random motion of
electrons. We have deterministic model for weather prediction; it takes into account of the factors
affecting weather. We can locally predict the temperature or the rainfall of a place on the basis of
previous data. Probabilistic models are established from observation of a random phenomenon.
While probability is concerned with analysis of a random phenomenon, statistics help in building
such models from data.

3
Deterministic versus probabilistic models

A deterministic model can be used for a physical quantity and the process generating it provided
sufficient information is available about the initial state and the dynamics of the process
generating the physical quantity. For example,

 We can determine the position of a particle moving under a constant force if we know the
initial position of the particle and the magnitude and the direction of the force.

 We can determine the current in a circuit consisting of resistance, inductance and

capacitance for a known voltage source applying Kirchoff's laws.

Many of the physical quantities are random in the sense that these quantities cannot be predicted
with certainty and can be described in terms of probabilistic models only. For example,

 The outcome of the tossing of a coin cannot be predicted with certainty. Thus, the
outcome of tossing a coin is random.

 The number of ones and zeros in a packet of binary data arriving through a
communication channel cannot be precisely predicted is random.

 The ubiquitous noise corrupting the signal during acquisition, storage and transmission
can be modelled only through statistical analysis.

How to Interpret Probability

Mathematically, the probability that an event will occur is expressed as a number between 0 and
1. Notationally, the probability of event A is represented by P (A).

 If P (A) equals zero, event A will almost definitely not occur.

 If P (A) is close to zero, there is only a small chance that event A will occur.
 If P (A) equals 0.5, there is a 50-50 chance that event A will occur.
 If P(A) is close to one, there is a strong chance that event A will occur.
 If P(A) equals one, event A will almost definitely occur.

In a statistical experiment, the sum of probabilities for all possible outcomes is equal to one. This
means, for example, that if an experiment can have three possible outcomes (A, B, and C), then
P(A) + P(B) + P(C) = 1.

4
Applications

Probability theory is applied in everyday life in risk assessment and in trade on financial markets.
Governments apply probabilistic methods in environmental regulation, where it is called pathway
analysis

Another significant application of probability theory in everyday life is reliability. Many consumer
products, such as automobiles and consumer electronics, use reliability theory in product design to
reduce the probability of failure. Failure probability may influence a manufacturer's decisions on a
product's warranty.

THE BASIC CONCEPTS OF SET THEORY

Some of the basic concepts of set theory are:

Set: A set is a well-defined collection of objects. These objects are called elements or members
of the set. Usually, uppercase letters are used to denote sets.

The set theory was developed by George Cantor in 1845-1918. Today, it is used in almost every
branch of mathematics and serves as a fundamental part of present-day mathematics.

In everyday life, we often talk of the collection of objects such as a bunch of keys, flock of birds,
pack of cards, etc. In mathematics, we come across collections like natural numbers, whole
numbers, prime and composite numbers.

We assume that,

● the word set is synonymous with the word collection, aggregate, class and comprises of elements.

● Objects, elements and members of a set are synonymous terms.

● Sets are usually denoted by capital letters A, B, C, . , etc.

● Elements of the set are represented by small letters a, b, c, . , etc.

If ‗a ‘is an element of set A, then we say that ‗a‘belongs to A. We denote the phrase ‗belongs
to ‘by the Greek symbol ‗∈‗ (epsilon). Thus, we say that a ∈ A.

If ‗b‘is an element which does not belong to A, we represent this as b ∉ A.

Examples of sets:

1. Describe the set of vowels.

If A is the set of vowels, then A could be described as A = {a, e, i, o, u}

5
1.Describe the set of positive integers.

Since it would be impossible to list all of the positive integers, we need to use a rule to describe
this set. We might say A consists of all integers greater than zero.

2. Set A = {1, 2, 3} and Set B = {3, 2, 1}. Is Set A equal to Set B?

Yes. Two sets are equal if they have the same elements. The order in which the elements are
listed does not matter.

3. What is the set of men with four arms?

Since all men have two arms at most, the set of men with four arms contains no elements. It is
the null set (or empty set).

4. Set A = {1, 2, 3} and Set B = {1, 2, 4, 5, 6}. Is Set A a subset of Set B?

Set A would be a subset of Set B if every element from Set A were also in Set B. However, this
is not the case. The number 3 is in Set A, but not in Set B. Therefore, Set A is not a subset of
Set B

Some important sets used in mathematics are

N: the set of all natural numbers = {1, 2, 3, 4, .. }

Z: the set of all integers = {....., -3, -2, -1, 0, 1, 2, 3, .. }

Q: the set of all rational numbers

R: the set of all real numbers

Z+: the set of all positive integers

W: the set of all whole numbers

The different types of sets are explained below with examples.

1. Empty Set or Null Set:

A set which does not contain any element is called an empty set, or the null set or the void set
and it is denoted by ∅ and is read as phi. In roster form, ∅ is denoted by {}. An empty set is a
finite set, since the number of elements in an empty set is finite, i.e., 0.

For example: (a) the set of whole numbers less than 0.

Clearly there is no whole number less than 0. Therefore, it is an empty set.

6
 2. Singleton Set: A set which contains only one element is called a singleton set.

For example:

• A = {x : x is neither prime nor composite}

It is a singleton set containing one element, i.e., 1.

• B = {x : x is a whole number, x < 1}

This set contains only one element 0 and is a singleton set.

• Let A = {x : x ∈ N and x² = 4}

Here A is a singleton set because there is only one element 2 whose square is 4.

• Let B = {x : x is a even prime number}

Here B is a singleton set because there is only one prime number which is even, i.e., 2.

3. Finite Set:

A set which contains a definite number of elements is called a finite set. Empty set is also
called a finite set.

For example:

• The set of all colors in the rainbow.

• N = {x : x ∈ N, x < 7}

• P = {2, 3, 5, 7, 11, 13, 17, ..... 97}

4. Infinite Set:

The set whose elements cannot be listed, i.e., set containing never-ending elements is called an
infinite set.

For example:

• Set of all points in a plane

• A = {x : x ∈ N, x > 1}

• Set of all prime numbers

• B = {x : x ∈ W, x = 2n}

Note:

7
 All infinite sets cannot be expressed in roster form.

For example:

The set of real numbers since the elements of this set do not follow any particular pattern.

5. Cardinal Number of a Set:

The number of distinct elements in a given set A is called the cardinal number of A. It is
denoted by n(A). And read as the number of elements of the set‘.

For example:

• A {x: x ∈ N, x < 5}

A = {1, 2, 3, 4}

Therefore, n(A) = 4

• B = set of letters in the word

ALGEBRA B = {A, L, G, E, B, R}

Therefore, n(B) = 6

6. Equivalent Sets:

Two sets A and B are said to be equivalent if their cardinal number is same, i.e., n(A) = n(B).
The symbol for denoting an equivalent set is ‗↔‘.

For example:

A = {1, 2, 3} Here n(A) = 3

B = {p, q, r} Here n(B) = 3

Therefore, A ↔ B

7. Equal sets:

Two sets A and B are said to be equal if they contain the same elements. Every element of A is
an element of B and every element of B is an element of A.

For example:

A = {p, q, r, s}

B = {p, s, r, q}

Therefore, A = B

8. Disjoint Sets:
8
Two sets A and B are said to be disjoint, if they do not have any element in common.

For example;

A = {x : x is a prime number}

B = {x : x is a composite number}.

Clearly, A and B do not have any element in common and are disjoint sets.

9. Overlapping sets:

Two sets A and B are said to be overlapping if they contain at least one element in common.

For example;

• A = {a, b, c, d}

B = {a, e, i, o, u}

• X = {x : x ∈ N, x < 4}

Y = {x : x ∈ I, -1 < x <

4}

Here, the two sets contain three elements in common, i.e., (1, 2, 3)

10. Definition of Subset:

If A and B are two sets, and every element of set A is also an element of set B, then A is called
a subset of B and we write it as A ⊆ B or B ⊇ A

The symbol ⊂ stands for ‗is a subset of‘ or ‗is contained in‘

• Every set is a subset of itself, i.e., A ⊂ A, B ⊂ B.

• Empty set is a subset of every set.

• Symbol ‗⊆‘ is used to denote ‗is a subset of‘ or ‗is contained in‘.

• A ⊆ B means A is a subset of B or A is contained in B.

• B ⊆ A means B contains A.

Examples;

1. Let A = {2, 4, 6}

B = {6, 4, 8, 2}

9
Here A is a subset of B

Since, all the elements of set A are contained in set B.

But B is not the subset of A

Since, all the elements of set B are not contained in set A.

Notes:

If ACB and BCA, then A = B, i.e., they are equal sets.

Every set is a subset of itself.

Null set or ∅ is a subset of every set.

2. The set N of natural numbers is a subset of the set Z of integers and we write N ⊂ Z.

3. Let A = {2, 4, 6}

B = {x : x is an even natural number less than

8} Here A ⊂ B and B ⊂ A.

Hence, we can say A = B

4. Let A = {1, 2, 3, 4}

B = {4, 5, 6, 7}

Here A ⊄ B [⊄ denotes ‗not a subset of‘]

11. Proper Subset:

If A and B are two sets, then A is called the proper subset of B if A ⊆ B but B ⊇ A i.e., A ≠ B.
The symbol ‗⊂‘ is used to denote proper subset. Symbolically, we write A ⊂ B.

For example;

1. A = {1, 2, 3, 4}

Here n(A) = 4

B = {1, 2, 3, 4, 5}

Here n(B) = 5

We observe that, all the elements of A are present in B but the element ‗5‘ of B is not present in

10
A. So, we say that A is a proper subset of B.
Symbolically, we write it as A ⊂ B

Notes:

No set is a proper subset of itself.

Null set or ∅ is a proper subset of every set.

2. A = {p, q, r}

B = {p, q, r, s, t}

Here A is a proper subset of B as all the elements of set A are in set B and also A ≠ B.

12. Power Set:

The collection of all subsets of set A is called the power set of A. It is denoted by P(A). In
P(A), every element is a set.

For example;

If A = {p, q} then all the subsets of A will be

P(A) = {∅, {p}, {q}, {p, q}}

Number of elements of P(A) = n[P(A)] = 4 = 22

In general, n[P(A)] = 2m where m is the number of elements in set A.

13. Universal Set

A set which contains all the elements of other given sets is called a universal set. The symbol
for denoting a universal set is ∪ or ξ.

For example;

1. If A = {1, 2, 3} B = {2, 3, 4} C = {3, 5, 7}

then U = {1, 2, 3, 4, 5, 7}

[Here A ⊆ U, B ⊆ U, C ⊆ U and U ⊇ A, U ⊇ B, U ⊇ C]

2. If P is a set of all whole numbers and Q is a set of all negative numbers then the universal set
is a set of all integers.

3. If A = {a, b, c} B = {d, e} C = {f, g, h, i}

then U = {a, b, c, d, e, f, g, h, i} can be taken as universal set.

11
Operations on sets:

1. Definition of Union of Sets:

Union of two given sets is the smallest set which contains all the elements of both the sets.

To find the union of two given sets A and B is a set which consists of all the elements of A
and all the elements of B such that no element is repeated.

The symbol for denoting union of sets is ∪‘.

Some properties of the operation of union:

(i) A∪B = B∪A (Commutative law)

(ii) A∪(B∪C) = (A∪B)∪C (Associative law)

(iii) A ∪ Ф = A (Law of identity element, is the identity of ∪)

(iv) A∪A = A (Idempotent law)

(v) U∪A = U (Law of ∪) ∪ is the universal set.

Notes:

A ∪ Ф = Ф∪ A = A i.e. union of any set with the empty set is always the set itself.

Examples:

1. If A = {1, 3, 7, 5} and B = {3, 7, 8, 9}. Find union of two set A and B.

Solution:

A ∪ B = {1, 3, 5, 7, 8, 9}
No element is repeated in the union of two sets. The common elements 3, 7 are taken only once.

2. Let X = {a, e, i, o, u} and Y = {ф}. Find union of two given sets X and Y.

Solution:

X ∪ Y = {a, e, i, o, u}

Therefore, union of any set with an empty set is the set itself.

2. Definition of Intersection of Sets:

Intersection of two given sets is the largest set which contains all the elements that are
common to both the sets.

To find the intersection of two given sets A and B is a set which consists of all the elements
which are common to both A and B.

12
The symbol for denoting intersection of sets is ∩. Some properties of the operation of intersection

(i) A∩B = B∩A (Commutative law)

(ii) (A∩B)∩C = A∩ (B∩C) (Associative law)

(iii) Ф ∩ A = Ф (Law of Ф)

(iv) U∩A = A (Law of ∪)

(v) A∩A = A (Idempotent law)

(vi) A∩(B∪C) = (A∩B) ∪ (A∩C) (Distributive law) Here ∩ distributes over

∪ Also, A∪(B∩C) = (AUB) ∩ (AUC) (Distributive law) Here ∪ distributes

over ∩ Notes:

A ∩ Ф = Ф ∩ A = Ф i.e. intersection of any set with the empty set is always the empty

set. Solved examples :

1. If A = {2, 4, 6, 8, 10} and B = {1, 3, 8, 4, 6}. Find intersection of two set A and B.

Solution:

A ∩ B = {4, 6, 8}

Therefore, 4, 6 and 8 are the common elements in both the sets.

2. If X = {a, b, c} and Y = {ф}. Find intersection of two given sets X and Y.

Solution:

X∩Y={}

3. Difference of two sets

If A and B are two sets, then their difference is given by A - B or B - A.

• If A = {2, 3, 4} and B = {4, 5, 6}

A - B means elements of A which are not the elements of B.

i.e., in the above example A - B = {2, 3}

In general, B - A = {x : x ∈ B, and x ∉ A}

13
• If A and B are disjoint sets, then A – B = A and B – A =

B Solved examples to find the difference of two sets:

1. A = {1, 2, 3} and B = {4, 5, 6}.

Find the difference between the two sets:

(i) A and B

(ii) B and A

Solution:

The two sets are disjoint as they do not have any elements in

common. (i) A - B = {1, 2, 3} = A

(ii) B - A = {4, 5, 6} = B

2. Let A = {a, b, c, d, e, f} and B = {b, d, f, g}.

Find the difference between the two sets:

(i) A and B

(ii) B and A

Solution:

(i) A - B = {a, c, e}

Therefore, the elements a, c, e belong to A but not to B

(ii) B - A = {g)

Therefore, the element g belongs to B but not A.

4. Complement of a Set

In complement of a set if S be the universal set and A a subset of S then the complement of
A is the set of all elements of S which are not the elements of A.

Symbolically, we denote the complement of A with respect to S as A‘

14
Some properties of complement sets

(i) A ∪ A' = A' ∪ A = ∪ (Complement law)

(ii) (A ∩ B') = ϕ (Complement law) - The set and its complement are disjoint sets.

(iii) (A ∪ B) = A' ∩ B' (De Morgan ‘s law)

(iv) (A ∩ B)' = A' ∪ B' (De Morgan ‘s law)

(v) (A')' = A (Law of complementation)

(vi) Ф' = ∪ (Law of empty set - The complement of an empty set is a universal set.

(vii) ∪' = Ф and universal set) - The complement of a universal set is an empty set.

For Example; If S = {1, 2, 3, 4, 5, 6, 7}

A = {1, 3, 7} find A'.

Solution:

We observe that 2, 4, 5, 6 are the only elements of S which do not belong to A.

Therefore, A' = {2, 4, 5, 6}

Algebraic laws on sets:

1. Commutative Laws:

For any two finite sets A and B;

(i) A U B = B U A

(ii) A ∩ B = B ∩ A

2. Associative Laws:

For any three finite sets A, B and C;

(i) (A U B) U C = A U (B U C)

(ii) (A ∩ B) ∩ C = A ∩ (B ∩ C)

Thus, union and intersection are associative.

3. Idempotent Laws:

For any finite set A;

(i) A U A = A

15
(ii) A ∩ A = A

4. Distributive Laws:

For any three finite sets A, B and C;

(i) A U (B ∩ C) = (A U B) ∩ (A U C)

(ii) A ∩ (B U C) = (A ∩ B) U (A ∩ C)

Thus, union and intersection are distributive over intersection and union respectively.

5. De Morgan’s Laws:

For any two finite sets A and B;

(i) A – (B U C) = (A – B) ∩ (A – C)

(ii) A - (B ∩ C) = (A – B) U (A – C)

De Morgan‘s Laws can also we written as:

(i) (A U B)‘ = A' ∩ B'

(ii) (A ∩ B)‘ = A' U B'

More laws of algebra of sets:

6. For any two finite sets A and B;

(i) A – B = A ∩ B'

(ii) B – A = B ∩ A'

(iii) A – B = A ⇔ A ∩ B = ∅

(iv) (A – B) U B = A U B

(v) (A – B) ∩ B = ∅

(vi) (A – B) U (B – A) = (A U B) – (A ∩ B)

Definition of De Morgan’s law:

The complement of the union of two sets is equal to the intersection of their complements and
the complement of the intersection of two sets is equal to the union of their complements.
These are called De Morgan’s laws.

For any two finite sets A and B;

(i) (A U B)' = A' ∩ B' (which is a De Morgan's law of union).

(ii) (A ∩ B)' = A' U B' (which is a De Morgan's law of intersection).

16
Venn Diagrams:

Pictorial representations of sets represented by closed figures are called set diagrams or Venn
diagrams.

Venn diagrams are used to illustrate various operations like union, intersection and

difference. We can express the relationship among sets through this in a more significant

way.

In this,

• A rectangle is used to represent a universal set.

• Circles or ovals are used to represent other subsets of the universal set.

Venn diagrams in different situations

In these diagrams, the universal set is represented by a rectangular region and its subsets by
circles inside the rectangle. We represented disjoint set by disjoint circles and intersecting sets
by intersecting circles.

S.No Set &Its relation Venn

Diagram

1 Intersection of A and B

Union of A and B

3 Difference: A-B

17
4 Difference: B-A

5 Complement of set A

6 A ∪ B when A ⊂ B

A ∪ B when neither A ⊂ B
nor B ⊂ A

8 A ∪ B when A and B are

disjoint sets

9 (A ∪ B)’ (A union B dash)

10 (A ∩ B)’ (A intersection B
dash)

18
11 B’ (B dash)

12 (A - B)’ (Dash of sets A

minus B)

13 (A ⊂ B)’ (Dash of A subset

B)

Problems of set theory:

1. Let A and B be two finite sets such that n(A) = 20, n(B) = 28 and n(A ∪ B) = 36, find n(A ∩ B).

Solution:

Using the formula n(A ∪ B) = n(A) + n(B) - n(A ∩ B).

then n(A ∩ B) = n(A) + n(B) - n(A ∪ B)

= 20 + 28 - 36

= 48 - 36

= 12

2. If n(A - B) = 18, n(A ∪ B) = 70 and n(A ∩ B) = 25, then find n(B).

Solution:

Using the formula n(A∪B) = n(A - B) + n(A ∩ B) + n(B -

A) 70 = 18 + 25 + n(B - A)

70 = 43 + n(B - A)

n(B - A) = 70 - 43

n(B - A) = 27
19
Now n(B) = n(A ∩ B) + n(B - A)

= 25 + 27

= 52

3. In a group of 100 persons, 72 people can speak English and 43 can speak French. How
many can speak English only? How many can speak French only and how many can speak
both English and French?

Solution:

Let A be the set of people who speak

English. B be the set of people who speak

French.

A - B be the set of people who speak English and not French.

B - A be the set of people who speak French and not English.

A ∩ B be the set of people who speak both French and

English. Given,

n(A) = 72 n(B) = 43 n(A ∪ B) =

100 Now, n(A ∩ B) = n(A) + n(B) - n(A ∪

B)

= 72 + 43 - 100

= 115 - 100

= 15

Therefore, Number of persons who speak both French and English =

15 n(A) = n(A - B) + n(A ∩ B)

⇒ n(A - B) = n(A) - n(A ∩ B)

= 72 - 15

= 57

and n(B - A) = n(B) - n(A ∩ B)

= 43 - 15

20
 = 28

Therefore, Number of people speaking English only = 57

Number of people speaking French only = 28

Probability Concepts

Before we give a definition of probability, let us examine the following concepts:

Experiment:

In probability theory, an experiment or trial (see below) is any procedure that can be infinitely
repeated and has a well-defined set of possible outcomes, known as the sample space. An
experiment is said to be random if it has more than one possible outcome, and deterministic if
it has only one. A random experiment that has exactly two (mutually exclusive) possible outcomes
is known as a Bernoulli trial.

Random Experiment:

An experiment is a random experiment if its outcome cannot be predicted precisely. One out of a
number of outcomes is possible in a random experiment. A single performance of the random
experiment is called a trial.
Random experiments are often conducted repeatedly, so that the collective results may be
subjected to statistical analysis. A fixed number of repetitions of the same experiment can be
thought of as a composed experiment, in which case the individual repetitions are called trials.
For example, if one were to toss the same coin one hundred times and record each result, each toss
would be considered a trial within the experiment composed of all hundred tosses.

Mathematical description of an experiment:

A random experiment is described or modeled by a mathematical construct known as a
probability space. A probability space is constructed and defined with a specific kind of
experiment or trial in mind.
A mathematical description of an experiment consists of three parts:

1. A sample space, Ω (or S), which is the set of all possible outcomes.
2. A set of events, where each event is a set containing zero or more outcomes.
3. The assignment of probabilities to the events—that is, a function P mapping from events
to probabilities.
An outcome is the result of a single execution of the model. Since individual outcomes might be
of little practical use, more complicated events are used to characterize groups of outcomes. The
collection of all such events is a sigma-algebra. Finally, there is a need to specify each event's
likelihood of happening; this is done using the probability measure function,P.

21
Sample Space: The sample space is the collection of all possible outcomes of random experiment.
The elements of are called sample points.

 A sample space may be finite, countably infinite or uncountable.

 A finite or countably infinite sample space is called a discrete sample space.

 An uncountable sample space is called a continuous sample space

Ex:1. For the coin-toss experiment would be the results ―Head and ―Tail which we may
represent by S={H T}.

Ex. 2. If we toss a die, one sample space or the set of all possible outcomes is S = { 1, 2, 3, 4, 5, 6}

The other sample space can be S = {odd, even}

Types of Sample Space:

1. Finite/Discrete Sample Space:

Consider the experiment of tossing a coin twice. The sample space can be

S = {HH, HT, T H , TT} the above sample space has a finite number of sample points.
It is called a finite sample space.

2. Countably Infinite Sample Space:

Consider that a light bulb is manufactured. It is then tested for its life length by inserting it into a
socket and the time elapsed (in hours) until it burns out is recorded. Let the measuring instrument is
capable of recording time to two decimal places, for example 8.32 hours.

Now, the sample space becomes count ably infinite i.e.

S = {0.0, 0.01, 0.02}

The above sample space is called a countable infinite sample space.

3. Un Countable/ Infinite Sample Space:

If the sample space consists of unaccountably infinite number of elements, then it is called Un

Countable/ Infinite Sample Space.

Event: An event is simply a set of possible outcomes. To be more specific, an event is a subset
A of the sample space S.


 For a discrete sample space, all subsets are events.

Ex: For instance, in the coin-toss experiment the events A={Heads} and B={Tails} would be
mutually exclusive.

22
An event consisting of a single point of the sample space 'S' is called a simple event or
elementary event.

Some examples of event sets:

Example 1: tossing a fair coin

The possible outcomes are H (head) and T (tail). The associated sample space is It
is a finite sample space. The events associated with the sample space are: and
.

Example 2: Throwing a fair die:

The possible 6 outcomes are:

. . .
The associated finite sample space is .Some events are

And so on.

Example 3: Tossing a fair coin until a head is obtained

We may have to toss the coin any number of times before a head is obtained. Thus the possible
outcomes are:
H, TH, TTH, TTTH,
How many outcomes are there? The outcomes are countable but infinite in number. The
countably infinite sample space is .

Example 5: Drawing cards

Drawing 4 cards from a deck: Events include all spades, sum of the 4 cards is (assuming face
cards have a value of zero), a sequence of integers, a hand with a 2, 3, 4 and 5. There are many
more events.

Types of Events:

1. Exhaustive Events:

A set of events is said to be exhaustive, if it includes all the possible events.

Ex. In tossing a coin, the outcome can be either Head or Tail and there is no other possible
outcome. So, the set of events { H , T } is exhaustive.

2. Mutually Exclusive Events:

Two events, A and B are said to be mutually exclusive if they cannot occur together.

23
i.e. if the occurrence of one of the events precludes the occurrence of all others, then such a set
of events is said to be mutually exclusive.

If two events are mutually exclusive then the probability of either occurring is

Ex. In tossing a die, both head and tail cannot happen at the same time.

3.Equally Likely Events:

If one of the events cannot be expected to happen in preference to another, then such
events are said to be Equally Likely Events. (Or) Each outcome of the random
experiment has an equal chance of occurring.
Ex. In tossing a coin, the coming of the head or the tail is equally likely.

4. Independent Events:

Two events are said to be independent, if happening or failure of one does not affect the
happening or failure of the other. Otherwise, the events are said to be dependent.

If two events, A and B are independent then the joint probability is

5.Non-. Mutually Exclusive Events:

If the events are not mutually exclusive then

Probability Definitions and Axioms:

1. Relative frequency Definition:

Consider that an experiment E is repeated n times, and let A and B be two events
associated w i t h E. Let nA and nB be the number of times that the event A and the event
B occurred among the n repetitions respectively.

The relative frequency of the event A in the 'n' repetitions of E is defined

as f( A) = nA /n f(A)=nA/n

The Relative frequency has the following properties:

1.0 ≤f(A) ≤ 1

2. f(A) =1 if and only if A occurs every time among the n repetitions.

24
 If an experiment is repeated times under similar conditions and the event occurs
in times, then the probability of the event A is defined as

Limitation:
Since we can never repeat an experiment or process indefinitely, we can never know the
probability of any event from the relative frequency definition. In many cases we can't even
obtain a long series of repetitions due to time, cost, or other limitations. For example, the
probability of rain today can't really be obtained by the relative frequency definition since today
can‘t be repeated again.

2.. The classical definition:

Let the sample space (denoted by ) be the set of all possible distinct outcomes to
an experiment. The probability of some event is

provided that all points in S are equally likely.

Limitation:

What does "equally likely" mean? This appears to use the concept of probability while trying to
define it! We could remove the phrase "provided all outcomes are equally likely", but then the
definition would clearly be unusable in many settings where the outcomes in S did not tend
to occur equally often.

Example1:A fair die is rolled once. What is the probability of getting a ‗6‘ ?

Here and

Example2:A fair coin is tossed twice. What is the probability of getting two ‗heads'?

Here and .
Total number of outcomes is 4 and all four outcomes are equally likely.

Only outcome favourable to is {HH}

25
Probability axioms:

Axiom1: The probability of any event A is positive or zero. Namely P(A)≥0.

Axiom2: The probability of the sure event is 1. Namely P(Ω)=1. And so, the probability is
always greater than 0 and smaller than 1: probability zero means that there is no possibility for it
to happen (it is an impossible event), and probability 1 means that it will always happen (it is a
sure event).i.e .

Axiom3: The probability of the union of any set of two by two incompatible events is the sum
of the probabilities of the events. That is, if we have, for example, events A,B,C, and these are
two by two incompatible, then P(A∪B∪C)=P(A)+P(B)+P(C). i.e Additivity:

, where and are mutually exclusive.

for , 2, ..., where , , ... are mutually exclusive

(i.e., ).

Main properties of probability: If A is any event of sample space S then

1. P(A)+P(𝐴̅)=1
2. The probability of the impossible event is 0, i.e P(Ø)=0
2. If A⊂B, then P(A)≤P(B).
3. If A and B are two incompatible events, and therefore,
P(A−B)=P(A)−P(A∩B).and P(B−A)=P(B)−P(A∩B).
4. Addition Law of probability:
P(AUB)=P(A)+P(B)-P(A∩B)

PERMUTATIONS and COMBINATIONS:

S.No. PERMUTATIONS COMBINATIONS:

1 Arrangement of things in a In permutations, the order of arrangement of
specified order is called objects is important. But, in combinations,
permutation. Here all things are order is not important, but only selection of
taken at a time objects.
2 Arrangement of ‗r‘ things taken at
a time from ‗n‘ things ,where r < n
in a
specified order in called r-permutation.
3 Consider the letters a,b and c .
Considering all the three letters at a
time, the possible permutations are
ABC ,a c b , b c a , b a c , c b a and c a
b
.

26
 4 The number of permutations taking r The number of combinations taking r things at
things at a time from ‗n‘ available a time from ‗n‘ available things is denoted as
things is denoted as p ( n , r ) or n pr C( n , r ) or n Cr

5 nPr= r!/nCr= n!/(n-r)! nCr=P(n,r)/r!=n!/r!(n-r)!

Conditional probability

The conditional probability of B given A denoted by . We shall develop the

concept of the conditional probability and explain under what condition this
conditional probability is same as .

Let us consider the case of equiprobable events discussed earlier. Let sample points be
favourable for the joint event A∩B

This concept suggests us to define conditional probability. The probability of an event B under the
condition that another event A has occurred is called the conditional probability of B given A and
defined by

We can similarly define the conditional probability of A given B , denoted by .

27
 From the definition of conditional probability, we have the joint probability of two
events A and B as follows

Problems:

Example 1 Consider the example tossing the fair die. Suppose

Example 2 A family has two children. It is known that at least one of the children is a girl.
What is the probability that both the children are girls?

A = event of at least one girl

B = event of two girls

Properties of Conditional probability:

1. If , then

We have,

28
 2. We have,

3. Chain Rule of Probability/Multiplication theorem:

We have,

We can generalize the above to get the chain rule of probability for n events as

Joint probability

Joint probability is defined as the probability of both A and B taking place, and is
denoted by P (AB) or P(A∩B)

Joint probability is not the same as conditional probability, though the two concepts are
often confused. Conditional probability assumes that one event has taken place or will take
place, and then asks for the probability of the other (A, given B). Joint probability does not have
such conditions; it simply asks for the chances of both happening (A and B). In a problem, to
help distinguish between the two, look for qualifiers that one event is conditional on the other
(conditional) or whether they will happen concurrently (joint).

Probability definitions can find their way into CFA exam questions. Naturally, there may
also be questions that test the ability to calculate joint probabilities. Such computations require
use of the multiplication rule, which states that the joint probability of A and B is the product
of the conditional probability of A given B, times the probability of B. In probability notation:

P(AB) = P(A | B) * P(B)

Given a conditional probability P(A | B) = 40%, and a probability of B = 60%, the joint
probability P(AB) = 0.6*0.4 or 24%, found by applying the multiplication rule.

P(AUB)=P(A)+P(B)-P(A‫ח‬B)

For independent events: P(AB) = P(A) * P(B)

Moreover, the rule generalizes for more than two events provided they are all independent of one
another, so the joint probability of three events P(ABC) = P(A) * (P(B) *P(C), again assuming
independence.

29
 Total Probability theorem:

Let be n events such that

Then for any event B,

Proof: We have

Remark

(1) A decomposition of a set S into 2 or more disjoint nonempty subsets is called a partition of
S.The subsets form a partition of S if
30
 (2) The theorem of total probability can be used to determine the probability of a complex
event in terms of related simpler events. This result will be used in Bays' theorem to be
discussed to the end of the lecture.

Bayes' Theorem:

This result is known as the Baye's theorem. The probability is called the a priori
probability and is called the a posteriori probability. Thus the Bays' theorem enables us
to determine the a posteriori probability from the observation that B has occurred. This
result is of practical importance and is the heart of Baysean classification, Baysean estimation
etc.

Example1:
In a binary communication system a zero and a one is transmitted with probability 0.6 and 0.4
respectively. Due to error in the communication system a zero becomes a one with a
probability
0.1 and a one becomes a zero with a probability 0.08. Determine the probability (i) of
receiving a one and (ii) that a one was transmitted when the received message is one
Solution:
Let S is the sample space corresponding to binary communication. Suppose be event of
Transmitting 0 and be the event of transmitting 1 and and be corresponding events
of receiving 0 and 1 respectively.

31
 Given and

Example 7: In an electronics laboratory, there are identically looking capacitors of three makes
in the ratio 2:3:4. It is known that 1% of , 1.5% of are defective.
What percentages of capacitors in the laboratory are defective? If a capacitor picked at defective
is found to be defective, what is the probability it is of make ?

Let D be the event that the item is defective. Here we have to find .

32
 Independent events

Two events are called independent if the probability of occurrence of one event does
not affect the probability of occurrence of the other. Thus the events A and B are independent
if

and

where and are assumed to

be non-zero.

Equivalently if A and B are independent, we have

or --------------------

Two events A and B are called statistically dependent if they are not independent. Similarly, we
can define the independence of n events. The events are called independent if
and only if

Example: Consider the example of tossing a fair coin twice. The resulting sample space
is given by and all the outcomes are equiprobable.

Let be the event of getting ‗tail' in the first toss and be the
event of getting ‗head' in the second toss. Then

and

Again, so that

Hence the events A and B are independent.

33
Problems:

Example1: Roll a red die and a green die. Find the probability the total is
5. Solution: Let represent getting on the red die and on the green die.

Then, with these as simple events, the sample space is

The sample points giving a total of 5 are (1,4) (2,3) (3,2), and (4,1).
(total is 5) =

Example2: Draw 1 card from a standard well-shuffled deck (13 cards of each of 4 suits -
spades, hearts, diamonds, and clubs). Find the probability the card is a club.
Solution 1: Let = { spade, heart, diamond, club}. (The points of are generally listed between

brackets {}.) Then has 4 points, with 1 of them being "club", so (club) = .
Solution 2: Let = {each of the 52 cards}. Then 13 of the 52 cards are clubs, so

Example 3: Suppose we draw a card from a deck of playing cards. What is the
probability that we draw a spade?

Solution: The sample space of this experiment consists of 52 cards, and the probability of
each sample point is 1/52. Since there are 13 spades in the deck, the probability of drawing a
spade is

P(Spade) = (13)(1/52) = 1/4

Example 4: Suppose a coin is flipped 3 times. What is the probability of getting two tails
and one head?

Solution: For this experiment, the sample space consists of 8 sample points.

S = {TTT, TTH, THT, THH, HTT, HTH, HHT, HHH}

Each sample point is equally likely to occur, so the probability of getting any particular sample
point is 1/8. The event "getting two tails and one head" consists of the following subset of the
sample space.
34
 A = {TTH, THT, HTT}

The probability of Event A is the sum of the probabilities of the sample points in A. Therefore,

P(A) = 1/8 + 1/8 + 1/8 = 3/8

Example5: An urn contains 6 red marbles and 4 black marbles. Two marbles are
drawn without replacement from the urn. What is the probability that both of the marbles are
black?

Solution: Let A = the event that the first marble is black; and let B = the event that the second
marble is black. We know the following:

 In the beginning, there are 10 marbles in the urn, 4 of which are black. Therefore, P(A) =
4/10.
 After the first selection, there are 9 marbles in the urn, 3 of which are black. Therefore,
P(B|A) = 3/9.

Therefore, based on the rule of multiplication:

P(A ∩ B) = P(A) P(B|A)

P(A ∩ B) = (4/10) * (3/9) = 12/90 = 2/15

RANDOM VARIABLE

INTRODUCTION
In many situations, we are interested in numbers associated with the outcomes of a random
experiment. In application of probabilities, we are often concerned with numerical values which
are random in nature. For example, we may consider the number of customers arriving at a
service station at a particular interval of time or the transmission time of a message in a
communication system. These random quantities may be considered as real-valued function on
the sample space. Such a real-valued function is called real random variable and plays an
important role in describing random data. We shall introduce the concept of random variables in
the following sections.

Random Variable Definition

A random variable is a function that maps outcomes of a random experiment to real

numbers. (or)
A random variable associates the points in the sample space with real numbers

A (real-valued) random variable, often denoted by X(or some other capital letter), is a function
mapping a probability space (S; P) into the real line R. This is shown in Figure 1.Associated with
each point s in the domain S the function X assigns one and only one value X(s) in the range R.
(The set of possible values of X(s) is usually a proper subset of the real line; i.e., not all real
numbers need occur. If S is a finite set with m elements, then X(s) can assume at most an m
different value as s varies in S.)

35
Example1
A fair coin is tossed 6 times. The number of heads that come up is an example of a random
variable.
HHTTHT – 3, THHTTT -- 2.
These random variables can only take values between 0 and 6.
The Set of possible values of random variables is known as its
Range. Example2
A box of 6 eggs is rejected once it contains one or more broken eggs. If we examine 10 boxes
of eggs and define the randomvariablesX1, X2 as
1 X1- the number of broken eggs in the 10
boxes 2 X2- the number of boxes rejected
Then the range of X1 is {0, 1,2,3,4-------------- 60} and X2 is {0,1,2 --- 10}
Figure 2: A (real-valued) function of a random variable is itself a random variable, i.e., a
function mapping a probability space into the real line.

Example 3 Consider the example of tossing a fair coin twice. The sample space is S={
HH,HT,TH,TT} and all four outcomes are equally likely. Then we can define a random
variable as follows

Here .

Example 4 Consider the sample space associated with the single toss of a fair die. The
sample space is given by .

If we define the random variable that associates a real number equal to the number
on the face of the die, then .

36
Types of random variables:

There are two types of random variables, discrete and continuous.

1. Discrete random variable:

Continuous Random Variable: A continuous random variable is one having a continuous range of values
within one or more intervals; it is referred to as continuous random variable.
Ex: Measuring the height & weight of the person.
Mixed Random Variable: A mixed random variable is one for which some of its values are discrete and
some of its values are continuous.

Conditions for a Function to be a Random Variable:

37
 UNIT II
Distribution and density functions and Operations on One Random Variable

Distribution and density functions:

 Distribution function and its properties

 Density function and its properties
 Important Types of Distribution and density functions
 Binomial
 Uniform
 Exponential
 Gaussian
 Conditional Distribution & Conditional Density function and its properties
 Problems

Operation on One Random Variable:

 Expected value of a random variable, function of a random variable

 Moments about the origin
 Central moments - variance and skew
 Characteristic function
 Moment generating function
 Probability Distribution

The probability distribution of a discrete random variable is a list of probabilities associated with
each of its possible values. It is also sometimes called the probability function or the probability
mass function.

More formally, the probability distribution of a discrete random variable X is a function which
gives the probability p(xi) that the random variable equals xi, for each value xi:
p(xi) = P(X=xi)

It satisfies the following conditions:

a.
b.

Cumulative Distribution Function

All random variables (discrete and continuous) have a cumulative distribution function. It is a
function giving the probability that the random variable X is less than or equal to x, for every
value x.

Formally, the cumulative distribution function F(x) is defined to be:

for

For a discrete random variable, the cumulative distribution function is found by summing up the
probabilities as in the example below.

For a continuous random variable, the cumulative distribution function is the integral of its
probability density function.

Example
Discrete case : Suppose a random variable X has the following probability distribution p(xi):
xi 0 1 2 3 4 5
p(xi) 1/32 5/32 10/32 10/32 5/32 1/32
This is actually a binomial distribution: Bi(5, 0.5) or B(5, 0.5). The cumulative distribution
function F(x) is then:
xi 0 1 2 3 4 5

F(xi) 1/32 6/32 16/32 26/32 31/32 32/32

Probability Distribution Function

The probability is called the probability

distribution function ( also called the cumulative distribution function , abbreviated as
CDF ) of and denoted by . Thus
Properties of the probabilityDistribution Function

1.

2. is a non-decreasing function of . Thus, if

Is right continuous.

3.

.
4.
.
5.

We have,

6.

Example: Consider the random variable in the above example. We have

Thus we have seen that given , we can determine the probability of any
event involving values of the random variable .Thus is a complete
description of the random variable .
 Example 5 Consider the random variable defined by

Find a) .

b) .

c) .
d) .

Solutio

Probability Density Function

The probability density function of a continuous random variable is a function which can be
integrated to obtain the probability that the random variable takes a value in a given interval.

More formally, the probability density function, f(x), of a continuous random variable X is the
derivative of the cumulative distribution function F(x):

Since it follows that:

If f(x) is a probability density function then it must obey two conditions:

a. that the total probability for all possible values of the continuous random variable X is 1:

b. that the probability density function can never be negative: f(x) > 0 for all x.
Example 1

Consider the random variable with the distribution function

The plot of the is shown in Figure 7 on next page.

The probability mass function of the random variable is given by

 Value of the random
pX(x)
variable X =x
0

Properties of the Probability Density Function

1. .------- This follows from the fact that is a non-decreasing function

2.

3.

4.

Other Distribution and density functions of Random variable:

1. Bernoulli random variable:

Suppose X is a random variable that takes two values 0 and 1, with probability mass
functions

And

Such a random variable X is called a Bernoulli random variable, because it describes the
outcomes of a Bernoulli trial.

The typical CDF of the Bernoulli RV is as shown in Figure 2

Mean and variance of the Bernoulli random:

Remark

 The Bernoulli RV is the simplest discrete RV. It can be used as the building block for
many discrete RVs.

2. Binomial Distribution and density functions

Suppose X is a discrete random variable taking values from the set . is called
a binomial random variable with parameters n and if

where

The trials must meet the following requirements:

a. the total number of trials is fixed in advance;

b. there are just two outcomes of each trial; success and failure;
c. the outcomes of all the trials are statistically independent;
d. all the trials have the same probability of success.

As we have seen, the probability of k successes in n independent repetitions of the Bernoulli

trial is given by the binomial law. If X is a discrete random variable representing the number of
successes in this case, then X is a binomial random variable. For example, the number of heads
in ‗n
' independent tossing of a fair coin is a binomial random variable.

 The notation is used to represent a binomial RV with the parameters and

 .


 The sum of n independent identically distributed Bernoulli random variables is a
binomial random variable.
 The binomial distribution is useful when there are two types of objects - good, bad;
correct, erroneous; healthy, diseased etc

Example1:In a binary communication system, the probability of bit error is 0.01. If a block of 8 bits
are transmitted, find the probability that

(a) Exactly 2 bit errors will occur

(b) At least 2 bit errors will occur

(c) More than 2 bit errors will occur

(d) All the bits will be erroneous

Suppose is the random variable representing the number of bit errors in a block of 8 bits.
Then

Therefore,
 The probability mass function for a binomial random variable with n = 6 and p =0.8

Mean and Variance of the Binomial Random Variable

Uniform Distribution and density functions:

A continuous random variable X is called uniformly distributed over the interval [a, b],

, if its probability density function is given by

We use the notation to denote a random variable X uniformly distributed over the
interval [a,b]

Distribution function

Figure 2 illustrates the CDF of a uniform random variable.

Figure 2: CDF of a uniform random variable

Mean and Variance of a Uniform Random Variable:

 3. Normal or Gaussian Distribution and density functions

The normal distribution is the most important distribution used to model natural and man made
phenomena. Particularly, when the random variable is the result of the addition of large number of
independent random variables, it can be modeled as a normal random variable.

continuous random variable X is called a normal or a Gaussian random variable with

parameters and if its probability density function is given by,

Where and are real numbers.

We write that X is distributed.

If and , and the random variable X is called the standard normal variable.

Figure 3 illustrates two normal variables with the same mean but different variances.
Figure 3
  Is a bell-shaped function, symmetrical about .
 Determines the spread of the random variable X . If is small X is more
concentrated around the mean .

 Distribution function of a Gaussian variable is

Substituting , we get

where is the distribution function of the standard normal variable.

Thus can be computed from tabulated values of . The table was very useful in the
pre-computer days.

In communication engineering, it is customary to work with the Q function defined by,

Note that and

These results follow from the symmetry of the Gaussian pdf. The function is tabulated and the
tabulated results are used to compute probability involving the Gaussian random variable.
Using the Error Function to compute Probabilities for Gaussian Random Variables

The function is closely related to the error function and the complementary error
function .

Note that,

And the complementary error function is given

Mean and Variance of a Gaussian Random Variable

If X is distributed, then

Proof:
Exponential Random Variable

A continuous random variable is called exponentially distributed with the parameter

if the probability density function is of the

PDF of Exponential Random Variable is
Example 1

Suppose the waiting time of packets in in a computer network is an exponential RV with

Conditional Distribution and Density functions

We discussed conditional probability in an earlier lecture. For two events A and B

with , the conditional probability was defined as

Clearly, the conditional probability can be defined on events involving a Random Variable X

Conditional distribution function:

Consider the event and any event B involving the random variable X . The
conditional distribution function of X given B is defined as

We can verify that satisfies all the properties of the distribution function.
Particularly.

 And .
 .
 Is a non-decreasing function of .


Conditional Probability Density Function

In a similar manner, we can define the conditional density function of the

random variable X given the event B as

All the properties of the pdf applies to the conditional pdf and we can easily show that




OPERATION ON RANDOM VARIABLE-EXPECTATIONS

Expected Value of a Random Variable

 The expectation operation extracts a few parameters of a random variable and provides
a summary description of the random variable in terms of these parameters.
 It is far easier to estimate these parameters from data than to estimate the distribution or
density function of the random variable.
 Moments are some important parameters obtained through the expectation operation.

Expected value or mean of a random variable

 Is also called the mean or statistical average of the random variable and is denoted by

Note that, for a discrete RV with the probability mass function (pmf) the
pdf is given by

Thus for a discrete random variable with

Figure1 Mean of a random variable

Example 1

Suppose is a random variable defined by the pdf

 Then

Example 2

Consider the random variable with the pmf as tabulated below

Value of the random

0 1 2 3
variable x
pX(x)

Then

Example 3 Let X be a continuous random variable with

Then

=
 =
Hence EX does not exist. This density function is known as the Cauchy density function.

Expected value of a function of a random variable

Suppose is a real-valued function of a random variable as discussed in the last

class.
Then,

We shall illustrate the above result in the special case when is one-to-one and
monotonically increasing function of x In this case,

Figure 2
The following important properties of the expectation operation can be immediately
derived:

(a) If is a constant,

Clearly

(b) If are two functions of the random variable and are

constants,

The above property means that is a linear operator.

MOMENTS ABOUT THE ORIGIN:

Mean-square value
MOMENTS ABOUT THE MEAN

Variance

Second central moment is called as variance

For a random variable with the pdf and mean the variance of is denoted by
and

defined as

Thus for a discrete random variable with

The standard deviation of is defined as

Example 4

Find the variance of the random variable in the above example

Example 5

Find the variance of the random variable discussed in above example. As already computed
 For example, consider two random variables with pmf as shown below. Note that

each of has zero mean.The variances are given by and implying that
has more spread about the mean.

Properties of variance

(1)

(2) If then

(3) If is a constant,

nth moment of a random variable

We can define the nth moment and the nth central- moment of a random variable X by the
following relations
 Note that

 The mean is the first moment and the mean-square value is the
second moment
 The first central moment is 0 and the variance is the second
central moment

SKEWNESS

 The third central moment measures lack of symmetry of the pdf of a random variable

is called the coefficient of skewness and if the pdf is symmetric

this coefficient will be zero.
 The fourth central moment measures flatness or peakedness of the pdf of a random

variable. Is called kurtosis. If the peak of the pdf is sharper, then the
random variable has a higher kurtosis.

Characteristic function

Consider a random variable with probability density function The characteristic

function of denoted by is defined as

Note the following:

 is a complex quantity, representing the Fourier transform of and

traditionally using instead of This implies that the properties of the Fourier
transform applies to the characteristic function.

We can get
from by the inverse transform
Example 1

Consider the random variable X with pdf given by

= 0 otherwise. The characteristics function is given by

Solution:

Example 2

The characteristic function of the random variable with

Characteristic function of a discrete random variable

Suppose X is a random variable taking values from the discrete set with
corresponding probability mass function for the value

Then,
If Rx is the set of integers, we can write

In this case can be interpreted as the discrete-time Fourier transform with

substituting in the original discrete-time Fourier transform. The inverse relation is

Moments and the characteristic function

Given the characteristics function the nth moment is given by

To prove this consider the power series expansion of

Taking expectation of both sides and assuming to exist, we get

Taking the first derivative of with respect to at we get

Similarly, taking the derivative of with respect to at we get

Thus ,
 UNIT III
Multiple Random Variables and Operations on Multiple Random Variables

Multiple Random Variables:

 Vector Random Variables

 Joint Distribution Function and Properties
 Joint density Function and Properties
 Marginal Distribution and density Functions
 Conditional Distribution and density Functions
 Statistical Independence
 Distribution and density functions of Sum of Two Random Variables and Sum of Several
Random Variables

Operations on Multiple Random Variables:

 Expected Value of a Function of Random Variables

 Joint Moments about the Origin
 Joint Central Moments
 Joint Characteristic Functions
 UNIT-3
MULTIPLE RANDOM VARIABLES

Multiple Random Variables

In many applications we have to deal with more than two random variables. For example,
in the navigation problem, the position of a space craft is represented by three random variables
denoting the x, y and z coordinates. The noise affecting the R, G, B channels of colour video
may be represented by three random variables. In such situations, it is convenient to define the
vector-valued random variables where each component of the vector is a random variable.

In this lecture, we extend the concepts of joint random variables to the case of multiple
random variables. A generalized analysis will be presented for random variables defined on the
same sample space.

Jointly Distributed Random Variables

We may define two or more random variables on the same sample space. Let and be
two real random variables defined on the same probability space The mapping
such that for is called a joint random variable.

Figure 1

Joint Probability Distribution Function

Recall the definition of the distribution of a single random variable. The event was
used to define the probability distribution function . Given , we can find
the
probability of any event involving the random variable. Similarly, for two random variables
and , the event is considered as the representative event.
The probability is called the joint distribution function or the
joint cumulative distribution function (CDF) of the random variables and and denoted by

Figure 2

Properties of JPDF

satisfies the following properties:

1)

2)

3)

Note that

4)

5) is right continuous in both the variables.

6)
 Given ,we have a complete description
of the random variables and .

7)

To prove this

Similarly .

Given , each of is called a marginal

Distribution function or marginal cumulative distribution function (CDF).

Jointly Distributed Discrete Random Variables

If and are two discrete random variables defined on the same probability space
such that takes values from the countable subset and takes values from the
countable subset .Then the joint random variable can take values from the countable
subset in . The joint random variable is completely specified by their joint
probability mass function

Given , we can determine other probabilities involving the random variables and

Remark

•
 This is because

• Marginal Probability Mass Functions: The probability mass functions and

are obtained from the joint probability mass function as follows

and similarly

These probability mass functions and obtained from the joint probability mass
functions are called marginal probability mass functions .

Example 4 Consider the random variables and with the joint probability mass function as
tabulated in Table 1. The marginal probabilities and are as shown in the last column
and the last row respectively.

Table 1

Joint Probability Density Function

If and are two continuous random variables and their joint distribution function is
continuous in both and , then we can define joint probability density function by

provided it exists.
 Clearly

Properties of Joint Probability Density Function

• is always a non-negative quantity. That is,

• The probability of any Borel set can be obtained by

Marginal density functions

The marginal density functions and of two joint RVs and are given
by the derivatives of the corresponding marginal distribution functions. Thus

Example 5 The joint density function of the random variables in Example 3 is

Example 6 The joint pdf of two random variables and are given by

• Find .
• Find .
• Find and .
• What is the probability ?
Conditional Distributions

We discussed the conditional CDF and conditional PDF of a random variable conditioned on
some events defined in terms of the same random variable. We observed that

and

We can define these quantities for two random variables. We start with the conditional
probability mass functions for two random variables.

Conditional Probability Density Functions

Suppose and are two discrete jointly random variable with the joint PMF The
conditional PMF of given is denoted by and defined as

Similarly we can define the conditional probability mass function

 Conditional Probability Distribution Function

Consider two continuous jointly random variables and with the joint probability
distribution function We are interested to find the conditional distribution function of
one of the random variables on the condition of a particular value of the other random variable.

We cannot define the conditional distribution function of the random variable on the
condition of the event by the relation

as in the above expression. The conditional distribution function is defined in the

limiting sense as follows:

Conditional Probability Density Function

is called the conditional probability density function of

given

Let us define the conditional distribution function .

The conditional density is defined in the limiting sense as follows

 Because,

The right hand side of the highlighted equation is

Similarly we have

Two random variables are statistically independent if for all

Example 2 X and Y are two jointly random variables with the joint pdf given by

find,

(a)
(b)
(c)

Solution:
Since

We get

Independent Random Variables (or) Statistical Independence

Let and be two random variables characterized by the joint distribution function

and the corresponding joint density function

Then and are independent if and are independent events.
Thus,
 and equivalently

Sum of Two Random Variables

We are often interested in finding out the probability density function of a function of two or
more RVs. Following are a few examples.

• The received signal by a communication receiver is given by

where is received signal which is the superposition of the message signal and the noise .

• The frequently applied operations on communication signals like modulation,

demodulation, correlation etc. involve multiplication of two signals in the form Z = XY.

We have to know about the probability distribution of in any analysis of . More formally,
given two random variables X and Y with joint probability density function and a
function we have to find .

In this lecture, we shall address this problem.

Probability Density of the Function of Two Random Variables

 We consider the transformation

Consider the event corresponding to each z. We can find a variable subset

such that .

Figure 1

Probability density function of Z = X + Y .

Consider Figure 2
Figure 2

We have

Therefore, is the colored region in the Figure 2.

If X and Y are independent

Where * is the convolution operation.

Example 1

Suppose X and Y are independent random variables and each uniformly distributed over (a, b).
And are as shown in the figure below.

The PDF of is a triangular probability density function as shown in the figure.

Central Limit Theorem

Consider independent random variables .The mean and variance of
each of the random variables are assumed to be known. Suppose and

. Form a random variable

The mean and variance of are given by

 and
Thus we can determine the mean and the variance of .
Can we guess about the probability distribution of ?
The central limit theorem (CLT) provides an answer to this question.

The CLT states that under very general conditions converges in distribution to
as . The conditions are:

1. The random variables are independent and identically distributed.

2. The random variables are independent with same mean and variance,
but not identically distributed.
3. The random variables are independent with different means and
same variance and not identically distributed.
4. The random variables are independent with different means and
each variance being neither too small nor too large.

We shall consider the first condition only. In this case, the central-limit theorem can be stated as
follows:

Proof of the Central Limit Theorem:

We give a less rigorous proof of the theorem with the help of the characteristic function.
Further we consider each of to have zero mean. Thus,

Clearly,
The characteristic function of is given by
We will show that as the characteristic function is of the form of the characteristic
function of a Gaussian random variable.
Expanding in power series

Assume all the moments of to be finite. Then

Substituting

where is the average of terms involving and higher powers of .

Note also that each term in involves a ratio of a higher moment and a power of and
therefore,

which is the characteristic function of a Gaussian random variable with 0 mean and variance
.

OPERATIONS ON MULTIPLE RANDOM VARIABLES

Expected Values of Functions of Random Variables

If is a function of a continuous random variable then

 If is a function of a discrete random variable then

Suppose is a function of continuous random variables then the

expected value of is given by

Thus can be computed without explicitly determining .

We can establish the above result as follows.

Suppose has roots at . Then

Where

Is the differential region containing The mapping is illustrated in Figure 1

for .

Figure 1
Note that

As is varied over the entire axis, the corresponding (non-overlapping) differential regions
in plane cover the entire plane.

Thus,

If is a function of discrete random variables , we can similarly show that

Example 1 The joint pdf of two random variables is given by

 Find the joint expectation of

Example 2 If

Proof:

Thus, expectation is a linear operator.

Example 3

Consider the discrete random variables discussed in Example 4 in lecture 18.The

joint probability mass function of the random variables are tabulated in Table . Find the joint
expectation of .
Remark

(1) We have earlier shown that expectation is a linear operator. We can generally write

Thus
(2) If are independent random variables and ,then

Joint Moments of Random Variables

Just like the moments of a random variable provide a summary description of the random
variable, so also the joint moments provide summary description of two random variables. For
two continuous random variables , the joint moment of order is defined as

And the joint central moment of order is defined as

 where and

Remark
(1) If are discrete random variables, the joint expectation of order and is
defined as

(2) If and , we have the second-order moment of the random variables

given by

(3) If are independent,

Covariance of two random variables

The covariance of two random variables is defined as

Cov(X, Y) is also denoted as .

Expanding the right-hand side, we get

The ratio is called the correlation coefficient.

 If then are called positively correlated.

If then are called negatively correlated

If then are uncorrelated.

We will also show that To establish the relation, we prove the following result:

For two random variables

Proof:

Consider the random variable

Non-negativity of the left-hand side implies that its minimum also must be nonnegative.

For the minimum value,

so the corresponding minimum is

Since the minimum is nonnegative,

Now
 Thus

Uncorrelated random variables

Two random variables are called uncorrelated if

Recall that if are independent random variables, then

then

Thus two independent random variables are always uncorrelated.

Note that independence implies uncorrelated. But uncorrelated generally does not imply
independence (except for jointly Gaussian random variables).

Joint Characteristic Functions of Two Random Variables

The joint characteristic function of two random variables X and Y is defined by

If and are jointly continuous random variables, then

 Note that is same as the two-dimensional Fourier transform with the basis function

instead of

is related to the joint characteristic function by the Fourier inversion formula

If and are discrete random variables, we can define the joint characteristic function in terms
of the joint probability mass function as follows:

Properties of the Joint Characteristic Function

The joint characteristic function has properties similar to the properties of the chacteristic
function of a single random variable. We can easily establish the following properties:

1.
2.
3. If and are independent random variables, then

4. We have,
 Hence,

In general, the order joint moment is given by

Example 2 The joint characteristic function of the jointly Gaussian random variables and
with the joint pdf

Let us recall the characteristic function of a Gaussian random variable

If and is jointly Gaussian,

we can similarly show that

We can use the joint characteristic functions to simplify the probabilistic analysis as illustrated
on next page:

Jointly Gaussian Random Variables

Many practically occurring random variables are modeled as jointly Gaussian random variables.
Two random variables are called jointly Gaussian if their joint probability density
 UNIT IV
Stochastic Processes-Temporal Characteristics

 The Stochastic process Concept

 Classification of Processes, Deterministic and Nondeterministic Processes
 Distribution and Density Functions
 Statistical Independence
 Concept of Stationarity: First-Order Stationary Processes, Second-Order and Wide-Sense
Stationarity, Nth-Order and Strict-Sense Stationarity
 Time Averages and Ergodicity
 Mean-Ergodic Processes
 Correlation-Ergodic Processes
 Autocorrelation Function and Its Properties
 Cross-Correlation Function and Its Properties
 Covariance Functions and its properties
 Gaussian Random Processes.

Linear system Response:

 Mean
 Mean-squared value
 Autocorrelation
 Cross-Correlation Functions
 Random Process Concept

The concept of a random process is based on enlarging the random variable concept to
include time. The random variable is function of sample coins or sample space and the random
process is both sample space and time then it is called random process or stochastic process
and it is defined as X(t, s).
The random process X(t, s) has family of specific values x(t, s). A random process is sort
form can be represented as X(t), it has family of specific values x(t).
Random process can be represented in three methods.

Both time ‘t’ and sample space‘S’ are variable.

1. Time ‘t’ is fixed and sample space ‘S’is variable.

2. Time ‘t’ is fixed and sample space ‘S’is fixed.

t1 t2 = t 1 +

K.RAVEENDRA, Associate Professor, In-charge Examinations Branch, SVEW, TPT. Page 1

Example: Let us consider an experiment of measuring the temperature of a room with different or
collection of room temperature using thermometer. Each thermometer is a random variable which
can take on any value from the sample space ‘S’. Also, at different times the reading of
thermometers may be different. Thus, the room temperature is a function of a both the sample space
and time. In this example, the concept of random variable can be extended by taking into
consideration of time dimension. Here we assign atime function x(t, s) to every outcome ‘s’. There
will be a family of all such functions. The family X(t, S) is known as “random process” or
“stochastic process”. In place of x(t, s) and X(t, S), the sort form notation x(t) and X(t) are often
used.
The Fig. 1 shows random process, ‘S’ is sample space with sample S1, S2, S3. Sample S1 is
corresponds to thermometer1 readings i.e., x1(t). S2 and S3 are corresponds to thermometer2 and
thermometer3 readings respectively.
To determine the statistics of the room temperature, say mean value two methods are used.

The random variable corresponding to random process can be obtained by fixing time
T = t1, t2, t3, . . . tN The random variable X1 is obtained at fixing time t = t1, then

. t=t1 = X1 = {A 1, A2, A3}

X(t) similarly (= X (t1))

The random variable X2 is obtained at fixing time t = t2, then

. t=t2 = X2 = {B1, B2, B3}

X(t) similarly (= X (t2))

then the PDF of a random variable X1 and X2 can be obtained by calculating probability of a
random variable.
Let fX1 (x1) and fX2 (x2) are represents the PDFs of random variable X1 and X2. The CDF’s can
be obtained by integrating or adding the PDF’s FX1 (x1) and FX2 (x2)are represents the CDFs
of random variable’s X1 and X2.
∴ The statistical parameters of random process are mean value or expectation or statistical
average or ensemble average

Correlation of random process:

The random process X(t) is expected value of random variable X1 and X2 is

E[X(t1).X(t2)] = RX1X2 (t1, t2)

= E[X(t1).X(t + τ )]
= RX1X2
Classifications of Random Process

1.Non-Deterministic process

i. Continuous random process

ii. Discrete random process
iii. Continuous random sequence or Continuous sequence random process
iv. Discrete random sequence or Discrete sequence random process

2. Deterministic random process

3. Stationary random process

i. First order Stationary random process

ii. Second order Stationary random process
iii. Nth order Stationary random process
iv. Strict sense stationary random process (SSS)
v. Wide sense stationary random process (WSS)

Non-Deterministic process: If the future values of any sample function cannot be

predicted exactly from observed past values, the process is called “non-deterministic
process”.
i. Continuous random process: If the future values are not predicted in ad- vance
and values are continuously varying with respect to time then it is called
“continuous random process”. Examples are
– Temperature measured using thermometer.
– Thermal noise generated by resistor.

X(t)

Fig. 2

ii. Discrete random process: If X(t) is discrete with respect to time ‘t’ then
random process is called “Discrete random process”. It has only two set of values.
Ex: Logic ‘1’ and ‘0’ generated by personal computer.
 X(t)

5V

t
-5V

Fig..3
iii. Continuous sequence random process: A random process for which X(t) is
continuous but time has only discrete values is called a “continuous se- quence
random process”. This can be obtained by sampling continues ran- dom process.
X(t)

Fig. .4

iv. Discrete sequence random process: A random process for which X(t) and ‘t’ are
discrete is called a “discrete sequence random process”. This can be obtained by
sampling discrete random process.

X(t)

5V

t
-5V

Fig.5
2. Deterministic random process: If the future values of any sample function can be
predicted exactly from observed past values, the process is called “Deterministic
process”.

• Example: X(t) = A cos (ω0t + θ) = A cos (2πf0t + θ)

Here A, f0, or ω0 and θ are random variable.

3. Stationary random process: If the statistical parameters of a random process are

constant with respect to time, then it is called “stationary random process”. This means the
random process X(t) and X(t + τ ) possess the same statistical properties for any value of τ
(i.e., not affected by a shift in the time)
The physical meaning of stationary is that a time translation of a sample function results in
another sample function of the random process having the same probability.

i. First order Stationary random process: If the first order PDF and expectation
constant doesn’t change with respect to time, then the random process is called
“first order Stationary random process”. Ex:
– fX1 (x1) is constant. i.e., does not with respect to time.
– E[X1] is constant.
ii. Second order Stationary random process: If the second order PDF and ex-
pectation constant doesn’t change with respect to time, then the random process is
called “2nd order Stationary random process”. Ex:

– fX1X2 (x1, x2) is constant. i.e., does not with respect to time.
– E[X(t1)X(t1 + τ )] is constant.
iii. Nth order Stationary random process: If the Nth order PDF and expectation
constant doesn’t change with respect to time, then the it is called “Nth order
Stationary random process”. Ex:
– fX1X2...XN (x1, x2 . . . xN ) is constant. i.e., does not with respect to time.
– E[X(t1)X(t2) . . . X(tN )] is constant.
iv. Strict sense stationary random process (SSS): If all statistical parameters and
PDFs are does not change with respect to time then it is called strict sense
stationary random process.
v. Wide sense stationary random process (WSS): If expectation or mean is
constant and correlation is function of τ = t2 − t1 then it is called wide sense
stationary random process.
– E[X(t)] is constant.
– E[X(t)X(t + τ )] = RXX(τ ) is constant. or
– E[X(t1)X(t2)] = E[X(t1)X(t1 + τ )] is constant.
4. Non-Stationary random process: If any statistical parameters is changes withrespect
to time, then it is called “non-stationary random process”.

Problem 1: A random process X(t) = A cos(ω0t + θ) is stationary if A and ω0 are constants

and θ is a uniformly distributed variable on the interval (0, 2π). Show that itis WSS r.p.
Solution: Given X(t) = A cos(ω0t + θ); where A and ω0 are constants and
θ −→ (0, 2π);. The θ is uniformly distributed between 0 to 2π. The distribution shown in
Fig.

If mean value and auto correlation function of r.p is a function of time, ‘t’ then it isnot
stationary.
Correlation function

Correlation finds the similarities between two random variables in the random process.

Auto-correlation function
Let X(t) be the random process which contain X(t1) and X(t2) are random variables. Auto
correlation function is defined as

RXX(t1, t2) = E[X(t1)X(t2)]

RXX(t, t + τ ) = E[X(t1)X(t + τ )]
RXX(τ ) = E[X(t)X(t + τ )]
Properties of Auto-correlation

1. Mean square or total power of random process can be obtained at τ = 0. i.e.,

E[X2(t)] = RXX(0)

Proof. RXX(τ ) = E[X(t)X(t + τ )]

If τ = 0;
RXX(0) = E[X(t)X(t)]
= E[X2(t)]
= X2

2. Auto-correlation function is even function RXX(τ ) = RXX(−τ )

Proof. RXX(τ ) = E[X(t)X(t + τ )]

Let τ = −τ ;
RXX(−τ ) = E[X(t)X(t − τ )]

Let u = t − τ =⇒ t = u + τ

RXX(−τ ) = E[X(u + τ )X(u)]

= RXX(τ ).

3. Auto-correlation function has maximum value at origin.

|RXX (τ )| ≤ RXX (0)
Proof. Consider positive quantity,

4. If X(t) is is independent, then

lim RXX(τ ) = 0
|τ |→∞

or If X(t) is ergodic, zero mean and has no periodic component, then

lim RXX(τ ) = 0
|τ |→∞

5. If X(t) is periodic then RXX(τ ) will be a periodic with a same period.

 6. If a random process with a zero mean has DC component ‘A’; Y (t) = A + X(t)
then RY Y (τ ) = A2 + RXX(τ )

Proof.
Y (t) = A + X(t)
RY Y (τ ) = E[Y (t)Y (t + τ )]
= E[(A + X(t))(A + X(t + τ ))]

7. If the random process Z(t) is a sum of two random process X(t) and Y (t) that is
Z(t) = X(t) + Y (t) then RZZ(τ ) = RXX(τ ) + RXY (τ ) + RY X(τ ) + RY Y (τ ).

8. RXX(τ ) cannot have an arbitrary shape.

9. The auto-correlation of a r.p X(t) is a finite every function.

̅ ≠ 0 is ergodic with no periodic components then

10. If E[X(t)] = 𝑋
2
lim RXX(τ ) = X
|τ |→∞

Notes:
• Mean of X(t) = DC component

• E[X2(t)] = Total Power

• (E[X(t)])2 = DC Power

• Variance (σ2 ) = ACXPower

Standard deviation (σX) = rms valu

 Solution:
1. Mean square value E[X2(t)] : From Property (1)

4
E[X2(t)] = RXX (0) = 25 + = 29
1+0

2. Square of Mean value :(E[X(t)])2:

E[X(t)] = X(t) ≠ 0 then

2 4
lim RXX(τ ) = X = lim 25 +
|τ |→∞ |τ |→∞ 1 + τ2
6
∞ 0
= lim 25 + 1 = 25 + = 25
|τ |→∞
τ2 +6 0+6
 4τ 2 + 100
Problem 2: The auto correlation function of a WSS r.p is given by RXX(τ ) = .
τ2 + 4
Find mean and variance?
Solution:

Problem 3: Assume that X(t) is a WSS random process with an auto correlation function
RXX(τ ) = e−α|τ|. Determine the second moment of the random variable X(8) − X(5).
Solution: We know that E[X(t)X(t + τ )] = RXX(τ ); RXX(0) = E[X2(t)]
The second central moment of the r.v X is given by E[X2(t)].
E[X(8) − X (5)]2 = E[X 2 (8)] + E[X 2 (5)] − 2E[X 2 (8)X 2 (5)]

= E[X2(8)] + E[X2(5)] − 2E[X2(5)X2(5 + 3)]

= 1 + 1 − 2RXX(3) ∵ E[X2(t)] = RXX(0) = e0 = 1
= 2(1 − e−3α)

Cross Correlation Function

The correlation between two random variables which are obtain from two differentrandom
process is called cross correlation.
Let two random process X(t) and Y (t) with random variable X(t1) and Y (t2). The cross
correlation can be defined as

RXY (t1, t2) = E[X(t1)Y (t2)]

RXY (t, t + τ ) = E[X(t)Y (t + τ )]
RXY (τ ) = E[X(t)Y (t + τ )]
Properties of Cross Correlation Function
:

1. If X(t) and Y (t) are orthogonal process then RXY (τ ) = 0

2. If X(t) and Y (t) are independent and WSS random process, then,

RXY (τ ) = E[X]E[Y ] = X Y

3. If two random process X(t) and Y (t) have zero mean and independent, then

lim RXY (τ ) = 0
τ →∞

4. The cross correlation function satisfies the symmetric property i.e., RXY (-τ ) = RYX (τ )

Proof.
RXY (τ ) = E[X(t)Y (t + τ )]

Let τ = −τ
RXY (−τ ) = E[X(t)Y (t − τ )]

Let t − τ = u −→ t = u + τ

RXY (−τ ) = E[X(u + τ )Y

(u)]
= RYX(τ )

" #
1
 Covariance Function

We know that co-variance of two random variables X and Y are

CXY = µ11 = E[(x − X)(y − Y )] = RXY − X Y = E[XY ] − X Y

Auto covariance Function:

Let X(t) be a random process with random variable obtained at t1 and t2, the covariancecan
be defined as
Properties of Auto Covariance function:

1. If X(t1) and X(t2)2 are orthogonal, then E[X(t1)X(t2)] = 0 −→ RXX(0) = 0

∴ CXX (τ ) = −X (t)

2. IfX(t1) and X(t2) are independent then, CXX(τ ) = 0

3. If τ = 0 and CXX(τ ) = 0 then

2
CXX(0) = RXX(0) − X (t)
 2
E[X2(t)] − X (t)
CXX(0) = σ2

Cross covariance Function:

Let X(t) and Y (t) are two random processes. The covariance between two random variables X(t1) and X(t2 )
which are obtained from X(t) and Y (t) random process. The cross covariance can be written as
 UNIT V
Stochastic Processes-Spectral Characteristics

 The Power Spectrum and its Properties

 Relationship between Power Spectrum and Autocorrelation Function
 The Cross-Power Density Spectrum and Properties
 Relationship between Cross-Power Spectrum and Cross-Correlation Function.

Spectral characteristics of system response:

 Power density spectrum of response

 Cross power spectral density of input and output of a linear system
 CHAPTER 8

Spectral Characteristics

8.1 Spectral Representation

In previous sections studied the characteristics of random process in time domain. The
characteristics of random process can be represented in frequency domain also and the
function obtained in frequency domain is called the spectrum of random signal and
measured in ‘volts/Hertz’.
Let X(t) be a random process as shown in Fig.
The random process X(t) and XT (t) be defined as that portion of X(t) between
−T to +T i.e.,

X(t); −T < t < t
XT (t) =
0; otherwise

Fourier transforms are very useful in spectral in spectral representation of the ran-
dom signals. For example, consider a random signal x(t), the Fourier transform of x(t)
is X(ω) is given by

Z∞
x(t)e−jωt dt

X(ω) = F x(t) =
t=−∞

This function X(ω) is considered to the voltage density specturam of x(t).; But, the
problem is that X(ω) may not exist for many functions of a random process. Therefore,
the spectral representation of random process utilizing a voltage density spectrum is not
feasible always.
In such situation, we go for the power density spectrum of a random process which is
defined as the function which results when the power in the random process is described
as a function of frequency.

8.2 Power Spectral Density (PSD)

Let X(t) be a random process (r.p) as shown in Fig.

246
The random process X(t) between −T to +T can be written as

X(t); −T ≤ t ≤ T
XT (t) =
0; elsewhere

The Fourier Transform of XT (t) can be written as

Z∞
XT (t)e−jωt dt
 
F X(t) = XT (ω) =
t=−∞
ZT
= XT (t)e−jωt dt
t=−T
ZT
∴ XT (ω) = X(t)e−jωt dt
t=−T

The energy and power of a random process:

1. Time domain
RT 2
RT
• Energy E = X (t) dt = XT2 (t) dt
t=−T t=−T

1
RT 1
RT
• Power P = lim X 2 (t) dt = lim XT2 (t) dt
T →∞ 2T t=−T T →∞ 2T t=−T

2. Frequency domain

1
R∞ 1
R∞
• Energy E = 2π
X 2 (ω) dω = 2π
XT2 (ω) dω
ω=−∞ ω=−∞

1 1
R∞ 1 1
R∞ 2
• Power P = lim × 2π X 2 (ω) dω = lim × XT (ω) dω
T →∞ 2T ω=−∞ T →∞ 2T 2π
ω=−∞

The average power of random process can be written as

ZT
1
E X 2 (t) dt
 
PXX = lim
T →∞ 2T
t=−T
Z∞
1 1
E XT2 (ω) dω
 
= lim ×
T →∞ 2T 2π
ω=−∞
Z∞  
1 E XT2 (ω)
= lim dω
2π T →∞ 2T
ω=−∞
Z∞  
1 E XT2 (ω) h  i
= A E X 2 (t) dt dω = A E X 2 (t)


∵ lim
T →∞ 2π 2T
ω=−∞

247
 Z∞  
1 E XT2 (ω)
∴ PXX = lim
2π T →∞ 2T
ω=−∞
Z∞
1
= SX X(ω) dω
2π
ω=−∞

where SX X(ω) is called Power Spectral Density (PSD) or Power Density Spectrum
(PDS) and given by
 
E XT2 (ω)
SXX (ω) = lim
T →∞ 2T

8.2.1 Wiener Kinchin Relation

The Wiener Kinchin relation says that Power Spectral Density (PSD) and Auto-correlation
function from the ∞
Fourier Transform pair.
Z
SXX (ω) = RXX (τ )e−jωτ dτ
τ =−∞ F
Z∞ ∴ RXX (τ ) ←
→ SXX (ω)
1
RXX (τ ) = SXX (ω)e+jωτ dω
2π
ω=−∞

Proof. Let X(t) be the random process with PSD of

 2
E XT (ω)
SXX (ω) = lim
T →∞ 2T
1
E XT∗ (ω)XT (ω)
 
= lim
T →∞ 2T
( Z∞ Z∞ )
1
= lim E X(t1 )ejωt1 dt1 · X(t2 )e−jωt1 dt2
T →∞ 2T
t1 =−∞ t2 =−∞

Where X(t1 ) and X(t2 ) are two random variables obtained from random process
X(t) as t = t1 and t = t2

XT∗ (ω) = F [X(t1 )]

XT (ω) = F [X(t2 )]

ZT ZT
1
E X(t1 )X(t2 ) e−jω(t2 −t1 ) dt1 dt2
 
SXX (ω) = lim
T →∞ 2T
t1 =−T t2 =−T

let t1 = T, t2 = t1 + τ ⇒ τ = t2 + t1

248
 ZT ZT +t
1
E X(t)X(t + τ ) e−jωτ dt dτ
 
SXX (ω) = lim
T →∞ 2T
t=−T τ =T −t
T
 
Z ZT
 lim 1 E X(t)X(t + τ ) dt · e−jωτ dτ
 
=
T →∞ 2T
τ =−T t=−T
ZT
A RXX (τ ) e−jωτ dτ
 
=
τ =−T

When the random process X(t) is atleast Wide Sense Stationary random process
(WSS rp), we can write
 
A RXX (τ ) = RXX (τ )
Z∞
SXX (ω) = RXX (τ ) e−jωτ dτ
τ =−∞

Z∞
RXX (τ ) e−jωτ dτ = F RXX (τ )
 
SXX (ω) =
τ =−∞

Z∞
1
SXX (ω) ejωτ dω = F −1 SXX (ω)
 
RXX (τ ) =
2π
ω=−∞

8.2.2 Properties of Power Spectral Density (PSD)

1. Power spectral density is non-negative function.

Proof. " #
1 2
SXX (ω) = lim E XT (ω)
T →∞ 2T

Using the above equation we can say that PSD is a non-negative function.

2. Power spectral density is a real valued function.

Proof. " #
1 2
SXX (ω) = lim E XT (ω)
T →∞ 2T

Using the above equation we can say that PSD is a real valued function.

249
3. Power spectal density is even function

Proof.
Z∞
SXX (−ω) = RXX (τ )e−jωτ dτ (8.1)
τ =−∞

Let ω = −ω
Z∞
SXX (−ω) = RXX (τ )e+jωτ dτ
τ =−∞
Z∞
= RXX (τ ) e−jω(−τ ) dτ
| {z }
τ =−∞ even

Auto-correlation function is even function RXX (τ ) = RXX (−τ )

Z∞
SXX (−ω) = RXX (−τ ) e−jωτ dτ (8.2)
| {z }
τ =−∞ even

From equation (8.1) and (8.2)

SXX (−ω) = SXX (ω)

4. The total area of the auto-correlation function is equal to DC component (or)

average value of a random process.

Proof. From Wiener Kinchin relation

Z∞
SXX (ω) = RXX (τ )e−jωτ dτ
τ =−∞

The DC or average value of random process can be obtained by subtituting ω = 0

in PSD
Z∞
SXX (0) = RXX (τ ) dτ
τ =−∞

5. The total power or mean square value of random process is equal to the total area
of PSD.

250
 Proof. From Wiener Kinchin relation
Z∞
RXX (τ ) = f rac12π SXX (τ )ejωτ dω
ω=−∞

The total power of random process can be obtained by substituting τ = 0

 
RXX (τ ) = E X(t)X(t + τ )
 
If τ = 0 then RXX (0) = E X 2 (t)

Z∞
1
∴ RXX (τ ) = SXX (ω) dω
2π
ω=−∞

NOTE:

• Total power

Z∞
1
∴ RXX (0) = SXX (f ) df = E[X 2 (t)];
2π
ω=−∞

• DC power or Average power (power at zero freq)

Z∞
SXX (0) = RXX (τ ) dτ
τ =−∞

d
6. The PSD of derivation of the random process dt
X(t) is ω 2 times the PSD of the
random process.

SẊ Ẋ (ω) = ω 2 SXX (ω); . −→ denotes derivative

Proof. " #
1 2
SXX (ω) = lim E XT (ω)
T →∞ 2T

ZT
XT (ω) = XT (t)e−jωt dt
t=−T
ZT
d
XT (ω) = ẊT (ω) = XT (t)e−jωt (−jω) dt
dt
t=−T

251
 ẊT (ω) = −jω · XT (ω)

h i
2
E ẊT (ω)
SẊ Ẋ (ω) = lim
T →∞
h 2T i
E − jωXT (ω) · −jωXT (ω)
= lim
T →∞
h 2Ti
E XT (ω)|2
= ω 2 lim
T →∞ 2T
2
= ω SXX (ω)

SẊ Ẋ (ω) = ω 2 SXX (ω)

A2
Problem 1: Find the PSD of Auto-correlation function RXX (τ ) = 2
cos ω0 τ and
plot both Auto-correlation (ACF) and PSD.
Solution:
A2
RXX (τ ) = cos ω0 τ
2 
A2 ejω0 τ + e−jω0 τ

=
2 2
A jω0 τ A2 −jω0 τ
2
= e + e
4 4

The PSD is a Fourier transform of Autocorrelation function.

SXX (ω) = F [RXX (τ )]

 2
A jω0 τ A2 −jω0 τ

=F e + e
4 4
A2  jω0 τ  A2  −jω0 τ 
= F e + F e
4 4
A2 A2
= × 2πδ(ω − ω0 ) + × 2πδ(ω + ω0 )
4 4
A2
= π [δ(ω − ω0 ) + δ(ω + ω0 )]
2

A2
∴ SXX (ω) = π [δ(ω − ω0 ) + δ(ω + ω0 )]
2

252
 
1; ω=0 Let X(ω) = δ(ω − ω0 )
δ(ω) =
0; ω 6= 0
x(t) = F −1 X(ω)
 

Z∞ Z∞
1 1
F −1 [δ(ω)] = δ(ω)ejωτ dω = X(ω)ejωt dω
2π 2π
−∞
ω=−∞
Z∞
1
= δ(ω − ω0 )ejωt dω
 
 −1 1 2π
F F [δ(ω)] = F (1)
2π −∞

1
δ(ω) = F [1] From sampling property of the impulse
2π
2πδ(ω) = F [1] function

If ω = ω0 1 h jωt i
F −1 δ(ω − ω0 ) =
 
e
h i 2π ω=ω0
2πδ(ω − ω0 ) = F ejω0 τ 1 jω0 t
= e
2π
If ω = −ω0 1  jω0 t 
h i = F e
2πδ(ω + ω0 ) = F e−jω0 τ 2π
F ejω0 t = 2πδ(ω − ω0 )
 

Problem: 2 Find the PSD of Autocorrelation function

 h i
A 1 + |τ | ; −T ≤ τ ≤ T
T
RXX (τ ) =
0; otherwise

Solution: Given
 h i
τ


 A 1+ T ; −T ≤ τ ≤ 0
 h i
RXX (τ ) = A 1 − Tτ ; 0≤τ ≤T



0; otherwise

Method-1:
Using ramp function r(τ )

A 2A A
RXX (τ ) = r(τ + T ) − r(τ ) + r(τ − T )
T T T

First derivative of RXX (τ ) with respect to ‘τ ’

d A 2A A
RXX (τ ) = U (τ + T ) − U (τ ) + U (τ − T )
dτ T T T

Second derivative of RXX (τ ) with respect to ‘τ ’

d2 A 2A A
RXX (τ 2 ) = δ(τ + T ) − δ(τ ) + δ(τ − T ) (8.3)
dτ T T T
253
 d 2
We know that dτ RXX (τ 2 ) = (jω)2 SXX (ω)
Now, using differentiation and shifting property,
d
If x(t) ↔ X(ω) then x(t) ↔ jω X(ω);
dt
d2
x(t) ↔ (jω)2 X(ω) and
dt2
x(t − t0 ) ↔ X(ω) e−jωt0
From equation (8.3), apply Fourier transform on both sides

A   2A   A 
(jω)2 SXX (ω) =

F δ(τ + T ) − F δ(τ ) + F δ(τ − T )
T T T
A 2A A
−ω 2 SXX (ω) = ejωT − + e−jωT
T  T T
jωT −jωT

2A e +e
= −1
T 2
2A
= [cos ωT − 1]
T
2A
SXX (ω) = [1 − cos ωT ]
T ω2
4A ωT 1 − cos 2θ
= 2 sin2 ∵ sin2 θ =
ω T( 2 ) 2
2 ωT
sin 2
= AT ω2 T 2
2
" #2
sin ωT
2
= AT ωT
2
 
2 ωT
= AT sinc
2
 
2 ωT
SXX (ω) = AT sinc
2

Method-2:

ZT
RXX (τ )e−jωτ dτ
 
SXX (ω) = F RXX (τ ) =
τ =−T
Z0 ZT
 τ  −jωτ  τ  −jωτ
= A 1+ e dτ + A 1− e dτ
T T
τ =−T τ =0
ZT ZT
 τ  +jωτ  τ  −jωτ
= A 1− e dτ + A 1− e dτ
T T
τ =0 τ =0
ZT  τ  h jωτ i
= A 1− e + e−jωτ dτ
T
τ =0

254
 h i
ZT  ejωτ
+ e −jωτ
τ 
= 2A 1− dτ
T 2
τ =0
ZT  τ
= 2A 1− cos ωτ dτ
T
τ =0
ZT ZT
τ
= 2A cosωτ dτ − 2A cos ωτ dτ
T
τ =0 τ =0
 T  Z T
sin ωτ 2A sin ωτ sin ωτ
= 2A − τ − 1·
ω τ =0 T ω ω τ =0
   T
sin ωτ 2A sin ωτ cos ωτ
= 2A −0 − τ +τ
ω T ω ω 2 τ =0
  
sin ωT 2A sin ωT cos ωT 1
= 2A − T + − 0+ 2
ω T ω ω2 ω
sin ωT sin ωT 2A cos ωT 2A 1
= 2A − 2A − 2
+ · 2
ω ω T ω T ω
2A
= [1 − cos ωT ]
T ω2
4A ωT 1 − cos 2θ
= 2 sin2 ∵ sin2 θ =
ω T 2 2
sin2 ωT
= AT ω2 T 22
4
" #2
sin ωT
2
= AT ωT
2
 
2 ωT
= AT sinc
2
 
2 ωT
∴ SXX (ω) = AT sinc
2

Problem: 3 The autocorrelation function of the random telegraph process is given

by RXX (τ ) = e−2α|τ | . Find the power spectral density (PSD) ? Solution: Given
RXX (τ ) = e−2α|τ |

e2ατ ; −∞ ≤ τ ≤ 0
RXX (τ ) =
e−2ατ ; 0≤τ ≤∞

Z∞
1
SXX (ω) = RXX (τ )e−jωτ dτ
2π
τ =−∞

255
 Z∞
SXX (f ) = RXX (τ )e−j2πf τ dτ
τ =−∞
Z0 Z∞
−j2πf τ
= e2ατ
e dτ + e−2ατ e−j2πf τ dτ
τ =−∞ τ =0
Z0 Z∞
= e(2α−j2πf )τ dτ + e(−2α−j2πf )τ dτ
τ =−∞ τ =0
(2α−j2πf )τ
0 ∞
e(−2α−j2πf )τ
 
e
= +
2α − j2πf −∞ −2α − j2πf 0
1 1
= [1 − 0] + [0 − 1]
2α − j2πf −2α − j2πf
1 1
= +
2α − j2πf 2α + j2πf
2α +   + 2α − j2πf
j2πf
 
= 
(2α)2 − (j2πf )2
4α
=
4α − 4π 2 f 2
2

4α
∴ SXX (f ) =
4α2 − 4π 2 f 2

𝑆𝑋𝑋 (𝑓)

1
𝛼= 1

𝛼= 2

0 𝑓
3 2 1 1 2 3
− − −
𝜋 𝜋 𝜋 𝜋 𝜋 𝜋

Problem: 4 The power spectral density of a WSS white noise whose frequency
components are limited to −W ≤ f ≤ W is shown in the Fig.
𝑆𝑋𝑋 (𝑓)

𝜂
2
(a) Find average power of X(t)?
(b) Find auto-correlation of X(t)?

-W 0 W
f

256
 (b) Auto correlationfor this process is

Solution: Z∞
(a) Average power RXX (τ ) = SXX (f )ej2πf τ df
f =−∞
ZW ZW
E X 2 (t) = η j2πf τ
 
SXX (f ) df = e df
f =−W
2
f =−W
ZW W
η ej2πf τ

η
= df =
2 2 j2πτ −W
f =−W
η ej2πW τ − ej2πW τ
 
η W η =
= [f ]−W = [2W ] 2 j2πW τ
2 2
η
= ηW = sin(2πW τ )
2πτ
sin(2πW τ )
∴ E X 2 (t) = ηW
  = ηW ·
2πW τ
sin πx
RXX (τ ) = ηW sinc(2W τ ) = sincx ∵
πx
Problem: 5 For the random process X(t) = A sin(ω0 t+θ), where A and ω0 are real
constants and θ is a random variable distributed uniformly in the interval 0 < θ < π2 .
Find the average power PXX in X(t)?
𝑓𝛳 (𝛳)

2/𝜋 = 1
𝜋/2

𝛳
0 𝜋/2

Solution:
First Approach:
1 = cos 2θ
E X 2 (t) = E A2 sin2 (ω0 t + θ) ∵ sin2 θ =
   
2
 A2 A2 
=E − cos(2ω0 t + 2θ)
2 2
π
2
2 2 Z
A A 2
= − cos(2ω0 t + 2θ) · dθ
2 2 π
θ=0
2 2
 π
A A sin(2ω0 t + 2θ) 2
= −
2 π 2 θ=0
A2 A 2
= − [sin(2ω0 t + π) − sin 2ω0 t]
2 2π
A2 A2
= − {[sin(2ω0 t) cos π − cos(2ω0 t) sin π] − sin 2ω0 t}
2 2π
A2 A2
= − [− sin 2ω0 t − sin 2ω0 t]
2 2π
257
 A2 A2
= + sin 2ω0 t
2 π

Since E[X 2 (t)] is time dependent. So, X(t) is not WSS random process. Finally we
perform time averages is
ZT
1
PXX = lim E[X 2 (t)]dt
T →∞ 2T
t=−T
ZT 
A2 A2

1
= lim + sin 2ω0 t dt
T →∞ 2T 2 π
t=−T
2
A
=
2

A2
∴ PXX =
2

Second Approach:
Z∞
XT (ω) = XT (t)e−jωt dt
t=−∞
ZT
= A sin(ω0 t + θ) e−jωt dt
t=−T
ZT
ejω0 t+θ − e−jω0 t+θ −jωt
=A ·e dt
2j
t=−T

ZT ZT
A jθ j(ω0 −ω)t A −jθ
= e e dt − e e−j(ω0 +ω)t dt
2j 2j
t=−T t=−T
T T
A jθ ej(ω0 −ω)t A −jθ e−j(ω0 +ω)t
 
= e − e
2j j(ω0 − ω) t=−T 2j −j(ω0 + ω) t=−T
 j(ω0 −ω)T −j(ω0 −ω)T

A jθ e −e
= e
j(ω0 − ω) 2j
 −j(ω0 +ω)T
− ej(ω0 +ω)T

A −jθ e
+ e
j(ω0 + ω) 2j
jθ −jθ
AT e sin(ω0 − ω)T AT e sin(ω0 + ω)T
= −
j (ω0 − ω)T j (ω0 + ω)T
 
jθ sin(ω0 − ω)T −jθ sin(ω0 + ω)T
XT (ω) = jAT e +e
(ω0 − ω)T (ω0 + ω)T
 
2 jθ sin(ω0 − ω)T −jθ sin(ω0 + ω)T
XT (ω) = jAT (−jAT ) e +e
(ω0 − ω)T (ω0 + ω)T

258
  
−jθ sin(ω0− ω)T sin(ω0 + ω)T
× e + ejθ
(ω0 − ω)T (ω0 + ω)T
2 A2
XT (ω) = PXX =
2

A2
∴ Average power PXX =
2

By comparing both two methods, the direct method (second method) is tedious.
So, very easy to compute first method.

Problem: 6 For the stationary ergodic random process having the auto correlation
function as shown in Fig,. Find (a) E[X(t)] (b) E[X 2 (t)] 2
(c) σX of RXX (τ )
Solution:

(a) (E[X(t)])2 = lim RXX (τ ) = 20

√τ →∞
∴ E[X(t)] = 20

(b) E[X 2 (t)] = RXX (0) = 50

2
(c) σX = E[X 2 (t)] − (E[X(t)])2 = 50 − 20 = 30

Problem: 7 Assume that an ergodic random process X(t) has an auto-correlation

2
function RXX (τ ) = 18 + 6+τ 2 [1 + 4 cos(12τ )]. Find X(t) and what is average power

of X(t)?

(b) Average power:

Solution: (a) Square of Mean value:
PXX = E[X 2 (t)] = RXX (τ = 0)
h i2 2  
E[X(t)] = lim RXX (τ ) = 18 + 2
1 + 4 cos(12τ )
τ →∞
6+τ
τ =0
2 2 
= lim 18 + [1 + 4 cos(12τ )]

= 18 + 1 + 4(1)
τ →∞ 6 + τ2 6+0
2
∴ (X(t)) = 18 10 118
= 18 + =
√ 6 6
=⇒ X(t) = ± 18 59
=⇒ PXX = Watts
 3
 1 ; −π ≤ θ ≤ π
Problem: 8 Let X(t) = A cos(ω0 t + θ), fθ (θ) = 2π
0; elsewhere
b2
and Y (t) = B cos(ω0 t); where fB (b) = √1 e− 2 ; −∞ ≤ b ≤ ∞.
2π

259
 E[X(t)] E[Y (t)] CXX (τ )
E[X 2 (t)] E[Y 2 (t)] RXY (τ )
Find
2
σX σY2 CXY (τ )
RXX (τ ) RY Y (τ )
Solution:

1.
E[X(t)] = E[A cos(ω0 t + θ)]
= AE[cos ω0 t sin θ + sin ω0 t cos θ]
= A cos ω0 tE[sin θ] + A sin ω0 tE[cos θ]
Zπ Zπ
1 1
= A cos ω0 t sin θ dθ + A sin ω0 t cos θ dθ
2π 2π
θ=−π θ=−π
A  π A  π
= cos ω0 t cos θ θ=−π + sin ω0 t − sin θ θ=−π
2π 2π
A   A  
= cos ω0 t cos π − cos(−π) − sin ω0 t sin π − sin(−π)
2π 2π
A   A  
= cos ω0 t 1 − 1 − sin ω0 t 0 − 0
2π 2π
=0

2.
E[X 2 (t)] = E[A2 cos2 (ω0 t + θ)]
 
2 1 + cos 2(ω0 t + θ)
=A E
2
2
 
1 A
= A2 E + E [cos 2(ω0 t + θ)]
2 2
1
= A2 · + 0


2
2
A
=
2

2 A2 A2
3. σX = m2 − m21 = 2
− 02 = 2

4.
RXX (τ ) = E[X(t)X(t + τ )]
= Let t = t1 ; t + τ = t2 ;
= E[X(t1 )X(t2 )]
= E [A cos(ω0 t1 + θ) · A cos(ω0 t2 + θ)]
= A2 E [cos(ω0 t1 + θ) · cos(ω0 t2 + θ)]

260
 = A2 E [cos(ω0 t1 − ω0 t2 ) + cos(ω0 t1 + ω0 t2 + 2θ)]
A2 A2 ( =0
(
(
((((
= E [cos(ω0 t1 − ω0 t2 )] + (E([cos(ω ( (
(( 0 1t (+ ω t
0 2 + 2θ)]
2 (2
A2
= E [cos ω0 (t1 − t2 )]
2
A2
E [cos(ω0 τ )] ‘θ’ is a random variable
2
A2
∴ RXX (τ ) = cos(ω0 τ )
2

5.
 
CXX (τ ) = E (X(t) − X(t))(X(t + τ ) − X(t + τ )
 2
= RXX (τ ) − X(t)
A2
= cos(ω0 τ ) − 02
2
A2
∴ CXX (τ ) = cos(ω0 τ )
2
A2
If t1 = t2 = t then CXX (τ ) =
2

E[Y (t)] = E[B cos ω0 t] Here r.v is ‘B’

b2
given FB (b) = √12π e− 2 By comparing
= cos ω0 t E[B]
6. their mean value E[B] = 0 and variance
= cos ω0 t × 0
σB2 = 1
∴ E[Y (t)] = 0
2
⇒ σB2 = E[B 2 ] − E[B] ⇒ E[B 2 ] = 1

7.
E[Y 2 (t)] = E B 2 cos ω02 t
 

= cos ω02 tE B 2
 

= cos ω02 t

8.

2
σY2 = E[Y 2 (t)] − E[Y (t)]
= cos ω02 t − 0
= cos ω02 t

9.
RY Y (τ ) = E[Y (t)Y (t + τ )]
= E[B cos ω0 t B cos(ω0 t + τ )]
= cos ω0 tE[B 2 ]

261
 = cos ω0 t

10.
CY Y (τ ) = RY Y (τ ) − (Y )2
= cos ω0 t − 0
= cos ω0 t

11.
RXY (τ ) = E[A cos(ω0 t1 + θ) B cos(ω0 t2 + θ)]
= E[A cos(ω0 t1 + θ)]E[B cos(ω0 t2 + +θ)]
=0

12.
CXY (τ ) = RXY (τ ) − (X X) = 0

RXY (t) = X X thus X(t) and Y (t) are uncorrelated.

Two process X(t) and Y (t) are called orthogonal then E[X(t1 )X(t2 )] = 0

8.3 Types of random process

Two types:
• Baseband random process

• Bandpass random process

8.3.1 Baseband random process

If the power spectral density SXX (ω) of a random process X(t)have zero frequency
components then it is called baseband random process. The frequency plot of baseband
random process will be shown in Fig. Here 3dB bandwith can be written as
𝑆𝑋𝑋(ω)

-ω ω
−𝑊 0 𝑊
BW

v
u R∞
ω 2 SXX (ω)dω
u
u
uω=−∞
Wrms = rms bandwidth = u
u R∞
t SXX (ω)dω
ω=−∞

262
8.3.2 Bandpass random process

If the power spectral density SXX (ω) of a random process X(t) does not have zero
frequency components then it is called bandpass random process. The frequency plot
of baseband random process will be shown in Fig.
𝑆𝑋𝑋(ω)

-ω 0
ω
− ω0 ω0
BW BW

v
u R∞
2
ω − ω0 SXX (ω)dω
u
u4
ω=−∞
u
Wrms = rms bandwidth = u
u R∞
t SXX (ω)dω
ω=−∞

R∞
ωSXX (ω)dω
ω=0
where ω0 = R∞
SXX (ω)dω
ω=0

Question 1: What is the bandwidth of the power density spectrum?

Assume X(t) is a lowpass random process,i.e its spectral components are clustered
near w = 0 and have decreasing magnitude at higher frequencies.Except for the fact that
the area of SXX (ω) is not necessarily unity,SXX (ω) has characteristics similar to prob-
ability density function(PDF).Indeed,by dividing SXX (ω) by its area,a new function is
formed with area of unity that is analogous to a density function.
Standard deviation is a measure of the spread in a density function.The analogous
quantity for the normalized power spectrum is a measure of its spread that we call rms
bandwidth ,which we denoted by Wrms .
Now since SXX (ω) is an even function for a real process, its “mean value” is zero
and its standard deviation is the square root of the its second moment, thus upon nor-
malization, the rms bandwidth v is given by
u R∞ 𝑆 (ω) 𝑋𝑋

ω 2 SXX (ω)dω
u
u
uω=−∞
Wrms = rms bandwidth = u u R∞
t SXX (ω)dω
-ω ω
ω=−∞ −𝑊 0 𝑊
BW

It is also called Baseband random process.

263
 2
Problem 9: The PSD of a baseband random process X(t) is SXX (ω) = 
2 
ω
1+ 2

Find rms BW?

Solution:
R∞
ω 2 SXX (ω)dω
2 ω=−∞
Wrms = R∞
SXX (ω)dω
ω=−∞

Numerator part:

Z∞
ω 2 SXX (ω)dω
ω=−∞
Z∞
ω2 × 2
= h
i dω
ω 2
ω=−∞
1+ 2
Z∞
2ω 2
= 2  dω
1 + ω4

ω=−∞
Z∞
2 × 4ω 2
= dω
4 + ω2
ω=−∞
Z∞
ω2
=2×8 dω
4 + ω2
ω=0

put ω = 2 tan θ ⇒ dω = 2 sec2 θ dθ

Z∞
4 tan2 θ
= 16 2
· 2 sec2 θ
4 + 4 tan θ
ω=0
Z∞
4 tan2 θ 2
= 16 × 2 2
 ·
sec θ
4
(1+tan
 θ)
ω=0
Z∞
= 32 sin2 θ cos−2 θ dθ
ω=0
  Z∞  
3 1 m n m+1 n+1
= 32 β , − ∵ sin θ cos θ dθ = β ,
2 2 2 2
0
Γ 32 Γ − 12



Γ(m)Γ(n)
= 32 β(m, n) =
Γ(1) Γ(m + n)
Γ 1 + 21 Γ − 21



= 32
Γ(1)

264
 1 Γ 12 Γ − 12



= 32 × Γ(1 + n) = nΓ(n)
2 Γ(1)
Γ 1 − 12


1√ √ √
   
1 1
= 32 × π×2 π Γ − = = −2Γ = − 2 π
2 2 − 12 2
= 32π

Z∞
∴ ω 2 SXX (ω)dω = 32π
ω=−∞
Denominator part:
Z∞ Z∞
2×4
SXX (ω) dω = dω
22 + ω 2
ω=−∞ ω=−∞
 ∞
1 −1
=8 tan ω
2
h π  π iω=−∞
=4 − −
2 2
= 4π

R∞
ω 2 SXX (ω)dω
2 ω=−∞ 32π
Wrms = R∞ = =8
4π
SXX (ω)dω
ω=−∞
√
=⇒ ∴ Wrms = 8

1; for |ω| < B
Problem 10: Assume random process PSD SX X(ω) =
0; for |ω| ≥ B
Find the rms bandwidth?
𝑆𝑋𝑋 (ω)

0
ω
-B B

Solution: Given

1; −B ≤ ω ≤ B
SX X(ω) =
0; otherwise
R∞
ω 2 SXX (ω)dω
2 ω=−∞
Wrms = R∞
SXX (ω)dω
ω=−∞

265
 R∞ RB
ω 2 (1) dω ω 2 dω
ω=−∞ ω=−B
= R∞ =
RB
(1) dω 1 dω
ω=−∞ ω=−B
h iB
ω3
3 2B 3
−B 3
=  B =
ω −B 2B

B2 B
= =⇒ ∴ rms bandwidth = Wrms = √
3 3

B
1; |ω ± ω0 | < 2
Problem 11: Assume random process PSD SX X(ω) =
0; elsewhere
Find the rms bandwidth?
𝑆𝑋𝑋 (ω)

-ω ω
𝐵 − ω0 𝐵 0 𝐵 𝐵
− ω0 + − ω 0− ω 0− ω0 − ω0 +
2
2 2 2

Solution:
R∞
2 R∞
4 ω − ω0 SXX (ω)dω ωSXX (ω)dω
2 ω=−∞ ω=0
Wrms = R∞ ; where ω0 = R∞
SXX (ω)dω SXX (ω)dω
ω=−∞ ω=0

(i)
ω0 + B
Z∞ Z 2  
 ω0 + B2 B B
SXX (ω) dω = 1 · dω = ω ω − B = ω0 + − ω0 − =B
0 2 2 2
0 ω0 − B
2

(ii)
ω0 + B
Z∞ 2 ω0 + B2
ω2
Z 
ωSXX (ω) = ω · 1 · dω =
2 ω0 − B
0 2
ω0 − B
2
( 2  2 )
1 B B
= ω0 + − ω0 −
2 2 2
 
1 B
= × 4 ω0 · = Bω0
2 2

266
(iii)
R∞
ωSXX (ω)dω
ω=0 Bω0
∴ ω0 = R∞ = =B ∵ (i) and (ii)
B
SXX (ω)dω
ω=0

(iv)
R∞
2
4 ω − ω0 SXX (ω)dω
2 ω=−∞
Wrms = R∞ (8.4)
SXX (ω)dω
ω=−∞

By taking numerator term ω0 = B and SXX (ω) = 1

Z∞

2
ω − ω0 SXX (ω)dω
ω=−∞
ω0 + B
2
Z

2
= ω − ω0 · 1 · dω
ω=ω0 − B
2

ω0 + B
2
Z
ω 2 + ω02 − 2ωω0 dω
 
=
ω=ω0 − B
2

ω3 ω2
 
2
= + ω0 ω − 2ω0
3 2
( 3  3 )   
1 B B 2 B B
= ω0 + − ω0 − + ω0 ω0 + − ω0 −
3 2 2 2 2
( 2  2 )
B B
− ω0 ω0 + − ω0 −
2 2

∵ (a + b)3 − (a − b)3 = 6a2 b + 2b3 ; ∵ (a + b)2 − (a − b)2 = 4ab

(  3 )  
1 2B B 2 2B B
= 6ω0 + 2 + ω0 · − ω0 4ω0
2 2 2 2 2
B B3 4ω 2 B
 
1
= 6ω02 + + ω02 B − 0
2 2 4 2
3
B
= ω02 B + + ω02 B − 2ω02 B
12
3
 B
=2ω02 B + − 2ω2 B

12  0
B3
=
12

From the equation (8.4) and solution of (i) then

267
 3
2 4 · B12 B2
=⇒ Wrms = =
B 3

B
=⇒ ∴ Wrms = rms bandwidth = √
3

• Both, the ideal low pass and band pass process, rms bandwidth is equal i.e., √B3 .
This is the only the case if the factor is present 4 in bandwidth of band-pass
random process.

8.4 Cross correlation and cross PSD

Let two random process X(t) and Y (t), the sample function of random processcan be
written as 
X(t); −T ≤ t ≤ T
XT (t) =
0; elsewhere

y(t); −T ≤ t ≤ T
YT (t) =
0; elsewhere

The Fourier Transform of XT (t) can be written as

Z∞
XT (t)e−jωt dt
 
F X(t) = XT (ω) =
t=−∞
ZT
= XT (t)e−jωt dt
t=−T
ZT
∴ XT (ω) = X(t)e−jωt dt
t=−T

The Fourier Transform of YT (t) can be written as

Z∞
YT (t)e−jωt dt
 
F X(t) = YT (ω) =
t=−∞
ZT
= YT (t)e−jωt dt
t=−T
ZT
∴ YT (ω) = Y (t)e−jωt dt
t=−T

The cross-power between X(t) and Y (t) in interval (−T, T ) can be written as

268
 Z∞ Z∞
1 XT∗ (ω)YT (ω)
PXY = XT (t)YT (t) dt = dω
2π 2T
t=−∞ ω=−∞

The total cross-power can be written as

Z∞ Z∞
1 1 XT∗ (ω)YT (ω)
lim XT (t)YT (t) dt = lim dω
T →∞ 2T T →∞ 2π 2T
t=−∞ ω=−∞

The total average cross power can be written as

Z∞ Z∞
1 1 E[XT∗ (ω)YT (ω)]
∴ PXY = lim E[XT (t)YT (t)] dt = lim dω
T →∞ 2T T →∞ 2π 2T
t=−∞ ω=−∞

Z∞
  1
∴ PXY = A E[X( t)YT (t)] = SXY (ω) dω
2π
ω=−∞

where SXY is cross PSD can be written as

E[XT∗ (ω)YT (ω)]

SXY (ω) = lim dω
T →∞ 2T

8.4.1 Wiener Kinchin Relation

The Wiener Kinchin relation says that Cross Power Spectral Density (PSD) SXY (ω)
and Cross-correlation function RXY (τ ) from the Fourier Transform pair.
Z∞
SXY (ω) = RXY (τ )e−jωτ dτ
τ =−∞ F
Z∞ ∴ RXY (τ ) ←
→ SXY (ω)
1
RXY (τ ) = SXY (ω)e+jωτ dω
2π
ω=−∞

Proof. Let X(t) be the random process with PSD of

( ZT ZT )
1
SXY (ω) = lim E XT (t)ejωt dt · YT (t)e−jωt dt
T →∞ 2T
t=−T t=−T

Where XT (t) is obtained from random random process X(t) as t = t1 and Y (t) is
obtained t = t2 = t1 + τ then
( ZT ZT )
1
SXY (ω) = lim E XT (t1 )ejωt1 dt1 · YT (t2 )e−jωt2 dt2
T →∞ 2T
t1 =−T t2 =−T

269
 ZT ZT ( )
1
= lim E XT (t1 )YT (t2 ) e−jω(t2 −t1 ) dt2 dt1
T →∞ 2T
t1 =−T t2 =−T

let t1 = T, t2 = t1 + τ ⇒ τ = t2 − t1
ZT ZT +t ( )
1
= lim E XT (t)YT (t + τ ) e−jωτ dτ dt
T →∞ 2T
t=−T τ =T −t
T
 
Z ZT
=  lim 1 RXY (τ )e−jωτ dt dτ
T →∞ 2T
τ =−T t=−T
ZT
= A [RXY (τ )] e−jωτ dτ
τ =−T

The random process X(t) and Y (t) are WSS random process, then

Z∞
RXY (τ ) e−jωτ dτ = F RXY (τ )
 
SXY (ω) =
τ =−∞

Z∞
1
SXY (ω) ejωτ dω = F −1 SXY (ω)
 
RXY (τ ) =
2π
ω=−∞

8.4.2 Properties of Cross Power Spectral Density (PSD)

1. Power spectal density is even function, SXY (ω) = SXY (−ω)

Proof.
Z∞
SXY (−ω) = RXY (τ )e−jωτ dτ (8.5)
τ =−∞

Let ω = −ω
Z∞
SXY (−ω) = RXY (τ )e+jωτ dτ
τ =−∞
Z∞
= RXY (τ ) e−jω(−τ ) dτ
| {z }
τ =−∞ even

270
 Cross-correlation function is even function RXY (τ ) = RXY (−τ )

Z∞
SXY (−ω) = RXY (−τ )e−jω(−τ ) dτ (8.6)
τ =−∞

From equation (8.5) and (8.6)

SXY (−ω) = SXY (ω)

2. The real part of cross PSD is even and imaginary part is odd function.

Proof.
Z∞
SXY (ω) = RXY (τ )e−jωτ dτ
τ =−∞
Z∞
= RXY (τ ) [cos ωτ − j sin ωτ ] dτ
τ =−∞
Z∞
Re [SXY (ω)] = RXY (τ ) cos ωτ dτ =⇒ even function
τ =−∞
Z∞
Im [SXY (ω)] = − RXY (τ ) sin ωτ dτ =⇒ odd function
τ =−∞

3. If X(t) and Y (t) are orthogonal randomprocess then cross PSD is zero.

Proof.
Z∞
SXY (ω) = RXY (τ )e−jωτ dτ
τ =−∞

RXY (τ ) = E [X(t)X(t + τ )] = 0
IfX(t) and Y (t) are orthogonal.

∴ SXY (ω) = 0

4. If X(t) and Y (t) are uncorrelated and WSS r.p then SXY = 2πX Y δ(ω)

271
 Proof. From Wiener Kinchin relation
Z∞
SXX (ω) = RXX (τ )e−jωτ dτ
τ =−∞
Z∞
= E [X(t)Y (t + τ )] e−jωτ dτ
τ =−∞
Z∞
= E [X(t)] E [Y (t + τ )] e−jωτ dτ ∵ X(t), Y (t) are independent
τ =−∞
Z∞
= X Y e−jωτ dτ ∵ WSS E[X(t)] = X; E[Y (t + τ )] = Y
τ =−∞
Z∞
=XY e−jωτ dτ
τ =−∞

= X Y · 2πδ(ω)
∴ SXX (ω) = 2πX Y δ(ω)

8.5 White Noise

A white noise in which all frequency components from f = −∞ to f = ∞ are present

in equal measure i.e., whose PSD remains constant for all frequencies and is indepen-
dent of frequency, which is called “white noise”. It is shown in figure.
N0
∴ SN (f ) = ; −∞ ≤ f ≤ ∞; N0 is constant
2

∴ The white noise process is zero mean WSS process with PSD is constant (flat)
for all frequencies. It is strictly speaking if we take inverse fourier transform of a flat
function does not exist for all frequencies f .
 
−1 −1 N0 N0
F {SN (f )} = RW W (f ) = F = δ(τ )
2 2

But from definition,

Z∞ Z∞
1
RW W (τ ) = SW (f )dω = SW (f ) df
2π
ω=−∞ ω=−∞

If τ = 0, we will get mean square value, so,

272
 Z∞
N0
RW W (τ ) = W2 = df = ∞
2
ω=−∞

So, this mean square value (power) of white process is infinite. However it is not
possible to have a random process with infinite power, white noise does not exist in
the physical world. It is mathematical model can be used a close approximation toreal
world process.
Gaussian white noise often called white Gaussian noise, for any two (or several)
random variables from the process independent and long as they are not same random
variable, and uncorrelated their mean is zero.

8.6 Signal to Noise Ratio (SNR)

The SNR is defined as

Signal power
SN R =
Noise power
Here SNR is a ratio of powers and not of voltages. We may express SNR is in
decibels rather than just a ratio.

Signal power
(SN R)dB = 10 log10
Noise power

𝑥(𝑡) ℎ(𝑡) 𝑦(𝑡)

𝑆𝑖 System 𝑆o
𝑁𝑖 𝑁o

Here,
Si − input signal power So − input signal power
Ni − input signal power No − input signal power
G− system gain

The output signal powerS0 = GSi

and output noise powerN0 = GNi + Na

where Na − is additional noise power in the system.

 
S Input signal power Si
Input SNR = =
N Input noise power Ni
 i
S Output signal power GSi
Output SNR = =
N o Output noise power GNi + Na

The output (SNR)10 < Input (SNR)10

273
 (S/N )o
∴ SN R =
(S/N )i

For any circuit, contain some noise producing active/ passive elements in it. These
SNR at the output will always be less than the SNR at the input, i.e., there is a deterio-
ration of SNR. Thus an amplifier does not improve SNR, it only degrades it.
Noise figure:
(S/N )i
F = Noise figure =
(S/N )o

274
 CHAPTER 9

LTI Systemswith Random Inputs

9.1 Introduction

In application of random process, the input-output relation through linear system can
be described as follows.

𝑋(𝑡) ℎ(𝑡) 𝑌(𝑡) = 𝑋(𝑡)*ℎ(𝑡)

LTI System
𝑋(ω) 𝑌(ω)=𝑋(ω). 𝐻( ω )
𝐻( ω)

Here X(t) is a random process and h(t) (deterministic function) is the impulse
response of the linear system (Filter or another linear system).

9.1.1 Input-Output relation

1. Time domain: The output is the time domain is convolution of the input random
process X(t) and impulse response h(t) i.e.,

Z∞ Z∞
y(t) = X(t) ∗ h(t) = X(τ )h(t − τ ) dτ = h(τ )X(t − τ ) dτ
τ =−∞ τ =−∞

Q: Can you evaluate this convolution integral?

A: In general, we can not evaluate this convolution integral, because X(t) is
random process and there is no mathematical expression for X(t).

2. Frequency domain: The output in the frequency domain is the product of the input
Fourier transform of the impulse response X(t) is X(f ) and the fourier transform
of the impulse response h(t) is H(f ).
R∞
X(f ) = X(t)e−j2πf t dt =⇒ FT of the the input r.p X(t) is a r.p
−∞

R∞
H(f ) = h(t)e−j2πf t dt =⇒ FT of the the deterministic impulse response
−∞

Y (f ) = X(f )H(f ) or Y (ω) = X(ω)H(ω)

275
 Q: Can you evaluate the Fourier transform of input random porocess x(t), X(f )?
A: In genertal NO. Since, the X(t) is random in general and has no mathematical
expression.
Q: How can we describe the behavior of the random process and the output ran-
dom process through a linear system?
A: Case 1: Using auto-correlation function of random process X(t), RXX (τ ).
assume a WSS (constant mean and RXX (τ ) function is deterministic and only a
function of ‘τ ’.

RXX (τ ) = E[X(t)X(t + τ )]

The auto-correlation tell us how the random process is varying. It is a slow vary-
ing/fast varying process.
𝑅𝑋𝑋 (τ)

Slow varying r.p

Fast varying r.p

Case 2: Using power spectral density.

RXX (τ ) ←→ SXX (ω)

NOTE: The inout and output of the linear system as shown below in time and
frequency domain assuming the random process X(t) is WSS.
Random Function Random Function

𝑋(𝑡) ℎ(𝑡) 𝑌(𝑡) = 𝑋(𝑡)*ℎ(𝑡)

LTI System
𝑅𝑋𝑋 (τ) 𝑅YY (τ)
𝐻( ω)

None Random None Random

Deterministic Function Deterministic Function

9.1.2 Response of LTI system in time domain

1. Response of LTI system for mean value

Let X(t) is a random process with mean E[X(t)] and the response of the system

276
 for mean value can be written as
Z∞
y(t) = X(t) ∗ h(t) = h(τ )X(t − τ ) dτ
τ =−∞

Take expectation on both sides,

 ∞ 
 Z  Z∞
E[Y (t)] = E h(τ )X(t − τ ) dτ = h(τ )E {X(t − τ )} dτ
 
τ =−∞ τ =−∞

If X(t) be the WSS process, E[X(t)] = E[X(t + τ )]

Z∞
∴ E[Y (t)] == X(t) h(τ ) dτ
τ =−∞

The mean value of Y (t) is the multiplication of mean value of X(t) and the area
under the impulse response.

2. Response of LTI system for mean-square value

Let the output of LTI system,

Z∞
y(t) = X(t) ∗ h(t) = h(τ )X(t − τ ) dτ
τ =−∞

The mean-square value can be written as

 ∞ 2 
 Z 
E[Y 2 (t)] = E  h(τ )X(t − τ ) dτ 
 
τ =−∞
 ∞ 
 Z Z∞ 
=E h(τ )X(t − τ ) dτ · h(τ )X(t − τ ) dτ
 
τ =−∞ τ =−∞
 ∞ 
 Z Z∞ 
=E h(τ1 )X(t − τ1 ) dτ1 · h(τ2 )X(t − τ2 ) dτ2
 
τ1 =−∞ τ2 =−∞
Z∞ Z∞
= E[X(t − τ1 ) X(t − τ2 )]h(τ1 )h(τ2 ) dτ1 dτ1
τ1 =−∞ τ2 =−∞
Z∞ Z∞
= RXX (τ1 − τ2 )h(τ1 )h(τ2 ) dτ1 dτ2
τ1 =−∞ τ2 =−∞

Let τ1 = τ2 = τ

277
 Z∞ Z∞
2
E[Y (t)] = RXX (0)h(τ )h(τ ) dτ dτ
τ =−∞ τ =−∞

3. Response of LTI system for auto-correlation function (ACF)

R∞
Let the output function Y (t) = h(τ )X(t − τ ) dτ
τ =−∞
The auto-correlation of X(t) and Y (t) is

RY Y (τ ) = E[Y (t)Y (t + τ )]
( Z∞ Z∞ )
=E h(τ1 )X(t − τ1 ) dτ1 · h(τ2 )X(t − τ2 ) dτ2
τ1 =−∞ τ2 =−∞
| {z } | {z }
Y (t) Y (t+τ )
Z∞ Z∞
= E[X(t − τ1 )X(t + τ − τ2 )] h(τ1 )h(τ2 ) dτ1 dτ2
τ1 =−∞ τ2 =−∞
Z∞ Z∞
= RXX (τ + τ1 − τ2 ) h(τ1 )h(τ2 ) dτ1 dτ2
τ1 =−∞ τ2 =−∞

∴ RY Y (τ ) = RXX (τ ) ∗ h(−τ ) ∗ h(τ )

Other method:
( Z∞ Z∞ )
RY Y (τ ) = E h(τ )X(t − τ ) dτ · h(t + τ )X(t) dτ
τ =−∞ τ =−∞
Z∞ Z∞
= E[X(t)X(t − τ )] h(τ )h(t + τ ) dτ dτ
τ =−∞ τ =−∞

Z∞ Z∞
∴ RY Y (τ ) = RXX (τ )h(τ )h(t + τ ) dτ dτ
τ =−∞ τ =−∞

4. Cross-correlation function of input and output

The cross-correlation of X(t) and Y (t) is

RXY (τ ) = E[X(t)Y (t + τ )]
( Z∞ )
= E X(t) h(τ1 )X(t + τ − τ1 ) dτ1
τ1 =−∞

= E [X(t)X(t + τ − τ1 )] h(τ1 ) dτ1

278
 = RXX (τ − τ1 )h(τ! ) dτ1
∴ RXY (τ ) = RXX (τ ) ∗ h(τ )

9.1.3 Response of LTI system in frequency domain

1. Response of LTI systen for PSD

𝑋(𝑡) ℎ(𝑡) 𝑌(𝑡) = 𝑋(𝑡)*ℎ(𝑡)

LTI System
𝑅𝑋𝑋 (τ) 𝐻( ω) 𝑅YY (τ)

𝑆XX(ω) 𝑆𝑌𝑌(ω)

Z∞
RY Y (τ )e−jωτ dτ
 
The output PSD SY Y (ω) = F RY Y (τ ) = (9.1)
τ =−∞

 
ACF RY Y (ω) = E Y (t)Y (t + τ ) (9.2)

Z∞
Y (t) = X(t) ∗ h(t) = h(α1 )X(t − α1 ) dα1
α1 =−∞

Z∞
Y (t + τ ) = X(t + τ ) ∗ h(t + τ ) = h(α2 )X(t + τ − α2 ) dα2
α2 =−∞

From equation (9.2)

( Z∞ Z∞ )
RY Y (τ ) = E h(α1 )X(t − α1 ) dα1 · h(α2 )X(t + τ − α2 ) dα2
α1 =−∞ α2 =−∞
Z∞ Z∞
= h(α1 ) dα1 · h(α2 ) dα2 · E[X(t − α1 )X([t + τ ] − α2 )]
α1 =−∞ α2 =−∞

let T = t − α1 ; T + τ = t + τ − α2
T + τ − T ⇒ (t + τ ) − α2 − (t − α1 ) = τ + α1 − α2
Z∞ Z∞
RY Y (τ ) = h(α1 ) dα1 · h(α2 ) dα2 · RXX (τ + α1 − α2 )
α1 =−∞ α2 =−∞

From equation (9.1)

Z∞
SY Y (ω) = RY Y (τ )e−jωτ dτ
τ =−∞

279
 Z∞ Z∞ Z∞
 

=  h(α1 ) dα1 · h(α2 ) dα2 · RXX (τ + α1 − α2 )e−jωτ dτ 

τ =−∞ α1 =−∞ α2 =−∞

let α = τ + α1 − α2
Z∞ Z∞ Z∞
= h(α1 ) dα1 · h(α2 ) dα2 · RXX (α)e−jω(α−α1 +α2 ) dα
α1 =−∞ α2 =−∞ τ =−∞
Z∞ Z∞ Z∞
= h(α1 ) dα1 · h(α2 ) dα2 · RXX (α)e−jω(α) dα
α1 =−∞ α2 =−∞ α=−∞

= H(−ω) · H(ω) · SXX (ω)

2
= H(ω) · SXX (ω)
2
∴ SY Y (ω) = SXX (ω) · H(ω)

Alternate Method:
From response of LTI system for ACF
RY Y (τ ) = RXX (τ ) ∗ h(−τ ) ∗ h(τ )
↓ ↓ ↓ ↓ ↓ ↓
SY Y (ω) = SXX (ω) · H ∗ (ω) · H(ω)
2
∴ SY Y (ω) = SXX (ω) H(τ )

2. Power calculation at input and output of LTI system

Total power of the input

Z∞
PXX = E[X 2 (t)] = RXX (0) = SXX (f ) df
f =−∞

Total power of the output

Z∞ Z∞
2
PY Y = E[Y 2 (t)] = RY Y (0) = SY Y (f ) df = SXX (f ) H(f ) df
f =−∞ −∞

Problem 1: Let X(t) be the random process with PSD SXX (ω) is shown in Fig.
Find the output power of a LTI system whose frequency response is

1; |ω| ≤ ωc
H(ω) =
0; otherwise

Solution: (i) Average power in-terms of angular frequency

280
 Z∞ Zωc
1 1 η
PY Y = SXX (ω) dω = dω
2π 2π 2
ω=−∞ ω=−ωc
1 η  ωc 1 η 
= × ω −ωc = × 2ωc
2π 2 2π 2
ηωc
= W atts/rad/sec or V 2 /rad/sec
2π

(ii) Average power in-terms of linear frequency

Z∞ Zfc
η
PY Y = SXX (f ) df = df
2
f =−∞ f =−fc
η  fc η 
= f −fc = 2fc
2 2
= ηfc W atts/Hz or V 2 /Hz

Problem 2: Let X(t) be the random process with PSD SXX (ω) is shown in Fig.
Find the output power of a LTI system whose frequency response is

1; (ωc − B2 ) ≤ |ω| ≤ (ωc + B2 )
H(ω) =
0; otherwise

Solution: (i) Average power in-terms of angular frequency

Z∞
1
PY Y = SXX (ω) dω
2π
ω=−∞
ωc + B
2
Z
1 η
= ×2 dω
2π 2
ω=ωc − B
2

1 η h iωc + B2
= ×2× ω
2π 2 ωc − B2
"   #
1 η B B
= × ωc + − ωc −
2π 2 2 2
η
= B W atts/rad/sec or V 2 /rad/sec
2π

(ii) Average power in-terms of linear frequency

Z∞
1
PY Y = SXX (f ) df
2π
f =−∞
fc + B
2
Z
η
=2 df
2
f =fc − B
2

281
 η h ifc + B2
=2× f
2 fc − B
2
"   #
η B B
= fc + − fc −
2 2 2
= ηB W atts/Hz or V 2 /Hz

3
Problem 3: A random noise X(t) having power spectrum SXX (ω) = is
49 + ω 2
applied to a network for which h(t) = t2 Exp(−7t). The network response is denoted
by Y (t).

1. Find the average power of X(t)

2. Find the power spectrum of Y (t)

3. Find the average power of Y (t)

Solution:
Z∞
1
PY Y = SXX (ω) dω
2π
ω=−∞
Z∞
1 3
= dω
2π 49 + ω 2
ω=−∞
3 1 ω  ∞
= × tan−1
2π 7 7 ω=−∞
3 1 h π  π i
= × − −
2π 7 2 2
3 1
= × ×π
2π 7
3
= W atts
14

3
∴ PXX = W atts
14

(ii) Given h(t) = t2 Exp(−7t)

Z∞
H(ω) = F [h(t)] = t2 Exp(−7t) · e−jωt dt
ω=−∞
Z∞
= t2 e−(7+jω)t dt
ω=−∞
x
let x = (7 + jω)t =⇒ t = ;
7 + jω

282
 dx
dx = (7 + jω) dt =⇒ dt =
7 + jω
if x = ∞ ⇒ t = ∞; x = −∞ ⇒ t = −∞
Z∞
x2 dx
∴ H(ω) = 2
· e−x ·
(7 + jω) 7 + jω
x=−∞
Z∞ Z∞
1 −x 2
= e x dx ∵ e−x xn−1 dx = Γ(n)
(7 + jω)3
x=−∞ x=−∞
Z∞
1
= e−x x3−1 dx ∵ Γ(n) = nΓ(n − 1)
(7 + jω)3
x=−∞
1
= Γ(3) ∵ Γ(n + 1) = nΓ(n) = n!
(7 + jω)3
1
= Γ(2 + 1)
(7 + jω)3
1
× 2!
(7 + jω)3
2
∴ H(ω) =
(7 + jω)3

2
2 2
H(ω) =
(7 + jω)3
4
=
3 2

(49 + ω 2 ) 2
4
=
(49 + ω 2 )3

2
∴ SY Y (ω) = SXX (ω) · H(ω)
3 4
= ·
49 + ω 2 (49 + ω 2 )3
12
=
(49 + ω 2 )4

12
∴ SY Y (ω) =
(49 + ω 2 )4
(iii)
Z∞
1
PY Y (ω) = SXX (ω)
2π
ω=−∞
Z∞
1 12
= dω
2π (49 + ω 2 )4
ω=−∞

283
 let ω = 7 tan θ =⇒ dω = 7 sec2 θ
π
let ω = −∞ ⇒ θ = tan−1 (−∞) = − ;
2
−1 π
ω = ∞ ⇒ θ = tan (∞) =
2
π
Z 2
1 12
PY Y = 2 4
· 7 sec2 θ dθ
2π (1 + tan θ)
ω=− π2
π
Z2
1 12
= · 7 sec2 θ dθ
2π (sec2 θ)4
ω=− π2
π
Z2
12 × 7 1
= dθ
2π sec6 θ
ω=− π2
π
Z2
12 × 7 × 2
= cos6 θ dθ ⇒ 1
2π
ω=0

We know that

Z∞  n−1 · n−3
n n−2
· . . . 12 · π2 ; ‘n’ even
cosn θ dθ =
 n−1 · n−3
· . . . 23 ; ‘n’ odd
0 n n−2

Zπ/2
6−1 6−3 6−5 π
cos6 θ dθ = · · ·
6 6−2 6−4 2
0
5 3 1 π
= · · ·
6 4 2 2
12 × 7 5 3 1 π
1 ⇒ PY Y (ω) = × · · ·
2π 6 4 2 2
7×5×3 105
= = = 13.125 W atts
2×4 8

∴ PY Y = 13.125 W atts

Problem: 4 A random voltage modeled by a white noise process X(t) which power
spectral density η2 ia an input to RC network shown in Fig. Find

1. Output PSD SY Y (ω)

2. Auto-correlation function RY Y (τ )

3. Average output power [Y 2 (t)]

Solution: The frequency response of the system is given by

284
 1
jωC 1
H(ω) = 1 =
R+ jωC
1 + jωRC

(a)
2
SY Y (ω) = H(ω) SXX (ω)
1 N0
= ×
12 2 2
+ω R C 2 2

(b) Taking inverse Fourier transform both sides

N0 |τ |
RY Y (τ ) = e RC
4RC

(c) Average output power

N0
E[Y 2 (t)] = RY Y (0) =
4RC

10−4 ; |f | < 100
Problem 5: A WSS r.p X(t) with PSD SXX (f ) = is
0; otherwise
1
the input an RC filter with the frequency response H(f ) = 100π+j2πf . The filter output
is the stochastic process Y (t). What is the

(a) E[X 2 (t)] (b) SXY (f ) (c) SY X (f ) (d) SY Y (f ) (e) E[Y 2 (t)]
Solution:
(a) We know that

Z∞ Z∞
1 jωτ
RXX (τ ) = SXX (ω)e dω = SXX (f )ej2πf τ df
2π
−∞ −∞

Z∞
If τ = 0 RXX (0) = E[X 2 (t)] = SXX (f )e0 df
−∞

Mean square value

Z∞
2
E[X (t)] = SXX (f ) df
−∞
Z100 h i100
−4 −4
= 10 df = 10 f = 10−4 (200) = 0.02
−100
−100

∴ E[X 2 (t)] = 0.02

285
 
10−4 H(f ); |f | ≤ 100
(b) SXY (f ) = H(f )SXX (f ) =
0; otherwise

10−4

100π+j2πf
; |f | ≤ 100
∴ SXY (f ) =
0; otherwise

∗ ∗
(c) SY X (f ) = SXY (f ) and we know that RY X (τ ) = RXY (−τ )

10−4 H ∗ (f ); |f | ≤ 100
∗
SY X (f ) = SXY (f ) =
0; otherwise


10−4

100π−j2πf
; |f | ≤ 100
∴ SY X (f ) =
0; otherwise

(d) SY Y (f ) = H ∗ (f )SXY (f ) = |H(f )|2 SXX (f )


10−4

104 π 2 +(2πf )2
; |f | ≤ 100
∴ SY X (f ) =
0; otherwise

(e)

Z∞
E[Y 2 (t)] = SY Y (f ) df
−∞
Z100
10−4
= df
104 π 2 + 4π 2 f 2
−100
Z100
2 df
= 8 2
10 π f 2


1+ 50
0
 100
 Z
2 −1 f dx 1 −1 x
 
= 8 2 tan ∵ = tan
10 π 50 0 x 2 + a2 a a
2  −1
= 8 2 tan (2) − tan−1 (0)

10 π
2
= 8 2 × 63.4349
10 π
= 12.584 × 108
∴ E[Y 2 (t)] = 12.584 × 108

Problem 6: Let X(t) is a WSS Gaussian r.p with mean X(t) = 0 and auto-
correlation function RXX (τ ) = 10δ(τ ). where δ(·) is a Dirac delta function. The

286