PROBABILITY AND RANDOM

PROCESSES
(R20A0408)
LECTURE NOTES

B.TECH
(II YEAR – II SEM)
(2022-23)

Prepared by:
Mr. K.Suresh, Assistant Professor
Mr. K. Rasool Reddy, Assistant Professor

Department of Electronics and Communication Engineering

MALLA REDDY COLLEGE

OF ENGINEERING & TECHNOLOGY
(Autonomous Institution – UGC, Govt. of India)
Recognized under 2(f) and 12 (B) of UGC ACT 1956
(Affiliated to JNTUH, Hyderabad, Approved by AICTE - Accredited by NBA & NAAC – ‘A’ Grade - ISO 9001:2015 Certified)
Maisammaguda, Dhulapally (Post Via. Kompally), Secunderabad – 500100, Telangana State, India
 CONTENTS

 SYLLABUS

 UNIT-I-PROBABILITY AND RANDOM VARIABLE

 UNIT-II- DISTRIBUTION AND DENSITY FUNCTIONS AND OPERATIONS ON

ONE RANDOM VARIABLE

 UNIT-III-MULTIPLE RANDOM VARIABLES AND OPERATIONS

 UNIT-IV-STOCHASTIC PROCESSES-TEMPORAL CHARACTERISTICS

 UNIT-V- STOCHASTIC PROCESSES-SPECTRAL CHARACTERISTICS

 UNITWISE IMPORTANT QUESTIONS

 PROBABILITY AND RANDOM PROCESSES

Course Objectives:

 To provide mathematical background and sufficient experience so that student can read,
write and understand sentences in the language of probability theory.
 To introduce students to the basic methodology of “probabilistic thinking” and apply it to
problems.
 To understand basic concepts of Probability theory and Random Variables, how to deal
with multiple Random Variables.
 To understand the difference between time averages statistical averages.
 To teach students how to apply sums and integrals to compute probabilities, and
expectations.
UNIT I:
Probability and Random Variable
Probability: Set theory, Experiments and Sample Spaces, Discrete and Continuous Sample
Spaces, Events, Probability Definitions and Axioms, Joint Probability, Conditional Probability,
Total Probability, Bayes’ Theorem, and Independent Events, Bernoulli’s trials.
The Random Variable: Definition of a Random Variable, Conditions for a Function to be a
Random Variable, Discrete and Continuous.

UNIT II:
Distribution and density functions and Operations on One Random Variable
Distribution and density functions: Distribution and Density functions, Properties, Binomial,
Uniform, Exponential, Gaussian, and Conditional Distribution and Conditional Density function
and its properties, problems.
Operation on One Random Variable: Expected value of a random variable, function of a
random variable, moments about the origin, central moments, variance and skew, characteristic
function, moment generating function.

UNIT III:
Multiple Random Variables and Operations on Multiple Random Variables
Multiple Random Variables: Joint Distribution Function and Properties, Joint density Function
and Properties, Marginal Distribution and density Functions, conditional Distribution and density
Functions, Statistical Independence, Distribution and density functions of Sum of Two Random
Variables.
Operations on Multiple Random Variables: Expected Value of a Function of Random
Variables, Joint Moments about the Origin, Joint Central Moments, Joint Characteristic
Functions, and Jointly Gaussian Random Variables: Two Random Variables case Properties.
UNIT IV:
Stochastic Processes-Temporal Characteristics: The Stochastic process Concept,
Classification of Processes, Deterministic and Nondeterministic Processes, Distribution and
Density Functions, Statistical Independence and concept of Stationarity: First-Order Stationary
Processes, Second-Order and Wide-Sense Stationarity, Nth-Order and Strict-Sense Stationarity,
Time Averages and Ergodicity, Mean-Ergodic Processes, Correlation-Ergodic Processes
Autocorrelation Function and Its Properties, Cross-Correlation Function and Its Properties,
Covariance Functions and its properties.
Linear system Response: Mean and Mean-squared value, Autocorrelation, Cross-Correlation
Functions.

UNIT V:
Stochastic Processes-Spectral Characteristics: The Power Spectrum and its Properties,
Relationship between Power Spectrum and Autocorrelation Function, the Cross-Power Density
Spectrum and Properties, Relationship between Cross-Power Spectrum and Cross-Correlation
Function.
Spectral characteristics of system response: power density spectrum of response, cross power
spectral density of input and output of a linear system

TEXT BOOKS:

1. Probability, Random Variables & Random Signal Principles -Peyton Z. Peebles, TMH,
4th Edition, 2001.
2. Probability and Random Processes-Scott Miller, Donald Childers,2Ed,Elsevier,2012

REFERENCE BOOKS:

1. Theory of probability and Stochastic Processes-Pradip Kumar Gosh, University Press

2. Probability and Random Processes with Application to Signal Processing - Henry Stark
and John W. Woods, Pearson Education, 3rd Edition.
3. Probability Methods of Signal and System Analysis- George R. Cooper, Clave D. MC
Gillem, Oxford, 3rd Edition, 1999.
4. Statistical Theory of Communication -S.P. Eugene Xavier, New Age Publications 2003
5. Probability, Random Variables and Stochastic Processes Athanasios Papoulis and
S.Unnikrishna Pillai, PHI, 4th Edition, 2002.
 UNIT I
Probability and Random Variable
Probability:

 Set theory
 Experiments
 Sample Spaces, Discrete and Continuous Sample Spaces
 Events
 Probability Definitions and Axioms
 Joint Probability
 Conditional Probability
 Total Probability
 Bayes’ Theorem
 Independent Events
 Bernoulli’s trials

Random Variable:

 Definition of a Random Variable

 Conditions for a Function to be a Random Variable
 Discrete and Continuous Random Variables
 UNIT – 1
PROBABILITY AND RANDOM VARIABLE

PROBABILITY

Introduction

It is remarkable that a science which began with the consideration of games of chance
should have become the most important object of human knowledge. Probability is simply how
likely something is to happen. Whenever we’re unsure about the outcome of an event, we can
talk about the probabilities of certain outcomes —how likely they are. The analysis of events
governed by probability is called statistics. Probability theory, a branch
of mathematics concerned with the analysis of random phenomena. The outcome of a random
event cannot be determined before it occurs, but it may be any one of several possible
outcomes. The actual outcome is considered to be determined by chance.

How to Interpret Probability

Mathematically, the probability that an event will occur is expressed as a number between 0 and 1.
Notationally, the probability of event A is represented by P (A).

 If P (A) equals zero, event A will almost definitely not occur.

 If P (A) is close to zero, there is only a small chance that event A will occur.
 If P (A) equals 0.5, there is a 50-50 chance that event A will occur.
 If P(A) is close to one, there is a strong chance that event A will occur.
 If P(A) equals one, event A will almost definitely occur.

In a statistical experiment, the sum of probabilities for all possible outcomes is equal to one. This
means, for example, that if an experiment can have three possible outcomes (A, B, and C), then
P(A) + P(B) + P(C) = 1.
 Applications

Probability theory is applied in everyday life in risk assessment and in trade on financial markets.
Governments apply probabilistic methods in environmental regulation, where it is called pathway
analysis

Another significant application of probability theory in everyday life is reliability. Many consumer
products, such as automobiles and consumer electronics, use reliability theory in product design to
reduce the probability of failure. Failure probability may influence a manufacturer's decisions on a
product's warranty.

The range of applications extends beyond games into business decisions, insurance, law, medical tests,
and the social sciences The telephone network, call centers, and airline companies with their randomly
fluctuating loads could not have been economically designed without probability theory.

Uses of Probability in real life

 Sports – be it basketball or football or cricket a coin is tossed and both teams have 50/50 chances
of winning it.

 Board Games – The odds of rolling one die and getting and even number there is a 50% chance
since three of the six numbers on a die are even.

 Medical Decisions – When a patient is advised to undergo surgery, they often want to know the
success rate of the operation which is nothing but a probability rate. Based on the same the patient
takes a decision whether or not to go ahead with the same.

 Life Expectancy – this is based on the number of years the same groups of people have lived in the past.

 Weather – when planning an outdoor activity, people generally check the probability of rain.
Meteorologists also predict the weather based on the patterns of the previous year, temperatures and
natural disasters are also predicted on probability and nothing is ever stated as a surety but a possibility
and an approximation.
SET THEORY:

Set: A set is a well defined collection of objects. These objects are called elements or members of the
set. Usually uppercase letters are used to denote sets.

The set theory was developed by George Cantor in 1845-1918. Today, it is used in almost every
branch of mathematics and serves as a fundamental part of present-day mathematics.

In everyday life, we often talk of the collection of objects such as a bunch of keys, flock of birds,
pack of cards, etc. In mathematics, we come across collections like natural numbers, whole numbers,
prime and composite numbers.

We assume that,

● the word set is synonymous with the word collection, aggregate, class and comprises of elements.

● Objects, elements and members of a set are synonymous terms.

● Sets are usually denoted by capital letters A, B, C, ...... , etc.

● Elements of the set are represented by small letters a, b, c, ..... , etc.

If ‘a’ is an element of set A, then we say that ‘a’ belongs to A and it is mathematically
represented as aϵ A
If ‘b‘ is an element which does not belong to A, we represent this as b ∉ A.

Examples of sets:

1. Describe the set of vowels.

If A is the set of vowels, then A could be described as A = {a, e, i, o, u}.

2.Describe the set of positive integers.

Since it would be impossible to list all of the positive integers, we need to use a rule to describe this
set. We might say A consists of all integers greater than zero.

3. Set A = {1, 2, 3} and Set B = {3, 2, 1}. Is Set A equal to Set B?

Yes. Two sets are equal if they have the same elements. The order in which the elements are listed
does not matter.

4. What is the set of men with four arms?

Since all men have two arms at most, the set of men with four arms contains no elements. It is the
null set (or empty set).
5. Set A = {1, 2, 3} and Set B = {1, 2, 4, 5, 6}. Is Set A a subset of Set B?

Set A would be a subset of Set B if every element from Set A were also in Set B. However, this is
not the case. The number 3 is in Set A, but not in Set B. Therefore, Set A is not a subset of Set B

Some important sets used in mathematics are

N: the set of all natural numbers = {1, 2, 3, 4, ...... }

Z: the set of all integers = {....., -3, -2, -1, 0, 1, 2, 3, ..... }

Q: the set of all rational numbers

R: the set of all real numbers

Z+: the set of all positive integers

W: the set of all whole numbers

Types of sets:

1. Empty Set or Null Set:

A set which does not contain any element is called an empty set, or the null set or the void set and it
is denoted by ∅ and is read as phi. In roster form, ∅ is denoted by {}. An empty set is a finite set,
since the number of elements in an empty set is finite, i.e., 0.

For example: (a) the set of whole numbers less than 0.

(b) Clearly there is no whole number less than 0.

Therefore, it is an empty set.

(c) N = {x : x ∈ N, 3 < x < 4}

• Let A = {x : 2 < x < 3, x is a natural number}

Here A is an empty set because there is no natural number between

2 and 3
 2. Singleton Set:

A set which contains only one element is called a singleton set.

For example:

• A = {x : x is neither prime nor composite}

It is a singleton set containing one element, i.e., 1.

• B = {x : x is a whole number, x < 1}

This set contains only one element 0 and is a singleton set.

• Let A = {x : x ∈ N and x² = 4}

Here A is a singleton set because there is only one element 2 whose square is 4.

• Let B = {x : x is a even prime number}

Here B is a singleton set because there is only one prime number which is even, i.e., 2.

3. Finite Set:

A set which contains a definite number of elements is called a finite set. Empty set is also called a
finite set.

For example:

• The set of all colors in the rainbow.

• N = {x : x ∈ N, x < 7}

• P = {2, 3, 5, 7, 11, 13, 17, ....... 97}

4. Infinite Set:

The set whose elements cannot be listed, i.e., set containing never-ending elements is called an
infinite set.

For example:

• Set of all points in a plane

• A = {x : x ∈ N, x > 1}

• Set of all prime numbers

• B = {x : x ∈ W, x = 2n}