Chapter 1: Probability
• Sample Space.
• Events.
• Counting Techniques.
• Probability of an Event.
• Additive Rules.
• Conditional Probability.
• Independence, and the Product Rule.
• Total Probability Rule.
• Bayes’ Rule.
2
Total Probability Rule (1/10)
Total Probability Rule:
𝑷 𝑨 = 𝑷 𝑬 ∩ 𝑨 ∪ 𝑷 𝑬′ ∩ 𝑨
= 𝑷 𝑬 ∩ 𝑨 + 𝑷 𝑬′ ∩ 𝑨
= 𝑷 𝑨 𝑬 𝑷 𝑬 + 𝑷 𝑨 𝑬′ 𝑷 𝑬′
3
Total Probability Rule (2/10)
Example10:
Consider the information about contamination in the
following table.
𝑃 𝐹 𝐻 = 0.1
𝑃 𝐹 𝐻′ = 0.005
𝑃(𝐻) = 0.2
𝑃(𝐻′) = 0.8
𝑃 𝐹 ?
Let F denote the event that the product fails, and let H
denote the event that the chip is exposed to high levels of
contamination.
4
Total Probability Rule (3/10)
Example10:
Consider the information about contamination in the
following table.
𝑃 𝐹 𝐻 = 0.1
𝑃 𝐹 𝐻′ = 0.005
𝑃(𝐻) = 0.2
𝑃(𝐻′) = 0.8
𝑃 𝐹 ?
𝑃 𝐹 = 𝑃 𝐹 𝐻 𝑃 𝐻 + 𝑃 𝐹 𝐻′ 𝑃 𝐻′
= 0.1 0.2 + 0.005 0.8 = 0.024
5
Total Probability Rule (4/10)
Total Probability Rule (Multiple Events):
𝑃(𝐵) = 𝑃(𝐵 ∩ 𝐸1 ) + 𝑃(𝐵 ∩ 𝐸2 ) + 𝑃(𝐵 ∩ 𝐸3 ) + 𝑃(𝐵 ∩ 𝐸4 )
6
Total Probability Rule (5/10)
Example11:
In a certain assembly plant, three machines, 𝐵1 , 𝐵2 , and 𝐵3 ,
make 30%, 45%, and 25%, respectively, of the products. It
is known from past experience that 2%, 3%, and 2% of the
products made by each machine, respectively, are
defective. Now, suppose that a finished product is
randomly selected. What is the probability that it is
defective?
7
Total Probability Rule (5/10)
Example11:
In a certain assembly plant, three machines, 𝐵1 , 𝐵2 , and 𝐵3 ,
make 30%, 45%, and 25%, respectively, of the products. It
is known from past experience that 2%, 3%, and 2% of the
products made by each machine, respectively, are
defective. Now, suppose that a finished product is
randomly selected. What is the probability that it is
defective?
𝑫: the product is defective. Find 𝑷(𝑫) ?
8
Total Probability Rule (6/10)
Example11: 𝑃 𝐵1 = 0.3, 𝑃 𝐵2 = 0.45, 𝑃 𝐵3 = 0.25
In a certain assembly plant, three machines, 𝐵1 , 𝐵2 , and 𝐵3 ,
make 30%, 45%, and 25%, respectively, of the products. It
is known from past experience that 2%, 3%, and 2% of the
products made by each machine, respectively, are
defective. Now, suppose that a finished product is
randomly selected. What is the probability that it is
defective?
𝑃 𝐷|𝐵1 = 0.02,
𝑃 𝐷|𝐵2 = 0.03,
𝑃 𝐷|𝐵3 = 0.02.
9
Total Probability Rule (7/10)
Example11: 𝑃 𝐵1 = 0.3, 𝑃 𝐵2 = 0.45, 𝑃 𝐵3 = 0.25
𝑃 𝐷|𝐵1 = 0.02,
𝑃 𝐷|𝐵2 = 0.03,
𝑃 𝐷|𝐵3 = 0.02.
Applying the total probability rule , we can write
𝑃 𝐷
= 𝑃 𝐷 𝐵1 𝑃 𝐵1 + 𝑃 𝐷 𝐵2 𝑃 𝐵2 + 𝑃 𝐷 𝐵3 𝑃(𝐵3 )
= 0.02 0.3 + 0.03 0.45 + 0.02 0.25 = 0.0245
10
Total Probability Rule (8/10)
Example12:
Box#1 contains 2 red balls and 3 blue balls; Box#2
contains 5 red balls and 2 blue balls. If the selection of two
boxes is equally likely, and you selected one ball, what is
the probability that it is red?
11
Total Probability Rule (8/10)
Example12:
Box#1 contains 2 red balls and 3 blue balls; Box#2
contains 5 red balls and 2 blue balls. If the selection of two
boxes is equally likely, and you selected one ball, what is
the probability that it is red?
Box#1 Box#2
12
Total Probability Rule (9/10)
Example12:
Box#1 contains 2 red balls and 3 blue balls; Box#2
contains 5 red balls and 2 blue balls. If the selection of two
boxes is equally likely, and you selected one ball, what is
the probability that it is red?
Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
Find 𝑃(𝑅) ?
13
Total Probability Rule (10/10)
Example12: Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
𝑃 𝑅 𝐵1 = 2ൗ5 = 0.4
𝑃 𝑅 𝐵2 = 5ൗ7 = 0.7143
𝑃 𝑅 = 𝑃 𝑅 𝐵1 𝑃 𝐵1 + 𝑃 𝑅 𝐵2 𝑃 𝐵2
𝑃 𝑅 = 0.4 0.5 + 0.7143 0.5 = 0.55715
14
Bayes’ Rule (1/11)
From the definition of conditional probability,
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴|𝐵)𝑃(𝐵) = 𝑃(𝐵|𝐴)𝑃(𝐴)
Now, considering the second and last terms in the
preceding expression, we can write
15
Bayes’ Rule (2/11)
If 𝐸1 , 𝐸2 , … , 𝐸𝑘 are 𝑘 mutually exclusive and exhaustive
events and 𝐵 is any event,
16
Bayes’ Rule (3/11)
Example1:
Box#1 contains 2 red balls and 3 blue balls; Box#2
contains 5 red balls and 2 blue balls. If the selection of two
boxes is equally likely, and the selected ball was red, what
is the probability that it is from Box#1?
Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
Find 𝑷(𝑩𝟏 |𝑹) ?
17
Bayes’ Rule (4/11)
Example1: Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
𝑃 𝑅 𝐵1 = 2ൗ5 = 0.4
𝑃 𝑅 𝐵2 = 5ൗ7 = 0.7143
18
Bayes’ Rule (5/11)
Example1: Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
𝑃 𝑅 𝐵1 = 2ൗ5 = 0.4
𝑃 𝑅 𝐵2 = 5ൗ7 = 0.7143
𝑃 𝑅 𝐵1 𝑃 𝐵1 0.4 0.5
𝑃 𝐵1 𝑅 = =
𝑃 𝑅 𝑃 𝑅
19
Bayes’ Rule (6/11)
Example1: Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
𝑃 𝑅 𝐵1 = 2ൗ5 = 0.4
𝑃 𝑅 𝐵2 = 5ൗ7 = 0.7143
𝑃 𝑅 𝐵1 𝑃 𝐵1 0.4 0.5
𝑃 𝐵1 𝑅 = =
𝑃 𝑅 𝑃 𝑅
20
Bayes’ Rule (7/11)
Example1: Box#1 (𝑩𝟏 ) Box#2 (𝑩𝟐 )
𝑃 𝐵1 = 𝑃 𝐵2 = 0.5
𝑅: 𝑟𝑒𝑎𝑑, 𝐵: 𝑏𝑙𝑢𝑒
𝑃 𝑅 𝐵1 = 2ൗ5 = 0.4
𝑃 𝑅 𝐵2 = 5ൗ7 = 0.7143
𝑃 𝑅 𝐵1 𝑃 𝐵1 0.4 0.5 0.2
𝑃 𝐵1 𝑅 = = = = 0.35897
𝑃 𝑅 𝑃 𝑅 0.55715
21
Bayes’ Rule (8/11)
Example2:
In a certain assembly plant, three machines, 𝐵1 , 𝐵2 , and 𝐵3 ,
make 30%, 45%, and 25%, respectively, of the products. It
is known from past experience that 2%, 3%, and 2% of the
products made by each machine, respectively, are
defective. If a product was chosen randomly and found to
be defective, what is the probability that it was made by
machine 𝐵3 ?
𝑫: the product is defective. Find 𝑷(𝑩𝟑 |𝑫) ?
22
Bayes’ Rule (8/11)
Example2: 𝑃 𝐵1 = 0.3, 𝑃 𝐵2 = 0.45, 𝑃 𝐵3 = 0.25
In a certain assembly plant, three machines, 𝐵1 , 𝐵2 , and 𝐵3 ,
make 30%, 45%, and 25%, respectively, of the products. It
is known from past experience that 2%, 3%, and 2% of the
products made by each machine, respectively, are
defective. If a product was chosen randomly and found to
be defective, what is the probability that it was made by
machine 𝐵3 ?
𝑃 𝐷|𝐵1 = 0.02,
𝑃 𝐷|𝐵2 = 0.03,
𝑃 𝐷|𝐵3 = 0.02.
23
Bayes’ Rule (9/11)
Example2: 𝑃 𝐵1 = 0.3, 𝑃 𝐵2 = 0.45, 𝑃 𝐵3 = 0.25
𝑃 𝐷|𝐵1 = 0.02,
𝑃 𝐷|𝐵2 = 0.03,
𝑃 𝐷|𝐵3 = 0.02.
Using Bayes’ rule to write
𝑃 𝐷 𝐵3 𝑃 𝐵3
𝑃 𝐵3 |𝐷 =
𝑃 𝐷
24
Bayes’ Rule (10/11)
Example2: 𝑃 𝐵1 = 0.3, 𝑃 𝐵2 = 0.45, 𝑃 𝐵3 = 0.25
𝑃 𝐷|𝐵1 = 0.02,
𝑃 𝐷|𝐵2 = 0.03,
𝑃 𝐷|𝐵3 = 0.02.
Using Bayes’ rule to write
𝑃 𝐷 𝐵3 𝑃 𝐵3
𝑃 𝐵3 |𝐷 =
𝑃 𝐷
25
Bayes’ Rule (11/11)
Example2: 𝑃 𝐵1 = 0.3, 𝑃 𝐵2 = 0.45, 𝑃 𝐵3 = 0.25
𝑃 𝐷|𝐵1 = 0.02,
𝑃 𝐷|𝐵2 = 0.03,
𝑃 𝐷|𝐵3 = 0.02.
Using Bayes’ rule to write
𝑃 𝐷 𝐵3 𝑃 𝐵3 0.02 0.25
𝑃 𝐵3 |𝐷 = = = 0.2041
0.0245 0.0245
26
Chapter 2: Random Variable
• Discrete Random Variables.
• Probability Mass Functions.
• Cumulative Distribution Functions.
• Discrete R.V. (Mean and Variance).
• Continuous Random Variables.
• Probability Density Functions.
• Continuous R.V. (Mean and Variance).
• Joint Probability Distributions.
27
Discrete Random Variables (1/3)
Random Variable
• Is a function that assigns a real number to each outcome
in the sample space of random experiment. Denoted by
an uppercase letter such as X
A Discrete Random Variable
• Is a random variable with a finite (or countable infinite)
range.
• The possible values of X may be listed as 𝑥1 , 𝑥2 , . . .
28
Discrete Random Variables (2/3)
Example1
• Flipping a coin of two times. Let 𝑋 is the number of
heads.
29
Discrete Random Variables (3/3)
Example1
• Flipping a coin of two times. Let 𝑋 is the number of
heads.
Answer:
𝑆 = 𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇
𝟐 𝟏 𝟏 𝟎
𝑥 = 0, 1, 2
1 2 1
𝑃 0 = , 𝑃 1 = , 𝑃 2 =
4 4 4
30
Probability Mass Fun. (1/14)
Probability Mass Function
𝑥𝑖 𝒙𝟏 𝒙𝟐 𝒙𝟑 𝒙𝟒 𝒙𝟓
𝑓 𝑥𝑖 = 𝑃(𝑥𝑖 ) 𝑷 𝒙𝟏 𝑷 𝒙𝟐 𝑷 𝒙𝟑 𝑷 𝒙𝟒 𝑷 𝒙𝟓
31
Probability Mass Fun. (2/14)
Example1
Verify that the function is a probability mass function:
𝒙 -2 -1 0 1 2
𝒇 𝒙 = 𝑷(𝑿 = 𝒙) 1/8 2/8 2/8 2/8 1/8
32
Probability Mass Fun. (3/14)
Example1
Verify that the function is a probability mass function:
𝒙 -2 -1 0 1 2
𝒇 𝒙 = 𝑷(𝑿 = 𝒙) 1/8 2/8 2/8 2/8 1/8
Answer:
𝑷 𝒙𝒊 = 𝟏 , 𝑷 𝒙𝒊 ≥ 𝟎
33
Probability Mass Fun. (4/14)
Example2 𝒙 -2 -1 0 1 2
𝒇 𝒙 = 𝑷(𝑿 = 𝒙) 1/8 2/8 2/8 2/8 1/8
Find:
a. 𝑷 𝑿 ≤ 2 b. 𝑷 𝑿 > −2
c. 𝑷 −1 ≤ 𝑿 ≤ 1 d. 𝑷 𝑿 ≤ −1 or 𝑿 = 2
34
Probability Mass Fun. (5/14)
Example2 𝒙 -2 -1 0 1 2
𝒇 𝒙 = 𝑷(𝑿 = 𝒙) 1/8 2/8 2/8 2/8 1/8
Answer:
a. 𝑷 𝑿 ≤ 2 = 1
7
b. 𝑷 𝑿 > −2 =
8
6
c. 𝑷 −1 ≤ 𝑿 ≤ 1 =
8
3 1 4
d. 𝑷 𝑿 ≤ −1 or 𝑿 = 2 = + =
8 8 8
35
Probability Mass Fun. (6/14)
Example3
Two balls are drawn in succession without replacement
from a box containing 4 red balls and 3 black balls. The
possible outcomes and the values 𝑦 of the random variable
𝑌 , where 𝑦 is the number of red balls, are
36
Probability Mass Fun. (7/14)
Example3
No Red Balls
4 3
𝑓 0 =𝑃 𝑌=0 = 0 2 = 3 =1
7 21 7
2
37
Probability Mass Fun. (7/14)
Example3
One Red Ball
4 3
𝑓 1 =𝑃 𝑌=1 = 1 1 = 12 = 4
7 21 7
2
38
Probability Mass Fun. (7/14)
Example3
Two Red Balls
4 3
𝑓 2 =𝑃 𝑌=2 = 2 0 = 6 =2
7 21 7
2
39
Probability Mass Fun. (7/14)
Example3
𝑦 0 1 2
𝑓 𝑦 = 𝑃(𝑌 = 𝑦) 1/7 4/7 2/7
40
Probability Mass Fun. (8/14)
Example4
A shipment of 20 similar laptop computers to a retail outlet
contains 3 that are defective. If a school makes a random
purchase of 2 of these computers, find the probability
distribution for the number of defectives.
Note: Let 𝑋 be a random variable whose values 𝑥 are the
possible numbers of defective computers purchased by the
school.
41
Probability Mass Fun. (8/14)
Example4
A shipment of 20 similar laptop computers to a retail outlet
contains 3 that are defective. If a school makes a random
purchase of 2 of these computers, find the probability
distribution for the number of defectives.
Note: Let 𝑋 be a random variable whose values 𝑥 are the
possible numbers of defective computers purchased by the
school. Then 𝒙 can only take the numbers 0, 1, and 2.
42
Probability Mass Fun. (9/14)
Example4
A shipment of 20 similar laptop computers to a retail outlet
contains 3 that are defective. If a school makes a random
purchase of 2 of these computers, find the probability
distribution for the number of defectives.
3 17
𝑓 0 =𝑃 𝑋=0 = 0 2 = 136
20 190
2
43
Probability Mass Fun. (10/14)
Example4
A shipment of 20 similar laptop computers to a retail outlet
contains 3 that are defective. If a school makes a random
purchase of 2 of these computers, find the probability
distribution for the number of defectives.
3 17
𝑓 1 =𝑃 𝑋=1 = 1 1 = 51
20 190
2
44
Probability Mass Fun. (11/14)
Example4
A shipment of 20 similar laptop computers to a retail outlet
contains 3 that are defective. If a school makes a random
purchase of 2 of these computers, find the probability
distribution for the number of defectives.
3 17
𝑓 2 =𝑃 𝑋=2 = 2 0 = 3
20 190
2
45
Probability Mass Fun. (12/14)
Example4
A shipment of 20 similar laptop computers to a retail outlet
contains 3 that are defective. If a school makes a random
purchase of 2 of these computers, find the probability
distribution for the number of defectives.
𝑥 0 1 2
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 136/190 51/190 3/190
46
Probability Mass Fun. (13/14)
Example5
There is a chance that a bit transmitted through a digital
transmission channel is received in error. Let 𝑋 equal the
number of bits in error in the next four bits transmitted. The
possible values for 𝑋 are {0, 1, 2, 3, 4}.
Suppose that the probabilities are
47
Probability Mass Fun. (14/14)
Example5
48
Cumulative Distribution (1/9)
The cumulative distribution function (cdf), denoted by
𝐹(𝑥), measures the probability that the random variable 𝑋
assumes a value less than or equal to 𝑥, that is,
49
Cumulative Distribution (2/9)
If 𝑋 is discrete, then
𝑥 -2 -1 0 1 2
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 1/8 2/8 2/8 2/8 1/8
50
Cumulative Distribution (3/9)
If 𝑋 is discrete, then
𝑥 -2 -1 0 1 2
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 1/8 2/8 2/8 2/8 1/8
𝑭 𝒙 =𝑷 𝑿≤𝒙 1/8 3/8 5/8 7/8 8/8
51
Cumulative Distribution (4/9)
Example1
𝑥 0 1 2 3 4
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 0.6561 0.2916 0.0486 0.0036 0.0001
52
Cumulative Distribution (5/9)
Example1
𝑥 0 1 2 3 4
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 0.6561 0.2916 0.0486 0.0036 0.0001
𝑭 𝒙 =𝑷 𝑿≤𝒙 0.6561 0.9477 0.9963 0.9999 1
53
Cumulative Distribution (6/9)
Example1
𝑥 0 1 2 3 4
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 0.6561 0.2916 0.0486 0.0036 0.0001
𝑭 𝒙 =𝑷 𝑿≤𝒙 0.6561 0.9477 0.9963 0.9999 1
54
Cumulative Distribution (7/9)
Example2
𝑥 0 1 2 3 4
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 1/16 4/16 6/16 4/16 1/16
55
Cumulative Distribution (8/9)
Example2
𝑥 0 1 2 3 4
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 1/16 4/16 6/16 4/16 1/16
𝑭 𝒙 =𝑷 𝑿≤𝒙 1/16 5/16 11/16 15/16 16/16
56
Cumulative Distribution (9/9)
Example2
𝑥 0 1 2 3 4
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 1/16 4/16 6/16 4/16 1/16
𝑭 𝒙 =𝑷 𝑿≤𝒙 1/16 5/16 11/16 15/16 16/16
57
