Applied Statistics
Lecture (3)
Basic Probability Concepts
Khaled H. Hamed Lecture 3 – Page 1
Random Experiments
◼ Random Experiment
◼ Sample Space
Ahmed Helmi Lecture 3 – Page 2
Sample Space
◼ Deck of Cards
◼ 52 cards in a deck of cards: {c1, …, c52}
◼ Value: {K, Q, J, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1}
◼ spades (♠), hearts (♥), diamonds (♦) and clubs (♣) : {S, H, D, C}
◼ Red and black cards: {R, B}
◼ Throwing a die (dice)
◼ Value: {1, 2, 3, 4, 5, and 6}
◼ Value odd or even: {O, E}
Ahmed Helmi Lecture 3 – Page 3
Events
◼ Simple event
◼ Outcome from a sample space with one characteristic
◼ e.g.: A red card from a deck of cards
◼ e.g.: An odd number from a dice throw
◼ Joint event
◼ Involves two outcomes simultaneously
◼ e.g.: A King that is also red from a deck of cards
◼ e.g.: An even number less than 3 from a dice throw
Ahmed Helmi Lecture 3 – Page 4
Visualizing Events
◼ Venn Diagram
◼ Sample space S
◼ Events A and B
Ahmed Helmi Lecture 3 – Page 5
Special Events
◼ Impossible event
e.g.: getting “7” from throwing one die
drawing a card “0” from a deck of cards
◼ Complement of event
◼ Complement of event A: all events not in A
◼ Denoted as A’
◼ For example
◼ A: all defective bulbs in a box
◼ A’: all non-defective bulbs in the box
Ahmed Helmi Lecture 3 – Page 6
Special Events
◼ Complement of an event
◼ Sample space S
◼ Event A A A’
◼ Complement of A → A’ S
◼ eg.: Throwing dice
◼ A = even number = {2, 4, 6}
◼ A’ = odd number = {1, 3, 5}
◼ B = number < 3 = {1, 2}
◼ B’ = number 3 = {3, 4, 5, 6}
Ahmed Helmi Lecture 3 – Page 7
Special Events
Union: A B
(A or B)
A B
S
Intersection:
A B A B
(A and B)
S
• Mutually Exclusive Events
A B = = {} A B
S
A and B cannot occur together
Ahmed Helmi Lecture 3 – Page 8
Special Events
(continued)
◼ Collectively exhaustive events
◼ One of the events must occur
◼ The set of events covers the whole sample space
◼ e.g.: -- A: all the aces; B: all the black cards; C: all the
diamonds; D: all the hearts
◼ Events A, B, C and D are collectively
exhaustive
◼ Events B, C and D are also collectively
exhaustive A
C D
B ♦ ♥
Ahmed Helmi Lecture 3 – Page 9
Probability
◼ Probability is a numerical 1 Certain
measure of the likelihood S
that an event will occur
◼ Value is between 0 and 1
.5
◼ Sum of the probabilities of
all mutually exclusive and
collectively exhaustive events
is equal to 1
0 Impossible
Ahmed Helmi Lecture 3 – Page 10
Computing Probabilities
◼ The probability of an event E:
number of event outcomes
P( E ) =
total number of possible outcomes in the sample space
X
=
T e.g. P( ) = 2/36
(There are 2 ways to get one 6 and the other 4)
◼ Each of the outcomes in the sample space is equally
likely to occur (i.e., has the same probability to
occur)
Ahmed Helmi Lecture 3 – Page 11
Computing probabilities
◼ If an event can occur (happen) in h different
ways out of a total number of n possible
ways, all of which are equally likely, then the
probability of the event is h/n
Example: What is the probability of drawing a
king from a set of 52 playing cards?
Solution:
◼ h = 4 kings in the deck of cards
◼ n = 52
◼ n/h = 4/52 = 1/13
Ahmed Helmi Lecture 3 – Page 12
Computing Joint Probability
◼The probability of a joint event, A and B:
P (A and B ) = P (A B )
number of outcomes in A and B at the same time
=
total number of possible outcomes in sample space
e.g. P (Red Card and King)
2 Red Kings 1
= =
52 Total Number of Cards 26
Ahmed Helmi Lecture 3 – Page 13
Computing Compound (or
Multiple) Probability
◼ Probability of a compound event, A or B:
P ( A or B ) = P ( A B )
number of outcomes from either A or B or both
=
total number of outcomes in sample space
E.g. P (Red Card or King)
4 Kings + 26 Red Cards - 2 Red Kings
=
52 total number of cards
28 7
= =
52 13
Ahmed Helmi Lecture 3 – Page 14
Compound Probability
(Addition Rule)
P(A1 or B1 ) = P(A1) + P(B1) - P(A1 and B1)
P(A1 B1 ) = P(A1) + P(B1) - P(A1 B1)
For Mutually Exclusive Events (Events that cannot
occur at the same time):
P(A or B) = P(A) + P(B)
Ahmed Helmi Lecture 3 – Page 15
Computing probabilities
◼ Complement of an Event, P(A’)
A A '=S
P (A ) + P (A ') − P (A A ') = P (S )
A and A ' are mutually exclusive
P (A A ') = P ( ) = 0
P (A ) + P (A ') = P (S ) = 1
P (A ') = 1 − P (A )
Ahmed Helmi Lecture 3 – Page 16
Conditional Probability
◼ The probability of event A given that event B
has occurred:
P( A and B)
P( A | B) =
P( B)
E.g.
P (Red Card given that it is a King)
2 Red Kings 2 / 52 1
= = =
4 Kings 4 / 52 2
Ahmed Helmi Lecture 3 – Page 17
Conditional Probability and
Statistical Independence
◼ Conditional probability:
P( A and B)
P( A | B) =
P( B)
◼ Multiplication rule:
P( A and B) = P( A | B) P( B)
= P( B | A) P( A)
Ahmed Helmi Lecture 3 – Page 18
Conditional Probability and
Statistical Independence
(continued)
◼ Events A and B are independent when the
probability of one event, A, is not affected by
another event, B
◼ Events A and B are independent if
P ( A | B ) = P ( A)
or P( B | A) = P ( B )
or P( A and B) = P( A) P( B)
Ahmed Helmi Lecture 3 – Page 19
Example 1
◼ A water treatment plant may fail for two reasons: inadequacy of
materials (event A) or mechanical failure (event B).
If P(A) = 2 P(B), P(A | B) = 0.8 and the probability of failure of
the treatment plant equals 0.001 , what is the probability that a
mechanical failure occurs? What is the probability of a failure
due to inadequate materials?
The probability of failure P (A U B) may be written as:
P (A U B) = P (A) + P (B) – P (A ∩ B)
= 2 P (B) + P (B) – P (B) P (A | B)
= 3 P (B) – 0.8P (B)
0.001 = 3 P (B) – 0.8P (B)
→ P (B) = 0.00045
→ P (A) = 0.0009
Ahmed Helmi Lecture 3 – Page 20
Example 2
◼ Each of two pumps Q1 and Q2 may not operate. We may observe
one of the outcomes of the following set: {(ƒ, ƒ), (ƒ, o), (o, ƒ), (o, o)}.
The notation (ƒ, o) means that pump Q1 fails and pump Q2 operates.
The sample space has four elements. From experience it is known
that:
◼ P ((ƒ, ƒ)) = 0.1, P ((ƒ, o )) = 0.2 A B
◼ P ((o, ƒ)) = 0.3, P ((o, o )) = 0.4
(o,f) (o,o) (f,o)
◼ Let
0.3 0.4 0.2
◼ A: Q1 operates
◼ B: Q2 operates
◼ C: at least one of the pumps operates (f,f) 0.1
◼ Compute P(A), P(B), P(A ∩ B), P(C)
Ahmed Helmi Lecture 3 – Page 21
Example 2 (Cont.)
A B
◼ P ( A ) = 0.7
◼ P ( B ) = 0.6 (o,f) (o,o) (f,o)
◼ P ( A ∩ B ) = P (( o , o )) = 0.4 0.3 0.4 0.2
◼ P(C)=(AUB) (f,f) 0.1
=P(A)+P(B)–P(A∩B)
= 0.9
Ahmed Helmi Lecture 3 – Page 22
Total Probability
◼ Total Probability Rule (multiple events)
E1 E2 Ek
B
Ahmed Helmi Lecture 3 – Page 23
Example 3
◼ The demand of a water supply system can be low (event L) ,
moderate (event M) or high (event H) with known probabilities
P(L) = 0.1, P(M) = 0.7 and P(H) = 0.2. Failure of the system (event F)
can only occur if a certain pump fails to function. From past
experience the following conditional probabilities are known:
P(F | L) = 0.05, P(F | M) = 0.10 and P(F | H) = 0.25. What is the
probability of a pump failure P(F).
First we observe that the events L, M and H are clearly mutually
exclusive and that their union is just the sure event S. Hence
P ( F ) = P (( L ∩ F ) U ( M ∩ F ) U ( H ∩ F ))
= P (L ∩ F) + P (M ∩ F) + P (H ∩ F)
Using the definition of conditional probability, we get
P ( F ) = P (L) P (F | L) + P (M) P (F | M) + P (H) P (F | H)
= 0.1 0.05 + 0.7 0.10 + 0.2 0.25
= 0.125
Ahmed Helmi Lecture 3 – Page 24
Bayes’s Theorem
◼ B1, B2, …, Bk are mutually exclusive and
collectively exhaustive
◼ Suppose it is known that event A has occurred
◼ What is the (conditional) probability that
event Bi has occurred?
B1 B2 Bk
Ahmed Helmi Lecture 3 – Page 25
Bayes’s Theorem
P ( B i and A )
P (Bi | A ) =
P (A )
P (A | Bi ) P (Bi )
=
P ( A | B1 ) P ( B1 ) + • • • + P ( A | B k ) P ( B k )
B1 B2 Bk
Ahmed Helmi Lecture 3 – Page 26
Example 3(b)
◼ In example 3, If a failure occurs, what is the
probability that it occurred during moderate demand?
P (F | M ) P (M )
P (M / F ) =
P (F | L )P (L ) + P (F | M )P (M ) + P (F | H )P (H )
P (F | M ) P ( M ) (0.1)(0.7) 0.07
= = = = 0.28
P (F ) 0.125 0.25
Ahmed Helmi Lecture 3 – Page 27
Example 4
Fifty percent of graduating engineers in a certain year worked as
water resources (WR) managers. Out of those who worked as WR
managers, 40% had a Masters degree. Ten percent of those
graduating engineers who did not work as WR managers had a
Masters degree. What is the probability that a randomly selected
engineer who has a Masters degree is a WR manager?
P (WR ) = 0.5, P (M /WR ) = 0.4, P (M /WR ) = 0.1, P (WR / M ) = ?
P (M |WR ) P (WR )
P (WR / M ) =
P (M |WR )P (WR ) + P (M |WR )P (WR )
(0.4)(0.5) 0.2
= = = 0.8
(0.4)(0.5) + (0.1)(0.5) 0.25
Ahmed Helmi Lecture 3 – Page 28
