PROBABILITY

PROBABILITY 1 / 45
CONTENTS:
1 Introduction
2 Sample space, events
3 Operations on events
4 Assigning probability to events
5 Conditional probability

PROBABILITY 2 / 45
INTRODUCTION

This part discusses the language and techniques of probability. The

probability theory describes random events and then deduce their
consequences. The term ”stochastic” is often used for meaning
”chance” or ”random effect” for indicating the effect that can arise
unpredictably. The interpretation of the word ”probability” involves other
synonyms such as odds, uncertainty, prevalence, risk and expectancy.
Probability theory is especially fundamental to Mathematical Statistics.
Mathematical Statistics is concerned with the development of methods
and their applications for collecting, analysing and interpreting data in
such a way that the reliability of a conclusion based on data may be
evaluated objectively by means of probability statements.

PROBABILITY 3 / 45
In statistics:
1 Descriptive statistics are used to describe the basic features of the
data in a study. They provide simple summaries about the sample,
and the quantitative measures also known as statistics. Together
with simple graphics analysis, descriptive statistics display
quantitative analysis of the available data.
2 Inferential statistics are used for making inferences from samples to
more general conditions.

The mathematical theory of probability starts with the analysis of games

by Gerolamo Cardano in the sixteenth century. Pierre de Fermat and
Blaise Pascal consolidated the theory in the seventeenth century. Initially,
probability theory mainly considered discrete events, and its methods were
mainly combinatorial. Eventually, analytical considerations compelled the
incorporation of continuous variables into the theory. The modern
probability theory has been then initiated by Andrey Nikolaevich
Kolmogorov in 1933.

PROBABILITY 4 / 45
EXPERIMENTS, SAMPLE SPACE, EVENTS

The basic language of probability theory comes from set theory. The
probability theory describes events (possible occurrences) by assigning
numbers (probabilities) to the events.

Definition
The sample space S of an experiment is the set of all possible results of an
experiment.
The elements of S are called ”sample points”.
If there is a finite number of sample points, the total number of sample
points is denoted by #S, and we say that S is a finite sample space.

Definition
An event E is any subset of the sample space S of the experiment.

Examples

PROBABILITY 5 / 45
1 Two different coins are tossed, and the result (H or T ) for each coin
is observed. Determine a sample space.
Solution

S = {HH, HT , TH, TT }
2 A bag contains four jelly beans: one red, one pink, one black, and one
white. A jelly bean is withdrawn at random, its color is noted, and it
is put back in the bag. Then for the second time, a jelly bean is
randomly withdrawn and its color is noted. The experiment is then
stopped.
(1) Describe a sample space and determine the number of sample points.
Solution
Using a tree:

PROBABILITY 6 / 45
S = {RR, RP, RB, RW , PR, PP, PB, PW , BR,
BP, BB, BW , WR, WP, WB, WW }

#S = 16

PROBABILITY 7 / 45
(2) Describe the following events:
(i) E1 : At least a red jelly bean is picked.
Solution

E1 = {RR, RP, RB, RW , PR, BR, WR}

#E1 = 7

(ii) E2 : At most one pink jelly bean is picked.

Solution

E2 = {RR, RP, RB, RW , PR, PB, PW , BR,

BP, BB, BW , WR, WP, WB, WW }

#E2 = 15

(iii) E3 : No black jelly been is picked.

Solution

E3 = {RR, RP, RW , PR, PP, PW , WR, WP, WW }

#E3 = 9

PROBABILITY 8 / 45
 (iv) E4 : Neither red nor pink jelly bean is picked.
Solution

E2 = {BB, BW , WB, WW }
#E4 = 4

(3) A pair of dice is rolled once, and for each dice, the number that turns
up is observed. Determine the number of sample points.
Solution
D2 1 2 3 4 5 6
D1
1 (1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
2 (2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
3 (3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
4 (4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
5 (5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
6 (6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
S = {(1, 1), (1, 2), . . . , (6, 6)}
#S = 36

PROBABILITY 9 / 45
OPERATIONS ON EVENTS

The usual operations on sets are applied to the events.

Sets are visually presented by Venn diagram by incorporating the elements
in dartboards. The same procedure is used for presenting events as shows
the following figure.

PROBABILITY 10 / 45
The region inside the rectangle represents the whole sample space S which
represents all the possible outcomes.The regions A, B, and C represent
the 3 events to which probabilities are assigned. Note that φ and S are
also the events of the sample space S.
There three basic operations on sets are applied to events:
1 Intersection: The intersection of sets is the region contained in both
particular sets. The intersection of sets A and B is written A ∩ B and
read ”A intersection B. In terms of the events, we say the events A
and B occur simultaneously.

PROBABILITY 11 / 45
 1 The intersection is commutative: A ∩ B=B ∩ A.
2 If A ∩ B=Φ, then A and B are mutually exclusive. Mutually exclusive
events are then events that can not occur simultaneously.
2 Union: The union of sets is the region contained in either set. The
union of sets A and B is written A ∪ B and read ”A union B”. In
terms of the events, we say that either event A or B may occur,
including the possibility that both events may occur.

PROBABILITY 12 / 45
3 Complement: The complement of a set is represented by a region
outside the set in the sample space. The complement of a set A can
be denoted by Ā; Ac or by A0 . In terms of events, the complement of
A refers to ” not A”. The following figure shows Ā.

PROBABILITY 13 / 45
PROBABILITY 14 / 45
In class-1-
Given the usual sample space for rolling a dice, let E , F , and G be the
events defined by E = {1, 3, 5}; F = {3, 4, 5, 6} and G = {1}. Determine
each of the following events:
(1) E
(2) E ∪ F
(3) E ∩ F
(4) F ∩ G
(5) E ∪ E
(6) E ∩ E

PROBABILITY 15 / 45
Properties of events
Consider the events E , F and G of the sample space S. The table bellow
describes the common properties of the events.
Table: Properties of events
Operation Property Note
1 E ∩E =E
2 E ∩E =Φ
Intersection 3 E ∩S =E
4 E ∩Φ=Φ
5 E ∩F =F ∩
E
Commutative property of intersection
6 E ∩ F ∩G = E ∩F ∩G Associative property of intersection
1 E ∪E =E
2 E ∪E =S
Union 3 E ∪S =S
4 E ∪Φ=E
5 E ∪F =F ∪
E
Commutative property of union
6 E ∪ F ∪ G
= E ∪ F
∪ G
Associative property of union
Union and 1 E ∩ F ∪G = E ∩F ∪ E ∩G Distributive property of intersection
intersection



2 E ∪ F ∩G = E ∪F ∩ E ∪G Distributive property of union
1 E =E
Complement 2 E ∪F =E ∩F De Morgan’s law
3 E ∩F =E ∪F De Morgan’s law

PROBABILITY 16 / 45
ASSIGNING PROBABILITY TO EVENTS
A probability is a number assigned to each events of a sample space S. It
measures the likelihood of that event to occur. The probability of an event
A is written by P(A) and read ”the probability of A” Each event of the
sample space S is assigned to a probability by a probability model. This
part uses Venn diagrams for presenting the events and introduces a
presentation of a probability by the area in a Venn diagram.

A probability model requires the following four axioms:

PROBABILITY 17 / 45
1 If S is the sample space, P(S) = 1.

PROBABILITY 18 / 45
2 For any event A, 0 ≤ P(A) ≤ 1.

PROBABILITY 19 / 45
3 If A and B are mutually exclusive events, P(A ∪ B) = P(A) + P(B).

PROBABILITY 20 / 45
4 If Ā is the complement of A, P(Ā) = 1 − P(A).

The probability ”0” means that the event cannot happen while the
probability ”1” means that the event happens always in the sample
space.

PROBABILITY 21 / 45
The Axiom 3 is applicable for mutually exclusive events. In general,
for events A and B such that A ∩ B 6= Φ,

P(A ∪ B = P(A) + P(B) − P(A ∩ B).

Consider the sample space S associated with simple mutually

exclusive events E1 , E2 , . . . , En as indicated in the following figure.

PROBABILITY 22 / 45
Due to Axiom 2:

P(Ei ) ≥ 0. ∀ i ∈ [1, n]
Using Axiom 3 yields:

P(E1 ∪ E2 ∪ · · · ∪ En ) = P(E1 ) + P(E2 ) + · · · + P(En )

where E1 ∪ E2 ∪ · · · ∪ En = S
Xn
= P(Ei )
i=1
n
X
⇒ P(S) = P(Ei )
i=1

Axiom 1 gives the result

n
X
P(Ei ) = 1.
i=1

PROBABILITY 23 / 45
 If all the events are equally likely to occur in a sample space S, S is
said to be an equiprobable sample space.
For the equiprobable sample space of elementary events
Ei , ∀ i ∈ [1, n] , we have:

1
P(Ei ) = ∀ i ∈ [1, n].
n

In general, given any event E of the sample space S,

#E
P(E ) = .
#S

PROBABILITY 24 / 45
In class-2-
In a certain population, 10% of the people are rich, 5% are famous, and
3% are both rich and famous. A person is randomly selected from this
population.
Use a Venn diagram to compute the probability that a person chosen
randomly is:
(a) Not rich.
(b) Rich but not famous.
(c) Either rich or famous.

PROBABILITY 25 / 45
6.3 Exercises : 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16

PROBABILITY 26 / 45
QUIZ-1-

PROBABILITY 27 / 45
CONDITIONAL PROBABILITY
The conditional probability is used to compute a probability of one event
conditional on knowing that another event has occurred. The concept of
conditional probability is used to deal with the law of total probability, the
Bayes’ theorem and the independence in probability.

Definition
The probability of event A conditional on event B is denoted by P(A|B)
and read ”probability of A if B”. P(A|B) is given by

P(A ∩ B)
P(A|B) = , provided that P(B) 6= 0. (1)
P(B)

Two important particular cases arise:

PROBABILITY 28 / 45
1 Event A and event B are mutually exclusive:

A∩B =Φ
and

P(Φ)
P(A|B) =
P(B)
0
=
P(B)
= 0

2 Event A includes event B (denoted by B ⊂ A)

A∩B =B

PROBABILITY 29 / 45
and

P(B)
P(A|B) =
P(B)
= 1.

PROBABILITY 30 / 45
In class-3-
1 A fair coin is tossed three times.
(1) What is the probability of observing three heads, HHH?
(2) What is the probability of observing exactly one head?
(3) Given that at least one head is observed, what is the probability that at
least two heads are observed?
2 The probability of color-blindness depends on a person’s gender. A
person can be male (event M) or female (event F ). Assume that
P(M) = P(F ) = 0.5. Let the event C indicate that a person is
color-blind, and let the event N indicate that he or she is not.
Suppose that P(M ∩ C ) = 0.025, P(F ∩ C ) = 0.005, meaning that
2.5% of people are color-blind males and 0.5% of people are
color-blind females.
(1) What is the probability that a male is color-blind?
(2) What is the probability that a color blind person is a male?
(3) For 10 000 people, calculate the number of males, females, color-blind
males, color-blind female and check the probabilities calculated in (1)
and (2).

PROBABILITY 31 / 45
Definition
The events E1 , E2 , . . . , En of the same sample space S form a set of
mutually exclusive and collectively exhaustive events (see Figure bellow) if
no pair intersect and the union is the sample space.

PROBABILITY 32 / 45
Theorem: The law of total probability
Suppose that E1 , E2 , . . . , En form a set of mutually exclusive and
collectively exhaustive events in the sample space S. Then for any event A
of the sample space S,
n
X
P(A) = P(A|Ei )P(Ei ),
i=1

for any event Ei ∀ i ∈ [1, n] of mutually exclusive and collectively

exhaustive events in sample space S.

Proof (page 77): In class-4-

PROBABILITY 33 / 45
In class-5-

1 Suppose 1% of females and 5% of males are color-blind. What is the

probability that a person chosen at random is color-blind for the
following situation?
(1) The number of males and females are equal.
(2) The number of female is the double of the number of males.
2 Individuals are diagnosed with particular type of cancer that can take
on three different forms, D1 , D2 and D3 . It is known that 20% of
people will eventually be diagnosed with disease 1, 30% with disease
2, and 50% with disease 3. The probability of requiring chemotherapy
differs among the three forms of disease: 80% with disease 1, 30%
with disease 2 and 10% with disease 3. Based solely on the
preliminary test, what is the probability of requiring chemotherapy?

PROBABILITY 34 / 45
 3 Consider three baskets A, B and C. Basket A contains two white balls
and one black ball. Basket B contains three white balls and one black
ball and Basket C contains two white balls and 2 black balls. We
choose a basket at random and draw one ball. Find the probability
that this ball is white.
Theorem: Bayes’ theorem
For any events A and B of the same sample space S where P(B) 6= 0;
P(A) 6= 0,
P(A|B)P(B)
P(B|A) = .
P(A)

Proof (page 79): In class-6-

Assume that B1 , B2 , . . . , Bn are mutually exclusive and collectively

exhaustive events in the sample space S. Using the Bayes theorem,

PROBABILITY 35 / 45
 P(A|Bk )P(Bk )
P(Bk |A) = ; k ∈ [1, n].
P(A)
Applying the law of the total probability yields:

P(A|Bk )P(Bk )
P(Bk |A) = n ; k ∈ [1, n].
P
P(A|Bi )P(Bi )
i=1

In particular, if P(B) 6= 0; P(B) 6= 0 and P(A) 6= 0; then

P(A|B)P(B)
P(B|A) = .
P(A|B)P(B) + P(A|B)P(B)

PROBABILITY 36 / 45
In class-7-
1 A person is male with probability 0.5, males are color-blind with
probability 0.05, and any person chosen at random is color-blind with
probability 0.03. Use Bayes’ theorem to find the probability that a
color-blind person is male.
2 Suppose that a disease infects 1% of people. A diagnostic test always
picks up the disease, but generates 5% false positive. A patient tests
positive.
(1) Use a sample of 10 000 individuals to calculate the probability that the
person has the disease.
(2) Use Bayes’ theorem to calculate the probability that the person has the
disease.
(3) Repeat (2) if a disease infects 40% of people.

PROBABILITY 37 / 45
6.4 Exercises : 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16, 17, 18,
21,22

PROBABILITY 38 / 45
QUIZ-2-

Definition: Independent events

Event A is independent to event B if

P(A|B) = P(A).

Theorem: Multiplication rule for independent events

Suppose that A and B are any two independent events of the same sample
space S. Then
P(A ∩ B) = P(A)P(B).

Proof (page 74): In class-8-

PROBABILITY 39 / 45
Generally for any n independent events E1 , E2 , E3 , . . . , En of the
sample space S,

P(E1 ∩ E2 ∩ · · · ∩ En ) = P(E1 ) · P(E2 ) · · · · · P(En )

Yn
= P(Ei ).
i=1

Theorem
If A and B are independent, then the following pairs of events are also
independent: A and B, Ā and B, and Ā and B.

PROBABILITY 40 / 45
In class-9-
The probability that the machine A fails during any day is 0.025; similarly
for the machine B, that probability is 0.015. Given that the machines
operate independently, find
(1) The probability that both machines fail simultaneously.
(2) The probability that no machine fails.
(3) The probability that one or the other has a fault.

PROBABILITY 41 / 45
6.5 Exercises: 1,2,3,4,5,6,7,8,9,10,11,12,19, 20

PROBABILITY 42 / 45
2.3 Exercises (Page 84) : 1, 2, 9, 20, 23,24

QUIZ-3-

PROBABILITY 43 / 45
PROBABILITY 44 / 45
PROBABILITY 45 / 45