Chapter 2

Basic Probability Concepts

 Contents
 2-1 Sample space
 2-2 Events
 2-3 Counting Sample points
 2-4 Probability of an Event
 2-5 Additive Rules
 2-6 Conditional Probability
 2-7 Bayes’ Rule
 2-8 Probability Space
 2-1 Sample space

Definition 2.1:

The set of all possible outcomes of a statistical experiment

is called the sample space (S).

[Example] Tossing a die

S1 = {1, 2, 3, 4, 5, 6} ; S2 = {even, odd}

► depending on the problem of interest.
 [Example]

S3={ x | x is a city with population >106 }

S4={ (x, y) | x2+y2 ≤ 4 }

Consider an experiment of coin flipping / die tossing:

S5={HH,HT,T1,T2,T3,T4,T5,T6}
H
Tree H
T
diagram 1
T 2
3
4
5
6
 2-2 Events

Definition 2.2:

An event is a subset of a sample space.

[Example]
event A={outcomes of a die tossing divisible by 3}
={ 3 , 6 }
 Set Operations
 Union: A∪B={ x | x∈A or x∈B}

 Intersection: A∩B={ x | x∈A and x∈B}

 Difference: A - B={ x | x∈A and x B}

 Complement : Ac = S-A = { x | x∈S and x A} ,

 Two events A and B are mutually exclusive, or

disjoint, if A∩B=Ø, that is, if A and B have no
elements in common.
 2-3 Counting Sample Points

Theorem 2.1: multiplication rule

If an operation can be performed in n1 ways, and if for
each of these ways a second operation can be performed
in n2 ways, then the two operations can be performed
together in n1* n2 ways.

[Example] Tossing 2 dice

The first die can land in any one of n1=6 ways. The second
die can also land in n2 =6 ways. Therefore, the pair of dice
can land in n1* n2=6*6=36 ways.
 Permutation
A permutation is an arrangement of a set of objects in an ordered fashion .
[Theorem 2.2]

The number of permutation of n distinct objects is n!.

The number of permutation of n distinct objects taken r at
a time is nPr = n! / (n-r)!.
[Example]
Three awards (research, teaching, and service) will be given
for a department of 25 professors. Each can receive at most 1
award. How many possible selections are there?
25 3 P = 25! / (25-3)!=(25)(24)(23)=13800.
[Theorem 2.3] (Circular permutation)
The number of permutation of n distinct objects
arranged in a circle is (n-1)!.

[Theorem 2.4]
The number of distinct permutation of n objects of
which n1 are of one kind, n2 of a second kind,…, nk of a
kth kind is n!/ (n1! n2! ... nk!).
 Combination
A combination is a selection of objects without regard to
order, i.e., partition into cells.
[Theorem 2.5]
The number of combinations of n distinct objects taken r at a
time is nCr = n! / r! (n - r)!.
The number of ways of partitioning a set of n objects into r
cells with n1 elements in the first cell, n2 elements in the
second, and so forth, is
 n  n!
 n , n ,..., n   , (n1  n2  ...  nr  n).
 1 2 r n1 !n2 !...nr !
 [Example]

In how many ways can 7 students be assigned to

one triple and two double hotel rooms during a
field trip?

(Solution)
The total number of possible ways would be
 7  7!
   210
 3, 2, 2  3! 2! 2!
 [Example]

John asks Tom to get 5 game-boy cartridges from his collection

of 10 arcade and 5 sport games. How many ways are there that
Tom will get 3 arcade and 2 sport games, respectively?

(Solution)
The number of ways of selecting 3 cartridges from 10 is
10C3 = 10! / (3! 7!) = 120.

The number of ways of selecting 2 cartridges from 5 is

5C2 = 5! / (2! 3!.) = 10.

Using the multiplication rule with n1=120 and n2=10,

there are (120)(10)=1200 ways.
 2-4 Probability of an Event

Definition 2.3
The probability of an event A is the sum of the probabilities
of all sample points (elementary events) in A.

0  P( A)  1, P( )  0, and P( S )  1,
If A1, A2, A3,…are mutually exclusive events, then
P( A1  A 2  A 3 ...)  P( A 1 )  P( A 2 )  P( A3 )  ...
* A more rigorous treatment of the probability can be
described by Kolmogorov’s axioms and the definition of
probability spaces.
 Kolmogorov’s Axioms
(rules or principles that are generally accepted)

 First axiom:
the probability of an event A is a non-negative real number:
P ( A)  0.
 Second axiom:
the probability that a certain event in the entire sample space
will occur is 1. More specifically, there are no events outside
the sample space:
P ( S )  1.
 Third axiom:

If A1, A2, A3,…are mutually exclusive events, then

P( A1  A 2  A 3 ...)  P( A 1 )  P( A 2 )  P( A3 )  ...
 Probability Space

 A probability space is a sample space together with

an assignment of probabilities (0 ~ 1) to events.
 A probability space, denoted by (, B, P), is a set 
(sample space) , a  – field B (event space) and a
probability measure P defined on B.
 An alternative approach to formalizing probability,
favored by some Bayesians, is given by Cox’s
theorem.
 Uniform probability space

[Theorem]
If an experiment can result in any one of N different
equally likely outcomes (assumption of uniform
probability space), and if exactly n of these outcomes
correspond to event A, then the probability of event A
is given by
n
P( A) 
N
 [Example] tossing a die

S= {1, 2, 3, 4, 5, 6 }

P(1)=P(2)=P(3)=P(4)=P(5)=P(6)=1/6
P(S)=1/6+1/6+1/6+1/6+1/6+1/6=1
What about tossing 2 dice?
Is it true that P(n)=1/11, for n = 2~12?
S= {(1,1), (1,2),(2,1), (1,3),(3,1),(2,2), (1,4),……}
 Example

In a poker hand consisting of 5 cards, find the probability

of holding 2 aces and 3 jacks.
(Solution)
The no. of ways of being dealt 2 aces from 4 is 4C2 = 6.
The no. of ways of being dealt 3 jacks from 4 is 4C3 = 4.
By the multiplication rule, there are (6)(4)=24 hands with
2 aces and 3 jacks. The total no. of 5-card hands, all of
which are equally likely, is 52C5 = 52!/(5! 47!) = 2,598,960.
Thus, the probability of getting 2 aces and 3 jacks is
24/ 2,598,960=0.9*10-5.
 2-5 Additive Rules
[Theorem] B
If A and B are two events, then
P ( A  B )  P ( A)  P ( B )  P ( A  B ) A∩B

[Theorem]
A
For three events A, B, and C,
P ( A  B C )  P ( A)  P ( B )  P (C )  P ( A  B )
 P ( A C )  P ( B C )  P ( A  B C ).
[Theorem]
If A and Ac are complementary events, then P(A) + P(Ac) =
1.
 Example

After interviewing with 2 companies, John assesses that his

probability of getting an offer from company A is 0.8, and
the probability of getting an offer from company B is 0.6.
What is the probability that he will get at least one offer from
the 2 companies?
(Solution)
P ( A  B )  P ( A)  P ( B )  P ( A  B )
 0.8  0.6  ?
Obvious, the probability that John will get offers from both
companies is at least 0.4 (subjective probability).
 2-6 Conditional Probability

Definition 2.4
The conditional probability of B, given A, denoted by
P(B|A) is defined as
P( A  B)
P ( B | A)  , if P(A) > 0.
P ( A)

Thus,
P ( A  B )  P ( A)  P ( B | A),
 Example

The probability that a regularly scheduled flight departs on

time is P(D)=0.83; the probability that it arrives on time is
P(A)=0.82; and the probability that it departs and arrives on
time is P ( D  A)  0.78. Find the probability that a plane
(a) arrives on time given that it departed on time, and
(b) departed on time given that it has not arrived on time.
(Solution)
P ( A  D) 0.78
(a) P( A | D)    0.94.
P( D) 0.83
c
(b) P ( D | A ) 
c P ( D  A ) p ( D )  P ( D  A) 0.83  0.78
c
   0.28.
P( A ) 1  P ( A) 1  0.82
 Independent Events

Definition 2.5
Two events A and B are independent if and only if
P( B | A)  P( B ) or P ( A | B )  P ( A)

[Theorem]
Two events A and B are independent if and only if

P ( A  B )  P ( A) P ( B )
 Example

2 poker cards are drawn in succession from an ordinary deck

of 52 cards with replacement. Consider the following 2 events:
A: the first card is an ace,
B: the second card is a spade.
Are the 2 events independent?
(Solution)
Since the first card is replaced, the sample space for both
draws are the same with 52 cards containing 4 aces and 13
spades. Hence
P( B | A)  13
52
 P( B).
Therefore, the 2 events are independent.
 Example

One bag contains 4 white and 3 black balls, and a second bag
contains 3 white and 5 black balls. One ball is drawn from the
first bag and placed unseen in the second bag. What is the
probability that a ball now drawn from the second bag is black?
(Solution)
Tree diagram
 2-7 Bayes’ Rule

[Theorem] Theorem of total probability

If B1, B2,…, Bk constitute a partition of the sample space S
such that P(Bi)≠0 for i =1,2,…,k, then for any event A of S,
k k
P ( A)   P ( Bi  A)   P ( Bi )P ( A | Bi ).
i 1 i 1
 [Theorem] Bayes’ Rule
If the events B1, B2,…, Bk constitute a partition of the sample
space S such that P(Bi)≠0 for i =1,2,…,k, then for any event
A in S such that P(A)≠0,
P ( Br  A) P ( Br  A) P ( Br ) P ( A | Br )
P ( Br | A)   k  k
P ( A)
 P( Bi  A)  P( Bi )P( A | Bi )
i 1 i 1
for r =1,2,…,k.
 Example

In a plant, 3 machines B1, B2, B3 make 30%, 45%,

25%, respectively, of the products. If 2%, 3%, 2% of
the products made by each machine are defective.
(a) Suppose a product is randomly selected. What is
the probability that it is defective?

(b) If a product were chosen randomly and found to be

defective, what is the probability that it was made by B3?
 (Solution)
Consider the following events
A: the product is defective.
Bi: the product is made by machine Bi.
3 3
(a) P ( A)   P ( Bi  A)   P ( Bi )P ( A | Bi )
i 1 i 1
= (0.3)(0.02)+(0.45)(0.03)+(0.25)(0.02)
= 0.0245
P ( B3  A) P ( B3  A) P ( B3 ) P ( A | B3 )
(b) P ( B3 | A)   k  k
P ( A)
 P( Bi  A)  P( Bi )P( A | Bi )
i 1 i 1
0.25  0.02
  0.2041
0.0245
 2-8 Probability Space
 The concept of probability space is developed to form the
foundation for the mathematical treatment of random phenomena.
 Sample space the set of outcomes for a random experiment
 – Field B (event space, signifies subset)
A nonempty collection of subsets of a set  is called a  – field B
if it is closed under compliments and countable unions, i.e.
i) A  B implies Ac B.
ii) Ai B, i=1,2,… implies i 1 Ai B.


 The members in  – field B are events, to which probability will

be assigned.
 The pair [, B] is called a measurable space.
 Probability Measure P
a real-valued function defined on a – field Bsuch that
i) P ()  0 and P ()  1.
ii) 0  P ( Ai )  1 for all Ai  B.
iii) If Ai, i = 1,2,3,…, are mutually disjoint sets in B, then

i 1 Ai )   P ( Ai ).
P (
i 1
 A Probability Space, denoted by (, B, P), is a set 
(sample space) , a  – field B (event space) and a probability
measure P defined on B.
 Uniform probability space

 Some of the oldest problems in probability involve the

idea of picking a point “at random” from a set S.
 The intuition is that if A and B are 2 sets of the same
“size” (having the same number of points) then the
chance of picking a point from A is the same as from B.
 Under the assumption of uniform probability space, the
probability measure P is defined such that P(A) is
proportional to the size (volume, area,…) of event (set) A.
 Bertrand's paradox (Self-contradictory or
counter-intuitive statement or argument)

Bertrand's paradox (1888) is a problem within

the classical interpretation of probability theory.
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen “at random”.
What is the probability that the chord is longer than a
side of the triangle?
 Bertrand's Paradox Solution

 Selection method 1 - Random Endpoints:

For points on the arc between the endpoints of the side
opposite the first point, the chord is longer than a side of the
triangle. P = 1/3.

 Selection method 2 - Random Radius:

The chord is longer than a side of the triangle if the chosen
point is nearer the center of the circle. P = 1/2.

 Selection method 3 - Random Midpoints:

The chord is longer than a side of the triangle if the chosen
point falls within a concentric circle of radius 1/2. P = 1/4.

The solution thus hinges on the means by which a chord

is chosen “ at random". Once the method of selection is
specified, the problem has a well-defined solution.