CHAPTER 2

Introduction to Probability

PART 1 Axioms of Probability

I and Events
Sample Space

Def The Sample Space Son r associated with

random experimenta

is the set of all possible outcomes of the experiment I may be

a discrete or continuous set

Example 1 Throw a die R 21 2,3 4,5 65 discrete

2 a coin until Tails
Keep tossing you get
I IT HT HAT HH HT discrete

3 A train arrives at a station between 10 00 and 11 00 so

the time of arrival can be modelled as D 10 ID continuous

Arriving precisely at 10 15 is the outcome 10.25

Def An event E is subset of the sample space Eer An event

any
is said to be simple if it consists of exactly one outcome Ws
Singleton otherwise it is said to be compound
Example 1 Throw a die The event obtaining 3 E 231 is simple The
event the number obtained is odd Ez 1.3.55 is
compound Notice E E Ee meaning that whenever E occurs
then E occurs

2 The event Tails appear before the 4th trial is the

subset H AT HATS
3 The event The train arrives after 10 30 is E 10.5 ID

Two special events E O is the impossible event or null event

Er is the certain event

general that every subset of r is an event

Note It is not the case in
We cannot measure the probability of some subsets of r We
therefore distinguish the collection of all events on r from

the collection of all subsets of r denoted 21 E e 21 For

discrete r we will usually consider E 21
If the outcome oftheexperiment to the set E we the event E occurs
belongs say

Set Theory Some relationships between events are studied

using settheory

Intersection A NB we r we A and we B Both A and B occur

Union AUB we r we A or we BS At least one of A and B occurs

Complement A or F or A we r we Al Event A does not occur

Difference A IB L we r we A and wet B A occurs but B does not occur

AN B A
A B A B

AUB AIB
A B A B
Some Facts from Settheory
E E EVE R EN E 0
EUF FUE and EMF FAE commutativity
EV FUG EU F UG and EN FRG LEN F NG associativity
ERF UG EU G N FOG and EUF NG LEAGUING
distributivity

De Morgan's Laws AUB ANB and ANB AUB useful

In fact these laws
apply to any finite or even countably infinite sequence

of events

Prop Let Ai be a sequence of events ie subsets of r Then

for new we have
every

Ai D Ai and Ai Ai Ai
 We also have

QI Ai I Ai and
I Ai Ai

Def Two events A and B are said to be mutually exclusive disjoint

if An B O This means that A and B cannot both occur
A sequence of events Anti is said to be mutually exclusive pairwise

disjoint if Ain Aj 0
fig
Exercise Two dice are rolled Consider the
following events
E The sum of outcomes is odd
F At least one 1 appears
Interpret the events E NF En f Ent and FIE

Note We can write En f as EF

 Solution First of all all the outcomes are represented here with
the sum associated with each outcome
growth I 2 3 4 5 6
1 1,172 1,213 1,374 1,475 1,516 1,617
2 21173 12,274 2,315 2,476 2,577 2,678
3 3,174 13,215 3,316 3,417 3,518 13,619
4 4,175 14,276 43 7 4,478 4,579 4 6 to
5 15,176 15,277 15,318 5,479 5,516 5,6711
6 6,177 16,278 16,379 16,416 16,5711 6 6 12

EMF OMI and one even

EMF 412 1,4 1,6 1211 14,1 1611
ÉnF Oms and one odd
EDF A1 1.3 115 1311 5,1
ENE Both numbers are even on both belong to 13.51
FIE FAE ERF as above
 I Axioms and Properties of Probability

The probability of an event A is a number PLA that assigns a certain

likelihood to the event A We need to formalize this mathematically

Def Let r be a sample space and E the collection of events on R

A Probability Function is a function P E R that obeys the
following 3 axioms
A1 For
any
event Ae E PLA 0
A2 PCR 1
AZ Countable Additivity let Anti be a sequence of mutually exclusive
events all Ane E and Ain Then
ie
fig Aj 0

IP LIAN
E PLAN
 Properties of Probability
The following properties can be proved
using the axioms

Prop The Impossible Event has Probability 0 PC03 0

Proof Define the infinite sequence of event A Az by Ai Of fit in
Using A3
PCO PL Ai ET PLA EE IPLO
This happens if and only if
IPLO O D

Prop Finite Additivity let Ai An An be a finite sequence of pairwise

disjoint events Then
IPL Ai
É PLAi

In particular if ANB O PEAU B PLA PCB

 Proof Extend the sequence Ai it to an infinite sequence Ai s.t

Anti 0 Antz 0 Antz 0 Using A3 and P 07 0

and the fact that Ai Ai we have

Pl A IPL AI É PLA
E PLAN Ing PCA

É PlAi In O É PLAIT D

Prop PLA 1 PLA for event A

any
Proof

By definition we have A RIA so AUF r and An F 0

Using countable additivity and A2

IE PLA PLA UAT PLA PLAT PLAT L PLA

D
Prop For event A IPCA e 1
any
Proof Usingthe above I PLA PLAT PCA since PCA so D

Prop For any events A B IPC Bl A IPL B PLAN B

Proof
B BIA U CANB and the sets events BIA and ANB are disjoint

Using countable additivity

IPC B IPLBIA PLAN B as desired D

The next proposition is an important result that allows us to compute

the union of 2 events that are not mutually exclusive

Prop For events A B er PCAUB PLAT PCB PLANB

any
Proof
AUB A U RIA and these events are disjoint By countable additivity
and the result before IP AUB PCAT IDIBIA PCA PCB PCAUB D
Exercise In a certain country club 40 of participants
play tennis
30 play squash and 55 play at least one of the 2
sports
A member is selected at random
awhat is the probability that they only play tennis
b what is the probability that they neither
sport
play
Exercise In a certain country club 40 of participants
play tennis
30 play squash and 55 play at least one of the 2
sports
A member is selected at random
awhat is the probability that they only play tennis
b what is the probability that they neither
sport
play

Solution
T S
a PIT US PCT PLS PITAS
PITAS PIT PLS ALTUS
0.4 0 3 0 55
0.15
We want PITS PLT IS PITT PLAS 0.4 0.15
O 25

b Pl neither sport IP In 5 É IPL TUT 1 IPC Tus

I O 55 0 45
Prop For events A B S t AEB we have PCA e PIB
Proof

Using
a
proposition above IP BIA IPL B IPLBPA but since
Ae B AN B A PL BI A PCB PLA o since PIBIA O

Now what if we would like to compute the union of n events

Al Az An e r We can generalize the result about PCAUB

Thm Inclusion Exclusion Principle For a finite sequence Ailien of events

IPC Ai
É H EE PfAi n n Ain

No need to know the proof An important special case is n 3

PLA VA VA PLA PLAz PIA37 IPCA N Az

PLAz nAs IPLA N Az IPL A N AzNAs

II Probabilities in a Discrete Sample Space

In many real world applications the number of outcomes is finite or

countably infinite lie the outcomes can be listed in a sequence bijection
with the natural numbers IN How to compute probabilities in this context

let A wifi wi wz wz Since I is the countable union

of all the simple events singletons wi A2
using

R IFLWil 1 11427 IT IPKwis

Thus the sum of the probabilities of all outcomes is 1 in a discrete
main tools for calculations
sample space This is one of your

Also use countable additivity if A w wa wa then since all singletons are

disjoint PLA Pfw Pl wa Plus

 Note For simple
In general
events we write
IP A wef Pl wi
an Pl wi instead of
IPLwill for simplicity
Exercise Example 2.15 in book A train has 5 cars Suppose a commuter is
twice as likely to select con 3 as to select con 2 or car 4 He is
also twice as likely to select cons 2 on 4 as to select either car I a 5

Compute the probability that he picks one of the 3 middle cons

 Note For simple
In general
events we write
IP A Pl wi
a a Pl wi instead of
IPLwill for simplicity
Exercise Example 2.15 in book A train has 5 cars Suppose a commuter is
twice as likely to select can 3 as to select con 2 or car 4 He is
also twice as likely to select cons 2 or 4 as to select either car I a 5

Compute the probability that he picks one of the 3 middle cons

Solution
I Carl Carl can3 Carl can51 let Ei IPLcani selected simple
Notice PLE3 2 PLED Z PLEy 4 PIE U PLEs
I
É PLEB ME ZIPLE 4 PLED 2 PLED IPE TO

4 10
ME
PLED P Es Yo IPL Ez IPE 410 PCE

PL 3 middleand PIED PIE PLED 2 2 8170 0.8

 Equiprobability

let El denote the cardinality of a set E number of elements

Suppose I is finite ie I wi un i e R l n

Def A Uniform Probability Space is a finite sample space r where all

outcomes wi er are equiprobable have the same probability

let denote this probability i.e let Pl wi p fit in

p
what is
p
I PED IE PI wi IE p n
p
I Irl p

Pl wi Yr ti h in in a uniform probability
space

How to compute the probabilities of events then First make sure that

your representation of the sample space correctly yields efuiprobability

 If A w wa wa ie I Al 3 then as before

IP A IPL w Plur PLwa 3

p
3 x 1
111
El favorable outcomes
In general H Ee I we have PLE for E
111
total outcomes
When outcomes are
equally likely
the problem reduces to counting
the number of outcomes in the event E This will be the central
topic of the next part where we explore all the combinatorial
methods that let us count this number of outcomes

Example Recall the exercise before about rolling 2 dice We want to compute
the probability of the event G the sum is equal to 7 Check the table
We have There are 6 favourable outcomes and 36 in total
qui
probability
P G 36g Yo Does this work if we choose unordered
outcomes for r
 I Interpreting Probability

You have noticed that we define probability strictly as a mathematical

object with no reference whatsoever to the concept of likelihood chance

However this definition of
probability is proved to satisfy the law of large
numbers later which verifies the following interpretation of probability

Frequentist Interpretation objective does not rely on individuals

The probability of an event A is the relative frequency of occurrence of the

event A in a number of independent and identical experiments
large

In a fair coin toss we have PLH PLT Yz If I toss the coin 10

times I
get 8 Hand 2T However in the long run one cannot
may
the odds For n I million we will have 501 Heads
escape approx
and 501 Tails
 We formalize the above A Bernoulli Trial is a random experiment with
two outcomes success with probability and failure with probability
p
I Consider nidentical and Bernoulli trials and let Sn
p independent
count the number of successes among these trials If success is defined
as occurrence of a certain event A then PLAT
p
Theorem Special case of the
Strong Law of large Numbers We have

no
PC Sun s 1
p

This means that if n A counts the occurrence of event A out of n

replications
n
IA is the relative frequency of occurrence of A and

PLA him Y
What does this entail in practice To estimate the probability of A run
a large number n of independent simulations and record NCA n this ratio
fluctuates at first but then stabilizes around PCA So for lay en

MI e PLA

Example We run the following simulation Flip a fair coin let

A be the simpleevent Heads We have etuiprobability so PLA 42
We repeat this experiment and record the relative frequency of
NCA
occurrence of A lie A as n in aliases This is the
graph