Random Events and

Probability
Prof. Miloš Stanković
Statistical experiment
• We start from the definition of the statistical (or random) experiment
to better connect the axioms of probability theory with events in the
real world that we are modeling

• Properties:
• It can be repeated many arbitrary many times under the same conditions
• All of the possible outcomes are predefined
• The outcome of any single experiment is not known in advance

• For example, a coin toss

Events
• In describing the statistical experiment we use sets:
• Each possible outcome is called elementary event (only one can occur)
• The set of all elementary events is denoted with Ω (events are the elements
of that set)
• An event is each subset of Ω - a set containing some elementary events. An
event has occurred if one of the elementary events contained in the subset
occurred
• Ω is a sure event
• Empty set ∅ is impossible event
Examples
1. In one coin toss the outcomes are 𝐻 (head) and 𝑇 (tail)

Ω = {𝐻, 𝑇}

2. Tossing of two coins: Ω = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}

• An event is e.g. A = {𝐻𝐻, 𝐻𝑇} – head in first toss
• Elementary events can be defined differently : Ω = 𝜔1 , 𝜔2 , 𝜔3
𝜔1 : “zero tails” , 𝜔2 : “one tail” , 𝜔3 : “two tails”
- “head in first toss” is not an event!

3. The measurements in engineering and physics - each measurement gives slightly different value
• Set Ω is in this case infinite (countable or uncountable)
Algebra of events
• Algebra of events is the algebra of sets
• Event A or B is denoted by 𝐴 ∪ 𝐵
• Event A and B (simultaneously) is denoted by 𝐴 ∩ 𝐵 (or AB)
• Event A or not B is denoted by 𝐴\𝐵 (or 𝐴 − 𝐵)
• The opposite event to event: 𝐴𝑐 (complement)
• 𝐴 ⊂ 𝐵 – event A implies event B

• Events are exclusive if 𝐴 ∩ 𝐵 = 𝐴𝐵 = ∅

Example
• In a box we have
• 3 white balls denoted by 1,2,3
• 5 black balls denoted by 4,5,6,7,8
• 2 blue balls denoted by 9,10
• We pick one ball, set of elementary events is:
Ω = 1,2,3,4,5,6,7,8,9,10
• Let W,B,P be the events that white, black or blue ball is picked,
respectively
• Lets define events 𝐷even = {2,4,6,8,10}, 𝐷odd = 1,3,5,7,9
• Then, 𝑊𝐵 = ∅, 𝐷even 𝑐 = 𝐷odd , 𝑃 ⊂ 𝐷even ∪ 𝐷odd = Ω
Axioms of probability
• We assign specific probabilities to events
• The following three rules the probability function 𝑃() must satisfy (Kolmogorov 1933.):

A1) 𝑃 Ω = 1
A2) 0 ≤ 𝑃 𝐴 ≤ 1 , for 𝐴 ⊂ Ω
A3) 𝑃 𝐴1 ∪ 𝐴2 ∪ ⋯ = 𝑖 𝑃(𝐴𝑖 ) if events 𝐴𝑖 are mutually exclusive

• The developed powerful mathematical apparatus is applied regardless of the selected set
of events, as well as the selected probability distribution of events
• We see that the axioms are not tied to specific probabilities of events - this mathematical
apparatus "distanced itself" from the application to the specific problems in reality
Example
• Two coins tossing Ω = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}
• Lets define probabilities in the following way:
1 1 1 1
𝑃 𝐻𝐻 = ,𝑃 𝐻𝑇 = , 𝑃 𝑇𝐻 = , 𝑃 𝑇𝑇 =
4 12 2 6
• All axioms hold
1
• We can calculate, e. g. , 𝑃 𝐻1 = 𝑃 𝐻𝐻 ∪ 𝐻𝑇 = (according to A3)
3
• The selected probability function is legitimate, but it is not a good
model of fair coins tossing!
Statistical estimation of probability
• One way to determine the probability is to repeat the statistical experiment
n times
• We record the number of occurrences of an event 𝐴: it is m(n)
• Then, it holds that
𝑚(𝑛)
𝑃 𝐴 = lim
𝑛→∞ 𝑛

• This rule can be proved starting from axioms – the law of large numbers

• An important assumption is that the experiments are repeated under

identical conditions
Classical (a priori, theoretical) determination
of the probability space
• Laplace 18th century
• Based on the assumption that elementary events are equally likely

• Let n be the number of elementary events

• If event 𝐴 contains 𝑚 elementary events, then

𝑚 Favorable outcomes for event A

𝑃 𝐴 =
𝑛
• This can be derived starting from axioms and the assumption on equally
likely events
Example – dice roll paradox
• We are rolling two dice and we want to find probability that the sum is 9
and that the sum is 10
Number of outcomes giving the sum 9
• Hence 𝑃 𝐴9 = , similar for 𝑃 𝐴10
Total number of outcomes
• 9=6+3=4+5
• 10 = 4 + 6 = 5 + 5
• Are these events equally likely?

• No! because 9 = 6 + 3 = 4 + 5 = 5 + 4 = 3 + 6 (4 equally likely

elementary events)
• 10 = 4 + 6 = 5 + 5 = 6 + 4 (3 equally likely elementary events! – 5+5
can happen in only one way!)
Properties of probability
• Derived from axioms!
• 𝑃 𝐴𝑐 = 1 − 𝑃(𝐴) B
• 𝑃 ∅ =1−𝑃 Ω =0 𝐴
B\A
• 𝑃 𝐴\B = 𝑃 𝐴 − 𝑃(𝐴𝐵) 𝐴\B
𝐴 ∩ B =AB

• ⇒ 𝑃 𝐴𝑈B = 𝑃 𝐴 + 𝑃(𝐵) − 𝑃(𝐴𝐵)

• 𝐴 ⊂ 𝐵 ⇒ 𝑃 𝐴 ≤ 𝑃(𝐵)
Properties of probability
• General equation for the probability of union of 𝑛 events:

𝑃 𝐴𝑛1 ∪ 𝐴2 ∪ ⋯ 𝐴𝑛
𝑛+1
= 𝑃(𝐴𝑖 ) − 𝑃 𝐴𝑖 𝐴𝑗 + 𝑃 𝐴𝑖 𝐴𝑗 𝐴𝑘 + ⋯ + −1 𝑃 𝐴1 𝐴2 … 𝐴𝑛
𝑖=1 1≤𝑖<𝑗≤𝑛 1≤𝑖<𝑗<𝑘≤𝑛

• Boole’s inequality 𝑛

𝑃 𝐴1 ∪ 𝐴2 ∪ ⋯ 𝐴𝑛 ≤ 𝑃(𝐴𝑖 )
𝑖=1
Example
• What is the probability that, when rolling a dice, we get a number
that is divisible by 2 or 3?
• Let A be the event that the obtained number is divisible by 2, and B
the event that the obtained number is divisible by 3. Events A and B
are not mutually exclusive - 6 is divisible by both 2 and 3

3 2 1 2
• Hence, 𝑃 𝐴𝑈B = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴𝐵 = + − =
6 6 6 3
Application of combinatorics
• To obtain probabilities when dealing with experiments with equally likely
outcomes, counting the outcomes is needed
• Sometimes we can use computer, but combinatorial equations give precise
answers
• All equation are derived from two rules:
• Rule of summation: If an object of type 1 can be chosen in 𝑛1 ways, an object of type
2 in 𝑛2 ways, ... , object of type 𝑘 in 𝑛𝑘 ways, then an object of any type can be
chosen in 𝒏𝟏 + 𝒏𝟐 + ⋯ + 𝒏𝒌 ways.

• Product rule: If an object of type 1 can be chosen in 𝑛1 ways, after that an object of
type 2 in 𝑛2 ways, ... , and at the end object of type 𝑘 in 𝑛𝑘 ways, then, k objects one
by one can be chosen in 𝑛1 𝑛2 ⋯ 𝑛𝑘 ways.
Example
• In a box we have 2 red balls, 3 white and 6 blue, number of ways to
choose one ball (of any type) is 10

• We toss 3 coins one by one. What is the probability of obtaining

exactly 2 tails?

• Product rule: total number of outcomes is 2 ⋅ 2 ⋅ 2 = 8

• Number of favorable outcomes id 3 – HHT, HTH, THH
3
• Hence, the probability is
8
Example
• In a box we have 5 white, 3 black and 2 red balls. 3 balls are taken
randomly with return. What is the probability that first ball is white
and second black?

• Total number of outcomes 103 = 1000

• Number of favorable outcomes: first white ball can be chosen in 5
ways, second black in 3, and third doesn’t matter - 10 ways. Hence,
5 ⋅ 3 ⋅ 10
5⋅3⋅10
• The wanted probability is = 0.15
1000
Variations
• Variations with repetition:
• From a box of 𝑛 balls we take 𝑘 with return. What is the number of ways of
doing this?

• Solution: 𝑛𝑘

• Variations without repetition (k-permutations):

• From a box of 𝑛 balls we take 𝑘 without returning. What is the number of
ways of doing this?
𝑛!
• Solution: 𝑛 𝑛 − 1 … 𝑛 − 𝑘 + 1 =
(𝑛−𝑘)!
Permutations
• Permutations without repetition:
• In how many ways we can place n different objects to n places?

• Solution: 𝑛! = 𝑛 𝑛 − 1 𝑛 − 2 ⋯ 2 ⋅ 1

• Permutations with repetition:

• We have r types of objects, 𝑛𝑖 objects of type 𝑖. Total number of objects is n. In how
many ways we can place them to n places?

• Solution: If objects were different - 𝑛! ways. Since objects of the same type do not
differ – their permuting doesn’t change the outcome. Number of these permutaions
is 𝑛1 ! 𝑛2 ! … 𝑛𝑟 ! .
𝑛!
• Hence, the solution is
𝑛1 !𝑛2 !…𝑛𝑟 !
Combinations without repetitions
• In how many ways we can choose k elements from a set of n
elements?

• Solution: The same as variations without repetitions, but the order is

irrelevant. Hence, 𝑘! ways in which we can permute k elements are
treated as the same outcome:
𝑛 𝑛−1 … 𝑛−𝑘+1 𝑛! 𝑛
= =
𝑘! 𝑛 − 𝑘 ! 𝑘! 𝑘
Example
• In a wardrobe we have 10 different pairs of shoes. We randomly take
4 shoes. What is the probability that among them there is at least one
pair of the same shoes?

10 4
2 99
4
• Solution: 1 − 20 =
323
4