SAMPLE SPACES

Learning Objectives

1. To learn the concept of the sample space associated with a

random experiment.
2. To learn the concept of an event associated with a random
experiment.
3. To learn the concept of the probability of an event and how to
compute it.
Source: Lectures on Probability and Statistics by Dr. Eusebius Doedel
 SAMPLE SPACES

DEFINITION :
The sample space is the set of all possible outcomes of an experiment.
EXAMPLE : When we flip a coin then sample space is
S = {H , T },
where
H denotes that the coin lands ”Heads up”
and
T denotes that the coin lands ”Tails up”.

For a ”fair coin ” we expect H and T to have the same ”chance ” of

occurring, i.e., if we flip the coin many times then about 50 % of the
outcomes will be H.
We say that the probability of H to occur is 0.5 (or 50 %) .
The probability of T to occur is then also 0.5.

1
EXAMPLE :

When we roll a fair die then the sample space is

S = { ......., ........, ........., ........, ........., ......... } .

The probability the die lands with k up , (k = 1, 2, · · · , 6).

is ?

When we roll it 1200 times we expect a 5 up about ? times.

What is the probability it lands with an odd number up ?

2
Application :
When we toss a coin 3 times and record the results in the sequence
that they occur, then the sample space is

S = ?

Elements of S are ”vectors ”, ”sequences ”, or ”ordered outcomes ”.

We may expect each of the outcomes to be equally likely.

Thus the probability of any sequence is ?

What is the probability of a sequence to contain precisely two Heads

3
EXAMPLE : When we toss a coin 3 times and record the results
without paying attention to the order in which they occur, e.g., if we
only record the number of Heads, then the sample space is
 
S = {H, H, H} , {H, H, T } , {H, T, T } , {T, T, T } .
The outcomes in S are now sets ; i.e., order is not important.
Recall that the ordered outcomes are
{ HHH , HHT , HT H , HT T , T HH , T HT , T T H , T T T } .

Note that
{H, H, H} corresponds to one of the ordered outcomes,
{H, H, T } ,, three ,,
{H, T, T } ,, three ,,
{T, T, T } ,, one ,,

Thus {H, H, H} and {T, T, T } each occur with probability 18 ,

while {H, H, T } and {H, T, T } each occur with probability 38 .

4
Events

In Probability Theory subsets of the sample space are called events.

EXAMPLE : The set of basic outcomes of rolling a die once is

S = {1, 2, 3, 4, 5, 6},

so the subset E = { 2 , 4 , 6 } is an example of an event.

If a die is rolled once and it lands with a 2 or a 4 or a 6 up then we

say that the event E has occurred.

We have already seen that the probability that E occurs is

1 1 1 1
P (E) = + + = .
6 6 6 2

5
The Algebra of Events

Since events are sets, namely, subsets of the sample space S, we can
do the usual set operations :

If E and F are events then we can form

Ec the complement of E
E∪F the union of E and F
EF the intersection of E and F

We write E ⊂ F if E is a subset of F .

REMARK : In Probability Theory we use

Ec instead of Ē ,

EF instead of E ∩ F ,

E⊂F instead of E ⊆ F .

6
If the sample space S is finite then we typically allow any subset of
S to be an event.

EXAMPLE : If we randomly draw one character from a box con-

taining the characters a, b, and c, then the sample space is
S = {a , b , c} ,
and there are 8 possible events, namely, those in the set of events
 
E = { } , {a} , {b} , {c} , {a, b} , {a, c} , {b, c} , {a, b, c} .

If the outcomes a, b, and c, are equally likely to occur, then

1 1 1
P ({ }) = 0 , P ({a}) = , P ({b}) = , P ({c}) = ,
3 3 3
2 2 2
P ({a, b}) = , P ({a, c}) = , P ({b, c}) = , P ({a, b, c}) = 1 .
3 3 3
For example, P ({a, b}) is the probability the character is an a or a b.

7
We always assume that the set E of allowable events includes the
complements, unions, and intersections of its events.

EXAMPLE : If the sample space is

S = {a , b , c , d} ,

and we start with the events

 
E0 = {a} , {c, d} ,

then this set of events needs to be extended to (at least)

 
E = { } , {a} , {c, d} , {b, c, d} , {a, b} , {a, c, d} , {b} , {a, b, c, d} .

EXERCISE : Verify E includes complements, unions, intersections.

8
Axioms of Probability

A probability function P assigns a real number (the probability of E)

to every event E in a sample space S.

P (·) must satisfy the following basic properties :

• 0 ≤ P (E) ≤ 1 ,

• P (S) = 1 ,

• For any disjoint events Ei , i = 1, 2, · · · , n, we have

P (E1 ∪ E2 ∪ · · · ∪ En ) = P (E1 ) + P (E2 ) + · · · P (En ) .

9
Further Properties

PROPERTY 1 :

P (E ∪ E c ) = P (E) + P (E c ) = 1 . ( Why ? )

Thus
P (E c ) = 1 − P (E) .

EXAMPLE :
What is the probability of at least one ”H” in four tosses of a coin?

HINT : The sample space S will have 16 outcomes. (Which?)

Find : P (at least one H) =

10
PROPERTY 2 :
P (E ∪ F ) = P (E) + P (F ) − P (EF ) .

PROOF (using the third axiom) :

P (E ∪ F ) = P (EF ) + P (EF c ) + P (E c F )

= [P (EF ) + P (EF c )] + [P (EF ) + P (E c F )] − P (EF )

= P(E) + P(F) - P(EF) . ( Why ? )

NOTE :
• Draw a Venn diagram with E and F to see this !

• The formula is similar to the one for the number of elements :

n(E ∪ F ) = n(E) + n(F ) − n(EF ) .

11
So far our sample spaces S have been finite.
S can also be countably infinite, e.g., the set Z of all integers.
S can also be uncountable, e.g., the set R of all real numbers.

EXAMPLE : Record the low temperature in Montreal on January

8 in each of a large number of years.
We can take S to be the set of all real numbers, i.e., S = R.
(Are there are other choices of S ?)
What probability would you expect for the following events to have?

(a) P ({π}) (b) P ({x : − π < x < π})

(How does this differ from finite sample spaces?)

We will encounter such infinite sample spaces many times · · ·

12
Counting Outcomes

We have seen examples where the outcomes in a finite sample space

S are equally likely , i.e., they have the same probability .

Such sample spaces occur quite often.

Computing probabilities then requires counting all outcomes and

counting certain types of outcomes .

The counting has to be done carefully!

We will discuss a number of representative examples in detail.

Concepts that arise include permutations and combinations.

13
Permutations

• Here we count of the number of ”words ” that can be formed

from a collection of items (e.g., letters).

• (Also called sequences , vectors , ordered sets .)

• The order of the items in the word is important;

e.g., the word acb is different from the word bac .

• The word length is the number of characters in the word.

NOTE :
For sets the order is not important. For example, the set {a,c,b} is
the same as the set {b,a,c} .

14
EXAMPLE : Suppose that four-letter words of lower case alpha-
betic characters are generated randomly with equally likely outcomes.
(Assume that letters may appear repeatedly.)

(a) How many four-letter words are there in the sample space S ?
SOLUTION :

(b) How many four-letter words are there are there in S that start
with the letter ”s ” ?
SOLUTION : .

(c) What is the probability of generating a four-letter word that

starts with an ”s ” ?
SOLUTION :

Could this have been computed more easily?

15
EXAMPLE : How many re-orderings (permutations) are there of
the string abc ? (Here letters may appear only once.)
SOLUTION : Six, namely, abc , acb , bac , bca , cab , cba .

If these permutations are generated randomly with equal probability

then what is the probability the word starts with the letter ”a ” ?
SOLUTION :
2 1
= .
6 3

EXAMPLE : In general, if the word length is n and all characters

are distinct then there are n! permutations of the word. ( Why ? )

If these permutations are generated randomly with equal probability

then what is the probability the word starts with a particular letter ?

SOLUTION :
(n − 1)! 1
= . ( Why ? )
n! n

16
EXAMPLE : How many

words of length k

can be formed from

a set of n (distinct) characters ,

(where k ≤ n ) ,

when letters can be used at most once ?

SOLUTION :

n!
( Why ? )
(n − k)!

17
EXAMPLE : Three-letter words are generated randomly from the
five characters a , b , c , d , e , where letters can be used at most
once.

(a) How many three-letter words are there in the sample space S ?
SOLUTION : 6 0

(b) How many words containing a , b are there in S ?

SOLUTION : First place the characters
a,b
i.e., select the two indices of the locations to place them.
This can be done in
3 × 2 = 6 ways . ( Why ? )
There remains one position to be filled with a c , d or an e .
Therefore the number of words is 3 × 6 = 18 .

18
(c) Suppose the 60 solutions in the sample space are equally likely .

What is the probability of generating a three-letter word that

contains the letters a and b ?

SOLUTION :
18
= 0.3 .
60

19
EXERCISE :

Suppose the sample space S consists of all five-letter words

having distinct alphabetic characters .

• How many words are there in S ?

• How many ”special” words are in S for which only the second
and the fourth character are vowels, i.e., one of {a, e, i, o, u, y} ?

• Assuming the outcomes in S to be equally likely, what is the

probability of drawing such a special word?

20
Combinations

Let S be a set containing n (distinct) elements.

Then
a combination of k elements from S ,
is
any selection of k elements from S ,

where order is not important .

(Thus the selection is a set .)

NOTE : By definition a set always has distinct elements .

21
EXAMPLE :

There are three combinations of 2 elements chosen from the set

S = {a , b , c} ,

namely, the subsets

{a, b} , {a, c} , {b, c} ,

whereas there are six words of 2 elements from S ,

namely,

ab , ba , ac , ca , bc , cb .

22
In general, given
a set S of n elements ,

the number of possible subsets of k elements from S equals

 
n n!
≡ .
k k! (n − k)!

 
n
REMARK : The notation is referred to as
k
”n choose k ”.

 
n n! n!
NOTE : = = = 1,
n n! (n − n)! n! 0!

since 0! ≡ 1 (by “convenient definition” !) .

23
PROOF :

First recall that there are

n!
n (n − 1) (n − 2) · · · (n − k + 1) =
(n − k)!

possible sequences of k distinct elements from S .

However, every sequence of length k has k! permutations of itself,

and each of these defines the same subset of S.

Thus the total number of subsets is

 
n! n
≡ .
k! (n − k)! k

24
EXAMPLE :
In the previous example, with 2 elements chosen from the set

{a , b , c} ,

we have n = 3 and k = 2 , so that there are

3!
= 6 words ,
(3 − 2)!
namely
ab , ba , ac , ca , bc , cb ,
while there are
 
3 3! 6
≡ = = 3 subsets ,
2 2! (3 − 2)! 2
namely
{a, b} , {a, c} , {b, c} .

25
EXAMPLE : If we choose 3 elements from {a , b , c , d} , then
n = 4 and k = 3 ,
so there are
4!
= 24 words, namely :
(4 − 3)!

abc , abd , acd , bcd ,

acb , adb , adc , bdc ,
bac , bad , cad , cbd ,
bca , bda , cda , cdb ,
cab , dab , dac , dbc ,
cba , dba , dca , dcb ,
while there are
 
4 4! 24
≡ = = 4 subsets ,
3 3! (4 − 3)! 6
namely,
{a, b, c} , {a, b, d} , {a, c, d} , {b, c, d} .

26
EXAMPLE :

(a) How many ways are there to choose a committee of 4 persons

from a group of 10 persons, if order is not important?
SOLUTION :
 
10 10!
= = 210 .
4 4! (10 − 4)!

(b) If each of these 210 outcomes is equally likely then what is the
probability that a particular person is on the committee?
SOLUTION :
   
9 10 84 4
/ = = . ( Why ? )
3 4 210 10

Is this result surprising?

27
(c) What is the probability that a particular person is not on the
committee?
SOLUTION :
   
9 10 126 6
/ = = . ( Why ? )
4 4 210 10

Is this result surprising?

(d) How many ways are there to choose a committee of 4 persons

from a group of 10 persons, if one is to be the chairperson?
SOLUTION :
     
10 9 9 9!
= 10 = 10 = 840 .
1 3 3 3! (9 − 3)!

QUESTION : Why is this four times the number in (a) ?

28
EXAMPLE : Two balls are selected at random from a bag with
four white balls and three black balls, where order is not important.

What would be an appropriate sample space S ?

SOLUTION : Denote the set of balls by

B = {w1 , w2 , w3 , w4 , b1 , b2 , b3 } ,
where same color balls are made “distinct” by numbering them.

Then a good choice of the sample space is

S = the set of all subsets of two balls from B ,
because the wording ”selected at random ” suggests that each such
subset has the same chance to be selected.

The number of outcomes in S (which are sets of two balls) is then

 
7
= 21 .
2

29
EXAMPLE : ( continued · · · )
(Two balls are selected at random from a bag with four white balls
and three black balls.)
• What is the probability that both balls are white?
SOLUTION :    
4 7 6 2
/ = = .
2 2 21 7

• What is the probability that both balls are black?

SOLUTION :    
3 7 3 1
/ = = .
2 2 21 7

• What is the probability that one is white and one is black?

SOLUTION :      
4 3 7 4·3 4
/ = = .
1 1 2 21 7
(Could this have been computed differently?)

30
EXAMPLE : ( continued · · · )
In detail, the sample space S is

{w1 , w2 }, {w1 , w3 }, {w1 , w4 }, | {w1 , b1 }, {w1 , b2 }, {w1 , b3 },
{w2 , w3 }, {w2 , w4 }, | {w2 , b1 }, {w2 , b2 }, {w2 , b3 },
{w3 , w4 }, | {w3 , b1 }, {w3 , b2 }, {w3 , b3 },
| {w4 , b1 }, {w4 , b2 }, {w4 , b3 },
———– ———– ———–
{b1 , b2 }, {b1 , b3 }, 
• S has 21 outcomes, each of which is a set . {b2 , b3 }
1
• We assumed each outcome of S has probability 21
.
• The event ”both balls are white” contains 6 outcomes.
• The event ”both balls are black” contains 3 outcomes.
• The event ”one is white and one is black” contains 12 outcomes.
• What would be different had we worked with sequences ?

31
EXERCISE :

Three balls are selected at random from a bag containing

2 red , 3 green , 4 blue balls .

What would be an appropriate sample space S ?

What is the the number of outcomes in S ?

What is the probability that all three balls are red ?

What is the probability that all three balls are green ?

What is the probability that all three balls are blue ?

What is the probability of one red, one green, and one blue ball ?

32
EXAMPLE : A bag contains 4 black balls and 4 white balls.
Suppose one draws two balls at the time, until the bag is empty.
What is the probability that each drawn pair is of the same color?

SOLUTION : An example of an outcome in the sample space S is

 
{w1 , w3 } , {w2 , b3 } , {w4 , b1 } , {b2 , b4 } .

The number of such doubly unordered outcomes in S is

    
1 8 6 4 2 1 8! 6! 4! 2! 1 8!
= = 4
= 105 (Why?)
4! 2 2 2 2 4! 2! 6! 2! 4! 2! 2! 2! 0! 4! (2!)
The number of such outcomes with pairwise the same color is
     
1 4 2 1 4 2
· = 3 · 3 = 9. ( Why ? )
2! 2 2 2! 2 2

Thus the probability each pair is of the same color is 9/105 = 3/35 .

33
EXAMPLE : ( continued · · · )
The 9 outcomes of pairwise the same color constitute the event

 
{w1 , w2 } , {w3 , w4 } , {b1 , b2 } , {b3 , b4 } ,
 
{w1 , w3 } , {w2 , w4 } , {b1 , b2 } , {b3 , b4 } ,
 
{w1 , w4 } , {w2 , w3 } , {b1 , b2 } , {b3 , b4 } ,
 
{w1 , w2 } , {w3 , w4 } , {b1 , b3 } , {b2 , b4 } ,
 
{w1 , w3 } , {w2 , w4 } , {b1 , b3 } , {b2 , b4 } ,
 
{w1 , w4 } , {w2 , w3 } , {b1 , b3 } , {b2 , b4 } ,
 
{w1 , w2 } , {w3 , w4 } , {b1 , b4 } , {b2 , b3 } ,
 
{w1 , w3 } , {w2 , w4 } , {b1 , b4 } , {b2 , b3 } ,

 
{w1 , w4 } , {w2 , w3 } , {b1 , b4 } , {b2 , b3 } .

34
EXERCISE :

• How many ways are there to choose a committee of 4 persons

from a group of 6 persons, if order is not important?
• Write down the list of all these possible committees of 4 persons.
• If each of these outcomes is equally likely then what is the
probability that two particular persons are on the committee?

EXERCISE :

Two balls are selected at random from a bag with three white balls
and two black balls.

• Show all elements of a suitable sample space.

• What is the probability that both balls are white?

35
EXERCISE :

We are interested in birthdays in a class of 60 students.

• What is a good sample space S for this purpose?

• How many outcomes are there in S ?

• What is the probability of no common birthdays in this class?

• What is the probability of common birthdays in this class?

36
EXAMPLE :
How many nonnegative integer solutions are there to

x1 + x2 + x3 = 17 ?

SOLUTION :
Consider seventeen 1’s separated by bars to indicate the possible
values of x1 , x2 , and x3 , e.g.,

111|111111111|11111 .

The total number of positions in the “display” is 17 + 2 = 19 .

The total number of nonnegative solutions is now seen to be

 
19 19! 19 × 18
= = = 171 .
2 (19 − 2)! 2! 2

37
EXAMPLE :
How many nonnegative integer solutions are there to the inequality

x1 + x2 + x3 ≤ 17 ?

SOLUTION :
Introduce an auxiliary variable (or ”slack variable ” )

x4 ≡ 17 − (x1 + x2 + x3 ) .

Then
x1 + x2 + x3 + x4 = 17 .

Use seventeen 1’s separated by 3 bars to indicate the possible values

of x1 , x2 , x3 , and x4 , e.g.,

111|11111111|1111|11 .

38
 111|11111111|1111|11 .

The total number of positions is

17 + 3 = 20 .

The total number of nonnegative solutions is therefore

 
20 20! 20 × 19 × 18
= = = 1140 .
3 (20 − 3)! 3! 3×2

39
EXAMPLE :
How many positive integer solutions are there to the equation
x1 + x2 + x3 = 17 ?

SOLUTION : Let
x1 = x̃1 + 1 , x2 = x̃2 + 1 , x3 = x̃3 + 1 .

Then the problem becomes :

How many nonnegative integer solutions are there to the equation
x̃1 + x̃2 + x̃3 = 14 ?
111|111111111|11
The solution is
 
16 16! 16 × 15
= = = 120 .
2 (16 − 2)! 2! 2

40
EXAMPLE :
What is the probability the sum is 9 in three rolls of a die ?

SOLUTION : The number of such sequences of three rolls with

sum 9 is the number of integer solutions of
x1 + x2 + x3 = 9 ,
with
1 ≤ x1 ≤ 6 , 1 ≤ x2 ≤ 6 , 1 ≤ x3 ≤ 6 .
Let
x1 = x̃1 + 1 , x2 = x̃2 + 1 , x3 = x̃3 + 1 .

Then the problem becomes :

How many nonnegative integer solutions are there to the equation
x̃1 + x̃2 + x̃3 = 6 ,
with
0 ≤ x̃1 , x̃2 , x̃3 ≤ 5 .

41
EXAMPLE : ( continued · · · )
Now the equation
x̃1 + x̃2 + x̃3 = 6 , ( 0 ≤ x̃1 , x̃2 , x̃3 ≤ 5 ) ,

1|111|11
has  
8
= 28 solutions ,
2

from which we must subtract the 3 impossible solutions

(x̃1 , x̃2 , x̃3 ) = (6, 0, 0) , (0, 6, 0) , (0, 0, 6) .
111111|| , |111111| , ||111111

Thus the probability that the sum of 3 rolls equals 9 is

28 − 3 25 ∼
= = 0.116 .
63 216

42
EXAMPLE : ( continued · · · )

The 25 outcomes of the event ”the sum of the rolls is 9” are

{ 126 , 135 , 144 , 153 , 162 ,

216 , 225 , 234 , 243 , 252 , 261 ,

315 324 , 333 , 342 , 351 ,

414 , 423 , 432 , 441 ,

513 , 522 , 531 ,

612 , 621 }.

The ”lexicographic” ordering of the outcomes (which are sequences)

in this event is used for systematic counting.

43
EXERCISE :

• How many integer solutions are there to the inequality

x1 + x2 + x3 ≤ 17 ,

if we require that

x1 ≥ 1 , x2 ≥ 2 , x3 ≥ 3 ?

EXERCISE :

What is the probability that the sum is less than or equal to 9

in three rolls of a die ?

44