Probability

Relative frequency vs degree of belief

 Give me an example
of a random
Random Experiment experiment

› An experiment is a procedure that is

– carried out under controlled conditions, and
– executed to discover an unknown result

Section 2.1.1 Random Experiment

2
Sample Spaces

The set of all possible outcomes of a random experiment is

called the sample space, 𝑆𝑆.

𝑆𝑆 is discrete if it consists of a finite or countable infinite set of

outcomes.
𝑆𝑆 is continuous if it contains an interval (either finite or
infinite) of real numbers.

Section 2.1.2 Sample Spaces

3
 The time taken
to ready the
camera for

Example 2.1 | Camera Flash

another flash

› Randomly select a camera and record the recycle time of a

flash. 𝑆𝑆 = 𝑅𝑅+ = {𝑥𝑥 | 𝑥𝑥 > 0}, the positive real numbers.
› Suppose it is known that all recycle times are between 1.5 and
5 seconds. Then 𝑆𝑆 = {𝑥𝑥 | 1.5 < 𝑥𝑥 < 5} is continuous.
› It is known that the recycle time has only three values (low,
medium or high). Then 𝑆𝑆 = {𝑙𝑙𝑙𝑙𝑙𝑙, 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚, ℎ𝑖𝑖𝑖𝑖𝑖} is discrete.
› Does the camera conform to minimum recycle time
specifications? Suppose, there are
2 cameras. What is
𝑆𝑆 = {𝑦𝑦𝑦𝑦𝑦𝑦, 𝑛𝑛𝑛𝑛} is discrete. the sample space?

Section 2.1.2 Sample Spaces

4
Example 2.3 | Message Delays

3 Messages are classified as on-time or late within the time

specified by the system design. Use a tree diagram to represent
the sample space of possible outcomes.

Section 2.1.2 Sample Spaces

5
Events

Recall set
operations

Event combinations
– The union of two events consists of all outcomes that are contained in either of the
two events, denoted as 𝐸𝐸1 ∪ 𝐸𝐸2.
– The intersection of two events is the event that consists of all outcomes that are
contained in both of the two events, denoted as 𝐸𝐸1 ∩ 𝐸𝐸2.
– The complement of an event in a sample space is the set of outcomes in the sample
space that are not in the event. We denote the complement of the event 𝐸𝐸 as 𝐸𝐸𝐸.

Section 2.1.3 Events

6
Example 2.4 | Events
› Suppose that the recycle times of two cameras are recorded. Consider only whether or not
the cameras conform to the manufacturing specifications. We abbreviate yes and no as y
and n. Consider the sample space 𝑆𝑆 = {𝑦𝑦𝑦𝑦, 𝑦𝑦𝑦𝑦, 𝑛𝑛𝑛𝑛, 𝑛𝑛𝑛𝑛}.
– Suppose that the subset of outcomes for which at least one camera conforms is denoted as
𝐸𝐸1. Then, 𝐸𝐸1 = {𝑦𝑦𝑦𝑦, 𝑦𝑦𝑦𝑦, 𝑛𝑛𝑛𝑛}.
– Suppose that the subset of outcomes for which both cameras do not conform, denoted as
𝐸𝐸2, contains only the single outcome, 𝐸𝐸2 = {𝑛𝑛𝑛𝑛}.
– Other examples of events are 𝐸𝐸3 = Ø, the null set, and 𝐸𝐸4 = 𝑆𝑆, the sample space.
– If 𝐸𝐸5 = {𝑦𝑦𝑦𝑦, 𝑛𝑛𝑛𝑛, 𝑛𝑛𝑛𝑛}, 𝐸𝐸1 ∪ 𝐸𝐸5 = 𝑆𝑆, 𝐸𝐸1 ∩ 𝐸𝐸5 = 𝑦𝑦𝑦𝑦, 𝑛𝑛𝑛𝑛 , 𝐸𝐸1′ = 𝑛𝑛𝑛𝑛

Section 2.1.3 Events

7
Example 2.5 | Camera Recycle Time
Camera recycle times might use the sample space S = R+.
Let E1 = {x | 10 ≤ x < 12} and E2 = {x | 11 < x < 15}
Then,
and
Also

And

Section 2.1.3 Events

8
 Venn Diagrams
We can use Venn diagrams to represent a sample space and events
in a sample space.

Section 2.1.3 Events

9
 They cannot
happen
simultaneously i.e.
Mutually Exclusive Events no overlap, no
outcome is shared

Section 2.1.3 Events

Mutually Exclusive Events - Laws
› The definition of the complement of an event implies that 𝐸𝐸 ′ ′
= 𝐸𝐸

› The distributive law for set operations implies that

𝐴𝐴 ∪ 𝐵𝐵 ∩ 𝐶𝐶 = 𝐴𝐴 ∩ 𝐶𝐶 ∪ (𝐵𝐵 ∩ 𝐶𝐶) and 𝐴𝐴 ∩ 𝐵𝐵 ∪ 𝐶𝐶 = 𝐴𝐴 ∪ 𝐶𝐶 ∩ (𝐵𝐵 ∪ 𝐶𝐶)

› DeMorgan’s laws imply that

𝐴𝐴 ∪ 𝐵𝐵 ′
= 𝐴𝐴′ ∩ 𝐵𝐵𝐵 and 𝐴𝐴 ∩ 𝐵𝐵 ′
= 𝐴𝐴′ ∪ 𝐵𝐵𝐵

› Also, remember that

𝐴𝐴 ∩ 𝐵𝐵 = 𝐵𝐵 ∩ 𝐴𝐴 and 𝐴𝐴 ∪ 𝐵𝐵 = 𝐵𝐵 ∪ 𝐴𝐴

Section 2.1.3 Events

11
Counting Techniques
• In complicated examples, determining the outcomes in the sample space (or an
event) can become difficult
• In these cases, counts of the numbers of outcomes in the sample space and
various events are used to analyze the random experiments.
• These methods are referred to as counting techniques.
• Multiplication Rule
• Permutations
• Combinations

Section 2.2 Counting Techniques

12
Example 2.7 | Web Site Design

The design for a Website is to consist of four colors, three

fonts, and three positions for an image.

From the multiplication rule, 4 x 4 x 3 = 36 different designs

are possible
Practical Interpretation: The use of the multiplication rule
and other counting techniques enables one to easily
determine the number of outcomes in a sample space or
event and this, in turn, allows probabilities of events to
be determined
Section 2.2 Counting Techniques

13
Counting – Multiplication Rule
Assume an operation can be described as a sequence of k steps, and
› The number of ways to complete step 1 is n1, and
› The number of ways to complete step 2 is n2 for each way to complete step
1, and
› The number of ways to complete step 3 is n3 for each way to complete step
2, and so fourth

The total number of ways to complete the operation is 𝑛𝑛1 × 𝑛𝑛2 × ⋯ × 𝑛𝑛𝑘𝑘

Section 2.2 Counting Techniques

14
Example 2.8 | Printed Circuit Board

A printed circuit board has eight different locations in which

a component can be placed. If four different components
are to be placed on the board, how many different designs
are possible?

Answer: P48 = 8! 8 ⋅ 7 ⋅ 6 ⋅ 5 ⋅ 4!
= = 8 ⋅ 7 ⋅ 6 ⋅ 5 = 1,680 different designs are possible
(8 − 4)! 4!

n! = n(n-1)(n-2)… Section 2.2 Counting Techniques

15
Counting – Permutations
Permutations of Subsets
The number of permutations of subsets of r elements selected from a set of n
different elements is

n!
P = n(n − 1)(n − 2)...(n − r + 1)=
n

(n − r )!
r

Section 2.2 Counting Techniques

16
Counting – Permutations

A permutation of the elements is an ordered sequence of the elements

Consider a set of elements, such as S = {a, b, b}.

abc, acb, bac, bca, cab, cba are all the permutations of the
elements of S.

Section 2.2 Counting Techniques

17
Example 2.9 | Hospital Schedule
A hospital operating room needs to schedule three knee surgeries and two hip
surgeries in one day. We denote a knee surgery as k, and hip surgery as h. The
number of possible sequences of three knee and hip surgeries is

The 10 sequences are easily summarized as

{𝑘𝑘𝑘𝑘𝑘𝑘𝑘𝑘, 𝑘𝑘𝑘𝑘𝑘𝑘𝑘𝑘, 𝑘𝑘𝑘𝑘𝑘𝑘𝑘𝑘, 𝑘𝑘𝑘𝑘𝑘𝑘𝑘𝑘, 𝑘𝑘𝑘𝑘𝑘𝑘𝑘𝑘, 𝑘𝑘𝑘𝑘𝑘𝑘𝑘𝑘, ℎ𝑘𝑘𝑘𝑘𝑘𝑘𝑘, ℎ𝑘𝑘𝑘𝑘𝑘𝑘𝑘, ℎ𝑘𝑘𝑘𝑘𝑘𝑘𝑘, ℎℎ𝑘𝑘𝑘𝑘𝑘𝑘}

Section 2.2 Counting Techniques

18
Counting – Permutations
Permutations of Similar Objects

The number of permutations of n = n1 + n2 + … + nr objects of which 𝑛𝑛1 are of one

type, 𝑛𝑛2 are of a second type, … , and 𝑛𝑛𝑟𝑟 are of an 𝑟𝑟 𝑡𝑡𝑡 type is

n!
n1 ! n2 ! ... nr !

Section 2.2 Counting Techniques

19
Example 2.11a | Sampling without
Replacement
› A bin of 50 parts contains 3 defectives and 47 non-defective parts. A sample of 6
parts is selected from the 50 without replacement. How many samples of size 6
contain 2 defective parts?
› The first step is to select 2 parts form the 3 defective parts in the bin The number
of different ways this step can be completed is

Section 2.2 Counting Techniques

20
Example 2.11b | Sampling without
Replacement

The second step is to select the remaining 4 parts form the 47 acceptable parts in
the bin. The number of different ways the second step can be completed is

Section 2.2 Counting Techniques

21
Example 2.11c | Sampling without
Replacement
Therefore, from the multiplication rule, the number of subsets of size
6 that contain exactly 2 defective parts is

As an additional computation, the total number of different subsets

of size 6 is

Section 2.2 Counting Techniques

22
 Simply choose
(instead of

Counting – Combinations
arrange in any
order)

› In combinations, order is not important

Section 2.2 Counting Techniques

23
Interpretations and Axioms of Probability
› Probability is used to quantify the likelihood, or chance, that an outcome of a
random experiment will occur
› The likelihood of an outcome is quantified by assigning a number from the
interval [0,1] to the outcome (or a percentage from 0 to 100%)
› 0 indicates an outcome will not occur
› 1 indicates that an outcome will occur with certainty

Section 2.3 Interpretations and Axioms of Probability

24
Interpretations and Axioms of Probability

• Subjective probability, or degree of belief

• Different individuals will no doubt assign different probabilities to the same
outcomes
• Ex: “The chance of rain today is 30%”

• Relative frequency probability

• Interpreted as the limiting value of the proportion of times the outcome
occurs in n repetitions of the random experiment as n increases beyond
all bounds
• Ex: If we assign probability 0.2 to the outcome that there is a corrupted
pulse in a digital signal, we might interpret this assignment as implying that,
if we analyze many pulses, approximately 20% of them will be corrupted

Section 2.3 Interpretations and Axioms of Probability

25
Interpretations and Axioms of Probability

For example, suppose that we select 1 laser diode randomly from a batch of
100. Randomly implies that it is reasonable to assume that each diode in the
batch has an equal chance of being selected. The probability model for this
experiment assigns probability of 0.01 to each of the 100 outcomes, because
each outcome in the sample space is equally likely

Section 2.3 Interpretations and Axioms of Probability

26
Interpretations and Axioms of Probability
For a discrete sample space, the probability of an event can be defined by the
reasoning used in the preceding example

Section 2.3 Interpretations and Axioms of Probability

27
Example 2.13 | Probabilities of Events
A random experiment can result in one of the outcomes {a, b, c, d} with probabilities 0.1, 0.3,
0.5, 0.1, respectively
Let A denote the event {a, b}, B the event {b, c, d}, and C the event {d}

𝑃𝑃(𝐴𝐴) = 0.1 + 0.3 = 0.4

𝑃𝑃(𝐵𝐵) = 0.3 + 0.5 + 0.1 = 0.9
𝑃𝑃(𝐶𝐶) = 0.1
𝑃𝑃(𝐴𝐴′ ) = 0.6 𝑎𝑎𝑎𝑎𝑎𝑎 𝑃𝑃(𝐵𝐵′ ) = 0.1 𝑎𝑎𝑎𝑎𝑎𝑎 𝑃𝑃(𝐶𝐶′ ) = 0.9

Because 𝐴𝐴 ∩ 𝐵𝐵 = {𝑏𝑏}, 𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵) = 0.3

Because 𝐴𝐴 ∪ 𝐵𝐵 = {𝑎𝑎, 𝑏𝑏, 𝑐𝑐, 𝑑𝑑}, 𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑃𝑃(𝐴𝐴 ∪ 𝐵𝐵) = 1.0
Because 𝐴𝐴 ∩ 𝐶𝐶 = {𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛}, 𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑃𝑃(𝐴𝐴 ∩ 𝐶𝐶 ) = 0

Section 2.3 Interpretations and Axioms of Probability

28
Axioms of Probability
Probability is a number that is assigned to each member of a collection of events
from a random experiment that satisfies the following properties:
1. P(S) = 1
2. 0 ≤ P(E) ≤ 1
3. For any two events E1 and E2 with 𝐸𝐸1 ∩ 𝐸𝐸2 = Ø,
𝑃𝑃(𝐸𝐸1 ∪ 𝐸𝐸2 ) = 𝑃𝑃(𝐸𝐸1) + 𝑃𝑃(𝐸𝐸2)

The axioms imply that:

– 𝑃𝑃(Ø) = 0 𝑎𝑎𝑎𝑎𝑎𝑎 𝑃𝑃(𝐸𝐸𝐸 ) = 1 – 𝑃𝑃(𝐸𝐸)
– If E1 is contained in E2, then 𝑃𝑃(𝐸𝐸1) ≤ 𝑃𝑃(𝐸𝐸2).

Section 2.3 Interpretations and Axioms of Probability

29
Unions of Events and Addition Rules
Joint events are generated by applying basic set operations to individual events,
specifically:
• Unions of events, 𝐴𝐴 ∪ 𝐵𝐵
• Intersections of events, 𝐴𝐴 ∩ 𝐵𝐵
• Complements of events, 𝐴𝐴′

Probabilities of joint events can often be determined from the probabilities of the
individual events it comprises

Section 2.4 Unions of Events and Addition Rules

30
Example 2.15a | Semiconductor Wafers
A wafer is randomly selected from a batch of 940 wafers in a
semiconductor manufacturing process
– Let H denote the event that the wafer contains high levels of contamination
› Then P(H) = 358/940.
– Let C denote the event of the wafer is in center of a sputtering tool
› Then P(C) = 626/940.

Location of Tool
Challenge:
Contamination Total
Center Edge
Can you
represent the Low 514 68 582
table as Venn
or tree High 112 246 358
diagram
Total 626 314 940
Section 2.4 Unions of Events and Addition Rules

31
Example 2.15b | Semiconductor Wafers
𝑃𝑃(𝐻𝐻 ∩ 𝐶𝐶) is the probability that the wafer is from the center of the sputtering tool and
contains high levels of contamination
𝑃𝑃(𝐻𝐻 ∩ 𝐶𝐶) = 112/940
The event (𝐻𝐻 ∪ 𝐶𝐶) is the event that a wafer is from the center of the sputtering tool
or contains high levels of contamination (or both)
𝑃𝑃 𝐻𝐻 ∪ 𝐶𝐶 = 𝑃𝑃 𝐻𝐻 + 𝑃𝑃 𝐶𝐶 − 𝑃𝑃(𝐻𝐻 ∩ 𝐶𝐶)
= (358 + 626 − 112)/940 = 872/940 Contamination Location of Tool Total
Center Edge
Low 514 68 582
Practical Interpretation: To better High 112 246 358
understand the sources of contamination,
Total 626 314 940
yield from different locations on wafers are
routinely aggregated
Section 2.4 Unions of Events and Addition Rules

32
Addition Rule

Section 2.4 Unions of Events and Addition Rules

33
Addition Rule
Three or More Events

Section 2.4 Unions of Events and Addition Rules

34
Conditional Probability
› The probability of an event B under the knowledge that the outcome will
be in event A is called the conditional probability of B given A, denoted
as 𝑃𝑃(𝐵𝐵 | 𝐴𝐴)
› A digital communications channel has an error rate of 1 per 1000 bits
transmitted. Errors are rare, occur in bursts. If a single bit is transmitted, we
might model the probability of an error as 1/1000. However, if the previous
bit was in error because of the bursts, we might believe that the probability
that the next bit will be in error is greater than 1/1000.

Section 2.5 Conditional Probability

35
Conditional Probability

This definition can be understood in a special case in which all outcomes of a random
experiment are equally likely. If there are N total outcomes,
𝑃𝑃(𝐴𝐴) = (number of outcomes in A) / N
𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵) = (number of outcomes in 𝐴𝐴 ∩ 𝐵𝐵) / N
𝑃𝑃(𝐵𝐵 | 𝐴𝐴) = (number of outcomes in 𝐴𝐴 ∩ 𝐵𝐵) / (number of outcomes in A)

Section 2.5 Conditional Probability

36
Example 2.17 | Surface Flaws and
Defectives
Table 2.2 provides an example of 400 Parts Classified
parts classified by surface flaws and as Surface Flaws
(functionally) defective. Of the parts with Defective Total
surface flaws (40 parts), the number of Yes (F ) No (F' )
defective ones is 10. Therefore, Yes (D ) 10 18 28
𝑃𝑃(𝐷𝐷 | 𝐹𝐹) = 10/40 = 0.25 No (D' ) 30 342 372
Total 40 360 400
And of the parts without surface flaws
Practical Interpretation
(360 parts), the number of defective ones
The probability of being defective is five times greater
is 18. Therefore,
for parts with surface flaws. This calculation illustrates
𝑃𝑃(𝐷𝐷 | 𝐹𝐹𝐹) = 18/360 = 0.05 how probabilities are adjusted for additional
information. The result also suggests that there may
be a link between surface flaws and functionally
defective parts, which should be investigated

Section 2.5 Conditional Probability

37
Random Samples

To select randomly implies that at each step of the sample, the

items that remain in the batch are equally likely to be selected

Section 2.5 Conditional Probability

38
Intersections of Events and Multiplication and
Total Probability Rules
› The conditional probability definition can be rewritten to provide a
formula known as the multiplication rule for probabilities

› This expression is obtained by interchanging A and B

Section 2.6 Intersections of Events and Multiplication and Total Probability Rules

39
Example 2.19 | Machining Stages
The probability that the 1st stage of a numerically controlled machining operation
meets specifications is 0.90. The probability that it meets specifications in the 2nd
stage, given that met specifications in the first stage is 0.95.

What is the probability that both stages meet specifications?

› Let A and B denote the events that the 1st and 2nd stages meet specifications,
respectively.
› 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵) = 𝑃𝑃(𝐵𝐵 | 𝐴𝐴) · 𝑃𝑃(𝐴𝐴) = 0.95 · 0.90 = 0.855

Section 2.6 Intersections of Events and Multiplication and Total Probability Rules

40
Total Probability Rule
› A and A′ are mutually exclusive.
› 𝐴𝐴 ∩ 𝐵𝐵 and 𝐴𝐴′ ∩ 𝐵𝐵 are mutually
exclusive
› 𝐵𝐵 = 𝐴𝐴 ∩ 𝐵𝐵 ∪ (𝐴𝐴′ ∩ 𝐵𝐵)

Total Probability Rule

For any two events A and B

Section 2.6 Intersections of Events and Multiplication and Total Probability Rules

41
Example 2.20 | Semiconductor
Contamination
Information about product failure based on chip manufacturing process contamination is
given below. Find the probability of failure.

Let F denote the event that the product fails

Let H denote the event that the chip is exposed to high contamination. Then
› P(F | H) = 0.10 and P(F | H′ ) = 0.005
› P(H) = 0.20 and P(H′ ) = 0.8
› P(F) = 0.10(0.20) + 0.005(0.80) = 0.024

Section 2.6 Intersections of Events and Multiplication and Total Probability Rules 42
Total Probability Rule

Section 2.6 Intersections of Events and Multiplication and Total Probability Rules

43
Independence

Knowledge that the outcome of the experiment is in event A

does not affect the probability that the outcome is in event
B
commonly arise
as different trials
in an experiment Section 2.7 Independence

44
Independence

Section 2.7 Independence

45
Bayes’ Theorem
› Thomas Bayes addressed essential question in 1700s of
finding the probability that a condition was present (high
contamination) given an outcome (a semiconductor failure)
› Bayes’ theorem states that,

Section 2.8 Bayes’ Theorem

46
Example 2.26
The conditional probability that a high level of contamination was present when a
failure occurred is to be determined. The information is summarized here.

Section 2.8 Bayes’ Theorem

47
Bayes’ Theorem

Note:
Numerator expression is always one of the terms in the sum of the denominator.

Section 2.8 Bayes’ Theorem

48