Probability

Basics of Probability …

• Probability is the study of randomness.

• A Probability is a Numerical Measure of the Likelihood of

Occurrence of an event; more it is , more is the likelihood of
happening of an event.
 Probability is:

⚫ A quantitative measure of uncertainty

⚫ A measure of the strength of belief in the occurrence of
an uncertain event
⚫ A measure of the degree of chance or likelihood of
occurrence of an uncertain event
⚫ Measured by a number between 0 and 1 (or between 0%
and 100%)
What makes to define PROBABILITY?

SAMPLE
RANDOM
OUTCOME EXPERIMENT

EVENT

SAMPLE
SPACE

PROBABILITY
 Random Experiment

• … is a process that lead to more than one possible outcome and

each outcome is uncertain.

• … is a process leading to at least two possible outcomes with

uncertainty as to which will occur.

• … is any procedure
1. that can be repeated, theoretically or otherwise, an infinite number of times;
2. that has a well-defined set of possible outcomes; and
3. the outcome that will prevail in a given experience is uncertain.
 Random Experiment

• Tossing a Coin: Imagine flipping a coin. We can't predict

whether it will land heads or tails until it's actually tossed.

• Rolling a Dice: when we roll a standard six-sided dice, we

don't know which number will show up on top.

• Drawing a Card: When we draw a card from the deck, we

can't be certain which card we'll get.
 Random Experiment

• Each trial of an experiment has a single observed outcome.

• The precise outcome of a random experiment is unknown

before a trial.
 SAMPLE OUTCOME

• Each of the potential eventualities of a random experiment

is defined as sample outcome.

• Possible outcomes of a random experiment is called

Sample Outcome.
 SAMPLE SPACE

• The totality of all eventualities of a random experiment is

called Sample Space.

• Complete set of all possible outcomes of a random

experiment is called Sample Space.
 EVENT

• … is any subset of a sample space.

• … is any designated collection of sample outcomes,

including individual outcomes, the entire sample space, and
the null set.

• … is a subset of basic outcomes of the sample space.

 Events : Definition

⚫ Sample Space
✓Set of all possible outcomes (universal set) for a given experiment
⚫ E.g.: Roll a regular six-sided dice
• S = {1,2,3,4,5,6}
 Events : Definition

⚫ Sample Space
✓Set of all possible outcomes (universal set) for a given experiment
⚫ E.g.: Roll a regular six-sided dice
• S = {1,2,3,4,5,6}

⚫ Event
✓Collection of outcomes having a common characteristic
⚫ E.g.: Even number
• A = {2,4,6}
• Event A occurs if an outcome in the set A occurs
 Events : Definition

⚫ Sample Space
✓Set of all possible outcomes (universal set) for a given experiment
⚫ E.g.: Roll a regular six-sided dice
• S = {1,2,3,4,5,6}

⚫ Event
✓Collection of outcomes having a common characteristic
⚫ E.g.: Even number
• A = {2,4,6}
• Event A occurs if an outcome in the set A occurs
⚫ Probability of an event
✓Sum of the probabilities of the outcomes of which it consists
⚫ P(A) = P(2) + P(4) + P(6)
 Types of Probability

Objective or Classical Probability

 Types of Probability

Objective or Classical Probability

✓based on equally-likely events

✓not based on personal beliefs

✓ is the same for all observers (objective)

✓examples: toss a coin, throw a dice, pick a card

 Equally-likely Probabilities
(Hypothetical or Ideal Experiments)
• For example:
✓Throw a dice
• Six possible outcomes {1,2,3,4,5,6}
• If each is equally-likely, the probability of each is 1/6 = 0.1667 = 16.67%
1
• P( e) =
n( S )
• Probability of each equally-likely outcome is 1 divided by the number of possible
outcomes
 Equally-likely Probabilities
(Hypothetical or Ideal Experiments)
• For example:
✓Throw a dice
• Six possible outcomes {1,2,3,4,5,6}
• If each is equally-likely, the probability of each is 1/6 = 0.1667 = 16.67%
1
• P( e) =
n( S )

• Probability of each equally-likely outcome is 1 divided by the number of possible

outcomes

✓Event A (even number) P ( A ) =  P ( e)

• P(A) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 1/2 n( A ) 3 1
• = = =
n( S ) 6 2
 Types of Probability (Continued)

Subjective Probability
 Types of Probability (Continued)

Subjective Probability

✓based on personal beliefs, experiences, prejudices, intuition -

personal judgment

✓different for all observers (subjective)

✓examples: Elections, new product introduction, snowfall

 Basic Definitions

Set - a collection of elements or objects of interest

✓Empty set (denoted by )

⚫ a set containing no elements

✓Universal set (denoted by S)

⚫ a set containing all possible elements

✓Complement (Not). The complement of A is ( A)

⚫ a set containing all elements of S not in A
 Complement of a Set

Venn Diagram illustrating the Complement of an event

 Basic Definitions (Continued)

✓Intersection (And) ( A  B)
– a set containing all elements in both A and B

✓Union (Or) ( A  B)
– a set containing all elements in A or B or both
Sets: A Intersecting with B

A
B

A B
Sets: A Union B

A
B

A B
 Pick a Card: Sample Space
Hearts Diamonds Clubs Spades
Union of A
K
A
K
A
K
A
K
Event ‘Ace’
Events ‘Heart’ Q Q Q Q
n ( Ace ) 4 1
and ‘Ace’ J J J J
P ( Ace ) = = =
10 10 10 10
P ( Heart  Ace ) = 9 9 9 9
n(S ) 52 13

8 8 8 8
n ( Heart  Ace ) 7 7 7 7
= 6 6 6 6
n(S ) 5 5 5 5
4 4 4 4
16 4 3 3 3 3
=
2 2 2 2
52 13

The intersection of the

events ‘Heart’ and ‘Ace’
Event ‘Heart’
comprises the single point
n ( Heart ) 13 1
P ( Heart ) = = = circled twice: the ace of hearts
n(S ) 52 4 n ( Heart  Ace ) 1
P ( Heart  Ace ) = =
n(S ) 52
 Basic Definitions (Continued)

• Mutually exclusive or disjoint sets

–sets having no elements in common, having no
intersection, whose intersection is the empty set

• Partition
–a collection of mutually exclusive sets which together
include all possible elements, whose union is the
universal set
Mutually Exclusive or Disjoint Sets

Sets have nothing in common

B
A
 Sets: Partition

S
A3
A1

A2 A4

A5
Basic Rules for Probability
 Basic Rules for Probability

⚫ Range of Values for P(A): 0  P( A) 1

⚫ Complements - Probability of not A

P( A) = 1− P( A)
⚫ Intersection - Probability of both A and B
P( A  B) = n( A  B)
n( S )
✓Mutually exclusive events (A and C) :

P( A  C) = 0
 Basic Rules for Probability (Continued)

• Union - Probability of A or B or both (rule of unions)

P( A  B) = n( A  B) = P( A) + P( B) − P( A  B)
n( S )

✓Mutually exclusive events: If A and B are mutually exclusive, then

P( A  B) = 0 so P( A  B) = P( A) + P(B)
Sets: P(A Union B)

A
B

P( A  B)
 Example 1

1. ShopperTrak is a hidden electric eye designed to count

the number of shoppers entering a store. When two
shoppers enter a store together, one walking in front of the
other, the following probabilities apply: There is a 0.98
probability that the first shopper will be detected, a 0.94
probability that the second shopper will be detected, and a
0.93 probability that both of them will be detected by the
device. What is the probability that the device will detect at
least one of two shoppers entering together?
 Example 2

2. Following are age and gender data for 20 midlevel

managers at a service company: 34 F, 49 M, 27 M, 63 F, 33
F, 29 F, 45 M, 46 M, 30 F, 39 M, 42 M, 30 F, 48 M, 35 F, 32 F,
37 F, 48 F, 50 M, 48 F, 61 F. A manager must be chosen at
random to serve on a companywide committee that deals
with personnel problems. What is the probability that the
chosen manager will be either a woman or over 50 years old
or both?
 Conditional Probability

• Conditional Probability - Probability of A given B

P( A  B)
P( A B) = , where P( B)  0
P( B)

✓Independent events:
P( A B) = P( A)
P( B A) = P( B)
 Conditional Probability (continued)

Rules of conditional probability:

P( A B) = P( A  B) so P( A  B) = P( A B) P( B)
P( B)
= P( B A) P( A)

If events A and D are statistically independent:

P( A D) = P( A)
so P( A  D) = P( A) P( D)
P( D A) = P( D)
 Contingency Table

• Example: Drug Efficacy Study

• Suppose we are conducting a drug efficacy study to determine the

relationship between taking a new drug (Drug A) and the presence of
a specific health condition (Condition B).

• The participants are divided into two groups: Group 1 (receives Drug
A) and Group 2 (receives a placebo).

• We measure whether the condition improves ("Success") or not

("Failure") for each group after the treatment period.
 Contingency Table

Group 1 Group 2
(Drug A) (Placebo) Total
Success
30 15 45
(Condition)

Failure (No
10 25 35
Change)
Total 40 40 80
 Probability Table

Group 1 (Drug A) Group 2 (Placebo)

Probability
30/40 = 0.75 15/40 = 0.375
(Success)

Probability
10/40 = 0.25 25/40 = 0.625
(Failure)
 Contingency Table - Example

Counts
AT& T IBM Total

Telecommunication 40 10 50

Computers 20 30 50

Total 60 40 100

Probabilities
AT& T IBM Total

Telecommunication .40 .10 .50

Computers .20 .30 .50

Total .60 .40 1.00

 Contingency Table - Example

Counts
AT& T IBM Total

Telecommunication 40 10 50 Probability that a project

Computers 20 30 50
is undertaken by IBM
given it is a
Total 60 40 100
telecommunications
Probabilities project:
AT& T IBM Total

Telecommunication .40 .10 .50

Computers .20 .30 .50

Total .60 .40 1.00

 Contingency Table - Example

Counts
AT& T IBM Total

Telecommunication 40 10 50 Probability that a project

Computers 20 30 50
is undertaken by IBM
given it is a
Total 60 40 100
telecommunications
Probabilities project:
P( IBM  T )
AT& T IBM Total P( IBM T ) =
P(T )
Telecommunication .40 .10 .50
0.10
= = 0.2
Computers .20 .30 .50 0.50

Total .60 .40 1.00

 Example

3. A financial analyst believes that if interest rates decrease

in a given period, then the probability that the stock market
will go up is 0.80. The analyst further believes that interest
rates have a 0.40 chance of decreasing during the period in
question. Given the above information, what is the
probability that the market will go up and interest rates will
go down during the period in question?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
b. If a stock is selected at random, what is the probability that it paid dividends?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
b. If a stock is selected at random, what is the probability that it paid dividends?
c. If a stock is randomly selected, what is the probability that it both increased
in price and paid dividends?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
b. If a stock is selected at random, what is the probability that it paid dividends?
c. If a stock is randomly selected, what is the probability that it both increased
in price and paid dividends?
d. What is the probability that a randomly selected stock neither paid dividends nor
increased in price?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
b. If a stock is selected at random, what is the probability that it paid dividends?
c. If a stock is randomly selected, what is the probability that it both increased
in price and paid dividends?
d. What is the probability that a randomly selected stock neither paid dividends nor
increased in price?
e. Given that a stock increased in price, what is the probability that it also paid
dividends?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
b. If a stock is selected at random, what is the probability that it paid dividends?
c. If a stock is randomly selected, what is the probability that it both increased
in price and paid dividends?
d. What is the probability that a randomly selected stock neither paid dividends nor
increased in price?
e. Given that a stock increased in price, what is the probability that it also paid
dividends?
f. If a stock is known not to have paid dividends, what is the probability that it
increased in price?
 Example

4. An investment analyst collects data on stocks and notes whether or not

dividends were paid and whether or not the stocks increased in price over a given period.
Data are presented in the following table.
Price No Price
Increase Increase Total
Dividends paid 34 78 112
No dividends paid 85 49 134
Total 119 127 246
a. If a stock is selected at random out of the analyst’s list of 246 stocks, what is
the probability that it increased in price?
b. If a stock is selected at random, what is the probability that it paid dividends?
c. If a stock is randomly selected, what is the probability that it both increased
in price and paid dividends?
d. What is the probability that a randomly selected stock neither paid dividends nor
increased in price?
e. Given that a stock increased in price, what is the probability that it also paid
dividends?
f. If a stock is known not to have paid dividends, what is the probability that it
increased in price?
g. What is the probability that a randomly selected stock was worth holding
during the period in question; that is, what is the probability that it increased
in price or paid dividends?
 Example

The probability that a consumer will be exposed to an

advertisement for a certain product by seeing a commercial
on television is 0.04. The probability that the consumer will
be exposed to the product by seeing an advertisement on a
billboard is 0.06. The two events, being exposed to the
commercial and being exposed to the billboard ad, are
assumed to be independent. (a) What is the probability that
the consumer will be exposed to both advertisements? (b)
What is the probability that
he or she will be exposed to at least one of the ads?
Independence of Events – Example

Events Television (T) and Billboard (B) are

assumed to be independent.

a )P(T  B) = P(T ) P( B)
= 0.04 * 0.06 = 0.0024
b)P(T  B) = P(T ) + P( B) − P(T  B)
= 0.04 + 0.06 − 0.0024= 0.0976
Product Rules for Independent Events

The probability of the intersection of several independent events

is the product of their separate individual probabilities:
P( A  A  A  An ) = P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3
The probability of the union of several independent events
is 1 minus the product of probabilities of their complements:
P( A  A  A  An ) = 1− P( A ) P( A ) P( A ) P( An )
1 2 3 1 2 3
 Example

5. A package of documents needs to be sent to a given

destination, and delivery within one day is important. To
maximize the chances of on-time delivery, three copies of
the documents are sent via three different delivery services.
Service A is known to have a 90% on-time delivery record,
service B has an 88% on-time delivery record, and service C
has a 91% on-time delivery record. What is the probability
that at least one copy of the documents will arrive at its
destination on time?
The Law of Total Probability and Bayes’ Theorem

The law of total probability:

P( A) = P( A  B) + P( A  B )

In terms of conditional probabilities:

P( A) = P( A  B) + P( A  B )
= P ( A B) P ( B ) + P ( A B ) P ( B )

More generally (where Bi make up a partition):

P( A) =  P( A  B )
i
=  P( AB ) P( B )
i i
 Example

• An analyst believes the stock market has a 0.75 probability

of going up in the next year if the economy should do well,
and a 0.30 probability of going up if the economy should
not do well during the year.

• The analyst further believes there is a 0.80 probability that

the economy will do well in the coming year.

• What is the probability that the stock market will go up next

year (using the analyst’s assessments)?
The Law of Total Probability- Example

Event U: Stock market will go up in the next year

Event W: Economy will do well in the next year
P(U W ) =.75
P(U W ) = 30
P(W ) =.80  P(W ) = 1−.8 =.2

P(U ) = P(U W ) + P(U W )

= P(U W ) P(W ) + P(U W ) P(W )
= (.75)(.80) + (.30)(.20)
=.60+.06 =.66
 Example

• 7. A drug manufacturer believes there is a 0.95 chance that the Food

and Drug Administration (FDA) will approve a new drug the company
plans to distribute if the results of current testing show that the drug
causes no side effects.

• The manufacturer further believes there is a 0.50 probability that the

FDA will approve the drug if the test shows that the drug does cause
side effects.

• A physician working for the drug manufacturer believes there is a

0.20 probability that tests will show that the drug causes side effects.
What is the probability that the drug will be approved by the FDA?
 Bayes’ Theorem…

• Bayes' Theorem relates the conditional and marginal

probabilities of two random events. It is often used to
compute posterior probabilities given observations.

It provides a mechanism to REVISE

our priori probabilities in the light
of NEW INFORMATION!!!
 Bayes’ Theorem

• Bayes’ theorem enables you, knowing just a little more

than the probability of A given B, to find the probability of B
given A.
• Based on the definition of conditional probability and the
law of total probability.

P( A  B)
P( B A) =
P( A)
P( A  B) Applying the law of total
= probability to the denominator
P( A  B) + P( A  B )
P( A B) P( B) Applying the definition of
=
P( A B) P( B) + P( A B ) P( B ) conditional probability throughout
 Bayes’ Theorem

P( A & B)
P( A | B) =  P( A | B) P( B) = P( A & B)
P( B)
P( A & B)
P( B | A) =  P( B | A) P( A) = P( A & B)
P( A)

So: P( A | B) P( B) = P( B | A) P( A)
P( A | B) P( B) P( A | B) P( B)
P( B | A) = =
P( A) P( A | B) P( B) + P( A | B ) P( B )

The above formula is referred to as Bayes’ theorem. It is

extremely useful in decision analysis when using
information.
 Bayes’ Theorem - General

P( Ei & A) P( Ei ) P( A | Ei )
P( Ei | A) =  n
 P( Ei ) P( A | Ei )
P( A)
i =1
 Seeing it in Tabular form…
POSTERI
PRIORI CONDITIONAL JOINT OR
EVENTS
PROBABILITIES PROBABILITIES PROBABILITIES PROBABI
LITIES
 Example

• Consider a test for an illness. The test has a known reliability:

1. When administered to an ill person, the test will indicate so with

probability 0.92.

2. When administered to a person who is not ill, the test will

erroneously give a positive result with probability 0.04.

Suppose the illness is rare and is known to affect only 0.1% of the
entire population. If a person is randomly selected from the entire
population and is given the test and the result is positive, what is the
posterior probability (posterior to the test result) that
the person is ill?
 Bayes’ Theorem - Example

• Let Z denote the positive result and I is for ill person.

• A medical test for a rare disease (affecting 0.1% of the population

[P(I) = 0.001]) is imperfect:

✓When administered to an ill person, the test will indicate so with probability 0.92
[ P(Z|I) = 0.92 ]

✓When administered to a person who is not ill, the test will erroneously give a
positive result (false positive) with probability 0.04
.
P( Z I ) = 0.04
 Example (continued)

P( I ) = 0.001 P( I  Z )
P( I Z ) =
P( Z )
P( I  Z )
=
P( I ) = 0.999 P( I  Z ) + P( I  Z )
P( Z I ) P( I )
=
P( Z I ) P( I ) + P( Z I ) P( I )
P( Z I ) = 0.92 =
(.92)(0.001)
(.92)(0.001) + (0.04)(.999)
0.00092 0.00092
= =
0.00092 + 0.03996 .04088
P( Z I ) = 0.04 = .0225
 Bayes’ Theorem Extended

• Given a partition of events B1,B2 ,...,Bn:

P( A  B )
P( B A) =
1
1

P( A)
Applying the law of total
P( A  B ) probability to the denominator
= 1

 P( A  B ) i
Applying the definition of
P( A B ) P( B ) conditional probability throughout
= 1 1

 P( A B ) P( B )
i i
 Bayes’ Theorem Extended - Example

⚫ An economist believes that during periods of high economic growth, the

U.S. dollar appreciates with probability 0.70; in periods of moderate
economic growth, the dollar appreciates with probability 0.40; and during
periods of low economic growth, the dollar appreciates with probability
0.20.

⚫ During any period of time, the probability of high economic growth is 0.30,
the probability of moderate economic growth is 0.50, and the probability of
low economic growth is 0.20.

⚫ Suppose the dollar has been appreciating during the present period. What is
the probability we are experiencing a period of high economic growth?
 The Joint Probability Table

⚫ A joint probability table is similar to a contingency table ,

except that it has probabilities in place of frequencies.

⚫ The joint probability for Example is shown on the next slide.

⚫ The row totals and column totals are called marginal

probabilities.
 The Joint Probability Table: Example

⚫ The joint probability table for Example is summarized

below.

High Medium Low Total

$
Appreciates
$Depreciates

Total 0.30 0.5 0.20 1.00

Marginal probabilities are the row totals and the column totals.
 The Joint Probability Table: Example

⚫ The joint probability table for Example is summarized

below.

High Medium Low Total

$
Appreciates
0.21 0.2 0.04 0.45
$Depreciates
0.09 0.3 0.16 0.55

Total 0.30 0.5 0.20 1.00

Marginal probabilities are the row totals and the column totals.
 Example

Q8: When the economic situation is “high,” a certain

economic indicator rises with probability 0.6. When the
economic situation is “medium,” the economic indicator
rises with probability 0.3. When the economic situation is
“low,” the indicator rises with probability 0.1. The economy
is high 15% of the time, it is medium 70% of the time, and it
is low 15% of the time. Given that the indicator has just
gone up, what is the probability that the economic situation
is high?
 Example

Q9. Saflok is an electronic door lock system made in Troy, Michigan,

and used in modern hotels and other establishments. To open a door,
you must insert the electronic card into the lock slip. Then a green light
indicates that you can turn the handle and enter; a yellow light indicates
that the door is locked from inside, and you cannot enter. Suppose that
90% of the time when the card is inserted, the door should open
because it is not locked from inside. When the door should open, a
green light will appear with probability 0.98. When the door should not
open, a green light may still appear (an electronic error) 5% of the time.
Suppose that you just inserted the card and the light is green. What is
the probability that the door will actually open?
 Example 10

Q10. An aircraft emergency locator transmitter (ELT) is a device

designed to transmit a signal in the case of a crash. The Altigauge
Manufacturing Company makes 80% of the ELTs, the Bryant
Company makes 15% of them, and the Chartair Company makes the
other 5%. The ELTs made by Altigauge have a 4% rate of defects, the
Bryant ELTs have a 6% rate of defects, and the Chartair ELTs have a
9% rate of defects (which helps to explain why Chartair has the
lowest market share).
a. If an ELT is randomly selected from the general population of all
ELTs, find the probability that it was made by the Altigauge
Manufacturing Company.
b. If a randomly selected ELT is then tested and is found to be
defective, find the probability that it was made by the Altigauge
Manufacturing Company.