Introduction to Probability Theory
INTRODUCTION TO PROBABILITY THEORY Basic Concepts of Probability Theory any process of measurement or observation of different outcomes
Experiment
Sample Space
set of all possible outcomes for an experiment
Element
members in the sample space
Event
any subset from the sample space
Simple Event Compound Event
event that contain only one element from sample space event that contain more than one element from sample space
Figure 1: Basic Concept of Probability Theory
Types of Events Event Complement Intersection Union Mutually exclusive Independent
Figure 2: Types of Events
A ={x | x A} A B = {x | x A and x B} A B = {x | x A or x B} AB=
Introduction to Probability Theory
Set Notation 1. Sample space, S represented by sample points in a square, while event, E represented by sample points in a circle.
S E
2. E is set complement of E. This means event E does not occur.
3. E1 E2 is the event that E1 occur or E2 occur or both occur.
E1
E2
4. E1 E2 is the event that both of E1 and E2 are occur.
E1
E2
5. E1 and E2 do not combine, i.e. E1 and E2 are mutually exclusive if E1 E2 = . This means that E1 and E2 do not have the same outcome.
E1
E2
6. E1, E2, E3, , En are mutually exclusive events if and only if (i) Ei Ej = , i, j. (ii) E1 E2 E3 En = S.
E1 E2
E6 E5 E3 E4
Identity of Set 1. 2. 3. 4. 5 6. 7. AA=A A=A AS=S AA=A A= AS=A AB=BA 8. 9. 10. 11. 12. 13. 14. AB=BA A (B C) = (A B) (A C) A (B C) = (A B) (A C) (A B) = A B (A B) = A B A B = A (A B) B = (A B) (A B)
Introduction to Probability Theory
Probability of an Event Probability of an event E defined by P(E), is a real number between 0 and 1 that give to each of the simple event E in a sample space.
Conditions (a) (b) If E is a simple event, then 0 P(E) 1. Sum of probability for all simple events in a sample space is equal to 1, i.e.
n
P( E ) = 1.
i =1 i
Formula
P(E) =
amount of elements in set E n( E ) . = n( S ) amount of elements in sample space S
Two Extreme Events
(a) For a sure event S, n( S ) P(S) = =1 n( S ) the event sure occur. (b) For a null event , n( ) P() = =0 n( S ) the event will not occur.
Probability of Combined Events
1. If A and B are any two events from same experiment, then P(A B) = P(A) + P(B) P(A B) 2. Two events are said mutually exclusive if one of the events occurs, but the other not occurs. If A and B are two mutually exclusive events, then A B = and n(A B) = 0 P(A B) = 0. Hence, P(A B) = P(A) + P(B) 3. Probability of union mutually exclusive events is given as below: (a) (b) (c) P(A B) = P(A) P(A B) P(A B) = P(A) + P(A B) P(A B) = 1 P(A B)
Introduction to Probability Theory
Venn Diagrams If A and B are two events and A B , then Venn diagrams can be used to represent the events below: (a) A (b) B (c) A B
(d) A B
(e) (A B)
(f) (A B)
(g) A B
(h) A B
(i) A B
(j) A B
(k) A B
(l) A B
Introduction to Probability Theory
Conditional Probability Probability of an event A occur, given that an event B was occurred is called conditional probability and denoted by P(A | B), i.e. P(A | B) = P( A B) , P(B) 0. P( B)
Independent Events Two events A and B are independent if P(A) = P(A | B) and P(B) = P(B | A). Hence, P(A B) = P(A) P(B).
Test of Independent Events Three relations below used to test the independent of events A and B: (a) (b) (c) P(A | B) = P(A) P(B | A) = P(B) P(A B) = P(A) P(B)
If one of three relations is valid, then events A and B are independent. Otherwise, A and B are dependent.
Contingency Table B1 A1 A2 Total P(A1 B1) P(A2 B1) P(B1) B2 P(A1 B2) P(A2 B2) P(B2) Total P(A1) P(A2) 1.00
Theorem of Total Probability If the events B1, B2, B3, , Bk, constitute a partition of the sample space S such that P(Bi) 0 for i = 1, 2, 3, , k, then for any event A of S,
k
P(A) =
P( B
i =1
A) = P ( Bi ) P ( A | Bi ) .
i =1
Introduction to Probability Theory
Bayes Rule If the events B1, B2, B3, , Bk, constitute a partition of the sample space S, where P(Bi) 0 for i = 1, 2, 3, , k, then for any event A in S such that P(A) 0, P(Br | A) =
P( Br A)
k
P ( Br ) P ( A | Br )
k
P( B
i =1
A)
P( B ) P( A | B )
i i i =1
for r = 1, 2, 3, , k.
Tree Diagram for Bayes Rule
P(B1) P(A | B1)
B1 P(B1)
P(A | B1)
P(B2)
B2
P(A | B2)
P(B1) P(A | B1)
. . .
P(Br) Br P(A | Br)
. . .
A
. . .
P(B1) P(A | B1)
Tabular Approach for Bayes Rule
Events Bi Prior Probabilities P( Bi ) . . 1.00 Conditional Probabilities P( A | Bi ) . . Joint Probabilities P( Bi A) . .
P ( A)
B1 B2
Posterior Probabilities P( Bi | A) . .
1.00
Formula: (a) P ( Bi A) = P ( Bi ) P ( A | Bi ) (b) P ( Bi | A) = P ( Bi A) P ( A)
Introduction to Probability Theory
Some Definitions
Probability A value between zero and one, inclusive, describing the relative possibility (chance or likelihood) an event will occur.
Experiment A process that leads to the occurrence of one and only one of several possible observations.
Outcome A particular result of an experiment.
Event A collection of one or more outcomes of an experiment.
Mutually exclusive (disjoint) The occurrence of one event means that none of the other events can occur at the same time.
Independence The occurrence of one event has no effect on the probability of the occurrence of another event.
Conditional probability The probability of a particular event occurring given that another event has occurred.
Contingency table A table used to classify sample observations according to two or more identifiable characteristics.
