SINGLE VARIABLE PROBABILITY

DeMorgans Laws
( A B )C AC B C
( A B )C AC B C

Probability Basics
Pr( A B ) Pr( A) Pr( B ) Pr( A B)
Pr( A B C ) Pr( A) Pr( B ) Pr(C ) Pr( A B) Pr( A C ) Pr( B C ) Pr( A B C )
Law of Total Probability
If E1 , E2 ,..., En are partitions of probability space S
Pr( A) Pr( A E1 ) Pr( A E2 ) Pr( A En )

Conditional Probability
Pr( A B )
Pr( A B)
Pr( B )
Pr( A B ) Pr( A B ) Pr( B )

Independence
Events A and B are considered independent if
Pr( A B) Pr( A) Pr( B )

Bayes Theorem
Pr( A Ei ) Pr( Ei )
Pr( Ei A) n

Pr( A E ) Pr( E )
j 1
j j

N choose k
n n!

k k ! (n k )!

Hazard Rate
f ( x)
x ( x )
s( x)

s ( x) e
x ( u ) du

Expectation

E[ g ( x)] g ( x) f ( x )dx


E[ X ] s ( x)dx

x

E[ X 2 ] 2 x sx ( x)dx

E[aX bY c] aE[ X ] bE[Y ] c expectation is linear

Variance
Var[ X ] E[ X 2 ] E[ X ]
Var[aX bY c] a 2Var[ X ] b 2Var[Y ] only if X and Y are independent
Var[aX bY c] a Var[ X ] b Var[Y ] 2abCov[ X , Y ]
2 2

Moment Generating Function

M x (t ) E[ X tX ]

E[ X n ] n
M x (t ) t power series expansion
n0 n!

Cumulant Generating Function

x (t ) ln( M x (t )) ln( E[ X tX ])
dk
x (t ) t 0 k x (0) E[( X E[ X ]) k ]
dt k
aX b (t ) x (at ) bt

3
X
Skewness E[ ]

4
X
Kurtosis E[ ]

x
Coefficient of Variation
x

Jensens Inequality
d2
For h(x) such that 2 h( x) 0 and f ( x) 0 for all x
dx
E[h( x)] h( E[ X ])

Markovs Inequality
For x > 0 and a > 0
E[ X ]
Pr( X a )
a

Chebyshevs Inequality
 X 1
Pr( r) 2
r
2
Pr( X r )
r2
1
Pr( r X r ) 1
r2
Transformation (single variable)
Y ( x) Y ( x)
X 1 (Y ) X ( y )
dx
f y ( y ) f x ( x ( y ))
dy

MULTI-VARIABLE PROBABILITY

fx

f x , y ( x, y )dy

Conditional Distribution
f x , y ( x, y )
f y x x
f x ( x)

Covariance
Cov[ X , Y ] E[( X x )(Y y )] E[ XY ] x y
Cov[aX 1 bX 2 , Y ] aCov[ X 1 , Y ] bCov[ X 2 , Y ]

Correlation Coefficient
Cov[ X , Y ]
x, y 1 1
f x ( x)

Moment Generating Function of a Joint Distribution

M x , y ( s, t ) E[e sX tY ]
d mn
m n
M x , y ( s, t ) s 0,t 0 E[ X m , Y n ]
ds dt
M x , y ( s, t ) M x ( s ) M y (t ) if x and y are independent

Multi-Variable Transformation (Jakobian Tech)

U ,V ( X , Y )
X , Y 1 (U , V )
fu ,v f x , y ( x (u , v), y (u , v)) J
 dx dx
du dv
J
dy dy
du dv

Central Limit Theorem

x x x xn
If Fz is the CDF of random variable z where x 1 and
x n
x1 ,..., xn are iid. Then for every x lim n Fz ( x) ( x)