Generating Random Samples Using Probability Integral Transformation
1. Concept of Probability Integral Transformation
- If X is a continuous random variable with cumulative distribution function (CDF) FX (x), then:
U = FX (X)
where U is uniformly distributed on [0, 1]. - Conversely, if U ∼ Uniform(0, 1), then the random variable:
−1
X = FX (U )
has the desired distribution FX (x).
2. Steps to Generate Random Samples
1. Understand the Target Distribution: Identify the CDF FX (x) of the target random variable.
2. Generate Uniform Random Numbers: Use a random number generator to produce U ∼ Uniform(0, 1).
−1 −1
3. Apply the Inverse CDF Transformation: Compute X = FX (U ), where FX is the inverse of the CDF. This
transformation ensures X follows the desired distribution.
3. Mathematical Foundation
−1
By the probability integral theorem, the transformation X = FX (U ) is valid because:
−1
P (X ≤ x) = P (FX (U ) ≤ x) = P (U ≤ FX (x)) = FX (x).
4. Example: Generating Exponential Random Variables
• Distribution: fX (x) = λe−λx , x ≥ 0, with CDF FX (x) = 1 − e−λx .
• Steps:
1. Generate U ∼ Uniform(0, 1).
2. Solve FX (x) = U :
1
1 − e−λx = U =⇒ x = − ln(1 − U ).
λ
3. Output X = − λ1 ln(1 − U ).
5. Applications
• Simulation of Random Variables: Widely used in Monte Carlo simulations to generate samples from arbitrary
distributions.
• Model Testing: To test theoretical models by generating data with known distributions.
• Bootstrapping: Generate samples to perform resampling for statistical inference.
6. Advantages
• Simple and effective for distributions with invertible CDFs.
• Does not require numerical integration or complex algorithms.
7. Limitations
• The method is limited to distributions with a closed-form or easily computable inverse CDF.
• For complicated distributions, alternative methods like the accept-reject algorithm or numerical approximations of
−1
FX are required.
1
8. R Implementation
R Code:
# Generating exponential random samples using probability integral transformation
set.seed(42) # For reproducibility
lambda <- 2 # Rate parameter
n <- 1000 # Number of samples
# Step 1: Generate uniform random numbers
U <- runif(n, min = 0, max = 1)
# Step 2: Transform using the inverse CDF
X <- -log(1 - U) / lambda
# Histogram of generated samples
hist(X, breaks = 30, main = "Generated Exponential Samples",
col = "lightblue", xlab = "x")
9. Generalization
For a multivariate distribution, the method can be extended if the joint CDF is available, but it becomes computa-
tionally challenging.
