3.3.
3 Likelihood – Detailed Explanation for Diploma
Students
1. What is Likelihood?
Likelihood is a function that measures how well a hypothesis or parameter explains observed data.
It is used in statistics and machine learning to estimate the most probable parameters for a given
dataset.
2. Likelihood is NOT Probability
Though likelihood and probability use similar formulas, their meanings differ:
- Probability: Model is fixed, data varies.
- Likelihood: Data is fixed, model varies.
3. Likelihood Function (L)
Given data D = {x■, x■, ..., x■} and parameter θ, the likelihood function is:
L(θ) = P(D | θ)
This function gives how likely the observed data is for each value of θ.
4. Maximum Likelihood Estimation (MLE)
MLE is used to find the parameter θ that maximizes L(θ). For example, in coin toss data:
If we observe H, H, T, H, H, T, H, we define:
L(θ) = θ■ × (1 - θ)²
The θ that gives the maximum value is the MLE estimate.
5. Log-Likelihood
To avoid dealing with very small likelihood values, we take the logarithm:
log L(θ) = 5 log(θ) + 2 log(1 - θ)
Used in loss functions and optimization in ML.
6. Likelihood in Model Selection
Likelihood is used in model comparison metrics:
- AIC (Akaike Information Criterion)
- BIC (Bayesian Information Criterion)
These help choose models with the best balance of fit and simplicity.
7. Likelihood Function vs Likelihood Value
Likelihood Function: L(θ) = θ■ × (1 - θ)² (general formula)
Likelihood Value: Value after substituting θ (e.g., θ = 0.7)
8. How to Write in Exams
Format:
- Definition
- Formula: L(θ) = P(Data | θ)
- Key Differences with Probability
- Example (Coin Toss)
- Uses in MLE, ML models
- Advanced concepts like log-likelihood
- Real-life applications
9. Real-Life Applications
- Voice recognition
- Medical diagnosis
- Spam filtering
