Statistical Modeling, Lecture 3

Melike Efe
July 16, 2025
Sabancı University
Table of contents

1. F-Distribution

2. Order Statistic

3. Point Estimation
Unbiased Estimator

1
F-Distribution
F-Distribution

What is the F-Distribution?

▶ The F-distribution is used to compare variances of two independent

samples.
▶ Defined as the ratio of two chi-square distributed random variables,
each divided by its degrees of freedom.

Definition: Let X and Y be independent chi-square distributed random

variables with ν1 and ν2 degrees of freedom. Then,

X /ν1
R=
Y /ν2

has an F-distribution with ν1 and ν2 degrees of freedom.

2
Theorem: The density of F-distribuiton with ν1 and ν2 degrees of
freedom is
ν1 + ν2   ν1 − 21 (ν1 +ν2 )
Γ( ) ν 2 ν1

ν1
f (r ) = ν1 2 1
r 2 −1
1+ r
Γ( 2 )Γ( ν22 ) ν2 ν2

for r > 0 and it is zero elsewhere.

3
Constructing an F-Distributed Variable

Understanding the F-Distribution Through Standard Normal

Variables:

▶ Consider two independent standard normal random variables:

d d
Z = N(0, 1), W = N(0, 1).

▶ Squaring each variable gives chi-square distributed variables with one

degree of freedom:
d d
Z 2 = χ21 , W 2 = χ21 .
▶ Taking their ratio results in an F-distributed variable:

Z2 d
R= = F1,1 .
W2

4
Inverse Property of the F-Distribution

Key Property: Relationship Between F and Its Reciprocal

▶ If R follows an F -distribution with degrees of freedom (ν1 , ν2 ):
d
R = Fν1 ,ν2
then its inverse follows an F-distribution with swapped degrees of
freedom:
1 d
= Fν2 ,ν1 .
R
▶ This property is useful when looking up F -tables and calculating
critical values.
Why does this hold?
▶ Recall that the F -distribution is based on the ratio of two chi-square
variables.
▶ Reversing the ratio effectively swaps the degrees of freedom, leading
to a new F -distribution.
5
6
Understanding Critical Points

▶ The critical points of the F-distribution are tabulated in statistical

tables.
▶ For an F-distributed random variable X with degrees of freedom
(ν1 ,ν2 ), the critical value fα,ν1 ,ν2 satisfies:

P(X > fα,ν1 ,ν2 ) = α

7
Critical points of the F distribution
Points fα,ν ,ν for α = 0.01.
1 2
ν2 ν1
1 2 3 4 5 6 7 8
1 4052.2 4999.5 5403.4 5624.6 5763.6 5859.0 5928.3 5981.1
2 98.50 99.00 99.17 99.25 99.30 99.33 99.36 99.37
3 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49
4 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80
5 16.26 13.27 12.06 11.39 10.97 10.67 10.46 10.29
6 13.75 10.92 9.78 9.15 8.75 8.47 8.26 8.10
7 12.25 9.55 8.45 7.85 7.46 7.19 6.99 6.84
8 11.26 8.65 7.59 7.01 6.63 6.37 6.18 6.03
9 10.56 8.02 6.99 6.42 6.06 5.80 5.61 5.47
10 10.04 7.56 6.55 5.99 5.64 5.39 5.20 5.06
11 9.65 7.21 6.22 5.67 5.32 5.07 4.89 4.74
12 9.33 6.93 5.95 5.41 5.06 4.82 4.64 4.50
13 9.07 6.70 5.74 5.21 4.86 4.62 4.44 4.30
14 8.86 6.51 5.56 5.04 4.69 4.46 4.28 4.14
15 8.68 6.36 5.42 4.89 4.56 4.32 4.14 4.00
16 8.53 6.23 5.29 4.77 4.44 4.20 4.03 3.89
17 8.40 6.11 5.19 4.67 4.34 4.10 3.93 3.79
18 8.29 6.01 5.09 4.58 4.25 4.01 3.84 3.71
19 8.19 5.93 5.01 4.50 4.17 3.94 3.77 3.63
20 8.10 5.85 4.94 4.43 4.10 3.87 3.70 3.56
30 7.56 5.39 4.51 4.02 3.70 3.47 3.30 3.17
40 7.31 5.18 4.31 3.83 3.51 3.29 3.12 2.99
50 7.17 5.06 4.20 3.72 3.41 3.19 3.02 2.89
60 7.08 4.98 4.13 3.65 3.34 3.12 2.95 2.82
70 7.01 4.92 4.07 3.60 3.29 3.07 2.91 2.78
80 6.96 4.88 4.04 3.56 3.26 3.04 2.87 2.74
90 6.93 4.85 4.01 3.54 3.23 3.01 2.84 2.72
100 6.90 4.82 3.98 3.51 3.21 2.99 2.82 2.69
∞ 6.63 4.61 3.78 3.32 3.02 2.80 2.64 2.51

8
Points fα,ν ,ν for α = 0.025.
1 2
ν2 ν1
1 2 3 4 5 6 7 8
1 647.79 799.50 864.16 899.58 921.85 937.11 948.22 956.65
2 38.51 39.00 39.17 39.25 39.30 39.33 39.36 39.37
3 17.44 16.04 15.44 15.10 14.88 14.73 14.62 14.54
4 12.22 10.65 9.98 9.60 9.36 9.20 9.07 8.98
5 10.01 8.43 7.76 7.39 7.15 6.98 6.85 6.76
6 8.81 7.26 6.60 6.23 5.99 5.82 5.70 5.60
7 8.07 6.54 5.89 5.52 5.29 5.12 4.99 4.90
8 7.57 6.06 5.42 5.05 4.82 4.65 4.53 4.43
9 7.21 5.71 5.08 4.72 4.48 4.32 4.20 4.10
10 6.94 5.46 4.83 4.47 4.24 4.07 3.95 3.85
11 6.72 5.26 4.63 4.28 4.04 3.88 3.76 3.66
12 6.55 5.10 4.47 4.12 3.89 3.73 3.61 3.51
13 6.41 4.97 4.35 4.00 3.77 3.60 3.48 3.39
14 6.30 4.86 4.24 3.89 3.66 3.50 3.38 3.29
15 6.20 4.77 4.15 3.80 3.58 3.41 3.29 3.20
16 6.12 4.69 4.08 3.73 3.50 3.34 3.22 3.12
17 6.04 4.62 4.01 3.66 3.44 3.28 3.16 3.06
18 5.98 4.56 3.95 3.61 3.38 3.22 3.10 3.01
19 5.92 4.51 3.90 3.56 3.33 3.17 3.05 2.96
20 5.87 4.46 3.86 3.51 3.29 3.13 3.01 2.91
30 5.57 4.18 3.59 3.25 3.03 2.87 2.75 2.65
40 5.42 4.05 3.46 3.13 2.90 2.74 2.62 2.53
50 5.34 3.97 3.39 3.05 2.83 2.67 2.55 2.46
60 5.29 3.93 3.34 3.01 2.79 2.63 2.51 2.41
70 5.25 3.89 3.31 2.97 2.75 2.59 2.47 2.38
80 5.22 3.86 3.28 2.95 2.73 2.57 2.45 2.35
90 5.20 3.84 3.26 2.93 2.71 2.55 2.43 2.34
100 5.18 3.83 3.25 2.92 2.70 2.54 2.42 2.32
∞ 5.02 3.69 3.12 2.79 2.57 2.41 2.29 2.19

9
Points fα,ν ,ν for α = 0.05.
1 2
ν2 ν1
10 12 15 20 24 30 40 100
1 241.88 243.90 245.95 248.01 249.05 250.09 251.14 253.04
2 19.40 19.41 19.43 19.45 19.45 19.46 19.47 19.49
3 8.79 8.74 8.70 8.66 8.64 8.62 8.59 8.55
4 5.96 5.91 5.86 5.80 5.77 5.75 5.72 5.66
5 4.74 4.68 4.62 4.56 4.53 4.50 4.46 4.41
6 4.06 4.00 3.94 3.87 3.84 3.81 3.77 3.71
7 3.64 3.57 3.51 3.44 3.41 3.38 3.34 3.27
8 3.35 3.28 3.22 3.15 3.12 3.08 3.04 2.97
9 3.14 3.07 3.01 2.94 2.90 2.86 2.83 2.76
10 2.98 2.91 2.85 2.77 2.74 2.70 2.66 2.59
11 2.85 2.79 2.72 2.65 2.61 2.57 2.53 2.46
12 2.75 2.69 2.62 2.54 2.51 2.47 2.43 2.35
13 2.67 2.60 2.53 2.46 2.42 2.38 2.34 2.26
14 2.60 2.53 2.46 2.39 2.35 2.31 2.27 2.19
15 2.54 2.48 2.40 2.33 2.29 2.25 2.20 2.12
16 2.49 2.42 2.35 2.28 2.24 2.19 2.15 2.07
17 2.45 2.38 2.31 2.23 2.19 2.15 2.10 2.02
18 2.41 2.34 2.27 2.19 2.15 2.11 2.06 1.98
19 2.38 2.31 2.23 2.16 2.11 2.07 2.03 1.94
20 2.35 2.28 2.20 2.12 2.08 2.04 1.99 1.91
30 2.16 2.09 2.01 1.93 1.89 1.84 1.79 1.70
40 2.08 2.00 1.92 1.84 1.79 1.74 1.69 1.59
50 2.03 1.95 1.87 1.78 1.74 1.69 1.63 1.52
60 1.99 1.92 1.84 1.75 1.70 1.65 1.59 1.48
70 1.97 1.89 1.81 1.72 1.67 1.62 1.57 1.45
80 1.95 1.88 1.79 1.70 1.65 1.60 1.54 1.43
90 1.94 1.86 1.78 1.69 1.64 1.59 1.53 1.41
100 1.93 1.85 1.77 1.68 1.63 1.57 1.52 1.39
∞ 1.83 1.75 1.67 1.57 1.52 1.46 1.39 1.24

10
Inverse Property of the F-Distribution

Show that the critical value of the F-distribution satisfies the identity:
1
f1−α,ν1 ,ν2 =
fα,ν1 ,ν2

▶ This property is particularly useful when working with F-tables and

hypothesis testing.

11
Distribution of the Ratio of Sample Variances

Theorem: If S12 and S22 are the variances of independent random

samples of sizes n1 and n2 from two normal populations with variances
σ12 and σ22 , respectively. Then

S12 /σ12
F =
S22 /σ22

is a random variable having a F-distribution with n1 − 1 and n2 − 1

degrees of freedom.

12
Example: Let S12 and S22 denote the variances of independent random
samples of size n1 = 8 and n2 = 12, taken from normal populations with
the same standard deviation. Estimate:
S12
P( < 4.89)
S22

13
Order Statistic
Order statistic

▶ Consider a random sample X1 , X2 , . . . , Xn from a continuous

population. We arrange the values in increasing order

Y1 , Y2 , . . . , Yn

and refer to them as order statistics. That is Y1 := min{X1 , X2 , . . . , Xn }

is the smallest and Yn := max{X1 , X2 , . . . , Xn } is the largest. The term
Yr is called r ’th order statistics.
▶ We denote the sample median X̃ . In a random sample of size
n = 2m + 1, it is Ym+1 . In a random sample of size n = 2m, it is defined
as 12 (Ym + Ym+1 ).

14
Example

Let f and F denote the population density and distribution function

respectively. In terms of f and F , derive the density of Y1 and Yn .

15
Theorem: The distribution of r ’th order statistic Yr is given by
n  
X n
FYr (y ) = [F (y )]k [1 − F (y )]n−k
k
k=r

and the density is

n!
fYr (y ) = f (y )[F (y )]r −1 [1 − F (y )]n−r
(n − 1)!(n − r )!

for −∞ < y < ∞.

16
Example: Let X1 , X2 , . . . , Xn be a random sample taken from a
continuous population with the density
(
1 −x/θ
e for x > 0
f (x) = θ
0 elsewhere

Find the density and expected value of Y1 and Yn .

17
Example: Let X1 , X2 , . . . , Xn be a random sample taken from a
continuous population with the density
(
1
for 0 < x < θ
f (x) = θ
0 elsewhere

Find the density and expected value of Y1 and Yn .

18
Example: Consider a random sample X1 , X2 , X3 , X4 from a uniform
distribution on the interval (0, 1). Calculate the expected value E(Y3 ) of
the third order statistics.

19
Point Estimation
Introduction

There are two major topics in statistics:

▶ Estimation: Point estimation (Ch. 10) and Interval estimation (Ch. 11).
▶ Testing hypotheses (Ch. 12).

First we discuss point estimation.

▶ What is point estimation?

▶ We have a random sample from a probability distribution.
▶ The distribution is known, except for one or more parameters, usually
denoted θ (or θ1 , . . . , θk ).
▶ Question: can we estimate the value of the parameter(s)?
▶ The “true” value remains usually unknown, even after taking a sample.
From the sample we calculate the estimate θ̂ of the parameter.

20
Statistics

Definition: Let X1 , X2 , . . . , Xn be a random sample. A statistic is a

function of our observations and also possibly of other known parameters.
That is, it is a random variable of the form h(X1 , X2 , . . . , Xn ) for a given
function h that is allowed to depend on known parameters as well.
Example of statistics:

▶ Sample mean: X
▶ Sample median: S 2
▶ Sample median
▶ Order statistics

21
Point Estimation

Definition: Using the value of a sample statistic to estimate the value of

a population parameter is called point estimation. We refer to the value
of the statistic as a point estimate. Correspondingly, we refer to the
statistics themselves as point estimators.
Remark: For some unknown parameters, there are natural candidates as
estimators. For example, when the population mean µ is unknown, the
sample mean X can be used as an estimator. When population variance
σ 2 must be estimated, the sample variance S 2 can be used as an
estimator. Sometimes there are more than one candidate; for example,
when µ is unknown, you can use the sample mean X , the sample median,
or X1 + Xn /2. In such cases, it is a good idea to compare the candidates
based on their properties.

22
Choosing the best estimator

What makes a good estimator? - We need properties that help us

determine which estimator performs best.
1. Unbiasedness
2. Minimum Variance (Efficiency)
3. Consistency
4. Sufficiency
5. Robustness
These properties may or may not be attained by a point estimator.
Since point estimators are random variables, they have their sampling
distributions. The above properties are directly related to their sampling
distributions. Sampling distributions are discussed in Ch 8.

23
Definition: A statistic Θ̂ is an unbiased estimator of the parameter θ of
a given distribution if and only if E[Θ̂] = θ for all possible values of θ.

24
Example: If X has the binomial distribution with the parameters n and
θ, show that the sample proportion, Xn , is an unbiased estimator of θ.

25
Example: Let X1 , . . . , Xn be a random sample from a continuous
population with the density function
(
e −(x−λ) for x > λ
f (x) =
0 elsewhere

for some unknown parameter λ > 0.

(a) Determine whether X is an unbiased estimator of λ.
(b) Determine whether X − 1 is an unbiased estimator of λ.

26
Asymptotically Unbiased Estimator

Defrinition: An estimator Θ̂ is biased if E[Θ̂] ̸= θ.

Definition: The bias of an estimator is given by:

bn (θ) = E[Θ̂] − θ

▶ If bn (θ) > 0, the estimator tends to overestimate.

▶ If bn (θ) < 0, the estimator tends to underestimate.
Definition: The estimator Θ̂ of θ is asymptotically unbiased if

lim bn (θ) = 0
n→∞

27
Example: Suppose X1 , X2 , . . . , Xn is a random sample from a uniform
distribuiton on (0, θ), where θ is an unknown parameter. Show that the
estimator Yn = max{X1 , . . . , Xn } is asymptotically unbiased.

28