Stat 400, section 6.

1b Point Estimates of Mean and Variance

notes by Tim Pilachowski
What we have so far:
Researchers often know a lot about a population, including the probability distribution, but the value of the
( )
population parameter remains unknown. Examples of common parameters are mean (µ), variance σ 2 , median
(µ~ ) , and proportion (p). A population parameter has a value, however we usually don’t know what that value is.
“A point estimate of a parameter θ is a single number that can be regarded as a sensible value for θ … The
selected statistic is called the point estimator of θ.” The symbol θˆ is used for both the random variable and the
calculated value of the point estimate.
()
Ideally, the point estimator θˆ is unbiased, i.e. E θˆ = θ . In words, the sampling distribution based on the
statistic has an expected value equal to the actual (but unknown) population parameter.
Task 1: Show that the point estimator µ̂ = X (sample mean) is an unbiased estimator of the population
parameter µ. That is, show E (X ) = µ .
We already did this in Lecture 5.4b, as part of the development of the Central Limit Theorem.
Random variables X1, X2, … , Xn form a (simple) random sample of size n if they meet two (important)
requirements:
1. The Xi’s are independent random variables.
2. Every Xi has the same probability distribution.
Given a linear transformation/change of variables which is a sum of n independent random variables,
Y = a1 X1 + a2 X 2 + K + an X n ⇒ E (Y ) = µY = a1µ X 1 + a2 µ X 2 + K + an µ X n .
notes on the proof:
X1 + X 2 + K + X n
X =
n
1
E (X ) = [E ( X1 ) + E ( X 2 ) + K + E ( X n )]
n
1
= [µ + µ + K + µ ]
n
1
= [n ∗ µ ]
n
=µ
Note that, if we were to pick a single, randomly chosen element of the population, it would be true that our
statistic would be unbiased, i.e. E(X) = µ. Why don’t we just do that? Why is it better to choose a random
sample of size n?

Principle: Among all unbiased estimators of a population parameter θ, choose one that has the minimum
variance.
Task 2: Show that the point estimator σˆ 2 = S 2 (sample variance) is an unbiased estimator of the population
( )
parameter σ 2 . That is, show E S 2 = σ 2 .
First, we take a short side trip, using the formula for sample mean.
1 n n
X = ∑ X i ⇒ nX = ∑ X i
n i =1 i =1

We begin with the “sum of squares” formula.

notes on the proof:
n n
∑ (X i − X ) ∑ (X i 2 − 2 X i X + X 2 )
2
=
i =1 i =1
n n n
= ∑ X i2 − ∑ 2 X i X + ∑ X 2
i =1 i =1 i =1
n n n
= ∑ X i2 − 2 X ∑ X i + X 2 ∑ 1
i =1 i =1 i =1
n
= ∑ X i 2 − 2 X (nX ) + nX 2
i =1
n
= ∑ X i 2 − 2 nX 2 + nX 2
i =1
n
= ∑ X i 2 − nX 2
i =1
2
n 1 n 
= ∑ Xi 2
− n ∑ Xi 
 n i =1 
i =1  
2
n
n  n 
= ∑ X i2 − 2  ∑ X i 
i =1 n  i = 1 
2
n
1 n 
= ∑ X i2 −  ∑ X i 
n  i =1 
i =1  
We now substitute into to the formula for random variable S2, sample variance.
S2 =
1
n −1
[
∑ (X i − X )2 ]
1  1 2
= 
n −1 
∑ X i 2 − (∑ X i ) 
n 
2
How is the formula for S (sample variance) different from the formula for V(X) (population variance)?
Next is another short side trip, using the shortcut formula for population variance.
( )
V (Y ) = E Y 2 − [E (Y )]2
V (Y ) + [E (Y )]2 = E (Y )
2

σ 2 + µ2 = E (Y )
2

Finally, we substitute into the shortcut formula for sample variance and simplify.
notes on the proof:
1  1 2
S2 = 
n −1
∑ X i 2 − (∑ X i ) 
n 

( )
E S2 =
1 
n −1
2 1
∑ E X i − E
n
( ) [(∑ X ) ]
i
2

=
1 
n −1
( ) {
2 1
∑ E X i − V (∑ X i ) + E (∑ X i ) 
n
[ 2 

]}
1  2 
=
n −1
( 2
) {
1
∑ σ + µ − nσ + (nµ ) 
2
n
2

}
1  nσ 2
n µ 
2 2
=
n − 1 
( 2
n σ + µ − )2
n
− 
n 
1
=
n −1
{ nσ 2 + nµ 2 − σ 2 − nµ 2 }
1
=
n −1
{ }
(n − 1) ∗ σ 2
( )
E S2 = σ 2