1 One and two sample estimation problems
The distributions associated with populations are often known except for one
or more parameters. Estimation problems deal with how best to estimate the
value of these parameters and equally important, how to provide a measure of
the confidence in that estimate. In other words, what can one say about the
range of certainty in that estimate. We shall be concerned with the estimation
of the following parameters: the mean, the variance and the proportion.
Notation1 We shall denote parameters by Greek letters such as θ. The statistic
which serves to estimate θ is indicated by the capital Θ̂ and it is a random
ˆ An
variable. A specific value of that random variable is indicated by θ.
estimator is a rule which produces a point estimate based on the observed
data sample X1 , ..., Xn .
Notation2 The population mean will be denoted by µ, the population variance
by σ 2 , and the population proportion by p.
Notation3 There is some variation to the above notation. The sample mean
based on a sample of size n is denoted by X̄n and a specific observed value
is denoted by x̄n . The sample variance based on a sample of size n is
denoted by Sn2 or simply S 2 if the meaning is clear. a specific observed
value is denoted by s2 .
Notation4 The population proportion is denoted by p whereas the point esti-
mator is denoted by P̂ . A specific observed value is denoted by p̂.
1
Definition A statistic Θ̂ is said to an unbiased estimator of a parameter θ if
E Θ̂ = θ
The estimator with the smallest variance among all unbiased estimators
is called the most efficient estimator of θ.
Theorem The sample mean X̄n is an unbiased estimator for the population
mean µ.
Theorem The sample variance S 2 is an unbiased estimator for the population
variance σ 2 .
1.1 Variance of a point estimator
Sometimes we have more than point estimator for the same parameter. In
order to make a choice we impose certain restrictions. One is that the estimator
should be unbiased. The other is the estimator should have a smaller variance.
Among all unbiased estimators, the one with the smallest variance is called the
most efficient estimator. As an example, for a sample of size 3, both X̄3 and
X1 +2X2 +3X3
Θ̂ = 6 are unbiased. But X̄3 is more efficient since
σ2 14
V ar X̄3 = , V ar Θ̂ = σ2
3 36
2
1.2 Interval estimates
In addition to point estimators for parameters, we are also interested in interval
estimates which provide some measure of uncertainty. In order to construct
interval estimates, we need to find two random variables, Θ̂L , Θ̂U such that
P Θ̂L < θ < Θ̂U = 1 − α, 0 < α < 1.
In other words, we would like the parameter to be contained in an interval.
The interval computed from the sample, θ̂L < θ < θ̂U is called a 100 (1 − α) %
confidence interval. The endpoints of the interval are called the confidence
limits. In general, we would like a short confidence interval with a high degree
of confidence.
1.3 Estimating the mean of a single population when the
variance is known
Assume that either the sample comes from a normal distribution or the sample
size n is very large. In either case, if X̄n is used as an estimator of µ, then with
probability (1 − α) , the mean µ will be included in the interval X̄n ±zα/2 √σn .
Hence,
σ σ
Θ̂L = X̄n − zα/2 √ , Θ̂U = X̄n + zα/2 √
n n
3
The width of the confidence interval is
σ
Θ̂U − Θ̂L = 2zα/2 √
n
The error defined as the half width, will not exceed e when the sample size is
z
α/2 σ
2
n=
e
We note that the width of the confidence interval decreases as the sample
size increases. It increases as the standard deviation increases and as the the
confidence level increases.
Defintion The standard error of an estimate is the standard deviation of the
estimate.
Hence, the standard error of X̄ deboted s.e. X̄ = √σn
Example The amount of butterfat in pounds produced by a cow during a
305 day milk production period is normally distributed with mean µ and
standard deviation 89.75. Construct a 95% confidence for the mean µ on
the basis of a random szmple of size 20 when the sample mean is 507.5.
Here, x̄20 = 507.5, σ = 89.75, α = 0.05. Hence z0.025 = 1.96. The interval
becomes
89.75
507.5 ± 1.96 √ = 507.5 ± 39.34
20
4
It is felt that the sample size is too large. If the level of confidence is
decreased to 90%, and the margin of error increased to e = 45, what would be
the sample size necessary?
2
1.645 (89.75)
n= = 10.76 ∼
= 11
45
1.4 Estimating the mean of a single population when the
variance is unknown
When the variance of a normal population is unknown, we may construct a
confidence interval based on the distribution of the random variable
√
n X̄n − µ
T =
S
which is Student t with n−1 degrees of freedom. In that case, we have that with
probability (1 − α) , the mean µ will be included in the interval X̄n ±tα/2 √Sn .
Hence,
S S
Θ̂L = X̄n − tα/2 √ , Θ̂U = X̄n + tα/2 √
n n
Example The amount of butterfat in pounds produced by a cow during a 305
day milk production period is normally distributed with mean µ. On the
basis of a random szmple of size 20, the sample mean is 507.5. and the
sample standard deviation is 89.75. Construct a 95% confidence for the
mean µ.
5
Here, x̄20 = 507.5, s = 89.75, α = 0.05. Hence t0.025 = 2.093 when the number
of degrees of freedom is n − 1 = 19. The interval becomes
89.75
507.5 ± 2.093 √ = 507.5 ± 42.00
20
We see that the interval is wider when the variance is unknown. We refer to √σ
n
as the standard error and to √s as the estimate of the standard error.
n
1.5 Estimating the difference between two means
Example Suppose that we have two different feeding programs for fattening
beef cattle. We are interested in seeing if there is a significant difference
between them.
Suppose that we have two normal populations with means µ1 , µ2 and variances
σ12 ,σ22 respectively. We would like a confidence interval for the difference µ1 −
µ2 . Several cases present themselves which we can tabulate as follows. Let
X̄1 , X̄2 represent the sample means form the two populations based on samples
n1 n2 respectively.
Variances Interval Definitions d.f.
q 2
σ σ2
σ12 , σ22 known X̄1 − X̄2 ± zα/2 n11 + n22
(n1 −1)S12 +(n2 −1)S22
q
σ12 = σ22 = σ 2 unknown X̄1 − X̄2 ± tα/2 SP n11 + n12 SP2 = n1 +n2 −2 n1 + n2 − 2
q 2
S S2
σ12 , σ22 unknown X̄1 − X̄2 ± tα/2 n11 + n22 *
6
2
2
S1 S2
+ n2
n1 2
* = S2 2 2 2
S
1 /(n1 −1)+ n2 /(n2 −1)
n1 2
1.6 Paired observations
Example An experiment is conducted to compare people’s reaction time to a
red light vs a green light in seconds.
Red (x) Green (y) d=x−y
1 0.30 0.43 -0.13
2 0.23 0.32 -0.09
3 0.41 0.58 -0.17
4 0.53 0.46 0.07
5 0.24 0.27 -0.03
6 0.36 0.41 -0.05
7 0.38 0.38 0
8 0.51 0.61 -0.10
In the case of paired samples, we can no longer assume that the X 0 s and Y 0 s
are independent. Instead we may assume that the differences D1 , ..., Dn constitute
a random sample from say a normal distribution with mean µD and variance
2
σD . In that case we are back to the one sample problem. In our example, a
confidence interval for the mean is given by
Sd
D̄ ± tα/2 √
n
7
Here, we have
0.0765
−0.0625 ± 2.365 √
8
or
(−0.1265, 0.0015)
We see that 0 is in the interval so that we conclude there is no difference between
the two reaction times. However, the upper limit is very small. Had it been
negative, we would have concluded that people react faster to a red light.
Exercise Show that if this example were treated as a two sample problem, the
variance would increase, the d.f. would increase but the t-value would
decrease.
1.7 Single sample: estimating a proportion
Suppose that X is a random variable having a binomial distribution with param-
eters n, p. We are interested in estimating p. Here X represents the number of
successes in mn repetitions of a Bernoulli experiment. As examples, we may be
interested in estimating the proportion of voters in favour of a certain candidate
or perhaps the success rate of a certain drug in curing the common cold.The
construction of a confidence interval is based on the Central Limit approxima-
tion.
If P̂ is used as an estimator of p, then a 100 (1 − α) % confidence interval for
q
p is given by P̂ ± zα/2 P̂ Q̂/n.
8
The error will not exceed e when the sample size is
z 2
α/2
n = p̂q̂
e
1 zα/2 2
M ax n =
4 e
Example From a random sample of 400 citizens in Ottawa, there were 136 who
indicated that the city’s transpotation system is adequate. Construct a
99% confidence interval for the population proportion who feel the trans-
portation system is adequate.
136
√
Here, p̂ = 400 = 0.34, p̂q̂=0.2244,zα/2 = 2.575. The confidence interval is then
r
0.34 (0.66)
0.34 ± 2.575
400
0.34 ± 0.061
Example What is the sample size necessary to estimate the proportion of vot-
ers in favor of a candidate if we allow a maximum error of 3% and would
like a confidence of 95%?
z 2
α/2
n = maxp p (1 − p)
e
2
1 1.96 ∼
= = 1068
4 0.03
9
1.8 Estimating the difference between two proportions
For two proportions P1 , P2 a point estimate of the difference P1 − P2 is given by
P̂1 − P̂2
A 100 (1 − α) % confidence interval for p1 − p2 is given by
s
P̂1 Q̂1 P̂2 Q̂2
P̂1 − P̂2 ± zα/2 +
n1 n2
Example Two detergents were tested for their ability to remove grease stains.
Type A was successful on 63 stains out of 91 whereas type B was successful
on 42 stains out of 79. We construct a 90% confidence interval for the
difference Type A-Type B and obtain
(0.038, 0.282)
Since the interval does not include 0, it seems that
PA − PB > 0
and hence Type A is better than Type B.
10
1.9 Estimating the variance
Suppose that we are given a random sample of size n from a normal population
with variance σ 2 . Estimation of the variance is based on the statistic
(n − 1) S 2
∼ χ2n−1
σ2
We obtain from the chi square distribution two values χ2n−1 (1 − α/2) , χ2n−1 ((α/2))
such that
(n − 1) S 2
2 2
P χn−1 (1 − α/2) < < χn−1 ((α/2)) = 1 − α
σ2
Reworking the inner expression we see that a 100 (1 − α) %confidence interval
for σ 2 is given by
(n − 1) S 2 (n − 1) S 2
2 , 2
χn−1 ((α/2)) χn−1 (1 − α/2)
Example The time required in days based on a sample of size 13 for a mat-
uration of seeds for a certain type of plant is 18.97 days with a sample
variance of 10.7.
A 90% confidence interval for the population variance is
128.41 128.41
, = (6.11, 24.57)
21.03 5.226
11
2 One and two-sample tests of hypotheses
A statistical hypothesis is a conjecture concerning one or more populations. For
example, we may believe that a coin is balanced or that the average height of
soldiers in the Canadian army is 5’9”. Statistical methods can be used to verify
such claims.
There are two possible actions that one may take with respect to a claim or
hypothesis:
i) rejection of the null hypothesis when it is true is a Type I error
ii) non rejection of the null hypothesis when it is false is a Type II error
H0 is true H0 is false
Do not reject H0 Correct decision Type II error
Reject H0 Type I error Correct decision
We may illustrate the situation using the normal distribution for the sample
mean. Suppose that under the null hypothesis
X̄n ∼ N 25, 42
whereas under the alternative
X̄n ∼ N 50, 52
We illustrate graphically the various types of error as well as the notion of
p-values and power.
Note that Table 6.3 on page 264 of the text summarizes all the tests con-
12
cerning means from normal distributions, i.e.
1. single mean, variance known
2. single mean,variance unknown
3. difference of two means, variances known
4. difference of two means, varainces equal to a common but unknown mean
5. difference of two means, variances unknown
6. paired observations
13
