Chapter 2 Design and Analysis of Experiments 1

10E 2020 Montgomery

 Design of Engineering Experiments
Chapter 2 – Some Basic Statistical Concepts
• Describing sample data
– Random samples
– Sample mean, variance, standard deviation
– Populations versus samples
– Population mean, variance, standard deviation
– Estimating parameters
• Simple comparative experiments
– The hypothesis testing framework
– The two-sample t-test
– Checking assumptions, validity

Chapter 2 Design & Analysis of Experiments 2

10E 2020 Montgomery
Portland Cement Formulation (Section 2.1)

Chapter 2 Design & Analysis of Experiments 3

10E 2020 Montgomery
 Graphical View of the Data
Dot Diagram, Figure 2.1

The dot diagram enables the experimenter to see quickly the general location or central
tendency of the observations and their spread or variability. It is a very useful device
for displaying a small body of data (say up to about 20 observations)

For example, in the Portland cement tension bond experiment, the dot diagram reveals
that the two formulations may differ in mean strength but that both formulations produce
about the same variability in strength.

Chapter 2 Design & Analysis of Experiments 4

10E 2020 Montgomery
 If you have a large sample, a
histogram may be useful
Histograms should not
be used with fewer than
75–100 observations

Chapter 2 Design & Analysis of Experiments 5

10E 2020 Montgomery
 Box Plots, Figure 2.3, Section 2.2

The box plots indicate that both formulations produce reasonably symmetric
distributions of strength with similar variability or spread.

Chapter 2 Design & Analysis of Experiments 6

10E 2020 Montgomery
• Dot diagrams, histograms, and box plots are useful for summarizing the
information in a sample of data.
• To describe the observations that might occur in a sample more completely,
we use the concept of the probability distribution.

Chapter 2 Design & Analysis of Experiments 7

10E 2020 Montgomery
• Probability Distributions. The probability structure of a random variable, say
y, is described by its probability distribution.
• If y is discrete, we often call the probability distribution of y, say p(y), the
probability mass function of y.
• If y is continuous, the probability distribution of y, say f (y), is often called the
probability density function for y.

Chapter 2 Design & Analysis of Experiments 8

10E 2020 Montgomery
 The Hypothesis Testing Framework

• Statistical hypothesis testing is a useful

framework for many experimental
situations
• Origins of the methodology date from the
early 1900s
• We will use a procedure known as the two-
sample t-test

Chapter 2 Design & Analysis of Experiments 13

10E 2020 Montgomery
 The Hypothesis Testing Framework

• Sampling from a normal distribution

• Statistical hypotheses: is a statement either about the parameters of a
probability distribution or the parameters of a model. The hypothesis
reflects some conjecture about the problem situation

Chapter 2 Design & Analysis of Experiments 14

10E 2020 Montgomery
 The Hypothesis Testing Framework
In the Portland cement experiment, we may think that the mean tension
bond strengths of the two mortar formulations are equal. This may be
stated formally as
H 0 : 1 = 2
H1 : 1  2

• 𝜇1 is the mean tension bond strength of the modified mortar

• 𝜇2 is the mean tension bond strength of the unmodified mortar.

The statement H0∶𝜇1 = 𝜇2 is called the null hypothesis

The statement H1∶𝜇1 ≠ 𝜇2 is called the alternative hypothesis.
The alternative hypothesis specified here is called a two-sided alternative
hypothesis because it would be true if 𝜇1 < 𝜇2 or if 𝜇1 > 𝜇2.

Chapter 2 Design & Analysis of Experiments 15

10E 2020 Montgomery
 The Hypothesis Testing Framework

𝛼 = P(type I error) = P(reject H |H is true)

0 0

𝛽 = P(type II error) = P(fail to reject H |H is false)

0 0

Sometimes it is more convenient to work with the power

of the test, where
Power = 1 − 𝛽 = P(reject H |H is false)
0 0

The general procedure in hypothesis testing is to specify a

value of the probability of type I error 𝛼, often called the
significance level of the test, and then design the test
procedure so that the probability of type II error 𝛽 has a
suitably small value.

Chapter 2 Design & Analysis of Experiments 16

10E 2020 Montgomery
 Summary Statistics (page 34)
Modified Mortar Unmodified Mortar

“New recipe” “Original recipe”

y1 = 16.76 y2 = 17.04
S = 0.100
1
2
S 22 = 0.061
S1 = 0.316 S 2 = 0.248
n1 = 10 n2 = 10

Chapter 2 Design & Analysis of Experiments 17

10E 2020 Montgomery
 Estimation of Parameters
1 n
y =  yi estimates the population mean 
n i =1
n
1
S =
2

n − 1 i =1
( yi − y ) estimates the variance 
2 2

Chapter 2 Design & Analysis of Experiments 18

10E 2020 Montgomery
 How the Two-Sample t-Test Works:
Use the sample means to draw inferences about the population means
y1 − y2 = 16.76 − 17.04 = −0.28
Difference in sample means
Standard deviation of the difference in sample means

2  12  22
 =
2
y , and  2
y1 − y2 = + , y1 and y2 independent
n n1 n2
This suggests a statistic:
y1 − y2
Z0 =
 12  22
+
n1 n2
If the variances were known we could use the normal distribution as the basis of a test
Z0 has a N(0,1) distribution if the two population means are equal

Chapter 2 Design & Analysis of Experiments 19

10E 2020 Montgomery
 If we knew the two variances how would we use Z0 to test H0?

Suppose that σ1 = σ2 = 0.30. Then we can calculate

y1 − y2 −0.28 −0.28
Z0 = = = = −2.09
 12  22 0.32 0.32 0.1342
+ +
n1 n2 10 10

How “unusual” is the value Z0 = -2.09 if the two population means are equal?
It turns out that 95% of the area under the standard normal curve (probability)
falls between the values Z0.025 = 1.96 and - Z0.025 = -1.96.
So the value Z0 = -2.09 is pretty unusual in that it would happen less that 5%
of the time if the population means were equal

Chapter 2 Design & Analysis of Experiments 20

10E 2020 Montgomery
 Standard Normal Table (see appendix)

Z0.025 = 1.96

Chapter 2 Design & Analysis of Experiments 21

10E 2020 Montgomery
 So if the variances were known we would conclude that we should reject
the null hypothesis at the 5% level of significance
H 0 : 1 = 2
H1 : 1  2
and conclude that the alternative hypothesis is true.
This is called a fixed significance level test, because we compare the value
of the test statistic to a critical value (1.96) that we selected in advance
before running the experiment.
The standard normal distribution is the reference distribution for the test.
Another way to do this that is very popular is to use the P-value approach.
The P-value can be thought of as the observed significance level.
For the Z-test it is easy to find the P-value.

Chapter 2 Design & Analysis of Experiments 22

10E 2020 Montgomery
 Normal Table

Find the probability above Z0

= -2.09 from the table.
This is 1 – 0.98169 = 0.01832
The P-value is twice this
probability, or 0.03662.
So we would reject the null
hypothesis at any level of
significance that is less than or
equal to 0.03662.
Typically 0.05 is used as the
cutoff.

Chapter 2 Design & Analysis of Experiments 23

10E 2020 Montgomery
 The t -Test
• The Z-test just described would work perfectly if we knew
the two population variances.
• Since we usually don’t know the true population variances,
what would happen if we just plugged in the sample
variances?
• The answer is that if the sample sizes were large enough
(say both n > 30 or 40) the Z-test would work just fine. It
is a good large-sample test for the difference in means.
• But many times that isn’t possible (as Gosset wrote in
1908, “…but what if the sample size is small…?).
• It turns out that if the sample size is small we can no longer
use the N(0,1) distribution as the reference distribution for
the test.
Chapter 2 Design & Analysis of Experiments 24
10E 2020 Montgomery
 How the Two-Sample t-Test Works:
Use S and S to estimate  and 
1
2 2
2
2
1
2
2

y1 − y2
The previous ratio becomes
2 2
S S
+
1 2
n1 n2
However, we have the case where  =  =  2
1
2
2
2

Pool the individual sample variances:

( n − 1) S 2
+ ( n − 1) S 2
Sp = 1
2 1 2 2
n1 + n2 − 2
Chapter 2 Design & Analysis of Experiments 25
10E 2020 Montgomery
 How the Two-Sample t-Test Works:
The test statistic is
y1 − y2
t0 =
1 1
Sp +
n1 n2
• Values of t0 that are near zero are consistent with the null
hypothesis
• Values of t0 that are very different from zero are consistent
with the alternative hypothesis
• t0 is a “distance” measure-how far apart the averages are
expressed in standard deviation units
• Notice the interpretation of t0 as a signal-to-noise ratio
Chapter 2 Design & Analysis of Experiments 26
10E 2020 Montgomery
 The Two-Sample (Pooled) t-Test
(n1 − 1) S12 + (n2 − 1) S 22 9(0.100) + 9(0.061)
S =
2
= = 0.081
n1 + n2 − 2 10 + 10 − 2
p

S p = 0.284

y1 − y2 16.76 − 17.04
t0 = = = −2.20
1 1 1 1
Sp + 0.284 +
n1 n2 10 10

The two sample means are a little over two standard deviations apart
Is this a "large" difference?

Chapter 2 Design & Analysis of Experiments 27

10E 2020 Montgomery
William Sealy Gosset (1876, 1937)

Gosset's interest in barley cultivation led

him to speculate that design of
experiments should aim, not only at
improving the average yield, but also at
breeding varieties whose yield was
insensitive (robust) to variation in soil and
climate.

Developed the t-test (1908)

Gosset was a friend of both Karl Pearson

and R.A. Fisher, an achievement, for each
had a monumental ego and a loathing for
the other.

Gosset was a modest man who cut short

an admirer with the comment that “Fisher
would have discovered it all anyway.”

Chapter 2 Design & Analysis of Experiments 28

10E 2020 Montgomery
 The Two-Sample (Pooled) t-Test
• We need an objective
basis for deciding how t0 = -2.20
large the test statistic t0
really is
• In 1908, W. S. Gosset
derived the reference
distribution for t0 …
called the t distribution
• Tables of the t distribution
– see textbook appendix

Chapter 2 Design & Analysis of Experiments 29

10E 2020 Montgomery
Chapter 2 Design & Analysis of Experiments 30
10E 2020 Montgomery
 The Two-Sample (Pooled) t-Test
• A value of t0 between
–2.101 and 2.101 is t0 = -2.20
consistent with
equality of means
• It is possible for the
means to be equal and
t0 to exceed either
2.101 or –2.101, but it
would be a “rare
event” … leads to the
conclusion that the
means are different
• Could also use the
P-value approach
Chapter 2 Design & Analysis of Experiments 31
10E 2020 Montgomery
 The Two-Sample (Pooled) t-Test

t0 = -2.20

• The P-value is the area (probability) in the tails of the t-distribution beyond -2.20 + the
probability beyond +2.20 (it’s a two-sided test)
• The P-value is a measure of how unusual the value of the test statistic is given that the null
hypothesis is true
• The P-value the risk of wrongly rejecting the null hypothesis of equal means (it measures
rareness of the event)
• The exact P-value in our problem is P = 0.042 (found from a computer)

Chapter 2 Design & Analysis of Experiments 32

10E 2020 Montgomery
 Approximating the P-value
Our t-table only gives probabilities greater than positive values of t.
So take the absolute value of t0 = -2.20 or |t0|= 2.20.
Now with 18 degrees of freedom, find the values of t in the table that
bracket this value.
These are 2.101 < |t0|= 2.20 < 2.552. The right-tail probability for t =
2.101 is 0.025 and for t = 2.552 is 0.01. Double these probabilities
because this is a two-sided test.
Therefore the P-value must lie between these two probabilities, or
0.05 < P-value < 0.02
These are upper and lower bounds on the P-value.
We know that the actual P-value is 0.042.

Chapter 2 Design & Analysis of Experiments 33

10E 2020 Montgomery
 Computer Two-Sample t-Test Results

Chapter 2 Design & Analysis of Experiments 34

10E 2020 Montgomery
 Checking Assumptions –
The Normal Probability Plot

Chapter 2 Design & Analysis of Experiments 35

10E 2020 Montgomery
 Importance of the t-Test
• Provides an objective framework for simple
comparative experiments
• Could be used to test all relevant hypotheses
in a two-level factorial design, because all
of these hypotheses involve the mean
response at one “side” of the cube versus
the mean response at the opposite “side” of
the cube
Chapter 2 Design & Analysis of Experiments 36
10E 2020 Montgomery
Confidence Intervals (Section 2.4.1)
• Hypothesis testing gives an objective statement
concerning the difference in means, but it doesn’t
specify “how different” they are
• General form of a confidence interval
L    U where P( L    U ) = 1 − 
• The 100(1- α)% confidence interval on the
difference in two means:
y1 − y2 − t / 2, n1 + n2 − 2 S p (1/ n1 ) + (1/ n2 )  1 −  2 
y1 − y2 + t / 2,n1 + n2 − 2 S p (1/ n1 ) + (1/ n2 )

Chapter 2 Design & Analysis of Experiments 37

10E 2020 Montgomery
Chapter 2 Design & Analysis of Experiments 38
10E 2020 Montgomery
 What if the Two Variances are Different?

Chapter 2 Design & Analysis of Experiments 39

10E 2020 Montgomery
Chapter 2 Design & Analysis of Experiments 40
10E 2020 Montgomery
Chapter 2 Design & Analysis of Experiments 41
10E 2020 Montgomery
Chapter 2 Design & Analysis of Experiments 42
10E 2020 Montgomery
 Other Chapter Topics
• Hypothesis testing when the variances are
known
• One sample inference (t and Z tests)
• Hypothesis tests on variances (F tests)
• Paired experiments – this is an example of
blocking

Chapter 2 Design & Analysis of Experiments 43

10E 2020 Montgomery