Design of Experiments
introduction
introduction
• A designed experiment is a test or series of tests
in which purposeful changes are made to the
input factors of a process so that we may
observe and identify corresponding changes in
the output response
• The objectives of the experiment may include:
– Determining which variables are most influential on response y
– Determining where to set the influential x’s so that y is near
nominal requirement
– Determining where to set the influential x’s so that variability in
y is small
– Determining where to set the influential x’s so that the effects
of the uncontrollable variables z are minimized
introduction
• Statistical process control (SPC) and experimental
design are closely interrelated
• SPC passive statistical method
Experimental design active statistical method
• Application of experimental design techniques
early in process development can result in
– Improved yield
– Reduced variability and closer conformance to nominal
– Reduced development time
– Reduced overall costs
introduction
• Experimental design methods can also play
major role in engineering design activities,
where new products are developed and existing
ones improved
• Some application of statistical experimental
design in engineering design include:
– Evaluation and comparison of basic design
configuration
– Evaluation of material alternatives
– Determination of key product design parameters that
impact performance
ANOVA :: single factor
• The simplest case would be experimental design
that involves only single factor
• It considers the effect of one controlled
parameter upon the performance of a product or
process one-way ANOVA
• Experiment is performed by choosing several (a)
levels for the factor, and conducting n
observations for each level
• Any output response yij may be described:
yij i ij i 1, 2,..., a
j 1, 2,..., n
ANOVA :: single factor
• Total variation can be decomposed into:
– Variation of the average under each factor level ( yi)
around the average of all observations ( y )
SS FACTOR n1 y1 y n2 y2 y ... na ya y
2 2 2
a a yi2. y..2
SS FACTOR n ( yi y )
2
i 1 i 1 n an
– Variation of individual observations ( yij) around the
average under each factor level ( yi)
SS ERROR y11 y1 y12 y1 ... yan ya
2 2 2
a n
SS ERROR ( yij yi ) 2
i 1 j 1
ANOVA :: two factors
• When there are two factors considered (A and B),
value of output response is also affected by
interaction of the two factors
i 1, 2,..., a
yijk i j ( )ij ijk j 1, 2,..., b
k 1, 2,..., n
• We are interested in testing the hypotheses of no
significant factor A effect, no significant factor B
effect, and no significant AB interaction effect
• F-test is utilized within two-way ANOVA
ANOVA :: two factors
• Total sum of squares:
a b
y...2
n
SST y 2
ijk
i 1 j 1 k 1 abn
• Main effects:
a
yi2.. y...2 b y.2j .
y...2
SS A SS B
i 1 bn abn j 1 an abn
• Interaction:
a b yij2. y...2
SS AB SS A SS B
i 1 j 1 n abn
• Error:
SS E SST SS A SS B SS AB
ANOVA :: two factors
• The ANOVA table
Source of Sum of Degrees of
Mean Square F0
variation squares freedom
A SSA a-1 SS A MS A
MS A F0
a 1 MS E
B SSB b-1 SS MS B
MS B B F0
b 1 MS E
Interaction SSAB (a-1)(b-1) SS AB MS AB
MS AB F0
(a 1)(b 1) MS E
Error SSE ab(n-1) SS E
MS E
ab(n 1)
Total SST abn-1
• The F0 values are compared to value from F table for the
specified degrees of freedoms and required risk
ANOVA :: two factors
Example:
Aircraft primer paints are applied to aluminum surfaces
by two methods – dipping and spraying. The purpose of
the primer is to improve paint adhesion; some parts can
be primed using either application method. An engineer
interested in learning whether three different primers
differ in their adhesion properties performed a factorial
experiment to investigate the effect of paint primer type
and application method on paint adhesion.
Application Method
Primer type Dipping Spraying yi..
1 4.0, 4.5, 4.3 --> 12.8 5.4, 4.9, 5.6 --> 15.9 28.7
2 5.6, 4.9, 5.4 --> 15.9 5.8, 6.1, 6.3 --> 18.2 34.1
3 3.8, 3.7, 4.0 --> 11.5 5.5, 5.0, 5.0 --> 15.5 27
y.j. 40.2 49.6 y… = 89.8
ANOVA :: two factors
• Calculate correction factor:
y...2 89.82
CF 448
abn 18
• Calculate total sum of squares:
2
a b n
y
SST yijk 2
...
i 1 j 1 k 1 abn
SST (4.0) 2 (4.5) 2 ... (5.0) 2 448 10.72
• Calculate main effects sum of squares:
a
yi2.. y...2 28.7 2 34.12 27 2
SS primers 448 4.58
i 1 bn abn 23
b y.2j .
y...2 40.22 49.62
SS methods 448 4.91
j 1 an abn 9
ANOVA :: two factors
• Calculate interaction sum of squares:
b y2
a
y...2
SS AB
ij .
SS primers SSmethods
i 1 j 1 n abn
12.82 15.92 ... 15.52
SS AB 488 4.58 4.91 0.24
3
• Calculate error sum of squares:
SS E SST SS A SS B SS AB
SS E 10.72 4.58 4.91 0.24 0.99
ANOVA :: two factors
• ANOVA table for the example:
Source of Sum of Degrees of
Mean Square F0
variation squares freedom
Primer types 4.58 2 2.29 28.63
Methods 4.91 1 4.91 61.38
Interaction 0.24 2 0.12 1.5
Error 0.99 12 0.08
Total 10.72 17
• Suppose that we use α risk 5% F0.05,1,12 = 3.89 and
F0.05,2,12 = 4.75
• We conclude that:
– primer type and application method affect adhesion force
– There is no indication of interaction between these factors
ANOVA :: two factors
• Exercise: Problem 12-1
Phosphor Type
Glass Type 1 2 3
280 300 290
1 290 310 285
285 295 290
230 260 220
2 235 240 225
240 235 230
factorial design :: 22
• In factorial design, factors are varied together
all possible combinations of the levels of the
factors are investigated
• A very useful type of factorial designs is one
which involves k factors, each at two levels 2k
factorial design
• The simplest type is 22 design two factors A
and B, each at two levels: ‘high’ (+) and ‘low’ (-)
High b ab
(+) Factorial effect
Run I A B AB
B 1 (1) + - - +
(1) a
2 a + + - -
Low
(-) 3 b + - + -
Low A High 4 ab + + + +
(-) (+)
factorial design :: 22
• To estimate the amount of effect, compute
average of observations at ‘high’ level, subtract
from it the average at ‘low’ level
a ab b (1) 1
A y A y A a ab b (1)
2n 2n 2n
1
B b ab a (1)
2n
1
AB ab (1) a b
2n
• The quantities in brackets are called contrasts
CONTRASTA a ab b (1)
CONTRASTB b ab a (1)
CONTRASTAB ab (1) a b
factorial design :: 22
• The value of contrast can be used to compute
sums of squares
contrast 2 contrast 2
SS
n (contrast _ coeff ) 2
4n
[a ab b (1)]2
SS A
4n
[b ab a (1)]2
SS B
4n
[ab (1) a b]2
SS AB
4n
factorial design :: 22
Example:
A router is used to cut registration nothes in
PCBs. Variability in notch dimension causes
improper board registration. Two factors are
considered: bit size (A) and speed (B). Two
levels were chosen for each factor, and 22 design
was setup.
Factors
Run A B Vibration Total
1 (1) - - 18.2 18.9 12.9 14.4 64.4
2 a + - 27.2 24 22.4 22.5 96.1
3 b - + 15.9 14.5 15.1 14.2 59.7
4 ab + + 41 43.9 36.3 39.9 161.1
factorial design :: 22
• Calculate numerical effect:
1 1
A a ab b (1) [96.1 161.1 59.7 64.4] 16.64
2n 2(4)
B 7.54
AB 8.71
• ANOVA table:
Source of Sum of Degrees of
Mean Square F0
variation squares freedom
Bit size (A) 1107.226 1 1107.226 185.25
Speed (B) 227.256 1 227.256 38.03
AB 303.631 1 303.631 50.80
Error 71.723 12 5.977
Total 1709.836 15
factorial design :: 22
• Conclusion: …
• Model of vibration on PCB surface:
y 0 1 A 2 B 12 AB
Using the model to predict vibration level:
16.64 7.54 8.71
yˆ 23.83 A B AB
2 2 2
For example, consider small bit and low speed:
16.64 7.54 8.71
yˆ 23.83 ( 1) ( 1) (1)(1) 16.1
2 2 2
factorial design :: 2k, k ≥ 3
• The method for 22 factorial design can be
extended to more than 2 factors
• For example, consider k = 3 factors, each at two
levels eight factor-level combinations (23
factorial design)
bc abc A B C
- - -
c ac + - -
- + -
b + + -
ab
C - - +
B + - +
(1)
a - + +
+ + +
A
factorial design :: 2k, k ≥ 3
• Estimation of main effects:
1
A y A y A a ab ac abc b c bc (1)
4n
1
B yB yB b ab bc abc a c ac (1)
4n
1
C yC yC c ac bc abc a b ab (1)
4n
• Estimation of interaction AB:
1 1
AB (C _ low) [ab b] [a (1)]
2n 2n
1 1
AB (C _ high) [abc bc] [ac c]
2n 2n
1
AB [ab (1) abc c b a bc ac]
4n
factorial design :: 2k, k ≥ 3
• Signs for effects in the 23 design
Treatment Factorial effect
combination I A B AB C AC BC ABC
(1) + - - + - + + -
a + + - - - - + +
b + - + - - + - +
ab + + + + - - - -
c + - - + + - - +
ac + + - - + + - -
bc + - + - + - + -
abc + + + + + + + +
1
ABC [......]
4n
• General formula for effect and sum of squares:
contrast contrast 2
Effect SS
n 2k 1 n2k
factorial design :: 2k, k ≥ 3
Example:
An experiment was performed to investigate the surface
finish of a metal part. The experiment is a 23 factorial
design in the factors feed rate (A), depth of cut (B), and
tool angle (C), with n = 2 replicates.
Design factors Surface
Run A B C finish Totals
1 (1) - - - 9, 7 16
2 a + - - 10, 12 22
3 b - + - 9, 11 20
4 ab + + - 12, 15 27
5 c - - + 11, 10 21
6 ac + - + 10, 13 23
7 bc - + + 10, 8 18
8 abc + + + 16, 14 30
factorial design :: 2k, k ≥ 3
• Estimation of factor A effect:
1 1
A a ab ac abc b c bc (1) 22 27 23 30 20 21 18 16 3.375
4n 4(2)
• Sum of squares for A:
contrast A2 27 2
SS A k
45.5625
n2 2(8)
• ANOVA table:
Source of Sum of Degrees of
Mean Square F0
variation squares freedom
A 45.5625 1 45.5625 18.69
B 10.5625 1 10.5625 4.33
C 3.0625 1 3.0625 1.26
AB 7.5625 1 7.5625 3.10
AC 0.0625 1 0.0625 0.03
BC 1.5625 1 1.5625 0.64
ABC 5.5625 1 5.5625 2.08
Error 19.5 8 2.4375
Total 92.9375 15
factorial design :: 2k, k ≥ 3
• Conclusion: …
• Model of surface finish of metal part:
y 0 1 A 2 B 12 AB
Using corresponding effect estimates:
3.375 1.625 1.375
ˆy 11.0625 A B AB
2 2 2
yˆ 11.0625 1.6875 A 0.8125B 0.6875 AB
For example, when A, B, C are at low level:
yˆ 11.0625 1.6875(1) 0.8125(1) 0.6875(1)(1) 9.25
factorial design :: 2k, k ≥ 3
• All observations can be graphed in a cube:
9.5 14.25
14.25
9.5
9.25
11.25
B (depth)
9.25
C (tool angle)
11.25
A (feed rate)
• The analysis suggests that both feed rate (A) and
depth of cut (B) should be run at low level
factorial design :: 2k, k ≥ 3
• Exercise: Problem 12-2
– 23 factorial design
– n=2
• Run (1): 221, 311
• Run a: 325, 435
• Run b: 354, 348
• Run ab: 552, 472
• Run c: 440, 453
• Run ac: 406, 377
• Run bc: 605, 500
• Run abc: 392, 419
a. Do any of the three factors affect tool life?
b. What combination of factor levels gives highest output?
c. Is there combination of A and C that always gives good results
regardless of B?
factorial design :: 2k, k ≥ 3
• As the number of factors k grows: (1) more
interactions come into play, (2) more runs are
required
• Sparsity of effects principle the system is
usually dominated by main effects and low-order
interactions
• Higher order interactions can be pooled as
errors, this allows single replicate of runs (n = 1)
factorial design :: fractional :: 2k-1
• As opposed to full factorial design, fractional
factorial design requires less than 2k runs in
exchange of resolution of information
• One-half (2k-1) fractional factorial design requires
half as many runs as full 2k factorial design
– 3 factors, observed in 22 = 4 runs
– 4 factors, observed in 23 = 8 runs, etc
factorial design :: fractional :: 2k-1
• For example, 2k-1 design of four factors:
Treatment Factorial effect
combination I A B AB C AC BC ABC
a + + - - - - + +
b + - + - - + - +
c + - - + + - - +
abc + + + + + + + +
ab + + + + - - - -
ac + + - - + + - -
bc + - + - + - + -
(1) + - - + - + + -
• Design is formed by selecting only those runs that yield a
plus on the ABC effect ABC is generator of this
particular fraction
• The defining relation for this design:
I ABC
factorial design :: fractional :: 2k-1
• We obtain the estimates of the main effects:
2
A 1 a b c abc
2
B 1 a b c abc
2
C1 a b c abc
• We obtain also estimates of two-factor interactions:
2
BC 1 a b c abc
2
AC 1 a b c abc
2
AB 1 a b c abc
• Thus linear combination in column A, say lA, estimates A
+ BC, lB estimates B + AC, and lC estimates C + AB
• Two or more effects that have this property are called
aliases
factorial design :: fractional :: 2k-1
• Multiplying any effect by the defining relation yields the
aliases for that effect
A ABC A2 BC BC
B ABC AB 2C AC
C ABC ABC 2 AB
• Now suppose that we had chosen the other one-half
fraction ( I = -ABC )
• The aliases are A = -BC, B = -AC, and C = -AB
• It is possible to select the fraction so that main effects
and low-order interactions of interest will be aliased with
high-order interactions
factorial design :: fractional :: 2k-1
Example
Suppose there are four factors: A, B, C, D to investigate.
We decide to use a 24-1 design with I = ABCD
Run A B C D = ABC Etch rate
1 (1) - - - - 550
2 a + - - + 749
3 b - + - + 1052
4 ab + + - - 650
5 c - - + + 1075
6 ac + - + - 642
7 bc - + + - 601
8 abc + + + + 729
factorial design :: fractional :: 2k-1
• Aliases of main effects:
A ABCD A2 BCD BCD
B ABCD AB 2CD ACD
C ABCD ABC 2 D ABD
D ABCD ABCD 2 ABC
• Aliases of two-factor interactions:
AB ABCD A2 B 2CD CD
AC ABCD A2 BC 2 D BD
AD ABCD A2 BCD 2 BC
• Estimates of main effects and aliases:
l A A BCD 1
4
550 749 1052 650 1075 642 601 729 127
lB B ACD 4
lC C ABD 11.5
lD D ABC 290.51
factorial design :: fractional :: 2k-1
• lA and lD are large, and if we believe that three-factor
interactions are negligible, then the main effects A and
D are ‘significant’
• Estimates of interactions effect (and their aliases):
l AB AB CD 1
4
550 749 1052 650 1075 642 601 729 10
l AC AC BD 25.5
l AD AD BC 197.5
• From this experiment, we may conclude that factor B
and C are not significant, thus may be removed from
further experiments
• Experiments that seek to identify a relatively few
significant factors from large number of factors are
called screening experiments
factorial design :: fractional :: 2k-1
• Resolution III (example: 23-1)
– Main effects are aliased with 2f interactions
– 2f interactions may be aliased with each other
• Resolution IV (example: 24-1)
– No main effects are aliased with other main effect or
2f interactions
– 2f interactions may be aliased with each other
• Resolution V (example: 25-1)
– No main effect or 2f interaction is aliased with any
other main effect or 2f interaction
– 2f interactions are aliased with 3f interactions
NOTE: 2f 2-factors interaction; 3f 3-factors interaction
factorial design :: fractional :: 2k-p
• Less runs are required with smaller fractions
experiments
• 2k design may be run in a (½)p fraction 2k-p
fractional factorial design
• Different choices of p for the same k factors yield
different resolutions
For example: k = 6 factors
- p = 1 32 runs, resolution VI 2VI6-1
- p = 2 16 runs, resolution IV 2IV6-2
- p = 3 8 runs, resolution III 2III6-3
factorial design :: fractional :: 2k-p
• Construction of 26-2 design:
– Factors: ABCDEF
– Number of runs = 24 = 16 create full factorial design of 4
factors A, B, C, D
– Define generators; ex. I = ABCE = BCDF, thus E=ABC and F=BCD
Other generator = ABCE * BCDF = AB2C2DEF = ADEF
– Find aliases; ex. Factor A:
• A*ABCE = A2BCE = BCE
• A*BCDF = ABCDF
• A*ADEF = A2DEF = DEF
Complete alias structure for A A = BCE = DEF = ABCDF
Complete alias structure for AB AB = CE = ACDF = BDEF
– Resolution = … …
factorial design :: fractional :: 2k-p
Factorial effect
Run I A B C D E=ABC F=BCD ?
1 + - - - - - -
2 + + - - - + -
3 + - + - - + +
4 + + + - - - +
5 + - - + - + +
6 + + - + - - +
7 + - + + - - -
8 + + + + - + -
9 + - - - + - +
10 + + - - + + +
11 + - + - + + -
12 + + + - + - -
13 + - - + + + -
14 + + - + + - -
15 + - + + + - +
16 + + + + + + +
factorial design :: fractional :: 2k-p
• Had we used generator I = ABDE = ABEF
– Factors: ABCDEF
– Number of runs = 24 = 16 create full factorial design of 4
factors A, B, C, D
– Define generators; ex. I = ABDE = ABEF, thus E=ABD and F=ABE
Other generator = ABDE * ABEF = DF
– Find aliases; ex. Factor D:
• D*ABDE = ABD2E = ABE
• D*ABEF = ABDEF
• D*DF = D2F = F
Complete alias structure for D D = ABE = ABDEF = F factor
D is aliased with factor F !
…
…
factorial design :: fractional :: 2k-p
• Exercise 12-14: Set up a 28-4 fractional factorial design. Verify that
this is a resolution IV design.
• Exercise 12-11: A 25-2 design is used to investigate the effect of 5
factors. The results obtained are as follows:
– e = 23.2 cd = 23.8
– ab = 15.5 ace = 23.4
– ad = 16.9 bde = 16.8
– bc = 16.2 abcde = 18.1
a. Verify that the design generators used were I = ACE and I = BDE
b. Write down the complete defining relation and the aliases from
this design
c. Estimate the main effects
d. Prepare ANOVA. Verify that AB and AD are available to use as
error
e. Plot the residuals versus the fitted values
factorial design :: residual analysis
• Knowing the significant factors/interactions, we
can construct regression model to predict the
output response under any factor-level
combinations
• Residual is the difference between actual
observation and the predicted value
• Normal probability plot of residuals checks
normality assumption of data
• Plot of residuals against predicted values shows
variability under different factor-level
combinations
factorial design :: residual analysis
• In exercise 12-11, factor B is known to have significant
effect (low B results in high yield)
• Assuming that it is the only significant effect:
5.175
yˆ 19.2375 B
2
• Predicted value when B is low ( = -1 ):
5.175
ˆy 19.2375 (1) 21.825
2
• Residuals when B is low: e( e ) 23.2 21.825 1.375
e( ad ) 16.9 21.825 4.925
e( cd ) 23.8 21.825 1.975
e( ace ) 23.4 21.825 1.575
factorial design :: residual analysis
• Residuals when B is high can be computed
similarly
• Plot of residuals versus fitted values:
Residuals vs fitted
4
2
Residual
0
-2 0 5 10 15 20 25
-4
-6
Fitted value
• What can be concluded from this graph? …
factorial design :: residual analysis
• Factor A is known to have no significant effect on the
output (average)
• Residual plot may be helpful in choosing the level of A to
operate
• Residuals when A is low: Residuals vs A-value
e( e ) 23.2 21.825 1.375 4
e(bde ) 16.8 16.65 0.15
Residual
0
e( cd ) 23.8 21.825 1.975 -2
-4
e(bc ) 16.2 16.65 0.45 -6
A value
• Residuals when A is high can be computed similarly
further topics
• Experiment with more than two levels 3k-p
• Second (and higher) order regression models
• Inclusion of uncontrollable (noise) factors into
experiment process robustness
• Finding the set of operating conditions that
result in the best process performance
process optimization
• Experiment for attribute data
• And more …
ujian akhir semester
• Hari senin, 18-05-2009, 12:30 – selesai
• Ruang: ……
• Open-book
• Siapkan:
– Alat tulis
– Kalkulator
– Tabel: normal standar, Poisson, F, dsb.
