See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/321975479

Using Design of Experiments in Method Development and Validation

Conference Paper · December 2017

CITATIONS READS

0 1,258

4 authors, including:

Ron Snee
Snee Associates, LLC
428 PUBLICATIONS   7,824 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Working to Bring Analytics to Construction Industry Safety Data View project

Quality by Design View project

All content following this page was uploaded by Ron Snee on 21 December 2017.

The user has requested enhancement of the downloaded file.

 Using Design of Experiments in
Method Development and Validation
Ronald D. Snee, PhD
Snee Associates, LLC
Newark, DE
IVT Laboratory Week
Philadelphia, PA
December 4-6, 2017
Snee Associates, LLC
1
 About the Speaker …..
He is also an Adjunct Professor in the
pharmaceutical programs at Temple and Rutgers
Universities. He worked at DuPont for 24 years prior
to starting his consulting career.
Ron received his BA from Washington and
Jefferson College and MS and PhD degrees from
Rutgers University. He is an academician in the
International Academy for Quality and Fellow of the
American Society of Quality, American Statistical
Ron Snee, PhD is Founder of Snee Associates,
Association, and American Association for the
LLC, a firm dedicated to the successful
Advancement of Science.
implementation of process and organizational
improvement initiatives. He provides guidance to Ron is an Honorary Member of ASQ and has been
pharmaceutical and biotech senior executives in awarded ASQ’s Shewhart, Grant and Distinguished
their pursuit of improved business performance Service Medals, and ASA’s Deming Lecture, Dixon
using Quality by Design, Lean Six Sigma and Consulting and Gerald J. Hahn Quality and
other improvement approaches that produce Productivity Achievement Awards.
bottom line results. He is a frequent speaker and has published 6 books
He has authored several articles on how to and more than 300 papers in the fields of quality,
successfully implement QbD, coauthored 3 books performance improvement, management, and
on QbD tools and speaks regularly at statistics. He recently received the Institute of
pharmaceutical and biotech conferences. Validation Technology’s Speaker of the Year Award.
Snee Associates, LLC
2
 Abstract
Pharmaceutical industry is experiencing a growing need to improve
performance driven by global competition and the increasing impact of
information technology. How to go about this improvement in a systematic,
focused and sustainable manner is the question. Fortunately a world-class
body of improvement technology exists known as Quality by Design (QbD); a
science and data-based approach that builds quality into products and
processes during development and validation as well as operations.
A characteristic of good science is good data. Quality data are arguably more
important today than ever before. Design of experiments, an important QbD
tool, is very effective in developing test methods that produce quality data. It
will be shown how DOE can be used to effectively design and improve
laboratory processes enabling prompt, successful method validation.
Methods discussed in this session include assessment of method
repeatability and reproducibility, improving method robustness and
measurement process control. Methods for assessing that amount of product
variation that can be attributed to the manufacturing process, sampling
procedures and test method are also presented. The concepts and methods
involved will be introduced and illustrated with pharmaceutical and biotech
case studies and examples.
Snee Associates, LLC
3
 Session Outline
I. A Look at Today’s Realities
- Importance and need to improve measurement quality
- Need to Reduce Risk in method validation
- The Promise of QbD – Building Quality into Test Methods
- Understanding measurement processes
II. Using Design of Experiments in Method Validation
- Test method development
- Measurement quality - Repeatability and reproducibility
- Measurement system capability
- Improving test method robustness
- Measurement process control – Continued process verification
III. How to Troubleshoot Measurement Test Methods
- Is the variation due to sampling method or test method?
- Using designed experiments (DOE) to quantify variation
- Using data to identify opportunities for improvement
- Graphical methods to visualize and understand variation

Snee Associates, LLC

4
 Lab Results
Tell Us How Our Processes Are Performing
 Test labs produce the data that are used to:
 Develop products and processes
 Control and improve processes
 Assess product quality
 Improving test methods improves the quality of the
data used to design, control and improve products and
processes:
 Risk of poor quality product getting to patients and
ineffective and inefficient process performance is
reduced in the process

Improved Test Methods Performance Reduces Risk

Snee Associates, LLC

5
 Improving Lab Performance
Reducing Risk in Test Method Measurements
 Developing test method design
 Improve the quality of measurements process:
 Gage R&R Study
 Improve the stability of the measurement process:
 Process control charting of control samples over time
 Reducing the sensitivity of the measurement process to
uncontrollable variables:
 Robustness studies using 2-level fractional-factorial
designs
 Improving flow through the lab
 Use lean operating principles for the flow of individual
methods as well as the overall lab flow

Quality by Design Addresses These Issues

Snee Associates, LLC

6
 Quality by Design
Generic Definition

“A Systematic Process for

Building Quality into a Product
from the Inception to Final Output”
“Product” of Analytical Laboratory Processes:
Test results that have the desired precision and
accuracy delivered in a cost-effective and timely
manner compliant with GLPs

Snee Associates, LLC

7
 Test Method QbD Building Blocks

Life Cycle Management and Continuous Improvement

Continued Process Verification
Snee Associates, LLC
8
 Use of QbD in Analytical Methods
 Method Design
 Meet method design performance criteria
 Method Evaluation
 Risk assessment using Failure Modes and Effects Analysis
(FMEA)
 Assess method repeatability and reproducibility (Gage R&R)
 Assess robustness of method design space (DOE)
 Assess sampling and testing variation
 Method Control and Continued Verification
 Control of method using control samples and statistical
process control techniques (SPC)
 Continued process verification using an integrated system
of method control and improvement techniques

Snee Associates, LLC

9
 Test Method and Its Variables
Controlled Variables
• Test Time and Temperature
• Mixing Speed and Time
• Reagent Concentrations
• Apparatus
Process Inputs Process Outputs
• Raw Materials Test • Quality Measurements
• Instruments • Test Results on Time
Method
• Personnel • Cost Effective Operations

Environmental Variables
• Ambient Temperature and Humidity
• Reagent Quality
• Analyst
• Day of Week
• Season of Year
Snee Associates, LLC
• Shift 10
 Test Method Design
Experimentation Strategy
Screening Optimization
Experiment Experiment

 Screening Experiment for 6 or more method input variables

 Select variables with largest effects
 Study selected variables in an optimization experiment
 Optimization experiment – Identify Method Design Space
 Design Space – Settings of the variables that will give
the best method performance

Snee Associates, LLC

11
 Example – Test Method Development
 Screening Experiment
 11 Design Variables evaluated – Each at 2 levels
 Experiment Design
 24 run Plackett-Burman design in 4 blocks
 6 runs in each of 4 weeks
 4 critical variables were identified
 Optimization Experiment
 4 variables from screening experiment were optimized
 Experiment Design
 28-Run subset of a 3x3x2x3 factorial design
 Runs identified by computer-aided selection procedure enabling
estimation of main effects and two-factor interactions

Snee Associates, LLC

12
 Example – Test Method Development
Results Summary
 Assay Method is robust to variation in the four principal
control parameters.
 Assay precision is 2- 4% depending on the number of samples
tested and the number of tests made on each sample.
 Reaction time and reaction temperature had the largest effects
 Increasing reaction temperature significantly decreases
method variation
 Method design space can be expressed as a function of the
four principal control parameters
 Next Step: Method raw material optimization in progress

Snee Associates, LLC

13
 Test Method Development
Experimental Strategy– Underlying Theory
 Study a large number of variables to ensure that no
important variables are missed
 Pareto principle applies
 Only a few variables (3-6) will have major effects
 For critical variables - linear (1st Order) effects are typically
much larger than interaction and curvature effects (2nd
Order)
 Studying a variable at high and low levels will identify
critical variables
 Studying the critical variables in optimization experiments
is an effect way to optimize a test method.

Snee Associates, LLC

14
 Reducing Test Method Risk
Improving Test Method
Repeatability and Reproducibility
Using
Gage R&R Studies

Snee Associates, LLC

15
 Sources of Variation

Total
Variation

Process or Measurement
Product System
Variation Variation

Accuracy Precision

Snee Associates, LLC

16
 Sources of Variation

Total
Variation
Gage R&R Studies
Process or Measurement Focus on
Product System Repeatability and
Variation Variation Reproducibility

Accuracy Precision

Repeatability Reproducibility

Snee Associates, LLC

17
Gage Repeatability and Reproducibility (R&R)
Gage R&R Study:
 Procedure used to determine how much of the observed
total variation is due to the measurement system
precision variation
 “Gage” is any device used to obtain measurements
Why is Gage R&R Important?
 Pharmaceutical R&D and Operations are data driven
 Before data are collected, it is crucial that the gage is
adequate
 Gage R&R should also be used whenever a new
measurement device is released to manufacturing

Snee Associates, LLC

18
 Repeatability and Reproducibility
Gage Repeatability Gage Reproducibility

Operator B

Operator C

Repeatability

Operator A

Reproducibility

Gage Repeatability is the Gage Reproducibility is the

variation in measurements variation in the average of
obtained when one measurements made by
operator uses the same different operators using
gage several times for the same gage when
measuring the identical measuring identical
characteristics of the same characteristics of the same
sample or part. sample or part. Snee Associates, LLC
19
 Gage R&R Study Design?
 Select N Samples that represent the full range of long-term
variation for your process
 Select K Analysts that customarily test the samples
 Each Analyst will measure each sample R Times
Analyst 1 Analyst 2 Analyst 3
Sample T1 T2 T1 T2 T1 T2
1
2
3 Example Design
4 • 10 Samples
• 3 Analysts
5
• 2 Tests by Each Analyst
6 on Each Sample
7 • 10x3x2=60 Test Results
8
9
10 Snee Associates, LLC
20
 HPLC Gage R&R Study
Sampling Design
- 6 Samples
- 4 Analysts per Sample – Total of 24 sub-samples
- 4 Tests per sub-sample – Total of 96 tests

Sample Sample 1 …… Sample 6

Analyst A1 A2 A3 A4 …………..
A21 A22 A23 A24

Tests T1 T5 T9 T13 …………..

T81 T85 T89 T93
T2 T6 T10 T14 …………..
T82 T86 T90 T94
T3 T7 T11 T15 T83 T87 T91 T95
T4 T8 T12 T16 T84 T88 T92 T96

Adapted from Ahuga and Dong (2005) Handbook of Pharmaceutical Analysis by HPLC

Snee Associates, LLC

21
 HPLC Gage R&R Study
Partial Data List - Samples 1 and 6
Sample Analyst Test Assay% Sample Analyst Test Assay%
1 1 1 99.3 6 1 1 98.1
1 1 1 99.4 6 1 1 98.4
1 1 1 99.9 6 1 1 97.2
1 1 1 99.4 6 1 1 97.1
1 2 2 99.5 6 2 2 96.4
1 2 2 99.2 6 2 2 97.9
1 2 2 98.6 6 2 2 94.4
1 2 2 101.6 6 2 2 97.3
1 3 3 97.8 6 3 3 95.4
1 3 3 97.4 6 3 3 96.1
1 3 3 98.1 6 3 3 96.9
1 3 3 99.1 6 3 3 96.4
1 4 4 97.9 6 4 4 95.9
1 4 4 98.3 6 4 4 95.2
1 4 4 99.1 6 4 4 97.2
1 4 4 98.9 6 4 4 96.8
Snee Associates, LLC
22
 HPLC Gage R&R Study
Two-Way ANOVA Table

Source DF SS MS F P
Sample 5 187.501 37.5001 39.9815 0.000
Analyst 3 15.623 5.2076 5.5522 0.009
Sample*Analyst 15 14.069 0.9379 1.4339 0.155
Repeatability 72 47.097 0.6541
Total 95 264.290

P< .05 indicates significant effects

• Significant differences between
• Samples (as expected)
• Analysts
• Analyst differences are the same for all Samples
• Sample-Analyst interaction is not significant
Snee Associates, LLC
23
 HPLC Gage R&R Study
Variance Components
Source VarComp % of Total
Total Gage R&R 0.90299 28.32
Repeatability 0.65413 20.52
Reproducibility 0.24885 7.81
Analyst 0.17790 5.58
Analyst*Sample 0.07095 2.23
Part-To-Part 2.28514 71.68
Total Variation 3.18812 100.00
• Total Gage R&R should be <30%
• Majority of Gage Variation is due to Test-to-Test

Number of Distinct Categories = 2

• Number of categories should be >4)

Snee Associates, LLC

24
 80
% Study Var 100.0

Percent
HPLC Gage
40 R&R Study 97.5

Repeatability – Test-to-Test Variation 95.0

0
Gage R&R Repeat Reprod Part-to-Part 1

R Chart by Analyst
1 2 3 4
4 UCL=3.802
Sample Range 100.0

2 _ 97.5
R=1.667
95.0

0 LCL=0

Xbar Chart by Analyst

1 2 3 4
100.0
Control Chart for Range 100.0

• Repeatability
Sample Mean

is stable across Analysts

UCL=97.782

Average
97.5 _
_ 97.5
X=96.568
LCL=95.354 95.0
95.0

Snee Associates, LLC

25
 Number of Distinct Categories
 Reflects measurement resolution
 Ability to detect differences between samples
 Number of distinct categories

>4 distinct data

 partto part  categories is
  1.41
  measurement 
desired

Snee Associates, LLC

26
Number of Distinct Categories
The number of distinct levels that can be measured by
the measuring device – also known as resolution
1 Distinct Category –
unacceptable for estimating
process parameters; only
indicates whether the process is
producing conforming or
nonconforming samples

2-4 Distinct Categories –

generally unacceptable for
estimating process parameter;
provides only coarse estimates

5 or more Distinct Categories –

acceptable for estimating
process parameters
Snee Associates, LLC
27
 HPLC Gage R&R Study
Measurement Variation vs Total Variation

Measurement
Variation

Measurement Variation Takes up 63% of the Total Variation

Method Not Useful for Distinguishing Between Different Samples

Snee Associates, LLC

28
 Thickness Measurement System
Number of Categories = 5 (Acceptable Resolution)

Measurement
Variation

Measurement Variation Takes up 23% of the Total Variation

Method Useful for Distinguishing Between Different Samples
Snee Associates, LLC
29
 Sample and Operator Differences
Study A
Operator 2
Test-to-Test
Variation Is
Higher than
Other Operators

Study B

Operators
Do Not Agree
on Length of
Sample 6

Snee Associates, LLC

30
 Improving Measurement Methods

 My measurement method is not adequate. The Gage R&R

statistics for repeatability and reproducibility are too large.
 Now what do I do?
 One possibility is to evaluate the measurement process /
procedure for ruggedness
 Effects of small variations in the how the method is used.
 There are other sources of variation in a measurement
method in addition to instruments and analysts which are
typically the subject of Gage R&R studies.

Snee Associates, LLC

31
 Measurement Method Ruggedness
 Measurement method is “rugged,” if it is immune to
modest (and inevitable) departures from the conditions
specified in the method (Youden 1961)
 Ruggedness/Robustness can be evaluated using two-
level fractional-factorial designs including Plackett-
Burman designs
Example – Dissolution Method Validation
 Objective: How robust (insensitive) is the dissolution
method to small changes in its execution.
 Procedure:
 Make a large batch of tablets to assure uniformity
 Vary 8 factors of the method using a 12-run Plackett-
Burman design
 Measure the dissolution time for each run Snee Associates, LLC
32
 Dissolution Method Validation
Ruggedness Test Variables

Variable (X) Low Level (-) High Level (+)

X1=Acid Conc (x1000) 9 11
X2=Polysorb Conc (x100) 4 6
X3=Stirring Speed (rpm) 45 55
X4=Temperature (deg C) 35 39
X5=Degasssing No Yes
X6=Filter Position Low High
X7=Operator A B
X8=Apparatus V W

Mixture of Quantitative and Qualitative Variables

Snee Associates, LLC

33
 Dissolution Method Validation
Ruggedness Test Design (Plackett- Burman)

Poly- De- Oper- Appa- Disso

Run Acid sorb Stir Temp gas Filter ator ratus D1 D2 D3 Time
1 11 6 45 39 Yes High A V -1 1 -1 20.5
2 9 6 55 35 Yes High B V -1 -1 1 16.5
3 11 4 55 39 No High B W -1 -1 -1 16.7
4 9 6 45 39 Yes Low B W 1 -1 -1 19.3
5 9 4 55 35 Yes High A W 1 1 -1 14.7
6 9 4 45 39 No High B V 1 1 1 18.1
7 11 4 45 35 Yes Low B W -1 1 1 15.5
8 11 6 45 35 No High A W 1 -1 1 17.9
9 11 6 55 35 No Low B V 1 1 -1 16.5
10 9 6 55 39 No Low A W -1 1 1 19.1
11 11 4 55 39 Yes Low A V 1 -1 1 16.7
12 9 4 45 35 No Low A V -1 -1 -1 20.5

Adapted from Lewis, Mathieu and Phan-Tan-Luu (1999)

Snee Associates, LLC

34
Dissolution Method Validation
Pareto Analysis of Method Variable Effects

Pareto Chart of the Effects

(response is Dissolution Time, Alpha = .05)
4.030
Stirring Speed
Temp
Polysorb Conc
Operator
Dummy 1
Term

Apparatus
Degassing
Acid Conc
Dummy 3
No Significant
Filter Position
Dummy 2
Effects
0 1 2 3 4
Effect
Lenth's PSE = 1.4

Snee Associates, LLC

35
 Dissolution Method Validation
Analysis of Test Method Variable Effects
Low High
Variable (X) Level Level Effect %
Acid Conc 9 11 -0.73 -4.1
Polysorb Conc 4 6 1.27 7.2
Stirring Speed 45 55 -1.93 -10.9
Temperature 35 39 1.47 8.3
Degasssing No Yes -0.93 -5.3
Filter Position Low High -0.53 -3.0
Operator A B -1.13 -6.4
Apparatus V W -0.93 -5.3
Dummy 1 -1 1 -0.93 -5.3
Dummy 2 -1 1 -0.53 -3.0
Dummy 3 -1 1 -0.73 -4.1

Effect = Change in dissolution time when variable moves low to high

% = Effect relative to average dissolution time=17.7 min.
Snee Associates, LLC
36
 Dissolution Method Validation
Conclusion
 Method is rugged (robust)
 None of the factor effects is significant
 Observed method dissolution time
 Standard deviation is 1.9 min
 2 standard deviation limits +/- 3.8 min
 Coefficient of variation 10.7%
 If variation is a concern consider tighter control on the
variables with the largest observed effects
Factor Low High Effect Pct
Polysorb Conc 4 6 1.27 7.2
Stirring Speed 45 55 -1.93 -10.9
Temperature 35 39 1.47 8.3

Snee Associates, LLC

37
Method Robustness – Underlying Theory
 Departures from test method standard operating
procedures happen frequently
 Human variation is a significant cause
 Small variations in operating parameters have no
practical effect on performance
 Interactions are small or non-existent
 Additive main effect models are adequate
 Plackett-Burman and two-level fractional factorial
designs work well
 Small run size provides adequate sensitivity

Snee Associates, LLC

38
 Process Variation Is Too High
Where is the Source of the Variation?

Process?
Sampling Method?
Analytical Lab?
Some or All of the Above?

Snee Associates, LLC

39
 Process Variation Issues

Excessive variation in Source of Process Variation

process output results Category
Manufacturing
Sampling

may be due to: Test Method

 Manufacturing Process
 Sampling Method
 Measurement Method
 A combination of these
sources of variation

Need a Procedure for Identifying Major Sources of Variation

Snee Associates, LLC

40
 Chemical Analysis Variation

Runs were made by 3 operators to locate a source of

variation in a chemical analysis. The procedure
consisted of:
 Taking a specimen,
 Heat treating the specimen
 Performing a chemical analysis.
In the test, 3 operators each took 2 specimens and
made 3 combustion runs on each specimen and
titrated each run in duplicate.

Snee Associates, LLC

41
Snee Associates, LLC
42
• Large Operator Differences
• Operator 2 Has More
Run-to-Run Variation than
Other Operators

Snee Associates, LLC

43
 Pigment Paste Production
Box, Hunter & Hunter (2005)

 Batch Process:
 A sample is taken form each batch and analyzed in the
lab to determine the moisture content of the product
 It is desired to understand the sources of
variation for the moisture content measurement
so that it can be determined the nature and
magnitude of the needed improvements
 Three sources of variation are identified:
 Batch-to-batch
 Sample-to-sample within a batch
 Test-to-test within a sample

Snee Associates, LLC

44
 Pigment Paste Production

 Sampling Study:
 15 Batches
 2 Samples per batch
 2 Analytical tests per sample
 Total of 30 samples and 60 tests

Batch B1 B2 ……….. B15

Sample S1 S2 S3 S4 ………….. S29 S30

Test T1 T3 T5 T7 ……….. T57 T59

T2 T4 T6 T8 ……….. T58 T60

Snee Associates, LLC

45
 Pigment Paste Production

Individual Value Plot of MOISTURE CONTENT

40

35
MOISTURE CONTENT

30

25

20

15

10
SAMPLE 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2
BATCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Snee Associates, LLC

46
 Pigment Paste Production
Analysis of Variance for MOISTURE

Source DF SS MS F P
BATCH 14 1210.9333 86.4952 1.492 0.226
SAMPLE 15 869.7500 57.9833 63.255 0.000
TEST 30 27.5000 0.9167
Total 59 2108.1833
Source of Process Variation
Test
Variance Components 3% Batch
19%
Source Var Comp. % of Total StDev
BATCH 7.128 19.49 2.670
SAMPLE 28.533 78.01 5.342
TEST 0.917 2.51 0.957 Sample
78%

Sampling Variation is Large and Statistically Significant

Snee Associates, LLC
47
 Pigment Paste Production
Interpretation of Results

 Largest variance component is due to Sampling

 Sampling procedure:
1. Randomly select 5 drums from a batch of 80
drums
2. Use a special sampling tube to sample
material from all levels of the drum
3. Thoroughly mix the 5 samples
4. Test a sample of the aggregate mixture

Snee Associates, LLC

48
 Pigment Paste Production
Interpretation of Results
 Investigation found:
 Procedures not followed and unknown
 Test samples were routinely taken from a single
drum
 When the proper sampling method was used the
“sample” variance component reduced dramatically
Component Before After
Batches 7.1 4.3
Samples 28.5 16.0
Tests 0.9 1.3
Snee Associates, LLC
49
 Pigment Paste Production

 Corrective Action:
 Operators were trained to use the proper sampling
methods
 Sampling process was more closely supervised
 Studies were initiated to further improve the
sampling procedure
 Alternative sampling schemes were investigated:
Guideline:
Sample those sources with the largest variance

Snee Associates, LLC

50
Raw Material Supplier Selection Study
( Adapted from Gonzalez-de la Parra and Rodriguez-Loaiza, Quality Engg 2003)

 Purpose
 Identify and quantify the sources of variability in a critical
raw material used to manufacture an API
 Nested Sampling Design was used
 Material from 2 suppliers was analyzed
 3 lots were evaluated from each supplier
 4 containers were sampled from each lot
 3 measurements (assay %) were made on each container
 Total of 2x3x4x3 = 72 assay results

Snee Associates, LLC

51
 Raw Material Supplier Selection Study
Analysis of Variance Results
Source DF SS MS F P
Supplier 1 1.8412 1.8412 1.494 0.289
Lot 4 4.9301 1.2325 1.570 0.225
Container 18 14.1317 0.7851 5.597 0.000
Assay 48 6.7330 0.1403
Significant
Total 71 27.6361 Container
Variance Components Variation
Source Var Comp. % of Total StDev
Supplier 0.017 4.13 0.130
Lot 0.037 9.11 0.193
Container 0.215 52.50 0.464
Assay 0.140 34.26 0.375
Total 0.409 0.640

Let’s Look at a Plot of the Data

Snee Associates, LLC
52
Raw Material Supplier Selection Study
I Chart of Result(%) by Supplier
1 2
1
102.5
Supplier 1 Supplier 2 UCL=102.231
102.0

101.5
Individual Value

101.0
_
100.5 X=100.523

100.0

99.5

99.0
LCL=98.814

1 8 15 22 29 36 43 50 57 64 71
Observation

Snee Associates, LLC

53
Raw Material Supplier Selection Study
Boxplot of Result(%)
102.5

102.0

101.5
Result(%)

101.0

100.5

100.0

99.5

99.0
1 2
Supplier

Snee Associates, LLC

54
 Raw Material Supplier Selection Study
Supplier 1 Supplier 2
Variance % of Std Variance % of Std
Source Component Total Dev Component Total Dev
Batch 0.030 50.0 0.174 0.044 6.1 0.211
Container 0.005 7.6 0.068 0.425 58.7 0.652
Assay 0.026 42.4 0.160 0.255 35.2 0.505
Total 0.061 100.0 0.246 0.725 100.0 0.851

• Supplier 2
• Has Greater Variation than Supplier 1
• Variation is due to sampling (59%) and testing (35%)
• Next Steps
• Assess sampling and test methods used by Supplier 2
• Consider evaluating what Supplier 1 is doing differently

Snee Associates, LLC

55
 Method Control

Reducing Analytical Risk

Maintaining Stable Analytical Methods
Using
Shewhart Control Charts

Enables Continued Measurement

Process Verification

Snee Associates, LLC

56
Use of Blind Controls (Reference Samples)
 Blind Control – Samples from a common source are
regularly submitted for analysis along with routine
production samples in a way that the analyst can not
determine the difference between the production samples
and the control samples
 “ The use of blind controls is rare in the pharmaceutical
industry. The roots of this probably lie in the compliance
aspects of the study. However there is no better way to
understand the true variability of the analytical method”

B. K Nunnally and J. S McConnell (2007)

Six Sigma in the Pharmaceutical Industry
Understanding, Reducing and Controlling Variation in Pharma and Biologics

Snee Associates, LLC

57
Measurement of Potency (%) on a QC
Control Material

Potency (%) on a QC
Control Material

84 Rows of Data
42 QC Runs
2 Measurements / Run

Snee Associates, LLC

VII-58
58
Measurement of Potency (%) on a QC
Control Material (Continued)
Variables Control Chart
XBar of Potency
96.0
95.5
Mean of Potency

95.0 UC L=95.101

94.5
94.0
Av g=94.178 Out-of-Control Runs
93.5 Detected. Process Is
LCL=93.254
93.0 Not Stable; Long-Term
92.5
Variation = 58%
12
15
18
21
24
27
30
33
36
39
42
3
6
9

Run
Note: T he s ig ma wa s cal c ul ated us in g th e ra nge .

R of Potency

UC L=1.604
1.5
Good Repeatability
Range of Potency

1.0
Within-Run Variation in
0.5 Av g=0.491 Control
0.0 LCL=0.000
12
15
18
21
24
27
30
33
36
39
42
3
6
9

Snee Associates, LLC

Run VII-59
59
Assessing Measurement Process Stability
 Q: When Should I worry about measurement stability?
 A: When Long-Term Variation represents more than 20% of
the total variation
 Total Variation = Long-Term Variation (Reproducibility)
+ Short-Term Variation (Repeatability)
 This guideline is based on the assumption that
 Well-controlled process will detect a shift of 1.5 short-
term standard deviations
Measurement Process Stability Long-Term Variation
Not a Problem < 20%
May be A Problem 20 – 30%
Corrective Action May Needed? > 30%
Ref: Snee and Hoerl, Quality Progress, May 2012, 39-41
Snee Associates, LLC
60
Assessing Measurement Process Stability
Example – Stable and Unstable Processes

• Stable Process
• Good Reproducibility
• Long-Term
Variance = 21%

• Un-Stable Process
• Poor Reproducibility
• Long-Term
Variance = 58%

Snee Associates, LLC

61
 Using QbD Techniques Improves
Test Method Performance
 Screening experiments followed by optimization studies
is an effective way to design effective test methods
 Measurement process can be controlled using Control
samples and Statistical Process Control techniques
 Measurement quality can be improved using Gage
Repeatability and Reproducibility studies
 Creating robust measurement systems using statistical
design of experiments
 Product variation: Be sure to separate sampling and
process variation from test method variation
Risk is Reduced when
Test Method Performance is
Stable, Capable, Efficient and Cost-Effective
Snee Associates, LLC
62
References – Improving Measurement Systems
Borman, P., M., etal (2007) “Application of Quality by Design to Analytical
Methods”, Pharmaceutical Technology, October 2007, 142-152.
Box, G. E. P. , J. S. Hunter and W. G. Hunter (2005), Statistics for
Experimenters, 2nd Edition, John Wiley and Sons, New York, NY, 345-353
Mason, R. L., Gunst, R. F. and Hess, J. L. (2003), Statistical Design and
Analysis of Experiments, 2nd Edition, John Wiley , New York, NY, Ch13
Montgomery, D. C. (2009), Design and Analysis of Experiments,7th Edition,
John Wiley and Sons, New York, NY, Chapter 13.
Schweitzer, M., etal (2010) “Implications and Opportunities of Applying QbD
Principles to Analytical Measurements”, Pharma Tech, Feb 2010, 52-59.
Snee, R. D. (1983), “Graphical Analysis of Process Variation Studies”, J.
Quality Technology, 15, 76-88.
Snee, R. D. (2005) “Are We Making Decisions in a Fog? The Measurement
Process Must Be Continually Measured, Monitored and Improved”, Quality
Progress, December 2005, 75-77
Snee, R. D. (2006) “If You’re Not Keeping Score, It’s Just Practice. If you’re Not
Measuring, You’re Not Competing”, Quality Progress, May 2006, 72-74.
Snee, R. D. (2014) “QbD in Test Method Development and Validation”,
Pharmaceutical Processing, May 2014, 28-29.
Snee, R. D. & R. W. Hoerl (2012) “Going on Feel: Monitor and Improve Process
Stability to Make Customers Happy”, Quality Progress, May 2012, 39-41
Snee Associates, LLC
63
 For Further Information,
Please Contact:
Ronald D. Snee, PhD
Snee Associates, LLC
Newark, DE
(610) 213-5595
[email protected]

Please visit our website at:

www.SneeAssociates.com

Snee Associates, LLC

View publication stats 64