CH 501- Analytical Calculations

1
 Method validation
• This is the process used to prove that an analytical method is acceptable for the
intended use

• Method validation parameters (May change slightly based on regulatory unit and
the study)

• Specificity
• Linearity
• Accuracy
• Precision
• Range
• Limit of detection
• Limit of quantitation
• Robustness
2
Specificity
• Specificity is the ability of an analytical method to distinguish analyte from everything
else that might be in the sample.

Electropherogram of the drug cefotaxime spiked with 0.2 wt% of

known impurities normally present from the synthesis
3
Assay development considerations

• What impurities need to be added to test for specificity.

• Analysis of a drug formulation: Pure drug should be compared with

• one containing additions of all possible

(a) synthetic by-products and intermediates
(b) degradation products
(c) excipients (substances added to give desirable form or consistency).

Degradation products may results due to:

Exposing pure material to heat, light, humidity, acid, base, and oxidant

4
Selectivity
Ability to assess an analyte in the presence of interferences that may be expected to be
present in the sample matrix.

Linearity
• This measures how well a calibration curve follows a straight line and showing that
the analytical response is directly proportional to the concentration of the analyte in
the sample

• A common method of testing the linearity is the square of the correlation coefficient
(R2) 𝟐
σ (𝒙𝒊 − ഥ
𝒙 )(𝒙𝒊 − ഥ
𝒙 )
Square of the correlation coefficient: 𝑹𝟐 =
σ(𝒙𝒊 − 𝒙ഥ)𝟐 σ(𝒚𝒊 − 𝒚 ഥ)𝟐

• A value of R2 above 0.995 or sometimes 0.999 can be considered as a good fit for
5
many purposes
Accuracy
Accuracy is “nearness to the truth.”

Ways to demonstrate accuracy include,

1. Analyze a certified reference material in a matrix similar to that of the unknown.

Newly developed method should provide the certified value for analyte in the
reference material, within the precision (random uncertainty) of your method.

2. Compare results from two or more different analytical methods. They should agree
within their expected precision.

3. Analyze a blank sample spiked with a known addition of analyte.

(a) The matrix must be the same as the unknown.
(b) When assaying a major component, three replicate samples at each of three
levels ranging from 0.5 to 1.5 times the expected sample concentration are
customary. 6
(c) For impurities, spikes could cover three levels spanning an expected range of
concentrations, such as 0.1 to 2 wt%.

4. If you cannot prepare a blank with the same matrix as the unknown, then it is
appropriate to make standard additions of analyte to the unknown. An accurate
assay will find the known amount of analyte that was added.

• Spiking is the most common method to evaluate accuracy because reference

materials are not usually available and a second analytical method may not be
readily available.

• Spiking ensures that the matrix remains nearly constant.

• An example of a specification for accuracy is that the analysis will recover 100 ±2%
of the spike of a major constituent. For an impurity, the specification might be that
recovery is within 0.1 wt% absolute or ± 10% relative.

7
Precision
• Precision is how well replicate measurements agree with one another, usually
expressed as a standard deviation.

• Many types of precision are distinguished.

▪ Instrument precision (injection precision) is the reproducibility observed when the

same quantity of one sample is repeatedly introduced ( ≥ 10 times) into an
instrument. Variability could arise from variation in the injected quantity and variation
of instrument response.

▪ Intra-assay precision is evaluated by analyzing aliquots of a homogeneous material

several times by one person on one day with the same equipment. Each analysis
is independent, so the intra-assay precision is telling us how reproducible the
analytical method can be.

8
 Intraassay variability is greater than instrument variability, because more steps are
involved.

Examples of specifications: instrument precision is 1% and intra-assay precision is

2%.

• Intermediate precision, formerly called ruggedness, is the variation observed when

an assay is performed by different people on different instruments on different
days in the same lab.

Each analysis might incorporate fresh reagents and different chromatography

columns.

• Inter-laboratory precision (reproducibility), is the most general measure of

reproducibility observed when aliquots of the same sample are analyzed by
different people in different laboratories.

Interlaboratory precision becomes poorer as the level of analyte decrease 9

Range
• Range is the concentration interval over which linearity, accuracy, and precision are
all acceptable

• An example of a specification for range for a major component of a mixture is the

concentration interval providing a correlation coefficient of R2 ≥ 0.995 (a measure of
linearity), spike recovery of 100 ± 2% (a measure of accuracy), and interlaboratory
precision of ± 3%.

• For an impurity, an acceptable range might provide a correlation coefficient of R2 ≥

0.98, spike recovery of 100 ± 10%, and inter-laboratory precision of ±15%.

10
Confusing terms:

Linear range: concentration range over

which calibration curve is linear

Dynamic range: concentration range over

which there is measurable response

Range: concentration range over which

linearity, accuracy, and precision meet
specifications for analytical method

11
Limits of Detection and Quantitation
Every analytical measurement is made up of two components

• Signal
• Noise

Signal
• Signal, carries information about the analyte that is of interest to the chemist

• Analytical signal is used for all the signals which are produced by analytical methods
and used (treated, evaluated and interpreted) in any form in analytical chemist

Noise
Noise, is made up of extraneous information that is unwanted because it degrades
the accuracy and precision of an analysis and also places a lower limit on the
amount of analyte that can be detected.
12
The signal to noise ratio (S/N)
The signal to noise ratio is a representative marker it that is used in describing the quality
of an analytical method or the performance of an instrument

There are many ways to find the S/N ratio

Estimation of noise

• Peak-peak noise:

Evaluate the maximum height (hmax) of the variation of the background noise (

• RMS noise (standard deviation)

Find the standard deviation of all small fluctuations: S/N

13
 0.92
Peak to peak noise
0.828

0.9 Signal-01
0.824

0.88 0.820
98 99 100 101 102
Signal

0.86
Signal-02 Signal-03
Signal-04
0.84

0.82

time/s
0.8
50 55 60 65 70 75 80 85 90 95 100

14
Limits of detection (LOD)
𝐿𝑂𝐷 = 3 × (S/N)

Limits of Quantitation (LOQ)

𝐿𝑂𝑄 = 10 × (S/N)

15
• The detection limit (also called the lower limit of detection) is the smallest quantity of
analyte that is “significantly different” from the blank.

• Here is a procedure that produces a detection limit with chance of being greater than
the blank.

• That is, only 1% of samples containing no analyte

will give a signal greater than the detection limit.

• We say that there is a rate of false positives.

• We assume that the standard deviation of the

signal from samples near the detection limit is
similar to the standard deviation from blanks.

16
Detection limit

Distribution of measurements for a blank

and a sample whose concentration is at
the detection limit.

The area of any region is proportional to

the number of measurements in that
region.

Only ~1% of measurements for a blank

are expected to exceed the detection limit.

However, 50% of measurements for a

sample containing analyte at the detection
limit will be below the detection limit.

17
• There is a 1% chance of concluding that a blank has analyte above the detection limit
(false positive).

• If a sample contains analyte at the detection limit, there is a 50% chance of

concluding that analyte is absent because its signal is below the detection limit (false
negative).

• Curves are Student’s t distributions for 6 degrees of freedom and are broader than
the corresponding Gaussian distributions.

The procedure explain in the next slide for the determination of detection limit

Calculate a value with a 99% confidence that the signal above the detection limit
produce by a sample that really contains an analyte

18
1. After estimating the detection limit from previous experience with the method,
prepare a sample whose concentration is ~1 to 5 times the detection limit.

2. Measure the signal from n replicate samples (n ≥ 7).

3. Compute the standard deviation (s) of the n measurements.

4. Measure the signal from n blanks (containing no analyte) and find the mean value,
yblank.

5. The minimum detectable signal, ydl, is defined as

𝑺𝒊𝒈𝒏𝒂𝒍 𝒅𝒆𝒕𝒆𝒄𝒕𝒊𝒐𝒏 𝒍𝒊𝒎𝒊𝒕: 𝒚𝐝𝐥 = 𝒚𝐛𝐥𝐚𝐧𝐤 + 𝟑𝑺

6. The corrected signal, ysample - yblank, is proportional to sample concentration:

Calibration line: 𝒚𝐬𝐚𝐦𝐩𝐥𝐞 − 𝒚𝐛𝐥𝐚𝐧𝐤 = 𝒎 × 𝒔𝒂𝒎𝒑𝒍𝒆 𝒄𝒐𝒏𝒄𝒆𝒏𝒕𝒓𝒂𝒕𝒊𝒐𝒏

19
where;

ysample is the signal observed for the sample m is the slope of the linear calibration
curve

The minimum detectable concentration, also called the detection limit, is obtained by
substituting ydl from the equation 𝒚𝐝𝐥 = 𝒚𝐛𝐥𝐚𝐧𝐤 + 𝟑𝒔 for ysample in the equation

𝒚𝐬𝐚𝐦𝐩𝐥𝐞 − 𝒚𝐛𝐥𝐚𝐧𝐤 = 𝒎 × 𝒔𝒂𝒎𝒑𝒍𝒆 𝒄𝒐𝒏𝒄𝒆𝒏𝒕𝒓𝒂𝒕𝒊𝒐𝒏

3s
Detection limit: Minimum detectable concentration =
m

20
Another common way to define detection limit is from the least-squares equation of
a calibration curve:

𝒔𝒊𝒈𝒏𝒂𝒍 𝒅𝒆𝒕𝒆𝒄𝒕𝒊𝒐𝒏 𝒍𝒊𝒎𝒊𝒕 = 𝒃 + 𝟑𝒔𝒚

where b is the y-intercept and sy is given by;

σ 𝒅𝒊𝟐
𝒔𝒚 =
𝒏−𝟐

The lower limit of detection in the equation, Minimum detectable concentration =

3s
is 3s/m,
m

where; s is the standard deviation of a low-concentration sample

m is the slope of the calibration curve

21
• The standard deviation is a measure of the noise (random variation) in a blank or a
small signal.

• When the signal is 3 times greater than the noise, it is detectable, but still too small
for accurate measurement.

• A signal that is 10 times greater than the noise is defined as the lower limit of
quantitation, or the smallest amount that can be measured with reasonable
accuracy.

10s
Lower limit of quantitation ≡
m

22
• The instrument detection limit is obtained by replicate measurements (n ≥ 7) of
aliquots from one sample.

• The method detection limit, which is greater than the instrument detection limit,
is obtained by preparing n ≥ 7 individual samples and analyzing each one once.

• The reporting limit is the concentration below which regulations say that a given
analyte is reported as “not detected,” which does not mean that analyte is not
observed.

• It means that analyte is below a prescribed level.

• Reporting limits are set at least 5 to 10 times higher than the detection limit, so
that detecting analyte at the reporting limit is not ambiguous.

23
example;
• Labels on U.S. packaged foods must state how much trans
fat is present.

• This type of fat is derived mainly from partial hydrogenation

of vegetable oil and is a major component of margarine
and shortening.

• The reporting limit for trans fat is 0.5 g per serving.

• If the concentration is 0.5 g/serving, it is reported as 0.

• By reducing the serving size, a manufacturer can state that

trans fat content is 0.

• If your favorite snack food is made with partially

hydrogenated oil, it contains trans fat even if the label says
24
otherwise.
EXAMPLE; Detection Limit

From previous measurements of a low concentration of analyte, the signal detection

limit was estimated to be in the low nanoampere range.

Signals from seven replicate samples with a concentration about three times the
detection limit were 5.0, 5.0, 5.2, 4.2, 4.6, 6.0, and 4.9 nA. Reagent blanks gave values
of 1.4, 2.2, 1.7, 0.9, 0.4, 1.5, and 0.7 nA. The slope of the calibration curve for higher
nA
concentrations is 𝑚 = 0.229 m
μM

(a) Find the signal detection limit and the minimum detectable concentration.

(b) What is the concentration of analyte in a sample that gave a signal of 7.0 nA?

25
(b) To find the concentration of a sample whose signal is 7.0 nA, use the equation
𝒚𝐬𝐚𝐦𝐩𝐥𝐞−𝒚𝐛𝐥𝐚𝐧𝐤=𝒎 × 𝒔𝒂𝒎𝒑𝒍𝒆 𝒄𝒐𝒏𝒄𝒆𝒏𝒕𝒓𝒂𝒕𝒊𝒐𝒏 :

𝑐𝑜𝑛𝑐𝑒𝑛𝑡𝑟𝑎𝑡𝑖𝑜𝑛= (𝑦sample−𝑦blank)/𝑚= (7.0 nA − 1.26 nA)/(0.229 nA/μM)=𝟐𝟓.𝟏

Robustness
➢ Robustness is the ability of an analytical method to be unaffected by small,
deliberate changes in operating parameters.

➢ For example, a chromatographic method is robust if it gives acceptable results when

small changes are made in solvent composition, pH, buffer concentration,
temperature, injection volume, and detector wavelength.

26
• In tests for robustness, the organic solvent content in the mobile phase could be
varied by, say, ±2%, the eluent pH varied by ±0.1, and column temperature varied
by ±5°C.

• If acceptable results are obtained, the written procedure should state that these
variations are tolerable.

• Capillary electrophoresis requires such small volumes that a given solution could
conceivably be used for months before it is used up.

• Therefore, solution stability (shelf life) should be evaluated for robustness.

27
Graphing techniques
• The purpose of the graph is to visually display relationships which may not be
apparent from data tables

• Experimental results are always associated with errors; best fit line averages out the
errors

• A graph paper with high quality/appropriate quality should be selected (rectangular

coordinate graph paper is recommended for a general work)

• Before plotting data points apply Q-test and reject data points if necessary

• A title should be given to the graph that includes all important information

• The scale for independent variable should be plotted along X-axis (abscissa)

• The scale for dependent variable should be plotted along Y-axis (ordinate)
28
• The scales should be chosen so that coordinates of any point on the plot should be
read quickly and easily (It may not be necessary to include zero on either side of the
axes)

• Graph should fills the entire sheet of the graph paper

• Main coordinate axes should be labelled with the values they present in ink. The name
of the quantity represented should be given along the axes together with the units

• Divisions on axes needs to be evenly spaced

• Each data point should be surrounded by a suitable symbol

29
Graphing of data using a conventional graph sheet

Best Graph
The best graph is the straight line which goes through the mean point (𝑥,ҧ 𝑦)
ത and the
maximum number of points keeping other points symmetrically on both sides
Worse Graph
The worse graph is the straight line which goes through the mean point (𝑥,ҧ 𝑦)
ത and the
most deviated point which is not an outlier
Error of the gradient
𝐸𝑟𝑟𝑜𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑔𝑟𝑎𝑑𝑖𝑒𝑛𝑡 = 𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑏𝑒𝑠𝑡 𝑔𝑟𝑎𝑝ℎ (𝑚𝑏𝑒𝑠𝑡 ) − 𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑤𝑜𝑟𝑠𝑒 𝑔𝑟𝑎𝑝ℎ 𝑚𝑤𝑜𝑟𝑠𝑒

𝐸𝑟𝑟𝑜𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 = 𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑏𝑒𝑠𝑡 𝑔𝑟𝑎𝑝ℎ (𝑐𝑏𝑒𝑠𝑡 ) − 𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡 𝑜𝑓 𝑡ℎ𝑒 𝑤𝑜𝑟𝑠𝑒 𝑔𝑟𝑎𝑝ℎ 𝑐𝑤𝑜𝑟𝑠𝑒

30
 Experimental data
Graph 01: The plot of ln Ct Vs t
ln Ct
-5.552

(84.00, -5.568)
-5.568
(46.00, -5.564)
Best graph

-5.584
(−5.556 − −5.568 )
𝑚𝑏𝑒𝑠𝑡 =
(328.00 − 84.00) -5.600
𝑥,ҧ 𝑦ത
(−5.556 − −5.568 ) Worse graph
= -5.616
(328.00 − 84.00)

= 4.918 × 10−4 𝑠 −1 -5.632

(−5.552 − −5.564 ) -5.648 (348.00, -5.552)

𝑚𝑤𝑜𝑟𝑠𝑒 =
(348.00 − 46.00)
(328.00, -5.556)
(−5.552 − −5.564 ) -5.664
= 20.00 60.00 100.00 140.00 180.00 220.00 260.00 300.00 340.00 380.00 420.00
(348.00 − 46.00)
time/ s
= 3.974 × 10−4 𝑠 −1
∆𝑚 = 4.918 − 3.974 × 10−4 𝑠 −1 = 0.944 × 10−4 𝑠 −1 31
Graphing techniques -The Method of Least Squares

• In quantitative chemical analysis, a calibration curve is often prepared

• Known standards are used to construct a calibration curve

• Response of an unknown solution can be interpreted based on the calibration curve

• Method of least squares is one of the statistical methods to draw the “best” line
through the experimental data points that are slightly scattered and do not lie
perfectly on a straight line

• The best line is the line where some of the points lie above and some lie below the
line

32
Finding the equation of the line
Assumptions:

• Uncertainties in y values are substantially greater than uncertainties in x values

(This assumption is often satisfied when the experimental response i.e y values is
less uncertain than the quantity of analyte i.e x values)

• Uncertainties (standard deviations) of all y values are similar

• Let us say the equation of the line is 𝑦 = 𝑚𝑥 + 𝑏 m − slope b − intercept

• The vertical deviation for the point 𝑥𝑖 , 𝑦𝑖 = 𝑦𝑖 − 𝑦, 𝑦 − ordinate is the staright line
when 𝑥 = 𝑥𝑖

Vertical deviation = 𝑑𝑖 = 𝑦𝑖 − 𝑦 = 𝑦𝑖 − (𝑚𝑥𝑖 + b) 33

• There are positive deviations as well as negative deviations

• As the goal is to minimize the deviations, all deviations are squared such that there
are only positive numbers to deal with

𝑑𝑖2 = 𝑦𝑖 − 𝑦 2
= (𝑦𝑖 −(𝑚𝑥𝑖 + b)) 2

• Since, the squares of deviations are minimized in this method, this method is known
as method of least squares

• Calculus can be used obtain values for m and b while minimizing sum of the squares
of the vertical deviations

𝑛 σ(𝑥𝑖 𝑦𝑖 ) − σ 𝑥𝑖 σ 𝑦𝑖 σ(𝑥𝑖2 ) σ 𝑦𝑖 − σ(𝑥𝑖 𝑦𝑖 ) σ 𝑥𝑖

𝑚= 𝑏= 2
𝑛σ 𝑥𝑖2 − (σ 𝑥𝑖 )
2 𝑛 σ 𝑥𝑖2 − (σ 𝑥𝑖 )

34
This can also be written using determinants
෍ 𝑥𝑖𝑦𝑖 ෍ 𝑥𝑖 ෍ 𝑥𝑖 2 ෍ 𝑥𝑖
𝑆𝑙𝑜𝑝𝑒; 𝑚 = ÷𝐷 𝑊ℎ𝑒𝑟𝑒 𝐷 =
෍ 𝑦𝑖 𝑛 ෍ 𝑥𝑖 𝑛
Least − squares
best line
෍ 𝑥𝑖 2 ෍ 𝑥𝑖𝑦𝑖
𝐼𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡; 𝑏= ÷𝐷
෍ 𝑥𝑖 ෍ 𝑦𝑖

Determinant
𝑒 𝑓 6 5
= 𝑒ℎ − 𝑔𝑓 So for an example = 6 × 3 − 4 × 5 = 18 − 20 = −2
𝑔 ℎ 4 3

35
Example:
Data set

𝒙𝒊 𝒚𝒊 𝒙𝒊 𝒚𝒊 𝒙𝟐𝒊 𝒅𝒊 (= 𝒚𝒊 − 𝒎𝒙𝒊 − 𝒃) 𝒅𝟐𝒊

1 2 2 1 0.03846 0.0014793
3 3 9 9 -0.19231 0.036982
4 4 16 16 0.19231 0.036982
6 5 30 36 -0.03846 0.0014793

෍ 𝑥𝑖 = 14 ෍ 𝑦𝑖 = 14 ෍(𝑥𝑖 𝑦𝑖 ) = 57 ෍ 𝑥𝑖2 = 62 ෍(𝑑𝑖2 ) = 0.076923

σ 𝑥𝑖𝑦𝑖 σ 𝑥𝑖 σ 𝑥𝑖 2 σ 𝑥𝑖 57 14 62 14 57×4 −(14×14) 32

𝑆𝑙𝑜𝑝𝑒; 𝑚 = ÷ = ÷ = = = 0.61538
σ 𝑦𝑖 𝑛 σ 𝑥𝑖 𝑛 14 4 14 4 62×4 −(14×14) 52

σ 𝑥𝑖 2 σ 𝑥𝑖𝑦𝑖 σ 𝑥𝑖 2 σ 𝑥𝑖 62 57 62 14 62×14 −(57×14) 70

𝐼𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡; 𝑏 = ÷ = ÷ = = = 1.34615
σ 𝑥𝑖 σ 𝑦𝑖 σ 𝑥𝑖 𝑛 14 14 14 4 62×4 −(14×14) 52

Therefore, the equation for the best line is 𝒚 = 𝟎. 𝟔𝟏𝟓𝟑𝟖 𝒙 + 𝟏. 𝟑𝟒𝟔𝟏𝟓

36
Reliability of least square method
• First estimate the standard deviation that describes the population of y values

• The standard deviation, σy , characterizes the Gaussian Curve

• We estimate σy , the population standard deviation of all y values, by calculating sy

, the standard deviation for the four measured values of y

• The deviation of each value of yi from the center of its Gaussian curve is,

𝒅𝒊 = 𝒚𝒊 − 𝒚 = 𝒚𝒊 − (𝒎𝒙𝒊 + 𝒃)

37
• The standard deviation of these vertical deviations is,

𝟐
σ 𝒅𝒊 −𝒅
𝝈𝒚 ≈ 𝒔𝒚 =
(𝒅𝒆𝒈𝒓𝒆𝒆𝒔 𝒐𝒇 𝒇𝒓𝒆𝒆𝒅𝒐𝒎)

• But the average deviation, 𝑑 , is 0 for the best straight line.

• Therefore, the numerator of the above equation reduces to 𝚺𝒅𝒊𝟐
• The degrees of freedom is the number of independent pieces of information
available
• we begin with n points and Two degrees of freedom are lost in determining the slope
and the intercept.

38
• Therefore, n-2 degrees of freedom remain and the equation becomes,

σ 𝒅𝒊 𝟐
Standard deviation of y : 𝒔𝒚 =
(𝒏−𝟐)

𝒙𝒊 𝒚𝒊 𝒙𝒊 𝒚𝒊 𝒙𝟐𝒊 𝒅𝒊 (= 𝒚𝒊 − 𝒎𝒙𝒊 − 𝒃) 𝒅𝟐𝒊

1 2 2 1 0.03846 0.0014793

3 3 9 9 -0.19231 0.036982

4 4 16 16 0.19231 0.036982

6 5 30 36 -0.03846 0.0014793

෍ 𝑥𝑖 = 14 ෍ 𝑦𝑖 = 14 ෍(𝑥𝑖 𝑦𝑖 ) = 57 ෍(𝑥𝑖 𝑦𝑖 ) = 62 ෍(𝑑𝑖2 ) = 0.076923

39
Uncertainty analysis for the equations, ෍ 𝑥𝑖𝑦𝑖 ෍ 𝑥𝑖
𝑆𝑙𝑜𝑝𝑒; 𝑚= ÷𝐷
෍ 𝑦𝑖 𝑛
Least − squares
best line
෍ 𝑥𝑖 2 ෍ 𝑥𝑖𝑦𝑖
𝐼𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡; 𝑏= ÷𝐷
෍ 𝑥𝑖 ෍ 𝑦𝑖

Leads to following results, 𝟐𝒏

𝒔𝒚
𝒔𝒕𝒂𝒏𝒅𝒂𝒓𝒅 𝒖𝒏𝒄𝒆𝒓𝒕𝒂𝒊𝒏𝒍𝒚 𝒖𝒎 𝟐 =
𝒐𝒇 𝒔𝒍𝒐𝒑𝒆 𝑫
𝟐 σ 𝟐
𝒂𝒏𝒅 𝒊𝒏𝒕𝒆𝒓𝒄𝒆𝒑𝒕 𝒔𝒚 𝒙 𝒊
𝒖𝒃 𝟐 =
𝑫

um is the standard uncertainty of the slope, ub is the standard uncertainty of the intercept, sy is the
standard deviation of y and D is given by, ෍ 𝑥𝑖 2 ෍ 𝑥𝑖
𝐷=
෍ 𝑥𝑖 𝑛 40
• Standard uncertainty (um and ub ) is the standard deviation of the mean

• If you double the number of calibrations points, um and ub decrease by ~ 1ൗ 2

• The standard deviation sy is a characteristic of the population of measurements and is

independent of the number of calibration points

• If you double the number of points , sy is nearly constant

• At last, we can assign significant figures to the slope and the intercept

41
Consider σ 𝑑𝑖 2 = 0.076923

σ 𝒅𝒊 𝟐
Putting the numbers in to the equation s𝒚 = gives,
(𝒏−𝟐)

0.076 923
𝑠𝑦 2 = = 0.038462
4 −2

𝒔𝒚𝟐 𝒏 𝒔𝒚𝟐 σ 𝒙𝒊𝟐

Plugging numbers in to 𝒖𝒎𝟐 = and 𝒖𝒃𝟐 =
𝑫 𝑫

𝑠𝑦 2 𝑛 0.038462 4
𝑢𝑚 2 = = = 0.002 958 6 ⇒ 𝑢𝑚 = 0.054 39
𝐷 52

𝑠𝑦 2 σ 𝑥𝑖2 0.038 462 62

𝑢𝑏 2 = = = 0.045 859 ⟹ 𝑢𝑏 = 0.21415
𝐷 52
42
 Combining the results for m, um, b, and ub, we write;

0.615 38
slop: = 0.62 ± 0.05 or 0.615 ± 0.054
±0.054 39

The first digit of the uncertainty is the last significant figure. We often retain
extra, insignificant digits to prevent round-off errors in further calculations

1.346 15
Intercept: = 1.3 ± 0.2 or 1.35 ± 0.21
±0.214 15

• Where the uncertainties are um and ub

• The first decimal place of the uncertainty is the last significant figure of the slope or
intercept
• Many scientists write results such as 1.35±0.21 to avoid excessive round-off 43
 𝑡𝑠
To express uncertainty as a confidence interval, equation 𝑥ҧ ± tells us to multiply the
𝑛

standard uncertainty in the following equations by Student’s t for n-2 degrees of

freedom.
0.615 38
𝐬𝐥𝐨𝐩: = 0.62 ± 0.05 or 0.615 ± 0.054
±0.054 39

1.346 15
𝐈𝐧𝐭𝐞𝐫𝐜𝐞𝐩𝐭: = 1.3 ± 0.2 or 1.35 ± 0.21
±0.214 15

44
Example; Finding sy, sm, and sb with a Spreadsheet
• The Excel function LINEST returns the slope and intercept and their uncertainties in
a table (a matrix).
• As an example, enter x and y values in columns A and B.
• Then highlight the 3-row × 2-column region E3:F5 with your mouse.
• This block of cells is selected for the output of the LINEST.
• On the Formulas ribbon, go to Insert Function.
• In the window that appears, in “Or select a category” select Statistical and double
click on LINEST.
• The new window asks for four inputs to the function.
• For y values, enter B2:B5.

45
➢ Then enter A2:A5 for x values.
➢ The next two entries are both “TRUE”
➢ The first TRUE tells Excel that we want to compute the y-intercept of the
least-squares line and not force the intercept to be 0.
➢ The second TRUE tells Excel to print out the uncertainties as well as the
slope and intercept.
➢ The formula you have just entered is
“=LINEST(B2:B5,A2:A5,TRUE,TRUE)”.
➢ Now press CONTROL+SHIFT+ENTER on a PC or
CONTROL+SHIFT+RETURN on a Mac.

46
• Excel prints out a matrix in cells E3:F5.
• Write labels around the block to indicate what is in each cell.
• The slope and intercept are on the top line.
• The second line contains um and ub.
• Cell F5 contains sy and cell E5 contains a quantity called R2, which is
a measure of the goodness of fit of the data to the line.
• The closer R2 is to unity, the better the fit.

47
 Calibration Curves
Spectrophotometer data used to construct calibration curve

Amount of Absorbance of independent

Range Corrected absorbance
protein (μg) standards

0.0 0.099 0.099 0.100 0.001 -0.0003 -0.0003 0.0007

5.0 0.185 0.187 0.188 0.003 0.0857 0.0877 0.0887
10.0 0.282 0.272 0.272 0.010 0.1827 0.1727 0.1727
15.0 0.345 0.347 0.392 0.047 0.2457 0.2477 _

20.0 0.425 0.425 0.430 0.005 0.3257 0.3257 0.3307

25.0 0.483 0.488 0.496 0.013 0.3837 0.3887 0.3967
48
 Constructing a Calibration Curve
General procedure for constructing a calibration curve:

1. Prepare known samples of analyte covering the range (0 - 150%) of concentrations

expected for unknowns. Measure the response of the analytical procedure to these
standards to generate data like the left half of the given table.

2. Subtract the average absorbance of the blank solutions from each measured
absorbance to obtain corrected absorbance. The blank measures the response of
the procedure when no analyte is present.

49
4. Make a graph of corrected absorbance versus quantity of analyte analyzed. Inspect
the graph to determine over what range the data are linear, whether there are any
outliers, and that the uncertainty is approximately the same over the range.

5. Use the least-squares procedure to find the best straight line through the linear
portion of the data. Find the slope and intercept and their standard uncertainties with
Equations

6. If you analyze an unknown at a future time, run a blank at that time, Subtract the new
blank absorbance from the unknown absorbance to obtain corrected absorbance.

50
 EXAMPLE
Using a Linear Calibration Curve
1. An unknown protein sample gave an absorbance of 0.406 and a blank had an
absorbance of 0.104. How many micrograms of protein are in the unknown?

2. What mass of protein gives a corrected absorbance of 0.250?

51
• We prefer calibration procedures with a linear response, in which the corrected
analytical signal (= signal from sample - signal from blank) is proportional to the
quantity of analyte.

• Although we try to work in the linear range, you can obtain valid results beyond the
linear region The dashed curve comes from a least-squares fit of the data to the
equation
• y = ax2 + bx + c

• The linear range of an analytical method is the analyte concentration range over
which response is proportional to concentration.
52
A related quantity in the following figure is dynamic range - the concentration range
over which there is a measurable response to analyte, even if the response is not
linear.

53
 Propagation of Uncertainty with a Calibration Curve

In the preceding example, an unknown with a corrected absorbance of y = 0.302 had

a protein content of x = 18.24 µg. What is the uncertainty in the number 18.24?
Propagation of uncertainty for the equation y = mx + b (but not y = m) gives the
following result:

Standard uncertainty in x = standard deviation of the mean =

𝑠𝑦 1 1
µx = + +((y − 𝑦)
lj 2)/(m2 Σ(xi − 𝑥)
lj 2)
𝑚 𝑘 𝑛

54
where s, is the standard deviation of y

|m| is the absolute value of the slope,

k is the number of replicate measurements of the unknown,

n is the number of data points for the calibration line

𝑦lj is the mean value of y for the points on the calibration line,

xi are the individual values of x for the points on the calibration line, and

𝑥lj is the mean value of x for the points on the calibration line

55
• For a single measurement of the unknown (k = 1), Equation gives ux = ±0.39 µg.
Four replicate unknowns (k = 4) with an average corrected absorbance of 0.302
reduce the uncertainty to ux = ±0.23 µg

• The confidence interval for x is ±tux , where t is Student's t for n-2 degrees of
freedom. If ux = ±0.23 µg and n= 14 points (12 degrees of freedom), the 95%
1
confidence interval for x is ±tux = +(2.179)(0.23) = ± 0.50, µg. There is no in the
𝑛

expression for confidence interval because ux is the standard deviation of the

mean.

56