DESIGN OF EXPERIMENTS FOR NON-LINEAR

MODELS
Part 2

Barbara Bogacka
Queen Mary, University of London
D - the most popular optimality criterion
The criterion, introduced by Wald (1943), is

ΦD = det(M −1 ).

Properties:
I it minimises the general variance of the parameter
estimator,
I it minimises the volume of the parameter confidence
ellipsoid,
I it is invariant under linear transformations of the
parameters,
I it is equivalent to G-optimality, what is given in so called
Equivalence Theorem,
I it has at most p(p + 1)/2 + 1 points of support
(Carathéodory’s Theorem)
D - the most popular optimality criterion
Geometrical Interpretation - volume of confidence ellipsoid

Confidence Intervals for θ1 and θ2

Estimator of θ = (θ1 θ 2)

Confidence ellipse for θ = (θ1 θ2)

100(1 − α)% confidence region of for parameter estimates is

b T M(θ − θ)
(θ − θ) b ≤ ps2 Fp,ν,α ,

where s2 is an estimate of σ 2 , and Fp,ν,α is 100α% point of the F

distribution on p and ν degrees of freedom.
1/2
The volume of a p-dim. ellipsoid is proportional to det M −1

.
D - the most popular optimality criterion
Geometrical Interpretation - design locus

Locally optimum designs for non-linear models with p

parameters usually have p support points. Then the weights are
all equal to 1/p.

Design locus is derived on the basis that the volume of a

simplex in Rp , formed by p points xi ∈ Rp and the origin, is
proportional to the determinant of the (p × p)-dimensional
matrix formed by these points.

So, to maximise det M, we find p points in the space of

derivatives, which together with the origin, form a simplex of
largest volume.
D - the most popular optimality criterion
Geometrical Interpretation - design locus: PK model, p = 2

f2(t,θo)

0.0

-0.5

-1.0

-1.5

-2.0
0.0 0.2 0.4 0.6 0.8 1.0 f1(t,θo)

Design Locus
D - the most popular optimality criterion
Geometrical Interpretation - design locus: PK model, p = 2

f2(t,θo)

0.0

-0.5

-1.0

-1.5

-2.0
0.0 0.2 0.4 0.6 0.8 1.0 f1(t,θo)

Design Locus, optimum points and the simplex

D - the most popular optimality criterion
Geometrical Interpretation - design locus: PK model, p = 2

f2(t,θo)

0.0

-0.5

-1.0

-1.5

-2.0
0.0 0.2 0.4 0.6 0.8 1.0 f1(t,θo)

Design Locus, optimum and non-optimum solution

D - the most popular optimality criterion
Geometrical Interpretation - parameter sensitivities

f1
1 f2

0
t1=1.23 t2=6.86 time

-1

-2

3 8 13 18

We find t1 and t2 such that det X = f1 (t1 )f2 (t2 ) − f2 (t1 )f1 (t2 ) is
maximum.
D - the most popular optimality criterion
The Equivalence Theorem

Kiefer and Wolfowitz (1960)

A design ξ ∗ is D-optimum if and only if it is G-optimum, that is
the following conditions are equivalent:

det(M −1 (ξ ∗ )) = min det(M −1 (ξ))

ξ

max d(x, ξ ∗ ) = min max d(x, ξ),

x ξ x

where d(x, ξ) = f T (x)M −1 (ξ)f (x) is the variance of prediction at

a point x.The third equivalent condition says

max d(x, ξ ∗ ) ≤ p,
x

where p is the number of parameters.

Equality is achieved at the support points of ξ ∗ .
D - the most popular optimality criterion
The Equivalence Theorem, an Illustration
Let the model response be

η(x, ϑ) = ϑ0 + ϑ1 x + ϑ2 x2 , on [−1, 1].

Then, the D-optimum design is

 
? −1 0 1
ξ = 1 1 1
3 3 3

The design does not depended on N, but instead on the

weights.

The information matrix can then be written as

   1   
1 1 1 3 0 0 1 −1 1
M(ξ ? , ϑo ) = X T WX =  −1 0 1 × 0 1
3 0  × 1 0 0 
1 0 1 1 1 1 1
0 0 3
D - the most popular optimality criterion
The Equivalence Theorem, an Illustration

Hence,  
3 0 2
1
M=  0 2 0 
3
2 0 2
and the variance function is

d(x, ξ ? ) = f T (x)M −1 f (x)

   
1 0 −1 1
= 3(1, x, x2 ) ×  0 0.5 0  ×  x 
−1 0 1.5 x2
= 3 − 4.5x2 + 4.5x4 .

Note, that d(x, ξ ? ) = 3 at x = −1, 0, 1

D - the most popular optimality criterion
The Equivalence Theorem, an Illustration

 
? −1 0 1
ξ = 1 1 1
3 3 3
D - the most popular optimality criterion
The Equivalence Theorem - PK model
 
? 1.23 6.86
ξ = 1 1
2 2

d
2.0

1.5

1.0

0.5

0.0
0 t1 5 t2 10 15 20 t
Example 4
Enzyme Kinetics Model, p = 2,

In a typical enzyme kinetics reaction enzymes bind substrates

and turn them into products. The binding step is reversible
while the catalytic step irreversible:

S + E ←→ ES → E + P,

S, E and P denote substrate, enzyme and product, respectively.

Example 4
Enzyme Kinetics Model, p = 2,

The reaction rate is represented by the Michaelis-Menten

model
Vmax [S]
v= ,
Km + [S]
where [S] is the concentration of the substrate and Vmax and Km
are the model parameters:
I Vmax denotes the maximum velocity of the enzyme and
I Km is Michaelis-Menten constant, it is the value of [S] at
which half of the maximum velocity Vmax is reached.
Example 4
Enzyme Kinetics Model, p = 2,

η([S],V max,K m)

Vmax

Vmax /2

Km [S]

Michaelis-Menten Model. The response function:

o = 1, Kmo = 1.


η [S]; Vmax , Km for the point priors Vmax
D optimality
Enzyme Kinetics Model, p = 2, Parameter Sensitivities

f1
f2
Parmeter
Sensitivities

[S] 1 [S] 2 [S]

f1 does not have a proper maximum; the largest value is at the

border of the design region.
D optimality
Enzyme Kinetics Model, p = 2, Design Locus

0.00

-0.05

-0.10
f2

-0.15

-0.20

-0.25

0.1 0.3 0.5 0.7 0.9

f1

Design Locus does not form a loop.

D optimality
Enzyme Kinetics Model, p = 2, Design Locus

0.00 (0,0)

-0.05

-0.10
f2

-0.15

-0.20

-0.25

0.1 0.3 0.5 0.7 0.9

f1

Design Locus: one vertex must be at the end of the locus.

D optimality
Enzyme Kinetics Model, p = 2, Design Locus

0.00 (0,0)

-0.05

-0.10
f2

-0.15

-0.20

-0.25

0.1 0.3 0.5 0.7 0.9

f1

Design Locus: the triangle of maximum area.

D optimality
Enzyme Kinetics Model, p = 2, Design Locus

0.00 (0,0)

-0.05
(f1([S] 2),f2([S] 2))

-0.10
f2

-0.15

-0.20
(f1([S] 1),f2([S] 1))

-0.25

0.1 0.3 0.5 0.7 0.9

f1

Design Locus: Optimum design points.

D optimality
Enzyme Kinetics Model, p = 2, The Equivalence Theorem

The variance function has only one proper maximum; it also

reaches p = 2 at the border of the design region.
D optimality
Enzyme Kinetics Model, p = 2, COURSE-WORK 1

Obtain a locally D-optimum design points for the

Michaelis-Menten model for the point prior values of the
o
parameters equal to Vmax = 1, Kmo = 1.
Example 5. Two Consecutive Chemical Reactions
Model.

Atkinson and Bogacka (2002), Chemometrics

k k
A →1 B →2 C.
The kinetic differential equations for [A], [B] and [C], the
concentrations of the chemical compounds A, B and C as
functions of time t are
d[A]
= −k1 [A]λ1
dt
d[B]
= k1 [A]λ1 − k2 [B]λ2
dt
d[C]
= k2 [B]λ2 . (1)
dt
Interest is in estimation of the orders λ1 , λ2 as well as of the
rates k1 , k2 .
Example 5. Two Consecutive Chemical Reactions
Model

The first of the three equations can be solved analytically to

give the concentration of chemical A at time t as

[A] = {1 − (1 − λ1 )k1 t}1/(1−λ1 ) (λ1 , k1 , t ≥ 0; λ1 6= 1),

if it is assumed that the initial concentration of A is 1.

This gives the following differential equation for the

concentration of the compound B
λ1
d[B]
= k1 {1 − (1 − λ1 )k1 t} 1−λ1 − k2 [B]λ2
dt
which has to be solved numerically.
Example 5. Two Consecutive Chemical Reactions
Model

0.6 General Consecutive Reaction

response
0.4
0.2
0.0

0 10 20 30 40 50

time

Concentration of B. Reading upward at t = 20:

(λo1 , λo2 ) = (1, 1), (2, 1), (1, 2) and (2, 2), (k1o , k2o ) = (0.7, 0.2).
Example 5. Two Consecutive Chemical Reactions
Model derivatives with respect to the parameters

General Consecutive Reaction

0.5
0.0
derivatives
-0.5
-1.0
-1.5

0 10 20 30 40 50

time

Derivatives (parameter sensitivities) as a function of time.

Reading upward at t = 10: f2 , f1 , f3 , f4 for k2 , k1 , λ1 and λ2 ,
respectively. Here (λo1 , λo2 ) = (1, 1), (k1o , k2o ) = (0.7, 0.2).
Example 5. Two Consecutive Chemical Reactions
D-optimum designs

These designs were found by searching over the four

continuous values of time, but with the weights held known at
0.25. The design region is T = [0,50].

Prior orders of reaction time

(k1o , k2o , λo1 , λo2 ) t1∗ t2∗ t3∗ t4∗
(0.7, 0.2, 1,1) 0.80 2.85 7.05 15.90
(0.7, 0.2, 2,1) 0.51 2.36 7.30 18.26
(0.7, 0.2, 1,2) 0.83 2.91 8.05 40.39
(0.7, 0.2, 2,2) 0.57 2.65 9.68 50.00

Table 1. D-optimum designs for both rate and order. The

weights are 0.25 at each design point.
Example 5. Two Consecutive Chemical Reactions
D-optimum designs

A -> B -> C: lambda = (1,1)

4
3
variance

2
1

• • • •
0

0 5 10 15 20 25

time

The variance of prediction d(t, ξ ∗ , ϑ) for prior

(k1o , k2o , λo1 , λo2 ) = (0.7, 0.2, 1, 1).
Example 5. Two Consecutive Chemical Reactions
D-optimum designs

A -> B -> C: lambda = (2,2)

4
3
variance

2
1

•• • •
0

0 10 20 30 40 50

time

Responses for various priors and the variance of prediction

d(t, ξ ∗ , ϑ) for prior (k1o , k2o , λo1 , λo2 ) = (0.7, 0.2, 2, 2).