M347

Mathematical statistics

Univariate continuous distribution

theory
Contents
1 Pdfs and cdfs 3
1.1 Densities and normalising constants 3
1.1.1 Examples 4
1.2 Distribution functions and densities 6
1.2.1 Examples 8
1.3 Probabilities of lying within intervals 10

2 Moments 11
2.1 Expectation 11
2.2 The mean 13
2.3 Raw moments 14
2.4 Central moments 15
2.5 The variance 16
2.6 Linear transformation 18
2.6.1 Expectation and variance 18
2.6.2 Effects on the pdf 19
2.7 The moment generating function 21
2.7.1 Examples 22

Solutions 24
This publication forms part of an Open University module. Details of this and other
Open University modules can be obtained from the Student Registration and Enquiry Service, The
Open University, PO Box 197, Milton Keynes MK7 6BJ, United Kingdom (tel. +44 (0)845 300 6090;
email [email protected]).
Alternatively, you may visit the Open University website at www.open.ac.uk where you can learn
more about the wide range of modules and packs offered at all levels by The Open University.
To purchase a selection of Open University materials visit www.ouw.co.uk, or contact Open
University Worldwide, Walton Hall, Milton Keynes MK7 6AA, United Kingdom for a brochure
(tel. +44 (0)1908 858793; fax +44 (0)1908 858787; email [email protected]).

Note to reader
Mathematical/statistical content at the Open University is usually provided to students in
printed books, with PDFs of the same online. This format ensures that mathematical notation
is presented accurately and clearly. The PDF of this extract thus shows the content exactly as
it would be seen by an Open University student. Please note that the PDF may contain
references to other parts of the module and/or to software or audio-visual components of the
module. Regrettably mathematical and statistical content in PDF files is unlikely to be
accessible using a screenreader, and some OpenLearn units may have PDF files that are not
searchable. You may need additional help to read these documents.

The Open University, Walton Hall, Milton Keynes, MK7 6AA.

First published 2015.
Copyright c 2015 The Open University
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, transmitted
or utilised in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without
written permission from the publisher or a licence from the Copyright Licensing Agency Ltd. Details of such
licences (for reprographic reproduction) may be obtained from the Copyright Licensing Agency Ltd, Saffron
House, 6–10 Kirby Street, London EC1N 8TS (website www.cla.co.uk).
Open University materials may also be made available in electronic formats for use by students of the
University. All rights, including copyright and related rights and database rights, in electronic materials and
their contents are owned by or licensed to The Open University, or otherwise used by The Open University as
permitted by applicable law.
In using electronic materials and their contents you agree that your use will be solely for the purposes of
following an Open University course of study or otherwise as licensed by The Open University or its assigns.
Except as permitted above you undertake not to copy, store in any medium (including electronic storage or
use in a website), distribute, transmit or retransmit, broadcast, modify or show in public such electronic
materials in whole or in part without the prior written consent of The Open University or in accordance with
the Copyright, Designs and Patents Act 1988.
Edited, designed and typeset by The Open University, using the Open University TEX System.
Printed in the United Kingdom by Martins the Printers, Berwick upon Tweed.
ISBN 978 1 4730 0341 5
1.1
 1 Pdfs and cdfs

1 Pdfs and cdfs

The distributions of continuous random variables are described by their
‘probability density functions (pdfs)’ and ‘cumulative distribution
functions (cdfs)’. Pdfs are the topic of Subsection 1.1, where their basic
properties are described and examples are developed. Cdfs, including their
properties, some examples and their relationship to pdfs, are considered in
Subsection 1.2. Finally, you will be reminded how differences between cdf
values give probabilities of a random variable lying within an interval in
Subsection 1.3.

1.1 Densities and normalising constants

A continuous random variable X follows a distribution with probability
density function, or pdf (or sometimes just density function or This pdf has nothing to do with
density), f say. the ‘portable document format’
PDF files on your computer.

The key properties of a pdf f are that it is non-negative and

integrates to 1, that is,
Z ∞
f (x) ≥ 0 for all x ∈ R and f (x) dx = 1.
−∞

f (x)
area = 1

0 x

Figure 2.1 A pdf f (x) plotted as a function of x

Note that in Figure 2.1, f (x) ≥ 0 for all x. Also, its integral, which is the
area of the shaded region under the pdf, is 1.
In fact, any mathematical function which is non-negative, positive on at
least one interval of values of x, and has a finite integral can be made into
a pdf. (A function whose integral does not exist cannot be made into a
pdf.) Suppose that g is such a function with
Z ∞
g(x) dx = C,
−∞
where 0 < C < ∞. Then
g(x)
f (x) =
C
is a pdf. To see this, note that
Z ∞ Z ∞
g(x) 1 ∞ 1
Z
f (x) dx = dx = g(x) dx = × C = 1.
−∞ −∞ C C −∞ C
C is called the normalising (or normalisation) constant. Some statisticians use the term
‘normalising constant’ for the
The following nomenclature is not entirely standard in statistics, but can quantity 1/C instead of C.
be useful. The part g of the pdf f that contains all the dependence of f on
x will be referred to as the density core, or just core for short, in M347.

3
 Univariate continuous distribution theory

One thing to bear in mind is that a pdf is not a probability itself. In

particular, f (x) 6= P (X = x). Indeed, for a continuous distribution,
P (X = x) equals zero! However, there is a closely related probability. The
probability that X lies within a specific interval [x0 , x0 + ε), where ε > 0 is
small, is approximately equal to ε times the density at x0 (see Figure 2.2).
Formally,
P (x0 ≤ X < x0 + ε)
→ f (x0 ) as ε → 0.
ε
f (x) f (x0 )

0
x
x0 x0 +

Figure 2.2 The pdf f (x) of Figure 2.1 with the area under the pdf and
between x0 and x0 + ε shaded

1.1.1 Examples

Example 2.1 The pdf of the uniform distribution on (0, 1)

The distribution with density f which is constant on 0 < x < 1 and zero
otherwise is called the uniform distribution on (0, 1), denoted U (0, 1).
The constant value, k say, must be positive for f to be a density. What is
the value of k? Well, the density is 0 for x ≤ 0, k for 0 < x < 1 and 0 again
for x ≥ 1. So, splitting the range of integration into three parts gives
Z ∞ Z 0 Z 1 Z ∞
f (x) dx = f (x) dx + f (x) dx + f (x) dx
−∞ −∞ 0 1
Z 0 Z 1 Z ∞
= 0 dx + k dx + 0 dx
−∞ 0 1
Z 1
=0+ k dx + 0
0
Z 1
= k dx = [kx]10 = k(1 − 0) = k.
0 Integration of a constant
However, this integral should be 1, so it must be that k = 1. That is,

1 if 0 < x < 1,
f (x) =
0 otherwise.
For short, it can be said that
f (x) = 1 on 0 < x < 1.

4
 1 Pdfs and cdfs

The uniform pdf is shown in Figure 2.3.

f (x)
1

0
−1 0 1 2 x

Figure 2.3 Graph of the pdf of the uniform distribution on (0, 1)

Notice that you could have replaced the integration step in calculating k
by using the area of the square in Figure 2.3.

When a pdf only takes positive values on a subset of R, then the limits of
integration reduce to the limits of that subset. In Example 2.1, these limits
were 0 and 1. In general, the domain on which f takes positive values is
called the support of the distribution. Thus, in Example 2.1, the support
of f is (0, 1) and the density is written in the form
f (x) = {function of x} on x in its support.
And, in general, the integral of f is the integral of f over its support. In
M347, the support of f will always be a single interval, although one or
both endpoints of that interval may be ±∞.

Example 2.2 The pdf of the exponential distribution

The exponential distribution, M (λ), with parameter λ > 0, is claimed

to have density
f (x) = λe−λx on x > 0.
(Thus, implicitly, f (x) = 0 for x ≤ 0.) Is this truly a density? Well the
exponential function is non-negative everywhere, so f certainly is. Does it
integrate to 1? Yes, it does: noting that the support of f is (0, ∞),
Z ∞ Z ∞
f (x) dx = λe−λx dx
−∞ 0 Integration of an exponential
Z ∞
=λ e−λx dx
0
  ∞
1 −λx
h i∞
=λ − e = − e−λx = 0 − (−1) = 1.
λ 0 0

5
 Univariate continuous distribution theory

The exponential pdf is shown in Figure 2.4 for λ = 1.

f (x)
1

0
−1 0 1 2 3 4 x

Figure 2.4 Graph of the pdf of the exponential distribution with λ = 1

Exercise 2.1

For each of the following four functions, decide whether or not they are, or
can be made into, densities. If they are non-negative and integrable,
calculate what their correct normalising constants should be.
(a) fa (x) = x(1 − x) on 0 < x < 1.
(b) fb (x) = ex (1 − x) on x > 0.

1 + x on −1 < x < 0,
(c) fc (x) =
1 − x on 0 ≤ x < 1.
(d) fd (x) = 1/x on x > 1.

1.2 Distribution functions and densities

The cumulative distribution function, distribution function or

cdf, F say, of the random variable X is defined as
F (x) = P (X ≤ x).

That is, F (x) is the probability that the random variable X takes any
value less than or equal to the fixed value x.

6
 1 Pdfs and cdfs

Exercise 2.2

Explain why, in the continuous case, F (x) is also equal to P (X < x).

In this unit, the continuous case only is studied.

The key properties of F are:

• 0 ≤ F (x) ≤ 1 (because F (x) is a probability);
• for any c < d in the support, F (c) < F (d) (that is, the probability
of being less than or equal to a particular value is less than the
probability of being less than or equal to a larger value);
• limx→−∞ F (x) = 0 and limx→∞ F (x) = 1 (that is, X is certain to
lie somewhere between −∞ and ∞).

The cdf and pdf are intimately related as shown below.

x
d
Z
F (x) = f (y) dy; f (x) = F (x) = F ′ (x).
−∞ dx

It is valid to think of this relationship as defining the pdf f .

The audio below accompanies Animation 2.1, and you should listen to this
audio while using the animation.
Audio 2.1

Interactive content appears here. Please visit the website to use it.

Animation 2.1 Linked pair of plots of f and F

Interactive content appears here. Please visit the website to use it.
Rx
It is important to notice that in the relationship F (x) = −∞ f (y) dy, x is
the upper limit of integration, and something else – here, y – is the variable
with respect to which you integrate. Any symbol will do for the variable of
integration in place of y, except for x as this has to be the upper
R x limit of
integration. It is both wrong and confusing to write F (x) = −∞ f (x) dx.

Exercise 2.3

Use the relationships between a pdf and cdf to check mathematically that:
(a) limx→∞ F (x) = 1;
(b) F is an increasing function of x on its support.

7
 Univariate continuous distribution theory

1.2.1 Examples

Example 2.3 The cdf of the uniform distribution on (0, 1)

From Example 2.1,

f (x) = 1 on 0 < x < 1.
For any value of x ≤ 0,
Z x Z x
F (x) = f (y) dy = 0 dy = 0.
−∞ −∞
For any value of 0 < x < 1,
Z x Z 0 Z x
F (x) = f (y) dy = 0 dy + 1 dy
−∞ −∞ 0
Z x
=0+ 1 dy = [y]x0 = x − 0 = x.
0
For any value of x ≥ 1,
Z x Z 0 Z 1 Z x
F (x) = f (y) dy = 0 dy + 1 dy + 0 dy
−∞ −∞ 0 1
Z 1
=0+ 1 dy + 0 = [y]10 = 1 − 0 = 1.
0
That is,

0 if x ≤ 0,
F (x) = x if 0 < x < 1,
1 if x ≥ 1.


You can check this calculation by differentiating F to get f (x) = F ′ (x) = 1

if 0 < x < 1, f (x) = 0 otherwise.
f (x)
1

0
−1 0 1 2 x
(a)

F (x)
1

0
−1 0 1 2 x
(b)

Figure 2.5 (a) Graph of the pdf of the uniform distribution on (0, 1);
(b) graph of the cdf of the uniform distribution on (0, 1)

8
 1 Pdfs and cdfs

It is always the case that if the support of f , (a, b) say, is not the whole
of R but an interval subset thereof, then F (x) = 0 for x ≤ a and F (x) = 1
for x ≥ b. This was illustrated for the uniform distribution with a = 0 and
b = 1 in Figure 2.5. In such cases, for short, just write
F (x) = {function of x} on a < x < b.
For example, for the uniform distribution of Example 2.3, write
F (x) = x on 0 < x < 1.
When doing so, do not lose sight of the fact that F is zero ‘below’ its
support and F is one ‘above’ its support. If a = −∞, then there is no
interval of values for which F (x) = 0, and if b = ∞, then there is no
interval of values with F (x) = 1.

Exercise 2.4

From Example 2.2, the pdf of the exponential distribution with parameter
λ > 0 is
f (x) = λe−λx on x > 0.
This exercise concerns the cdf of the exponential distribution.
(a) What is the value of F (x) for x ≤ 0?
(b) Show that the cdf of the exponential distribution is
F (x) = 1 − e−λx on x > 0.
(c) Check this result by differentiation to obtain the corresponding pdf.

A distribution is introduced now that may be new to you but will be used
as an example sufficiently often in M347 to warrant a name: the power
distribution. It has density function
f (x) = βxβ−1 on 0 < x < 1.
The parameter β can take any positive value.
Animation 2.2 Graph of the pdf of a power distribution, with a
slider to change the value of β > 0
Interactive content appears here. Please visit the website to use it.

Exercise 2.5

Using Animation 2.2, describe the main qualitative behaviour of the power
density for values of β < 1, β = 1 and β > 1, respectively.

9
 Univariate continuous distribution theory

Exercise 2.6

For each of the following two distributions, obtain their cdfs.

(a) The power distribution.
(b) The continuous uniform distribution on (a, b), which has density
1
f (x) = on a < x < b.
b−a

Notice that the uniform distribution on (0, 1) is a special case of both of

the distributions in Exercise 2.6. It is the special case of the uniform
distribution on (a, b) obtained by setting a = 0, b = 1; and it is also the
special case of the power distribution corresponding to setting β = 1 (as
x0 = 1 for any x).

1.3 Probabilities of lying within intervals

Let c < d. It is the case that

Z d
P (c ≤ X ≤ d) = F (d) − F (c) = f (x) dx.
c

Also,
P (c ≤ X ≤ d) = P (c ≤ X < d) = P (c < X ≤ d) = P (c < X < d)
since X is continuous so that P (X = c) = P (X = d) = 0. The formula in
the box above is most easily seen with the aid of an animated figure:
Animation 2.3 Consecutive showing of images corresponding to
F (d), then F (c), then F (d) − F (c)
Interactive content appears here. Please visit the website to use it.

Exercise 2.7
(a) For the distribution with cdf
F (x) = x2 (3 − 2x) on 0 < x < 1,
calculate P ( 14 ≤X≤ 5
8 ).
(b) For the exponential distribution with λ = 1 which has cdf
F (x) = 1 − e−x on x > 0,
show that
P (log 2 ≤ X ≤ log 4) = 14 .

10
 2 Moments

Exercise 2.8
(a) Write P (x0 ≤ X < x0 + ε), ε > 0, as a function of F .
(b) Hence explain why
P (x0 ≤ X < x0 + ε)
lim = f (x0 ).
ε→0 ε

2 Moments
The mean and variance are special cases of a concept known as the
‘moments’ of a distribution. These in turn are special cases of the more
general concept of ‘expectation’, or ‘expected value’, which is described in
Subsection 2.1. The familiar notion of the mean is then considered in
Subsection 2.2. The mean is also known as the first moment of a
distribution, which leads, in Subsections 2.3 and 2.4, to two general
definitions of moments. Following these, another familiar notion, the
variance, is studied in Subsection 2.5. Subsection 2.6 concerns random
variables linked by ‘linear transformation’ and, in particular, the means,
variances and densities thereof. Finally, in Subsection 2.7, you will see how
to deal with many moments all in one go through the medium of
something called the ‘moment generating function’.

‘Doing the best at this moment puts you in the best place for the
next moment.’
(Oprah Winfrey, American television personality, actress and producer)

2.1 Expectation
Informally, as you might guess, the expected value, or expectation, of a
random variable X is some kind of average or mean or typical value of X.
The formal definition of expected value is given here in general terms for
the expected value of a general function h of X. This expected value is
denoted by E{h(X)} and is defined by another integral.

Let (a, b) denote the support of pdf f , remembering that a can be

−∞ and b can be ∞. Then
Z b
E{h(X)} = h(x) f (x) dx. (2.1) The specific type of brackets
a used is not part of the notation
for expectation, which is just
the E.
There is an implicit assumption here that the integral is finite, but this
need not necessarily be so; if the integral is not finite, then the expected
value of h(X) is said not to exist.
Here is a first, particularly simple but useful, special case of the above.
Suppose that h does not actually depend on X at all, that is, h(X) = k
where k is a constant. Then
Z b Z b
E(k) = k f (x) dx = k f (x) dx = k × 1 = k.
a a

11
 Univariate continuous distribution theory

(This particular expectation always exists because f is a pdf.) This is very

reasonable: the expected (or ‘average’) value of a constant is none other
than the constant value itself (what else could it be?).
Some other simple choices of h are especially important in statistics, and
these will be the principal focus in the following subsections. Some of them
will be familiar to you already, for example, the mean and the variance.
However, before looking at them, it will be useful to obtain a general result
and some important consequences thereof.

Let h1 and h2 be two functions of X, and let k1 and k2 be two

constants. Then the linearity of expectation property is that
E{k1 h1 (X) + k2 h2 (X)} = k1 E{h1 (X)} + k2 E{h2 (X)}. (2.2)

Exercise 2.9

Since k1 h1 (X) + k2 h2 (X) is itself just a function of X, its expected value

is
Z b
E{k1 h1 (X) + k2 h2 (X)} = {k1 h1 (x) + k2 h2 (x)} f (x) dx,
a
where (a, b) is the support of f . Show that Equation (2.2) holds.

Exercise 2.10

Write down a formula for E{k1 h(X) + k2 } in terms of k1 , k2 and E{h(X)}.

The result of Exercise 2.10 is a particularly important consequence of the

linearity of expectation:

E{k1 h(X) + k2 } = k1 E{h(X)} + k2 . (2.3)

Exercise 2.11

Write down a formula for E{k h(X)} in terms of k and E{h(X)}.

In fact, linearity of expectation holds for a linear combination of functions

of X consisting of any finite number of terms (and not just two terms as
above). The result is as follows.

( n ) n
X X
E ki hi (X) = ki E{hi (X)}. (2.4)
i=1 i=1

12
 2 Moments

This result follows from the linearity of integration in the same way that
Equation (2.2) does, but no further details of the proof are given.

2.2 The mean

Perhaps the most important measure of the location of a distribution is
the mean, or arithmetic mean, of X. This is nothing other than the
expected value, or expectation, of the random variable X itself,
i.e. E{h(X)} where h(X) = X. The usual notation for the mean is µ.

If X is defined on (a, b), then

Z b
µ = E(X) = x f (x) dx. The mean does not exist if its
a defining integral does not.

Notice that this subsection concerns the population mean of a

continuous distribution rather than the sample mean briefly reviewed in
Subsection 1.2 of Unit 1 or the population mean of a discrete distribution
briefly reviewed in Subsection 1.4 of Unit 1.

Example 2.4 The mean of the uniform distribution

The uniform distribution on (a, b) was introduced in Exercise 2.6(b). The

mean of the uniform distribution on (a, b) is
Z b  
1
E(X) = x dx
a b−a
Z b
1
= x dx
b−a a
 2 b
1 x
=
b−a 2 a
1
(b + a)(b − a)
= b2 − a2 = = 12 (a + b).
2(b − a) 2(b − a)
The mean of the uniform distribution on (0, 1) is therefore 12 (0 + 1) = 12 .

The next example requires integration by parts as well as by substitution.

13
 Univariate continuous distribution theory

Example 2.5 The mean of the exponential distribution

Using the pdf of the exponential distribution given in Example 2.2,

Z ∞
E(X) = xλe−λx dx
0 Integration by substitution
1 ∞ −u
Z
= ue du Integration by parts
λ 0
(using the substitution u = λx, du = λ dx)
  Z ∞ 
1  ∞
= −ue−u 0 − − e−u du
λ 0
(integrating by parts using f (u) = u, g ′ (u) = e−u
so that f ′ (u) = 1, g(u) = −e−u )
1  ∞ 1 1 1
= 0 − 0 − e−u 0 = {−(0 − 1)} = × 1 = .
λ λ λ λ

Exercise 2.12

Calculate the mean of the distribution with density

f (x) = 6x(1 − x) on 0 < x < 1.

2.3 Raw moments

The mean is also known as the first moment of a distribution. The second
and third moments are defined to be E(X 2 ) and E(X 3 ), respectively, and
so on.

Generally, the rth moment is defined to be E(X r ), r = 1, 2, . . .,

where Since X 0 = 1,
Z b E(X 0 ) = E(1) = 1, although
r
E(X ) = xr f (x) dx. one rarely refers to the zeroth
a moment.

As with the expected value, these moments are subject to the existence of
their defining integrals.
To distinguish them from other versions of moments, one of which will be
introduced in the next section, these moments are sometimes called the
raw moments. Broadly speaking, the lower the value of r, the more
important the corresponding moment is in statistics!
For some distributions, it is as easy to determine the general rth moment
as it is the mean, so all the (raw) moments might as well be calculated at
once.

14
 2 Moments

Example 2.6 Raw moments of the uniform distribution on (0, 1)

The rth moment of the uniform distribution on (0, 1) is

Z 1  r+1 1 Uniform distribution on (0, 1)
r r x 1 1
E(X ) = x dx = = (1 − 0) = .
0 r+1 0 r+1 r+1
(Remember, f (x) = 1 on 0 < x < 1.) In particular, setting r = 1 gives
E(X) = 12 , as previously shown in Example 2.4.

Exercise 2.13

Calculate the rth moment of the power distribution, which has density
f (x) = βxβ−1 on (0, 1). Can you deduce the formula for the rth moment of
the uniform distribution on (0, 1) (Example 2.6) from your result? Power distribution

2.4 Central moments

For r = 2, 3, . . ., it is sometimes useful to consider the central

moments, or moments about the mean, defined by
µr = E{(X − µ)r }.

It will make life easier below to also define µ0 = E{(X − µ)0 } and
µ1 = E{(X − µ)1 }, but these both take particular constant values.

Exercise 2.14

What are the values of µ0 and µ1 ?

The rth central moment can be written in terms of the mean µ and the
raw moments up to and including the rth, and the rth raw moment can be
written in terms of the mean and the central moments up to the rth. The
latter formula will be derived next. The derivation starts with a handy
little trick: write X = X − µ + µ so that
X r = (X − µ + µ)r = {(X − µ) + µ}r .
Now the binomial expansion
r  
r
X r i r−i
(a + b) = ab
i
i=0
can be used. Set a = X − µ, b = µ to get
r  
r
X r
X = (X − µ)i µr−i
i
i=0

15
 Univariate continuous distribution theory

so that, by the linearity of expectation and the fact that µ is a constant,

( r   ) r  
X r X r
r i r−i
E(X ) = E (X − µ) µ = E{(X − µ)i } µr−i
i i
i=0 i=0
or
r   Linearity of expectation
r
X r r−i
E(X ) = µi µ .
i
i=0

Thus the rth raw moment E(X r ) has been written in terms of the mean µ,
µ0 = 1, µ1 = 0 and the central moments µ2 , µ3 , . . . , µr .

Exercise 2.15

Show that the rth central moment µr can be written in terms of the
mean µ and the raw moments up to the rth as follows:
r  
X r
µr = (−1)r−i E(X i ) µr−i .
i
i=0

2.5 The variance

In the same way that the mean holds a special place in statistics as a
leading measure of the location of a distribution, so the variance – and its
very close relation the standard deviation – holds an equivalent place as
the leading measure(s) of the scale or spread of a distribution. You may
well have recognised the variance as being precisely the same as the second
central moment, µ2 , in Subsection 2.4, although this notation is rarely used
for the variance. Instead, the variance is denoted either by σ2 or by V (X).

The variance is given by

σ2 = V (X) = µ2 = E{(X − µ)2 }.
The standard deviation, σ, is the square root of the variance,
p
σ = V (X).

Again, this is the population, rather than the sample, variance.

16
 2 Moments

Example 2.7 Variance and standard deviation of the uniform

distribution on (0, 1)

In Example 2.4, it was shown that for the uniform distribution on (0, 1),
µ = 12 . This can be used in calculating the variance of the uniform
distribution:
V (X) = E{(X − µ)2 } = E{(X − 12 )2 }
Z 1
= (x − 12 )2 dx
0
Z 1
= (x2 − x + 14 ) dx
0
1
x3 x2 x

= − +
3 2 4 0

= ( 13 − 1
2 + 1
4) −0
1
= 12 .
(Remember, f (x) = 1 on 0 < x < 1.)
The standard deviation is therefore
q
1
p
σ = V (X) = 12 = 0.29 (correct to two decimal places).

If, having worked out the variance directly in Example 2.7, you were
concerned that such calculations could quickly start to become tricky, you
will be pleased to find that there is an alternative, easier, route to the
same answer. It depends on the following simple link between the second
central moment, the variance, and the first and second raw moments, µ
and E(X 2 ).

V (X) = E(X 2 ) − {E(X)}2 = E(X 2 ) − µ2 . (2.5)

This result is, of course, obtainable from the relationship derived in

Exercise 2.15, although it is instructive to verify its truth from scratch:
V (X) = E{(X − µ)2 }
= E(X 2 − 2µX + µ2 )
= E(X 2 ) − 2µE(X) + µ2 (by linearity of expectation) Linearity of expectation

= E(X 2 ) − 2µ2 + µ2
= E(X 2 ) − µ2 .

17
 Univariate continuous distribution theory

Example 2.8 Variance of the uniform distribution revisited

1
For the uniform distribution on (0, 1), µ = 2 and, from Example 2.6,
1 1
E(X 2 ) = = .
2+1 3
Using Equation (2.5), it is therefore the case that
 2
2 2 1 1 1 1 1
V (X) = E(X ) − µ = − = − = .
3 2 3 4 12
Note that this is the same result as that calculated earlier in Example 2.7
by directly evaluating the second central moment.

Exercise 2.16

In Exercise 2.13 you showed that for the power distribution,

Power distribution
β
E(X r ) = .
r+β
Calculate the variance of the power distribution.

The variance and standard deviation are positive quantities. This is

because the variance is an expectation, or average, of positive quantities,
2
Rnamely the squared values h(X) = (X − µ) . Equivalently, the integral
h(x) f (x) dx involves positive quantities only and hence must be positive
(the integral is also the area under a positive function). So if your Basic ideas of integration
calculations result in a negative variance you know you must have gone
wrong!

2.6 Linear transformation

Let X be a random variable and let Y be another random variable defined
through the linear transformation Y = aX + b. This subsection is split
into two parts: the first concerns the mean and variance of Y , the second
concerns the density of Y .

2.6.1 Expectation and variance

Let X be a random variable and let L be a new random variable defined by
L = X + b, b ∈ R.
Then Equation (2.3) tells us immediately that
E(L) = E(X + b) = E(X) + b = µ + b.
Also,
V (L) = E{(L − E(L))2 }
= E{(X + b − (µ + b))2 }
= E{(X − µ)2 } = V (X).

18
 2 Moments

On the other hand, let S be another random variable defined in terms of X

by
S = aX, a ∈ R.
Then, again by Equation (2.3),
E(S) = E(aX) = a E(X) = aµ
and
V (S) = E{(S − E(S))2 }
= E{(aX − aµ)2 }
= E{a2 (X − µ)2 } = a2 E{(X − µ)2 } = a2 V (X).
If the standard deviation of S is denoted by σS and that of X by σX , this
means that
p p
σS = V (S) = a2 V (X) = |a| σX .
Note the absolute value signs on a: this is because σS must be positive,
and σX is positive but there is no sign restriction on a.

Exercise 2.17

Write Y = aX + b.
(a) What is E(Y ) in terms of a, b, E(X) and V (X)?
(b) What is V (Y ) in terms of a, b, E(X) and V (X)?

The results of Exercise 2.17 are summarised in the following box.

If Y = aX + b, then
E(Y ) = a E(X) + b and V (Y ) = a2 V (X). (2.6)

2.6.2 Effects on the pdf

Subsection 2.6.1 specifically concerned the effect of the linear
transformation Y = aX + b on the mean and the variance of Y as
compared with those of X. More generally, it can be said that this linear
transformation effects a location and scale change: the quantity b
moves X along to a new location, to the right if b > 0, to the left if b < 0;
the quantity |a| changes the scale of X by expanding it if |a| > 1 or by
contracting it if |a| < 1. This is illustrated in the following animation.
For the remainder of this subsection, restrict attention to cases where
a > 0.
Animation 2.4 Exploring the effects of changing the location b and
the scale a

Interactive content appears here. Please visit the website to use it.

Suppose now that X has a continuous distribution with pdf fX and

cdf FX , and let Y = aX + b once more, with a > 0.

19
 Univariate continuous distribution theory

The distribution of Y can be figured out in terms of the distribution of X.

The cdf, FY , of Y is given in terms of FX by
FY (y) = P (Y ≤ y) = P (aX + b ≤ y)
   
y−b y−b
=P X≤ = FX .
a a
Note that the manipulation of the inequality is correct because a > 0.
Differentiating the first and last terms of the above equation gives the
density, fY , of Y in terms of fX :
d
fY (y) = FY (y)
dy
 
d y−b
= FX
dy a Chain rule
 
d y−b d
= × FX (x)
dy a dx (y−b)/a
 
1 y−b
= fX
a a
(by the chain rule of differentiation). This relationship between the
densities of linearly transformed (Y ) and original (X) random variables is
an important one.

If Y = aX + b, a > 0, then
 
1 y−b
fY (y) = fX . (2.7)
a a
If fX has support (c, d), then fY has support (ac + b, ad + b).

Exercise 2.18

Suppose now that X follows the standard normal distribution and define
Y = σX + µ. Standard normal distribution
(a) What are E(Y ) and V (Y )?
(b) Let the pdf of the standard normal distribution be designated φ(x).
What, in terms of µ, σ and φ, is the pdf of Y ?
(c) Use the result of part (b) and the formula for φ(x) to show that the
density of Y is that of the general normal distribution with mean µ Normal distribution
and variance σ2 (a fact with which you should already be familiar).

The density of Y = aX + b in Equation (2.7) is said to be written in

location–scale form. The quantity b is then said to be a location
parameter of the distribution of Y , and a is said to be its scale
parameter. Often, the density of X will be in standardised form: this
means that E(X) = 0 and V (X) = 1. In such cases E(Y ) = b and σY = a,
so the location and scale parameters coincide with the mean and standard
deviation of the distribution.
These ideas were illustrated in Exercise 2.18 for the normal distribution
where the density of X – the standard normal distribution – is indeed the
standardised form of the normal distribution with mean 0 and standard

20
 2 Moments

deviation 1. As in Subsection 1.5.2 of Unit 1, Y = σX + µ has the general

normal distribution with mean µ and variance σ2 . But the point here is
that the location–scale notion is quite general and by no means confined to
the normal distribution.

2.7 The moment generating function

A single formula for the moments for all r is one way of providing a whole
host of moment information at once. Another way, and one that proves to
be especially useful in statistics (and in parts of the rest of M347), is to
summarise all the moment information by calculating the moment
generating function, usually abbreviated to mgf.

If E(etX ) exists for all t in some interval (−δ, δ), δ > 0, then the For fixed t, etX is just a
function particular choice of function
h(X) for which expectation can
MX (t) = E(etX ), t ∈ (−δ, δ), be considered as in
Subsection 2.1.
is called the moment generating function of X.

Why the name? Well, here is a mathematically somewhat informal

explanation. First, the Maclaurin expansion of etX – see Subsection 2.4 of
Unit 1 – is employed:
MX (t) = E(etX )
(tX)2 (tX)3 (tX)r
 
= E 1 + tX + + + ··· + + ···
2! 3! r!
 2 3 r 
2t 3t rt
= E 1 + Xt + X +X + ··· + X + ··· .
2! 3! r!
Now invoke the linearity of expectation result (2.4), noting that it was
claimed to be true only for finite sums. Here there is an infinite series and
so such a result cannot be applied trivially. However, it turns out that
under fairly general conditions, which you need not worry about in M347,
it is valid to write the moment generating function in the following form.

t2 t3
MX (t) = 1 + E(X) t + E(X 2 ) + E(X 3 ) + · · ·
2! 3!
tr
+ E(X r ) + ···. (2.8)
r!

The reason for the name is now apparent: the function MX (t) generates all
the (raw) moments of a distribution by attaching them as coefficients of
tr /r! in a power series expansion of itself.
Moreover, there is a neat way to recover the individual raw moments from
the mgf. First, differentiate (2.8) with respect to t to get
t2
′
MX (t) = E(X) + E(X 2 ) t + E(X 3 ) + ··· Each prime, ′ , denotes
2! differentiating once with respect
and again to get to t.
′′
MX (t) = E(X 2 ) + E(X 3 ) t + · · · .
Then set t = 0:
′
MX (0) = E(X), ′′
MX (0) = E(X 2 ).

21
 Univariate continuous distribution theory

In general, the rth derivative of the mgf at t = 0 is the rth raw

moment, i.e.
(r) (r)
E(X r ) = MX (0) = MX (t) |t=0 ,
(r)
where MX (t) means differentiation r times with respect to t.

2.7.1 Examples
The mgf is particularly straightforward to obtain for the standard normal
and exponential distributions.

Example 2.9 The mgf of the standard normal distribution

The standard normal distribution, N (0, 1), has density

1
f (x) = √ exp − 12 x2


on R.
2π
Using Equation (2.1), the mgf is therefore
Z ∞
tX
MX (t) = E(e ) = exp(tx) f (x) dx
−∞
∞
1
Z
exp(tx) √ exp − 12 x2 dx


=
−∞ 2π
Z ∞
1
exp − 12 x2 + tx dx


=√
2π −∞
Z ∞
1
exp − 12 (x2 − 2tx) dx.

=√
2π −∞
Completing the square and then integrating by substitution gives
Z ∞
1
exp − 12 (x − t)2 + 12 t2 dx

MX (t) = √
2π −∞ Completing the square
Z ∞
1 2 √1 Integration by substitution
exp − 12 (x − t)2 dx

= exp( 2 t )
2π −∞
Z ∞
1 2 √1
exp − 12 z 2 dz


= exp( 2 t )
2π −∞
(using the substitution z = x − t, dz = dx)
Z ∞
1 2
= exp( 2 t ) f (x) dx
−∞

= exp( 12 t2 ) × 1 = exp( 21 t2 ).
Now, the derivative of MX (t) = exp( 12 t2 ) with respect to t is
′
MX (t) = t exp( 21 t2 ),
so µ = E(X) = MX ′ (0) = 0, confirming the value of the mean of the

standard normal distribution.

Also,
Product rule
d ′
′′
MX (t) = MX (t) = exp( 12 t2 ) + t2 exp( 12 t2 ) = (1 + t2 ) exp( 12 t2 ),
dt

22
 2 Moments

so E(X 2 ) = MX
′′ (0) = 1. In addition, it follows that the variance of the

standard normal distribution is E(X 2 ) − µ2 = 1 − 02 = 1.

Exercise 2.19

If X follows the standard normal distribution, then E(X 4 ) = 3. Use the

mgf of the standard normal distribution to prove this result.

Exercise 2.20
(a) Calculate the mgf of the exponential distribution. (You may assume
that t < λ, which turns out to be the condition necessary for the Exponential distribution
exponential mgf to exist.)
(b) Hence verify that the mean and variance of the exponential
distribution are 1/λ and 1/λ2 , respectively.
(r−1)
(c) The formula for the (r − 1)th derivative of MX (t), denoted MX (t),
is
(r−1) (r − 1)!λ
MX (t) = , t < λ.
(λ − t)r
(r)
Hence obtain the formula for MX (t), t < λ.
(d) Using the result of part (c), verify that, for the exponential
distribution,
r!
E(X r ) = .
λr

23
 Solutions

Solutions
Solution 2.1
(a) fa (x) ≥ 0 for all x ∈ (0, 1). Also,
Z 1 Z 1
x(1 − x) dx = (x − x2 ) dx
0 0 Integration of a power
2 3 1
 
x x
= −
2 3 0
= ( 21 − 13 ) − (0 − 0) = 16 .
The correct density associated with this function is therefore
fa (x) x(1 − x)
= = 6x(1 − x) on 0 < x < 1.
1/6 1/6

f (x)

1.5

0 1 x

Graph of the pdf 6x(1 − x) on 0 < x < 1

(b) The exponential function is positive for all x > 0, but 1 − x is not: it
is negative when x > 1. It follows that fb (x) is negative when x > 1
and hence is not a probability density function on x > 0.
(c) This function is non-negative for all x ∈ (−1, 1). Its integral is
Z 0 Z 1 0 1
x2 x2
 
(1 + x) dx + (1 − x) dx = x + + x−
−1 0 2 −1 2 0
= 0 − ((−1) + 12 ) + ((1 − 12 ) − 0)
1 1
= 2
+ 2
= 1.
Alternatively, you might recognise fc as the triangular function shown
below. If so, the integral, which is the area under the triangle, is
1 1
2 + 2 = 1. This is because the area under the left-hand half of the
triangle is half that of the unit square (which is 21 × 1), and similarly
for the right-hand half.
f (x)
1

−1 0 1 x

Graph of the function fc

24
 Univariate continuous distribution theory

Either way, being non-negative and integrating to 1, this function is a

density function as it stands.
(d) fd (x) is non-negative for all x > 1. However,
Z ∞
1 ∞
dx = [log x]1 = log ∞ − log 1 = ∞ − 0 = ∞.
1 x
This function is not, and cannot be made into, a density, because its
integral is infinity.

Solution 2.2
F (x) = P (X ≤ x) = P (X < x) + P (X = x) = P (X < x)
in the continuous case because then P (X = x) = 0.

Solution 2.3
(a) As f is a pdf and must therefore integrate to 1,
Z x
lim F (x) = lim f (y) dy
x→∞ x→∞ −∞
Z ∞
= f (y) dy = 1.
−∞

(b) The derivative of F (x) is the density f (x), which is positive for all x
on its support; it follows that F is increasing on its support.

Solution 2.4
(a) F (x) = 0 for all x ≤ 0. Explicitly, for x ≤ 0,
Z x
F (x) = 0 dy = 0.
−∞

(b) For x > 0,

Z x Z x
f (y) dy = λe−λy dy
−∞ −∞ Integration of an exponential
Z x
= λe−λy dy
0

 x
1 −λy
=λ − e
λ 0
 −λy x
=− e 0
= −e−λx − (−1) = 1 − e−λx .
So

0 if x ≤ 0,
F (x) =
1 − e−λx if x > 0,
which can be written
F (x) = 1 − e−λx on x > 0.
(c) For x > 0,
d
F ′ (x) = (1 − e−λx ) = −(−λ)e−λx = λe−λx = f (x).
dx
The pdf and cdf of the exponential distribution are shown below (for
reference).

25
 Solutions

f (x)
1

0
−1 0 1 2 3 4 x
(a)

F (x)
1

0
−1 0 1 2 3 4 x
(b)

(a) Graph of pdf of exponential distribution with λ = 1;

(b) graph of cdf of exponential distribution with λ = 1

Solution 2.5
For β < 1, the power density is decreasing. For β = 1, the power density is
flat/constant. For β > 1, the power density is increasing.

Solution 2.6
(a) On 0 < x < 1,
Z x  β x
β−1 y
F (x) = βy dy = β = xβ − 0 = xβ .
0 β 0 Integration of a power
(b) On a < x < b,
Z x
1 1 x x−a
F (x) = dy = [y]a =
a b−a b−a b−a
(or you could have evaluated the area of the appropriate rectangle).

Solution 2.7
(a) P ( 14 ≤ X ≤ 85 ) = F ( 85 ) − F ( 14 )
= ( 85 )2 (3 − 2( 58 )) − ( 41 )2 (3 − 2( 14 ))
25
= 64 (3 − 54 ) − 1
16 (3 − 12 )
25 7 1
= ( 64 )( 4 ) − ( 16 )( 25 )
175−40 135
= 256
= 256
= 0.53
(correct to two decimal places).
(b) P (log 2 ≤ X ≤ log 4) = F (log 4) − F (log 2)
Exponentials and logarithms
= (1 − e− log 4 ) − (1 − e− log 2 )
= (1 − 14 ) − (1 − 12 )
3 1
= 4 − 2 = 14 .

Solution 2.8
(a) P (x0 ≤ X < x0 + ε) = P (x0 ≤ X ≤ x0 + ε)
= F (x0 + ε) − F (x0 ).

26
 Univariate continuous distribution theory

P (x0 ≤ X < x0 + ε) F (x0 + ε) − F (x0 )

(b) lim = lim
ε→0 ε ε→0 ε
which equals the derivative of F at x0 by the definition of
differentiation given in Subsection 2.2 of Unit 1. And the derivative of
F at x0 is f (x0).

Solution 2.9
Z b
E{k1 h1 (X) + k2 h2 (X)} = {k1 h1 (x) + k2 h2 (x)} f (x) dx
a
Z b Z b
= k1 h1 (x)f (x) dx + k2 h2 (x)f (x) dx
a a Linearity of integration
Z b Z b
= k1 h1 (x)f (x) dx + k2 h2 (x)f (x) dx
a a
= k1 E{h1 (X)} + k2 E{h2 (X)}.

Solution 2.10
Set h1 (X) = h(X) and h2 (X) = 1. Then Equation (2.2) shows that
E{k1 h(X) + k2 } = k1 E{h(X)} + k2 E(1) = k1 E{h(X)} + k2
since E(1) = 1.

Solution 2.11
E{k h(X)} = k E{h(X)} by setting k1 = k and k2 = 0 in Equation (2.3).

Solution 2.12
Z 1
E(X) = 6x2 (1 − x) dx
0
Z 1
=6 (x2 − x3 ) dx
0
1
x3 x4

=6 − = 6(( 13 − 41 ) − (0 − 0)) = 6 × 1
12
= 12 .
3 4 0

Solution 2.13
Z 1
r
E(X ) = xr βxβ−1 dx
0
Z 1
=β xr+β−1 dx
0
1
xr+β

β β
=β = (1 − 0) = .
r+β 0 (r + β) r+β
The formula for the uniform distribution arises when β = 1, namely
1/(r + 1).

Solution 2.14
µ0 = E{(X − µ)0 } = E(1) = 1.
By Equation (2.3),
µ1 = E(X − µ) = E(X) − µ = µ − µ = 0.

27
 Solutions

Solution 2.15
( r  
)
r
X r
µr = E{(X − µ) } = E X i (−µ)r−i
i=0
i Binomial expansion
(by setting a = X and b = −µ in the binomial expansion) Linearity of expectation
r  
X r
= (−1)r−i E(X i ) µr−i
i=0
i
(by the linearity of expectation).

Solution 2.16
β β
E(X) = , E(X 2 ) = ,
1+β 2+β
and so
V (X) = E(X 2 ) − {E(X)}2
 2
β β
= −
2+β 1+β
β(1 + β)2 − β2 (2 + β)
=
(2 + β)(1 + β)2
β + 2β2 + β3 − 2β2 − β3
=
(2 + β)(1 + β)2
β
= .
(2 + β)(1 + β)2

Solution 2.17
(a) From Equation (2.3), E(Y ) = E(aX + b) = a E(X) + b.
(b) The calculation of V (Y ) is similar to the calculation of V (L) and
V (S) above:
V (Y ) = E{(Y − E(Y ))2 }
= E{(aX + b − (aµ + b))2 }
= E{(aX − aµ)2 }
= E{a2 (X − µ)2 } = a2 E{(X − µ)2 } = a2 V (X).

Solution 2.18
(a) By Equations (2.6),
E(Y ) = σ E(X) + µ = σ × 0 + µ = µ,
V (Y ) = σ2 V (X) = σ2 × 1 = σ2 .
(b) By Equation (2.7),
 
1 y−µ
fY (y) = φ .
σ σ
1
φ(x) = √ exp − 12 x2


(c)
2π
gives
(  2 )
1 1 y−µ
fY (y) = √ exp − ,
σ 2π 2 σ

28
 Univariate continuous distribution theory

which is indeed the pdf of the normal distribution with mean µ and
variance σ2 .

Solution 2.19
Starting from MX′′
(t) = (1 + t2 ) exp( 12 t2 ) as in Example 2.9, you should
find that
(3) d ′′ Product rule
MX (t) = MX (t)
dt
= 2t exp( 12 t2 ) + (1 + t2 )t exp( 12 t2 )
= (3t + t3 ) exp( 12 t2 )
and
(4) d (3)
MX (t) = M (t)
dt X
= (3 + 3t2 ) exp( 12 t2 ) + (3t + t3 )t exp( 12 t2 )
= (3 + 6t2 + t4 ) exp( 12 t2 ).
Thus
(4)
E(X 4 ) = MX (0) = (3 + 0 + 0) × 1 = 3.

Solution 2.20
(a) MX (t) = E(etX )
Z ∞
= exp(tx) f (x) dx
0
Z ∞
= exp(tx) λ exp(−λx) dx
0 Integration of an exponential
Z ∞
=λ exp{(t − λ)x} dx
0
 ∞
1
=λ exp{(t − λ)x}
t−λ 0
λ λ
= (0 − 1) =
t−λ λ−t
(note that t − λ < 0 gives e(t−λ)x = 0 for x = ∞).
′ λ ′ λ 1
(b) MX (t) = 2
, so E(X) = MX (0) = 2 = ,
(λ − t) λ λ
2λ 2λ 2
MX ′′
(t) = 3
, so E(X 2 ) = MX ′′
(0) = 3 = 2 ,
(λ − t) λ λ
and therefore
 2
2
2 2 1 1
V (X) = E(X ) − {E(X)} = 2 − = 2.
λ λ λ
(r) d (r−1) r(r − 1)!λ r!λ
(c) MX (t) = MX (t) = r+1
= , t < λ.
dt (λ − t) (λ − t)r+1
(r) r!λ r!
(d) E(X r ) = MX (0) = r+1 = λr ,
λ
as required.

29