Chapter 3

Multidimensional Random Variables

3.1 What Is a Multidimensional Random Variable?

We have already explained that one random variable is obtained by mapping a

sample space S to the real x-axis. However, there are many problems in which the
outcomes of the random experiment need to be mapped from the space S to a two-
dimensional space (x1, x2), leading to a two-dimensional random variable (X1, X2).
Since the variables X1 and X2 are the result of the same experiment, then it is
necessary to make a joint characterization of these two random variables.
In general, the mapping of the outcomes from S to a N-dimensional space leads
to a N-dimensional random variable.
Fortunately, many problems in engineering can be solved by considering
only two random variables [PEE93, p. 101]. This is why we emphasize the two
random variables and the generalization of a two-variable case will lead to a
N-dimensional case.

3.1.1 Two-Dimensional Random Variable

Consider the sample space S, as shown in Fig. 3.1, and a two-dimensional space
where x1 and x2 are real axes,
1 < x1 < 1,
1 < x2 < 1. Mapping of the
outcome si to a point (x1, x2) in a two-dimensional space, leads to a two-dimensional
random variable (X1, X2).
If the sample space is discrete, the resulting two-dimensional random variable
will also be discrete. However, as in the case of a one variable, the continuous
sample space may result in continuous, discrete, or mixed two-dimensional random
variables. The range of a discrete two-dimensional random variable is made of
points in a two-dimensional space, while the range of a continuous two-dimensional
variable is a continuous area in a two-dimensional space. Similarly, a mixed
two-dimensional random variable has, aside from continuous area, additional

G.J. Dolecek, Random Signals and Processes Primer with MATLAB, 155
DOI 10.1007/978-1-4614-2386-7_3, # Springer Science+Business Media New York 2013
156 3 Multidimensional Random Variables

Fig. 3.1 Obtaining a two-dimensional random variable

discrete points. Figure 3.2 shows examples of discrete, continuous, and mixed
ranges.
The following example illustrates this concept.
Example 3.1.1 Two messages have been sent through a communication system.
Each of them, independently of another, can be transmitted with or without error.
Define the two-dimensional variable for this case.
Solution The sample space has four outcomes, shown in Fig. 3.3.
We used the following denotation:
s1 ¼ fThe first and second messages are transmitted correctlyg;
s2 ¼ fThe first message is transmitted correctly; while the second one incurs an errorg;
s3 ¼ fThe second message is transmitted correctly; while the first message incurs an errorg;
s4 ¼ fBoth messages incur an error in transmissiong:
(3.1)

Fig. 3.2 Examples of discrete, continuous and mixed ranges. (a) Discrete. (b) Continuous. (c) Mixed
3.1 What Is a Multidimensional Random Variable? 157

We can see here that, in order to mathematically model this experiment, it is

more convenient to use the numerical assignations to the outcomes si, instead of
words in (3.1). Similarly, as in the case of one variable, it is convenient to assign the
numbers 0 and 1, for binary outcomes (the message is either correct or incorrect).
Let us assign the number 1 if the message is correct. Otherwise, 0 is assigned.
The first message is mapped to the x1-axis, while the second message is mapped to
the x2-axis, as shown in Fig. 3.3. The range of two-dimensional variable (X1, X2)
is the following points in the (x1, x2) space:

ðx1 ¼ 1; x2 ¼ 1Þ; ðx1 ¼ 1; x2 ¼ 0Þ; ðx1 ¼ 0; x2 ¼ 1Þ; ðx1 ¼ 0; x2 ¼ 0Þ: (3.2)

Example 3.1.2 Consider the experiment of tossing two coins. There are four
outcomes in the sample space S:

s1 ¼ fFirst coin comes up heads; Second coin comes up headsg;

s2 ¼ fFirst coin comes up heads; Second coin comes up tailsg;
s3 ¼ fFirst coin comes up tails; Second coin comes up headsg;
s4 ¼ fFirst coin comes up tails; Second coin comes up tailsg: (3.3)

Adopting the denotation from Example 1.1.1, we rewrite the outcomes (3.3) in
the following form:

s1 ¼ fH; Hg;
s2 ¼ fH; Tg;
s3 ¼ fT; Hg;
s4 ¼ fT; Tg: (3.4)

Fig. 3.3 Sample space and two-dimensional r.v. in Example 3.1.1

158 3 Multidimensional Random Variables

Let X1 and X2 indicate the occurrence of heads and tails for the first and second
coins, respectively.
The values of the random variable X1 for the first coin tossing are x1 ¼ 1 if heads
occurs, and x1 ¼ 0, if tails occurs.
Similarly, for the variable X2, the value x2 ¼ 1 indicates the occurrence of heads,
and x2 ¼ 0 indicates tails in the second coin toss.
The mapping from the sample space S to the (x1, x2) space is shown in Fig. 3.4.
Note that the values of the random variables are the same as in Example 3.1.1.
Therefore, one can obtain the same two-dimensional random variable, from differ-
ent random experiments.
The next example illustrates that, like in the one variable example, one can
obtain different two-dimensional random variables from the same experiment.
Example 3.1.3 Consider the same outcomes as in Example 3.1.1 for the following
mapping:

X1 indicates at least one head occurs:

X2 indicates at least one tail occurs: (3.5)

Therefore, the values of the random variable X1 are x1 ¼ 1, if at least one heads
occurs and x1 ¼ 0 if no heads occurs. Similarly, the values of X2 are x2 ¼ 1 if at
least one tails occurs and x2 ¼ 0 if no tails occurs. This mapping is shown in
Fig. 3.5.
In a similar way, one can define a N-dimensional random variable by mapping
the outcomes of the space S to N-dimensional space, thus obtaining the N-dimen-
sional random variable,
ðX1 ; X2 ; . . . ; XN Þ (3.6)
with the range:

ðx1 ; x2 ; . . . ; xN Þ: (3.7)

Fig. 3.4 Mapping the sample space S to (x1, x2) space in Example 3.1.2
3.2 Joint Distribution and Density 159

Fig. 3.5 Mapping from the space S to the space (x1, x2) in Example 3.1.3

3.2 Joint Distribution and Density

3.2.1 Joint Distribution

The cumulative distribution function, joint distribution function or, shortly, joint
distribution of a pair of random variables X1 and X2 is defined as the probability of
joint events:

fX1  x1 ; X2  x2 g; (3.8)

and is denoted as FX1 X2 ðx1 ; x2 Þ.

Therefore, we have:

FX1 X2 ðx1 ; x2 Þ ¼ PfX1  x1 ; X2  x2 g;
1 < x1 < 1;

1 < x2 < 1; (3.9)

where x1 and x2 are values within the two-dimensional space as shown in (3.9). The
joint distribution is also called two-dimensional distribution, or second distribution.
In this context, distribution of one random variable is also called one-dimensional
distribution, or first distribution.
From the properties of a one-dimensional distribution, we easily find the follow-
ing properties for a two-dimensional distribution:
P.1 0  FX1 X2 ðx1 ; x2 Þ  1: (3.10)

P.2 FX1 X2 ð
1;
1Þ ¼ PfX1 
1; X2 
1g ¼ 0: (3.11a)

FX1 X2 ð
1; x2 Þ ¼ PfX1 
1; X2  x2 g ¼ 0: (3.11b)

160 3 Multidimensional Random Variables

FX1 X2 ðx1 ;
1Þ ¼ PfX1  x1 ; X2 
1g ¼ 0: (3.11c)

P.3 FX1 X2 ð1; 1Þ ¼ PfX1  1; X2  1g ¼ 1: (3.12)

P.4 Two-dimensional distribution is a nondecreasing function of both x1 and x2.

P.5 FX1 X2 ðx1 ; 1Þ ¼ PfX1  x1 ; X2  1g ¼ FX1 ðx1 Þ: (3.13)

FX1 X2 ð1; x2 Þ ¼ PfX1  1; X2  x2 g ¼ FX2 ðx2 Þ: (3.14)

The one-dimensional distributions in (3.13) and (3.14) are called marginal

distributions.
Next, we express the probability that the pair of variables X1 and X2 is in a given
space, in term of its two-dimensional distribution function:

Pfx11 < X1  x12 ; x21 < X2  x22 g ¼ Pfx11 < X1  x12 ; X2  x22 g

Pfx11 < X1  x12 ; X2 < x21 g ¼ PfX1  x12 ; X2  x22 g
PfX1 < x11 ; X2  x22 g

½PfX1  x12 ; X2 < x21 g
PfX1 < x11 ; X2 < x21 g
¼ FX1 X2 ðx12 ; x22 Þ
FX1 X2 ðx11 ; x22 Þ
FX1 X2 ðx12 ; x21 Þ þ FX1 X2 ðx11 ; x21 Þ:
(3.15)

If a two-dimensional variable is discrete, then its distribution has two-dimensional

unit step functions u(x1i, x2j) ¼ u(x1i)u(x2j) in the corresponding discrete points
(x1i, x2j), as shown in the following equation (see [PEE93, pp. 358–359]):
XX
FX1 X2 ðx1 ; x2 Þ ¼ PfX1 ¼ x1i ;X2 ¼ x2j guðx1
x1i Þuðx2
x2j Þ: (3.16)
i j

Example 3.2.1 Find the joint distribution function from Example 3.1.1 considering
that the probability that the first and second messages are correct (denoted as p1 and
p2, respectively), and that they are independent.

Pfx1 ¼ 1; x2 ¼ 1g ¼ Pfx1 ¼ 1gPfx2 ¼ 1g ¼ p1 p2 ;

Pfx1 ¼ 1; x2 ¼ 0g ¼ Pfx1 ¼ 1gPfx2 ¼ 0g ¼ p1 ð1
p2 Þ;
Pfx1 ¼ 0; x2 ¼ 1g ¼ Pfx1 ¼ 0gPfx2 ¼ 1g ¼ p2 ð1
p1 Þ;
Pfx1 ¼ 0; x2 ¼ 0g ¼ Pfx1 ¼ 0gPfx2 ¼ 0g ¼ ð1
p1 Þð1
p2 Þ: (3.17)

The joint distribution is:

FX1 X2 ðx1 ;x2 Þ ¼ þ p1 p2 uðx1
1Þuðx2
1Þ þ p1 ð1
p2 Þuðx1
1Þuðx2 Þ

þ ð1
p1 Þp2 uðx1 Þuðx2
1Þ þ ð1
p1 Þð1
p2 Þuðx1 Þuðx2 Þ: (3.18)
3.2 Joint Distribution and Density 161

Example 3.2.2 Find the joint distribution in Example 3.1.2.

Solution All outcomes have the same probability of occurrence, resulting in:

PfX1 ¼ 1; X2 ¼ 1g ¼ PfX1 ¼ 0; X2 ¼ 0g ¼ PfX1 ¼ 1; X2 ¼ 0g

¼ PfX1 ¼ 0; X2 ¼ 1g ¼ 1=4: (3.19)

From (3.19), we have:

FX1 X2 ðx1 ; x2 Þ ¼ þ 1=4½uðx1
1Þuðx2
1Þ þ uðx1
1Þuðx2 Þ þ uðx1 Þuðx2
1Þ

þ uðx1 Þuðx2 Þ:
(3.20)

Example 3.2.3 Find the joint distribution in Example 3.1.3.

Solution All outcomes have the same probability of occurrence. However, the
probabilities of the values of the random variables are:

Pfx1 ¼ 1; x2 ¼ 1g ¼ PfH; Tg þ PfT; Hg ¼ 1=4 þ 1=4 ¼ 1=2;

Pfx1 ¼ 1; x2 ¼ 0g ¼ PfH; Hg ¼ 1=4;
Pfx1 ¼ 0; x2 ¼ 1g ¼ PfT; Tg ¼ 1=4: (3.21)

The joint distribution is:

FX1 X2 ðx1 ; x2 Þ ¼ 1=2uðx1
1Þuðx2
1Þ þ 1=4uðx1
1Þuðx2 Þ þ 1=4uðx1 Þuðx2
1Þ:

(3.22)

Example 3.2.4 Find the probability that a random point (x1, x2) will fall in the area
A, as shown in Fig. 3.6.
Solution Area A is defined as:

fx1  a; x2  cg
fx1  a; x2  bg: (3.23)

Fig. 3.6 Area A in Example 3.2.4

162 3 Multidimensional Random Variables

From here, we have:

PfðX1 X2 Þ 2 Ag ¼ Pfx1  a; x2  cg
Pfx1  a; x2  bg: (3.24)

Using (3.9), (3.24) can be rewritten as:

PfðX1 ; X2 Þ 2 Ag ¼ FX1 X2 ða; cÞ
FX1 X2 ða; bÞ: (3.25)

In general, for N random variables

ðX1 ; X2 ; . . . ; XN Þ;

the joint distribution, denoted as FX1 ;...; XN ðx1 ; . . . ; xN Þ, is then defined as:

FX1 ;...; XN ðx1 ; . . . ; xN Þ ¼ PfX1  x1 ; . . . ; X1  xN g: (3.26)

3.2.2 Joint Density Function

The joint probability density function (PDF) or joint density function or two-
dimensional density function or shortly, joint PDF, for pair of random variables
X1 and X2, is denoted as, fX1 X2 ðx1 ; x2 Þ, and defined as:

@ 2 FX1 X2 ðx1 ; x2 Þ
fX1 X2 ðx1 ; x2 Þ ¼ : (3.27)
@x1 @x2

For discrete variables the derivations are not defined in the step discontinuities,
implying the introduction of delta functions at pairs of discrete points (x1i, x2j).
Therefore, the joint PDF is equal to (see [PEE93, pp. 358–359], [HEL91, p. 147]):
XX
fX1 X2 ðx1 ; x2 Þ ¼ PfX1 ¼ x1i ;X2 ¼ x2j gdðx1
x1i Þdðx2
x2j Þ: (3.28)
i j

Example 3.2.5 We can find the joint density functions for Examples 3.2.1, 3.2.2,
and 3.2.3. Using the distribution (3.18), and (3.27), we have:

fX1 X2 ðx1 ; x2 Þ ¼ þ p1 p2 dðx1
1Þdðx2
1Þ þ p1 ð1
p2 Þdðx1
1Þdðx2 Þ

þ ð1
p1 Þp2 dðx1 Þdðx2
1Þ þ ð1
p1 Þð1
p2 Þdðx1 Þdðx2 Þ: (3.29)

Similarly, from (3.20) we have:

fX1 X2 ðx1 ;x2 Þ ¼ þ 1=4½dðx1
1Þdðx2
1Þ þ dðx1
1Þdðx2 Þ

þ dðx1 Þdðx2
1Þ þ dðx1 Þdðx2 Þ: (3.30)
3.2 Joint Distribution and Density 163

From (3.22), we get:

fX1 X2 ðx1 ; x2 Þ ¼ 1=2dðx1
1Þdðx2
1Þ þ 1=4dðx1
1Þdðx2 Þ þ 1=4dðx1 Þdðx2
1Þ:

(3.31)

Next, in Fig. 3.7 we compare one-dimensional and two-dimensional PDFs.

The shaded area in Fig. 3.7a represents the probability that the random variable
X is in the infinitesimal interval [x, x + dx]:

A ¼ Pfx < X  x þ dxg: (3.32)

From here, considering that dx is an infinitesimal interval, the PDF in the interval
[x, x + dx] is constant, resulting in:

fX ðxÞdx ¼ A ¼ Pfx < X  x þ dxg: (3.33)

Similarly, the volume in Fig. 3.7b presents the probability that the random
variables X1 and X2 are in the intervals [x1, x1 + dx1] and [x2, x2 + dx2],
respectively.
The equivalent probability

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g (3.34)

corresponds to the elemental volume V, with a base of (dx1 dx2) and height of
fX 1 X2 ðx1 ; x2 Þ:

fX 1 X2 ðx1 ; x2 Þdx1 dx2 ¼ V ¼ Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g: (3.35)

Fig. 3.7 One-dimensional and two-dimensional PDFs

164 3 Multidimensional Random Variables

Therefore, as opposed to one-dimensional random variable, where the probability

represents the area below the density function, for two-dimensional variable, the
probability is a volume below a joint density function.
The properties of one-dimensional PDF can be easily applied to the
two-dimensional PDF:
P.1 fX1 X2 ðx1 ; x2 Þ  0: (3.36)

P.2 ð
1 ð
1

fX1 X2 ðx1 ; x2 Þdx1 dx2 ¼1: (3.37)

1
1

xð1 xð2
P.3
FX1 X2 ðx1 ; x2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2: (3.38)

1
1

xð1 ð
1
P.4
FX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2: (3.39)

1
1

xð2 ð
1

FX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2: (3.40)

1
1

xð22 xð12
P.5
Pfx11 < X1  x12 ; x21 < X2  x22 g ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2: (3.41)
x21 x11

P.6 ð
1

fX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx2 : (3.42a)

1

ð
1

fX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 : (3.42b)

1

Example 3.2.6 A two-dimensional random variable has a constant joint density

function in the shaded area A, shown in Fig. 3.8. Find the expression for the joint
density function.
3.2 Joint Distribution and Density 165

Fig. 3.8 The range of (X1, X2) in Example 3.2.6

Solution From (3.37), we have:


1=ab for ðx1 ; x2 Þ 2 A
fX1 X2 ðx1 ; x2 Þ ¼ : (3.43)
0 otherwise

For N random variables, N-dimensional density function is defined as,

@ N FX1 X2 ;...;XN ðx1 ; x2 ; . . . ; xN Þ

fX1 X2 ;...; XN ðx1 ; x2 ; . . . ; xN Þ ¼ : (3.44)
@x1 @x2 ; . . . ; @xN

3.2.3 Conditional Distribution and Density

In some practical problems, it is necessary to find the distribution or PDF of one

variable given the specific value of another variable. The corresponding conditional
distribution denoted as, FX1 ðx1 jX2 ¼ x2 Þ, can be expressed in terms of the joint
density function.
We consider a general case in which X1 and X2 are both continuous variables and
thus the conditional event B is defined as:

B ¼ fx2 < X2  x2 þ dx2 g: (3.45)

From (1.73) and (3.45), we have:

PfX1  x1 ; Bg PfX1  x1 ; x2 < X2  x2 þ dx2 g

FX1 ðx1 jBÞ ¼ ¼ : (3.46)
PfBg Pfx2 < X2  x2 þ dx2 g
166 3 Multidimensional Random Variables

Using (3.45), (3.46) is rewritten as:

Ðx1 Ð 2
x2 þdx
fX1 X2 ðx1 ; x2 Þdx1 dx2

1 x2
FX1 ðx1 jx2 < X2  x2 þ dx2 Þ ¼
Ð 2
x2 þdx
fX2 ðx2 Þdx2
x2
Ðx1 Ðx1
fX1 X2 ðx1 ; x2 Þdx1 dx2 fX1 X2 ðx1 ; x2 Þdx1

1
1
¼ ¼ :
fX2 ðx2 Þdx2 fX2 ðx2 Þ
(3.47)

If dx2 is approaching zero, and for each x2 for which fX 2 ðx2 Þ 6¼ 0, we finally
obtain:

Ðx1
fX1 X2 ðx1 ; x2 Þdx1

1
FX1 ðx1 jX2 ¼ x2 Þ ¼ : (3.48)
fX2 ðx2 Þ

Similarly, we have:

Ðx2
fX1 X2 ðx1 ; x2 Þdx2

1
FX2 ðx2 jX1 ¼ x1 Þ ¼ : (3.49)
fX1 ðx1 Þ

From (3.48) and (3.49), using (3.27), we obtain the corresponding PDFs:

fX1 X2 ðx1 ; x2 Þ
fX1 ðx1 jX2 ¼ x2 Þ ¼ : (3.50)
fX2 ðx2 Þ

fX1 X2 ðx1 ; x2 Þ
fX2 ðx2 jX1 ¼ x1 Þ ¼ : (3.51)
fX1 ðx1 Þ

Consider now that the condition for event B is defined as an event in which the
other variable X2 lies in the given interval [x21, x22], resulting in:

Ðx1 xÐ22
fX1 X2 ðx1 ; x2 Þdx1 dx2

1 x21
FX1 ðx1 jx21 < X2  x22 Þ ¼ xÐ22
fX2 ðx2 Þdx2
x21
FX1 X2 ðx1 ; x22 Þ
FX1 X2 ðx1 ; x21 Þ
¼ ; (3.52)
FX2 ðx22 Þ
FX2 ðx21 Þ
3.2 Joint Distribution and Density 167

where the denominator in (3.52) is:

FX2 ðx22 Þ
FX2 ðx21 Þ ¼ Pfx21 < X2  x22 g 6¼ 0: (3.53)

From (3.27) and (3.52), the corresponding conditional density is:

xÐ22
fX1 X2 ðx1 ; x2 Þdx2
x21
fX1 ðx1 jx21 < X2  x22 Þ ¼ xÐ22 : (3.54)
Ð
1
fX1 X2 ðx1 x2 Þdx1 dx2
x21
1

Example 3.2.7 Consider a random variable X1 with the density function:


l e
lx1 for x1 > 0
fX1 ðx1 Þ ¼ (3.55)
0 otherwise

and the conditional density


x1 e
x1 x2 for x1 > 0; x2 > 0
fX2 ðx2 jx1 Þ ¼ : (3.56)
0 otherwise

Find the conditional density fX1 ðx1 jx2 Þ.

Solution From (3.51), (3.55), and (3.56), we get:

fX1 X2 ðx1 ; x2 Þ ¼ fX2 ðx2 jx1 ÞfX1 ðx1 Þ: (3.57)

Placing (3.55) and (3.56) into (3.57), we arrive at:


lx1 e
x1 ðlþx2 Þ for x1 > 0; x2 > 0
fX1 X2 ðx1 ; x2 Þ ¼ : (3.58)
0 otherwise

From here, using (3.42b), we find:

ð
1 ð
1

fX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þ dx1 ¼ lx1 e
x1 ðlþx2 Þ dx1 : (3.59)
0 0

Using integral 1 from Appendix A, we obtain:

 l
for x2 > 0
fX2 ðx2 Þ ¼ ðlþx2 Þ2 : (3.60)
0 otherwise
168 3 Multidimensional Random Variables

The desired conditional density is obtained using (3.50):

fX1 X2 ðx1 ; x2 Þ
fX1 ðx1 jx2 Þ ¼ : (3.61)
fX2 ðx2 Þ

Finally, placing (3.58) and (3.60) into (3.61), we have:


x1 ðl þ x2 Þ2 e
x1 ðlþx2 Þ for x1 > 0
fX1 ðx1 jx2 Þ ¼ : (3.62)
0 otherwise

3.2.4 Independent Random Variables

The two random variables, X1 and X2, are independent if the events

fX1  x1 g and fX2  x1 g

are independent for any value of x1 and x2.

Using (1.105), we can write:

PfX1  x1 ; X2  x1 g ¼ PfX1  x1 gPfX2  x1 g: (3.63)

From (3.9) and (2.10), the joint distribution is:

FX1 X2 ðx1 ; x2 Þ ¼ FX1 ðx1 ÞFX1 ðx1 Þ: (3.64)

Similarly, for the joint density we have:

fX1 X2 ðx1 ; x2 Þ ¼ fX1 ðx1 Þ fX1 ðx1 Þ: (3.65)

Therefore, if the random variables X1 and X2 are independent, then their joint
distributions and joint PDFs are equal to the products of the marginal distributions
and densities, respectively.
Example 3.2.8 Determine whether or not the random variables X1 and X2 are
independent, if the joint density is given as:

1=2 for 0 < x1 < 2; 0 < x2 < 1
fX1 X2 ðx1 ; x2 Þ ¼ : (3.66)
0 otherwise

Solution The joint density (3.66) can be rewritten as:

fX1 X2 ðx1 ; x2 Þ ¼ fX1 ðx1 ÞfX2 ðx2 Þ; (3.67)

3.3 Expected Values and Moments 169

where

1=2 for 0 < x2 < 2
fX1 ðx1 Þ ¼ : (3.68)
0 otherwise

1 for 0 < x2 < 1
fX2 ðx2 Þ ¼ : (3.69)
0 otherwise

From (3.65) and (3.67)–(3.69), we can conclude that the variables are
independent.
The result (3.65) can be generalized to N jointly independent random variables:

Y
N
FX1 ;...;XN ðx1 ; . . . ; xN Þ ¼ FXi ðxi Þ: (3.70)
i¼1

Y
N
fX1 ;...;XN ðx1 ; . . . ; xN Þ ¼ fXi ðxi Þ: (3.71)
i¼1

3.3 Expected Values and Moments

3.3.1 Expected Value

In order to find the mean value of two joint random variables X1 and X2, we will
apply the similar procedure to that which we used in the case of one random
variable (see Sect. 2.7), starting with a random experiment.
Consider two discrete random variables X1 and X2 with the possible values x1i
and x2j, respectively.
The experiment is performed N times under the same conditions,

N1 X
X N2
N¼ Nij ; (3.72)
i¼1 j¼1

and as a result the following values are obtained:

X1 ¼ x11 ; and X2 ¼ x21 ; N11 times:

  
X1 ¼ x1i; and X2 ¼ x2j ; Nij times: (3.73)
  
X1 ¼ x1N1 ; and X2 ¼ x2N2 ; NN1 N2 times:
170 3 Multidimensional Random Variables

As indicated in Sect. 2.7.2, we now have a finite set of values x1i and x2j, and we
can calculate the arithmetic mean value of the products, also called the empirical
mean value, since it is obtained from the experiment,

P
N1 P
N2
Nij x1i x2j
i¼1 j¼1
X1 X2emp ¼ : (3.74)
N

For a high enough value N, the ratio Nij/N becomes a good approximation of the
probability,

Nij
! PfX1 ¼ x1i ; X2 ¼ x2j g; (3.75)
N

and the empirical mean value becomes independent on experiment and approaches
the mean value of joint random variables X1 and X2,
XX
X 1 X2 ¼ x1i x2j PfX1 ¼ x1i ; X2 ¼ x2j g; (3.76)
i j

or, in a general case,

X
1 X
1
X1 X2 ¼ x1i x2j PfX1 ¼ x1i ; X2 ¼ x2j g: (3.77)
i¼
1 j¼
1

This result can be generalized for N discrete random variables,

X1 ; . . . ; XN ; (3.78)

as given in the following equation:

X
1 X
1
X 1 . . . XN ¼ ... x1i . . . xN j PfX1 ¼ x1i ; . . . ; XN ¼ xN j g: (3.79)
i¼
1 j¼
1

Similarly,

X
1 X
1
gðX1 ...XN Þ ¼ .. . gðx1i ...xN j ÞPfX1 ¼ x1i ; ...; XN ¼ xN j g: (3.80)
i¼
1 j¼
1
3.3 Expected Values and Moments 171

Example 3.3.1 The discrete random variables X1 and X2, both take the discrete
values
1 and 1, with the following probabilities:

PfX1 ¼ x11 ¼
1; X2 ¼ x21 ¼
1g ¼ 1=4;

PfX1 ¼ x12 ¼ 1; X2 ¼ x22 ¼ 1g ¼ 1=2;
PfX1 ¼ x11 ¼
1; X2 ¼ x22 ¼ 1g ¼ 1=8;
PfX1 ¼ x12 ¼ 1; X2 ¼ x21 ¼
1g ¼ 1=8: (3.81)

Find the mean values:

(a) EfX1 X2 g: (3.82)

(b) EfX12 þ X2 g: (3.83)

Solution
(a) From (3.76), we have:

X1 X2 ¼ ð
1Þ  ð
1Þ  1=4 þ 1  1  1=2 þ ð
1Þ  1  1=8 þ 1  ð
1Þ  1=8 ¼ 1=2:

(3.84)

(b) Using (3.80), we get:

2 X
X 2
X12 þ X2 ¼ ðx2i1 þ x2j ÞPfX1 ¼ x1i ; X2 ¼ x2j g
i¼1 j¼1
h i
¼ ð
1Þ2 þ ð
1Þ  1=4 þ ½12 þ 1  1=2
h i  
þ ð
1Þ2 þ 1Þ  1=8 þ 12 þ ð
1Þ  1=8 ¼ 5=4: (3.85)

Using a similar approach to that taken in Sect. 2.7.3, we can express the mean
value of the joint continuous random variables X1 and X2, using the joint density
function fX1 X2 ðx1 ; x2 Þ:
ð
1 ð
1

X 1 X2 ¼ x1 x2 fX1 X2 ðx1 ; x2 Þdx1 dx2: (3.86)

1
1

Instead, we can consider the function g(X1, X2) resulting in:

ð
1 ð
1

gðX1 X2 Þ ¼ gðx1 x2 ÞfX1 X2 ðx1 ; x2 Þdx1 dx2: (3.87)

1
1

The mean value for the two random variables can be generalized for N continu-
ous random variables:

X 1 ; . . . ; XN : (3.88)
172 3 Multidimensional Random Variables

ð
1 ð
1

X1 ; . . . ; XN ¼ ... x1 ; . . . ; xN fX1 ;...;XN ðx1 ; . . . ; xN Þdx1 ; . . . ; dxN : (3.89)

1
1

Similarly, the expression (3.87) can be generalized for N random variables (the
mean value of a function of N variables),

gðX1 ; . . . ; XN Þ (3.90)

ð
1 ð
1

gðX1 ; . . . ; XN Þ ¼ ... gðx1 ; . . . ; xN ÞfX1 ;...;XN ðx1 ; . . . ; xN Þdx1 ; . . . ; dxN : (3.91)

1
1

3.3.1.1 Mean Value of the Sum of Random Variables

Consider the sum of two random variables X1 and X2. We can write:

gðX1 ; X2 Þ ¼ X1 þ X2 ; (3.92)

and use (3.87) to find the desired mean value.

ð
1 ð
1

gðX1 X2 Þ ¼ X1 þ X2 ¼ ðx1 þ x2 ÞfX1 X2 ðx1 ; x2 Þdx1 dx2

1
1
ð
1 ð
1 ð
1 ð
1

¼ x1 fX1 X2 ðx1 ; x2 Þdx1 dx2 þ x2 fX1 X2 ðx1 ; x2 Þdx1 dx2

1
1
1
1
ð
1 ð
1 ð
1 ð
1

¼ x1 dx1 fX1 X2 ðx1 ; x2 Þdx2 þ x2 dx2 fX1 X2 ðx1 ; x2 Þdx1 : (3.93)

1
1
1
1

Using (3.42a) and (3.42b), we have:

ð
1 ð
1

fX1 X2 ðx1 ; x2 Þdx2 ¼ fX1 ðx1 Þ; fX1 X2 ðx1 ; x2 Þdx1 ¼ fX2 ðx2 Þ: (3.94)

1
1

Placing (3.94) into (3.93) and using (2.261) we obtain:

ð
1 ð
1

X1 þ X2 ¼ x1 fX1 ðx1 Þdx1 þ x2 fX2 ðx2 Þdx2 ¼X1 þ X2 : (3.95)

1
1
3.3 Expected Values and Moments 173

Equation (3.95) shows the following statement:

The mean value of the sum of two random variables is equal to the sum of the
corresponding mean values.
Note that no conditions have been imposed to obtain the result (3.95). A more
general result, which includes the N random variables, can be expressed as:

X
N X
N
Xk ¼ Xk : (3.96)
k¼1 k¼1

Example 3.3.2 Verify the relation (3.95) for the discrete random variables from
Example 3.3.1.
Solution From (3.80), we have:
2 X
X 2
X1 þ X2 ¼ ðx1i þ x2j ÞPfX1 ¼ x1i ; X2 ¼x2j g ¼ 3=4
1=4 ¼ 1=2: (3.97)
i¼1 j¼1

To verify the result (3.97), we found the following probabilities for random
variables X1 and X2, using (1.67):

PfX1 ¼ x11 ¼
1g ¼ PfX1 ¼ x11 ¼
1; X2 ¼ x21 ¼
1g

þ PfX1 ¼ x11 ¼
1; X2 ¼ x22 ¼ 1g ¼ 3=8: (3.98)

PfX1 ¼ x12 ¼ 1g ¼ PfX1 ¼ x12 ¼ 1; X2 ¼ x21 ¼
1g

þ PfX1 ¼ x12 ¼ 1; X2 ¼ x22 ¼ 1g ¼ 5=8: (3.99)

PfX2 ¼ x21 ¼
1g ¼ PfX2 ¼ x21 ¼
1; X1 ¼ x11 ¼
1g

þ PfX2 ¼ x21 ¼
1; X1 ¼ x12 ¼ 1g ¼ 3=8: (3.100)

PfX2 ¼ x21 ¼ 1g ¼ PfX2 ¼ x21 ¼ 1; X1 ¼ x11 ¼
1g

þ PfX2 ¼ x21 ¼ 1; X1 ¼ x12 ¼ 1g ¼ 5=8: (3.101)

From (2.220) and (3.98) and (3.99), we have:

X
2
X1 ¼ x1i PfX1 ¼x1i g ¼
1  3=8 þ 1  5=8 ¼ 1=4: (3.102)
i¼1

Similarly, from (3.100) and (3.101), it follows:

X
2
X2 ¼ x2i PfX2 ¼x2i g ¼
1  3=8 þ 1  5=8 ¼ 1=4: (3.103)
i¼1
174 3 Multidimensional Random Variables

From (3.95), (3.97), (3.102), and (3.103), we arrive at:

X1 þ X2 ¼ 1=2 ¼ X1 þ X2 ¼ 1=4 þ 1=4 ¼ 1=2: (3.104)

3.3.2 Joint Moments Around the Origin

The expected value of joint random variables X1 and X2,

 
E X1n X2k : (3.105)
is called the joint moment mr of the order r, where

r ¼ n þ k: (3.106)

Equation (3.105) presents the expected value of the function g(X1, X2) of the
random variables X1 and X2, and thus can be obtained using (3.87):
ð
1 ð
1

mr ¼ EfX1n X2k g ¼ xn1xk2 fX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.107)

1
1

This result can be generalized for N random variables X1, . . ., XN in order to

obtain the joint moment around the origin of the order
X
N
r¼ ni ; (3.108)
i¼1
ð
1 ð
1
 
E X1n1 ; . . . ; XNnN ¼ ... xn11 ; . . . ;xnNN fX1 ;...;XN ðx1 ; . . . ; xN Þdx1 ; . . . ; dxN : (3.109)

1
1

Special importance has been placed on the second moment, where n ¼ k ¼ 1,

which is called the correlation and has its proper denotation RX 1 X2 ,
ð
1 ð
1

RX1 X2 ¼ EfX1 X2 g ¼ x1x2 fX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.110)

1
1

Some important relations between variables X1 and X2 can be expressed using

the correlation.
For example, if the correlation can be written as a product of the expected values
of X1 and X2,
EfX1 ; X2 g ¼ EfX1 gEfX2 g; (3.111)
then it is said that the variables are uncorrelated.
3.3 Expected Values and Moments 175

If the correlation is zero, the variables are said to be orthogonal,

RX 1 X2 ¼ EfX1 ; X2 g ¼ 0: (3.112)

Note that if the variables are uncorrelated and one or both variables have a zero
mean value, it follows that they are also orthogonal.
Example 3.3.3 The random variables X1 and X2 are related in the following form:

X2 ¼
2X1 þ 5: (3.113)

Determine whether or not the variables are correlated and orthogonal, if the
random variable X1 has the mean, and the squared mean values equal to 2 and 5,
respectively.
Solution In order to determine if the variables are correlated and orthogonal, we
first have to find the correlation:

RX 1 X2 ¼ EfX1 X2 g ¼ EfX1 ð
2X1 þ 5Þg ¼ Ef
2X12 þ 5X1 g

¼
2EfX12 g þ 5EfX1 g ¼
2  5 þ 5  2 ¼ 0: (3.114)

Therefore, according to (3.112) the variables are orthogonal.

Next, we have to verify that the condition (3.111) is satisfied. To this end, we
have to find the mean value of X2, being as the mean value of X1 has been found to
be equal to 2.

EfX2 g ¼
2EfX1 g þ 5 ¼
4 þ 5 ¼ 1: (3.115)

Since

EfX1 X2 g ¼ RX1 X2 ¼ 0 6¼ EfX1 gEfX2 g ¼ 2  1 ¼ 2; (3.116)

the variables are correlated.

3.3.3 Joint Central Moments

The joint central moment of order r, of two random variables X1 and X2, with
corresponding mean values E{X1} and E{X2}, is defined as:
n o
mnk ¼ E ðX1
EfX1 gÞn ðX2
EfX2 gÞk ; (3.117)

where the order r is:

r ¼ n þ k: (3.118)
176 3 Multidimensional Random Variables

Using the expression for the mean value of the function of two random variables
(3.87), we arrive at:
ð
1 ð
1
k
mnk ¼ ðx1
X1 Þn ðx2
X2 Þ fX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.119)

1
1

This expression can be generalized for N random variables X1, . . ., XN:

ð
1 ð
1
n
mn1 ;...;nN ¼ ... ðx1
X1 Þn1 ; . . . ; ðxN
XN Þ N fX1 ;...;XN ðx1 ; . . . ; xN Þdx1 ; . . . ; dxN :

1
1
(3.120)

The second central moment m11, called covariance is especially important:

 
CX1 X2 ¼ m11 ¼ E ðX1
X1 ÞðX2
X2 Þ
ð 1
1 ð
¼ ðx1
X1 Þðx2
X2 ÞfX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.121)

1
1

Let us first relate the covariance to the independent variables, where the joint
PDF is equal to the product of the marginal PDFs, resulting in the following
covariance:

ð
1 ð
1

CX1 X2 ¼ m11 ¼ ðx1
X1 Þðx2
X2 ÞfX1 ðx1 ÞfX2 ðx2 Þdx1 dx2

1
1
21 32 3
ð ð
1 ð
1 ð
1

¼4 x1 fX1 ðx1 Þdx1
X1 fX1 ðx1 Þdx154 x2 fX2 ðx2 Þdx2
X2 fX2 ðx2 Þdx25

1
1
1
1
  
¼ X1
X1 X2
X2 ¼ 0:
(3.122)

From (3.122) it follows that the covariance is equal to zero for the independent
variables.
Using (3.95), (3.121) can be simplified as:

CX1 X2 ¼ X1 X2
X1 X2 : (3.123)

Let us now relate covariance with the correlated and orthogonal random
variables.
From (3.123) and (3.111), it follows that the covariance is equal to zero if the
random variables are uncorrelated.
3.3 Expected Values and Moments 177

Therefore, from summing the above statements it follows that the covariance
equals to zero if the random variables are either independent or dependent but
uncorrelated.
Additionally, from (3.112) it follows that if the random variables are orthogo-
nal, then the covariance is equal to the negative product of their mean values.

CX 1 X2 ¼
EfX1 gEfX2 g: (3.124)

3.3.3.1 Variance of the Sum of Random Variables

Consider the sum of two random variables X1 and X2, which is itself a random
variable X:
X ¼ X1 þ X2 : (3.125)

By applying the definition (2.333) of the variance of the random variable X and
using (3.88) and (3.125), we get:

2 2
s2X ¼ ðX
XÞ ¼ ðX1 þ X2
X1 þ X2 Þ
2 2
¼ ðX1
X1 Þ þ ðX2
X2 Þ þ 2ðX1
X1 ÞðX2
X2 Þ: (3.126)

The first two terms in (3.126) are the corresponding variances of the variables X1
and X2, respectively, while the third averaged product is the covariance.
Therefore, (3.126) reduces to:

s2X ¼ s2X1 þ s2X2 þ 2CX1 X2 : (3.127)

Equation (3.127) states that the variance of the sum of the variables X1 and X2 is
equal to the sum of the corresponding variances if their covariance is equal to zero
(i.e., the variables are either independent or uncorrelated).
Therefore, if the random variables X1 and X2 are either independent or uncorre-
lated ðCX 1 X2 ¼ 0Þ, then

s2X1 þX2 ¼ s2X1 þ s2X2 : (3.128)

The result (3.128) can be generalized to the sum of N either independent or

uncorrelated variables X1, . . ., XN:
X
N
s2P
N ¼ s2Xi : (3.129)
Xi i¼1
i¼1
178 3 Multidimensional Random Variables

3.3.4 Independence and Correlation

Random variables are independent if one variable does not have any influence
over the values of another variable, and vice versa. For nonrandom variables,
dependency means that if we know one variable, we can find the exact values of
another variable.
However, dependence between random variables can have different degrees of
dependency.
If we can express the relation between random variables X1 and X2 with an exact
mathematical expression, then this dependency is called functional dependency, as
seen for the example (X2 ¼ 2X1 + 2). In the opposite case, we do not have the exact
mathematical expression but the tendency, as for example, if X1 is the height and X2
is the weight of people in a population. In the majority of cases, higher values of X1
correspond to higher values of X2, but there is no mathematical relation to express
this relation.
To this end, the dependence between variables is expressed using some
characteristics, like covariance. However, the covariance contains the information,
not only of dependency of random variables, but also the information about the
dissipation of variables around their mean values. If, for example, the dissipations
of random variables X1 and X2 around their mean values were very small, then the
covariance CX 1 X2 would be small for any degree of dependency in the variables.
This problem is solved by introducing the correlation coefficient ðrX 1 X2 Þ:

C X1 X2 ðX1
X1 ÞðX2
X2 Þ X1 X2
X1 X2
rX1 X2 ¼ ¼ ¼ : (3.130)
sX1 sX2 sX1 sX2 sX1 sX2

What are the values that the correlation coefficient can take?
(a) The variables are equal
Consider a case in which the random variables are equal, X2 ¼ X1, and there-
fore, there is a maximum degree of dependency between them. From (3.130),
and using the definition of variance (2.333), we have:

2
CX1 X2 ðX1
X1 ÞðX1
X1 Þ ðX1
X1 Þ s2X1
rX1 X2 ¼ ¼ ¼ ¼ ¼ 1: (3.131)
sX1 sX2 sX1 sX1 s2X1 s2X1

Therefore, for the maximum dependency of random variables, the correlation

coefficient is equal to 1.
(b) Linear dependency of random variables
Consider the linear dependency for random variables,

X2 ¼ aX1 þ b; (3.132)

where a and b are deterministic constants. Figure 3.9 shows this dependency for
a > 0 and a < 0.
3.3 Expected Values and Moments 179

Fig. 3.9 Linear dependency

The covariance is equal to:

CX1 X2 ¼ ðX1
X1 ÞðaX1 þ b
aX1 þ bÞ ¼ ðX1
X1 ÞaðX1
X1 Þ

2
¼ aðX1
X1 Þ ¼ as2X1 : (3.133)

From (2.355), the variance of the random variable X2 is

s2X2 ¼ a2 s2X1 : (3.134)

Finally, using (3.133) and (3.134), we can obtain the correlation coefficient:

CX1 X2 as2X1 a 1 for a>0
rX1 X2 ¼ ¼ ¼ ¼ : (3.135)
sX1 sX2 sX1 jajsX1 jaj
1 for a<0

Therefore, the maximum absolute value of the correlation coefficient is equal

to 1 (if the variables are linearly dependent), as given in (3.135).
If

rX1 X2 > 0; (3.136)

then there is a positive correlation, as shown in Fig. 3.10a, in contrast, to the case
in which

rX1 X2 < 0; (3.137)

there is a negative correlation.

Note that in (3.136) and (3.137) there is no functional relation between variables,
but only tendency: if one variable increases, in the majority of cases, the other
variable either increases (positive correlation) or decreases (negative correlation).
Therefore, the coefficient of correlation presents the degree of linear dependency.
If there is no linear dependency between the variables (like in Fig. 3.11a) the
coefficient of correlation is equal to zero. As a consequence, the random variables
are dependent, but uncorrelated.
180 3 Multidimensional Random Variables

Fig. 3.10 Positive and negative correlations

Fig. 3.11 Dependent uncorrelated and independent variables

If the random variables are independent, then obviously there is no relation

between them (not even a linear relation), resulting in a zero value of the coefficient
of correlation, as shown in Fig. 3.11b.
From (3.135) to (3.137), the values of the correlation coefficient are:

1  rX1 X2  1: (3.138)

Example 3.3.4 Consider dependent random variables X and Y, where

Y ¼ X2 : (3.139)

Determine whether or not the random variables X and Y are correlated and find
the coefficient of correlation for the following two cases:
(a) The random variable X is uniform in the interval [
1, 1].
(b) The random variable X is uniform in the interval [0, 2].
3.3 Expected Values and Moments 181

Solution
(a) The expected values for the random variables X and Y are:

ð1
1 2 1
X ¼ 0; Y ¼ X2 ¼ x dx ¼ : (3.140)
2 3

1

From (3.121) and (3.140), the covariance is equal to:

CXY ¼ ðX
XÞðY
YÞ ¼ XðX2
1=3Þ ¼ X3
1=3X ¼ 0: (3.141)

The obtained result shows that the covariance is equal to zero, and thus the
coefficient of correlation (3.130) is also zero. As such, the random variables are
uncorrelated.
Therefore, variables X and Y are dependent and uncorrelated.
(b) In this case, the random variables are also dependent.
The corresponding mean values are:

ð2
1 2 4
X ¼ 1; Y¼ X2 ¼ x dx ¼ : (3.142)
2 3
0

From (3.121) and (3.142), the covariance is equal to:

CXY ¼ ðX
XÞðY
YÞ ¼ ðX
1ÞðX2
4=3Þ ¼ X3
X2
4=3X þ 4=3: (3.143)

To calculate (3.143), we first need the third moment of the random variable X:

ð2
1 3
X3 ¼ x dx ¼ 2: (3.144)
2
0

Placing (3.142) and (3.144) into (3.143), we get:

CXY ¼ 2
4=3
4=3 þ 4=3 ¼ 2=3: (3.145)

This result indicates that the random variables are correlated. To find the
coefficient of correlation we need the corresponding variances.
From (3.142), we have:

2
s2X ¼ X2
X ¼ 4=3
1 ¼ 1=3; (3.146)

2 2
s2Y ¼ Y 2
Y ¼ X4
X2 ; (3.147)
182 3 Multidimensional Random Variables

ð2
1 4
X4 ¼ x dx ¼ 16=5: (3.148)
2
0

Placing (3.148) and (3.142) into (3.147), we can obtain the variance of the
random variable Y:

s2Y ¼ 16=5
ð4=3Þ2 ¼ 1:422: (3.149)

The standard deviations are obtained from (3.146) and (3.149):

qffiffiffiffiffiffi pffiffiffiffiffiffiffiffi
sX ¼ s2X ¼ 1=3 ¼ 0:5774; (3.150)

qffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffi
sY ¼ s2Y ¼ 1:422 ¼ 1:1925: (3.151)

Using values for the covariance (3.145) and standard deviations (3.150) and
(3.151), we calculate the coefficient of correlation using (3.130):

CXY 2=3
rXY ¼ ¼ ¼ 0:9682: (3.152)
sX sY 0:5774  1:1925

As opposed to case (a), in case (b), the variables are dependent and correlated.
Based on the previous discussion, we can conclude that the dependence is a
stronger condition than correlation. That is, if the variables are independent, they
are also uncorrelated. However, if the random variables are dependent they can be
either correlated or uncorrelated, as summarized in Fig. 3.12.

INDEPENDENT R.V. DEPENDENT R.V.

fXY = fX fY

UNCORRELATED UNCORRELATED CORRELATED

CXY = 0 CXY = 0 CXY 0
XY
=0 XY
=0 XY
0

Positive Negative
Correlation Correlation
rXY >0 rXY <0

Fig. 3.12 Correlation and dependency between r.v.

3.4 Transformation of Random Variables 183

Therefore, if random variables are uncorrelated, they can be either dependent or

independent. However, there are some exceptions such as the case of uncorrelated
normal random variables. If normal random variables are uncorrelated it follows
that they are independent (see Chap. 4).

3.4 Transformation of Random Variables

3.4.1 One-to-One Transformation

Consider two random variables X1 and X2 with a known joint PDF fX 1 X2 ðx1 ; x2 Þ. As a
result of the transformation

Y1 ¼ g1 ðX1 ; X2 Þ
Y2 ¼ g2 ðX1 ; X2 Þ (3.153)

new random variables Y1 and Y2 are obtained, as shown in Fig. 3.13.

We are looking for the joint density of the transformed random variables (3.153).
The result depends on the type of transformation. Here we consider a simple case in
which the infinitesimal area in the (x1, x2) system has a one-to-one correspondence
to the infinitesimal area in the (y1, y2) system, as shown in Fig. 3.14. In other words,

Fig. 3.13 Transformation of two random variables

Fig. 3.14 Mapping from (x1, x2) space onto (y1, y2) space
184 3 Multidimensional Random Variables

the elementary area (dx1 dx2) is mapped one-to-one onto a corresponding infinites-
imal area (dy1 dy2). As a result, the corresponding probabilities are equal:

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g ¼ Pfy1 < Y1

 y1 þ dy1 ; y2 < Y2  y2 þ dy2 g: (3.154)

The probabilities in (3.154) can be expressed in terms of their corresponding

joint densities:

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2 ;

Pfy1 < Y1  y1 þ dy1 ; y2 < Y2  y2 þ dy2 g ¼ fY1 Y2 ðy1 ; y2 Þdy1 dy2 : (3.155)

From (3.154) and (3.155), we have:

fY1 Y2 ðy1 ; y2 Þdy1 dy2 ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.156)

The infinitesimal areas (dy1 dy2) and (dx1 dx2) are related as,

dy1 dy2
dx1 dx2 ¼ ; (3.157)
Jðx1 x2 Þ

where J(x1, x2) is the Jacobian of the transformation (3.153) [PAP65, p. 201]:
 
 @g1 @g1 
 
 @x2 :
Jðx1 ; x2 Þ ¼  @x1 (3.158)
 @g2 @g2 
 
@x1 @x2

Finally, from (3.156) and (3.157) we get:


fX1 X2 ðx1 ; x2 Þ
fY1 Y2 ðy1 ; y2 Þ ¼ ; (3.159)
jJðx1 ; x2 Þj x1 ¼g
1
1
ðy1 ;y2 Þ
x2 ¼g
1 ðy1 ;y2 Þ
2

where g
1
i , i ¼ 1, 2 is the inverse transformation of (3.153),

x1 ¼ g
1
1 ðy1 ; y2 Þ
(3.160)
x2 ¼ g
1
2 ðy1 ; y2 Þ:

Note that in (3.159) the absolute value of the Jacobian must be used because the
joint density cannot be negative as opposite to the Jacobian which can be either
positive or negative.
If for certain values y1, y2, there is no real solution (3.160), then

fY1 Y2 ðy1 ; y2 Þ ¼ 0: (3.161)

3.4 Transformation of Random Variables 185

Example 3.4.1 The joint density function of random variables X1 and X2 is

given as:

x21 þ x22
1

fX1 X2 ðx1 ; x2 Þ ¼ e 2s2 ;
1 < x1 < 1;
1 < x2 < 1: (3.162)
2ps2

Find the joint density function of the random variables Y1 and Y2 if,

X1 ¼ Y1 cos Y2 ;
(3.163)
X2 ¼ Y1 sin Y2 :

Solution The given transformation (3.163) can be rewritten as:

x2
y2 ¼ tan
1 ¼ g2 ðx1 ; x2 Þ
x1 (3.164)
qffiffiffiffiffiffiffiffiffiffiffiffiffiffi
y1 ¼ x21 þ x22 ¼ g1 ðx1 ; x2 Þ:

The Jacobian of the transformation (3.164), according to (3.158), is:

 
 pffiffiffiffiffiffiffiffiffiffiffiffiffiffi
x1
pffiffiffiffiffiffiffiffiffiffiffiffiffiffi 
x2

 x1 þ x2
2 2 x1 þ x2 
2 2
x21 þ x22 1 1
J ¼  ¼
 pffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ : (3.165)

x 2 x 1  ðx 2 þ x 2 Þ x2 þ x2 x 2 þ x2 y 1
 x2 þ x2 x 2 þ x2 
1 2 1 2 1 2
1 2 1 2

Using (3.159), we get:

8
< y1
y21
fY1 Y2 ðy1 ; y2 Þ ¼ 2ps2 e 2s
2
for y1  0; 0  y2  2p; (3.166)
:
0 otherwise:

In the following, we demonstrate how the expression (3.159) can be used to

obtain the PDF of a random variable which is the function of two random variables:

Y ¼ gðX1 ; X2 Þ: (3.167)

In order to apply the expression (3.159), we introduce the auxiliary variable r

[PAP65, p. 204], as shown in Fig. 3.15.
The variable r will later be eliminated and therefore we chose it for
convenience:

r ¼ X1 or r ¼ X2 : (3.168)
186 3 Multidimensional Random Variables

Y
X1
X2

Fig. 3.15 Transformation of two input random variables

Now we have the transformation:

Y ¼ gðX1 ; X2 Þ;
r ¼ X1 ; (3.169)

or
Y ¼ gðX1 ; X2 Þ:
r ¼ X2 : (3.170)

The joint PDF fYr(y, z) is obtained from (3.159). Finally, the required PDF fY(y)
is obtained from fYr(y, z), using (3.42a, 3.42b):
ð
1

fY ðyÞ ¼ fYr ðy; zÞ dz: (3.171)

1

Example 3.4.2 Find the PDF of the variable Y

Y ¼ X1 X 2 ; (3.172)

if it is known then the joint PDF is fX 1 X2 ðx1 ; x2 Þ.

Solution The auxiliary variable

r ¼ X1 (3.173)

is introduced, resulting in the following transformation:

y ¼ x1 x2 :
z ¼ x1 : (3.174)

From here,

x1 ¼ z; x2 ¼ y=z: (3.175)
3.4 Transformation of Random Variables 187

Using (3.159), (3.158), and (3.175) we get:

 
 x2 x1 
J¼   ¼
x1 ¼
z; (3.176)
1 0

and

fX1 X2 ðx1 ; x2 Þ fX1 X2 ðz; y=zÞ

fYr ðy; zÞ ¼ ¼ : (3.177)
jzj jzj

Finally, the desired PDF is obtained from (3.171) and (3.174):

ð
1
1
fY ðyÞ ¼ fX X ðz; y=zÞdz: (3.178)
j zj 1 2

1

3.4.2 Nonunique Transformation

In this case, the infinitesimal area (dy1 dy2) corresponds to two or more infinitesi-
mal areas (dx1 dx2), resulting in:
P 
fX1 X2 ðxi1 ; xi2 Þ

fY1 Y2 ðy1 ; y2 Þ ¼ i
  
Jðxi ; xi Þ xi ¼ g
1 ðy ; y Þ : (3.179)
1 2 1 1 1 2
xi2 ¼ g
1
2 ðy 1 ; y2 Þ

Example 3.4.3 Find the joint PDF of the random variables Y1 and Y2,
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
X1
Y1 ¼ X12 þ X22 ; Y2 ¼ ; (3.180)
X2

if the given joint PDF of the random variables X1 and X2 is given as:

x21 þ x22
1

fX1 X2 ðx1 ; x2 Þ ¼ e 2s2 : (3.181)
2ps2

Solution From (3.180), we have:

qffiffiffiffiffiffiffiffiffiffiffiffiffiffi
x1
y1 ¼ x21 þ x22 ; y2 ¼ : (3.182)
x2
188 3 Multidimensional Random Variables

The Jacobian of the transformation (3.180) is:

  2
 x1 x2 
 pffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffi  x1
þ1
 x þx
2 2 x1 þ x2 
2 2
Jðx1 ; x2 Þ ¼ 
x2
1
1
2
x1  ¼
p ffiffiffiffiffiffiffiffiffiffiffiffiffiffi : (3.183)

2  x21 þ x22
 x2 x2

Placing (3.182) into (3.183), we get:

y22 þ 1
Jðy1 ; y2 Þ ¼
: (3.184)
y1

Equations (3.182) have two solutions for x1 and x2:

y1 y2 y1
x11 ¼ pffiffiffiffiffiffiffiffiffiffiffiffi2ffi ; x12 ¼ pffiffiffiffiffiffiffiffiffiffiffiffi2ffi ;
1 þ y2 1 þ y2
y1 y2 y1 (3.185)
x1 ¼
pffiffiffiffiffiffiffiffiffiffiffiffi2ffi ; x2 ¼
pffiffiffiffiffiffiffiffiffiffiffiffi2ffi :
2 2
1 þ y2 1 þ y2

From (3.179), (3.181), and (3.185), it follows:

fX1 X2 x11 ; x12 þ fX1 X2 x21 ; x22

fY1 Y2 ðy1 ; y2 Þ ¼
jJðy1 ; y2 Þj
y21 (3.186)
y1 y1 1
2
¼ 2fX X ðy1 ; y2 Þ ¼ e 2s :
1 þ y22 1 2 1 þ y22 ps2

Finally, we have:
8
>
< y21
y1 1
2
fY1 Y2 ðy1 ; y2 Þ ¼ e 2s for y1  0; (3.187)
: 1 þ y2 ps
> 2 2
0 for y1 < 0:

3.4.3 Generalization for N Variables

Given N random variables

X1 ; X2 ; . . . ; XN ; (3.188)

with the joint density function fX1 X2 ;...;XN ðx1 ; x2 ; . . . ; xN Þ, are transformed into new
random variables

Y1 ; Y2 ; . . . ; YN ; (3.189)
3.5 Characteristic Functions 189

where an unique transformation of the input variables (3.189) is defined as:

Y1 ¼ g1 ðX1 ; . . . ; XN Þ
Y2 ¼ g2 ðX1 ; . . . ; XN Þ
.. (3.190)
.
YN ¼ gN ðX1 ; . . . ; XN Þ

Similarly as in (3.157), we have:

dy1 ; . . . ; dyN
dx1 ; . . . ; dxN ¼ ; (3.191)
Jðx1 ; . . . ; xN Þ

where J is the Jacobian of the transformation (3.190), defined as:

 
 @g1 ::: @g1 
 @x1 @xN 
 .. :
Jðx1 ; . . . ; xN Þ ¼  ... ..
. .  (3.192)
 @gN
 @x ::: @g
@x
N 
1 N

Similarly, like (3.159), using (3.190)–(3.192), we get:


fX1 X2 ;...;XN ðx1 ; x2 ;.. .;xN Þ
fY1 Y2 ;...;YN ðy1 ; y2 ;.. .;yN Þ ¼ x ¼g
1 ðy ;y ;...;y Þ : (3.193)
jJðx1 ;x2 ; ...; xN Þj  x1 ¼g1
1 ðy1 ;y2 ;...;yN Þ
2 2 1 2 N
..
.
xN ¼g
1 ðy1 ;y2 ;...;yN Þ
N

3.5 Characteristic Functions

3.5.1 Definition

The joint characteristic function of two random variables X1 and X2, denoted as
fX1 X2 ðo1 ; o2 Þ, is defined as the expected value of the complex function
ejðo1 X1 þo2 X2 Þ ,
n o
fX1 X2 ðo1 ; o2 Þ ¼ E ejðo1 X1 þo2 X2 Þ : (3.194)

According to (3.87), the expression (3.194) is equal to:

ð
1 ð
1

fX1 X2 ðo1 ; o2 Þ ¼ ejðo1 x1 þo2 x2 ÞfX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.195)

1
1

The obtained expression (3.195) can be interpreted with the exception of

the exponent sign, as a two-dimensional Fourier transform of the joint PDF
190 3 Multidimensional Random Variables

fX1 X2 ðx1 ; x2 Þ. This means that using the inverse two-dimensional Fourier transform,
one can obtain the joint PDF from its joint characteristic function, as shown in the
following expression:
ð
1 ð
1
1
fX1 X2 ðx1 ; x2 Þ ¼ fX1 X2 ðo1 ; o2 Þe
jðo1 x1 þo2 x2 Þdo1 do2 : (3.196)
ð2pÞ2

1
1

If random variables X1 and X2 are independent, then their joint density is equal to
the product of the marginal densities. Thus the joint characteristic function (3.195)
becomes:
ð
1 ð
1

fX1 X2 ðo1 ; o2 Þ ¼ ejo1 x1 ejo2 x2 fX1 ðx1 ÞfX2 ðx2 Þdx1 dx2 ¼

1
1
ð
1 ð
1

ejo1 x1 fX1 ðx1 Þdx1 ejo2 x2 fX2 ðx2 Þdx2 ¼ fX1 ðo1 ÞfX2 ðo2 Þ: (3.197)

1
1

Therefore, for the independent random variables, the joint characteristic function
is equal to the product of the marginal characteristic functions. The reverse is also
true, that is if the joint characteristic function is equal to the product of marginal
characteristic functions, then the corresponding random variables are independent.
The marginal characteristic functions are obtained by making o1 or o2 equal to
zero in the joint characteristic function, as demonstrated in (3.198):

fX1 ðo1 Þ ¼ fX1 X2 ðo1 ; 0Þ;

fX2 ðo2 Þ ¼ fX1 X2 ð0; o2 Þ: (3.198)

3.5.2 Characteristic Function of the Sum

of the Independent Variables

Consider the sum of two random variables X1 and X2,

Y ¼ X 1 þ X2 : (3.199)

The characteristic function of the variable Y, according to (2.386) and the

definition (3.194), is equal to:

fY ðoÞ ¼ EfejoðX1 þX2 Þ g ¼ fX1 X2 ðo; oÞ: (3.200)

3.5 Characteristic Functions 191

If the random variables X1 and X2 are independent, then the joint characteristic
function of their sum is equal to the product of the marginal characteristic functions,

fY ðoÞ ¼ fX1 ðoÞfX2 ðoÞ: (3.201)

This result can be applied for the sum of N independent random variables Xi,

X
N
Y¼ Xi : (3.202)
i¼1

The joint characteristic function of (3.202) is

Y
N
fY ðoÞ ¼ fXi ðoÞ: (3.203)
i¼1

Example 3.5.1 Find the characteristic function of the random variable Y, where

Y ¼ aX1 þ bX2 (3.204)

and where X1, and X2 are independent.

Solution Using (2.386) and the definition of the characteristic function (3.194) and
the condition of independence (3.201), we have:

fY ðoÞ ¼ EfejoðaX1 þbX2 Þ g ¼ EfejoaX1 gEfejobX2 g ¼ fX1 ðaoÞfX2 ðboÞ: (3.205)

Example 3.5.2 The random variables X1 and X2 are independent. Find the joint
characteristic function of the variables X and Y, as given in the following equations:

X ¼ X1 þ 2X2 ;
Y ¼ 2X1 þ X2 : (3.206)

Solution From the definition (3.194), and using (3.206), we have:

fXY ðo1 ; o2 Þ ¼ Efejðo1 Xþo2 Y g ¼ Efejo1 ðX1 þ2X2 Þþjo2 ð2X1 þX2 Þ g
¼ EfejX1 ðo1 þ2o2 ÞþjX2 ð2o1 þo2 Þ g: (3.207)

Knowing that the random variables X1 and X2 are independent from (3.207) and
(3.201), we arrive at:

fXY ðo1 ; o2 Þ ¼ EfejX1 ðo1 þ2o2 Þ gEfejX2 ð2o1 þo2 Þ g

¼ fX1 ðo1 þ 2o2 ÞfX2 ð2o1 þ o2 Þ: (3.208)
192 3 Multidimensional Random Variables

3.5.3 Moment Theorem

Using the moment theorem for one random variable, as an analogy, we arrive at
the moment theorem that finds joint moments mnk from the joint characteristic
function, as

nþk @ nþk fX1 X2 ðo1 ; o2 Þ
mnk ¼ ð
jÞ o1 ¼ 0 : (3.209)
@o1 n @o2 k
o2 ¼ 0

3.5.4 PDF of the Sum of Independent Random Variables

The random variable Y is equal to the sum of the independent random variables
X1 and X2,

Y ¼ X 1 þ X2 : (3.210)

The PDF of the variable Y (see (2.387)) is expressed by its characteristic

function as,

ð
1
1
fY ðyÞ ¼ fY ðoÞe
joy do: (3.211)
2p

1

Placing (3.201) into (3.211), we have:

ð
1
1
fY ðyÞ ¼ fX1 ðoÞfX2 ðoÞe
joy do: (3.212)
2p

1

By applying the relation (2.386) to the characteristic function fX1 ðoÞ in (3.212),
and interchanging the order of the integrations, we arrive at:

ð
1 ð
1
1
fY ðyÞ ¼ fX2 ðoÞe
joy do fX1 ðx1 Þejox1 dx1
2p

1
1
ð
1 ð
1
1
¼ fX1 ðx1 Þdx1 fX2 ðoÞe
joðy
x1 Þ do: (3.213)
2p

1
1
3.5 Characteristic Functions 193

Using (2.387), we get:

ð
1
1
fX2 ðoÞe
joðy
x1 Þ do ¼ fX2 ðy
x1 Þ; (3.214)
2p

1

From (3.213) and (3.214), we arrive at:

ð
1

fY ðyÞ ¼ fX1 ðx1 ÞfX2 ðy
x1 Þdx1 : (3.215)

1

This expression can be rewritten as:

ð
1

fY ðyÞ ¼ fX2 ðx2 Þ fX1 ðy
x2 Þdx2 : (3.216)

1

The expressions (3.215) and (3.216) present the convolution of the PDFs of
random variables X1 and X2, and can be presented as:

fY ðyÞ ¼ fX1 ðx1 Þ  fX2 ðx2 Þ ¼ fX2 ðx2 Þ  fX1 ðx1 Þ; (3.217)

where * stands for the convolution operation.

This result can be generalized for the sum of N independent variables Xi,

X
N
Y¼ Xi ; (3.218)
i¼1

fY ðyÞ ¼ fX1 ðx1 Þ  fX2 ðx2 Þ      fXN ðxN Þ: (3.219)

Example 3.5.3 Two resistors R1 and R2 are in a serial connection (Fig. 3.16a). Each
of them randomly changes its value in a uniform way for 10% about its nominal
value of 1,000 O. Find the PDF of the equivalent resistor R,

R ¼ R1 þ R2 (3.220)

if both resistors have uniform density in the interval [900, 1,100] O.

Solution Denote the equivalent resistor R as a random variable X and the particular
resistors R1 and R2, as the random variables X1 and X2. Then,

X ¼ X1 þ X 2 : (3.221)
194 3 Multidimensional Random Variables

Fig. 3.16 Convolution of uniform PDFs

From (3.217), the PDF of the random variable X is equal to the convolution of
the PDFs of the random variables X1 and X2. In this case, it is convenient to present
the convolution graphically, as shown in Fig. 3.16.

3.6 Numerical Exercises

Exercise 3.1 The joint random variables X1 and X2 are defined in a circle of a
radius r ¼ 2, as shown in Fig. 3.17. Their joint PDF is constant inside the circle.
Find and plot the joint PDF and the marginal PDFs. Determine whether or not the
random variables X1 and X2 are independent.
Answer The area A in Fig. 3.17 is:

A ¼ r 2 p ¼ 4p: (3.222)

The volume below the joint density is the height of the cylinder which, according
to (3.37) must be unity, is shown in Fig. 3.18.
The joint density is:

1=4p for x21 þ x22  4;
fX1 X2 ðx1 ; x2 Þ ¼ (3.223)
0 otherwise:
3.6 Numerical Exercises 195

Fig. 3.17 Range in Exercise 3.1

Fig. 3.18 Joint PDF in Exercise 3.1

From (3.223), we have:

qffiffiffiffiffiffiffiffiffiffiffiffiffi
x11;2 ¼ 4
x22 : (3.224)
qffiffiffiffiffiffiffiffiffiffiffiffiffi
x21;2 ¼ 4
x21 : (3.225)

Using (3.42a, 3.42b), and (3.223)–(3.225), we get the marginal densities:

8 pffiffiffiffiffiffiffiffiffiffiffiffi
>
< Ð2
x2
1 4
x21
fX1 X2 ðx1 ; x2 Þdx2 ¼ ðx21
x22 Þ ¼ for jx1 j  2;
fX1 ðx1 Þ ¼ x2 4p 2p
>
: 1
0 otherwise,
(3.226)
196 3 Multidimensional Random Variables

8 pffiffiffiffiffiffiffiffiffiffiffiffi
>
< Ð2
x1
1 4
x22
fX1 X2 ðx1 ; x2 Þdx1 ¼ ðx11
x12 Þ ¼ for jx2 j  2;
fX2 ðx2 Þ ¼ 4p 2p
> x1
: 1
0 otherwise:
(3.227)

The marginal PDFs are shown in Fig. 3.19.

The random variables are dependent because their joint PDF cannot be presented
as the product of the marginal PDFs.
Exercise 3.2 The joint random variables are defined in the area A, as shown in
Fig. 3.20. Find and plot the joint density function and marginal PDFs if the joint
PDF is constant. Find the probability P{0 < x1 < 2, 0 < x2 < 1} as well. Deter-
mine whether or not the random variables are dependent.
Answer The area A is: A ¼ 3  2 ¼ 6: (3.228)

From (3.36), the joint PDF is:


1=6 for
1 < x1 < 2;
1 < x2 < 1;
fX1 X2 ðx1 ; x2 Þ ¼ (3.229)
0 otherwise:

This is shown in Fig. 3.21.

Fig. 3.19 Marginal PDFs in Exercise 3.1

Fig. 3.20 Range of variables in Exercise 3.2

3.6 Numerical Exercises 197

Fig. 3.21 Joint PDF in Exercise 3.2

The marginal PDFs are

8
< Ð1
1=6 dx2 ¼ 1=3 for
1 < x1 < 2;
fX1 ðx1 Þ ¼
1 (3.230)
:
0 otherwise:
8
< Ð2
1=6 dx1 ¼ 1=2 for
1 < x2 < 1;
fX2 ðx2 Þ ¼
1 (3.231)
:
0 otherwise:

The marginal densities are shown in Fig. 3.22.

The desired probability is:

ð2 ð1
Pf0 < x1 < 2; 0 < x2 < 1g ¼ 1=6 dx1 dx2 ¼ 1=3: (3.232)
0 0

Fig. 3.22 Marginal densities in Exercise 3.2

198 3 Multidimensional Random Variables

The random variables are independent because the joint PDF can be presented as
a product of the corresponding marginal PDFs.
Exercise 3.3 Find the conditional density fX1 ðx1 jx2 Þ for the joint variables from
Exercise 3.1 and find whether the variables are dependent.
Answer From (3.50), (3.223), and (3.227) we have:

fX1 X2 ðx1 ; x2 Þ
fX1 ðx1 jX2 ¼ x2 Þ ¼
fX2 ðx2 Þ
( pffiffiffiffiffiffiffiffiffiffiffiffiffi
p1ffiffiffiffiffiffiffi2ffi for jx1 j < 4
x22 ; jx2 j < 2;
¼ 2 4
x2 (3.233)
0 otherwise:

The conditional density (3.233) is different from fX1 ðx1 Þ, given in (3.226), thus
confirming that the variables are dependent.
Exercise 3.4 Find the conditional density fX1 ðx1 jX2 ¼ x2 Þ for the joint density from
Exercise 3.2 and find whether the variables are independent.
Answer From (3.50), (3.223), and (3.227), we have:
 1=6
fX X ðx1 ; x2 Þ ¼ 1=3 for
1 < x1 < 2;
fX1 ðx1 jX2 ¼ x2 Þ ¼ 1 2 ¼ 1=2 (3.234)
fX2 ðx2 Þ 0 otherwise:

The result (3.234) shows that

fX1 jX2 ðx1 jx2 Þ ¼ fX1 ðx1 Þ; (3.235)

thus confirming that the variables X1 and X2 are independent.

Exercise 3.5 In a two-dimensional discrete random variable (X1 X2), the random
variable X1 has two possible values:

x11 ¼ 1; x12 ¼
1; (3.236)

while the random variable X2 has three possible values:

x21 ¼ 0; x22 ¼ 2; x23 ¼ 5: (3.237)

Find the corresponding marginal distributions and densities if the following

probabilities are known:

Pfx11 ; x21 g ¼ 0:1; Pfx12 ; x21 g ¼ 0:15:

Pfx11 ; x22 g ¼ 0:15; Pfx12 ; x22 g ¼ 0:25: (3.238)
Pfx11 ; x23 g ¼ 0:20; Pfx12 ; x22 g ¼ 0:15:
3.6 Numerical Exercises 199

Answer From (3.238), we have:

X
3
PfX1 ¼ x11 g ¼ PfX1 ¼ x11 ; X2 ¼ x2j g ¼ 0:1 þ 0:15 þ 0:2 ¼ 0:45:
j¼1

X
3
PfX1 ¼ x12 g ¼ PfX1 ¼ x12 ; X2 ¼ x2j g ¼ 0:15 þ 0:25 þ 0:15 ¼ 0:55: (3.239)
j¼1

The marginal distribution and density are, respectively:

FX1 ðx1 Þ ¼ 0:45uðx1
1Þ þ 0:55uðx1 þ 1Þ; (3.240)

fX1 ðx1 Þ ¼ 0:45dðx1
1Þ þ 0:55dðx1 þ 1Þ: (3.241)

Similarly,

X
2
PfX2 ¼ x21 g ¼ PfX1 ¼ x1j ; X2 ¼ x21 g ¼ 0:1 þ 0:15 ¼ 0:25;
j¼1

X
2
PfX2 ¼ x22 g ¼ PfX1 ¼ x1j ; X2 ¼ x22 g ¼ 0:15 þ 0:25 ¼ 0:4;
j¼1

X
2
PfX2 ¼ x23 g ¼ PfX1 ¼ x1j ; X2 ¼ x23 g ¼ 0:2 þ 0:15 ¼ 0:4; (3.242)
j¼1

FX2 ðx2 Þ ¼ 0:25uðx2 Þ þ 0:4uðx2
2Þ þ 0:35uðx2
5Þ: (3.243)

fX2 ðx2 Þ ¼ 0:25dðx2 Þ þ 0:4dðx2
2Þ þ 0:35dðx2
5Þ: (3.244)

Exercise 3.6 Find the conditional distributions,

FX1 ðx1 jX2 ¼ x21 ¼ 0Þ; and FX2 ðx2 jX1 ¼ x11 ¼ 1Þ; (3.245)

for the random variables from Exercise 3.5.

Answer From (3.46), we have:

P
2
PfX1 ¼ x1i ; X2 ¼ x21 ¼ 0guðx1
x1i Þ
i¼1
FX1 ðx1 jX2 ¼ x21 ¼ 0Þ ¼ : (3.246)
PfX2 ¼ x21 ¼ 0g

From (3.238), we have:

PfX1 ¼ 1; X2 ¼ 0g ¼ 0:1; PfX1 ¼
1; X2 ¼ 0g ¼ 0:15: (3.247)

200 3 Multidimensional Random Variables

From (3.242), it follows:

PfX2 ¼ 0g ¼ 0:25: (3.248)

Finally, from (3.246)–(3.248) we get the desired conditional density:

FX1 ðx1 jX2 ¼ x21 ¼ 0Þ ¼ 0:1=0:25uðx1
1Þ þ 0:15=0:25uðx1 þ 1Þ: (3.249)

Similarly, we have:

FX2 ðx2 jX1 ¼ x11 ¼ 1Þ ¼ 0:1=0:45uðx2 Þ þ 0:15=0:45uðx2
2Þ

þ 0:2=0:45uðx2
5Þ: (3.250)

Exercise 3.7 Two-dimensional random variable (X1, X2), has a uniform joint PDF
in the area A, as shown in Fig. 3.23. Find the marginal PDFs.
Answer Knowing that the area A ¼ 1/2, the corresponding joint PDF is:

2 for ðx1 ; x2 Þ 2 A;
fX1 X2 ðx1 ; x2 Þ ¼ (3.251)
0 otherwise:

The marginal density is:

ð
1

fX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx2 : (3.252)

1

From Fig. 3.23, we see that x2 changes from 0 to x1, where x1 is in the interval
[0, 1]:
ð
1 xð1

fX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx2 ¼ 2dx2 ¼ 2x1 : (3.253)

1 0

Fig. 3.23 Range in Exercise 3.7

3.6 Numerical Exercises 201

Therefore,

2x1 for 0 < x1 < 1;
fX1 ðx1 Þ ¼ (3.254)
0 otherwise:

Similarly, we have:
ð
1 xð2

fX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 ¼ 2dx1 ¼ 2x2 (3.255)

1 0

and

2x2 for 0 < x2 < 1;
fX2 ðx2 Þ ¼ (3.256)
0 otherwise:

Exercise 3.8 The random variables X1 and X2 are related as:

X2 ¼
2X1 þ 5: (3.257)

Determine whether or not the random variables are orthogonal and correlated if,

EfX1 g ¼ 2; and s2X1 ¼ 1: (3.258)

Answer The random variables are orthogonal if

EfX1 X2 g ¼ RX 1 X2 ¼ 0: (3.259)

From (3.257), we have:

 
RX 1 X2 ¼Efð
2X1 þ 5ÞX1 g ¼
2E X12 þ 5EfX1 g: (3.260)

From (3.258), it follows:

 
s2X1 ¼ 1 ¼ E X12
EfX1 g2 ¼ EfX12 g
4 (3.261)

and
 
E X12 ¼ 5: (3.262)

Finally,

RX 1 X2 ¼
2  5 þ 5  2 ¼ 0: (3.263)

This result confirms that the variables X1 and X2 are orthogonal.

From (3.257), we can see that there is a linear relation between the random
variables X1 and X2. It results in a maximum correlation factor. Due to the
202 3 Multidimensional Random Variables

coefficient
2, the correlation is negative and the correlation factor is equal to
1.
This is confirmed in the following numerical calculation.
From (3.130), the correlation coefficient is:

C X1 X2 ðX1
2ÞðX2
X2 Þ
rX 1 X 2 ¼ ¼
sX1 sX2 sX1 sX2
X1 X2
2X2
X1 X2 þ 2X2 X1 X2
X1 X2
¼ ¼ : (3.264)
sX1 sX2 sX1 sX2

From (3.259) and (3.263), we have:

X1 X2 ¼ 0: (3.265)

From (3.257), we find the mean value and the variance of the variable X2:

X2 ¼
2X1 þ 5 ¼ 1;
s2X2 ¼ 4s2X1 ¼ 4: (3.266)

Finally, placing (3.265) and (3.266) into (3.264), we arrive at:

2  1
rX1 X2 ¼ ¼
1: (3.267)
12

Exercise 3.9 The random variable X is uniform in the interval [0, 2p]. Show that
the random variables Y and Z are dependent and uncorrelated, if

Y ¼ sin X; Z ¼ cos X: (3.268)

Answer From (3.268), we have: Y 2 þ Z 2 ¼ 1: (3.269)

This indicates that the variables are dependent. We also notice that the depen-
dence is squared, that is, it does not contain any degree of linear relation. Therefore,
the correlation is zero and the random variables are uncorrelated. This is confirmed
in the following calculation.
The mean values and the autocorrelation are:

ð
2p
1
Y¼ sin x dx ¼ 0;
2p
0
2ðp
1
Z¼ cos x dx ¼ 0; (3.270)
2p
0
2ðp 2ðp
1 1
YZ ¼ sin x cos x dx ¼ sin 2x dx ¼ 0:
2p 4p
0 0
3.6 Numerical Exercises 203

Using (3.270) in the middle, the covariance is:

CYZ ¼ ðY
YÞðZ
ZÞ ¼ YZ ¼ 0: (3.271)

From here, it follows that the random variables are uncorrelated.

Exercise 3.10 The discrete random variables X1 and X2 have the possible values:

1, 0, 1. Find the mean values for the variables X1 and X2, if it is known:

PfX1 ¼ 1; X2 ¼ 1g ¼ 0; PfX1 ¼ 1; X2 ¼ 0g ¼ 1=15; PfX1 ¼ 1;X2 ¼
1g ¼ 4=15;

PfX1 ¼ 0; X2 ¼ 1g ¼ 2=15; PfX1 ¼ 0; X2 ¼ 0g ¼ 2=15; PfX1 ¼ 0;X2 ¼
1g ¼ 1=15;
PfX1 ¼
1;X2 ¼ 1g ¼ 0; PfX1 ¼
1;X2 ¼ 0g ¼ 1=15; PfX1 ¼
1; X2 ¼
1g ¼ 4=15:
(3.272)

Answer In order to find the mean values for X1 and X2, we need the corresponding
probabilities P(X1i) and P(X2j), i ¼ 1, . . ., 3; j ¼ 1, . . ., 3.
From (1.67), we have:

PðX1 ¼ 1Þ ¼ PfX1 ¼ 1; X2 ¼ 1g þ PfX1 ¼ 1; X2 ¼ 0g

þ PfX1 ¼ 1; X2 ¼
1g ¼ 1=15 þ 4=15 ¼ 5=15 ¼ 1=3: (3.273)

PðX1 ¼ 0Þ ¼ PfX1 ¼ 0; X2 ¼ 1g þ PfX1 ¼ 0; X2 ¼ 0g þ PfX1 ¼ 0; X2 ¼
1g

¼ 2=15 þ 2=15 þ 1=15 ¼ 5=15 ¼ 1=3:
(3.274)

PðX1 ¼
1Þ ¼ PfX1 ¼
1;X2 ¼ 1g þ PfX1 ¼
1;X2 ¼ 0 þ PfX1 ¼
1;X2 ¼
1gg

¼ 1=15 þ 4=15 ¼ 5=15 ¼ 1=3:
(3.275)

PðX2 ¼ 1Þ ¼ PfX2 ¼ 1; X1 ¼ 1g þ PfX2 ¼ 1; X1 ¼ 0g

(3.276)
þPfX2 ¼ 1; X1 ¼
1g ¼ 2=15:

PðX2 ¼ 0Þ ¼ PfX2 ¼ 0; X1 ¼ 1g þ PfX2 ¼ 0; X1 ¼ 0g þ PfX2 ¼ 0; X1 ¼
1g

¼ 1=15 þ 2=15 þ 1=15 ¼ 4=15:
(3.277)

PðX2 ¼
1Þ ¼ PfX2 ¼
1; X1 ¼ 1g þ PfX2 ¼
1; X1 ¼ 0g

þ PfX2 ¼
1; X1 ¼
1g ¼ 4=15 þ 1=15 þ 4=15 ¼ 9=15:
(3.278)

From (3.273) to (3.275), we get:

EfX1 g ¼ 1 PfX1 ¼ 1g þ 0 PfX1 ¼ 0g þ ð
1Þ PfX1 ¼
1g ¼ 1=3
1=3 ¼ 0: (3.279)

204 3 Multidimensional Random Variables

Similarly from (3.276) to (3.278), we obtain:

EfX2 g ¼ 1 P fX2 ¼ 1g þ 0 P fX2 ¼ 0g þ ð
1Þ P fX2 ¼
1g

¼ 2=15
9=15 ¼
7=15: (3.280)

Exercise 3.11 The discrete random variable X1 has 0 and 1 as its discrete values,
whereas X2 has 0, 1, and
1 as its discrete values. Find the mean value and the
variance of the random variable X, if

X ¼ 2X1 þ X22 : (3.281)

The corresponding probabilities are:

PfX1 ¼ 0; X2 ¼ 0g ¼ 0:2; PfX1 ¼ 0;X2 ¼ 1g ¼ 0; PfX1 ¼ 0; X2 ¼
1g ¼ 0:1;

PfX1 ¼ 1; X2 ¼ 0g ¼ 0:3; PfX1 ¼ 1;X2 ¼ 1g ¼ 0:2; PfX1 ¼ 1;X2 ¼
1g ¼ 0:2:
(3.282)

Answer The variance of the random variable X is:

2 2 2
s2X ¼ X2
X ¼ ð2X1 þ X22 Þ
ð2X1 þ X22 Þ : (3.283)

From (3.281), we get the mean value of the variable X:

2 X
X 3
X ¼ 2X1 þ X22 ¼ ð2x1i þ x22j ÞPfX1 ¼ x1i ; X2 ¼ x2j g
i¼1 j¼1

¼ 0:1 þ 2  0:3 þ ð2 þ 1Þ  0:2 þ ð2 þ 1Þ  0:2 ¼ 0:1 þ 0:6 þ 0:6 þ 0:6 ¼ 1:9:

(3.284)

Similarly, the mean squared value of X is:

2
2 X
X 3
X2 ¼ ð2X1 þ X22 Þ ¼ ð2x1i þ x22j Þ2 PfX1 ¼ x1i ; X2 ¼ x2j g
i¼1 j¼1

¼ 0:1 þ 1:2 þ 1:8 þ 1:8 ¼ 4:9: (3.285)

Placing (3.284) and (3.285) into (3.283), we arrive at:

2
s2X ¼ X2
X ¼ 4:9
1:92 ¼ 1:29: (3.286)
3.6 Numerical Exercises 205

Exercise 3.12 The random variables X1 and X2 are independent and have the
density functions

fX1 ðx1 Þ ¼ e
x1 uðx1 Þ;

fX2 ðx2 Þ ¼ e
x2 uðx2 Þ: (3.287)

Determine whether or not the random variables

Y1 ¼ X1 þ X2 (3.288)

and

X1
Y2 ¼ (3.289)
X 1 þ X2
are independent.
Answer From (3.288) to (3.289), we write:

y1 ¼ x1 þ x2 ;
x1
y2 ¼ : (3.290)
x1 þ x2

The random variables X1 and X2 are independent, and their joint PDF from
(3.287) is:

e
ðx1 þx2 Þ for x1  0; x2  0;
fX1 X2 ðx1 ; x2 Þ ¼ fX1 ðx1 ÞfX2 ðx2 Þ ¼ (3.291)
0 otherwise:

Using (3.289), we can present the joint density (3.291) in the following form:

e
y1 for y1  0; 0  y2  1;
fX1 X2 ðy1 ; y2 Þ ¼ (3.292)
0 otherwise:

The Jacobian of the transformation (3.290) is:

 
 1 1  1 1
J ¼  x2
x1  ¼
¼
: (3.293)
ðx1 þx2 Þ2 ðx1 þx2 Þ 2 x 1 þ x 2 y 1

The joint density of Y1 and Y2 is obtained from (3.159), (3.291), and (3.293) as:

fX1 X2 ðy1 ; y2 Þ e
y1

fY1 Y2 ðy1 ; y2 Þ ¼ ¼  

jJðy1 ; y2 Þj 
1  y1

y1
y1 e for y1  0; 0  y2  1;
¼ (3.294)
0 otherwise:
206 3 Multidimensional Random Variables

The joint density (3.294) can be rewritten as:

fY1 Y2 ðy1 ; y2 Þ ¼ fY1 ðy1 ÞfY2 ðy2 Þ; (3.295)

where

fY1 ðy1 Þ ¼ y1 e
y1 ; for y1  0;

fY2 ðy2 Þ ¼ 1; for 0  y2  1: (3.296)

Therefore, from (3.295) it follows that the random variables Y1 and Y2 are also
independent.
Exercise 3.13 The random variables X1 and X2 are independent and have the
following density functions:
 
e
x1 for x1 > 0 e
x2 for x2 > 0
fX1 ðx1 Þ ¼ ; f ðx Þ ¼ : (3.297)
0 otherwise X2 2 0 otherwise

Find the PDF of the random variable X, if

X1
X¼ : (3.298)
X2

Solution We have two input random variables and one output variable. In order to
apply the expression (3.159) we must first introduce the auxiliary variable Y,

Y ¼ X1 : (3.299)

Now we have the transformation defined in the following two equations:

x1
x¼ ;
x2
y ¼ x1 : (3.300)

This system of equations (3.300) has one solution:

x11 ¼ y; and x21 ¼ y=x: (3.301)

The Jacobian of the transformation (3.300) is:

1 

xx12  x1 y x2
J ¼  x2 2
 ¼ 2¼ 2 2¼ : (3.302)
1 0 x2 y =x y
3.6 Numerical Exercises 207

Using (3.159), the joint density of the variables X and Y is:

fX1 X2 ðx; yÞ y
fXY ðx; yÞ ¼ ¼ 2 e
y e
y=x ; y > 0; x > 0: (3.303)
jJðx; yÞj x

The desired density is obtained from (3.42a):

ð
1 ð
1 ð
1
y
yð1þ1=xÞ 1
fX ðxÞ ¼ fXY ðx; yÞdy ¼ 2
e dy ¼ 2 y e
ay dy; (3.304)
x x
0 0 0

where

a ¼ 1 þ 1=x: (3.305)

Using integral 1 from Appendix A and from (3.304) and (3.305), we get:

1 x2 1
fX ðxÞ ¼ ¼ ; x0 (3.306)
x ðx þ 1Þ2 ðx þ 1Þ2
2

Exercise 3.14 Find the PDF of the random variable X,

X ¼ X1 þ a cos X2 ; (3.307)

if the joint density of the variables X1 and X2 is known.

Answer After introducing the auxiliary random variable Y ¼ X2, we have the
following equations:
x ¼ x1 þ a cos x2 ;
y ¼ x2 : (3.308)

The Jacobian of the transformation (3.308) is:

 
1
a sin x2 
J ¼   ¼ 1: (3.309)
0 1

The joint density of the variables X and Y is:

fXY ðx; yÞ ¼ fX1 X2 ðx1 ; x2 Þ ¼ fX1 X2 ðx
a cos y; yÞ: (3.310)

From here,

ð
1

fX ðxÞ ¼ fX1 X2 ðx
a cos y; yÞdy: (3.311)

1
208 3 Multidimensional Random Variables

Exercise 3.15 The random variables X1 and X2 are independent. Find the PDF of
the random variable X, if

X1
X¼ : (3.312)
X2

The marginal densities are:

x21 x22


x1
2 x2 2
fX1 ðx1 Þ ¼ e 2a uðx1 Þ; fX2 ðx2 Þ ¼ 2 e 2b uðx2 Þ: (3.313)
a2 b

Answer The auxiliary variable Y ¼ X2 is introduced. The corresponding set of the

transformation equations

x ¼ x1 =x2 ;
y ¼ x2 ; (3.314)

has the unique solution x1 ¼ xy and x2 ¼ y. The Jacobian of the transformation

(3.314) is
1 

xx12  1 1
J¼  x ¼ ¼ : (3.315)

2 2
0 1 x2 y

The joint density function of the variables X and Y is given as:

1 x21 x22 y2 x2 1
fX X ðx; yÞ yx1 x2 2 a2 þ b2


xy 3

2
þ 2
fXY ðx; yÞ ¼ 1 2 ¼ e ¼ e 2 a b : (3.316)
jJðx; yÞj a2 b2 a2 b2

Denoting
1 x2 1
a¼ þ ; (3.317)
2 a2 b2

from (3.316) and (3.317), we have:

ð
1 ð
1
x
y3 e
ay dy:
2
fX ðxÞ ¼ fXY ðx; yÞ dy ¼ (3.318)
a b2
2
0 0

Using the integral 4 from Appendix A and (E.2.15.6) we have:

x 1 2a2 x
fX ðxÞ ¼ ¼ ; x0 (3.319)
a2 b2 2a2 b2 a2
2
x2 þ 2
b
3.6 Numerical Exercises 209

Exercise 3.16 The random variables X1 and X2 are independent and uniform in the
intervals [1, 2] and [3, 5], respectively (as shown in Fig. 3.24). Find the density of
their sum:
X ¼ X1 þ X2 : (3.320)

Answer The density of the sum of the independent random variables is equal to the
convolution of their densities. This result can be easily obtained graphically, as
shown in Fig. 3.25.
Exercise 3.17 The random variable X1 uniformly takes values around its nominal
value 100 in the interval [100 10%]. Similarly, the variable X2 changes uni-
formly in the interval [200 10%]. Find the probability that the sum of the
variables X ¼ X1 + X2 is less than 310, if the variables X1 and X2 are independent.
Answer The density of the sum is the convolution of the corresponding densities
and is obtained graphically, as shown in Fig. 3.26. The desired probability is
presented in the shaded area, and is equal to:

PfX < 310g ¼ 1
ð330
310Þ=ð2  40Þ ¼ 1
1=4 ¼ 3=4: (3.321)

Fig. 3.24 Densities of the random variables X1 and X2

Fig. 3.25 Convolution of densities

210 3 Multidimensional Random Variables

Fig. 3.26 Convolution of the uniform PDFs

Exercise 3.18 The random variables X1 and X2 are independent. Find the density of
their sum if the corresponding densities are given as:

fX1 ðx1 Þ ¼ a e
ax1 uðx1 Þ; fX2 ðx2 Þ ¼ b e
bx2 uðx2 Þ; (3.322)

where a and b are constants.

Solution The density of the sum
X ¼ X1 þ X 2 (3.323)

is equal to the convolution of the densities (3.322):

fX ðxÞ ¼ fX1 ðx1 Þ  fX2 ðx2 Þ: (3.324)

For x < 0,

fX ðxÞ ¼ 0: (3.325)

For x > 0,

ð
1

fX ðxÞ ¼ fX1 ðx1 ÞfX2 ðx
x1 Þdx1

1
ð
1
ab
ax
¼ ab e
ax1 e
bðx
x1 Þ dx1 ¼ e : (3.326)
a
b
x

Exercise 3.19 Find the characteristic function of the variable X with the density
function shown in Fig. 3.27 in terms of the characteristic function of the variables
X1 and X2. The variable X is the sum of X1 and X2, and the variables X1 and X2 are
independent.
3.6 Numerical Exercises 211

Fig. 3.27 PDF of the variable X

Fig. 3.28 Convolution of PDFs

Answer The characteristic function of the sum of independent variables is equal to

the product of their characteristic functions:

fX ðoÞ ¼ fX1 ðoÞfX2 ðoÞ; (3.327)

where
ð ð
fX1 ðoÞ ¼ ejox1 fX1 ðx1 Þdx1 ; fX2 ðoÞ ¼ ejox2 fX2 ðx2 Þdx2 : (3.328)
x1 x2

The PDF from Fig. 3.27 is obtained by the convolution of the densities of the
variables X1 and X2, as shown in Fig. 3.28.
The characteristic functions are:

ð1 ð1
1 jox1 1 jox2 sin o
fX1 ðoÞ ¼ e dx1 ¼ fX2 ðoÞ ¼ e dx2 ¼ : (3.329)
2 2 o

1
1

Finally, from (3.327) and (3.329) we have:

2
sin o
fX ðoÞ ¼ : (3.330)
o
212 3 Multidimensional Random Variables

3.7 MATLAB Exercises

Exercise M.3.1 (MATLAB file exercise_M_3_1.m) Generate uniform and normal

random variables X and Y using the MATLAB functions rand and randn. Deter-
mine whether or not X and Y are independent observing the plot Y vs. X.
Solution The random variable X is uniform in the range [0, 1] and the variable Y is
a Gaussian variable with m ¼ 0 and the variance 1. The plot shown in Fig. 3.29
indicates that the variables are independent.
Exercise M.3.2 (MATLAB file exercise_M_3_2.m)
(a) Generate uniform random variable X over range [
2, 2]. Random variable
Y ¼ X2. Determine whether or not X and Y are correlated observing the plot Y
vs. X. Estimate the coefficient of correlation.
(b) Generate uniform random variable X over range [0, 0.5]. Random variable
Y ¼ X2. Determine whether or not X and Y are correlated observing the plot Y
vs. X. Estimate the coefficient of correlation.
(c) Generate uniform random variable X over range [5, 10]. Random variable
Y ¼ X2. Determine whether or not X and Y are correlated observing the plot Y
vs. X. Estimate the coefficient of correlation.
(d) Generate uniform random variable X over range [
10,
5]. Random variable
Y ¼ X2. Determine whether or not X and Y are correlated observing the plot Y
vs. X. Estimate the coefficient of correlation.
Solution
(a) The plot in Fig. 3.30a indicates that the variables are dependent but
uncorrelated.

1
Y

-1

-2

-3

-4
0 0.2 0.4 0.6 0.8 1
X

Fig. 3.29 Independent random variables

3.7 MATLAB Exercises 213

a b
4 0.25
3.5
0.2
3
2.5 0.15

Y1
2
Y

1.5 0.1
1
0.05
0.5
0 0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 0 0.1 0.2 0.3 0.4 0.5
X X1

c d
100 100
90 90
80 80
70 70
Y2

60
Y3

60
50 50
40 40
30 30
20 20
5 6 7 8 9 10 -10 -9 -8 -7 -6 -5
X2 X3

Fig. 3.30 Uncorrelated and correlated variables

The coefficient of correlation is estimated using the MATLAB file corrcoef.m.

The estimated coefficient of correlation is
0.0144.
(b) The variables are correlated as shown in Fig. 3.30b.
The estimated coefficient of correlation is 0.9679.
(c) The variables are correlated as shown in Fig. 3.30c.
The estimated coefficient of correlation is 0.9964.
(d) The variables are negative correlated as shown in Fig. 3.30d.
The estimated coefficient of correlation is
0.9964.
Exercise M.3.3 (MATLAB file exercise_M_3_3.m) Generate uniform random
variables X1 and X2 using rand.m.
(a) Determine whether or not X1 and X2 are independent observing the plot X2 vs. X1.
(b) Plot the sum
Y ¼ X 1 þ X2 : (3.331)

and estimate the PDF of the sum.

(c) Find the mathematical PDF of Y as the convolution of the corresponding PDFs,

fY ðyÞ ¼ fX1 ðx1 Þ  fX2 ðx2 Þ: (3.332)

214 3 Multidimensional Random Variables

Solution The plot in Fig. 3.31a indicates that the variables are independent. Their
sum is shown in Fig. 3.31b.
The estimated and the mathematical PDFs are shown in Fig. 3.32.
Exercise M.3.4 (MATLAB file exercise_M_3_4.m) Generate the uniform random
variables X1 and X2 in the intervals [1, 6] and [
2, 2], respectively.
(a) Determine whether or not X1 and X2 are independent observing the plot X2 vs. X1.
(b) Plot the sum
Y ¼ X 1 þ X2 : (3.333)
and estimate the PDF of the sum.

a b X1+X2
1 2
0.9 1.8
0.8 1.6
0.7 1.4
0.6 1.2
X2

0.5 1
0.4 0.8
0.3 0.6
0.2 0.4
0.1 0.2
0 0
0 0.2 0.4 0.6 0.8 1 0 200 400 600 800 1000
X1

Fig. 3.31 Independent variables and their sum in Exercise M.3.3

X1+X2
PDF estimation

0.5

0
0 0.5 1 1.5 2
cells
CONVOLUTION OF PDFs

1
PDF

0.5

0
0 0.5 1 1.5 2
range

Fig. 3.32 Estimated and mathematical PDFs in Exercise M.3.3

3.7 MATLAB Exercises 215

(c) Find the mathematical PDF of Y as the convolution of the corresponding PDFs,

fY ðyÞ ¼ fX1 ðx1 Þ  fX2 ðx2 Þ: (3.334)

Solution The plot in Fig. 3.33a indicates that the variables are independent. Their
sum is shown in Fig. 3.32b.
The estimated and the mathematical PDFs are shown in Fig. 3.34.

a b X1+X2
2 8
1.5 7
1 6
5
0.5
4
X2

0
3
-0.5
2
-1 1
-1.5 0
-2 -1
1 2 3 4 5 6 0 200 400 600 800 1000
X1

Fig. 3.33 Independent variables and their sum in Exercise M.3.4

X1+X2
PDF estimation

0.2
0.15
0.1
0.05
0
-1 0 1 2 3 4 5 6 7 8
cells
CONVOLUTION OF PDFs

0.2
0.15
PDF

0.1
0.05
0
-1 0 1 2 3 4 5 6 7 8
range

Fig. 3.34 Estimated and mathematical PDFs in Exercise M.3.4

216 3 Multidimensional Random Variables

Exercise M.3.5 (MATLAB file exercise_M_3_5.m) Plot the joint density for
independent uniform random variables X1 and X2:
(a) X1 and X2 are in the intervals [1, 2] and [1, 2], respectively.
(b) X1 and X2 are in the intervals [2, 4] and [1, 6], respectively.
Solution The joint densities are shown in Fig. 3.35.
Exercise M.3.6 (MATLAB file exercise_M_3_6.m) Plot the joint density for
independent normal random variables X1 and X2:
(a) X1 ¼ N(0, 1); X2 ¼ N(0, 1).
(b) X1 ¼ N(4, 4); X2 ¼ N(3, 9).
Solution The joint densities are shown in Fig. 3.36.

1.5
JOINT PDF

0.5

0 2
2 1.8 1.5
1.6
1.4
1.2 1
1
x2 x1

1.5

1
JOINT PDF

0.5

-0.5

-1 4
6
5 3
4
3
2 2
1
x2 x1

Fig. 3.35 Joint PDFs (a) 1  X1  2; 1  X2  2 (b) 2  X1  4; 1  X2  6

3.7 MATLAB Exercises 217

0.2

0.15
JOINT PDF

0.1

0.05

0 5
4 2 0
0
-2
-4 -5
x2 x1

0.03
0.025
JOINT PDF

0.02
0.015
0.01
0.005
0 10
15 10 5
5 0 0
-5
-10 -5
x2 x1

Fig. 3.36 Joint PDFs (a) X1 ¼ N(0, 1); X2 ¼ N(0, 1) (b) X1 ¼ N(4, 4); X2 ¼ N(3, 9)

Exercise M.3.7 (MATLAB file exercise_M_3_7.m) Plot the joint density for the
variables Y1 and Y2 from Example 3.4.1:
8
>
< y21
y1
2
fY1 Y2 ðy1 ; y2 Þ ¼ e 2s for y1  0; 0  y2  2p; : (3.335)
>
: 2ps2
0 otherwise:

Solution The joint density is shown in Fig. 3.37.

218 3 Multidimensional Random Variables

0.1

JOINT PDF 0.05

-0.05

-0.1
8
6 10
4 5
2 0
0 -5
y2 y1

Fig. 3.37 Joint PDF in Exercise M.3.7

3.8 Questions

Q.3.1. Can the same marginal density functions possibly result in different joint
density functions?
Q.3.2. In general, is a knowledge of the marginal PDFs sufficient to specify the
joint PDF?
Q.3.3. The random variables X1 and X2 are independent. Does it mean that the
transformed variables Y1 ¼ g1(X1) and Y2 ¼ g2(X2) are also independent?
Q.3.4. In which condition is the conditional distribution equal to the marginal
distribution FX1 ðx1 jX2  x2 Þ ¼ FX1 ðx1 Þ?
Q.3.5. Is the necessary condition fX1 X2 ðx1 ; x2 Þ ¼ fX1 ðx1 ÞfX2 ðx2 Þ, for the indepen-
dence of two random variables X1 and X2, also a sufficient condition?
Q.3.6. Is the following true?

Pfa < X1  b; c < X2  dg 6¼ FX1 X2 ðb; dÞ
FX1 X2 ða; cÞ: (3.336)

Q.3.7. Is that the following probability true?

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.337)

Q.3.8. For three random variables X1, X2, and X3 we have:

fX1 X2 ðx1 ; x2 Þ ¼ fX1 ðx1 ÞfX2 ðx2 Þ; (3.338)

fX1 X3 ðx1 ; x3 Þ ¼ fX1 ðx1 ÞfX3 ðx3 Þ; (3.339)

3.9 Answers 219

fX2 X3 ðx2 ; x3 Þ ¼ fX2 ðx2 ÞfX3 ðx3 Þ: (3.340)

Are the random variables X1, X2, and X3 independent?

Q.3.9. Is it possible for joint random variables to be of different types? For
example, one discrete random variable and one continuous random
variable?
Q.3.10. When is the correlation equal to the covariance?
Q.3.11. Is the coefficient of correlation zero, if random variables X and Y are
related as in Y ¼ X2?
Q.3.12. Why is the covariance not zero if the variables are correlated?

3.9 Answers

A.3.1. Yes. As illustrated in the following example, two discrete random variables
X1 and X2 have the following possible values:

ðX1 ¼ 1; X2 ¼ 1Þ; ðX1 ¼ 1; X2 ¼ 0Þ; ðX1 ¼ 0; X2 ¼ 1Þ; ðX1 ¼ 0; X2 ¼ 0Þ: (3.341)

We consider two cases:

ðaÞ PfX1 ¼ 1; X2 ¼ 1g ¼ PfX1 ¼ 0; X2 ¼ 0g ¼ 1=8;

(3.342)
PfX1 ¼ 1; X2 ¼ 0g ¼ PfX1 ¼ 0; X2 ¼ 1g ¼ 3=8:

ðbÞ PfX1 ¼ 1;X2 ¼ 1g ¼ PfX1 ¼ 0;X2 ¼ 0g ¼ PfX1 ¼ 1; X2 ¼ 0g

(3.343)
¼ PfX1 ¼ 0;X2 ¼ 1g ¼ 1=4:

The joint density function in case (a) is found to be:

fX1 X2 ðx1 ; x2 Þ ¼ 1=8½dðx1
1Þdðx2
1Þ þ dðx1 Þdðx2 Þ

(3.344)
þ 3=8½dðx1
1Þdðx2 Þ þ dðx1 Þdðx2
1Þ:

Similarly, in case (b), we have:

fX1 X2 ðx1 ; x2 Þ ¼ 1=4½dðx1
1Þdðx2
1Þ þ dðx1 Þdðx2 Þ þ dðx1
1Þdðx2 Þ

(3.345)
þ dðx1 Þdðx2
1Þ:

Note that the joint PDFs (3.344) and (3.345) are different.
Next we find the marginal densities.
220 3 Multidimensional Random Variables

For case (a), we have:

ð
1

fX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx2 ¼ 1=8½dðx1
1Þ þ dðx1 Þ þ 3=8½dðx1
1Þ þ dðx1 Þ

1
¼ 1=2½dðx1
1Þ þ dðx1 Þ;
ð
1

fX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 ¼ 1=8½dðx2
1Þ þ dðx2 Þ þ 3=8½dðx2
1Þ þ dðx2 Þ

1
¼ 1=2½dðx2
1Þ þ dðx2 Þ:
(3.346)

Similarly, in case (b), we get:

ð
1

fX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx2 ¼ 1=4½dðx1
1Þ þ dðx1 Þþdðx1
1Þ þ dðx1 Þ

1
¼ 1=2½dðx1
1Þ þ dðx1 Þ
(3.347)
ð
1

fX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 ¼ 1=4½dðx2
1Þ þ dðx2 Þþdðx2
1Þ þ dðx2 Þ

1
¼ 1=2½dðx2
1Þ þ dðx2 Þ:

Note that the marginal densities (3.346) and (3.347) are equal in both
cases, but the joint densities are different.
Therefore, the same marginal density functions may result in different
joint density functions.
A.3.2. In the previous example, we concluded that knowledge of the marginal
density functions does not provide all of the information about the relations
of the random variables.
In case (a),

fX1 ðx1 ÞfX2 ðx2 Þ ¼ 1=4½dðx1
1Þdðx2
1Þ þ dðx1
1Þdðx2 Þ

þ dðx1 Þdðx2
1Þ þ dðx1 Þdðx2 Þ 6¼ fX1 X2 ðx1 ; x2 Þ: (3.348)

This result shows us that if variables X1 and X2 are dependent, it is not

possible to obtain the joint density from the marginal densities.
However, in case (b) we have:

fX1 ðx1 ÞfX2 ðx2 Þ ¼ 1=4½dðx1
1Þdðx2
1Þ þ dðx1
1Þdðx2 Þ

þ dðx1 Þdðx2
1Þ þ dðx1 Þdðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þ; (3.349)

which demonstrates that the random variables are independent.

3.9 Answers 221

This result shows us that if the random variables are independent, we

can obtain the joint density from the marginal densities.
A.3.3. Let us suppose that

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g ¼ Pfy1 < Y1

 y1 þ dy1 ; y2 < Y2  y2 þ dy2 g: (3.350)

From (3.350), we have:

fX1 X2 ðx1 ; x2 Þdx1 dx2 ¼ fY1 Y2 ðy1 ; y2 Þdy1 dy2 : (3.351)

Knowing that the random variables X1 and X2 are independent, we get:

fX1 ðx1 ÞfX2 ðx2 Þdx1 dx2 ¼ fY1 Y2 ðy1 ; y2 Þdy1 dy2 : (3.352)

The joint distribution of Y1 and Y2 is:

yð1 yð2 ð1 Þ gðxð2 Þ

gðx

FY1 Y2 ðy1 ;y2 Þ ¼ fY1 Y2 ðy1 ;y2 Þdy1 dy2 ¼ fX1 X2 ðx1 ;x2 Þdx1 dx2

1
1
1
1

ð1 Þ gðxð2 Þ
gðx yð1 yð2

¼ fX1 ðx1 ÞfX2 ðx2 Þdx1 dx2 ¼ fX1 ðx1 Þdx1 fX2 ðx2 Þdx2 ¼ FY1 ðy1 ÞFY2 ðy2 Þ:

1
1
1
1

(3.353)

The joint distribution function is equal to the product of the marginal

distributions. As a consequence, the variables Y1 and Y2 are independent.
A.3.4. The conditional distribution can be rewritten as:

PfX1  x1 ; X2  x2 g FX1 X2 ðx1 ; x2 Þ

FX1 ðx1 jX2  x2 Þ ¼ ¼ : (3.354)
PfX2  x2 g FX2 ðx2 Þ

If the random variables X1 and X2 are independent, then

FX1 X2 ðx1 ; x2 Þ ¼ FX1 ðx1 ÞFX2 ðx2 Þ; (3.355)

resulting in:

FX1 ðx1 ÞFX2 ðx2 Þ

FX1 ðx1 jX2  x2 Þ ¼ ¼ FX1 ðx1 Þ: (3.356)
FX2 ðx2 Þ

If the random variables X1 and X2 are independent, then the conditional

distribution of variable X1, given the other variable X2, is equal to the
marginal distribution of X1.
222 3 Multidimensional Random Variables

A.3.5. The joint density function for the independent random variables X1 and X2
can be written as (see Q.3.3. and [THO71, p. 71]):

fX1 X2 ðx1 ; x2 Þ ¼ g1 ðx1 Þg2 ðx2 Þ: (3.357)

The marginal densities can be expressed as:

ð
1 ð
1

fX1 ðx1 Þ ¼ fX1 X2 ðx1 ; x2 Þdx2 ¼ g1 ðx1 Þg2 ðx2 Þdx2

1
1
ð
1 ð
1

¼ g1 ðx1 Þ g2 ðx2 Þdx2 ¼g1 ðx1 ÞK1 ; g2 ðx2 Þdx2 ¼K1 ; (3.358)

1
1

ð
1 ð
1

fX2 ðx2 Þ ¼ fX1 X2 ðx1 ; x2 Þdx1 ¼ g1 ðx1 Þg2 ðx2 Þdx1

1
1
ð
1 ð
1

¼ g2 ðx2 Þ g1 ðx1 Þdx1 ¼g2 ðx2 ÞK2 ; g1 ðx1 Þdx1 ¼K2 : (3.359)

1
1

From (3.358) and (3.359), we have:

g1 ðx1 Þ ¼ fX1 ðx1 Þ=K1

g2 ðx2 Þ ¼ fX2 ðx2 Þ=K2 : (3.360)

Using (3.357) and (3.360), we get:

fX1 ðx1 ÞfX2 ðx2 Þ

fX1 X2 ðx1 ; x2 Þ ¼ : (3.361)
K1 K2

From (3.37), (3.357) and (3.361), we have:

ð
1 ð
1

1¼ fX1 X2 ðx1 ; x2 Þdx1 dx2

1
1
ð 1
1 ð ð
1 ð
1
1
¼ g1 ðx1 Þg2 ðx2 Þdx1 dx2 ¼ fX1 ðx1 ÞfX2 ðx2 Þdx1 dx2 : (3.362)
K1 K2

1
1
1
1

From (3.362), it follows:

K1 K2 ¼ 1: (3.363)
3.9 Answers 223

Finally, from (3.357) and (3.363) we arrive at:

g1 ðx1 Þ ¼ fX1 ðx1 Þ;

g2 ðx2 Þ ¼ fX2 ðx2 Þ; (3.364)

which confirms that the condition is also a sufficient condition for the
independence.
A.3.6. Yes. It is true, as explained in the following:
The event

A ¼ fa < X1  b; c < X2  dg: (3.365)

can be presented as a difference between the events A1 and A2, as shown in

Fig. 3.38.

A1 ¼ fa < X1  b; X2  dg; A2 ¼ fa < X1  b; X2  cg; (3.366)

PfAg ¼ PfA1 g
PfA2 g: (3.367)

Placing (3.365) and (3.366) into (3.367), we get:

Pfa < X1  b; c < X2  dg ¼ Pfa < X1  b; X2  dg
Pfa < X1  b; X2  cg:

(3.368)

Fig. 3.38 Illustration of A.3.6

224 3 Multidimensional Random Variables

Using (3.9), we can rewrite (3.368) in the following form:

Pfa < X1  b; c < X2  dg ¼ FX1 X2 ðb; dÞ
FX1 X2 ða; dÞ
½FX1 X2 ðb; cÞ
FX1 X2 ða; cÞ:
(3.369)

Finally, from (3.369) we obtain (3.15):

Pfa < X1  b; c < X2  dg ¼ FX1 X2 ðb; dÞ
FX1 X2 ða; dÞ
FX1 X2 ðb; cÞ þ FX1 X2 ða; cÞ:
(3.370)

A.3.7. Yes. Using (3.15), the probability (3.337) is expressed as:

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g

¼ ½FX1 X2 ðx1 þ dx1 ; x2 þ dx2 Þ
FX1 X2 ðx1 ; x2 þ dx2 Þ

½FX1 X2 ðx1 þ dx1 ; x2 Þ
FX1 X2 ðx1 ; x2 Þ: (3.371)

The expression in the first bracket of (3.371) can be rewritten as:

FX1 X2 ðx1 þ dx1 ; x2 þ dx2 Þ
FX1 X2 ðx1 ; x2 þ dx2 Þ

FX X ðx1 þ dx1 ; x2 þ dx2 Þ
FX1 X2 ðx1 ; x2 þ dx2 Þ
¼ 1 2 dx1
dx1
@FX1 X2 ðx1 ; x2 þ dx2 Þ
¼ dx1 : (3.372)
@x1
Similarly, we have:

FX1 X2 ðx1 þ dx1 ; x2 Þ þ FX1 X2 ðx1 þ dx1 ; x2 Þ

FX1 X2 ðx1 þ dx1 ; x2 Þ
FX1 X2 ðx1 ; x2 Þ @FX1 X2 ðx1 ; x2 Þ
¼ dx1 ¼ dx1 : (3.373)
dx1 @x1

Placing (3.372) and (3.373) into (3.371), and using the definition (3.27),
we get:

Pfx1 < X1  x1 þ dx1 ; x2 < X2  x2 þ dx2 g

 
@FX1 X2 ðx1 ; x2 þ dx2 Þ
@FX1 X2 ðx1 ; x2 Þ 1
¼ dx1 dx2
@x1 dx2
@ 2 FX1 X2 ðx1 ; x2 Þ
¼ dx1 dx2 ¼ fX1 X2 ðx1 ; x2 Þdx1 dx2 : (3.374)
@x1 @x2

A.3.8. A set of N random variables is statistically independent “if any joint PDF of
M variables, M  N, is factored into the product of the corresponding
marginal PDFs” [MIL04, p. 209]. According to this statement, the
conditions (3.338)–(3.340) do not assure that the all three variables are
3.9 Answers 225

independent because the condition is satisfied only for M < N ¼ 3.

Therefore, the condition for M ¼ N ¼ 3 should be added:

fX1 X2 ðx1 ; x2 ; x3 Þ ¼ fX1 ðx1 ÞfX2 ðx2 ÞfX3 ðx3 Þ: (3.375)

However, the condition (3.375) itself includes the conditions

(3.338)–(3.340). As a consequence, the condition (3.375) is a necessary
and sufficient condition for the independence of three random variables.
A.3.9. Yes. However, in this case, it is more comfortable to work with the
corresponding joint probabilities rather than either joint PDFs or
distributions [LEO94, pp. 206–207]. The following example, adapted
from [LEO94, pp. 206–207], illustrates the concept.
The input to the communication channel (Fig. 3.39) is a discrete random
variable X with the discrete values: U and –U (the polar signal) and the
corresponding probabilities P{U} ¼ P{
U} ¼ 1/2. The uniform noise N
over the range [
a, a] is added to the signal X, resulting in the output of the
channel Y ¼ X + N. Find the probabilities

PfðX ¼ UÞ \ ðY  0Þg
PfðX ¼
UÞ \ ðY  0Þg: (3.376)

Using the conditional probabilities we can rewrite the probabilities

(3.376) as:

PfðX ¼ UÞ \ ðY  0Þg ¼ PfY  0jX ¼ Ug PfX ¼ Ug ¼ PfY  0jX ¼ Ug1=2:

PfðX ¼
UÞ \ ðY  0Þg ¼ PfY  0jX ¼
Ug PfX ¼
Ug ¼ PfY  0jX ¼
Ug1=2
(3.377)

The input random variable X is a discrete random variable, while the output
random variable Y is a continuous random variable. The conditional
variables are also continuous: Y1 ¼ Y|U and Y2 ¼ Y|
U.
The PDFs of the noise and the variables Y1 and Y2 are shown in
Fig. 3.40.
From Fig. 3.40, we get:

PfY  0jX ¼ Ug ¼ jU
aj=2a ¼ PfY  0jX ¼
Ug ¼ ja
U j=2a: (3.378)

Finally, from (3.377) and (3.378), we have:

Fig. 3.39 Communication

channel with added noise
226 3 Multidimensional Random Variables

Fig. 3.40 PDFs of (a) noise (b) r.v. Y1, and (c) r.v. Y2

PfðX ¼ UÞ \ ðY  0Þg ¼ PfðX ¼ UÞ \ ðY  0Þg ¼ ja
U j=4a: (3.379)

A.3.10. Recall that from (3.110) and (3.123):

CX1 X2 ¼ X1 X2
X1 X2 ¼ RX1 X2
X1 X2 : (3.380)

From (3.380), it follows that the covariance and correlation are equal if
either mean value of X1 or X2–or of both of them–is zero.
A.3.11. It is not necessarily be zero. It depends on the range of the variable X. See
Example 3.3.4.
A.3.12. Let us recall of the definition of covariance (3.121):

CX1 X2 ¼ EfðX1
X1 ÞðX2
X2 Þg: (3.381)

When variables X1 and X2 are correlated, they have either the same
tendency (positive correlation), or the opposite tendency (negative corre-
lation). Therefore, if there is a positive correlation between them, the
dissipation of the random variable X1 around its mean value will have
the same sign (in a majority of cases) as the dissipation of the random
variable X2 around its mean value. As a consequence, the expected value of
the product of (X1
E{X1}) and (X2
E{X2}) cannot equal zero, and the
covariance is not zero.
Similarly, if variables X1 and X2 have a negative correlation, the signs of
the dissipations of the values of the random variables will be opposite. As a
consequence, the expected value of the product of (X1
E{X1}) and
(X2
E{X2}) cannot equal zero, and the covariance is not zero.