Generating random numbers

Lecturer: Dmitri A. Moltchanov

E-mail: [email protected]
http://www.cs.tut.fi/kurssit/ELT-53606/

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

OUTLINE:
Why do we need random numbers;
Basic steps in generation;
Uniformly distributed random numbers;
Statistical tests for uniform random numbers;
Random numbers with arbitrary distributions;
Statistical tests for random numbers with arbitrary distribution;
Multidimensional distributions.

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

1. The need for random numbers

Examples of randomness in telecommunications:
interarrival times between arrivals of packets, tasks, etc.;
service time of packets, tasks, etc.;
time between failure of various components;
repair time of various components;
...
Importance for simulations:
random events are characterized by distributions;
simulations: we cannot use distribution directly.
For example, M/M/1 queuing system:
arrival process: exponential distribution with mean 1/;
service times: exponential distribution with mean 1/.

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Discrete-event simulation of M/M/1 queue

INITIALIZATION
time:=0;
queue:=0;
sum:=0;
throughput:=0;
generate first interarrival time;
MAIN PROGRAM
while time < runlength do
case nextevent of
arrival event:
time:=arrivaltime;
add customer to a queue;
start new service if the service is idle;
generate next interarrival time;
departure event:
time:=departuretime;
throughput:=throughtput + 1;
remove customer from a queue;
if (queue not empty)
sum:=sum + waiting time;
start new service;
OUTPUT
mean waiting time = sum / throughput

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

2. General notes
General approach nowadays:
transforming one random variable to another one;
as a reference distribution a uniform distribution is often used.

Note the following:

most simulators contain generator of uniformly distributed numbers in interval (0, 1).
may not contain arbitrarily distributed random numbers you want.

The procedure is to:

generate RN with inform distribution between a and b, b >>>> a;
transform it somehow to random number with uniform distribution on (0, 1);
transform it somehow to a random number with desired distribution.

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

2.1. Pseudo random numbers

All computer generated numbers are pseudo ones:
we know the method how they are generated;
we can predict any random sequence in advance.

The goal is then: imitate random sequences as good as possible.

Requirements for generators:

must be fast;
must have low complexity;
must have sufficiently long cycles;
must allow to generate repeatable sequences;
must be independent;
must closely follow a given distribution.

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

2.2. Step 1: uniform random numbers in (a, b)

Basic approach:
generate random number with uniform distribution on (a, b);
transform these random numbers to (0, 1);
transform it somehow to a random number with desired distribution.
Uniform generators:
old methods: mostly based on radioactivity;
Von Neumanns algorithm;
congruential methods.
Basic approach: next number is some function of previous one
i+1 = F (i ),

i = 0, 1, . . . ,

(1)

recurrence relation of the first order;

0 is known and directly computed from the seed.

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

2.3. Step 2: transforming to random numbers in (0, 1)

Basic approach:
generate random number with uniform distribution on (0, 1);
transform these random numbers to (0, 1);
transform it somehow to a random number with desired distribution.
Uniform U (0, 1) distribution has the following pdf:

1, 0 x 1
.
f (x) =
0, otherwise

Lecture: Generating random numbers

(2)

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Mean and variance are given by:

Z
E[X] =
0

2 [X] =

1
x 
1
xdx =  = ,
2
2
2

1
.
12

(3)

How to get U (0, 1):

by rescaling from U (0, m) as follows:
yi = i /m,

(4)

where m is the modulo in linear congruential algorithm.

What we get:
something like: 0.12, 0.67, 0.94, 0.04, 0.65, 0.20, . . . ;
sequence that appears to be random...

Lecture: Generating random numbers

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

2.4. Step 3: non-uniform random numbers

Basic approach:
generate random number with uniform distribution on (a, b);
transform these random numbers to (0, 1);
transform it somehow to a random number with desired distribution.
If we have generator U (0, 1) the following techniques are avalable:
discretization: bernoulli, binomial, poisson, geometric;
rescaling: uniform;
inverse transform: exponential;
specific transforms: normal;
rejection method: universal method;
reduction method: Erlang, Binomial;
composition method: for complex distributions.

Lecture: Generating random numbers

10

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3. Uniformly distributed random numbers

The generator is fully characterized by (S, s0 , f, U, g):
S is a finite set of states;
s0 S is the initial state;
f (S S) is the transition function;
U is a finite set of output values;
g(S U ) is the output function.
The algorithm is then:
let u0 = g(s0 );
for i = 1, 2, . . . do the following recursion:
si = f (si1 );
ui = g(si ).
Note: functions f () and g() influence the goodness of the algorithm heavily.

Lecture: Generating random numbers

11

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

user choice s0
u0=g(s0)

u4

u1=g(s1)

s0

s1=f(s0)
s1

u1

s4
s3

s2=f(s1)

u0

u4=g(s4)
s4=f(s3)

u3

u3=g(s3)
u2

s2 s3=f(s2)
u2=g(s2)

Figure 1: Example of the operations of random number generator.

Here s0 is a random seed:
allows to repeat the whole sequence;
allows to manually assure that you get different sequence.

Lecture: Generating random numbers

12

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.1. Von Neumanns generator

The basic procedure:
start with some number u0 of a certain length x (say, x = 4 digits, this is seed);
square the number;
take middle 4 digits to get u1 ;
repeat...
example: with seed 1234 we get 1234, 5227, 3215, 3362, 3030, etc.
Shortcoming:
sensitive to the random seed:
seed 2345: 2345, 4990, 9001, 180, 324, 1049, 1004, 80, 64, 40... (will always < 100);
may have very short period:
seed 2100: 2100, 4100, 8100, 6100, 2100, 4100, 8100,... (period = 4 numbers).
To generate U (0, 1): divide each obtained number by 10x (x is the length of u0 ).
Note: this generator is also known as midsquare generator.

Lecture: Generating random numbers

13

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.2. Congruential methods

There are a number of versions:
additive congruential method;
multiplicative congruential method;
linear congruential method;
Tausworthe binary generator.
General congruential generator:
ui+1 = f (ui , ui1 , . . . )

mod m,

(5)

ui , ui1 , . . . are past numbers.

For example, quadratic congruential generator:
ui+1 = (a1 u2i + a2 ui1 + c) mod m.

(6)

Note: if here a1 = a2 = 1, c = 0, m = 2 we have the same as midsquare method.

Lecture: Generating random numbers

14

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.3. Additive congruential method

Additive congruential generator is given:
ui+1 = (a1 ui + a2 ui1 + + ak uik ) mod m.

(7)

The common special case is sometimes used:

ui+1 = (a1 ui + a2 ui1 ) mod m.

(8)

Characteristics:
divide by m to get U (0, 1);
maximum period is mk ;
note: rarely used.
Shortcomings: consider k = 2:
consider three consecutive numbers ui2 , ui1 , ui ;
we will never get: ui2 < ui < ui1 and ui1 < ui < ui2 (must be 1/6 of all sequences).

Lecture: Generating random numbers

15

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.4. Multiplicative congruential method

Multiplicative congruential generator is given:
ui+1 = (aui )

mod m.

(9)

Characteristics:
divide by m to get U (0, 1);
theoretical maximum period is m;
note: rarely used.
Shortcomings:
can never produce 0.
Choice of a, m is very important:
recommended m = (2p 1) with p = 2, 3, 5, 7, 13, 17, 19, 31, 61 (Fermat numbers);
if m = 2q , q 4 simplifies the calculation of modulo;
practical maximum period is at best no longer than m/4.

Lecture: Generating random numbers

16

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.5. Linear congruential method

Linear congruential generator is given:
ui+1 = (aui + c)

mod m,

(10)

where a, c, m are all positive.

Characteristics:
divide by m to get U (0, 1);
maximum period is m;
frequently used.
Choice of a, c, m is very important. To get full period m choose:
m and c have no common divisor;
c and m are prime number (distinct natural number divisors 1 and itself only);
if q is a prime divisor of m then a = 1, mod q;
if 4 is a divisor of m then a = 1, mod 4.

Lecture: Generating random numbers

17

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

The step-by-step procedure is as follows:

set the seed x0 ;
multiply x by a and add c;
divide the result by m;
the reminder is x1 ;
repeat to get x2 , x3 , . . . .

Examples:
x0 = 7, a = 7, c = 7, m = 10 we get: 7,6,9,0,7,6,9,0,... (period = 4);
x0 = 1, a = 1, c = 5, m = 13 we get: 1,6,11,3,8,0,5,10,2,7,12,4,9,1... (period = 13);
x0 = 8, a = 2, c = 5, m = 13 we get: 8,8,8,8,8,8,8,8,... (period = 1!).

Recommended values: a = 314, 159, 269, c = 453, 806, 245, m = 231 for 32 bit machine.

Lecture: Generating random numbers

18

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Complexity of the algorithm: addition, multiplications and division:

division is slow: to avoid it set m to the size of the computer word.
Overflow problem when m equals to the size of the word:
values a, c and m are such that the result axi + c is greater than the word;
it may lead to loss of significant digits but it does not hurt!
How to deal with:
register can accommodate 2 digits at maximum;
the largest number that can be stored is 99;
if m = 100: for a = 8, u0 = 2, c = 10 we get (aui + c) mod 100 = 26;
if m = 100: for a = 8, u0 = 20, c = 10 we get (aui + c) mod 100 = 170;
aui = 8 20 = 160 causing overflow;
first significant digit is lost and register contains 60;
the reminder in the register (result) is: (60 + 10) mod 70 = 70.
the same as 170 mod 100 = 70.

Lecture: Generating random numbers

19

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.6. How to get good congruental generator

Characteristics of good generator:
should provide maximum density:
no large gaps in [0, 1] are produced by random numbers;
problem: each number is discrete;
solution: a very large integer for modulus m.
should provide maximum period:
achieve maximum density and avoid cycling;
achieve by: proper choice of a, c, m, and x0 .
effective for modern computers:
set modulo to power of 2.

Lecture: Generating random numbers

20

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.7. Tausworthe generator

Tausworthe generator (case of linear congruential generator or order k):
!
k
X
zi = (a1 zi1 + a2 zi2 + + ak zik + c) mod 2 =
aj zij + c
mod 2.

(11)

j=1

where aj {0, 1}, j = 0, 1, . . . , k;

the output is binary: 0011011101011101000101...

Advantages:
independent of the system (computer architecture);
independent of the word size;
very large periods;
can be used in composite generators (we consider in what follows).

Note: there are several bit selection techniques to get numbers.

Lecture: Generating random numbers

21

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

A way to generate numbers:

choose an integer l k;
split in blocks of length l and interpret each block as a digit:
un =

l1
X

znl+j 2(j+1) .

(12)

j=0

In practice, only two ai are used and set to 1 at places h and k. We get:
zn = (zih + zik ) mod 2.

(13)

Example:
h = 3, k = 4, initial values 1,1,1,1;
we get: 110101111000100110101111...;
period is 2k 1 = 15;
if l = 4: 13/16, 7/16, 8/16, 9/16, 10/16, 15/16, 1/16, 3/16...

Lecture: Generating random numbers

22

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

3.8. Composite generator

Idea: use two generators of low period to generate another with wider period.
The basic principle:
use the first generator to fill the shuffling table (address - entry (random number));
use random numbers of second generator as addresses in the next step;
each number corresponding to the address is replaced by new random number of first generator.
The following algorithm uses one generator to shuffle with itself:
1. create shuffling table of 100 entries (i, ti = i , i = 1, 2, . . . , 100);
2. draw random number k and normalize to the range (1, 100);
3. entry i of the table gives random number ti ;
4. draw the next random number k+1 and update ti = k+1 ;
5. repeat from step 2.
Note: table with 100 entries gives fairly good results.

Lecture: Generating random numbers

23

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4. Tests for random number generators

What do we want to check:
independence;
uniformity.
Important notes:
if and only if tests passed number can be treated as random;
recall: numbers are actually deterministic!
Commonly used tests for independence:
runs test;
correlation test.
Commonly used tests for uniformity:
Kolmogorovs test;
2 test.

Lecture: Generating random numbers

24

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4.1. Independence: runs test

Basic idea:
compute patterns of numbers (always increase, always decrease, etc.);
compare to theoretical probabilities.

1/3

1/3

1/3

1/3

1/3

1/3

Figure 2: Illustration of the basic idea.

Lecture: Generating random numbers

25

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Do the following:
consider a sequence of pseudo random numbers: {ui , i = 0, 1, . . . , n};
consider unbroken subsequences of numbers where numbers are monotonically increasing;
such subsequence is called run-up;
example: 0.78,081,0.89,0.81 is a run-up of length 3.
compute all run-ups of length i:
ri , i = 1, 2, 3, 4, 5;
all run-ups of length i 6 are grouped into r6 .
calculate:
1 X
(ri nbi )(rj nbj )aij ,
R=
n 1i,j6

1 i, j 6,

(14)

where
(b1 , b2 , . . . , b6 ) =

1 5 11 19 29
1
, ,
,
,
,
6 24 120 720 5040 840

Lecture: Generating random numbers

(15)

26

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Coefficients aij must be chosen as an element of the matrix:

Statistics R has 2 distribution:

number of freedoms: 6;
n > 4000.

If so, observations are i.i.d.

Lecture: Generating random numbers

27

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4.2. Independence: correlation test

Basic idea:
compute autocorrelation coefficient for lag-1;
if it is not zero and this is statistically significant result, numbers are not independent.
Compute statistics (lag-1 autocorrelation coefficient) as:
N
N
X
X
R=
(uj E[u])(uj+1 E[u])/
(uj E[j])2 .
j=1

(16)

j=1

Practice: if R is relatively big there is serial correlation.

Important notes:
exact distribution of R is unknown;

for large N : if uj uncorrelated we have: P r{2/ N R 2/ N };

therefore: reject hypotheses of non-correlated at 5% level if R is not in {2/ N , 2/ N }.

Notes: other tests for correlation Ljung and Box test, Portmanteau test, etc.

Lecture: Generating random numbers

28

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4.3. Uniformity: 2 test

The algorithm:
divide [0, 1] into k, k > 100 non-overlapping intervals;
compute the relative frequencies of falling in each category, fi :
ensure that there are enough numbers to get fi > 5, i = 1, 2, . . . , k;
values fi > 5, i = 1, 2, . . . , k are called observed values.
if observations are truly uniformly distributed then:
these values should be equal to ri = n/k, i = 1, 2, . . . , k;
these values are called theoretical values.
compute 2 statistics for uniform distribution:
k

k X
n 2
=
fi
.
n i=1
k
2

(17)

that must have k 1 degrees of freedom.

Lecture: Generating random numbers

29

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Hypotheses:
H0 observations are uniformly distributed;
H1 observations are not uniformly distributed.
H0 is rejected if:
computed value of 2 is greater than one obtained from the tables;
you should check the entry with k 1 degrees of freedom and 1-a level of significance.

Lecture: Generating random numbers

30

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4.4. Kolmogorov test

Facts about this test:
compares empirical distribution with theoretical ones;
empirical: FN (x) number of smaller than or equal to x, divided by N ;
theoretical: uniform distribution in (0, 1): F (x) = x, 0 < x < 1.
Hypotheses:
H0 : FN (x) follows F (x);
H1 : FN (x) does not follow F (x).
Statistics: maximum absolute difference over a range:
R = max |F (x) FN (x)|.

(18)

if R > R : H0 is rejected;
if R R : H0 is accepted.
Note: use tables for N , (significance level), to find R .

Lecture: Generating random numbers

31

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Example: we got 0.44, 0.81, 0.14, 0.05, 0.93:

H0 : random numbers follows uniform distribution;
we have to compute:

R(j)

0.05

0.14

0.44

0.81

0.93

j/N

0.20

0.40

0.60

0.80

1.00

j/N R(j)

0.15

0.26

0.16

0.07

R(j) (j-1)/N

0.05

0.04

0.21

0.13

compute statistics as: R = max |F (x) FN (x)| = 0.26;

from tables: for = 0.05, R = 0.565 > R;
H0 is accepted, random numbers are distributed uniformly in (0, 1).

Lecture: Generating random numbers

32

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4.5. Other tests

The serial test:
consider pairs (u1 , u2 ), (u3 , u4 ), . . . , (u2N 1 , u2N );
count how many observations fall into N 2 different subsquares of the unit square;
apply 2 test to decide whether they follow uniform distribution;
one can formulate M -dimensional version of this test.
The permutation test
look at k-tuples: (u1 , uk ), (uk+1 , u2k ), . . . , (u(N 1)k+1 , uN k );
in a k-tuple there k! possible orderings;
in a k-tuple all orderings are equally likely;
determine frequencies of orderings in k-tuples;
apply 2 test to decide whether they follow uniform distribution.

Lecture: Generating random numbers

33

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

The gap test

let J be some fixed subinterval in (0, 1);
if we have that:
un+j not in J, 0 j k, and both un1 J, un+k+1 J;
we say that there is a gap of length k.
H0 : numbers are independent and uniformly distributed in (0, 1):
gap length must be geometrically distributed with some parameter p;
p is the length of interval J:
P r{gap of length k} = p(1 p)k .

(19)

practice: we observe a large number of gaps, say N ;

choose an integer and count number of gaps of length 0, 1, . . . , h 1 and h;
apply 2 test to decide whether they independent and follow uniform distribution.

Lecture: Generating random numbers

34

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

4.6. Important notes

Some important notes on seed number:
do not use seed 0;
avoid even values;
do not use the same sequence for different purposes in a single simulation run.

Note: these instruction may not be applicable for a particular generator.

General notes:
some common generators are found to be inadequate;
even if generator passed tests, some underlying pattern might still be undetected;
if the task is important use composite generator.

Lecture: Generating random numbers

35

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5. Random numbers with arbitrary distribution

Discrete distributions:
discretization;
for any discrete distribution.
rescaling:
for uniform random numbers in (a, b).
methods for specific distributions.

Continuous distributions:
inverse transform;
rejection method;
composition method;
methods for specific distributions.

Lecture: Generating random numbers

36

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.1. Discrete distributions: discretization

Consider arbitrary distributed discrete RV:
P r{X = xj } = pj ,

j = 0, 1, . . .,

pj = 1.

(20)

j=0

The following method can be applied:

generate uniformly distributed RV;
use the following to generate discrete RV:

this method can be applied to any discrete RV;

there are some specific methods for specific discrete RVs.

Lecture: Generating random numbers

37

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Figure 3: Illustration of the proposed approach.

Lecture: Generating random numbers

38

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

The step-by-step procedure:

compute probabilities pi = P r{X = xi }, i = 0, 1, . . . ;
generate RV u with U (0, 1);
if u < p0 , set X = x0 ;
if u < p0 + p1 , set X = x1 ;
if u < p0 + p1 + p2 , set X = x2 ;
...

Note the following:

this is inverse transform method for discrete RVs:
we determine the value of u;
we determine the interval [F (xi1 ), F (xi )] in which it lies.
complexity depends on the number of intervals to be searched.

Lecture: Generating random numbers

39

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Example: p1 = 0.2, p2 = 0.1, p3 = 0.25, p4 = 0.45:

determine generator for P r{X = xj } = pj .
Algorithm 1:
generate u = U (0, 1);
if u < 0.2, set X = 1, return;
if u < 0.3, set X = 2;
if u < 0.55, set X = 3;
set X = 4.
Algorithm 2 (more effective):
generate u = U (0, 1);
If u < 0.45, set X = 4;
if u < 0.7, set X = 3;
if u < 0.9, set X = 1;
set X = 2.

Lecture: Generating random numbers

40

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.2. Example of discretization: Poisson RV

Poisson RV have the following distribution:
i e
pi = P r{X = i} =
,
i!

i = 0, 1, . . . .

(21)

We use the property:

pi+1 =

pi ,
i+1

i = 1, 2, . . . .

(22)

The algorithm:
1. generate u = U (0, 1);
2. i = 0, p = e , F = p;
3. if u < F , set X = i;
4. p = p/(i + 1), F = F + p, i = i + 1;
5. go to step 3.

Lecture: Generating random numbers

41

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.3. Example of discretization: binomial RV

Binomial RV have the following distribution:
pi = P r{X = i} =

n!
pi (1 p)ni ,
i!(n i)!

i = 0, 1, . . . .

(23)

We are going to use the following property:

pi+1 =

ni p
pi ,
i+11p

i = 0, 1, . . . .

(24)

The algorithm:
1. generate u = U (0, 1);
2. c = p/(1 p), i = 0, d = (1 p)n, F = d;
3. if u < F , set X = i
4. d = [c(n i)/(i + 1)]d, F = F + d, i = i + 1;
5. go to step 3.

Lecture: Generating random numbers

42

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.4. Continuous distributions: inverse transform method

Inverse transform method:
applicable only when cdf can be inversed analytically;
works for a number of distributions: exponential, unform, Weibull, etc.

Assume:
we would like to generate numbers with pdf f (x) and cdf F (x);
recall, F (x) is defined on [0, 1].

The generic algorithm:

generate u = U (0, 1);
set F (x) = u;
find x = F 1 (u),
F 1 () is the inverse transformation of F ().

Lecture: Generating random numbers

43

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Example:
we want to generate numbers from the following pdf f (x) = 2x, 0 x 1;
calculate the cdf as follows:
Z
F (x) =

2tdt = x2 ,

0 x 1.

(25)

let u be the random number, we have u = x2 or u =

x;

get the random number.

Lecture: Generating random numbers

44

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.5. Inverse transform method: uniform continuous distribution

Uniform continuous distribution has the following pdf and cdf:

1 , a<x<b
(xa) , a < x < b
f (x) = (ba)
,
F (x) = (ba)
.
0,

otherwise
0,
otherwise

(26)

The algorithm:
generate u = U (0, 1);
set u = F (x) = (x a)/(b a);
solve to get x = a + (b a)u.

Lecture: Generating random numbers

45

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.6. Inverse transform method: exponential distribution

Exponential distribution has the following pdf and cdf:
f (x) = ex ,

F (x) = 1 ex ,

> 0,

x 0.

(27)

The algorithm:
generate u = U (0, 1);
set u = F (x) = ex ;
solve to get x = (1/) log u.

Lecture: Generating random numbers

46

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.7. Inverse transform method: Erlang distribution

Erlang distribution:
convolution of k exponential distributions.

The algorithm:
generate u = U (0, 1);
sum of exponential variables x1 , . . . , xk with mean 1/;
solve to get:
x=

k
X
i=1

X
1X
1
xi =
log ui = log
ui .
i=1

i=1

Lecture: Generating random numbers

(28)

47

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.8. Specific methods: normal distribution

Normal distribution has the following pdf:
(x)2
1
12
2
f (x) = e
,
2

< x < ,

(29)

where and are the standard deviation and the mean.

Standard normal distribution (RV Z = (X /)) has the following pdf:

1 2
1
f (z) = e 2 z ,
2

< z < ,

Lecture: Generating random numbers

where

= 0, = 1.

(30)

48

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Central limit theorem:

if x1 , x2 , . . . , xn are independent with E[xi ] = , 2 [xi ] = 2 , i = 1, 2, . . . , n;
P
P
2
the sum of them approaches normal distribution if n : E[ xi ] = n, [ xi ] = n 2 .
The approach:
generate k random numbers ui = U (0, 1), i = 0, 1, . . . , k 1;
each random numbers has: E[ui ] = (0 + 1)/2 = 1/2, 2 [ui ] = (1 0)2 /12 = 1/12;
sum of these number follows normal distribution with:
P


X
k k
ui k/2

ui N
,
,
or
N (0, 1).
2 12
k/ 12

(31)

if the RV we want to generate is x with mean and standard deviation :

x
N (0, 1).

finally (note that k should be at least 10):

P
ui k/2
x

,
or
=

k/ 12

r
x=

Lecture: Generating random numbers

(32)

12
k

X

k
ui
2


+ .

(33)

49

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.9. Specific method: empirical continuous distributions

Assume we have a histogram:
xi is the midpoint of the interval i;
f (xi ) is the length of the ith rectangle.

Note: the task is different from sampling from discrete distribution.

Lecture: Generating random numbers

50

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Construct the cdf as follows:

F (xi ) =

f (xk ),

(34)

k{F (xi1 ),F (xi )}

which is monotonically increasing within each interval [F (xi1 ), F (xi )].

Lecture: Generating random numbers

51

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

The algorithm:
generate u = U (0, 1);
assume that u {F (xi1 ), F (xi )};
use the following linear interpolation to get:
x = xi1 + (xi xi1 )

u F (xi1 )
.
F (xi ) F (xi1 )

(35)

Note: this approach can also be used for analytical continuous distribution.
get (xi , f (xi )), i = 1, 2, . . . , k and follow the procedure.

Lecture: Generating random numbers

52

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.10. Rejection method

Works when:
pdf f (x) is bounded;
x has a finite range, say a x b.

The basic steps are:

normalize the range of f (x) by a scale factor such that cf (x) 1, a x b;
define x as a linear function of u = U (0, 1), i.e. x = a + (b a)u;
generate pairs of random numbers (u1 , u2 ), u1 , u2 = U (0, 1);
accept the pair and use x = a + (b a)u1 whenever:
the pair satisfies u2 cf (a + (b a)u1 );
meaning that the pair (x, u2 ) falls under the curve of cf (x).

Lecture: Generating random numbers

53

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

The underlying idea:

P r{u2 cf (x)} = cf (x);
if x is chosen at random from (a, b):
we reject if u2 > cf (x);
we accept if u2 cf (x);
we match f (x).

Lecture: Generating random numbers

54

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Example: generate numbers from f (x) = 2x, 0 x 1:

1. select c such that cf (x) 1:
for example: c = 0.5.
2. generate u1 and set x = u1 ;
3. generate u2 :
if u2 < cf (u1 ) = (0.5)2u1 = u1 then accept x;
otherwise go back to step 2.

Lecture: Generating random numbers

55

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

5.11. Convolution method

The basis of the method is the representation of cdf F (x):
F (x) =

pj Fj (x),

(36)

j=1

pj 0, j = 1, 2, . . . ,

j=1

pj = 1.

Works when:
it is easy to to generate RVs with distribution Fj (x) than F (x);
hyperexponential RV;
Erlang RV.
The algorithm:
1. generate discrete RV J, P r{J = j} = pj ;
2. given J = j generate RV with Fj (x);
P
3. compute
j=1 pj Fj (x).

Lecture: Generating random numbers

56

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Example: generate from exponential distribution:

divide (0, ) into intervals (i, i + 1), i = 0, 1, . . . ;
the probabilities of intervals are given:
pi = P r{i X < i + 1} = ei e(i+1) = ei (1 e1 ),

(37)

gives geometric distribution.

the conditional pdfs are fiven by:
fi (x) = e(xi) /(1 e1 ),

i x < i + 1.

(38)

in the interval i(X i) has the pdf ex /(1 e1 ), 0 x < 1.

The algorithm:
get I from geometric distribution pi = ei /(1 e1 ), i = 0, 1, . . . ;
get Y from ex /(1 e1 ), 0 x < 1;
X =I +Y.

Lecture: Generating random numbers

57

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

6. Statistical tests for RNs with arbitrary distribution

What we have to test for:
independence;
particular distribution.

Tests for independence:

correlation tests:

Portmanteau test, modified Portmanteau test, 2/ n, etc.

note: here we test only for linear dependence...

Tests for distribution:

2 test;
Kolmogorovs test.

Lecture: Generating random numbers

58

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

7. Multi-dimensional distributions
Task: generate samples from RV (X1 , X2 , . . . , Xn ).
Write the joint density function as:
f (x1 , x2 , . . . , xn ) = f1 (x1 )f2 (x2 |x1 ) . . . f (xn |x1 . . . xn1 ).

(39)

f1 (x1 ) is the marginal distribution of X1 ;

fk (xk |x1 , . . . , xk1 ) is the conditional pdf of Xk with condition on X1 = x1 , . . . , Xk1 = xk1 .
The basic idea: generate one number at a time:
get x1 from f1 (x1 );
get x2 from f2 (x2 |x1 ), etc.
The algorithm:
get n random numbers ui = U (0, 1), i = 0, 1, . . . , n;
subsequently get the following RVs:
F1 (X1 ) = u1 ,

F2 (X2 |X1 ) = u2 ,

...

Lecture: Generating random numbers

Fn (Xn |X1 , . . . , Xn1 ) = un .

(40)

59

Network analysis and dimensioning I

D.Moltchanov, TUT, 2013

Example: generate from f (x, y) = x + y:

marginal pdf and cdf of X are given by:
Z 1
1
f (x) =
f (x, y)dy = x + ,
2
0

Z
F (x) =
0

1
f (x0 )dx0 = (x2 + x).
2

(41)

1 2
xy
+
y
2
f (y 0 |x)dy 0 =
.
x + 12

(42)

conditional pdf and cdf of Y are given by:

f (x, y)
x+y
f (y|x) =
,
=
f (x)
x + 12

Z
F (y|x) =
0

by inversion we get:
1
x = ( 8u1 + 1 1),
2
p
y = x2 + u2 (1 + 2x) x.

Lecture: Generating random numbers

(43)

60