CHAPTER 3

METHODOLOGY
3.1 Testing a Random Number Generator
Many sequences that appear random may actually be easy to predict. For this

reason, it is important to thoroughly test any generator that claims to produce

random results. A fitting description of random in this context is as follows: a

random sequence is one that cannot be described by a sequence shorter than

itself (L’Ecuyer, 2007). Attempting to find these patterns by intuition would be

difficult if not impossible. Fortunately, many tests exist that can suggest if

numbers in a sequence are random, and the algorithms in these tests are

widely used outside of random numbers. Whenever professional forecasters

make data-driven predictions, they apply formulas to determine the probability

that the results were not random. These formulas can likewise be used to

verify that a result was random, and the only thing that must change is the

passing criteria. No test can conclusively prove randomness; the best that can

be accomplished is that with enough testing, users of the generators can be

confident that the sequence is random enough. There are scholars who believe

that the source of a random sequence should dictate what tests to run. For

example, true random generators tend to exhibit bias toward values, and this
trait worsens as the hardware wears down. As a result, these scholars claim

that if a random sequence comes from a true random generator, extra tests

should be performed that check for bias (Kenny, 2005). Other scholars believe

that random is random regardless of where it came from, and that it is

appropriate to test all random sequences the same. In the following

subsections several statistical and exploratory tests will be examined, as well as

the major test suites NIST and Diehard.

3.2. Statistical Tests

One of the approaches to testing random number generators is leveraging the

wide array of statistical formulas. With this approach, each test examines a

different quality that a random number generator should have. For example, a

random generator would go through a chi-squared test to ensure a uniform

distribution, and then a reverse- arrangements test to see if the sequences

contained any trends. Confidence in the generator’s randomness is only gained

after it passes an entire suite of tests which comes at it from different

directions. These tests should be run on more than just a single sequence to

ensure that the test results are accurate. Making the act of testing more

difficult is the fact that failing a test does not indicate that a generator is not

random. When outputs are truly random, then there will be some isolated

sequences produced that appear non-random (Haahr, 2011). Tests need to be

picked carefully and tailored to the context generators are needed in. The chi-

squared, runs, next bit, and matrix based tests will be examined because of

their popularity.

CHI-SQUARED TEST

The chi-squared test is used to ensure that available numbers are uniformly

utilized in a sequence. This test is easy to understand and set up, so it is

(Oi−Ei) 2
commonly used (Foley, 2001). The formula for the test is: X = ∑
Ei

Where,

o the summation is over all the available categories.

o O represents the actual number of entries in the category, and

o E is the expected number of entries. For example, if the random

numbers were scattered one through six, then there would be six

categories, and the number of times each value appeared in the

sequence would become the values of O. When the resulting value is

above the chosen significance level, then it can be said that the values in

the sequence are uniformly distributed. Because this test is simple, it

can be run many times on different sequences with relative ease to

increase the chance of its accuracy (Foley, 2001).

The runs test

The Runs Test is a non-parametric statistical test used to examine whether

a sequence of data points is random. It analyzes the number of runs in a

sequence, where a “run” is a series of consecutive similar observations (e.g., all

positive, negative, above or below the mean, etc.).

An important trait for a random sequence is that it does not contain patterns.

The test of runs above and below the median can be used to verify this

property (Foley, 2001). In this test, the number of runs, or streaks of numbers,

above or below the median value are counted. If the random sequence has an

upward or downward trend, or some kind of cyclical pattern, the test of runs

will pick up on it. The total number of runs and the number of values above

and below the median are recorded from the sequence. Then, these values are

used to compute a z-score to determine if numbers are appearing in a random

R−µR
order. The formula for the score is: Z = σR
, where σ denotes the standard

deviation of the sequence. (Foley, 2001).

For the mean of runs, the expected number of runs under randomness is :

2N 1 N2
µR = N
+1

Also, the variance of runs under the randomness is:

2 N 1 N 2(2 N 1 N 2−N )
σ 2R =
N 2 (N −1)
The formula for the Runs Test involves the following steps:

 N1 : Number of observations of one type (e.g., positive or above the

median).

 N2: Number of observations of the other type (e.g., negative or below

the median).

 R: Total number of runs in the sequence.

 N: Total number of observations, N = N1 + N2.

Next bit test

The Next-Bit Test is a theoretical criterion used to evaluate the unpredictability

of a pseudo-random number generator (PRNG). Informally, it states that a

PRNG passes the test if, given the first k bits of an output sequence, the k + 1 -

th bit cannot be predicted with a probability significantly greater than 50%.

When testing pseudo random generators for cryptography applications, the

next bit test is a staple. In its theoretical form, the next bit test declares that a

generator is not random if given every number in the generated sequence up

to that point there is an algorithm that can predict the next bit produced with

significantly greater than 50% accuracy (Lavasani & Eghlidos, 2009). This

definition makes the next bit test virtually impossible to implement, because it

would require trying every conceivable algorithm to predict the next bit.

Instead, it can be used after a pattern is discovered to cement the fact that a

generator is insecure. Several attempts have been made to alter the next bit

test so that it can be used as an actual test. The universal next bit test

developed in 1996 was the first to allow the next bit test to be administered,

but it was shown that this test would pass non-random generators. Later, the

practical next bit test was developed and was shown to be as accurate as the

NIST test suite at the time, if not more so (Lavasani & Eghlidos, 2009). However,

this test required a large amount of resources to run, limiting its usefulness.

The next bit test remains relevant in cryptography because it has been proven

that if a generator can pass the theoretical next bit test, then it will pass every

other statistical test for randomness.

The formal mathematical representation is:

P(b k+1= 1 |b 1, b 2, ….,b k) ≈ 0.5

where:

 b 1, b 2, ….,b k are the first k bits of the PRNG output.

  b k+1 is the next bit to be predicted.

 P denotes the conditional probability.

3.3. NIST

Of the available suites for testing random number generators, the NIST suite

reigns as the industry standard (Kenny, 2005). The NIST suite was designed to

test bit sequences, with the idea that passing all NIST tests means that a

generator is fit for cryptographic purposes. Even new true random number

generators have their preliminary results run through the NIST battery to

demonstrate their potential (Li, Wang, & Zhang, 2010). The NIST suite contains

fifteen well-documented statistical tests (NIST.gov, 2008). Because

cryptography has the most stringent requirements for randomness out of all

the categories, a generator that passes the NIST suite is also random enough

for all other applications. However when a generator fails the NIST suite, it

could still be random enough to serve in areas such as gaming and simulation,

since the consequences of using less than perfectly random information is

small. NIST does not look at factors such as rate of production, so passing the

NIST suite should not be the only factor when determining a generator’s

quality.
3.4. Diehard

Another widely used suite of random number tests is known as Diehard. This

suite was invented by George Marsaglia in 1995 (Kenny, 2005). It was made to

be an update for the original random number test suite, Knuth. Knuth is named

after Donald Knuth and was published in the 1969 book The Art of Computer

Programming, Volume 2. Knuth’s tests were designed before cryptography

became a major industry, and the suite was later considered to be too easy to

pass for situations where vast quantities of random numbers were needed.

Diehard was designed to be more difficult to pass than Knuth’s suite, fulfilling

the role of a general-purpose battery for detecting non- randomness. All of the

tests are available free online, so they can be easily used to test any number

sequence (Marsaglia, 2005). The Diehard suite has not been updated since its

inception in 1995, but is still a widely used test suite (Kenny, 2005).

3.5. LIMITATIONS

Pseudo-Random Number Generators (PRNGs)

 Depend on algorithms, so they are not truly random.

 Sequences repeat after a finite period.

  Output quality depends on the initial seed; weak seeds lead to

predictable results.

 Not secure enough for cryptographic purposes.

True Random Number Generators (TRNGs)

 Require specialized hardware, which can be expensive or complex to

build.

 Output can be affected by environmental conditions like noise or

interference.

 Slower than PRNGs, especially for high-speed applications.

 May produce biased results that need correction.

3.6. SUMMARY

This chapter discusses the methodology used to test random number

generators (RNGs) to ensure that their outputs are truly random and

unpredictable. It emphasizes the importance of testing because many

sequences that appear random may be predictable. Several statistical tests and

test suites are examined, including:

1. Randomness Testing: Randomness is defined as a sequence that cannot be

described by a shorter sequence. Tests help identify whether a sequence is

random enough, even if no test can conclusively prove randomness.

2. Statistical Tests: These tests examine different qualities of RNGs:

 Chi-Squared Test: Assesses if a sequence is uniformly distributed by

comparing actual and expected frequencies of outcomes.

 Runs Test: Checks if the sequence shows any trends or patterns by

analyzing streaks (or "runs") of similar values. A z-score is used to

determine randomness.

 Next-Bit Test: Evaluates the unpredictability of pseudo-random

number generators (PRNGs), specifically in cryptography, by testing if

the next bit in the sequence can be predicted with accuracy greater

than 50%.

3. Test Suites:

 NIST Test Suite: The industry standard for testing RNGs, consisting of 15

tests. It is primarily designed for cryptographic applications, but

passing the NIST suite ensures the RNG is sufficiently random for most

uses.
  Diehard Test Suite: A general-purpose suite, developed by George

Marsaglia, that tests RNGs with greater difficulty than earlier methods.

It was designed for applications where high-quality randomness is

required but hasn't been updated since 1995.

These tests and suites are critical for verifying the randomness of number

generators, particularly for applications in cryptography, simulations, and

gaming.

Overall, testing RNGs is a complex but necessary task, as it ensures the

reliability and security of systems that rely on random sequences. These

systems are found in diverse applications, from cryptographic systems to

gaming and simulations. By using a combination of statistical tests and

established test suites like NIST and Diehard, users can be more confident in

the randomness of the generators they employ. Although these tests cannot

definitively prove that a sequence is random, they help verify that the

sequence is random enough for practical purposes, especially in high-stakes

fields such as cryptography.