Order Statistics (Math313)

Prof. Magdy E. El-Adll

Department of Mathematics

Faculty of Science

Helwan University
Contents

1 Introduction 2

1.1 Chapter Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Some Basic Concepts of Statistics . . . . . . . . . . . . . . . . . . 3

1.2.1 What Is Statistics? . . . . . . . . . . . . . . . . . . . . . . 4

1.2.2 Why We Study Statistics? . . . . . . . . . . . . . . . . . . 5

1.2.3 Types of Statistics . . . . . . . . . . . . . . . . . . . . . . 6

1.2.4 Types of variables . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Basic Elements from Probability Theory . . . . . . . . . . . . . . 9

1.3.1 The Concept of Random Variable . . . . . . . . . . . . . . 10

1.3.2 Discrete Random Variables . . . . . . . . . . . . . . . . . . 11

1.3.3 Continuous Random Variables . . . . . . . . . . . . . . . 16

1
Chapter 1

Introduction

1.1 Chapter Objectives

After completing this chapter, you should be able to:

1. Demonstrate knowledge of statistical terms.

2. Distinguish between the two branches of statistics.

3. Identify types of data.

2
Order Statistics (Math 313) Prof. Magdy E. El-Adll

4. Identify the measurement level for each variable.

5. Identify the concept of random variables.

6. Distinguish between discrete and continuous random variables.

7. Find the mean, variance, standard deviation, for random variable.

1.2 Some Basic Concepts of Statistics

The study of statistics has become more popular than ever over the

past four decades or so. The increasing availability of computers and

statistical software packages has enlarged the role of statistics as a tool

for empirical research. As a result, statistics is used for research in almost

all professions, from medicine to sports. Today, college students in almost

all disciplines are required to take at least one statistics course. Almost

all newspapers and magazines these days contain graphs and stories on
3
Order Statistics (Math 313) Prof. Magdy E. El-Adll

statistical studies. After you finish reading this course, it should be

much easier to understand these graphs and stories. Every field of study

has its own terminology. Statistics is no exception. This introductory

chapter explains the basic terms of statistics. These terms will bridge our

understanding of the concepts and techniques presented in subsequent

chapters.

1.2.1 What Is Statistics?

The word statistics has two meanings. In the more common usage, statis-

tics refers to numerical facts. The numbers that represent the income of

a family, the age of a student, the percentage of passes completed by the

quarterback of a football team, and the starting salary of a typical college

graduate are examples of statistics in this sense of the word.

4
Order Statistics (Math 313) Prof. Magdy E. El-Adll

Definition 1.2.1. Statistics is the science of collecting, organizing,

presenting, analyzing, and interpreting data to assist in making de-

cisions.

1.2.2 Why We Study Statistics?

No matter what line of work you select, you will find yourself faced with

decisions where an understanding of data analysis is helpful. In order to

make an informed decision, you will need to able to:

1. Determine whether the existing information is sufficient or addition-

al information is required.

2. Gather additional information, if it is needed, in such away that it

is does not provide misleading results.

3. Summarize the information in a useful informative manner.

5
Order Statistics (Math 313) Prof. Magdy E. El-Adll

4. Analyze the available information.

5. Draw conclusions and make inferences.

In summary, there are at least three reasons for studying statistics:

1. Data is everywhere,

2. Statistical techniques are used to make many decisions that effect

our lives,

3. No matter what your future line of work, you will make decisions

that involve data.

1.2.3 Types of Statistics

The study of statistics is usually divided into two categories: descriptive

statistics and inferential statistics.

6
Order Statistics (Math 313) Prof. Magdy E. El-Adll

Definition 1.2.2. Descriptive statistics are methods of organizing,

summarizing and presenting data in an informative way.

Definition 1.2.3. Inferential statistics are methods used to deter-

mine something about a population , based on a sample.

Definition 1.2.4. A population is a collection of all possible, ob-

jects, or measurements of interest.

Definition 1.2.5. A sample is a portion, or a part, of the population

of interest.

1.2.4 Types of variables

There are two types of variables

A. A qualitative variable is nonnumeric.

7
Order Statistics (Math 313) Prof. Magdy E. El-Adll

1. Usually we are interested in the number or percent of the ob-

servations in each category.

2. qualitative data are usually summarized in graphs and par

charts.

B. There are two types of quantitative variables and they are usually

reported numerically.

1. Discrete variables can assume only certain values, and there are

usually gaps between values.

2. A continuous variables can assume any value within a specific

range or interval.

8
Order Statistics (Math 313) Prof. Magdy E. El-Adll

1.3 Basic Elements from Probability Theory

Regarded as the founder of modern probability theory, Kolmogorov was a

Soviet mathematician whose work was also influential in several other sci-

entific areas, notably in topology, constructive logic, classical mechanics,

mathematical ecology, and algorithmic information theory.

He earned his Doctor of Philosophy (PhD) degree from Moscow State

University in 1929, and two years later, he was appointed a professor in

that university. In his book, “Foundations of the Theory of Probability”,

which was published in 1933 and which remains a classic text to this day,

he built up probability theory from fundamental axioms in a rigorous

manner, comparable to Euclids axiomatic development of geometry.

9
Order Statistics (Math 313) Prof. Magdy E. El-Adll

1.3.1 The Concept of Random Variable

Before probability distribution is defined formally, the definition of a vari-

able is reviewed. In Chapter 1, a variable was defined as a characteristic

or attribute that can assume different values. Various letters of the al-

phabet, such as X, Y, or Z, are used to represent variables. Since the

10
Order Statistics (Math 313) Prof. Magdy E. El-Adll

variables in this chapter are associated with probability, they are called

random variables.

Statistics is concerned with making inferences about populations and

population characteristics. Experiments are conducted with results that

are subject to chance.

1.3.2 Discrete Random Variables

The testing of a number of electronic components is an example of

a statistical experiment, a term that is used to describe any process

by which several chance observations are generated. It is often im-

portant to allocate a numerical description to the outcome. For ex-

ample, the sample space giving a detailed description of each possi-

ble outcome when three electronic components are tested may be writ-

ten S = {N N N, N N D, N DN, DN N, N DD, DN D, DDN, DDD},

11
Order Statistics (Math 313) Prof. Magdy E. El-Adll

where N denotes nondefective and D denotes defective. One is naturally

concerned with the number of defective that occur. Thus, each point

in the sample space will be assigned a numerical value of 0, 1, 2, or 3.

These values are, of course, random quantities determined by the out-

come of the experiment. They may be viewed as values assumed by the

random variable X, the number of defective items when three electronic

components are tested.

Definition 1.3.1. A random variable is a variable that associates a

real number with each element in the sample space.

In other words, a random variable is a variable whose values are de-

termined by chance.

We shall use a capital letter, say X, to denote a random variable and

its corresponding small letter, x in this case, for one of its values. In

12
Order Statistics (Math 313) Prof. Magdy E. El-Adll

the electronic component testing illustration above, we notice that the

random variable X assumes the value 2 for all elements in the subset

E = {DDN, DN D, N DD} of the sample space S. That is, each possi-

ble value of X represents an event that is a subset of the sample space

for the given experiment.

Recall that variables can be classified as discrete or continuous by

observing the values the variable can assume. If a variable can assume

only a specific number of values, such as the outcomes for the number

of defective items when three electronic components are tested, then the

variable is called a discrete variable.

Definition 1.3.2. The set of ordered pairs (x, f (x)) is a probability

mass function, probability function, or probability distribution of the

discrete random variable X if, for each possible outcome x,

13
Order Statistics (Math 313) Prof. Magdy E. El-Adll

1. f (x) > 0,

P
2. f (x) = 1,
x

3. P (X = x) = f (x).

For the previous example of defective items, a probability distribution

can be constructed by listing the outcomes and assigning the probability

of each outcome, as shown here.

P
Number of defective items X 0 1 2 3

1 3 3 1
f (x) = P (X = x) 8 8 8 8 1

14
Order Statistics (Math 313) Prof. Magdy E. El-Adll

Definition 1.3.3. The cumulative distribution function (cdf ) F (x)

of a discrete random variable X with probability distribution f (x) is

X
F (x) = P (X 6 x) = f (t), for − ∞ < x < ∞.
t6x

Definition 1.3.4. The mean and the variance of the discrete random

variable X can be obtained, respectively, by

X
µ = E [X] = xf (x).
all x

and
X
2
 2 2
σ =E X −µ = x2f (x) − µ2.
all x

Definition 1.3.5. The moment-generating function of the discrete

random variable X is given by

 tX  X tx
M (t) = E e = e f (x).
all x

Clearly, M (0) = 1. Explain?

15
Order Statistics (Math 313) Prof. Magdy E. El-Adll

1.3.3 Continuous Random Variables

Definition 1.3.6. We say that the integrable function f (x) is a

probability density function (pdf ) of a random variable X of the

continuous type, with space R that is an interval or union of in-

tervals, if the following three conditions are satisfied:

1. f (x) > 0, x ∈ R.

R
2. R f (x)dx = 1.

3. The probability of the event X ∈ A is

Z
P (X ∈ A) = f (x)dx.
A

Example 1.3.1. Let the random variable X be the distance in

millimeters between bad records on a used computer tap. Suppose

16
Order Statistics (Math 313) Prof. Magdy E. El-Adll

that a reasonable probability model for X is given by the pdf

1
f (x) = e−x/2, 0 ≤ x ≤ ∞.
2

Show that f (x) is a pdf and find P (X > 2)

Solution. Not that

R = {x : 0 ≤ x ≤ ∞} and f (x) > 0, for x ∈ R.

Also,
Z Z ∞
1 −x/2
f (x)dx = e dx
R 0 2
h ib
−x/2
= lim −e
b→∞ 0

= 1 − lim e−b/2 = 1.
b→∞

The probability that the distance between bad records is greater than 2

millimeters is
Z ∞
1 −x/2
P (X > 2) = e dx = e−1 = 0.368.
2 2
17
Order Statistics (Math 313) Prof. Magdy E. El-Adll

We extend the definition of the pdf f (x) to the entire set of real numbers

by letting it equal zero when x 6∈ R. For example,


 1
 e−x/2, 0 ≤ x < ∞,


f (x) = 2

 0, elsewhere,



has the properties of a pdf of a continuous-type random variable X

having support {x : 0 ≤ x < ∞}. It always be understood that

f (x) = 0 when x 6∈ R, even when this is not explicitly written out.

The cumulative distribution function (cdf) of a random variable X

of the continuous type, defined in terms of the pdf pdf of X, is given

by
Z x
F (x) = P (X ≤ x) = f (t)dt.
−∞

Example 1.3.2. Let Y be a continuous random variable with pdf

18
Order Statistics (Math 313) Prof. Magdy E. El-Adll

g(y) = 2y, 0 < y < 1. Find the cdf of Y and

   
1 3 1
P <Y ≤ and P ≤Y ≤2 .
2 4 4

Solution. The cdf of Y is given by






 0, y < 0;



Ry
G(y) = 2
 0 2tdt = y , 0 ≤ y < 1;




y ≥ 1.

 1,


we can evaluate the required probabilities using the cdf G(y) as follows
     
1 3 3 1
P <Y ≤ = G −G
2 4 4 2
 2  2
3 1 5
= − =
4 2 16

and
   
1 1
P ≤Y ≤2 = G (2) − G
4 4
 2
1 15
= 1− = .
4 16
19
Order Statistics (Math 313) Prof. Magdy E. El-Adll

The expected value of X or mean of X is

Z ∞
µ = E[X] = xf (x)dx.
−∞

The variance of X is
Z ∞
2 2
σ = V ar(X) = E[(X − µ) ] = (x − µ)2f (x)dx.
−∞

The standard deviation of is

p
σ = V ar(X).

The moment-generating function of the continuous random variable X

is given by
Z ∞
tX
M (t) = E[e ] = etxf (x)dx, −h < t < h.
−∞

20
Order Statistics (Math 313) Prof. Magdy E. El-Adll

21
Order Statistics (Math 313) Prof. Magdy E. El-Adll

22