WORKLOAD CHARACTERIZATION

This refers to the demands placed by the requests on the various system resources. An important
feature of computer systems is that their performance depends dramatically on the workload
(or simply load) they are subjected to. The load characterizes the quantity and the nature of
requests submitted to the system.
Workload characterization is one of the central issues in performance evaluation because it is
not always clear what aspects of the workload are important, in how much detail the workload
should be recorded, and how the workload should be represented and used.
Regardless of which performance evaluation technique is used, we need to provide input to the
model or real system under study. Many new computer and network applications and
programming paradigms are constantly emerging. Understanding the characteristics of today’s
emerging workloads is essential to design efficient and cost-effective architectures for them. It
is important to characterize web servers, database systems, and transaction processing systems,
multimedia, networks, ATM switches, and scientific workloads.
It is also useful to design models for workloads. An accurate characterization of application
and operation system behavior leads to improved architectures and designs. Analytical
modeling of workloads is a challenge and needs to be performed carefully. This is because it
takes significant amounts of time to perform trace-driven or execution-driven simulations due
to the increased complexity of the processor, memory subsystem, and the workload domain.
Quantitative characterization of workloads can help significantly in the creation and validation
of analytic models. They can capture the essential features of systems and workloads, which
can be helpful in providing early predication about the design. Moreover, quantitative and
analytical characterization of workloads is important in understanding and exploiting their
interesting features. Figure 1.1 depicts an overall block diagram of workload characterization
process.

FIGURE 1.1. Overall workload characterization process.

In this context, there are two types of relevant inputs: (a) parameters that can be controlled by
the system designer, such as resource allocation buffering technique and scheduling schemes,
and (b) input generated by the environments in which the system under study is used such as
interarrival times. Such inputs are used to drive the real system if the measurement technique
or the simulation model is used. They also can be used to determine adequate distributions for
the analytic and simulation models. In the published literature, such inputs are often called
workloads.
Workload characterization is considered an important issue in performance evaluation, as it is
not always clear what
(a) level of detail the workload should have
(b) aspects of the workload are significant, and
(c) method to be used to represent the workload.

Workload Model
 Workload characterization only builds a model of the real workload, since not every
aspect of the real workload may be captured or is relevant.
 A workload model may be executable or non-executable. For example, recording the
arrival instants and service durations of jobs creates an executable model, whereas only
determining the distributions creates a non-executable model.
 An executable model need not be a record of inputs, it can also be a program that
generates the inputs.
 Executable workloads are useful in direct measurements and trace-driven simulations,
whereas non-executable workloads are useful for analytic modeling and distribution-
driven simulations.
In workload characterization, the term ‘‘user’’ may or may not be a human being. In most
related literature, the term ‘‘workload component’’ or ‘‘workload unit’’ is used instead of user.
This means that workload characterization attempts to characterize a typical component.
Examples of workload components include (a) applications such as website, e-mail service, or
program development (b) sites such as several sites for the same company, and (c) user sessions
such as monitoring complete sessions from user login and logout and applications that can be
run during such sessions. Measured quantities, requests, and resource demands used to
characterize the workload are called parameters. Transaction types include (a) packet sizes, (b)
source and destination of packets, and (c) instructions.
In general, workload parameters are preferable over system parameters for the characterization
of workloads. The parameters of significant impact are included, whereas those of minor
impact are usually excluded.
Among the techniques that can be used to specify workload are:
(a) Averaging
The averaging is the simplest scheme. It relies on presenting a single number that summarizes
the parameter values observed, such as arithmetic mean, median/mode/geometric or harmonic
means. The arithmetic means may not be appropriate for certain applications. In such cases,
the median, mode, geometric means, and harmonic means are used. For example, in the case
of addresses in a network, the mean or median is meaningless, therefore, the mode is often
chosen.
(b) Single-parameter histogram
In the single-parameter histogram scheme, we use histograms to show the relative frequencies
of various values of the parameter under consideration. The drawback of using this scheme is
that when using individual-parameter histograms, these histograms ignore the correlation
among various parameters.
(c) Multi-parameter histogram
To avoid the problem of correlation among different parameters in the single-parameter
scheme, the multi-parameter scheme is often used. In the latter scheme, a k-dimensional
histogram is constructed to describe the distribution of k workload parameters. The difficulty
with the same technique is that it is not easy to construct joint histograms for more than two
parameters.

(d) Markov models.

Markov models are used in cases when the next request is dependant only on the last request.
In general, we can say that if the next state of the system under study depends only on the
current state, then the overall systems’ behavior follows the Markov model. Markov models
are often used in queuing analysis. We can illustrate the model by a transition matrix that gives
the values of the probabilities of the next state given present state.
(e) Clustering
The clustering scheme is used when the measured workload is made of a huge number of
components. In such a case, these huge components are categorized into a small number of
clusters such that the components in one cluster are as similar to each other as possible. This is
almost similar to what is used in clustering in pattern recognition. One class member may be
selected from each cluster to be its representative and to conduct the needed study to find out
what system design decisions are needed for that cluster/group.
(f) use of dispersion measures such as coefficient of variation (COV)
The use of dispersion measure can give better information about the variability of the data, as
the mean scheme alone is insufficient in cases where the variability in the data set is large. The
variability can be quantified using the variance, standard deviation or the COV.
𝑛

𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 = 𝑆 2 = 1/(𝑛 − 1) ∑(𝑥𝑖 − 𝑥̅ )2

𝑖=1

and 𝐶𝑂𝑉 = 𝑆/𝑥̅

where
𝑆 2 = sample variance
𝑥𝑖 = the value of the one observation
𝑥̅ = the sample mean or the mean value of all observations
𝑛 = the number of observations
A high COV means high variance, which means in such a case, the mean is not sufficient. A
zero COV means that the variance is zero, and in such a case, the mean value gives the same
information as the complete data set.
(g) Principal component analysis.
The principal-component analysis is used to categorize workload components using the
weighted sum of their parameter values. If 𝑑𝑖 is the weight for the 𝑖 𝑡ℎ parameter 𝑥𝑖 , then the
weighted sum W is as follows:
𝑘

𝑊 = ∑ 𝑑𝑖 𝑥𝑖
𝑖=1

The value of 𝑊𝑖 is called the principal factor or principal component. In general, if we are given
a set of k parameters, such as x1, x2,…, xn, then the principal component analysis produces a
set of factors and W1, W2,…, Wk, such that: (a) the W’s are linear combinations of x’s, (b) the
W’s form an orthogonal set, which means that their inner product is zero: Inner Product =
∑ 𝑊𝑗 × 𝑊𝑗 = 0.

1.7.2 Case Study: Website Characterization

The phenomenal growth of the World-Wide Web (WWW), in both the volume of information
on it and the numbers of users desiring access to it, is dramatically increasing the performance
requirements for large-scale information servers. WWW server performance is a central issue
in providing universal, reliable, and efficient information access.
It is important that the WWW traffic workload be understood as it is crucial in the analysis of
a server’s performance. Capturing the main characteristics of such systems, such as the
distributions of file sizes and buffering schemes, is vital to provide a quantitative measure of
the aggregate overall advantage of a particular server system’s optimization. Workload
generators that can be used for such systems include SpecWeb96, WebStone, and SURGE.
In the characterization of a web server, we need to choose parameters that best describe the
characteristics of the workload of the servers and system software used, monitor the systems
to obtain some raw performance data, analyze performance data, and finally construct a
workload model of the system under investigation. Workload characterization allows us to
understand the current state of the system under investigation. Characterizing workload is also
essential to the design of new system components.
1.8 PERFORMANCE EVALUATION CHECKLIST
PE1 Define your goal. For example: dimension of the system, find the overload behaviour;
evaluate alternatives. Do you need a performance evaluation study? Aren’t the results obvious?
Are they too dependent on the input factors, which are arbitrary?
PE2 Identify the factors. What are all the factors? Are there external factors which need to be
controlled?
PE3 Define your metrics. For example: response time, server occupancy, number of
transactions per hour, Joule per Megabyte. Define not only what is measured but also under
which condition or sampling method. If the metric is multidimensional, different metric values
are not always comparable and there may not be a best metric value. However, there may be
non-dominated metric values.
PE4 Define the offered load. How is it expressed: transactions per second, number of users,
number of visits per hour? Is it measured on a real system? Artificial load generated by a
simulator, by a synthetic load generator? Load model in a theoretical model?
PE5 Know your bottlenecks. The performance often depends only on a small number of
factors, often those whose utilization (= load/capacity) is high. Make sure what you are
evaluating is one of them.
PE6 Know your system well. Know the system you are evaluating and list all factors. Use
evaluation tools that you know well. Know common performance patterns for your system.