Software Testing and Quality Assurance
Theory and Practice
Chapter 4
Control Flow Testing
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 1
2
• Basic Idea
© Naik & Tripathy
• Outline of Control Flow Testing
• Control Flow Graph
• Paths in a Control Flow Graph
• Path Selection Criteria
• Generating Test Input
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
Containing Infeasible Paths
Outline of the Chapter
•
• Summary
Two kinds of basic program statements:
3
•
– Assignment statements (Ex. x = 2*y; )
© Naik & Tripathy
– Conditional statements (Ex. if(), for(), while(),
…)
• Control flow
– Successive execution of program statements is
viewed as flow of control. Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– Conditional statements alter the default flow.
• Program path
– A program path is a sequence of statements from
entry to exit.
Basic Idea
– There can be a large number of paths in a
program.
– There is an (input, expected output) pair for each
path.
– Executing a path requires invoking the program
unit with the right test input.
– Paths are chosen by using the concepts of path
selection criteria.
• Tools: Automatically generate test inputs
from program paths.
Outline of Control Flow Testing
Figure 4.1: The process of generating test input data for control flow testing.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 4
Inputs to the test generation process
5
•
– Source code
© Naik & Tripathy
– Path selection criteria: statement, branch, …
• Generation of control flow graph (CFG)
– A CFG is a graphical representation of a program
unit.
Outline of Control Flow Testing
– Compilers are modified to produce CFGs. (You Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
can draw one by hand.)
• Selection of paths
– Enough entry/exit paths are selected to satisfy
path selection criteria.
• Generation of test input data
– Two kinds of paths
• Executable path: There exists input so that
the path is executed.
• Infeasible path: There is no input to execute
the path.
– Solve the path conditions to produce test input
for each path.
Control Flow Graph
•
Figure 4.2: Symbols in a control flow graph
Symbols in a CFG
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 6
Control Flow Graph
Figure 4.4: A high-level CFG representation of
openfiles().
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 7
• Example code: ReturnAverage()
8
public static double ReturnAverage(int value[], int AS, int MIN, int MAX){
/* Function: ReturnAverage Computes the average of all those numbers in
© Naik & Tripathy
the input array in
the positive range [MIN, MAX]. The maximum size of the array is AS.
But, the array size
could be smaller than AS in which case the end of input is represented by -
999. */
int i, ti, tv, sum;
double av;
i = 0; ti = 0; tv = 0; sum = 0;
while (ti < AS && value[i] != -999) {
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
ti++;
if (value[i] >= MIN && value[i] <= MAX) {
tv++;
Control Flow Graph
sum = sum + value[i];
}
i++;
}
if (tv > 0)
av = (double)sum/tv;
else
av = (double) -999;
return (av);
}
Figure 4.6: A function to compute the average of
selected integers in an array.
Control Flow Graph
Figure 4.7: A CFG representation of ReturnAverage().
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 9
10
• Statement coverage criterion
© Naik & Tripathy
– Statement coverage means executing individual
program statements and observing the output.
– 100% statement coverage means all the statements
1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13
have been executed at least once.
• Cover all assignment statements.
• Cover all conditional statements.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– Less than 100% statement coverage is unacceptable.
Path Selection Criteria
1-2-3(F)-10(F)-11-13
Table 4.4: Paths for statement coverage of the CFG of
Figure 4.7.
SCPath1
SCPath2
Branch coverage criterion
11
•
– A branch is an outgoing edge from a node in a
© Naik & Tripathy
CFG.
• A condition node has two outgoing branches
– corresponding to the True and False values
of the condition.
– Covering a branch means executing a path that
contains the branch. Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– 100% branch coverage means selecting a set of
Path Selection Criteria
paths such that each branch is included on some
path.
Path Selection Criteria
Figure 4.8: The dotted arrows represent the branches not covered
by the statement covering in Table 4.4.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 12
13 © Naik & Tripathy
1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-12-13
• Branch coverage criterion
1-2-3(T)-4(T)-5-6(T)-7(F)-9-3(F)-10(F)-11-13
– A branch is an outgoing branch (edge) from a node in a
CFG.
1-2-3(T)-4(T)-5-6(F)-9-3(F)-10(F)-11-13
• A condition node has two outgoing branches –
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
corresponding to the True and False values of the
condition.
Path Selection Criteria
– Covering a branch means executing a path that
contains the branch.
1-2-3(T)-4(F)-10(F)-11-13
– 100% branch coverage means selecting a set of paths
such that each branch is included on some path.
1-2-3(F)-10(F)-11-13
Table 4.5: Paths for branch coverage of the flow graph of
Figure 4.7.
BCPath 1
BCPath 2
BCPath 3
BCPath 4
BCPath 5
Predicate coverage criterion
14
•
– If all possible combinations of truth values of the
© Naik & Tripathy
conditions affecting a path have been explored
under some tests, then we say that predicate
coverage has been achieved.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
Path Selection Criteria
Path Selection Criteria
Figure 4.9: Partial control flow graph with (a) OR operation and (b)
AND operation.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing) © Naik & Tripathy 15
Having identified a path, a key question is
16
•
how to make the path execute, if possible.
© Naik & Tripathy
– Generate input data that satisfy all the conditions
on the path.
• Key concepts in generating test input data
– Input vector
– Predicate Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– Path condition
Generating Test Input
– Predicate interpretation
– Path predicate expression
– Generating test input from path predicate
expression
Input vector
17
•
– An input vector is a collection of all data entities
© Naik & Tripathy
read by the routine whose values must be fixed
prior to entering the routine.
– Members of an input vector can be as follows.
• Input arguments to the routine
• Global variables and constants
• Files
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
• Contents of registers (in Assembly language
Generating Test Input
programming)
• Network connections
• Timers
– Example: An input vector for openfiles() consists
of individual presence or absence of the files
“files1,” “file2,” and “file3.”
– Example: The input vector of ReturnAverega()
shown in Figure 4.6 is <value[], AS, MIN,
MAX>.
Predicate
18
•
– A predicate is a logical function evaluated at a
© Naik & Tripathy
decision point.
– Example: ti < AS is a predicate in node 3 of
Figure 4.7.
– Example: The construct OB is a predicate in node
5 in Figure 4.9.
• Path predicate
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– A path predicate is the set of predicates
Generating Test Input
associated with a path.
– Figure 4.10: An example path from Fig. 4.7:
• 1-2-3(T)-4(T)-5-6(T)-7(T)-8-9-3(F)-10(T)-
12-13.
– Figure 4.11: The path predicate for the path
shown in Figure 4.10.
ti < AS ≡ True
value[i] != -999 ≡ True
value[i] >= MIN ≡ True
value[i] <= MAX ≡ True
ti < AS ≡ False
tv > 0 ≡ True
Predicate interpretation
19
•
– A path predicate may contain local variables.
© Naik & Tripathy
– Example: <i, ti, tv> in Figure 4.11 are local
variables.
– Local variables play no role in selecting inputs
that force a path to execute.
– Local variables can be eliminated by a process
called symbolic execution.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– Predicate interpretation is defined as the process
Generating Test Input
of
– symbolically substituting operations along a
path in order to express the predicate solely in
terms of the input vector and a constant
vector.
– A predicate may have different interpretations
depending on how control reaches the predicate.
Path predicate expression
20
•
– An interpreted path predicate is called a path
© Naik & Tripathy
predicate expression.
– A path predicate expression has the following
attributes.
• It is void of local variables.
• It is a set of constraints in terms of the input
vector, and, maybe, constants.
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
• Path forcing inputs can be generated by
Generating Test Input
solving the constraints.
• If a path predicate expression has no
solution, the path is infeasible.
– Figure 4.13: Path predicate expression for the
path shown in Figure 4.10.
0 < AS ≡ True …… (1)
value[0] != -999 ≡ True …… (2)
value[0] >= MIN ≡ True …… (3)
value[0] <= MAX ≡ True …… (4)
1 < AS ≡ False …… (5)
1>0 ≡ True …… (6)
Path predicate expression
21
•
– An example of infeasible path
© Naik & Tripathy
– Figure 4.14: Another example of path from
Figure 4.7.
• 1-2-3(T)-4(F)-10(T)-12-13
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
– Figure 4.15: Path predicate expression for the
path shown in Figure 4.14.
Generating Test Input
0 < AS ≡ True …… (1)
value[0] != -999 ≡ True …… (2)
0>0 ≡ True …… (3)
Generating input data from a path predicate
22
•
expression
© Naik & Tripathy
– Consider the path predicate expression of Figure
4.13 (reproduced below.)
0 < AS ≡ True …… (1)
value[0] != -999 ≡ True …… (2)
value[0] >= MIN ≡ True …… (3)
value[0] <= MAX ≡ True …… (4)
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
1 < AS ≡ False …… (5)
Generating Test Input
1>0 ≡ True …… (6)
– One can solve the above equations to obtain the
following test input data
AS =1
MIN = 25
MAX = 35
Value[0] = 30
– Note: The above set is not unique.
A program unit may contain a large number
23
•
of paths.
© Naik & Tripathy
– Path selection becomes a problem. Some selected
paths may be infeasible.
– Apply a path selection strategy:
• Select as many short paths as possible.
• Choose longer paths. Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
Containing Infeasible Paths
– There are efforts to write code with fewer/no
infeasible paths.
Control flow is a fundamental concept in
24
•
program execution.
© Naik & Tripathy
• A program path is an instance of execution
of a program unit.
• Select a set of paths by considering path
selection criteria.
• Statement coverage
Software Testing and QA Theory and Practice (Chapter 4: Control Flow Testing)
• Branch coverage
• Predicate coverage
• All paths
• From source code, derive a CFG (compilers
Summary
are modified for this.)
• Select paths from a CFG based on path
selection criteria.
• Extract path predicates from each path.
• Solve the path predicate expression to
generate test input data.
• There are two kinds of paths.
• feasible
• infeasible
