Techniques 9.
Path testing
Functional testing Structural testing
Control Flow Graph Graphs DD-paths Independent paths McCabes Cyclomatic Metric Derivation of test cases
boundary value equivalence class decision tables
path testing data flow testing
Control Flow Graph (CFG) Path testing
Structural testing method Based on the source code / pseudocode of the program or the system, and NOT on its specification Primarily used for testing imperative-style programs/designes Can be applied at different levels of granularity
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 program TRIANGLE input (a) input (b) input (c) if (a<b+c) AND (b<a+c) AND (c<a+b) then IsATriangle = T else IsATriangle = F endif if IsATriangle then if (a=b) AND (b=c) then Output = Equilateral else if (a != b) AND (b != c) AND (a != c) then Output = Scalene else Output = Isosceles endif endif else Output = Not a triangle endif end TRIANGLE
2 3 4 5
7 8 9 17 11
10 12 13 16 15 14
18 19
Source node
1 2 3 4 5 7 8 9 17 11 13 16 18 15 10 12 6
Source node Sequence nodes
1 3 4 5 6
Sequence nodes
if-then node
7 8 9 12 11
if-then-else nodes
if-then-else nodes
13 14 14 15
Sink node
19
Sink node
16
Summary of the node types
...
Src A
Graphs - terminology
Nodes (n) and edges (e) Directed / undirected graph Degree of a node deg(n)
The number of edges that have that node as an endpoint
Sequence
If-then-else
If-then
Case
B C D F H G E
Indegree of a node
The number of edges that have the node as a terminal node
Outdegree of a node
The number of edges that have the node as a start node
Number of components (p)
Component is a maximal set of connected nodes
Pre-test loop Post-test loop
Snk
Cyclomatic number of graph G V(G) = e n + 2p
Graphs - exercise
Graphs - exercise
Specify degree of nodes A to H
A 3 B 4 C 3 D 4 E F G 3 4 3 indeg(D) = 3 outdeg(D) = 1 H 0
Specify degree of nodes A to H Specify indeg of nodes B and D Specify outdeg of nodes B and D What is indeg of the source node? What is outdeg of the sink node? Compute the cyclomatic number of the graph
Specify indeg of nodes B and D
indeg(B) = 1 outdeg(B) = 3 Always 0
Specify outdeg of nodes B and D What is indeg of the source node? What is outdeg of the sink node?
Always 0
Compute the cyclomatic number of the graph
V(G) = e n + 2p = 13 10 + 4 = 7
DD path (Logical Branch)
DD: Decision to Decision path A DD-path is a sub-path in a program graph fulfilling one of the conditions below:
1. It consists of a single node with indeg = 0 2. It consists of a single node with outdeg = 0
Ensures the unique source and sink nodes
DD-path graphs
1 2 3 4 5
A CFG can be broken into DD-paths Each DD-path is collapsed into an individual node The resulting graph is called a DD-path graph of the program Every node in a DD-path graph is equivalent to a predicate Case 1 Src Other cases: C3
C4 C4 C3 C3 C4 C3
3. It consists of a single node with indeg 2 or outdeg 2
No node is contained in more than one DD-path
4. It consists of a single node with indeg = 1 and outdeg = 1
Ensures the one fragment one DD path mapping
Case 5
5. It is a maximal chain of length 1
Single entry single exit sequence of nodes
Case 2
1 2 3 4 5 7 8 9 17 11
Control flow graph
DD-path graph
Src A B
Independent (basis) paths
D
C
6
Independent path is a path through a DD-path graph of the program
(i.e. it a graph which has at least one source node and one sink node)
E F
10 12 13 16 15 14
G I
H J K N M L
which cannot be reproduced from other paths by
Addition (i.e. one path following another) Repetition (e.g. loop)
The concept is similar to that of the basis functions
18 19
O Snk
Src A B
Independent paths
Find at least one independent path in the graph
Src A B D
Src-A-B-D-E-F-H-J-K-M-N-O-Snk Src-A-B-D-E-F-H-J-L-M-N-O-Snk Src-A-B-D-E-F-H-I-N-O-Snk Src-A-B-C-E-F-G-O-Snk Src A B D E F G I H J K N O Snk M O Snk L N G I K M O Snk C E F H J L N G I K M O Snk D C E F H J L N G I K M Src A B D C E F H J L Src A B D
C E F G I
H J K N M L
P1: Src-A-B-D-E-F-H-J-K-M-N-O-Snk P2: Src-A-B-D-E-F-H-J-L-M-N-O-Snk P3: Src-A-B-D-E-F-H-I-N-O-Snk P4: Src-A-B-C-E-F-G-O-Snk
P1: Scalene P2: Isosceles P3: Equilateral P4: Not a triangle
O Snk
?
Independent paths
Src A B C E F G I K N O Snk M O Snk H J L N G I K M O Snk D C E F H J L N G I K M O Snk Src A B D C E F H J L N if IsATriangle G I K M Src A B D C E F H J L IsATriangle = F Src A B D
Lesson: Paths must be feasible Generating independent paths Generate one feasible path (a baseline path) Generate further paths by flipping each decision point in turn
Decision point is a node with outdegree 2 Flipping is taking a different edge than those taken previously A technically feasible path may not be feasible logically (according to the logic of the program)
Decision points: nodes with outdeg 2
McCabe cyclomatic metric
The number of independent paths can be predicted from a DD-path graph of the program Cyclomatic number of graph G
...
If-then-else
If-then
Case
V(G) = e n + 2p
gives (approximately!) the number of independent paths
Pre-test loop
Post-test loop
Src A B C E F G I K N O Snk G M H J L C F D
McCabe cyclomatic metric
Exercise
Compute McCabe cyclomatic metric for the graph
V(G) = e n + 2p
V(G) = 20 17 + 2 = 5 Five independent paths
B
A D E
V(G) = e n + 2p
V(G) = 10 7 + 2 = 5 5 independent paths
P1: P2: P3: P4: P5: A-B-C-G A-B-C-B-C-G A-B-E-F-G A-D-E-F-G A-D-F-G
McCabe cyclomatic metric
In practical terms the McCabes cyclomatic metric defines a lower bound on the number of tests for the Path Coverage The metric also gives an intuitive feel for program complexity in terms of the number of decision nodes and loops
Derivation of test cases
Determine a DD-path graph for the program Determine the cyclomatic number V(G)
this tells you approximately how many tests to generate
Generate test cases in the same way as the Independent Paths are generated
i.e. each test case will represent a different combination of the states of the Gate Variables
There exist tools for semi-automation of the Path Testing
Derivation of test cases
Src A program TRIANGLE input (a) input (b) input (c) if (a<b+c) AND (b<a+c) AND ((c<a+b) then IsATriangle = T else IsATriangle = F endif if IsATriangle then if (a=b) AND (b=c) then Output = Equilateral else if (a != b) AND (b != c) AND (a != c) then Output = Scalene else Output = Isosceles endif endif else Output = Not a triangle endif end TRIANGLE
Src A B C E F G I K N O M H J L D Src A
Exercise: derive test cases
program TRIANGLE input (a) input (b) input (c) if (a<b+c) AND (b<a+c) AND ((c<a+b) then IsATriangle = T else IsATriangle = F endif if IsATriangle then if (a=b) AND (b=c) then Output = Equilateral else if (a != b) AND (b != c) AND (a != c) then Output = Scalene else Output = Isosceles endif endif else Output = Not a triangle endif end TRIANGLE
Src A B
B D C E F H I J K L M N G O Snk
Snk
B D C E F H I J K L M N G O Snk
C E F G I
H J K N M L
O Snk
DD-path testing
Testing which covers every DD-path is a minimum industry accepted level of test coverage of the source code It is called path coverage metric, C1 This and other metrics (C0, C0p, C2, Cd, CMCC, Cjk, Cstat, C) are primarily criteria that measure the quality of testing and not a procedure to identify test cases. [McCabe]
DD-path testing: loops
Src A B C D E F Snk Src A B C D E F Snk
Concatenated loop
Nested loop
Both loops are post-test loops
DD-path testing: loops How many paths?
Src A B C D E Snk Src A B C D E Snk P1: Src-A-B-C-A-D-E-Snk Src A B C D E Snk P2: Src-A-D-E-Snk
DD-path testing: loop coverage
C2 metric: measures C1 coverage and loop coverage Every loop involves a decision Necessary to test both outcomes of the decision
Test inside the loop Do not enter the loop
Additional testing (modified boundary approach) by testing the loop index at
Minimum value Nominal value Maximum value
pre-test loop
For nested loop, repeat from the innermost loop and work outwards
Path Testing - conclusions
Based on code complementary to functional methods Provides useful metrics, especially valuable for discovering redundancy in the number of test cases Metrics also useful for software testing quality assurance Cumbersome to use Does not make distinction between the feasible and infeasible paths
Next lecture
Data Flow Testing
Homework Further reading
Re-write the Triangle program segment 7-14 so that the compound conditions are replaced by nested if-then-else statements. Compare the cyclomatic complexity of the new program with that of the existing version. Use the whiteBox.exe program (the Course resource page) to experiment with various sets of test cases to determine DD-path coverage for the Triangle problem and the NextDate problem Draw a CFG for two nested pre-test loops and specify all the possible paths through the graph.
Src A B C D F H G Snk E
Additional material on the web http://www.cs.bham.ac.uk/~exc/Teaching/STesting
Using the Cyclomatic Complexity Metric BCS Standard for Software Component Testing
Specify all the directed paths between nodes C and Snk for the graph on the left.
