Unit 3 – Domain Testing
Domain Testing U3
We will see in this part of Unit 3:
• Domains & Paths
• Nice & Ugly Domains
• Domain Testing
• Domains & Interfaces Testing
• Domains & Testability
ref boris beizer 2
Transaction-Flow Testing U3
Contents of the unit
• Domains
• Model, as a set, paths
• Predicates
• closure, dimensionality, assumptions
• Nice & ugly domains
• specified & implemented domains
• Nice domains – properties, testing implications
• Ugly domains, simplification by programmers & testers
• Domain testing
• Bugs & testing for those bugs
• Testing procedure
• Variations, tools, effectiveness
• Domains & Interface Testing
• Range, closure, span
• Interface range, domain compatibility
• Finding the values
• Domains & Testability
• Linearizing, coordinate transformations.
ref borisCanonical
beizer program form 3
Domain Testing - Domains & Paths U3
Domain Testing Model
D1
D2
INPUT CLASSIFY DO CASE 1 OUTPUT
D3
(x,y,z,…) Dn { Outcome }
DO CASE 2
ref boris beizer
(1, 2, 3, 4, 5,…)
DO CASE 3
DO CASE n
4
Domain Testing U3
Domain Testing
Views Programs as input data classifiers
ref boris beizer
Verifies if the classification is correct
5
Domain Testing - Domains & Paths U3
Domain Testing Model
Two Views
• Based on Specs
ref boris beizer
• Functional Testing
• Based on Implementation information
• Structural Technique
6
Domain Testing - Domains & Paths U3
Domain Testing Model
• Variables
• Simple combinations of two variables
• Numbers from - to +
ref boris beizer
• Input variables as numbers
• Loop-free Programs
7
Domain Testing - Domains & Paths U3
Domain Testing Model
• Structural Knowledge is not needed. Only Specs.
ref boris beizer
• For each Input Case,
• A Hypothetical path for Functional testing
• An Actual path for Structural testing
8
Domain Testing - Domains & Paths U3
Domain (simplified)
• A Single Connected Set of numbers D1
• No arbitrary discrete sets
• Defined by the boundaries
ref boris beizer
• One or more boundaries
• Specified by predicates
• Bugs likely at the boundaries
• One or more variables
9
Domain Testing - Domains & Paths U3
Predicates
• Interpretation
• Structural Testing - CFG
ref boris beizer
• Functional Testing - DFG
• Specifies the Domain Boundary
• n Predicates in Sequence 2n domains
• Or, just two domains A .AND. B .AND. C 10
Domain Testing - Domains & Paths U3
Paths
• At least One PATH thru the Program
• More paths for disconnected domains
ref boris beizer
11
Domain Testing - Domains & Paths U3
Summary:
• Domain for a loop-free program corresponds to a set of
numbers defined over the input vector
ref boris beizer
• For every domain, there is at least one path thru the routine,
along which that domain’s processing is done
• The set of interpreted predicates traversed on that path (ie.,
the path’s predicate expression) defines the domain’s
boundaries. 12
Domain Testing - Domains & Paths - Domain Closure U3
Boundary: Closed / Open
X >= MIN X <= MAX
MAX
MIN
D1 D2 D3
Closed
ref boris beizer
X >= MIN D2 X < MAX D3
D1
Closed Open
MIN MAX
13
Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open
MAX
MIN
D2 D3
D1
ref boris beizer
Open
X > MIN X < MAX
14
Domain Testing - Domains & Paths - Domain Dimensionality U3
• One dimension per variable
• At least one predicate
ref boris beizer
• Slices thru previously defined Domain
• Slicing Boundary
• N-spaces are cut by Hyperplanes 15
Domain Testing - Domains & Paths – Bug Assumptions U3
• Processing is OK. Domain definition may be wrong.
Boundaries are wrong.
Predicates are wrong.
ref boris beizer
• Once input vector is set on the right path, it’s correctly
processed.
• More bugs causing domain errors … 16
Domain Testing - Domains & Paths – Bug Assumptions.. U3
• Double-zero representation
• Floating point zero check
ref boris beizer
• Contradictory domains
defined
17
• Ambiguous domains
undefined
Domain Testing - Domains & Paths – Bug Assumptions.. U3
• Over-specified domains
X > 3 .AND. X < 2 .AND. Y > 3
• Boundary errors
ref boris beizer
Boundary closure
18
Shifted
Domain Testing - Domains & Paths - Bug Assumptions U3
• Boundary errors….
Tilted
ref boris beizer
Missing
19
Extra boundary
Domain Testing - Domains & Paths - Bug Assumptions U3
X >= 3
• Closure Reversal
ref boris beizer
• Faulty Logic
Simplification of compound 20
predicates
Domain Testing - Domains & Paths - Bug Assumptions U3
ref boris beizer
21
Strategy for domain testing in 2-dim
Domain Testing - Domains & Paths - Restrictions U3
In General…
1. Coincidental Correctness
DT cannot detect
ref boris beizer
Example F1(x) : +1
D1
• Representative outcome
Partition testing D2
F2(x) : -1 22
Input Equivalence
Domain Testing - Domains & Paths U3
Domain Testing Model
Function f1
D1
INPUT CLASSIFY DO CASE 1 OUTPUT
D2
(x, y, z, …) { Outcome }
DO CASE 2
ref boris beizer
(1, 2, 3, 4, 5,…) f2
D3
DO CASE 3 f3
Dn
f4
DO CASE n
23
Domain Testing - Domains & Paths - - Restrictions U3
3. Simple boundaries & Compound predicates
D1
X X >=0 .AND. X <= 16
0 16
ref boris beizer
D1
Y Y = 1 .AND. X >=0 .AND. X <= 16
0 16
24
Domain Testing - Domains & Paths - - Restrictions U3
Compound predicates….
• Impact of .OR.
• Concave, Disconnected
ref boris beizer
• Adjacent domains with same function
• Example
ABC + DEF
25
• Eliminate compound predicates
Domain Testing - Domains & Paths - Restrictions U3
4. Functional Homogeneity of Bugs
• Functional form still Retained
ref boris beizer
a x + b y >= c
• Bugs are only in a, b, c
26
Domain Testing - Domains & Paths - - Restrictions U3
5.Linear Vector Space
• Linear boundary predicate, Interpreted
• Simple relational operators
ref boris beizer
• Conversion to linear vector space
• 2-d Polar co-ordinates
• Polynomials
• Problems with Non-linear Boundaries 27
Domain Testing - Domains & Paths - Restrictions U3
6. Loop-free Software
Predicate for each iteration
ref boris beizer
Loop over the entire transaction
Definite loop
28
Domain Testing - Domains & Paths – Nice & Ugly domains U3
Nice Domains
Requirements
• Bugs ill-defined domains
ref boris beizer
Before DT
• Analyze specs
29
• Make the Boundary Specs Consistent & Complete
Domain Testing - Domains & Paths – Nice domains U3
Implemented Domains
• Complete, Consistent & Process all inputs
ref boris beizer
Specified Domains
• Incomplete, Inconsistent
30
Programmer’s / Designer’s Effort
Domain Testing - Domains & Paths – Nice domains U3
Nice Domains
• Linear, Complete, Systematic, Orthogonal, Consistently Closed,
& Convex
U1 U2
ref boris beizer
V1 D11 D12
V2 D21 D22
31
Domain Testing - Domains & Paths – Nice domains U3
Nice Domains
Advantages
ref boris beizer
• Ease of DT
• Fewer Bug Occurrences
32
Domain Testing - Domains & Paths – Nice domains U3
Nice Domains
Boundaries are
5. Closure consistency
1.Linear
ref boris beizer
1.Complete
6. Convex
2.Systematic
1.Orthogonal 7. Simply connected
33
Domain Testing - Domains & Paths – Nice domains U3
1.Linear Boundaries
Interpreted linear inequalities
n-dim Hyperplane:
ref boris beizer
n+1 Points
n+1 + 1 Test Cases
Non-Linear
34
• Transform
Domain Testing - Domains & Paths – Nice domains U3
2.Complete Boundaries
• Span the total number space (-, +)
• One set of Tests
ref boris beizer
Incomplete…
• Reasons
35
• Tests
Domain Testing - Domains & Paths - Nice domains U3
3. Systematic Boundaries
• Linear Inequalities differing by a constant
fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c
ref boris beizer
• Parallel lines
• Identical Sectors in a Circle
• DT
• Test a domain Tests for other Domains 36
Domain Testing - Domains & Paths - Nice domains U3
4. Orthogonal Boundaries
• Two boundaries or, boundary sets Uj
• Parallel to axes
ref boris beizer
Vj
• DT
• Each Set Independently
• # Tests O (n) 37
Domain Testing - Domains & Paths - Nice domains U3
Orthogonal Boundaries
• Tilted sets
• transformation
• Test cases: O(n)
ref boris beizer
• Concentric circles with radial lines
• Rectangular coordinates
• Polar
r ≥ aj .AND. r < aj+1 .AND.
≥ j .AND. < j+1 38
Domain Testing - Domains & Paths - Nice domains U3
Non-Orthogonal Boundaries
• Test Intersections O ( n2 ) + O (n)
ref boris beizer
39
Domain Testing - Domains & Paths - Nice domains U3
5. Closure Consistency
• A Simple pattern in all boundary closures
• Example
ref boris beizer
• Same relational operator for systematic boundaries
D3 D1 D2 D4
40
Domain Testing - Domains & Paths - Nice domains U3
6. Convex Domain
• Line joining any two points lies with in the domain
• DT
• n on-points & 1 off-pt
ref boris beizer
Concave
• “ But, However, Except, Or … “ in Specs
• Handle with special care
41
Domain Testing - Domains & Paths - Nice domains U3
7. Simply Connected Domain
• In a single piece D2 D1 D3
D2
D1
ref boris beizer
• 2 complementary domains
• D1: Convex D2: Concave, not-Connected
• Programmer / Designer
42
• Convex part First
Domain Testing - Domains & Paths – Ugly Domains U3
Generally,
From Bad Specs
ref boris beizer
Programmer / Designer Simplifies => Ugly to Good
possibility of Introduction of more bugs ?!?
43
Domain Testing - Domains & Paths – Ugly Domains U3
Causes
1. Non-linear Boundaries
ref boris beizer
2. Ambiguities & Contradictions
3. Simplifying the topology
44
4. Rectifying boundary closures
Domain Testing - Domains & Paths – Ugly Domains U3
1. Non-linear Boundaries
• Transform
ref boris beizer
45
Domain Testing - Domains & Paths – Ugly Domains U3
2. Ambiguities
• Holes in input vector space.
• Missing boundary
ref boris beizer
• Detected by Specification languages & tools.
46
Domain Testing - Domains & Paths – Ugly Domains U3
2. Contradictions..
• Overlapping of
• Domain Specs
ref boris beizer
• Closure Specs
D3
D1 D2
47
Domain Testing - Domains & Paths – Ugly Domains U3
3. Simplifying the Topology….
• Complexity !
ref boris beizer
• Concavity, Holes, Disconnectedness
48
Domain Testing - Domains & Paths – Ugly Domains U3
3. Simplifying the Topology….
• Smoothing out concavity
ref boris beizer
• Filling in Holes
49
Domain Testing - Domains & Paths – Ugly Domains U3
3. Simplifying the Topology….
• Joining the pieces
ref boris beizer
Correct:
• Connect disconnected boundary segments
• Extend boundaries to infinity 50
Domain Testing - Domains & Paths – Ugly Domains U3
4. Rectifying Boundary Closures.
• make closures in one direction for parallel boundaries with
closures in both directions
• Force a Bounding Hyperplane to belong to the Domain.
ref boris beizer
51
Consistent Direction Inclusion / Exclusion Consistency
Domain Testing - Domains & Paths – Domain Testing U3
General DT Strategy
1. Select test points near the boundaries.
2. Define test strategy for each possible bug related to boundary
ref boris beizer
3. Test points for a domain useful to test its adjacent domain.
4. Run the tests. By post test analysis determine if any boundaries
are faulty & if so how?
5. Run enough tests to verify every boundary of every domain 52
Domain Testing - Domains & Paths – Domain Testing U3
DT for Specific Domain Bugs
Generally,
• Interior point
ref boris beizer
• Exterior point
• Epsilon neighborhood
• Extreme point
• On point 53
Domain Testing - Domains & Paths – Domain Testing U3
DT for Specific Domain Bugs
Off Points
Extreme
point
ref boris beizer
Boundary point
Epsilon neighborhood
On Points
54
Domain D1
Domain Testing - Domains & Paths – Domain Testing U3
1. 1-d Domains
2. 2-d Domains
ref boris beizer
3. Equality & inequality Predicates
4. Random Testing
55
5. Testing n-dimensional Domains
Domain Testing - Domains & Paths – Domain Testing U3
Testing 1-d Domains : Bugs with open boundaries
B A B A
Closure Bug
x x
ref boris beizer
B A
Shift left Bug
x1 x
B A
Shift Right Bug
x x1
56
Domain Testing - Domains & Paths – Domain Testing U3
Testing 1-d Domains : Bugs with open boundaries
B A B A
Missing Boundary x
x x
ref boris beizer
Extra Boundary B A C
x x
x x1
Bugs with Closed Boundaries : Similar to the above
57
Domain Testing - Domains & Paths – Domain Testing U3
2. Testing 2-d Domains
• Closure bug
• Boundary Shift : up / down
ref boris beizer
• Tilted Boundary
• Extra Boundary
• Missing Boundary
58
Domain Testing - Domains & Paths – Domain Testing U3
ref boris beizer
2 n : Domains share tests
3 n : no sharing of tests by domains
59
Strategy for domain testing in 2-dim
Domain Testing - Domains & Paths – Domain Testing U3
3. Equality & Inequality Predicates
• An Equality predicate defines a line in 2-d
a
A
ref boris beizer
c c’
c
d
B
b
To avoid bugs
60
Domain Testing - Domains & Paths – Domain Testing U3
4. Random Testing
• A Point in the center : verifies computation
ref boris beizer
61
Domain Testing - Domains & Paths – Domain Testing U3
5. Testing n-Dimensional Domains (strategy)
n-dimensions, p boundary segments
• (n+1)*p test cases : n on points & 1 off point
ref boris beizer
• Extreme pt shared : 2 * p points
• Equalities over m-dimensions create a subspace of n-m
dimensions
• Orthogonal domains with consistent boundary closures,
orthogonal to the axes & complete boundaries
• Independent testing 62
Domain Testing - Domains & Paths – Domain Testing U3
Procedure for solving for values
Simple procedure. Need tools.
1. Identify input variables
ref boris beizer
2. Identify variables which appear in domain-defining predicates,
such as control-flow predicates
63
Domain Testing - Domains & Paths – Domain Testing U3
Procedure
3. Interpret all domain predicates in terms of input variables:
• Transform non-linear to linear
• Find data flow path
ref boris beizer
4. Predicate expression with p # predicates.
Find # domains : < 2 p
5. Solve inequalities for extreme points
6. Use extreme points to solve for nearby on points …
64
Domain Testing - Domains & Paths – Domain Testing U3
Effectiveness
• Cost effective
• Bugs on boundaries, extreme points
ref boris beizer
• Hardware logic testing – tool intensive
65
Domain Testing - Domains & Paths – Domain Testing U3
Domains & Interface Testing
• Domain
• Range
ref boris beizer
Range
Variable Domain
Function /
Classify
Routine
66
Domain Testing - Domains & Paths – Domain Testing U3
Domains & Interface Testing
Variable
Range for Domain for Range for
Routine 1 Routine1
Routine 2 Routine2
Domain Routine2
ref boris beizer
• Span compatibility
67
Domain Testing - Domains & Paths – Domain Testing U3
Domains & Interface Testing
• Closure compatibility
ref boris beizer
68
Domain Testing - Domains & Paths – Domain Testing U3
Interface Range/Domain Compatibility Testing
• Test each var independently
ref boris beizer
• Find an inverse function
• Build a super domain
69
Domain Testing - Domains & Paths – Domain Testing U3
Finding the values / solving inequalities
ref boris beizer
70
Domain Testing - Domains & Paths – Domain Testing U3
Domains and Testability
• Linearizing transformations
• Polynomial
• Rectangular to polar
ref boris beizer
• Generic & rational => Taylor series
71
Domain Testing - Domains & Paths – Domain Testing U3
Domains and Testability
Perform transformations for better testing.
• Co-ordinate transformations
ref boris beizer
72
Domain Testing - Domains & Paths – Domain Testing U3
Domains and Testability
• Canonical Program Form
ref boris beizer
73
Domain Testing – Questions from the previous year’s exams U3
1. Explain Nice & Ugly Domains.
2. What is domain testing? Discuss applications of domain testing
3. Explain the domain boundary bugs for two dimensional domains
ref boris beizer
4. What is the purpose of Domain Testing? Give its schematic representation
5. Define the following concepts.
1. Domains 2. Domain closure 3. Domain dimensionality
4. Bug assumptions for Domain Testing
6. Explain simple domain boundaries & compound predicates. 74
7. Classify what can go wrong with boundaries, then define a test strategy for
