Module - Test Plan Development
© Nex-G Exuberant Solutions Pvt. Ltd. 1
Test Plan Development
Test Scoping
Two questions you can expect from your clients when discussing potential test
engagements are
1. "How long will this testing take?"
2. "How much will it cost?"
You would be wise to refrain from giving an answer to either question until you have a
good understanding of the scope of what you will be testing. Defining the test scope will
help you estimate the extensiveness of the test process and help you forecast costs.
For example, a bug fix verification test is a test with a narrow scope and would not
normally require a complicated lab topology to be built or an extended set of test cases
to complete. The test scope would broaden as more devices, features, and test cases
are added to the requirements.
© Nex-G Exuberant Solutions Pvt. Ltd. 2
Test Plan Development
It is sometimes difficult to define the scope of a test when you do not have a technical
understanding of the solution or design you are being asked to verify. For this reason,
consider involving your most senior technical people during the scoping exercise,
despite their protestations to avoid it.
Whenever possible, you should spend time with the network architects to review the
proposed network design prior to scoping a test so that you can accurately estimate the
necessary tools, people, and equipment.
© Nex-G Exuberant Solutions Pvt. Ltd. 3
Test Plan Development
The sections that follow describe some considerations when scoping a test engagement,
the steps for which are as follows:
• Step 1. Categorize the type of test to be completed.
• Step 2. Identify project stakeholders.
• Step 3. Identify indicators of test success.
• Step 4. Estimate the resources required to complete the test.
• Step 5. Identify risks.
• Step 6. Identify the timeline for completion.
© Nex-G Exuberant Solutions Pvt. Ltd. 4
Test Plan Development
Step 1: Categorize the Type of Test to Be Completed
Identify the reasons for testing and try to determine whether the type of test fits into one
of the categories described earlier in the chapter. Try to clearly articulate the trigger and
objectives in one sentence, such as:
• "Proof of concept test, with the goal of gaining confidence and experience with the
new voice gateway platform."
• "Network ready for use test to certify the new data center in Syracuse is operationally
ready to be cut over into production and carry live customer data."
• "Design verification test to ensure that the next-generation WAN design will work and
support company business needs for the next 3 to 5 years."
Categorizing a test as shown in the previous examples will allow you to consider and
compare a potential test with similar engagements you may have completed in the past.
© Nex-G Exuberant Solutions Pvt. Ltd. 5
Test Plan Development
Step 2: Identify Project Stakeholders
Stakeholders are people or groups who have a vested interest in the outcome of the test
initiative. They may include the company leadership, representatives from the business
units, application developers, network architects, network operations staff, and
contracted professional services firms.
Early in the project, work with your project sponsor to create a list of all possible
stakeholders so that you can develop an effective communications plan. The goal is to
gain and sustain commitment to the test initiative, and to avoid negative behaviors such
as stalling or undermining from people who demand to be "in the loop."
It is a good idea to solicit input to the test plan from a wide variety of stakeholders so that
a comprehensive approach can be taken.
© Nex-G Exuberant Solutions Pvt. Ltd. 6
Test Plan Development
For very large test efforts requiring input from multiple stakeholders, it may be helpful to
assign and designate them using a well-known method from organizational design
known as RACI (Responsible, Accountable, Consulted, Informed):
Responsible: This person is responsible for completing a task.
Accountable: This person will be called to account if the task is not completed and may
manage the person who is responsible for completing the task. Project managers often
have this role.
Consulted: Though not accountable or responsible for completion, this person is
consulted about aspects of the task.
Informed: The holder of this passive role is kept informed but isn't accountable or
responsible for tasks.
© Nex-G Exuberant Solutions Pvt. Ltd. 7
Test Plan Development
Step 3: Identify Indicators of Test Success
It is critical to get agreement from the stakeholders on what constitutes testing success;
otherwise, exiting the test may become difficult. You don't need to identify success
criteria for each and every test case during the Scoping Phase—that will come later
during test planning.
What you need to understand at this point is what constitutes overall test success, so
that you have a good idea of which elements are important to build test cases around.
Here are some examples of success criteria for various types of tests.
© Nex-G Exuberant Solutions Pvt. Ltd. 8
Test Plan Development
Network Design Verification Test
• Test Pass Criteria:
• 100 percent of test cases have been executed.
• Network design meets customer requirements with respect to feature
functionality, performance, and convergence around failures.
• No Severity 1 or 2 defects encountered.
• All Severity 3 defects (if any) have been documented/filed and workarounds
have been provided.
• Test Fail Criteria:
• Severity 1 defects are found with no workaround in an area critical to the
solution.
• Severity 2 defects are found that can put in jeopardy the on-time deployments of
the solution.
• One or more "key" features are missing, are not operational, or don't meet
customer-specific requirements.
© Nex-G Exuberant Solutions Pvt. Ltd. 9
Test Plan Development
Step 4: Estimate the Resources Required to Complete the Test
An estimation of the people, lab equipment, and test tools needed to complete the
testing activities will be necessary to develop a reasonably accurate project timeline, and
to provide pricing for the test if procurement is necessary. When estimating the people
required, be sure to account for the skill level and number of people required for each of
the necessary tasks; for example:
• Test plan development (high skill level)
• Equipment cabling and lab assembly (low skill level)
• Test plan execution (medium to high skill level, depending on technology and scale of
test)
• Test results development (medium to high skill level, depending on technology and
scale of the test)
© Nex-G Exuberant Solutions Pvt. Ltd. 10
Test Plan Development
Step 5: Identify Risks
A risk is an event or condition that, if it occurs, could negatively impact your test project.
Sometimes risks are outside your control, such as when an equipment or software
manufacturer cannot ship critical hardware or tools on time. This type of risk can be
mapped to external dependencies.
In other cases, risks can be mapped to internal dependencies, as in the case of having
to hire staff with the appropriate skill set to complete a test. Tests that do not have clear
objectives or well-documented designs present a risk of going over budget as the test
team wastes time testing the wrong features or design scenarios.
© Nex-G Exuberant Solutions Pvt. Ltd. 11
Test Plan Development
Step 6: Identify the Timeline for Completion
Large testing projects often require contributions from multiple people across different
departments and even companies. Good teamwork, clear communications, and
dedication to the project are necessary so that testing of a particular design or solution
does not last forever.
Because network testing is commonly triggered by a proposed network deployment,
deadlines are often readily available and documented in an overall project plan.
A challenge you will often face is developing a realistic test timeline that allows you to
execute your testing thoroughly, while at the same time meeting the deadlines of the
project.
© Nex-G Exuberant Solutions Pvt. Ltd. 12
Test Plan Development
It is often helpful to allocate time to each of the various test phases when constructing
your timeline; for example:
• Assessment of Objectives: 1 week
• Test Case Planning: 2 weeks
• Test Lab Setup: 2 weeks
• Test Execution: 2 weeks
• Test Documentation and Analysis: 1 week
Understand that there are dependencies that may affect your timeline, causing it to slip
past the eight calendar weeks given in the preceding
© Nex-G Exuberant Solutions Pvt. Ltd. 13
Test Plan Development
Test Planning
Now that you and your client clearly understand and agree on the test scope, objectives,
and criteria for success, it is finally time to roll up your sleeves and start working on the
test plan.
As always, it is important to collaborate with the stakeholders on the test plan to
determine specifics regarding the application characteristics, behaviors, and new
features that are expected of the new system.
The prototype network system, equipment specifications, test cases, test tools, data to
be collected, and results format must also be discussed and agreed upon. This is an
important step in the process because it requires many decisions and significant
teamwork.
© Nex-G Exuberant Solutions Pvt. Ltd. 14
Test Plan Development
Design the Functional Prototype Network System
For most types of tests, a working prototype network system of the intended design will
serve as the platform upon which functionality, operation, and performance of the new
system will be evaluated. A prototype network system is commonly illustrated in a set of
network topology diagrams that represent a miniaturized version of the end-state
network.
Designing the working prototype for the lab requires experience, creativity, ingenuity, and
a deep understanding of the network design, solutions, and technologies to be
evaluated. An awareness of the test triggers and motivations of the clients requesting the
test is also necessary so that the right aspects of the design are represented and
modeled.
The first challenge you will face when designing the prototype is how much of the
network system will need to be implemented to convince your client that the design
meets business and technical requirements.
© Nex-G Exuberant Solutions Pvt. Ltd. 15
Test Plan Development
The working prototype must be functional and able to demonstrate performance under
stress and scale conditions, but it rarely needs to be a full-scale implementation of the
new system. The design and scale of a prototype will vary, depending on the type of test
you are conducting and what features or components of the design must be showcased
for your clients.
How little or much of the network is represented in the prototype will be bounded by the
people, money, equipment, and time you have to complete the test.
Use the information you collected during the Scoping Phase to help you determine the
size and design of the network prototype, particularly the input and requirements from
the key stakeholders.
Whereas the design architect may be primarily focused on technical issues, such as
validating a routing design or gaining performance benchmarks, network operations may
be concerned only with the manageability and usability benefits of the new design.
© Nex-G Exuberant Solutions Pvt. Ltd. 16
Test Plan Development
Constructing a High-Level Lab Topology Diagram
Early in the Plan Phase, you need to process all the input gathered from the Scoping
Phase and develop a high-level topology diagram to start the test plan dialog with your
clients.
The high-level topology diagram is your initial draft of what the final lab network topology
will look like, and it should be one of the first documents you share with your customer
when discussing your approach.
This diagram should include the major devices, including routers, switches, firewalls,
servers, workstations, telephony, video, WAN simulators, test equipment, and so forth,
that will be necessary to conduct the tests.
Connections between the various devices should be shown, using "clouds" where
appropriate to represent elements that will provide connectivity to the system, but not
necessarily be under evaluation, such as a provider Multiprotocol Label Switching
(MPLS) backbone or the Internet.© Nex-G Exuberant Solutions Pvt. Ltd. 17
Test Plan Development
Figure - High-Level Lab Topology Diagram
© Nex-G Exuberant Solutions Pvt. Ltd. 18
Test Plan Development
Identifying the Test Suites and Test Cases
A test case in the context of internetworking is a set of conditions or variables under
which a tester will determine whether or not a design element (network service,
component, or feature) is working correctly.
Test cases are the essence of the test plan, and they are sometimes collected or
aggregated into test suites. Identifying the right test cases and expected output is an art
form in itself, often requiring a series of conversations with the project stakeholders,
architects, and operators of the network.
A simple description of the test cases you expect to run, and accompany the network
topology diagram you have prepared, is sufficient to begin the test plan dialog. Your
client may already have some ideas of the kinds of tests they want to see, so you should
request their input right away.
© Nex-G Exuberant Solutions Pvt. Ltd. 19
Test Plan Development
Test Suite # Test Suite Test Case
1 Open Shortest Path First (OSPF) •ABR Summarization
Routing •Default Route Announce
•OSPF NSSA
•Redistribution BGP to OSPF
•Timer Optimizations
2 Border Gateway Protocol (BGP) •Data Center iBGP Optimizations
Routing •CE-PE eBGP Optimizations
•BGP Aggregation
•BGP Policy (Communities and Local-Pref)
3 Quality of Service (QoS) •Marking
•Queuing
•Traffic Shaping
•Policing
•Remarking at CE-PE to MPLS
•QoS Transparency Across MPLS
•CoPP
4 Negative Testing •Circuit Path Failover
•Line Card Failover
•Route Processor Failover
© Nex-G Exuberant Solutions Pvt. Ltd. 20
