Software testing

49 languages

 Article
 Talk
 Read
 Edit
 View history
Tools


















Appearance
hide

Text

Small

Standard

Large

Width

Standard

Wide

Color (beta)


Automatic

Light

Dark

From Wikipedia, the free encyclopedia

Part of a series on

Software development

hide

Core activities

 Data modeling
 Processes
 Requirements
 Design
 Construction
 Engineering
 Testing
 Debugging
 Deployment
 Maintenance

show

Paradigms and models

show

Methodologies and frameworks

show

Supporting disciplines

show

Practices
 show

Tools

show

Standards and bodies of knowledge

show

Glossaries

show

Outlines

 v
 t
 e

TestingCup – Polish Championship in Software

Testing, Katowice, May 2016

Software testing is the act of checking whether software satisfies expectations.

Software testing can provide objective, independent information about the quality of
software and the risk of its failure to a user or sponsor.[1]

Software testing can determine the correctness of software for specific scenarios but
cannot determine correctness for all scenarios.[2][3] It cannot find all bugs.

Based on the criteria for measuring correctness from an oracle, software testing
employs principles and mechanisms that might recognize a problem. Examples of
oracles include specifications, contracts,[4] comparable products, past versions of the
same product, inferences about intended or expected purpose, user or customer
expectations, relevant standards, and applicable laws.

Software testing is often dynamic in nature; running the software to verify actual
output matches expected. It can also be static in nature; reviewing code and its
associated documentation.
Software testing is often used to answer the question: Does the software do what it
is supposed to do and what it needs to do?

Information learned from software testing may be used to improve the process by
which software is developed.[5]: 41–43

Software testing should follow a "pyramid" approach wherein most of your tests
should be unit tests, followed by integration tests and finally end-to-end (e2e)
tests should have the lowest proportion.[6][7][8]

Economics
[edit]

A study conducted by NIST in 2002 reported that software bugs cost the U.S.
economy $59.5 billion annually. More than a third of this cost could be avoided if
better software testing was performed.[9][dubious – discuss]

Outsourcing software testing because of costs is very common, with China, the
Philippines, and India being preferred destinations.[citation needed]

History
[edit]

Glenford J. Myers initially introduced the separation of debugging from testing in

1979.[10] Although his attention was on breakage testing ("A successful test case is
one that detects an as-yet undiscovered error."[10]: 16 ), it illustrated the desire of the
software engineering community to separate fundamental development activities,
such as debugging, from that of verification. Software testing typically includes
handling software bugs – a defect in the code that causes an undesirable result.[11]:
31
Bugs generally slow testing progress and involve programmer assistance
to debug and fix.

Not all defects cause a failure. For example, a defect in dead code will not be
considered a failure.

A defect that does not cause failure at one point in time may lead to failure later due
to environmental changes. Examples of environment change include running on
new computer hardware, changes in data, and interacting with different software.[12]

Goals
[edit]

Software testing is typically goal driven.

Finding bugs
[edit]

Software testing typically includes handling software bugs – a defect in the code that
causes an undesirable result.[11]: 31 Bugs generally slow testing progress and
involve programmer assistance to debug and fix.
Not all defects cause a failure. For example, a defect in dead code will not be
considered a failure.

A defect that does not cause failure at one point in time may lead to failure later due
to environmental changes. Examples of environment change include running on
new computer hardware, changes in data, and interacting with different software.[13]

A single defect may result in multiple failure symptoms.

Ensuring requirements are satisfied

[edit]

Software testing may involve a Requirements gap – omission from the design for a
requirement.[5]: 426 Requirement gaps can often be non-functional requirements such
as testability, scalability, maintainability, performance, and security.

Code coverage
[edit]

A fundamental limitation of software testing is that testing under all combinations of

inputs and preconditions (initial state) is not feasible, even with a simple product. [3]: 17–18
[14]
Defects that manifest in unusual conditions are difficult to find in testing. Also, non-
functional dimensions of quality (how it is supposed to be versus what it is supposed
to do) – usability, scalability, performance, compatibility, and reliability – can be
subjective; something that constitutes sufficient value to one person may not to
another.

Although testing for every possible input is not feasible, testing can
use combinatorics to maximize coverage while minimizing tests.[15]

Categories
[edit]

Main article: Software testing tactics

Testing can be categorized many ways.[16]

Automated testing
[edit]

This paragraph is an excerpt from Test automation.[edit]

Test automation is the use of software (separate from the software being tested) for
controlling the execution of tests and comparing actual outcome with predicted.[17] Test
automation supports testing the system under test (SUT) without manual interaction which
can lead to faster test execution and testing more often. Test automation is key aspect
of continuous testing and often for continuous integration and continuous delivery (CI/CD).[18]

Levels
[edit]
Software testing can be categorized into levels based on how much of the software
system is the focus of a test.[19][20][21][22]

Unit testing
[edit]

This paragraph is an excerpt from Unit testing.[edit]

Unit testing, a.k.a. component or module testing, is a form of software testing by which
isolated source code is tested to validate expected behavior.[23]

Integration testing
[edit]

This paragraph is an excerpt from Integration testing.[edit]

Integration testing is a form of software testing in which multiple software components,

modules, or services are tested together to verify they work as expected when combined.
The focus is on testing the interactions and data exchange between integrated parts, rather
than testing components in isolation.

System testing
[edit]

This paragraph is an excerpt from System testing.[edit]

System testing, a.k.a. end-to-end (E2E) testing, is testing conducted on a complete software
system.

Static, dynamic, and passive testing

[edit]

There are many approaches to software testing. Reviews, walkthroughs,

or inspections are referred to as static testing, whereas executing programmed code
with a given set of test cases is referred to as dynamic testing.[24][25]

Static testing is often implicit, like proofreading, plus when programming tools/text
editors check source code structure or compilers (pre-compilers) check syntax and
data flow as static program analysis. Dynamic testing takes place when the program
itself is run. Dynamic testing may begin before the program is 100% complete in
order to test particular sections of code and are applied to discrete functions or
modules.[24][25] Typical techniques for these are either using stubs/drivers or execution
from a debugger environment.[25]

Static testing involves verification, whereas dynamic testing also involves validation.
[25]

Passive testing means verifying the system's behavior without any interaction with
the software product. Contrary to active testing, testers do not provide any test data
but look at system logs and traces. They mine for patterns and specific behavior in
order to make some kind of decisions.[26] This is related to offline runtime
verification and log analysis.