Software Testing
Software Testing Foundations
Agenda
1. ISTQB and IEEE
2. Quality, Testing, QA, QC
3. Testing Principles
4. Who is who in the team?
5. Software Development Life Cycle
6. Specification
7. Why do we test?
8. Common misconceptions
9. The role of Software QA
ISTQB, IEEE
• ISTQB - The International Software Testing Qualifications Board (ISTQB) is a software testing
qualification certification organization that operates internationally. Founded in Edinburgh in
November 2002, ISTQB is a non-profit association legally registered in Belgium.
• IEEE - The Institute of Electrical and Electronics Engineers (IEEE) is a professional association with
its corporate office in New York City and its operations center in Piscataway, New Jersey.
Quality
What is quality?
Quality
Definitions of Quality:
By ISTQB:
The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.
By IEEE:
• The degree to which a system, component, or process meets specified requirements.
• The degree to which a system, component, or process meets customer or user needs or expectations.
QA, QC, Testing
Quality Assurance:
• A set of activities designed to ensure that the development and/or maintenance process is adequate to ensure a
system will meet its objectives.
Quality Control:
• A set of activities designed to evaluate a developed work product.
Testing:
• The process of executing a system with the intent of finding defects.
QA, QC, Testing
Quality Quality Control Testing
Assurance
Definition Activities which ensure the Activities which ensure the Activities which ensure the
implementation of verification of developed identification of
processes, procedures and software with respect to bugs/error/defects in the
standards in context to documented (or not in Software.
verification of developed some cases) requirements
software and specified
requirements.
Focuses on Focuses on processes and Focuses on actual testing Focuses on actual testing
procedures rather then by executing Software with
conducting actual testing intend to identify
on the system. bug/defect through
implementation of
procedures and process.
Relate to Process oriented Product oriented Product oriented
activities. activities. activities.
Goal Prevention Detection Detection
Audit
Audit
Standardization organizations
- ISO (International Organization for Standardization)
- IEC (International Engineering Consortium)
- IEEE (Institute of Electrical and Electronics Engineers)
Testing vs Debugging
Testing Debugging
Purpose Finding and locating of a Identifying, isolating and
defects in a Testing phase fixing the defects/bug
Executed by Done by Testing Team Done by Development
Team
Goal To find as many defects To remove the defects
as possible
Testing Principles
7 principles of testing
Testing shows presence of defects
Exhaustive testing is impossible
Early testing
Defect clustering
Pesticide paradox
Testing is context depending
Absence of errors fallacy
Who is who in the team
1. QAs
2. Developers
3. Business Analysts
4. Project Manager
5. Solution Designed/Solution Architect/Solution Strategist
6. Technical Writers
SDLC
Example
Cost of errors
• The later the a fault is found,
the higher the cost
• Small error in specification can result
in huge error in production
• Error in production can
ruin the company
Examples
1. In October of 1999 the $125 million NASA Mars Climate Orbiter spacecraft was believed to be lost in
space due to a simple data conversion error. It was determined that spacecraft software used certain
data in English units that should have been in metric units.
2. A small town in Illinois received an unusually large monthly electric bill of $7 million in March of 1999.
This was about 700 times larger than its normal bill. It turned out to be due to faults in new software
that had been purchased by the local power company to deal with Y2K software issues.
3. In early 1999 a major computer game company recalled all copies of a popular new product due to
software problems. The company made a public apology for releasing a product before it was ready
Mistake and Error
• Humans make mistakes: No one is perfect. We all make mistakes or omissions
• Aggressive schedules and pressure: The more pressure we are under the more likely we are
to make mistakes. In IT development we have time and budgetary deadlines to meet
• Poor Communication: Poor Training. Poor Communication
• Poor Documentation: Requirements not clearly defined. Requirements change & requirements
not properly documented. Data specifications not complete
• Assumptions
Specification
• Description of how software should work/look
- A document that specifies, ideally in a complete, precise and
verifiable manner, the requirements, design, behavior, or other
characteristics of a component or system, and, often, the procedures
for determining whether these provisions have been satisfied. [IEEE 610]
• Different types of documents
- Use cases, User Stories, Technical Design, Mock-ups
• Bug in specification
• Lack of specification
TROUBLESHOOT!
• Try to find where is the problem
• What caused the problem
• How to prevent its reappearance
Why do we test?
• Find defects before the customer
• Measure quality
• Gain confidence in the level of quality
• Increase reliability
• Increase profitability
Misconceptions
• MYTH: Software quality depends on the testing team
FACT: Software quality depends on the development team
• MYTH: The objective of Testing is to ensure a 100% defect- free product.
FACT: The objective of testing is to uncover as many defects as possible while
ensuring that the software meets the requirements. Identifying and getting rid of all
defects is impossible.
• MYTH: Testing is easy.
FACT: Testing can be difficult and challenging (sometimes, even more so than
coding).
Misconceptions
• MYTH: Anyone can test.
FACT: Testing is a rigorous discipline and requires many kinds of skills.
• MYTH: There is no creativity in testing.
FACT: Creativity can be applied when formulating test approaches, when designing tests,
and even when executing tests.
• MYTH: Automated testing eliminates the need for manual testing.
FACT: 100% test automation cannot be achieved. Manual Testing, to some level, is always
necessary.
Misconceptions
MYTH: Quality Assurance = Testing
Tester vs QA
Tester QA Engineer
• Takes part in all stages of SDLC and TLC
• Participates in the testing phase of the project • Determines testing strategy and approach
• Writes and verifies defects • System technical knowledge
• Test case execution • Gives suggestions and ideas how to improve the
• Reports results product
• Facilitates implementation of processes and procedures
The role of QA
What is special about QA?
• Broad competence
• Special attitude
Levels of testing team independence
• No testing team – developers do the testing
• Test specialists as part of development team
• Separate testing team inside the organization (internal)
• Separate testing team outside the organization (external/outsource)
QUESTIONS?
