Unit testing and code
coverage for C++
1
cppUnit
Coding an individual test
Use assertion macros defined in unit test library
Elements for building tests
TestFixture, TestSuite, TestRunner
2
Code in an individual test
Use assertion macros defined in cppunit
If assertion fails, an exception is thrown
CPPUNIT_ASSERT(condition):
Checks the condition and throws an exception if it's false.
CPPUNIT_ASSERT_MESSAGE(message, condition):
Checks the condition and throws an exception and showing specified message if
it is false.
CPPUNIT_ASSERT_EQUAL(expected,current):
Checks whether the expected condition is the same as current, and raises an
exception showing the expected and current values.
CPPUNIT_ASSERT_EQUAL_MESSAGE(message,expected,c
urrent)
Checks whether the expected is the same as the actual, and raises an exception
showing the expected and current values, and specified message.
CPPUNIT_ASSERT_DOUBLES_EQUAL(expected,current,delta):
Checks whether the expected and current difference is smaller than delta. If it 3
fails, the expected and current values are shown.
TestFixture
Each test should be derived from the abstract
class TestFixture
Override two functions:
setUp
operations that would be invoked prior to each individual test,
e.g. create new clean instances of class ready for next test
tearDown
operations that would be invoked after to each individual test,
e.g. deletes instances
4
TestSuite, TestRunner
TestSuite
Groups a number of tests (using TestFixture) so
that they can all be run
TestRunner
Driver program
5
Test driven development
Test twice, code once
“The style here is to write a few lines of
code, then a test that should run, or even
better, to write a test that won't run, then
write the code that will make it run.”
6
Test driven development – step 0
Define class Bitmap
Owns an array of unsigned long integers
Methods
boolean Identity(const Bitmap& other)
Are we the same Bitmap
Compare our address with other’s address!
boolean Equals(const Bitmap& other)
Are we equal?
What do you mean equal?
• This is the part we test twice – code once!
void Zero()
Set my bits all to zero.
7
#ifndef __MYBITSCLASS__
#define __MYBITSCLASS__
Class Bitmap
// Code assumes 32-bit unsigned long integers
#define MAXBITS 512
#define NUMWORDS 16
typedef unsigned long Bits;
class Bitmap {
public:
Bitmap();
void Zero(void);
bool Equals(const Bitmap &other) const;
bool Identity(const Bitmap &other) const;
private:
Bits fBits[NUMWORDS];
};
#endif
8
#include "Bitmap.h"
Bitmap::Bitmap() Class Bitmap
{
Zero();
}
void Bitmap::Zero(void)
{
for(int i=0;i<NUMWORDS; i++)
fBits[i] = 0;
}
bool Bitmap::Identity(const Bitmap& other) const
{
return (this == &other);
}
bool Bitmap::Equals(const Bitmap& other) const
{
return Identity(other);
// Incorrect implementation
}
9
Now write a test for Bitmap -- TestFixture
#ifndef BITMAPTEST_H
#define BITMAPTEST_H
#include <cppunit/TestCase.h>
#include <cppunit/extensions/HelperMacros.h>
#include "Bitmap.h"
class BitmapTest : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE (BitmapTest);
CPPUNIT_TEST (testEquals);
CPPUNIT_TEST_SUITE_END ();
private:
Bitmap bitmap1;
Bitmap bitmap2;
public:
void setUp();
void tearDown();
protected:
void testEquals();
};
#endif 10
#include "BitmapTest.h"
#include "Bitmap.h"
CPPUNIT_TEST_SUITE_REGISTRATION (BitmapTest);
void BitmapTest::setUp()
{
// Put the bitmaps back in standard state
// (Need at least two as doing things like
// checking equality)
bitmap1.Zero();
bitmap2.Zero();
}
void BitmapTest::tearDown()
{
// Nothing to do
}
11
void BitmapTest::testEquals()
{
// Identity should work
CPPUNIT_ASSERT(bitmap1.Identity(bitmap1));
CPPUNIT_ASSERT(bitmap2.Identity(bitmap2));
CPPUNIT_ASSERT(!bitmap1.Identity(bitmap2));
// At this stage - equality incorrectly defined
// It is identity
// (It shouldn't be!)
// This test should succeed - I'm equal to me
CPPUNIT_ASSERT(bitmap1.Equals(bitmap1));
// so should this
CPPUNIT_ASSERT(bitmap2.Equals(bitmap2));
// But this should fail
CPPUNIT_ASSERT(bitmap1.Equals(bitmap2));
}
12
The Driver -- TestRunner
int main (int argc, char* argv[])
{
// Result collection
CPPUNIT_NS :: TestResult testresult;
CPPUNIT_NS :: TestResultCollector collectedresults;
testresult.addListener (&collectedresults);
CPPUNIT_NS :: TextTestProgressListener tracker;
testresult.addListener(&tracker);
// Setting up runner and running test
CPPUNIT_NS :: TestRunner testrunner;
testrunner.addTest (CPPUNIT_NS :: TestFactoryRegistry :: getRegistry ().makeTest
());
testrunner.run (testresult);
// Reporting
CPPUNIT_NS :: CompilerOutputter compileroutputter (&collectedresults, std::cerr);
compileroutputter.write ();
return collectedresults.wasSuccessful () ? 0 : 1; 13
}
Build and run
!!!FAILURES!!!
Test Results:
Run: 1 Failures: 1 Errors: 0
1) test: BitmapTest::testEquals(F) line: 34
BitmapTest.cc
assertion failed
Expression: bitmap1.Equals(bitmap2)
14
“Refactor”
bool Bitmap::Equals(const Bitmap& other) const
{
for(int i = 0; i < NUMWORDS; i++)
if(this->fBits[i] != other.fBits[i]) return false;
return true;
}
15
Next increment – more functionality in
Bitmap
class Bitmap {
public:
Bitmap();
void Zero(void);
void SetBit(int bitnum);
int Count(void) const;
bool Equals(const Bitmap &other) const;
bool Identity(const Bitmap &other) const;
private:
Bits fBits[NUMWORDS];
};
16
BitmapTest
class BitmapTest : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE (BitmapTest);
CPPUNIT_TEST (testEquals);
CPPUNIT_TEST (testSetBit);
CPPUNIT_TEST (testSetBitAndCount);
CPPUNIT_TEST_SUITE_END ();
private:
Bitmap bitmap1;
Bitmap bitmap2;
public:
void setUp();
void tearDown();
protected:
void testEquals();
void testSetBit();
void testSetBitAndCount();
};
17
BitmapTest
Extended equality test
Can now make Bitmaps that are not equal (by
using set bit operation)
Add two more extensive tests on new test
and count operations
18
void BitmapTest::testEquals()
{ BitmapTest
// Identity should work
CPPUNIT_ASSERT(bitmap1.Identity(bitmap1));
CPPUNIT_ASSERT(bitmap2.Identity(bitmap2));
CPPUNIT_ASSERT(!bitmap1.Identity(bitmap2));
CPPUNIT_ASSERT(bitmap1.Equals(bitmap1));
CPPUNIT_ASSERT(bitmap2.Equals(bitmap2));
CPPUNIT_ASSERT(bitmap1.Equals(bitmap2));
// Have some functionality to change the bits
// See if equals works after that
// Set a different bit
bitmap1.SetBit(22);
bitmap2.SetBit(123);
// Shouldn't be equals any more!
CPPUNIT_ASSERT(!bitmap1.Equals(bitmap2));
}
19
void BitmapTest::testSetBit()
{ BitmapTest
// If set bit 0 in one, they should not be equal
bitmap1.SetBit(0);
CPPUNIT_ASSERT(!bitmap1.Equals(bitmap2));
// Just being pedantic - reverse roles in test
CPPUNIT_ASSERT(!bitmap2.Equals(bitmap1));
bitmap1.Zero();
// But should be equal after it got cleared
CPPUNIT_ASSERT(bitmap1.Equals(bitmap2));
CPPUNIT_ASSERT(bitmap2.Equals(bitmap1));
// If set a non-existent bit it isn't supposed
// to change anything
bitmap1.SetBit(-1);
bitmap2.SetBit(513);
CPPUNIT_ASSERT(bitmap2.Equals(bitmap1));
}
20
void BitmapTest::testSetBitAndCount()
{ BitmapTest
// Set some bits and check that count equals number of bits set
int bits1[] = { 0, 3, 17, 21, 33, 54, 68, 77, 91, 103, 211, 304 };
int bits2[] = { 0, 31, 32, 33, 63, 64, 65, 300, 400, 500, 511 };
int count1 = sizeof(bits1)/sizeof(int);
int count2 = sizeof(bits2)/sizeof(int);
for(int i=0;i<count1;i++)
bitmap1.SetBit(bits1[i]);
for(int i=0;i<count2;i++)
bitmap2.SetBit(bits2[i]);
CPPUNIT_ASSERT_EQUAL(count1, bitmap1.Count());
// Deliberate error to get a failed test
CPPUNIT_ASSERT_EQUAL(count1, bitmap2.Count());
}
21
Rebuild, rerun
!!!FAILURES!!! Text
Test Results:
Run: 3 Failures: 1 Errors: 0
1) test: BitmapTest::testSetBitAndCount (F) line: 77
BitmapTest.cc
equality assertion failed
- Expected: 12
- Actual : 11
22
cppunit
Available at
http://sourceforge.net/projects/cppunit
Is installed on Linux machines in lab used
for CSCI222
23
Tutorials …
http://www.comp.nus.edu.sg/~cs3214s/too
ls/cppunitSol.html
http://www.evocomp.de/tutorials/tutorium_
cppunit/howto_tutorial_cppunit_en.html
Documentation with cppUnit
Docs folder in /share/cs-pub/222/testing
“Money” tutorial
24
Code coverage
g++ compiler and
gcov analysis tool
25
Compilation for code coverage
Set extra compile flags
In NetBeans setup, it is simplest just to specify
--coverage
in the "command line" options for both the compilation
and linker steps
26
Run, then use gcov to analyze
NetBeans can be used to run the application
under test
Best to run several times with different input data so as
to try to test all options
gcov analysis should be done in terminal
session
cd to directory with C++ source
Identify directory where gcov files (.gcno, .gcda) were
created – should be ./build/Debug/GNU-Linux-x86
Run gcov specifying the data directory and file for
analysis
27
Gcov analysis : function summaries
gcov –o ./build/Debug/GNU-Linux-x86 -f Bitmap.cc
100.00% of 3 source lines executed in function void Bitmap::Zero()
100.00% of 7 source lines executed in function void Bitmap::SetBit(int)
100.00% of 8 source lines executed in function void Bitmap::ClearBit(int)
0.00% of 4 source lines executed in function void Bitmap::SetAs(int, int)
85.71% of 7 source lines executed in function int Bitmap::TestBit(int) const
100.00% of 7 source lines executed in function void Bitmap::FlipBit(int)
100.00% of 2 source lines executed in function void
Bitmap::ReadFrom(std::fstream&)
…
100.00% of 4 source lines executed in function int Bitmap::Equals(const
Bitmap&) const
94.32% of 88 source lines executed in file Bitmap.cc
28
Gcov analysis: branch counts
$ gcov –o ./build/Debug/GNU-Linux-x86 -c Bitmap.cc
…
94.32% of 88 source lines executed in file Bitmap.cc
Creating Bitmap.cc.gcov.
$ cat Bitmap.cc.gcov
#include "Bitmap.h"
Bitmap::Bitmap()
14 {
14 Zero();
}
void Bitmap::Zero(void)
15 {
255 for(int i=0;i<NUMWORDS; i++)
240 fBits[i] = 0;
}
void Bitmap::SetBit(int bitnum)
113 {
113 if((bitnum < 0) || (bitnum >= MAXBITS))
113 return;
113 int word = bitnum / 32;
…
29
Inspect profiles
Profiles and function summaries
immediately reveal untested code.
30
