Simple unit tests with C++ and CMake
Tags: programming, projects, research
A warning upfront: this post is sort of an advertisement for Aleph, my C++ library for topological data analysis. In this blog post I do not want to cover any of the mathematical algorithms present in Aleph—I rather want to focus on small but integral part of the project, viz. unit tests.
If you are not familiar with the concept of unit testing, the idea is (roughly) to write a small, self-sufficient test for every new piece of functionality that you write. Proponents of the methodology of test-driven development (TDD) even go so far as to require you to write the unit tests before you write your actual code. In this mindset, you first think of the results your code should achieve and which outputs you expect prior to writing any “real” code. I am putting the word real in quotation marks here because it may seem strange to focus on the tests before doing the heavy lifting.
However, this way of approaching software development may actually be quite beneficial, in particular if you are working on algorithms with a nice mathematical flavour. Here, thinking about the results you want to achieve with your code ensures that at least a few known examples are processed correctly by your code, making it more probable that the code will perform well in real-world scenarios.
When I started writing Aleph in 2016, I also wanted to add some unit
tests, but I did not think that the size of the library warranted the
inclusion of one of the big players, such as Google
Test or
Boost.Test. While
arguably extremely powerful and teeming with more features than I could
possibly imagine, they are also quite heavy and require non-trivial
adjustments to any project.
Thus, in the best tradition of the
not-invented-here-syndrome,
I decided to roll my own testing framework, base on pure CMake and
small dash of C++. My design decisions were rather simple:
- Use
CTest, the testing framework ofCMaketo run the tests. This framework is rather simple and just uses the return type of a unit test program to decide whether the test worked correctly. - Provide a set of routines to check the correctness of certain calculations within a unit test, throwing an error if something unexpected happened.
- Collect unit tests for the “larger” parts of the project in a single executable program.
Yes, you read that right—my approach actually opts for throwing an
error in order to crash the unit test program. Bear with me, though, for
I think that this is actually a rather sane way of approaching unit
tests. After all, if the tests fails, I am usually not interested in
whether other parts of a test program—that may potentially depend
on previous calculations—run through or not. As a consequence,
adding a unit test to Aleph is as simple as adding the following lines
to a CMakeLists.txt file, located in the tests subdirectory of the project:
ADD_EXECUTABLE( test_io_gml test_io_gml.cc )
ADD_TEST( io_gml test_io_gml )
While in the main CMakeLists.txt, I added the following lines:
ENABLE_TESTING()
ADD_SUBDIRECTORY( tests )
So far, so good. A test now looks like this:
#include <tests/Base.hh>
void testBasic()
{
// do some nice calculation; store the results in `foo` and `bar`,
// respectively
ALEPH_ASSERT_THROW( foo != bar );
ALEPH_ASSERT_EQUAL( foo, 2.0 );
ALEPH_ASSERT_EQUAL( bar, 1.0 );
}
void testAdvanced()
{
// a more advanced test
}
int main(int, char**)
{
testBasic();
testAdvanced();
}That is basically the whole recipe for a simple unit test. Upon
execution, main() will ensure that all larger-scale test routines,
i.e. testSimple() and testAdvanced() are called. Within each of
these routines, the calls to the corresponding macros—more on that
in a minute— ensure that conditions are met, or certain values are
equal to other values. Else, an error will be thrown, the test will
abort, and CMake will throw an error upon test execution.
So, how do the macros look like? Here is a copy of the current version of Aleph:
#define ALEPH_ASSERT_THROW( condition ) \
{ \
if( !( condition ) ) \
{ \
throw std::runtime_error( std::string( __FILE__ ) \
+ std::string( ":" ) \
+ std::to_string( __LINE__ ) \
+ std::string( " in " ) \
+ std::string( __PRETTY_FUNCTION__ ) \
); \
} \
}
#define ALEPH_ASSERT_EQUAL( x, y ) \
{ \
if( ( x ) != ( y ) ) \
{ \
throw std::runtime_error( std::string( __FILE__ ) \
+ std::string( ":" ) \
+ std::to_string( __LINE__ ) \
+ std::string( " in " ) \
+ std::string( __PRETTY_FUNCTION__ ) \
+ std::string( ": " ) \
+ std::to_string( ( x ) ) \
+ std::string( " != " ) \
+ std::to_string( ( y ) ) \
); \
} \
}
Pretty simple, I would say. The ALEPH_ASSERT_EQUAL macro actually
tries to convert the corresponding values to strings, which may not
always work. Of course, you could use more complicated string conversion
routines, as
Boost.Test does.
For now, though, these macros are sufficient to make up the unit test
framework of Aleph, which at the time of me writing this, encompasses
more than 4000 lines of code.
The only remaining question is how this framework is used in practice.
By setting ENABLE_TESTING(), CMake actually exposes a new target
called test. Hence, in order to run those tests, a simple make test
is sufficient in the build directory. This is what the result may look
like:
$ make test
Running tests...
Test project /home/bastian/Projects/Aleph/build
Start 1: barycentric_subdivision
1/36 Test #1: barycentric_subdivision ............ Passed 0.00 sec
Start 2: beta_skeleton
[...]
34/36 Test #34: union_find ......................... Passed 0.00 sec
Start 35: witness_complex
35/36 Test #35: witness_complex .................... Passed 1.82 sec
Start 36: python_integration
36/36 Test #36: python_integration ................. Passed 0.07 sec
100% tests passed, 0 tests failed out of 36
Total Test time (real) = 5.74 sec
In addition to being rather lean, this framework can easily be integrated into an existing Travis CI workflow by adding
- make test
as an additional step to the script target in your .travis.yml file.
If you are interested in using this testing framework, please take a look at the following files:
- The
testssubdirectory ofAleph - The basic configuration file of all unit tests
- A simple example of a unit test for a single class
That is all for now, until next time—may your unit tests always work the way you expect them to!