CMake with Check Unit Tests

Introduction

Working on a legacy C autoconf system really makes you wonder how life could be easier. So, as that project draws to a close, I stole some time to explore alternatives.

At the same time, I’ve been reading The Architecture of Open Source Applications (warning: the “few fearless hacks” byline can trigger derisive giggles in non-programming bystanders) and CMake is mentioned there. It seemed worth investigating as a simpler alternative to autoconf.

And after a happy, rather productive day of hacking, I think I finally have a minimal, coherent example of a C project that compiles with CMake and uses Check for unit tests.

The code is visible on BitBucket - incidentally, this is also connected to the C ORM work I mentioned earlier, although as yet there’s not any ORM-related code.

In the rest of the post I’ll walk through that directory structure, explaining how things work.

CMake

I don’t really have enough experience to pronounce on the advantages or otherwise of CMake yet, but I can explain some of how it works.

First, it is a layer above make (or whatever the appropriate build system is on your platform). It takes a specification, given in various CMakeLists.txt files, and uses those to generate a build directory (where Makefile lives, along with compiled code). So the standard incantation is something like:

cmake
make

In practice, the build directory is wherever you run cmake, and it’s full of junk files, so you don’t want to run cmake in your top level directory. Instead, you do something more like:

cd build
cmake ..
make

And what I have actually been doing, while working out exactly what is happening, is more like:

rm -fr `find . -name CMakeFiles`
rm -f `find . -name CMakeCache.txt`
rm -f `find . -name Makefile`
rm -f `find . -name cmake_install.cmake`
rm -f `find . -name "*.a"`
rm -fr build
mkdir build

cd build
cmake ..
make
ctest --output-on-failure .

(I’ll get to ctest later).

Project Structure

This is the simplified project structure:

.
├── build
│   └── [stuff]
├── CMakeLists.txt
├── CMakeModules
│   └── FindCheck.cmake
├── src
│   ├── CMakeLists.txt
│   ├── isti.h
│   ├── isti_str.c
│   └── isti_str.h
└── tests
    ├── CMakeLists.txt
    ├── CTestTestfile.cmake
    └── test_isti_str.c

And you can see that there’s a CMakeLists.txt at each level in the project/source/test tree.

In the src directory, things are quite unexciting:

1 include_directories(.)
2 add_library(isti_clib isti_str.c)

where line 2 defines a library, along with the (single, in this case) source file(s). That library, isti_clib, will be built when cmake and make run.

At the top level we have:

1 cmake_minimum_required(VERSION 2.8)
2 set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/CMakeModules/")
3 project(isti_clib)
4 enable_testing()
5 add_subdirectory(src)
6 add_subdirectory(tests)

where line 2 is configuring the subdirectory CMakeModules as a place to look for alternative files. There I have added a file I found elsewhere by googling for “cmake findcheck”, where “check” is the name of the library you want, and “find” is magic.

The motivation for this will be explained below, where I address testing, but you can see, I hope, how this is vaguely like extending autoconf with instructions on how to check for particular libraries. It turns out that CMake has a bunch of checks built-in (in which case, you use find_package), but the Check library is not one of them.

Configuring Tests

OK, so all the above is fairly standard and google-able. What comes next required a little more sweat.

The CMakeLists.txt in the tests directory looks like this:

1 enable_testing()
2 find_package(Check REQUIRED)
3 include_directories(${CHECK_INCLUDE_DIRS})
4 set(LIBS ${LIBS} ${CHECK_LIBRARIES} isti_clib)
5 include_directories(. ../src)
6 add_executable(test_isti_str test_isti_str.c)
7 target_link_libraries(test_isti_str ${LIBS})
8 add_test(test_isti_str ${CMAKE_CURRENT_BINARY_DIR}/test_isti_str)

Line 2 triggers the code in FindCheck.cmake mentioned earlier. This does clever things, the results of which we are placed in CHECK_INCLUDE_DIRS and CHECK_LIBRARIES.

Lines 3 and 4 add dependencies for our code (on the Check library, just found, and the library we are going to build and test).

Line 5 allows us to include from src (where the library headers live).

Lines 6 and 7defines test_isti_str as a simple C program, defined in test_isti_str.c. I will show that code below - the important thing to see here is that a C unit test is simple a C program, configured as you would configure any other program in CMake.

Line 8 connects the test program, when built, with the conceptual test itself, called “test_isti_str”. This connects up to CTest, which is some kind of test runner (but not a test framework - see below).

Test Source

The test code is unaware of CMake. It uses Check in a completely standard way:

#include <stdio.h>
#include <check.h>

#include "isti.h"
#include "isti_str.h"


START_TEST (test_str)
{
        isti_str *s = NULL;
        fail_if(isti_str_alloc(&s), "Could not allocate s");
        fail_if(!s, "s null after allocation");
        fail_if(s->free(&s, ISTI_OK), "Could not free s");
        fail_if(s, "s not null after free");
}
END_TEST

Suite* str_suite (void) {
        Suite *suite = suite_create("isti_str");
        TCase *tcase = tcase_create("case");
        tcase_add_test(tcase, test_str);
        suite_add_tcase(suite, tcase);
        return suite;
}

int main (int argc, char *argv[]) {
        int number_failed;
        Suite *suite = str_suite();
        SRunner *runner = srunner_create(suite);
        srunner_run_all(runner, CK_NORMAL);
        number_failed = srunner_ntests_failed(runner);
        srunner_free(runner);
        return number_failed;
}

There’s not much to add here. If the above is unclear, Check’s excellent docs should help.

Running Tests

Once the above is complete, running a test looks like this:

> cd build
> cmake ..
-- Configuring done
-- Generating done
-- Build files have been written to: /home/andrew/projects/isti/bitbucket/c-orm/clib/build
> make
[ 50%] Built target isti_clib
[100%] Built target test_isti_str
> ctest --output-on-failure .
Test project /home/andrew/projects/isti/bitbucket/c-orm/clib/build
    Start 1: test_isti_str
1/1 Test #1: test_isti_str ....................***Failed    0.00 sec
Running suite(s): isti_str
0%: Checks: 1, Failures: 1, Errors: 0
/home/andrew/projects/isti/bitbucket/c-orm/clib/tests/test_isti_str.c:15:F:case:test_str:0:
s not null after free


0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.04 sec

The following tests FAILED:
          1 - test_isti_str (Failed)
Errors while running CTest

where we have a useful error message from a failed test. Hurray!

Note that we use ctest to run the test, but the test itself is structure using the Check library. CTest is only a test runner;it detects whether a test has failed or passed by simply checking the return value (error on non-zero exit).

Conclusions

Hopefully the above clarifies how to configure a simple CMake project with unit tests. The same Check package that works well in autoconf can be used for tests in CMake.

As for CMake. Well, I am not hugely impressed. The documentation is pretty poor (it exists, but finding answers is hard). On the other hand, I guess it’s better than autoconf because almost anything would be…


Related Posts

blog comments powered by