Skip to main content

An Introduction to CTest

I've seen a lot of people (I'm looking at you Daniel Lemire) praise newer languages like Go, which makes me sad as a C++ programmer. Well, that's until I realise that most of the features touted could be easily incorporated in C++ with just a bit of elbow grease. In this post, I'll show how to add an automated testing system using CTest.

I love CMake, it allows me to write cross-platform code and be confident that the build system would work across a choice of compilers, IDEs, and operating systems. When writing large projects, however, it is imperative to have a series of tests that could indicate regression bugs whenever new features are added. While I love the testing support built into languages such as Java, writing tests is not all that hard in C++ either. For example, I could simply write a function that mimics a unit test and prints out either "Test passed" or "Test failed" depending on the result of the test. All I now need is a way to automatically run these tests.

This is where CTest comes into the picture. I'm assuming here that you are using CMake as your build system. If you are not, then you are clearly wrong, a terrible person, and the type who would probably not write tests anyway. So stop reading this post. 😄

Okay, back from that little digression.

My setup for testing is thus: I am building a library that defines all the required functions. Each class/function/API must be tested, so I write tests for each of these. The tests are written so that they would indicate success using the keyword "Test passed" or failure using the keyword "Test failed". I put all the tests in a separate tests folder. Within the tests folder, I write my CMakeLists.txt thus.


file(GLOB files "test_*.cc")

foreach(file ${files})
    string(REGEX REPLACE "(^.*/|\\.[^.]*$)" "" file_without_ext ${file})
    add_executable(${file_without_ext} ${file})
    target_link_libraries(${file_without_ext} ${PROJECT_LIBS})
    add_test(${file_without_ext} ${file_without_ext})
        PASS_REGULAR_EXPRESSION "Test passed")
        FAIL_REGULAR_EXPRESSION "(Exception|Test failed)")
        TIMEOUT 120)

This is a really simple script. It loops over all files in the tests folder that match the pattern test_*.cc, i.e. C++ files that start with the test_ prefix. It generates the executable name by stripping out all the leading directories until the path, while also stripping out the extension for the file. It compiles the file and links it against the project libraries. Finally, it tells CMake/CTest that the compiled binary is a test which on passing would have in its output the text "Test passed", and on failure could have the text "Test failed" or "Exception". Finally, I add a timeout of 120 seconds, or two minutes. If the test runs longer than this time, it will be automatically terminated and marked as a failure. This is not ideal -- some of my tests run for just seconds, while the longest test could run for just under a minute. However, the purpose of the timeout here is not to detect performance bugs, rather to prevent the machine running the tests from fritzing out because of a bug that results in the tests doing something really crazy.

Oh, and in order to enable testing, I simply change the CMakeLists.txt on the very top level of the project (the one that defines project name, version, etc.) to include this one extra line.


Simple, isn't it? Now why would you go running to Go (see what I did there? 😄), when C++ with CMake offers cross platform builds, testing infrastructure, and parallel execution with memory consistency models?

In the next posts on the series, I might just describe how to test for memory leaks using Valgrind/MemCheck or how to test test-coverage (alliteration FTW) using gcov. Stay tuned.


Popular posts from this blog

Progressive Snapshot: Is it worth it?

I turned 25 last year, which in the highly mathematical and calculating eyes of the US insurance industry meant that I had suddenly matured into a much more responsible driver than I was at 24 years and 364 days of age. As a result, I expected my insurance rates to go down. Imagine my surprise when my insurance renewal notice from GEICO actually quoted a $50 increase in my insurance rates. To me, this was a clear signal that it was time to switch companies.Typically, I score really high on brand loyalty. I tend to stick with a brand for as long as possible, unless they really mess up. This qualified as a major mess up. As a result, I started shopping for insurance quotes.Two companies that quoted me significantly lower rates (30%–40% lower) were Progressive and Allstate. Both had an optional programme that could give me further discounts based on my consenting to the companies tracking my driving habits. Now, I am a careful driver – I hardly ever accelerate hard. I hate using the brak…

Build those noise cancelling headphones

So, here's another DIYLet me start by putting the cart before the horse. I shall start with the credits. This project was done while I was working on my Electronics Design Lab, along with my friends, Srujan M and Indrasen Bhattacharya. The work would not have been possible without the generous help received from the staff at Wadhwani Electronics Laboratory, who ensured that the only thing we did right was to leave the lab on time. This project would also not have been possible without the guidance of our dear and learned professors. It would probably have just about become additional dead weight on the head.Enough with the credits, now, I need to dive right into noise cancellation and how it works.The essence of sound is a pressure wave. The pressure wave, when incident on the eardrum sets into motion the complex mechanisms inside the ear, and after a long path, rather like the Cog advertisement, ends up making some nerves vibrate. The nerves send electrical signals to the brain, …

The joy of receiving a handwritten letter...

I receive around 20 emails a day. I hit delete for most.While studying letter writing in school, I often used to wonder, is letter writing relevant any more? I mean, who sends snail mail? Isn't it much more convenient to write an email?Fast forward to a few days ago... I received a note, not really a letter, from a friend, whom I had the pleasure to know for over three months. The pleasure of reading the note really changed my perception about the composition exercise learnt in school.So, what is it that a handwritten letter has which email lacks? Maybe it is the personal touch, the realization that a person has written the letter, and that it has not been written by a computer. Handwriting just happens to add a personal touch which the cold hard sans-serif font of email just cannot capture.I also think that handwritten letters take time and effort into composition. This means that they generally have a better content than email, which is often written casually, in a hurry with l…