Using OpenCV's Test Framework (With CMake)
This post explains how to set up tests making use of OpenCV’s built in framework.
Promoting Chilitags, our C++ library for detecting fiducial markers, from “research code” to published code means that we have no excuse any more for not having any test at all. After a short comparisons for testing frameworks, it seemed that Google Test would best fit our need.
Chilitags is heavily based on OpenCV. It turns out that OpenCV already integrates Google Test, and adds their own features, so you don’t have to write a function to assert that two matrices are equal for example. I’m more and more conviced that “OpenCV” means “we already did everything you may and will need” in some secret language.
However, I failed to find documentation on OpenCV’s
ts module, even though they use it to test all the other modules. It is not listed in OpenCV’s documentation, and Google didn’t help either. I asked on StackOverflow how to use OpenCV’s test framework with CMake, but haven’t received an answer. This post aims at answering myself the question with what I found.
First, I’ll show how to configure CMake to compile an OpenCV’s executable, and then explain how to run several of them as part of CMake’s tests.
Compiling an OpenCV test executable
This section gives a minimal example of an OpenCV based library being tested by the OpenCV test framework. We give the code of a dummy library, short test codes, and the CMake set up to build them all.
The code to be tested
Let’s say we want to test a dummy library, whose interface is declared in
1 2 3 4 5 6 7 8 9 10
and implemented in
1 2 3 4 5
Right… Obviously, this library is not the focus of this article.
The test code
We first write a simple test case in
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
This test cases will be automagically run in a specified folder if they are compiled with such
1 2 3
Structure and CMake Files
All these files will be structured typically as follow:
1 2 3 4 5 6 7 8 9 10 11 12
The general CMakeLists is standard…
1 2 3 4 5 6 7 8 9 10 11 12 13
…and so is the one to compile the library (in
1 2 3 4 5
Last but not least, it is not much more complicated to compile an OpenCV test:
1 2 3 4 5 6 7
And we’re done.
Running the tests
Finally we can compile averything and run the test in a standard CMake process. Here is the one if you use Makefile:
1 2 3 4
This creates a
mytest executable in
build/test which can be run to produce this output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Configuring CMake to run the tests
Now we may want to use several test executables for various reason. For example, we may want to split unrelated test, or split a test that takes too long to run, or make another kind of test (unit tests, performances tests, internationalisation tests or what not), or simply run non C++ executable.
It would be a bit cumbersome to launch them all by hand, especially when CMake integrates testing facilites.
Instead, we simply tell CMake which programs are test programs by adding an
enable_testing() directive to the general
1 2 3 4 5 6 7 8 9 10 11 12 13 14
We also specify an
add_test(testname executablename) directive for each test executable:
1 2 3 4 5 6 7 8 9
This way, CMake will create a test target which can be
built by your IDE to run the tests. For example when I run
make test, I get the following output:
1 2 3 4 5 6 7 8 9 10 11 12
This output is a lot less detailed than the previous one, which makes sense if there are a lot of tests happing, There is also an option to display the output of each test program (as shown before). With make, it’s
make test ARGS=-V.
Of course, there is much more to say about CMake’s test facilities or OpenCV’s test framework, not to mention the embedded Google Test framework. While the documentation for the former and the latter can be found easily, I have yet to find the documentation for OpenCV’s
ts module. As an alternative, I guess I’ll stick to reading the source of the module, and look at the examples in the test folders of the other modules.
Do not hesitate to comment on the StackOverflow question if you know more !