Cmocka - should we all be creating one executable per test for isolating static variables?

Question

Failure

Given the following cmocka test with a static variable in a dependency:

main.c

#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>
#include <cmocka.h>
#include "dependency.h"

void test1(void** state) {
    increment();
    assert_int_equal(1, get_incrementer());
}

void test2(void** state) {
    increment();
    assert_int_equal(1, get_incrementer());
}

int main() {
    const struct CMUnitTest tests[] = {
        cmocka_unit_test(test1),
        cmocka_unit_test(test2),
    };
    return cmocka_run_group_tests(tests, NULL, NULL);
}

dependency.h

void increment();
int get_incrementer();

dependency.c

static int incrementer = 0;

void increment() {
    ++incrementer;
}

int get_incrementer() {
    return incrementer;
}

(compile with gcc main.c dependency.c -lcmocka and run)

I get the following output:

[==========] Running 2 test(s).
[ RUN      ] test1
[       OK ] test1
[ RUN      ] test2
[  ERROR   ] --- 0x1 != 0x2
[   LINE   ] --- main.c:14: error: Failure!
[  FAILED  ] test2
[==========] 2 test(s) run.
[  PASSED  ] 1 test(s).
[  FAILED  ] 1 test(s), listed below:
[  FAILED  ] test2

 1 FAILED TEST(S)

This is because incrementer is not "reset" between each test.

This is just a minimal example to demonstrate a point. In reality, my dependency is deep down into the code which I'm calling in my test.

Alternative to Just Make It Work

But If I separate each test into its own executable with its own test group, I'm able to reset incrementer for each test as one would expect.

Questions

• Is the only way to achieve test isolation with cmocka to separate each test into its own executable?

It shocked me when I found that this was happening. I'm new to C and C testing and I had always thought that test isolation was a critical part of a unit testing framework. It doesn't make sense to me why there would even be test groups to begin with, as most of the time you will be testing code with dependencies where a static variable is used somewhere, and would therefore be making one executable per test.

Answer 1

Yes, if you have a static variable in the tested object, you need to compile separated tests if they expect a "fresh" test object.

I would do this anyway, each test in a separated executable, since you have a Singleton in your test object.

This is a fine example why the Singleton pattern is an anti-pattern today. It is hard to test.