Test organization or the house that Jack built

> The Unit Test Framework > User's guide > Test organization >

Test organization or the house that Jack built

If you look at many legacy test modules, big chance is that it's implemented as one big test function that consists of a mixture of check and output statements. Is there anything wrong with it? Yes. There are various disadvantages in single test function approach:

One big function tends to become really difficult to manage if the number of checks exceeds a reasonable limit (true for any large function). What is tested and where - who knows?
Many checks require similar preparations. This results in code repetitions within the test function.
If a fatal error or an exception is caused by any checks within the test function the rest of tests are skipped and there is no way to prevent this.
No way to perform only checks for a particular subsystem of the tested unit.
No summary of how different subsystems of the tested unit performed under in the test.

The above points should make it clear that it's preferable to split test module into smaller units. These units are test cases. A test case has to be constructed based on some kind of function and registered in a test tree, so that the test runner knows how to invoke it. There are different possible designs for the test case construction problem: inheritance from the predefined base class, specifically named member function and so on. The UTF opted to avoid classed altogether and to use the least intrusive " generic callback" approach. The facility, the UTF provides, requires specific function signature and allows adopting any function or function object that matches the signature as a test case. The signatures the UTF supports are:

To solve test tree creation problem the UTF provides facilities for test suite creation.

Generic test case construction design used by the UTF causes the test case implementation (test function body) and test case creation/registration points to be remote. As a result you may forget to register the test case and it's never going to be executed, even though it's present in test file. To alleviate this issue the UTF presents facilities for automated (in place) test case creation and registration in the test tree. These facilities sacrifice some generality and work for selected test function signatures only. But the result is that library users are relieved from the necessity to manually register test cases. These facilities are the most user-friendly and are recommended to be used whenever possible. In addition they support automated registration of test suites, thus allowing whole test tree to be created without any use of manual registration.

The single test module may mix both automated and manual test case registration. In other words, within the same test module you can have both test cases implemented remotely and registered manually in the test module initialization function and test cases that are registered automatically at implementation point.

In some cases it's desirable to allow some "expected" failures in test case without failing a test module. To support this request The UTF allows specifying the number of expected failures in a test case.

Boost C++ Libraries

Test organization or the house that Jack built