...one of the most highly
regarded and expertly designed C++ library projects in the
world.
— Herb Sutter and Andrei
Alexandrescu, C++
Coding Standards
Some tests are required to be repeated for a series of different input parameters. One way to achieve this is manually register a test case for each parameter. You can also invoke a test function with all parameters manually from within your test case, like this:
void single_test( int i )
{
BOOST_TEST
( /* test assertion */ );
}
void combined_test()
{
int params[] = { 1, 2, 3, 4, 5 };
std::for_each( params, params+5, &single_test );
}
The approach above has several drawbacks:
single_test
in the above example
is run from the test case combined_test
while its execution would be better handled by the Unit Test
Framework
param
array above (say a failure in BOOST_TEST_REQUIRE
), the test
combined_test
is aborted
and the next test-case in the test tree is executed.
In some circumstance, one would like to run a parametrized test over an arbitrary large set of values. Enumerating the parameters by hand is not a solution that scales well, especially when these parameters can be described in another function that generates these values. However, this solution has also limitations
func(float f)
, where f
is any number in [0, 1]. We are not interested that much in the exact
value, but we would like to test func
.
What about, instead of writing the f
for which func
will
be tested against, we choose randomly f
in [0, 1]? And also what about instead of having only one value for
f
, we run the test
on arbitrarily many numbers? We easily understand from this small example
that tests requiring parameters are more powerful when, instead of
writing down constant values in the test, a generating function is
provided.
func1
, on which
we test N
values written
as constant in the test file. What does the test ensure? We have the
guaranty that func1
is working on these N
values. Yet in this setting N
is necessarily finite and usually small. How would we extend or scale
N
easily? One solution
is to be able to generate new values, and to be able to define a test
on the class of possible inputs for
func1
on which the
function should have a defined behavior. To some extent, N
constant written down in the test
are just an excerpt of the possible inputs of func1
,
and working on the class of inputs gives more flexibility and power
to the test.
Composition: suppose we already have
test cases for two functions func1
and func2
, taking as
argument the types T1
and T2
respectively.
Now we would like to test a new functions func3
that takes as argument a type T3
containing T1
and
T2
, and calling func1
and func2
through a known algorithm. An example of such a setting would be
// Returns the log of x // Precondition: x strictly positive. double fast_log(double x); // Returns 1/(x-1) // Precondition: x != 1 double fast_inv(double x); struct dummy { unsigned int field1; unsigned int field2; }; double func3(dummy value) { return 0.5 * (exp(fast_log(value.field1))/value.field1 + value.field2/fast_inv(value.field2)); }
In this example,
func3
inherits
from the preconditions of fast_log
and fast_inv
:
it is defined in (0, +infinity)
and in [-C,
+C] -
{1}
for field1
and field2
respectively
(C
being a constant
arbitrarily big).
func3
should be close to 1 everywhere on its definition domain.
fast_log
and fast_inv
in the compound function func3
and assert that func3
is well defined over an arbitrary large definition domain.
Having parametrized tests on func3
hardly tells us about the possible numerical properties or instabilities
close to the point {field1
= 0, field2
= 1}
. Indeed, the parametrized test may
test for some points around (0,1), but will fail to provide an asymptotic behavior of the function close to
this point.
The facilities provided by the Unit Test Framework addressed the issues described above:
BOOST_DATA_TEST_CASE
, is used
for the declaration and registration of a test case over a collection
of values (samples),
The remainder of this section covers the notions and feature provided by the Unit Test Framework about the data-driven test cases, in particular: