Boost Test Library: The Test Tools

Introduction
Reference
Floating point comparison algorithms
Output testing tool
Custom predicate support
Implementation
Examples and tests

Introduction

Boost Test Library's Test Tools component supply a toolbox of instruments to ease a creation and a maintenance of test programs and provide a uniform error reporting mechanism. The toolbox supplied in most part in a form of macro and function declarations. While the functions can be called directly, the usual way to use Test Tools is via convenience macros. All macros arguments are calculated once, so it's safe to pass complex expressions in their place. All tools automatically supply an error location: a file name and a line number. Boost Test Library's Test Tools are intended for unit test code rather than library or production code, where throwing exceptions, using assert(), boost::concept_check or BOOST_STATIC_ASSERT() may be more suitable ways to detect and report errors. For list of all supplied Test Tools and usage examples see the reference.

Test Tools flavours

All the tools supplied (with one exclusion) in three flavours(levels): WARN, CHECK and REQUIRE (for example BOOST_WARN_EQUAL, BOOST_CHECK_EQUAL, BOOST_REQUIRE_EQUAL). If an assertion designated by the tool passes, confirmation message could be printed in log output (note: to manage what messages appear in the test log stream set the proper log level). If an assertion designated by the tool failed, depending on the level following will happened (in some cases log message could be slightly different to reflect failed tool specifics):

Level	Output log content	Errors counter	Test execution
WARN	warning in <test case name>: condition <assertion description> is not satisfied	not affected	continues
CHECK	error in <test case name>: test <assertion description> failed	increased	continues
REQUIRE	fatal error in <test case name>: critical test <assertion description> failed	increased	aborts

Regularly you should use CHECK level tools to implement your assertions.You could use WARN level tools to validate aspects less important then correctness: performance, portability, usability etc. You should use REQUIRE level tools only if continuation of the test case doesn't make sense if this assertions fails.

Output testing tool

In addition to the checking tools toolbox, the Test Tools also contains class ostream_test_stream. This is a class designed to significantly simplify correctness testing of the ostream based output procedures. Detailed description of interface and example usage you could see on the ostream_test_stream page.

Logging tool arguments

Most of test tools direct values of their arguments to the output stream in some form of log statement. If arguments type does not support operator<<(std::ostream&,ArgumentType const&) interface you will get a compilation error. You could either implement above interface or prohibit the Test Tools from logging argument values for specified type. To do so use following statement on file level before first test case that includes statement failing to compile:

BOOST_TEST_DONT_PRINT_LOG_VALUE( ArgumentType ).

Logging floating point numbers

It may appear that floating-point numbers are displayed by Boost.Test with an excessive number of decimal digits. However the number of digits shown is chosen to avoid apparently nonsensical displays like [1.00000 != 1.00000] when comparing exactly unity against a value which is increased by just one least significant binary digit using the default precision for float of just 6 decimal digits, given by std::numeric_limits<float>::digits10. The function used for the number of decimal digits displayed is that proposed for a future C++ Standard, A Proposal to add a max significant decimal digits value, to be called numeric_limits::max_digits10();. For 32-bit floats, 9 decimal digits are needed to ensure a single bit change produces a different decimal digit string.

So a much more helpful display using 9 decimal digits is thus: [1.00000000 != 1.00000012] showing that the two values are in fact different.

For IEEE754 32-bit float values - 9 decimal digits are shown. For 64-bit IEEE754 double - 17 decimal digits.
For IEEE754 extended long double using 80-bit - 21 decimal digits. For IEEE754 quadruple long double 128-bit, and Sparc extended long double 128-bit - 36 decimal digits. For floating-point types, a convenient formula to calculate max_digits10 is:
  2 + std::numeric_limits<FPT>::digits * 3010/10000;

Note that a user defined floating point type UDFPT must define numeric_limits<UDFPT>::is_specialized = true and provide an appropriate value for std::numeric_limits<UDFPT>::digits, the number of bits used for the significand or mantissa. For example, for the Sparc extended long double 128, 113 bits are used for the significand (one of which is implicit).

Custom predicate support

Even though supplied test tools cover wide range of possible checks and provide detailed report on cause of error in some cases you may want to implement and use custom predicate that perform complex check and produce intelligent report on failure. To satisfy this need the Test Tools implement custom predicate support.

Implementation

The Test Tools are implemented in three modules: two header files and one source file.

boost/test/test_tools.hpp:

contains definition for the convenience macros, some template based Test Tools implementation functions and class ostream_test_stream.

boost/test/floating_point_comparison.hpp:

contains implementation for the floating point comparison algorithms, used by floating point comparison tools. They also could be used directly.

libs/test/test_tools.cpp:

contains definition for the most Test Tools implementation functions and class ostream_test_stream implementation.

Since this component is not intended to be used standalone, there are no special compilation instruction for it.

The Test Tools are implemented in namespace boost::test_tools. But in majority of the cases you don't need to know about that, since the tool are used through convinience macros that fully qualify implementation functions access.

Examples and Tests

test_tools_test
output_test_stream_test
boost_check_equal_str
test_fp_comparisons