Floating point comparison

Enabling tolerance for user-defined types
Tolerance-based comparisons
Theory behind floating point comparisons

Unless specified otherwise, when a value of floating-point type is compared inside a BOOST_TEST assertion, operators ==, != , < etc. defined for this type are used. However for floating point type, in most cases what is needed is not an exact equality (or inequality), but a verification that two numbers are sufficiently close or sufficiently different. For that purpose, a tolerance parameter that will instruct the framework what is considered sufficiently close needs to provided.

	Note
	How the tolerance parameter is processed in detail is described here.

Test-unit tolerance

It is possible to define a per-test unit tolerance for a given floating point type by using decorator tolerance:

Example: specifying tolerance per test case

Code
#define BOOST_TEST_MODULE tolerance_01 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001)) { double x = 10.0000000; double y = 10.0000001; double z = 10.001; BOOST_TEST(x == y); // irrelevant difference BOOST_TEST(x == z); // relevant difference }

Code

#define BOOST_TEST_MODULE tolerance_01
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;

BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001))
{
  double x = 10.0000000;
  double y = 10.0000001;
  double z = 10.001;
  BOOST_TEST(x == y); // irrelevant difference
  BOOST_TEST(x == z); // relevant difference
}

Output
> tolerance_01 Running 1 test case... test.cpp(11): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005] *** 1 failure is detected in the test module "tolerance_01"

Output

> tolerance_01
Running 1 test case...
test.cpp(11): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005]

*** 1 failure is detected in the test module "tolerance_01"

Assertion tolerance

It is possible to specify floating point comparison tolerance per single assertion, by providing the manipulator boost::test_tools::tolerance as the second argument to BOOST_TEST:

Example: specifying tolerance per assertion

Code
#define BOOST_TEST_MODULE tolerance_02 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001)) { double x = 10.0000000; double y = 10.0000001; double z = 10.001; BOOST_TEST(x == y); // irrelevant by default BOOST_TEST(x == y, tt::tolerance(0.0)); BOOST_TEST(x == z); // relevant by default BOOST_TEST(x == z, tt::tolerance(0.001)); }

Code

#define BOOST_TEST_MODULE tolerance_02
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;
namespace tt = boost::test_tools;

BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001))
{
  double x = 10.0000000;
  double y = 10.0000001;
  double z = 10.001;
  BOOST_TEST(x == y); // irrelevant by default
  BOOST_TEST(x == y, tt::tolerance(0.0));

  BOOST_TEST(x == z); // relevant by default
  BOOST_TEST(x == z, tt::tolerance(0.001));
}

Output
> tolerance_02 Running 1 test case... test.cpp(12): error: in "test1": check x == y has failed [10 != 10.000000099999999] test.cpp(14): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005] *** 2 failures are detected in the test module "tolerance_02"

Output

> tolerance_02
Running 1 test case...
test.cpp(12): error: in "test1": check x == y has failed [10 != 10.000000099999999]
test.cpp(14): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005]

*** 2 failures are detected in the test module "tolerance_02"

	Caution
	Manipulators requires a compiler that supports variadic macros, `auto` for type deduction and `decltype`. These are C++11 features, but are also available on some pre-C++11 compilers. On compilers that are lacking these features, resort to defining tolerance per test unit or to compatibility test assertions: `BOOST_CHECK_CLOSE` and `BOOST_CHECK_SMALL`.

Tolerance expressed in percentage

It is possible to specify the tolerance as percentage. At test unit level, the decorator syntax is:

* boost::unit_test::tolerance( boost::test_tools::fpc::percent_tolerance(2.0) )
// equivalent to: boost::unit_test::tolerance( 2.0 / 100 )

At assertion level, the manipulator syntax is:

2.0% boost::test_tools::tolerance()
boost::test_tools::tolerance( boost::test_tools::fpc::percent_tolerance(2.0) )
// both equivalent to: boost::test_tools::tolerance( 2.0 / 100 )

Type of the tolerance

Manipulator tolerance specifies the tolerance only for a single floating-point type. This type is deduced from form the numeric value passed along the manipulator:

expression	semantics
`tolerance(0.5)`	tolerance for type `double` changed to 0.5
`tolerance(float(0.5))`	tolerance for type `float` changed to 0.5
`tolerance(0.5f)`	tolerance for type `float` changed to 0.5
`tolerance(0.5L)`	tolerance for type `long double` changed to 0.5
`tolerance(Decimal("0.5"))`	tolerance for a user-defined type `Decimal` changed to the supplied value
`5.0% tolerance()`	tolerance for type `double` changed to 0.05 (`5.0 / 100`)
`5.0f% tolerance()`	tolerance for type `float` changed to 0.05
`Decimal("5.0")% tolerance()`	tolerance for type `Decimal` changed to value `(Decimal("5.0") / 100)`

This is also the case for decorator tolerance. In the case of the decorator however, it is possible to apply multiple decorators tolerance defining the tolerance for different types.

When values of two different floating point types T and U are compared, BOOST_TEST uses the tolerance specified for type boost::common_type<T, U>::type. For instance, when setting a tolerance for mixed float-to-double comparison, the tolerance for type double needs to be set.

Given two floating point types T and U and their common type C, the tolerance specified for type C is applied only when types T and U appear as sub-expressions of the full expression inside assertion BOOST_TEST. It is not applied when T and U are compared inside a function invoked during the evaluation of the expression:

Example: tolerance applied to different types

Code
#define BOOST_TEST_MODULE tolerance_05 #include <boost/test/included/unit_test.hpp> #include <boost/optional.hpp> #include <boost/optional/optional_io.hpp> BOOST_AUTO_TEST_CASE(test, * boost::unit_test::tolerance(0.02)) { double d1 = 1.00, d2 = 0.99; boost::optional<double> o1 = 1.00, o2 = 0.99; BOOST_TEST(d1 == d2); // with tolerance (double vs. double) BOOST_TEST(o1 == o2); // without tolerance (optional vs. optional) BOOST_TEST(o1 == d2); // without tolerance (optional vs. double) BOOST_TEST(o1 == o2); // with tolerance (double vs. double) }

Code

#define BOOST_TEST_MODULE tolerance_05
#include <boost/test/included/unit_test.hpp>
#include <boost/optional.hpp>
#include <boost/optional/optional_io.hpp>

BOOST_AUTO_TEST_CASE(test, * boost::unit_test::tolerance(0.02))
{
  double                  d1 = 1.00, d2 = 0.99;
  boost::optional<double> o1 = 1.00, o2 = 0.99;

  BOOST_TEST(d1 == d2);   // with tolerance    (double vs. double)
  BOOST_TEST(o1 == o2);   // without tolerance (optional vs. optional)
  BOOST_TEST(o1 == d2);   // without tolerance (optional vs. double)
  BOOST_TEST(*o1 == *o2); // with tolerance    (double vs. double)
}

Output
> tolerance_05 Running 1 test case... ../doc/examples/tolerance_05.run.cpp(20): error: in "test": check o1 == o2 has failed [ 1 != 0.99] ../doc/examples/tolerance_05.run.cpp(21): error: in "test": check o1 == d2 has failed [ 1 != 0.98999999999999999] *** 2 failures are detected in the test module "tolerance_05"

Output

> tolerance_05
Running 1 test case...
../doc/examples/tolerance_05.run.cpp(20): error: in "test": check o1 == o2 has failed [ 1 !=  0.99]
../doc/examples/tolerance_05.run.cpp(21): error: in "test": check o1 == d2 has failed [ 1 != 0.98999999999999999]

*** 2 failures are detected in the test module "tolerance_05"

Type promotion of the operands

Given two types T and U being compared inside an assertion BOOST_TEST, tolerance based comparison is invoked

whenever the types T and U are both tolerance based types
whenever T is tolerance based and U is arithmetic, in the sense that std::numeric_limits<U>::value evaluates to true (or the other way round)

In all cases, the type of the tolerance is deduced as boost::common_type<T, U>::type, and both type may be cast to this tolerance type.

Note

This behavior has been introduced in Boost 1.70 / Unit Test Framework 3.10. Previously tolerance based comparison was used only when the type of the two operands were tolerance based types, which was silently ignoring the tolerance for expressions such as

double x = 1E-9;
BOOST_TEST(x == 0); // U is int

Example: operands type promotion

Code
#define BOOST_TEST_MODULE tolerance_06 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.1415 / 3)) // == 0.047166667 { double x = 3.141592404915836; // x is 'double' which is tolerance based, 3 is 'int' which is arithmetic: // tolerance based comparison will be used. // Type of tolerance for this comparison will be boost::common_type<double, int>::type == double // Value for this tolerance type is set by the decorator. BOOST_TEST(x == 3); }

Code

#define BOOST_TEST_MODULE tolerance_06
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;
namespace tt = boost::test_tools;

BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.1415 / 3)) // == 0.047166667
{
  double x = 3.141592404915836;
  // x is 'double' which is tolerance based, 3 is 'int' which is arithmetic:
  // tolerance based comparison will be used.
  // Type of tolerance for this comparison will be boost::common_type<double, int>::type == double
  // Value for this tolerance type is set by the decorator.
  BOOST_TEST(x == 3);
}

Output
> tolerance_06 Running 1 test case... test.cpp(21): error: in "test1": check x == 3 has failed [3.1415924049158361 != 3]. Relative difference exceeds tolerance [0.0471975 > 0.0471667] *** 1 failure is detected in the test module "tolerance_06"

Output

> tolerance_06
Running 1 test case...
test.cpp(21): error: in "test1": check x == 3 has failed [3.1415924049158361 != 3]. Relative difference exceeds tolerance [0.0471975 > 0.0471667]

*** 1 failure is detected in the test module "tolerance_06"

Other relational operators

Finally, note that comparisons for tolerance are also applied to operator< with semantics less by more than some tolerance, and other relational operators. Also, the tolerance-based comparisons are involved when a more complicated expression tree is processed within the assertion body. The section on relational operators defines how operator< relates to tolerance.

Example: tolerance applied in more complex expressions

Code
#define BOOST_TEST_MODULE tolerance_03 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; double x = 10.000000; double d = 0.000001; BOOST_AUTO_TEST_CASE(passing, * utf::tolerance(0.0001)) { BOOST_TEST(x == x + d); // equal with tolerance BOOST_TEST(x >= x + d); // ==> greater-or-equal BOOST_TEST(d == .0); // small with tolerance } BOOST_AUTO_TEST_CASE(failing, * utf::tolerance(0.0001)) { BOOST_TEST(x - d < x); // less, but still too close BOOST_TEST(x - d != x); // unequal but too close BOOST_TEST(d > .0); // positive, but too small BOOST_TEST(d < .0); // not sufficiently negative }

Code

#define BOOST_TEST_MODULE tolerance_03
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;

double x = 10.000000;
double d =  0.000001;

BOOST_AUTO_TEST_CASE(passing, * utf::tolerance(0.0001))
{
  BOOST_TEST(x == x + d); // equal with tolerance
  BOOST_TEST(x >= x + d); // ==> greater-or-equal

  BOOST_TEST(d == .0);    // small with tolerance
}

BOOST_AUTO_TEST_CASE(failing, * utf::tolerance(0.0001))
{
  BOOST_TEST(x - d <  x); // less, but still too close
  BOOST_TEST(x - d != x); // unequal but too close

  BOOST_TEST(d > .0);     // positive, but too small
  BOOST_TEST(d < .0);     // not sufficiently negative
}

Output
> tolerance_03 Running 2 test cases... test.cpp(18): error: in "failing": check x - d < x has failed [10 - 1.0000000000000001e-05 >= 10]. Relative difference is within tolerance [1e-06 < 0.001] test.cpp(19): error: in "failing": check x - d != x has failed [10 - 1.0000000000000001e-05 == 10]. Relative difference is within tolerance [1e-06 < 0.001] test.cpp(21): error: in "failing": check d > .0 has failed [1.0000000000000001e-05 <= 0]. Absolute value is within tolerance [\|1e-05\| < 0.001] test.cpp(22): error: in "failing": check d < .0 has failed [1.0000000000000001e-05 >= 0] *** 4 failures are detected in the test module "tolerance_03"

Output

> tolerance_03
Running 2 test cases...
test.cpp(18): error: in "failing": check x - d < x has failed [10 - 1.0000000000000001e-05 >= 10]. Relative difference is within tolerance [1e-06 < 0.001]
test.cpp(19): error: in "failing": check x - d != x has failed [10 - 1.0000000000000001e-05 == 10]. Relative difference is within tolerance [1e-06 < 0.001]
test.cpp(21): error: in "failing": check d > .0 has failed [1.0000000000000001e-05 <= 0]. Absolute value is within tolerance [|1e-05| < 0.001]
test.cpp(22): error: in "failing": check d < .0 has failed [1.0000000000000001e-05 >= 0]

*** 4 failures are detected in the test module "tolerance_03"