libs/test/doc/testing_tools/boost_test_collection_comparison.qbk
[/ / Copyright (c) 2015 Boost.Test contributors / / Distributed under the Boost Software License, Version 1.0. (See accompanying / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) /] [section:collections Collections comparison] Instead of comparing a single value against another, there is often a need for comparing /collections/ of values. A collection and indirectly the values it contains may be considered in several ways: * collection as a /sequence of values/: this is the case for instance when `N` values are stored in a container. Containers in this case are used for storing several values, and iterating over the containers yields sequences that can be compared *element-wise*. The iteration should be in an order that is /a priori/ known [footnote this might not be the case for e.g. `std::unordered_map`, for which the buckets might be filled differently depending on the insertion order.], for being able to compare the sequences. The values in the collection are independent each other, and subsets can be compared as well. * collection as an /ensemble/: this is the case where the elements of the collection define an /entity/, and no element can be dissociated from the others. An example would be a collection of letters for a specific word in the natural language; in this settings any of the character in the word/collection depends /semantically/ on the other and it is not possible to take a subset of it without breaking the meaning of the word. Another example would be a vector of size `N` representing a point in a `N` dimensional space, compared to another point with the relation "`<`": the comparison is application specific and a possible comparison would be the lexicographical ordering [footnote in this case `v_a < v_b` means that the point `v_a` is inside the rectangle (origin, `v_b`)]. The following observations can be done: * the methods employed for comparing collections should be chosen adequately with the meaning of the collection, * comparing sequences *element-wise* often involves writing loops in the test body, and if a dedicated tool is already in place the test body would gain in clarity and expressiveness (including the report in case of failure), * some comparison methods such as the lexicographical one, have good general behavior (e.g. total ordering, defined for collections of different size), but are sometimes inappropriate. __BOOST_TEST__ provides specific tools for comparing collections: * using the /native/ [footnote either defined by the container or by the user] operator of the container of the collection, which is mentioned as the [link ref_boost_test_coll_default_comp /default behavior/]. * using [link boost_test_coll_perelement element-wise] comparison for which extended failure diagnostic is provided, * and using [link boost_test_coll_default_lex lexicographical] comparison for which extended failure diagnostic is provided, More details about the concept of /collection/ in the __UTF__ is given [link what_is_a_collection /here/]. [#ref_boost_test_coll_default_comp][h3 Default comparison] The default comparison dispatches to the existing overloaded comparison operator. The __UTF__ distinguishes two use cases # none of the comparison operand is a C-Array, in which case we use the [link ref_boost_test_coll_default_comp_container container default behaviour] # one of the comparison operand is a C-array, in which case we [link ref_boost_test_coll_c_arrays mimic `std::vector`] behaviour [#ref_boost_test_coll_default_comp_container][h4 Container default behaviour] Given two containers `c_a` and `c_b` that are not C-arrays, `` BOOST_TEST(c_a op c_b) `` is equivalent, in terms of test success, to `` auto result = c_a op c_b; BOOST_TEST(result); `` In the example below, `operator==` is not defined for `std::vector` of different types, and the program would fail to compile if the corresponding lines were uncommented (`std::vector` uses lexicographical comparison by default). [note In the case of default comparison, there is no additional diagnostic provided by the __UTF__. See the section [link ref_boost_test_coll_special_macro `BOOST_TEST_SPECIALIZED_COLLECTION_COMPARE`] below.] [bt_example boost_test_container_default..BOOST_TEST containers comparison default..run-fail] [#ref_boost_test_coll_c_arrays][h4 C-arrays default behaviour] As soon as one of the operands is a C-array, there is no /default behaviour/ the __UTF__ can dispatch to. This is why in that case, the comparison mimics the `std::vector` behaviour. [bt_example boost_test_macro_container_c_array..BOOST_TEST C-arrays..run-fail] [#boost_test_coll_perelement][h3 Element-wise comparison] By specifying the manipulator [classref boost::test_tools::per_element], the comparison of the elements of the containers are performed /element-wise/, in the order given by the forward iterators of the containers. This is a comparison on the /sequences/ of elements generated by the containers, for which the __UTF__ provides advanced diagnostic. In more details, let `c_a = (a_1,... a_n)` and `c_b = (b_1,... b_n)` be two sequences of same length, but not necessarily of same type. Those sequences correspond to the content of the respective containers, in the order given by their iterator. Let `op` be one of the [link boost_test_statement_overloads binary comparison operators]. `` BOOST_TEST(c_a op c_b, boost::test_tools::per_element() ); `` is equivalent to `` if(c_a.size() == c_b.size()) { for(int i=0; i < c_a.size(); i++) { __BOOST_TEST_CONTEXT__("index " << i) { BOOST_TEST(a_i op b_i); } } } else { BOOST_TEST(c_a.size() == c_b.size()); } `` [warning this is fundamentally different from using the containers' default comparison operators (default behavior).] [warning this is not an order relationship on containers. As a side effect, it is possible to have ``BOOST_TEST(c_a == c_b)`` and ``BOOST_TEST(c_a != c_b)`` failing at the same time] Sequences are compared using the specified operator `op`, evaluated on the left and right elements of the respective sequences. The order of the compared elements is given by the iterators of the respective containers [footnote the containers should yield the same sequences for a fixed set of elements they contain]. In case of failure, the indices of the elements failing `op` are returned. [bt_example boost_test_sequence_per_element..BOOST_TEST sequence comparison..run-fail] [h4 Requirements] For the sequences to be comparable element-wise, the following conditions should be met: * the containers should meet the [link what_is_a_collection sequence] definition, * the containers should yield the same number of elements, * `op` should be one of the comparison operator `==`, `!=`, `<`, `<=`, `>`, `>=` * the `a_i op b_i` should be defined, where the type of `a_i` and `b_i` are the type returned by the dereference operator of the respective collections. [caution the resulting type of "`c_a == c_b`" is an [classref boost::test_tools::assertion_result assertion_result]: it is not possible to compose more that one comparison on the `BOOST_TEST` statement: `` BOOST_TEST(c_a == c_b == 42, boost::test_tools::per_element() ); // does not compile ``] [#boost_test_coll_default_lex][h3 Lexicographic comparison] By specifying the manipulator [classref boost::test_tools::lexicographic], the containers are compared using the /lexicographical/ order and for which the __UTF__ provides additional diagnostic in case of failure. `` BOOST_TEST(c_a op c_b, boost::test_tools::lexicographic() ); `` The comparison is performed in the order given by forward iterators of the containers. [tip lexicographic comparison yields a total order on the containers: the statements `c_a < c_b` and `c_b <= c_a` are mutually exclusive.] [note The equality `==` and inequality `!=` are not available for this type of comparison.] [bt_example boost_test_container_lex..BOOST_TEST container comparison using lexicographical order..run-fail] [#ref_boost_test_coll_special_macro][h3 Extended diagnostic by default for specific containers] As seen above, * for testing equality, the `==` relation is either explicit (using `boost::test_tools::per_element()`) or implicit when the container overloads/implements this type of comparison, * for testing inequality, lexicographical comparison for `<` (and derived operations) is either explicit (using `boost::test_tools::lexicographic()`) or implicit when the container overloads/implements uses this type of comparison. When the default is to using the container implementation, it is not possible to benefit from an extended failure diagnostic. The __UTF__ provides a mechanism for performing the same comparisons through the __UTF__ instead of the container operator, through the macro `BOOST_TEST_SPECIALIZED_COLLECTION_COMPARE` that might be used as follow: [bt_example boost_test_container_lex_default..Default `std::vector<int>` to lexicographic with extended diagnostic..run-fail] [h4 Requirements] * the containers should meet the [link what_is_a_collection sequence] definition, * the containers should be of the exact same type * `op` should be one of the comparison operator `==`, `!=`, `<`, `<=`, `>`, `>=` [note Note that the operation `!=` is in this case not an element-wise comparison, ] [#what_is_a_collection][h3 What is a sequence?] A /sequence/ is given by the iteration over a /forward iterable/ container. A forward iterable container is: * either a C-array, where the underlying type is neither `char` or `wchar_t`, * or a `class`/`struct` that implements the member functions `begin` and `end` and for which `value_type` is neither `char` or `wchar_t` When the stored type `char` or `wchar_t`, then the container is considered as a string and is dispatched on [link boost_test.testing_tools.extended_comparison.strings string comparison] instead. [note Since Boost.Test 3.6 (Boost 1.65) the requirements for the collection concepts have been relaxed to include C-arrays as well] The detection of the types that meet these requirements containers is delegated to the class [classref boost::unit_test::is_forward_iterable], which for C++11 detects the required member functions and fields. However for C++03, the types providing the sequences should be explicitly indicated to the __UTF__ by a specialization of [classref boost::unit_test::is_forward_iterable] [footnote Standard containers of the `STL` are recognized as /forward iterable/ container.]. [endsect] [/ sequences]