Boost C++ Libraries

How would you perform correctness test for

operator<< ( std::ostream &, ... )

operations?

You can print into the standard output stream and manually check that it is matching your expectations. Unfortunately, this is not really acceptable for the regression testing and doesn't serve a long term purpose of a unit test.

You can use std::stringstream and compare resulting output buffer with the expected pattern string, but you are required to perform several additional operations with every check you do. So it becomes tedious very fast.

The class boost::test_tools::output_test_stream is designed to automate these tasks for you. This is a simple, but powerful tool for testing standard std::ostream based output operation. The class output_test_stream complies to std::ostream interface so it can be used in place of any std::ostream parameter. It provides several test methods to validate output content, including test for match to expected output content or test for expected output length. Flushing, synchronizing, string comparison and error message generation is automated by the tool implementation.

All output_test_stream validation member functions by default flush the stream once the check is performed. If you want to perform several checks with the same output, specify parameter flush_stream with value false ^[16].

In some cases manual generation of expected output is either too time consuming or is impossible at all because of sheer volume. A possible way to address that issue is to split the test in two steps:

The class output_test_stream allows both the matching of the output content versus a pattern file and generation of this pattern file. The command line parameter save_pattern may be used to either generate a new pattern file, or to check against an existing pattern.

Usage

There are two ways to employ the class output_test_stream:

Explicit output checks

Use the instance of class output_test_stream as an output stream and check output content using tool's methods.

Example: Explicit output checks with output_test_stream

#define BOOST_TEST_MODULE example
#include <boost/test/included/unit_test.hpp>
#include <boost/test/tools/output_test_stream.hpp>
using boost::test_tools::output_test_stream;

BOOST_AUTO_TEST_CASE( test )
{
  output_test_stream output;
  int i=2;
  output << "i=" << i;
  BOOST_TEST( !output.is_empty( false ) );
  BOOST_TEST( output.check_length( 3, false ) );
  BOOST_TEST( output.is_equal( "i=3" ) );
}

> example
Running 1 test case...
test.cpp(15): error in "test": check output.is_equal( "i=3" ) has failed. Output content: "i=2"

*** 1 failures is detected in test suite "example"

	Note
	Use of false to prevent output flushing in first two invocation of check functions. Unless you want to perform several different checks for the same output you wouldn't need to use it though. Your test will look like a sequence of output operators followed by one check.

	Tip
	Try to perform checks as frequently as possible. It not only simplifies patterns you compare with, but also allows you to more closely identify possible source of failure.

Pattern file matching

The pattern file is a companion file containing the patterns that the stream should match. Your testing will look like a series of output operators followed by match pattern checks repeated several times.

In the example below, the file pattern_file contains the patterns that should match.

i=2
File: test.cpp Line:XXX

Example: Pattern file matching with output_test_stream

#define BOOST_TEST_MODULE example
#include <boost/test/included/unit_test.hpp>
#include <boost/test/tools/output_test_stream.hpp>
using boost::test_tools::output_test_stream;

BOOST_AUTO_TEST_CASE( test )
{
  output_test_stream output( "pattern_file", !boost::unit_test::runtime_config::save_pattern() );
  int i=2;
  output << "i=" << i;
  BOOST_TEST( output.match_pattern() );

  output << "\nFile: " << __FILE__ << " Line:YYY";
  BOOST_TEST( output.match_pattern() ); 
}

> example
Running 1 test case...
test.cpp(16): error in "test": check output.match_pattern() has failed. Mismatch at position 23
...5...
...4...

*** 1 failures is detected in test suite "example"

	Tip
	Try to perform checks as frequently as possible, because it allows you to more closely identify possible source of failure