Home > The Unit Test Framework > User's guide >

Unit Test Framework: User's guide

Table of Contents

Usage variants
Static library
Dynamic library
Single header
External test runner
Supplied test runners
External test runner
Test module initialization
Test organization
Nullary function based test case
Manual registration
Automated registration
Unary function based test case
Test case template
Manual registration
Automated registration
Test suite
Manual registration
Automated registration
Master Test Suite
Expected failures specification
Generic model
Per test case
Test suite shared
Global fixture
Test Output
Test log
Logging floating point type numbers
Human readable format
XML based log output format
Compile time configuration
Test report output
Progress display
Runtime configuration
Run by name
Parameters reference

Introduction or what's your name?

Without further ado, let's define terms regularly used by the UTF.

The test module

This is a single binary that performs the test. Physically a test module consists of one or more test source files, which can be built into an executable or a dynamic library. A test module that consists of a single test source file is called single-file test module. Otherwise it's called multi-file test module. Logically a test module consists of four parts: test setup (or test initialization), test body, test cleanup and test runner. The test runner part is optional. If a test module is built as an executable the test runner is built-in. If a test module is built as a dynamic library, it is run by an external test runner.

The test body

This is the part of a test module that actually performs the test. Logically test body is a collection of test assertions wrapped in test cases, which are organized in a test tree .

The test tree

This is a hierarchical structure of test suites (non-leaf nodes) and test cases (leaf nodes).

The test unit

This is a collective name when referred to either test suite or test case

Test assertion

This is a single binary condition (binary in a sense that is has two outcomes: pass and fail) checked by a test module.

There are different schools of thought on how many test assertions a test case should consist of. Two polar positions are the one advocated by TDD followers - one assertion per test case; and opposite of this - all test assertions within single test case - advocated by those only interested in the first error in a test module. The UTF supports both approaches.

The test case

This is an independently monitored function within a test module that consists of one or more test assertions. The term "independently monitored" in the definition above is used to emphasize the fact, that all test cases are monitored independently. An uncaught exception or other normal test case execution termination doesn't cause the testing to cease. Instead the error is caught by the test case execution monitor, reported by the UTF and testing proceeds to the next test case. Later on you are going to see that this is on of the primary reasons to prefer multiple small test cases to a single big test function.

The test suite

This is a container for one or more test cases. The test suite gives you an ability to group test cases into a single referable entity. There are various reasons why you may opt to do so, including:

  • To group test cases per subsystems of the unit being tested.
  • To share test case setup/cleanup code.
  • To run selected group of test cases only.
  • To see test report split by groups of test cases
  • To skip groups of test cases based on the result of another test unit in a test tree.

A test suite can also contain other test suites, thus allowing a hierarchical test tree structure to be formed. The UTF requires the test tree to contain at least one test suite with at least one test case. The top level test suite - root node of the test tree - is called the master test suite.

The test setup

This is the part of a test module that is responsible for the test preparation. It includes the following operations that take place prior to a start of the test:

  • The UTF initialization
  • Test tree construction
  • Global test module setup code

Per test case" setup code, invoked for every test case it's assigned to, is also attributed to the test initialization, even though it's executed as a part of the test case.

The test cleanup

This is the part of test module that is responsible for cleanup operations.

The test fixture

Matching setup and cleanup operations are frequently united into a single entity called test fixture.

The test runner

This is an "executive manager" that runs the show. The test runner's functionality should include the following interfaces and operations:

  • Entry point to a test module. This is usually either the function main() itself or single function that can be invoked from it to start testing.
  • Initialize the UTF based on runtime parameters
  • Select an output media for the test log and the test results report
  • Select test cases to execute based on runtime parameters
  • Execute all or selected test cases
  • Produce the test results report
  • Generate a test module result code.

An advanced test runner may provide additional features, including interactive GUI interfaces, test coverage and profiling support.

The test log

This is the record of all events that occur during the testing.

The test results report

This is the report produced by the UTF after the testing is completed, that indicates which test cases/test suites passed and which failed.