Preprocessor library limitations

The Boost preprocessor library has always had certain specific limitations, which are specifically explained in other areas of this documentation and are implemented as object-like macros in the header file config/limits.hpp. These limitations largely detail the maximum number for certain features in the library. These maximums have been chosen in order to accommodate the large number of compilers and their preprocessors which the library supports, in order to provide a lowest common set of limits with which the vast majority of these compilers can work successfully. This reflects the situation that the C/C++ standard rarely specifies preprocessor limitations for compilers but rather allows each compiler to choose whatever limitations it deems necessary in its preprocessor. While this allows individual compilers to largely choose what preprocessor limitations it deems necessary, it makes it more difficult for a preprocessor library like this one to work with a large number of preprocessors across a number of operating system platforms. The difficulty of this situation means that this Boost preprocessor library has chosen fairly conservative limitations.

In fact the limitation macros, although indeed existing, have never previously been used internally in the preprocessor library code. Instead while the limitation macros do indeed exist, the library has internally used fixed numbers equivalent to what these macros represent. This historical situation most probably reflects the fact that the library did not want the end-user's possibly changing the limitations, by redefining the macros in config/limits.hpp, to change how the library works and therefore cause problems in using the library.

This situation has now changed and a number of the limitation macros specified in config/limits.hpp are now used internally in the library code when the preprocessor is C++ standard conforming. This topic is therefore about how an end-user can change the limitations of the library.

Type of limitations

The type of limitations in the Boost preprocessor library can be divided as:

The library has support for numbers. This supports basic arithmetic, comparisons, and logical operations. This support is documented by the macro BOOST_PP_LIMIT_MAG, where 'MAG' is short for "MAGNITUDE'. The current fixed maximum value is 256. This means that the library supports numerical operations in the range between 0 and 256.

The library has support for the maximum sizes of higher level types. These types are arrays, lists, seqs, tuples, and variadic data. By sizes for these higher level constructs it is meant the maximum number of elements of which a higher level construct can consist. For arrays and tuples this value is documented by the macro BOOST_PP_LIMIT_TUPLE while for variadic data this value is documented by the macro BOOST_PP_LIMIT_VARIADIC. In both these cases the current fixed maximum value is 64. For seqs the value is documented by the macro BOOST_PP_LIMIT_SEQ, whose current fixed value is 256. For lists there has never been a limitation macro but in actual fact the limitation on the maximum number of list elements is defined by the maximum number given above for BOOST_PP_LIMIT_MAG. So a list, like a seq, has a default maximum of 256 elements. The reason that the maximum number of elements for arrays, tuples, and variadic data is 64 while for seqs and lists it is the much larger value of 256 is because the implementation of the former constructs often involves passing the likewise number as parameters to internal macros and a number of compilers have placed limitations on the number of parameters a macro can have; so the much safer 64 was chosen so as to be able to work with these compilers.

The looping constructs consist of the control structure BOOST_PP_WHILE and the repetition constructs BOOST_PP_FOR and BOOST_PP_REPEAT. Each of these have a limitation macro, all currently set to 256. This means that the loop can run at maximum 256 times. The corresponding limitation macros are BOOST_PP_LIMIT_WHILE, BOOST_PP_LIMIT_FOR, and BOOST_PP_LIMIT_REPEAT.

Iteration constructs consist of local iteration and file iteration constructs.The specifics are more fully discussed in their own topics. The corresponding macro is BOOST_PP_LIMIT_ITERATION whose value is 256.

Limitation dependencies

A number of the limitations are actually dependent on other limitations. These are:

Changing the limitations

Limitations can be changed by the end-user defining a limitation macro prior to including preprocessor library header files. This can be done either in a source file or as part of a compiler command line.

Before specifying which limitations can be changed a number of items need to be emphasized:

All the limitations whose default value is 256 can be changed to either 512 or 1024, except for BOOST_PP_LIMIT_WHILE which can not be changed.
The two limitations whose default value is 64 can be changed to 128 or 256.

For the individual limitations:

Other considerations

There is no necessity to change any limitation if you are happy with its default value. However if you decide to increase a limitation you should be aware that doing so can affect both the amount of memory used by the preprocessor and the speed in which preprocessing is accomplished. In the first case some preprocessors have been known to run out of memory if a limitation value is increased, even though this should really never happen in modern computer systems. In the latter case preprocessing might become so slow that trying to use the library with some increased limitation, and have your preprocessing finish in some reasonable amount of time, becomes impossible. The latter can occur when you are using lists or seqs and while cycling through a large number of elements you are also doing time consuming operations on each element value which generates further macro looping. Nearly all arithmetic and comparison operations involve further macro looping.

For lists and seqs there is functionality which uses the BOOST_PP_FOR macro. If the maximum size for lists and seqs is increased, by using BOOST_PP_LIMIT_MAG macro, you may also need to define BOOST_PP_LIMIT_FOR to increase the maximum number of 'for' loops in order to cycle through all lists or seqs using this functionality.

The BOOST_PP_WHILE construct is used internally for looping by nearly all arithmetic and comparison macros as well as internally by a number of other macros in the library which loop through elements. While the construct can be called from within an already running BOOST_PP_WHILE loop, it is possible to run out of BOOST_PP_WHILE loops when this happens. In order to solve this problem you can specify a BOOST_PP_LIMIT_MAG which is larger than the maximum number you will ordinarily use. This will give you twice the number of BOOST_PP_WHILE loops and will keep you from running out of loops if you have to do arithmetic computations while cycling through lists and seqs and the number of lists and/or seqs you cycle through is anywhere near your original  maximum.

Testing limitations

The default testing of the preprocessor library, using Boost Build's b2 command in the preprocessor test directory, only tests the default limitations. If you want to test, within the test directory, any of the non-default limitations which may be set, you can do so by invoking the b2 command with any of these explicit targets:

If you invoke b2 with the last preprocessor_limits target your testing may take awhile for your compiler. When you choose one of the above targets, as in 'b2 preprocessor_limits', the default tests are not run. If you want the default tests run, along with any of the targets such as preprocessor_limits, your command would be 'b2 . preprocessor_limits', where the '.' notation means all the non-explicit ( aka default ) targets.
See Also

Copyright Edward Diener 2020

Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at www.boost.org/LICENSE_1_0.txt)