Boost C++ Libraries

...one of the most highly regarded and expertly designed C++ library projects in the world. Herb Sutter and Andrei Alexandrescu, C++ Coding Standards

This is the documentation for a snapshot of the develop branch, built from commit 0f79ae966a.
PrevUpHomeNext

Macros for working with data types

Specific macros for working with data types
Emptiness
Macro constraints
Identifiers
Numbers
Types
VMD and Boost PP data types
Identifying macros and BOOST_VMD_IS_EMPTY
Summing up the specific VMD macros
Generic macros for working with data types
Parsing sequences
Converting sequences
Accessing a sequence element
Getting the type of data
Testing for equality and inequality
Summing up the generic macros

VMD macros for working with data types which VMD understands can be divided into specific and generic macros.

The specific macros ask whether some input data is a particular data type. The generic macros work with input data as any data type while also allowing the programmer to separately query the type of data.

Both specific and generic macros have their place and the macro programmer can decide which to use for any given situation.

VMD has a number of specific macros for parsing data types. Each of these macros asks if some input is a particular VMD data type.

Passing empty arguments

It is possible to pass an empty argument to a macro. The official terminology for this in the C++ standard is an argument "consisting of no preprocessing tokens". Passing an empty argument to a macro is officially part of the C++11 standard, but a number of compilers also support passing an empty argument to a macro in C++98/C++03 mode, when not in strict compliance to the standard.

Let us consider a number of cases without worrying too much what the macro output represents.

Consider these two function-like macros:

#define SMACRO() someoutput
#define EMACRO(x) otheroutput x

The first macro takes no parameters so invoking it must always be done by

SMACRO()

and passing any arguments to it would be invalid.

The second macro takes a single parameter. it can be evoked as

EMACRO(somedata)

but it also can be invoked as

EMACRO()

In the second invocation of EMACRO we are passing an empty argument to the macro. Similarly for any macro having 1 or more parameters, an empty argument can be validly passed for any of the parameters, as in

#define MMACRO(x,y,z) x y z

MMACRO(1,,2)

An empty argument is an argument even if we are passing nothing.

Because an empty argument can be passed for a given parameter of a macro does not mean one should do so. Any given macro will specify what each argument to a macro should represent, and it is has normally been very rare to encounter a macro which specifies that an empty argument can logically be passed for a given argument. But from the perspective of standard C++ it is perfectly valid to pass an empty argument for a macro parameter.

The notion of passing empty arguments can be extended to passing empty data which "consists of no preprocessing tokens" in slightly more complicated situations. It is possible to pass empty data as an argument to a variadic macro in the form of variadic macro data, as in

#define VMACRO(x,...) x __VA_ARGS__

invoked as

VMACRO(somedata,)

Here one passes empty data as the variadic macro data and it is perfectly valid C++. Please notice that this different from

VMACRO(somedata)

which is not valid C++, prior to C++20, since something must be passed for the variadic argument. In C++20 the above invocation is valid and is exactly the same as in our previous example of 'VMACRO(somedata,)' where one passes empty data as the variadic macro data. Similarly one could invoke the macro as

VMACRO(somedata,vdata1,,vdata3)

where one is passing variadic macro data but an element in the variadic macro data is empty.

Furthermore if we are invoking a macro which expects a Boost PP data type, such as a tuple, we could also validly pass empty data for all or part of the data in a tuple, as in

#define TMACRO(x,atuple) x atuple

TMACRO(somedata,())

In this case we are passing a 1 element tuple where the single element itself is empty.

or

TMACRO(somedata,(telem1,,telem2,teleem3))

In this case we are passing a 4 element tuple where the second element is empty.

Again either invocation is valid C++ but it is not necessarily what the designed of the macro has desired, even if in both cases the macro designer has specified that the second parameter must be a tuple for the macro to work properly.

Returning emptiness

Similar to passing empty arguments in various ways to a macro, the data which a macro returns ( or 'generates' may be a better term ) could be empty, in various ways. Again I am not necessarily promoting this idea as a common occurrence of macro design but merely pointing it out as valid C++ preprocessing.

#define RMACRO(x,y,z)

RMACRO(data1,data2,data3)

It is perfectly valid C++ to return "nothing" from a macro invocation. In fact a number of macros in Boost PP do that based on the preprocessor metaprogramming logic of the macro, and are documented as such.

Similarly one could return nothing as part or all of a Boost PP data type or even as part of variadic macro data.

#define TRETMACRO(x,y,z) ()
#define TRETMACRO1(x,y,z) (x,,y,,z)
#define VRETMACRO(x,y,z) x,,y,,z

Here again we are returning something but in terms of a Boost PP tuple or in terms of variadic data, we have elements which are empty.

Emptiness in preprocessor metaprogramming

In the examples given above where "emptiness" in one form of another is passed as arguments to a macro or returned from a macro, the examples I have given were created as simplified as possible to illustrate my points. In actual preprocessor metaprogramming, using Boost PP, where complicated logic is used to generate macro output based on the arguments to a macro, it might be useful to allow and work with empty data if one were able to test for the fact that data was indeed empty.

Testing for empty data

Currently Boost PP has an undocumented macro for testing whether a parameter is empty of not. The macro is called BOOST_PP_IS_EMPTY. The macro is by its nature flawed, since there is no generalized way of determining whether or not a parameter is empty using the C++ preprocessor prior to C++20. But the macro will work correctly with almost all non-empty input or if the input is actually empty.

Paul Mensonides, the developer of Boost PP, wrote the code for the BOOST_PP_IS_EMPTY macro, for determining whether or not a parameter is empty or not using variadic macros, which he originally published on the Internet in response to a discussion about emptiness. I have adapted his code for the Boost PP library and for VMD developed my own very slightly different code based on his original example.

The macro is called BOOST_VMD_IS_EMPTY and will return 1 if its input is empty or 0 if its input is not empty. The macro is a variadic macro which make take any input [1].

Macro Flaw with a standard C++ compiler

The one situation prior to C++20 where the macro does not work properly is if its input resolves to a function-like macro name or a sequence of preprocessor tokens ending with a function-like macro name and the function-like macro takes two or more parameters.

Here is a simple example:

#include <boost/vmd/is_empty.hpp>

#define FMACRO(x,y) any_output

BOOST_VMD_IS_EMPTY(FMACRO)
BOOST_VMD_IS_EMPTY(some_input FMACRO)

In the first case the name of a function-like macro is being passed to BOOST_VMD_IS_EMPTY while in the second case a sequence of preprocessing tokens is being passed to BOOST_VMD_IS_EMPTY ending with the name of a function-like macro. The function-like macro also has two ( or more ) parameters. In both the cases above a compiler error will result from the use of BOOST_VMD_IS_EMPTY.

Please note that these two problematical cases are not the same as passing an invocation of a function-like macro name to BOOST_VMD_IS_EMPTY, as in

#include <boost/vmd/is_empty.hpp>

BOOST_VMD_IS_EMPTY(FMACRO(arg1,arg2))
BOOST_VMD_IS_EMPTY(someinput FMACRO(arg1,arg2))

which always works correctly, unless of course a particular function-like macro invocation resolves to either of our two previous situations.

Another situation where the macro may not work properly is if the previously mentioned function-like macro takes a single parameter but creates an error when the argument passed is empty. An example of this would be:

#define FMACRO(x) BOOST_PP_CAT(+,x C);

When nothing is passed to FMACRO undefined behavior will occur since attempting to concatenate '+' to ' C' is UB in C++ preprocessor terms.

So for a standard conforming compiler, prior to C++20, we have essentially two corner cases where the BOOST_VMD_IS_EMPTY does not work and, when it does not work it, produces a compiler error rather than an incorrect result. Essentially what is desired for maximum safety is that we never pass input ending with the name of a function-like macro name when testing for emptiness.

Macro Flaw with Visual C++

The VC++ default preprocessor is not a standard C++ conforming preprocessor in at least two relevant situations to our discussion of emptiness. These situations combine to create a single corner case which causes the BOOST_VMD_IS_EMPTY macro to not work properly using VC++ with its default preprocessor when the input resolves to a function-like macro name.

The first situation, related to our discussion of emptiness, where the VC++ default preprocessor is not a standard C++ conforming preprocessor is that if a macro taking 'n' number of parameters is invoked with 0 to 'n-1' parameters, the compiler does not give an error, but only a warning.

#define FMACRO(x,y) x + y

FMACRO(1)

should give a compiler error, as it does when using a C++ standard-conforming compiler, but when invoked using VC++ with its default preprocessor it only gives a warning and VC++ continues macro substitution with 'y' as a placemarker preprocessing token. This non-standard conforming action actually eliminates the case where BOOST_VMD_IS_EMPTY does not work properly with a standard C++ conforming compiler. But of course it has the potential of producing incorrect output in other macro processing situations unrelated to the BOOST_VMD_IS_EMPTY invocation, where a compiler error should occur.

A second general situation, related to our discussion of emptiness, where the VC++ default preprocessor is not a standard C++ conforming preprocessor is that the expansion of a macro works incorrectly when the expanded macro is a function-like macro name followed by a function-like macro invocation, in which case the macro re-expansion is erroneously done more than once. This latter case can be seen by this example:

#define FMACRO1(parameter) FMACRO3 parameter()
#define FMACRO2() ()
#define FMACRO3() 1

FMACRO1(FMACRO2)

should expand to:

FMACRO3()

but in VC++ with its default preprocessor it expands to:

1

where after initially expanding the macro to:

FMACRO3 FMACRO2()

VC++ erroneously rescans the sequence of preprocessing tokens more than once rather than rescan just one more time for more macro names.

What these two particular preprocessor flaws in the VC++ compiler with its default preprocessor mean is that although BOOST_VMD_IS_EMPTY does not fail with a compiler error in the same case as with a standard C++ conforming compiler given previously, it fails by giving the wrong result in another situation.

The failing situation is:

when the input to BOOST_VMD_IS_EMPTY resolves to only a function-like macro name, and the function-like macro, when passed a single empty argument, expands to a Boost PP tuple, BOOST_VMD_IS_EMPTY will erroneously return 1 when using the Visual C++ compiler rather than either give a preprocessing error or return 0.

Here is an example of the failure:

#include <boost/vmd/is_empty.hpp>

#define FMACRO4() ( any_number_of_tuple_elements )
#define FMACRO5(param) ( any_number_of_tuple_elements )
#define FMACRO6(param1,param2) ( any_number_of_tuple_elements )

BOOST_VMD_IS_EMPTY(FMACRO4) // erroneously returns 1, instead of 0
BOOST_VMD_IS_EMPTY(FMACRO5) // erroneously returns 1, instead of 0
BOOST_VMD_IS_EMPTY(FMACRO6) // erroneously returns 1, instead of generating a preprocessing error

As with a standard C++ conforming compiler prior to C++20, we have a rare corner case where the BOOST_VMD_IS_EMPTY will not work properly, but unfortunately in this very similar but even rarer corner case with VC++ with its default preprocessor, we will silently get an incorrect result rather than a compiler error.

I want to reiterate that for all compilers prior to C++20 there is no perfect solution in C++ to the detection of emptiness even for a C++ compiler whose preprocessor is completely conformant, which VC++ with its default preprocessor obviously is not.

Testing emptiness in C++20 mode

A few compilers can currently operate in C++20 mode, by which I mean that you can pass a compiler flag when compiling with such a compiler which enforces the upcoming C++20 standard. One of the features of the C++20 standard is the addition of a preprocessor construct called __VA_OPT__. Because of the specification of how the __VA_OPT__ construct works in C++20, it is now possible to have the BOOST_VMD_IS_EMPTY macro work perfectly to test for emptiness without any of the flaws that exist in the macro for levels of the C++ standard before C++20. But the macro will only do a 100% reliable test for emptiness when the compiler is compiling in C++20 mode. For all levels of the C++ standard before C++20, such as C++98, C++03, C++11, C++14, and C++17, the testing for emptiness has the corner cases which prevent it from wroking perfectly which have already been discussed.

Furthermore in C++20 mode it is possible that a compiler still does not yet support the __VA_OPT__ construct, even though it is part of the C++20 standard. Luckily it is possible to test whether or not a compiler supports the __VA_OPT__ construct in C++20 mode, and the macro implementation of BOOST_VMD_IS_EMPTY does that before using the construct to provide a perfectly reliable implementation for testing emptiness.

The result of all this is that when a compiler is compiling source using the C++20 standard, and supports the C++20 __VA_OPT__ preprocessor construct, the implementation provides a completely reliable way of testing for emptiness using the BOOST_VMD_IS_EMPTY macro. Otherwise the BOOST_VMD_IS_EMPTY macro has the corner cases previously discussed which make the macro less than 100% reliable in testing for emptiness. The good news of course is that more compilers will be implementaing the C++20 standard and more C++ programmers will be using the C++20 standard to compile their code.

The programmer may know whether the compiler is being used in C++20 mode from the command line parameters he passes to the compiler, and the programmer may know whether the compiler in C++20 mode supports the __VA_OPT__ construct of C++20 from the compiler's documentation. But from the preprocessor programming perspective it would be good to find out using a macro whether or not C++20 mode with the __VA_OPT__ construct is being used so that the BOOST_VMD_IS_EMPTY macro can be considered completely reliable in testing for emptiness. Such a macro does already exist in the Boost Preprocessor library, and it is called BOOST_PP_VARIADIC_HAS_OPT. You can read the documentation for this macro in the Boost Preprocessor library documentation, but I will give a quick rundown of how this works here. The macro is a function-like macro taking no parameters and returns 1 if the compiler is in C++20 mode and __VA_OPT__ is supported, otherwise returns 0. The header file needed to invoke the macro as BOOST_PP_VARIADIC_HAS_OPT() is included as:

#include <boost/preprocessor/variadic/has_opt.hpp>

The programmer does not have to be compiling in C++20 mode to invoke the BOOST_PP_VARIADIC_HAS_OPT macro. When the programmer is not in C++20 mode invoking the macro always returns 0. When the programmer is in C++20 mode invoking the macro returns 1 when the __VA_OPT__ construct is supported and returns 0 when the __VA_OPT__ construct is not supported. It does this latter step through clever preprocessor programming.

Macro Flaw conclusion

With all of the above mentioned, the cases where BOOST_VMD_IS_EMPTY will work incorrectly are very small, even with the erroneous VC++ default preprocessor, and I consider the macro worthwhile to use since it works correctly with the vast majority of possible preprocessor input, and always works correctly in C++20 mode with __VA_OPT__ preprocessor support.

The case where it will not work, with both a C++ standard conforming preprocessor or with Visual C++, occurs when the name of a function-like macro is part of the input to BOOST_VMD_IS_EMPTY. Obviously the macro should be used by the preprocessor metaprogrammer when the possible input to it is constrained to eliminate the erroneous case.

Since emptiness can correctly be tested for in nearly every situation, the BOOST_VMD_IS_EMPTY macro can be used internally when the preprocessor metaprogrammer wants to return data from a macro and all or part of that data could be empty.

Therefore I believe the BOOST_VMD_IS_EMPTY macro is quite useful, despite the corner case flaws which makes it imperfect. Consequently I believe that the preprocessor metaprogrammer can use the concept of empty preprocessor data in the design of his own macros.

Using the macro

The macro BOOST_VMD_IS_EMPTY is used internally throughout VMD and macro programmers may find this macro useful in their own programming efforts despite the slight flaw in the way that it works in pre C++20 mode.

You can use the general header file:

#include <boost/vmd/vmd.hpp>

or you can use the individual header file:

#include <boost/vmd/is_empty.hpp>

for the BOOST_VMD_IS_EMPTY macro.

When discussing the BOOST_VMD_IS_EMPTY macro I mentioned constraining input to the macro. Now I will discuss what this means in terms of preprocessor metaprogramming and input to macros in general.

Constrained input

When a programmer designs any kinds of callables in C++ ( functions, member functions etc. ), he specifies what the types of input and the return value are. The C++ compiler enforces this specification at compile time. Similarly at run-time a callable may check that its input falls within certain documented and defined boundaries and react accordingly if it does not. This is all part of the constraints for any callable in C++ and should be documented by any good programmer.

The C++ preprocessor is much "dumber" than the C++ compiler and even with the preprocessor metaprogramming constructs which Paul Mensonides has created in Boost PP there is far less the preprocessor metaprogrammer can do at preprocessing time to constrain argument input to a macro than a programmer can do at compile-time and/or at run-time to constrain argument input to a C++ callable. Nevertheless it is perfectly valid to document what a macro expects as its argument input and, if a programmer does not follow the constraint, the macro will fail to work properly. In the ideal case in preprocessor metaprogramming the macro could tell whether or not the constraint was met and could issue some sort of intelligible preprocessing error when this occurred, but even within the reality of preprocessor metaprogramming with Boost PP this is not always possible to do. Nevertheless if the user of a macro does not follow the constraints for a macro parameter, as specified in the documentation of a particular macro being invoked, any error which occurs is the fault of that user. I realize that this may go against the strongly held concept that programming errors must always be met with some sort of compile-time or run-time occurrence which allows the programmer to correct the error, rather than a silent failure which masks the error. Because the preprocessor is "dumber" and cannot provide this occurrence in all cases the error could unfortunately be masked, despite the fact that the documentation specifies the correct input constraint(s). In the case of the already discussed macro BOOST_VMD_IS_EMPTY, this masking of the error could only occur with a preprocessor ( Visual C++ ) which is not C++ standard conformant.

The Boost PP library does have a way of generating a preprocessing error, without generating preprocessor output, but once again this way does not work with the non-conformant preprocessor of Visual C++. The means to do so using Boost PP is through the BOOST_PP_ASSERT macro. As will be seen and discussed later VMD has an equivalent macro which will work with Visual C++ by producing incorrect C++ output rather than a preprocessing error, but even this is not a complete solution since the incorrect C++ output produced could be hidden.

Even the effort to produce a preprocessing error, or incorrect output inducing a compile-time error, does not solve the problem of constrained input for preprocessor metaprogramming. Often it is impossible to determine if the input meets the constraints which the preprocessor metaprogrammer places on it and documents. Certain preprocessing tokens cannot be checked reliably for particular values, or a range of values, without the checking mechanism itself creating a preprocessing error or undefined behavior.

This does not mean that one should give up attempting to check macro input constraints. If it can be done I see the value of such checks and a number of VMD macros, discussed later, are designed as preprocessing input constraint checking macros. But the most important thing when dealing with macro input constraints is that they should be carefully documented, and that the programmer should know that if the constraints are not met either preprocessing errors or incorrect macro results could be the results.

The VMD library, in order to present more preprocessor programming functionality and flexibility, allows that erroneous results could occur if certain input constraints are not met, whether the erroneous results are preprocessing errors or incorrect output from a VMD macro. At the same time the VMD does everything that the preprocessor is capable of doing to check the input constraints, and carefully documents for each macro in the library what the input for each could be in order to avoid erroneous output.

Documented macro input constraints are just as valid in the preprocessor as compile-time/run-time constraints are valid in C++, even if the detection of such constraints and/or the handling of constraints that are not met are far more difficult, if not impossible, in the preprocessor than in the compile-time/run-time processing of C++.

The VMD library uses constraints for most of it macros and the documentation for those macros mentions the constraints that apply in order to use the macro.

An identifier in VMD is either of two lower-level preprocessor possibilities:

  • a preprocessing token 'identifier', which is essentially a sequence of alphanumeric characters and the underscore character with the first character not being a numeric character.
  • a preprocessing token 'pp-number' that is an integral literal token.

Here are some examples:

SOME_NAME
_SOME_NAME
SOME_123_NAME
some_123_name
sOMe_123_NAmE
2367
43e11
0
22
654792
0x1256
Problem testing any identifier

One of the difficulties with identifiers in preprocessor metaprogramming is safely testing for a particular one. VMD has a means of doing this within a particular constraint for the characters that serve as the input to test.

The constraint is that the input must either be empty or that the beginning input character, ignoring any whitespace, passed as the input to test must be either:

  • an identifier character, ie. an alphanumeric or an underscore
  • the left parenthesis of a tuple

and if the first character is not the left parenthesis of a tuple the remaining characters must be alphanumeric or an underscore until a space character or end of input occurs

If this is not the case the behavior is undefined, and most likely a preprocessing error will occur.

Given the input:

's_anything' : can be tested
'S_anything' : can be tested
's_anYthiNg' : can be tested
'_anything' : can be tested
'_Anything' : can be tested
'_anytHIng' : can be tested
'24' : can be tested
'245e2' : can be tested
'(anything)' : can be tested, tuple
'(anything) anything' : can be tested, tuple and further input
'anything anything' : can be tested, identifier followed by space character

'%_anything' : undefined behavior and most likely a preprocessing error due to the constraint
'(_anything' : undefined behavior and most likely a preprocessing error due to the constraint, since a single '(' does not form a tuple
'44.3' : undefined behavior and most likely a preprocessing error due to the constraint since '.' is not alphanumeric
Identifying an identifier

There are three levels of testing identifiers in VMD.

  1. You can test for any general identifier which matches the identifier syntax
  2. You can test for a specific identifier
  3. You can test that a specific identifier is equal to another specific identifier
Testing for any general identifier

The macro for testing for any identifier is called BOOST_VMD_IS_GENERAL_IDENTIFIER.

You invoke it with input which may be an identifier according to the rules given above, and it returns 1 if the input has the form of an identifier and returns 0 if the input does not have the form of an identifier. Be aware that input which does not follow the above constraints will probably lead to undefined behavior. Also the input to BOOST_VMD_IS_GENERAL_IDENTIFIER is meant to be a single identifier and not a VMD sequence of preprocessor tokens, else the return may be incorrect. Sample code:

#include <boost/vmd/is_generaL_identifier.hpp>

BOOST_VMD_IS_GENERAL_IDENTIFIER(some_input) // returns 1 if 'some_input' is in the form of an identifier, else returns 0

This form of an identifier which follows the identifier syntax is called a "general" identifier. General identifiers can be tested successfully using the VMD specific macros but can not be tested successfully using the VMD generic macros. For the latter you need specific identifiers.

Testing for a specific identifier

Although testing whether macro input is in the form of any general identifier given above may be useful sometimes it is valuable to find out whether macro input is one of a number of specific identifiers in which you are particularly interested. In order to do this you first create a user-defined macro which 'registers' specific identifiers. The user-defined macro is an object-like macro whose form is:

#define BOOST_VMD_REGISTER_identifier (identifier)

where 'identifier' is a specific identifier we wish to identify. This is called in VMD a registration macro.

The macro for testing for a specific identifier is called BOOST_VMD_IS_IDENTIFIER.

#include <boost/vmd/is_identifier.hpp>

#define BOOST_VMD_REGISTER_yellow (yellow)
#define BOOST_VMD_REGISTER_green  (green)
#define BOOST_VMD_REGISTER_blue   (blue)

BOOST_VMD_IS_IDENTIFIER(some_input) // returns 1 if 'some_input' is 'yellow','green', or 'blue'
BOOST_VMD_IS_IDENTIFIER(some_input) // returns 0 if 'some_input' is 'purple'

It is recommended that registration macros be created in a header file which can be included before the end-user uses the identifier macros of VMD, but of course you can create registration macros directly in a source file if you wish. If a particular registration macro occurs more than once it is not a preprocessing error, so duplicating a registration macro will not lead to any problems since each registration macro of the same name will have the exact same object-like macro expansion. Within a given translation unit it could potentially happen that registration macros have been included by header files which a particular end-user of VMD has not created. This should also not lead to particular problems since registration is a process for adding specific identifiers for any particular translation unit.

As we shall see further in the documentation only specific identifiers can be tested for when we work with VMD generic macros and to parse VMD sequences, so having an identifier which is not a specific identifier but only a general identifier has a limited use in VMD.

Testing for identifier equality

Although registering an identifier allows VMD to recognize the string of characters as a specific VMD identifier when using the BOOST_VMD_IS_IDENTIFIER, the ability to detect whether or not a specific identifier is exactly the same or not than another specific identifier needs the end-user to define another macro:

#define BOOST_VMD_DETECT_identifier_identifier

where 'identifier' is a specific identifier we wish to detect. This object-like macro must expand to no output.

Like the registration macro, multiple detection macros of the same identifier in a translation unit does not cause a compiler problem since the exact same object-like macro occurs.

The term for creating this macro is that we have potentially 'pre-detected' the identifier and I will use the term pre-detected as the process of creating the BOOST_VMD_DETECT macro and the term 'detect' as the ability to find out if a specific identifier is the same as another specific identifier or not.

The ability to detect that a VMD identifier is a particular specific identifier is used in VMD macros when preprocessor data is compared for equality/inequality using the VMD macros BOOST_VMD_EQUAL and BOOST_VMD_NOT_EQUAL as well as when we want to match an identifier against a specific set of other registered identifiers using the macro BOOST_VMD_IS_IDENTIFIER with more than the single variadic argument given above.

These situations will be explained later in the documentation in other topics discussing VMD macro functionality. If the programmer never uses the functionality which these situations encompass, ie. in which we match a specific identifier against another to see if they are equal or not, there is no need to use pre-detection for a registered identifier. However once you provide a registration macro in order to test for a specific identifier in preprocessor input also providing a pre-detection macro for that identifier costs very little in terms of macro notation and nothing in terms of preprocessor time. So the end-user might want to add a pre-detection macro for a specific identifier to the same place or header file where the corresponding registration macro is created, even if the end-user initially does not think that he might want to test that identifier for equality with another identifier.

To sum up the three levels of identifier functionality:

  • General identifiers can be used with the VMD specific macros and will return the correct result, but can not be used with VMD generic macros.
  • Specific identifiers, ie. registered identifiers, can be used with VMD specific macros and nearly all VMD generic macros.
  • Pre-detected specific identifiers can be used with all VMD macros, whether specific or generic.
Parsing identifier constraints and undefined behavior

The reason that the identifier constraints mentioned above exist is that the technique for parsing identifiers, once it is determined that the input being parsed is not empty or does not begin with a set of parentheses, uses preprocessor concatenation in its parsing. This technique involves the preprocessor '##' operator to concatenate input, and examine the results of that concatenation. When preprocessor concatenation is used the result of the concatenation must be a valid preprocessing token, else the behavior of the preprocessor is undefined. In C++ 'undefined behavior' in general means that anything can happen. In practical use when preprocessor concatenation does not produce a valid preprocessing token, a compiler is most likely to generate a preprocessing error. If the compiler chooses not to issue a preprocessing error when concatenation does not produce a valid preprocessor token the outcome of the VMD macros will always be correct and parsing an identifier will fail. But because the outcome is undefined behavior there is no absolute way that the programmer can determine what the outcome will be when preprocessor concatenation is used and the input being parsed contains does not meet the constraints for parsing an identifier mentioned at the beginning of this topic.

In this documentation I will be using the abbreviation 'UB' as the shortened form of 'undefined behavior' to denote the particular occurrence where VMD attempts to parse preprocessor input using preprocessor concatenation and undefined behavior will occur.

Usage

To use the BOOST_VMD_IS_GENERAL_IDENTIFIER macro and/or the BOOST_VMD_IS_IDENTIFIER macro you can either include the general header:

#include <boost/vmd/vmd.hpp>

or include the specific headers:

#include <boost/vmd/is_general_identifier.hpp>
#include <boost/vmd/is_identifier.hpp>

A number in VMD is a preprocessing 'pp-number', limited to a Boost PP number. This is an integral literal between 0 and BOOST_PP_LIMIT_MAG. The form of the number does not contain leading zeros. Acceptable as numbers are:

0
127
33
254
18

but not:

033
06
009
00
Problem testing any number

As can be seen from the explanation of an identifier, a number is merely a small subset of all possible identifiers, for which VMD internally provides registration macros and pre-detection macros for its use. Therefore the particular constraint on the input to test is exactly the same as for identifiers.

The constraint is that the beginning input character, ignoring any whitespace, passed as the input to test must be either:

  • an identifier character, ie. an alphanumeric or an underscore
  • the left parenthesis of a tuple

and if the first character is not the left parenthesis of a tuple the remaining characters must be alphanumeric or an underscore until a space character or end of input occurs.

If this is not the case the behavior is undefined, and most likely a preprocessing error will occur.

Testing for a number macro

The macro used to test for any number in VMD is called BOOST_VMD_IS_NUMBER. The macro takes a single variadic parameter, the input to test against.

The macro returns 1 if the parameter is a Boost PP number, otherwise the macro returns 0.

The Boost PP library has a great amount of functionality for working with numbers, so once you use VMD to parse/test for a number you can use Boost PP to work with that number in various ways. The VMD library makes no attempt to duplicate the functionality of numbers that in the Boost PP library.

Any number is also an identifier, which has been registered and pre-detected, so you can also use the VMD functionality which works with identifiers to work with a number as an identifier if you like.

Example

Let us look at an example of how to use BOOST_VMD_IS_NUMBER.

#include <boost/vmd/is_number.hpp>

BOOST_VMD_IS_NUMBER(input)

returns:

if input = 0, 1
if input = 44, 1
if input = SQUARE, 0
if input = 44 DATA, 0 since there are tokens after the number
if input = 044, 0 since no leading zeros are allowed for our Boost PP numbers
if input = 256, 1
if input = 1025, 0 since it falls outside the Boost PP number range of 0-BOOST_PP_LIMIT_MAG
if input = %44, does not meet the constraint therefore undefined behavior
if input = 44.0, does not meet the constraint therefore undefined behavior
if input = ( 44 ), 0 since the macro begins with a tuple and this can be tested for
Usage

To use the BOOST_VMD_IS_NUMBER macro either include the general header:

#include <boost/vmd/vmd.hpp>

or include the specific header:

#include <boost/vmd/is_number.hpp>

A subset of identifiers is VMD types, called a 'v-type'. These are identifiers which represent all of the preprocessor data types which VMD can parse. This subset of identifiers is automatically registered and pre-detected by VMD. Each identifier type begins with the unique prefix 'BOOST_VMD_TYPE_'.

The actual types are:

  • BOOST_VMD_TYPE_EMPTY, represents emptiness, ie. "empty data"
  • BOOST_VMD_TYPE_ARRAY, a Boost PP array
  • BOOST_VMD_TYPE_LIST, a Boost PP list
  • BOOST_VMD_TYPE_SEQ, a Boost PP seq
  • BOOST_VMD_TYPE_TUPLE, a Boost PP tuple
  • BOOST_VMD_TYPE_IDENTIFIER, identifier
  • BOOST_BMD_TYPE_NUMBER, a number
  • BOOST_VMD_TYPE_TYPE, a type itself
  • BOOST_VMD_TYPE_SEQUENCE, a sequence
  • BOOST_VMD_TYPE_UNKNOWN, an unknown type

Since a v-type is itself an identifier the particular constraint on the input to test is exactly the same as for identifiers.

The constraint is that the beginning input character, ignoring any whitespace, passed as the input to test must be either:

  • an identifier character, ie. an alphanumeric or an underscore
  • the left parenthesis of a tuple

and if the first character is not the left parenthesis of a tuple the remaining characters must be alphanumeric or an underscore until a space character or end of input occurs.

If this is not the case the behavior is undefined, and most likely a preprocessing error will occur.

The macro used to test for a particular type in VMD is called BOOST_VMD_IS_TYPE. The macro takes a single variadic parameter, the input to test against.

The macro returns 1 if the parameter is a v-type, otherwise the macro returns 0.

A v-type is also an identifier, which has been registered and pre-detected, so you can also use the VMD functionality which works with identifiers to work with a v-type as an identifier if you like.

Example

Let us look at an example of how to use BOOST_VMD_IS_TYPE.

#include <boost/vmd/is_type.hpp>

BOOST_VMD_IS_TYPE(input)

returns:

if input = BOOST_VMD_TYPE_SEQ, 1
if input = BOOST_VMD_TYPE_NUMBER, 1
if input = SQUARE, 0
if input = BOOST_VMD_TYPE_IDENTIFIER DATA, 0 since there are tokens after the type
if input = %44, does not meet the constraint therefore undefined behavior
if input = ( BOOST_VMD_TYPE_EMPTY ), 0 since the macro begins with a tuple and this can be tested for
Usage

To use the BOOST_VMD_IS_TYPE macro either include the general header:

#include <boost/vmd/vmd.hpp>

or include the specific header:

#include <boost/vmd/is_type.hpp>

VMD is able to determine whether or not preprocessing input is a given Boost PP data type. The VMD macros to do this are:

  • BOOST_VMD_IS_ARRAY for an array
  • BOOST_VMD_IS_LIST for a list
  • BOOST_VMD_IS_SEQ for a seq
  • BOOST_VMD_IS_TUPLE for a tuple

Each of these macros take a single variadic parameter as input and return 1 if the parameter is the appropriate data type and 0 if it is not.

Syntax anomalies

Both an array and a non-empty list are also a tuple. So if one has:

#define ANARRAY (3,(a,b,c))
#define ALIST (a,(b,(c,BOOST_PP_NIL)))
#define ATUPLE (a,b,c)
#define ASEQ (a)(b)(c)

then

#include <boost/vmd/is_tuple.hpp>

BOOST_VMD_IS_TUPLE(ANARRAY) returns 1
BOOST_VMD_IS_TUPLE(ALIST) returns 1
BOOST_VMD_IS_TUPLE(ATUPLE) returns 1
BOOST_VMD_IS_TUPLE(ASEQ) returns 0

A list whose first element is the number 2 and whose second element is not the end-of-list marker BOOST_PP_NIL is also an array. So if one has:

#define ALIST (2,(3,BOOST_PP_NIL))
#define ALIST2 (2,(3,(4,BOOST_PP_NIL)))
#define ALIST3 (2,BOOST_PP_NIL)

#include <boost/vmd/is_array.hpp>
#include <boost/vmd/is_list.hpp>

BOOST_VMD_IS_LIST(ALIST) returns 1
BOOST_VMD_IS_LIST(ALIST2) returns 1
BOOST_VMD_IS_LIST(ALIST3) returns 1
BOOST_VMD_IS_ARRAY(ALIST) returns 1
BOOST_VMD_IS_ARRAY(ALIST2) returns 1
BOOST_VMD_IS_ARRAY(ALIST3) returns 0

A single element tuple is also a one element seq. So if one has:

#define ASE_TUPLE (a)

then

#include <boost/vmd/is_seq.hpp>
#include <boost/vmd/is_tuple.hpp>

BOOST_VMD_IS_TUPLE(ASE_TUPLE) returns 1
BOOST_VMD_IS_SEQ(ASE_TUPLE) returns 1

However if our data consists of more than one consecutive tuple of a single element the data is always a seq:

#include <boost/vmd/is_seq.hpp>
#include <boost/vmd/is_tuple.hpp>

#define ST_DATA (somedata)(some_other_data)

BOOST_VMD_IS_SEQ(ST_DATA) will return 1
BOOST_VMD_IS_TUPLE(ST_DATA) will return 0
Problem when testing an array

The form of an array is a two element tuple, where the first element is a number and the second element is a tuple. The number specifies the size of the tuple. Since when using variadic macros it is never necessary to specify the size of a tuple, an array is largely obsolete. However VMD still supports it.

The problem when testing for an array is that if the first element does not obey the constraint on testing for a number, you will get UB.

#include <boost/vmd/is_array.hpp>
#include <boost/vmd/is_tuple.hpp>

#define A_TUPLE (&anything,(1,2))

BOOST_VMD_IS_ARRAY(A_TUPLE) will give UB due to the constraint
BOOST_VMD_IS_TUPLE(A_TUPLE) will return 1

When VMD attempts to parse for an array, as it does when the BOOST_VMD_IS_ARRAY is used, if first looks to see if the syntax represents a tuple with two elements. Next it looks to see if the second element itself is a tuple. Finally if it is satisfied that the previous checks are valid it tests whether the first element is a number or not. It is in this final test, that the first element is not a valid number, where the UB could occur as explained in the topic 'Numbers'.

Problem when testing a list

The form of a non-empty list is a two element tuple, where the first element is the head of the list and can be anything and the second element is itself a list or the end-of-list identifier BOOST_PP_NIL.

The problem when testing for a list is that if the second element does not obey the constraint on testing for an identifier, since BOOST_PP_NIL is an identifier and is tested as such, you will get UB.

#include <boost/vmd/is_list.hpp>
#include <boost/vmd/is_tuple.hpp>

#define A_TUPLE (element,&anything)

BOOST_VMD_IS_LIST(A_TUPLE) will give UB due to the constraint
BOOST_VMD_IS_TUPLE(A_TUPLE) will return 1

The form of an empty list is the identifier BOOST_PP_NIL. Therefore:

#include <boost/vmd/is_identifier.hpp>
#include <boost/vmd/is_list.hpp>

#define A_BAD_EMPTY_LIST &BOOST_PP_NIL

BOOST_VMD_IS_LIST(A_BAD_EMPTY_LIST) will give UB due to the constraint
BOOST_VMD_IS_IDENTIFIER(A_BAD_EMPTY_LIST) will give UB due to the constraint

When VMD attempts to parse for a list, as it does when the BOOST_VMD_IS_LIST is used, if first looks to see if the syntax represents a tuple with two elements. If it is not a tuple with two elements it will check for the end-of-list. If it is a tuple with two elements it looks to see if the second element is a list. In both these paths it must always eventually check for the end-of-list notation BOOST_PP_NIL, which is an identifier in VMD. It is in this final test, that the end-of-list notation exists as a VMD identifier, where the UB could occur as explained in the topic 'Identifiers'.

Empty Boost PP data types

An array and a list can be empty.

An empty array has the form '(0,())', and is a perfectly valid array.

You can test for an empty array using the macro BOOST_VMD_IS_EMPTY_ARRAY.

#include <boost/vmd/is_array.hpp>
#include <boost/vmd/is_empty_array.hpp>

#define AN_ARRAY (1,(1))
#define AN_EMPTY_ARRAY (0,())

BOOST_VMD_IS_ARRAY(AN_ARRAY) will return 1
BOOST_VMD_IS_ARRAY(AN_EMPTY_ARRAY) will return 1

BOOST_VMD_IS_EMPTY_ARRAY(AN_EMPTY_ARRAY) will return 1
BOOST_VMD_IS_EMPTY_ARRAY() will return 0
BOOST_VMD_IS_EMPTY_ARRAY(AN_ARRAY) will return 0

An empty list has the form 'BOOST_PP_NIL', and is a perfectly valid list.

You can test for an empty list using the macro BOOST_VMD_IS_EMPTY_LIST.

#include <boost/vmd/is_empty_list.hpp>
#include <boost/vmd/is_list.hpp>

#define A_LIST (1,BOOST_PP_NIL)
#define AN_EMPTY_LIST BOOST_PP_NIL

BOOST_VMD_IS_LIST(A_LIST) will return 1
BOOST_VMD_IS_LIST(AN_EMPTY_LIST) will return 1

BOOST_VMD_IS_EMPTY_LIST(AN_EMPTY_LIST) will return 1
BOOST_VMD_IS_EMPTY_LIST() will return 0
BOOST_VMD_IS_EMPTY_LIST(A_LIST) will return 0

Neither seqs or tuples can be empty when using Boost PP. Because of this if you convert from an empty array or list to a seq or tuple using Boost PP macros to do so you will get undefined behavior.

The syntax '()', which is called an empty parenthesis, is neither a zero-element seq or a tuple consisting of no elements. Rather it is either a one-element seq whose content is emptiness or a single-element tuple whose content is emptiness.

VMD supports the syntax of an empty parenthesis. You can test for it using the macro BOOST_VMD_IS_PARENS_EMPTY.

#include <boost/vmd/is_parens_empty.hpp>
#include <boost/vmd/is_seq.hpp>
#include <boost/vmd/is_tuple.hpp>

#define EMPTY_PARENS ()
#define TUPLE (0)
#define SEQ (0)(1)

BOOST_VMD_IS_TUPLE(EMPTY_PARENS) will return 1
BOOST_VMD_IS_SEQ(EMPTY_PARENS) will return 1

BOOST_VMD_IS_PARENS_EMPTY(EMPTY_PARENS) will return 1
BOOST_VMD_IS_PARENS_EMPTY() will return 0
BOOST_VMD_IS_PARENS_EMPTY(TUPLE) will return 0
BOOST_VMD_IS_PARENS_EMPTY(SEQ) will return 0

The VC++8 compiler ( Visual Studio 2005 ), which is the oldest VC++ version which VMD supports, has trouble working with the empty parenthesis syntax. Therefore if you have to use VC++8 avoid its use, otherwise you should be fine using it if you desire.

Using a tuple instead of an array

When using variadic macros, the fact that an array can be empty is its only advantage over a tuple. Otherwise using a tuple is always easier since the syntax is simpler; you never have to notate the tuple's size.

Since VMD fully supports passing and returning emptiness you could use a tuple instead of an array in all situations and simply pass or return emptiness to represent an "empty" tuple, and as an equivalent to an empty array.

This notion of using emptiness to represent an "empty" tuple can also be extended to using emptiness to represent an "empty" seq. However functionality in Boost PP will not recognize emptiness as an empty tuple or seq, nor can you work with emptiness to represent an empty tuple or empty seq using the Boost PP functionality for a tuple or a seq. For a solution to using emptiness to represent an "empty" tuple or an "empty" seq VMD has functionality which will be explained when we look at our last area of functionality in VMD, useful variadic macros not in Boost PP.

Usage

You can use the general header file:

#include <boost/vmd/vmd.hpp>

or you can use individual header files for each of these macros. The individual header files are:

#include <boost/vmd/is_array.hpp> //  for the BOOST_VMD_IS_ARRAY macro
#include <boost/vmd/is_list.hpp> //  for the BOOST_VMD_IS_LIST macro
#include <boost/vmd/is_seq.hpp> //  for the BOOST_VMD_IS_SEQ macro
#include <boost/vmd/is_tuple.hpp> // for the BOOST_VMD_IS_TUPLE macro.

#include <boost/vmd/is_empty_array.hpp> // for the BOOST_VMD_IS_EMPTY_ARRAY macro.
#include <boost/vmd/is_empty_list.hpp> // for the BOOST_VMD_IS_EMPTY_LIST macro.
#include <boost/vmd/is_parens_empty.hpp> // for the BOOST_VMD_IS_PARENS_EMPTY macro.

The various macros for identifying VMD data types complement the ability to identify emptiness using BOOST_VMD_IS_EMPTY. The general name I will use in this documentation for these specific macros is "identifying macros." The identifying macros also share with BOOST_VMD_IS_EMPTY the inherent flaw mentioned when discussing BOOST_VMD_IS_EMPTY prior to the C++20 standard, since they themselves use BOOST_VMD_IS_EMPTY to determine that the input has ended.

To recapitulate the flaw with BOOST_VMD_IS_EMPTY prior to C++20:

  • using a standard C++ compiler if the input ends with the name of a function-like macro, and that macro takes two or more parameters, a preprocessing error will occur.
  • using the VC++ compiler with its default preprocessor if the input consists of the name of a function-like macro, and that macro when invoked with no parameters returns a tuple, the macro erroneously returns 1, meaning that the input is empty.
  • even if the function-like macro takes one parameter, passing emptiness to that macro could cause a preprocessing error.

The obvious way to avoid the BOOST_VMD_IS_EMPTY problem with the identifying macros is to design input so that the name of a function-like macro is never passed as a parameter. This can be done, if one uses VMD and has situations where the input could contain a function-like macro name, by having that function-like macro name placed within a Boost PP data type, such as a tuple, without attempting to identify the type of the tuple element using VMD. In other word if the input is:

( SOME_FUNCTION_MACRO_NAME )

and we have the macro definition:

#define SOME_FUNCTION_MACRO_NAME(x,y) some_output

VMD can still parse the input as a tuple, if desired, using BOOST_VMD_IS_TUPLE without encountering the BOOST_VMD_IS_EMPTY problem. However if the input is:

SOME_FUNCTION_MACRO_NAME

either directly or through accessing the above tuple's first element, and the programmer attempts to use BOOST_VMD_IS_IDENTIFIER with this input, the BOOST_VMD_IS_EMPTY problem will occur.

The VMD specific identifying macros test the input for VMD data types. The macros are represented by this table:

Table 1.2. Specific identifying macros

Name

Function

BOOST_VMD_IS_EMPTY

Tests if the input is empty

BOOST_VMD_IS_GENERAL_IDENTIFIER

Tests if the input has the general VMD identifier syntax

BOOST_VMD_IS_IDENTIFIER

Tests if the input is a specific VMD identifier

BOOST_VMD_IS_NUMBER

Tests if the input is a VMD number

BOOST_VMD_IS_TYPE

Tests if the input is a VMD type

BOOST_VMD_IS_ARRAY

Tests if the input is a Boost PP array

BOOST_VMD_IS_LIST

Tests if the input is a Boost PP list

BOOST_VMD_IS_SEQ

Tests if the input is a Boost PP seq

BOOST_VMD_IS_TUPLE

Tests if the input is a Boost PP tuple

BOOST_VMD_IS_EMPTY_ARRAY

Tests if the input is a Boost PP empty array

BOOST_VMD_IS_EMPTY_LIST

Tests if the input is a Boost PP empty list

BOOST_VMD_IS_PARENS_EMPTY

Tests if the input is empty parenthesis




[1] For VC++ 8 the input is not variadic data but a single parameter


PrevUpHomeNext