Boost C++ Libraries

...one of the most highly regarded and expertly designed C++ library projects in the world. Herb Sutter and Andrei Alexandrescu, C++ Coding Standards

This is the documentation for an old version of Boost. Click here to view this page for the latest version.
PrevUpHomeNext

Functions

Call Policies
Overloading
Default Arguments
Auto-Overloading

In this chapter, we'll look at Boost.Python powered functions in closer detail. We will see some facilities to make exposing C++ functions to Python safe from potential pifalls such as dangling pointers and references. We will also see facilities that will make it even easier for us to expose C++ functions that take advantage of C++ features such as overloading and default arguments.

Read on...

But before you do, you might want to fire up Python 2.2 or later and type >>> import this.

>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than right now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

In C++, we often deal with arguments and return types such as pointers and references. Such primitive types are rather, ummmm, low level and they really don't tell us much. At the very least, we don't know the owner of the pointer or the referenced object. No wonder languages such as Java and Python never deal with such low level entities. In C++, it's usually considered a good practice to use smart pointers which exactly describe ownership semantics. Still, even good C++ interfaces use raw references and pointers sometimes, so Boost.Python must deal with them. To do this, it may need your help. Consider the following C++ function:

X& f(Y& y, Z* z);

How should the library wrap this function? A naive approach builds a Python X object around result reference. This strategy might or might not work out. Here's an example where it didn't

>>> x = f(y, z) # x refers to some C++ X
>>> del y
>>> x.some_method() # CRASH!

What's the problem?

Well, what if f() was implemented as shown below:

X& f(Y& y, Z* z)
{
    y.z = z;
    return y.x;
}

The problem is that the lifetime of result X& is tied to the lifetime of y, because the f() returns a reference to a member of the y object. This idiom is is not uncommon and perfectly acceptable in the context of C++. However, Python users should not be able to crash the system just by using our C++ interface. In this case deleting y will invalidate the reference to X. We have a dangling reference.

Here's what's happening:

  1. f is called passing in a reference to y and a pointer to z
  2. A reference to y.x is returned
  3. y is deleted. x is a dangling reference
  4. x.some_method() is called
  5. BOOM!

We could copy result into a new object:

>>> f(y, z).set(42) # Result disappears
>>> y.x.get()       # No crash, but still bad
3.14

This is not really our intent of our C++ interface. We've broken our promise that the Python interface should reflect the C++ interface as closely as possible.

Our problems do not end there. Suppose Y is implemented as follows:

struct Y
{
    X x; Z* z;
    int z_value() { return z->value(); }
};

Notice that the data member z is held by class Y using a raw pointer. Now we have a potential dangling pointer problem inside Y:

>>> x = f(y, z) # y refers to z
>>> del z       # Kill the z object
>>> y.z_value() # CRASH!

For reference, here's the implementation of f again:

X& f(Y& y, Z* z)
{
    y.z = z;
    return y.x;
}

Here's what's happening:

  1. f is called passing in a reference to y and a pointer to z
  2. A pointer to z is held by y
  3. A reference to y.x is returned
  4. z is deleted. y.z is a dangling pointer
  5. y.z_value() is called
  6. z->value() is called
  7. BOOM!

Call Policies

Call Policies may be used in situations such as the example detailed above. In our example, return_internal_reference and with_custodian_and_ward are our friends:

def("f", f,
    return_internal_reference<1,
        with_custodian_and_ward<1, 2> >());

What are the 1 and 2 parameters, you ask?

return_internal_reference<1

Informs Boost.Python that the first argument, in our case Y& y, is the owner of the returned reference: X&. The "1" simply specifies the first argument. In short: "return an internal reference X& owned by the 1st argument Y& y".

with_custodian_and_ward<1, 2>

Informs Boost.Python that the lifetime of the argument indicated by ward (i.e. the 2nd argument: Z* z) is dependent on the lifetime of the argument indicated by custodian (i.e. the 1st argument: Y& y).

It is also important to note that we have defined two policies above. Two or more policies can be composed by chaining. Here's the general syntax:

policy1<args...,
    policy2<args...,
        policy3<args...> > >

Here is the list of predefined call policies. A complete reference detailing these can be found here.

  • with_custodian_and_ward: Ties lifetimes of the arguments
  • with_custodian_and_ward_postcall: Ties lifetimes of the arguments and results
  • return_internal_reference: Ties lifetime of one argument to that of result
  • return_value_policy<T> with T one of:
    • reference_existing_object: naive (dangerous) approach
    • copy_const_reference: Boost.Python v1 approach
    • copy_non_const_reference:
    • manage_new_object: Adopt a pointer and hold the instance

Remember the Zen, Luke:

"Explicit is better than implicit"

"In the face of ambiguity, refuse the temptation to guess"

The following illustrates a scheme for manually wrapping an overloaded member functions. Of course, the same technique can be applied to wrapping overloaded non-member functions.

We have here our C++ class:

struct X
{
    bool f(int a)
    {
        return true;
    }

    bool f(int a, double b)
    {
        return true;
    }

    bool f(int a, double b, char c)
    {
        return true;
    }

    int f(int a, int b, int c)
    {
        return a + b + c;
    };
};

Class X has 4 overloaded functions. We will start by introducing some member function pointer variables:

bool    (X::*fx1)(int)              = &X::f;
bool    (X::*fx2)(int, double)      = &X::f;
bool    (X::*fx3)(int, double, char)= &X::f;
int     (X::*fx4)(int, int, int)    = &X::f;

With these in hand, we can proceed to define and wrap this for Python:

.def("f", fx1)
.def("f", fx2)
.def("f", fx3)
.def("f", fx4)

Boost.Python wraps (member) function pointers. Unfortunately, C++ function pointers carry no default argument info. Take a function f with default arguments:

int f(int, double = 3.14, char const* = "hello");

But the type of a pointer to the function f has no information about its default arguments:

int(*g)(int,double,char const*) = f;    // defaults lost!

When we pass this function pointer to the def function, there is no way to retrieve the default arguments:

def("f", f);                            // defaults lost!

Because of this, when wrapping C++ code, we had to resort to manual wrapping as outlined in the previous section, or writing thin wrappers:

// write "thin wrappers"
int f1(int x) { return f(x); }
int f2(int x, double y) { return f(x,y); }

/*...*/

    // in module init
    def("f", f);  // all arguments
    def("f", f2); // two arguments
    def("f", f1); // one argument

When you want to wrap functions (or member functions) that either:

  • have default arguments, or
  • are overloaded with a common sequence of initial arguments

BOOST_PYTHON_FUNCTION_OVERLOADS

Boost.Python now has a way to make it easier. For instance, given a function:

int foo(int a, char b = 1, unsigned c = 2, double d = 3)
{
    /*...*/
}

The macro invocation:

BOOST_PYTHON_FUNCTION_OVERLOADS(foo_overloads, foo, 1, 4)

will automatically create the thin wrappers for us. This macro will create a class foo_overloads that can be passed on to def(...). The third and fourth macro argument are the minimum arguments and maximum arguments, respectively. In our foo function the minimum number of arguments is 1 and the maximum number of arguments is 4. The def(...) function will automatically add all the foo variants for us:

def("foo", foo, foo_overloads());

BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS

Objects here, objects there, objects here there everywhere. More frequently than anything else, we need to expose member functions of our classes to Python. Then again, we have the same inconveniences as before when default arguments or overloads with a common sequence of initial arguments come into play. Another macro is provided to make this a breeze.

Like BOOST_PYTHON_FUNCTION_OVERLOADS, BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS may be used to automatically create the thin wrappers for wrapping member functions. Let's have an example:

struct george
{
    void
    wack_em(int a, int b = 0, char c = 'x')
    {
        /*...*/
    }
};

The macro invocation:

BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(george_overloads, wack_em, 1, 3)

will generate a set of thin wrappers for george's wack_em member function accepting a minimum of 1 and a maximum of 3 arguments (i.e. the third and fourth macro argument). The thin wrappers are all enclosed in a class named george_overloads that can then be used as an argument to def(...):

.def("wack_em", &george::wack_em, george_overloads());

See the overloads reference for details.

init and optional

A similar facility is provided for class constructors, again, with default arguments or a sequence of overloads. Remember init<...>? For example, given a class X with a constructor:

struct X
{
    X(int a, char b = 'D', std::string c = "constructor", double d = 0.0);
    /*...*/
}

You can easily add this constructor to Boost.Python in one shot:

.def(init<int, optional<char, std::string, double> >())

Notice the use of init<...> and optional<...> to signify the default (optional arguments).

It was mentioned in passing in the previous section that BOOST_PYTHON_FUNCTION_OVERLOADS and BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS can also be used for overloaded functions and member functions with a common sequence of initial arguments. Here is an example:

void foo()
{
   /*...*/
}

void foo(bool a)
{
   /*...*/
}

void foo(bool a, int b)
{
   /*...*/
}

void foo(bool a, int b, char c)
{
   /*...*/
}

Like in the previous section, we can generate thin wrappers for these overloaded functions in one-shot:

BOOST_PYTHON_FUNCTION_OVERLOADS(foo_overloads, foo, 0, 3)

Then...

.def("foo", (void(*)(bool, int, char))0, foo_overloads());

Notice though that we have a situation now where we have a minimum of zero (0) arguments and a maximum of 3 arguments.

Manual Wrapping

It is important to emphasize however that the overloaded functions must have a common sequence of initial arguments. Otherwise, our scheme above will not work. If this is not the case, we have to wrap our functions manually.

Actually, we can mix and match manual wrapping of overloaded functions and automatic wrapping through BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS and its sister, BOOST_PYTHON_FUNCTION_OVERLOADS. Following up on our example presented in the section on overloading, since the first 4 overload functins have a common sequence of initial arguments, we can use BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS to automatically wrap the first three of the defs and manually wrap just the last. Here's how we'll do this:

BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(xf_overloads, f, 1, 4)

Create a member function pointers as above for both X::f overloads:

bool    (X::*fx1)(int, double, char)    = &X::f;
int     (X::*fx2)(int, int, int)        = &X::f;

Then...

.def("f", fx1, xf_overloads());
.def("f", fx2)

PrevUpHomeNext