Boost C++ Libraries

Call Policies

In C++, we often deal with arguments and return types such as pointers and references. Such primitive types are rather, ummmm, low level and they really don't tell us much. At the very least, we don't know the owner of the pointer or the referenced object. No wonder languages such as Java and Python never deal with such low level entities. In C++, it's usually considered a good practice to use smart pointers which exactly describe ownership semantics. Still, even good C++ interfaces use raw references and pointers sometimes, so Boost.Python must deal with them. To do this, it may need your help. Consider the following C++ function:

    X& f(Y& y, Z* z);

How should the library wrap this function? A naive approach builds a Python X object around result reference. This strategy might or might not work out. Here's an example where it didn't

    >>> x = f(y, z) ##x refers to some C++ X
    >>> del y
    >>> x.some_method() ##CRASH!

What's the problem?

Well, what if f() was implemented as shown below:

    X& f(Y& y, Z* z)
    {
        y.z = z;
        return y.x;
    }

The problem is that the lifetime of result X& is tied to the lifetime of y, because the f() returns a reference to a member of the y object. This idiom is is not uncommon and perfectly acceptable in the context of C++. However, Python users should not be able to crash the system just by using our C++ interface. In this case deleting y will invalidate the reference to X. We have a dangling reference.

Here's what's happening:

1. f is called passing in a reference to y and a pointer to z
2. A reference to y.x is returned
3. y is deleted. x is a dangling reference
4. x.some_method() is called
5. BOOM!

We could copy result into a new object:

    >>> f(y, z).set(42) ##Result disappears
    >>> y.x.get()       ##No crash, but still bad
    3.14

This is not really our intent of our C++ interface. We've broken our promise that the Python interface should reflect the C++ interface as closely as possible.

Our problems do not end there. Suppose Y is implemented as follows:

    struct Y
    {
        X x; Z* z;
        int z_value() { return z->value(); }
    };

Notice that the data member z is held by class Y using a raw pointer. Now we have a potential dangling pointer problem inside Y:

    >>> x = f(y, z) ##y refers to z
    >>> del z       ##Kill the z object
    >>> y.z_value() ##CRASH!

For reference, here's the implementation of f again:

    X& f(Y& y, Z* z)
    {
        y.z = z;
        return y.x;
    }

Here's what's happening:

1. f is called passing in a reference to y and a pointer to z
2. A pointer to z is held by y
3. A reference to y.x is returned
4. z is deleted. y.z is a dangling pointer
5. y.z_value() is called
6. z->value() is called
7. BOOM!

Call Policies

Call Policies may be used in situations such as the example detailed above. In our example, return_internal_reference and with_custodian_and_ward are our friends:

    def("f", f,
        return_internal_reference<1,
            with_custodian_and_ward<1, 2> >());

What are the 1 and 2 parameters, you ask?

    return_internal_reference<1

Informs Boost.Python that the first argument, in our case Y& y, is the owner of the returned reference: X&. The "1" simply specifies the first argument. In short: "return an internal reference X& owned by the 1st argument Y& y".

    with_custodian_and_ward<1, 2>

Informs Boost.Python that the lifetime of the argument indicated by ward (i.e. the 2nd argument: Z* z) is dependent on the lifetime of the argument indicated by custodian (i.e. the 1st argument: Y& y).

It is also important to note that we have defined two policies above. Two or more policies can be composed by chaining. Here's the general syntax:

    policy1<args...,
        policy2<args...,
            policy3<args...> > >

Here is the list of predefined call policies. A complete reference detailing these can be found here.

• with_custodian_and_ward
  Ties lifetimes of the arguments
• with_custodian_and_ward_postcall
  Ties lifetimes of the arguments and results
• return_internal_reference
  Ties lifetime of one argument to that of result
• return_value_policy<T> with T one of:
  
• reference_existing_object
  naïve (dangerous) approach
• copy_const_reference
  Boost.Python v1 approach
• copy_non_const_reference
  
• manage_new_object
  Adopt a pointer and hold the instance

Remember the Zen, Luke: "Explicit is better than implicit" "In the face of ambiguity, refuse the temptation to guess"

Copyright © 2002-2003 David Abrahams
Copyright © 2002-2003 Joel de Guzman

Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided "as is" without express or implied warranty, and with no claim as to its suitability for any purpose.