Support nb::init(<lambda>) as syntactic sugar for custom constructors

Author: oremanj

docs/api_core.rst

@@ -2191,9 +2191,14 @@ Class binding
 
    .. cpp:function:: template <typename... Args, typename... Extra> class_ &def(init<Args...> arg, const Extra &... extra)
 
-      Bind a constructor. The variable length `extra` parameter can be used to
+      Bind a C++ constructor that takes parameters of types ``Args...``.
+      The variable length `extra` parameter can be used to
       pass a docstring and other :ref:`function binding annotations
-      <function_binding_annotations>`.
+      <function_binding_annotations>`. You can also bind a custom constructor
+      (one that does not exist in the C++ code) by writing
+      ``.def(nb::init(<lambda>))``, provided the lambda returns an instance of
+      the class by value. If you need to wrap a factory function that returns
+      a pointer or shared pointer, see :cpp:struct:`nb::new_() <new_>` instead.
 
    .. cpp:function:: template <typename Arg, typename... Extra> class_ &def(init_implicit<Arg> arg, const Extra &... extra)
 
@@ -2567,62 +2572,96 @@ Class binding
    constructor. It is only meant to be used in binding declarations done via
    :cpp:func:`class_::def()`.
 
-   Sometimes, it is necessary to bind constructors that don't exist in the
-   underlying C++ type (meaning that they are specific to the Python bindings).
-   Because `init` only works for existing C++ constructors, this requires
-   a manual workaround noting that
-
-   .. code-block:: cpp
-
-      nb::class_<MyType>(m, "MyType")
-          .def(nb::init<const char*, int>());
-
-   is syntax sugar for the following lower-level implementation using
-   "`placement new <https://en.wikipedia.org/wiki/Placement_syntax>`_":
+   To bind a constructor that exists in the C++ class, taking ``Args...``, write
+   ``nb::init<Args...>()``.
+
+   To bind a constructor that is specific to the Python bindings (a
+   "custom constructor"), write ``nb::init(<some function>)`` (write a
+   lambda expression or a function pointer inside the
+   parentheses). The function should return a prvalue of the bound
+   type, by ending with a statement like ``return MyType(some,
+   args);``. If you write a custom constructor in this way, then
+   nanobind can construct the object without any extra copies or
+   moves, and the object therefore doesn't need to be copyable or movable.
+
+   If your custom constructor needs to take some actions after constructing
+   the C++ object, then nanobind recommends that you eschew
+   :cpp:struct:`nb::init() <init>` and instead bind an ``__init__`` method
+   directly. By convention, any nanobind method named ``"__init__"`` will
+   receive as its first argument a pointer to uninitialized storage that it
+   can initialize using `placement new
+   <https://en.wikipedia.org/wiki/Placement_syntax>`_:
 
    .. code-block:: cpp
 
       nb::class_<MyType>(m, "MyType")
           .def("__init__",
                [](MyType* t, const char* arg0, int arg1) {
                    new (t) MyType(arg0, arg1);
+                   t->doSomething();
                });
 
    The provided lambda function will be called with a pointer to uninitialized
    memory that has already been allocated (this memory region is co-located
    with the Python object for reasons of efficiency). The lambda function can
    then either run an in-place constructor and return normally (in which case
    the instance is assumed to be correctly constructed) or fail by raising an
-   exception.
+   exception. If an exception is raised, nanobind assumes the object *was not*
+   constructed; in the above example, if ``doSomething()`` could throw, then you
+   would need to take care to call the destructor explicitly (``t->~MyType();``)
+   in case of an exception after the C++ constructor had completed.
+
+   When binding a custom constructor using :cpp:struct:`nb::init() <init>` for
+   a type that supports :ref:`overriding virtual methods in Python
+   <trampolines>`, you must return either an instance of the trampoline
+   type (``PyPet`` in ``nb::class_<Pet, PyPet>(...)``) or something that
+   can initialize both the bound type and the trampoline type (e.g.,
+   you can return a ``Pet`` if there exists a ``PyPet(Pet&&)`` constructor).
+   If that's not possible, you can alternatively write :cpp:struct:`nb::init()
+   <init>` with two function arguments instead of one. The first returns
+   an instance of the bound type (``Pet``), and will be called when constructing
+   an instance of the C++ class that has not been extended from Python.
+   The second returns an instance of the trampoline type (``PyPet``),
+   and will be called when constructing an instance that does need to consider
+   the possibility of Python-based virtual method overrides.
+
+   .. note:: :cpp:struct:`nb::init() <init>` always creates Python ``__init__``
+      methods, which construct a C++ object in already-allocated Python object
+      storage. If you need to wrap a constructor that performs its own
+      allocation, such as a factory function that returns a pointer, you must
+      use :cpp:struct:`nb::new_() <new_>` instead in order to create a Python
+      ``__new__`` method.
 
 .. cpp:struct:: template <typename Arg> init_implicit
 
    See :cpp:class:`init` for detail on binding constructors. The main
-   difference between :cpp:class:`init`  and `init_implicit` is that the latter
-   only supports constructors taking a single argument `Arg`, and that it marks
-   the constructor as usable for implicit conversions from `Arg`.
+   difference between :cpp:class:`init` and `init_implicit` is that the latter
+   only supports constructors that exist in the C++ code and take a single
+   argument `Arg`, and that it marks the constructor as usable for implicit
+   conversions from `Arg`.
 
    Sometimes, it is necessary to bind implicit conversion-capable constructors
    that don't exist in the underlying C++ type (meaning that they are specific
-   to the Python bindings). This can be done manually noting that
+   to the Python bindings). This can be done manually, noting that
 
    .. code-block:: cpp
 
-      nb::class_<MyType>(m, "MyType")
-          .def(nb::init_implicit<const char*>());
+       nb::class_<MyType>(m, "MyType")
+           .def(nb::init_implicit<const char*>());
 
    can be replaced by the lower-level code
 
    .. code-block:: cpp
 
        nb::class_<MyType>(m, "MyType")
-           .def("__init__",
-                [](MyType* t, const char* arg0) {
-                    new (t) MyType(arg0);
-                });
+           .def(nb::init<const char*>());
 
        nb::implicitly_convertible<const char*, MyType>();
 
+   and that this transformation works equally well if you use one of the forms
+   of :cpp:class:`nb::init() <init>` that cannot be expressed by
+   :cpp:class:`init_implicit`.
+
 .. cpp:struct:: template <typename Func> new_
 
    This is a small helper class that indicates to :cpp:func:`class_::def()`

docs/changelog.rst

@@ -73,6 +73,17 @@ Version TBD (not yet released)
   binding abstractions that "feel like" the built-in ones.
   (PR `#884 <https://github.com/wjakob/nanobind/pull/884>`__)
 
+- :cpp:struct:`nb::init() <init>` may now be written with a function argument
+  and no template parameters to express a custom constructor that doesn't exist
+  in C++. For example, you could use this to adapt Python strings to a
+  pointer-and-length argument convention:
+  ``.def(nb::init([](std::string_view sv) { return MyType(sv.data(), sv.size()); }))``.
+  This feature is syntactic sugar for writing a custom ``"__init__"`` binding
+  using placement new, which remains fully supported, and which should continue
+  to be used in cases where the custom constructor cannot be written as a
+  function that finishes by returning a prvalue (``return MyType(some, args);``).
+  (PR `#885 <https://github.com/wjakob/nanobind/pull/885>`__)
+
 Version 2.4.0 (Dec 6, 2024)
 ---------------------------
 

docs/classes.rst

@@ -543,7 +543,8 @@ propagated to Python:
 
 To fix this behavior, you must implement a *trampoline class*. A trampoline has
 the sole purpose of capturing virtual function calls in C++ and forwarding them
-to Python.
+to Python. (If you're reading nanobind's source code, you might see references
+to an *alias class*; it's the same thing as a trampoline class.)
 
 .. code-block:: cpp
 

docs/porting.rst

@@ -146,30 +146,66 @@ accepts ``std::shared_ptr<T>``. That means a C++ function that accepts
 a raw ``T*`` and calls ``shared_from_this()`` on it might stop working
 when ported from pybind11 to nanobind. You can solve this problem
 by always passing such objects across the Python/C++ boundary as
-``std::shared_ptr<T>`` rather than as ``T*``. See the :ref:`advanced section
+``std::shared_ptr<T>`` rather than as ``T*``, or by exposing all
+constructors using :cpp:struct:`nb::new_() <new_>` wrappers that
+return ``std::shared_ptr<T>``. See the :ref:`advanced section
 on object ownership <enable_shared_from_this>` for more details.
 
 Custom constructors
 -------------------
 In pybind11, custom constructors (i.e. ones that do not already exist in the
 C++ class) could be specified as a lambda function returning an instance of
-the desired type.
+the desired type or a pointer to it.
 
 .. code-block:: cpp
 
    py::class_<MyType>(m, "MyType")
-       .def(py::init([](int) { return MyType(...); }));
+       .def(py::init([](int) { return MyType(...); }))
+       .def(py::init([](std::string_view) {
+           return std::make_unique<MyType>(...);
+       }));
+
+nanobind supports only the first form (where the lambda returns by value). Note
+that thanks to C++17's guaranteed copy elision, it now works even for types that
+are not copyable or movable, so you may be able to mechanically convert custom
+constructors that return by pointer into those that return by value.
+
+.. note:: If *any* of your custom constructors still need to return a pointer or
+   smart pointer, perhaps because they wrap a C++ factory method that only
+   exposes those return types, you must switch *all* of them to use
+   :cpp:struct:`nb::new_() <new_>` instead of :cpp:struct:`nb::init() <init>`.
+   Be aware that :cpp:struct:`nb::new_() <new_>` cannot construct in-place, so
+   using it gives up some of nanobind's performance benefits (but should still be
+   faster than ``py::init()`` in pybind11). It comes with some other caveats
+   as well, which are explained in the documentation on :ref:`customizing
+   Python object creation <custom_new>`.
+
+Guaranteed copy elision only works if the object is constructed as a temporary
+directly within the ``return`` statement. If you need to do something to the
+object before you return it, as in this example:
 
-Unfortunately, the implementation of this feature was quite complex and
-often required further internal calls to the move or copy
-constructor. nanobind instead reverts to how pybind11 originally
-implemented this feature using in-place construction (`placement
-new <https://en.wikipedia.org/wiki/Placement_syntax>`_):
+.. code-block:: cpp
+
+   py::class_<MyType>(m, "MyType")
+       .def(py::init([](int value) {
+           auto ret = MyType();
+           ret.value = value;
+           return ret;
+       }));
+
+then ``MyType`` must be movable, and depending on compiler optimizations the move
+constructor might actually be called at runtime, which is more expensive than
+in-place construction. In such cases, nanobind recommends instead that you
+directly bind a ``__init__`` method using `placement new
+<https://en.wikipedia.org/wiki/Placement_syntax>`_:
 
 .. code-block:: cpp
 
    nb::class_<MyType>(m, "MyType")
-       .def("__init__", [](MyType *t) { new (t) MyType(...); });
+       .def("__init__", [](MyType *t, int value) {
+           auto* self = new (t) MyType(...);
+           self->value = value;
+       });
 
 The provided lambda function will be called with a pointer to uninitialized
 memory that has already been allocated (this memory region is co-located
@@ -178,15 +214,6 @@ then either run an in-place constructor and return normally (in which case
 the instance is assumed to be correctly constructed) or fail by raising an
 exception.
 
-To turn an existing factory function into a constructor, you will need to
-combine the above pattern with an invocation of the move/copy-constructor,
-e.g.:
-
-.. code-block:: cpp
-
-   nb::class_<MyType>(m, "MyType")
-       .def("__init__", [](MyType *t) { new (t) MyType(MyType::create()); });
-
 Implicit conversions
 --------------------
 

include/nanobind/nb_class.h

@@ -278,6 +278,8 @@ struct is_copy_constructible : std::is_copy_constructible<T> { };
 template <typename T>
 constexpr bool is_copy_constructible_v = is_copy_constructible<T>::value;
 
+struct init_using_factory_tag {};
+
 NAMESPACE_END(detail)
 
 // Low level access to nanobind type objects
@@ -365,6 +367,106 @@ template <typename... Args> struct init : def_visitor<init<Args...>> {
     }
 };
 
+template <typename... Args>
+struct init<detail::init_using_factory_tag, Args...> {
+    static_assert(sizeof...(Args) == 2 || sizeof...(Args) == 4,
+                  "Unexpected instantiation convention for factory init");
+    static_assert(sizeof...(Args) != 2,
+                  "Couldn't deduce function signature for factory function");
+    static_assert(sizeof...(Args) != 4,
+                  "Base factory and alias factory accept different arguments, "
+                  "or we couldn't otherwise deduce their signatures");
+};
+
+template <typename Func, typename Return, typename... Args>
+struct init<detail::init_using_factory_tag, Func, Return(Args...)>
+  : def_visitor<init<detail::init_using_factory_tag, Func, Return(Args...)>> {
+    std::remove_reference_t<Func> func;
+
+    init(Func &&f) : func((detail::forward_t<Func>) f) {}
+
+    template <typename Class, typename... Extra>
+    NB_INLINE void execute(Class &cl, const Extra&... extra) {
+        using Type = typename Class::Type;
+        using Alias = typename Class::Alias;
+        constexpr bool has_alias = !std::is_same_v<Type, Alias>;
+        if constexpr (!has_alias) {
+            static_assert(std::is_constructible_v<Type, Return>,
+                          "nb::init() factory function must return an instance "
+                          "of the type by value, or something that can "
+                          "direct-initialize it");
+        } else {
+            static_assert(std::is_constructible_v<Alias, Return>,
+                          "nb::init() factory function must return an instance "
+                          "of the alias type by value, or something that can "
+                          "direct-initialize it");
+        }
+        cl.def(
+            "__init__",
+            [func_ = (detail::forward_t<Func>) func](pointer_and_handle<Type> v, Args... args) {
+                if constexpr (has_alias && std::is_constructible_v<Type, Return>) {
+                    if (!detail::nb_inst_python_derived(v.h.ptr())) {
+                        new (v.p) Type{ func_((detail::forward_t<Args>) args...) };
+                        return;
+                    }
+                }
+                new ((void *) v.p) Alias{ func_((detail::forward_t<Args>) args...) };
+            },
+            extra...);
+    }
+};
+
+template <typename CFunc, typename CReturn, typename AFunc, typename AReturn,
+          typename... Args>
+struct init<detail::init_using_factory_tag, CFunc, CReturn(Args...),
+            AFunc, AReturn(Args...)>
+  : def_visitor<init<detail::init_using_factory_tag, CFunc, CReturn(Args...),
+                     AFunc, AReturn(Args...)>> {
+    std::remove_reference_t<CFunc> cfunc;
+    std::remove_reference_t<AFunc> afunc;
+
+    init(CFunc &&cf, AFunc &&af)
+      : cfunc((detail::forward_t<CFunc>) cf),
+        afunc((detail::forward_t<AFunc>) af) {}
+
+    template <typename Class, typename... Extra>
+    NB_INLINE void execute(Class &cl, const Extra&... extra) {
+        using Type = typename Class::Type;
+        using Alias = typename Class::Alias;
+        static_assert(!std::is_same_v<Type, Alias>,
+                      "The form of nb::init() that takes two factory functions "
+                      "doesn't make sense to use on classes that don't have an "
+                      "alias type");
+        static_assert(std::is_constructible_v<Type, CReturn>,
+                      "nb::init() first factory function must return an "
+                      "instance of the type by value, or something that can "
+                      "direct-initialize it");
+        static_assert(std::is_constructible_v<Alias, AReturn>,
+                      "nb::init() second factory function must return an "
+                      "instance of the alias type by value, or something that "
+                      "can direct-initialize it");
+        cl.def(
+            "__init__",
+            [cfunc_ = (detail::forward_t<CFunc>) cfunc,
+             afunc_ = (detail::forward_t<AFunc>) afunc](pointer_and_handle<Type> v, Args... args) {
+                if (!detail::nb_inst_python_derived(v.h.ptr()))
+                    new (v.p) Type{ cfunc_((detail::forward_t<Args>) args...) };
+                else
+                    new ((void *) v.p) Alias{ afunc_((detail::forward_t<Args>) args...) };
+            },
+            extra...);
+    }
+};
+
+template <typename Func>
+init(Func&& f) -> init<detail::init_using_factory_tag,
+                       Func, detail::function_signature_t<Func>>;
+
+template <typename CFunc, typename AFunc>
+init(CFunc&& cf, AFunc&& af) -> init<detail::init_using_factory_tag,
+                                     CFunc, detail::function_signature_t<CFunc>,
+                                     AFunc, detail::function_signature_t<AFunc>>;
+
 template <typename Arg> struct init_implicit : def_visitor<init_implicit<Arg>> {
     template <typename T, typename... Ts> friend class class_;
     NB_INLINE init_implicit() { }