Porting Guide

Porting PyObject * to HPy API constructs

While in CPython one always uses PyObject * to reference to Python objects, in HPy there are several types of handles that should be used depending on the life-time of the handle: HPy, HPyField, and HPyGlobal.

  • HPy represents short lived handles that live no longer than the duration of one call from Python to HPy extension function. Rule of thumb: use for local variables, arguments, and return values.

  • HPyField represents handles that are Python object struct fields, i.e., live in native memory attached to some Python object.

  • HPyGlobal represents handles stored in C global variables. HPyGlobal can provide isolation between subinterpreters.

Warning

Never use a local variable of type HPyField, for any reason! If the GC kicks in, it might become invalid and become a dangling pointer.

Warning

Never store HPy handles to a long-lived memory, for example: C global variables or Python object structs.

The HPy/HPyField dichotomy might seem arbitrary at first, but it is needed to allow Python implementations to use a moving GC, such as PyPy. It is easier to explain and understand the rules by thinking about how a moving GC interacts with the C code inside an HPy extension.

It is worth remembering that during the collection phase, a moving GC might move an existing object to another memory location, and in that case it needs to update all the places which store a pointer to it. In order to do so, it needs to know where the pointers are. If there is a local C variable which is unknown to the GC but contains a pointer to a GC-managed object, the variable will point to invalid memory as soon as the object is moved.

Back to HPy vs HPyField vs HPyGlobal:

  • HPy handles must be used for all C local variables, function arguments and function return values. They are supposed to be short-lived and closed as soon as they are no longer needed. The debug mode will report a long-lived HPy as a potential memory leak.

  • In PyPy and GraalPy, HPy handles are implemented using an indirection: they are indexes inside a big list of GC-managed objects: this big list is tracked by the GC, so when an object moves its pointer is correctly updated.

  • HPyField is for long-lived references, and the GC must be aware of their location in memory. In PyPy, an HPyField is implemented as a direct pointer to the object, and thus we need a way to inform the GC where it is in memory, so that it can update its value upon moving: this job is done by tp_traverse, as explained in the next section.

  • HPyGlobal is for long-lived references that are supposed to be closed implicitly when the module is unloaded (once module unloading is actually implemented). HPyGlobal provides indirection to isolate subinterpreters. Implementation wise, HPyGlobal will usually contain an index to a table with Python objects stored in the interpreter state.

  • On CPython without subinterpreters support, HPy, HPyGlobal, and HPyField are implemented as PyObject *.

  • On CPython with subinterpreters support, HPyGlobal will be implemented by an indirection through the interpreter state. Note that thanks to the HPy design, switching between this and the more efficient implementation without subinterpreter support will not require rebuilding of the extension (in HPy universal mode), nor rebuilding of CPython.

Note

If you write a custom type using HPyField, you MUST also write a tp_traverse slot. Note that this is different than the old Python.h API, where you need tp_traverse only under certain conditions. See the next section for more details.

Note

The contract of tp_traverse is that it must visit all members of type HPyField contained within given struct, or more precisely owned by given Python object (in the sense of the owner argument to HPyField_Store), and nothing more, nothing less. Some Python implementations may choose to not call the provided tp_traverse if they know how to visit all members of type HPyField by other means (for example, when they track them internally already). The debug mode will check this contract.

tp_traverse, tp_clear, Py_TPFLAGS_HAVE_GC

Let’s quote the Python.h documentation about GC support

Python’s support for detecting and collecting garbage which involves circular references requires support from object types which are “containers” for other objects which may also be containers. Types which do not store references to other objects, or which only store references to atomic types (such as numbers or strings), do not need to provide any explicit support for garbage collection.

A good rule of thumb is that if your type contains PyObject * fields, you need to:

  1. provide a tp_traverse slot;

  2. provide a tp_clear slot;

  3. add the Py_TPFLAGS_GC to the tp_flags.

However, if you know that your PyObject * fields will contain only “atomic” types, you can avoid these steps.

In HPy the rules are slightly different:

  1. if you have a field of type HPyField, you always MUST provide a tp_traverse. This is needed so that a moving GC can track the relevant areas of memory. However, you MUST NOT rely on tp_traverse to be called;

  2. tp_clear does not exist. On CPython, HPy automatically generates one for you, by using tp_traverse to know which are the fields to clear. Other implementations are free to ignore it, if it’s not needed;

  3. HPy_TPFLAGS_GC is still needed, especially on CPython. If you don’t specify it, your type will not be tracked by CPython’s GC and thus it might cause memory leaks if it’s part of a reference cycle. However, other implementations are free to ignore the flag and track the objects anyway, if their GC implementation allows it.

tp_dealloc and Py_DECREF

Generally speaking, if you have one or more PyObject * fields in the old Python.h, you must provide a tp_dealloc slot where you Py_DECREF all of them. In HPy this is not needed and will be handled automatically by the system.

In particular, when running on top of CPython, HPy will automatically provide a tp_dealloc which decrefs all the fields listed by tp_traverse.

See also, Deallocator slot Py_tp_dealloc.

Direct C API to HPy mappings

In many cases, migrating to HPy is as easy as just replacing a certain C API function by the appropriate HPy API function. Table Safe API function mapping gives a mapping between C API and HPy API functions. This mapping is generated together with the code for the CPython ABI mode, so it is guaranteed to be correct.

Safe API function mapping

C API function

HPY API function

PyBool_FromLong

HPyBool_FromLong()

PyBytes_AS_STRING

HPyBytes_AS_STRING()

PyBytes_AsString

HPyBytes_AsString()

PyBytes_Check

HPyBytes_Check()

PyBytes_FromString

HPyBytes_FromString()

PyBytes_GET_SIZE

HPyBytes_GET_SIZE()

PyBytes_Size

HPyBytes_Size()

PyCallable_Check

HPyCallable_Check()

PyCapsule_IsValid

HPyCapsule_IsValid()

PyContextVar_Get

HPyContextVar_Get()

PyContextVar_New

HPyContextVar_New()

PyContextVar_Set

HPyContextVar_Set()

PyDict_Check

HPyDict_Check()

PyDict_Copy

HPyDict_Copy()

PyDict_Keys

HPyDict_Keys()

PyDict_New

HPyDict_New()

PyErr_Clear

HPyErr_Clear()

PyErr_ExceptionMatches

HPyErr_ExceptionMatches()

PyErr_NewException

HPyErr_NewException()

PyErr_NewExceptionWithDoc

HPyErr_NewExceptionWithDoc()

PyErr_NoMemory

HPyErr_NoMemory()

PyErr_SetFromErrnoWithFilename

HPyErr_SetFromErrnoWithFilename()

PyErr_SetFromErrnoWithFilenameObjects

HPyErr_SetFromErrnoWithFilenameObjects()

PyErr_SetObject

HPyErr_SetObject()

PyErr_SetString

HPyErr_SetString()

PyErr_WarnEx

HPyErr_WarnEx()

PyErr_WriteUnraisable

HPyErr_WriteUnraisable()

PyEval_EvalCode

HPy_EvalCode()

PyEval_RestoreThread

HPy_ReenterPythonExecution()

PyEval_SaveThread

HPy_LeavePythonExecution()

PyFloat_AsDouble

HPyFloat_AsDouble()

PyFloat_FromDouble

HPyFloat_FromDouble()

PyImport_ImportModule

HPyImport_ImportModule()

PyList_Append

HPyList_Append()

PyList_Check

HPyList_Check()

PyList_New

HPyList_New()

PyLong_AsDouble

HPyLong_AsDouble()

PyLong_AsLong

HPyLong_AsLong()

PyLong_AsLongLong

HPyLong_AsLongLong()

PyLong_AsSize_t

HPyLong_AsSize_t()

PyLong_AsSsize_t

HPyLong_AsSsize_t()

PyLong_AsUnsignedLong

HPyLong_AsUnsignedLong()

PyLong_AsUnsignedLongLong

HPyLong_AsUnsignedLongLong()

PyLong_AsUnsignedLongLongMask

HPyLong_AsUnsignedLongLongMask()

PyLong_AsUnsignedLongMask

HPyLong_AsUnsignedLongMask()

PyLong_AsVoidPtr

HPyLong_AsVoidPtr()

PyLong_FromLong

HPyLong_FromLong()

PyLong_FromLongLong

HPyLong_FromLongLong()

PyLong_FromSize_t

HPyLong_FromSize_t()

PyLong_FromSsize_t

HPyLong_FromSsize_t()

PyLong_FromUnsignedLong

HPyLong_FromUnsignedLong()

PyLong_FromUnsignedLongLong

HPyLong_FromUnsignedLongLong()

PyNumber_Absolute

HPy_Absolute()

PyNumber_Add

HPy_Add()

PyNumber_And

HPy_And()

PyNumber_Check

HPyNumber_Check()

PyNumber_Divmod

HPy_Divmod()

PyNumber_Float

HPy_Float()

PyNumber_FloorDivide

HPy_FloorDivide()

PyNumber_InPlaceAdd

HPy_InPlaceAdd()

PyNumber_InPlaceAnd

HPy_InPlaceAnd()

PyNumber_InPlaceFloorDivide

HPy_InPlaceFloorDivide()

PyNumber_InPlaceLshift

HPy_InPlaceLshift()

PyNumber_InPlaceMatrixMultiply

HPy_InPlaceMatrixMultiply()

PyNumber_InPlaceMultiply

HPy_InPlaceMultiply()

PyNumber_InPlaceOr

HPy_InPlaceOr()

PyNumber_InPlacePower

HPy_InPlacePower()

PyNumber_InPlaceRemainder

HPy_InPlaceRemainder()

PyNumber_InPlaceRshift

HPy_InPlaceRshift()

PyNumber_InPlaceSubtract

HPy_InPlaceSubtract()

PyNumber_InPlaceTrueDivide

HPy_InPlaceTrueDivide()

PyNumber_InPlaceXor

HPy_InPlaceXor()

PyNumber_Index

HPy_Index()

PyNumber_Invert

HPy_Invert()

PyNumber_Long

HPy_Long()

PyNumber_Lshift

HPy_Lshift()

PyNumber_MatrixMultiply

HPy_MatrixMultiply()

PyNumber_Multiply

HPy_Multiply()

PyNumber_Negative

HPy_Negative()

PyNumber_Or

HPy_Or()

PyNumber_Positive

HPy_Positive()

PyNumber_Power

HPy_Power()

PyNumber_Remainder

HPy_Remainder()

PyNumber_Rshift

HPy_Rshift()

PyNumber_Subtract

HPy_Subtract()

PyNumber_TrueDivide

HPy_TrueDivide()

PyNumber_Xor

HPy_Xor()

PyObject_ASCII

HPy_ASCII()

PyObject_Bytes

HPy_Bytes()

PyObject_Call

HPy_CallTupleDict()

PyObject_DelItem

HPy_DelItem()

PyObject_GetAttr

HPy_GetAttr()

PyObject_GetAttrString

HPy_GetAttr_s()

PyObject_GetItem

HPy_GetItem()

PyObject_HasAttr

HPy_HasAttr()

PyObject_HasAttrString

HPy_HasAttr_s()

PyObject_Hash

HPy_Hash()

PyObject_IsTrue

HPy_IsTrue()

PyObject_Length

HPy_Length()

PyObject_Repr

HPy_Repr()

PyObject_RichCompare

HPy_RichCompare()

PyObject_RichCompareBool

HPy_RichCompareBool()

PyObject_SetAttr

HPy_SetAttr()

PyObject_SetAttrString

HPy_SetAttr_s()

PyObject_SetItem

HPy_SetItem()

PyObject_Str

HPy_Str()

PyObject_Type

HPy_Type()

PyObject_TypeCheck

HPy_TypeCheck()

PyObject_Vectorcall

HPy_Call()

PyObject_VectorcallMethod

HPy_CallMethod()

PySequence_Contains

HPy_Contains()

PySlice_AdjustIndices

HPySlice_AdjustIndices()

PySlice_Unpack

HPySlice_Unpack()

PyTuple_Check

HPyTuple_Check()

PyType_IsSubtype

HPyType_IsSubtype()

PyUnicode_AsASCIIString

HPyUnicode_AsASCIIString()

PyUnicode_AsLatin1String

HPyUnicode_AsLatin1String()

PyUnicode_AsUTF8AndSize

HPyUnicode_AsUTF8AndSize()

PyUnicode_AsUTF8String

HPyUnicode_AsUTF8String()

PyUnicode_Check

HPyUnicode_Check()

PyUnicode_DecodeASCII

HPyUnicode_DecodeASCII()

PyUnicode_DecodeFSDefault

HPyUnicode_DecodeFSDefault()

PyUnicode_DecodeFSDefaultAndSize

HPyUnicode_DecodeFSDefaultAndSize()

PyUnicode_DecodeLatin1

HPyUnicode_DecodeLatin1()

PyUnicode_EncodeFSDefault

HPyUnicode_EncodeFSDefault()

PyUnicode_FromEncodedObject

HPyUnicode_FromEncodedObject()

PyUnicode_FromString

HPyUnicode_FromString()

PyUnicode_FromWideChar

HPyUnicode_FromWideChar()

PyUnicode_ReadChar

HPyUnicode_ReadChar()

PyUnicode_Substring

HPyUnicode_Substring()

Py_FatalError

HPy_FatalError()

Reference Counting Py_INCREF and Py_DECREF

The equivalents of Py_INCREF and Py_DECREF are essentially HPy_Dup() and HPy_Close(), respectively. The main difference is that HPy_Dup() gives you a new handle to the same object which means that the two handles may be different if comparing them with memcmp but still reference the same object. As a consequence, you may close a handle only once, i.e., you cannot call HPy_Close() twice on the same HPy handle, even if returned from HPy_Dup. For examples, see also sections Handles and Handles vs PyObject *

Calling functions PyObject_Call and PyObject_CallObject

Both PyObject_Call and PyObject_CallObject are replaced by HPy_CallTupleDict(callable, args, kwargs) in which either or both of args and kwargs may be null handles.

PyObject_Call(callable, args, kwargs) becomes:

HPy result = HPy_CallTupleDict(ctx, callable, args, kwargs);

PyObject_CallObject(callable, args) becomes:

HPy result = HPy_CallTupleDict(ctx, callable, args, HPy_NULL);

If args is not a handle to a tuple or kwargs is not a handle to a dictionary, HPy_CallTupleDict will return HPy_NULL and raise a TypeError. This is different to PyObject_Call and PyObject_CallObject which may segfault instead.

Calling Protocol

Both the tp_call and vectorcall calling protocols are replaced by HPy’s calling protocol. This is done by defining slot HPy_tp_call. HPy uses only one calling convention which is similar to the vectorcall calling convention. In the following example, we implement a call function for a simple Euclidean vector type. The function computes the dot product of two vectors.

typedef struct {
    long x;
    long y;
} EuclideanVectorObject;
HPyType_HELPERS(EuclideanVectorObject)
HPyDef_SLOT(call, HPy_tp_call)
static HPy
call_impl(HPyContext *ctx, HPy self, const HPy *args, size_t nargs,
          HPy kwnames)
{
    static const char *keywords[] = { "x1", "y1", NULL };
    long x1, y1;
    HPyTracker ht;
    if (!HPyArg_ParseKeywords(ctx, &ht, args, nargs, kwnames, "ll", keywords,
                &x1, &y1)) {
        return HPy_NULL;
    }
    EuclideanVectorObject *data = EuclideanVectorObject_AsStruct(ctx, self);
    return HPyLong_FromLong(ctx, data->x * x1 + data->y * y1);
}

Positional and keyword arguments are passed as C array args. Argument nargs specifies the number of positional arguments. Argument kwnames is a tuple containing the names of the keyword arguments. The keyword argument values are appended to positional arguments and start at args[nargs] (if there are any).

In the above example, function call_impl will be used by default to call all instances of the corresponding type. It is also possible to install (maybe specialized) call function implementations per instances by using function HPy_SetCallFunction(). This needs to be done in the constructor of an object. For example:

HPyDef_CALL_FUNCTION(special_call)
static HPy
special_call_impl(HPyContext *ctx, HPy self, const HPy *args, size_t nargs,
                  HPy kwnames)
{
    HPy tmp = call_impl(ctx, self, args, nargs, kwnames);
    HPy res = HPy_Negative(ctx, tmp);
    HPy_Close(ctx, tmp);
    return res;
}

HPyDef_SLOT(new, HPy_tp_new)
static HPy
new_impl(HPyContext *ctx, HPy cls, const HPy *args, HPy_ssize_t nargs, HPy kw)
{
    static const char *keywords[] = { "x", "y", "use_special_call", NULL };
    HPyTracker ht;
    long x, y;
    HPy use_special_call = ctx->h_False;
    if (!HPyArg_ParseKeywordsDict(ctx, &ht, args, nargs, kw, "ll|O", keywords,
                &x, &y, &use_special_call)) {
        return HPy_NULL;
    }
    EuclideanVectorObject *vector;
    HPy h_point = HPy_New(ctx, cls, &vector);
    if (HPy_IsNull(h_point)) {
        HPyTracker_Close(ctx, ht);
        return HPy_NULL;
    }
    if (HPy_IsTrue(ctx, use_special_call) &&
            HPy_SetCallFunction(ctx, h_point, &special_call) < 0) {
        HPyTracker_Close(ctx, ht);
        HPy_Close(ctx, h_point);
        return HPy_NULL;
    }
    HPyTracker_Close(ctx, ht);
    vector->x = x;
    vector->y = y;
    return h_point;
}

Limitations

  1. It is not possible to use slot HPy_tp_call for a var object (i.e. if HPyType_Spec.itemsize is greater 0). Reason: HPy installs a hidden field in the object’s data to store the call function pointer which is appended to everything else. In case of EuclideanVectorObject, a field is implicitly appended after member y. This is not possible for var objects because the variable part will also start after the fixed members.

  2. It is also not possible to use slot HPy_tp_call with a legacy type that inherits the basicsize (i.e. if HPyType_Spec.basicsize is 0) for the same reason as above.

To overcome these limitations, it is still possible to manually embed a field for the call function pointer in a type’s C struct and tell HPy where this field is. In this case, it is always necessary to set the call function pointer using HPy_SetCallFunction() in the object’s constructor. This procedure is less convenient than just using slot HPy_tp_cal but still not hard to use. Consider following example. We define a struct FooObject and declare field HPyCallFunction call_func which will be used to store the call function’s pointer. We need to register the offset of that field with member __vectorcalloffset__ and in the constructor Foo_new, we assign the call function Foo_call_func.

typedef struct {
    void *a;
    HPyCallFunction call_func;
    void *b;
} FooObject;
HPyType_HELPERS(FooObject)
HPyDef_MEMBER(Foo_call_func_offset, "__vectorcalloffset__", HPyMember_HPYSSIZET,
                  offsetof(FooObject, call_func), .readonly=1)

HPyDef_CALL_FUNCTION(Foo_call_func)
static HPy
Foo_call_func_impl(HPyContext *ctx, HPy self, const HPy *args, size_t nargs,
                   HPy kwnames)
{
    return HPyUnicode_FromString(ctx,
            "hello manually initialized call function");
}

HPyDef_SLOT(Foo_new, HPy_tp_new)
static HPy Foo_new_impl(HPyContext *ctx, HPy cls, const HPy *args,
        HPy_ssize_t nargs, HPy kw)
{
    FooObject *data;
    HPy h_obj = HPy_New(ctx, cls, &data);
    if (HPy_IsNull(h_obj))
        return HPy_NULL;
    data->call_func = Foo_call_func;
    return h_obj;
}

Note

In contrast to CPython’s vectorcall protocol, nargs will never have flag PY_VECTORCALL_ARGUMENTS_OFFSET set. It will only be the positional argument count.

Incremental Migration to HPy’s Calling Protocol

In order to support incremental migration, HPy provides helper function HPyHelpers_PackArgsAndKeywords() that converts from HPy’s calling convention to CPython’s tp_call calling convention. Consider following example:

// function using legacy 'tp_call' calling convention
static HPy
Pack_call_legacy(HPyContext *ctx, HPy self, HPy args, HPy kwd)
{
    // use 'args' and 'kwd'
    return HPy_Dup(ctx, ctx->h_None);
}

// function using HPy calling convention
HPyDef_SLOT(Pack_call, HPy_tp_call)
static HPy
Pack_call_impl(HPyContext *ctx, HPy self, const HPy *args, size_t nargs,
          HPy kwnames)
{
    HPy args_tuple, kwd;
    HPy result;
    if (!HPyHelpers_PackArgsAndKeywords(ctx, args, nargs, kwnames,
             &args_tuple, &kwd)) {
        return HPy_NULL;
    }
    result = Pack_call_legacy(ctx, self, args_tuple, kwd);
    HPy_Close(ctx, args_tuple);
    HPy_Close(ctx, kwd);
    return result;
}

In this example, args, nargs, and kwnames are used to create a tuple of positional arguments args_tuple and a keyword arguments dictionary kwd.

PyModule_AddObject

PyModule_AddObject is replaced with a regular HPy_SetAttr_s(). There is no HPyModule_AddObject function because it has an unusual refcount behavior (stealing a reference but only when it returns 0).

Deallocator slot Py_tp_dealloc

Py_tp_dealloc essentially becomes HPy_tp_destroy. The name intentionally differs because there are major differences: while the slot function of Py_tp_dealloc receives the full object (which makes it possible to resurrect it) and while there are no restrictions on what you may call in the C API deallocator, you must not do that in HPy’s deallocator.

The two major restrictions apply to the slot function of HPy_tp_destroy:

  1. The function must be thread-safe.

  2. The function must not call into the interpreter.

The idea is, that HPy_tp_destroy just releases native resources (e.g. by using C lib’s free function). Therefore, it only receives a pointer to the object’s native data (and not a handle to the object) and it does not receive an HPyContext pointer argument.

For the time being, HPy will support the HPy_tp_finalize slot where those tight restrictions do not apply at the (significant) cost of performance.

Special slots Py_tp_methods, Py_tp_members, and Py_tp_getset

There is no direct replacement for C API slots Py_tp_methods, Py_tp_members, and Py_tp_getset because they are no longer needed. Methods, members, and get/set descriptors are specified flatly together with the other slots, using the standard mechanisms of HPyDef_METH, HPyDef_MEMBER, and HPyDef_GETSET. The resulting HPyDef structures are then accumulated in HPyType_Spec.defines.

Creating lists and tuples

The C API way of creating lists and tuples is to create an empty list or tuple object using PyList_New(n) or PyTuple_New(n), respectively, and then to fill the empty object using PyList_SetItem / PyList_SET_ITEM or PyTuple_SetItem / PyTuple_SET_ITEM, respectively.

This is in particular problematic for tuples because they are actually immutable. HPy goes a different way and provides a dedicated builder API to avoid the (temporary) inconsistent state during object initialization.

Long story short, doing the same in HPy with builders is still very simple and straight forward. Following an example for creating a list:

PyObject *list = PyList_New(5);
if (list == NULL)
    return NULL; /* error */
PyList_SET_ITEM(list, 0, item0);
PyList_SET_ITEM(list, 1, item0);
...
PyList_SET_ITEM(list, 4, item0);
/* now 'list' is ready to use */

becomes

HPyListBuilder builder = HPyListBuilder_New(ctx, 5);
HPyListBuilder_Set(ctx, builder, 0, h_item0);
HPyListBuilder_Set(ctx, builder, 1, h_item1);
...
HPyListBuilder_Set(ctx, builder, 4, h_item4);
HPy h_list = HPyListBuilder_Build(ctx, builder);
if (HPy_IsNull(h_list))
    return HPy_NULL; /* error */

Note

In contrast to PyList_SetItem, PyList_SET_ITEM, PyTuple_SetItem, and PyTuple_SET_ITEM, the builder functions HPyListBuilder_Set() and HPyTupleBuilder_Set() are NOT stealing references. It is necessary to close the passed item handles (e.g. h_item0 in the above example) if they are no longer needed.

If an error occurs during building the list or tuple, it is necessary to call HPyListBuilder_Cancel() or HPyTupleBuilder_Cancel(), respectively, to avoid memory leaks.

For details, see the API reference documentation Building tuples and lists.

Buffers

The buffer API in HPy is implemented using the HPy_buffer struct, which looks very similar to Py_buffer (refer to the CPython documentation for the meaning of the fields):

typedef struct {
    void *buf;
    HPy obj;
    HPy_ssize_t len;
    HPy_ssize_t itemsize;
    int readonly;
    int ndim;
    char *format;
    HPy_ssize_t *shape;
    HPy_ssize_t *strides;
    HPy_ssize_t *suboffsets;
    void *internal;
} HPy_buffer;

Buffer slots for HPy types are specified using slots HPy_bf_getbuffer and HPy_bf_releasebuffer on all supported Python versions, even though the matching PyType_Spec slots, Py_bf_getbuffer and Py_bf_releasebuffer, are only available starting from CPython 3.9.

Multi-phase Module Initialization

HPy supports only multi-phase module initialization (PEP 451). This means that the module object is typically created by interpreter from the HPyModuleDef specification and there is no “init” function. However, the module can define one or more HPy_mod_exec slots, which will be executed just after the module object is created. Inside the code of those slots, one can usually perform the same initialization as before.

Example of legacy single phase module initialization that uses Python/C API:

static struct PyModuleDef mod_def = {
    PyModuleDef_HEAD_INIT,
    .m_name = "legacyinit",
    .m_size = -1
};

PyMODINIT_FUNC
PyInit_legacyinit(void)
{
    PyObject *mod = PyModule_Create(&mod_def);
    if (mod == NULL) return NULL;

    // Some initialization: add types, constants, ...

    return mod;
}

The same code structure ported to HPy and multi-phase module initialization:

HPyDef_SLOT(my_exec, HPy_mod_exec)
int my_exec_impl(HPyContext *ctx, HPy mod) {

    // Some initialization: add types, constants, ...

    return 0; // success
}

static HPyDef *Methods[] = {
    &my_exec, // HPyDef_SLOT macro generated `my_exec` for us
    NULL,
};

static HPyModuleDef mod_def = {
    .defines = Methods
};

HPy_MODINIT(hpyinit, mod_def)