Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Cython's freethreading_compatible directive #35

Merged
merged 4 commits into from
Jul 16, 2024
Merged

Document Cython's freethreading_compatible directive #35

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2f787b9
Document Cython's freethreading_compatible directive
lysnikolaou Jul 15, 2024
cd2ba23
Address feedback
lysnikolaou Jul 15, 2024
a94ec7d
Add build examples
lysnikolaou Jul 16, 2024
51c813c
Address feedback; don't use Ellipsis & use add_project_arguments
lysnikolaou Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 61 additions & 6 deletions docs/porting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,72 @@ Extension modules need to explicitly indicate they support running with the GIL
disabled, otherwise a warning is printed and the GIL is re-enabled at runtime
after importing a module that does not support the GIL.

!!! note

Currently it is not possible for extensions written in Cython to declare
they support running without the GIL. Work is under way to add support
(see [cython#6242](https://github.com/cython/cython/pull/6242)).

C++ extension modules making use of `pybind11` can easily declare support for
running with the GIL disabled via the
[`gil_not_used`](https://pybind11.readthedocs.io/en/stable/reference.html#_CPPv4N7module_23create_extension_moduleEPKcPKcP10module_def16mod_gil_not_used)
argument to `create_extension_module`.

Starting with Cython 3.1.0 (only available via the nightly wheels or the `master`
branch as of right now), extension modules written in Cython can also do so using the
[`freethreading_compatible`](https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#compiler-directives)
compiler directive. It can be enabled either per module as a directive
(`# cython: freethreading_compatible=True`) in `.pyx` files, or globally by adding
`-Xfreethreading_compatible=True` to the Cython arguments via the project's
build system.
lysnikolaou marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's some example code to do this via build configuration with setuptools and meson:

compiler_directives = {'freethreading_compatible': True}

setup(
    ext_modules=cythonize(
        extensions,
        compiler_directives=compiler_directives)
)
cython_args = []
if cy.version().version_compare('>=3.1.0')
  cython_args += ['-Xfreethreading_compatible=True']
endif

Meson compiler versions don't include alpha or beta qualifiers, so no need to check that for cython.

For the setuptools one, you could note that you can use e.g. packaging.version to check for a minimum Cython version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of including example build config code - that'll definitely be helpful for users.

The Meson code should include an extension_module call as well I think, something like:

py.extension_module('modulename'
    'source.pyx',
    cython_args: cython_args,
    ...
)

Even nicer if we had a CMake/scikit-build-core version of it as well. Maybe @henryiii could suggest an idiomatic snippet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks both for the suggestions! I added setuptools and Meson examples, but I don't know enough about CMake/scikit-build-core to do that as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re scikit-build-core, I think this is a good example: https://github.com/scikit-build/scikit-build-sample-projects/tree/main/projects/hello-free-threading (or the hello-cython next to it). However, it doesn't have the code yet to mark extension modules as compatible and I'm not sure what the idiomatic way to do it is, so let's do this as a follow-up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be better once we finish scikit-build/cython-cmake#19. Will try to do that this week and then update here.

Here are a few examples of how to globally enable the directive in a few popular
build systems:

=== "setuptools"

When using setuptools, you can pass the `compiler_directives` keyword argument
to `cythonize`:

```python
from Cython.Compiler.Version import version as cython_version
from packaging.version import Version

compiler_directives = {...}
lysnikolaou marked this conversation as resolved.
Show resolved Hide resolved
if Version(cython_version) >= Version("3.1.0a1"):
compiler_directives["freethreading_compatible"] = True

setup(
ext_modules=cythonize(
extensions,
compiler_directives=compiler_directives,
)
)
```

=== "Meson"

When using Meson, you can add the directive to the `cython_args` you're
passing to `py.extension_module`:

```meson
cy = meson.get_compiler('cython')

cython_args = [...]
lysnikolaou marked this conversation as resolved.
Show resolved Hide resolved
if cy.version().version_compare('>=3.1.0')
cython_args += ['-Xfreethreading_compatible=True']
endif

py.extension_module('modulename'
'source.pyx',
cython_args: cython_args,
...
)
```

You can also globally add the directive for all Cython extension modules:

```meson
cy = meson.get_compiler('cython')
if cy.version().version_compare('>=3.1.0')
add_global_arguments('-Xfreethreading_compatible=true', language : 'cython')
lysnikolaou marked this conversation as resolved.
Show resolved Hide resolved
endif
```

C or C++ extension modules using multi-phase initialization can specify the
[`Py_mod_gil`](https://docs.python.org/3.13/c-api/module.html#c.Py_mod_gil)
module slot like this:
Expand Down