Infrastructure Proposal: use tox or nox for test executions

[sirosen]

This is a feature request / proposal for changes I would like to make, if the project is open to them.

What

Add noxfile.py, tox.ini, or other config (e.g., tox.toml) to the project, and update documentation to instruct contributors to use nox -s test or tox run to execute tests.
Update CI to also run tests via the designated workflow tool.

Why

As a new contributor looking to get involved in the project, it's not clear exactly how to run the tests. I can dig information out of CI configs and look at the test scripts, but it's non-obvious what the contributor setup should be.
(I did find that uv sync --all-extras; ./scripts/test.sh works)

Furthermore, if we want to test the code under multiple conditions (different interpreters, different extras installed), a matrix-style runner is needed.

Additional Background

Because I'm a long-time tox user, I would be most inclined to tackle this by setting up a tox.ini.
Tox and nox are almost entirely interchangeable, and nox has significant usage as well (e.g., pip's own tests use it).

Both tools also support use of uv as a speedup, e.g. via the tox-uv plugin.

My overall goal in proposing this is twofold:

1. Make local development of isort easier and more pleasant
2. Allow these environment managers to be used to script and automate non-testsuite tasks -- e.g. a new tox run -e benchmark suite to profile performance

---

Okay, that's my proposal. Would this be interesting to the project? I'd start small with a config which only handles the testing.

[DanielNoord]

I see @staticdev gave this as 👍 so I guess they agree with this?

Personally I'm not the biggest fan of tox and nox as I find it hides a lot of useful interaction and commands. I consider it a powerful "skill" to be able to run pytest yourself with the correct environment and arguments. That is something that is also applicable to other projects whereas learning all the intricacies of nox might not be.

That said, doesn't uv cover all of this? It can be told to use a certain interpreter and run certain commands?

[staticdev]

@DanielNoord I like this discussion.. I still think Python is not there yet for a complete tool.
With UV we are still relying on scripts and don't have minimal sessions for each workflow job like tox and NOx provide.
I worked with both tools and also they have pros and cons, still complementing what we have with UV.
Currently nox is a bit more modern and flexible as all the configuration is in python. I am doing some reseache for the lower complexity and maintenance cost we can have in 2025 with available tools. Also worth mentioning https://rye.astral.sh/ and https://github.com/pypa/hatch are also contenders for improving test, linters and other side job executions.