Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: use pyproject.toml to declare configs #112

Merged
merged 13 commits into from
Mar 4, 2025

Conversation

jsstevenson
Copy link
Contributor

@jsstevenson jsstevenson commented Feb 26, 2025

  • incorporate build configs into pyproject.toml. This is the PyPA's recommended declaration model and lets you consolidate a couple of files and settings into one place.
  • use setuptools-scm to define versioning. Rather than manually updating a version number in code (which can be easy to forget), you increment the version by making a new release on GitHub, and the package gets versioning from there. This is nice because it also automatically creates a changelog and static copies of released code. There are some good integrations from here as well.
  • add a GitHub actions release workflow. This automates the twine/setuptools build process for uploading releases to PyPI, and is triggered on new GitHub releases. There's a small amount of setup to do on the PyPI side (I think it amounts to clicking a box once, @korikuzma would've done it most recently) but once it's ready, it just works and is super convenient.
  • small precommit tweaks. update hooks version (silences a warning), pick a line ending standard, set a minimum precommit version.
  • add some style and quality checks to GitHub Actions, which can run every time a PR is made or pushed to:
    • A stub that tests that the package installs properly. Next, I'd like to build this out into full fledged unit tests that can run in CI.
    • Style checks (black/flake8). If yall are up for it, I'd propose moving this over to Ruff in a future commit, it runs much faster (nice for local development) and can do a more expansive set of checks. I've actually learned a lot about modern Python features from Ruff's linting.
    • A subset of precommit checks. We've found that some less-regular contributors will forget to install precommit locally and will inadvertently introduce stuff like trailing whitespace or mixed line endings into the codebase, so it's nice to run these checks against incoming PRs.

@jsstevenson jsstevenson marked this pull request as ready for review February 28, 2025 17:00
@jsstevenson jsstevenson requested a review from quinnwai February 28, 2025 17:00
Copy link
Collaborator

@quinnwai quinnwai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just minor comments. Checks on the PR and commit look good to me.

One question: could you do a random pypi release now to ensure that the setuptools-scm or whatnot works as expected? Or do we have to do this after this PR gets merged

run: |
python3 -m pip install ".[tests]"

# TODO -- build CI-ready unit tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preempted my thoughts here: would it be an easy win to add the current unit tests (pytest tests/unit) to the CI in this PR? Or were you considering it out of scope for the current PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think too hard about how nontrivial it would be to get these working, but I think there's still some configuration that would have to happen for the CI instance to be able to run them. Here's an example run trying every test: https://github.com/gks-anvil/vrs_anvil_toolkit/actions/runs/13637983594/job/38121227703

Copy link
Collaborator

@quinnwai quinnwai Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay fair enough, we can make it out of scope then. I think the main problem is the supposed "unit tests" we created are pointing to an already downloaded seqrepo dir when testing occurs. Not sure how you do it in vrs-python but would be inclined to do it that same way.

Copy link
Contributor Author

@jsstevenson jsstevenson Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah. For SeqRepo, we use the REST dataproxy in tests and then use something like pytest vcr to capture + stash the HTTP requests so that they can be used during CI/CD. It's a little painful but it works. I can make another issue to work that out.

@jsstevenson
Copy link
Contributor Author

jsstevenson commented Mar 3, 2025

One question: could you do a random pypi release now to ensure that the setuptools-scm or whatnot works as expected? Or do we have to do this after this PR gets merged

Yeah good question @quinnwai . We've been using this exact workflow in all of our libs for a few years and it's held pretty steady, I took it straight from our template repo https://github.com/GenomicMedLab/software-templates/blob/main/python/%7B%7Bcookiecutter.project_slug%7D%7D/.github/workflows/release.yaml. See eg https://github.com/GenomicMedLab/cool-seq-tool/actions/workflows/release.yml

We can do a partial test right now. Just for kicks, I made a new release from the front of this branch. You can see progress here: https://github.com/gks-anvil/vrs_anvil_toolkit/actions/runs/13639823319. Basically, it builds successfully, but hits an error why submitting to PyPI because GitHub hasn't been added as a trusted publisher on the PyPI side yet.

Before it can completely run, someone with management access to the project on PyPI needs to do a few configurations: https://docs.pypi.org/trusted-publishers/adding-a-publisher/. We could briefly talk about this at the meeting tomorrow.

Copy link
Collaborator

@quinnwai quinnwai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, created a trusted publisher and tested that the PyPi release works as expected. Ping me your username I can add you to the admin to the pypi as well

🚀

@jsstevenson jsstevenson merged commit 27ead76 into development Mar 4, 2025
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants