This is a hybrid python-rust package using Maturin/pyo3. The setup based on this minimal example. Heavy calculations are done in PyTorch, for lots of string processing rust with rayon should be used. Everything should have type hints and docstrings for nice developer experience. Rust functions should be wrapped in typed python functions/methods. Tests are all in python with a PyTest suite. A RTD documentation is built using MkDocs.
- python/ python package
- rust/ rust library
- tests/ test suite
- scripts/ some bash scripts for development
- performance/ measuring performance
- docs/ markdown files and visualizations for documentation
- .github/ workflow that builds, tests (fast), releases
maturin develop # quick build, creates python/magicsoup/_lib.*.so
maturin develop --release # build with performance optimizations
bash scripts/test.sh tests/fast # only quick tests
bash scripts/test.sh tests/ # all tests
There is docs/create_figures.py which creates a lot of plots. Some of those plots also serve as sanity checks. E.g. reaction kinetics and biochemical patterns should still make sense, overall energy has to eventually decrease during a simulation, etc. They should be checked before major updates. Also run scripts/check-performance.sh for checking performance of all major simulation steps. For performance-related updates check this on CPU and GPU and also run performance/run.py and look at the times with Tensorboard.
bash scripts/release.sh # creates a tag and pushes it
Update version in pyproject.toml first. If rust files changed also update Cargo.toml to the same version. The build pipeline in .github/ will be triggered by pushing the version tag.
bash scripts/serve-docs.sh # builds and serves documentation
Markdown files and images are in docs/. Docstrings from python are parsed and mounted in docs/reference.md. This is configured in mkdocs.yml. Note that docs/index.md is the index for RTD and README.md is the start page on PyPI. Docs are built for RTD on every push to main. See if docs/create_figures.py or other plots should be recreated.