Planning 1.0 release #275
Open
Planning 1.0 release #275
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Deprecate and remove `exists` and `forall` functions from the codebase. * Remove `exists` and `forall` functions from `quinn/functions.py`. * Remove import statements for `exists` and `forall` from `quinn/__init__.py`. * Remove tests related to `exists` and `forall` functions from `tests/test_functions.py`.
…emove-athena-func Remove the print Athena function
…sts-forall Issue 49 remove exists forall
Related to #237 Remove deprecated extension functions from the codebase * Delete `quinn/extensions/dataframe_ext.py` and `quinn/extensions/spark_session_ext.py` * Remove import statements for `dataframe_ext` and `spark_session_ext` in `quinn/extensions/__init__.py` * Remove per-file ignores for `quinn/extensions/dataframe_ext.py` and `quinn/extensions/__init__.py` in `pyproject.toml` * Delete test files `tests/extensions/test_dataframe_ext.py` and `tests/extensions/test_spark_session_ext.py`
Since the extensions functions have been removed, remove the corresponding package/directory and the `__init__.py` file too.
The following functions have been removed from the init file * print_athena_create_table * exists * forall
* Create the `extensions/` directory. * Create the `__init__.py` file within it with the license header. * Create the `spark_session_ext.py` file within the extensions directory * Update the doc-string in create_df to align with the parameter list. * Format modified files & lint the code using ruff (successful)
As a matter of fact, a create_df function already exists under the dataframe_helpers.py Following are the changes introduced by this commit. * Update `dataframe_helpers.py` by updating the doc-string in the create_df function. * Remove quinn/extensions/ directory along with the two `.py` files involved. * Remove the ruff check from the pyproject.toml. * Finally, format the dataframe_helpers.py with ruff
- Update deps - Update Ruff - Corresponding updates of pyproject - Slightly update Makefile - Drop Support of Spark2 - Drop Support of Spark 3.1-3.2 - Minimal python is 3.10 from now - Apply new ruff rules - Apply ruff linter On branch 202-drop-pyspark2 Changes to be committed: modified: .github/workflows/ci.yml modified: Makefile modified: poetry.lock modified: pyproject.toml modified: quinn/append_if_schema_identical.py modified: quinn/dataframe_helpers.py modified: quinn/keyword_finder.py modified: quinn/math.py modified: quinn/schema_helpers.py modified: quinn/split_columns.py modified: quinn/transformations.py
…emove-extensions Remove the extensions functions
Drop Spark-2 support and update dependencies
This commit introduces the following changes: * Updates the `ci.yml` file by introducing a new step under the `test` job to perform tests using Spark-Connect. * Creates a shell-script that downloads & installs Spark binaries and then runs the Spark-Connect server. * Creates a pytest module/file that tests a very simple function on Spark-Connect. * Updates the Makefile to add a new step for the Spark-Connect tests.
update makefile add make command for ruff
Improve pre-commit hooks and use of ruff
… GitHub Actions local setup
…rk-Connect <3.5.2
This was included to perform the GH Actions on the feature/ branches to speed up the development & test when working on the Spark-Connect tests upgrade.
…ntegrate-sc-tests Integrate the Spark-Connect tests with the existing test suite & CI
…m-makefile Remove linting from Makefile
…262) * Add configuration files for release-please and update contributing guide * **release-please-config.json**: Add configuration file with release type set to `python`, changelog path to `CHANGELOG.md`, and packages to include `quinn`. * **.release-please-manifest.json**: Add manifest file with `quinn` version set to `1.0.0`. * **.github/workflows/release-please.yml**: Add workflow file for release-please action * **CONTRIBUTING.md**: Add a section for the "Release Process" * feat: Update the documentation to elaborate the release process * feat: Update the release GH action to publish the package to PyPI
|
I will work on resolving conflicts and preparing for the merge. |
|
Thanks for merging the previous pull request @SemyonSinchenko . Glad to see the GitHub Actions pipeline working |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Describe the big picture of your changes here to communicate to the maintainers. If it fixes a bug or resolves a feature request, please provide a link to that issue.
Types of changes
What types of changes does your code introduce to quinn?
Put an
xin the boxes that applyFurther comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...