Skip to content

Commit

Permalink
Merge pull request #135 from regro-cf-autotick-bot/2.9.0_hbcf50c
Browse files Browse the repository at this point in the history
ray-packages v2.9.0
  • Loading branch information
mattip authored Jan 13, 2024
2 parents 255a794 + 7ceb365 commit 0aef696
Show file tree
Hide file tree
Showing 19 changed files with 241 additions and 425 deletions.
61 changes: 0 additions & 61 deletions .azure-pipelines/azure-pipelines-win.yml

This file was deleted.

21 changes: 0 additions & 21 deletions .ci_support/win_64_python3.10.____cpython.yaml

This file was deleted.

21 changes: 0 additions & 21 deletions .ci_support/win_64_python3.11.____cpython.yaml

This file was deleted.

21 changes: 0 additions & 21 deletions .ci_support/win_64_python3.8.____cpython.yaml

This file was deleted.

21 changes: 0 additions & 21 deletions .ci_support/win_64_python3.9.____cpython.yaml

This file was deleted.

25 changes: 23 additions & 2 deletions .gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 0 additions & 6 deletions .scripts/build_steps.sh

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions .scripts/run_docker_build.sh

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 0 additions & 28 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion azure-pipelines.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 6 additions & 24 deletions ray-patching.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,23 +47,10 @@ Rebase to new version or ray:
* remove all patches in the `patches` folder
* copy patches produced by `git format-patch` over to recipe and update `meta.yaml`
## How to prepare or adapt patches for third party Ray components (using Redis as the example)
## How to prepare the 0001-patch-redis-for-ar-ranlib.patch
The above process is complicated by the fact that ray has patches for third party
dependencies (like redis) in its source code - in other words, we need to patch
the patches.
The process we just described then repeats for each dependency we need to patch
more or less as-is, with the exception that we now need to check in the patches
to third-party code into the ray source-code, and include _that_ in the ray-patches
we check into the feedstock. For added fun, it may happen that upstream ray changes
their patches from version to version in ways that conflict with ours.
Since patch application within the ray build is done by bazel (and may be subject
to destructive options like `-pN`, which changes the file hierarchy level the
patches apply to), it's best - for single patches at least - to extend the
ray patch directly to suit our purposes. This makes it also much clearer (well...)
when reviewing the diff(-of-the-diff-of-the-diff) on the feedstock.
The above process is complicated by the fact that ray has a patch for redis in
its source code. We want to add another patch on top of this.
### Prepare sources
1. Get Ray repository (using Ray 2.1.0 for example)
Expand All @@ -89,18 +76,13 @@ git commit -m 'Upstream Ray patches'
```
### Make changes
1. Make new branch and make the necessary changes (maybe adapting existing conda recipe patches, etc.)
```sh
# do the necessary changes - edit files, etc.
git commit -m 'Fix the issue'
# if ray carries a single commit, squash it (and our changes) into one by interactive rebasing
git rebase -i 7.0.5
```
1. Make new branch 7.0.5-patched2 and make the necessary changes (maybe
adapting existing conda recipe patches, etc.)
2. Prepare the patch for Redis in Ray sources (note that if the patch application
in bazel does not have a `-p1`, the call to `git format-patch` below also
needs a `--no-prefix` option):
```
git format-patch 7.0.5-patched..7.0.5 --stdout > ../ray/thirdparty/patches/redis-quiet.patch
git format-patch 7.0.5-patched2..7.0.5-patched --stdout > ../ray/thirdparty/patches/new-name.patch
```
3. Commit the patch for Redis:
```sh
Expand Down
12 changes: 9 additions & 3 deletions recipe/build-core.bat
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
cd python
echo on
set SKIP_THIRDPARTY_INSTALL=1
set IS_AUTOMATED_BUILD=1
Expand All @@ -15,10 +14,17 @@ echo dir %BAZEL_VC%
dir "%BAZEL_VC%"

echo ==========================================================
echo calling pip install
echo calling bash to build
echo ==========================================================

"%PYTHON%" -m pip install . -vv
rem cd python
rem "%PYTHON%" -m pip install . -vv

powershell ci/pipeline/fix-windows-bazel.ps1
%BAZEL_SH% echo "startup --output_user_root=c:/tmp" >> ~/.bazelrc
%BAZEL_SH% ci/ci.sh init
%BAZEL_SH% ci/ci.sh build

rem remember the return code
set RETCODE=%ERRORLEVEL%

Expand Down
14 changes: 7 additions & 7 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
{% set version = "2.8.1" %}
{% set version = "2.9.0" %}

package:
name: ray-packages
version: {{ version }}

source:
url: https://github.com/ray-project/ray/archive/ray-{{ version }}.tar.gz
sha256: 371bfec66dbc41f6a50930fdc7c7793308b683f320fb2e1916ba3590284e2c23
sha256: bebe4413dd1a856cd854308d76d14fb412727b6228bfbdea5f3b5a45ef3f5acc
patches:
- patches/0001-Redis-deps-now-build-but-do-not-link.patch
- patches/0001-patch-redis-for-ar-ranlib.patch
- patches/0002-Disable-making-entry-scripts.patch
- patches/0003-Ignore-warnings-in-event.cc-and-logging.cc.patch
- patches/0004-Remove-all-dependencies-from-setup.py.patch
- patches/0005-ci-remove-boost-as-a-dependency-42226.patch

build:
number: 1
number: 0
skip: true # [win]

# Need these up here for conda-smithy to handle them properly.
requirements:
Expand Down Expand Up @@ -77,7 +79,6 @@ outputs:
- frozenlist
- jsonschema
- msgpack-python >=1.0.0,<2.0.0
- numpy >=1.20
- packaging
- protobuf >=3.15.3,!=3.19.5
- psutil
Expand Down Expand Up @@ -121,7 +122,6 @@ outputs:
run:
- python
- {{ pin_subpackage('ray-core', exact=True) }}
- numpy >=1.20
- pandas >=1.3
- pyarrow >=6.0.1
- fsspec
Expand Down Expand Up @@ -153,7 +153,7 @@ outputs:
- async-timeout
- colorful
- gpustat >=1.0.0
- grpcio >=1.50,<1.59
- grpcio
- opencensus
- prometheus_client >=0.7.1
- py-spy >=0.2.0
Expand Down
Loading

0 comments on commit 0aef696

Please sign in to comment.