Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel Stream File Parsing with ray.io #260

Merged
merged 31 commits into from
Aug 5, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
ebb7c75
Merge remote-tracking branch 'origin/main' into parstream
kmdalton Jun 19, 2024
4ebaea4
add ray dep for parallelization
kmdalton Jun 19, 2024
2c29aa8
add ray as testing dep
kmdalton Jun 19, 2024
08ba5cd
fix up docstrings
kmdalton Jun 19, 2024
315fccb
add proper capitalization, fix dtypes in A / eV
kmdalton Jun 19, 2024
9418c3b
use Union instead of |
kmdalton Jun 19, 2024
db9bfaa
Merge branch 'main' of ssh://github.com/rs-station/reciprocalspaceshi…
kmdalton Jun 19, 2024
a2587c9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 19, 2024
7007266
maybe fix build issue
Jun 19, 2024
8dbeead
remove unused imports, attr
Jun 21, 2024
e928b9a
add average cell method
Jun 21, 2024
6a37e86
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 21, 2024
c784115
fix wavelength for empty peak lists
Jun 24, 2024
88f1caa
fix numpy v1 support
Jun 24, 2024
df663cd
add numpy v1 to matrix
Jun 24, 2024
69ed824
fix numpy versioning
Jun 24, 2024
3b4f4b2
Merge branch 'npv1' of ssh://github.com/rs-station/reciprocalspaceshi…
Jun 24, 2024
0f34c9d
pass through kwargs to ray.init
kmdalton Jun 27, 2024
d5913b9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 27, 2024
c11e99f
enable read only stream parsing
Jul 19, 2024
0f1f547
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2024
b5729e6
remove unused imports
Jul 19, 2024
84b83ad
fix column spec, add option to disable ray, make num_cpus explicit
Jul 19, 2024
612ef0a
test with and without ray
Jul 19, 2024
dff3c28
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2024
a1e44a3
add error message
Jul 19, 2024
a03780a
clean up dataset creation
Jul 19, 2024
1082a92
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2024
f1770ac
remove unused variable
Aug 2, 2024
33b57be
removed unused timing stuff
Aug 2, 2024
ec6ad73
removed unused DataSeries
Aug 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
numpy-version: ['numpy<2.0', 'numpy>=2.0']
python-version: ['3.9', '3.10', '3.11']

# Skip CI if 'skip ci' is contained in latest commit message
Expand All @@ -23,15 +24,17 @@ jobs:
steps:

- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
- name: Set up Python ${{ matrix.python-version }} and ${{ matrix.numpy-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install "${{ matrix.numpy-version }}"
pip install -e .
pip install ray

- name: Test with pytest
run: |
Expand Down
6 changes: 5 additions & 1 deletion reciprocalspaceship/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -1234,7 +1234,11 @@ def hkl_to_asu(self, inplace=False, anomalous=False):
# Compute new HKLs and phase shifts
hkls = dataset.get_hkls()
compressed_hkls, inverse = np.unique(hkls, axis=0, return_inverse=True)
inverse = inverse.squeeze(-1)

# The behavior of np.unique changed with v2.0. This block maintains v1 compatibility
if inverse.shape[-1] == 1:
inverse = inverse.squeeze(-1)

asu_hkls, isym, phi_coeff, phi_shift = hkl_to_asu(
compressed_hkls, dataset.spacegroup, return_phase_shifts=True
)
Expand Down
Loading
Loading