Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

speedup: customize a number od jobs #895

Open
matyaskopp opened this issue Jan 23, 2025 · 2 comments · May be fixed by #894
Open

speedup: customize a number od jobs #895

matyaskopp opened this issue Jan 23, 2025 · 2 comments · May be fixed by #894
Assignees
@matyaskopp
Labels
enhancement New feature or request
Milestone
Future

Comments

@matyaskopp
Copy link
Collaborator

The parlamintp-.*.pl scripts have a hard-coded number of jobs. They should be parametrised to use all the power of larger computers.
@matyaskopp matyaskopp added this to the Future milestone Jan 23, 2025
@matyaskopp matyaskopp self-assigned this Jan 23, 2025
@matyaskopp matyaskopp linked a pull request Jan 23, 2025 that will close this issue
Speedup validation and creating derived formats #894
Open
@matyaskopp matyaskopp added the enhancement New feature or request label Jan 23, 2025
matyaskopp added a commit that referenced this issue Jan 23, 2025
@matyaskopp
customize number of jobs in parallel #895
cc94650
matyaskopp added a commit that referenced this issue Jan 28, 2025
@matyaskopp
add missing getopt ##895
85c4656
matyaskopp added a commit that referenced this issue Jan 28, 2025
@matyaskopp
set halt 0 #895
96df102
matyaskopp added a commit that referenced this issue Jan 30, 2025
@matyaskopp
reduce paralelism, when meta script is run (not enough memory) #895
5ff8fc2
TomazErjavec added a commit that referenced this issue Feb 1, 2025
@TomazErjavec
Parametrise number of parallel jobs (#895).
5645212
@TomazErjavec
Copy link
Collaborator

OK, made the relevant scripts have a parameter with number of jobs (on devel branch).
It was a bit more work, as I had to introduce proper parameters, so also -in and -out for input and output directories.
Hm, but I see only now that this issue has been closed, I hope you haven't done the same, ups.

@matyaskopp matyaskopp reopened this Feb 1, 2025
@matyaskopp
Copy link
Collaborator Author

I did almost similar things in a separate branch; I am now testing it before merging it to develop...
#894

matyaskopp added a commit that referenced this issue Feb 4, 2025
@matyaskopp
reduce paralelism in vert #895
76963a8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matyaskopp matyaskopp

Labels
enhancement New feature or request
Projects
None yet
Milestone
Future
Development

Successfully merging a pull request may close this issue.

Speedup validation and creating derived formats clarin-eric/ParlaMint
2 participants
@matyaskopp @TomazErjavec