Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support parsing of job options from UDP definition #366

Closed
jdries opened this issue Jan 20, 2025 · 6 comments
Closed

support parsing of job options from UDP definition #366

jdries opened this issue Jan 20, 2025 · 6 comments
Assignees

Comments

@jdries
Copy link
Contributor

jdries commented Jan 20, 2025

Support functionality described here:
Open-EO/openeo-api#471

The goal is to allow UDP's to advertise job options, avoiding failures due to for instance not providing udf dependencies or sufficient python-memory.

@soxofaan soxofaan transferred this issue from Open-EO/openeo-geopyspark-driver Feb 6, 2025
soxofaan added a commit that referenced this issue Feb 6, 2025
soxofaan added a commit that referenced this issue Feb 6, 2025
soxofaan added a commit that referenced this issue Feb 6, 2025
soxofaan added a commit that referenced this issue Feb 6, 2025
… defaults from remote process definitions
soxofaan added a commit that referenced this issue Feb 6, 2025
@soxofaan
Copy link
Member

soxofaan commented Feb 6, 2025

PR #367 is ready I think. I'll give it a final look tomorrow and merge

@soxofaan
Copy link
Member

soxofaan commented Feb 6, 2025

Open-EO/openeo-api#471 was just merged too (after some more review iterations)

@soxofaan
Copy link
Member

soxofaan commented Feb 7, 2025

merged #367 now

@soxofaan
Copy link
Member

before closing (and as proof that it works), when feature is available on a public deployment (e.g. CDSE staging)

  • work out example in context of APEx benchmarks

soxofaan added a commit to ESA-APEx/apex_algorithms that referenced this issue Feb 14, 2025
related to Open-EO/openeo-python-driver#366

And yes, this job option "logging-threshold" is actually deprecated
(Open-EO/openeo-geopyspark-driver#909),
but this is just proof-of-concept
soxofaan added a commit to ESA-APEx/apex_algorithms that referenced this issue Feb 14, 2025
@soxofaan
Copy link
Member

soxofaan commented Feb 14, 2025

Usage example in APEX:

https://github.com/ESA-APEx/apex_algorithms/blob/11c27fb1a90cfc8f2eb285b881d1db8a96c358f4/openeo_udp/examples/max_ndvi/max_ndvi.json#L190-L195

Here I added these default job options to the UDP JSON doc:

  "default_job_options": {
    "logging-threshold": "info"
  },
  "default_synchronous_options": {
    "logging-threshold": "warning"
  }

When this UDP is executed, these default job options will be used (unless explicitly overridden)

@soxofaan
Copy link
Member

the official spec/doc of the processing parameters extension also has an example by the way: https://github.com/Open-EO/openeo-api/tree/draft/extensions/processing-parameters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants