Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: CDAT Migration: Add support for opening multiple time series datasets #861

Closed
tomvothecoder opened this issue Oct 1, 2024 · 2 comments
Assignees

Comments

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Oct 1, 2024

Is your feature request related to a problem?

The CDAT codebase supports XML files to open up multiple time series datasets. The cdat-migration-fy24 branch uses xc.open_dataset() for single variables that don't need to be derived.

  • Zppy specifies input paths as directories of .nc files (related comment)
  • There might be some cases where variables are split up across multiple files and need to be concatenated as a single dataset object.

Describe the solution you'd like

Update _get_time_series_dataset_obj() to use xc.open_mfdataset()

ds = xc.open_dataset(
filepath, add_bounds=["X", "Y", "T"], decode_times=True, use_cftime=True
)

Describe alternatives you've considered

No response

Additional context

No response

@forsyth2
Copy link
Collaborator

forsyth2 commented Oct 1, 2024

Zppy specifies input paths as directories of .nc files

Just to elaborate a bit:

  • The core of Remove cdscan for e3sm_diags zppy#598 is the bash file change
  • That takes zppy from A) concatenating variables into a txt file and then using cdscan to create an xml file to B) taking an entire directory of nc files. I.e., "process all the variables in this directory" rather than "process all the variables I listed in this file".
  • This is incompatible with how Diags (on main) currently functions, which is why Remove cdscan for e3sm_diags zppy#598 can't be merged into zppy's main until Diags has completed its CDAT migration.

@tomvothecoder
Copy link
Collaborator Author

Closed by #866

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants