Estimating the memory needed for ESMF_RegridWeightGen

[davidhassell]

Requirements

• Reviewed ESMF Reference Manual
• Searched GitHub Discussions

Affiliation(s)

NCAS

ESMF Version

v8.8.0

Issue

Hello,

Is it possible to estimate physical memory required for a weights calculation?

I'm using

ESMF_RegridWeightGen -i --ignore_degenerate -s src.nc  -d dst.nc  -m bilinear -w w.nc

where src.nc is 768 x 768 = 589,824 grid points, and dst.nc is 43200 x 4200 = 181,440,000 grid points.

I have attempted to run this on 3072 PEs where each of the 48 groups of 64 PEs has shared access to 512 GB of RAM, giving a total of ~24 TB, but I'm still getting out-of-memory errors.

It's wholly possible that my parallelised set up is wrong (!), but to try to diagnose if that's the case, or if I just need more resources, it would be useful to know what the memory requirement ought to be.

Many thanks,
David

Autotag

@oehmke

[oehmke]

Hi David, That seems like a lot of memory for that case. Would you mind trying it with the argument: -p none ? That turns off some extra extrapolation at the pole which could potentially add more memory. If it gives you an unmapped point error use the flag: -i to turn it off. Let me know if that helps. If not, is there someplace where I can get the files to try (e.g. do you have an account on Derecho)? - Bob

…

On Feb 21, 2025, at 5:46 AM, David Hassell ***@***.***> wrote: Requirements Reviewed ESMF Reference Manual <https://earthsystemmodeling.org/doc/> Searched GitHub Discussions <https://github.com/orgs/esmf-org/discussions?discussions_q=> Affiliation(s) NCAS ESMF Version v8.8.0b0 Issue Hello, Is it possible to estimate physical memory required for a weights calculation? I'm using ESMF_RegridWeightGen -i --ignore_degenerate -s src.nc -d dst.nc -m bilinear -w w.nc where src.nc is 768 x 768 = 589,824 grid points, and dst.nc is 43200 x 4200 = 181,440,000 grid points. I have attempted to run this on 3072 PEs where each of the 48 groups of 64 PEs has shared access to 512 GB of RAM, giving a total of ~24 TB, but I'm still getting out-of-memory errors. It's wholly possible that my parallelised set up is wrong (!), but to try to diagnose if that's the case, or if I just need more resources, it would be useful to know what the memory requirement ought to be. Many thanks, David Autotag @oehmke <https://github.com/oehmke> — Reply to this email directly, view it on GitHub <#357>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7UYZ4A5T2JSWYYQUT2D2Q4NZBAVCNFSM6AAAAABXTE6H4OVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZXHE4TQMZZGU>. You are receiving this because you were mentioned.

[davidhassell]

Hi Bob,

Thanks! Both grids include the south pole (not the north) - I shall try with -p none. I'll let you know how that goes ...

David

[davidhassell]

Hi Bob, still fails on memory. I don't have anywhere handy to put the src and dst files - they're only 8MB - I could upload them to the discussion, if that's OK.

Thanks,
David

[oehmke]

Too bad that didn’t work, but yep, that’s fine, just attach them and I’ll take a look. Thanks, - Bob

…

On Feb 24, 2025, at 6:46 AM, David Hassell ***@***.***> wrote: Hi Bob, still fails on memory. I don't have anywhere handy to put the src and dst files - they're only 8MB - I could upload them to the discussion, if that's OK. Thanks, David — Reply to this email directly, view it on GitHub <#357 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7U4KLQSK4RQGSLRJHVD2RMPDFAVCNFSM6AAAAABXTE6H4OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMZQGA4TSMI>. You are receiving this because you were mentioned.

[davidhassell]

Thank you!

Files: src.nc.gz, dst.nc.gz

with command:

ESMF_RegridWeightGen -p none -i --ignore_degenerate -s src.nc  -d dst.nc  -m bilinear -w w.nc -t CFGRID --netcdf4 --src_regional

David