Regridding performance with no MPI

[AbhiHampiholi]

Requirements

• Reviewed ESMF Reference Manual
• Searched GitHub Discussions

Affiliation(s)

No response

ESMF Version

8.6.0

Issue

Hello,
I am trying to use ESMC to regrid a 2 dimensional grid of floating point numbers from a source grid size of 1440 x 720 to a destination grid of size 360 × 240. I am using the ESMF_COMM=mpiuni option and the regridding algorithm is bilinear. This takes about 3 minutes with almost all of that time in the ESMC_FieldRegridStore method. The output grid looks correct.

ESMC_Initialize(NULL, ESMC_InitArgLogKindFlag(ESMC_LOGKIND_NONE), ESMC_ArgLast);
//create src grid using ESMC_GridCreateNoPeriDim - add two coordinates to it (latitude and longitude) and fill it up
//create dest grid - fill it up 
//create src field using ESMC_FieldCreateGridTypeKind - fill up src field
//create dest field
//regrid between srcfield and destfield using ESMC_FieldRegridStore to generate the weights and ESMC_FieldRegrid to apply them
//read from destField to get "regridded data"
ESMC_FieldDestroy(&srcfield);
ESMC_FieldDestroy(&destfield);
ESMC_GridDestroy(&srcgrid);
ESMC_GridDestroy(&destgrid);
ESMC_Finalize();

I am running this on a machine with 32 GB of RAM and 3 minutes seems like too long for bilinear interpolation on a rectilinear grid. Is this expected? And any way to find if I am doing something wrong would be helpful.

Thanks!

Autotag

@oehmke

[oehmke]

That does sound longer than I would expect. Was the ESMF library compiled in optimized mode (i.e. with ESMF_BOPT set to O)? What machine is this on? Thanks!

[AbhiHampiholi]

Yes, the library is compiled with ESMF_BOPT=O and I'm running on a Lenovo Thinkpad, 11th Gen Intel i9 @ 2.60 GHz and 8 cores. I'm expecting this regridding operation to take less than a second (since that is how long a simple implementation of bilinear interpolation takes on these array sizes). Is my expectation off?

[shailesh1210]

@oehmke I have had a similar issue when calling ESMC_FieldRegridStore as @AbhiHampiholi has pointed out. I have a source grid of size {4064, 6844} and destination grid of size {3501, 7001}. It took almost ~800 secs and used up ~ 60 GB of memory. It was run Intel(R) Xeon(R) Platinum 8488C machine with 252 GB of memory. However, when I used the ESMF's CLI tool (ESMF_RegridWeightGen) to generate weights, the memory usage didn't exceed 20 GB. Has this been addressed yet?

[oehmke]

Sorry, not yet. We entered a release phase and had some emergency issues come up, so I haven't had a chance to circle back to this yet. That's interesting that ESMF_RegridWeightGen would take so much less memory since they both go through the same interface underneath. Let me take a look and get back.

I saw that you also made a support ticket, but since you made this discussion issue first, I'll follow it here. Let me know if you'd rather the reverse.

Would you mind sending the grid files that you used for ESMF_RegridWeightGen? Just send them with the same subject as you sent the support email. Thanks!

[shailesh1210]

@oehmke I wouldn't be able to send the grid files due to proprietary reasons. I have ran a system test posted here - https://github.com/esmf-org/esmf/blob/develop/src/system_tests/ESMC_FieldRegrid/ESMC_FieldRegridSTest.c - with the same grid sizes from my use case (for both source and destination grid), and memory usage topped ~ 70GB. Please let me know if you are able to reproduce it. Here's the screenshot from that run -