Remove the eomg job #3331

DavidGrumm-NOAA · 2025-02-14T19:22:42Z

Description

The eomg job is no longer supported for the GSI and the GDASApp has its own jobs to handle observational diagnostics (i.e. when lobsdiag=".true."): atmensanlobs and atmensanlsol. Thus, the eomg job will be removed from the global workflow without affecting the new atmensanlobs and atmensanlsol jobs.

This PR resolves issue #2525.

Type of change

Bug fix (fixes something broken)
New feature (adds functionality)
Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? YES/NO (If YES, please add a link to any PRs that are pending.)
- [NO] EMC verif-global
- [NO] GDAS
- [NO] GFS-utils
- [NO] GSI
- [NO] GSI-monitor
- [NO] GSI-utils
- [NO] UFS-utils
- [NO] UFS-weather-model
- [NO] wxflow

How has this been tested?

Checklist

Any dependent changes have been merged and published
[X ] My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have documented my code, including function, input, and output descriptions
My changes generate no new warnings
New and existing tests pass with my changes
This change is covered by an existing CI test or a new one has been added
Any new scripts have been added to the .github/CODEOWNERS file with owners
I have made corresponding changes to the system documentation if necessary

DavidGrumm-NOAA · 2025-02-18T19:21:36Z

@WalterKolczynski-NOAA @DavidHuber-NOAA @aerorahul : this PR is ready for review. There are some differences between the baseline output and the updated code output, which is unexpected. @DavidHuber-NOAA has suggested we go ahead with this PR and then rerun the test once approvals are made.

DavidHuber-NOAA

In addition to the comments, there are some additional places where eomg needs to be removed. Specifically:

parm/config/gfs/config.resources*
parm/config/gfs/config.eobs (this config was used by eomg and there are a couple of eomg-specific variables)
parm/archive/enkf.yaml.j2

Also, there is still a mention of eomg in gfs_tasks.py (under the eupd method).

DavidHuber-NOAA · 2025-02-18T19:38:48Z

workflow/applications/gfs_cycled.py

-            configs += ['ecen', 'esfc', 'efcs', 'echgres', 'epos', 'earc_vrfy']
-
-            if options['do_archtar']:
-                configs += ['earc_tars']
+            configs += ['ecen', 'esfc', 'efcs', 'echgres', 'epos', 'earc']


I think this should be reverted.

Suggested change

configs += ['ecen', 'esfc', 'efcs', 'echgres', 'epos', 'earc_vrfy']

if options['do_archtar']:

configs += ['earc_tars']

configs += ['ecen', 'esfc', 'efcs', 'echgres', 'epos', 'earc']

configs += ['ecen', 'esfc', 'efcs', 'echgres', 'epos', 'earc_vrfy']

if options['do_archtar']:

configs += ['earc_tars']

DavidHuber-NOAA · 2025-02-18T19:40:52Z

workflow/applications/gfs_cycled.py

+                    if options['lobsdiag_forenkf']:
+                        task_names[run] += ['ediag']


I think that we should just assume we are always running ediag and ignore the lobsdiag_forenkf flag when running GSI-based analyses (i.e. when do_jediatmens = False).

DavidHuber-NOAA · 2025-02-18T20:03:36Z

Also, there are some checks for lobsdiag_forenkf in the following files that should be checked. If these checks are for JEDI-based analyses, then they're fine, otherwise the code should be simplified to remove the contents inside of the lobsdiag_forenkf logical blocks. Just a reminder, eomg used to be triggered when lobsdiag_forenkf = .false.

parm/archive/enkf_restarta_grp.yaml.j2
parm/archive/enkf_grp.yaml.j2
parm/config/gfs/config.anal
scripts/exgdas_enkf_update.sh
scripts/exglobal_atmos_analysis.sh

DavidHuber-NOAA · 2025-02-18T20:39:11Z

@DavidGrumm-NOAA Per our discussion, please just remove eomg and lobsdiag references from GSI-specific files and not those files that are shared by both the GSI and GDASApp. Specifically, please target these files:

parm/config/gfs/config.anal
scripts/exgdas_enkf_update.sh
scripts/exglobal_atmos_analysis.sh
parm/config/gfs/config.resources
parm/config/gfs/config.resources.HERCULES
parm/config/gfs/config.eobs (this config was used by eomg and there are a couple of eomg-specific variables)

…eomg_1

DavidHuber-NOAA

Overall it looks pretty good. I just have a couple of suggestions on the namelists since we cannot control the default values of lobsdiag_forenkf and one other place in the setup scripts that needs the flag removed.

DavidHuber-NOAA · 2025-02-27T14:06:12Z

scripts/exgdas_enkf_update.sh

@@ -288,7 +275,7 @@ cat > enkf.nml << EOFnml
   univaroz=.false.,adp_anglebc=.true.,angord=4,use_edges=.false.,emiss_bc=.true.,
   letkf_flag=${letkf_flag},nobsl_max=${nobsl_max},denkf=${denkf},getkf=${getkf}.,
   nhr_anal=${IAUFHRS_ENKF},nhr_state=${IAUFHRS_ENKF},
-   lobsdiag_forenkf=${lobsdiag_forenkf},taperanalperts=${taperanalperts},
+   taperanalperts=${taperanalperts},


An lobsdiag_forenkf entry may still be required when writing the EnKF namelist. It depends on whether the EnKF initializes lobsdiag_forenkf to. I'd suggest just setting it to a constant .true..

Suggested change

taperanalperts=${taperanalperts},

lobsdiag_forenkf=.true.,taperanalperts=${taperanalperts},

DavidHuber-NOAA · 2025-02-27T14:07:53Z

scripts/exglobal_atmos_analysis.sh

@@ -723,7 +722,6 @@ cat > gsiparm.anl << EOF
  diag_precon=.true.,step_start=1.e-3,emiss_bc=.true.,nhr_obsbin=${nhr_obsbin:-3},
  cwoption=3,imp_physics=${imp_physics},lupp=${lupp},cnvw_option=${cnvw_option},cao_check=${cao_check},
  netcdf_diag=${netcdf_diag},binary_diag=${binary_diag},
-  lobsdiag_forenkf=${lobsdiag_forenkf},
  write_fv3_incr=${write_fv3_increment},


Same here, lobsdiag_forenkf may still be required in the analysis namelist:

Suggested change

write_fv3_incr=${write_fv3_increment},

lobsdiag_forenkf=.true.,

write_fv3_incr=${write_fv3_increment},

DavidHuber-NOAA · 2025-02-27T14:10:19Z

workflow/rocoto/gfs_tasks.py

        if self.options['lobsdiag_forenkf']:
            dep_dict = {'type': 'task', 'name': f'{self.run}_ediag'}
-        else:
-            dep_dict = {'type': 'metatask', 'name': f'{self.run}_eomg'}
-        deps.append(rocoto.add_dependency(dep_dict))
+            deps.append(rocoto.add_dependency(dep_dict))


Suggested change

if self.options['lobsdiag_forenkf']:

dep_dict = {'type': 'task', 'name': f'{self.run}_ediag'}

else:

dep_dict = {'type': 'metatask', 'name': f'{self.run}_eomg'}

deps.append(rocoto.add_dependency(dep_dict))

deps.append(rocoto.add_dependency(dep_dict))

dep_dict = {'type': 'task', 'name': f'{self.run}_ediag'}

deps.append(rocoto.add_dependency(dep_dict))

First commits for "Remove the eomg job issue NOAA-EMC#2525"

0df74d7

DavidGrumm-NOAA requested review from WalterKolczynski-NOAA, aerorahul and DavidHuber-NOAA as code owners February 14, 2025 19:22

DavidGrumm-NOAA self-assigned this Feb 14, 2025

DavidGrumm-NOAA added 4 commits February 14, 2025 15:10

Merge branch 'develop' into eomg_1

c24048d

Fixed whitespace

ecbdb87

Merge in develop

f4e8441

Merge remote-tracking branch 'origin/develop' into eomg_1

73523e5

DavidHuber-NOAA requested changes Feb 18, 2025

View reviewed changes

DavidHuber-NOAA changed the title ~~Remove the eomg job issue~~ Remove the eomg job Feb 19, 2025

Updates based on reviewer comments

29122b0

DavidGrumm-NOAA requested review from RussTreadon-NOAA, CoryMartin-NOAA and CatherineThomas-NOAA as code owners February 25, 2025 18:37

DavidGrumm-NOAA added 2 commits February 25, 2025 14:35

Merge branch 'develop' into eomg_1

02d3f87

Merge in develop; Merge remote-tracking branch 'origin/develop' into …

c667866

…eomg_1

DavidGrumm-NOAA requested a review from DavidHuber-NOAA February 25, 2025 21:45

DavidHuber-NOAA requested changes Feb 27, 2025

View reviewed changes

Address additional comments

0162dc7

DavidGrumm-NOAA requested a review from DavidHuber-NOAA February 28, 2025 23:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove the eomg job #3331

Remove the eomg job #3331

DavidGrumm-NOAA commented Feb 14, 2025 •

edited

Loading

DavidGrumm-NOAA commented Feb 18, 2025

DavidHuber-NOAA left a comment •

edited

Loading

DavidHuber-NOAA Feb 18, 2025

DavidGrumm-NOAA Feb 25, 2025

DavidHuber-NOAA Feb 18, 2025

DavidGrumm-NOAA Feb 25, 2025

DavidHuber-NOAA commented Feb 18, 2025

DavidHuber-NOAA commented Feb 18, 2025

DavidHuber-NOAA left a comment

DavidHuber-NOAA Feb 27, 2025

DavidGrumm-NOAA Feb 28, 2025

DavidHuber-NOAA Feb 27, 2025

DavidGrumm-NOAA Feb 28, 2025

DavidHuber-NOAA Feb 27, 2025

DavidGrumm-NOAA Feb 28, 2025

	taperanalperts=${taperanalperts},
	lobsdiag_forenkf=.true.,taperanalperts=${taperanalperts},

	write_fv3_incr=${write_fv3_increment},
	lobsdiag_forenkf=.true.,
	write_fv3_incr=${write_fv3_increment},

Remove the eomg job #3331

Are you sure you want to change the base?

Remove the eomg job #3331

Conversation

DavidGrumm-NOAA commented Feb 14, 2025 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

Checklist

DavidGrumm-NOAA commented Feb 18, 2025

DavidHuber-NOAA left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DavidHuber-NOAA commented Feb 18, 2025

DavidHuber-NOAA commented Feb 18, 2025

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DavidGrumm-NOAA commented Feb 14, 2025 •

edited

Loading

DavidHuber-NOAA left a comment •

edited

Loading