rose stem "validate_rose_meta" fails when running rose stem after running the apply_macros.py script

[MichaelWhitall]

Hi folks, sorry its me again!

Many thanks for updating the instructions for applying the new lfric upgrade macros to a test branch :)
WorkingPractices/macros

Having successfully run the wrapper script and applied my upgrade macro to the rose-stem apps (for this ticket: lfric_apps:#467), I'm now finding that when I run rose stem, the app
validate_rose_meta
fails with a lot of errors like this:

Failures validating lfric_atm
[FAIL] Could not find metadata for applications/lfric_atm/rose-meta/lfric-lfric_atm/vn2.0_t467, using applications/lfric_atm/rose-meta/lfric-lfric_atm/HEAD
[FAIL] To suppress these warnings run 'rose edit --no-warn version'
[FAIL] 
[V] metomi.rose.macros.DefaultValidators: issues: 3

Unlike when I successfully ran these tests with the same meta-data from the vn1.2 branch for this ticket (where I had to just manually add my new namelist inputs to all the apps), the key thing that's changed since then is that rose app-upgrade (called from the wrapper script) has changed the meta= line at the top of each rose-app.conf file:

>> fcm diff rose-stem/app/lfric_atm/rose-app.conf
Index: rose-stem/app/lfric_atm/rose-app.conf
===================================================================
--- rose-stem/app/lfric_atm/rose-app.conf	(revision 5800)
+++ rose-stem/app/lfric_atm/rose-app.conf	(working copy)
@@ -1,4 +1,4 @@
-meta=applications/lfric_atm/rose-meta/lfric-lfric_atm/vn2.0
+meta=applications/lfric_atm/rose-meta/lfric-lfric_atm/vn2.0_t467

But the edited rose-meta in my branch is in
applications/lfric_atm/rose-meta/lfric-lfric_atm/HEAD
and the directory vn2.0_t467 does not exist, so validate_rose_meta is falling over because the meta-data directories now pointed at by all the rose-stem apps don't exist.

Does anyone know what I should do to get validate_rose_meta to succeed? e.g. in my test branch should I just make a copy of the HEAD meta-data in a new directory vn2.0_t467? Or should I just ignore the validate_rose_meta fails for now?

Cheers!
Miker

[jennyhickson]

Hi Mike,

A fix for the validate rose meta script has been included at vn2.0.1. Hopefully this sorts this for you.

Thanks
Jenny

[MichaelWhitall]

Awesome no prob and thanks for this Jenny; having rebranched from vn2.0.1, validate_rose_meta now succeeds in my tests :)

In case anyone else is about to try this, one odd thing I found was that when branching from vn2.0.1, where you specify the BEFORE_TAG and AFTER_TAG in your upgrade macro, these still have to point to vn2.0 not vn2.0.1. e.g. if I put what I'd expect to put in an upgrade macro at vn2.0.1:

class vn201_t467(MacroUpgrade):
    """Upgrade macro for ticket #467 by Mike Whitall."""
 
    BEFORE_TAG = "vn2.0.1"
    AFTER_TAG = "vn2.0.1_t467"
 
    def upgrade(self, config, meta_config=None):
        ...

The apply_macros.py script fails with the following:

Traceback (most recent call last):
  File "/home/h01/frum/SimSys_Scripts/lfric_macros/apply_macros.py", line 1050, in <module>
    main()
  File "/home/h01/frum/SimSys_Scripts/lfric_macros/apply_macros.py", line 1024, in main
    macro_object.preprocess_macros()
  File "/home/h01/frum/SimSys_Scripts/lfric_macros/apply_macros.py", line 774, in preprocess_macros
    last_after_tag = self.find_last_macro(macros, meta_dir)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/h01/frum/SimSys_Scripts/lfric_macros/apply_macros.py", line 515, in find_last_macro
    raise Exception(
Exception: Couldn't find the before tag 'vn2.0' in macros in the versions.py file at /net/home/h05/mwhitall/UM/branches/7815_improve_precfrac/vn2.0.1_improve_precfrac/science/um_physics_interface/rose-meta/um-microphysics

But this problem goes away if I keep the upgrade macro the same as it was in the vn2.0 branch, even though this is vn2.0.1:

class vn20_t467(MacroUpgrade):
    """Upgrade macro for ticket #467 by Mike Whitall."""
 
    BEFORE_TAG = "vn2.0"
    AFTER_TAG = "vn2.0_t467"
 
    def upgrade(self, config, meta_config=None):
        ...

Cheers!
Mike

[jennyhickson]

Good spot Mike. We didn't do the full release process for vn2.0.1 and that includes creating a vn2.0 - vn2.0.1 macro, so you're right - continue to treat macros as being vn2.0.

Cheers