'title' attribute in files misses mip_era when using mip-cmor-tables

[matthew-mizielinski]

Data that we've published for CMIP6Plus has the title attribute HadGEM3-GC31-LL data prepared for , i.e. it is missing the mip_era which should have been appended.

I think this is down to us having removed the mip_era attribute from the tables within mip-cmor-tables in order to make them more generic.

Would be good to pick up the mip era attribute from the global attributes rather than the MIP tables themselves.

Thanks to @klauswyser for pointing this out

[durack1]

This relates to the "Header" change from cmip6-cmor-tables to mip-cmor-tables and future sources

PCMDI/cmip6-cmor-tables/Tables/CMIP6_3hr.json

"Header": {
  "data_specs_version": "01.00.33", 
  "cmor_version": "3.5", 
  "table_id": "Table 3hr", 
  "realm": "atmos", 
  "table_date": "18 November 2020", 
  "missing_value": "1e20", 
  "int_missing_value": "-999", 
  "product": "model-output", 
  "approx_interval": "0.125000", 
  "generic_levels": "", 
  "mip_era": "CMIP6", 
  "Conventions": "CF-1.7 CMIP-6.2"
}, 

vs

PCMDI/mip-cmor-tables/Tables/MIP_ACmon.json

"Header": {
  "Conventions": "CF-1.7 CMIP-6.5",
  "approx_interval": 30.0,
  "checksum": "",
  "cmor_version": "3.8.0",
  "data_specs_version": "6.5.0.0",
  "generic_levels": "",
  "int_missing_value": "-999",
  "missing_value": "1e20",
  "product": "model-output",
  "table_date": "2023-11-16",
  "table_id": "ACmon"
},

Some prototype python code to generate a table checksum can be found at https://gist.github.com/matthew-mizielinski/643ca1841dc760edf230473a8b81d396

[taylor13]

mip_era is defined in the cmor input file: #776 . And it is stored as a global attribute in files written through CMOR, so there should be no problem removing mip_era from the header. I guess the problem is that if CMOR doesn't find mip_era in the header, then in constructing the "title", it inserts an empty string.

For the new CMOR 3.10 table format, the plan is to eliminate mip_era from the header, but keep it in the input.json file, as described here.

[taylor13]

@matthew-mizielinski : I'm checking something. Is HadGEM3-GC31-LL data prepared for the entire title now written (incorrectly)? The original CMOR wrote something like: HadGEM3-GC31-LL data prepared for the CMIP5 historical experiment. Was it changed to HadGEM3-GC31-LL data prepared for CMIP6? Or something else?

[klauswyser]

If I may chip in here: the header in the netCDF file now says

    :title = "HadGEM3-GC31-LL output prepared for" ;

while for any CMIP6 dataset it would have been ...prepared for CMIP6

[taylor13]

Thanks @klauswyser . If we modify CMOR, we could probably write anything we want here (given the available global attributes). Any votes on whether we should mention experiment_id? activity_id? mip_era? institution_id? or ??

I don't know how "title" has been relied on by users in the past, so a little difficult to say what should be mentioned here. Clearly "prepare for" by itself is useless.

[klauswyser]

It's unclear how the title attribute has been used, it's not showing up in any of the search facets at CMIP6 search portals. Is it necessary at all? mip_era is already present the metadata (at least for CMIP6Plus), isn't that sufficient? I don't see what extra value we'd get from title.

[taylor13]

Originally, we expected title might be displayed by a user on a plot or something. There was no anticipated use of it in search interfaces or for any "official" purpose. "titles" were not meant to necessarily be interpreted by machines either.

[klauswyser]

Indeed, some applications may use title like for example ncview:

[durack1]

Indeed, some applications may use title like for example ncview:

Yes, @klauswyser Ferret is another package that uses the title global attribute, there are likely others, but not any that I am aware of, see example below

[matthew-mizielinski]

@matthew-mizielinski : I'm checking something. Is HadGEM3-GC31-LL data prepared for the entire title now written (incorrectly)? The original CMOR wrote something like: HadGEM3-GC31-LL data prepared for the CMIP5 historical experiment. Was it changed to HadGEM3-GC31-LL data prepared for CMIP6? Or something else?

@klauswyser has hopefully clarified this. Personally, I would remove it or make it configurable via the CVs