Update cmor-cvs.json

[matthew-mizielinski]

@znichollscr, these are the changes I've had to make to use the CVs json file with the examples in https://github.com/WCRP-CMIP/cmip7-cmor-tables/

This isn't for merging, but it would be good to fix the easy items in the code that generates the CVs Json file for cmor, and get an update on the automatic generation of this file as updates to the CVs come in.

[znichollscr]

Thanks @matthew-mizielinski, all looks good

Only one small question. I assume that the steps needed to add MOHC stuff are clear enough (we have to alter the CVs, then regenerate the table, I can help with this)?

[znichollscr]

Was the removal of time range on purpose? As I understand it, that is meant to be there so it should be something like

"filename_template": "<variable_id><branding_suffix><frequency><region><grid_label><source_id><experiment_id><variant_label>[<timeRange>]"

I also don't know whether the trailing .nc is meant to be there or not..?

[matthew-mizielinski]

@znichollscr, I don't think <timeRange> is understood by CMOR at all. If I add it in then we end up with file names like

MIP-DRS7/CMIP7/CMIP/CCCma/CanESM6-MR/1pctCO2/r9i1p1f3/glb/mon/tos/tavg-u-hxy-sea/g99/v20260127/tos_tavg-u-hxy-sea_mon_glb_g99_CanESM6-MR_1pctCO2_r9i1p1f3_timeRange_201801-201802.nc

if I add just .nc to the end I get

MIP-DRS7/CMIP7/CMIP/CCCma/CanESM6-MR/1pctCO2/r9i1p1f3/glb/mon/tos/tavg-u-hxy-sea/g99/v20260127/tos_tavg-u-hxy-sea_mon_glb_g99_CanESM6-MR_1pctCO2_r9i1p1f3_.nc_201801-201802.nc

So it seems leaving these two items out is necessary for recent versions of CMOR (in older versions this string wasn't picked up from the CV json file at all).

[taylor13]

So, is the bottom line that CMOR is currently hard-wired to append the time-range to whatever filename structure is specified by the data provider in the cmor-cvs.json file? If so, is the time-range something we should put on the list of things that should be handled as input in a future release? Note that for fixed fields (when frequency = "fx"), the time-range is omitted.

[znichollscr]

So it seems leaving these two items out is necessary for recent versions of CMOR (in older versions this string wasn't picked up from the CV json file at all).

Ok great, thanks for checking. Updated now

So, is the bottom line that CMOR is currently hard-wired to append the time-range to whatever filename structure is specified by the data provider in the cmor-cvs.json file?

This is my understanding

If so, is the time-range something we should put on the list of things that should be handled as input in a future release?

Up to the CMOR developers I would say. We're probably not going to change this convention anytime soon so it's not a big issue right now (if we ever do this change the convention, then CMOR will need an update - up to the CMOR developers whether they do the work now 'just in case' or wait until it's actually a problem).

[znichollscr]

Thanks @matthew-mizielinski. In terms of how we can implement this, it's a little bit complicated. I've made some pull requests to show you how this works:

1. fix up generation of the CMOR CVs file (currently the CI for checking the CMOR CVs file in the CMIP7 CVs repo has been ignored, so I had to get that going again first)
   • fixes to the CVs: Update to fix CMOR table export #337
   • updates needed to esgvoc (these aren't normally needed, but esgvoc and the CVs have got a bit out of sync): Update to fix CMOR table export ESGF/esgf-vocab#187
2. update the data specs version entry
   • CMIP7 CVs entry (this normally isn't needed, but as we're changing the name, we have to do minor updates to the CMIP7 CVs file): Update CMIP7 data specs version entry znichollscr/CMIP7-CVs#8
   • universe updates: Update CMIP7 data specs version entry WCRP-universe#106
   • esgvoc update (not strictly needed, but useful I think to clarify what is going on with data specs version): Update docstring for data specs version znichollscr/esgf-vocab#1
3. update the UKMO information
   • CMIP7 CVs addition: Add UKMO entries znichollscr/CMIP7-CVs#9
   • universe updates: Update UKMO entries for CMIP7 CMOR CVs demo znichollscr/WCRP-universe#6

Once you do all of that, then you end up with a CMOR CVs file like what you have here, see https://github.com/znichollscr/CMIP7-CVs/blob/add-ukmo-entries/cmor-cvs.json

[znichollscr]

For further context about how to keep the CMOR CVs file up to date, see #271

[matthew-mizielinski]

Only one small question. I assume that the steps needed to add MOHC stuff are clear enough (we have to alter the CVs, then regenerate the table, I can help with this)?

Correct, I think we need a pretty minimal set of details for each model that could be constructed from the information collected by EMD.

[znichollscr]

I'm going to close this PR as I think we've got everything out of it now. #336 (comment) will still be there - up to you whether you continue the conversation in that thread or start a dedicated issue