CMIP7 Data Request Spreadsheet Content

[tylov]

Goals:

1. Gather the CMIP7 data we need in order to map model output variables to the CMIP7 names?
2. Create a list variables for each experiment (group) and map select which are to be output for CMORization?
3. Associate one or multiple Opportunities to each experiment?

Questions:

1. Which new columns are needed compared to the CMIP6 DR, and which may be removed?
2. How should we best use the Air table and the and the DReq software to extract needed information?
3. Should we rather or additionally organize the spreadsheet on experiments rather than variables?

The CMIP7 Data Request adds new features like Opportunities and a new organization of the variables, i.e.:

Alongside discussing individual variables, author teams have been asked to define:

• Experiment groups: non-exclusive grouping of experiments (e.g. ‘Fast Track’, ‘DECK’, ‘Scenarios’ etc.)
• Variable groups: non-exclusive grouping of variables (e.g. monthly time slices of the baseline variables).
• Opportunities: intended use-case/justification for one or multiple variable groups. Opportunities are also linked to relevant experiment groups.

Here is the current CMIP7 DR spreadsheet. This will be updated from the latest revision of the CMIP7 air-table.

[tylov]

Data to be used for testing:

New simulations:
https://github.com/NorESMhub/noresm3_dev_simulations/issues

Most follow the setup from Tomas:
NorESMhub/noresm3_dev_simulations#226

Data is stored on NIRD:
/datalake/NS9560K/noresm3/cases

[tylov]

I have finished a version 1.0 (beta) of the CMIP7 DR spreadsheet, and it should be ready to be used by model developers. It merges @YanchunHe's CMIP6 DR spreadsheet by matching CMIP6 DR rows with CMIP7 DR rows.

I duplicated the modeler editable columns (green background) from the CMIP6 DR columns at the right side of the sheet, which contains the dependent NorESM model variables and default / desired CMORization support. Leave the original CMIP6 columns as a reference.

Question: Should we duplicate (or just use/update) the columns for each MIP at the right end of the sheet as well?

To make it more manageable, I removed many columns when importing from the Airtable. Let me know if any important columns are missing: @matsbn, @MichaelSchulzMETNO, @maritsandstad, @hgoelzer, @adagj, @DirkOlivie, @JorgSchwinger, @TomasTorsvik, @tjiputra, @rosiealice, @JensBDebernard, @mvdebolskiy, @mvertens, @gold2718, @oyvindseland, @lisesg

[oyvindseland]

A question about the categories CMOR required and CMOR default. What do these categoriesimply
Sometimes both are ticked off. My interpretation is that CMOR desired is what should added in the cmorization script in whihc case it should be moved to default. That does not explain why sometimes both boxes are ticked off. @tylov @YanchunHe

[oyvindseland]

Some of the sums in the cmorizationn formulas are now done in the model both for simplicity and to reduce the amount of output. My plan is too modify these as needed.

[oyvindseland]

Are there any versions of the google-sheet that can be edited. I can not edit the document included in this discusion so if people are going to edit it it has to either be opened for editing or there need to be another copy somewhere that can be edited. I can make a copy myself, but it is better to agree first how it should be done. @tylov @MichaelSchulzMETNO @matsbn

[MichaelSchulzMETNO]

Has been opened for you, Øyvind.

[oyvindseland]

For people who want to edit. A number of the sorting mechanisms in the document only works inn google-chrome. Not other browsers.

[rosiealice]

FWIW, I have an issue and a copy of the land part of the table in here:
NorESMhub/CTSM#188

[YanchunHe]

First of all, I strongly recommend all work (EDIT) on the same Google sheet, but only leave in the note who made the changes. Do not make copies. I do not own the Google sheet, so I can not change the editors.

The CMOR desired [ ] is one which is assumed to have model output and need to cmorized; while CMOR default [ ] is a confirmation such variable is implemented in the CMOR tool so that I can or can not be cmorized (due to various reasons as shown in the notes). If both are not ticked, it means that this variable is in the CMIP data request, but it is not available/not desired by the modeller/MIP participants and not implemented to CMORize.

Hope this answers you questions @oyvindseland

[lisesg]

I agree with Yanchun, it's best to work on the same spreadsheet. I've updated the editing rights for a number of people, but if someone wants to edit and find themselves without access please let me know and I will fix it.

[oyvindseland]

So when I know changes the variable names (and the equations) what should I tick?

[YanchunHe]

So when I know changes the variable names (and the equations) what should I tick?

So if you know there are changes in the models/variables, you update the corresponding entries in the sheet, also best to add some notes/log in the 'Notes' column; The CMOR desired [ ] can already be ticked. Do not tick the CMOR default [ ], which is done by the one who implement the CMORizaton tool (currently me). Now I have ticked off all the boxes of CMOR default [ ]; and I will tick one by one when each is implemented and tested.

I suggest Tyge remove the columns for the CMIP6, but may add some where a link to the CMIP6 sheet. @tylov

[rosiealice]

Hi all,

I'm afraid that ship has sailed already - last week we determined that, because the sheet was locked for all of us, it was probably intended that we made our own copy. But it was also extremely unwieldy to work with the whole sheet in terms of sorting the variables for more effective viewing and so on. I will work at adding my land changes back in once we are happy with the majority of them.

[oyvindseland]

At the moment I am working directly in the sheet, but I can of course just add it as a comment.
I hope that we can avoid using personal copies of a sheet.

[YanchunHe]

you should have the write permission now, as I see Lise added you as editor

[oyvindseland]

Yes I can edit the question is more if I can do anything thatmake it easier to update the cmor script at the end.

E.g In line 197 I replaced the old sum of variables with a new variable name as seen from the change log and ticked of cmorization desired and removed the mark in CMOR by default. It should be by default but will not work now. Should I then also tick off change since CMIP6. The CMOR definition is as before but the CAM variable name has changed

[YanchunHe]

Yes I can edit the question is more if I can do anything thatmake it easier to update the cmor script at the end.

Anyway, now I tick off all the boxes in the CMOR by default, and likely I will need to import manually, at least at first, to the CMOR tool, and check each one and then tick them one by one again.

I don't know what you referred to by the change since CMIP6. However, I would say, currently, you don't need to think too much about how it was done in CMIP6.

[tylov]

@rosiealice The most efficient way for sorting variables is to use filtering, "Filter by cell value". Click on the striped triangle in the column header.

@oyvindseland "Should I then also tick off change since CMIP6. " No, this has a special meaing, see the description in the column header. Only the columns with green background should be edited. We can add more columns to for e.g. conversion formulas if needed.

[MichaelSchulzMETNO]

....maybe we should have a zoom session on how to filter, sort and combine tables in one google sheet? I could show some tricks.

[JorgSchwinger]

I would be interested in learning some tricks... (and maybe also dos and don'ts of editing the sheet)

[rosiealice]

I already went through all the land variables - so I'll need to push those answers back into the main table. (There are a few I am still thinking about, and some outputs we could add if we wanted to... )

[maritsandstad]

So not very active here, but @mvertens and I are working on converting the sheet to a yaml-file for processing atmosphere and land variables. We will have multiple questions down the line, I think, but here is one: The dimensions column is presumably what was originally there from the CMIP7 data request, correct? Is this how we want this to be in final output netcdf (i.e. do we want alevel rather than ilev etc @MichaelSchulzMETNO f.i.)?

[lisesg]

It's my understanding that the dimensions column should match the CMIP7 data request. If the vertical coordinate is alevel, then the final output should be as defined here in the airtable. @tylov and @YanchunHe , please comment if this is not correct.

[YanchunHe]

Yes, I agree with @lisesg

ilev defines the layer interfaces, which at sometimes can/must be provided as 'layer/level' boundaries (as the bounds attribute).

[maritsandstad]

Well yes, was more asking about alevel versus lev

[lisesg]

When the vertical dimension is "alevel" in the request, the vertical coordinate should be based on the native vertical coordinate "lev" in NorESM. It should also be called "lev" in the CMOR-ized files. alevel was also used for CMIP6, you can look at the files for AERmon variables, f.eks. /projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-LM/historical/r1i1p1f1/AERmon/ua/gn/latest/ua_AERmon_NorESM2-LM_historical_r1i1p1f1_gn_185001-185912.nc.

[maritsandstad]

How do I get write access to the sheet @tylov @lisesg ? When converting to yaml I have found one typo, and the header of column K has two line shifts in it which makes the file very annoying to parse as a csv for no particularly good reason...

[gold2718]

@maritsandstad (and others), please be sparing in editing column labels. I already have a script that looks for certain column headings in order to parse the spreadsheet. Changing the column labels breaks the tool.

[maritsandstad]

I will not do it again, but I would much prefer that we don't have line shifts in the column label... (That's also really hard to parse...)

[gold2718]

I'm all for cleanup and improvements, I just think this sheet has become a communal resource so more notice and discussion would help. PRs for spreadsheets? ;)

[MichaelSchulzMETNO]

we could protext some parts of the sheet against changes, I believe the owner has to do that....

[maritsandstad]

FYI, I've made a similar change cutting a lineshift in the CMIP7 freq. column header.