Guidelines for adding variables to the register

[taylor13]

I would like to get input on what rules we should adopt soon to help us determine whether a proposed variable should be allowed into the variable register. This is prompted by a remark made in #28 (comment):

The variable registry is currently a piece of software and a github repo and as such should not have the power to impose a veto on variables requested by the scientific community.

I think we all agree we do not want to prevent science from being done, but we also are trying to impose some uniformity across WCRP activities

• to make it easy for modeling groups to develop post-processing procedures that will minimize their effort in contributing data across all projects.
• to make it easy for those with interest in the data from the WCRP activities to write codes that can be applied across projects and not have to deal with unneeded diversity in the variables they access.

As examples of the kind of rules that were established long ago for MIP "standard output" (subsequent renamed "data request"), there are some obvious ones:

• if a vertical flux has been registered by an established WCRP activity, then with rare exceptions we would not allow the negative of that vertical flux to be registered. (e.g., if surface_downward_sensible_heat_flux exists, we would require a very strong argument to add surface_upward_sensible_heat_flux.)
• if eastward_wind_speed has been defined with units of m s-1, we would not allow a new variable, eastward_wind_speed, to be registered with different units (e.g., cm s-1).
• if the area_fraction of vegetation were registered with a cell_methods including the directive "where land", we would not register a new vegetation area_fraction without the "where" directive, but rather ask that users to calculate the new variable from the existing area_fraction plus the area fraction of land (i.e., area fraction of vegetation in the cell = (area fraction of vegetation where land) * (area fraction of land).

Along these lines, I have argued through a number of CMIP phases (most recently in #28 (comment)) that rather than requesting surface_albedo as a variable, we get more bang for the buck by requesting the incident and reflected shortwave radiative fluxes.

I hope to get input on how we might best make these "unwritten" rules more generally known. Contributions are also welcome as to what additional rules might be added. In establishing "rules", there may always be exceptions where we would break a rule, but there would have to be some strong justification with a use-case that clearly demanded it.

[znichollscr]

The line between science and data management feels very blurry to me. I think it's good to try and get clarity where we can, but accept that some things are just going to be borderline.

(Not the point of this issue, but as an example) On the issue of #28

For me at least, the balance seems to be somewhere between #28 (comment) and #28 (comment). I would probably say that, if the teams really want albedo, fine include it (I think Karl makes a good point about the efficiency and benefits of reporting fluxes instead, but my feeling in that is that rejecting the use of albedo at all is probably going too far). Having said that, the argument, the "variable registry is currently a piece of software and a github repo" is also clearly too simple. There is also the responsibility to manage that registry and keep it as sensible and tidy as possible, not just blindly accepting anything. Karl brings up a nice example, we would be doing a disservice if we accepted a flux that was simply the negative of an already available flux i.e. we should vet and occasionally reject proposed variables where there are clear grounds i.e. we have a responsibility to go beyond just software and a github repo in such cases.

It is probably true that we are lacking clarity around what happens when things are not clearly in one camp or the other. As above, I'd probably lean towards accepting proposals rather than rejecting wherever there is not a clear consensus.

[eleanororourke]

I think getting as many of these "unwritten" rules written down somewhere for everyone to be aware of (and potentially review/comment) would be really helpful @taylor13 and could then represent general principles to be followed. Having this available would most likely be of particular use for those new to the community or new MIPs/MIP leadership for example.

As @znichollscr suggests there will always be edge cases and having open discussion of these is important but we then need a clear and transparent process for how an outcome is determined (perhaps leaning on the CF conventions example).

[taylor13]

Yes, writing these rules down is in the plan and yes they won't be hard and fast rules and we'll need a process to deal with edge cases (and other exceptions).

Regarding control of what goes into the register, I advocate a rather vigilant stance. If there is a compelling science question that can only be addressed if a given variable is saved (and it cannot be easily calculated from other variables saved), then it would, of course, never get vetoed. If, however, a variable already exists which can be substituted for a newly proposed variable without comprising its scientific value, then compelling justification would be required before accepting it.

The past practice is based largely on years of input from modeling groups urging us to limit the variety of variables requested and processing procedures required to prepare variables. One way we have managed to do this is by avoiding variables that are not typically saved by modeling groups when they run their models. We also avoid collecting variables that are easily calculated from other reported variables.

Thus, I think there is good reason to prevent unnecessary proliferation of variables in the data register. That way, modeling groups who have developed software to contribute existing variables will not need to modify the software when a new data request is constructed from a subset of the existing variables.

In the case of surface albedo, it must be considered in the mix of other variables that might be included in the data request. Variable considered very high priority in their own right are those that enable analysis of energy budgets of the atmosphere and subsurface region (ocean, seaice, land ...). Thus incident and reflected SW radiation at the surface have always been included in data requests. Of course albedo can be calculated from them as a ratio, so does not needed to be included in a data request. One could, of course, save albedo and one of the radiative fluxes, from which the other could be calculated. The reason albedo was the variable omitted is that calculation of the mean albedo requires careful weighting of samples by the incident SW radiation, which requires special post-processing procedures, which modeling groups oppose. There has never been a use case requiring that albedo be recorded in preference to simply calculating it as the ratio of two other reported variables.

[znichollscr]

Regarding control of what goes into the register, I advocate a rather vigilant stance

I feel a bit nervous about this. I fear the CVs/variable registry team would end up being the 'guardian of the modelling teams', which I think could be a very awkward place to be in given that a lot of the data request is written by modelling teams in the first place i.e. we would end up blocking teams from doing things they want to do on the grounds that it's not in their best interest, even though they're the ones who want to do the thing in the first place.

I would advocate for a process more like this:

1. Person/group asks to add a variable
2. CVs/variable registry team review requests and provide feedback based on experience to date
3. Person/group considers feedback
4. If the CVs/variable registry team and the person/group asking to add the variable disagree, then a consultation with modelling groups occurs. If modelling teams are happy to provide the variable, then the CVs/variable registry team should add it (irrespective of their own misgivings). Otherwise, the variable is not added

I think a process like this means that the CVs/variable registry team can provide quick feedback which will lead to a fast resolution in the vast majority (my guess would be >95%) of cases. However, ultimately the variable register is about serving users and modelling teams. If things are grey and both users and modelling teams really want a particular variable to be included, then I don't think the CVs/variable registry team should/wants to be a blocker. I think a process something like the proposal above would serve this intent.

As one other note, it's a data request. Ultimately, if modelling groups want to ignore the request, they can (or do I misunderstand and it's actually a data mandate?).

In the case of surface albedo...

Understood. Let's see how #28 plays out. There seems to be some agreement with you, and some push back (it would be interesting to see the previous discussions and differing views mentioned here #28 (comment)).

[eleanororourke]

If I remember correctly, the CVs TT committed to providing a recommendation for governance of the variable registry and process for how the community can suggest variables to be added. The WIP, and CMIP Panel, would then review this recommendation and direct implementation of the appropriate structure for this process in the longer term. The process @znichollscr has outlined above looks like a good start for this recommendation.

[taylor13]

Thanks, @znichollscr, for writing down a process that we can seriously consider.

To be clear, I agree that both modeling groups and those requesting model output need to have input to the decision making. And if many modeling groups really want to save a variable, we shouldn't stand in the way of that.

My impression is that currently the modeling groups have not been alerted to certain important details regarding the variables they are asked to review, so, for example, they haven't notice when certain (nearly?) duplicative variables were being requested. If they knew of such cases when asked to review some 2000 variables, they might object to some variables that currently appear in the data request.

So, I like the idea that we go to the modeling groups on a case-by-case basis (or with a small collection of cases) and ask for their input and provide a summary of discussion surrounding each of those variables.

In general, I think modeling groups should have a chance to review any new variables proposed, whether or not they are considered controversial. I don't think we want to include too many variables in the register that modeling groups have no intention of writing.

For your step 2 above, the written "rules" or "guidelines" regarding what is generally considered eligible for inclusion in the data register will be essential.

[jesusff]

Thank you for the interesting discussion. I agree that a basic set of written rules would be really helpful.

I wonder about the current scope of this repo. Would you be willing to discuss variable proposals beyond the CMIP7 DR? I'm specifically thinking of the CORDEX DR, which we try to keep aligned with CMIP as much as we can, but we sometimes are ahead of CMIP in the request of certain variables (e.g CMIP-Data-Request/Harmonised-Public-Consultation#74 cmip6dr/CMIP6_DataRequest_VariableDefinitions#415).

I fully understand if you prefer to focus on the CMIP7 DR, as the repo is in a very early stage. Beyond the acceptance or actual registration of the variables, the step proposed by Zeb above where we could get feedback based on experience to date, (especially concerning naming) would be really helpful to define new variables in a way that would be consistent with CMIP if they are ever requested there. Such a central discussion forum for climate model output variables would be really welcome.

A very recent example from the CORDEX wave model data request (from @gkara00):

Also, there has been interest in variables not currently included in the
CFTables or CMIP request, such as "phioc; energy by wave-breaking
inducing mixing" and "tauoc; Ocean stress modified by wave induced
stress". These are available, for instance, in the ERA5 dataset.

[taylor13]

thanks @jesusff for bringing this up. The intention all along is that the data register would serve all the WCRP-coordinated modeling and observational activities. So once established, based on the CMIP7 fasttrack data request, we'll open it up to other projects. This way the definition of variables across projects will be consistent. For any individual MIP, only a subset of the the variables in the register are expected to be relevant.

Note that since the register doesn't specify frequency, region, or grid, a particular request is not limited to the frequencies, grids, and regions defined for CMIP7. That being said, if a global GCM is expected to contribute results, we would hope the already defined options will be sufficient. This will make it easier for modeling groups to participate.

[bethdingley]

I just want to copy in @martinjuckes's comment from #21 into here, as it should be taken into account during the development of the variable registry guidance:

'I'm concerned that much of this discussion is based on the misleading premise that the use of "surface" in the CF standard name of ocean variables has a specific meaning. This was not the case last time I checked the CF Standard Name list and I'm not aware of any plans to change it. If we are removing vertical coordinate information from variables we need to be clear that consequent introduction of ambiguity in the variable specification is in line with scientific requirements.'

[github-actions]

This issue is stale after 30 days of inactivity.

[github-actions]

This issue was closed after 14 more days of inactivity.