String/Char labels (section 6) mismatch between Conformance and CF Document

[DocOtak]

Hi CF Folks,

I have a situation where we label each data point using a string, currently using the char data type. The data being labeled are 2 dimensional and incomplete (DSG as defined in H.3.2). What we are doing is recording a sample id and these sometimes have non numeric chars in them due to historic reasons. These labels may or may not be unique for each data point. This leads me to the following apparent contradiction in the Conformance doc when related to the CF 1.8 document itself:

In the conformance doc section 6.1 is the following:

If the variable is of type char it must have one or two dimensions, where the trailing (CDL order) or sole dimension is for the maximum string length.

However the CF 1.8 document itself has the following in section 6.1:

If the variable has a type of char, the inner dimension (last dimension in CDL terms) is the maximum length of each string, and the other dimensions are axis dimensions.

Which seems to allow any number of dimensions as long as the CDL last dimension is the string length.

For some additional context, I'm using these labels as auxiliary coordinates. I do not have any Coordinate Variables in the file.

Thanks for any help/clarification you can provide.

[JonathanGregory]

Dear Andrew @DocOtak

I agree, this looks like an inconsistency in Sect 6.1.

If the variable has a type of char, the inner dimension (last dimension in CDL terms) is the maximum length of each string, and the other dimensions are axis dimensions.

allows for the possibility of multidimensional string-valued auxiliary coordinate variables, but the section begins with

Character strings can be used to provide a name or label for each element of an axis

which is concerned only with one-dimensional coordinate variables. There aren't any multidimensional cases in Sect 6.1. I think the text should be rewritten as

If the variable has a type of char, the inner dimension (second and last dimension in CDL terms) is the maximum length of each string, and the outer dimension (first dimension in CDL terms) is the axis dimension.

You say "the data being labeled are 2 dimensional". Do you mean your sample label is a 2D array of strings (or a 3D char array), or is it only a 1D string array (or 2D char array), because it's a DSG?

Cheers

Jonathan

[DocOtak]

Hi @JonathanGregory

The labels themselves are a 3d char array, in CDL order the dimensions represent: profile, vertical level, char array length. The vertical axis is a 2d array of values as allowed/required by H.3.2. The data being labeled are 2 dimensional, i.e. they only have the profile and vertical level dims. Unfortunately H.3.2 doesn't actually have its own example.

A multidimensional case I could find for an axis is H.6.2. Profiles along a single trajectory. Which is almost the same as H.3.2 except we aren't making an assertion that these profiles occurred along a given trajectory. We are trying to provide alternative labels for what is in the float alt(profile, z) variable of that example. If I was trying to provide char array labels for Example H.6.1, it would result in a 4 dimensional variable.

[JonathanGregory]

Dear @DocOtak

I think your data is a collection of profiles (like section H3) rather than a trajectoryProfile (like section H6) because, as you say, you are not stating that there is any relationship among the profiles. Example H8 is logically similar to what you want, but it uses the orthogonal multidimensional representation, whereas you are using the incomplete multidimensional representation. Is that because the profiles don't have a common set of levels? It would of course be useful to have a separate example of the incomplete multidimensional representation for a collection of profiles. Perhaps your actual case could be added to the standard document to provide an example.

By the way, I see that section H3.2 begins, "If there are the same number of levels in each profile, but they do not have the same set of vertical coordinates, one can use the incomplete multidimensional array representation." I think the "If" condition here is incorrect and should be deleted. The incomplete representation allows missing elements, so that the profiles don't all have to have the same number of levels (see Sect 9.3.2). Do you agree?

If I understand correctly, you want to attach labels char sample_id(profile,z,maxstringlen) to the individual data elements X(profile,z), where X is the data variable, z is the level (the element dimension) and profile indexes the profiles (the instance dimension). I agree that this makes sense. However, it is more general than currently allowed in Sect 6.1. The text

If the variable has a type of char, the inner dimension (last dimension in CDL terms) is the maximum length of each string, and the other dimensions are axis dimensions.

is OK for your case, but the conformance document is too restrictive in expecting there to be only one axis.

Sect 6.1 envisages attaching labels to elements of an axis, but you want to label elements of the two-dimensional data array. Your sample_id variable is a string-valued multidimensional auxiliary coordinate variable. I don't think there's any logical problem with this in the CF data model, but it needs some more general wording in the preamble of Sect 5 and in Sect 6.1, as well as an example.

Is this what you mean?

Best wishes

Jonathan

[DocOtak]

Hi @JonathanGregory

I think you understand my data correctly, these are discrete samples (niskin bottle) where the Z axis variable is the pressure which the bottle was triggered at. So while there tends to be the same number of Z-levels (e.g. 24 place rosette), the actual value of the Z axis will be different for each profile.

I've never really considered the "if", since section H3.2 ends with:

This representation also allows one to have a variable number of elements in different profiles, at the cost of some wasted space. In that case, any unused elements of the data and auxiliary coordinate variables must contain missing data values (section 9.6).

You also understand my label attaching desire. I think the conformance document section 6.1 is too restrictive in expecting there to only be one dimension for an axis variable (see examples H.12, H.16, H.18, and H.20 for axis variables with more than one dimension). Section 5 of the conformance document does specifically address labeling of auxiliary coordinate variables:

The dimensions of each auxiliary coordinate must be a subset of the dimensions of the variable they are attached to, with two exceptions. First, a label variable which will have a trailing dimension for the maximum string length. [...]

Should I suggest text changes for the relevant sections?

[JonathanGregory]

Dear @DocOtak

Yes, please propose some text changes, in a new issue in the conventions repository as an enhancement. The usual practice (as you will have seen) is to describe the proposed changes in the issue first, and proceed to a pull request once the changes are agreed in principle. On the basis of our discussion, it seems that we need to generalise 6.1 to allow labels to be string-valued auxiliary coordinate variables with any combination of dimensions of the data variable, although the usual case is that they are one-dimensional. I agree that sect 5 of the conformance doc is already OK for this case but I feel it would be better if it said "char variable containing a string-valued auxiliary coordinate variable" instead of "label variable", to be more general.

Thanks and best wishes

Jonathan