Add apiVersion field to config files

[qzb]

Version v2 of Klio required "apiVersion" field in all its config files. Since this field was neither used nor well specified, I've removed it while implementing multiple registries support. That isn't changing the fact that having apiVersion field could be very useful, but it has to be thought through, not only copy-pasted from k8s.

Why we need a version field?

Currently, I see three use cases for having such a field:

1. ensuring that Klio has a minimal feature set to run a command;
2. managing unknown properties to improve forward compatibility of the files;
3. helping to identify Klio-related files.

Let's discuss each point separately.

1. Ensuring minimal feature set

This use case applies mainly to a "manifest.yaml" file. Klio provides various helpers to make creating commands easier. If some command uses, for example, JSON output to print logs, it will not work correctly with a version which doesn't support this feature.

We could use apiVersion to specify which version of Klio's API is required to run a command correctly. It would require using some variation of semantic versioning.

Examples in other projects:

• docker-compose.yml
• Chart.yaml

2. Improving forward compatibility

Managing unknown properties could be tricky when it comes to ensuring forward-compatibility. Forbidding them breaks compatibility with future versions, but allowing them could also be harmful since it enables to using some 3rd-party extensions to the file format. This unknown fields could potentially be incompatible with some future additions to file structure.

ApiVersion field can be used to circumvent this issue. When a particular version is known to Klio, it can deny using file if it has unknown properties. If a version is higher than currently supported one, Klio can simply ignore them, assuming that there is some newer version of Klio which supports them. For example, let's assume that Klio supports following apiVersions: 1.0, 1.1, 1.2:

• If it reads the file with apiVersion set to 1.2, it fails if a file contains some unknown properties.
• If it reads a file with apiVersion set to 1.3, it ignores unknown properties.
• If it reads a file with apiVersion set to 2.0, it fails, because there are some breaking changes which could affect how currently supported properties work.

3. Helping to identify Klio files

This use case doesn't affect Klio itself, since when Klio discovers paths, it is using a file's name, not its content. But it can help 3rd party tools (like CI pipelines) to check if the project is using Klio. For this to work, a version has to be prefixed, in a similar manner to k8s or skaffold apiVersion fields.

Examples in other projects:

• skaffold.yaml

How should a version field work?

We need to establish not only why also how versions should work.

Field name

The field doesn't have to be named apiVersion, neither we have to use a single field to describe version.

Field name	Good for	Notes
version	API compatibility File format	It's confusing, since there are also command versions, which are not described by this field.
apiVersion	API compatibility	This name is used by k8s and a few other projects.
configVersion	File format

I'm going to update this table if some new ideas came out.

Version format

Generally, versions should follow semantic versioning, but we can use a different number of segments:

• Some projects (like Helm) use only major part to version their files.
• Other (e.g. docker-compose) use both major and minor part.
• Using whole semver version is probably an overkill.

Additionally, a version can be prefixed with some kind of "namespace" (like "skaffold/v2").

Looking at our use cases, it would be best to use versions with two semver segments (e.g. "1.2"). If we want to address the 3rd use case (files discovery), versions have to be prefixed (e.g. "klio/1.2").

If we are going to use prefixes, it's worth to mention that they cannot be subject to branding, since it would break compatibility with public registries and commands.

Should be there a single version for all files?

Since Klio API is going to change more often than files formats, "klio.yaml" and "registry.yaml" should be versioned separately from "manifest.yaml".

Conclusion

We definitely need to include apiVersion in "manifest.yaml". I'm not sure about other files. Adding version to them increases the complexity and size of a boilerplate, but there aren't significant gains.

Please let me know what do you think about it.

[kstasik]

"Version v2 of Klio required "apiVersion" field in all its config files"

Can't we just have one config file (with combined functionality of config file and lock file) with version field?

EDIT:
I think i understand. But still i suggest using klio.yaml in both klio and subcommands. Similar to npm/cargo/composer/etc. It would be simplier and we could have type(optional, schema discriminator) + version field.

[wopol]

I agree with @kstasik, we have to minimize number of configuration files. I suppose so we should have two files: one per project and one per command(combination manifest + registry). In my opinion this file should be versioned with some kind of "namespace" (like "skaffold/v2").

[qzb]

In my opinion we should use apiVersion field name with klio/<major>.<minor> format and add it to registry.yaml (command.yaml schema already includes version). Adding version field to klio.yaml is difficult, since it breaks compatibility with most of the commands used in G2A (they parse apiVersion).