Add IfNotPresent and Ignore SSA policies

[stefanprodan]

This PR extends the apply behaviour with two policies IfNotPresent and Ignore.

Fix: #936
Implementation and tests: fluxcd/pkg#625

Controlling the apply behavior of resources

To change the apply behaviour for specific Kubernetes resources, you can annotate them with:

Annotation	Default	Values	Role
kustomize.toolkit.fluxcd.io/ssa	Override	- Override - Merge - IfNotPresent - Ignore	Apply policy
kustomize.toolkit.fluxcd.io/force	Disabled	- Enabled - Disabled	Recreate policy
kustomize.toolkit.fluxcd.io/prune	Enabled	- Enabled - Disabled	Delete policy

kustomize.toolkit.fluxcd.io/ssa

Override

The Override policy instructs the controller to reconcile the Kubernetes resources with the desired state (YAML manifests) defined in the Flux source (Git, OCI, Bucket).

Merge

The Merge policy instructs the controller to preserve the fields added by other tools to the Kubernetes resources managed by Flux.

The fields defined in the manifests applied by the controller will always be overridden, the Merge policy works only for adding new fields that don’t overlap with the desired state.

IfNotPresent

The IfNotPresent policy instructs the controller to only apply the Kubernetes resources if they are not present on the cluster.

This policy can be used for Kubernetes Secrets and ValidatingWebhookConfigurations managed by cert-manager, where Flux creates the resources with fields that are later on mutated by other controllers.

Ignore

The Ignore policy instructs the controller to skip applying Kubernetes resources even if they present in a Flux source (Git, OCI, Bucket).

[hiddeco]

Awesome to have the annotations documented in one place now 💯 🥇

[hiddeco]

You may want to link from https://fluxcd.io/flux/faq/#how-to-patch-coredns-and-other-pre-installed-addons to the new documentation once this is available on the website.

[JeremyMordkoff]

I am struggling to get this to work for a configmap.

We are using helm and it includes a configmap to which I have added this annotation. Then my master pod overwrites this configmap. This version also has the annotation. But flux still reconciles the configmap back the helm chart version.

Does the annotation need to be in version 0 of this resource or can it be introduced in an upgrade?
Could the issue be that the master pod is re-creating the configmap (as opposed to editing it ... I'm not sure about this part...just guessing that's what we do.

I hope this is an appropriate place to ask ....

[stefanprodan]

SSA has nothing to do with Helm, these annotations are only for resources managed by Flux Kustomizations https://fluxcd.io/flux/components/kustomize/kustomizations/#controlling-the-apply-behavior-of-resources