Variable substitution failed: missing closing brace when using variable substitution in Kustomization

[ilya-git]

Describe the bug

I have set up a simple variable substitution as described in the documentation from config map:

  postBuild:
    substituteFrom:
      - kind: ConfigMap
        name: env-config

Then I use it in helm release values:

values:
   secrets:
      userAssignedIdentityID: "${kubelet-identity-client-id}"

kubectl get configmap

apiVersion: v1
data:
  ...
  kubelet-identity-client-id: <a guid in here>
kind: ConfigMap

When Kustomization controller runs, I get the following error:

var substitution failed for 'my-app': variable substitution failed: missing closing brace

If I write a random name (var does not exist in config map) instead of kubelet-identity-client-id, then an empty string is written.

Expected behavior

No error or another error that describes what is the problem (the closing brace seems to be in place at least right now)

Additional context

• Kubernetes version: 1.20.2
• Git provider: Bitbucket
• Container registry provider: Azure Container Registry

Below please provide the output of the following commands:

flux --version: 0.8.2
flux check

► checking prerequisites
✗ flux 0.8.2 <0.9.0 (new version is available, please upgrade)
✔ kubectl 1.20.0 >=1.18.0-0
✔ Kubernetes 1.20.2 >=1.16.0-0
► checking controllers
✔ helm-controller: healthy
► ghcr.io/fluxcd/helm-controller:v0.7.0
✔ kustomize-controller: healthy
► ghcr.io/fluxcd/kustomize-controller:v0.8.1
✔ notification-controller: healthy
► ghcr.io/fluxcd/notification-controller:v0.8.0
✔ source-controller: healthy
► ghcr.io/fluxcd/source-controller:v0.8.1
✔ all checks passed

[hiddeco]

This is likely because the variable name contains a hyphen (-), try to use underscores instead like you would in a Bourne-style shell.

[ilya-git]

Thanks a lot, I have actually just found exactly the same! It works with underscores, perhaps it's worth adding to documentation (all the limitations)? And maybe have a better error handling / validation of characters?

[ilya-git]

This is a super interesting and useful feature by the way, e.g. I use terraform to create all resources like public IP, then I create a config map and can reference the IP from config map instead of hardcoding it.

Really well though through 👍

[pmialon]

Hi,

I have a similar problem, I generate ConfigMap in my kustomization.

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
  - files:
      - application.yml=app-conf
    name: app-conf

but the file app-conf contain spring placeholders like this :

app.name=MyApp
app.description=${app.name} is a Spring Boot application

I have the same error message:
var substitution failed for 'my-app': variable substitution failed: missing closing brace

The only way I found is to entirely disable envsubst for this configmap.

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
  - files:
      - application.yml=app-conf
    name: app-conf
    options:
      annotations:
        kustomize.toolkit.fluxcd.io/substitute: disabled

[xvzf]

@pmialon did you find a solution?

[Tzrlk]

From the docs:

Note that if you want to avoid var substitutions in scripts embedded in ConfigMaps or container commands, you must use the format $var instead of ${var}. If you want to keep the curly braces you can use $${var} which will print out ${var}.

[old-guy-coder]

I know this is an old ticket but I am facing a similar problem, I am creating a config map using configMapGenerator in a Kustomization which is triggered by a Flux Kustomization called k8s-infrastructure-configs

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
generatorOptions:
  disableNameSuffixHash: true
  labels:
    app.kubernetes.io/part-of: flux
    app.kubernetes.io/component: infrastructure
configMapGenerator:
  - name: environment-release-values
    namespace: flux-system
    files:
      - environment=../../config/environment-values.yaml
  - name: geography-release-values
    namespace: flux-system
    files:
      - geography=../../config/eu/geography-values.yaml

The format of the geography-values.yaml is:

adxClusterName: ""
adxClusterUri: ""
adxDatabaseName: ""
adxLogsMigratorClientIdName: ""
adxLogsMigratorClientSecretName: ""
adxLogsReaderClientIdName: ""
adxLogsReaderClientSecretName: ""
applicationInsightsConnectionStringSecretName: ###
applicationInsightsInstrumentationKeySecretName: ###
applicationInsightsName: ###
asoWorkloadIdentityClientId: ###
asoWorkloadIdentityTenantId: ###
logAnalyticsWorkspaceName: ###
monitoringActionGroupId: ###
name: ###
primaryRegion: uksouth
prometheusQueryEndpoint: ###
prometheusQueryEndpointClientId: ###
prometheusQueryEndpointClientSecret: ###
regionNames:
- uksouth
resourceGroupName: ###
serviceBusConnectionStringSecretName: ###
serviceBusName: ###

and the resulting config map looks like this:

apiVersion: v1
data:
  geography: |
    adxClusterName: ""
    adxClusterUri: ""
    adxDatabaseName: ""
    adxLogsMigratorClientIdName: ""
    adxLogsMigratorClientSecretName: ""
    adxLogsReaderClientIdName: ""
    adxLogsReaderClientSecretName: ""
    applicationInsightsConnectionStringSecretName: ###
    applicationInsightsInstrumentationKeySecretName: ###
    applicationInsightsName: ###
    asoWorkloadIdentityClientId: ###
    asoWorkloadIdentityTenantId: ###
    logAnalyticsWorkspaceName: ###
    monitoringActionGroupId: ###
    name: ###
    primaryRegion: uksouth
    prometheusQueryEndpoint: ###
    prometheusQueryEndpointClientId: ###
    prometheusQueryEndpointClientSecret: ###
    regionNames:
    - uksouth
    resourceGroupName: ###
    serviceBusConnectionStringSecretName: ###
    serviceBusName: ###
kind: ConfigMap

Now, I have another Flux Kustomization that depends on the k8s-infrastructure-configs which I am trying to do a postBuild substitution from the config map defined above on a resource created by this config map and getting the same error message, in the resource I want to pick a few of the variables from the config map for replacement, e.g.

apiVersion: alertsmanagement.azure.com/v1api20230301
kind: PrometheusRuleGroup
metadata:
  name: flux-suspended-hr
  namespace: flux-system
spec:
  location: ${geography.primaryRegion}
  // omitted for brevity

Note the variable referred to above should map to the same variable from the config map as primaryRegion is nested under geography but no matter what I try (${geograhpy_primaryRegion}, ${geography\\primaryRegion}, ${geography/primaryRegion}, ${geography["primaryRegion"]}) I am getting this same error, is what I am trying to do even possible? If I just enter ${geography} then I get a different error which talks about JsonToYaml conversion so it certainly looks like the values are getting picked up but I do not seem to be able to select the nested value.

Any help with this would be greatly valued.

EDIT
It is worth noting that if I create the configmap manually and without the nesting under geography then that works as expected