Using $KUBECONFIG with several files, logging into the cluster dumps all contexts into the default config file

[adietish]

Steps:

1. ASSERT: have several config files, all of them listed on the KUBECONFIG env var

• $KUBECONFIG

export KUBECONFIG=${HOME}/.kube/config:${HOME}/.kube/minikube.yaml:${HOME}/.kube/sandbox.yaml

• config

apiVersion: v1
current-context: rh-sandbox
kind: Config
preferences: {}

• minikube.yaml

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /Users/adietish/.minikube/ca.crt
    extensions:
    - extension:
        provider: minikube.sigs.k8s.io
        version: v1.34.0
      name: cluster_info
    server: https://127.0.0.1:41825
  name: minikube
contexts:
- context:
    cluster: minikube
    extensions:
    - extension:
        last-update: Wed, 08 Jan 2025 15:05:54 CET
        provider: minikube.sigs.k8s.io
        version: v1.34.0
      name: context_info
    namespace: kube-node-lease
    user: minikube
  name: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /Users/adietish/.minikube/profiles/minikube/client.crt
    client-key: /Users/adietish/.minikube/profiles/minikube/client.key

• sandbox.yaml

apiVersion: v1
clusters:
- cluster:
    server: https://api.XXXX.openshiftapps.com:6443
  name: rh-sandbox
contexts:
- context:
    cluster: rh-sandbox
    namespace: adietish-dev
    user: adietish/rh-sandbox
  name: rh-sandbox
current-context: ""
kind: Config
preferences: {}
users:
- name: adietish/rh-sandbox
  user:
    token: sha256~XXXXXXXXXXXXXXXXXXXEMhxfNi614txI

2. EXEC: pick ctx menu action "Log in to Cluster"
3. ASSERT: Login dialog is show
4. EXEC: get the token and paste into the Login dialog into the field "Token:"
5. EXEC: hit "OK"
6. ASSERT: The plugin is logged into the RH sandbox
7. EXEC: cat ~/.kube/config

Result:
~/.kube/config was altered. All the above contexts were dumped into it. Interestingly, Only the context were dumped. Neiter the users, nor clusters were. Additionally a new context was created for the cluster using the url that was listed in the dialog.

apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: https://api.rm1.0a51.p1.openshiftapps.com:6443
  name: api-rm1-0a51-p1-openshiftapps-com:6443
contexts:
- context:
    cluster: apiXXXXopenshiftapps-com:6443
    namespace: adietish-dev
    user: adietish/apiXXXXXopenshiftapps-com:6443
  name: adietish-dev/apiXXXXXopenshiftapps-com:6443/adietish
- context:
    cluster: minikube
    extensions:
    - extension:
        provider: minikube.sigs.k8s.io
        version: v1.34.0
      name: context_info
    namespace: kube-node-lease
    user: minikube
  name: minikube
- context:
    cluster: rh-sandbox
    namespace: adietish-dev
    user: adietish/rh-sandbox
  name: rh-sandbox
current-context: adietish-dev/api-rm1-0a51-p1-openshiftapps-com:6443/adietish
kind: Config
preferences: {}
users:
- name: adietish/apiXXXXXopenshiftapps-com:6443
  user:
    token: sha256XXXXXXXXXEMhxfNi614txI

[adietish]

This is caused by calling oc login. The new context is created in the first file listed in the $KUBECONFIG:

https://docs.openshift.com/container-platform/4.17/cli_reference/openshift_cli/managing-cli-profiles.html#load-and-merge-rules_managing-cli-profiles

When a value is created, it is created in the first file that exists.

Why all the existing contexts are dumped into this file is beyond me currently

[adietish]

oc login is currently implemented at

intellij-openshift-connector/src/main/java/org/jboss/tools/intellij/openshift/utils/oc/OcCli.java

Lines 62 to 68 in 2474d44

	public void login(String url, String userName, char[] password, char[] token) throws IOException {
	if (userName != null && !userName.isEmpty()) {
	execute(new File(HOME_FOLDER), command, envVars, "login", url, "-u", userName, "-p", String.valueOf(password), "--insecure-skip-tls-verify");
	} else {
	execute(new File(HOME_FOLDER), command, envVars, "login", url, "--token", String.valueOf(token), "--insecure-skip-tls-verify");
	}
	}

As a workaround, we could

1. check if $KUBECONFIG exists
2. if it exists (if it doesn't we dont need to proceed differently):
3. create a new file and specify --kubeconfig= when logging in, so that a new file is created.

oc login <cluster-url> --kubeconfig=<new file> --token=<token> --insecure-skip-tls-verify

or

oc login <cluster-url> --kubeconfig=<new file> -u=<username> -p=<password> --insecure-skip-tls-verify

I would think that the contexts that exist in $KUBECONFIG are visible and thus not dumped into the new file (not tested).

With a new file, we have the following problem though: the new file is not in the env. We can add it to the running system but there's no way for us to persist it.
Maybe we should simply use the first file in the list of file of the $KUBECONFIG and only that file (--kubeconfig=) hoping that oc would not dump the contexts of the other ones into it.

@sbouchet, @msivasubramaniaan, @vrubezhny: thoughts?

[adietish]

On github, this issue was filed at openshift/oc#283
Reading the discussion one can see that the behaviour is intentional and that one should use --kubeconfig to avoid it.
A support case was opened though as of August 2024.