fix : remove kube-rbac-proxy sidecar metric proxy container from Operator deployment

[rohanKanojia]

What does this PR do?

• ☑️ Requires chore (deps) : bump controller-runtime to v0.18.7 #1435 to be merged first

As mentioned in #1352 and #1343 (comment) , using kube-rbac-proxy sidecar container is deprecated and is causing issues in configuring Operator resource/limits via Operator subscription.

This PR removes this sidecar container and uses controller-runtime's inbuilt WithAuthenticationAndAuthorization. Now that we don't have a proxy, we can directly post metrics on 8443 and 9443 ports for devworkspace-controller-manager and devworkspace-webhook-server respectively.

Signed-off-by: Rohan Kumar rohaan@redhat.com

What issues does this PR fix or reference?

#1343

Is it tested? How?

After making changes I made sure operator is running and metrics are accessible on 8443 and 9443 ports.

• setup OpenShift cluster
• Install DWO from this PR:

oc apply -f - <<EOF
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: devworkspace-operator-catalog
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: docker.io/rohankanojia/devworkspace-operator-index:next
  publisher: Red Hat
  displayName: DevWorkspace Operator Catalog
  updateStrategy:
    registryPoll:
      interval: 5m
EOF

oc apply -f - <<EOF
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: devworkspace-operator
  namespace: openshift-operators
spec:
  channel: next
  installPlanApproval: Automatic
  name: devworkspace-operator
  source: devworkspace-operator-catalog
  sourceNamespace: openshift-marketplace
EOF

• Check operator pods, they'd have only one running container (previously all pods have 2 containers each)

  devworkspace-operator : $ oc get pods
  NAME                                               READY   STATUS    RESTARTS      AGE
  devworkspace-controller-manager-56679dd464-lnh2g   1/1     Running   0             15m
  devworkspace-webhook-server-5874545c5c-58n4s       1/1     Running   0             15m
  devworkspace-webhook-server-5874545c5c-ntfmt       1/1     Running   1 (15m ago)   15m
  

• Port forward metrics service and try to access metrics

  kubectl  port-forward service/devworkspace-controller-metrics 8443:8443 &
  kubectl port-forward service/devworkspace-webhookserver 9443:9443
  
  # [In separate session]
   NAMESPACE=devworkspace-controller 
   kubectl create clusterrolebinding dw-metrics \
  --clusterrole=devworkspace-controller-metrics-reader \
  --serviceaccount=${NAMESPACE}:devworkspace-controller-serviceaccount
  TOKEN=$(kubectl create token devworkspace-controller-serviceaccount -n ${NAMESPACE} --duration=1h)
  
  curl -k -H "Authorization: Bearer ${TOKEN}"   https://localhost:9443/metrics
  
  curl -k -H "Authorization: Bearer ${TOKEN}" https://localhost:8443/metrics

Configuring memory and CPU requests/limits

• Patch the existing Subscription to have CPU requests and limits

kubectl patch subscription  devworkspace-operator \
  -n openshift-operators \
  --type merge \
  -p '{
    "spec": {
      "config": {
        "resources": {
          "requests": {
            "cpu": "100m",
            "memory": "256Mi"
          },
          "limits": {
            "cpu": "500m",
            "memory": "512Mi"
          }
        }
      }
    }
  }'

• After applying abovementioned changes, check state of DevWorkspace Operator Pods to see if memory requests/limits have been added or not:

kubectl get pods -l app.kubernetes.io/name=devworkspace-controller   -o custom-columns=NAME:.metadata.name,CPU_REQ:.spec.containers[*].resources.requests.cpu,CPU_LIM:.spec.containers[*].resources.limits.cpu,MEM_REQ:.spec.containers[*].resources.requests.memory,MEM_LIM:.spec.containers[*].resources.limits.memory

NAME                                               CPU_REQ   CPU_LIM   MEM_REQ   MEM_LIM
devworkspace-controller-manager-6ddc89c564-4fxwt   100m      500m      256Mi     512Mi

PR Checklist

• E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)
  • v8-devworkspace-operator-e2e: DevWorkspace e2e test
  • v8-che-happy-path: Happy path for verification integration with Che

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[rohanKanojia]

/ok-to-test

[rohanKanojia]

/ok-to-test

[rohanKanojia]

/ok-to-test

[rohanKanojia]

/ok-to-test

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dkwon17, rohanKanojia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [dkwon17]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dkwon17]

Since kube-rbac-proxy was configured to use HTTPS with --secure-listen-address=0.0.0.0:8443, could we also set the metrics server to use HTTPS?

[dkwon17]

Basically, this documentation should work the same way as before: https://eclipse.dev/che/docs/stable/administration-guide/monitoring-the-dev-workspace-operator/

[rohanKanojia]

Good catch, I've added SecureServing: true in metricsserver.Options. It enables https in metrics server

[openshift-ci]

New changes are detected. LGTM label has been removed.