[SPARK-55399][K8S] Improve KubernetesDriverEndpoint to use patch instead of edit API

### What changes were proposed in this pull request?

This PR aims to improve `KubernetesDriverEndpoint` to use `patch` instead of `edit` API

### Why are the changes needed?

**Network Efficiency**

- `edit` requires fetching the entire resource and sending the full updated resource back.
- `patch` only transmits the specific changes, making it much more network-efficient.

**Concurrency & Conflict Resolution**

- `edit` typically follows a Get -> Modify -> Update (PUT) pattern. Using this pattern creates a race condition where, if another client modifies the resource in between, a 409 Conflict error occurs due to a mismatched resourceVersion.
- `patch` sends only the changes (delta) to the server, where the merge is handled server-side. This significantly reduces the risk of conflicts, especially for simple operations like adding an annotation.

### Does this PR introduce _any_ user-facing change?

This will reduce the overhead of K8s control plane and the chance of 409 error.

### How was this patch tested?

Pass the CIs with newly updated test case.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: `Sonnet 4.5` on `Claude Code`

Closes #54179 from dongjoon-hyun/SPARK-55399.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

Author: dongjoon-hyun

resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala

@@ -353,10 +353,11 @@ private[spark] class KubernetesClusterSchedulerBackend(
             kubernetesClient.pods()
               .inNamespace(namespace)
               .withName(x.podName)
-              .edit({p: Pod => new PodBuilder(p).editMetadata()
+              .patch(PATCH_CONTEXT, new PodBuilder()
+                .withNewMetadata()
                 .addToLabels(SPARK_EXECUTOR_ID_LABEL, newId)
                 .endMetadata()
-                .build()})
+                .build())
           }
         }
         executorService.execute(labelTask)