Concurrent map writes error

[devdattakulkarni]

Saw the following error in a production deployment of KubePlus (version: 4.0.2)

HTTP method:CREATE
2025/01/21 21:50:41 http: TLS handshake error from 10.0.2.30:47306: read tcp 10.0.2.123:443->10.0.2.30:47306: read: connection reset by peer
fatal error: concurrent map writes

goroutine 11266 [running]:
main.handleCustomAPIs(0x2258c20?, {0xc00070139a, 0x6})
/home/vagrant/go/src/github.com/cloud-ark/kubeplus/mutating-webhook/webhook.go:1410 +0x665
main.(*WebhookServer).mutate(0xc0003df080?, 0xc0003df110, {0xc00070139a, 0x6})
/home/vagrant/go/src/github.com/cloud-ark/kubeplus/mutating-webhook/webhook.go:269 +0x6ba
main.(*WebhookServer).serve(0xc00050e028, {0x1787348, 0xc0007ee2a0}, 0xc000325200)
/home/vagrant/go/src/github.com/cloud-ark/kubeplus/mutating-webhook/webhook.go:1832 +0x2f4
net/http.HandlerFunc.ServeHTTP(0xc000512230?, {0x1787348?, 0xc0007ee2a0?}, 0x10?)
/usr/local/go/src/net/http/server.go:2166 +0x29
net/http.(*ServeMux).ServeHTTP(0x411d65?, {0x1787348, 0xc0007ee2a0}, 0xc000325200)
/usr/local/go/src/net/http/server.go:2683 +0x1ad
net/http.serverHandler.ServeHTTP({0x1782610?}, {0x1787348?, 0xc0007ee2a0?}, 0x6?)
/usr/local/go/src/net/http/server.go:3137 +0x8e
net/http.(*conn).serve(0xc000490000, {0x1792e98, 0xc0006a0150})
/usr/local/go/src/net/http/server.go:2039 +0x5e8
created by net/http.(*Server).Serve in goroutine 50
/usr/local/go/src/net/http/server.go:3285 +0x4b4

goroutine 1 [chan receive, 22 minutes]:
main.main()
/home/vagrant/go/src/github.com/cloud-ark/kubeplus/mutating-webhook/main.go:99 +0x6fc

[devdattakulkarni]

In the helmer container, saw following errors:

ABC helm install cmd:helm install atomiccore1-1-1-testdev111 ./atomic-core -f /chart/atomic-core/overrides.yaml -n testdev111
StdOutput:
StdErr: Error: Error: INSTALLATION FAILED: 24 errors occurred:
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": read tcp 10.0.2.30:47360->10.0.2.123:443: read: connection reset by peer
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": EOF
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": EOF
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": dial tcp 10.0.2.123:443: connect: connection refused
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": dial tcp 10.0.2.123:443: connect: connection refused
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": dial tcp 10.0.2.123:443: connect: connection refused
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"
* Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": no endpoints available for service "crd-hook-service"