If the SPIRE server API is placed behind a load balancer that doesn't support or can't have client IP preservation enabled, it will rate limit on the load balancer IPs instead of caller (agent) IPs which can lead to limits being incorrectly applied.
Because SPIRE server terminates TLS, modifying requests before they reach the server isn't feasible. One possible solution would be to add proxy protocol support which would allow the caller IP to be encoded in to requests and allow the server to enforce rate limits properly.