We added a flag in https://github.com/spiffe/spire-controller-manager/pull/547 to up the grpc message size as a workaround for lack of pagination. In the longer run we should support pagination so there is no longer an arbitrary limit.
