Add an example of application auth using JWTs

[drewwells]

There's beautiful implementations of transport level security in this package https://pkg.go.dev/github.com/spiffe/go-spiffe/v2@v2.3.0/spiffegrpc/grpccredentials. I think these even rotate credentials (not verified) in some way so they work over time as certificates expire.

Since Spire also supports JWTs, implement methods that we can use to create Application level auth using PerRPCCredentials. It also needs to be able to refresh tokens over time, which has been tricky for us to implement correctly and caused many outages in clients. The client may have clockskew with Spire, so expiry in the JWT can not be trusted. I've added an example interceptor that would re-attempt non-streaming calls when 401 errors are encountered.

Here's a rough example of what I am talking about.

Create and cache tokens

type source struct {}

var _ credentials.PerRPCCredentials = &source{}

func (c *source) RequireTransportSecurity() bool { 
    return true 
}
func (s *source) GetRequestMetadata(ctx context.Context, uri ...string) (map[string]string, error) {
    // retrieve a token from workload api if it does not exist
    // some logic to cache token so we don't call spire-agent continuously
    if s.cache == "" {
      s.cache = GetTokenFromSpire()
    }
    return map[string]string{
        "authorization": "Bearer " + s.cache,
    }, nil
}

Mechanism to refresh token on auth failure.

func UnaryInterceptor(s *source) grpc.UnaryClientInterceptor {
    return func(
        ctx context.Context,
        method string,
        req, reply interface{},
        cc *grpc.ClientConn,
        invoker grpc.UnaryInvoker,
        opts ...grpc.CallOption,
    ) error {
        err := invoker(ctx, method, req, reply, cc, opts...)
        if err != nil && grpc.Code(err) == codes.Unauthenticated {
            s.cache = ""
            // Retry the request with the new token which GetRequestMetadata will create
            return invoker(ctx, method, req, reply, cc, opts...)
        }
        return err
    }
}

dialOpts := []grpc.DialOption{
        grpc.WithBlock(),
        grpc.WithUnaryInterceptor(UnaryInterceptor(s)),
}

grpc.DialContext(ctx, hostport, dialOpts...)