Skip to content

Issue 605 grpc host matching#632

Merged
nathanejohnson merged 1 commit into
fabiolb:masterfrom
tommyalatalo:issue-605-grpc-host-matching
Jul 15, 2022
Merged

Issue 605 grpc host matching#632
nathanejohnson merged 1 commit into
fabiolb:masterfrom
tommyalatalo:issue-605-grpc-host-matching

Conversation

@tommyalatalo

Copy link
Copy Markdown

We have implemented grpc host matching by using client-side metadata which is used to match hosts. If the specified metadata host is not found, the request will be routed according to the current default behavior, i.e. to any host serving the grpc service requested.

In doing this we also found that the GRPC max message size is hardcoded in fabio to the default 4mb, we increased this to 1Gb.

@CLAassistant

CLAassistant commented Mar 29, 2019

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@mwalters-workmarket

Copy link
Copy Markdown

This would be very useful for me.
Any timeline on getting merged?

@aaronhurt aaronhurt closed this Jan 27, 2020
@aaronhurt aaronhurt reopened this Jan 27, 2020
@aaronhurt

Copy link
Copy Markdown
Member

Triggering a rebuild ... will merge if tests come back clean. We have Travis setup again.

@aaronhurt

Copy link
Copy Markdown
Member

@tommyalatalo tests pass and I've looked over the code. Would you be willing to add some documentation updates to describe the metadata options?

@ishworg

ishworg commented Apr 7, 2020

Copy link
Copy Markdown

Would be good to have the metadata doco updated to get a better understanding.

cheers

@rocwenlinux

Copy link
Copy Markdown
Contributor

doc is updated, please check if that is enough.

Comment thread docs/content/feature/grpc-proxy.md Outdated
Comment thread docs/content/feature/grpc-proxy.md Outdated
Comment thread docs/content/feature/grpc-proxy.md Outdated
Comment thread proxy/grpc_handler.go Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it is outside the scope of the review right now but this can be more concisely written as:

Suggested change
}
kv := map[string]string{"dstHost":"www.example.com"}
if _, ok := kv["dstHost"]; !ok {
fmt.Print("fail")
}
fmt.Println(kv["dstHost"])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The print "fmt.Println("hosts", md)" is only for debug purpose, it shouldn't be enabled/included in production. Otherwise each request will trigger this print, cause the performance issue.

@ishworg ishworg left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work bud. I have commented few small things.

@rocwenlinux rocwenlinux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed.

@codyja

codyja commented Jun 11, 2020

Copy link
Copy Markdown

We hit the 4mb limit also. Any chance we can get this merged in and a release cut? Thanks!

@nathanejohnson

Copy link
Copy Markdown
Member

@rocwenlinux can you squash this? If so I'll merge this and slate it for 1.6

…ng 'dsthost' in metadata in grpc client side.;set max message size for grpc proxy server and client;
@rocwenlinux

rocwenlinux commented Dec 3, 2020

Copy link
Copy Markdown
Contributor

@rocwenlinux can you squash this? If so I'll merge this and slate it for 1.6

@nathanejohnson squashed.

@nathanejohnson nathanejohnson self-requested a review December 3, 2020 21:30

@nathanejohnson nathanejohnson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make max message sizes configurable?

Comment thread main.go
@@ -161,6 +161,8 @@ func newGrpcProxy(cfg *config.Config, tlscfg *tls.Config) []grpc.ServerOption {
grpc.UnknownServiceHandler(handler),
grpc.StreamInterceptor(proxyInterceptor.Stream),
grpc.StatsHandler(statsHandler),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to have these values be configurable. Could you add these to config/config.go in the Config struct, maybe as GRPCMaxRxMsgSize and GRPCMaxTxSize , and then setting sane default values in config/default.go ? Perhaps setting the current default of 4MB is still sane here, 1gb seems like it might be excessive for many people? And if it's configurable, this allows people to relax the limits.

Comment thread proxy/grpc_handler.go

func (p *grpcConnectionPool) newConnection(ctx context.Context, target *route.Target) (*grpc.ClientConn, error) {
opts := []grpc.DialOption{
grpc.WithDefaultCallOptions(grpc.CallCustomCodec(grpc_proxy.Codec())),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above about the configurable limits

@nathanejohnson

Copy link
Copy Markdown
Member

I'm going to merge this and do a subsequent PR to make these limits configurable. Some good work in here and I'm guessing the original authors have moved on.

@nathanejohnson nathanejohnson merged commit 03f0889 into fabiolb:master Jul 15, 2022
nathanejohnson added a commit that referenced this pull request Jul 15, 2022
nathanejohnson added a commit that referenced this pull request Jul 15, 2022
…able

add configurable grpc message sizes to #632
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants