Excessive consul logging

[nkissebe]

Maybe I have something misconfigured in consul (very possible) but fabio logs a lot of messages like

2017/12/12 20:27:27 [INFO] consul: Manual config changed to #4980182

nearly every second. If that is normalish then I don't think it is very useful at the INFO log level and should be at the DEBUG level as it provides no useful information unless you were heavily into debugging something. Just my opinion.

[magiconair]

That means your consul state changes every second. Check for flapping services or - more likely - volatile data in the health check output. Like time stamps or process ids or something else that changes every time the check is executed. There are mitigation procedures on the consul side. It in general I advise to try and fix the root cause first.

[ctlajoie]

@magiconair I also have an excessive number of these in my logs. Every few seconds or so, and sometimes several in rapid succession. None of my services are flapping. I think maybe the consul state changes every time a check is performed. I have many service checks that are performed every 10 seconds. I agree with OP, I think DEBUG level would be more appropriate for these messages.

[magiconair]

@ctlajoie no time stamps in the health check output? I’ll switch that to debug and work on the consul change rate indicator as described in #368.

[slackpad]

This issue (hashicorp/consul#3642) fixed in 1.0.2 could cause a lot of extra churn for services with tags.

[magiconair]

Thanks @slackpad