Fabio immediately drop routes when consul agent unavailable for a while

[andrekzn]

There is a problem, when consul agent on the host have been restarted, for in-place version update. At this time fabio detect consul unavailability, and delete routes. At this moment we see lots of 5xx errors, which is expected. All this happen about to 2-5 seconds. When agent is started, all routes has back again.
There in no big deal about it, but would be great if we can set some timeout on the consul agent availability. Mb there is some settings, but I can't find it.

[tristanmorgan]

I'm taking a guess but does registry.timeout = 10s do anything for this?

[andrekzn]

Unfortunately increasing the value to registry.timeout = 20s did not solve the problem. Imo, default value 10s is enough. Consul starts much faster.
Docs says that this option affect during startup, apparently this is true.

[nathanejohnson]

@andrekzn do you have the ability to test against master?

#811

Setting these new options might possibly alleviate the issue.

[andrekzn]

@nathanejohnson hi, sorry for delay.
I build a test docker image from master branch, added options in the fabio startup config from the above PR. But the behavior remains the same, routes are still deleted, when consul agent unavailable for a short period. Maybe I'm doing something wrong...

[andrekzn]

In general, I think I found another reason of this behavior. Nevertheless, the consul sends a request to remove the service from the catalog when restarting. And from here the specific route is removed. Not all are deleted, but only those apps that are running on the same host as fabio. In our infra it is ok to run load-balancer and apps on the same nomad agent.

I will test it again and post the results. Thanks anyway.

[andrekzn]

I close this issue, because it is not directly related to fabio. In my case this is the default behavior of the consul agent.