Bug in consul 1.4.3

[sielaq]

Hi Frank
how are you :) long time... ;)

Fabio: 1.5.11
with
Consul: 1.4.2 - Fabio works all fine
but
Consul: 1.4.3 - Fabio does not update routes or even loose routes randomly

registering 4 services:

2019/03/09 10:36:17 [INFO] Config updates
+ route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31226/
2019/03/09 10:36:22 [INFO] Config updates
+ route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31767/
2019/03/09 10:36:22 [INFO] Config updates
+ route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31788/
2019/03/09 10:36:24 [INFO] Config updates
+ route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31508/

deregistering 3 services, fabio removes 4

2019/03/09 10:36:45 [INFO] Config updates
- route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31226/
2019/03/09 10:36:45 [INFO] Config updates
- route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31788/
- route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31767/
- route add python-smooth python-smooth.service.consul/ http://192.168.99.100:31508/

[sielaq]

suspecting this, but not 100% sure:
hashicorp/consul@2aac4d5

[sielaq]

and might be connected with
hashicorp/consul#5457

[magiconair]

Hi @sielaq, is this a Fabio or Consul bug?

[magiconair]

I'm fine. Thanks for asking. It has been a while indeed :)

[sielaq]

@magiconair, At the begging I was suspecting an incompatibility - sort of change in api behavior.
But now, I suspect, that this is a Consul bug. Since this is a serious accusation (A consul bug? can't be) I would prefer also you to verify that :)

[sielaq]

ps: My suspicious is: that Consul remove all health checks even if only one service was removed, so Fabio loose all routes.
(The reason, why in my case, was so hard to understand what was going on, was that Registrator constantly was trying to heal the situation...)

[magiconair]

My time to work on this project is quite limited. Maybe someone from the community can pick this up? — Frank Schröder

…

On 10. Mar 2019, at 07:28, Wojciech Sielski ***@***.***> wrote: ps: My suspicious is: that Consul remove all health checks even if only one service was removed, so Fabio loose all routes. (The reason, why in my case, was so hard to understand what was going on, was that Registrator constantly was trying to heal the situation...) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[aaronhurt]

We recently updated a lab to 1.4.3 ... I’ll take a look Monday and see if I can reproduce this behavior.

…

-- Aaron

On Mar 10, 2019, at 12:39 AM, Frank Schröder ***@***.***> wrote: My time to work on this project is quite limited. Maybe someone from the community can pick this up? — Frank Schröder > On 10. Mar 2019, at 07:28, Wojciech Sielski ***@***.***> wrote: > > ps: My suspicious is: that Consul remove all health checks even if only one service was removed, so Fabio loose all routes. > (The reason, why in my case, was so hard to understand what was going on, was that Registrator constantly was trying to heal the situation...) > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[pschultz]

This does seem to be an issue with Consul, actually.

# register services
for port in 31508 31226 31767 31788; do
    jq --arg port $port -n '{
        "Address": "127.0.0.1",
        "Port": $port|tonumber,
        "Check": {
            "Interval": "5s",
            "HTTP": ("http://localhost:"+$port+"/smooth")
        },
        "Name": "python-smooth",
        "ID": ("python-smooth-"+$port),
        "Tags": [
            "urlprefix-/smooth"
        ]
    }' | curl -XPUT localhost:8500/v1/agent/service/register -d@-
done

Consul logs and UI show four healthy instances:

    2019/03/11 09:39:17 [INFO] agent: Synced node info
    2019/03/11 09:39:19 [INFO] agent: Synced service "python-smooth-31508"
    2019/03/11 09:39:19 [INFO] agent: Synced service "python-smooth-31226"
    2019/03/11 09:39:19 [INFO] agent: Synced service "python-smooth-31767"
    2019/03/11 09:39:19 [INFO] agent: Synced service "python-smooth-31788"
    2019/03/11 09:39:20 [INFO] agent: Synced check "service:python-smooth-31226"
    2019/03/11 09:39:21 [INFO] agent: Synced check "service:python-smooth-31767"
    2019/03/11 09:39:23 [INFO] agent: Synced check "service:python-smooth-31508"
    2019/03/11 09:39:24 [INFO] agent: Synced check "service:python-smooth-31788"

However after deregistering one service, Consul 1.4.3 removes all four checks where 1.4.2 removes only the service:python-smooth-31508 check (as expected):

$ consul services deregister -id python-smooth-31508

    2019/03/11 09:40:31 [INFO] agent: Deregistered service "python-smooth-31508"
    2019/03/11 09:40:31 [INFO] agent: Deregistered check "service:python-smooth-31226"
    2019/03/11 09:40:31 [INFO] agent: Deregistered check "service:python-smooth-31767"
    2019/03/11 09:40:31 [INFO] agent: Deregistered check "service:python-smooth-31788"
    2019/03/11 09:40:31 [INFO] agent: Deregistered check "service:python-smooth-31508"

[pschultz]

hashicorp/consul#5456 fits the bill.

[kaspergrubbe]

Consul 1.4.4 incoming, thanks for noticing @sielaq

[magiconair]

Can someone update the readme, please?

[aaronhurt]

@magiconair will do

Sorry, guess you meant Consul readme ;p