Fabio hangs for 30+ seconds for 204 response

[discobean]

Hi,

I have an interesting issue that has come up, this has occurred in 1.1.1 and is also occurring in current latest build.

We have fabio routing to a mulesoft API in the backend, when this mulesoft API returns a 204 fabio is delayed 30+ seconds. I have done ngrep's to confirm that this is the case:

Mule ngrep from fabio <-> mule:

T 10.90.42.170:46254 -> 10.90.34.187:8004 [AP]
  POST /api/Vessel/v1/Schedule/VisitSearch HTTP/1.1..Host: api-20.prod..User-Agent: curl/7.37.1
  ..Content-Length: 51..Accept: */*..Cache-Control: no-cache..Content-Type: application/json..Forwarded: f
  or=172.31.14.127; proto=http; by=10.90.42.170..Postman-Token: a42a153f-0755-9902-1168-3d9d50f16076..Toke
  n: c9c1355f-70f3-4659-ae50-XXX..X-Forwarded-For: 172.31.14.127..X-Forwarded-Port: 56790..X-Forw
  arded-Proto: http..X-Real-Ip: 172.31.14.127..Accept-Encoding: gzip....{."TerminalCode": "CTLPB",."VisitI
  D": "123456789".}                                                                                       
##
T 10.90.34.187:8004 -> 10.90.42.170:46254 [AP]
  HTTP/1.1 204 No Content..Transfer-Encoding: chunked..Content-Type: application/json;charset=UTF-8..Date:
   Wed, 22 Jun 2016 03:20:33 GMT....              

Fabio ngrep Client <-> Fabio:

T 172.31.14.127:56790 -> 10.90.42.170:9999 [AP]
  POST /api/Vessel/v1/Schedule/VisitSearch HTTP/1.1..User-Agent: curl/7.37.1..Accept: */*..Content-Type: application/json..token: c9c1355f-70f3-4659-ae50-8
  f08ae90126b..Host: api-20.prod...Cache-Control: no-cache..Postman-Token: a42a153f-0755-9902-1168-XXX..Content-Length: 51....{."Termina
  lCode": "CTLPB",."VisitID": "123456789".}                                                                                                                
##
T 10.90.42.170:9999 -> 172.31.14.127:56790 [AP]
  HTTP/1.1 204 No Content..Content-Type: application/json;charset=UTF-8..Date: Wed, 22 Jun 2016 03:20:33 GMT....       
^^^^ This second packet is not generated by fabio until 30+ seconds passes

I can see the packet hit fabio, then immediately routes it to mule, which immediately responds with the 204, but it takes 30+ seconds for fabio to generate and send the packet back to the client (Packet 2 in the second section)

When I hit the service directly it works fine. Have you seen anything like that before?

I will open some ports and let give you access to test the API via the web if you wish.

[discobean]

Actually after more debugging I think it is mule keeping the connection open

[discobean]

Just an update, confirmed this is an issue with mule 204 response, there is a scenario where mulesoft will return a 204 response, but the connection will remain open waiting for more response data.

ie. when I try the mule service via curl and no proxies, I get the 204 response, but then mule just keeps the connection open and curl times out after 30 seconds. Which explains the 30+ seconds delay.