Native STOMP#9141
Conversation
ikavgo
force-pushed
the
ik-native-stomp
branch
from
7856889 to
41ce7a9
Compare
ikavgo
force-pushed
the
ik-native-stomp
branch
from
41ce7a9 to
cb0eda2
Compare
ikavgo
force-pushed
the
ik-native-stomp
branch
from
cb0eda2 to
64b59ef
Compare
|
Regarding Connection closure, UTF, queues utilization - the idea is to merge this first, then tweak standard support and resources. |
|
Thank you @ikvmw. Overall this PR goes into the right direction. There is some more work to do to make it production ready in order to merge it into
creates a quorum queue on creates a quorum queue on There is even a test file called Same applies to streams. |
| start(normal, []) -> | ||
| Config = parse_configuration(), | ||
| Listeners = parse_listener_configuration(), | ||
| rabbit_global_counters:init([{protocol, stomp}]), |
There was a problem hiding this comment.
I'd vote to differentiate by STOMP versions, i.e. 1.0, 1.1, 1.2 since that's what we currently do for MQTT (3.1, 3.1.1, 5.0).
There was a problem hiding this comment.
Where are the global counters per protocol and queue type initialised? Are they missing?
| SMsgSeqNos = lists:usort(MsgSeqNos), | ||
| UnconfirmedCutoff = case rabbit_confirms:is_empty(UC) of | ||
| true -> lists:last(SMsgSeqNos) + 1; | ||
| false -> rabbit_confirms:smallest(UC) | ||
| end, | ||
| Cutoff = lists:min([UnconfirmedCutoff | NegativeMsgSeqNos]), | ||
| {Ms, Ss} = lists:splitwith(fun(X) -> X < Cutoff end, SMsgSeqNos), | ||
| State1 = case Ms of | ||
| [] -> State; | ||
| _ -> MkMsgFun(lists:last(Ms), true, State) | ||
| end, | ||
| lists:foldl(fun(SeqNo, StateN) -> | ||
| MkMsgFun(SeqNo, false, StateN) | ||
| end, State1, Ss). |
There was a problem hiding this comment.
Where is this code tested? If it's not tested, please add tests.
| {eol, Actions} -> | ||
| State1 = handle_queue_actions(Actions, State), | ||
| State2 = handle_consuming_queue_down_or_eol(QRef, State1), | ||
| {ConfirmMXs, UC1} = | ||
| rabbit_confirms:remove_queue(QRef, State1#proc_state.unconfirmed), | ||
| %% Deleted queue is a special case. | ||
| %% Do not nack the "rejected" messages. | ||
| State3 = record_confirms(ConfirmMXs, | ||
| State2#proc_state{unconfirmed = UC1}), | ||
| {ok, State3#proc_state{queue_states = rabbit_queue_type:remove(QRef, QStates0)}}; |
There was a problem hiding this comment.
Where is this code tested? If it's not tested, please add tests.
| ({rejected, _QRef, MsgSeqNos}, S0) -> | ||
| {U, Rej} = | ||
| lists:foldr( | ||
| fun(SeqNo, {U1, Acc}) -> | ||
| case rabbit_confirms:reject(SeqNo, U1) of | ||
| {ok, MX, U2} -> | ||
| {U2, [MX | Acc]}; | ||
| {error, not_found} -> | ||
| {U1, Acc} | ||
| end | ||
| end, {S0#proc_state.unconfirmed, []}, MsgSeqNos), | ||
| S = S0#proc_state{unconfirmed = U}, | ||
| %% Don't send anything, no nacks in STOMP | ||
| record_rejects(Rej, S); | ||
| ({queue_down, QRef}, S0) -> | ||
| handle_consuming_queue_down_or_eol(QRef, S0); | ||
| %% TODO: I have no idea about the scope of credit_flow | ||
| ({block, QName}, S0) -> | ||
| credit_flow:block(QName), | ||
| S0; | ||
| ({unblock, QName}, S0) -> | ||
| credit_flow:unblock(QName), | ||
| S0; | ||
| %% TODO: in rabbit_channel there code for handling | ||
| %% send_drained and send_credit_reply | ||
| %% I'm doing catch all here to not crash? | ||
| (_, S0) -> | ||
| S0 |
There was a problem hiding this comment.
Where is this code tested? If it's not tested, please add tests.
| parse_endpoint(undefined) -> | ||
| parse_endpoint("/queue"); | ||
| parse_endpoint(Destination) when is_binary(Destination) -> | ||
| parse_endpoint(unicode:characters_to_list(Destination)); |
There was a problem hiding this comment.
Where is this code tested? If it's not tested, please add tests.
There was a problem hiding this comment.
parse_endpoint called on each SEND and SUBSCRIBE. Should be many dozens hits across our test suites
There was a problem hiding this comment.
I observe 0 hits for these 2 clauses
| rabbit_core_metrics:channel_queue_down({self(), QName}), | ||
| erase({queue_stats, QName}), | ||
| [begin | ||
| rabbit_core_metrics:channel_queue_exchange_down({self(), QX}), | ||
| erase({queue_exchange_stats, QX}) | ||
| end || {{queue_exchange_stats, QX = {QName0, _}}, _} <- get(), | ||
| QName0 =:= QName]. |
There was a problem hiding this comment.
Why are these stats erased? It looks like there aren't written in the first place?
|
|
||
| maybe | ||
| {ok, User} ?= rabbit_access_control:check_user_login(Username, AuthProps), | ||
| {ok, AuthzCtx} ?= check_vhost_access(VHost, User, Addr), |
There was a problem hiding this comment.
Should we additionally check for whether the vhost exists and whether the vhost is alive? (see Native MQTT).
| end, | ||
|
|
||
| maybe | ||
| {ok, User} ?= rabbit_access_control:check_user_login(Username, AuthProps), |
There was a problem hiding this comment.
rabbit_core_metrics:auth_attempt_failed() is missing if user login is refused.
| Delivery = #'basic.deliver'{consumer_tag = ConsumerTag, | ||
| delivery_tag = DeliveryTag, | ||
| redelivered = Redelivered, | ||
| exchange = ExchangeNameBin, | ||
| routing_key = RoutingKey}, |
There was a problem hiding this comment.
Creating an AMQP 0.9.1 #basic.deliver{} is unnecessary. The STOMP protocol is used to send the message to the client, not the AMQP 0.9.1 protocol. I think you should restructure the code, specifically rabbit_stomp_util:headers_extra/4 should not accept a #basic.deliver{} anymore.
| @@ -20,10 +20,6 @@ | |||
| -include("rabbit_stomp_frame.hrl"). | |||
| -include_lib("amqp_client/include/amqp_client.hrl"). | |||
ikavgo
force-pushed
the
ik-native-stomp
branch
3 times, most recently
from
f61964c to
256810d
Compare
ikavgo
force-pushed
the
ik-native-stomp
branch
4 times, most recently
from
b565f15 to
b6c9849
Compare
ikavgo
force-pushed
the
ik-native-stomp
branch
2 times, most recently
from
a786625 to
4c55f0e
Compare
…nary headers
Remove the AMQP 0-9-1 client dependency from the STOMP plugin, making it a
truly native protocol implementation (following the MQTT native pattern).
Key changes:
- Remove amqp_client from DEPS; replace #amqp_adapter_info{} with #conn_info{}
record storing connection endpoints directly via rabbit_net:socket_ends/2
- Replace #'basic.deliver'{} construction with direct parameter passing
- Add per-version global counters (STOMP 1.0/1.1/1.2) and per-queue-type
initialization, publisher/consumer lifecycle counters
- Add force_event_refresh handler, rabbit_trace integration (tap_in/tap_out),
consumer timeout handling, vhost existence and connection limit checks
- Add structured logger metadata (connection, vhost, user) for traceable logs
- Rewrite frame parser: bulk binary scanning for commands and headers (zero-copy
fast path), byte-by-byte fallback only for escape sequences, bounded
accumulation in all phases, unknown commands produce frames (not parse errors)
- Convert all header representation from charlists to binaries throughout:
frame parser, serializer, header accessors, rabbit_stomp_util,
rabbit_stomp_processor, and all test files
- Restructure python_SUITE: auto-generate per-function CT test cases from
Python test files, organized into nested groups by class
- Fix rabbitmq_cli plugin tests to reflect STOMP's independence from amqp_client
- Update rabbit_web_stomp_handler for all API changes
`command/2` returned a 3-tuple `{ok, State, none}` when an unauthenticated
client sent a non-CONNECT frame. The reader expects a 2-tuple, `{ok, State}`.
1. A negative content-length crashed the parser in `parse_known_body/4`.
2. A non-numeric content-length (e.g. "abc") crashed `integer_header/2`
with a `badarg` exception
3. A content-length pointing past the NUL
terminator resulted in a `badmatch`.
All three are pre-auth exceptions.
With these changes:
1. Negative values now return `{error, {invalid_content_length, N}}`
2. Non-numeric values are treated as absent
3. A missing NUL terminator returns `{error, missing_body_terminator}`
5 participants
NOTE: Bazel tests pass locally, I don't like the code yet though.
TODO: Double check Management UI correctly shows Connection information.
Proposed Changes
Don't use AMQP 0-9-1 as backend for STOMP.
Types of Changes
What types of changes does your code introduce to this project?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply.You can also fill these out after creating the PR.
If you're unsure about any of them, don't hesitate to ask on the mailing list.
We're here to help!
This is simply a reminder of what we are going to look for before merging your code.
CONTRIBUTING.mddocumentFurther Comments
Documentation update: rabbitmq/rabbitmq-website#1713