Allow direct on MQTT retained message stream

[joda01]

Proposed change

Activate the allow_direct flag per default on the MQTT retained message stream.

Use case

This allows the application to request retained messages directly from NATS JetStream, which is necessary for emulating retained message behaviour when interacting with MQTT messages using NATS rather than an MQTT client.

Without this possibility a retained message form the stream can only be captured by connecting via MQTT client.

Contribution

No response

[MauriceVanVeen]

Is it strictly necessary to be done via DirectGet, doesn't the usual MsgGet suffice?

That uses the $JS.API.STREAM.MSG.GET.<stream> API versus $JS.API.DIRECT.GET.<stream>, which is also what the CLI uses with nats stream get.

[joda01]

I was wondering because the recommended way for high performance is the Direct Message API, yet I found no reason why it is deactivated in the MQTT JetStream.

However, a second reason for using the direct API is that I can use it with multiple accounts via export/import, which is not possible with the $JS.API.STREAM.MSG.GET method.

Use Case: Having an IOT account and an INTERNAL account. IoT devices are using MQTT and scooped to the IOT account. From the INTERNAL account, where the businesses logic is running I want to access the messages from the IOT account. To get the retained messages on startup of my service I need to request it. With the direct API and import/export from IOT to INTERNAL account this works quiet fine.

[neilalexander]

I think there's no problem if you want to enable direct gets, nats stream update \$MQTT_rmsgs --allow-direct.

[joda01]

Yes this is working but the stream is only created when the fist MQTT message is received. This is a problem when the NATS server is deployed because I have to wait for the first MQTT message received before I can send the --allow-direct.

In addition this has to be done for each account which makes the deployment quiet complicated.
If not enabling the direct feature per default probably a config flag in the mqtt config would be fine.