Merge pull request #15756 from rabbitmq/mergify/bp/v4.3.x/pr-15696

ra-v3 branch / QQ v8 release notes (backport #15696)

Author: kjnilsson

release-notes/4.3.0.md

@@ -11,7 +11,7 @@ This release removes the original classic queue storage implementation these day
 known as CQv1. A 2nd generation implementation called CQv2 has been adopted
 as the default starting with `4.2.0`.
 
-This means that attemptes to declare a queue using the following [optional queue arguments](https://www.rabbitmq.com/docs/queues#optional-arguments) will fail:
+This means that attempts to declare a queue using the following [optional queue arguments](https://www.rabbitmq.com/docs/queues#optional-arguments) will fail:
 
  * `x-queue-mode` set to any value
  * `x-queue-version` set to `1`
@@ -20,9 +20,30 @@ Existing classic queues upgraded to CQv2 during an earlier upgrade to `4.2.x` wi
 operating as usual.
 
 
+### Consumer timeouts are no longer evaluated for classic queues and streams.
+
+This release moves consumer timeout handling responsibility into the queues 
+themselves. Also all protocols (except the stream protocol) now evaluate
+consumer timeout for queue types that support them. Classic queues and streams
+never evaluate consumer timeouts as their use cases cause less demand for it.
+
+
 ## Release Highlights
 
-TBD
+This release upgrades the [Ra](https://github.com/rabbitmq/ra) dependency to `3.0.1` and introduces
+`rabbit_fifo` v8, a new version of the quorum queue state machine with several new features and optimisations:
+
+ * **Strict priority queues** with per-priority message counts, correct
+   redelivery ordering, and priority-aware message expiration
+ * **Delayed retry** for quorum queues: configurable increasing backoff when
+   messages are returned
+ * **Consumer timeout** for quorum queues: configurable timeout for
+   unacknowledged messages, with protocol-specific handling for AMQP 1.0 and
+   MQTT
+ * **Recovery snapshots** and **snapshot throttling** to reduce recovery time
+   and improve snapshotting decisions
+ * **Memory optimisations** including compact message references, optimised
+   tuple storage for delayed keys, and removal of `rabbit_fifo_index` usage
 
 
 ## Upgrading to 4.3.0
@@ -86,25 +107,65 @@ compared to other versions.
 
    GitHub issue: [#15075](https://github.com/rabbitmq/rabbitmq-server/pull/15075)
 
-#### Bug Fixes
+ * Quorum queues now support **strict priority queues** with per-priority message counts,
+   correct redelivery ordering across priorities, and priority-aware message expiration scans.
 
- * Quorum queue at-most-once dead lettering for the overflow behaviour `drop-head` now happens in the correct order.
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
 
-   GitHub issue: [#14926](https://github.com/rabbitmq/rabbitmq-server/pull/14926)
+ * Quorum queues now support **delayed retry** with configurable backoff based on delivery count. When messages
+   are returned (via reject, nack, or modify), they can be held in a delayed state before becoming
+   available again. The delay is based on delivery count: `min(min_delay * delivery_count, max_delay)`.
 
-#### Enhancements
+   Configuration is available via queue arguments (`x-delayed-retry-type`, `x-delayed-retry-min`,
+   `x-delayed-retry-max`) or policy keys (`delayed-retry-type`, `delayed-retry-min`,
+   `delayed-retry-max`). The retry type can be set to `disabled`, `all`, `failed`, or `returned`.
 
- * When a message is rejected by a queue, RabbitMQ now provides the queue name and rejection reason to AMQP 1.0 publishers
-   in the `Rejected` outcome. This is particularly useful when multiple queues are bound to an exchange, as it allows
-   publishers to identify which specific queue out of several target queues rejected the message and why
-   (e.g., maximum queue length reached or queue unavailable). Previously, publishers had no way to determine which queue
-   rejected their message or the reason for rejection.
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
 
-   The queue name and reason are included in the `info` field of the `Rejected` outcome's `error` field:
-   * `queue: <queue name>`
-   * `reason: maxlen | unavailable`
+ * Quorum queues now support a configurable **consumer timeout**. When a consumer holds unacknowledged
+   messages beyond the timeout, the messages are returned to the queue. For AMQP 1.0 clients,
+   timed-out deliveries are released via `DISPOSITION(state=released)` instead of detaching the link,
+   allowing the consumer to recover without re-attaching. MQTT consumers are also supported.
 
-   GitHub issue: [#15075](https://github.com/rabbitmq/rabbitmq-server/pull/15075)
+   The timeout can be set via the `x-consumer-timeout` consumer argument, queue argument, `consumer-timeout`
+   policy key, or the global `consumer_timeout` setting in `rabbitmq.conf`.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
+ * A new `consumer_disconnected_timeout` setting controls how long quorum queues wait before returning
+   messages when a consumer's node becomes unreachable due to a network partition. The default is 60 seconds.
+   Configurable via `consumer_disconnected_timeout` in `rabbitmq.conf`, the `consumer-disconnected-timeout`
+   policy key, or the `x-consumer-disconnected-timeout` queue argument.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
+ * Quorum queue **recovery snapshots** reduce recovery time after a member restart by avoiding
+   the need to replay all enqueue commands from the log.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
+ * Quorum queue **snapshot throttling** now uses WAL fill ratio and reclaimable byte tracking
+   to make smarter snapshotting decisions, yielding roughly one snapshot per queue per WAL cycle
+   instead of excessive snapshots in shallow, fast-flowing queues.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
+ * Quorum queue **memory optimisations**: message references now use a compact packed integer
+   representation (fitting into a 59-bit immediate term) when possible, halving per-message
+   memory overhead in many scenarios. The `rabbit_fifo_index` module is no longer used by the
+   main state machine.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
+ * Quorum queues now allow unlimited explicit message returns. The delivery limit is based on
+   `delivery-count` rather than `acquired-count`, so messages can be explicitly returned to the
+   queue without counting towards the delivery limit.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
+ * Purging a quorum queue now also removes at-least-once dead-lettered messages that were pending delivery.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
 
 #### Bug Fixes
 
@@ -114,7 +175,7 @@ compared to other versions.
 
  * Feature flag state in the registry and on disk were not consistent for a period of time during node boot.
 
-  GitHub issue: [#14943](https://github.com/rabbitmq/rabbitmq-server/pull/14943)
+   GitHub issue: [#14943](https://github.com/rabbitmq/rabbitmq-server/pull/14943)
 
 
 ### Stream Plugin
@@ -147,6 +208,11 @@ compared to other versions.
 
    GitHub issue: [#14923](https://github.com/rabbitmq/rabbitmq-server/discussions/14923)
 
+ * Quorum queue delayed retry configuration and status, per-priority message counts, and consumer
+   timeout state are now displayed in the management UI.
+
+   GitHub issue: [#13885](https://github.com/rabbitmq/rabbitmq-server/pull/13885)
+
 
 ### MQTT Plugin
 
@@ -234,6 +300,7 @@ auth_http.authorization_failure_disclosure = true
 
 ### Dependency Changes
 
+ * `ra` was upgraded from [`2.17.2`](https://github.com/rabbitmq/ra/releases/tag/v2.17.2) to [`3.0.1`](https://github.com/rabbitmq/ra/releases/tag/v3.0.1)
  * `cuttlefish` was upgraded to [`3.6.0`](https://github.com/kyorai/cuttlefish/releases)
 
 ## Source Code Archives