STREAM.UPDATE with replica scale-down drops internal subscriptions for re-added subjects

[johnweldon]

Observed behavior

When a clustered R3 stream receives a STREAM.UPDATE with num_replicas: 0 (normalized to R1), the stream scales down from R3 to R1. If a subsequent STREAM.UPDATE restores the original subjects and sets num_replicas: 3, the stream scales back up to R3 and the config correctly reflects all subjects, but internal subscriptions for subjects that were removed during the scale-down are never recreated. Messages published to those subjects are silently dropped (no responders). The stream config is misleading: it shows the subjects are present, but no routing occurs.

A leader step-down immediately resolves the issue by forcing subscribeToStream() to run with mset.active = false.

The root cause is the mset.active guard in subscribeToStream() (stream.go). During R3->R1 scale-down, mset.active is set to true with subscriptions for the reduced subject list. During R1->R3 scale-up, subscribeToStream() is called again but returns immediately because mset.active is already true, skipping subscription creation for the restored subjects.

Expected behavior

After a STREAM.UPDATE that restores subjects and scales the stream back to R3, all subjects in the config should have active internal subscriptions. Messages published to any configured subject should be routed to the stream.

Server and client version

• nats-server 2.12.6

Host environment

Reproduced locally on macOS (3-node cluster, localhost).

Steps to reproduce

Minimal repro (requires nats-server and nats CLI):

# 1. Start a 3-node cluster (any standard 3-node config)

# 2. Create an R3 stream with multiple subjects
nats stream add test --subjects "A.>,B.>,C.>" --replicas 3 --retention work \
  --storage file --discard old --max-msgs=-1 --max-bytes=104857600 \
  --max-msg-size=10485760 --max-msgs-per-subject=1 --dupe-window=2m \
  --no-allow-direct --defaults

# 3. Verify baseline
nats req A.test "hello" --timeout=3s   # => JS ack (works)
nats req C.test "hello" --timeout=3s   # => JS ack (works)

# 4. Send STREAM.UPDATE with reduced subjects and num_replicas:0
#    (num_replicas:0 is normalized to Replicas:1, triggering R3->R1 scale-down)
nats req '$JS.API.STREAM.UPDATE.test' \
  '{"name":"test","subjects":["A.>","B.>"],"retention":"workqueue","max_consumers":0,"max_msgs":0,"max_msgs_per_subject":1,"max_bytes":104857600,"max_age":0,"max_msg_size":10485760,"storage":"file","num_replicas":0}' \
  --raw --timeout=5s
sleep 8   # wait for R3->R1 scale-down

# 5. Restore full config with num_replicas:3
nats req '$JS.API.STREAM.UPDATE.test' \
  '{"name":"test","subjects":["A.>","B.>","C.>"],"retention":"workqueue","max_consumers":-1,"max_msgs":-1,"max_msgs_per_subject":1,"max_bytes":104857600,"max_age":0,"max_msg_size":10485760,"storage":"file","num_replicas":3}' \
  --raw --timeout=5s
sleep 10  # wait for R1->R3 scale-up

# 6. Verify: stream config shows all subjects, but C.> has no subscriptions
nats stream info test                  # => shows subjects: [A.> B.> C.>] (correct)
nats req C.test "hello" --timeout=3s   # => no responders (BUG)
nats req A.test "hello" --timeout=3s   # => JS ack (works)

# 7. Step-down fixes it
nats stream cluster step-down test
sleep 3
nats req C.test "hello" --timeout=3s   # => JS ack (works)

An automated reproduction script with 4 test cases (3 reproduce the bug at 100%, 1 control) is available. The control case uses num_replicas: 3 throughout (no scale-down) and passes, confirming the R3->R1->R3 transition is the trigger.