Skip to content

Commit 41ea407

Browse files
konardkonard
authored
Merge pull request #1892 from link-assistant/issue-1891-81a6f573f174
fix(telegram): de-duplicate /queue display + markdown-safe message splitting (#1891)
2 parents 83e411e + 530a4df commit 41ea407

18 files changed

Lines changed: 1349 additions & 120 deletions

‎.changeset/issue-1891-queue-deduplication.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
'@link-assistant/hive-mind': minor
3+
---
4+
5+
fix(telegram): de-duplicate `/queue` display and split long messages without breaking markdown (#1891)
6+
7+
The `/queue` (alias `/solve_queue`) detailed display repeated the same words on every
8+
line — every executing row said `(processing, …)`, every waiting row said
9+
`(waiting, …)`, and the (almost always identical) per-item waiting reason was printed
10+
once per item. Empty queues were also still printed. This wasted vertical space and
11+
pushed real data off screen.
12+
13+
Display changes (`formatDetailedStatus` + queue helpers):
14+
15+
- Executing rows now render compactly as `• owner/repo#number (▶️ <dur>)` and pending
16+
rows as `• owner/repo#number (⏳ <dur>)` — the status word is replaced by the emoji
17+
marker inside the duration parenthesis.
18+
- Processing, pending, completed, and failed entries are split into distinct
19+
compact lists per tool, with counts only on those list labels instead of a
20+
duplicated `(pending: n, processing: n)` tool-header summary.
21+
- The shared waiting reason is shown **once per tool** (only when all pending items
22+
agree on it) instead of once per item.
23+
- Empty queues are skipped entirely.
24+
- All queued items are listed (no per-queue truncation on the active lists).
25+
26+
Message-splitting changes (`splitTelegramMessageText` in `telegram-safe-reply.lib.mjs`,
27+
the single universal splitter every Telegram send path funnels through):
28+
29+
- Splitting now happens only on line boundaries, so inline Markdown entities
30+
(bold/italic/links) are never cut in half.
31+
- Fenced code blocks stay balanced per chunk: a split inside a code block closes the
32+
fence at the end of one chunk and reopens it — repeating the language — at the start
33+
of the next. The original fence marker (``` vs `~~~`) and indentation are preserved.
34+
- Pathologically long single lines are hard-split as a fallback.
35+
36+
Both behaviours are covered by extensive new tests
37+
(`tests/test-telegram-message-split-1891.mjs`, `tests/test-queue-compact-display-1891.mjs`).

‎docs/case-studies/issue-1891/README.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Case Study — Issue #1891
2+
3+
**Title:** We have too much duplication in `/queue` display
4+
5+
**Issue:** https://github.com/link-assistant/hive-mind/issues/1891
6+
**Pull Request:** https://github.com/link-assistant/hive-mind/pull/1892
7+
**Status:** Implemented
8+
9+
This folder is the deep case study for issue #1891, compiled as required by the
10+
issue itself ("make sure we compile that data to `./docs/case-studies/issue-{id}`
11+
folder, and use it to do deep case study analysis"). It contains:
12+
13+
| File | Purpose |
14+
| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
15+
| [`README.md`](./README.md) | Overview, the verbatim problem, before/after, and the shipped solution at a glance |
16+
| [`requirements.md`](./requirements.md) | The exhaustive, numbered list of every requirement extracted from the issue, each mapped to where it is satisfied |
17+
| [`analysis.md`](./analysis.md) | Timeline, root-cause framing for each problem, design decisions, and trade-offs |
18+
| [`existing-components.md`](./existing-components.md) | Survey of existing in-repo components reused, plus external prior art / libraries evaluated (with online research) |
19+
| [`issue.json`](./issue.json) | Raw issue payload downloaded via `gh issue view --json` (timeline source data) |
20+
| [`assets/queue-screenshot.png`](./assets/queue-screenshot.png) | The screenshot attached to the issue showing the duplicated old display |
21+
22+
---
23+
24+
## The problem (verbatim from the issue)
25+
26+
> We have too much duplication in `/queue` display.
27+
>
28+
> For example we can just have links to issues like we have
29+
>
30+
> ```
31+
> ▶️ uselessgoddess/ryzr#3 (processing, 2h 14m 16s)
32+
> ▶️ link-foundation/box#99 (processing, 53m 52s)
33+
> ▶️ link-assistant/hive-mind#1886 (processing, 51m 50s)
34+
> ▶️ link-assistant/hive-mind#1885 (processing, 50m 30s)
35+
> ```
36+
>
37+
> But if we split processing and pending, that will help us make it more compact (with links as now):
38+
>
39+
> ```
40+
> • uselessgoddess/ryzr#3 (▶️ 2h 14m 16s)
41+
> • link-foundation/box#99 (▶️ 53m 52s)
42+
> • link-assistant/hive-mind#1886 (▶️ 51m 50s)
43+
> • link-assistant/hive-mind#1885 (▶️ 50m 30s)
44+
> ```
45+
>
46+
> Waiting/pending does not need to show waiting reason, it usually all the same for all of them.
47+
> [...] We should also try to fit more data (it will be fine if it does not fit in one message, we can
48+
> split it by our universal message sending method, split should be done by lines, without breaking the
49+
> markdown (especially code blocks) [...] add lots of tests for that). And we don't need to show empty queues.
50+
51+
## The problem in one sentence
52+
53+
The `/queue` (alias `/solve_queue`) Telegram command rendered **the same words over
54+
and over** — every executing line said `(processing, …)`, every waiting line said
55+
`(waiting, …)`, and the (almost always identical) waiting reason was repeated once
56+
per item — which wasted vertical space, pushed real data off the screen, and risked
57+
hitting Telegram's 4096-character limit without a markdown-safe way to split.
58+
59+
## Before → After
60+
61+
**Before** (duplicated, verbose; reason repeated per item):
62+
63+
```
64+
*claude* (pending: 2, processing: 4)
65+
▶️ uselessgoddess/ryzr#3 (processing, 2h 14m 16s)
66+
▶️ link-foundation/box#99 (processing, 53m 52s)
67+
⏳ link-assistant/hive-mind#1900 (waiting, 5m 2s) — Claude 5 hour session limit is 95% (threshold: 90%)
68+
⏳ link-assistant/hive-mind#1901 (waiting, 4m 1s) — Claude 5 hour session limit is 95% (threshold: 90%)
69+
*agent* (pending: 0, processing: 0)
70+
*codex* (pending: 0, processing: 0)
71+
*gemini* (pending: 0, processing: 0)
72+
```
73+
74+
**After** (compact; split lists; shared reason once; empty queues hidden):
75+
76+
```
77+
*claude*
78+
*Processing* (2):
79+
• uselessgoddess/ryzr#3 (▶️ 2h 14m 16s)
80+
• link-foundation/box#99 (▶️ 53m 52s)
81+
*Pending* (2):
82+
• link-assistant/hive-mind#1900 (⏳ 5m 2s)
83+
• link-assistant/hive-mind#1901 (⏳ 4m 1s)
84+
⏳ Claude 5 hour session limit is 95% (threshold: 90%)
85+
```
86+
87+
## The shipped solution at a glance
88+
89+
Two independent changes, both required by the issue:
90+
91+
1. **Compact, de-duplicated `/queue` display** (`formatDetailedStatus`):
92+
- executing rows render as `• owner/repo#number (▶️ <dur>)`,
93+
- pending rows render as `• owner/repo#number (⏳ <dur>)`,
94+
- each tool renders as separate labeled lists (`*Processing* (n):`,
95+
`*Pending* (n):`, `*Completed* (n):`, `*Failed* (n):`) with no duplicated
96+
`(pending: n, processing: n)` tool-header summary,
97+
- the shared waiting reason is printed **once per tool** (only when all pending
98+
items agree on it) instead of once per item,
99+
- **empty queues are skipped entirely**,
100+
- all queued items are listed (no per-queue truncation) — the universal sender
101+
splits the message if it grows past Telegram's limit.
102+
103+
2. **A markdown-safe, line-based universal message splitter**
104+
(`splitTelegramMessageText` in `src/telegram-safe-reply.lib.mjs`):
105+
- splits only on line boundaries so inline entities (bold/italic/links — none of
106+
which may span a newline in Telegram Markdown) are never cut in half,
107+
- keeps **fenced code blocks balanced per chunk**: a split inside a code block
108+
closes the fence (` ``` `) at the end of one chunk and **reopens it,
109+
repeating the language**, at the start of the next,
110+
- preserves the original fence marker (` ``` ` vs `~~~`) and indentation,
111+
- hard-splits pathologically long single lines as a fallback.
112+
113+
Both behaviours are covered by extensive new tests
114+
(`tests/test-telegram-message-split-1891.mjs`,
115+
`tests/test-queue-compact-display-1891.mjs`) and by updates to the existing queue
116+
display tests.

‎docs/case-studies/issue-1891/analysis.md‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# Analysis — Issue #1891
2+
3+
## Timeline / sequence of events
4+
5+
1. **2026-06-10 23:17 UTC** — `konard` opens issue #1891 ("We have too much
6+
duplication in `/queue` display"), labelled `bug`, with a screenshot
7+
(`assets/queue-screenshot.png`) of the live `/queue` output and concrete
8+
before/after format examples. No comments follow; the issue body is the entire
9+
specification.
10+
2. The issue traces back to the display introduced/extended for the `/queue`
11+
(`/solve_queue`) command:
12+
- #1232 added `/solve_queue`,
13+
- #1267 added per-tool grouping, human-readable durations, and the
14+
`MAX_DISPLAY_ITEMS_PER_QUEUE` cap with "… and N more",
15+
- #1837 added the short `/queue` alias, clickable `owner/repo#number` links, and
16+
the executing-list merge with detached sessions.
17+
Each step added information; none removed the per-line status word or the
18+
repeated waiting reason, so the duplication accumulated.
19+
3. **PR #1892** (this work) addresses every requirement (see `requirements.md`).
20+
21+
## Problem 1 — Duplication in the display
22+
23+
### Root cause
24+
25+
The old `formatDetailedStatus` rendered each item through a single helper that
26+
embedded the _status word_ and the _waiting reason_ on every line:
27+
28+
```
29+
▶️ owner/repo#n (processing, 2h 14m 16s)
30+
⏳ owner/repo#n (waiting, 5m 2s) — <reason>
31+
```
32+
33+
Three sources of duplication:
34+
35+
1. **The status word is constant within its list.** Everything in the executing
36+
list is "processing"; everything in the pending list is "waiting". Printing the
37+
word on every line conveys zero marginal information.
38+
2. **The waiting reason is (almost) constant within a tool.** The queue blocks a
39+
whole tool on the same limit (e.g. "Claude 5 hour session limit is 95%"), so the
40+
reason is identical for every pending item — yet it was printed once per item.
41+
3. **Empty queues were still printed** (`*agent* (pending: 0, processing: 0)` …),
42+
adding four dead lines to almost every message.
43+
44+
### Solution
45+
46+
- Move the status signal **into the duration parenthesis as an emoji**:
47+
`(▶️ <dur>)` for executing, `(⏳ <dur>)` for pending. The emoji is the marker, so
48+
the word "processing"/"waiting" disappears.
49+
- Split the queue into explicit labeled lists (Processing, Pending, Completed,
50+
Failed) so the reader groups items visually without a per-line status word.
51+
- Render the tool name as a plain header; counts live only on the individual list
52+
labels, e.g. `*Pending* (2):`, so `(pending: N, processing: N)` is not repeated
53+
above the same lists.
54+
- Print the **shared waiting reason once per tool**, and only when all pending items
55+
agree on it (`distinctReasons.length === 1`). Divergent reasons suppress the
56+
shared line rather than print a misleading one.
57+
- **Skip empty queues** with an early `continue`.
58+
59+
## Problem 2 — Splitting long messages without breaking markdown
60+
61+
### Root cause
62+
63+
Removing the per-queue truncation (R6: "try to fit more data") means a `/queue`
64+
message can now exceed Telegram's **4096-character** limit. The previous
65+
`splitTelegramMessageText` split on arbitrary separators (including mid-line), which
66+
can:
67+
68+
- cut an inline entity in half (e.g. `[label](ur` … `l)`), producing
69+
`can't parse entities` errors, and
70+
- split **inside a fenced code block**, leaving one chunk with an unclosed ` ``` `
71+
and the next chunk starting in a broken state — exactly the failure the issue
72+
calls out ("without breaking the markdown (especially code blocks)").
73+
74+
### Solution
75+
76+
A line-based, fence-aware splitter:
77+
78+
- **Split only on `\n`.** Telegram's legacy-Markdown inline entities (bold, italic,
79+
code span, link) cannot span a newline, so a line boundary is always a safe place
80+
to cut an inline entity — there are none crossing it.
81+
- **Track fenced code blocks.** A regex (`/^(\s*)(```+|~~~+)(.*)$/`) recognises a
82+
fence line and captures its indentation, marker, and language/info string. A
83+
running `openFence` toggles as fences are seen.
84+
- **Close + reopen across a split.** When a chunk is flushed mid-code-block, the
85+
splitter appends a closing fence (same indent + marker) to the current chunk and
86+
seeds the next chunk with a reopening fence that **repeats the language** — so each
87+
chunk is independently valid Markdown and the code block renders continuously.
88+
- **Reserve headroom** (`FENCE_HEADROOM`) so adding a close/reopen pair never pushes
89+
a chunk past the limit, and **hard-split** any single physical line that alone
90+
exceeds the budget (pathological input) using the existing
91+
`findTelegramSplitIndex`.
92+
- **Preserve the marker kind and indentation** — a `~~~python` block reopens as
93+
`~~~python`, an indented block keeps its indent — so we never silently rewrite the
94+
author's fence style.
95+
96+
This lives in the one universal splitter (`src/telegram-safe-reply.lib.mjs`) that
97+
**every** Telegram send path already funnels through, satisfying R11 ("perfect
98+
across the codebase") without touching each call site.
99+
100+
## Codebase-wide audit (R18)
101+
102+
- **Send paths** — `safeReply`, the wrapped `telegram.sendMessage`, and the wrapped
103+
`telegram.editMessageText` all call `splitTelegramMessageText`. Fixing the splitter
104+
fixes every outbound message, including edits (which also forward overflow chunks
105+
via `sendFollowUpChunk`).
106+
- **Queue formatters** — `formatDetailedStatus` (the `/queue` detail) is the only
107+
formatter that listed items with a status word + reason; it is fixed.
108+
`formatStatus` (one line per tool, used by `/limits`) never listed items, so it has
109+
no duplication and is intentionally untouched.
110+
- **History sections** — `formatQueueHistorySection` (Completed/Failed) keeps its cap
111+
by design (see `requirements.md`, "out of scope").
112+
113+
## Scope: other repositories (R17)
114+
115+
The bug is entirely internal: it is in this repo's Telegram rendering
116+
(`telegram-solve-queue*.mjs`) and message-splitting (`telegram-safe-reply.lib.mjs`)
117+
code. The `owner/repo#number` strings in the screenshot (e.g.
118+
`uselessgoddess/ryzr`, `link-foundation/box`) are just _queued work items_, not the
119+
source of the bug. There is therefore **no upstream/third-party repository to file a
120+
report against**. Prior art in external projects is catalogued in
121+
`existing-components.md` for design validation, not as a defect to report.
122+
123+
## Risk / trade-off notes
124+
125+
- **Listing all items** can make a message long; mitigated by the markdown-safe
126+
splitter. Worst case is several messages instead of a truncated one — the issue
127+
explicitly accepts this ("it will be fine if it does not fit in one message").
128+
- **Shared-reason heuristic**: if two pending items genuinely have different reasons
129+
we show none (rather than a wrong single one). The per-item status is still implied
130+
by the ⏳ marker; the detailed reason remains available in each item's own status
131+
message. This matches the issue's "it usually [is] all the same".
132+
- **Backward compatibility**: `formatQueueProcessingItems` is kept as a deprecated
133+
alias of `formatQueueExecutingItems` so any external caller keeps working.

‎docs/case-studies/issue-1891/assets/queue-screenshot.png‎

578 KB
Loading

0 commit comments

Comments
 (0)
⚡