Skip to content

Commit 69e91db

Browse files
committed
Tooling: add docs-reachable check + --docs-graph view to keep the doc tree connected from AGENTS.md
Every CLAUDE.md, DETAILS.md, and `docs/` file should be discoverable by an agent (or human) entering at `AGENTS.md` and walking references. Nothing enforced that before, so docs silently went orphan (14 unlinked guides, 4 subsystem CLAUDE.md). Two tools on one shared graph core (`docs_graph.go`): - **`docs-reachable`** (`IsFast`, error not warn): fails when an enforced doc can't be reached from `AGENTS.md`. BFS over references, where a reference is any mention (Markdown link, `@import`, backtick path, bare path token) treated equally. A `CLAUDE.md` also counts as reached when a reachable doc names its *directory* (architecture.md lists subsystems as `` `dir/` ``, and Claude Code auto-injects from the dir anyway); every other doc must be named. Matching is generous (relative, root-relative, ≥2-segment suffix) so over-connecting hides a would-be orphan rather than inventing a false CI failure. `docs/specs` + `docs/notes` (self-declared ephemeral scratch) and the repo-root loader `CLAUDE.md` are excluded. Wired into the CI `hygiene` job. Shrink-wrapping allowlist (`docs-reachable-allowlist.json`) for intentional exemptions; goal is an empty list. - **`pnpm check --docs-graph`**: renders the discoverability tree from `AGENTS.md` in the same box-drawing style as `--graph`, closest-to-root placement, `(dir reference)` tags, and a red orphan list at the bottom. A "show me" tool for spotting deeply-nested or orphaned docs. Surfaces 18 real orphans today (left untouched here): they're for connecting, not exempting. Docs: new § "Docs reachable" in `checks/DETAILS.md`, module-map + check-count rows, `--docs-graph` in the runner CLAUDE.md, and the allowlist consent contract extended in the file-length-allowlist rule.
1 parent a011de5 commit 69e91db

12 files changed

Lines changed: 722 additions & 12 deletions

File tree

.claude/rules/file-length-allowlist.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,12 @@ reason and the same consent rule applies to adding one.
1515

1616
The file-length check is warn-only — it doesn't fail the suite — so leaving the warning is always safe.
1717

18+
The same contract also applies to `scripts/check/checks/docs-reachable-allowlist.json` (the `docs-reachable` check): the
19+
check shrink-wraps entries that are gone or now reachable, and adding/keeping an entry needs explicit consent. Prefer
20+
connecting the doc from a reachable one (a `CLAUDE.md` counts as reached when a reachable doc names its directory) over
21+
exempting it. Unlike the length checks, `docs-reachable` is an **error**, so a real orphan fails the suite until it's
22+
linked or (with consent) allowlisted.
23+
1824
The same contract applies to `scripts/check/checks/claude-md-length-allowlist.json` (the `claude-md-length` check, which
1925
caps push-tier CLAUDE.md word counts): the check shrink-wraps stale `files` entries on local runs (remove gone /
2026
under-threshold, ratchet >10% slack down), and adding or raising an entry needs explicit user consent. If a CLAUDE.md

.github/workflows/ci.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -577,6 +577,9 @@ jobs:
577577
- name: Check CHANGELOG commit links
578578
run: ./scripts/check/check --check changelog-commit-links --ci
579579

580+
- name: Check docs reachable from AGENTS.md
581+
run: ./scripts/check/check --check docs-reachable --ci
582+
580583
- name: Check workflow hardening
581584
run: ./scripts/check/check --check workflows-hardening --ci
582585

scripts/check/CLAUDE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ see [DETAILS.md](DETAILS.md).
1515
and misses before pnpm/SMB; record passes after the run).
1616
- `checks/inputs.go`: shared `Inputs` building blocks (mined from ci.yml filters).
1717
- `smb_orchestrator.go` + `smblease/` + `smb-lease/`: runner-level SMB Docker lifecycle behind a machine-wide lease.
18-
- `freestyle.go`: all freestyle.sh remote-VM execution. `graph.go`: `--graph` renderer. `stats.go`: CSV stats logging.
18+
- `freestyle.go`: all freestyle.sh remote-VM execution. `graph.go`: `--graph` (check dependency tree) renderer.
19+
`docs_graph_render.go`: `--docs-graph` (doc-discoverability tree) renderer. `stats.go`: CSV stats logging.
1920

2021
## Must-knows
2122

scripts/check/checks/CLAUDE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,8 @@ see [`../CLAUDE.md`](../CLAUDE.md).
1717
- `inputs.go`: shared `Inputs` building blocks. `allowlist.go` / `directives.go`: allowlist shrink-wrap + opt-out
1818
tracking plumbing.
1919
- Warn-only scanners with their JSON allowlists: `file-length.go`, `claude-md-length.go`, `e2e-durations.go`,
20-
`website-bundle-size.go`.
20+
`website-bundle-size.go`. Error-level allowlist scanner: `docs-reachable.go` (+ shared `docs_graph.go`), which fails
21+
when a `CLAUDE.md` / `DETAILS.md` / `docs` file isn't reachable from `AGENTS.md`.
2122

2223
## Must-knows
2324

scripts/check/checks/DETAILS.md

Lines changed: 40 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@ freestyle.sh remote execution), see [`../CLAUDE.md`](../CLAUDE.md).
1818
| `file-length-allowlist.json` | Allowlist for file-length check: `{ "exempt": { "path": reason }, "files": { "path": lineCount } }`. See § File-length allowlist. |
1919
| `claude-md-length.go` | Warn-only push-tier scanner: warns when a `CLAUDE.md` exceeds 600 words (`DETAILS.md` is the unlimited pull tier, not scanned). Allowlist with the same shrink-wrap semantics as file-length. See § CLAUDE.md length. |
2020
| `claude-md-length-allowlist.json` | Allowlist for claude-md-length: `{ "files": { "path": wordCount } }`. Same ratchet/consent rules as file-length. |
21+
| `docs_graph.go` | Shared doc-discoverability graph: reachability from `AGENTS.md` over references between docs. Powers both the `docs-reachable` check and the `--docs-graph` renderer. See § Docs reachable. |
22+
| `docs-reachable.go` | Errors (not warn-only) when any `CLAUDE.md` / `DETAILS.md` / `docs/` file can't be reached from `AGENTS.md`. Allowlist with the same shrink-wrap/consent semantics as file-length. See § Docs reachable. |
23+
| `docs-reachable-allowlist.json` | Allowlist for docs-reachable: `{ "files": { "path": reason } }` of docs intentionally unreachable. Goal is empty. Shrink-wraps gone/now-reachable entries; adding one needs David's OK. |
2124
| `e2e-durations.go` | E2E test duration flagger (warn-only): parses the Playwright JSON reports after each E2E run and flags tests over the 2 s budget. Embedded in both E2E checks, not a registry check. See § E2E test duration flagger. |
2225
| `e2e-duration-allowlist.json` | Per-platform (`macos` / `linux`) allowlist for the duration flagger: `{ "<spec>::<describe chain>::<title>": reason }`. Entries need a reason; new entries need David's OK. |
2326
| `website-bundle-size.go` | Warn-only website `dist/` size budget: warns when the total grows >10% over the committed baseline. Self-skips without `dist/`. See § Website bundle-size baseline. |
@@ -185,6 +188,33 @@ local runs drop dead/under-threshold entries and ratchet >10%-slack entries down
185188
change. Adding or raising an entry needs David's OK (`.claude/rules/file-length-allowlist.md`); the fix for an oversized
186189
`CLAUDE.md` is to move depth into `DETAILS.md`, not bump the number.
187190

191+
## Docs reachable
192+
193+
`docs-reachable` (`IsFast`, an **error** not a warn: the doc tree must stay connected) enforces that every `CLAUDE.md`,
194+
`DETAILS.md`, and `docs/` file is discoverable from the single root `AGENTS.md` by link-walking, so a reader entering
195+
there can find every doc. `docs_graph.go` builds the graph (shared with the `--docs-graph` renderer in
196+
`../docs_graph_render.go`); `docs-reachable.go` is the check shell + allowlist.
197+
198+
How reachability is decided (`BuildDocGraph`):
199+
200+
- **One root, `AGENTS.md`.** A doc is reached when a doc already reached from the root references it. BFS, so each doc
201+
is placed under its closest-to-root reference (a cycle just hits an already-reached node and stops).
202+
- **A reference is any mention, syntax-agnostic:** Markdown link, `@import`, backtick path, or bare path token are all
203+
equal. We watch intent, not form. Matching is generous (relative-to-source, repo-root-relative, and ≥2-segment path
204+
suffix), because over-connecting only hides a would-be orphan, while a false orphan would be a noisy CI failure.
205+
- **The CLAUDE.md asymmetry:** a `DETAILS.md` or `docs/` file must be named, but a `CLAUDE.md` also counts as reached
206+
when a reachable doc mentions its _directory_ (`architecture.md` lists most subsystems as `` `some/dir/` ``, and
207+
Claude Code auto-injects a `CLAUDE.md` from its directory regardless). Such edges are tagged `ViaDir`; the renderer
208+
shows "(dir reference)".
209+
- **Ephemeral dirs are excluded** from the enforced candidate set: `docs/specs` and `docs/notes` self-declare in their
210+
READMEs as temporary scratch "wiped periodically", so requiring each to be linked fights their purpose
211+
(`ephemeralDocDirs`). The repo-root `CLAUDE.md` (the loader shim that only `@import`s the entry docs) is excluded too.
212+
213+
`docs-reachable-allowlist.json` maps a doc path → the reason it's intentionally unreachable. The goal is an empty list:
214+
connect docs rather than exempt them. Shrink-wrap drops entries whose file is gone or which became reachable; adding or
215+
keeping one needs David's OK (`.claude/rules/file-length-allowlist.md`). To inspect the whole tree and spot
216+
deeply-nested or orphaned docs visually, run `pnpm check --docs-graph`.
217+
188218
## E2E test duration flagger
189219

190220
The E2E suites were hard-won down to under 2 s per test; `e2e-durations.go` defends that. After a successful E2E run,
@@ -253,16 +283,16 @@ RUSTSEC ignores — that's a quarterly task in `docs/maintenance.md`.
253283

254284
## Apps and check counts
255285

256-
| App | Tech | Checks |
257-
| ---------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
258-
| Desktop | Rust | rustfmt, clippy, cargo-audit, cargo-deny, cargo-machete, cargo-udeps (CI-only), jscpd, log-error-macro, error-string-match, lock-poison, bindings-fresh, ipc-enum-camelcase, tests, integration-tests (Docker SMB), tests-linux (slow) |
259-
| Desktop | Svelte | prettier, eslint, svelte-kit-sync, eslint-typecheck-svelte, eslint-typecheck-typescript, stylelint, css-unused, a11y-contrast, btn-restyle, bare-poll, svelte-check, import-cycles, knip, type-drift, tests, e2e-linux-typecheck, e2e-linux (slow), e2e-playwright (slow) |
260-
| Website | Astro | prettier, eslint, typecheck, build, html-validate, bundle-size (warn-only), e2e |
261-
| Website | Docker | docker-build |
262-
| API server | TS | oxfmt, eslint, typecheck, tests |
263-
| Scripts | Go | gofmt, go-vet, staticcheck, ineffassign, misspell, gocyclo, nilaway, deadcode, go-tests, govulncheck |
264-
| Other | Metrics | file-length (warn-only), CLAUDE.md-reminder (warn-only), claude-md-length (warn-only), changelog-commit-links, workflows-rustup (forbids `rustup target/component add` in workflows), ci-coverage (registry-to-workflows contract) |
265-
| Other | Security | workflows-hardening (SHA-pinning, no `pull_request_target`, job-scoped `id-token: write`) |
286+
| App | Tech | Checks |
287+
| ---------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
288+
| Desktop | Rust | rustfmt, clippy, cargo-audit, cargo-deny, cargo-machete, cargo-udeps (CI-only), jscpd, log-error-macro, error-string-match, lock-poison, bindings-fresh, ipc-enum-camelcase, tests, integration-tests (Docker SMB), tests-linux (slow) |
289+
| Desktop | Svelte | prettier, eslint, svelte-kit-sync, eslint-typecheck-svelte, eslint-typecheck-typescript, stylelint, css-unused, a11y-contrast, btn-restyle, bare-poll, svelte-check, import-cycles, knip, type-drift, tests, e2e-linux-typecheck, e2e-linux (slow), e2e-playwright (slow) |
290+
| Website | Astro | prettier, eslint, typecheck, build, html-validate, bundle-size (warn-only), e2e |
291+
| Website | Docker | docker-build |
292+
| API server | TS | oxfmt, eslint, typecheck, tests |
293+
| Scripts | Go | gofmt, go-vet, staticcheck, ineffassign, misspell, gocyclo, nilaway, deadcode, go-tests, govulncheck |
294+
| Other | Metrics | file-length (warn-only), CLAUDE.md-reminder (warn-only), claude-md-length (warn-only), docs-reachable (errors when a CLAUDE.md/DETAILS.md/docs file isn't reachable from AGENTS.md), changelog-commit-links, workflows-rustup (forbids `rustup target/component add` in workflows), ci-coverage (registry-to-workflows contract) |
295+
| Other | Security | workflows-hardening (SHA-pinning, no `pull_request_target`, job-scoped `id-token: write`) |
266296

267297
## Key decisions
268298

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"$comment": "Docs intentionally NOT reachable from AGENTS.md, each mapped to its reason. The goal is an empty list: connect docs instead of exempting them. The docs-reachable check shrink-wraps stale entries (file gone or now reachable) on local runs; adding or keeping an entry needs David's explicit consent (see .claude/rules/file-length-allowlist.md).",
3+
"files": {}
4+
}
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
package checks
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
"os"
7+
"path/filepath"
8+
"strings"
9+
)
10+
11+
// docsReachableAllowlist is the on-disk shape of docs-reachable-allowlist.json.
12+
// `Files` maps a repo-relative doc path to the reason it's intentionally NOT
13+
// reachable from AGENTS.md. The goal is an empty allowlist: every doc connected.
14+
type docsReachableAllowlist struct {
15+
Comment string `json:"$comment,omitempty"`
16+
Files map[string]string `json:"files"`
17+
}
18+
19+
func docsReachableAllowlistPath(rootDir string) string {
20+
return filepath.Join(rootDir, "scripts", "check", "checks", "docs-reachable-allowlist.json")
21+
}
22+
23+
// loadDocsReachableAllowlist reads the allowlist JSON. A missing or unparsable
24+
// file yields an empty allowlist (every orphan gets reported).
25+
func loadDocsReachableAllowlist(rootDir string) docsReachableAllowlist {
26+
list := docsReachableAllowlist{Files: map[string]string{}}
27+
data, err := os.ReadFile(docsReachableAllowlistPath(rootDir))
28+
if err != nil {
29+
return list
30+
}
31+
if err := json.Unmarshal(data, &list); err != nil {
32+
return docsReachableAllowlist{Files: map[string]string{}}
33+
}
34+
if list.Files == nil {
35+
list.Files = map[string]string{}
36+
}
37+
return list
38+
}
39+
40+
// shrinkwrapDocsReachableAllowlist drops allowlist entries that no longer earn
41+
// their place: the doc is gone, or it's now reachable (so it needs no exemption).
42+
// Mutates list in place and returns one human-readable line per change.
43+
func shrinkwrapDocsReachableAllowlist(rootDir string, list *docsReachableAllowlist, orphanSet map[string]bool) []string {
44+
var changes []string
45+
for _, docPath := range sortedKeys(list.Files) {
46+
switch {
47+
case !fileExists(filepath.Join(rootDir, filepath.FromSlash(docPath))):
48+
delete(list.Files, docPath)
49+
changes = append(changes, fmt.Sprintf("removed %s (file no longer exists)", docPath))
50+
case !orphanSet[docPath]:
51+
delete(list.Files, docPath)
52+
changes = append(changes, fmt.Sprintf("removed %s (now reachable from AGENTS.md)", docPath))
53+
}
54+
}
55+
return changes
56+
}
57+
58+
// RunDocsReachable fails when any enforced doc (CLAUDE.md, DETAILS.md, or a
59+
// docs/ file outside the ephemeral scratch dirs) can't be reached from AGENTS.md
60+
// by walking references between docs. A CLAUDE.md counts as reached when a
61+
// reachable doc mentions its directory; everything else must be named. Allowlist
62+
// entries (intentionally-unreachable docs) are suppressed; stale ones shrink-wrap
63+
// away outside CI. Unlike the length scanners this is an error, not a warning:
64+
// the doc tree must stay connected.
65+
func RunDocsReachable(ctx *CheckContext) (CheckResult, error) {
66+
g, err := BuildDocGraph(ctx.RootDir)
67+
if err != nil {
68+
return CheckResult{}, fmt.Errorf("failed to build doc graph: %w", err)
69+
}
70+
71+
orphanSet := make(map[string]bool, len(g.Orphans))
72+
for _, o := range g.Orphans {
73+
orphanSet[o] = true
74+
}
75+
76+
allowlist := loadDocsReachableAllowlist(ctx.RootDir)
77+
staleChanges := shrinkwrapDocsReachableAllowlist(ctx.RootDir, &allowlist, orphanSet)
78+
if len(staleChanges) > 0 && !ctx.CI {
79+
if err := writeJSONAllowlist(docsReachableAllowlistPath(ctx.RootDir), allowlist); err != nil {
80+
return CheckResult{}, err
81+
}
82+
reformatWithOxfmt(ctx.RootDir, "scripts/check/checks/docs-reachable-allowlist.json")
83+
}
84+
85+
var reported []string
86+
for _, o := range g.Orphans {
87+
if _, ok := allowlist.Files[o]; !ok {
88+
reported = append(reported, o)
89+
}
90+
}
91+
92+
staleMsg := formatDocsStaleMsg(ctx.CI, staleChanges)
93+
if len(reported) == 0 {
94+
okMsg := fmt.Sprintf("All docs reachable from AGENTS.md (%d in graph)", len(g.Reached))
95+
if len(allowlist.Files) > 0 {
96+
okMsg = fmt.Sprintf("%s (%d allowlisted)", okMsg, len(allowlist.Files))
97+
}
98+
if staleMsg != "" {
99+
if ctx.CI {
100+
return CheckResult{Code: ResultWarning, Message: okMsg + "; " + staleMsg, Total: -1, Issues: -1, Changes: -1}, nil
101+
}
102+
return SuccessWithChanges(okMsg + "; " + staleMsg), nil
103+
}
104+
return Success(okMsg), nil
105+
}
106+
107+
body := formatOrphans(reported, len(allowlist.Files))
108+
if staleMsg != "" {
109+
body += "\n" + staleMsg
110+
}
111+
return CheckResult{}, fmt.Errorf("%s", body)
112+
}
113+
114+
// formatDocsStaleMsg renders the shrink-wrap note (or the CI-mode equivalent).
115+
func formatDocsStaleMsg(ci bool, staleChanges []string) string {
116+
if len(staleChanges) == 0 {
117+
return ""
118+
}
119+
verb := "Shrink-wrapped allowlist"
120+
if ci {
121+
verb = "Stale allowlist entries (a local run shrink-wraps them)"
122+
}
123+
return fmt.Sprintf("%s:\n - %s", verb, strings.Join(staleChanges, "\n - "))
124+
}
125+
126+
// formatOrphans builds the failure body listing the unreachable docs.
127+
func formatOrphans(orphans []string, allowlisted int) string {
128+
suffix := ""
129+
if allowlisted > 0 {
130+
suffix = fmt.Sprintf(" (%d allowlisted)", allowlisted)
131+
}
132+
var sb strings.Builder
133+
for _, o := range orphans {
134+
sb.WriteString(" - ")
135+
sb.WriteString(o)
136+
sb.WriteString("\n")
137+
}
138+
return fmt.Sprintf(
139+
"%d %s unreachable from AGENTS.md%s. Link each from a doc that's already reachable (a CLAUDE.md also counts as reached when a reachable doc mentions its directory):\n%s",
140+
len(orphans), Pluralize(len(orphans), "doc", "docs"), suffix,
141+
strings.TrimRight(sb.String(), "\n"))
142+
}

0 commit comments

Comments
 (0)