Update add-model skill: lagging-provider checks and push-gate rules

[tawnymanticore]

What does this PR do?

The stop hook in Claude Code web is super annoying. "hopeful" fix.

Checklists

• Tests have been run locally and passed
• New tests have been added to any work in /lib

Summary by CodeRabbit

• Documentation
  • Added operational guidelines: a Phase 1B backfill cross-check on every skill run that validates recent models and flags proposed additions instead of silently changing the registry.
  • Expanded Phase 5 with explicit gating of execution when tests or phases fail, plus an “abandon” recovery path to clean up local changes and branches.
  • Clarified guidance for using provider endpoints and requesting API keys.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds Phase 1B guidance to run a provider-specific backfill cross-check on every skill invocation (pulling the 10 newest built_in_models entries and validating against Fireworks, Together, and SiliconFlow), extends Phase 5 with gating/abandon rules for Claude Code Web’s stop hook, and updates Lagging Providers usage guidance.

Changes

Cohort / File(s)

Summary

Skill doc .agents/skills/claude-maintain-models/SKILL.md

Added Phase 1B: require provider-specific backfill cross-check on every invocation (fetch 10 newest built_in_models/git history, validate model availability via Fireworks, Together, SiliconFlow; flag/propose PRs instead of silently adding). Extended Phase 5 with subsections 5.0/5.1 describing gating rules (prevent commit/push/branch creation when tests/smokes fail or phases are incomplete), an explicit abandon path (revert local changes, delete add-model/* branches), and updated Lagging Providers guidance including WebFetch/Together API-key handling and prompting the user for keys.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• Move Cursor add model skill to .agents and rename #1260 — Related edits to the same claude-maintain-models Phase 5 PR/commit workflow; overlaps gating/PR creation behavior.
• Qwen 3.5, Gemini 3.1 Flash Lite, GPT 5.3 Chat/Instant, GPT 5.4 #1103 — Changes touching built_in_models and ModelName enum; relevant to the backfill/cross-check guidance.

Suggested reviewers

• scosman
• leonardmq
• chiang-daniel

Poem

🐰 I hopped through docs to check the list,
I sniffed each model in the mist,
If a provider lags behind,
I flag and bundle, thoughtful-minded,
Branches pruned, the garden's kissed 🌿

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is largely incomplete. It lacks detail about what the PR actually does, omits the Related Issues section, doesn't adequately explain the stop hook fix, and provides minimal context despite significant documentation changes.	Expand the description to explain the Phase 1B backfill cross-check addition, Phase 5 operational behavior changes, and lagging provider guidance; include Related Issues link if applicable.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title directly and clearly summarizes the main changes: updates to the add-model skill focusing on lagging-provider checks and push-gate rules, which aligns with the documentation changes adding cross-check validation and stop hook behavior gates.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch mike/update-claude-maintain

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request updates the SKILL.md documentation to provide guidance on handling the Claude Code Web stop hook and defines a gating process before pushing changes. The feedback suggests enhancing the abandonment procedure by explicitly deleting local branches to ensure a clean state and including authentication errors as critical failures in the push gate criteria.

[github-actions]

📊 Coverage Report

Overall Coverage: 91%

Diff: origin/main...HEAD

No lines with coverage information in this diff.

---

• 📊 HTML Coverage Report - Interactive coverage report
• 📈 Diff Coverage Report - Detailed diff analysis
• Github Actions Run - View the full coverage report

[coderabbitai]

🧹 Nitpick comments (3)

.agents/skills/claude-maintain-models/SKILL.md (3)

537-537: Minor wording: simplify the Together API key instruction.

Line 537 says "ask the user before prompting them to export it" — the "before prompting them" is redundant. Consider:

-If the key isn't set, ask the user before prompting them to export it — don't fail silently onto models.dev.
+If the key isn't set, ask the user to export it — don't fail silently onto models.dev.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In @.agents/skills/claude-maintain-models/SKILL.md at line 537, Edit the
SKILL.md text that currently reads "If the key isn't set, ask the user before
prompting them to export it — don't fail silently onto models.dev." and simplify
the phrasing by removing the redundant "before prompting them" (e.g., "If the
key isn't set, ask the user to export it — don't fail silently onto
models.dev."). Update the single line at or around the sentence in the Together
API key instructions to the simplified wording.

---

60-75: Phase 1B design looks good; consider frequency optimization.

The backfill check design is sound and addresses the "lagging providers" gap mentioned in the PR objectives. Running it on every invocation ensures recent models don't miss provider additions, which aligns with keeping the model list current.

One observation: checking 10 models across 3 providers means ~30 API calls every time the skill runs. If this becomes a bottleneck, consider throttling (e.g., only run Phase 1B if the user explicitly invokes discovery mode, or cache results for 24 hours). For now, the thoroughness seems justified.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In @.agents/skills/claude-maintain-models/SKILL.md around lines 60 - 75, Phase
1B currently runs the lagging-provider backfill check on every invocation by
pulling the top 10 entries from built_in_models in ml_model_list.py and
cross-checking Fireworks/Together/SiliconFlow against KilnModel entries, which
can cost ~30 API calls per run; modify the skill to reduce frequency by either
gating Phase 1B behind an explicit "discovery mode" flag or adding a 24-hour
cache for the backfill results keyed by provider+model (invalidate on model-add
operations), and ensure the logic that reads built_in_models and compares
against KilnModel honors this gate/cache so behavior remains identical when the
check runs.

---

522-524: Consider adding language specifiers to code blocks.

Static analysis flags these WebFetch command blocks as missing language specifiers (MD040). While WebFetch isn't bash, you could either:

• Add ```bash if these are meant to be run in a shell context
• Add a generic ```text or ```shell for consistency
• Leave unmarked if these are intentionally tool-specific commands

Other WebFetch examples in the document (lines 522-524, 540-542) are inconsistent, so standardizing would improve clarity.

Also applies to: 540-542

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In @.agents/skills/claude-maintain-models/SKILL.md around lines 522 - 524,
Update the fenced code blocks that contain the WebFetch command (e.g., the block
with "WebFetch https://fireworks.ai/models/fireworks/{model-slug}") to include a
language specifier for consistency (choose one such as ```text, ```shell, or
```bash) and apply the same specifier to the other WebFetch examples in this
document so all WebFetch blocks use the same tag; ensure you only modify the
triple-backtick opening fence to add the chosen specifier for each WebFetch
block (search for the literal "WebFetch" blocks to find all occurrences).

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.agents/skills/claude-maintain-models/SKILL.md:
- Line 537: Edit the SKILL.md text that currently reads "If the key isn't set,
ask the user before prompting them to export it — don't fail silently onto
models.dev." and simplify the phrasing by removing the redundant "before
prompting them" (e.g., "If the key isn't set, ask the user to export it — don't
fail silently onto models.dev."). Update the single line at or around the
sentence in the Together API key instructions to the simplified wording.
- Around line 60-75: Phase 1B currently runs the lagging-provider backfill check
on every invocation by pulling the top 10 entries from built_in_models in
ml_model_list.py and cross-checking Fireworks/Together/SiliconFlow against
KilnModel entries, which can cost ~30 API calls per run; modify the skill to
reduce frequency by either gating Phase 1B behind an explicit "discovery mode"
flag or adding a 24-hour cache for the backfill results keyed by provider+model
(invalidate on model-add operations), and ensure the logic that reads
built_in_models and compares against KilnModel honors this gate/cache so
behavior remains identical when the check runs.
- Around line 522-524: Update the fenced code blocks that contain the WebFetch
command (e.g., the block with "WebFetch
https://fireworks.ai/models/fireworks/{model-slug}") to include a language
specifier for consistency (choose one such as ```text, ```shell, or ```bash) and
apply the same specifier to the other WebFetch examples in this document so all
WebFetch blocks use the same tag; ensure you only modify the triple-backtick
opening fence to add the chosen specifier for each WebFetch block (search for
the literal "WebFetch" blocks to find all occurrences).

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 937d76cc-beeb-4011-89c5-43264b43b076

📥 Commits

Reviewing files that changed from the base of the PR and between e8fe6b4 and 2975906.

📒 Files selected for processing (1)

• .agents/skills/claude-maintain-models/SKILL.md