You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|`.github/prompts/claude-code-review.md`| Active CI prompt (comprehensive) | Updated with fixes|
152
+
|`plugins/scm-utils/.../prompt-template.md`| Template for other repos | Updated with fixes|
153
+
|`shared/skills/self-review/SKILL.md`| Shared review procedure |**NEW** — extracted from project skill|
154
+
|`shared/skills/parallel-review/SKILL.md`| Agent Teams fan-out |**NEW**|
155
+
|`.claude/skills/code-review/SKILL.md`| (Removed)|**REMOVED** — replaced by shared skills |
146
156
147
157
The CI prompt and plugin template still need to be kept in sync manually. A future improvement could generate both from a single source.
148
158
149
159
### 3.2 Single-Agent CI Review Bottleneck
160
+
150
161
The CI workflow runs a single Claude session. For large PRs, this means:
162
+
151
163
- Long review times (often 10-20+ minutes)
152
164
- Risk of context window exhaustion on complex changes
153
165
- No parallelism for independent review concerns (security vs. quality vs. patterns)
154
166
155
167
The project-level SKILL.md already describes a multi-agent approach, but this architecture hasn't been brought to the CI workflow.
156
168
157
169
### 3.3 No Structured Output Validation
170
+
158
171
The review output format (badges, details/summary, footnotes) relies entirely on prompt instructions. There's no post-processing to validate that the output is well-formed markdown before posting. This has led to issues like CDATA wrappers appearing in posted reviews.
159
172
160
173
### 3.4 COMMENT Verdict Overuse
174
+
161
175
The original guidance made it too easy to use `COMMENT` (non-blocking) when `REQUEST_CHANGES` would be more appropriate. The threshold was "won't break if merged" rather than "would the code be better if this were addressed." This resulted in reviews that identified real improvements but didn't enforce them.
162
176
163
177
### 3.5 Template Synchronization Problem
178
+
164
179
The `interpolate-prompt` action uses `envsubst`, which means the prompt template is a flat markdown file with `${VAR}` placeholders. There's no mechanism to:
180
+
165
181
- Share common sections between the CI prompt and scm-utils template
166
182
- Version the prompt independently from the workflow
167
183
- Test prompt changes before deploying
168
184
169
185
### 3.6 Missing Review State Persistence
186
+
170
187
The CI review runs in a fresh environment every time. While the prompt instructs the agent to review previous reviews, there's no persistent memory of past review decisions. Each review re-evaluates from scratch, which can lead to:
188
+
171
189
- Score inconsistency across reviews
172
190
- Re-raising issues that were already discussed and resolved
173
191
- Missing context from conversations that happened between reviews
174
192
175
193
### 3.7 No Inline Comment Deduplication
194
+
176
195
The prompt says "future reviews don't need to re-post in-line comments" but there's no mechanism to enforce this. The agent must discover its previous inline comments via API calls during the review, adding complexity and unreliability.
177
196
178
197
---
@@ -181,13 +200,13 @@ The prompt says "future reviews don't need to re-post in-line comments" but ther
181
200
182
201
### scm-utils (Plugin) vs. ai-mktpl (Project)
183
202
184
-
| Aspect | scm-utils Plugin | ai-mktpl Project |
185
-
|--------|-----------------|------------------|
186
-
|**Purpose**| Distributable review setup for any repo | This repo's specific review instance |
187
-
|**Skill**|`code-review` — triggers CI or falls back to local review |`code-review` — multi-agent local review |
188
-
|**Prompt**| Template for new repos | Active CI prompt |
189
-
|**Workflow**| Template for new repos | Active CI workflow |
The scm-utils plugin is the **distribution mechanism**. It contains templates and a skill that knows how to trigger the CI bot (via label) or fall back to local review. The ai-mktpl project has its own instantiated copies that have evolved beyond the templates.
193
212
@@ -203,32 +222,42 @@ The scm-utils plugin is the **distribution mechanism**. It contains templates an
203
222
## 5. Suggested Improvements
204
223
205
224
### 5.1 Unify Prompt Templates (High Priority)
225
+
206
226
Extract the shared review logic into a single source of truth. The scm-utils template should be the canonical version, with the ai-mktpl prompt generated from it (or symlinked).
207
227
208
228
### 5.2 Multi-Agent CI Review (High Priority)
229
+
209
230
Bring the parallel agent architecture from the project SKILL.md into the CI workflow. Use Claude Code's Agent Teams or sub-agent fan-out to parallelize review by category. See the migration guide for details.
210
231
211
232
### 5.3 Output Validation Gate
233
+
212
234
Add a post-processing step (or hook) that validates the review markdown before posting:
235
+
213
236
- Strip any CDATA wrappers
214
237
- Verify `<details>/<summary>` structure
215
238
- Validate shields.io badge URLs return 200
216
239
- Check footnote references are complete
217
240
218
241
### 5.4 Review State Persistence
242
+
219
243
Store review decisions in a structured format (e.g., `.claude/pr-reviews/<pr>/<timestamp>/decision.json`) that subsequent reviews can reference. This enables:
244
+
220
245
- Consistent scoring across iterations
221
246
- Awareness of which issues were previously raised and addressed
Version the review prompt independently from the workflow. Tag prompt versions and include the version in the review footnotes so authors know which review criteria were applied.
226
252
227
253
### 5.6 Strengthen Non-Blocking Criteria
254
+
228
255
The non-blocking feedback fix applied in this changeset raises the bar for `COMMENT` verdicts. Monitor subsequent reviews to confirm the model respects the new criteria and adjusts to REQUEST_CHANGES for actionable improvements.
229
256
230
257
### 5.7 Review Metrics Dashboard
258
+
231
259
Instrument reviews to track:
260
+
232
261
- Review duration
233
262
- Verdict distribution (APPROVE/COMMENT/REQUEST_CHANGES)
Copy file name to clipboardExpand all lines: docs/review-system-v2-migration-guide.md
+25-11Lines changed: 25 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,6 +25,7 @@ v2 of the review system introduces:
25
25
**What changed:** The project-level `.claude/skills/code-review/SKILL.md` has been removed. Its content has been refactored into the shared `self-review` skill.
26
26
27
27
**Action required:**
28
+
28
29
```bash
29
30
# If you have a project-level code-review skill, remove it
|`self-review`| Multi-agent review procedure (scoring, dimensions, output format) |`shared/skills/self-review/`| CI workflow, local fallback, any plugin|
186
+
|`parallel-review`| Agent Teams fan-out orchestration |`shared/skills/parallel-review/`| Interactive use with agent teams enabled |
187
+
|`code-review`| Trigger CI review or fall back to local review |`plugins/scm-utils/skills/code-review/`| User-facing entry point|
175
188
176
189
### How They Interact
177
190
@@ -208,7 +221,7 @@ Currently, the CI workflow runs a single Claude session. A future v3 could adopt
208
221
- name: Run Claude Code Review
209
222
uses: anthropics/claude-code-action@v1
210
223
env:
211
-
CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS: '1'
224
+
CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS: "1"
212
225
with:
213
226
prompt: ${{ steps.prompt.outputs.prompt }}
214
227
settings: |
@@ -221,6 +234,7 @@ Currently, the CI workflow runs a single Claude session. A future v3 could adopt
221
234
This would enable true parallel review in CI, reducing review time for large PRs from 15-20 minutes to 5-8 minutes.
222
235
223
236
**Blockers for CI agent teams:**
237
+
224
238
- `claude-code-action` needs to support `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS` environment variable
225
239
- In-process teammate mode must work in GitHub Actions runners
226
240
- Token cost needs to be justified by review quality improvement
@@ -229,11 +243,11 @@ This would enable true parallel review in CI, reducing review time for large PRs
229
243
230
244
## Breaking Changes
231
245
232
-
| Change | Impact | Migration |
233
-
|--------|--------|-----------|
234
-
| Project-level `code-review` skill removed | Local `/review` commands may not trigger | Use scm-utils `code-review` or `self-review` skill instead |
235
-
| Non-blocking feedback requires justification | Reviews will have fewer COMMENT verdicts | Update prompt templates per Step 3 |
236
-
| Footnote structure changed | Existing prompt templates won't match new format | Replace prompt template per Step 3 |
0 commit comments