Docs <-> Galaxy Review Agents

Author: jmchilton

.claude/CLAUDE.md

@@ -105,8 +105,7 @@ Each topic is defined by three files:
    - Related topics and code paths
 
 2. **content.yaml** - Ordered sequence of content blocks
-   - Each block: type (prose/slide), id, heading, content source
-   - Smart defaults: prose renders in docs only; slides render everywhere
+   - Each block: type, id, heading, content source
    - Content from inline, single file, or multiple fragments
 
 3. **fragments/** - Actual content (optional for granular organization)
@@ -115,6 +114,20 @@ Each topic is defined by three files:
 
 All content validated with Pydantic v2 models before build.
 
+### Content Block Types
+
+| Type | Slides | Sphinx Docs | Agent Commands | Use Case |
+|------|--------|-------------|----------------|----------|
+| `slide` | ✅ | ✅ | ✅ | Training slides, also rendered in docs |
+| `prose` | ❌ | ✅ | ✅ | Extended docs content, not for slides |
+| `agent-context` | ❌ | ❌ | ✅ | Context only for agent command generation |
+
+**slide** - Primary content type. Rendered in training slides AND Sphinx docs. Use for core architectural explanations.
+
+**prose** - Extended documentation. Rendered in Sphinx docs only. Use for detailed explanations too verbose for slides.
+
+**agent-context** - Agent-only context. NOT rendered anywhere. Use for guidance, examples, anti-patterns, and tips specifically for AI agents generating code or performing reviews. This content enriches generated agentic operations without cluttering human-facing docs.
+
 ## Build Artifacts
 
 - **outputs/training-slides/generated/** - GTN-compatible Remark.js slides

.claude/commands/generate-agentic-op.md

@@ -29,6 +29,8 @@ Generate Claude slash command or skill from topic content by reading and reasoni
 
 2. **Understand the operation:**
    - Read the operation's `prompt` field to understand the development task
+   - If the prompt contains a TODO item, it is a indication that this is information you should
+     generate from the topic content and embed in the generated artifact.
    - Determine the operation type (claude-slash-command or claude-skill)
 
 3. **Reason about relevant content:**
@@ -61,6 +63,8 @@ Generate Claude slash command or skill from topic content by reading and reasoni
 ## Important Notes
 
 - **DO NOT mechanically copy** all content - use judgment about what's relevant
+- **DO** If either the files you're asked to generate already exist, do not read them, back them up,
+  and your new summary to the supplied path.
 - **DO** synthesize and reformulate content appropriately for the operation
 - **DO** prominently feature related_code_paths (Galaxy available at these paths)
 - **DO** fail if the operation doesn't exist or content is insufficient

.claude/commands/harmonize.md

@@ -0,0 +1,138 @@
+# Harmonize Agentic Operation
+
+Improve alignment between architecture documentation and generated agentic operations.
+
+This command orchestrates the positive feedback loop between architecture documentation in this repository and agentic operations for acting on the Galaxy codebase. The goal is to make commands as sharp and intelligent as possible while being completely generated from architecture documentation.
+
+## Usage
+
+```
+/harmonize <existing-command-path> <topic-id> <operation-name>
+```
+
+## Arguments
+
+- **existing-command-path**: Path to current slash command (e.g., `~/.claude/commands/review-di.md`)
+- **topic-id**: Topic in this repo (e.g., `dependency-injection`)
+- **operation-name**: Operation from metadata.yaml (e.g., `review-di`)
+
+## Your Task
+
+### Phase 1: Load and Analyze
+
+1. **Read the existing command** from the supplied path
+2. **Read topic content**: `topics/<topic>/metadata.yaml` and `topics/<topic>/content.yaml`
+3. **Read the operation's prompt** from metadata.yaml
+4. **Create directory** `topics/<topic>/harmonize/` if needed
+5. **Write analysis** to `topics/<topic>/harmonize/<op>_analysis.md`:
+
+```markdown
+# Harmonization Analysis: <operation-name>
+
+## Information in Command NOT in Docs
+
+<!-- List specific patterns, examples, guidance present in command but missing from content.yaml -->
+
+- [ ] ...
+
+## Information in Docs NOT in Command
+
+<!-- List architectural content from content.yaml not reflected in the command -->
+
+- [ ] ...
+
+## Gaps in Operation Prompt
+
+<!-- What does the prompt ask for that the docs can't provide? -->
+
+- [ ] ...
+```
+
+### Phase 2: Generate Fresh Command
+
+Launch a **subagent** to generate a fresh command in isolation:
+
+```
+Use the Task tool with:
+- subagent_type: "general-purpose"
+- prompt: |
+    Run the /generate-agentic-op skill with arguments: <topic-id> <operation-name>
+
+    This will read topics/<topic>/metadata.yaml and content.yaml, then generate
+    a command to generated_agentic_operations/commands/<topic>-<op>.md
+
+    Return the path to the generated file when complete.
+```
+
+**Why a subagent?** The generation task should reason fresh from the documentation without being influenced by knowledge of the existing command or harmonization goals. This isolation ensures we see what the docs *actually* produce.
+
+After the subagent completes, read the generated file at `generated_agentic_operations/commands/<topic>-<op>.md` for Phase 3 comparison.
+
+### Phase 3: Compare Commands
+
+Read both commands and append a comparison section to the analysis file:
+
+```markdown
+## Command Comparison
+
+| Dimension | Original | Generated | Winner |
+|-----------|----------|-----------|--------|
+| Specificity of code patterns | ... | ... | ... |
+| Coverage of anti-patterns | ... | ... | ... |
+| Actionability of guidance | ... | ... | ... |
+| Use of related_code_paths | ... | ... | ... |
+| Alignment with docs | ... | ... | ... |
+
+## Key Differences
+
+1. ...
+2. ...
+3. ...
+```
+
+### Phase 4: Generate Recommendations
+
+Write recommendations to `topics/<topic>/harmonize/<op>_recommendations.md`.
+
+Split into two categories - this separation is critical:
+
+```markdown
+# Harmonization Recommendations: <operation-name>
+
+## Documentation Improvements
+
+<!-- Changes to content.yaml to add missing reference info -->
+<!-- These become the source of truth that commands are generated from -->
+
+1. Add prose block about X
+2. Expand slide Y to include Z pattern
+3. Add agent-context block for anti-patterns
+4. Add example code showing pattern W
+
+## Prompt Improvements
+
+<!-- Changes to metadata.yaml operation prompt -->
+<!-- NOT content duplication - prompts should reference docs, not repeat them -->
+<!-- Focus on: how to interact with agent, prods to pull specific info from docs -->
+
+1. Add instruction to consult related_code_paths for examples
+2. Add TODO marker for agent to extract patterns from docs
+3. Refine scope/focus of the operation
+4. Clarify input format expectations
+```
+
+### Phase 5: Summary
+
+Output a brief summary to the user:
+- What gaps were found
+- Key differences between original and generated commands
+- Top 3 actionable recommendations
+- Paths to created artifacts
+
+## Important Notes
+
+- **Documentation is source of truth** - prompts should reference docs, not duplicate content
+- **Create concrete artifacts** - don't just think, write files
+- **Isolation matters** - Phase 2 subagent must not see the existing command
+- **Prompt changes are about interaction** - how to guide the agent to use docs, not what to embed
+- If prose content is too verbose for slides, suggest prose blocks or agent-context blocks

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "review/galaxy-plugins"]
+	path = review/galaxy-plugins
+	url = https://github.com/galaxyproject/claude-galaxy-plugins

README.md

@@ -192,6 +192,77 @@ galaxy-architecture/
 - **Hub Articles**: Generate galaxyproject.org articles
 - **Galaxy Repo Migration**: Move content into main Galaxy repository
 
+## Agentic Code Review
+
+Architecture documentation enables **agentic code review** - generating AI-powered review commands from architectural knowledge.
+
+### The Plugin Marketplace
+
+The `review/` directory builds the `claude-galaxy-plugins` marketplace containing slash commands for reviewing Galaxy contributions. Commands come from two sources:
+
+- **Static commands**: Hand-written review prompts (`review/static_commands/`)
+- **Generated commands**: Created from architecture topics via `/generate-agentic-op`
+
+Build and use:
+```bash
+cd review && make
+claude --plugin-dir review/galaxy-plugins
+/gx-arch-review:gx-review <pr-or-commit>
+```
+
+Or install from GitHub:
+```bash
+/plugin marketplace add galaxyproject/claude-galaxy-plugins
+/plugin install gx-arch-review@claude-galaxy-plugins
+```
+
+See [review/galaxy-plugins/README.md](review/galaxy-plugins/README.md) for full marketplace documentation.
+
+### The Feedback Loop
+
+Architecture documentation, agentic commands, and code reviews form a **positive feedback loop**:
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                                                                     │
+│   ┌──────────────────┐         ┌──────────────────┐                │
+│   │   Architecture   │────────▶│ Agentic Commands │                │
+│   │  Documentation   │         │   (gx-review)    │                │
+│   └────────▲─────────┘         └────────┬─────────┘                │
+│            │                            │                          │
+│            │                            ▼                          │
+│   ┌────────┴─────────┐         ┌──────────────────┐                │
+│   │   Suggestions    │◀────────│   Code Reviews   │                │
+│   │ (topic/suggests/)│         │                  │                │
+│   └──────────────────┘         └──────────────────┘                │
+│                                                                     │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+**Documentation → Commands**: Generating agentic commands from architecture docs produces `suggestions/` files identifying documentation gaps. If a topic lacks detail to produce a useful review command, that's signal to improve the docs.
+
+**Commands → Reviews**: Better architecture documentation yields more comprehensive review commands that catch more issues.
+
+**Reviews → Documentation**: When agentic reviews miss something a human reviewer catches, this reveals gaps in architecture documentation. The missed pattern should be documented, improving future command generation.
+
+This virtuous cycle means:
+- Every documentation improvement makes reviews better
+- Every review gap improves documentation
+- The system self-improves through use
+
+### Slash Commands
+
+| Command | Source | Description |
+|---------|--------|-------------|
+| `/generate-agentic-op` | Built-in | Generate review command from topic |
+| `/gx-arch-review:gx-review` | Plugin | Orchestrator - runs applicable sub-reviews |
+| `/gx-arch-review:review-di` | Generated | Dependency injection patterns |
+| `/gx-arch-review:review-business-logic-organization` | Generated | Controller/Service/Manager layers |
+| `/gx-arch-review:py-challenge-patches` | Static | Mock/patch quality in tests |
+| `/gx-arch-review:gx-vitest-review` | Static | Vue/TypeScript test review |
+
+See `review/galaxy-plugins/plugins/gx-arch-review/README.md` for the complete command list.
+
 ## Philosophy
 
 - **Clean Content First**: Source of truth is markdown, not presentation markup