Skip to content

Latest commit

 

History

History
251 lines (192 loc) · 11.1 KB

File metadata and controls

251 lines (192 loc) · 11.1 KB

Apollo Supergraph Audit System

Agent-based auditing tools for Apollo GraphQL supergraphs. Performs parallel, phased analysis of federated schemas and produces structured findings reports.

Architecture

AGENTS.md                              ← You are here. Orchestration + reference index.
.claude/
  agents/
    audit-federation.md                ← Entity design & optimization, directives, composition, security
    audit-schema-design.md             ← Naming, type design, pagination, mutations, docs
    audit-access-patterns.md           ← Usage-aware: operation hotspots, field usage, subgraph fetches
    audit-operations.md                ← Client operation efficiency (TODO)
    references/
      federation-error-codes.md        ← Full GraphOS composition error catalog
      graphos-linting-rules.md         ← Complete linting rule reference
  commands/
    audit-supergraph.md                ← /audit-supergraph entry point

Agents perform audit work. Each is self-contained with embedded rules. References provide overflow detail agents can Read when a check needs deeper context than the embedded rules provide. Commands are user-facing entry points.

No external skills are required. All audit knowledge is embedded directly in agent files. This ensures 100% context availability — agents never need to decide whether to load reference material.

Input Modes

The audit system supports two input modes:

Mode 1: Graph Ref (preferred)

Point directly at a graph in Apollo GraphOS using a graph ref:

/audit-supergraph my-graph@production
/audit-supergraph my-graph@staging

This uses Rover CLI to pull schemas from GraphOS — the single source of truth. The orchestrator will:

  1. rover subgraph list {graph_ref} — enumerate all subgraphs
  2. rover subgraph fetch {graph_ref} --name {name} — pull each subgraph SDL
  3. rover supergraph fetch {graph_ref} — pull the composed supergraph schema
  4. Write all schemas to .claude/output/schemas/{session}/
  5. Dispatch audit agents against the fetched files

Requirements: Rover CLI installed and authenticated (APOLLO_KEY set or rover config auth completed). The graph ref must be accessible.

Mode 2: Local Files (fallback)

If no graph ref is provided, the orchestrator falls back to scanning the local repository for schema files:

/audit-supergraph
/audit-supergraph ./path/to/schemas/

This uses Glob/Grep to discover .graphql, .gql, and code-embedded schemas in the repo. Less authoritative than Mode 1 since local files may be stale or incomplete.

Rover Commands Reference

These are the Rover commands the audit system uses. Agents have Bash(rover:*) in their allowed tools.

rover subgraph list {graph}@{variant}
  → Lists all subgraphs with names, routing URLs, last updated timestamps

rover subgraph fetch {graph}@{variant} --name {subgraph_name}
  → Fetches a single subgraph's SDL from GraphOS
  → Use --output {path} to write to file

rover supergraph fetch {graph}@{variant}
  → Fetches the composed supergraph schema (includes routing metadata)
  → Use --output {path} to write to file

rover subgraph check {graph}@{variant} --name {name} --schema {path}
  → Runs schema checks (build + operations + linting) against GraphOS
  → Used in Phase 5 of audit-federation for linting alignment

rover subgraph lint {graph}@{variant} --name {name} --schema {path}
  → Runs GraphOS linter rules against a subgraph schema
  → Validates naming, composition rules, best practices

Audit Workflow

When a user invokes /audit-supergraph:

Step 1: Resolve Input

Determine the input mode:

  • If argument matches {graph}@{variant} pattern → Graph Ref mode
  • If argument is a file path → Local Files mode
  • If no argument → Local Files mode, scan current directory

Step 2: Fetch Schemas (Graph Ref mode)

SESSION=$(date +%Y%m%d-%H%M%S)
SCHEMA_DIR=".claude/output/schemas/$SESSION"
mkdir -p "$SCHEMA_DIR"

# List all subgraphs
rover subgraph list {graph_ref} > "$SCHEMA_DIR/subgraph-list.txt"

# Fetch each subgraph SDL
for name in $(parse subgraph names from list); do
  rover subgraph fetch {graph_ref} --name "$name" \
    --output "$SCHEMA_DIR/$name.graphql"
done

# Fetch composed supergraph schema
rover supergraph fetch {graph_ref} \
  --output "$SCHEMA_DIR/supergraph.graphql"

Step 2 (alt): Discover Schemas (Local Files mode)

Use the Explore agent to find schema files in the repo. Group by subgraph based on directory structure.

Step 3: Parallel Audit Dispatch

Spawn named audit agents in parallel with the schema file paths:

@audit-federation       → Entity design & optimization, directives, composition, query planning, security
@audit-schema-design    → Naming, type design, pagination, mutations, documentation
@audit-access-patterns  → Operation hotspots, field usage, subgraph fetch patterns (graph ref only)
@audit-operations       → Client operations (only if operation files detected)

Each agent receives:

  • The list of subgraph schema file paths
  • The supergraph schema path (if available)
  • The graph ref (if available, for running rover subgraph lint)
  • The session ID

Each agent writes findings to .claude/output/audit-{domain}-{session}.json.

Step 4: Aggregation and Report Generation

After all agents complete:

  1. Read all .claude/output/audit-*-{session}.json files
  2. Deduplicate findings that overlap between agents
  3. Aggregate into a single summary report
  4. Sort findings by severity (P0 first)
  5. Write aggregated JSON to .claude/output/audit-supergraph-{session}.json
  6. Generate markdown report to .claude/output/audit-supergraph-{session}.md

Step 5: Present Results

Present to the user:

  • Graph ref and variant audited (or "local files")
  • Subgraph count and names
  • Total finding counts by severity
  • P0 findings listed individually
  • P1 findings summarized by category
  • P2 findings counted only
  • Link to full JSON report and markdown report

If any P0 findings exist, state clearly: "This supergraph has blocking issues that should be resolved before deployment."

Severity Definitions

Consistent across all audit agents:

Level Meaning Action
P0 Breaks composition, causes runtime failure, or exposes security vulnerability Must fix before deploy
P1 Violates Apollo documented best practice; may cause performance degradation Fix before next release
P2 Recommendation that improves maintainability or developer experience Track in backlog

SAF Pillar Integration

All findings are tagged with one of the five Supergraph Architecture Framework pillars. The aggregated report includes a per-pillar scorecard with Good / Needs Attention / At Risk assessments.

SAF Pillar What It Covers Example Checks
Operational Excellence Schema standards, linting, documentation, schema evolution FED-L*, SD-N*, SD-D*, SD-ENT004, SD-ENT005
Security Auth, PII, DoS, demand control, introspection FED-S*, FED-ENT006–007, FED-ENT009–010, AP-O005, AP-C003, SD-ENT007, SD-ENT009
Reliability Composition, migration safety, error rates FED-E004–E005, FED-D003, FED-D007, FED-Q005, FED-ENT004, AP-O003, AP-S003, SD-ENT006
Performance Entity caching, query planning, @defer, caching hints FED-E007–E012, FED-ENT001–003, FED-ENT005, FED-ENT011–012, AP-O001–O004, AP-C001–C002, SD-ENT001–002
Developer Experience Contracts, deprecation, naming, DX patterns FED-ENT008, SD-ENT003, SD-ENT008, AP-C006

Reference Index

Canonical Apollo documentation pages. Agents cite these URLs in findings.

Entity best practices         → apollographql.com/docs/graphos/schema-design/federated-schemas/entities/best-practices
Define advanced keys           → apollographql.com/docs/graphos/schema-design/federated-schemas/entities/define-keys
Contribute entity fields       → apollographql.com/docs/graphos/schema-design/federated-schemas/entities/contribute-fields
Value types and @shareable     → apollographql.com/docs/graphos/schema-design/federated-schemas/sharing-types
Schema composition             → apollographql.com/docs/graphos/schema-design/federated-schemas/composition
Federation directives          → apollographql.com/docs/graphos/schema-design/federated-schemas/reference/directives
Federation error codes         → apollographql.com/docs/graphos/schema-design/federated-schemas/reference/errors
Query planning best practices  → apollographql.com/docs/graphos/routing/query-planning/query-planning-best-practices
Entity caching                 → apollographql.com/docs/graphos/routing/performance/entity-caching
Authorization directives       → apollographql.com/docs/graphos/routing/security/authorization
Graph security                 → apollographql.com/docs/graphos/platform/security/overview
Schema linting rules           → apollographql.com/docs/graphos/platform/schema-management/linting/rules
Schema checks                  → apollographql.com/docs/graphos/platform/schema-management/checks
Deployment best practices      → apollographql.com/docs/graphos/platform/production-readiness/deployment-best-practices
Demand-oriented schema design  → apollographql.com/docs/graphos/schema-design/guides/demand-oriented-schema-design
Errors as data                 → apollographql.com/docs/graphos/schema-design/guides/errors-as-data-explained
Relay-style connections        → apollographql.com/docs/graphos/schema-design/guides/relay-style-connections
Rover subgraph commands        → apollographql.com/docs/rover/commands/subgraphs
Rover supergraph commands      → apollographql.com/docs/rover/commands/supergraphs
Platform API (GraphOS)         → graphql.api.apollographql.com/api/graphql

# Enterprise features
Demand control                → apollographql.com/docs/graphos/routing/security/demand-control
Progressive @override         → apollographql.com/docs/graphos/schema-design/federated-schemas/entities/migrate-fields
@defer support                → apollographql.com/docs/graphos/routing/performance/defer
Contracts                     → apollographql.com/docs/graphos/delivery/contracts
Persisted queries             → apollographql.com/docs/graphos/routing/security/persisted-queries
SAF overview                  → apollographql.com/docs/graphos/resources/saf
SAF assessment                → saf.apollographql.com

Rules for All Agents

  1. Read-only. Audit agents never modify files.
  2. Self-contained. All rules are embedded in the agent file. No external skill loading required.
  3. Cite sources. Every finding includes a reference URL from the index above.
  4. Be specific. Findings reference file paths, type names, and field names.
  5. Structured output. All findings are JSON. Human-readable summaries come from the orchestrator, not individual agents.
  6. Fail open. If an agent can't determine whether something is a violation, report it as P2 with "requires runtime verification."
  7. Fast scanning. Use Grep to identify hot files before doing deep Reads.