Skip to content

Latest commit

 

History

History
261 lines (189 loc) · 9.91 KB

File metadata and controls

261 lines (189 loc) · 9.91 KB

Supergraph Auditor

Supergraph Auditor

Find security gaps, performance bottlenecks, and best-practice violations in your Apollo Federation supergraph -- before they hit production.

One command. 100+ checks. Actionable results.

Experimental -- This is an unofficial project under active development. Checks and output formats may change.


See It In Action

claude /audit-supergraph my-graph@production

That's it. Point it at your GraphOS graph and get a prioritized report in minutes:

Supergraph Audit Results

Graph:      se-ecom-demo@current
Subgraphs:  3 (products, customers, orders)
Findings:   40 total

  P0  4 blocking       P1  22 best practice       P2  14 recommendation

SAF Pillar Scores
  Operational Excellence:  Needs Attention  (0/3/1)
  Security:                At Risk          (4/6/2)
  Reliability:             Good             (0/0/0)
  Performance:             At Risk          (0/12/9)
  Developer Experience:    Good             (0/1/2)

Every finding tells you exactly what's wrong, where it is, and how to fix it:

FED-S002: No @authenticated directive usage on sensitive queries  [P0]

  Subgraph:  customers
  Type:      Query
  Field:     users, user

  Query fields 'users' and 'user' return PII data (email, phone, address)
  but lack @authenticated or @requiresScopes directives. These directives
  are imported in the schema but never applied.

  Ref: apollographql.com/docs/graphos/routing/security/authorization

Full example output is in example/.


Get Started

Prerequisites

  • Claude Code CLI
  • Rover CLI (for GraphOS mode)
  • APOLLO_KEY environment variable (for usage-based checks)

Install & Run

git clone https://github.com/apollosolutions/supergraph-auditor.git
cd supergraph-auditor

# Audit a graph from GraphOS (recommended -- includes live usage analysis)
claude /audit-supergraph my-graph@production

# Or audit local schema files
claude /audit-supergraph ./path/to/schemas/

How It Works

A Claude Code agent system -- no application code. The audit rules, orchestration, and report generation are all defined in agent markdown files that Claude Code executes directly.

The /audit-supergraph command fetches your schemas, then dispatches three audit agents in parallel:

/audit-supergraph my-graph@production
        |
        v
  Fetch schemas via Rover (or discover locally)
        |
        +---> @audit-federation        Entity design, directives, composition,
        |                              query planning, security, enterprise
        |                              optimizations
        |
        +---> @audit-schema-design     Naming, type design, pagination,
        |                              mutations, documentation, enterprise
        |                              readiness
        |
        +---> @audit-access-patterns   Operation hotspots, field usage,
        |                              subgraph fetch patterns (graph ref only)
        v
  Aggregate & deduplicate findings
        |
        v
  JSON report + Markdown report + SAF pillar scores

Input Modes

Mode Trigger Source
Graph Ref my-graph@production Pulls schemas from GraphOS via Rover CLI
Local path ./schemas/ Reads .graphql/.gql files from the directory
Local scan (no argument) Discovers schema files in the current repo

Graph Ref mode is preferred -- it pulls the authoritative schemas and enables the access patterns agent, which cross-references live usage data from GraphOS.

Severity Levels

Level Meaning Action
P0 Breaks composition, runtime failure, or security vulnerability Must fix before deploy
P1 Violates Apollo best practice; may cause performance issues Fix before next release
P2 Improves maintainability or developer experience Track in backlog

Audit Checks

Federation Topology (@audit-federation) -- 6 phases, 45+ checks
Phase Check IDs Examples
Entity Design & Optimization FED-E001 -- FED-E012 Missing @key, entity caching candidates, compound key vs simple ID
Directive Usage FED-D001 -- FED-D007 Incorrect @shareable, missing @external, @override without migration plan
Composition & Query Planning FED-Q001 -- FED-Q005 Circular references, excessive entity hops, unbounded @requires
Security Posture FED-S001 -- FED-S006 Missing auth on mutations, PII exposure, unbounded lists
Enterprise Optimizations FED-ENT001 -- FED-ENT012 Entity cache readiness, @defer candidates, demand control, contracts/@tag gaps
Schema Linting FED-L001 -- FED-L003 Rover lint violations, naming rule alignment
Schema Design (@audit-schema-design) -- 7 phases, 50+ checks
Phase Check IDs Examples
Naming Conventions SD-N001 -- SD-N010 Non-PascalCase types, missing Input suffix
Type Design SD-T001 -- SD-T008 JSON scalar abuse, all-optional inputs, god types (30+ fields)
Query & Mutation Design SD-M001 -- SD-M006 Coarse-grained mutations, missing error types
Pagination SD-P001 -- SD-P006 Unbounded list fields, missing connection types
Documentation SD-D001 -- SD-D005 Low description coverage, undocumented root fields
Autogeneration Detection SD-A001 -- SD-A005 Excessive nullability, REST-shaped schemas
Enterprise Readiness SD-ENT001 -- SD-ENT009 Missing @cacheControl, @contact, enterprise directive imports
Access Patterns (@audit-access-patterns) -- 3 phases, 17 checks (graph ref only)

Requires a graph ref and APOLLO_KEY. Queries the Apollo Platform API for runtime usage data and cross-references with schema structure.

Phase Check IDs Examples
Operation Hotspots AP-O001 -- AP-O005 High-traffic ops spanning 3+ subgraphs, high p99 latency
Schema Coordinate Usage AP-C001 -- AP-C006 Hot fields on non-entity types, PII without auth, @requires on hot paths
Subgraph Fetch Patterns AP-S001 -- AP-S006 N+1 fetch patterns, subgraph latency hotspots

SAF Pillar Scoring

All findings map to Apollo's Supergraph Architecture Framework pillars. The report includes a per-pillar scorecard (Good / Needs Attention / At Risk).

Pillar Focus
Operational Excellence Schema standards, documentation, evolution
Security Auth, PII, demand control, introspection
Reliability Composition, migration safety, error rates
Performance Entity caching, query planning, @defer, cache hints
Developer Experience Contracts, deprecation guidance, naming consistency

Output

Reports are written to .claude/output/:

File Contents
audit-supergraph-{session}.json Machine-readable aggregated findings, metrics, and metadata
audit-supergraph-{session}.md Human-readable report with executive summary and findings by severity
schemas/{session}/ Fetched schema snapshots from the audit run

Token Usage

Each audit spawns 3 parallel agents plus orchestrator aggregation. Estimates based on the example audit (3 subgraphs, ~15KB schemas).

Agent Input Output Total
@audit-federation ~12,700 ~3,500 ~16,200
@audit-schema-design ~10,300 ~3,600 ~13,900
@audit-access-patterns ~9,400 ~2,600 ~12,000
Orchestrator ~15,000 ~4,800 ~19,800
Lower bound ~47,400 ~14,500 ~61,900

Real usage is typically 2-4x the lower bound (reasoning, tool calls, multi-turn interactions).

Graph Size Estimated Total
Small (3 subgraphs) ~65K-105K tokens
Medium (5-10 subgraphs) ~100K-200K tokens
Large (10+ subgraphs) ~150K-300K tokens

Customization

Add a new audit agent:

  1. Create .claude/agents/audit-{domain}.md following the pattern in existing agents
  2. Add a spawn block in .claude/commands/audit-supergraph.md (Step 3)
  3. The orchestrator automatically picks up audit-*-{session}.json files during aggregation

Modify checks: Each agent embeds its full ruleset. Edit the check definitions directly in the agent markdown file -- no external config needed.


Project Structure

.claude/
  agents/
    audit-federation.md              Federation topology & entity optimization
    audit-schema-design.md           Schema design quality
    audit-access-patterns.md         Usage-aware access patterns (graph ref only)
    references/                      Supplemental reference material
  commands/
    audit-supergraph.md              /audit-supergraph entry point
  output/                            Generated reports (gitignored)
example/                             Full example audit (se-ecom-demo@current)
AGENTS.md                            Orchestration guide & reference index

License

Licensed under the Elastic License 2.0 (ELv2). License is subject to change as the project matures.

Disclaimer: This tool is provided as-is for informational purposes. Audit findings are AI-generated recommendations and should be reviewed by your team before acting on them. Apollo and the authors assume no liability for decisions made based on audit output.