Context Hub (chub) - Design Document

What is Context Hub?

Context Hub bridges the gap between rapidly evolving APIs and LLM knowledge cutoffs. It's a repository of curated, LLM-optimized documentation and skills that AI agents (and humans) can search and retrieve via a CLI.

There are two kinds of content, with a fundamental distinction:

Docs ("what to know") — API/SDK reference documentation, factual knowledge that fills knowledge cutoff gaps. Large, detailed, fetched on-demand for a specific task.
Skills ("how to do it") — Behavioral instructions, coding patterns, automation playbooks. Smaller, actionable, can be installed into agent skill directories for persistent availability.

Each entry also has a source field (official | maintainer | community) for trust/quality signaling. Users control which sources agents see via ~/.chub/config.yaml.

Architecture

Content repo (source of truth)
  ↓ chub build → registry.json + content tree
CDN (serves registry + individual files + optional full bundle)   ← remote source
  ↓ CLI fetches from here
~/.chub/ (local cache)                                            ← cached remote data
  ↓ CLI reads from here
Agent/Human (consumes docs via stdout or -o file)
  ↑ CLI also reads directly from
Local folders (private/internal docs)                             ← local source

The CLI supports multiple sources — both remote CDNs and local folders. Entries from all sources are merged.

Design Decisions & Rationale

Why separate "docs" and "skills"?

We initially treated all content uniformly — just tags, no rigid types. But docs and skills have fundamentally different access patterns:

	Docs	Skills
Purpose	Reference knowledge ("what to know")	Behavioral instructions ("how to do it")
Size	Large (10K-50K+ tokens)	Small (<500 lines entry point)
Lifecycle	Ephemeral, fetched per-task	Can be persistent, installed into agent
Discovery	Agent explicitly searches and fetches	Agent can auto-discover from filesystem
Install target	`.context/` or any file	`.claude/skills/`, `.cursor/skills/`, etc.
Language/version	Yes — per-language, per-version variants	No — skills are typically language-agnostic

This distinction drives the registry format split into docs[] and skills[]. The CLI uses a single chub get <id> command that auto-detects the type.

Why `docs[]` and `skills[]` in the registry (not `entries[]`)?

The original format had a single entries[] array with a provides field to indicate doc/skill. We split it because:

Different schemas: Docs need languages[].versions[] nesting. Skills are flat — no language or version, just name, path, files.
Array membership IS the type: No need for a provides field. A doc is in docs[], a skill is in skills[].
Bundled entries: When a topic has both DOC.md and SKILL.md, they appear as separate items in their respective arrays. Clean separation.
CLI mapping: chub get searches both arrays and auto-detects type. chub search searches both.

Why skills have no language or version?

Skills are behavioral instructions ("how to integrate Stripe", "how to write Playwright login flows"). They're typically language-agnostic or written for a single context. Adding language/version nesting would add complexity without value — a skill author who needs Python and TypeScript variants can create two separate skill entries.

Docs, on the other hand, have fundamentally different content per language (Python SDK vs JavaScript SDK) and evolve with API versions.

Why a single `get` command (not `get docs` / `get skills`)?

We originally had chub get docs <id> and chub get skills <id> as separate subcommands. We simplified to chub get <id> because the CLI can auto-detect the type from the registry (docs have languages[], skills don't). The user shouldn't need to know internal taxonomy to fetch content. --lang and --version flags apply when the entry is a doc and are silently ignored for skills.

Why DOC.md and SKILL.md (not just SKILL.md)?

We considered using SKILL.md for everything since the Agent Skills spec is the format standard. But calling a 50K API reference "SKILL.md" is semantically misleading — agents that scan for skills would load doc descriptions into their system prompt (wasting ~100 tokens per doc entry), and might "activate" a doc when the user just wants to write code.

Why `--lang` flag instead of positional argument?

Originally: chub get openai-chat python. Changed to: chub get openai/chat-api --lang python.

Reasons:

Multi-id support (chub get openai/chat-api stripe/payments) would make a positional language argument ambiguous
Language can be auto-inferred when an entry has only one — the flag is only needed for disambiguation
Flags are self-documenting; a bare python after an id is ambiguous to readers

Why multi-id support?

Agents often need multiple entries in one operation. Rather than looping, chub get openai/chat-api stripe/payments fetches both. Output is concatenated with --- separators for stdout, or written as separate files when -o points to a directory.

Why one CLI, not two?

We considered separate tools for docs and skills. Rejected because they share the same registry, config, sources, search, and cache infrastructure.

Why 5+1 commands?

We started with 8 commands and trimmed to 5 core + 1 build: search, get, update, cache, feedback, and build. list and info were merged into search. get docs and get skills were merged into get with auto-detection.

Why `source` field + config-level filtering?

Each entry has source: "official" | "maintainer" | "community". The human controls trust policy via ~/.chub/config.yaml. An enterprise can restrict agents to source: official,maintainer without the agent needing to know about quality tiers.

Why tags instead of rigid categories?

Rather than rigid sub-types, entries use free-form tags. This is flexible — new categories emerge without schema changes.

Why progressive disclosure?

A monolithic 50K-token doc file wastes context. Each entry is a directory with a small entry point (DOC.md or SKILL.md, ~500 lines max) that links to detailed reference files. The agent reads the overview first, then selectively loads only what it needs.

The --full flag exists for when you want everything. With -o <dir>, --full writes individual files preserving directory structure so relative links resolve on disk. Without -o, it concatenates to stdout with # FILE: headers.

Why hybrid data strategy?

Three approaches were considered:

Full bundle (download everything) — simple but doesn't scale
Index + on-demand (fetch individual docs) — lightweight but needs network per doc
Hybrid (chosen) — registry-only by default, on-demand doc fetching, optional full bundle

Why author-prefixed IDs?

IDs are always author/name — e.g., openai/chat-api, stripe/payments, playwright-community/login-flows. The author is the top-level directory name in the content repo; the name comes from frontmatter. This eliminates name collisions by construction — two authors can both have a chat entry, but their ids differ (openai/chat-api vs mycompany/chat). This is the same pattern as npm scopes, Docker images, and GitHub repos.

Why `source:` prefix (not `source/`) for multi-source?

When multiple sources define the same id, the user disambiguates with a source: prefix: internal:openai/chat-api vs community:openai/chat-api. We use colon instead of slash because ids already contain slashes (author/name). Using source/author/name would be ambiguous — is internal the source or the author?

Why multi-source?

Teams often have internal/proprietary docs alongside the public community registry. The CLI supports multiple sources — remote CDNs and local folders. Entries are merged, and IDs are namespaced with source: only when there's a collision across sources.

Content Repository & Build Pipeline

Content repo structure

Content is organized by author directories. Each author gets a top-level directory and organizes their docs and skills inside it:

content-repo/
├── stripe/                              # author directory
│   ├── registry.json                    # OPTIONAL: author manages own index
│   ├── docs/
│   │   └── payments/                    # entry directory
│   │       ├── DOC.md                   # frontmatter: name, description, languages, versions
│   │       ├── references/
│   │       │   └── webhooks.md
│   │       └── examples/
│   │           └── checkout.py
│   └── skills/
│       └── integration/                 # entry directory
│           ├── SKILL.md
│           └── scripts/
│               └── setup.sh
├── openai/                              # no registry.json → auto-discover
│   └── docs/
│       └── chat/
│           ├── DOC.md                   # languages: "python,javascript", versions: "1.52.0"
│           └── references/
│               └── streaming.md
└── playwright-community/
    └── skills/
        └── login-flows/
            ├── SKILL.md
            └── helpers/
                └── login-util.ts

Convention: <author>/{docs,skills}/<entry-name>/ with DOC.md or SKILL.md at root.

The entry directory is the unit of content

Following the convention established by Anthropic and OpenAI skill repos:

SKILL.md (or DOC.md) lives in a directory
All other files in that directory are companions — installed together with --full
References use relative paths (e.g., [Auth](references/auth.md))
--full -o <dir> writes individual files preserving directory structure, so relative links resolve on disk
Without --full, only the entry point (DOC.md or SKILL.md) is fetched

Two discovery modes per author

1. Author provides `registry.json`

If an author directory contains registry.json, the build uses it directly. Same schema as the top-level registry (with docs[] and skills[]). Paths are prefixed with the author directory name during merge.

This is for authors with complex organization who want full control over their index.

2. Auto-discovery (no `registry.json`)

The build walks the author directory, finds all DOC.md and SKILL.md files, and parses frontmatter to generate registry entries.

DOC.md frontmatter:

---
name: chat-api
description: OpenAI Chat API - completions, streaming, function calling
metadata:
  languages: "python,javascript,typescript"    # comma-separated, multi-lang
  versions: "1.52.0"                           # comma-separated, multi-version
  updated-on: "2026-01-15"
  source: maintainer
  tags: "openai,chat,llm"
---

SKILL.md frontmatter (no language/version needed):

---
name: login-flows
description: Login flow automation patterns for Playwright
metadata:
  updated-on: "2026-01-15"
  source: community
  tags: "browser,playwright,automation"
---

Multi-language and multi-version in frontmatter

A single DOC.md can declare multiple languages and versions (comma-separated strings). The build expands this into the registry schema — multiple languages[] entries, each with multiple versions[], all pointing to the same directory path.

This means one doc file can serve Python, JavaScript, and TypeScript users if the content is language-agnostic enough. When content differs per language, authors create separate DOC.md files in separate directories — the build groups them by matching name.

Version-specific docs

When an API has breaking changes across versions, the author creates separate DOC.md files:

openai/
└── docs/
    └── chat/
        ├── v1/
        │   ├── DOC.md              # versions: "1.52.0,1.51.0", languages: "python,javascript"
        │   └── references/
        │       └── streaming.md
        └── v2/
            ├── DOC.md              # versions: "2.0.0", languages: "python,javascript"
            └── references/
                ├── streaming.md
                └── structured-outputs.md

Both DOC.md files have name: chat-api (under the openai/ author directory) — they get grouped into id: openai/chat-api, into one docs[] entry with multiple versions pointing to different paths. recommendedVersion is the highest semver. chub get openai/chat-api gets the latest; --version 1.52.0 gets the older docs.

Language-specific docs

For different content per language:

stripe/
└── docs/
    └── payments/
        ├── python/
        │   └── DOC.md          # languages: "python", versions: "14.0.0"
        └── javascript/
            └── DOC.md          # languages: "javascript", versions: "14.0.0"

Same name: payments under stripe/ → both contribute to id: stripe/payments. Different languages, different paths, different content.

The `chub build` command

chub build <content-dir> [options]

Options:

-o, --output <dir> — output directory (default: <content-dir>/dist)
--base-url <url> — set base_url in registry (for CDN deployment)
--validate-only — check frontmatter and structure without writing output
--json — output build summary as JSON

Build steps:

List top-level directories in <content-dir> (author directories)
For each author directory:
- If registry.json exists → use it directly, prefix paths
- Else → auto-discover DOC.md/SKILL.md, parse frontmatter, group by name
Merge all author entries into one registry (ids are author/name, so collisions are rare)
Write registry.json to output dir
Copy content tree to output dir (preserving structure)
Print summary: N docs, N skills, N warnings

Validation rules

DOC.md must have name, description, metadata.languages, metadata.versions
SKILL.md must have name, description (no language/version required)
Warn on missing metadata.source (default: "community")
Warn on missing metadata.tags
Error on duplicate id (rare since ids are author/name)
If both DOC.md and SKILL.md exist in the same directory, name must match

Publishing to CDN

The build output is a static directory ready to serve:

dist/
├── registry.json                              # Generated index
├── stripe/docs/payments/DOC.md               # Content files (copied)
├── stripe/docs/payments/references/...
├── openai/docs/chat-api/DOC.md
└── playwright-community/skills/login-flows/SKILL.md

Upload dist/ to any static file host (S3, CloudFlare R2, GitHub Pages). The CLI fetches registry.json first, then individual files on demand.

CLI Interface

Commands

Command	Purpose	Key Options
`chub search [query]`	Search (no query = list all, exact id = detail)	`--tags`, `--lang`, `--limit`, `--json`
`chub get <ids...>`	Fetch docs or skills (auto-detects type)	`--lang`, `--version`, `--full`, `-o <path>`, `--json`
`chub update`	Refresh cached registry	`--force`, `--full`
`chub cache status\|clear`	Manage local cache
`chub build <content-dir>`	Build registry from content	`-o`, `--base-url`, `--validate-only`, `--json`

How `search` works

chub search — lists all entries (replaces list)
chub search openai/chat-api — exact id match shows full detail (replaces info)
chub search "stripe" — fuzzy search across id, name, description, tags
chub search --tags browser — filtered listing
Results show [doc] or [skill] type labels

How `get` works

chub get openai/chat-api --lang python — auto-detects doc, fetches DOC.md
chub get openai/chat-api --full — fetch all files in the entry
chub get openai/chat-api --full -o .context/openai/ — write individual files preserving structure
chub get openai/chat-api stripe/payments --lang js — fetch multiple entries at once
chub get pw-community/login-flows — auto-detects skill, fetches SKILL.md
chub get nonexistent/thing → error: Entry "nonexistent/thing" not found.

Language inference

Entry has one language → auto-selected, no --lang needed
Entry has multiple languages, no --lang → error with suggestion
--lang applies to all ids in a multi-id command
--lang and --version apply to doc entries, silently ignored for skills

Output modes

Default: Human-friendly, colored terminal output
--json: Structured JSON to stdout (no color escapes)
-o <path>: Write content to file, print short confirmation to stderr
-o <dir>/: Write each entry as separate file when fetching multiple
--full -o <dir>: Write individual files preserving directory structure

Agent piping patterns

# Get the top search result's id
chub search "stripe payments" --json | jq -r '.results[0].id'

# Full pipeline: search → pick best → fetch → write to file
ID=$(chub search "stripe payments" --json | jq -r '.results[0].id')
chub get "$ID" --lang js -o .context/stripe.md

# Fetch top 3 results
chub search "stripe" --json | jq -r '.results[:3][].id' | xargs chub get -o .context/

# Fetch multiple at once
chub get openai/chat-api stripe/payments -o .context/

# Install a skill into Claude Code's skill directory
chub get pw-community/login-flows -o .claude/skills/login-flows/SKILL.md

# Install a skill with all companion files
chub get pw-community/login-flows --full -o .claude/skills/login-flows/

# Multi-source: disambiguate with source: prefix
chub get internal:openai/chat-api

Data Strategy

Content format: Agent Skills compatible

All content follows the Agent Skills spec. Both DOC.md and SKILL.md use the standard's frontmatter format (name, description, optional metadata). This makes chub content interoperable with Claude Code, Cursor, Codex, OpenCode, and 30+ agents.

What the CDN serves

cdn.aichub.org/v1/
├── registry.json                                        # Index (~100KB)
├── bundle.tar.gz                                        # Full bundle (optional)
├── stripe/docs/payments/DOC.md                         # Entry point
├── stripe/docs/payments/references/webhooks.md         # Supporting file
└── playwright-community/skills/login-flows/SKILL.md    # Skill

How the CLI uses it

chub update → fetches registry.json only (~100KB), caches locally
chub search → searches local registry (no network)
chub get <id> → auto-detects type, fetches entry point (DOC.md or SKILL.md), checks cache first
chub get <id> --full → fetches all files listed in registry
chub update --full → downloads entire bundle.tar.gz for offline use

Local cache layout

~/.chub/
├── config.yaml              # User config (optional, created manually)
└── sources/                 # Per-source cache (remote sources only)
    ├── community/
    │   ├── registry.json    # Cached index for this source
    │   ├── meta.json        # { lastUpdated, registryHash }
    │   └── data/            # Cached content (on-demand or full bundle)
    └── another-remote/
        └── ...

Local path sources are not cached — the CLI reads directly from the configured path.

Schemas

Registry (`registry.json`)

{
  "version": "1.0.0",
  "base_url": "https://cdn.aichub.org/v1",
  "generated": "2026-02-02T00:00:00.000Z",
  "docs": [
    {
      "id": "openai/chat-api",
      "name": "chat-api",
      "description": "Chat completions with GPT models",
      "source": "maintainer",
      "tags": ["openai", "chat", "llm"],
      "languages": [
        {
          "language": "python",
          "versions": [
            {
              "version": "1.52.0",
              "path": "openai/docs/chat-api/v1",
              "files": ["DOC.md", "references/streaming.md"],
              "size": 42000,
              "lastUpdated": "2026-01-15"
            }
          ],
          "recommendedVersion": "1.52.0"
        }
      ]
    }
  ],
  "skills": [
    {
      "id": "playwright-community/login-flows",
      "name": "login-flows",
      "description": "Login flow automation patterns for Playwright",
      "source": "community",
      "tags": ["browser", "playwright"],
      "path": "playwright-community/skills/login-flows",
      "files": ["SKILL.md", "helpers/login-util.ts"],
      "size": 12000,
      "lastUpdated": "2026-01-15"
    }
  ]
}

Doc entry fields:

id — unique identifier in author/name format, used by chub get <id>
name — short name from frontmatter (the part after the author prefix)
description — short description for search results
source — official (library author), maintainer (context-hub team), community
tags — free-form tags for filtering
languages[] — per-language grouping
- versions[] — per-version, each with path, files, size, lastUpdated
- recommendedVersion — default version to fetch

Skill entry fields:

name, description, source, tags — same as docs
path — directory path (relative to base_url or source root)
files — all files in the entry directory
size, lastUpdated — flat, no language/version nesting

Config (`~/.chub/config.yaml`)

# Multi-source (recommended)
sources:
  - name: community
    url: https://cdn.aichub.org/v1       # Remote CDN
  - name: internal
    path: /path/to/local/docs                # Local folder (build output)

# Trust policy: which entry sources to show
source: "official,maintainer,community"

# Optional
refresh_interval: 86400                       # Cache TTL in seconds (24h)

Backward compat: If no sources array, falls back to single cdn_url field (or CHUB_BUNDLE_URL env var) as a source named "default".

Local source: Can be either a raw content repo or a chub build output directory — both must contain registry.json at root with the standard schema.

Agent Skills Compatibility

Content follows the Agent Skills open standard, supported by Claude Code, Cursor, Codex, OpenCode, and 30+ agents.

How chub relates to the Agent Skills ecosystem

Layer	Agent Skills spec	npx skills (Vercel)	chub
Format	SKILL.md with frontmatter	SKILL.md	SKILL.md + DOC.md
Discovery	Local filesystem scan	`npx skills search` (git repos)	`chub search` (registry index)
Distribution	None (copy files)	Git repos	CDN + local folders
Versioning	None	None	Per-entry, per-language (docs)
Multi-language	None	None	Yes (docs)
Trust/quality	None	None	`source` field + config filtering
Build pipeline	None	None	`chub build`

Why adopt the standard?

Makes chub content interoperable with the broader agent ecosystem. A skill fetched via chub get can be piped directly into any agent's skill directory and discovered natively.

How chub extends it

Registry-based search and discovery over network
Multi-source aggregation (CDN + local folders)
Trust/quality filtering via source field
Progressive disclosure with --full flag
DOC.md for reference knowledge (uses same frontmatter format)
Build pipeline to generate registry from content directories

Project Structure

chub-first-draft/
├── cli/
│   ├── package.json              # npm package with bin entry
│   ├── bin/chub                  # #!/usr/bin/env node entry point
│   ├── src/
│   │   ├── index.js              # Commander setup, global --json, preAction cache hook
│   │   ├── commands/
│   │   │   ├── search.js         # search / list / info (all in one)
│   │   │   ├── get.js            # get command (auto-detects doc/skill)
│   │   │   ├── build.js          # build registry from content directory
│   │   │   ├── update.js         # refresh registry / full bundle
│   │   │   └── cache.js          # cache status / clear
│   │   └── lib/
│   │       ├── config.js         # Load config.yaml, merge env vars, defaults
│   │       ├── cache.js          # Registry fetch, on-demand doc fetch, bundle extract
│   │       ├── registry.js       # Load registry, search/filter/query, resolve paths
│   │       ├── frontmatter.js    # YAML frontmatter parser
│   │       ├── output.js         # Dual-mode output (human with chalk / JSON)
│   │       └── normalize.js      # Language aliases (js→javascript, py→python)
├── plans-for-reference/          # Archived design plans
├── NARRATIVE.md                  # Product pitch
├── DESIGN.md                     # This file
├── .gitignore
└── package.json                  # Root workspace

Dependencies

commander ^12 — CLI framework
chalk ^5 — Terminal colors
yaml ^2 — Config + frontmatter parsing
tar ^7 — Bundle extraction (for --full mode)
Node.js >= 18 (built-in fetch, no node-fetch needed)

Future considerations

skills_dir / docs_dir config — default output directories for skills and docs
Agent detection — auto-detect installed agents and write to the right skill directory
chub install — dedicated install command if the piping pattern proves too verbose
Usage telemetry — agents report which docs/skills they used, enabling quality signals
CI/CD integration — GitHub Action that runs chub build and publishes to CDN on push

Reference

Agent Skills specification: https://agentskills.io/specification
Vercel Skills CLI: https://github.com/vercel-labs/skills

FilesExpand file tree

design.md

Latest commit

History