Add agent-browser skills command with evals (#1225)

* Add `agent-browser skills` command

Adds a `skills` CLI command that serves bundled skill content at runtime,
always matching the installed CLI version. This solves the problem of
agents relying on stale cached SKILL.md files after CLI upgrades.

The `npx skills add vercel-labs/agent-browser` flow now installs a single
thin discovery skill with trigger words for all use cases (browser
automation, dogfooding, Electron apps, Slack, etc.) that directs agents
to `agent-browser skills get <name>` for current instructions. The other
five skills (dogfood, electron, slack, vercel-sandbox, agentcore) are
marked `metadata.internal: true` so they are not installed by default but
remain accessible via the CLI command.

Subcommands:
  skills [list]              List available skills
  skills get <name> [--full] Get skill content (with optional references)
  skills get --all           Get all skill content
  skills path [name]         Print skill directory path

* Fix skills command robustness: UTF-8 safety, flag handling, path output

- Make truncate_description UTF-8-safe using char_indices() instead of
  byte-indexed slicing that panics on multi-byte codepoints
- Pass get_all as a bool parameter to run_get instead of embedding
  --all as a sentinel string in the names list
- Canonicalize skills_dir path so `skills path` output is clean
- Warn on unrecognized flags in `skills get` instead of silently
  ignoring them

* Add evals framework and strengthen SKILL.md for better agent compliance

Strengthen SKILL.md loading instructions to require `skills get` before
running commands, and trim skill descriptions to prevent agents from
guessing at command syntax. Add TypeScript/Bun eval framework that tests
skill-loading, skill-selection, and command-usage via Claude CLI with
Vercel AI Gateway. Evals pass 20/20 (100%), up from 85% baseline.

* Fix formatting in skills.rs

* Add Codex provider to evals framework

Add multi-provider support with a shared Provider interface. Codex
provider spawns `codex exec --json`, parses JSONL output, and writes
~/.codex/config.toml for AI Gateway routing. Use `--provider codex`
to run evals with Codex (default model: openai/o3). First run scores
19/20 (95%) with 100% on skill-loading and skill-selection.

* Use scoped temp dir for Codex config instead of overwriting ~/.codex

Author: ctate

README.md

@@ -371,6 +371,19 @@ agent-browser install --with-deps     # Also install system deps (Linux)
 agent-browser upgrade                 # Upgrade agent-browser to the latest version
 ```
 
+### Skills
+
+```bash
+agent-browser skills                  # List available skills
+agent-browser skills list             # Same as above
+agent-browser skills get <name>       # Output a skill's full content
+agent-browser skills get <name> --full  # Include references and templates
+agent-browser skills get --all        # Output every skill
+agent-browser skills path [name]      # Print skill directory path
+```
+
+Serves bundled skill content that always matches the installed CLI version. AI agents use this to get current instructions rather than relying on cached copies. Set `AGENT_BROWSER_SKILLS_DIR` to override the skills directory path.
+
 ## Authentication
 
 agent-browser provides multiple ways to persist login sessions so you don't re-authenticate every run.

cli/Cargo.lock

@@ -42,6 +42,9 @@ libc = "0.2"
 [target.'cfg(windows)'.dependencies]
 windows-sys = { version = "0.52", features = ["Win32_System_Threading", "Win32_Foundation"] }
 
+[dev-dependencies]
+tempfile = "3"
+
 [build-dependencies]
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"

cli/Cargo.toml

@@ -6,6 +6,7 @@ mod flags;
 mod install;
 mod native;
 mod output;
+mod skills;
 #[cfg(test)]
 mod test_utils;
 mod upgrade;
@@ -685,6 +686,12 @@ fn main() {
         return;
     }
 
+    // Handle skills command (doesn't need daemon)
+    if clean.first().map(|s| s.as_str()) == Some("skills") {
+        skills::run_skills(&clean, flags.json);
+        return;
+    }
+
     // Handle session separately (doesn't need daemon)
     if clean.first().map(|s| s.as_str()) == Some("session") {
         run_session(&clean, &flags.session, flags.json);

cli/src/main.rs

@@ -2712,6 +2712,40 @@ Examples:
 "##
         }
 
+        "skills" => {
+            r##"
+agent-browser skills - List and retrieve bundled skill content
+
+Usage: agent-browser skills [subcommand] [options]
+
+Subcommands:
+  list                       List all available skills (default)
+  get <name> [name...]       Output a skill's full content
+  get <name> --full          Include references and templates
+  get --all                  Output every skill
+  path [name]                Print filesystem path to skill directory
+
+Options:
+  --json                     Output as JSON
+
+The skills command serves bundled skill content that always matches the
+installed CLI version. Agents should use this to get current instructions
+rather than relying on cached copies.
+
+Examples:
+  agent-browser skills
+  agent-browser skills list
+  agent-browser skills get agent-browser
+  agent-browser skills get electron --full
+  agent-browser skills get --all
+  agent-browser skills path agent-browser
+  agent-browser skills list --json
+
+Environment:
+  AGENT_BROWSER_SKILLS_DIR   Override the skills directory path
+"##
+        }
+
         _ => return false,
     };
     println!("{}", help.trim());
@@ -2844,6 +2878,12 @@ Setup:
   dashboard start            Start the observability dashboard
   profiles                   List available Chrome profiles
 
+Skills:
+  skills [list]              List available skills
+  skills get <name> [--full] Get skill content (--full includes references)
+  skills get --all           Get all skill content
+  skills path [name]         Print skill directory path
+
 Snapshot Options:
   -i, --interactive          Only interactive elements
   -c, --compact              Remove empty structural elements