Telemetry: Add agent detection#33675
Conversation
valentinpalkovic
added
maintenance
|
View your CI Pipeline Execution ↗ for commit 0d8afbe
☁️ Nx Cloud last updated this comment at |
|
View your CI Pipeline Execution ↗ for commit 017aced
☁️ Nx Cloud last updated this comment at |
| generatedAt?: number; | ||
| userSince?: number; | ||
| /** Whether this process is likely invoked by an AI agent */ | ||
| isAgent?: boolean; |
There was a problem hiding this comment.
Redundant to agent: { name: 'unknown' } IMO
📝 WalkthroughWalkthroughAdds runtime AI-agent detection and telemetry integration: a new detector inspects explicit environment signals and non‑TTY heuristics to produce agent metadata; telemetry now includes that agent info; tests validate precedence, TTY/CI behavior, and heuristic fallbacks. Changes
Sequence Diagram(s)mermaid CLI->>Detector: provide Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@code/core/src/telemetry/detect-agent.ts`:
- Around line 1-63: The KnownAgentName union includes 'opencode' but
detectExplicitAgent never returns it; update detectExplicitAgent to check for
the OpenCode marker (e.g. env.OPENCODE_CLIENT or the correct OpenCode env var)
and return { name: 'opencode' } when present, referencing the
detectExplicitAgent function and KnownAgentName type; alternatively, if you
prefer not to detect OpenCode, remove 'opencode' from the KnownAgentName union
to avoid an unreachable value.
🧹 Nitpick comments (1)
code/core/src/telemetry/detect-agent.ts (1)
80-83: Comment mismatch with actual CI behavior.
The comment says CI is excluded, but the logic (and tests) apply heuristics even in CI. Consider updating the comment to avoid future confusion.✍️ Suggested comment tweak
- // 2) Behavioral / fingerprint heuristics (exclude CI to reduce false positives) + // 2) Behavioral / fingerprint heuristics (no CI special-casing)
| export type KnownAgentName = | ||
| | 'claude-code' | ||
| | 'gemini-cli' | ||
| | 'cursor' | ||
| | 'codex' | ||
| | 'opencode' | ||
| | 'amp' | ||
| | 'unknown'; | ||
|
|
||
| export type AgentInfo = { | ||
| name: KnownAgentName; | ||
| }; | ||
|
|
||
| export type AgentDetection = { | ||
| isAgent: boolean; | ||
| agent?: AgentInfo; | ||
| }; | ||
|
|
||
| type DetectAgentOptions = { | ||
| stdoutIsTTY: boolean; | ||
| env: NodeJS.ProcessEnv; | ||
| }; | ||
|
|
||
| function detectExplicitAgent(env: NodeJS.ProcessEnv): AgentInfo | undefined { | ||
| // Amp | ||
| if (env.AGENT === 'amp') { | ||
| return { | ||
| name: 'amp', | ||
| }; | ||
| } | ||
|
|
||
| // Claude Code | ||
| if (env.CLAUDECODE) { | ||
| return { | ||
| name: 'claude-code', | ||
| }; | ||
| } | ||
|
|
||
| // Gemini CLI | ||
| if (env.GEMINI_CLI) { | ||
| return { | ||
| name: 'gemini-cli', | ||
| }; | ||
| } | ||
|
|
||
| // OpenAI Codex | ||
| if (env.CODEX_SANDBOX) { | ||
| return { | ||
| name: 'codex', | ||
| }; | ||
| } | ||
|
|
||
| // Cursor Agent (proposed / best-effort; Cursor often sets VSCode env vars too) | ||
| if (env.CURSOR_AGENT) { | ||
| return { | ||
| name: 'cursor', | ||
| }; | ||
| } | ||
|
|
||
| // Generic "AGENT" marker (unknown implementation) | ||
| if (env.AGENT) { | ||
| return { name: 'unknown' }; | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's examine the detect-agent.ts file to confirm the current state
cat -n code/core/src/telemetry/detect-agent.tsRepository: storybookjs/storybook
Length of output: 2555
🏁 Script executed:
# Search for any references to 'opencode' in the codebase
rg -i "opencode" --type ts --type tsx --type js --type jsx -B 2 -A 2Repository: storybookjs/storybook
Length of output: 92
🏁 Script executed:
# Also search for 'OPENCODE' environment variable references
rg "OPENCODE" -B 2 -A 2Repository: storybookjs/storybook
Length of output: 47
🏁 Script executed:
# Let me search more broadly for 'opencode' with simpler rg command
rg -i "opencode"Repository: storybookjs/storybook
Length of output: 119
🏁 Script executed:
# Also check git history to see when 'opencode' was added
git log --oneline --all -- code/core/src/telemetry/detect-agent.ts | head -20Repository: storybookjs/storybook
Length of output: 106
🏁 Script executed:
# Check the actual git diff for that file to understand the context
git log -p --all -- code/core/src/telemetry/detect-agent.ts | head -150Repository: storybookjs/storybook
Length of output: 47
🏁 Script executed:
# Check the full git history of that commit to understand intent
git show 017acedcc0Repository: storybookjs/storybook
Length of output: 50382
🏁 Script executed:
# Search for any TODO or FIXME comments near opencode
rg -i "opencode|TODO|FIXME" code/core/src/telemetry/ -B 2 -A 2Repository: storybookjs/storybook
Length of output: 1327
🌐 Web query:
OpenCode CLI environment variable identification agent
💡 Result:
In OpenCode CLI, the environment variable used to identify the client (agent/client identifier) is:
OPENCODE_CLIENT— “Client identifier (defaults tocli)” (set this if you want OpenCode to tag/identify where requests are coming from). [1]
To select which OpenCode “agent” to run, there isn’t an OPENCODE_* env var for that; you choose it via CLI flags:
opencode --agent <agent>(TUI) /opencode run --agent <agent>(non-interactive). [1]
Add explicit detection for opencode or remove it from KnownAgentName.
KnownAgentName includes 'opencode' (line 6), but detectExplicitAgent never returns it, so telemetry will never emit that value. Either add detection logic using the appropriate environment variable (OpenCode uses OPENCODE_CLIENT as a client identifier) or remove 'opencode' from the union to avoid a dead value.
🤖 Prompt for AI Agents
In `@code/core/src/telemetry/detect-agent.ts` around lines 1 - 63, The
KnownAgentName union includes 'opencode' but detectExplicitAgent never returns
it; update detectExplicitAgent to check for the OpenCode marker (e.g.
env.OPENCODE_CLIENT or the correct OpenCode env var) and return { name:
'opencode' } when present, referencing the detectExplicitAgent function and
KnownAgentName type; alternatively, if you prefer not to detect OpenCode, remove
'opencode' from the KnownAgentName union to avoid an unreachable value.
Package BenchmarksCommit: No significant changes detected, all good. 👏 |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@code/core/src/telemetry/detect-agent.test.ts`:
- Around line 77-79: The test description is misleading: update the test's "it"
description in detect-agent.test.ts to reflect the actual expected return value
of detectAgent (which is AgentInfo | undefined); change the string "returns
isAgent=false when there are no signals" to something like "returns undefined
when there are no signals" so it matches the assertion that detectAgent({
stdoutIsTTY: false, env: {} }) returns undefined and references the detectAgent
function in the test.
🧹 Nitpick comments (1)
code/core/src/telemetry/detect-agent.test.ts (1)
6-58: Good coverage of explicit agent precedence, but consider adding isolatedCLAUDECODEtest.The precedence tests thoroughly validate
AGENT=ampas highest priority and the fallthrough behavior. However, there's no dedicated test forCLAUDECODEdetection in isolation (it's only tested alongside other env vars in lines 20-31).Consider adding a test for completeness:
🔧 Suggested test case for CLAUDECODE isolation
+ it('detects claude-code via CLAUDECODE', () => { + expect(detectAgent({ stdoutIsTTY: true, env: { CLAUDECODE: '1' } })).toEqual({ + name: 'claude-code', + }); + }); + it('detects Gemini CLI via GEMINI_CLI', () => {
| it('returns isAgent=false when there are no signals', () => { | ||
| expect(detectAgent({ stdoutIsTTY: false, env: {} })).toEqual(undefined); | ||
| }); |
There was a problem hiding this comment.
Fix misleading test description.
The test description says "returns isAgent=false" but the assertion expects undefined. The detectAgent function returns AgentInfo | undefined, not an object with an isAgent field.
📝 Suggested fix
- it('returns isAgent=false when there are no signals', () => {
+ it('returns undefined when there are no signals', () => {
expect(detectAgent({ stdoutIsTTY: false, env: {} })).toEqual(undefined);
});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| it('returns isAgent=false when there are no signals', () => { | |
| expect(detectAgent({ stdoutIsTTY: false, env: {} })).toEqual(undefined); | |
| }); | |
| it('returns undefined when there are no signals', () => { | |
| expect(detectAgent({ stdoutIsTTY: false, env: {} })).toEqual(undefined); | |
| }); |
🤖 Prompt for AI Agents
In `@code/core/src/telemetry/detect-agent.test.ts` around lines 77 - 79, The test
description is misleading: update the test's "it" description in
detect-agent.test.ts to reflect the actual expected return value of detectAgent
(which is AgentInfo | undefined); change the string "returns isAgent=false when
there are no signals" to something like "returns undefined when there are no
signals" so it matches the assertion that detectAgent({ stdoutIsTTY: false, env:
{} }) returns undefined and references the detectAgent function in the test.
github-project-automation
bot
moved this from In Progress
to Done
in Core Team Projects
valentinpalkovic
deleted the
valentin/implement-agent-detection-telemetry
branch
…etection-telemetry Telemetry: Add agent detection (cherry picked from commit 945a876)
github-actions
bot
added
the
patch:done
2 participants
Closes #
What I did
Checklist for Contributors
Testing
The changes in this PR are covered in the following automated tests:
Manual testing
STORYBOOK_TELEMETRY_URL=http://localhost:6007/event-log npx storybook initin an agent and observe that the agent is appropriately detected (e.g. claude or amp agent)Documentation
MIGRATION.MD
Checklist for Maintainers
When this PR is ready for testing, make sure to add
ci:normal,ci:mergedorci:dailyGH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found incode/lib/cli-storybook/src/sandbox-templates.tsMake sure this PR contains one of the labels below:
Available labels
bug: Internal changes that fixes incorrect behavior.maintenance: User-facing maintenance tasks.dependencies: Upgrading (sometimes downgrading) dependencies.build: Internal-facing build tooling & test updates. Will not show up in release changelog.cleanup: Minor cleanup style change. Will not show up in release changelog.documentation: Documentation only changes. Will not show up in release changelog.feature request: Introducing a new feature.BREAKING CHANGE: Changes that break compatibility in some way with current major version.other: Changes that don't fit in the above categories.🦋 Canary release
This pull request has been released as version
0.0.0-pr-33675-sha-017acedc. Try it out in a new sandbox by runningnpx storybook@0.0.0-pr-33675-sha-017acedc sandboxor in an existing project withnpx storybook@0.0.0-pr-33675-sha-017acedc upgrade.More information
0.0.0-pr-33675-sha-017acedcvalentin/implement-agent-detection-telemetry017acedc1769526027)To request a new release of this pull request, mention the
@storybookjs/coreteam.core team members can create a new canary release here or locally with
gh workflow run --repo storybookjs/storybook publish.yml --field pr=33675Summary by CodeRabbit
New Features
Tests
✏️ Tip: You can customize this high-level summary in your review settings.