Upgrade Copilot SDK to v1.0.0 and surface premium-request credits on the dashboard#311
Merged
Merged
Conversation
Migrates from copilot-sdk v0.3.0 to v1.0.0. Breaking changes adopted: - SessionEvent.Type is now a method (Type()) instead of a field; updated all switch/comparison sites in models, utils, execution, orchestration. - SessionEvent.Data is the typed discriminator; emitter sites now build typed *Data structs (e.g. *copilot.AssistantMessageData) and rely on Data.Type() for the event type. - ClientOptions.CLIArgs/CLIPath/AutoStart/AutoRestart moved onto Connection (StdioConnection by value). AutoStart/AutoRestart are now managed internally by the SDK. - ProviderConfig.WireApi -> WireAPI (Go naming idiom). - PermissionRequest is now an interface; permission callbacks return rpc.PermissionDecision (e.g. *rpc.PermissionDecisionApproveOnce). - Pointer-ified fields: SessionConfig.Streaming, ResumeSessionConfig.Streaming, SessionShutdownData.TotalPremiumRequests, ShutdownModelMetricRequests.Count and Cost, AssistantUsageData.InputTokens/OutputTokens/Cost, ModelLimits.MaxContextWindowTokens. Test fixtures updated to wrap with copilot.Bool/Float64/Int helpers (no Int64 helper; use utils.Ptr). - streamingPtr() helper preserves pre-v1 semantics: only set Streaming when caller requested it, so tests that compare against nil keep working. - ExtractMessages now filters empty assistant messages — restores the behavior of the old test fixture (zero-value Data used to short-circuit). go test ./... and golangci-lint run are green; end-to-end sanity check against the mock engine (examples/code-explainer) passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a Credits column / KPI / trend chart / CSV column / compare metric that surfaces the premium-request count already collected from the Copilot SDK (UsageStats.PremiumRequests / shutdown.TotalPremiumRequests). Naming: "Credits" in UI, `premiumRequests` in JSON/Go. Tooltip clarifies: "Premium request count reported by the Copilot SDK — not dollars." Backend (internal/webapi): - types.go: RunSummary.PremiumRequests, SummaryResponse.AvgPremiumRequests - store.go: outcomeToSummary aggregates PremiumRequests from per-run SessionDigest.Usage - storage_adapter.go: Summary() aggregates total premium across runs and computes the average - handlers_test.go: fixtures + AvgPremiumRequests assertion Frontend (web/src): - api/client.ts: matching premiumRequests / avgPremiumRequests fields - lib/format.ts: formatCredits (integer with thousands separators) - components/RunsTable.tsx: Credits column with header + cell tooltip - components/KPICards.tsx: Avg Credits card (grid → 7 cols) with tooltip - components/RunDetail.tsx: Credits in run stat grid - components/CompareView.tsx: Credits metric row (lower-is-better) - components/TrendsPage.tsx: Credits per Run trend chart - lib/export.ts: Credits column in CSV export Tests: - Go: `go test ./...` green - Playwright: 53/53 chromium tests pass; updated dashboard mock data and KPI assertions to expect 7 cards including Avg Credits, plus a Credits column header check on the runs table - Screenshots regenerated to reflect new column / card / chart Docs: - site/src/content/docs/guides/dashboard-explore.mdx: documents Credits in the overview, comparison, and trends sections useSSE was left unchanged: the run_complete SSE event does not currently carry premiumRequests (only Status/DurationMs/Details), so a live update field would be a no-op until the backend payload is widened. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR (1) upgrades the Go Copilot SDK dependency to github.com/github/copilot-sdk/go v1.0.0 and adapts the codebase to its breaking API changes, and (2) plumbs Copilot “premium request” counts through the web API and dashboard UI as “Credits”.
Changes:
- Upgrade Copilot SDK v0.3.0 → v1.0.0 and refactor event typing, permission decisions, connection/client options, and pointer-typed payload fields.
- Add
premiumRequests/avgPremiumRequeststo the web API, aggregation, and Go tests; surface “Credits” across the dashboard (table, KPI card, run detail, compare, trends) plus CSV export and docs. - Update web e2e fixtures/mocks and regenerate dist/docs assets to reflect the new UI elements.
Show a summary per file
| File | Description |
|---|---|
| web/src/lib/format.ts | Add formatCredits for rendering premium-request counts as UI “Credits”. |
| web/src/lib/export.ts | Add Credits column to CSV export. |
| web/src/components/TrendsPage.tsx | Add “Credits per Run” trend series/chart. |
| web/src/components/RunsTable.tsx | Add Credits column with tooltip in runs table. |
| web/src/components/RunDetail.tsx | Add Credits stat and tooltip support in stat cards. |
| web/src/components/KPICards.tsx | Add “Avg Credits” KPI card and adjust grid sizing/tooltips. |
| web/src/components/CompareView.tsx | Add Credits metric to comparison grid with delta behavior. |
| web/src/api/client.ts | Extend client types with premiumRequests and avgPremiumRequests. |
| web/e2e/helpers/api-mock.ts | Update mocked /api/summary payload shape. |
| web/e2e/fixtures/mock-data.ts | Add credits fields to summary and run fixtures. |
| web/e2e/dashboard.spec.ts | Update KPI-card expectations and add Credits column test. |
| web/dist/index.html | Refresh built asset references for updated web bundle. |
| site/src/content/docs/guides/dashboard-explore.mdx | Document Credits in overview/table/compare/trends sections. |
| internal/webapi/types.go | Add PremiumRequests to run summary and AvgPremiumRequests to summary response. |
| internal/webapi/store.go | Aggregate/store premium requests in outcomeToSummary; update event type accessors. |
| internal/webapi/storage_adapter.go | Aggregate total/avg premium requests in Summary(). |
| internal/webapi/handlers_test.go | Update mock summary aggregation and assertions for avg premium requests. |
| internal/webapi/additional_test.go | Update Copilot SDK event fixture construction for v1 typing/discriminator behavior. |
| internal/utils/logging.go | Switch to event.Type() for v1 SDK event typing. |
| internal/utils/logging_test.go | Update event fixture construction for v1 SDK event discriminator changes. |
| internal/utils/copilot_log_iterator_test.go | Update tests to use Type() accessor. |
| internal/orchestration/runner_orchestration_test.go | Update transcript event assertions/fixtures for Type() behavior. |
| internal/models/events.go | Update transcript marshal/unmarshal to derive types via Type() and include raw discriminator when needed. |
| internal/models/events_test.go | Update tests for transcript type derivation via Type(). |
| internal/graders/prompt_grader_test.go | Update client option usage to match v1 SDK options. |
| internal/graders/inline_script_grader_test.go | Update event fixture construction for v1 SDK typing. |
| internal/execution/session_usage_collector.go | Handle pointer-typed shutdown premium/request fields and use Type() discriminator. |
| internal/execution/session_usage_collector_test.go | Update usage tests for pointer-typed SDK fields and revised event construction. |
| internal/execution/session_events_collector.go | Switch to Type() and adjust session error detection. |
| internal/execution/session_events_collector_test.go | Update event fixtures for v1 SDK typing/discriminator behavior. |
| internal/execution/sdkclient.go | Refactor to v1 flattened connection options (Connection: StdioConnection{Path, Args}). |
| internal/execution/sdkclient_test.go | Update assertions to validate Connection contents instead of deprecated fields. |
| internal/execution/engine.go | Adjust message extraction to use Type() and ignore empty assistant messages. |
| internal/execution/engine_response_test.go | Update event fixtures for message extraction tests under v1 typing. |
| internal/execution/copilot.go | Update provider config field names, permission decision return type, and streaming pointer semantics. |
| internal/execution/copilot_test.go | Update permission decision tests and provider config expectations for v1 changes. |
| internal/execution/copilot_engine_test.go | Update streaming assertions and connection args assertions for v1 options. |
| internal/copilotevents/events.go | Update Raw event data helper to include explicit event type discriminator. |
| go.mod | Bump Copilot SDK module to v1.0.0. |
| go.sum | Update checksums for Copilot SDK v1.0.0. |
| cmd/waza/newtask/converters.go | Switch session event discriminator usage to Type(). |
| cmd/waza/cmd_run_suggest.go | Switch session event discriminator usage to Type(). |
| cmd/waza/cmd_run_suggest_test.go | Update event fixtures for v1 SDK typing/discriminator behavior. |
| cmd/waza/cmd_models.go | Handle pointer-typed model limits (MaxContextWindowTokens). |
| cmd/waza/cmd_models_test.go | Update fixtures for pointer-typed model limits. |
Copilot's findings
- Files reviewed: 43/49 changed files
- Comments generated: 2
Resolve conflicts from #310 (cost-source pricing pipeline): - Combine Credits (premiumRequests) + CostSource fields on RunSummary/SummaryResponse - Aggregate both totalPremium and pricing.Compute in Summary() - StatCard/Card components support both title (Credits tooltip) and labelExtra (Cost InfoTooltip) - Strengthen Playwright assertions to check actual fixture values
- streamingPtr now always returns a non-nil *bool so SDK behavior is stable regardless of any future change to the SDK Streaming default - Credits stats switch from native title attribute on <div> to keyboard-reachable InfoTooltip (matches Cost stat pattern in KPICards/RunDetail/CompareView) - Drop redundant 'title?' prop from Card/StatCard/MetricCard - Update sessionConfigMatcher to treat *bool(false) as equivalent to nil for Streaming when expected fixture omits it
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #309
Two parts, landed in order on the same branch.
Part 1 — Copilot SDK v0.3.0 → v1.0.0
Bumps
github.com/github/copilot-sdk/goand adapts to the v1.0.0 breaking changes:SessionEvent.Typeis now a method (Type()); switches on the event discriminator updated accordingly.StdioConnectionpassed by value (not pointer) — the v1 SDK panics on*StdioConnection.ClientOptionsflattened: oldCLIArgs/CLIPath/AutoStart/AutoRestartreplaced with a singleConnection: StdioConnection{Path, Args}.ProviderConfig.WireApi→WireAPI.PermissionRequestis now an interface; callbacks return typed decisions like&rpc.PermissionDecisionApproveOnce{}.Streaming,TotalPremiumRequests,ShutdownModelMetricRequests.Count/Cost,AssistantUsageData.InputTokens/OutputTokens/Cost,ModelLimits.MaxContextWindowTokens); helpers added viacopilot.Bool/copilot.Float64/copilot.String/copilot.Int.RawSessionEventDatanow requires explicitEventTypeto preserve the discriminator.CLI bundler kept at 1.0.49.
make testandmake lintgreen; mock-engine eval verified end-to-end.(Already on the branch as commit
3b8bab51.)Part 2 — Surface premium-request credits on the dashboard
The Copilot SDK already reports
TotalPremiumRequestsand per-modelRequests.Count, andinternal/execution/session_usage_collector.goalready stores them inUsageStats.PremiumRequests/ModelUsage.RequestCount. This part wires them through the webapi into the dashboard.Naming decision: "Credits" in the UI,
premiumRequests/avgPremiumRequestsin JSON and Go. Tooltip clarifies "Premium request count reported by the Copilot SDK — not dollars."Backend (
internal/webapi/)types.go:RunSummary.PremiumRequests,SummaryResponse.AvgPremiumRequestsstore.go:outcomeToSummaryaggregatesr.SessionDigest.Usage.PremiumRequestsacross runsstorage_adapter.go:Summary()aggregates total premium and computes the averagehandlers_test.go: fixtures + assertion thatAvgPremiumRequests == 4.5for the test dataFrontend (
web/src/)api/client.ts: matchingpremiumRequests/avgPremiumRequestsfieldslib/format.ts:formatCredits(integer with thousands separators, no$)components/RunsTable.tsx: Credits column between Tokens and Cost, with header + cell tooltipcomponents/KPICards.tsx: Avg Credits card withCreditCardicon and tooltip; grid → 7 columnscomponents/RunDetail.tsx: Credits in the run stat grid (5 cols)components/CompareView.tsx: Credits metric row (lower-is-better)components/TrendsPage.tsx: "Credits per Run" trend chartlib/export.ts: Credits column in CSV exportTests
go test ./...— all packages greenweb/e2e/fixtures/mock-data.ts,web/e2e/helpers/api-mock.ts) updated to includepremiumRequests/avgPremiumRequests; existing "KPI cards" test renamed to expect 7 cards and now asserts "Avg Credits"; new "Credits column renders with values" test addedDocs
site/src/content/docs/guides/dashboard-explore.mdx: documents Credits in the overview KPI list, sortable columns, comparison metrics, and the trends section (now 5 charts)Intentionally skipped
web/src/hooks/useSSE.ts: therun_completeSSE event only carriesStatus/DurationMs/Detailstoday — there is nopremiumRequeststo read live. Per the spec, leaving the SSE wiring for a follow-up once the backend payload is widened.Coordination note
Companion issue #308 is still open and may overlap on
internal/webapi/types.go,web/src/components/RunsTable.tsx,web/src/components/KPICards.tsx, andweb/src/components/RunDetail.tsx. Reviewers / whoever lands second should expect to reconcile naming and grid-column counts in those files.