[token-consumption] Daily AIC Consumption Report - 2026-06-10

[github-actions]

Executive Summary

For the trailing 24h window, gh-aw emitted 15,577 gen_ai spans across 53+ workflows in the github/gh-aw Sentry project, exported simultaneously to Sentry and Grafana Cloud Tempo. Telemetry pipelines on both backends are healthy — spans, resource attributes, run status, and OTLP status.code are all present and correct.

However, AI Credits (AIC) are NOT queryable in either backend. No span in the window carries a usable gh-aw.aic value:

• Sentry: has:gh-aw.aic returns 0 results; the field exists in the EAP schema but is typed as a string, so numeric aggregation (sum/avg/p95) is rejected outright.
• Grafana/Tempo: gh-aw.aic is absent from the span attribute index entirely; { span."gh-aw.aic" != "" } matched 0 of 102 blocks.

This is an observability gap, not zero usage — actual AIC consumption is unknown. Total AIC, per-workflow AIC, avg, and P95 cannot be computed until the instrumentation gap is fixed. Companion errors and logs datasets were both explicitly checked and returned 0 events.

Key Metrics

Metric	Value
Events analyzed (Sentry gen_ai spans, canonical)	15,577
Events with AIC data	0 (gap)
Events with AIC data (Sentry)	0
Events with AIC data (Grafana)	0
Total AIC	Unknown — not queryable (do not read as 0)
Unique workflows (attributed)	53+
Avg AIC/event	N/A — no queryable AIC
P95 AIC/event	N/A — no queryable AIC
Events missing workflow attribution	11,406 (gh-aw.workflow.name = null on child spans)
Errors dataset (24h)	0
Logs dataset (24h)	0

Top 10 Workflows by AIC Consumption

⚠️ AIC per workflow is unavailable (no queryable gh-aw.aic). The table below ranks by gen_ai span activity as a proxy for relative LLM workload — it is not AIC and must not be interpreted as credit spend.

Workflow	gen_ai Spans	Total AIC	Avg AIC/Event
Smoke CI	660	Unknown	N/A
PR Code Quality Reviewer	312	Unknown	N/A
Design Decision Gate 🏗️	298	Unknown	N/A
Test Quality Sentinel	264	Unknown	N/A
PR Sous Chef	236	Unknown	N/A
Matt Pocock Skills Reviewer	231	Unknown	N/A
AI Moderator	154	Unknown	N/A
Issue Monster	140	Unknown	N/A
Auto-Triage Issues	108	Unknown	N/A
Contribution Check	60	Unknown	N/A

gen_ai activity by provider (gen_ai.system)

Provider	Spans
github	5,909
openai	4,478
github_models	2,891
mcp	1,280
anthropic	935
antigravity	30
google_vertex_ai	30
pi	24
Total	15,577

Grafana AIC Findings

Grafana AIC: NOT queryable (top-level gap — details below).

• Queryable? No. The gh-aw.aic attribute does not appear in tempo_get-attribute-names for the span scope. The closest consumption-related attribute present is gh-aw.action_minutes (numeric), but there is no AIC attribute.
• Exact issue: AIC attributes are not emitted onto the indexed run-level span, so Tempo has nothing to index or aggregate. Even where AIC would live (the gh-aw.activation.conclusion span), it is missing.
• Trace evidence: Full inspection of trace 7da7a19b94fdf888f4ab794f8766461 (run 27298535898, datasource grafanacloud-traces). The gh-aw.activation.conclusion span carries gh-aw.action_minutes=0.225, gh-aw.run.status=success, gh-aw.error_count=0, status.code=STATUS_CODE_OK, resource service.version=2.1.168 — but zero gh-aw.aic attribute.
• Query evidence: { span."gh-aw.aic" != "" } over 2026-06-09T18:00Z → 2026-06-10T19:00Z inspected 102 blocks / 105 jobs and matched 0 traces.

Data Quality and Gaps

• Events missing workflow identifiers: 11,406 gen_ai child spans have gh-aw.workflow.name = null. Workflow attribution is present on the resource and on the run-level parent spans (gh-aw.activation.* / gh-aw.agent.setup) but is not propagated onto per-LLM-call gen_ai child spans, so per-call rows cannot be attributed without a trace join.
• Events missing AIC attributes: 100% — no span in the 24h window (either backend) exposes a numeric gh-aw.aic.
• Root cause (emit-side, actions/setup/js/send_otlp_span.cjs): gh-aw.aic is pushed only when jobEmitsOwnTokenUsage is true and a numeric aiCredits resolves from one of GH_AW_AIC (env) → agent_usage.json (file) → engine metrics. In practice none of these resolve, so the attribute is omitted entirely. Separately, even when emitted, Sentry has inferred the field as string-typed, blocking numeric aggregation.
• Fallback fields used: Workflow name via gh-aw.workflow.name (resource + parent spans); run id via gh-aw.run.id; activity proxy via gen_ai span counts.
• Sentry-specific caveat: gh-aw.aic is a string-type EAP field → sum()/avg()/p95() return HTTP 400 (is a string type field). Spans are exported to both backends, so Sentry/Grafana AIC counts must not be added.
• Grafana-specific caveat: gh-aw.aic not indexed; only string-valued or numeric attributes actually emitted (e.g. gh-aw.action_minutes) are queryable. Grafana base URL/trace-explore deep links may be redacted by safe-outputs; datasource UID grafanacloud-traces + trace IDs are provided instead.
• Confirmed-healthy (not gaps): OTLP status.code (STATUS_CODE_OK), gh-aw.run.status (success), and resource service.version (2.1.168) are all correctly populated — these earlier-suspected attributes are fine; the gap is isolated to gh-aw.aic.

Recommendations

1. Fix the AIC emit gap (highest priority). In actions/setup/js/send_otlp_span.cjs, ensure gh-aw.aic is written onto the run-level gh-aw.activation.conclusion span whenever token usage is available. Verify the GH_AW_AIC env → agent_usage.json → engine-metrics resolution chain actually populates a value (the firewall proxy writing ai_credits=0 currently masks real usage). Without this, no daily AIC report can produce real numbers.
2. Emit gh-aw.aic as a numeric OTLP attribute, not a string. The Sentry EAP field is inferred as string, which blocks all numeric aggregation. Emit it as a double/int and re-check that Tempo indexes it so { span."gh-aw.aic" > 0 } becomes queryable.
3. Propagate workflow attribution to gen_ai child spans. Add gh-aw.workflow.name/gh-aw.run.id to per-LLM-call spans (or document the trace-join path) so the 11,406 unattributed child spans can be rolled up per workflow.
4. Once AIC is numeric, prioritize the heaviest LLM-activity workflows (Smoke CI, PR Code Quality Reviewer, Design Decision Gate, Test Quality Sentinel, PR Sous Chef) for credit-reduction review (model right-sizing, prompt/context trimming, caching) — these are the most likely top AIC consumers based on span volume.

References

• Sentry — gen_ai spans (24h): https://github.sentry.io/explore/traces/?query=span.op:gen_ai*&project=4511347087179777&statsPeriod=24h
• Sentry — empty has:gh-aw.aic (gap evidence): https://github.sentry.io/explore/traces/?query=has:gh-aw.aic&project=4511347087179777&statsPeriod=24h
• Grafana — datasource grafanacloud-traces, verified trace 7da7a19b94fdf888f4ab794f8766461 (no gh-aw.aic)
• Workflow run: https://github.com/github/gh-aw/actions/runs/27298535898

Generated by 📊 Daily AIC Consumption Report (Sentry + Grafana OTel) · 137.9 AIC · ⌖ 31.3 AIC · ⊞ 7.4K · ◷

• expires on Jun 11, 2026, 10:56 AM UTC-08:00