Traces in Trackio by abidlabs · Pull Request #518 · gradio-app/trackio

abidlabs · 2026-04-18T18:17:49Z

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

gradio-pr-bot · 2026-04-18T18:18:16Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`trackio`	`minor`

Traces in Trackio

‼️ Changeset not approved. Ensure the version bump is appropriate for all packages before approving.

Maintainers can approve the changeset by checking this checkbox.

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

gradio-pr-bot · 2026-04-18T18:18:17Z

🪼 branch checks and previews

•	Name	Status	URL
🦄	Changes	detected!	Details

HuggingFaceDocBuilderDev · 2026-04-18T18:18:42Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview

Install Trackio from this PR (includes built frontend)

pip install "https://huggingface.co/buckets/trackio/trackio-wheels/resolve/aa0f89bd16f476ebf7b7ea8e56544a89e3f148f5/trackio-0.24.2-py3-none-any.whl"

HuggingFaceDocBuilderDev · 2026-04-18T18:19:52Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

sergiopaniego

Thanks for the proposal!!
One question: would it support rendering images?
possible use case: you're training with GRPO + an env (e.g., OpenEnv), and the env returns a list of images (e.g., a browser env returning screenshots). It'd be nice to render them inline with the messages

qgallouedec · 2026-04-20T16:51:30Z

Very cool! looking forward to integrate this!

abidlabs · 2026-04-20T20:26:21Z

Thanks for the proposal!!
One question: would it support rendering images?
possible use case: you're training with GRPO + an env (e.g., OpenEnv), and the env returns a list of images (e.g., a browser env returning screenshots). It'd be nice to render them inline with the messages

yep can do! We already support images in tables, so we should be able to do the same here

# Conflicts: # trackio/frontend/src/App.svelte

abidlabs · 2026-04-20T21:05:31Z

Ok based on great feedback from everyone, have updated this PR.

Here's a basic example: python examples/traces/basic-trace.py

Screen.Recording.2026-04-20.at.2.00.32.PM.mov

(I've removed many of the earliers to make the UI less opinionated, thanks @adithya-s-k for the suggestion)

A more complex example including images and tool calls: python examples/traces/complex-trace.py

Screen.Recording.2026-04-20.at.2.03.29.PM.mov

cc @sergiopaniego @AmineDiro

And a potential example of how to use it with TRL: python examples/traces/trl-trace-integration.py (cc @qgallouedec)

Any other suggestions/improvements are welcome!

Copilot

Pull request overview

Adds first-class “trace” logging and a UI for browsing conversational/agent traces in Trackio, integrating with the existing metrics/log storage and dashboard routing.

Changes:

Introduce Trace payload type that serializes nested Trackio media inside messages/metadata.
Add SQLiteStorage.get_traces() + server API /get_traces to extract/search/sort traces from metric logs.
Add a new Svelte “Traces” page and navigation wiring (dynamic + static modes).

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
trackio/trace.py	New `Trace` object with nested media serialization.
trackio/run.py	Logs `Trace` instances and recursively queues nested media uploads.
trackio/sqlite_storage.py	Extracts trace records from metric logs; supports search/sort/limit/offset.
trackio/server.py	Exposes `get_traces` via the server API registry.
trackio/frontend/src/pages/Traces.svelte	New UI page to list/search/sort and expand trace conversations.
trackio/frontend/src/lib/api.js	Adds `getTraces()` client wrapper (static + server modes).
trackio/frontend/src/lib/staticApi.js	Implements static-mode trace extraction/search/sort from exported logs.
trackio/frontend/src/lib/router.js	Adds `/traces` route mapping.
trackio/frontend/src/components/Navbar.svelte	Adds “Traces” nav link.
trackio/frontend/src/App.svelte	Renders the Traces page and includes it in sidebar-enabled pages.
trackio/init.py	Exports `Trace` from the top-level package API.
tests/unit/test_trace.py	Unit coverage for trace serialization + storage search/sort.
tests/e2e-local/test_trace_e2e.py	E2E round-trip test for logging and reading traces.
examples/traces/basic-trace.py	Example: minimal trace logging.
examples/traces/complex-trace.py	Example: rich trace with tool calls + images.
examples/traces/trl-trace-integration.py	Example: TRL callback logging traces during training.
.changeset/easy-apes-hammer.md	Changeset marking a minor feature release.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        const trace = {
+          id: `${normalizeRun(run).id || normalizeRun(run).name || "run"}:${log.step}:${key}${traceIndex !== null ? `:${traceIndex}` : ""}`,
+          key,
+          index: traceIndex,
+          run: normalizeRun(run).name,
+          run_id: normalizeRun(run).id,
+          step: log.step,


+                elif isinstance(value, Trace):
+                    metrics[key] = value._to_dict(
+                        project=self.project, run=self.name, step=step
+                    )
+                    self._scan_and_queue_media_uploads(metrics[key], step)


abidlabs · 2026-04-20T21:30:41Z

+        offset: int = 0,
+        run_id: str | None = None,
+    ) -> list[dict[str, Any]]:
+        logs = SQLiteStorage.get_logs(project, run, max_points=None, run_id=run_id)


Good point — this is a real concern for very large runs, but filtering server-side is non-trivial because trace payloads are stored inline inside metric rows (no separate trace index), so SQLite has no cheap way to skip non-trace rows without a schema change. The input normalization is addressed in the follow-up commit; I'd like to defer the scan-reduction work to a dedicated PR that introduces a lightweight trace index table so pagination/sort can be pushed down to SQL.

+        if offset > 0:
+            traces = traces[offset:]
+        if limit is not None:
+            traces = traces[:limit]
+


+def get_traces(
+    project: str,
+    run: str | None = None,
+    run_id: str | None = None,
+    search: str | None = None,
+    sort: str | None = None,
+    limit: int | None = None,
+    offset: int | None = 0,
+) -> list[dict[str, Any]]:
+    return SQLiteStorage.get_traces(
+        project,
+        run,
+        search=search,
+        sort=sort,
+        limit=limit,
+        offset=offset or 0,
+        run_id=run_id,
+    )


+          {#each visibleTraces as trace}
+            <tr class="trace-row" onclick={() => toggleTrace(trace.id)}>
+              <td class="trace-id-cell">
+                <span class="trace-id">{trace.id}</span>
+              </td>
+              <td class="request-cell">


+  async function loadTraces() {
+    if (!project || selectedRuns.length === 0) {
+      traces = [];
+      expandedTraceId = null;
+      return;
+    }
+
+    loading = true;
+    try {
+      const batches = await Promise.all(
+        selectedRuns.map(async (run) => {
+          const runTraces = await getTraces(project, run);
+          return runTraces.map((trace) => normalizeTrace(trace, run.name));
+        }),
+      );
+      traces = batches.flat();
+      if (!traces.find((trace) => trace.id === expandedTraceId)) {
+        expandedTraceId = null;
+      }
+    } catch (error) {
+      console.error("Failed to load traces:", error);
+      traces = [];
+    } finally {
+      loading = false;
+    }


- Cache normalizeRun result in staticApi getTraces - Normalize step (None -> _next_step) before queuing trace/table media - Validate offset/limit/sort inputs in server.get_traces and storage - Make trace rows keyboard-accessible (role/tabindex/keydown) - Guard Traces.svelte loadTraces against stale responses via request id

znation

Overall looks good (I only skimmed, Claude reviewed more thoroughly). Left some optional comments for issues that Claude found.

znation · 2026-04-20T22:14:16Z

+        normalized_offset = max(0, int(offset)) if offset is not None else 0
+    except (TypeError, ValueError):
+        normalized_offset = 0
+    normalized_limit: int | None


Double-sanitization of offset/limit

trackio/server.py:843-856 normalizes offset and limit, then passes them to trackio/sqlite_storage.py:1968-1974 which normalizes them again with identical
logic. One layer should own this.

Recommendation: Remove sanitization from sqlite_storage.py and let the API layer (server.py) be the sole validator. The storage layer can trust its internal
callers.

znation · 2026-04-20T22:15:00Z

+                    self._queue_upload(absolute_path, step)
+                return
+            for nested in value.values():
+                self._scan_and_queue_media_uploads(nested, step)


Recursive _scan_and_queue_media_uploads has no depth limit

trackio/run.py:767-786 — The refactored _scan_and_queue_media_uploads now recurses into arbitrary dicts/lists. A deeply nested trace payload (or even an
accidental circular reference via a custom dict) could blow the stack. The old version was bounded to exactly 2 levels of nesting (table rows → values →
list items).

Recommendation: Add a max_depth parameter (e.g., 10) and stop recursing beyond it. This matches the practical ceiling for trace messages.

znation · 2026-04-20T22:16:01Z

+                        continue
+
+                    trace_index = index if isinstance(value, list) else None
+                    trace_id_parts = [run_id or run or "run", str(step), key]


Trace ID collisions across runs

trackio/sqlite_storage.py:1934-1937 — Trace IDs are constructed as run_id_or_name:step:key[:index]. When run_id is None and run is None, the fallback is the
string "run". If two different runs are both queried with run=None, run_id=None, they'll produce identical trace IDs, causing collisions in the frontend (the
expand/collapse toggle uses trace.id).

The frontend in Traces.svelte:57-61 fetches traces for multiple selectedRuns, flattening them into one array. If two runs share a step number + key, the IDs
will collide.

Recommendation: Include the actual run name or run ID in the trace ID unconditionally (the caller always has it from the selectedRuns list), or generate a
unique ID (e.g., hash).

znation · 2026-04-20T22:16:47Z

+    }
+  }
+
+  let visibleTraces = $derived.by(() => {


Client-side search duplicates server-side search

Traces.svelte:79-96 — visibleTraces does full client-side filtering/sorting on the loaded traces. But getTraces in api.js also passes search/sort options to
the server. Currently loadTraces() at line 59 calls getTraces(project, run) with no options — so the server-side search/sort/pagination is never used from
the UI. The toolbar controls only drive the client-side $derived block.

This means the server endpoint accepts search/sort/limit/offset parameters that the frontend never sends. The two code paths (server-side in
sqlite_storage.py and client-side in Traces.svelte) are duplicated logic that can drift.

Recommendation: Either remove the unused server-side filtering (YAGNI) or wire it up in the frontend and remove the client-side duplicate. Given Issue 1,
moving filtering server-side would also be the path to fixing the performance problem.

znation · 2026-04-20T22:18:13Z

+      <p>Try a different search query or model filter.</p>
+    </div>
+  {:else}
+    <div class="toolbar">


Toolbar duplication in Traces.svelte

Traces.svelte:175-189 and Traces.svelte:198-213 — The toolbar markup (search input, sort dropdown, count display) is duplicated verbatim in both the "no
matching traces" and "has traces" branches. If you change one, you'll need to change the other.

Recommendation: Extract the toolbar into its own {#snippet} or move it above the conditional so it renders once regardless of whether traces match.

abidlabs · 2026-04-22T16:33:40Z

Thanks so much for the review @znation! Cleaned up the frontend based on your comments, will merge this in once CI is green

Add proposal: trackio.Trace for GRPO rollout logging

018f84d

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

abidlabs requested review from NouamaneTazi, Saba9, qgallouedec, sergiopaniego and znation April 18, 2026 18:51

abidlabs and others added 2 commits April 18, 2026 11:52

changes

409d27d

add changeset

149112d

sergiopaniego reviewed Apr 20, 2026

View reviewed changes

abidlabs changed the title ~~Proposal: trackio.Trace for GRPO rollout logging~~ Traces in Trackio Apr 20, 2026

abidlabs and others added 2 commits April 20, 2026 13:21

Merge branch 'main' into trace-proposal

3f6e1ee

add changeset

4a9af83

abidlabs and others added 3 commits April 20, 2026 13:39

changes

7e58229

Merge remote-tracking branch 'origin/main' into trace-proposal

1fa956f

# Conflicts: # trackio/frontend/src/App.svelte

Merge branch 'main' into trace-proposal

e69690a

changes

9438de5

abidlabs marked this pull request as ready for review April 20, 2026 21:05

abidlabs added 3 commits April 20, 2026 14:09

changes

500b773

changes

8c0cf12

changes

eb0932b

abidlabs requested a review from Copilot April 20, 2026 21:21

Copilot started reviewing on behalf of abidlabs April 20, 2026 21:22 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

abidlabs added 2 commits April 20, 2026 14:31

Update nav-link count in UI tests for new Traces page

f52ef29

znation approved these changes Apr 20, 2026

View reviewed changes

abidlabs and others added 2 commits April 22, 2026 12:21

Merge branch 'main' into trace-proposal

5e6471e

changes

1e5003f

abidlabs added 2 commits April 22, 2026 09:33

changes

77e9c0a

changes

c1e8359

abidlabs enabled auto-merge (squash) April 22, 2026 17:10

changes

aa0f89b

abidlabs merged commit e7ed176 into main Apr 22, 2026
8 of 9 checks passed

gradio-pr-bot mentioned this pull request Apr 22, 2026

chore: update versions #532

Merged

Conversation

abidlabs commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gradio-pr-bot commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦄 change detected

This Pull Request includes changes to the following packages.

Something isn't right?

Uh oh!

gradio-pr-bot commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🪼 branch checks and previews

Uh oh!

HuggingFaceDocBuilderDev commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🪼 branch checks and previews

Uh oh!

HuggingFaceDocBuilderDev commented Apr 18, 2026

Uh oh!

sergiopaniego left a comment

Choose a reason for hiding this comment

Uh oh!

qgallouedec commented Apr 20, 2026

Uh oh!

abidlabs commented Apr 20, 2026

Uh oh!

abidlabs commented Apr 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

abidlabs Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

znation left a comment

Choose a reason for hiding this comment

Uh oh!

znation Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

znation Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

znation Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

znation Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

znation Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

abidlabs commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

abidlabs commented Apr 18, 2026 •

edited

Loading

gradio-pr-bot commented Apr 18, 2026 •

edited

Loading

gradio-pr-bot commented Apr 18, 2026 •

edited

Loading

HuggingFaceDocBuilderDev commented Apr 18, 2026 •

edited

Loading