Skip to content

Register Weave ops for trace call types instead of using raw string op_names #41

@wylerz

Description

@wylerz

Problem

The plugin creates Weave traces using saveCallStart/saveCallEnd with raw string op_name values like "claude_code.session", "claude_code.turn", "claude_code.tool.Bash", etc. These are not registered as Weave ops (operations), which limits what users can do with the traces in the Weave UI.

What Weave ops provide

In Weave, an op is a first-class construct (created via weave.op()) that represents a tracked function. Ops provide:

  • Versioning: Each code/schema change creates a new op version
  • Input/output schemas: Typed signatures visible in the UI
  • Evaluation support: Ops can be used as scorers, evaluated, and compared
  • Rich UI: Op-based calls get dedicated pages, version history, and filtering by op
  • Datasets & feedback: Calls linked to ops can be collected into datasets

Without ops, the traces show up as raw calls with string identifiers. They appear in the trace timeline but lack the structured metadata that enables Weave's more powerful features.

Evidence from traces

Querying all unique op_name values in the project wandb-smle/wyler-cc-history:

claude_code.session
claude_code.turn
claude_code.tool.Agent
claude_code.tool.Bash
claude_code.tool.Glob
claude_code.tool.Grep
claude_code.tool.Read
claude_code.tool.ToolSearch
claude_code.tool.WebFetch
claude_code.tool.WebSearch
claude_code.subagent.Explore
claude_code.permission_request

These are all plain strings — none are registered as Weave ops with the weave:///entity/project/op/name:version URI format.

Current architecture

The plugin is a daemon process that receives async hook events via Unix socket. It doesn't wrap its own functions — it reconstructs a trace tree from external events. This is fundamentally different from the typical @weave.op decorator pattern where you control the function being traced.

The Weave TS SDK exposes multiple APIs:

  • weave.op(fn) — wraps a function (not applicable for cross-process tracing)
  • weaveClient.createCall() / finishCall() — higher-level manual call creation
  • weaveClient.saveCallStart() / saveCallEnd() — batch API (currently used)

Proposed approach

Register ops programmatically at daemon startup for the known call types (session, turn, tool.*, subagent.*, permission_request). Then reference the registered op objects (or their URIs) in saveCallStart instead of bare strings. This should give the Weave UI the structured op metadata it needs while keeping the daemon's async event-driven architecture intact.

Key investigation needed:

  1. Can the TS SDK register an op without wrapping a real function? (e.g., weave.op(() => {}, { name: 'claude_code.session' }))
  2. Does saveCallStart accept an op reference/URI instead of a plain string?
  3. What's the minimum change to get calls recognized as op-backed in the Weave UI?

Branch

Wyler/add-weave-ops

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions