Skip to content

Capture intermediate assistant responses, not just the final one #40

@wylerz

Description

@wylerz

Problem

When Claude Code processes a user turn, the assistant often produces multiple text responses interspersed with tool calls. For example:

User: "Set up the dev environment"
  → Assistant text: "Let me explore the repo and understand the current setup."  ← NOT captured
  → Tool: Agent (Explore)
  → Tool: Bash (check settings)
  → Tool: Bash (check artifacts)
  → Assistant text: "OK, I've got the full picture. Here's the plan..."            ← NOT captured  
  → Tool: Bash (check node version)
  → Tool: Read (package.json)
  → Assistant text: "You don't have Node 24 installed..."                           ← CAPTURED

Only the final assistant text block is captured in the Weave trace output. All intermediate text responses are lost.

Evidence from traces

Looking at Turn 1 of a current session (trace 019d79e2-997a-777f-9684-76a0fad5d484), the turn output is:

"output": {
    "assistant_message": "You don't have Node 24 installed, and the package requires..."
}

This is only the final text block. The earlier responses ("Let me explore the repo...", "OK I've got the full picture...") which provide important context about the assistant's reasoning are not captured.

Root cause

The Stop hook event fires once per turn and provides last_assistant_message — a single string containing only the final text block. This is the only place assistant text is captured (daemon.ts:671):

output: { assistant_message: (payload['last_assistant_message'] as string | undefined) ?? '' }

There is no hook event that fires for each intermediate assistant text block between tool calls.

Proposed approach

Investigate whether Claude Code's transcript JSONL file contains all intermediate assistant messages. The parser (parser.ts) already reads the transcript and collects assistant messages — it aggregates them into turns but only extracts usage/model metadata, not the text content.

Option A: At Stop time, parse the transcript file to extract all assistant text blocks for the current turn (not just last_assistant_message). Change the output format to:

"output": {
    "assistant_messages": ["Let me explore...", "OK, I've got the full picture...", "You don't have Node 24..."],
    "assistant_message": "You don't have Node 24..."  // keep for backwards compat
}

Option B: If individual text blocks can be correlated with their position in the tool-call sequence, capture them as child spans of the turn trace.

Branch

Wyler/capture-intermediate-responses

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions