Skip to content

Commit 972d591

Browse files
dtmeadowsfelixfbeckerstainless-app[bot]mikecluck
authored
feat(tools): add AbortSignal support for tool runner (#848)
## Summary - Adds `AbortSignal` support to `BetaToolRunner` for cancelling API calls and tool execution - Introduces `BetaToolRunContext` type passed to `tool.run()` with `signal` and `toolUseBlock` - Adds `setRequestOptions()` method for updating signal/headers after runner creation ## Context Picks up the design from #877 (which was approved but stalled), adapted to the current codebase. Multiple users have requested the ability to cancel tool runner operations. The signal flows through: 1. Constructor options or `setRequestOptions()` → stored in `#options` 2. `#options` passed to API calls (`messages.create`/`messages.stream`) 3. `generateToolResponse(signal?)` → `tool.run(input, { toolUseBlock, signal })` Tools can check `context?.signal?.aborted` or pass the signal to downstream operations like `fetch()`. ## Test plan - [x] `./scripts/build` passes - [x] `./scripts/test -- tests/lib/tools/ToolRunner.test.ts` — 5 new tests - [x] Signal + toolUseBlock passed to tool.run via constructor options - [x] Undefined signal when none provided - [x] `setRequestOptions()` with direct object - [x] `setRequestOptions()` with mutator function - [x] Documentation updated in helpers.md --------- Co-authored-by: Felix Becker <felix@anthropic.com> Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com> Co-authored-by: Mike Cluck <mcluck90@gmail.com>
1 parent 0b536ae commit 972d591

File tree

6 files changed

+457
-15
lines changed

6 files changed

+457
-15
lines changed

helpers.md

Lines changed: 60 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,48 @@ console.log(await runner);
256256
See [`examples/tools-helpers-advanced-streaming.ts`](examples/tools-helpers-advanced-streaming.ts) for a more
257257
in-depth example.
258258
259+
#### Cancellation
260+
261+
The `BetaToolRunner` supports cancellation via `AbortSignal`. The signal is passed to both API calls and tool `run` methods via the `BetaToolRunContext`.
262+
263+
```ts
264+
const controller = new AbortController();
265+
266+
const runner = anthropic.beta.messages.toolRunner(
267+
{
268+
model: 'claude-sonnet-4-5-20250929',
269+
max_tokens: 1000,
270+
messages: [{ role: 'user', content: 'Do a long task' }],
271+
tools: [
272+
betaZodTool({
273+
name: 'long_task',
274+
inputSchema: z.object({ query: z.string() }),
275+
description: 'A long-running task',
276+
run: async (input, context) => {
277+
// Throws AbortError if already cancelled before run() was called
278+
context?.signal?.throwIfAborted();
279+
// Pass the signal to downstream operations for mid-flight cancellation
280+
const result = await fetch(url, { signal: context?.signal });
281+
return result.text();
282+
},
283+
}),
284+
],
285+
},
286+
{ signal: controller.signal },
287+
);
288+
289+
// Cancel after 5 seconds
290+
setTimeout(() => controller.abort(), 5000);
291+
292+
const finalMessage = await runner;
293+
```
294+
295+
You can also set or update the signal after creating the runner:
296+
297+
```ts
298+
runner.setRequestOptions({ signal: controller.signal });
299+
```
300+
259301
### `betaZodTool`
260302
261303
Zod schemas can be used to define the input schema for your tools:
@@ -386,9 +428,25 @@ runner.pushMessages(
386428
);
387429
```
388430
389-
#### `BetaToolRunner.generateToolResponse()`
431+
#### `BetaToolRunner.setRequestOptions()`
432+
433+
Updates the request options (e.g., headers, abort signal) for future API calls and tool executions.
434+
435+
```ts
436+
// Direct options update
437+
const controller = new AbortController();
438+
runner.setRequestOptions({ signal: controller.signal });
439+
440+
// Using mutator function to preserve existing options
441+
runner.setRequestOptions((prev) => ({
442+
...prev,
443+
signal: controller.signal,
444+
}));
445+
```
446+
447+
#### `BetaToolRunner.generateToolResponse(signal?)`
390448
391-
Gets the tool response for the last assistant message (if any tools need to be executed).
449+
Gets the tool response for the last assistant message (if any tools need to be executed). Accepts an optional `AbortSignal` parameter that will be passed to tool `run` methods; defaults to the signal from request options.
392450
393451
```ts
394452
for await (const message of runner) {

src/helpers/beta/json-schema.ts

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { FromSchema, JSONSchema } from 'json-schema-to-ts';
2-
import { Promisable, BetaRunnableTool } from '../../lib/tools/BetaRunnableTool';
2+
import { Promisable, BetaRunnableTool, BetaToolRunContext } from '../../lib/tools/BetaRunnableTool';
33
import { BetaToolResultContentBlockParam } from '../../resources/beta';
44
import { AutoParseableBetaOutputFormat } from '../../lib/beta-parser';
55
import { AnthropicError } from '../..';
@@ -16,7 +16,10 @@ export function betaTool<const Schema extends Exclude<JSONSchema, boolean> & { t
1616
name: string;
1717
inputSchema: Schema;
1818
description: string;
19-
run: (args: NoInfer<FromSchema<Schema>>) => Promisable<string | Array<BetaToolResultContentBlockParam>>;
19+
run: (
20+
args: NoInfer<FromSchema<Schema>>,
21+
context?: BetaToolRunContext,
22+
) => Promisable<string | Array<BetaToolResultContentBlockParam>>;
2023
}): BetaRunnableTool<NoInfer<FromSchema<Schema>>> {
2124
if (options.inputSchema.type !== 'object') {
2225
throw new Error(

src/helpers/beta/zod.ts

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ import type { infer as zodInfer, ZodType } from 'zod';
33
import * as z from 'zod/v4';
44
import { AnthropicError } from '../../core/error';
55
import { AutoParseableBetaOutputFormat } from '../../lib/beta-parser';
6-
import { BetaRunnableTool, Promisable } from '../../lib/tools/BetaRunnableTool';
6+
import { BetaRunnableTool, BetaToolRunContext, Promisable } from '../../lib/tools/BetaRunnableTool';
77
import { BetaToolResultContentBlockParam } from '../../resources/beta';
88
/**
99
* Creates a JSON schema output format object from the given Zod schema.
@@ -50,7 +50,10 @@ export function betaZodTool<InputSchema extends ZodType>(options: {
5050
name: string;
5151
inputSchema: InputSchema;
5252
description: string;
53-
run: (args: zodInfer<InputSchema>) => Promisable<string | Array<BetaToolResultContentBlockParam>>;
53+
run: (
54+
args: zodInfer<InputSchema>,
55+
context?: BetaToolRunContext,
56+
) => Promisable<string | Array<BetaToolResultContentBlockParam>>;
5457
}): BetaRunnableTool<zodInfer<InputSchema>> {
5558
const jsonSchema = z.toJSONSchema(options.inputSchema, { reused: 'ref' });
5659

src/lib/tools/BetaRunnableTool.ts

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import {
1111
BetaToolTextEditor20250124,
1212
BetaToolTextEditor20250429,
1313
BetaToolTextEditor20250728,
14+
BetaToolUseBlock,
1415
} from '../../resources/beta';
1516

1617
export type Promisable<T> = T | Promise<T>;
@@ -32,9 +33,17 @@ export type BetaClientRunnableToolType =
3233
| BetaToolTextEditor20250429
3334
| BetaToolTextEditor20250728;
3435

36+
export type BetaToolRunContext = {
37+
toolUseBlock: BetaToolUseBlock;
38+
signal?: AbortSignal | null | undefined;
39+
};
40+
3541
// this type is just an extension of BetaTool with a run and parse method
3642
// that will be called by `toolRunner()` helpers
3743
export type BetaRunnableTool<Input = any> = BetaClientRunnableToolType & {
38-
run: (args: Input) => Promisable<string | Array<BetaToolResultContentBlockParam>>;
44+
run: (
45+
args: Input,
46+
context?: BetaToolRunContext,
47+
) => Promisable<string | Array<BetaToolResultContentBlockParam>>;
3948
parse: (content: unknown) => Input;
4049
};

src/lib/tools/BetaToolRunner.ts

Lines changed: 52 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,8 @@ export class BetaToolRunner<Stream extends boolean> {
152152
max_tokens: this.#state.params.max_tokens,
153153
},
154154
{
155-
headers: { 'x-stainless-helper': 'compaction' },
155+
signal: this.#options.signal,
156+
headers: buildHeaders([this.#options.headers, { 'x-stainless-helper': 'compaction' }]),
156157
},
157158
);
158159

@@ -281,6 +282,40 @@ export class BetaToolRunner<Stream extends boolean> {
281282
this.#toolResponse = undefined;
282283
}
283284

285+
/**
286+
* Update the request options for future API calls.
287+
*
288+
* @param optionsOrMutator - Either new options or a function to mutate existing options
289+
*
290+
* @example
291+
* // Direct options update
292+
* runner.setRequestOptions({
293+
* signal: controller.signal,
294+
* });
295+
*
296+
* @example
297+
* // Using a mutator function
298+
* runner.setRequestOptions((prevOptions) => ({
299+
* ...prevOptions,
300+
* signal: controller.signal,
301+
* }));
302+
*/
303+
setRequestOptions(options: BetaToolRunnerRequestOptions): void;
304+
setRequestOptions(
305+
mutator: (prevOptions: BetaToolRunnerRequestOptions) => BetaToolRunnerRequestOptions,
306+
): void;
307+
setRequestOptions(
308+
optionsOrMutator:
309+
| BetaToolRunnerRequestOptions
310+
| ((prevOptions: BetaToolRunnerRequestOptions) => BetaToolRunnerRequestOptions),
311+
) {
312+
if (typeof optionsOrMutator === 'function') {
313+
this.#options = optionsOrMutator(this.#options);
314+
} else {
315+
this.#options = { ...this.#options, ...optionsOrMutator };
316+
}
317+
}
318+
284319
/**
285320
* Get the tool response for the last message from the assistant.
286321
* Avoids redundant tool executions by caching results.
@@ -293,19 +328,25 @@ export class BetaToolRunner<Stream extends boolean> {
293328
* console.log('Tool results:', toolResponse.content);
294329
* }
295330
*/
296-
async generateToolResponse() {
331+
async generateToolResponse(signal: AbortSignal | null | undefined = this.#options.signal) {
297332
const message = (await this.#message) ?? this.params.messages.at(-1);
298333
if (!message) {
299334
return null;
300335
}
301-
return this.#generateToolResponse(message);
336+
return this.#generateToolResponse(message, signal);
302337
}
303338

304-
async #generateToolResponse(lastMessage: BetaMessageParam) {
339+
async #generateToolResponse(
340+
lastMessage: BetaMessageParam,
341+
signal: AbortSignal | null | undefined = this.#options.signal,
342+
) {
305343
if (this.#toolResponse !== undefined) {
306344
return this.#toolResponse;
307345
}
308-
this.#toolResponse = generateToolResponse(this.#state.params, lastMessage);
346+
this.#toolResponse = generateToolResponse(this.#state.params, lastMessage, {
347+
...this.#options,
348+
signal,
349+
});
309350
return this.#toolResponse;
310351
}
311352

@@ -407,6 +448,7 @@ export class BetaToolRunner<Stream extends boolean> {
407448
async function generateToolResponse(
408449
params: BetaToolRunnerParams,
409450
lastMessage = params.messages.at(-1),
451+
requestOptions?: BetaToolRunnerRequestOptions,
410452
): Promise<BetaMessageParam | null> {
411453
// Only process if the last message is from the assistant and has tool use blocks
412454
if (
@@ -441,7 +483,10 @@ async function generateToolResponse(
441483
input = tool.parse(input);
442484
}
443485

444-
const result = await tool.run(input);
486+
const result = await tool.run(input, {
487+
toolUseBlock: toolUse,
488+
signal: requestOptions?.signal,
489+
});
445490
return {
446491
type: 'tool_result' as const,
447492
tool_use_id: toolUse.id,
@@ -491,4 +536,4 @@ export type BetaToolRunnerParams = Simplify<
491536
}
492537
>;
493538

494-
export type BetaToolRunnerRequestOptions = Pick<RequestOptions, 'headers'>;
539+
export type BetaToolRunnerRequestOptions = Pick<RequestOptions, 'headers' | 'signal'>;

0 commit comments

Comments
 (0)