Skip to content

support remote eval params#64

Open
Andrew Kent (realark) wants to merge 1 commit intomainfrom
ark/remote-eval-params
Open

support remote eval params#64
Andrew Kent (realark) wants to merge 1 commit intomainfrom
ark/remote-eval-params

Conversation

@realark
Copy link
Copy Markdown
Collaborator

@realark Andrew Kent (realark) commented Apr 3, 2026

support data and model eval params (prompt not yet supported)

@realark Andrew Kent (realark) changed the title Ark/remote eval params support remote eval params Apr 3, 2026
@realark Andrew Kent (realark) force-pushed the ark/remote-eval-params branch 7 times, most recently from 9adbe3c to 9e4f26d Compare April 6, 2026 18:25
@realark Andrew Kent (realark) marked this pull request as ready for review April 6, 2026 18:36
@realark Andrew Kent (realark) added the enhancement New feature or request label Apr 6, 2026
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no blocking reviews!

TaskResult<INPUT, OUTPUT> apply(DatasetCase<INPUT, OUTPUT> datasetCase) throws Exception;
default TaskResult<INPUT, OUTPUT> apply(DatasetCase<INPUT, OUTPUT> datasetCase)
throws Exception {
throw new UnsupportedOperationException(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is technically a breaking change because both methods are default. Is that fine?

Comment on lines +72 to +78
enum DataType {
STRING,
NUMBER,
BOOLEAN,
OBJECT,
ARRAY
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not used anywhere

Map<String, Object> parametersContainer = new LinkedHashMap<>();
parametersContainer.put("type", "braintrust.staticParameters");
parametersContainer.put("schema", schemaMap);
parametersContainer.put("source", NullNode.getInstance());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why NullNode.getInstance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants