Skip to content

stream Cargo JSON directly into BinaryList#3131

Merged
sunshowers merged 2 commits into
nextest-rs:mainfrom
cornmander:codex/issue-3129-stream-cargo-json
Mar 9, 2026
Merged

stream Cargo JSON directly into BinaryList#3131
sunshowers merged 2 commits into
nextest-rs:mainfrom
cornmander:codex/issue-3129-stream-cargo-json

Conversation

@cornmander

Copy link
Copy Markdown
Contributor

summary

  • add a BinaryListBuilder that can consume Cargo messages incrementally, and reuse it from BinaryList::from_messages
  • teach the cargo-nextest binary-list build path to process Cargo stdout in one pass instead of buffering it and reparsing it later
  • add a regression test for the line-by-line parsing path

Closes #3129.

testing

  • cargo xfmt
  • cargo nextest run -p nextest-runner -p cargo-nextest

experimental findings

I benchmarked the warmed end-to-end CLI path that this change targets:

cargo-nextest nextest list --workspace --all-targets --list-type binaries-only \
  --cargo-message-format json --cargo-quiet >/dev/null

using hyperfine --warmup 5 --runs 20 on two external repositories:

  • tokio (e67cd87): 426.3 ms ± 37.3 ms before, 400.6 ms ± 10.6 ms after, about 6.0% faster
  • rust-analyzer (51966da): 426.4 ms ± 36.4 ms before, 400.6 ms ± 11.9 ms after, about 6.1% faster

These runs intentionally benchmark the Cargo-JSON-forwarding list path, since that is the path changed by this patch.

Previously, cargo-nextest buffered Cargo JSON output into a BufList and reparsed it after the build finished. Now it feeds each line directly into a shared BinaryListBuilder, preserving JSON forwarding behavior while avoiding the extra allocation and second parse.

Also add a regression test for line-by-line parsing of Cargo output.
@cornmander cornmander marked this pull request as ready for review March 9, 2026 03:41
@codecov

codecov Bot commented Mar 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.44%. Comparing base (9531cb9) to head (0df9104).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3131      +/-   ##
==========================================
+ Coverage   84.43%   84.44%   +0.01%     
==========================================
  Files         156      156              
  Lines       42733    42811      +78     
==========================================
+ Hits        36080    36153      +73     
- Misses       6653     6658       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sunshowers sunshowers left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Just a few minor notes for myself that I'll fix in a followup.

@sunshowers sunshowers merged commit 3980fdc into nextest-rs:main Mar 9, 2026
23 checks passed
sunshowers added a commit that referenced this pull request Mar 9, 2026
Just a couple of minor cleanups.
sunshowers added a commit that referenced this pull request Mar 9, 2026
Just a couple of minor cleanups.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream Cargo JSON directly into BinaryList instead of buffering and reparsing it

2 participants