Add libtest json output option

[Jake-Shadle]

There are a couple of todos, but the biggest missing piece is combined stdout/err output.

Opening this as a draft while I make the combined output helper.

[codecov]

Codecov Report

Attention: 330 lines in your changes are missing coverage. Please review.

Comparison is base (b7c376e) 79.13% compared to head (54618a0) 78.00%.
Report is 91 commits behind head on main.

Files	Patch %	Lines
nextest-runner/src/reporter/structured/libtest.rs	0.00%	321 Missing ⚠️
cargo-nextest/src/errors.rs	0.00%	4 Missing ⚠️
nextest-runner/src/reporter/structured.rs	57.14%	3 Missing ⚠️
cargo-nextest/src/dispatch.rs	86.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1086      +/-   ##
==========================================
- Coverage   79.13%   78.00%   -1.13%     
==========================================
  Files          64       69       +5     
  Lines       16555    17329     +774     
==========================================
+ Hits        13100    13518     +418     
- Misses       3455     3811     +356     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sunshowers]

Thanks, will have a look at it over the weekend.

For stdout/stderr, can we just do something like:

*** STDOUT ***
<stdout>
*** STDERR ***
<stderr>

It's not great but it is emulation, after all. Seems like trying to maintain a combined buffer (or separate buffers with indexes) is possible but also adds some nontrivial complexity.

[sunshowers]

Just FYI I've had too may commitments to review this, but hope to get to it this weekend.

[Jake-Shadle]

No worries, I'm doing a bunch of other stuff atm.

[sunshowers]

Thanks for your patience. This looks pretty good overall. Let's get this in before working on the combined output.

[sunshowers]

Thanks for this. I really apologize for how long this has taken, I've been dealing with serious personal life issues.

I think this is good to go. The only question I have is whether it's ok for this to be experimental for at least a short while. You'd have to add experimental = ["libtest-json"] to .config/nextest.toml, similar to setup scripts. (I'm happy to implement this.)

[Jake-Shadle]

I could do that, but I don't understand the need for that additional hurdle when it already requires passing a new flag?

[sunshowers]

It's true that it requires passing a new flag, but as machine-readable output this would need to be a stable format (which experimental features are not). I think there's just a few more things we need to do before I'd feel comfortable stabilizing this:

• Use the format in our integration tests to shake out issues
• Document the format
• Get some real-world use to ensure that it's serving users' needs

Once this is stabilized the format will have to be maintained forever (per our stability policy), so it's worth spending some time making sure it's all good.

I'll land this and make it experimental, before getting a release out tomorrow or so. Thanks for all your work!