Commit ab37a05
authored
Improve instructions to agents for running tests (#23804)
## Summary
Several changes to our instructions that should, I think, make it easier
and faster for agents to run our tests:
- Suggest setting `CARGO_PROFILE_DEV_OPT_LEVEL=1` rather than
`--cargo-profile=fast-test`. `--cargo-profile` is only a valid command
if you're invoking the tests using `cargo nextest run`; if you're
invoking the tests using `cargo test`, you have to use
`--profile=fast-test` instead. That means that it sometimes takes three
tries for Claude to run our tests successfully if it's being invoked
from Claude Code for Web:
1. Try `cargo nextest run --cargo-profile=fast-test` <-- oh no, `cargo
nextest` isn't installed
2. Try `cargo test --cargo-profile=fast-test` <-- oh no,
`--cargo-profile` isn't a valid CLI argument for `cargo test`
3. Try `cargo test --profile=fast-test` <-- hooray, it finally worked!
- Set two environment variables for `cargo-insta` so that snapshot tests
are easier for agents to handle. Together these are equivalent to
running `cargo insta test --accept`, but they should work even if
`cargo-insta` isn't installed (which it won't be in a Claude Code for
Web environment):
- `INSTA_FORCE_PASS` means that the test runner will keep going even if
it encounters a failing snapshot in an mdtest
- `INSTA_UPDATE=always` means that the snapshots are automatically
accepted, so the agent doesn't need to do a separate `cargo insta
review` step; it can just review the changes that have been applied to
the existing snapshot files.
## Test Plan
Merge it and see?1 parent be5db3c commit ab37a05
1 file changed
Lines changed: 7 additions & 17 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
| 10 | + | |
17 | 11 | | |
18 | 12 | | |
19 | 13 | | |
20 | 14 | | |
21 | 15 | | |
22 | | - | |
| 16 | + | |
23 | 17 | | |
24 | 18 | | |
25 | | - | |
| 19 | + | |
26 | 20 | | |
27 | 21 | | |
28 | | - | |
| 22 | + | |
29 | 23 | | |
30 | 24 | | |
31 | 25 | | |
32 | 26 | | |
33 | 27 | | |
34 | | - | |
| 28 | + | |
35 | 29 | | |
36 | 30 | | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
| 31 | + | |
42 | 32 | | |
43 | 33 | | |
44 | 34 | | |
| |||
0 commit comments