Spun off from review feedback on #16986 (comment) by @Youssef1313.
Problem
In CreateMTPWorkItems (and previously in CreateXUnitV3WorkItems) the emitted Helix work item command unconditionally passes --roll-forward Major:
string command = `$"{PathToDotnet} exec --roll-forward Major " +`
`$"--runtimeconfig {assemblyBaseName}.runtimeconfig.json " +`
`$"--depsfile {assemblyBaseName}.deps.json " +`
`$"{assemblyName} {reporterArgs}"` + ...
Quoting @Youssef1313:
This [is] extremely extremely concerning to me. Why do we roll forward to Major?
I'm reading this as ""if I have a test project that targets net8.0 and net9.0, and .NET 8 runtime wasn't found, then both TFMs will run with .NET 9, causing silent loss of coverage.""
This behavior is identical to what was in the previous CreateXUnitV3WorkItems task, so #16986 is not the cause - it's a pre-existing issue that the rename inherited.
Proposed direction
- Investigate whether
--roll-forward Major is still required (it was likely added because Helix queues did not always have an exact runtime match available; we should confirm whether queue images now ship a matching runtime per TFM).
- If unconditional Major roll-forward is no longer needed, drop it (or make it opt-in via metadata on
MTPProject / a property).
- If it is still needed for some queues, narrow the policy (e.g.
Minor or LatestMinor) to avoid silently running net8.0 work items on a net9.0 runtime.
Impact
Silent loss of multi-TFM coverage is the worst-case outcome - failures look green because the assembly happily runs on a newer runtime instead of failing fast.
cc @Youssef1313 @ilyas1974
Spun off from review feedback on #16986 (comment) by @Youssef1313.
Problem
In
CreateMTPWorkItems(and previously inCreateXUnitV3WorkItems) the emitted Helix work item command unconditionally passes--roll-forward Major:Quoting @Youssef1313:
This behavior is identical to what was in the previous
CreateXUnitV3WorkItemstask, so #16986 is not the cause - it's a pre-existing issue that the rename inherited.Proposed direction
--roll-forward Majoris still required (it was likely added because Helix queues did not always have an exact runtime match available; we should confirm whether queue images now ship a matching runtime per TFM).MTPProject/ a property).MinororLatestMinor) to avoid silently running net8.0 work items on a net9.0 runtime.Impact
Silent loss of multi-TFM coverage is the worst-case outcome - failures look green because the assembly happily runs on a newer runtime instead of failing fast.
cc @Youssef1313 @ilyas1974