Rename XUnitV3Project -> MTPProject in Helix Sdk (breaking)

xUnit v3 with the Microsoft.Testing.Platform runner (default for v3) uses the exact same self-hosting 'dotnet exec ... <assembly>.dll' execution model as MSTest 4.x, NUnit-MTP, TUnit, and any other MTP-based test framework. The only difference between the previous XUnitV3Project Helix item type and the proposed MTPProject was the reporter args. This commit drops the duplication by:

* Renaming the item type XUnitV3Project to MTPProject, the task CreateXUnitV3WorkItems to CreateMTPWorkItems, and the folder/files xunitv3-runner -> mtp-runner / XUnitV3Runner -> MTPRunner.
* Dropping the UseMicrosoftTestingPlatformRunner switch and the legacy '-xml/-noAutoReporters' code path. MTP is now the only supported execution mode for this Helix item type; users on legacy non-MTP runners hand-author <HelixWorkItem>.
* Switching the generated reporter args to TRX ('--report-trx --report-trx-filename testResults.trx'), which is cross-framework and parsed natively by arcade's TRXFormat reporter. Microsoft.Testing.Extensions.TrxReport is required; MSTest.Sdk references it transitively and arcade's XUnitV3 SDK targets reference it implicitly under MTP mode.
* Adding MSTestProject as a discoverability shim that folds into MTPProject before any other target runs.

Migration: the only external in-tree consumer is dotnet/efcore's eng/helix.proj, which can rename '<XUnitV3Project ... />' to '<MTPProject ... />' as part of its next arcade SDK bump. Arcade's own tests/UnitTests.proj is updated in this commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: Evangelink

Documentation/AzureDevOps/SendingJobsToHelix.md

@@ -119,9 +119,11 @@ The list of available Helix queues can be found on the [Helix homepage](https://
       # condition: succeeded() - defaults to succeeded()
 ```
 
-### XUnit v3
+### Microsoft.Testing.Platform (MTP)
 
-XUnit v3 test projects are self-hosting executables and do not need an external console runner. Instead of `XUnitProjects`, use `XUnitV3Project` items directly in your Helix MSBuild project file (see [the SDK's readme](/src/Microsoft.DotNet.Helix/Sdk/Readme.md) for details). The `XUnitPublishTargetFramework`, `XUnitRuntimeTargetFramework`, and `XUnitRunnerVersion` parameters are not needed for v3 projects.
+Test projects that target [Microsoft.Testing.Platform](https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-intro) (MTP) are self-hosting executables and do not need an external console runner. This covers MSTest 4.x with the MTP runner, xUnit v3 with MTP (the default for v3), NUnit with the MTP runner, TUnit, and any custom MTP-based framework.
+
+Instead of `XUnitProjects`, use `MTPProject` items (or `MSTestProject` as a discoverability alias for MSTest projects) directly in your Helix MSBuild project file (see [the SDK's readme](/src/Microsoft.DotNet.Helix/Sdk/Readme.md) for details). Each project must reference `Microsoft.Testing.Extensions.TrxReport`; this is implicit for `MSTest.Sdk` projects and for xUnit v3 projects built with `Microsoft.DotNet.Arcade.Sdk`'s XUnitV3 targets. The `XUnitPublishTargetFramework`, `XUnitRuntimeTargetFramework`, and `XUnitRunnerVersion` parameters are not needed for MTP projects.
 
 ## The More Complex Case
 

src/Microsoft.DotNet.Helix/Sdk/CreateMTPWorkItems.cs

@@ -0,0 +1,165 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Threading.Tasks;
+using Microsoft.Build.Framework;
+
+namespace Microsoft.DotNet.Helix.Sdk
+{
+    /// <summary>
+    /// MSBuild custom task to create HelixWorkItems for test projects that target
+    /// Microsoft.Testing.Platform (MTP). Such projects ship as self-hosting executables
+    /// (a Main entry point is emitted by Microsoft.Testing.Platform.MSBuild) and can be
+    /// run directly with 'dotnet exec'. This applies to MSTest 4.x, xUnit v3 with MTP,
+    /// NUnit with MTP, TUnit, and any custom MTP-based test framework.
+    /// </summary>
+    public class CreateMTPWorkItems : BaseTask
+    {
+        /// <summary>
+        /// An array of MTP project work items containing the following metadata:
+        /// - [Required] PublishDirectory: the publish output directory of the test project
+        /// - [Required] TargetPath: the output dll path
+        /// - [Optional] Arguments: a string of arguments to be passed to the test executable
+        ///   *after* the auto-injected reporter flags
+        /// The two required parameters are populated automatically by MTPRunner.targets when
+        /// MTPProject.Identity is set to the path of the test csproj file.
+        /// </summary>
+        [Required]
+        public ITaskItem[] MTPProjects { get; set; }
+
+        /// <summary>
+        /// The path to the dotnet executable on the Helix agent. Defaults to "dotnet".
+        /// </summary>
+        public string PathToDotnet { get; set; } = "dotnet";
+
+        /// <summary>
+        /// Boolean true if this is a posix shell, false if not.
+        /// This does not need to be set by a user; it is automatically determined in
+        /// Microsoft.DotNet.Helix.Sdk.MonoQueue.targets. Currently unused (the dotnet exec
+        /// command is identical on every shell) but accepted for symmetry with the other
+        /// Create*WorkItems tasks and to allow future shell-specific tweaks without an
+        /// API break.
+        /// </summary>
+        [Required]
+        public bool IsPosixShell { get; set; }
+
+        /// <summary>
+        /// Optional timeout for all created work items.
+        /// Accepts any value parseable by <see cref="TimeSpan.TryParse(string, out TimeSpan)"/>.
+        /// Defaults to 5 minutes.
+        /// </summary>
+        public string MTPWorkItemTimeout { get; set; }
+
+        /// <summary>
+        /// Optional name of the TRX file produced by Microsoft.Testing.Extensions.TrxReport.
+        /// Defaults to "testResults.trx". Whatever value is used here is what the arcade
+        /// Python TRXFormat parser will pick up from the work item's working directory.
+        /// </summary>
+        public string TrxReportFilename { get; set; } = "testResults.trx";
+
+        /// <summary>
+        /// An array of ITaskItems of type HelixWorkItem.
+        /// </summary>
+        [Output]
+        public ITaskItem[] MTPWorkItems { get; set; }
+
+        public override bool Execute()
+        {
+            ExecuteAsync().GetAwaiter().GetResult();
+            return !Log.HasLoggedErrors;
+        }
+
+        private async Task ExecuteAsync()
+        {
+            // De-duplicate inputs by (Identity, AdditionalProperties). This collapses
+            // accidental duplicates such as a project listed under both MSTestProject and
+            // MTPProject, while preserving legitimate same-Identity entries that differ in
+            // AdditionalProperties (the canonical multi-TFM pattern).
+            var deduped = MTPProjects
+                .GroupBy(p => (p.ItemSpec, p.GetMetadata("AdditionalProperties") ?? string.Empty))
+                .Select(g => g.First())
+                .ToArray();
+
+            MTPWorkItems = (await Task.WhenAll(deduped.Select(PrepareWorkItem))).Where(wi => wi != null).ToArray();
+        }
+
+        /// <summary>
+        /// Prepares a HelixWorkItem for a single MTP test project.
+        /// </summary>
+        private async Task<ITaskItem> PrepareWorkItem(ITaskItem mtpProject)
+        {
+            // Forces this task to run asynchronously
+            await Task.Yield();
+
+            if (!mtpProject.GetRequiredMetadata(Log, "PublishDirectory", out string publishDirectory))
+            {
+                return null;
+            }
+            if (!mtpProject.GetRequiredMetadata(Log, "TargetPath", out string targetPath))
+            {
+                return null;
+            }
+
+            mtpProject.TryGetMetadata("Arguments", out string arguments);
+
+            string assemblyName = Path.GetFileName(targetPath);
+            string assemblyBaseName = assemblyName;
+            if (assemblyBaseName.EndsWith(".dll"))
+            {
+                assemblyBaseName = assemblyBaseName.Substring(0, assemblyBaseName.Length - 4);
+            }
+
+            // MTP test apps are self-hosting executables. Run the assembly directly with
+            // 'dotnet exec'. The reporter args below require the test project to reference
+            // Microsoft.Testing.Extensions.TrxReport. MSTest.Sdk references it transitively;
+            // xUnit v3 projects built with Microsoft.DotNet.Arcade.Sdk's XUnitV3 targets get
+            // it implicitly as well. Other MTP-based frameworks must add the package.
+            //
+            //   --results-directory .  -> TRX is written next to the assembly, which is the
+            //                             work item's cwd and is scanned by the arcade Python
+            //                             reporter / EnableHelixJobMonitor.
+            //   --report-trx           -> enable the TrxReport extension.
+            //   --report-trx-filename  -> deterministic filename so the parser finds it.
+            //
+            // Note about the AzureDevOps reporter (Microsoft.Testing.Extensions.AzureDevOpsReport):
+            // it activates only when '--report-azdo' is explicitly passed AND TF_BUILD=true.
+            // We never pass '--report-azdo' here, so there is no risk of a duplicate Azure
+            // DevOps test run on top of the one the Helix Sdk already opened. If a user has
+            // wired '--report-azdo' into their test project they should remove it for Helix
+            // runs (it conflicts with the Helix-managed run).
+            string reporterArgs =
+                $"--results-directory . --report-trx --report-trx-filename {TrxReportFilename}";
+
+            string command = $"{PathToDotnet} exec --roll-forward Major " +
+                $"--runtimeconfig {assemblyBaseName}.runtimeconfig.json " +
+                $"--depsfile {assemblyBaseName}.deps.json " +
+                $"{assemblyName} {reporterArgs}" +
+                (string.IsNullOrEmpty(arguments) ? "" : " " + arguments);
+
+            Log.LogMessage($"Creating MTP work item with properties Identity: {assemblyName}, PayloadDirectory: {publishDirectory}, Command: {command}");
+
+            TimeSpan timeout = TimeSpan.FromMinutes(5);
+            if (!string.IsNullOrEmpty(MTPWorkItemTimeout))
+            {
+                if (!TimeSpan.TryParse(MTPWorkItemTimeout, out timeout))
+                {
+                    Log.LogWarning($"Invalid value \"{MTPWorkItemTimeout}\" provided for MTPWorkItemTimeout; falling back to default value of \"00:05:00\" (5 minutes)");
+                }
+            }
+
+            var result = new Microsoft.Build.Utilities.TaskItem(assemblyName, new Dictionary<string, string>()
+            {
+                { "Identity", assemblyName },
+                { "PayloadDirectory", publishDirectory },
+                { "Command", command },
+                { "Timeout", timeout.ToString() },
+            });
+            mtpProject.CopyMetadataTo(result);
+            return result;
+        }
+    }
+}

src/Microsoft.DotNet.Helix/Sdk/CreateXUnitV3WorkItems.cs

@@ -331,17 +331,31 @@ Given a local folder `$(TestFolder)` containing `runtests.cmd`, this will run `r
   </PropertyGroup>
 
   <!--
-    XUnit v3 Runner
-      Enabling this will create one work item for each xunit v3 test project specified.
-      XUnit v3 tests are self-hosting executables and do not need an external console runner.
-      This is enabled by specifying one or more XUnitV3Project items.
+    Microsoft.Testing.Platform (MTP) Runner
+      Enabling this will create one Helix work item per MTP-based test project. This
+      covers MSTest 4.x with the MTP runner, xUnit v3 with MTP (the default for v3),
+      NUnit with the MTP runner, TUnit, and any custom MTP-based framework.
+
+      Each test project must reference Microsoft.Testing.Extensions.TrxReport so that
+      results can be reported as a TRX file (which arcade's reporter consumes natively).
+      Projects built with MSTest.Sdk, or with Microsoft.DotNet.Arcade.Sdk's XUnitV3
+      targets, get this reference implicitly.
+
+      For MSTest projects you can use the discoverability alias <MSTestProject .../>;
+      it is folded into @(MTPProject) before any other target runs.
   -->
   <ItemGroup>
-    <XUnitV3Project Include="..\tests\bar.Tests.csproj"/>
+    <!-- xUnit v3 / NUnit-MTP / TUnit / custom MTP -->
+    <MTPProject Include="..\tests\bar.Tests.csproj"/>
+
+    <!-- MSTest (folded into MTPProject) -->
+    <MSTestProject Include="..\tests\baz.Tests.csproj"/>
   </ItemGroup>
   <PropertyGroup>
-    <!-- Whether to use the Microsoft Testing Platform runner. Defaults to true. -->
-    <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
+    <!-- Optional: customize the TRX filename produced by Microsoft.Testing.Extensions.TrxReport. Defaults to testResults.trx. -->
+    <MTPTrxReportFilename>testResults.trx</MTPTrxReportFilename>
+    <!-- Optional: per-work-item timeout (TimeSpan format). Defaults to 5 minutes. -->
+    <MTPWorkItemTimeout>00:05:00</MTPWorkItemTimeout>
   </PropertyGroup>
 
   <ItemGroup>

src/Microsoft.DotNet.Helix/Sdk/Readme.md

@@ -46,7 +46,7 @@
   <UsingTask TaskName="FindDotNetCliPackage" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
   <UsingTask TaskName="GetHelixWorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
   <UsingTask TaskName="CreateXUnitWorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
-  <UsingTask TaskName="CreateXUnitV3WorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
+  <UsingTask TaskName="CreateMTPWorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
   <UsingTask TaskName="CreateXHarnessAndroidWorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
   <UsingTask TaskName="CreateXHarnessAppleWorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />
   <UsingTask TaskName="CreateTestsForWorkItems" AssemblyFile="$(MicrosoftDotNetHelixSdkTasksAssembly)" Runtime="NET" Architecture="*" TaskFactory="$(MicrosoftDotNetHelixSdkTasksFactory)" />

src/Microsoft.DotNet.Helix/Sdk/tools/Microsoft.DotNet.Helix.Sdk.props

@@ -2,7 +2,7 @@
 <Project>
 
   <PropertyGroup>
-    <_HelixMonoQueueTargets>$(_HelixMonoQueueTargets);$(MSBuildThisFileDirectory)XUnitV3Runner.targets</_HelixMonoQueueTargets>
+    <_HelixMonoQueueTargets>$(_HelixMonoQueueTargets);$(MSBuildThisFileDirectory)MTPRunner.targets</_HelixMonoQueueTargets>
   </PropertyGroup>
 
 </Project>