NuGet-based SDK resolver design spec

[jeffkl]

Overview

At the moment, there are two SDK resolvers, both of which assume that the SDKs are on disk. It would be very useful to have SDKs resolved as NuGet packages as well which would allow third party SDKs to be acquired.

Related issues:

• Adding a custom SDK resolver Adding a custom SDK resolver #2278
• Proposal: implement a SdkResolver for NuGet packages #5220

User Experience

Users would have to opt into the new behavior by specifying a version of the SDK to use. The resolver would never automatically get the latest because this would introduce very non-deterministic builds.

Users could control the version on a per-project basis:

<Project Sdk="My.Custom.Sdk/2.3.4">
  ...
</Project>

In this example, a NuGet package named My.Custom.Sdk version 2.3.4 would be retrieved and imported.

It would scale much better if users could specify a version per SDK for the whole repository. They could do this in their global.json:

{
  "sdk" : {
    "version:" "1.0.0"
  },
  "msbuild-sdks": {
    "My.Custom.Sdk" : "2.3.4"
  }
}

The sdk object controls which version of the .NET CLI to use and the msbuild-sdks object controls which versions of the MSBuild SDKs to use. If the resolver detected that multiple versions of the same SDK are requested for the same build episode, a warning would be logged and the first version specified would be used.

MSB4528: warning: A version of the SDK 'My.Custom.Sdk' was already specified in 'D:\Foo\global.json'.  The version specified at 'C:\Foo\bar.csproj (1.7)' was ignored.

In Visual Studio, users would see projects in a Loading state while SDKs are resolved. Any errors that occur during acquisition would be reported in the Output window just like other package restore or build errors. Projects would fail to load if an SDK could not be resolved. Users could fix the issue and reload the project in Visual Studio.

Since the SDKs would be NuGet packages, the standard caching mechanisms and configurations would be in place. Users would control which feeds to use and other NuGet settings in their NuGet.config.

Implementation

• 1. The current SdkResolvers infrastructure within MSBuild would need to be improved to support caching and parallelization. Currently, the same SDK is resolved multiple times per build episode which can slow down evaluations. Make SDK resolution a service #2847
• 2. Add functionality so that resolvers can maintain state in between resolutions. This would allow resolvers to only do expensive operations once per build. Allow SDK resolvers to preserve state #2849
• 3. Implement a new logging message to indicate to callers when an SDK is done resolving. This would allow UI bound threads to wait for the Project constructor to return and be notified when its ready. We already added a ProjectEvaluationFinished event.
• 4. NuGet needs to implement an API for requesting a package to be downloaded. This work is tracked here: NuGet Package Download API NuGet/Home#5919
• 5. We would make a new package type named Sdk or MSBuildSdk. This could help differentiate the packages so that users don't attempt to add a PackageReference to something like Microsoft.NET.Sdk. Create new package type for MSBuild project SDKs NuGet/Home#6484
• 6. Develop an SDK resolver that uses the NuGet object model to download packages into the global package cache and return the path to them. If the package was already on disk, it would not connect to any remote resource. Enough information would be logged that a user could diagnose problems like package feeds being offline, packages not existing, etc. Implement a NuGet-based SDK resolver #2850
  a. The resolver would ship with MSBuild so that the functionality is always available since it would be a core piece of project evaluation. It could be a new assembly or be in Microsoft.Build.dll
• 7. Document how to take advantage of the resolver and how to make SDK packages. This documentation should include the new msbuild-sdks section of global.json. Add documentation for using MSBuild project SDKs MicrosoftDocs/visualstudio-docs#487

If time permits, I'd like to add a command-line argument like /resolver which would allow users to specify custom resolvers to address #2278. Repository owners could control resolvers with Directory.Build.rsp. Resolvers would not be able to be specified in an import because resolvers load before projects are evaluated.

[jeffkl]

FYI @nguerrera

[jaredpar]

. It would be very useful to have SDKs resolved as NuGet packages as well which would allow third party SDKs to be acquired.

Think this design should encompass bringing in the .NET Core SDK via a NuGet package. Today it must be installed SxS with MSBuild and that creates a lot of friction in our initial build experience. Literally everything about the build except the .NET Core SDK can be bootstrapped. This is true even though the .NET Core SDK has supported xcopy distributions.

[jp2masa]

I'm working on 2 project systems, and I'd like to be able to create MSBuild SDKs which can be consumed by the project systems on both Visual Studio and the dotnet CLI. Installing the SDKs to the SDKs folder is not really easy (as it requires admin permission, and for the dotnet CLI it's even harder, as it would have to be installed on all the dotnet SDK installations). As one of the project systems is an extension for managed project systems, I use PackageReference, but it causes some problems on MEF parts import.

I see that the milestone for this feature is MSBuild 15.6, is it planned for the 15.6 release?

[jeffkl]

Yes I'm working to get this out in the next release

[jeffkl]

PR is out for a NuGet-based SDK resolver... #2850