Make it easy for mortals to generate packages.config for an app

[richlander]

@ellismg includes a pre-build packages.config in his very nice Linux Instructions.

How do mere mortals generate such files?

[kangaroo]

This. So much time lost today because of this. How do I find the latest versions of something or other for nuget? How do I get it to pull the latest versions to be in sync with corefx? boggle

[stephentoub]

cc; @ericstj

[kangaroo]

After a ton of fumbling, I finally realized the feed is now called dotnet-core, not dotnst-corefx. All the docs are out of date. I found this:

https://www.myget.org/gallery/dotnet-core

Of course everything requires nuget 3, which is incredibly difficult to find on google (a direct link to the exe).

I found all that, updated my packages.config to:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="System.Console" version="4.0.0-beta-23213" />
  <package id="System.Diagnostics.Contracts" version="4.0.1-beta-23213" />
  <package id="System.Diagnostics.Debug" version="4.0.11-beta-23213" />
  <package id="System.Diagnostics.Tools" version="4.0.1-beta-23213" />
  <package id="System.Globalization" version="4.0.11-beta-23213" />
  <package id="System.IO" version="4.0.11-beta-23213" />
  <package id="System.IO.FileSystem.Primitives" version="4.0.1-beta-23213" />
  <package id="System.Reflection" version="4.1.0-beta-23213" />
  <package id="System.Resources.ResourceManager" version="4.0.1-beta-23213" />
  <package id="System.Runtime" version="4.0.21-beta-23213" />
  <package id="System.Runtime.Extensions" version="4.0.11-beta-23213" />
  <package id="System.Runtime.Handles" version="4.0.1-beta-23213" />
  <package id="System.Runtime.InteropServices" version="4.0.21-beta-23213" />
  <package id="System.Text.Encoding" version="4.0.11-beta-23213" />
  <package id="System.Text.Encoding.Extensions" version="4.0.11-beta-23213" />
  <package id="System.Threading" version="4.0.11-beta-23213" />
  <package id="System.Threading.Tasks" version="4.0.11-beta-23213" />
</packages>

It still doesn't work:

plasma@diorite:~/coreclr-demo$ mono nuget.exe restore packages/packages.config -Source https://www.myget.org/F/dotnet-core/ -PackagesDirectory packages
Restoring NuGet package System.Diagnostics.Contracts.4.0.1-beta-23213.
Restoring NuGet package System.Diagnostics.Debug.4.0.11-beta-23213.
Restoring NuGet package System.Diagnostics.Tools.4.0.1-beta-23213.
Restoring NuGet package System.Globalization.4.0.11-beta-23213.
Restoring NuGet package System.IO.4.0.11-beta-23213.
Restoring NuGet package System.IO.FileSystem.Primitives.4.0.1-beta-23213.
Restoring NuGet package System.Reflection.4.1.0-beta-23213.
Restoring NuGet package System.Resources.ResourceManager.4.0.1-beta-23213.
Restoring NuGet package System.Runtime.4.0.21-beta-23213.
Restoring NuGet package System.Runtime.Extensions.4.0.11-beta-23213.
Restoring NuGet package System.Runtime.Handles.4.0.1-beta-23213.
Restoring NuGet package System.Runtime.InteropServices.4.0.21-beta-23213.
Restoring NuGet package System.Text.Encoding.4.0.11-beta-23213.
Restoring NuGet package System.Text.Encoding.Extensions.4.0.11-beta-23213.
Restoring NuGet package System.Threading.4.0.11-beta-23213.
WARNING: Unable to find version '4.0.11-beta-23213' of package 'System.Runtime.Extensions'.
WARNING: Unable to find version '4.0.21-beta-23213' of package 'System.Runtime'.
WARNING: Unable to find version '4.0.11-beta-23213' of package 'System.Text.Encoding'.
WARNING: Unable to find version '4.0.1-beta-23213' of package 'System.Runtime.Handles'.
WARNING: Unable to find version '4.1.0-beta-23213' of package 'System.Reflection'.
WARNING: Unable to find version '4.0.1-beta-23213' of package 'System.Resources.ResourceManager'.
WARNING: Unable to find version '4.0.11-beta-23213' of package 'System.Diagnostics.Debug'.
WARNING: Unable to find version '4.0.1-beta-23213' of package 'System.Diagnostics.Tools'.
WARNING: Unable to find version '4.0.1-beta-23213' of package 'System.IO.FileSystem.Primitives'.
WARNING: Unable to find version '4.0.11-beta-23213' of package 'System.IO'.
WARNING: Unable to find version '4.0.11-beta-23213' of package 'System.Globalization'.
WARNING: Unable to find version '4.0.11-beta-23213' of package 'System.Threading'.
WARNING: Unable to find version '4.0.21-beta-23213' of package 'System.Runtime.InteropServices'.
Adding package 'System.Diagnostics.Contracts.4.0.1-beta-23213' to folder '/home/plasma/coreclr-demo/packages'
Adding package 'System.Text.Encoding.Extensions.4.0.11-beta-23213' to folder '/home/plasma/coreclr-demo/packages'
Added package 'System.Diagnostics.Contracts.4.0.1-beta-23213' to folder '/home/plasma/coreclr-demo/packages'
Added package 'System.Text.Encoding.Extensions.4.0.11-beta-23213' to folder '/home/plasma/coreclr-demo/packages'
Unable to find version '4.0.1-beta-23213' of package 'System.Resources.ResourceManager'.
Unable to find version '4.0.11-beta-23213' of package 'System.Runtime.Extensions'.
Unable to find version '4.0.1-beta-23213' of package 'System.IO.FileSystem.Primitives'.
Unable to find version '4.0.21-beta-23213' of package 'System.Runtime'.
Unable to find version '4.0.11-beta-23213' of package 'System.Text.Encoding'.
Unable to find version '4.0.1-beta-23213' of package 'System.Runtime.Handles'.
Unable to find version '4.1.0-beta-23213' of package 'System.Reflection'.
Unable to find version '4.0.11-beta-23213' of package 'System.Diagnostics.Debug'.
Unable to find version '4.0.1-beta-23213' of package 'System.Diagnostics.Tools'.
Unable to find version '4.0.11-beta-23213' of package 'System.IO'.
Unable to find version '4.0.11-beta-23213' of package 'System.Globalization'.
Unable to find version '4.0.11-beta-23213' of package 'System.Threading'.
Unable to find version '4.0.21-beta-23213' of package 'System.Runtime.InteropServices'.

[ellismg]

The plan over the next month is to start producing true cross platform packages, which can be consumed by dnu to restore linux specific assets. At that point, you'll just author a project.json file for you project, DNU restore and then get everything you need.

For folks who are working outside the DNX app model and want to pull artifacts out of the packages (like folks who are using this document to try to build a corerun based environment), I believe that @richlander wrote a tool that walks over the project.lock.json file generated by DNU and yanks all that stuff out, instead of the hellish find magic that I wrote.

@richlander can you or @blackdwarf either update this document or just kill these sections and write something that does work?

[kangaroo]

@ellismg Great. In the mean time, how do I restore a current package set for a hello world? I picked the latest version and nuget.exe says it can't find them.

Should I just restort to downloading things from the build artifacts?

[ellismg]

I think perhaps you should just use DNU? I don't really understand the state that nuget.exe is in when it comes to consuming v3 pacakges, Eric would know more.

Anyway, if you have DNU, something like this would work. Write a project.json file with the assemblies you depend on.

{
    "dependencies": {
        "System.Console": "4.0.0-beta-*",
        "System.Runtime": "4.0.21-beta-*"
    }
}

Use dnu to restore them to a folder (e.g. packages):

dnu restore --source https://www.myget.org/F/dotnet-core --packages ./packages roject.json

The major issue now is that the location of the assets of the packages is no longer easy to understand, you really need to use the project.lock.json file to figure out where the assets you want are. That's where Rich's tool comes in but I don't know what the status of it is.

[kangaroo]

So I tried dnu. Now I have a VM which is looking for System.IO.TextWriter in System.IO.dll instead of mscorlib.dll for some reason.

Have we advanced the collective tooling state of dnvm/dnu/dnx/nuget/etc to the point where I'm intelligent enough to build and work on the vm, but too stupid to build hello world?

[kferrio]

Some of the observations in this post are strongly dependent on multiple moving targets. I'm posting this in case it helps anyway.

A couple more observations on the (actually not at all hellish) find command by @ellismg ...

Let's use a modified command to see what's going on:

find ../packages/ -path '*/netcore50/*.dll' -print

(N.B. the -wholename option is deprecated in favor of the more portable -path option.)

As of right now, the find will return lib, runtime, and even ref builds. So the final state of the destination directory potentially depends on the order in which results are returned:

user@host:~/coreclr-demo$ find ./packages/ -path '*/netcore50/*.dll' -print
./packages/System.Resources.ResourceManager/4.0.1-rc3-23805/lib/netcore50/System.Resources.ResourceManager.dll
./packages/System.Resources.ResourceManager/4.0.1-rc3-23805/runtimes/win8-aot/lib/netcore50/System.Resources.ResourceManager.dll
./packages/System.Resources.ResourceManager/4.0.1-rc3-23805/ref/netcore50/System.Resources.ResourceManager.dll
./packages/System.Diagnostics.Tools/4.0.1-rc3-23805/ref/netcore50/System.Diagnostics.Tools.dll
./packages/System.Reflection.Primitives/4.0.0/lib/netcore50/System.Reflection.Primitives.dll
./packages/System.Reflection.Primitives/4.0.0/runtimes/win8-aot/lib/netcore50/System.Reflection.Primitives.dll
./packages/System.Reflection.Primitives/4.0.0/ref/netcore50/System.Reflection.Primitives.dll
./packages/System.Globalization/4.0.11-rc3-23805/ref/netcore50/System.Globalization.dll
./packages/System.Diagnostics.Contracts/4.0.1-rc3-23805/lib/netcore50/System.Diagnostics.Contracts.dll
./packages/System.Diagnostics.Contracts/4.0.1-rc3-23805/runtimes/win8-aot/lib/netcore50/System.Diagnostics.Contracts.dll
./packages/System.Diagnostics.Contracts/4.0.1-rc3-23805/ref/netcore50/System.Diagnostics.Contracts.dll
./packages/System.Threading.Tasks/4.0.11-rc3-23805/ref/netcore50/System.Threading.Tasks.dll
./packages/System.Runtime/4.1.0-rc3-23805/ref/netcore50/System.Runtime.dll
./packages/System.Runtime.InteropServices/4.1.0-rc3-23805/ref/netcore50/System.Runtime.InteropServices.dll
./packages/System.Reflection/4.1.0-rc3-23805/ref/netcore50/System.Reflection.dll
./packages/System.Diagnostics.Debug/4.0.11-rc3-23805/ref/netcore50/System.Diagnostics.Debug.dll
./packages/System.Runtime.InteropServices.PInvoke/4.0.0-rc3-23805/runtimes/aot/lib/netcore50/System.Runtime.InteropServices.PInvoke.dll
./packages/System.Text.Encoding/4.0.11-rc3-23805/ref/netcore50/System.Text.Encoding.dll
./packages/System.Runtime.Extensions/4.1.0-rc3-23805/ref/netcore50/System.Runtime.Extensions.dll
./packages/System.Text.Encoding.Extensions/4.0.11-rc3-23805/ref/netcore50/System.Text.Encoding.Extensions.dll
./packages/System.IO/4.1.0-rc3-23805/ref/netcore50/System.IO.dll
./packages/System.Threading/4.0.11-rc3-23805/ref/netcore50/System.Threading.dll

Also, if you actually pick through this carefully, you'll find that the current packages (i.e. 4.0.1-rc3-23805) at MyGet are missing netcore50 builds of several components needed to run HelloWorld. Some -- but apparently not all -- of these can be safely replaced with DLLs from your own corefx build. So at the moment I am one or two assemblies shy of having HelloWorld fully resolve. Here is my project.json for reference:

{
    "dependencies": {
        "System.Console": "4.0.0-rc3-*",
        "System.Diagnostics.Contracts": "4.0.1-rc3-*",
        "System.Diagnostics.Debug":"4.0.11-rc3-*",
        "System.Diagnostics.Tools":"4.0.1-rc3-*",
        "System.Globalization":"4.0.11-rc3-*",
        "System.IO": "4.1.0-rc3-*",
        "System.IO.FileSystem.Primitives": "4.0.1-rc3-*" ,
        "System.Reflection": "4.1.0-rc3-*" ,
        "System.Resources.ResourceManager": "4.0.1-rc3-*" ,
        "System.Runtime": "4.1.0-rc3-*" ,
        "System.Runtime.Extensions": "4.1.0-rc3-*" ,
        "System.Runtime.Handles": "4.0.1-rc3-*" ,
        "System.Runtime.InteropServices": "4.1.0-rc3-*" ,
        "System.Text.Encoding": "4.0.11-rc3-*" ,
        "System.Text.Encoding.Extensions": "4.0.11-rc3-*" ,
        "System.Threading": "4.0.11-rc3-*" ,
        "System.Threading.Tasks": "4.0.11-rc3-*" 
    }
}

I based those deps on the current latest packages at MyGet dated Feb 5.

My whole reason for building Core myself is that I have a couple very specific fixes I would like to implement and test. So I would actually prefer to build all my dependencies and not rely on MyGet/Nuget at all until the packaging all gets sorted out.

Thanks.

[dls314]

@kferrio

Have you gotten a working HelloWorld using locally built dependencies?

I'm working through the instructions from building/linux-instructions.md, and I think I have a totally disorganized, wholly terrible set of notes that worked for me.

The dependencies were the hardest part, and some of the following list is probably surplus, but it's the ~/coreclr-demo/runtime contents as I have them.

corerun
HelloWorld.exe
libcoreclr.so
libcoreclrtraceptprovider.so
libdbgshim.so
libmscordaccore.so
libmscordbi.so
libryujit.so
libsosplugin.so
libsos.so
mscorlib.dll
mscorlib.ni.dll
System.Collections.dll
System.Console.dll
System.Diagnostics.Debug.dll
System.Globalization.Native.so
System.IO.Compression.Native.so
System.IO.dll
System.IO.FileSystem.Primitives.dll
System.Native.so
System.Net.Http.Native.so
System.Private.Uri.dll
System.Resources.ResourceManager.dll
System.Runtime.dll
System.Runtime.Extensions.dll
System.Runtime.Handles.dll
System.Runtime.InteropServices.dll
System.Security.Cryptography.Native.so
System.Text.Encoding.dll
System.Text.Encoding.Extensions.dll
System.Threading.dll
System.Threading.Tasks.dll

I'm still not sure where the dependency on System.IO.FileSystem.Primitives.dll came in. It's one I didn't find with dotPeek

[kferrio]

Yes, I did have HelloWorld built and running...until a few days ago when I
got latest. These are early days and my notes are in flux also. Thanks for
sharing your deps.; I also have some extras and have not systematically
trimmed them. Thanks!

@kferrio https://github.com/kferrio

Have you gotten a working HelloWorld using locally built dependencies?

I'm working through the instructions from building/linux-instructions.md
https://github.com/dotnet/coreclr/blob/master/Documentation/building/linux-instructions.md,
and I think I have a totally disorganized, wholly terrible set of notes
that worked for me.

The dependencies were the hardest part, and some of the following list is
probably surplus, but it's the ~/coreclr-demo/runtime contents as I have
them.

corerun
HelloWorld.exe
libcoreclr.so
libcoreclrtraceptprovider.so
libdbgshim.so
libmscordaccore.so
libmscordbi.so
libryujit.so
libsosplugin.so
libsos.so
mscorlib.dll
mscorlib.ni.dll
System.Collections.dll
System.Console.dll
System.Diagnostics.Debug.dllSystem.Globalization.Native.soSystem.IO.Compression.Native.so
System.IO.dll
System.IO.FileSystem.Primitives.dllSystem.Native.soSystem.Net.Http.Native.so
System.Private.Uri.dll
System.Resources.ResourceManager.dll
System.Runtime.dll
System.Runtime.Extensions.dll
System.Runtime.Handles.dll
System.Runtime.InteropServices.dllSystem.Security.Cryptography.Native.so
System.Text.Encoding.dll
System.Text.Encoding.Extensions.dll
System.Threading.dll
System.Threading.Tasks.dll

—
Reply to this email directly or view it on GitHub
https://github.com/dotnet/coreclr/issues/480#issuecomment-181974247.

[kingces95]

@blackdwarf Any update on when a new set of build instructions might be available to get helloworld off the ground? When first release, the instructions were good. But the nuget feeds seem to be missing files: 404 Not Found is what I get trying to restore them.