This guide will go more in-depth on how to do cross-building across multiple operating systems and architectures. It's worth mentioning this is not an any-to-any scenario. Only the combinations explained here are possible/supported. If/When any other combinations get supported/discovered, this document will get updated accordingly.
This section will go over cross-compiling on Windows. Currently, Windows allows you to cross-compile from x64 to basically any other architecture.
To do cross-compilation for ARM64 on Windows, first make sure you have the appropriate tools and Windows SDK installed. This is described in detail in the Windows requirements doc.
Once you have all the required dependencies, it is a straightforward process. Windows knows how to cross-build behind curtains, so all you have to do is specify which architecture you want to build for:
.\build.cmd -s clr -c Release -arch arm64Building for x86 doesn't require any additional software installed or configured, since all the x64 build tools also have the capability of building for x86. Just specify it when calling the build script:
.\build.cmd -s clr -c Release -arch x86This section will go over cross-compiling on macOS. Currently, macOS allows you to cross-compile between x64 and ARM64.
Similarly to targeting Windows x86, the native tooling you installed back in the macOS requirements doc has the capabilities to effectuate the cross-compilation. You have simply to pass the -cross flag, along with the designated architecture. For example, for an arm64 build on an Intel x64 Mac:
./build.sh -s clr -c Release --cross -a arm64This section will go over cross-compiling on Linux. Currently, Linux allows you to cross-compile from x64 to ARM32 and ARM64, as well as to other Unix-based operating systems, like FreeBSD and Alpine.
Before you can attempt to do any Linux cross-building, you will need to generate the ROOTFS corresponding to the platform you want to target. The script located in eng/common/cross/build-rootfs.sh is in charge of effectuating this task. Note that this script must be run with sudo, as it needs to make some symlinks to the system that would not be allowed otherwise.
For example, let's try generating a ROOTFS targeting Ubuntu 18 (Bionic) for ARM64:
sudo ./eng/common/cross/build-rootfs.sh arm64 bionicThe rootfs binaries will be placed in .tools/rootfs/<arch>. So, for this example, it would be .tools/rootfs/arm64. Note that the Linux codename argument is optional, and if you omit it, the script will pick its default one.
It is also possible to have build-rootfs.sh generate its output elsewhere. For that, you have to set the environment variable ROOTFS_DIR to the path where you want your rootfs binaries to be placed in.
Generating the ROOTFS for FreeBSD cross-compiling is virtually the same as for other Linux distributions in other architectures. The only difference is you have to specify it so. For example, for an x64 cross-compilation for FreeBSD 13:
sudo ./eng/common/cross/build-rootfs.sh x64 freebsd13Once you have your ROOTFS generated, make sure to set the environment variable ROOTFS_DIR to where your binaries are located if you didn't do so in the previous step. Then, build normally and pass the --cross flag to the build script:
export ROOTFS_DIR=/path/to/runtime/.tools/rootfs/arm64
./build.sh --subset clr --configuration Release --arch arm64 --crossLike with any other build, you'll find the built binaries at artifacts/bin/coreclr/Linux.<arch>.<configuration>. For our example, it would be artifacts/bin/coreclr/Linux.arm64.Release.
Very similarly to generating the ROOTFS, cross-building for FreeBSD follows the same process as for other architectures, which is described above. The only difference is that, in addition to the --cross flag, you also have to specify it is for FreeBSD by means of the --os flag:
export ROOTFS_DIR=/path/to/runtime/.tools/rootfs/x64
./build.sh --subset clr --configuration Release --cross --os freebsdThe default ARM compilation configuration for CoreCLR is armv7-a with thumb-2 instruction set, and VFPv3 floating point with 32 64-bit FPU registers.
CoreCLR JIT requires 16 64-bit or 32 32-bit FPU registers.
A set of FPU configuration options have been provided in the build scripts to accommodate different CPU types. These FPU configuration options are:
- CLR_ARM_FPU_TYPE: Translates to a value given to the
-mfpucompiler option. Please refer to your compiler documentation for possible options. - CLR_ARM_FPU_CAPABILITY: Used by the PAL code to decide which FPU registers should be saved and restored during context switches (the supported options are 0x3 and 0x7):
- Bit 0 unused always set to 1.
- Bit 1 corresponds to 16 64-bit FPU registers.
- Bit 2 corresponds to 32 64-bit FPU registers.
For example, if you wanted to support armv7 CPU with VFPv3-d16, you'd use the following compile options:
./build.sh --subset clr --configuration Release --cross --arch arm --cmakeargs "-DCLR_ARM_FPU_CAPABILITY=0x3" --cmakeargs "-DCLR_ARM_FPU_TYPE=vfpv3-d16"Certain parts of the build process need some native components that are built for the current machine architecture, regardless of whichever you are targeting. These tools are referred to as cross-targeting tools or "cross tools". There are two categories of these tools today:
- Crossgen2 JIT Tools
- Diagnostic Libraries
The Crossgen2 JIT tools are used to run Crossgen2 on libraries built during the current build, such as during the clr.nativecorelib stage. Under normal circumstances, you should have no need to worry about this, since these tools are automatically built when using the .\build.cmd or ./build.sh scripts at the root of the repo to build any of the CoreCLR native files.
However, you might find yourself needing to (re)build them because either you made changes to them, or you built CoreCLR in a different way using build-runtime.sh instead of the usual default script at the root of the repo. To build these tools, you need to run the src/coreclr/build-runtime.sh script, and pass the -hostarch flag with the architecture of the host machine, alongside the -component crosscomponents flag to specify that you only want to build the cross-targeting tools. Retaking our previous example of building for ARM64 using an x64 Linux machine:
./src/coreclr/build-runtime.sh -arm64 -hostarch x64 -component crosscomponents -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1"The output of running this command is placed in artifacts/bin/coreclr/linux.<target_arch>.<configuration>/<host_arch>. For our example, it would be artifacts/bin/coreclr/linux.arm64.Release/x64.
On Windows, you can build these cross-targeting diagnostic libraries with the linuxdac and alpinedac subsets from the root build.cmd script. That said, you can also use the build-runtime.cmd script, like with Linux. These builds also require you to pass the -os flag to specify the target OS. For example:
.\src\coreclr\build-runtime.cmd -arm64 -hostarch x64 -os linux -component crosscomponents -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1"If you're building the cross-components in powershell, you'll need to wrap "-DCLR_CROSS_COMPONENTS_BUILD=1" with single quotes (') to ensure things are escaped correctly for CMD.
When it comes to building, Docker offers the most flexibility when it comes to targeting different Linux platforms and other similar Unix-based ones, like FreeBSD. This is thanks to the multiple existing Docker images already configured for doing such cross-platform building, and Docker's ease of use of running out of the box on Windows machines with WSL enabled, installed, and up and running, as well as Linux machines.
As mentioned in the Linux Cross-Building section, the ROOTFS_DIR environment variable has to be set to the crossrootfs location. The prereqs Docker images already have crossrootfs built, so you only need to specify it when creating the Docker container by means of the -e flag. These locations are specified in the Docker Images table.
In addition, you also have to specify the --cross flag with the target architecture. For example, the following command would create a container to build CoreCLR for Linux ARM64:
docker run --rm \
-v <RUNTIME_REPO_PATH>:/runtime \
-w /runtime \
-e ROOTFS_DIR=/crossrootfs/arm64 \
mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net11.0-cross-arm64 \
./build.sh --subset clr --cross --arch arm64Using Docker to cross-build for FreeBSD is very similar to any other Docker Linux build. You only need to use the appropriate image and pass --os as well to specify this is not an architecture(-only) build. For example, to make a FreeBSD x64 build:
docker run --rm \
-v <RUNTIME_REPO_PATH>:/runtime \
-w /runtime \
-e ROOTFS_DIR=/crossrootfs/x64 \
mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net11.0-cross-freebsd-14-amd64 \
./build.sh --subset clr --cross --os freebsdCoreCLR builds a few tools, including NativeAOT compiler itself, using NativeAOT (or single file where NativeAOT is not supported). The build defaults to using a "Last Known Good" version of NativeAOT to build the tools. This "Last Known Good" version comes from the .NET SDK referenced in the global.json file. This default was chosen for a good local build experience of most repo contributors. Building with live NativeAOT version would make the local build longer and it would make debugging local changes that impact NativeAOT compiler complicated.
The runtime's build scripts provide an additional set of options to build with the live NativeAOT version instead of the "Last Known Good" version. This is useful for testing changes to NativeAOT or the tools that are built with it, and is required for building those tools for target platforms that are not known to the "Last Known Good" version of NativeAOT, such as FreeBSD, community architectures, or non-portable builds of .NET. This is not yet implemented for Windows.
To build the bootstrap subset of the runtime repo, you can build the bootstrap subset. To use the bootstrap components in the runtime repo build, you can pass the --use-bootstrap argument to the build script. This will use the bootstrap components instead of the "Last Known Good" version of NativeAOT.
For simplicity, a --bootstrap option is also provided. This option will build the bootstrap subset, clean up the artifacts directory, and then build the runtime repo with the --use-bootstrap option. This is useful for building the runtime repo with the live NativeAOT version without having to run two separate commands.
The --bootstrap option is automatically specified when building the runtime repo for .NET Source Build, as the vast majority of Source Build scenarios use non-portable RIDs.
For community-supported platforms where Microsoft does not publish targeting/runtime/apphost packs (e.g. FreeBSD, illumos, Haiku, linux-riscv64, linux-loongarch64, linux-ppc64le), the test builds need to consume the locally-built bootstrap artifacts instead of trying to download packs from NuGet. Both CoreCLR runtime tests and libraries tests support this via the --use-bootstrap flag.
The steps below use FreeBSD ARM64 as an example. The same flow applies to any other community-supported target by swapping --os/--arch and the docker image.
Start the cross-build container:
docker run --rm -it \
-v $(pwd):/runtime \
-w /runtime \
-e ROOTFS_DIR=/crossrootfs/arm64 \
mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net11.0-cross-freebsd-14-arm64 \
bashInside the container, set the target and build the product, then the tests:
os=freebsd
arch=arm64
# Initial product build (also produces the bootstrap subset).
./build.sh clr+libs --cross --arch $arch --os $os --bootstrap
# Subsequent rebuilds (e.g. after editing code) reuse the existing bootstrap.
./build.sh clr+libs --cross --arch $arch --os $os --use-bootstrap
# CoreCLR runtime tests.
src/tests/build.sh -cross -$arch -$os -p:LibrariesConfiguration=Debug --use-bootstrap
# Libraries tests (produces zipped per-library test archives under artifacts/helix/tests/).
./build.sh libs.tests --cross --arch $arch --os $os --use-bootstrap -p:ArchiveTests=true
# A single library's test project can also be built and archived on its own.
# The resulting zip lands under artifacts/helix/tests/.
./dotnet.sh build src/libraries/System.Formats.Nrbf/tests -p:CrossBuild=true -p:UseBootstrap=true -p:ArchiveTests=true -p:TargetOS=$os -p:TargetArchitecture=$archWithout --use-bootstrap, restore would fail with NU1102 errors because the apphost/runtime/targeting packs for freebsd-arm64 are not published to NuGet feeds.
From the host machine, pack and upload artifacts/tests/coreclr/<os>.<arch>.Debug to the target:
tar -czf coreclr-tests-freebsd-arm64-Debug.tar.gz artifacts/tests/coreclr/freebsd.arm64.Debug
scp coreclr-tests-freebsd-arm64-Debug.tar.gz $TargetMachine:/tmpOn the target machine, extract and run a test. For example, the interpreter tests:
ssh $TargetMachine
mkdir coreclr-tests && cd $_
tar -xzf /tmp/coreclr-tests-freebsd-arm64-Debug.tar.gz
CORE_ROOT=$(pwd)/artifacts/tests/coreclr/freebsd.arm64.Debug/Tests/Core_Root \
DOTNET_TieredCompilation=0 \
artifacts/tests/coreclr/freebsd.arm64.Debug/JIT/Interpreter/InterpreterTester/InterpreterTester.shExit code 100 means pass; any other value means fail.
Libraries tests need the test host (artifacts/bin/testhost/...) plus the per-library test archive. From the host machine:
# Pack and upload the test host.
tar -czf testhost_net11.0-freebsd-Debug-arm64.tar.gz artifacts/bin/testhost/net11.0-freebsd-Debug-arm64
scp testhost_net11.0-freebsd-Debug-arm64.tar.gz $TargetMachine:/tmp
# Copy a specific test's archive (paths and names vary; this example uses System.Text.RegularExpressions).
scp artifacts/helix/tests/freebsd.AnyCPU.Debug/System.Text.RegularExpressions.Unit.Tests.zip $TargetMachine:/tmpOn the target machine:
ssh $TargetMachine
mkdir testhost
tar -xzf /tmp/testhost_net11.0-freebsd-Debug-arm64.tar.gz -C testhost
mkdir regex-tests && cd $_
unzip /tmp/System.Text.RegularExpressions.Unit.Tests.zip
./RunTests.sh --runtime-path ~/testhost/artifacts/bin/testhost/net11.0-freebsd-Debug-arm64--bootstrapworks on any platform; on community-supported platforms it is the only way to build the tests because the targeting/runtime/apphost packs are not published to NuGet feeds.- If restore still fails after passing the flag, double-check that the
bootstrapsubset built successfully and producedartifacts/bootstrap/.