Testing zig solutions is slow on Exercism

[ee7]

82b56af is a significant speedup for a RUN zig test foo command that's added at the bottom of the Dockerfile. But it doesn't seem to be a speedup in production.

Some possible short-term ways to improve:

1. Ensure that Zig can fully use the cache. Maybe some combination of:
   • Set USER (done in b8b9d02)
   • Create the cache later (done in 3f2c0c2)
   • Create cache via bin/run.sh, not zig test directly (done in 0636420)
   • Specify -target (done in b58066f)
   • Change some paths of solution files
   • Look closer at the how Exercism run the container
   • Set zig cache environment variables, or pass zig cache location options to zig test
2. Put the cache on a tmpfs.
3. Update Zig. Maybe commits like ziglang/zig@020105d could help.

Longer term:

1. Use a bigger Zig cache.
2. Wait for https://www.github.com/ziglang/zig/issues/16270.
3. Use the x86_64 backend (currently marked as experimental).
4. Use wasm, like https://playground.zigtools.org/ and https://github.com/zigtools/playground. Exercism doesn't support this for now.
5. Wait for perf improvements in https://www.github.com/ziglang/zig/projects/6
6. Consider trying the C backend (no longer marked as experimental), and compiling with a fast C compiler. Note that the issue for generating tcc-compatible code (ziglang/zig#13576) is currently marked for Zig 1.1.0. Aside: the Nim test runner currently compiles Nim to C, and then uses tcc.

[ee7]

@ErikSchierboom is this and this the current and complete source of truth for how Exercism runs a container in production?

[ErikSchierboom]

Yes, that is correct.

[ee7]

It looks like b58066f helped. See #71 (comment).

Next I'll try to compile more of the zig stdlib and add it to the image's zig cache. (Edit: that doesn't seem to produce a significant speedup).