Option to run firecracker under jailer

[comunidadio]

Summary

Currently, matchlock launches firecracker directly as m.cmd = exec.CommandContext(ctx, "firecracker" in LinuxMachine.Start.
It may make sense to either default to launch using FireCracker's jailer or to have an option for jailer usage.

Why

jailer improves defense-in-depth for the firecracker process.

How (optional)

--jailer or perhaps this should be the default on Linux (with potentially a --no-jailer option then).

[jingkaihe]

The guest agent already provides the jailer-ish process level isolation.

At the moment I'm leaning towards relying on the pid 1 for isolation (admittedly not as rock solid as jailer), the main reason being there is no jailer equivalent in MacOS.

That said, I've been exploring a more production grade deployment model (disclaimer: completely a throwaway vibe proposal), as a side effect of "pod per sandbox" setup, you do get all the isolation bells and whistles for free from pod spec.

[comunidadio]

I think the jailer-ish isolation isolation done by guest agent is great defense-in-depth but in my understanding, by being inside the KVM, it protects against different potential vulnerabilities than jailer.

Jailer is outside the KVM, protecting the host against firecracker escape after hardware KVM vulnerability.
The guest agent (or guest agent cgroups etc) would likely be oblivious to such exploit (ie. the vulnerability may be triggered by a weird sequence/timing of CPU instructions).

So, given a 0day hardware KVM vulnerability, an attacker may jump directly from guest process to firecracker process:

guest process -> guest cgroup -> KVM -> firecracker -> host OS

With jailer, would need to exploit both firecracker and jailer's isolation to get to host OS:

guest process -> guest cgroup -> KVM -> firecracker -> jailer cgroup -> host OS

.

the main reason being there is no jailer equivalent in MacOS.

As above, I believe this is orthogonal to the issue, Virtualization.Framework may have its own "embedded/Apple-internal" jailer-like security mechanisms, but in the end it's also less likely to be used in high-risk multi-tenant scenarios (macOS server usage in general but more importantly as Virtualization.Framework is definitely less battle-tested than the full Firecracker+jailer solution)

[jingkaihe]

Did more investigation, the main blocker is that jailer setup needs elevated privileged capabilities. Since Matchlock runs without sudo, it is a tricky feature to add.

The only sane way to do it is perhaps run "matchlockd" as a daemon that handles all the privileged capabilities. For unprivileged user to run matchlock, it will communicate over a sock file or https, but this is something way down the line.

I will leave this issue open, as I agree this is a useful feature, but probably a low priority for the time being, as defending against kernel exploit/vm escape is a very low priority if not a non-goal.

Also as mentioned in the thread above, this will become trivial to solve once we have a pod per sandbox type of setup.