dockerContainers attribute?

[roberth]

Quoting @antifuchs #106 (comment)

Just came up against another scenario where I don't quite get how to set the arguments on a flake part: I'm trying to define dockerContainers, and so (AIUI) I need to use withSystem, as they're not supported by perSystem.

My flake part defining these docker containers does the following:

{
  self,
  lib,
  pkgs',
  inputs,
  withSystem,
  ...
}: {
  flake.dockerContainers.boinkornet-monitor = withSystem "x86_64-linux" ({pkgs, ...}: pkgs.callPackage ./monitor {});
  flake.dockerContainers.boinkornet-media-forwarder = withSystem "x86_64-linux" ({pkgs, ...}: pkgs.callPackage ./media-forwarder {});
}

And the monitor/default.nix calls dockerTools.buildLayeredImage; I'd like the withSystem's pkgs argument to be augmented with overlays, but I don't quite see where to put the _module.args.pkgs in this scheme such that flake.parts can define the container with the right pkgs definition.

Update: The trick seems to be to use perSystem in my flake part and set a different argument (pkgs is already getting overridden with overlays in another part, and setting that leads to an infinite recursion). The following seems to work:

{
  self,
  lib,
  pkgs',
  inputs,
  withSystem,
  ...
}: {
  perSystem = {
    config,
    system,
    self',
    inputs',
    pkgs,
    ...
  }: {
    _module.args = {
      finalPkgs = import inputs.nixpkgs-linux {
        inherit system;
        overlays = [
          self.overlays.default # Pulls in the overlay defined by this flake
        ];
      };
    };
  };
  flake.dockerContainers.boinkornet-monitor = withSystem "x86_64-linux" ({finalPkgs, ...}: finalPkgs.callPackage ./monitor {}); # Uses the packages augmented with our flake's overlay
  flake.dockerContainers.boinkornet-media-forwarder = withSystem "x86_64-linux" ({pkgs, ...}: pkgs.callPackage ./media-forwarder {});
}

[roberth]

Was adding these images to packages not feasible in your case? Does some tooling require a top-level attribute?
The CLI generally looks in a couple of places, so I would think that packages may be nicer to use here.

pkgs is already getting overridden with overlays in another part

Seems like a module that helps with consuming overlays may be helpful then.

finalPkgs

This name happens to be used by easyOverlay, which is for defining overlays instead of consuming them. I'd recommend to avoid this name unless you use that module. The name might come with some expectations for some readers.

pkgs is already getting overridden with overlays in another part, and setting that leads to an infinite recursion)

I wouldn't expect an infinite recursion in this case, but an error along the lines that pkgs is already defined. Maybe something else was involved in the loop.

[srid]

Was adding these images to packages not feasible in your case? Does some tooling require a top-level attribute?
The CLI generally looks in a couple of places, so I would think that packages may be nicer to use here.

I can imagine something like this being useful for a flake-app that looks at dockerContainers (or rather dockerImages which is what they are) and provides various functionality, like:

• Build and docker load -i that image (using git rev as tag)
• Push that image to a registry given login user/pass (knowing the tag used to build the image)

[antifuchs]

Was adding these images to packages not feasible in your case? Does some tooling require a top-level attribute? The CLI generally looks in a couple of places, so I would think that packages may be nicer to use here.

It can work, but the way my flake's packages are handled is that they get added to the flake's overlay under certain circumstances, and checking that with a layered image isn't working great. They're also not packages, but container images, and conceptually it makes sense to put them on the attribute that flakes specifically have for exactly these things.

(Besides, since I add packages to the flake's overlay, I don't want to create an infinite recursion - since one of the dockerImages' layers should include a package pulled from exactly that overlay).

@srid is right about the use case, too: These images get built as part of a CD build process that then uses skopeo to push the resulting image to fly.io's container registry, and then deploys these as containers to fly VMs. They don't get used in my nixos systems at all, but are something like a machine configuration like darwinConfigurations and nixosConfigurations represent.

pkgs is already getting overridden with overlays in another part

Seems like a module that helps with consuming overlays may be helpful then.

Yes - I would like that!

finalPkgs

This name happens to be used by easyOverlay, which is for defining overlays instead of consuming them. I'd recommend to avoid this name unless you use that module. The name might come with some expectations for some readers.

Oof. When you scope out an overlay-consuming module, it would be really good to have it ensure that I don't trivially stomp on other modules' global names /:

pkgs is already getting overridden with overlays in another part, and setting that leads to an infinite recursion)

I wouldn't expect an infinite recursion in this case, but an error along the lines that pkgs is already defined. Maybe something else was involved in the loop.

That'd also be great!

[srid]

How about dockerImages?

flake-schemas is adopting it:

https://github.com/DeterminateSystems/flake-schemas

[roberth]

Is flake-schemas really adopting it, or suggesting it? We should be wary about adding attributes that haven't been validated by actual use in the community. Also we have competing implementations of the container image concept, not just in the container ecosystem, but more importantly in the Nix ecosystem, and they produce distinct package attrsets, with various non-standard formats in their outputs.

In my opinion, it is up to an implementation to define the standard. Did Determinate Systems ask Docker?

Any permitted use must not falsely imply or suggest a sponsorship or endorsement by, or a partnership or affiliation with Docker. We reserve the right, in our sole discretion, to determine when this condition is met or violated.

Maybe they're not litigious, idk, but they're making it a bit more difficult for Docker to use Nix in the way they see fit for their next big thing.

Unlike "normal" packages, a Docker image can be multi-architecture. Flakes also have to deal with architectures. Did they incorporate this aspect into the schema?

So Docker is the wrong name and the wrong tech. It should be named after the specific thing inside, like nix2containerImages or whatever nlewo and the nix2container project think it should be called.

As for Nixpkgs dockerTools, it's most efficient to use the streaming functions, and those aren't images, so it's not quite the right attribute name for those.

Remember that basically everything except mkFlake is modular. Almost anything flake-parts can do, an external module can also do, and probably do it better.

If a de facto standard emerges, used by multiple Nix-based technologies, I'd be happy to merge it into extras/ aka inputs.flake-parts.flakeModules. Otherwise, it's probably best to have it as a flakeModule in the project that's responsible for the interface.