This is another piece of news in my quest for hermetic, ephemeral, and reproducible builds (“HER”). If you read my articles in the past months or so, you may have noticed that I am looking for good ways of creating completely hermetic, ephemeral, reproducible, but also practical builds.
Issues to solve
The following is a list of issues with my previous HER build approaches:
I do not want to manage a software repository. I think that instead I should be able to reuse an already worked out solution wholesale. Ideally, bringing an existing software repository would be the way to go. The “N+B” approach did this, but even ephemeral nix installation ended up being somewhat invasive, as it required a special installation step which could fail in hard to track down ways.
I do not want to introduce more coarse-grained sandboxing. Bazel has its own sandboxing approach, and any additional sandboxing requires double the care when writing build rules. This eventually can be done, but it sacrifices practicality, when we remember that we aren’t just after a solution today, but we want a solution that will last for the lifetime of our project. “N+B” runs bazel in a sandbox, which has led to issues with build termination. “BID” runs docker containers, which are not available everywhere, and carry a build performance penalty. It is also unclear how it interacts with remote builds.
Proposal
- Use an existing Debian software repository to bring in all binaries that are not otherwise available.
- Create rulesets which bring in the required pieces of software automatically.
- Provide Bazel build targets that represent properly configured such binaries
Approach
Here is how I envision this can work.
Use the existing rules_distroless to create a hermetic and reproducible container image from a preexisting Debian repository snapshot. The reproducibility is managed by using lockfiles which are part of rules_distroless.
Use the existing rules_oci to convert this container into a flat OCI container layer. This is a fancy way to say - a tar archive containing all the binaries we want to run.
Extract this flat layer into a “rootfs” directory: the “rootfs” contains exactly the directory layout that would have become the root filesystem of a “distroless” Debian system.
Create indirection scripts for any number of binaries that need to be brought in. This requires fixing up software and library paths to iron out the version mismatch between the runtime setup of the host system, and the system laid out in rootfs. This can either be done directly, or via local sandboxing, such as via PRoot. Note that this should not break the sandboxing rule above, since only the needed binaries would need to run under PRoot if required, not bazel itself. I would still avoid this if possible.
Create a configuration by which each would-be user can define their own rootfs with pre-made binaries to use.
Of course, if someone provides a different way to use any of the module deps, you can use those wholesale.
Next up… the presentation of the done deal.