I don't really like Nix but it's probably fine for this - you have files that come in, a build step, and output files. Your derivations (build recipes) are functions that can take arguments to e.g. provide a set of patches to apply. There are recipes for building a kernel in nixpkgs already which you could fork.

The main annoyance I'd have in your case is probably the number of file copies of large images it may do to keep things isolated, but most hermetic build systems will do something similar.