It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want.

I had a great experience with llama-cpp with Nvidia backend on NixOS.

(Sorry for being that guy.)