The interesting solution here is secure boot, only allow users to play from a set of trusted kernels.
Yep, a plenty of prior art on how to implement the necessary attestations. Valve could totally ship their boxes with support for anticheat kernel-attestation.
Is it possible to do this in a relatively hardware-agnostic, but reliable manner? Probably not.
What do you mean? Ship computer with preinstalled Linux that you can't tamper? Sounds like Android. For ordinary computers, secure boot is fully configurable, so it won't work: I can disable it, I can install my own keys, etc. Any for any userspace way to check it I'll fool you, if I own the kernel.
You can switch out the kernel in the running Linux desktop.
I wonder if you could use check-point and restore in userspace (https://criu.org/Main_Page) so that after the game boots and passes the checks on a valid system you can move it to an "invalid" system (where you have all the mods and all the tools to tamper with it).
I don't really care about games, but i do care about messing up people and companies that do such heinous crimes against humanity (kernel-level anti-cheat).
I'd be less antianticheat if I could just select the handcuffs at boot time for the rare occasion where I need them.
Although even then I'd still have qualms about paying for the creation of something that might pave the path for hardware vendors to work with authoritarian governments to restrict users to approved kernel builds. The potential harms are just not in the same league as whatever problems it might solve for gamers.