There are things like this.
The things I know of and can think of off the top of my head are:
1. appimage https://appimage.org/
2. nix-bundle https://github.com/nix-community/nix-bundle
3. guix via guix pack
4. A small collection of random small projects hardly anyone uses for docker to do this (i.e. https://github.com/NilsIrl/dockerc )
5. A docker image (a package that runs everywhere, assuming a docker runtime is available)
7. https://en.wikipedia.org/wiki/Snap_(software)
AppImage is the closest to what you want I think.
I wish AppImage was slightly more user friendly and did not require the user to specifically make it executable.
AppImage looks like what I need, thanks.
I wonder though, if I package say a .so file from nVidia, is that allowed by the license?
It should be noted that AppImages tend to be noticeably slower at runtime than other packaging methods and also very big for typical systems which include most libraries. They're good as a "compile once, run everywhere" approach but you're really accommodating edge cases here.
A "works in most cases" build should also be available for that that it would benefit. And if you can, why not provide specialized packages for the edge cases?
Of course, don't take my advice as-is, you should always thoroughly benchmark your software on real systems and choose the tradeoffs you're willing to make.