alt Hacker NewsEDIT: I was completely wrong, I have mostly worked with GGUF and related quantizations that are LUTs, thank you for correcting me.
EDIT: I was completely wrong, I have mostly worked with GGUF and related quantizations that are LUTs, thank you for correcting me.
> The quantized integer kernels aren't running true integer multiplication, the quantization is it's own thing, they're basically enums not integers
ELI-a-GPU-compiler-engineer-working-at-a-major-vendor (because I am). Ie I can pull up the design docs for our ALUs and literally see that you're wrong.