Why would you need Omlx? For speed up?
Has extra KV cache on SSD, and lots more options to tweak. There's experimental TurboQuant and multi token prediction support.
alt Hacker News
Has extra KV cache on SSD, and lots more options to tweak. There's experimental TurboQuant and multi token prediction support.