alt Hacker NewsWe should still expect significant performance improvements.
I believe Unsloth’s branch (and ggml’s) don’t support MTP or the ngram embedding yet.
We should still expect significant performance improvements.
I believe Unsloth’s branch (and ggml’s) don’t support MTP or the ngram embedding yet.
The active param count is so small I'm not sure how much advantage MTP will have, the current llama.cpp does load the ngram embeddings but I haven't verified it uses them. I expect to redeploy all this stuff every few days as the tooling gets improved.