alt Hacker NewsI'm working on a variation of MTP that recovers PP TPS (back to the same as with MTP disabled), keeping most of MTP's benefits to TG TPS.
https://news.ycombinator.com/item?id=48700782
Will propose a patch back to llama.cpp or provide it as a fork.
I'm working on a variation of MTP that recovers PP TPS (back to the same as with MTP disabled), keeping most of MTP's benefits to TG TPS.
https://news.ycombinator.com/item?id=48700782
Will propose a patch back to llama.cpp or provide it as a fork.