logoalt Hacker News

LoganDarktoday at 11:51 AM0 repliesview on HN

Optimize llama.cpp? Hmm.

WRT speculative decode, basically zero finetunes keep it. I'm testing with some ridiculous abliterated amalgamation so spec decode has been gone for most of its ancestry.

Fable recommended n-gram speculation so I'm working on that now.