alt Hacker NewsThanks so much. 27b is a favourite model and last night my dev ran some benchmarks to achieve 133tks about 3.2x which changes the economics.
Thanks so much. 27b is a favourite model and last night my dev ran some benchmarks to achieve 133tks about 3.2x which changes the economics.
great to hear, which quant + KV quant + KV size are you on? is it llama.cpp?