alt Hacker NewsYou can't afford the best model. What are your specs and what models + quants have you tried?
Qwen 3.6 35B A3B and Qwen 3.6 27B can both do reliable tool calls on Pi at Q4_K_M using llama.cpp
You can't afford the best model. What are your specs and what models + quants have you tried?
Qwen 3.6 35B A3B and Qwen 3.6 27B can both do reliable tool calls on Pi at Q4_K_M using llama.cpp
I'm on a 48 GB M5 Macbook Pro. I use 4-bit quants with a context window of 16-32k. I tried Qwen 3.6 27B, but I can only get around 10 tokens per second, but it's painfully slow, and it often fails during `write_file` tool calls, even with Qwen Code.