logoalt Hacker News

abhikul0yesterday at 5:34 PM2 repliesview on HN

Are you running it locally with llama.cpp? If so, is it working without any tweaking of the chat template? The tool calls fail for me when using the default chat template, however it seems to work a whole lot better with this: https://huggingface.co/Qwen/Qwen3.5-35B-A3B/discussions/9#69...


Replies

sosodevtoday at 1:36 AM

I’ve been running it via llama-server with no issues. Running the latest Bartowski 6-bit quant

show 2 replies
arcanemachineryesterday at 5:51 PM

Have you tried the '--jinja' flag in llama-server?

show 1 reply