logoalt Hacker News

mercutio2today at 2:48 AM1 replyview on HN

What toolchain are you going to use with the local model? I agree that’s a Strong model, but it’s so slow for be with large contexts I’ve stopped using it for coding.


Replies

embedding-shapetoday at 8:36 AM

I have my own agent harness, and the inference backend is vLLM.

show 1 reply