logoalt Hacker News

docheinestagestoday at 5:20 PM2 repliesview on HN

What we really need is a breakthrough in inference or LLM architecture to allow running GLM-5.2-level models at the size of Qwen 3.6 27b or smaller on consumer devices like a 48GB Macbook Pro, and at least at 100 tokens/second. My hypothesis is that a smaller, less capable but faster model paired with a good harness can run for longer and brute force its way out to solve problems that the bigger models can one-shot.


Replies

redwoodtoday at 5:29 PM

That would be great during the winter months

show 1 reply
cyanydeeztoday at 5:25 PM

im more expecting the harness to be a literal LLM, Like how you put vibration dampeners on all kinds of mechanical structures