logoalt Hacker News

mrashad74 • today at 5:48 PM • 1 reply • view on HN

This is an interesting question, so the idea is, based on experience, fine tuned small models can do narrow tasks very well (my experience running models fine-tuned on customer data as low as 1-2B is very promising) also the goal is to find a breakthrough were instead of the line of work (which I respect) that tries to squeeze much bigger models into smaller hardware like strata and maya, I'm working backward to discover/search for custom kernels and modifications till we exhaust the limit of what a smaller machine is capable of, so instead of people doing 64gb machines and old gpus to run a deepseek, I want to see what is the biggest usable qwen small model can run with reasonable speed on smaller devices, and by scaling as we discover more tricks, normal laptops can unlock more compute - for example a qwen 3.8 27B can do pretty serious work when coupled with a specialized harness - but we are yet to figure out what is the smallest machine that can run it with usable output and usable speed, and this can unlock interesting business.


Replies

yjftsjthsd-h • today at 7:56 PM

I'm told that qwen 3.8 27B is good enough for practical programming work, so if that's doable then I could easily see the utility. Or... the biggest qwen I see here is Qwen3.5-2B; is that actually capable of programming or agentic work with code?

➕ show 1 reply