logoalt Hacker News

eurekinlast Sunday at 10:25 PM1 replyview on HN

I'm still in awe that a local 3090 gpu was able to run the qwen3 coder instruct 30b-a3b exl3 q6 and...

Was able to create a sample page, tried starting a server, recognising a leftover server was running, killing it (and forced a prompt for my permission), retrying and finding out it's ip for me to open in the browser.

This isn't a demo anymore. That's actually very useful help for interns/juniors already.


Replies

mrheosuperlast Monday at 2:59 AM

How did you do your setup ?, right now the only way i know how to run LLM is through LM studio.

show 1 reply