logoalt Hacker News

jakswatoday at 1:58 PM0 repliesview on HN

Q3 results: unsloth/Muse-Glimmer-30B-GGUF:UD-Q3_K_XL gets down to 15.6GB VRAM and full context (131k) on the 4 parallel slots. Prompt/generation speeds about the same. Overall feeling like a nicer-fitting Qwen 3.6 27B, but want to test out MTP generation speeds once I can.

edit: My favorite bit of reasoning I saw go by in my "generate me a beautiful code snippet" anecdote: 'Could give a snippet of beautiful code: the "hello world" in brainfuck? No.'

edit2: my first dflash speculative model! no mtp. I'm up to ~60 tok/s on empty context with `--spec-type draft-dflash`