logoalt Hacker News

walrus01 • today at 8:27 PM • 1 reply • view on HN

Looking at the link to quantizations it looks like unsloth has already quantized it:

https://huggingface.co/thinkingmachines/Inkling-2-Mini

https://huggingface.co/unsloth/Inkling-2-Mini-GGUF

In terms of the active parameters and token/s on slow or less capable hardware it seems like it should be in the same general performance category as qwen 3.6 35B A3B.


Replies

jakswa • today at 8:47 PM

might need this if trying to test on llama.cpp https://github.com/ggml-org/llama.cpp/pull/32695 -- "Adds support for https://huggingface.co/thinkingmachines/Inkling-2-Mini - uses a looped architecture and this PR adds looped support"

➕ show 1 reply