alt Hacker NewsI wouldn’t say runs. More of a gentle stroll.
Replies
hasperdi • yesterday at 8:30 PM
With quantization, converting it to an MOE model... it can be a fast walk
I wouldn’t say runs. More of a gentle stroll.
With quantization, converting it to an MOE model... it can be a fast walk
I run it all the time, token generation is pretty good. Just large contexts are slow but you can hook a DGX Spark via Exo Labs stack and outsource token prefill to it. Upcoming M5 Ultra should be faster than Spark in token prefill as well.