logoalt Hacker News

tidbeck • today at 6:07 PM • 1 reply • view on HN

”Serve the model on either two Mac Studios, with M5 Ultra and 512GB of unified memory; or four Mac Studios with 256GB of unified memory, for a total of 1TB of unified memory, enough to hold a NVFP4 quantized version of Muse Spark Open”

Can not find any info on inference speed.


Replies

sourcecodeplz • today at 6:11 PM

its around 20tps