logoalt Hacker News

anon373839today at 2:15 PM1 replyview on HN

Does anyone have an idea how this might perform on a DGX Spark at longer contexts? I've been trying to investigate their performance with these medium-sized MoE models, but I'm seeing a lot of incomplete and conflicting information. The 273 GB/s bandwidth looks awfully bad on paper...


Replies

rohansood15today at 3:30 PM

Given the new architecture, speeds are harder to estimate. Max would be ~40 tok/s.

show 1 reply