logoalt Hacker News

cmrdporcupinetoday at 6:53 PM0 repliesview on HN

For inference, the prefill on the DGX Spark is like 5x the speed. Decode about the same (both bandwidth limited to about the same memory speed, unfortunately).

Difference is on the Spark you can use and learn CUDA, vLLM, SGlang etc which is the industry standard.

I bought mine (ASUS version) back in December with the intent to learn that stack of stuff. I've been on and off with it but it seems to have paid off, looks like I might be getting work in the inference serving space.