logoalt Hacker News

octoberfranklin • yesterday at 8:43 PM • 2 replies • view on HN

FWIW that is about $331,000 on low-onboarding-grief clouds (e.g. vast.ai) right now using the SXM variants they used.

You might be able to shave that down using a different flavor of H200 (like PCIe) but it would take more hours, so benefits are unclear.


Replies

mips_avatar • yesterday at 11:02 PM

You'd probably end up spending more on pcie h200s. They're not that much cheaper than 8x sxm nodes and they only have 4 way nvlink. At the scale these guys were training at you'd probably want at a minimum an 8x nvlinked node for the trainer, even though you could technically fit the full training and optimizer states on a 4x h200 node.

lumost • yesterday at 9:25 PM

Savings would likely be larger with alternative accelerators such as trainium, no?

➕ show 1 reply