Qualcomm Cloud AI 100 Ultra – 150W AI Accelerator Card with 128GB at 548GB/s

If the specs seem outdated, it’s because this chip launched in 2024. It’s available at cloud providers including Cirrascale

What's the price?

Curious how the memory bandwidth actually holds up under sustained inference loads compared to the spec sheet numbers.

So it's an M4 Max Mac Studio, RAM-wise, with a GPU's worth of compute?

Sort of interesting design It has half a terabyte of SRAM on the accelerator with a 128 GB of RAM replacing HBM with SRAM but keeping regular RAM.

Seems like a reasonable general-purpose card.

Ok so it’s significantly worse than Apple M6 and we don’t know if it’s cheaper or not

Edit: Ah sorry you’re right, the bandwidth is lower but the TOPS is a lot higher. I’m not sure how much effect you get from higher TOPS with such low bandwidth though.

Aren't these specs already outdated? Seems meh. I would think players would like to bring something new to the table instead of bringing what was okay two years ago or so.

Give me 1TB RAM and 5TB/s or something that exciting.