alt Hacker NewsTo be fair it's "only" half the throughput of a 4090 and a third of an RTX 6000. Significant but not an order of magnitude.
Replies
lowbloodsugar • today at 3:47 PM
An old ada Rtx 6000 maybe. A Blackwell RTX Pro 6000 is an order of magnitude faster and has 96gb.
➕ show 1 reply
Those are the ratios for memory bandwidth, but the GPUs have a much higher ratio for compute, and that affects prefill rate / TTFT, right?