2x RTX 3090 is not enough for proper use. ideal is 64GB+ so that you get proper cache and 256k context with good enough models. (e.g. Qwen 3.8 27B with MXFP4). And that is tight already.

You would need 3x RTX 3090 - but then the PCIe bandwidth comes an issue if you really want tensor parallelism with three cards. 3x PCIe5 x16 is not cheap with direct CPU access.

So surprisingly, 2x r9700 starts be a nice deal.

> Also these 20-30t/s jumping to 150-200... Watch out for the massaged numbers coming from vendors.

Well, luckily these are not vendor numbers. Prefill also scales almost linearly with the amount of GPUs.