alt Hacker NewsFrom the most unexpected place (but maybe expected if you believed they were paying attention)
Maxsun is releasing a a 48GB dual Intel Arc Pro B60 GPU. It's expected to cost ~$1000.
So for around $4k you should be able to build an 8 core 192GB local AI system, which would allow you to locally run some decent models.
This also assumes the community builds an intel workflow, but given how greedy Nvidia is with vram, it seems poised to be a hit.
The price of that system is unfortunately going to end up being a lot more than 4k, you'd need a CPU that has at least 64 lanes of PCIe. That's going to be either a Xeon W or a Threadripper CPU, with the motherboard RAM, etc you're probably looking at least another 2k.
Also kind of a nitpick, but I'd call that 8 GPU system, each BMG-G21 die has 20 Xe2 cores. Also even though it would be 4 PCIe cards it is probably best to think of it as 8 GPUs (that's how it will show up in stuff like pytorch), especially because their is no high-speed interconnect between the GPU dies colocated on the card. Also if you're going to do this make sure you get a motherboard with good PCIe bifurcation support.