Depends, you buy a used server off of eBay or 3d print your own fan shroud. Waiting for prices to drop some and then I’m looking to build my own rack in the basement, so I can use open weight models and not directly support horrible data center practices.

Not for gaming they don't have monitor outputs and most of them aren't even in a standard PCI-E format. Some of them might be useful as a home AI accelerator with adapters and jerry rigged cooling.

Some of them, sure. You can invest in a server mb, and put a rack in your shed or basement. If someone was selling B100 like that, I'd happily buy a few for 15k. I somehow doubt its going to happen though.

I am not sure how inference constrained we are, btw. As an enterprise customer, I have zero problems with 429s and capacity from google or aws. All large volume inference jobs - so customer journey bots that help our company make money - run on dirt cheap models kept on short leashes in custom harnesses. Message delivery costs us 10x more than inference (with meta price hikes of October). All long-horizon jobs such as coding are probably somewhat inference constrained, but only to the point that sometimes the devs and corporate users with their own agents wait a bit for something to be completed, but nobody is noticing it. Sure, one notices a difference between /fast mode and normal while building software, but nobody really has a reason to care much, /fast modes are a pure luxury thing outside of incident remediation. I don't see what would Anthropic actually do if they had 10x the compute or 100x the compute they are planning for. Surely, some new business models, trolling mathematicians at a larger scale maybe?