> IMO, Migrating to small-scale local LLMs would be a significant improvement over using data centers.

LLM serving is most efficient when you batch a lot of parallel requests together. Data center solutions also have the advantage of collecting queries from around the globe, so the hardware can be utilized around the clock.

Having everyone serve their own local LLMs would produce a lot more memory demand. Not less. The same memory would be idle most of the time, and when it was used it would be used for 1 person instead of a batch of requests.

There are other reasons to run local LLMs, but solving hardware demand problems is not one of them.