alt Hacker NewsMy bet is that the prices will crash once OpenAI (and/or Antrophic) IPO's have happened.
Right now the biggest threat to their IPO's is that people realize that local models are good enough for whatever they're peddling, what's the most important factor to even running good enough models? RAM since you want the models in memory to not be total slogs.
Replies
This...
Right now the biggest threat to their IPO's is that people realize that local models are good enough for whatever they're peddling...
...plus the recent price increases by AI companies, made me actually think the opposite: that there might be another additional "run" for memory and/or GPUs.
Therefore, yesterday I decided to order an additional RTX 5060 with 16 GiB VRAM for the ~500$ that I saved during the last months (to be added to the RTX 5070 12 GiB that I bought last year to play games in 4k + my old RTX 3060 12 GiB which I recycled a few months ago after noticing how nice it is to run llama.cpp locally without having to worry about subscription costs).
The original 24 GiB VRAM were actually quite enough for some of the stuff that I do (e.g. transcribe text of image scans of old magazines, coding with Aider, etc - I usually use Q5_K_M quantizations of Qwen & Gemma by Bartowski as lower ones delivered sometimes weird results and/or looped forever in "thinking"-mode), but I guess that with 40 GiB I should be bullet-proof for my pessimistic view of our future :o)
You are saying there will be even more demand for RAM and that will cause the prices to crash?
If local models are good enough, doesn't that increase demand for DRAM as everyone buys DRAM for their poorly utilized local machines?
Surely it is a more efficient use of DRAM to run inference on shared hardware with large batch sizes and more utilization.
My bet is that we're not gonna see any adjustments in RAM pricing until one of the planned data center projects collapses in a spectacular way.
> that local models are good enough for whatever they're peddling
they are not. Unless you are satisfied with plausible, but mostly garbage output.
[dead]
But remember that markets can stay irrational longer than anyone can hold his breath. If they get more funding there's a good chance they'll invest more in the destruction of the remaining production capacity. Admitting that with normal pricing anyone could have a decent AI-machine for 2K is hard - prices for acceptable AI-machines most likely will go >10K first.