Building a custom-designed ASIC is much easier than producing state of the art memory chips.
There's a reason why Micron and Nvidia are the crown jewels of American technology right now and for the foreseeable future.
Nvidia does not make RAM
That doesn't excuse them from wrecking the market for ordinary person.
Nvidia buys the memory it uses on its GPUs, same as all other ASICs.
To give some context, Intel started making DRAM, I think they were actually the company that came up with modern memory techniques. They exited the market and pursued a more lucrative moat with CPUs.