alt Hacker News> Ironic that these large LLMs are eroding Nividia's moat.
That's not what the quoted part meant. During v3 development they only had access to hardware limited variants of H series GPUs. Those had less interconnect bandwidth IIRC. So, at the time, the low-level wizards that ds employed bypassed the official APIs (i.e. the nvda ecosystem) and hand wrote alternatives to say nccl, to better use those limited GPUs. I remember them publishing some of it as well. It had to do with allocating memory, moving stuff around, etc. Basically bypassing some limitations by going lower than the official APIs support.
There is no eroding of their moat, as long as they sell GPUs. ANd they're selling GPUs like crazy. The moat speaks for itself, if I may :)
They talk about developing and using TileLang which they ported to Huawei's 950 GPUs.
And it makes sense, as these LLMs become more capable in coding abilities - people will use them to develop their own abstractions to work on different HW. You cannot have it otherwise. If SaaS companies get threatened that their SW doesn't have a moat why do you expect Nividia's SW to have moat ? The computing algorithms are not even proprietary. It is only a matter of whether someones cares about it and is committed. This should be encouraging for new AI chip development companies.
The very progress that Nividia enables also has a negative feedback that threatens it.