alt Hacker NewsCan anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
Replies
[facetious]
many yaks to shave. many bikesheds to paint.
if you're paid by the line, why not go to town...
For one, they have the internal models that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.
RL environments
Engineers are working on multiple sessions, and LLM is doing most of the commits/pushes. This is becoming norm at big techs.
If you are wondering what these changes are, you can take a peek at https://github.com/openclaw/openclaw/commits/main/
This isn't to say that such pace of iteration is good or bad. That's a separate discussion.
At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible
It's the billions of scripts we used not to commit. And billions more.
It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.
Anyone who has fallen into the ADR pit with agents could tell you this. Its just slop, slop to fix slop, docs about slop, slop to fix the docs, etc. 600 commits and 200k LOC/Docs for software that used to be 15 commits and 10k LOC +README and no real value to be found.
Or it's yet another iteration to get another test with overbuilt providence catalogling, boated code, overbuilt nonsense. Suddenly you're 2.5 Billion tokens deep and you just wonder "What the actual Feck did they do, how did they burn that many tokens? It's just a web app with a plugin system, my god."
Maybe it could be faking code and ai usage?
I've seen it on other workplaces, but from what I see from open ai and anthropic, the code shouldn't be as extended.
But maybe I'm crazy lol
What they do not have is a version of chatgpt.com that is not a slow, awkward, glitchy mess.
As a user of Astra, it writes code that appears to work. It does not necessarily write sensible code that is architected in a way that makes sense based on what it does. I just watched it do a major rework of a couple thousand lines of code by keeping the original organization and kludging up an adapter to make old code have new semantics.
I would expect that this approach, coupled with lots of fun asking for changes and none of the grunt work of actually thinking about the result, could result in exponentially increasing amounts of mess.
I haven’t used a recent Anthropic model - maybe it’s better at this.