How GLM built its own inference infrastructure

The first part of the article reads like Z.AI is trying to get their piece of the “national security concern” pie.

The way these “AI is too powerful now” articles read about Mythos, Fable, GLM, etc is completely incongruent with my experience using them. It feels like they are all trying to position themselves to influence government policy.

US chip export restrictions may actually be an advantage for China's AI Infrastructure. Chinese companies are forced to speed up developing their own AI chips

We built a complete production-grade inference service from scratch on a cluster of more than 100,000 Chinese-made AI accelerators. All production inference for GLM-5.3-Flash runs on this system.

    "We implemented a series of aggressive memory optimizations, including..."

Interesting that the tone of announcements between US and Chinese providers is converging.

GLM has in the past been more technical rather than speculation about future development on RSI etc.

Also curious whether those 100k accelerators are entirely locally made. If that's genuinely end to end on all components including lithography, memory, design etc then that is quite a feat.

If only this infrastructure could handle all the traffic. I've tried using glm via z.ai - and it's a snail kind of slow.

And at the same time you have pretty strict limits to your usage, so in many cases you can't even let it work all night, as you will reach your limit faster than that.

Time to tackle consumer GPUs next, since I’m not getting that Intel Arc B770.

I might be missing something but when I went to their site they are more expensive than Claude. Why would I pick GLM over Claude? Is it they just offer more tokens in their plans?

Given the huge amount of money being spent on AI chips in the US, what prevents US AI labs from doing the same level of software optimization? It could be a solve for some of the capacity constraints.

Maybe now we can stop posting the nonsense take that the frontier labs have hit a wall and are trying to distract from that for IPO reasons.

Also maybe we can stop saying "we can't slow down because China will never slow down" - I don't really think slowing down is right, BUT if slowing down is correct then maybe we should be talking about China slowing down instead of just saying "won't happen" without any evidence that Chinese labs don't have similar concerns.

I'm not feeling any of this speed optimization; it's dog slow.

Signed, a customer.

> As we develop GLM, the model sometimes exhibits capabilities that surprise us

Creators of known unreliable programs be surprised their programs are unreliable.

Different angle on the same model: the full GLM-5.3 (744B MoE, 4-bit experts, 434 GB on disk) runs on a single MacBook Pro M5 Max with 128 GB by streaming the experts from NVMe SSDs instead of keeping them in memory.

One drive gives about 2 tok/s; striped across four drives it reaches 3.5 tok/s with byte-identical output, and our best internal build with a not-yet-published patch does 4.2.

Method and numbers: https://github.com/argonautlabsai/argodrive (built on antirez/ds4).

[dead]

[dead]

[flagged]

[flagged]

I was gonna ask how people found their coding plans, and realized, have they massively ramped up the prices? Seems the middle plan is ~$80/month now, didn't that used to be like $20/month? Cheapest plan is ~$20/month currently.

They must have hit really hard scaling limits if the prices were hiked so much so quickly.

This is a really funny sounding post. They sound like they just found out that increasing your automation gives you increased capabilities at faster speeds. They also sound like they just realized AI makes hard things easier.

But what really kills me is the idea that these companies are using Python for production inference. I mean really? Have you seen how bloated and slow Python is? Do global locks really sound like a strategy for fast dynamic computation?

Necessity is the mother of invention. The shortsighted protections put on chips, etc., by the US has forced Chinese AI industry to adapt or die. Guess what their response to this fitness function has been? Kudos to Z.ai on their inventions and excellent write-up, which reads like humans wrote it.

Well, other than the infrastructure they got from illegally routing millions of paying customers' requests through Anthropic's Opus 4.8 in a distillation attack...

This article left me with one immediate question: "WTF is GLM?".

Honestly, I have no idea what z.ai is either (I'm aware of an AI-enabled editor called Zed, but that's under zed.dev), so it's a bit presumptuous from them to assume that everyone is familiar with their product...

Plot twist: the GLM optimization agent figured out that it can hack and use NVIDIA GPUs on a US Cloud provider and make the inference 10x faster.

I have a similar approach where I optimize kernels and find numerical differences between the CPU oracle and CUDA kernels using an automated AI agent in a feedback loop. Usually it solves numerical problems easily (it compares outputs of every layer and finds where they diverge), but so far no matter how many different SOTA models I throw at it, and even show it reference code from other inference engines, they aren't able to much the speed (my engine has a modification which is not found in reference code, although a lot of stuff is similar). Either I'm doing something wrong, or z.ai's Infra Agent is actually an agent swarm, i.e. a bruteforce with heuristics. My project is 2 weeks old so maybe I just need more time.