GLM builds proprietary inference infrastructure
Chinese AI company GLM has developed its own inference infrastructure rather than relying on third-party cloud providers.
What to know
- GLM has developed in-house inference infrastructure rather than depending on external cloud providers.
- The move represents vertical integration in the AI industry as companies seek control over deployment and serving of their models.
GLM AI company
How it unfolded 1 development · click the chart to see its coverage articlesposts
-
1
-
first by HN Best, 10d ago · also HN Frontpage
-
I have a similar approach where I optimize kernels and find numerical differences between the CPU oracle and CUDA kernels using an automated AI agent in a feedback loop. Usually it solves numerical problems easily (it compares outputs of every layer and finds where they diverge), but so far no matter how many different SOTA models I throw at it…
2 more of the top 3 · 23 posts in this stretch
-
Different angle on the same model: the full GLM-5.3 (744B MoE, 4-bit experts, 434 GB on disk) runs on a single MacBook Pro M5 Max with 128 GB by streaming the experts from NVMe SSDs instead of keeping them in memory.One drive gives about 2 tok/s; striped across four drives it reaches 3.5 tok/s with byte-identical output, and our best internal…
-
Nobody pretended that Chinese firms would just lay back and twiddle their thumbs when faced with import restrictions. The question was whether they would be far enough to be able to catch up without much issue or so far behind that they wouldn’t ever effectively catch up or that by the time they did, it wouldn’t matter.Half-arsed export…
-
What people are saying 20 voices from 1 site · best of 23 · verbatim
- Sep 18
-
Yes, but the lag time is crucial. The NY Times estimated that China has 10% the compute capacity of the US. They may catch up, but by that time the US labs could have too much of an advantage
- Sep 17
-
Pretty impressive to see the amount of performance they can squeeze out of the same hardware. I suspect the same process will play out for all combinations of LLMs, inference providers and hardware stacks. This should bring down the cost of inference for the providers by an order of magnitude in the next year and lead to fantastic margins for…
-
The absolute goddamn hubris of the USA as a whole would be astonishing if it weren't so fucking stupid
-
"Necessity is the mother of invention".If someone is capable of doing something, and your goal is to prevent them for doing it, the worst thing you can do is to make it necessary for them to do it.
-
The first part of the article reads like Z.AI is trying to get their piece of the “national security concern” pie.The way these “AI is too powerful now” articles read about Mythos, Fable, GLM, etc is completely incongruent with my experience using them. It feels like they are all trying to position themselves to influence government policy.
-
Of course it is - how else could Huawei and a number smaller companies compete with NVIDIA.
-
Where the US sees itself penalizing China with an export restriction, China sees the US gifting it with zero-political-cost “protective” import tariff.You can’t really hurt a country that has a culture with a positive attitude toward growth.
-
US chip export winners and losers:Winners: Huawei, SMIC, CXMT,Chinese ASML-competitors, OpenAI, Anthropic, Amazon, Microsoft, Google, Meta.Losers: Chinese AI labs, Nvidia, AMD, TSMC, Micron, SK Hynix, Samsung, Intel.Any company that depends on Nvidia hardware such as OpenAI, Anthropic, AWS are winners. It means less competition for Nvidia chips…
-
Yup. That was really short sighted. And good for China. And actually the overall global market market since supply will augment and competition will decrease pricing as well.
-
Plot twist: the GLM optimization agent figured out that it can hack and use NVIDIA GPUs on a US Cloud provider and make the inference 10x faster.
-
Given the huge amount of money being spent on AI chips in the US, what prevents US AI labs from doing the same level of software optimization? It could be a solve for some of the capacity constraints.
-
If only this infrastructure could handle all the traffic. I've tried using glm via z.ai - and it's a snail kind of slow.And at the same time you have pretty strict limits to your usage, so in many cases you can't even let it work all night, as you will reach your limit faster than that.
-
It was evident that this will happen.> Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically…
-
This is a really funny sounding post. They sound like they just found out that increasing your automation gives you increased capabilities at faster speeds. They also sound like they just realized AI makes hard things easier.But what really kills me is the idea that these companies are using Python for production inference. I mean really? Have you…
-
China themselves recognize this. After Trump relaxed sanctions and allowed NVIDIA H200 sales to China on a case by case basis, the Chinese government stepped in to essentially block it!In addition to Huawei who make the Ascend series that Ziphu are using, there are also at least a half dozen or so other Chinese companies also making their own AI…
-
"We implemented a series of aggressive memory optimizations, including..." This whole thing sounds like industrial scale auto-research, but done by people who actually know what they are doing.
-
I might be missing something but when I went to their site they are more expensive than Claude. Why would I pick GLM over Claude? Is it they just offer more tokens in their plans?
-
US chip export restrictions may actually be an advantage for China's AI Infrastructure. Chinese companies are forced to speed up developing their own AI chips
-
Interesting that the tone of announcements between US and Chinese providers is converging.GLM has in the past been more technical rather than speculation about future development on RSI etc.Also curious whether those 100k accelerators are entirely locally made. If that's genuinely end to end on all components including lithography, memory, design…
-
We built a complete production-grade inference service from scratch on a cluster of more than 100,000 Chinese-made AI accelerators. All production inference for GLM-5.3-Flash runs on this system.