> China’s Ministry of Commerce has led meetings over the past month with major AI companies, including Alibaba, ByteDance, and http://z.ai/, to discuss measures that would restrict overseas access to cutting-edge AI models, including models that have not yet been released.
> The discussions reportedly include not only closed-source models but also open-weight models.
> Future regulations could take the form of a tiered framework based on technological capability. Basic open-source AI models may be managed through a filing system, high-performance models may be subject to security reviews, and the most sensitive frontier models may be banned from public release or restricted to use within China
Reporting/news around AI is so very interesting these days.
It's really hard to tell what's propaganda from what's not.
For examples, this Reuters that you have included here has all tells tale signs of creating fear, uncertainty, and feeding into propaganda. But then again I can't be sure.
It absolutely blows my mind that everyone is abundantly aware that money is not shared freely, and yet they are already popping champagne that the CCP are surely going to share power freely.
They won't? They're profiting from every subscription to AI providers, because all companies are state-owned.
They're also well-positioned to control how groundbreaking US AI companies are allowed to appear to their investors, by offering open models that match what market leaders claim can only be attained with trillion-level spend. That's a strong control on US economy, considering how few stocks are propping up the US stock market, and how these stocks are all dependent on the same beliefs and factors.
Also, regarding China's tech investment in general, the five-year plan is just... available to read. It's not a secret. It explains the strategy, and you can draw a direct line from the plan to their now abundant solar infrastructures and tech achievements, including in AI, which is specifically named.
China's five year plan is whatever Xi wants whenever he wants it. It's just a formal guideline, not a law or constitution people can sue the CCP over.
Also, even if China decides it doesn't want to keep the crown jewels of productivity close to home, the US will ban their import. Maybe XzeRo_337 will be torrenting weights and have a VPN to access foreign providers, but Timmy and Ashley are just going to pay for their ChatGPT subs, and Mega corp will pay their Claude Legend token expenses.
China is an authoritarian state with a single leader who has unilateral uncontested control for life.
China has no real bureaucracy (or any other structure for that matter) because at the end of the day, it's one guy who can do whatever he wants whenever he wants. For commoners and officials there is this faux bureaucracy, but for the elite at the top making decisions, there is zero.
If Xi doesn't want models exported, he's not having a legal delegation go to the supreme court of China to fight for his ruling. It just happens, regardless of whatever anyone else or any piece of paper in the country says, and there is zero recourse anyone who doesn't like it can pursue.
More or less authoritarian than Imperial dynasties? Zhao Gao for Qin, Huang Hao for Shu Han, Yang Fugong for Tang, this list can just keep going.
The Ten Eunuchs, if you want one example. Which is to say, their bureaucracies have always worked with single leaders who ostensibly had unilateral uncontested control for life.
https://en.wikipedia.org/wiki/Ten_Attendants
I never thought the chinese were giving away valuable technology out of kindness. I thought having them be free and open weight was part of a strategic move to undermine US tech dominance. I suppose if that was the case something has changed or maybe they simply didn't care until recently
You could just as well read the european approach as a bet that frontier models will be unable to keep a significant edge over open competition (and thus not worth throwing subsidies at, because any economic advantage is fleeting at best).
Looking at the data and related past experience, this looks like a pretty solid bet (despite the "risk" being hard to quantify).
> You could just as well read the european approach as a bet that frontier models will be unable to keep a significant edge over open competition
And that's a bet they will lose 100%. Once the Chinese starts imposing export bans/controlling the access to their models, Europe would be at the mercy of US/China to allow them access or just rely on miserable mistral
Again, you are assuming that frontier models will stay meaningfully ahead long-term. Export controls/bans are pointless if this is not the case.
There is ton of strong indicators that they will not stay ahead: Assuming that technological progress of any kind follows some form of logistic function (where "gains", in this case "intelligence" become sub-linear at some point) is (long-term) a very conservative and proven assumption, and "automatically" negates your lead over time.
Similarly, purely "intellectual" advantage in disciplines like cryptography/computer chess/algorithm design never really stayed concentrated, either.
I would take the other side of this bet. While I agree that the impact of any given advance is likely to resemble a sigmoid curve, I think there is a material chance of "stacking sigmoids" creating something that looks exponential.
To take a simple example, look at the progress of technology over the last ~500 years - it seems to me that the rate of change continues to accelerate despite many of the logistic curves flattening along the way.
There are still huge unanswered questions about whether or not the stacking sigmoids will favor the incumbents. But I would not definitively bet against the people with the most compute data, talent and money.
I'm unsure what exactly you mean by sigmoid stacking.
I'd argue that a lot of important technologies (like circuit design!) started at basically zero in the last century, and the progress was actually exponential for a significant time (=> because we started so low on the curve).
But if you reduce things to a single metric (wealth per person? total energy available to humanity? global industrial/construction output in tons?) I can't think of anything where such "stacking" successful subverted the sigmoid trend (or looks like it will, long-term).
It is not about intellectual advantage, but about capabilities as well. Assuming that Europe would get all the knowledge they need on the cutting edge technology on how to train a frontier model (which they won't because US and China would guard this as national secrets), who is going to setup infrastructure to train the model? Who is building the data centers (multi-year project) and who is going to build the energy infrastructure? Just because you know how to do it doesn't mean you have the capability to do that as well.
One of the best example is nuclear reactors. By now the know how and technology is fairly mature and open, but not every country gets to build nuclear reactors. Same would be with the frontier models as well.
EU should have already started investing in the infrastructure side in-case they obtain the know how, but your politicians are still bickering on pension reforms and Ukrainian war, etc.
They can just operate and provide normal access to their services, just block AI access. This is already happening, apple would not release new Siri in EU (granted it's due to a regulation clash) but this would be a testing point.
If Europeans are still paying the same prices for sub par services/products to their American counter parts, it's win-win situation for those companies. Just sell a dumbed down non-AI version of service/product in EU for an inflated price.
Meta's FAIR has several R&D offices in the EU, yes. So you are saying their labs can conduct R&D on models in the EU, potentially even train them there, they just can't have production LLM inference serving or release the model weights? I'm just not seeing it.
A not-insignificant portion of the AI/ML research community is in the EU.
What you’re failing to understand is that meta is ultimately a US company. If meta unveils a powerful frontier model tomorrow and US impose a ban of its exports, regardless of how many research, data centres, meta has in EU, they will restrict access to the model for EU citizens, same way Anthropic did.
Regarding the open weights, I don’t see meta doing that for their future models, especially once they have their own frontier models. Open weight models are kinda marketing strategy where they use it as a bait and switch. A lot of Chinese companies became popular with their open wight models and once they build that reputation they have no incentive to keep on releasing new open weighted models
Pretty sure there would be a number of EU states who'd happily pump some money in if the EU were to mandate the ECB to (incentivise intermediary banks to) buy their corresponding long dated sovereign debt cheaply to fund it.
Why? Worst se we are 10 years behind them. Second mover advantage will also be there, it this is so critical indeed.
What happened to China because they were third movers in the race to the nuclear weapons? Nothing. They were pretty behind for a while technologically. The wheels have turned. Stop looking at the world as a short strategy game match, or like a Hollywood movie where everything is all-or-nothing. Many parts of south America are way behind in IT, automotive, space, yet many people have meaningful and happy lives there.
I'm yet to accept that the power-block I'm living within being first in the AI race provides a meaningful life to me.
EU's leadership doesn't have much power that way, the EU is a thin layer on the countries.
In the US the federal government has much more power but nobody is attributing the term good fortunes of the AI industry to the recent federal policy there. (There's long term stuff that does make a difference in hoovering up the global R&D talent and concentrating capital, but that's a different kettle)
There's very little betting on any particular AI models going on by the commission/EP. The Pleias article claims a 2020 eurocrat whitepaper determined how things go, but that's fantastical.
Makes sense. AI is a munition now, and nations tend to restrict them. Eventually there will be international open source communities building open weights (all you need's a few smart people and a lot of GPUs for training), making the restrictions moot. Until then this will become the norm.
1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples.
3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time.
4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
It's nobody gets fired for buying IBM all over again.
There's a huge case of survivorship bias when trying to recall historical analogues, because in every instance where margins collapsed and competition made the industry a commodity business, the big proprietary names are no longer with us. Here's a selection of examples, though:
1. Memory chip margins collapsed so much in the 80s that Intel exited the memory chip business entirely. At the time, they were known much more as a memory chip company than a microprocessor company.
2. Margins for high-end workstations collapsed in the face of cheaper IBM PC clones and an explosion of MS Windows software. This led directly to the deaths of SGI, Sun, Symbolics, Lucid, LMI, etc.
3. Proprietary UNIX variants like HP-UX, IRIX, AIX, and SCO Unix have basically completely died out, replaced by lower-cost proprietary OSes like Windows and MacOS, or by open-source descendants of Linux and BSD.
4. Many commercial database vendors like Oracle, dBase, Sybase, FoxPro, and Microsoft (SQL Server and Access) found themselves very much under margin pressure from PostGres, MySQL, and SQLite. Oracle survived thanks to their massive installed base and legal department, and Microsoft survived because they could cross-subsidize from their OS and Office monopolies, but dBase, Sybase, and FoxPro are no longer with us.
There's a really noticeable difference in time frame covered in your examples (80s and 90s) and the one in the comment you're replying to (2010s and 2020s).
Is that just two people with different go-to examples? Or is there something going on here?
(I don't mean this as a leading question to some conclusion in my back pocket, I genuinely have no clue.)
It was accelerated further with things like anti-circumvention clauses in Free Trade agreements (see Cory Doctrow's recent highlighting of this: https://pluralistic.net/2026/01/01/39c3/) and then had more gasoline thrown on the fire in the ZIRP/easy money era post GFC, culminating with the bazooka of stimulus unleashed post-covid.
My best guess is we are now going to witness ~20 years of slow unwind. You can already see signs of this in things like RoW/EM stocks outperforming the S&P, treasury yields diverging from other "safe haven" soverign bonds (e.g. swiss), gold price rising, Europe starting to get serious about addressing the Draghi report's findings, European defence spending increasing, China starting to act like the "adult in the room" wrt the recent Iran/US blow-up etc. Essentially, countries/blocs attempting to re-assert sovereignty that has been willingly diluted over the last ~30 years to mainly America's benefit.
Somewhere else in the comments here, someone else remarked "Individuals perhaps [move to the new models], but not organizations."
That's illustrative. The mechanism by which organizations are forced to update their technology, move to more competitive suppliers, and cut costs is a recession. In one, every business that doesn't do so goes bankrupt, and what's left are the more efficient businesses that have adopted technology effectively.
We haven't had a real recession since 2009. (2020 was an odd case, because it was effectively brought on by government edict and so it actually killed a number of efficient but unlucky sectors while doing nothing to clean out the dead wood in major corporations). The next one is likely to be a doozy, because the economy is filled with bullshit jobs, bullshit corporations, and bullshit products.
This argument is so flawed in many ways, economies are built by people for people. Extrapolate the numbers, let's say in the COVID Pandemic a country, take USA as example, has 10% percentage of their population killed by it, would that be better to the economy?
There's multiple ways to look at the economy, the raw exchange of dollar currency in a debt chase (shit I need to run faster & pay down this stuff!), there's the productivity of industrial output (shit we need to sell more junk and useless crap!), and there's the stock and productivity in optimal life cycles (Damn, that Nokia is a tank!)
If 10% of the population went away, it would affect 1 & 2, but in any true practical lens, there's a ton of cheap empty houses, while on the other hand building repairable stuff that lasts or enough cheaply is where economies move to more complex technologies by saving time and effort in useless endeavour of debt chases or consumption-oriented wasteful productivity
Not OP but, EU economy is being squeezed by China on the industrial and tech front, and by the US on the innovation/startup front. It is clear EU is no longer at the technological/economic frontier like it used to be 10-20 years ago. At the same time there are serious demographic, budgetary and political challenges all across the continent. Dragi's report covers some of these. It feels like the whole system might fall into a crisis soon if measures are not taken
We first became powerful because we did the industrial revolution before anyone else, and used more of that capacity to fight the world (and win) than to fight each other.
When we fought each other, after the industrial revolution, that was the Napoleonic Wars and the two World Wars.
> and ever since the creation of the EU it's been becoming less and less important on the world stage.
I wouldn't say it was "ever since the creation of the EU", but rather "roughly between WW1 and decolonisation". Post-Cold-War the EU has taken over from the former global importance of the member states, e.g. https://en.wikipedia.org/wiki/Brussels_effect
That said, east and South Asia are regaining their multi-millennia history of being the world's dominant power by virtue of having roughly half the total world population.
And to agree up-thread, there's plenty going that can rapidly turn the EU's economy into a disaster if not handled expertly.
There were a lot of European wars between the Napoleonic Wars and the First World War. I agree with your point overall but there was a lot of fighting each other during that timeframe.
Sure, but until then, proportional to each of [population, education/educated workers, capital, instantaneous industrial base, energy supply].
Asia's diverse, but I'd say they seem to be doing pretty well with rapid improvements across all fronts.
In comparison, the US's weaker (not weak-weak, just weaker) areas currently seem to be educated workers, instantaneous industrial base, and energy supply (relative to rapidly growing demand from compute); while the EU's weaker areas currently seem to be capital and energy supply (from supply shock, as it doesn't have the compute). The US and EU both have coming demographic issues, but not as soon as the other stuff becomes more important. People talk about China having demographic issues too, but they're a dictatorship, they can make it shift if they care to.
(And Russia's losing a lot of people, more educated people, capital markets, industrial base, and energy supply).
The big drop in Chinese fertility is going to be very disruptive in the near future, because it is much less gradual than European trends and the retiree/workers ratio is going to spike much harder because of that.
Having full authoritarian control is not gonna change anything now because it is already much too late (action would have been required like 35 years ago).
Best they can do is get through it somewhat smoothly.
edit: This is an even better visualization (projected working age population fraction)
The motivation that the USA entered WWII was not because they were generous, but because the 3rd Reich was effectively becoming a big European nation, so they had to do something to avoid it. A unified Europe is a thread to the USA and Russia and maybe somebody else too.
So you’re saying that the war-ravaged Europe of 1946 that was split by the Iron Curtain and needed Marshall Aid was more powerful and important than today’s EU?
Insane take. But somehow people will go to any lengths to disparage the EU.
More likely it just slowly declines like Japan. Or if anti-migrant sentiment continues growing at the current rate, it breaks into a race war when the AfD and PFN win the majority of votes in Germany and France respectively.
Including the warmongering angry midget next to the US, EU, and China is funny. Russia's economy, before they decided to shoot themselves in the face, was the size of the Netherlands. Whether they are in a recession or not is irrelevant to anyone but them.
Still, the clock is ticking. I don't know of any "new" company that would use Oracle instead of e.g. Postgres or would migrate to it. That's probably why they're pretty desperate to jump onto something new before the old source of income fizzles out.
There’s a difference between proprietary software thats highly profitable maintaining a stronghold over “cheaper” options and a massively overvalued and artificially inflated ecosystem having to confront economic realities.
It pivoted to server market and people who didn't think running linux was professional enough. Workstations were being held afloat by that pivot. Also they were general purpose instead if limiting their workstation market to just one niche.
Also cases where both happened, eg, Xerox wasn’t wiped out but copiers now have multiple vendors at the high end and have commodity via other brands at the low end.
Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them.
The UX is the same regardless the provider. You send in a prompt, it spits back an answer.
In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with. The transition is basically updating your current harnesses to know about the other models.
I think the comparison most apt is the rise of AMD. Sure, it never(?) achieved market dominance, but it did ultimately make a huge dent. And a big part of that was because AMD x86 was pretty close and pretty compatible with Intel x86 at a fraction of the cost.
If you're developing on top of LLM APIs directly, this is definitely not true. There are differences in how context caching works, in what's available through native harnesses, the types of tools you're fine-tuned on (GPT uses apply_patch while Claude uses edit, with different formats), the API surface (Agents SDK, Responses API, Managed Agents), cost structures, and best-practice guidance all around.
Not to mention the meta of account limits, billing, ZDR contracts, etc.
It really depends on what you're doing, but most LLM usage and agentic runs are pretty interchangeable in my experience, and it's usually trivial to switch.
If anything, you're better off supporting multiple LLMs as backup because most model providers have been so inconsistent with working all the time
Dude it’s not trivial to switch because the behaviors are different!
You’re clearly not building a product based on an LLM.
I’m still using various old Anthropic and OpenAI models for products I’ve built and released because I can’t risk the behavior changing in unpredictable ways and the users being pissed.
It’s much easier to switch out some deterministic software than an LLM which you’ve spent a ton of time on testing and benchmarking and understanding its nuances. Changing it is like replacing an employee who’s critical to the business.
The behavior of a single model and version can and does change. There’s not only built-in stochasticity, but closed hosted models like Claude are tweaked and changed all the time.
For the public facing consumer functionality I have Gemini Flash running on guardrails directed by a state machine that calls it statelessly everytime. For that, it's strictly locked to a version. I can't afford to suddenly get responses that the SM is not tuned for.
As for which model does the building... I'm not at all attached. Enough logic, and CI gates/tests live outside the whims of the LLM to be able to hotswap them any time.
I don't think they are saying it's trivial but compare say for example switching an organisation from Office or Windows the example that started this. They are not even in the same ballpark.
Makes sense but honestly if you've spent more time testing and working around the nuances to build consistent experience doesn't it mean you actually need more standardization to easily switch models if/when your trusted model is not viable for you provider?
Exactly, as in, really, will they? Where and at what price, especially across an actual enterprise that needs to deploy them to lots of devs? There's much more than just the actual model.
Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally with a bunch of open weight models.
I can run super tiny models on my 8GB NVIDIA card. They all suck (I have to use <=~5GB models if I want "usable" ~250k context that doesn't need to use system RAM and CPU (which makes things super slow).
I've also tried a GLM 4.7-flash, which even though it's super slow (in comparison) with ~250k context and it just doesn't cut it vs. the Claude Sonnet or Opus I get to use at work. All the while these are all touted as "totally usable, Claude/ChatGPT killer!" replacements.
It's just not "there" with tool use or building software for that matter. Like, just a simple Claude "web search" fails with it. So I asked it to build itself its own "web search" functionality and it just couldn't. It made so many mistakes its just not funny any more. And it couldn't recover from them either. I retried a few times (as I didn't have python installed and it wanted to implement it using that - this happens to be new system - never mind other attempts). I spent as much time doing this (and failing) as I spent building an actual full feature at work last week w/ Sonnet.
If it can't build itself a simple web search to .md file tool/skill, how am I supposed to trust this with actual coding? I'm used to being able to point Claude at our large code base and essentially work with it like a junior doing my bidding. Maybe 5.2 is a killer game changer vs. what I was able to try out (if slowly) but you really have to show me to convince me at this point. And not with synthetic benchmarks. In those, all of the models I tried are supposedly super awesome.
4.7 Flash is a small model that's almost a year old, which is ancient. And yes, your dinky GPU will not run anything worthwhile.
Just spend $5 on OpenCode Go and give GLM 5.2 a shot if you have the time. It's not quite as good as Opus, but it's more than good enough for many tasks.
The $5 is so they can see if open weights models are worth using, not so they can use it for a month. (Which you can't; The quota runs out way sooner than a month for any serious usage. Still worth the price of entry.)
If you use DeepSeek v4 Flash as a daily driver, with an occasional usage of DeepSeek V4 Pro and Glam 5.2 when necessary, the monthly quota practically never runs out.
Getting the pay-as-you-go plan from DeepSeek is also a good alternative. When motivation strikes you never get slowed down by quota, and it's cheap enough that even with mostly DeepSeek V4 Pro it's price-competitive with a $5/month subscription. Depending on how bursty your usage pattern is it might even be cheaper
True, but OpenCode Go gives 6x tokens on Flash and 1.5x tokens on DeepSeek Pro. After exhausting the monthly quota, Flash price is the same as directly from DeepSeek, while Pro is 4x pricier.
This is the conversation I plan to have with Okta sales soon. Wait till you see how easy AI makes it to switch to Entra ID or anyone else. It’s tedium not even problem solving.
My problem with the SSO providers is not the technical part, thats "easy". Its the coordinate with the 200+ external and internal vendors / support to redeploy the SSO part which is time consuming. I always say its a ~3 year project, which can be done in 6 months with the right amount of resources, especially if the platform has been running for years.
Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel <> Libre but still not "cheap and easy".
This is just another example of the bitter lesson. In a year a model will come out that will make none of these model specific optimizations you made matter.
Unlike all your examples, switching out an LLM is both cheap an easy.
Rolling out AI access in a large business is still hard, especially if you're trying to do it safely e.g stopping people throwing all your company data including user PII into a chat for productivity reasons.
It's more a staff training and guardrails issue than a choosing which LLM to use issue, but I imagine picking an open model like GLM would make it harder because the 'enterprise stuff' will be missing.
Plugins and skills are completely trivial to move and most work with any model. What is not trivial are Anthropic's new managed agents vendor lock-in offering.
Is it? I switched to Kiro and it's essentially identical.. well a bit better because you get a better idea of what the harness is doing, but otherwise identical.
I don’t exactly see orgs lining up to switch (and train) their employees between claude desktop and codex and whatever copilot is doing. There’s probably some inertia to those harnesses/integrations on top of the llms themselves.
Most large orgs do not need to train end users. They just need to add glm-5.2 to their router and their in house harness will pick it up. Then slowly limit usage on anthropic models and people will swap willingly. It's a simple /model command in every harness.
Yeah, most big orgs are pushing the idea of 'whitelabel' LLMs. Even if they choose to hang on to Claude Opus, they won't name it, they'll just call it the 'extra mode' and 'auto mode' will eventually switch to a local LLM in their harness.
The inertia is legal and financial. People are paying Anthropic through AWS accounts because the simple reason of not dealing making new contract and legal agreements is enough of reason of the inertia.
But, eventually, I’m quite sure that AWS will also provide open models with those contracts without any inertia. Copilot is already offering Kimi.
My company has a deal with Devin and they provide new models all the time, and open models are becoming the most used ones by our internal metrics, especially because the company is very worried about cost.
Say you have 20% of usecases that require the more expensive model — but in 80% you could just use Llama instead of Sonnet (eg, for basic queries of a document). That saves 80% of that 80%, or 65% of your total bill!
That is the kind of “swap” that’s likely to occur in automated tooling as pricing pressure kicks in — “can you save 65% on our AI bill by switching Bedrock over in 80% of uses?”
Bedrock is really out of date with the models it offers, to the extent that I'm not sure they even have plans to update what's on there now they have the deal with Anthropic. They're still offering Qwen 3, not even 3.5 and certainly not 3.6. GLM 5 is the newest z.AI model they have, when it's 5.2 that would be the one to worry Sonnet.
There are some ok models on there (Qwen 3 Coder Next is usable and fast, for instance) but the lack of updates in a fast-moving field makes it something I don't want to recommend to my org.
Maybe for a fantasy of legal liability of output produced. I haven't heard of any LLM corpo being held liable for any output they generate. Even NYT lawsuit is going nowhere for 3 years in courts already, despite being the most grounded.
What would "training" even entail for that? As far as I can tell, using these tools directly is basically identical in terms of what you need to know. If you happen to have a bunch of custom configurations, maybe you need to invest some time into porting them, but it's not clear to me why you think that anyone would need to be trained if they spent months using one tool and then suddenly had t switch to the other.
What "training" do you have to do to get a professional developer to switch LLMs or harnesses? Its literally just download the other one, point it to your code base and start typing into that text box instead of the other one.
Switching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self host a larger model that requires beefier hardware to host, plus all the complexity to scale that out to a bit internal user-base
Most of the AI companies have OpenAi compatible API's, so you just get a subscription from another provider and change the URL that your LLM Agent Harness uses to talk to the AI.
I use OpenRoutet which lets you switch between providers (Anthropic, ChatGPT, Z-AI) whenever you want. Sometimes I'll have two different models from different providers evaluate each other's answers.
They don't just need healthy margins, they need to make back almost a trillion dollars in a couple of years. Comparing that to elastic search and redis doesn't make much sense.
Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers.
Similar with Windows and macOS. Extremely high cost of switching to something different, if possible at all.
Same with office. Extremely high cost of switching due to compatibility issues and retraining of staff.
Your post primarily shows: It's all about lock-in. So far, it doesn't look like LLMs have any of that. So I don't think your points are valid here at all.
The companies don't necessarily need to make back $1T, the investors do, and those investors don't require $1T in profit to do so, they need an asset worth $1T.
Considering leaks suggest Anthropic's ARR would be $47B, that'd be a 20x valuation, but it wouldn't shock me if Anthropic doubles their revenue in the next year or two, in which a 10x revenue could easily support a $1T valuation, and boom there's your ROI, but considering they've raised $135B total, and their ARR is 30% of that, I'd consider that a pretty good ROI, especially if growth continues.
Wait what? Why are you measuring valuation as 20x revenue here? If its a public stock (which is what anthropic plans to be soon), it doesn't matter. Otherwise spacex's valuation should be... 18.67 billion x 20 by your logic but its current valuation is over 2 trillion dollars right now.
ARR literally doesn't mean much in terms of how these companies are valued by investors and it will mean little when it goes public. And yeah 10x their revenue in a year sure but they will also likely 10x their costs if they want to keep scaling
I was arguing a 20x ARR valuation based on a simple 'potential' justification for $1T.
If I was to go further into that, I'd say that Anthropic has grown from $9B ARR Dec 2025, to $47B at their Series H.
I'd say that Anthropic is still a growth stock, so their $1T valuation is based on expected ARR/growth over the next year, and if we assume a double in ARR (justified by their supply constraints as proof of demand), that's 10x Valuation to revenue.
We could consider valuing by P/E, but they're in a growth stage so that's a waste of time, hence why investors focus on growth, and hence ARR growth is hugely important. If they managed $100B ARR, the same P/E as other top software companies by marketcap, they'd fit in that lineup.
If Anthropic was to hit $100B ARR, they be in similar ratios of ARR:Valuation to Meta, MSFT, Apple, etc. If you assume per token price reduces, and 'per intelligence' prices to reduce, which bullish investors would, you'd also assume a good margin over time, (which rumours appear to support for Anthropic).
The losers are quickly forgotten. Palm, Blackberry, AOL, MySpace. Yahoo, etc.
Software gets replaced all the time too, you even listed one and didn't realize. 15 years ago you'd call office irreplaceable, now you have to add gsuite to the mix, in 15 years there might be others. I know people that have never had office installed on their PC and use spreadsheets daily.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Of course. But why pay $25 per million tokens for sonnet when you can pay $3 for GLM? Both probably running on AWS/Azure/Etc. under some third party.
Individuals, sure. For enterprise you can't get monthly plans. You have to pay per token.
It's a bit like saying "nobody pays for Microsoft Office". I certainly don't know anyone personally who has. Students get a free Education License and then your employer provides one for you...
Raw cloud computing costs have a fairly small margin over what they could be. Nobody with real purchasing power is paying anywhere near the listed retail rates. It's part of the reason smaller providers e.g. OVH, can remain quite competitive.
The other items have very strong lock-in and capture ecosytems. Microsoft Office is the first and only office suite anyone uses and its cheap enough for nobody to consider a real alternative. Microsoft could attempt to charge $10,000 a seat and while some will certainly stay, others would look for an alternative. But for just $10 a month, its a fair price to pay.
1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the cloud is prohibitively expensive. Furthermore, large businesses have sweet discounts.
2. ms office has immense networking effects due to its formats being quasi standards in many industries. try sending an odt to a government entity. As for gsuite, it uses open formats but it's classical google fashion a large suite of software bundled together and not that expensive for what it offers.
3. Linux is not a free alternative if you're a business, you still need to pay someone to support the computers with linux on it, and operating systems have the strongest network effects ever. Linux also has no stable ABI so one can't easily deploy third party software for it.
What's the LLM moat? Codex is OSS and Claude has gazillions of alternatives. Cursor is a nice app but it's a bunch of patches on top of vscode, a team of 5 people can vibecode it in 6 months.
Linux has a very stable userspace syscall ABI. About as stable as Windows, and much more stable than MacOS or the BSDs. I agree with everything else though.
Yeah, Linux-the-kernel does have a stable ABI indeed, but this is not relevant for most ISV desktop software out there. In my comment above I was referring to Linux-the-OS (aka GNU/Linux). The userspace libs don't have a stable ABI at all, and this is a widely discussed problem. Other operating systems built on top of Linux-the-kernel don't have this problem, Android has a really stable ABI.
Most people don't directly call Linux syscalls though but go through glibc. It might even be unavoidable if you want to ship desktop apps as the library will use it. If it's that easy there wouldn't be Python's manylinux, flatpak base packages or Steam Linux runtime
A lot of those things you mentioned have sticking power because they’re familiar to folks and migrating to something else is a big deal.
I can’t imagine most people would be able to tell the difference between Sonnet and GLM 5.2. If the infrastructure around the model you’re using doesn’t change, then swapping models is extremely easy.
I agree with swapping models making it easy. With openrouter, I just change the provider. With reasonix harness, cache hits are basically free. And that's with unsubsidized American providers like Digital Ocean or cloudflare.
Indeed, as it gets more commoditized it feels more like swapping electricity providers. Who cares whether you get your electricity from IBM or the state of Texas? An amp is an amp.
That's an interesting question. What if we did care? Is this amp from burning dinosaurs or from the sun or from fission? What if we could tag power as coming from oil vs renewables? how would that affect our habits?
We care indirectly through cost. Hydroelectric, solar, or wind power are often among the cheapest electricity sources, for example. Beyond that, no we don't care. That's why if people want change we leverage policy on cost, via subsidies, surcharges, taxes, tariffs, what have you.
To a consumer, an amp remains an amp — so they get the cheap one.
I'm using pi-coder with just the free-tier models I can get on openrouter / opencode / kilocode. When I run out of quota on one model I often switch to another model in the same session, and it generally works just fine.
To switch an LLM you just need to open another browser tab and type in your chat query. You cannot say the same for any other kind of software. Traditional software have a large switching cost which acts as a barrier.
LLM providers are like airlines. You only need when you have travel and most of the time you go for the cheapest one. Maybe LLM providers should start providing reward points :) .
In these examples cost of the solution does not generally scale with the use of the solution in the same way we see token use. In the case of LLMs the cost of use scales very differently than seat licensing.
Many corporations have found they have a new cost center drawing tens of millions or more with little direct evidence of productivity gain. Corporations are probably best positioned to either switch providers, leverage router solutions or at worst use the fact that they could to drive prices down from the proprietary providers.
Hyper scalers have a decent margin from a small number of their services and a much more normal if not a loss from many others. Additionally a massive part of their profit is support services and contracts.
They also benefit from the fact that developers do what is convenient for themselves and not what is necessarily computationally efficient (i.e. not pay attention to cross AZ egress/ingress, run an apache spark job when it could be done all within a normal database, build their entire product on irreplaceable/unswappable cloud provider specific databases and storage solutions).
AI will also experience a significant margin collapse, it's just not clear who will eat the brunt of it yet, the AI companies themselves or companies like Nvidia as more chip manufacturers/designers come into the arena and can meaningfully compete.
When a hyperscaler is viewed as a software company their stock and value multiplier is much higher than if they are viewed as a commodity with expensive infrastructure costs. There is now not enough compute resources to serve demand. It requires sustained capital to grow compute resources. The costs are uncollapsing due to the overall demand plus the pressure of LLMs. Capital costs matter.
The target audience is different. Coding is mainly a trade of the tech savvy, who like many on r/localllama users do not hesitate to deply on 16GB Vram gpus. Even if so, it is estimated that within 2 years we will be able to run Claude 4.8 on consumer hardware give the rate of improvement of open-weight LLMs, which will put more financial pressure on "paid" labs. It's just a matter of rate of improvement which is shrinking between open-closed models.
Nobody got fired for IBM, but it took some battles for IBM to reach that level. Same with AI. Brand images won't develop until the street battles are over and dust settled. Otherwise, Google wouldn't have taken over Yahoo and ChatGPT would have remained the king. That didn't happen. The street fights are still raging in AI and won't settle down any time soon. Cost-concious usage can kick-out Anthropic overnight. Ultimately it's only the cost that matters and that will blunt all other factors, including security concerns and risk aversion etc.
Also, note that even the highly-regulated sectors invited opensource products and services and allowed data transfer across their network perimeter. That required "blunting" of the security policies, and it did happen.
> I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Intelligence has diminishing returns, the analogues are with humans. It's a waste to hire Albert Einstein for $X million to operate the cash register in a gas station.
Artificial super intelligence will not have many customers.
1,2,3 are dominated by platform stickness or even active lock-in.
Can't say I see the same advantages to stop you switching the model you use.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Sure. Though it does depend on whether you need regular updates. If you want the model to be aware of the latest research - then fine. However it already does the job, you might prioritize stability over constant change.
> It's nobody gets fired for buying IBM all over again.
Except they when they did when IBM was no longer good value for money.
> but I don't see any historical analogues
None at all? You mentioned IBM - who is using AIX on IBM hardware in 2026? Who is using Solaris on Sun hardware? It's pretty much all gone to linux on commodity hardware.
Remember Netscape - thew browser company? Killed by Microsoft bundling of IE.
How hard would it be for Apple to bundle GLM based services?
> 4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
Elastic just had round of layoffs[1]. Elastic still runs operating losses till Q1 2026 [2], albeit a small one, however just breaking even in operating income is hardly "healthy margins". A P/E of 17 is not exactly signaling confidence given they are growing ~20% y-o-y.
The point in the article that you are not considering is how easy it is to switch to a different model provider right now.
You literally change a couple of env variables and you are done, your user experience is basically the same. I can try new models for an hour and be sure I can go back to the original model as quickly if I want.
That is not the case for the software you talked about. They all require way higher switching effort with more perceived risk.
I suspect the concern is that model serving is a stateless “simple” problem.
As yet, no one has identified a reliable moat in inference. If the moat is performance, then prices will collapse. Unlike traditional cloud moats around state, operations, and capex management - I can host a model reliably with less than 30 minutes effort.
I don't think the parallels are quite as clear for a few reasons:
1. Lock in - with an LLM, there is practically no lock in because of the inputs & outputs being text. You can move easily
2. Motivation - I think you underestimate just how high some of these bills are for companies. Finance departments are already getting mandates to reign in spending even at the high level of subsidization.
3. Political Meddling - we're now at the point where the US strategy seems to be to artificially limit access to powerful models. If China continues its trajectory they will have models as good as Fable in 6 months to a year, and they won't lock it off. So cheaper, better models that are available is a massive incentive to switch. China is much less motivated to ratchet prices up if it's winning them marketshare. I do think David Sacks + AI strategy for US Gov are being very short sighted and it's going to blow up in their faces.
> 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
Cloud opposes switch inertia. To setup a complex system in a different environment is a complex operation. Changing AI provider is switching an endpoint.
> It's nobody gets fired for buying IBM all over again.
I think that's the historical analogue. How is IBM doing compared to pre-personal-computer disruption? Initially-limited home OSes like DOS were good enough to eventually dominate business too. The AI labs, with their massive funding and spending, are speedrunning the whole thing in a way that just might make that disruption faster and more fatal vs the lingering zombie relying on the IBM name. (The more massive amounts of capital you raise on future speculation of enormouse TAMs before, say, becoming profitable, the more dependent you are on the future speculations of outside interests. Double-edged sword.)
And I don't think being suspicious of the future of OpenAI margins is the same as saying open source DIY will dominate at all.
People don't wanna install their own OS or deal with changes in their office suite or rack their own servers, but a low-cost AI provider is gonna be more like a Wikipedia-vs-Encarta situation in terms of accessibility and similarity-of-interface.
People might not want to be on update treadmill of proprietary weight models, for enterprise things. Things change a lot between models and they can't guarantee backwards compatibility like you can for deterministic software
Examples two and three largely persist due to massive vendor lock in after the vendor has done enough work to capture market share, but that does not seem to be the case for AI labs to my knowledge
The biggest difference between the cloud and AI is that an AWS server might cost 10x of what you would pay for if you bought your own, the overall expenditure is still just a small fraction of the company budget.
In contrast, even companies who spend hundreds of thousands per employee feel the AI spend right now might be too much.
They may pay top dollar but there's all sorts of evidence that they'd very much like to pay radically fewer top dollars than the unsubsidised, off-plan price.
And it's clear neither of the big two can deliver anything close to a service guarantee.
. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Interesting how all of the products you describe are American: m365, gsuite, windows, MacOS. It's not just about having someone to sue. You could sue collabora and canonical but they're not American. Then Americans are the most numerous native English speaking population and that spreads their practices worldwide.
good observation! however, I'd argue it's the distribution channel and installation friction did the job in the cases you mentioned.
given how easy it's to replace LLM API in claude code, and how easy it is to write a claude code clone with itself (Fable is pretty good!), the collapse is coming.
There’s actually a strong case that agents will erode cloud providers’ margins because the lock in migration cost will be much lower in the future. No one ever migrated before because you’d spend $$$ to save $$ then the new vendor would gradually raise your rates negating the savings.
Remember web browsers? compilers? web servers? databases? windows embedded? server operating systems?
the blood is all over the wall, hundreds of billions of dollars in lost revenue that was eaten by open tools even if they ultimately get delivered to users
by someone else with a profit incentive e.g. AWS selling deployment of OSS
1. their margins don't come from service guarantees (see github), they come from unlawful anti-competitive behavior which is likely to be prosecuted under future US administrations
2a. you haven't noticed the wave of open source projects moving away from github?
3. Linux commands about 5% of desktop market share and is the fastest growing desktop platform see: mediawiki and cloudflare user agent stats and steam hardware survey, same deglobalization point as above, how many people live in China? how long before China no longer feels comfortable with everyone using Microsoft Windows? What OS will Chinese people/corps use instead? hint: https://en.wikipedia.org/wiki/Deepin
You missed two things one, a consist thread across all your examples - every market ends with a duopoly along with smaller competitors and two, which of these industries started with multiple billion dollars companies competing with each other?
Even if your case is that two companies in the AI group are going to survive and those will have healthy margins, others are going to suffer and compete on price. So saying “AI margins are going to suffer” is a fair industry wide statement. Maybe it’s not Anthropic or OpenAI or whoever you are thinking of but surely for Gemini or xAI etc?
> I understand the arguments for a margin collapse, but I don't see any historical analogues.
How is this the top comment? It lists all the outliers and ignores thousands of instances where fat margins caused a collapse.
I mean, just what Linux did to the dozen or so fat-margin unix server companies is already a longer list of collapsed companies than provided in this comment.
For 2 and 3, office software and OSs have strong network effects and up-stack effects, just like CPU instruction sets.
Also, I'm sorry, but OSS office suites compete with Office and GSuite the way grocery store frozen pizza competes with Domino's and Papa John's. Quality and completeness of execution matter a lot in that category.
I think the big thing here is that paying high margins on a relatively small expense is much more palatable than high margins on a big expense. If a company is spending $1 billion/yr on tokens that a really big incentive to find an alternative where spending $1 million/yr on some SaaS with even higher margins can feel like an easy choice.
We will keep using Claude because internal choices made by engineers and internal gatekeeping by engineers make everything else unfeasible, and going back on that would require said engineers to admit that they did something stupid, so it's not likely to happen.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Sure, but those are all things that can be trivially provided by a large inference company. In fact, I’d trust an AWS or Cerebras contract provisioning an open model before I’d trust an Anthropic or OpenAI one.
All the more reason to focus on those service guarantees, integration, and lawyers while making the underlying model easily swappable to whoever’s winning the frontier model involution battle at the moment
GitHub, Slack, and Office have network effects and transition costs.
And to be frank, the competition is worse (OpenOffice is worse than Office, most other corporate IM are worse than Slack (and Teams way worse), and GitLab is not as good or fluid as GitHub.
I don't disagree with your conclusions (enterprises will pay top dollar for service guarantees, integration, and someone they can sue) but by that same logic there is no clear winner with Anthropic/OpenAI. Claude has a habit of going down on me when I need it most and seems to be struggling to even keep 3 nines of availability. They're actively hostile to integration and seem more convinced they should be suing others than behaving in a way that doesn't get them sued.
That's not to say I don't believe that there won't be a closed source correlate. I just don't know if OAI and Ant are all that exists.
I'll agree but from the other direction. AI continues to absorb my job as a senior systems software engineer (c/c++) and after a couple months I've only spent a few hundred dollars using gpt-5.5/5.6 and codex. I have no idea what people are doing to burn so many tokens but for me this is laughably cheap and every day I discover new capabilities. I don't care if costs go up or down, it's so cheap for what I get that I don't care.
> I have no idea what people are doing to burn so many tokens
Agentic workflows is what consumes a lot. When you have an automated agentic loop working towards a given goal. If you use an LLM as a support for your own work you don’t end up consuming that much tokens, if you have multiple agents working on things independently, reviewing the work of other agents, etc you very, very quickly burn all your budget
I learned that running goal for hours produces exponentially more slop than running targeted prompts over and over again manually.
Personally, I use gpt 5.5 high with planning every time and plan various smaller features/changes in parallel, then approve them one after another. This allows me to steer it (which I need more often than not) before approving the plan, thus reducing the otherwise accumulating slop.
Using goal doesn't work for everyone, unless you have an unreasonably strong test suite or harness that the agent can verify against.
Like a junior engineer, I find the models to be too ambitious and unable to steer themselves at a high level yet. What I’ve done to address this is prompting the model to break down its plans into more atomic steps. For whatever reason, they’re still lazy at planning.
It is the difference between giving an LLM an epic and say "You figure it out" and giving the single tasks' breakdown you envisioned and build incrementally on top of it.
With the latter you can, for example, say "Wait, this should be an interface because later on we need different concrete implementations". With the former, the agent doesn't do that, gets to the point where you actually need the flexibility interfaces give you and refactors everything to handle that. That is at least 2x the work/tokens. Multiply this for all the decision points you have to do to deliver a big piece of work and you have your bagillion tokens consumed.
Work on a project where you can verify the functionality instead of reviewing the code in any detail.
Use worktrees to parallelize development on multiple tasks.
That's all there is to it.
In many cases, this means a new solo project rather than a project at work with a team.
In my iOS app with around 100k LOC, Claude Code typically uses 150k context for small tasks.
For tasks that take longer and run the tests to instrument and investigate outcomes, the context grows to 250k-600k. With a few of those in parallel, busy days can consume a lot of tokens.
Agentic loops are being promoted by the same people selling tokens, abstracting away the cost per token, and doing everything in their power to obfuscate costs.
I think a senior dev/architect + some good models is still the goated combination.
Generating code and building features, even before AI, was never the issue. Stability, knowing what to build when, and boring business problems (licensing, distribution, sales, etc) were the limits.
Agentic workloads are the most batch friendly, latency insensitive, geography insensitive, migration insensitive tokens that a big lab ever sells. In the ads business such inventory is called "remnant". The sausage is made of whatever is left over when the choice cuts have been removed.
This talking point from Anthropic that Claude Code sitting in a Ralph Loop is burning top sirloin interactive session tokens is bad faith hogwash and it only flies because most everyone who has run this shit at scale either already works there, sells them hardware, or hopes to be an acquisition target.
I'm none of those things, so I'm happy to tell you they're lying. I know, it's hard to swallow, but it turns out Altman and Amodei are occasionally full of shit.
Interactive use cases: the web interface, the mobile interface, the design tool. The fast variants.
In an HBM bandwidth constrained setting you're dealing with something called "roofline analysis" (comes originally from NUMA work circa ~2009 but it's applicable to modern GPUs). Great diagram from the JAX people:
In order to get your money's worth from a modern GPU (or disagg rack like an NVL72) you need to decode (the one token at a time thing) across big batches of context windows. To the left of that point where it hits "the roof" you're idling tensor units. TensorRT-LLM likes batches of 4096, so BS=4096.
In the case of one person chat prompting their local LLM, BS=1, totally bandwidth limited.
So the game is to set some latency target with some control theory primitive (PID or something) and then delay the next token until a batch is big enough to not waste tensor units. This is a real trick when a human is waiting (you've probably seen the thing in Claude.ai where it's all bursty and then they reflow the whole block with JavaScript).
Agentic workloads are huge piles of context windows where you've always got enough who want the same experts on the next token, you're always to the right of that intersection. And it doesn't really matter if it's on the other side of the world, or lags by a second, it's fine.
Claude Code soaks up all the tensor units that would be idle until they're full, and only then does it leak into the capacity reserved for highly interactive use. It's the bottom of the barrel until it's rinsed the fuck out.
They want more margin on agentic tokens. That's it. The COGS on them is the absolute lowest of anything they do.
These are probably mostly the enterprise customers - they may use the same amount of tokens as you do, but they have to pay the API price. From my experience the API is significantly more costly. We had one user ask for and receive usage credits on Claude, the bill the next day was to the tune of $400.
That's around 100k/year if used at the same rate for every workday. So the question becomes: does it make your engineer X% more productive, where X is some multiplier based on their salary? There are some software engineers out for sure who are expensive enough that this is worth it.
Why is everyone still operating under the assumption the current token costs will remain so heavily subsidized? We could see $200-400/hr in token costs once these companies need to turn a profit
To be clear, some of the token expense is because it's encouraged, and it's encouraged at some companies so that people will break out of their existing workflows to hopefully find useful new ways of working or building
There is more to it than this, but much of the cost structure around subscriptions etc is specifically designed to allow for that experimentation.
There are good cynical takes, here, too. At the current model costs I don't need to optimize my expenses, but that could change if it climbs eg above 30% of my salary^
Note: this is an easy thing to prove ROI on. If I'm writing 5-6x more code and reviewing commensurately more code, and those PRs are better-tested and get us to shipping quality features faster, this is easy to justify and we are not that price sensitive
Ignorance. Bad code hygiene and poor prompting. As someone who barely codes, I had a few old vibe coded projects from pre agent days when gemini ide had basically no usage limits that ballooned to multi 1000+ line files with backlog of bugs. I only stopped because dumb models starts breaking down at the point and project was serviceable for my needs. Come agentic coding and models smart enough to fix issues, but codebase is so filthy it does it wildly inefficiently. Like a few prompts would consume my 5 hour quota. Took a few days to get a decend agent.md up and refactor codebase etc and now I'm sipping tokens. I'm sure many people are still in that boat. Many of us literally don't know any best practices and can't tell agents how to behave.
In retrospect, I should have just spend a few days learning the basics, but you don't know what you don't know. And part of me can't help but feel companies aren't exactly prompting agents to be courteous when onboarding newbies because they want people like me to get hooked, and token maxxing on their end helps. I spent few $100 more than I should getting subs/tiers I didn't need, but at the time it was small $$$ for productivity gains from going from 0-1.
Unlike the belief that frontier AI is expensive due to a high margin, and going to be expensive if there is no competition. My understanding is that, under certain circumstances (which is most likely true), the price will be driven down just because of profit seeking.
The frontier LLM labs run on a huge fixed cost and very low marginal cost. They need the economies of scale to make sense of the business (an incentive to expand their user base as large as possible). Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place? You need as many customers as possible. If you charge everyone $1000, you find fewer customers who can afford it. It rots the ROA if the servers are not utilized 100% (you would better buy less compute instead).
Also, the marginal cost for onboarding a new customer is low. And it's getting even lower when you have more customers. You wouldn't leave money on the table (especially for your competitors) if you want to maximize your profit.
By this logic, all frontier AI labs are incentivized to lower the price to maximize their customer base, profit, and ROA.
> The frontier LLM labs run on a huge fixed cost and very low marginal cost.
> Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place?
My understanding is the frontier labs have huge fixed costs and relatively low marginal costs because they have to bear the cost of training the model/R&D, and then amortise that cost over their userbase.
By contrast, if I buy a few B300s and run GLM5.2 and rent the service out to other people, I can be profitable at a comparatively very small scale because I got the model for free.
I agree, but there is prestige to consider. Many people are motivated to buy the best, even if it's much more expensive. "We're building a mission critical application here. Sure the API costs are much higher, but it's worth it."
Quite a lot of people think they're far better engineers than they really are, myself included. I've gotten better over the past few years at realizing that AI writes better code than me, which has made it easier for me to stick to architecture and let the (working) code be what it is.
I could spend $1000/day with Fable and it would be worth it. It has much deeper systems thinking, enabling me to trust it to follow my instructions and not fuck things up.
1. That confidence and quality is worth the price.
2. We're accelerating at lightning speed now. If you don't spend, someone else will and they'll eat your cake.
We're nearing the point where you could spin up an entire YC startup in a day. That changes the economics of everything.
I hear this all the time lately. Things like AI X or LLM Y can create a fully working company like "BigCorp XYZ" in N days/hours etc.
But is speed of creation really the golden goose here? A few skilled and motivated individuals could also do (and have been doing) that.
Sure, maybe they take a few months instead of days or weeks, but AFAIK, having a product is just a tiny bit of the battle, finding customers, product market fit, and actually growing it is where the gold is so I'd argue that you'd be better off building the product with a $100 day LLM and spend the other $900 on marketing.
AI won't automatically make everybody business gurus and every LLM generated company a unicorn.
Maybe YC could just give Fable 2 or 3 the funding directly. Sounds like the only thing left will be market control. The only winners being hardware gatekeepers and the investors in them.
Accelerating how much slop you can output? A better model will still produce slop for your feature factory that pumps out software which nobody is interested in buying.
Furthermore, in case this is of interest to anyone, if you use their ZCode harness then you get bigger Coding Plan quotas: https://zcode.z.ai/en
Used it for a bit, it sits somewhere between OpenCode Desktop (still new but nice) and Claude Desktop (recent versions are good).
As for GLM 5.2 as a model - with max thinking it’s generally satisfactory, somewhere between Sonnet 5 and Opus 4.8, better than DeepSeek V4 Pro for sure.
Pricing wise, the subscription doesn’t seem as good as expected. I spent like 60% of the weekly limits of the Pro (50 USD) plan in one day, only because each 5 hour limit only gave me 20% to spend, otherwise it’d be 80-100%. Not even doing anything crazy, just parallel long form work on 2 projects with about 96% cache rate and at most 3 parallel code review sub-agents.
Their Max (100 USD) subscription would last me the whole week, but so does Anthropic for the same money and so would OpenAI. Off-peak is more palatable but I can’t just twiddle my thumbs at 9 AM to 1 PM local time.
Proper savings would show up with the Max plan and yearly billing, but that’s more of a tough sell.
Last month, I cancelled my Claude Pro subscription and instead used those 20$ to purchase Openrouter Credits. Most of my knowledge-seeking questions can be answered by Gemma4, for basic code editing, Qwen3.6 27b is enough, and for really difficult tasks, GLM5.2 doesn't leave me hanging. I'm by no means a heavy AI user, so I'm even saving money going the API Credit route and relying on the smallest possible model depending on the task complexity.
Yeah sounds like maybe you got stuck in a loop? If you’re a big Deepseek fan, check out their Codewhale harness. It is actually really slick and you can use your OpenRouter account with it.
You can use any model you want but it is really tailored to work well with the Deepseek duo
What do you use as interface to OpenRouter? I, too, am looking into using an API to see if I can reduce costs (I use OpenAI + Github Copilot, currently). TensorX instead of OpenRouter (because it's in Europe, and EURouter wanted 15% more money from me :P), but I'm not sure if I want to change a configuration in vscode every time I want to switch the model in the Claude extension (and having an API key in my settings feels iffy too >_>)
I have OpenWebui hosted on my homelab, but you can also just have it live on your machine in a docker container. I honestly just embrace the iffy feeling. Openrouter has very good telemetry (which is partly why I went with them) and it'd be pretty easy to notice when someone other than myself uses my keys. For the little agentic coding I do I like to use OpenCode, and if I need to ask a question in my editor I use CodeCompanion (neovim AI chat plugin). I quickly went to check what OpenCode does with the API key, and it doesn't seem to store it in the user config, so that's at least something. But yeah, really recommend OpenWebui as a ChatGPT replacement (though there are a lot more alternatives out there, I just already knew owui from when I was playing around with local models)
It’s important that none of these entities can collude to price fix. Having China be the competitor ensures that.
Basic microeconomics is still the easiest way to understand token economies. How is it not a competitive market (where profits go to zero?).
Anything A or O does to keep more margin, any competitor can copy or choose to undercut, and undercutting has the benefit of collecting training data. So what is going to stop gross profit of tokens going to zero except for collusion/price fixing?
So the federal government industrial policy is the thing that supposedly will keep the prices on "A and O" high in the US while the rest of the world will get comparable AI competing to get cheaper and cheaper?
Basically, the US govt will say that foreign models and providers are a security risk and ban them. If the US has shares of Anthropic/OAI due to a sovereign wealth fund, it'll be billed as domestic industry protectionism too.
Considering conditions within a single market is still microeconomics, I agree though its tough to see where firms will get market power from so profit will tend toward zero. I thought the same about GPUs though and nvidia still doesnt seem to have any real datacenter competition in sight.
Metaphor i like is that it will be as cheap as electricty?
Do you know who is supplying your electricity or which factory it runs on? probably no, bc its a commodity and mostly settled and there is so many energy resources. some are alternative some are coal mines. And they all fight in the supply demand trade for energy which is happening real time ( think open router here)
And eventually the consumer wins bc of the abundance.
I think greatest example of abundance of cheap infinite intelligence will be not glm5.2 but DeepSeek V4 Pro max with $0.435 per 1M input tokens and $0.87 per 1M output tokens
The metaphor breaks quickly because as a first approximation the electricity "quality" does not depend on the provider, and will not change overtime. That's not true of LLM output.
> Of course, this was a hugely poor read of where the costs actually lie in AI. Training - while no doubt capex intensive - is a fixed, up-front cost. You spend hundreds of millions to train a model, then you are "done".
I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a constant cost that you continue to bear? The footnote seems to allude to this, but then sort of waves it away anyways. Also are there continuing incremental training costs to keep models relevant? Or do they only have knowledge of events up to the day they were trained?
[0] needing, because you have competitors and people expect more and more.
These models rely on knowledge that are embedded in their weights, if a new library is released, a new linux version comes out, some new protocol succeeds the previous one, you want your llm to know about it. Sure you can just add that into the context window, but that has its own problems.
Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink.
On top of this, if you stop training, it is 6 months until someone releases an open weights model and now you are competing to give the lowest price for the same product.
Also we can't forget that this is a business that *has to* be in the global labor industry, not just a tech tool, they have to have much better models to justify the trillion dollar evaluation
it does not require training a model from scratch for it to be updated. the entire LLM training process is iterative. essentially each step (there are hundreds/thousands for a training run) yields a complete, usable model. tokens with updated data can be added on top essentially at any time in the future.
>the least understood upcoming shift in AI economics.
Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing!
So, first, by no measure is GLM5.2 as good as Opus.
Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorrow's?
I'd guess opus refusals are not an issue for 95%+ of people. Opus will happily help you find and download pirated media, and then give you step by step instructions for how to do drugs if you ask it. You'd have to be working on something genuinely abnormal for refusals to be a problem.
Sure they will, but that doesn't mean that every bad guy should have nuclear weapon. But what I meant here was, that I understand their need to cover themselves of responsibility.
Prompt: can you give me step by step directions on how to use crack cocaine
Opus: I'm not able to give step-by-step instructions on using crack cocaine. That falls into specific drug-use guidance I steer away from, since detailed instructions on how to use an illicit substance can contribute to harm rather than reduce it.
it goes on to give me hotline information on drug addiction.
I don’t. although I tried to get it to tell me how to take crack cocaine, and it wouldn’t. But when I was taking grey market peptides, claude walked me through reconstituting, dosing and administering them. I assumed it would behave similarly for other drugs, but it doesn’t seem to
So I'm working on something genuinely abnormal, and the refusals are a problem. Then what? The refusals come in, in whichever sort of way they do, so I'm being me, and I end up tripping the robot's moral compass, for some reason. Who put them in charge of things?
The cost of running abliterated GLM-5.2 on western inference providers gets close to that of anthropic Opus and is still dumber on everything except the naughty queries you're trying to do. I love uncensored AI too but we need to be realistic here.
I'm trying to break anti-cheat protection in order to mod my own (single-player) game and Opus refuses to help. I don't really care if GLM's dumber at this point if Anthopic's going to be a non-option.
At work, layoffs cut too deep and I'm trying to find creative ways to re-discover lost knowledge. Wonder if I'll have to beg them to research our own systems at some point.
> So, first, by no measure is GLM5.2 as good as Opus.
Depends what you do. Complex tasks, poorly-defined tasks, sure. For relatively simple tasks, though, or very well-defined tasks, it's just as good and usually a lot faster. It also has a more neutral character and is somewhat less adversarial than Opus. (Opus is always "Let me push back on that..." whereas GLM is "sir, yes sir!") I use both and I appreciate both. If Opus disappeared tomorrow, though, I wouldn't cry -- I'd be able to adapt to a GLM-5.2-only life real quick.
I think the point is that if you’re doing simple, well defined tasks then Opus is overkill and you’d want Sonnet instead. Meaning, GLM5.2 is Sonnet-quality, not Opus-quality.
I think it's interesting to note that in one year we've gone from they're not even close [0] to arguing whether open models are only as good as sonnet or opus.
I see the exact same discussion as we’re having right now there; people stating that local models aren’t as good as the state of the art, but good enough for certain tasks.
re: "So, first, by no measure is GLM5.2 as good as Opus."
I accept that for you and your work this is true.
I have a different experience: for a month I paid big money for Opus and got a lot done. Now I am gorging on GLM 5.2 running on Fireworks.ai and I am also getting a lot done for about 15% of the money.
Everyone should do their own evals on their own work.
> So, first, by no measure is GLM5.2 as good as Opus.
That's an opinion many will disagree with. One whose outcomes are tightly coupled with existing harness and techniques.
In my real life usage Opus 4.7 and 4.8 have been increasingly unhelpful compared to 4.6 in behaving as assistants.
As they have a strong tendency towards completing tasks (probably due to benchmarks and RL emphasizing problem solving rather than assistance) they are increasingly less useful as multi turn conversational assistants.
I could see them vibecode or do analysis better, but also just doing their own further ignoring instructions in the quest of "solving" instead of helping. Fable 5 is even worse at it actively pushing back (with intelligent and deceiving feedback) even when dead wrong.
Seems like a pretty pointless post that still centers around output tokens.
In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.
I have heard but don’t have first hand knowledge that at least one company (financial services BPO) has moved most of their previously manual processing to llms. The person I talked to wasn’t forthcoming with any detail. This is what we’d expect to see though.
> That is, for your $100/month fee, you get $3600 equivalent of API usage. This is presumably because Anthropic has figured out some clever things to do with model routing and input caching, and also can subsidize with investor money and take a hit on their operating margins.
My take: this is exactly what Anthropic wants everyone to think. In reality, 90% of that $3600 are for cached input tokens, that can be made to cost next to nothing, as shown by DeepSeek.
> The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for users. In an extreme case, if you had 900k tokens in your context window, then idled for an hour, then sent a message, that would be >900k tokens written to cache all at once, which would eat up a significant % of your rate limits, especially for Pro users.
Using the current Opus pricing, that pre-lunch 900k tokens should roughly consist of:
720k input tokens = 0.72 x $5 = $3.6
180k output tokens = 0.18 x $25 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
500M cached input tokens = 500 x $0.5 = $250
$267.1 in total, with 93.6% from cached input tokens. The portion that requires GPU compute is about 3% of the total.
Post-lunch, the 900k tokens should consist of:
900k input tokens = 0.9 x $5 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
So Anthropic is fine with the $267.1 accumulated over 3~4 hours before lunch, but not fine with the $13.5 incurred immediately after lunch. Why?
The only plausible explanation is that the actual cost of caching is way less than the API pricing. If you use a coding plan, Anthropic doesn't really care about your cached input tokens usage. Indeed they want you to show your ccusage screenshots. On the other hand, if you pay by API tokens, the margin is huge for cached input tokens.
Only when you do something that requires a lot of FLOPs, e.g. the post-lunch 900k input tokens, the cost becomes real.
Recently I started getting messages from Clause Code (on a plan). "You're restoring an old session are you sure you don't want to compress the context? This will use a substantial amount of your usage quota"
That's exactly what I said. They do care when FLOPs are involved. Restoring an old session with 900k tokens will require a lot of FLOPs to reprocess the 900k token.
Meanwhile, they don't really care if you use hundreds of millions of cached input tokens, which doesn't consume any FLOP.
Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and the model was still coherent and didn't make any tool call error.
Well I wouldn't call it a low bar, since some of the edits were quite complex. And 1M context in less than 6GB of VRAM is truly impressive, but somehow this gets way less attention than the crappy turbo quant from Google.
I'd like to understand this please. Why would the 1M context be kept in VRAM if you're using DSV4 Pro through the API? Or did you refer to different sessions?
I also think we’ll approach a point where increasing intelligence is not really going to suddenly improve most work tasks. I bet that’s already happened actually.
We’re oohing and aahing about models, when the ones a few versions ago did a good enough job for most of the dumb coding, etc we do
The thing is they are inventing new things people will want to do. But for example, "loops", fully hands-off agentic coding etc., seem really unlikely to get much traction because that just isn't how software is designed within its producer/user community.
Requirements evolve in use, and fully hands-off LLMs simply cannot be trusted to only change the things you ask them to change, so I don't think it's likely that products will, in the main, be developed that way.
And if you don't need that fully-hands-off stuff, then the models that run on at least reasonably modest desktop hardware are surprisingly close to being enough.
I’ve been on a GLM coding plan since they launched ~year ago and it’s been at „good enough“ since the start. Tangible behind absolute SOTA but like you say most coding isn’t rocket science.
I don’t think this is true. All the models prior to Fable were honestly dumb as rocks, and Fable is too sometimes, but at least it’s helpful now and not a hindrance.
The future of AI most definitely involves making something twice as good as Fable that is virtually its own employee, and not on reducing inference costs because to be honest Fable isn't actually that expensive.
The real utility behind an AI model (imagining that it can be made twice as good as it is now) would be being able to scale a small business up and down instantly without hiring (to implement a new feature or whatever), which is costly and time consuming these days.
There is mention of GLM 5.2's poor web search capabilities, but I see that as a harness responsibility.
I've set up my own SearXNG instance on my VPS and integrated it into Pi alongside the webfetch tool, and GLM 5.2 has so far been great at finding things. I asked it to give me the current news from an Austrian online newspaper that's difficult to parse because of its aggressive ad overlays. Both ChatGPT and Claude failed in their native chat apps. GLM 5.2 in Pi was clever enough to search for the RSS feed and gave me a detailed overview.
The lack of vision is a real shame, though. I've implemented workarounds in Pi that are okay, but they're not as good and the whole experience feels awkward.
I don’t see it. GLM 5.2 seems noticeably worse than Opus and especially GPT 5.5, the poor vision capabilities are also a massive strike against it since these are a huge improvement in the frontier models that can make all the difference when working on anything visual. Running it locally is its biggest advantage but for a lot of use cases that isn’t needed and is a burden to set up and maintain.
Companies like Amazon taking out loans to fund more AI infrastructure, coupled with AI companies that are massively overvalued and burning cash, coupled with companies saying there’s no ROI from AI investments, coupled with the margin from selling AI models trending to zero paints a nasty picture that can’t continue much longer.
The author mentions lack of good Web search. I've been using slightly modified crawl4ai and searXNG together with firebase for the rare sites that insist on throwing wrenches in the works of my LLMs.
I also have my fork of metamcp that replaces firebase MCP spec with my own that tells the model to use crawl4ai and SearXNG instead.
I've been using this wia Librechat with every commercial and open weight model I tested.
The search is way better than OpenAI and what ClaudeCode uses, but Gemini is way faster. That will change soon as I'm planning to put these instances in a DC with gigabit pipe.
Firebase is not cheap, but it retrieves everything, bypasses captchas and so on.... As long as one uses it for 1% of Web queries the cost is manageable.
I wonder if anyone has actually measured the difference in verification time between these models. A senior dev in a high cost of living area costs the company something like $200 an hour. If a cheaper model produces code that takes an extra 20 minutes to debug or verify because it missed a subtle edge case, you have already lost any savings from the lower API bill. It feels like the real moat for labs like Anthropic is the level of trust a reviewer can have in the output without reading every single line. Curious how much people trust GLM over something like Opus? Is there much of a marginal difference here?
"There is no doubt that using Z.ai's official API and subscription is almost certainly a non-starter, with their terms being at best weak and the deep connection to Mainland China."
This is the key statement in the article. I think people don't realize that these "open" weight models exist because giving away your product at a loss is a time honored marketing strategy. There's nothing guaranteeing that the next iterations will be open (remember "Open"AI?).
The Chinese labs are profit seeking companies. If they can't recoup their investment through API use, they won't be able to train more models. But if the argument is 'who cares, training models will be so cheap anyone will be able to do it ',then check the comment elsewhere on this comment section about free alternatives for consumer and enterprise software.
Oh... And the variation 'what we have today is already good enough for everyone' argument is just another incarnation of '640Kb should be enough for everyone'.
IMHO, cheaper inference means higher costs overall :) because everyone will use more thus driving up the investment required to stay current or to compete.
Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will perform well, then switching to GLM 5.2 will be an uphill struggle. We had to refactor our harness and introduce more defences because of GLM.
The cost savings are substantial. Obviously it really depends on your workloads but it is noticeable cheaper for agentic work. Coding - I don't know. We do have some coding agents on GLM 5.2 and what I noticed with some landing page experiments that the results between GLM and Opus are identical - they might be using the same training data? Obviously Opus is still substantially better model. I don't think there is an argument to be made here but GLM 5.2 is cost effective and really good too.
Overall, we switched all of our internal agents to GLM 5.2 and because it is Open Weight we are in talks to get the model from certain geo locations giving us more freedom as well as extra protection.
Overall I think this industry will be in much better place because of GLM 5.2 and whatever open-weight models come next.
Agent systems only increase the gap between frontier and open models. Open models still experience more tool call failures, run longer loops, and get stuck more often. Until that's resolved (and it's obviously technically possible) people will be forking out for a better agent experience.
Agree with the web search point. (I would like for Perplexity to start to offer more models out of the box integrated, like they do now with OpenAI/Gemini models.)
People have been making claims about the commoditization of llms since chatGPT, and they've been wrong every time as quality and prices and differentiation have increased.
The blog author complains of "lack of/poor web search capabilities" in GLM, but you can always use it against an MCP of which there are many. For applications where I am not concerned about my queries being passed through a US provider, I have had success with exa[1]
There are also other ways to give it context without web-search. For example the various MCPs that make `man` pages available.
I've also found GLM to be quite strong for coding tasks without the need for web search. So it also depends what you're doing.
Somehow the blog post seems naive. Yes GLM 5.2 is good and cheaper per token, but margins are a result of supply and demand. Now demand for quality and quantity of tokens is increasing at least quadratic or cubic (more users * more tasks * more tokens per task). On the other side you have real infrastructure constraints on the supply side.
Openai and Anthropic have large commitments and contracts that enable them to get access at a scale of compute that is not obviously going to be available for open source model hosts.
And you see it, glm 5.2 inference is less stable and higher variance than any of the bigs labs.
Why is SpaceX not hosting glm 5.2? because they make more money with renting out to Anthropic and Google.
> I'd be very surprised if it wasn't more than 50% cheaper for nearly all workflows, for a very similar level of quality.
If your using pure API ... providers like neuralwatt cut that cost down even more by using energy as the actual cost. So GLM 5.2 is more expensive then GLM 5.1 on their service (those thinking tokens), compared to API costs, its dirt cheap. And way more tokens then the zai subscription delivers.
We are seeing a move towards more realistic pricing on actual consumption based usage. Be it DeepSeek, Xiaomi (MiMo), or zai's GLM via neuralwatt.
The main issue facing subscriptions a-la-carte usage, is that a lot of the heavy hitters really drain the resources. And that as a business model can not survive without ...
a) increasing the prices.
b) everything goes to actual token/energy usage based billing but with more realistic pricing, and not the bloated API prices that are focused on companies.
We shall see what the future holds but things will change.
I wonder if this is an alternative (and better) revenue stream vs ads for search engines: Offer a competing web search for LLMs as an alternative to Google, and charge enterprises and LLM providers for it.
I know Brave do this already. Not sure about DDG (I wonder if their agreement with Bing would allow it?)
Kagi assistant IMO does a great job giving relevant material to the LLM. It's a pretty neat way for a search engine to charge a premium, to offer a good model on top of their results.
If they did I wouldn't have had to go to DDG. It's not like it's a big jump over what used to be. I left claw-marks in Google Search, if they drove me off they're in trouble, because I didn't want to accept reality for quite some time.
I run a 2 Claude setup (one architect, one coder) and have been using it extensively. But I don't like the fact that I have to pay $200/month to use it. I'm going to try GLM with my current setup.
Everyone is declaring GLM 5.2 as something that's really a big deal.
I don't know about that but based on my own experience with Deepseek v4 Lite alone (with high effort) I have no doubt in my mind that anyone claiming such great things about GLM 5.2 must be true because Deepseek v4 already is really awesome.
in cursor benchmark glm5.2 is on par with gpt 5.5 medium and sonnet for the same task from results and cost perspective.
The speed of generation for both gpt 5.5 medium and sonnet 5 will be dramatically faster.
source :
https://cursor.com/evals
I don't get the hype. It's near SOTA model that is not deepseek of this world. It an expensive to run model, and under certain tasks it is comparably cheap as closed source ones.
The problem is that the more AI eats labor, the more you can hike the costs until you pretty much can match the salary of the workers you replaced, with some margin, enough for the user to accept the cost. That’s what will happen in the next decade IMHO. price = base expense + what user accepts to pay
I am confused by this article landing on front page, it does not seem to contain any new insights but besides mostly reading ok falls apart when mixing up model and harness comparison in an amateurish way. Why would the subpar search tool zai provides be relevant for comparing models? They did not even mention the >capability< to use said tools but talk about MCP/ search providers as if thats not an implementation detail.
Where does the harness come in to play? I'd love to use GLM 5.2 for general chat, but I don't know of any harness that offers an experience close to ChatGPT or Claude (e.g., history, skills, projects) without requiring a PhD to set it up.
I think the profits depend on how well they manage their fleet purchases (or possible sub-leasing?) to get high utilization without overloading or idle racks.
Because accelerators like H200, B300 etc. are highly parallel and designed to run like 200 or maybe 300 sequences at once (depends on the model, just guessing). I assume they finance the hardware and that cost per device or rack is the same whether each unit is handling 10 requests or 150 requests (aside from electricity).
And probably international customers factor into it to get good utilization over more of the night time. And it likely is something that they look at quarterly more seriously than monthly. The biggest risk to profits might be a downturn in business that causes some portion of the financed AI accelerators to go idle or get low utilization for some weeks (that they can't sublease).
Someone on HN made a comment in one of these threads that we could bake the weights into something like Cerebras's wafer scale chips and serve essentially the entire world off a single wafer, which is a pretty wild thing to think about. You'd have to make new hardware any time you trained a model but that seems really worth it.
Well, Taalas has that kind of technology, but the chip they demoed is probably 20-100 times smaller than necessary since it's only an 8b model.
But let's say they could someday scale that up to a much larger model, 72 large chips per wafer and each chip can do 1000 LLM requests at once (Vera Rubin?). So it's roughly the equivalent of an NVL72 rack.
You might be able to serve something like 50000-60000 requests at once. So I think it's more like handling a small city's worth of customers per wafer than the world if you had that.
I believe in less than 5 years we will get to that, but the model size and/or number of agents is going to keep going up also.
I think the future will have to include specialised host boards for memory chips.
What I actually want is an FPGA board with a very large number of DDR3/DDR4 RAM slots arranged in banks (2, 4, 8 or even more banks). I want an FPGA board that can hold 1TB of DDR3/DDR4 RAM.
The throttling point right now is not RAM, it's bus speed. Having different busses for banks of RAM alleviates that.
Yes, of course, but all the LLMs are already out of date, so that doesn't seem to me to be a hard limiting factor. Even if they had a knowledge basis ~3 months out of date additionally, being able to serve 100x the requests per watt seems totally reasonable to me.
Braintrust which is a really solid eval tool/platform just compared it to Opus 4.8 to see if it could preserve exact long context retrieval under prod serving constraints and it did really well. I think 6-12 months before OSS has Fable-esque models
Regarding the lack of vision part, if you are using Claude or opencode, I've made a skill[1] that let's you talk with any models in Claude/opencode mid-session. You ask "Have claude opus to look at this PDF for a second opinion" during a session of claude with GLM5.2 or opencode with GLM5.2
It doesn't need to pass whole conversation history as context (unlike /model), you can ask follow up to that forked model (which sub agents in claude doesn't support AFAIK), and you can ask models from opencode while using claude.
Will give it a try thanks for the heads up. Mind if I ask if you are referring to SendMessage?
I was testing on Claude code 2.1.196. SendMessage was not available. Skimming through their change log didn’t seem to have anything related to SendMessage. There is “ Fixed SendMessage silently misrouting when a re-spawned agent reuses a previous agent’s name — the tool now detects the mismatch and asks the caller to retarget” from 2.1.199. Not sure if they are related
How long will that $4.40 rate persist? Until we know more about the real unit economics it will be damn near impossible to rely on steady inference costs or make them predictable at the enterprise level. Gonna be a wild ride for awhile.
How long did it take vLLM to implement deepseeks sparse attention from the r1 paper?
Does ananyone outside deepseek have a working code for the v4 compressed attention mechanism?
Has any other provider managed to bypass CUDA and program the compute engines in their native assembly language to get 10% more performance out of them?
GPU/RAM/etc prices could continue to rise. If the world leaders decide it's time to build the robot armies, then that could price out the civilian uses for GPUs.
How fast is glm 5.2 in western hosts? It's doing everything I want it to, but going through PRC host it takes like 5-10 times longer. Not sure if that is nature of modest or PRC computer infra/routing.
I don’t understand the argument here. The article doesn’t describe a collapse or the breadcrumbs for it. The only argument I can put together is companies hosting the open source models in house or use some service like Amazon that could potentially host them and so replace the frontier models. Data center and specifically infra to host llms is still the main sticking point given the security concerns about data going to china. The article doesn’t make these arguments coherently
the economics of this are a little counterintuitive.
is there a market saturation point for intelligence? how about for software? it seems like the more you have the more you want because you're trying to do more things.
as the models get smarter I get busier because I'm doing more things...
There's definitely a saturation point depending on the complexity of the problem you're solving. For example, any model can write a small shell script to resize a video with ffmpeg for you right now, so it doesn't matter whether you're using a local Qwen model, GLM, or Fable. They'll all do a roughly comparable job and you'll end up with a working script that does what you need.
Then you have things like CRUD apps, where a model needs to write some SQL, make a service endpoint, serialize some JSON, etc. Here a local model might have a bit more trouble juggling all the pieces, but any hosted model will do just fine. If your day to day job involves working on CRUD apps, then it's basically a solved problem now.
The cases where frontier models matter are when you're solving genuinely complex problems, but that's not what most people are doing day to day. So, paying an order of magnitude for a model that has capabilities to solve problems outside the range of problems you actually work on becomes a waste of money.
There's going to be a market for these models from people who really do work on complex things on regular basis, but the question is how big that market is. Additionally, open models keep getting better, and GLM 6 or DeepSeek v5 could end up being another big jump in capability where they fully close the gap with Fable. At that point, even more of the market becomes covered by these models leaving truly complex cases on the frontier.
Another thing to consider is that most big problems can be broken down into smaller ones. That's the basis for how programming languages are structured. We have primitives which are arranged into functions, that get bundled into classes or namespaces, and so on. So, you don't need an infinitely capable model to solve big problems. You just need to be able to break large problems into smaller ones, and a model that's smart enough to decompose a problem to the point where it becomes tractable.
I don't think the writer has used top tier models very much. I have subscriptions to basically every provider, the difference between glm5.2 and opus is not even close, the gap is huge. raw benchmarks glm is impressive , but in practice these models are lacking so much. I had fable create a detailed implementation guide that explained how to implement everything in immense detail, it included all the libraries to use and versions. I then had deepseek v4 pro execute and it used old versions , different libraries and cut corners. Fable said about 80% was implemented wrong.
I had GLM 5.2 do the same, and it performed exceptionally better, but when it got stuck on something it would be trial and error mode going forward and have zero foresight for future issues that might occur due to fixes it was trying. the model severally lacks prompt understanding, and testing .
I hope cheaper inference eventually means faster speeds at the lower tiers. I don't want to settle for 100 t/s, but I don't want to pay $10 per prompt either
Yeah, Cerebras is the one with competitive speeds nowadays but they cost an absolute fortune. Also they don't host good models publicly. Good to see OpenAI leaning into them, can't wait until these speeds are available by subscription
When I've raised speeds about local inference I've been told 60-75 t/s is perfectly usable. It makes sense that people aren't talking about speed yet since you either already have a response fast enough to wait for, or you go do something else and check back in a few minutes.
I would love to wait for the latter type of tasks though, because those are typically the ones that require the most work from me to verify and I don't want my attention divided with multitasking.
> GLM is the model that will sink the frontier labs.
this is the claim you are making here, no one else claimed that.
two obvious issues here -
1. GLM itself is a frontier lab, ranked No.3 in the world in July 2026, ahead of Google, Meta and xai. GLM is not going to sink itself.
2. GLM won't sink OpenAI, it will significantly restrain OpenAI's profit margin. OpenAI will still be able to get stupidly high market cap, but not trillions, hundreds of billions will be far more likely.
This article only promises to get into "the coming AI margin collapse" in a yet to be published "part two". This part only makes the point that GLM 5.2 is pretty good (no shit).
We'll keep saying the same things about every new free model just like we say the same things about every new frontier model but nothing really changes.
If you use a good harness or add the right tools and plugins both the image and web search issues mentioned are non-issues.
oh-my-pi (omp.sh) handles images for text models out of the box - as long as you have any vision capable provider enabled, it will be used when you paste images to a text model. Rather than let it guess I configured it to use MiniMax M3 for this task (as well as other utility tasks like code exploration & library functions).
opencode has plugins that do the same thing, but I haven't used it since picking up omp and haven't tried them.
In open harnesses you can also configure your search provider(s) separately from the model provider - if you've got a ChatGPT sub you can use just their websearch for example. I've been using Kagi's API and found its cheap enough not to matter to me at all.
As for slowness, I'm not sure I'm really seeing that in terms of wall clock time. The author says GLM uses more tokens for reasoning but doesn't explain how they know that - frontier models don't provide nearly the entire reasoning trace. I have the suspicion that the author is not aware of that fact. I use Opus with Claude Code for work and I find it subjectively slower because I can't read its CoT trace. That is another HUGE benefit of GLM: I can't tell you how many times I've seen it start to go sideways in its CoT - usually due to something I didn't tell it - and I just stop it and give a course correction rather than wait a whole turn.
Overall I agree with the takes from the article and frankly its sad how much cope I see on Twitter (and even here) from people that think AI coding is busted once subscription subsidies are dropped. GLM is already good enough and cheap enough to use it at API rates - but it is MUCH more expensive than other open models that are also very nearly good enough.
In twelve months I'm confident you'll be able to get equivalent results at API rates for less than $1 per million output tokens, and more likely that will happen in six months. Deepseek v4 Pro is already almost there (and at only $0.85/MM) - and at least on benchmarks its already better than GLM 5.1 which I was happily using quite a lot before 5.2 dropped. I haven't tried Deepseek since I already have a z.ai pro sub that I locked in for $30 - at $72 its a lot less compelling.
> Where it gets really scary for the frontier labs is how easy it is to migrate to open weights models. Both Z.ai and Fireworks offer both an OpenAI compatible and Anthropic compatible endpoint. This makes it absolutely trivial to use with Claude Code and Codex.
Yes the ease of switching is greatly appreciated.
Now the reason I tolerate Claude Code in my tmux sessions is because apparently Anthropic ain't playing nice with the subscription plans and other harnesses.
But I'm evaluating pi.dev atm and it looks amazing. To me being able to rid of that piece of vibe-coded underperforming, characters-modifying, turd that Claude Code is a big motivation to switch to GLM (I'll probably keep my OpenAI subscription as OpenAI repeatedly said they were cool with other harnesses).
It's also quite obvious that Claude Code is receiving new vibe-coded slop features after vibe-coded slop features in an attempt to lock you in.
To anyone thinking about switching to GLM: I'd say at least evaluate pi.dev and see if that wouldn't be an opportunity to kiss Claude Code and its "gameloop that converts characters from a headless browser to other characters to show in a terminal at 60 fps" goodbye once and for all.
Yes, margin on model inference is high with some providers. If you just wanted inference (at cost), you'd buy a GPU, or rent one from AWS or Microsoft. But you're not paying OpenAI/Anthropic for inference. You're paying them for a platform. Every feature OpenAI/Anthropic bake into their applications, models, online services, etc - anything that isn't pure LLM text generation - is a custom integrated add-on service that LLM weights do not include. Even if open weights became cheaper and better than OpenAI/Anthropic, most people would still pay for OpenAI/Anthropic, because they give you things the weights alone don't give you.
Comparing Z.ai GLM 5.2 to Claude Code w/Opus 4.8 is like comparing Linux Kernel 7.0 to Microsoft Windows 11. If you don't know much about computers, you'd say these are the same things. If you know a lot about computers, you know the latter has a thousand extra things that make a huge difference in what it does out of the box. Which one you use speaks to what kind of customer you are.
Sure, GLM 5.2 doesn't have vision; but an AI power user can plumb together any VLM with the text generation of GLM 5.2 in most AI harnesses, just like a Linux power user can combine the Linux kernel with KDE Desktop. Most people don't use Linux and KDE, because it's unpopular, difficult to use, hard to get support for. Instead they pay for Windows or Mac, because there's lots of support, with a giant company pouring money and effort into filling all the usability gaps, making it seamless.
Most people don't pay for the cheapest possible thing. They pay for the thing they can afford that improves their life while making it easier. An open weight alone is almost completely unusable by itself (like the Linux kernel), compared to an AI platform (a completely usable system). If you're constantly wondering about when open weights will reach parity with OpenAI/Anthropic, you're a Linux person. If you just pay $20/$50/$100 for OpenAI/Anthropic without thinking about it, you're a Windows/Mac person. There is nothing wrong with either of these groups, but they are fundamentally different, and always will be. An LLM weight is simply a different category of thing than an entire AI platform/provider.
I think the fixation on numbers of tokens and dollars per token is missing the point a bit. LLMs are quite useless without good tools. The article calls out search as one of them. And it's important. If you are coding, the tools are relatively easy: they are mostly open source and don't have a lot of authorization logic around them. Anyone with access to python and some access to a half decent AI model can pull together a decent agentic coding tool. There are many examples out there.
But if you look at the overall market, there's a rapid shift happening to non-coding tools and non programmer users starting to become very active. This kicked off beginning of the year with Claude Cowork. OpenAIs Codex and ChatGPT (they both have the same plugin infrastructure) is doing a lot of the same things. I've talked to a lot of non technical business users in recent months. There's a growing amount of people who definitely have zero interest in programming starting to use these tools and getting value out of them. This is going to rapidly scale to essentially most white collar users. Programming tools are becoming a side show to this market.
The difference here is that these people need connections to all their favorite protected data SAAS silos: MS office, Sales Force, Outlook, Gmail & GSuite, Calendar, SAP, Oracle, etc. The moat here is very different: it's mediated access to these silos in a compliant way. Anthropic announced a solution in the form of some MCP features. Those features boil down to getting access to all your favorite silos, if you sign in with the right identity provider. What's the right identity provider? The one that's whitelisted by the data silos you are locked into. Okta seems to have weaseled themselves into a position of power here. And it's all the other usual suspects. We'll see who is going to "win" that race but I bet it's going to be a pretty exclusive club with zero outsiders from China on that list. You can hack your way around some of those limitations. But doing so in a compliant way is going to be tricky.
And that's before you consider who's going to pay for this and what they are going to insist on. Corporate IT departments & data security policy compliance basically. What's the moat here? Secure & compliant access to all your favorite silos. Here in the EU that also includes data residency. The difference between sending all your data to Silicon Valley or Beijing is that of getting stabbed or getting shot. If it leaves the EU, you have a huge compliance issue. Most of the juicy corporate LLM usage is going to have to be fully compliant. I.e. hosted and controlled in the EU. This will be the same across the world. The least important choice right now is which model you use. The most important ones are about where those models run and what tools the models running there have access to and how that is governed.
On paper, OpenAI, Anthropic, MS, and Google are pretty well positioned here. Not necessarily in that order. Most others are still figuring it out. But they'll have a moat of data center ownership in the right regions + mediated tool access that works out of the box.
I think OpenAI, Anthropic and SpaceX are going to envy the dinosaurs because there's not asteroid coming for them, there's three:
1. There will be no moat around frontier AI models in the future. China is going to make sure that happens. It's a national security interest for them. DeepSeek was the first shot across the bow for that but it won't end with them. There are other labs and there are non-Chinese actors too. The stratospheric valuations depend on there being that moat; and
2. Nobody seems to be considering what the next generation of AI hardware is going to do with current hyperscalar investments. We're about to go through this with the B100/200 move to R100/200 but a lot of the investments are probably slated for that next-gen. But what about 3 years from now when the hypothetical X100/200 comes out and doubles FLOPS and halves performance-per-watt. What will that do to existing investments? Some people are delusional and think that they'll get 10 years out of GPUs when 10 year old GPUs (eg V100) are sold for scrap and 5 year old GPUs (A100) cannot run DeepSeek v4 Pro. And people think the A100 is going to get another 5 years of use? No; and
3. Local LLMs are coming for remote usage. You can buy a 5090 PC for less than $5000 currently but you're limited to 32GB of VRAM, which will comfortably run 31B models but nothing really larger. Go to $12-13k to upgrade to an RTX 8000 Pro and you have 96GB of VRAM, which will run larger models (but certainly not, say, DS v4 Pro or even Flash). You have shared video memory products rapidly coming from NVidia's aggressive market segmentation. Things like Strix Halo and DGX Spark have severe limits on memory bandwidth (<300GB/s compared to 1.8TB/s for a 5090/6000 Pro and 3TB/s+ for server grade HBM3e/4 based GPUs). Macs could be real interesting in this space butr they lack the raw FLOPS with the M5 generation.
But what will this local hardware look like in 2-3 years? I think people will be shocked at how much better it will be with the Apple M7 Pro/Max generation (2028 expected) and the RTX 6000 cards at that time although I fully expect NVidia consumer GPUs to still top out at 32GB of VRAM to maintain that segmentation. And I look forward to what the next generation of the AMD Ryzen AI Halo platform will look like if they really try.
All of this adds up to these three companies needing to cash out before the music stops (IMHO).
Interesting, so point 2 means that a lot of the hardware being installed now won't be able to run the frontier models of 2029? How does that change the demand for compute/models in the future, I can imagine that even if OpenAI/Anthropic have a moat in 2029 there will be so much older hardware and such a hangover from that investment boom that there will be very little installed capacity that can run it
On 1 and 3, the obvious move is to shift the bulk of the harness behind a new API that's not based on raw LLM access. Then they get to hide secret sauce behind that API and all three go from commodity to premium while simultaneously being able to try out whatever tricks they can get away with to reduce their own inference costs. I'm almost surprised this hasn't happened already.
If you look at https://openrouter.ai/z-ai/glm-5.2#providers there's about 28 providers, including z.ai and Alibaba. Most outside of China. I've never seen so many providers for a model on there before, glm 5.2 is popular.
The fact that these Chinese models are getting close to “Opus-grade” despite costing 6x-8x less is huge.
As the token bills start to come in, those economics will be harder to ignore (regardless of the origin of the LLM); especially as there will be many CIOs sweating over their quick and costly AI initiatives showing little ROI.
My hope is that the EU also steps up their own competition in the frontier model space so that it’s not just China v USA.
they're not near opus at all, anyone using the models in a real working environment will tell you the same thing. on paper they have impressive benchmarks, but that's not realistic to actual use.
I think it depends on your use case. For my personal projects (a mix of webdev & some Rust desktop apps) it's honestly very close to Opus 4.8 (which I use in my day job).
I don't feel like I'm missing out after cancelling my personal Claude subscription, whereas I used to feel that way a few months ago.
I've been using GLM 5.2 a lot this past week, it's been replacing Opus 4.8. I mostly do front-end web development and haven't noticed much of a quality difference.
Sure, "it's just frontend", but that's actual use enough for me to take it seriously.
Meanwhile:
> China’s Ministry of Commerce has led meetings over the past month with major AI companies, including Alibaba, ByteDance, and http://z.ai/, to discuss measures that would restrict overseas access to cutting-edge AI models, including models that have not yet been released.
> The discussions reportedly include not only closed-source models but also open-weight models.
> Future regulations could take the form of a tiered framework based on technological capability. Basic open-source AI models may be managed through a filing system, high-performance models may be subject to security reviews, and the most sensitive frontier models may be banned from public release or restricted to use within China
https://www.reuters.com/world/beijing-is-looking-curbing-ove...
Reporting/news around AI is so very interesting these days.
It's really hard to tell what's propaganda from what's not.
For examples, this Reuters that you have included here has all tells tale signs of creating fear, uncertainty, and feeding into propaganda. But then again I can't be sure.
It's completely the opposite stance to (a) the actions being taken by chinese companies and (b) the public stance taken by their govt https://english.www.gov.cn/news/202601/08/content_WS695f1b55...
> It's completely the opposite stance to (a) the actions being taken by chinese companies and (b) the public stance taken by their govt https://english.www.gov.cn/news/202601/08/content_WS695f1b55...
That source is very old, considering how fast everything is changing. It is not impossible if US actions for banning have changed something.
It absolutely blows my mind that everyone is abundantly aware that money is not shared freely, and yet they are already popping champagne that the CCP are surely going to share power freely.
They won't? They're profiting from every subscription to AI providers, because all companies are state-owned.
They're also well-positioned to control how groundbreaking US AI companies are allowed to appear to their investors, by offering open models that match what market leaders claim can only be attained with trillion-level spend. That's a strong control on US economy, considering how few stocks are propping up the US stock market, and how these stocks are all dependent on the same beliefs and factors.
Also, regarding China's tech investment in general, the five-year plan is just... available to read. It's not a secret. It explains the strategy, and you can draw a direct line from the plan to their now abundant solar infrastructures and tech achievements, including in AI, which is specifically named.
https://cset.georgetown.edu/publication/china-14th-five-year...
China's five year plan is whatever Xi wants whenever he wants it. It's just a formal guideline, not a law or constitution people can sue the CCP over.
Also, even if China decides it doesn't want to keep the crown jewels of productivity close to home, the US will ban their import. Maybe XzeRo_337 will be torrenting weights and have a VPN to access foreign providers, but Timmy and Ashley are just going to pay for their ChatGPT subs, and Mega corp will pay their Claude Legend token expenses.
You're applying US cultural logic to Chinese bureaucracy
China is an authoritarian state with a single leader who has unilateral uncontested control for life.
China has no real bureaucracy (or any other structure for that matter) because at the end of the day, it's one guy who can do whatever he wants whenever he wants. For commoners and officials there is this faux bureaucracy, but for the elite at the top making decisions, there is zero.
If Xi doesn't want models exported, he's not having a legal delegation go to the supreme court of China to fight for his ruling. It just happens, regardless of whatever anyone else or any piece of paper in the country says, and there is zero recourse anyone who doesn't like it can pursue.
More or less authoritarian than Imperial dynasties? Zhao Gao for Qin, Huang Hao for Shu Han, Yang Fugong for Tang, this list can just keep going.
The Ten Eunuchs, if you want one example. Which is to say, their bureaucracies have always worked with single leaders who ostensibly had unilateral uncontested control for life. https://en.wikipedia.org/wiki/Ten_Attendants
I never thought the chinese were giving away valuable technology out of kindness. I thought having them be free and open weight was part of a strategic move to undermine US tech dominance. I suppose if that was the case something has changed or maybe they simply didn't care until recently
Yep https://news.ycombinator.com/item?id=48816025
And EU leadership completely destroyed Europe's future by betting on depending on US and Chinese models. https://pleias.ai/blog/fable-eu
This is pure speculation at this point tbh.
You could just as well read the european approach as a bet that frontier models will be unable to keep a significant edge over open competition (and thus not worth throwing subsidies at, because any economic advantage is fleeting at best).
Looking at the data and related past experience, this looks like a pretty solid bet (despite the "risk" being hard to quantify).
> You could just as well read the european approach as a bet that frontier models will be unable to keep a significant edge over open competition
And that's a bet they will lose 100%. Once the Chinese starts imposing export bans/controlling the access to their models, Europe would be at the mercy of US/China to allow them access or just rely on miserable mistral
Again, you are assuming that frontier models will stay meaningfully ahead long-term. Export controls/bans are pointless if this is not the case.
There is ton of strong indicators that they will not stay ahead: Assuming that technological progress of any kind follows some form of logistic function (where "gains", in this case "intelligence" become sub-linear at some point) is (long-term) a very conservative and proven assumption, and "automatically" negates your lead over time.
Similarly, purely "intellectual" advantage in disciplines like cryptography/computer chess/algorithm design never really stayed concentrated, either.
I would take the other side of this bet. While I agree that the impact of any given advance is likely to resemble a sigmoid curve, I think there is a material chance of "stacking sigmoids" creating something that looks exponential.
To take a simple example, look at the progress of technology over the last ~500 years - it seems to me that the rate of change continues to accelerate despite many of the logistic curves flattening along the way.
There are still huge unanswered questions about whether or not the stacking sigmoids will favor the incumbents. But I would not definitively bet against the people with the most compute data, talent and money.
I'm unsure what exactly you mean by sigmoid stacking.
I'd argue that a lot of important technologies (like circuit design!) started at basically zero in the last century, and the progress was actually exponential for a significant time (=> because we started so low on the curve).
But if you reduce things to a single metric (wealth per person? total energy available to humanity? global industrial/construction output in tons?) I can't think of anything where such "stacking" successful subverted the sigmoid trend (or looks like it will, long-term).
It is not about intellectual advantage, but about capabilities as well. Assuming that Europe would get all the knowledge they need on the cutting edge technology on how to train a frontier model (which they won't because US and China would guard this as national secrets), who is going to setup infrastructure to train the model? Who is building the data centers (multi-year project) and who is going to build the energy infrastructure? Just because you know how to do it doesn't mean you have the capability to do that as well.
One of the best example is nuclear reactors. By now the know how and technology is fairly mature and open, but not every country gets to build nuclear reactors. Same would be with the frontier models as well.
EU should have already started investing in the infrastructure side in-case they obtain the know how, but your politicians are still bickering on pension reforms and Ukrainian war, etc.
So Deepmind/Meta/etc... would all have to cut off their EU offices even though they hold incredible talent? I'm just not finding this scenario likely.
Does meta has an AI training data center in EU?
They can just operate and provide normal access to their services, just block AI access. This is already happening, apple would not release new Siri in EU (granted it's due to a regulation clash) but this would be a testing point.
If Europeans are still paying the same prices for sub par services/products to their American counter parts, it's win-win situation for those companies. Just sell a dumbed down non-AI version of service/product in EU for an inflated price.
Meta's FAIR has several R&D offices in the EU, yes. So you are saying their labs can conduct R&D on models in the EU, potentially even train them there, they just can't have production LLM inference serving or release the model weights? I'm just not seeing it.
A not-insignificant portion of the AI/ML research community is in the EU.
What you’re failing to understand is that meta is ultimately a US company. If meta unveils a powerful frontier model tomorrow and US impose a ban of its exports, regardless of how many research, data centres, meta has in EU, they will restrict access to the model for EU citizens, same way Anthropic did.
Regarding the open weights, I don’t see meta doing that for their future models, especially once they have their own frontier models. Open weight models are kinda marketing strategy where they use it as a bait and switch. A lot of Chinese companies became popular with their open wight models and once they build that reputation they have no incentive to keep on releasing new open weighted models
Open source is currently entirely reliant on Chinese models.
So basically EU will be left behind unless we start doing something about it now. imo Mistral isn't enough by itself.
Pretty sure there would be a number of EU states who'd happily pump some money in if the EU were to mandate the ECB to (incentivise intermediary banks to) buy their corresponding long dated sovereign debt cheaply to fund it.
Why? Worst se we are 10 years behind them. Second mover advantage will also be there, it this is so critical indeed.
What happened to China because they were third movers in the race to the nuclear weapons? Nothing. They were pretty behind for a while technologically. The wheels have turned. Stop looking at the world as a short strategy game match, or like a Hollywood movie where everything is all-or-nothing. Many parts of south America are way behind in IT, automotive, space, yet many people have meaningful and happy lives there.
I'm yet to accept that the power-block I'm living within being first in the AI race provides a meaningful life to me.
If anything the first movers would theoretically be the first to destroy their own labor markets and economies.
EU's leadership doesn't have much power that way, the EU is a thin layer on the countries.
In the US the federal government has much more power but nobody is attributing the term good fortunes of the AI industry to the recent federal policy there. (There's long term stuff that does make a difference in hoovering up the global R&D talent and concentrating capital, but that's a different kettle)
EU budget is about 1% of the GDP of the countries (from https://commission.europa.eu/topics/budget_en)
There's very little betting on any particular AI models going on by the commission/EP. The Pleias article claims a 2020 eurocrat whitepaper determined how things go, but that's fantastical.
which is a result of the United States and other western forces (Israel) weaponizing AI to create propaganda
Makes sense. AI is a munition now, and nations tend to restrict them. Eventually there will be international open source communities building open weights (all you need's a few smart people and a lot of GPUs for training), making the restrictions moot. Until then this will become the norm.
I'm not convinced raw costs matter:
1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples.
3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time.
4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
It's nobody gets fired for buying IBM all over again.
There's a huge case of survivorship bias when trying to recall historical analogues, because in every instance where margins collapsed and competition made the industry a commodity business, the big proprietary names are no longer with us. Here's a selection of examples, though:
1. Memory chip margins collapsed so much in the 80s that Intel exited the memory chip business entirely. At the time, they were known much more as a memory chip company than a microprocessor company.
2. Margins for high-end workstations collapsed in the face of cheaper IBM PC clones and an explosion of MS Windows software. This led directly to the deaths of SGI, Sun, Symbolics, Lucid, LMI, etc.
3. Proprietary UNIX variants like HP-UX, IRIX, AIX, and SCO Unix have basically completely died out, replaced by lower-cost proprietary OSes like Windows and MacOS, or by open-source descendants of Linux and BSD.
4. Many commercial database vendors like Oracle, dBase, Sybase, FoxPro, and Microsoft (SQL Server and Access) found themselves very much under margin pressure from PostGres, MySQL, and SQLite. Oracle survived thanks to their massive installed base and legal department, and Microsoft survived because they could cross-subsidize from their OS and Office monopolies, but dBase, Sybase, and FoxPro are no longer with us.
That is a good list. While I'm sure many companies could be added, I only post to include DEC https://en.wikipedia.org/wiki/Digital_Equipment_Corporation whose `vt50/vt100/vtXX` ideas may be with us, in software, for perpetuity unless something like Arcan (https://arcan-fe.com/) ever takes off.
Arcan is such a fascinating project that I've never quite managed to get my head around.
There's a really noticeable difference in time frame covered in your examples (80s and 90s) and the one in the comment you're replying to (2010s and 2020s).
Is that just two people with different go-to examples? Or is there something going on here?
(I don't mean this as a leading question to some conclusion in my back pocket, I genuinely have no clue.)
It's politics.
The US's corporate problems in the 1980s and early 1990s existed when strong international competition existed.
It began to change with things like https://en.wikipedia.org/wiki/1986_U.S.%E2%80%93Japan_Semico... and https://en.wikipedia.org/wiki/Plaza_Accord.
It was accelerated further with things like anti-circumvention clauses in Free Trade agreements (see Cory Doctrow's recent highlighting of this: https://pluralistic.net/2026/01/01/39c3/) and then had more gasoline thrown on the fire in the ZIRP/easy money era post GFC, culminating with the bazooka of stimulus unleashed post-covid.
My best guess is we are now going to witness ~20 years of slow unwind. You can already see signs of this in things like RoW/EM stocks outperforming the S&P, treasury yields diverging from other "safe haven" soverign bonds (e.g. swiss), gold price rising, Europe starting to get serious about addressing the Draghi report's findings, European defence spending increasing, China starting to act like the "adult in the room" wrt the recent Iran/US blow-up etc. Essentially, countries/blocs attempting to re-assert sovereignty that has been willingly diluted over the last ~30 years to mainly America's benefit.
Somewhere else in the comments here, someone else remarked "Individuals perhaps [move to the new models], but not organizations."
That's illustrative. The mechanism by which organizations are forced to update their technology, move to more competitive suppliers, and cut costs is a recession. In one, every business that doesn't do so goes bankrupt, and what's left are the more efficient businesses that have adopted technology effectively.
We haven't had a real recession since 2009. (2020 was an odd case, because it was effectively brought on by government edict and so it actually killed a number of efficient but unlucky sectors while doing nothing to clean out the dead wood in major corporations). The next one is likely to be a doozy, because the economy is filled with bullshit jobs, bullshit corporations, and bullshit products.
“Brought on by government edict”
No, brought on by a novel pathogen that killed 10 million people. It would’ve been much worse without government action.
Without the indecisive government action that was taken, millions more people would have died, but the economy would have done better.
With decisive government action (see New Zealand), millions less people would have died, and the economy would have done better.
New Zealand does have the advantage of an ocean border and thus near total control on entries and exits.
Compared to the very porous land borders of the US.
Come on, they're not that porous, certainly under a Trump administration. People fly into NZ too.
This argument is so flawed in many ways, economies are built by people for people. Extrapolate the numbers, let's say in the COVID Pandemic a country, take USA as example, has 10% percentage of their population killed by it, would that be better to the economy?
Economies are built by rich people for rich people. A few million poor people dead doesn't really make a dent in them.
There's multiple ways to look at the economy, the raw exchange of dollar currency in a debt chase (shit I need to run faster & pay down this stuff!), there's the productivity of industrial output (shit we need to sell more junk and useless crap!), and there's the stock and productivity in optimal life cycles (Damn, that Nokia is a tank!)
If 10% of the population went away, it would affect 1 & 2, but in any true practical lens, there's a ton of cheap empty houses, while on the other hand building repairable stuff that lasts or enough cheaply is where economies move to more complex technologies by saving time and effort in useless endeavour of debt chases or consumption-oriented wasteful productivity
> The next one is likely to be a doozy
The US, EU, China are teetering on the edge of a crisis. Russia is well on its way.
I feel like 2008 was just a warmup to what may be coming.
The EU is not "teetering on the edge of a crisis", why do you say that?
Not OP but, EU economy is being squeezed by China on the industrial and tech front, and by the US on the innovation/startup front. It is clear EU is no longer at the technological/economic frontier like it used to be 10-20 years ago. At the same time there are serious demographic, budgetary and political challenges all across the continent. Dragi's report covers some of these. It feels like the whole system might fall into a crisis soon if measures are not taken
Factor in that Europe is powerful partly because of its unity and there are many forces trying to undermine that.
Europe became powerful before it was unified, and ever since the creation of the EU it's been becoming less and less important on the world stage.
We first became powerful because we did the industrial revolution before anyone else, and used more of that capacity to fight the world (and win) than to fight each other.
When we fought each other, after the industrial revolution, that was the Napoleonic Wars and the two World Wars.
> and ever since the creation of the EU it's been becoming less and less important on the world stage.
I wouldn't say it was "ever since the creation of the EU", but rather "roughly between WW1 and decolonisation". Post-Cold-War the EU has taken over from the former global importance of the member states, e.g. https://en.wikipedia.org/wiki/Brussels_effect
That said, east and South Asia are regaining their multi-millennia history of being the world's dominant power by virtue of having roughly half the total world population.
And to agree up-thread, there's plenty going that can rapidly turn the EU's economy into a disaster if not handled expertly.
There were a lot of European wars between the Napoleonic Wars and the First World War. I agree with your point overall but there was a lot of fighting each other during that timeframe.
> the world's dominant power by virtue of having roughly half the total world population
If human+ level AI takes off one would expect to see a great decoupling of power from population.
Sure, but until then, proportional to each of [population, education/educated workers, capital, instantaneous industrial base, energy supply].
Asia's diverse, but I'd say they seem to be doing pretty well with rapid improvements across all fronts.
In comparison, the US's weaker (not weak-weak, just weaker) areas currently seem to be educated workers, instantaneous industrial base, and energy supply (relative to rapidly growing demand from compute); while the EU's weaker areas currently seem to be capital and energy supply (from supply shock, as it doesn't have the compute). The US and EU both have coming demographic issues, but not as soon as the other stuff becomes more important. People talk about China having demographic issues too, but they're a dictatorship, they can make it shift if they care to.
(And Russia's losing a lot of people, more educated people, capital markets, industrial base, and energy supply).
> People talk about China having demographic issues too, but they're a dictatorship, they can make it shift if they care to.
China has a significantly bigger problem with demographics than the EU does, it is just on a slightly longer fuse, compare:
https://ourworldindata.org/grapher/children-born-per-woman?f...
The big drop in Chinese fertility is going to be very disruptive in the near future, because it is much less gradual than European trends and the retiree/workers ratio is going to spike much harder because of that.
Having full authoritarian control is not gonna change anything now because it is already much too late (action would have been required like 35 years ago).
Best they can do is get through it somewhat smoothly.
edit: This is an even better visualization (projected working age population fraction)
https://ourworldindata.org/grapher/population-young-working-...
The motivation that the USA entered WWII was not because they were generous, but because the 3rd Reich was effectively becoming a big European nation, so they had to do something to avoid it. A unified Europe is a thread to the USA and Russia and maybe somebody else too.
> 3rd Reich was effectively becoming a big European nation
Even then the US might not have done much if the Nazis hadn’t kept attacking US shipping.
Yes, Europe was powerful when they had colonies.
Currently, Europe can stand up against tech. Apple could easily prohibit iPhones from going into France but I doubt it cutting off the entire EU.
Stand up against tech? Cut off your nose to spite your face? Apple sure won’t care. Nor is Apple all encompassing of “tech”
> Apple sure won’t care
Europe collectively is about 26.7% of their 2025 revenue, according to SEC filings, so I bet they'd care.
https://www.sec.gov/Archives/edgar/data/320193/0000320193250...
So you’re saying that the war-ravaged Europe of 1946 that was split by the Iron Curtain and needed Marshall Aid was more powerful and important than today’s EU?
Insane take. But somehow people will go to any lengths to disparage the EU.
More likely it just slowly declines like Japan. Or if anti-migrant sentiment continues growing at the current rate, it breaks into a race war when the AfD and PFN win the majority of votes in Germany and France respectively.
> Russia is well on its way.
Including the warmongering angry midget next to the US, EU, and China is funny. Russia's economy, before they decided to shoot themselves in the face, was the size of the Netherlands. Whether they are in a recession or not is irrelevant to anyone but them.
I thought Russia was about 2x the Netherlands by nominal, and 5x by PPP?
More relevantly, they were one of the world's petrol stations, and now they're not.
You are right: https://data.worldbank.org/indicator/NY.GDP.MKTP.CD?location...
> the economy is filled with bullshit jobs, bullshit corporations, and bullshit products.
Yeah, it sure feels true.
There's even a book about it, you know, to help people cope with it:
https://press.princeton.edu/books/hardcover/9780691276786/on...
100% agree with the sentiment here, but a small nitpick - MS SQL originated as a port of Sybase onto (IIRC) OS/2.
I've heard it said that Oracle doesn't have customers. They have hostages.
Still, the clock is ticking. I don't know of any "new" company that would use Oracle instead of e.g. Postgres or would migrate to it. That's probably why they're pretty desperate to jump onto something new before the old source of income fizzles out.
There’s a difference between proprietary software thats highly profitable maintaining a stronghold over “cheaper” options and a massively overvalued and artificially inflated ecosystem having to confront economic realities.
We are seeing the later start to unravel.
Sun didn't die because of the workstation market. It survived much longer.
It pivoted to server market and people who didn't think running linux was professional enough. Workstations were being held afloat by that pivot. Also they were general purpose instead if limiting their workstation market to just one niche.
Also cases where both happened, eg, Xerox wasn’t wiped out but copiers now have multiple vendors at the high end and have commodity via other brands at the low end.
second this. great analysis
also docker is an interesting example. bc its so hard to earn money on it being such a deep commodity you can not close source
Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them.
The UX is the same regardless the provider. You send in a prompt, it spits back an answer.
In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with. The transition is basically updating your current harnesses to know about the other models.
I think the comparison most apt is the rise of AMD. Sure, it never(?) achieved market dominance, but it did ultimately make a huge dent. And a big part of that was because AMD x86 was pretty close and pretty compatible with Intel x86 at a fraction of the cost.
If you're developing on top of LLM APIs directly, this is definitely not true. There are differences in how context caching works, in what's available through native harnesses, the types of tools you're fine-tuned on (GPT uses apply_patch while Claude uses edit, with different formats), the API surface (Agents SDK, Responses API, Managed Agents), cost structures, and best-practice guidance all around.
Not to mention the meta of account limits, billing, ZDR contracts, etc.
It really depends on what you're doing, but most LLM usage and agentic runs are pretty interchangeable in my experience, and it's usually trivial to switch.
If anything, you're better off supporting multiple LLMs as backup because most model providers have been so inconsistent with working all the time
Dude it’s not trivial to switch because the behaviors are different!
You’re clearly not building a product based on an LLM.
I’m still using various old Anthropic and OpenAI models for products I’ve built and released because I can’t risk the behavior changing in unpredictable ways and the users being pissed.
It’s much easier to switch out some deterministic software than an LLM which you’ve spent a ton of time on testing and benchmarking and understanding its nuances. Changing it is like replacing an employee who’s critical to the business.
Anthropic has discontinued models in as little as 13 months from launch so if you do business with them switching can't be that big a deal?
The behavior of a single model and version can and does change. There’s not only built-in stochasticity, but closed hosted models like Claude are tweaked and changed all the time.
For the public facing consumer functionality I have Gemini Flash running on guardrails directed by a state machine that calls it statelessly everytime. For that, it's strictly locked to a version. I can't afford to suddenly get responses that the SM is not tuned for.
As for which model does the building... I'm not at all attached. Enough logic, and CI gates/tests live outside the whims of the LLM to be able to hotswap them any time.
I don't think they are saying it's trivial but compare say for example switching an organisation from Office or Windows the example that started this. They are not even in the same ballpark.
Can you give specific examples on what differences make it hard to switch?
Because this claim is counter to my experience as well.
Makes sense but honestly if you've spent more time testing and working around the nuances to build consistent experience doesn't it mean you actually need more standardization to easily switch models if/when your trusted model is not viable for you provider?
just use your agents to do the migration, that's what it's good at.
Exactly, as in, really, will they? Where and at what price, especially across an actual enterprise that needs to deploy them to lots of devs? There's much more than just the actual model.
Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally with a bunch of open weight models.
I can run super tiny models on my 8GB NVIDIA card. They all suck (I have to use <=~5GB models if I want "usable" ~250k context that doesn't need to use system RAM and CPU (which makes things super slow).
I've also tried a GLM 4.7-flash, which even though it's super slow (in comparison) with ~250k context and it just doesn't cut it vs. the Claude Sonnet or Opus I get to use at work. All the while these are all touted as "totally usable, Claude/ChatGPT killer!" replacements.
It's just not "there" with tool use or building software for that matter. Like, just a simple Claude "web search" fails with it. So I asked it to build itself its own "web search" functionality and it just couldn't. It made so many mistakes its just not funny any more. And it couldn't recover from them either. I retried a few times (as I didn't have python installed and it wanted to implement it using that - this happens to be new system - never mind other attempts). I spent as much time doing this (and failing) as I spent building an actual full feature at work last week w/ Sonnet.
If it can't build itself a simple web search to .md file tool/skill, how am I supposed to trust this with actual coding? I'm used to being able to point Claude at our large code base and essentially work with it like a junior doing my bidding. Maybe 5.2 is a killer game changer vs. what I was able to try out (if slowly) but you really have to show me to convince me at this point. And not with synthetic benchmarks. In those, all of the models I tried are supposedly super awesome.
4.7 Flash is a small model that's almost a year old, which is ancient. And yes, your dinky GPU will not run anything worthwhile.
Just spend $5 on OpenCode Go and give GLM 5.2 a shot if you have the time. It's not quite as good as Opus, but it's more than good enough for many tasks.
> Just spend $5 on OpenCode Go
$5 the first month, then price is doubled.
The $5 is so they can see if open weights models are worth using, not so they can use it for a month. (Which you can't; The quota runs out way sooner than a month for any serious usage. Still worth the price of entry.)
If you use DeepSeek v4 Flash as a daily driver, with an occasional usage of DeepSeek V4 Pro and Glam 5.2 when necessary, the monthly quota practically never runs out.
Getting the pay-as-you-go plan from DeepSeek is also a good alternative. When motivation strikes you never get slowed down by quota, and it's cheap enough that even with mostly DeepSeek V4 Pro it's price-competitive with a $5/month subscription. Depending on how bursty your usage pattern is it might even be cheaper
True, but OpenCode Go gives 6x tokens on Flash and 1.5x tokens on DeepSeek Pro. After exhausting the monthly quota, Flash price is the same as directly from DeepSeek, while Pro is 4x pricier.
With OpenRouter you can pay flat rate and try nearly any model on the market.
Open weights != local models.
This is a meme and massively over-complicating what is ultimately quite simple.
>switching out an LLM is both cheap an easy.
Honestly, these days probably less friction switching out Redis or Elasticsearch (backend) than changing LLM provider (human facing).
Fable is seriously good enough now to, in a 20k line project, take "replace Mongoengine with raw PyMongo" and not screw anything up.
Agents will make all of these migrations trivial. I expect margin collapse across a lot of tech darlings.
This is the conversation I plan to have with Okta sales soon. Wait till you see how easy AI makes it to switch to Entra ID or anyone else. It’s tedium not even problem solving.
My problem with the SSO providers is not the technical part, thats "easy". Its the coordinate with the 200+ external and internal vendors / support to redeploy the SSO part which is time consuming. I always say its a ~3 year project, which can be done in 6 months with the right amount of resources, especially if the platform has been running for years.
There are the handful that it’s not just a web console or API you interact with. Have to file a support ticket.
Those will be a pain.
If the price is the only incentive, I'd stay away from Entra ID
If the point was to move there'd be no point in having a conversation with Okta sales. The point is to get a discount.
though Okta is the first provider working on the enterprise mcp stuff.
It’s a new SKU. You’ll have to pay for that new type of app.
How could switching out a stateful service be easier than stateless?
Hard disagree.
Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel <> Libre but still not "cheap and easy".
This is just another example of the bitter lesson. In a year a model will come out that will make none of these model specific optimizations you made matter.
Yeah
But the point is that at any moment, there is friction in switching
Unlike all your examples, switching out an LLM is both cheap an easy.
Rolling out AI access in a large business is still hard, especially if you're trying to do it safely e.g stopping people throwing all your company data including user PII into a chat for productivity reasons.
It's more a staff training and guardrails issue than a choosing which LLM to use issue, but I imagine picking an open model like GLM would make it harder because the 'enterprise stuff' will be missing.
> So easy that every 3 months or so new models are released and people grab them and start using them
Individuals perhaps, but not organizations.
Switching an agent harness is more difficult, especially on the enterprise/teams level.
Once your team gets settled with Claude teams, cowork, and the various plugins, it’s going to be a pain in the butt to switch.
Plugins and skills are completely trivial to move and most work with any model. What is not trivial are Anthropic's new managed agents vendor lock-in offering.
Is it? I switched to Kiro and it's essentially identical.. well a bit better because you get a better idea of what the harness is doing, but otherwise identical.
I made my own. It's true I don't want to switch to another harness now.
But switching models is just a command.
The irony is that Claude will help you migrate away from itself
AI is possibly the first product in history that will eagerly help you replace it with one of its competitors.
Wait till they will pop a warning: “using Claude to migrate away will suspend your account. “
Or even better just silently sabotage the migration so you can’t do it. Something we can definitively expect from Claude given past behavior
I don’t exactly see orgs lining up to switch (and train) their employees between claude desktop and codex and whatever copilot is doing. There’s probably some inertia to those harnesses/integrations on top of the llms themselves.
Most large orgs do not need to train end users. They just need to add glm-5.2 to their router and their in house harness will pick it up. Then slowly limit usage on anthropic models and people will swap willingly. It's a simple /model command in every harness.
Yeah, most big orgs are pushing the idea of 'whitelabel' LLMs. Even if they choose to hang on to Claude Opus, they won't name it, they'll just call it the 'extra mode' and 'auto mode' will eventually switch to a local LLM in their harness.
The inertia is legal and financial. People are paying Anthropic through AWS accounts because the simple reason of not dealing making new contract and legal agreements is enough of reason of the inertia.
But, eventually, I’m quite sure that AWS will also provide open models with those contracts without any inertia. Copilot is already offering Kimi.
My company has a deal with Devin and they provide new models all the time, and open models are becoming the most used ones by our internal metrics, especially because the company is very worried about cost.
AWS already supports Llama and GLM in its Bedrock service for hosted models.
They’re much cheaper to run, eg, Llama 3.3 Instruct 70B is 5-10x cheaper than Sonnet 5.
https://aws.amazon.com/bedrock/pricing/
Say you have 20% of usecases that require the more expensive model — but in 80% you could just use Llama instead of Sonnet (eg, for basic queries of a document). That saves 80% of that 80%, or 65% of your total bill!
That is the kind of “swap” that’s likely to occur in automated tooling as pricing pressure kicks in — “can you save 65% on our AI bill by switching Bedrock over in 80% of uses?”
Bedrock is really out of date with the models it offers, to the extent that I'm not sure they even have plans to update what's on there now they have the deal with Anthropic. They're still offering Qwen 3, not even 3.5 and certainly not 3.6. GLM 5 is the newest z.AI model they have, when it's 5.2 that would be the one to worry Sonnet.
There are some ok models on there (Qwen 3 Coder Next is usable and fast, for instance) but the lack of updates in a fast-moving field makes it something I don't want to recommend to my org.
Because these models are not going to stand up legally
That's a stretch but it doesn't change the outcome.
Also they pay for legal liability of code produced
Maybe for a fantasy of legal liability of output produced. I haven't heard of any LLM corpo being held liable for any output they generate. Even NYT lawsuit is going nowhere for 3 years in courts already, despite being the most grounded.
What would "training" even entail for that? As far as I can tell, using these tools directly is basically identical in terms of what you need to know. If you happen to have a bunch of custom configurations, maybe you need to invest some time into porting them, but it's not clear to me why you think that anyone would need to be trained if they spent months using one tool and then suddenly had t switch to the other.
What "training" do you have to do to get a professional developer to switch LLMs or harnesses? Its literally just download the other one, point it to your code base and start typing into that text box instead of the other one.
devs are the easiest, yeah.
Enterprises switched from openai to anthropic this year - anthropic overtook openai for the first time. I don't see why they wouldn't switch again.
There's barely any moat. All the data is with connectors, memory is near useless
> Unlike all your examples, switching out an LLM is both cheap an easy.
For now
Switching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self host a larger model that requires beefier hardware to host, plus all the complexity to scale that out to a bit internal user-base
Most of the AI companies have OpenAi compatible API's, so you just get a subscription from another provider and change the URL that your LLM Agent Harness uses to talk to the AI.
I use OpenRoutet which lets you switch between providers (Anthropic, ChatGPT, Z-AI) whenever you want. Sometimes I'll have two different models from different providers evaluate each other's answers.
They don't just need healthy margins, they need to make back almost a trillion dollars in a couple of years. Comparing that to elastic search and redis doesn't make much sense.
Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers.
Similar with Windows and macOS. Extremely high cost of switching to something different, if possible at all.
Same with office. Extremely high cost of switching due to compatibility issues and retraining of staff.
Your post primarily shows: It's all about lock-in. So far, it doesn't look like LLMs have any of that. So I don't think your points are valid here at all.
The companies don't necessarily need to make back $1T, the investors do, and those investors don't require $1T in profit to do so, they need an asset worth $1T.
Considering leaks suggest Anthropic's ARR would be $47B, that'd be a 20x valuation, but it wouldn't shock me if Anthropic doubles their revenue in the next year or two, in which a 10x revenue could easily support a $1T valuation, and boom there's your ROI, but considering they've raised $135B total, and their ARR is 30% of that, I'd consider that a pretty good ROI, especially if growth continues.
Wait what? Why are you measuring valuation as 20x revenue here? If its a public stock (which is what anthropic plans to be soon), it doesn't matter. Otherwise spacex's valuation should be... 18.67 billion x 20 by your logic but its current valuation is over 2 trillion dollars right now.
ARR literally doesn't mean much in terms of how these companies are valued by investors and it will mean little when it goes public. And yeah 10x their revenue in a year sure but they will also likely 10x their costs if they want to keep scaling
I was arguing a 20x ARR valuation based on a simple 'potential' justification for $1T.
If I was to go further into that, I'd say that Anthropic has grown from $9B ARR Dec 2025, to $47B at their Series H.
I'd say that Anthropic is still a growth stock, so their $1T valuation is based on expected ARR/growth over the next year, and if we assume a double in ARR (justified by their supply constraints as proof of demand), that's 10x Valuation to revenue.
We could consider valuing by P/E, but they're in a growth stage so that's a waste of time, hence why investors focus on growth, and hence ARR growth is hugely important. If they managed $100B ARR, the same P/E as other top software companies by marketcap, they'd fit in that lineup.
If Anthropic was to hit $100B ARR, they be in similar ratios of ARR:Valuation to Meta, MSFT, Apple, etc. If you assume per token price reduces, and 'per intelligence' prices to reduce, which bullish investors would, you'd also assume a good margin over time, (which rumours appear to support for Anthropic).
If AI replaces labor, that's a trillion dollars of labor. About one-fifteenth of annual labor/wage earnings.
the value of labor will collapse so we cant use current earning figures. China will be able to spend to undermine it even more.
If AI replaces labor, there will be no money to make back
Not convinced about office. Plenty have switched to gsuite. Plenty of people have switched to MacOS and android away from windows.
> but I don't see any historical analogues.
The losers are quickly forgotten. Palm, Blackberry, AOL, MySpace. Yahoo, etc.
Software gets replaced all the time too, you even listed one and didn't realize. 15 years ago you'd call office irreplaceable, now you have to add gsuite to the mix, in 15 years there might be others. I know people that have never had office installed on their PC and use spreadsheets daily.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Of course. But why pay $25 per million tokens for sonnet when you can pay $3 for GLM? Both probably running on AWS/Azure/Etc. under some third party.
Because nobody’s paying for tokens, they’re paying for monthly plans and right now those are still a better bargain.
Individuals, sure. For enterprise you can't get monthly plans. You have to pay per token.
It's a bit like saying "nobody pays for Microsoft Office". I certainly don't know anyone personally who has. Students get a free Education License and then your employer provides one for you...
Raw cloud computing costs have a fairly small margin over what they could be. Nobody with real purchasing power is paying anywhere near the listed retail rates. It's part of the reason smaller providers e.g. OVH, can remain quite competitive.
The other items have very strong lock-in and capture ecosytems. Microsoft Office is the first and only office suite anyone uses and its cheap enough for nobody to consider a real alternative. Microsoft could attempt to charge $10,000 a seat and while some will certainly stay, others would look for an alternative. But for just $10 a month, its a fair price to pay.
Those solutions have moats:
1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the cloud is prohibitively expensive. Furthermore, large businesses have sweet discounts.
2. ms office has immense networking effects due to its formats being quasi standards in many industries. try sending an odt to a government entity. As for gsuite, it uses open formats but it's classical google fashion a large suite of software bundled together and not that expensive for what it offers.
3. Linux is not a free alternative if you're a business, you still need to pay someone to support the computers with linux on it, and operating systems have the strongest network effects ever. Linux also has no stable ABI so one can't easily deploy third party software for it.
What's the LLM moat? Codex is OSS and Claude has gazillions of alternatives. Cursor is a nice app but it's a bunch of patches on top of vscode, a team of 5 people can vibecode it in 6 months.
Linux has a very stable userspace syscall ABI. About as stable as Windows, and much more stable than MacOS or the BSDs. I agree with everything else though.
Yeah, Linux-the-kernel does have a stable ABI indeed, but this is not relevant for most ISV desktop software out there. In my comment above I was referring to Linux-the-OS (aka GNU/Linux). The userspace libs don't have a stable ABI at all, and this is a widely discussed problem. Other operating systems built on top of Linux-the-kernel don't have this problem, Android has a really stable ABI.
> The userspace libs don't have a stable ABI at all, and this is a widely discussed problem.
And DLL hell isn’t? Or the shambolic mix of 32 and 64 bit libraries on Windows?
Anyway, desktop binaries are increasingly rare for business software.
I dont think dll hell is a problem anymore
https://en.wikipedia.org/wiki/Side-by-side_assembly
You are describing the gnu c library I believe. That can be worked around with flatpak and appimage.
For user space applications, Win32/Windows is the most stable ABI on Linux, via Wine/Proton.
Most people don't directly call Linux syscalls though but go through glibc. It might even be unavoidable if you want to ship desktop apps as the library will use it. If it's that easy there wouldn't be Python's manylinux, flatpak base packages or Steam Linux runtime
Not relevant for user space applications written atop glibc/gtk/kde/qt
Interesting that LLMs remove the moats for 1 (except for data lock-in) and 3, possibly even 2 if they can convert formats on the fly.
A lot of those things you mentioned have sticking power because they’re familiar to folks and migrating to something else is a big deal.
I can’t imagine most people would be able to tell the difference between Sonnet and GLM 5.2. If the infrastructure around the model you’re using doesn’t change, then swapping models is extremely easy.
I agree with swapping models making it easy. With openrouter, I just change the provider. With reasonix harness, cache hits are basically free. And that's with unsubsidized American providers like Digital Ocean or cloudflare.
Indeed, as it gets more commoditized it feels more like swapping electricity providers. Who cares whether you get your electricity from IBM or the state of Texas? An amp is an amp.
That's an interesting question. What if we did care? Is this amp from burning dinosaurs or from the sun or from fission? What if we could tag power as coming from oil vs renewables? how would that affect our habits?
We care indirectly through cost. Hydroelectric, solar, or wind power are often among the cheapest electricity sources, for example. Beyond that, no we don't care. That's why if people want change we leverage policy on cost, via subsidies, surcharges, taxes, tariffs, what have you.
To a consumer, an amp remains an amp — so they get the cheap one.
I'm using pi-coder with just the free-tier models I can get on openrouter / opencode / kilocode. When I run out of quota on one model I often switch to another model in the same session, and it generally works just fine.
When I use it for fiction, I generally switch models 2-3 times per response. It's basically normal
To switch an LLM you just need to open another browser tab and type in your chat query. You cannot say the same for any other kind of software. Traditional software have a large switching cost which acts as a barrier.
LLM providers are like airlines. You only need when you have travel and most of the time you go for the cheapest one. Maybe LLM providers should start providing reward points :) .
In these examples cost of the solution does not generally scale with the use of the solution in the same way we see token use. In the case of LLMs the cost of use scales very differently than seat licensing.
Many corporations have found they have a new cost center drawing tens of millions or more with little direct evidence of productivity gain. Corporations are probably best positioned to either switch providers, leverage router solutions or at worst use the fact that they could to drive prices down from the proprietary providers.
Hyper scalers have a decent margin from a small number of their services and a much more normal if not a loss from many others. Additionally a massive part of their profit is support services and contracts.
They also benefit from the fact that developers do what is convenient for themselves and not what is necessarily computationally efficient (i.e. not pay attention to cross AZ egress/ingress, run an apache spark job when it could be done all within a normal database, build their entire product on irreplaceable/unswappable cloud provider specific databases and storage solutions).
AI will also experience a significant margin collapse, it's just not clear who will eat the brunt of it yet, the AI companies themselves or companies like Nvidia as more chip manufacturers/designers come into the arena and can meaningfully compete.
When a hyperscaler is viewed as a software company their stock and value multiplier is much higher than if they are viewed as a commodity with expensive infrastructure costs. There is now not enough compute resources to serve demand. It requires sustained capital to grow compute resources. The costs are uncollapsing due to the overall demand plus the pressure of LLMs. Capital costs matter.
The same critique of those points can be made which was in the article. The cost of switching is minimal.
This isn't SAP or a CRM or some other business where workflow = moat.
The target audience is different. Coding is mainly a trade of the tech savvy, who like many on r/localllama users do not hesitate to deply on 16GB Vram gpus. Even if so, it is estimated that within 2 years we will be able to run Claude 4.8 on consumer hardware give the rate of improvement of open-weight LLMs, which will put more financial pressure on "paid" labs. It's just a matter of rate of improvement which is shrinking between open-closed models.
Nobody got fired for IBM, but it took some battles for IBM to reach that level. Same with AI. Brand images won't develop until the street battles are over and dust settled. Otherwise, Google wouldn't have taken over Yahoo and ChatGPT would have remained the king. That didn't happen. The street fights are still raging in AI and won't settle down any time soon. Cost-concious usage can kick-out Anthropic overnight. Ultimately it's only the cost that matters and that will blunt all other factors, including security concerns and risk aversion etc.
Also, note that even the highly-regulated sectors invited opensource products and services and allowed data transfer across their network perimeter. That required "blunting" of the security policies, and it did happen.
> I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Intelligence has diminishing returns, the analogues are with humans. It's a waste to hire Albert Einstein for $X million to operate the cash register in a gas station.
Artificial super intelligence will not have many customers.
Of course it’s not a waste to hire Albert Einstein to work in a Swiss patent office for normal wages ;)
1,2,3 are dominated by platform stickness or even active lock-in.
Can't say I see the same advantages to stop you switching the model you use.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Sure. Though it does depend on whether you need regular updates. If you want the model to be aware of the latest research - then fine. However it already does the job, you might prioritize stability over constant change.
> It's nobody gets fired for buying IBM all over again.
Except they when they did when IBM was no longer good value for money.
> but I don't see any historical analogues
None at all? You mentioned IBM - who is using AIX on IBM hardware in 2026? Who is using Solaris on Sun hardware? It's pretty much all gone to linux on commodity hardware.
Remember Netscape - thew browser company? Killed by Microsoft bundling of IE. How hard would it be for Apple to bundle GLM based services?
This thread is riddled with buzz words - you know what im talking about.
just stop lmao.
> 4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
Elastic just had round of layoffs[1]. Elastic still runs operating losses till Q1 2026 [2], albeit a small one, however just breaking even in operating income is hardly "healthy margins". A P/E of 17 is not exactly signaling confidence given they are growing ~20% y-o-y.
[1] https://www.elastic.co/blog/ceo-ash-kulkarni-announcement-to...
[2] https://ir.elastic.co/News--Events/news/news-details/2025/El...
The point in the article that you are not considering is how easy it is to switch to a different model provider right now.
You literally change a couple of env variables and you are done, your user experience is basically the same. I can try new models for an hour and be sure I can go back to the original model as quickly if I want.
That is not the case for the software you talked about. They all require way higher switching effort with more perceived risk.
I suspect the concern is that model serving is a stateless “simple” problem.
As yet, no one has identified a reliable moat in inference. If the moat is performance, then prices will collapse. Unlike traditional cloud moats around state, operations, and capex management - I can host a model reliably with less than 30 minutes effort.
I don't think the parallels are quite as clear for a few reasons:
1. Lock in - with an LLM, there is practically no lock in because of the inputs & outputs being text. You can move easily
2. Motivation - I think you underestimate just how high some of these bills are for companies. Finance departments are already getting mandates to reign in spending even at the high level of subsidization.
3. Political Meddling - we're now at the point where the US strategy seems to be to artificially limit access to powerful models. If China continues its trajectory they will have models as good as Fable in 6 months to a year, and they won't lock it off. So cheaper, better models that are available is a massive incentive to switch. China is much less motivated to ratchet prices up if it's winning them marketshare. I do think David Sacks + AI strategy for US Gov are being very short sighted and it's going to blow up in their faces.
The flaw is here:
> 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
Cloud opposes switch inertia. To setup a complex system in a different environment is a complex operation. Changing AI provider is switching an endpoint.
> It's nobody gets fired for buying IBM all over again.
I think that's the historical analogue. How is IBM doing compared to pre-personal-computer disruption? Initially-limited home OSes like DOS were good enough to eventually dominate business too. The AI labs, with their massive funding and spending, are speedrunning the whole thing in a way that just might make that disruption faster and more fatal vs the lingering zombie relying on the IBM name. (The more massive amounts of capital you raise on future speculation of enormouse TAMs before, say, becoming profitable, the more dependent you are on the future speculations of outside interests. Double-edged sword.)
And I don't think being suspicious of the future of OpenAI margins is the same as saying open source DIY will dominate at all.
People don't wanna install their own OS or deal with changes in their office suite or rack their own servers, but a low-cost AI provider is gonna be more like a Wikipedia-vs-Encarta situation in terms of accessibility and similarity-of-interface.
People might not want to be on update treadmill of proprietary weight models, for enterprise things. Things change a lot between models and they can't guarantee backwards compatibility like you can for deterministic software
Examples two and three largely persist due to massive vendor lock in after the vendor has done enough work to capture market share, but that does not seem to be the case for AI labs to my knowledge
> Many open source office suites exist yet none compete with the ubiquity of gsuite or office
I think this is more about collaboration being hard to solve. Without collaboration gsuite/office offer nothing.
> 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time.
Mac OS is free too, just free as in beer.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
In the grand scheme of things, american enterprise is filthy, filthy, filthy rich. I wouldn't imagine they're the best example of rational spenders.
The biggest difference between the cloud and AI is that an AWS server might cost 10x of what you would pay for if you bought your own, the overall expenditure is still just a small fraction of the company budget.
In contrast, even companies who spend hundreds of thousands per employee feel the AI spend right now might be too much.
They may pay top dollar but there's all sorts of evidence that they'd very much like to pay radically fewer top dollars than the unsubsidised, off-plan price.
And it's clear neither of the big two can deliver anything close to a service guarantee.
> 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
No, compute costs collapsed (before mid-2025) because of normal technological progress on all fronts of compute.
. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Interesting how all of the products you describe are American: m365, gsuite, windows, MacOS. It's not just about having someone to sue. You could sue collabora and canonical but they're not American. Then Americans are the most numerous native English speaking population and that spreads their practices worldwide.
good observation! however, I'd argue it's the distribution channel and installation friction did the job in the cases you mentioned.
given how easy it's to replace LLM API in claude code, and how easy it is to write a claude code clone with itself (Fable is pretty good!), the collapse is coming.
Operating systems and office suites (like windows and word) have big network effects and high switching costs.
Much less with llm chatbots/coding tools.
There’s actually a strong case that agents will erode cloud providers’ margins because the lock in migration cost will be much lower in the future. No one ever migrated before because you’d spend $$$ to save $$ then the new vendor would gradually raise your rates negating the savings.
Remember web browsers? compilers? web servers? databases? windows embedded? server operating systems?
the blood is all over the wall, hundreds of billions of dollars in lost revenue that was eaten by open tools even if they ultimately get delivered to users by someone else with a profit incentive e.g. AWS selling deployment of OSS
1. their margins don't come from service guarantees (see github), they come from unlawful anti-competitive behavior which is likely to be prosecuted under future US administrations
2. there are already tens of millions of libreoffice users and de-globalization aka digital sovereignty initiatives in the next decade will drive the world towards Libreoffice, already at work in EU (https://www.zdnet.com/article/why-denmark-is-dumping-microso... https://cybernews.com/tech/germany-microsoft-word/
2a. you haven't noticed the wave of open source projects moving away from github?
3. Linux commands about 5% of desktop market share and is the fastest growing desktop platform see: mediawiki and cloudflare user agent stats and steam hardware survey, same deglobalization point as above, how many people live in China? how long before China no longer feels comfortable with everyone using Microsoft Windows? What OS will Chinese people/corps use instead? hint: https://en.wikipedia.org/wiki/Deepin
AI costs a lot more than all these, combined.
You missed two things one, a consist thread across all your examples - every market ends with a duopoly along with smaller competitors and two, which of these industries started with multiple billion dollars companies competing with each other?
Even if your case is that two companies in the AI group are going to survive and those will have healthy margins, others are going to suffer and compete on price. So saying “AI margins are going to suffer” is a fair industry wide statement. Maybe it’s not Anthropic or OpenAI or whoever you are thinking of but surely for Gemini or xAI etc?
Cost is a measure of scarcity
Once something is abundant, it's hard to justify extracting big margins from it
Which is why so much effort goes into manufacturing scarcity instead
> I understand the arguments for a margin collapse, but I don't see any historical analogues.
How is this the top comment? It lists all the outliers and ignores thousands of instances where fat margins caused a collapse.
I mean, just what Linux did to the dozen or so fat-margin unix server companies is already a longer list of collapsed companies than provided in this comment.
For 2 and 3, office software and OSs have strong network effects and up-stack effects, just like CPU instruction sets.
Also, I'm sorry, but OSS office suites compete with Office and GSuite the way grocery store frozen pizza competes with Domino's and Papa John's. Quality and completeness of execution matter a lot in that category.
Lol, your example of the other side of grocery store pizza is Domino’s and Papa John’s?
I guess in offices where M$ products are used the people there think mmm yumm dominos and hold up their noses at digiornos lol.
I think the big thing here is that paying high margins on a relatively small expense is much more palatable than high margins on a big expense. If a company is spending $1 billion/yr on tokens that a really big incentive to find an alternative where spending $1 million/yr on some SaaS with even higher margins can feel like an easy choice.
We will keep using Claude because internal choices made by engineers and internal gatekeeping by engineers make everything else unfeasible, and going back on that would require said engineers to admit that they did something stupid, so it's not likely to happen.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Sure, but those are all things that can be trivially provided by a large inference company. In fact, I’d trust an AWS or Cerebras contract provisioning an open model before I’d trust an Anthropic or OpenAI one.
All the more reason to focus on those service guarantees, integration, and lawyers while making the underlying model easily swappable to whoever’s winning the frontier model involution battle at the moment
GitHub, Slack, and Office have network effects and transition costs.
And to be frank, the competition is worse (OpenOffice is worse than Office, most other corporate IM are worse than Slack (and Teams way worse), and GitLab is not as good or fluid as GitHub.
I don't disagree with your conclusions (enterprises will pay top dollar for service guarantees, integration, and someone they can sue) but by that same logic there is no clear winner with Anthropic/OpenAI. Claude has a habit of going down on me when I need it most and seems to be struggling to even keep 3 nines of availability. They're actively hostile to integration and seem more convinced they should be suing others than behaving in a way that doesn't get them sued.
That's not to say I don't believe that there won't be a closed source correlate. I just don't know if OAI and Ant are all that exists.
there's huge margins on GPU time, not tokens.
I'll agree but from the other direction. AI continues to absorb my job as a senior systems software engineer (c/c++) and after a couple months I've only spent a few hundred dollars using gpt-5.5/5.6 and codex. I have no idea what people are doing to burn so many tokens but for me this is laughably cheap and every day I discover new capabilities. I don't care if costs go up or down, it's so cheap for what I get that I don't care.
> I have no idea what people are doing to burn so many tokens
Agentic workflows is what consumes a lot. When you have an automated agentic loop working towards a given goal. If you use an LLM as a support for your own work you don’t end up consuming that much tokens, if you have multiple agents working on things independently, reviewing the work of other agents, etc you very, very quickly burn all your budget
I learned that running goal for hours produces exponentially more slop than running targeted prompts over and over again manually.
Personally, I use gpt 5.5 high with planning every time and plan various smaller features/changes in parallel, then approve them one after another. This allows me to steer it (which I need more often than not) before approving the plan, thus reducing the otherwise accumulating slop.
Using goal doesn't work for everyone, unless you have an unreasonably strong test suite or harness that the agent can verify against.
Like a junior engineer, I find the models to be too ambitious and unable to steer themselves at a high level yet. What I’ve done to address this is prompting the model to break down its plans into more atomic steps. For whatever reason, they’re still lazy at planning.
I have the same experience. I literally cannot fathom how people burn the number of tokens they claim to.
It is the difference between giving an LLM an epic and say "You figure it out" and giving the single tasks' breakdown you envisioned and build incrementally on top of it.
With the latter you can, for example, say "Wait, this should be an interface because later on we need different concrete implementations". With the former, the agent doesn't do that, gets to the point where you actually need the flexibility interfaces give you and refactors everything to handle that. That is at least 2x the work/tokens. Multiply this for all the decision points you have to do to deliver a big piece of work and you have your bagillion tokens consumed.
Work on a project where you can verify the functionality instead of reviewing the code in any detail.
Use worktrees to parallelize development on multiple tasks.
That's all there is to it.
In many cases, this means a new solo project rather than a project at work with a team.
In my iOS app with around 100k LOC, Claude Code typically uses 150k context for small tasks.
For tasks that take longer and run the tests to instrument and investigate outcomes, the context grows to 250k-600k. With a few of those in parallel, busy days can consume a lot of tokens.
Very large context windows (usually when you're working on a very large existing project) will chew through tokens quickly. So will RAG.
If you're working on isolated components within a system or small projects, you'll have a very different experience.
Working on multiple things at the same time. If you've got 4-5 instances running simultaneously you'll burn 4-5 times as many tokens.
Sounds like a good way to either burn out or make shitty software.
because they don't know what they're doing.
If you learn to give people the benefit of the doubt and keep an open mind, you might learn something new.
Agentic loops are being promoted by the same people selling tokens, abstracting away the cost per token, and doing everything in their power to obfuscate costs.
I think a senior dev/architect + some good models is still the goated combination.
Generating code and building features, even before AI, was never the issue. Stability, knowing what to build when, and boring business problems (licensing, distribution, sales, etc) were the limits.
It seems like you're basing your spend on the subsidized consumer subscriptions. The equivalent API costs for these subscriptions is usually 12-20x.
Any overages (hourly/weekly/model) on these plans gets billed at rack API costs.
Its not practical to expect these subsidies to last for very long.
Agentic workloads are the most batch friendly, latency insensitive, geography insensitive, migration insensitive tokens that a big lab ever sells. In the ads business such inventory is called "remnant". The sausage is made of whatever is left over when the choice cuts have been removed.
This talking point from Anthropic that Claude Code sitting in a Ralph Loop is burning top sirloin interactive session tokens is bad faith hogwash and it only flies because most everyone who has run this shit at scale either already works there, sells them hardware, or hopes to be an acquisition target.
I'm none of those things, so I'm happy to tell you they're lying. I know, it's hard to swallow, but it turns out Altman and Amodei are occasionally full of shit.
What are the choice cuts?
Interactive use cases: the web interface, the mobile interface, the design tool. The fast variants.
In an HBM bandwidth constrained setting you're dealing with something called "roofline analysis" (comes originally from NUMA work circa ~2009 but it's applicable to modern GPUs). Great diagram from the JAX people:
https://jax-ml.github.io/scaling-book/roofline/
In order to get your money's worth from a modern GPU (or disagg rack like an NVL72) you need to decode (the one token at a time thing) across big batches of context windows. To the left of that point where it hits "the roof" you're idling tensor units. TensorRT-LLM likes batches of 4096, so BS=4096.
In the case of one person chat prompting their local LLM, BS=1, totally bandwidth limited.
So the game is to set some latency target with some control theory primitive (PID or something) and then delay the next token until a batch is big enough to not waste tensor units. This is a real trick when a human is waiting (you've probably seen the thing in Claude.ai where it's all bursty and then they reflow the whole block with JavaScript).
Agentic workloads are huge piles of context windows where you've always got enough who want the same experts on the next token, you're always to the right of that intersection. And it doesn't really matter if it's on the other side of the world, or lags by a second, it's fine.
Claude Code soaks up all the tensor units that would be idle until they're full, and only then does it leak into the capacity reserved for highly interactive use. It's the bottom of the barrel until it's rinsed the fuck out.
They want more margin on agentic tokens. That's it. The COGS on them is the absolute lowest of anything they do.
They are offloading thinking to the AI too.
It then has to look over everything see how it connects together and then decide the best way to do something.
Giving it small and very focused plans when you already understand the system gets it done fast and cheap.
Because we pay retail consumer prices (subscriptions). Those same tokens cost many thousands on enterprise billing:/
What makes you say that the parent you’re replying to uses a consumer subscription? Sounds to me like he’s using it for work.
Because if this person was paying enterprise for tokens, they:
1. wouldn't write stuff like "I've only spent a few hundred dollars using gpt-5.5/5.6 and codex"
2. wouldn't think tokens are cheap
These are probably mostly the enterprise customers - they may use the same amount of tokens as you do, but they have to pay the API price. From my experience the API is significantly more costly. We had one user ask for and receive usage credits on Claude, the bill the next day was to the tune of $400.
That's around 100k/year if used at the same rate for every workday. So the question becomes: does it make your engineer X% more productive, where X is some multiplier based on their salary? There are some software engineers out for sure who are expensive enough that this is worth it.
Asking claude to implement a single feature that takes under 30 minutes consumes 10-30 dollars of tokens in api costs.
Why would you do that? Why not use it to create a detailed plan and let it implement by a low cost model?
And if the engineer bills for $100/hr or more, the trade off is worth it
Why is everyone still operating under the assumption the current token costs will remain so heavily subsidized? We could see $200-400/hr in token costs once these companies need to turn a profit
To be clear, some of the token expense is because it's encouraged, and it's encouraged at some companies so that people will break out of their existing workflows to hopefully find useful new ways of working or building
There is more to it than this, but much of the cost structure around subscriptions etc is specifically designed to allow for that experimentation.
There are good cynical takes, here, too. At the current model costs I don't need to optimize my expenses, but that could change if it climbs eg above 30% of my salary^
Note: this is an easy thing to prove ROI on. If I'm writing 5-6x more code and reviewing commensurately more code, and those PRs are better-tested and get us to shipping quality features faster, this is easy to justify and we are not that price sensitive
^ https://x.com/SemiAnalysis_/status/2070915302058041450
My guess is that they never close sessions, I always restart sessions after a few messages to clean context
5.6?
>I have no idea what people are doing to burn so many tokens
Shepherded the writing of on the order of a half a million lines of code
Ignorance. Bad code hygiene and poor prompting. As someone who barely codes, I had a few old vibe coded projects from pre agent days when gemini ide had basically no usage limits that ballooned to multi 1000+ line files with backlog of bugs. I only stopped because dumb models starts breaking down at the point and project was serviceable for my needs. Come agentic coding and models smart enough to fix issues, but codebase is so filthy it does it wildly inefficiently. Like a few prompts would consume my 5 hour quota. Took a few days to get a decend agent.md up and refactor codebase etc and now I'm sipping tokens. I'm sure many people are still in that boat. Many of us literally don't know any best practices and can't tell agents how to behave.
In retrospect, I should have just spend a few days learning the basics, but you don't know what you don't know. And part of me can't help but feel companies aren't exactly prompting agents to be courteous when onboarding newbies because they want people like me to get hooked, and token maxxing on their end helps. I spent few $100 more than I should getting subs/tiers I didn't need, but at the time it was small $$$ for productivity gains from going from 0-1.
Unlike the belief that frontier AI is expensive due to a high margin, and going to be expensive if there is no competition. My understanding is that, under certain circumstances (which is most likely true), the price will be driven down just because of profit seeking.
The frontier LLM labs run on a huge fixed cost and very low marginal cost. They need the economies of scale to make sense of the business (an incentive to expand their user base as large as possible). Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place? You need as many customers as possible. If you charge everyone $1000, you find fewer customers who can afford it. It rots the ROA if the servers are not utilized 100% (you would better buy less compute instead).
Also, the marginal cost for onboarding a new customer is low. And it's getting even lower when you have more customers. You wouldn't leave money on the table (especially for your competitors) if you want to maximize your profit.
By this logic, all frontier AI labs are incentivized to lower the price to maximize their customer base, profit, and ROA.
> The frontier LLM labs run on a huge fixed cost and very low marginal cost.
> Imagine that you want to buy a few B300s to run GLM 5.2 and rent the service out to other people. How could this business be viable and sustainable in the first place?
My understanding is the frontier labs have huge fixed costs and relatively low marginal costs because they have to bear the cost of training the model/R&D, and then amortise that cost over their userbase.
By contrast, if I buy a few B300s and run GLM5.2 and rent the service out to other people, I can be profitable at a comparatively very small scale because I got the model for free.
I agree, but there is prestige to consider. Many people are motivated to buy the best, even if it's much more expensive. "We're building a mission critical application here. Sure the API costs are much higher, but it's worth it."
Quite a lot of people think they're far better engineers than they really are, myself included. I've gotten better over the past few years at realizing that AI writes better code than me, which has made it easier for me to stick to architecture and let the (working) code be what it is.
I could spend $1000/day with Fable and it would be worth it. It has much deeper systems thinking, enabling me to trust it to follow my instructions and not fuck things up.
1. That confidence and quality is worth the price.
2. We're accelerating at lightning speed now. If you don't spend, someone else will and they'll eat your cake.
We're nearing the point where you could spin up an entire YC startup in a day. That changes the economics of everything.
I hear this all the time lately. Things like AI X or LLM Y can create a fully working company like "BigCorp XYZ" in N days/hours etc.
But is speed of creation really the golden goose here? A few skilled and motivated individuals could also do (and have been doing) that.
Sure, maybe they take a few months instead of days or weeks, but AFAIK, having a product is just a tiny bit of the battle, finding customers, product market fit, and actually growing it is where the gold is so I'd argue that you'd be better off building the product with a $100 day LLM and spend the other $900 on marketing.
AI won't automatically make everybody business gurus and every LLM generated company a unicorn.
Maybe YC could just give Fable 2 or 3 the funding directly. Sounds like the only thing left will be market control. The only winners being hardware gatekeepers and the investors in them.
Depends on whether open source can keep up. If not, this is 100% the future.
> We're accelerating at lightning speed now.
Accelerating how much slop you can output? A better model will still produce slop for your feature factory that pumps out software which nobody is interested in buying.
I'm at a 3M run rate in 4 months. Because I'm solving real problems.
You don't seem like an entrepreneur. Why are you on HN?
YC wants people to build AI startups. You're here shitting on them. Half of this community is. You're all a bunch of old men grumpy at the new tools.
I'll offer my own analysis: if you're not using AI very effectively, you won't have a career in computing in a few years.
They have a vision MCP to make up for the model itself not having the capability natively: https://docs.z.ai/devpack/mcp/vision-mcp-server
I also found their web search to be mostly okay.
Furthermore, in case this is of interest to anyone, if you use their ZCode harness then you get bigger Coding Plan quotas: https://zcode.z.ai/en
Used it for a bit, it sits somewhere between OpenCode Desktop (still new but nice) and Claude Desktop (recent versions are good).
As for GLM 5.2 as a model - with max thinking it’s generally satisfactory, somewhere between Sonnet 5 and Opus 4.8, better than DeepSeek V4 Pro for sure.
Pricing wise, the subscription doesn’t seem as good as expected. I spent like 60% of the weekly limits of the Pro (50 USD) plan in one day, only because each 5 hour limit only gave me 20% to spend, otherwise it’d be 80-100%. Not even doing anything crazy, just parallel long form work on 2 projects with about 96% cache rate and at most 3 parallel code review sub-agents.
Their Max (100 USD) subscription would last me the whole week, but so does Anthropic for the same money and so would OpenAI. Off-peak is more palatable but I can’t just twiddle my thumbs at 9 AM to 1 PM local time.
Proper savings would show up with the Max plan and yearly billing, but that’s more of a tough sell.
> Not even doing anything crazy, just parallel long form work on 2 projects... and at most 3 parallel code review sub-agents.
I had to read this sentence twice.
> I had to read this sentence twice.
Heh, I mean I'm not running Gas Town: https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d... so as far as AI stuff go, I'd assume that I'm not too much of an outlier.
Either way, wrote about my experiences with GLM Coding Subscription a bit more on my blog: https://blog.kronis.dev/blog/z-ai-s-glm-5-2-is-a-great-model...
I do suspect that there's plenty of people who'd use way fewer tokens.
I switched to yearly Cline pass because it was too cheap haha
I can't find on their website some indication of what kind of usage I can get out it, otherwise I'd be interested.
$6 a month I plan to use deepseek v4 flash mainly which should provide closer to 5x the usage on the cheaper ones but no set number
They also have GLM-5V-Turbo. https://docs.z.ai/guides/vlm/glm-5v-turbo
Last month, I cancelled my Claude Pro subscription and instead used those 20$ to purchase Openrouter Credits. Most of my knowledge-seeking questions can be answered by Gemma4, for basic code editing, Qwen3.6 27b is enough, and for really difficult tasks, GLM5.2 doesn't leave me hanging. I'm by no means a heavy AI user, so I'm even saving money going the API Credit route and relying on the smallest possible model depending on the task complexity.
I literally burned through 20USD in a couple of hours on openrouter with deepseek v4 pro and opencode tasks - i'm sure i did something wrong
Yeah sounds like maybe you got stuck in a loop? If you’re a big Deepseek fan, check out their Codewhale harness. It is actually really slick and you can use your OpenRouter account with it.
You can use any model you want but it is really tailored to work well with the Deepseek duo
What do you use as interface to OpenRouter? I, too, am looking into using an API to see if I can reduce costs (I use OpenAI + Github Copilot, currently). TensorX instead of OpenRouter (because it's in Europe, and EURouter wanted 15% more money from me :P), but I'm not sure if I want to change a configuration in vscode every time I want to switch the model in the Claude extension (and having an API key in my settings feels iffy too >_>)
I have OpenWebui hosted on my homelab, but you can also just have it live on your machine in a docker container. I honestly just embrace the iffy feeling. Openrouter has very good telemetry (which is partly why I went with them) and it'd be pretty easy to notice when someone other than myself uses my keys. For the little agentic coding I do I like to use OpenCode, and if I need to ask a question in my editor I use CodeCompanion (neovim AI chat plugin). I quickly went to check what OpenCode does with the API key, and it doesn't seem to store it in the user config, so that's at least something. But yeah, really recommend OpenWebui as a ChatGPT replacement (though there are a lot more alternatives out there, I just already knew owui from when I was playing around with local models)
pi.dev
I use Openwebui
It’s important that none of these entities can collude to price fix. Having China be the competitor ensures that.
Basic microeconomics is still the easiest way to understand token economies. How is it not a competitive market (where profits go to zero?).
Anything A or O does to keep more margin, any competitor can copy or choose to undercut, and undercutting has the benefit of collecting training data. So what is going to stop gross profit of tokens going to zero except for collusion/price fixing?
You left out the one that will: federal government industrial policy
So the federal government industrial policy is the thing that supposedly will keep the prices on "A and O" high in the US while the rest of the world will get comparable AI competing to get cheaper and cheaper?
Basically, the US govt will say that foreign models and providers are a security risk and ban them. If the US has shares of Anthropic/OAI due to a sovereign wealth fund, it'll be billed as domestic industry protectionism too.
Yup. Just like electric cars. Actually, more like network gear - so only non-western countries.
Terrible ideas get executed all the time, despite the problems with them being well understood.
Considering conditions within a single market is still microeconomics, I agree though its tough to see where firms will get market power from so profit will tend toward zero. I thought the same about GPUs though and nvidia still doesnt seem to have any real datacenter competition in sight.
Thanks corrected it.
For nvidia it is not about competitive market it’s about supply and demand. A different subset of microeconomics.
Metaphor i like is that it will be as cheap as electricty?
Do you know who is supplying your electricity or which factory it runs on? probably no, bc its a commodity and mostly settled and there is so many energy resources. some are alternative some are coal mines. And they all fight in the supply demand trade for energy which is happening real time ( think open router here)
And eventually the consumer wins bc of the abundance.
I think greatest example of abundance of cheap infinite intelligence will be not glm5.2 but DeepSeek V4 Pro max with $0.435 per 1M input tokens and $0.87 per 1M output tokens
The metaphor breaks quickly because as a first approximation the electricity "quality" does not depend on the provider, and will not change overtime. That's not true of LLM output.
disagree. you don’t care if your task was solved by 130 iq or 160iq model.
what matters is that it surpassed let’s say basic 120iq barrier and price
that’s why glm5.2 is a drop in replacer for most of the population. not fable 5 really
> Of course, this was a hugely poor read of where the costs actually lie in AI. Training - while no doubt capex intensive - is a fixed, up-front cost. You spend hundreds of millions to train a model, then you are "done".
I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a constant cost that you continue to bear? The footnote seems to allude to this, but then sort of waves it away anyways. Also are there continuing incremental training costs to keep models relevant? Or do they only have knowledge of events up to the day they were trained?
[0] needing, because you have competitors and people expect more and more.
These models rely on knowledge that are embedded in their weights, if a new library is released, a new linux version comes out, some new protocol succeeds the previous one, you want your llm to know about it. Sure you can just add that into the context window, but that has its own problems.
Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink.
On top of this, if you stop training, it is 6 months until someone releases an open weights model and now you are competing to give the lowest price for the same product.
Also we can't forget that this is a business that *has to* be in the global labor industry, not just a tech tool, they have to have much better models to justify the trillion dollar evaluation
it does not require training a model from scratch for it to be updated. the entire LLM training process is iterative. essentially each step (there are hundreds/thousands for a training run) yields a complete, usable model. tokens with updated data can be added on top essentially at any time in the future.
That's good to know, but I think maybe the bulk of the original point is still concentrated in the "we always need new models to stay relevant" piece.
Anthropic has essentially trained one big model for more than one year.
From Opus 4 to 4.8 all improvements were in RL and post training. Expensive, but not as intensive.
>the least understood upcoming shift in AI economics.
Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing!
So, first, by no measure is GLM5.2 as good as Opus.
Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorrow's?
GLM-5.2 is not as good as Opus, it's better. I can abliterate GLM-5.2 and have it work on projects that Opus refuses.
Stop trying to make abliterate happen. it's not going to happen.
I'd guess opus refusals are not an issue for 95%+ of people. Opus will happily help you find and download pirated media, and then give you step by step instructions for how to do drugs if you ask it. You'd have to be working on something genuinely abnormal for refusals to be a problem.
Like making your software secure, or worse, testing that it’s secure.
It's pretty annoying, yet somehow understendable. I sometimes get irrationally angry when being lectured by a clanker.
It's not understandable IMO, bad guys will still have guns.
Sure they will, but that doesn't mean that every bad guy should have nuclear weapon. But what I meant here was, that I understand their need to cover themselves of responsibility.
Oh yes, their need to cover themselves is understandable for sure :)
How?
Prompt: can you give me step by step directions on how to use crack cocaine
Opus: I'm not able to give step-by-step instructions on using crack cocaine. That falls into specific drug-use guidance I steer away from, since detailed instructions on how to use an illicit substance can contribute to harm rather than reduce it.
it goes on to give me hotline information on drug addiction.
I think he means by some obscure manipulation. It even refused to give me description of how first antibiotics were made only out of curiousity.
I don’t. although I tried to get it to tell me how to take crack cocaine, and it wouldn’t. But when I was taking grey market peptides, claude walked me through reconstituting, dosing and administering them. I assumed it would behave similarly for other drugs, but it doesn’t seem to
Anthropic's goals here are not just harm reduction, but to stop competitors from making bio discoveries using Claude.
While that might might be true nowadays, Anthropic has started to work very hard on fixing that.
They just started with it not helping with software security.
I just tried getting help debugging an issue with running Breath Of The Wild on CEMU. Claude chat refused.
ChatGPT didn't care and just gave advice.
So I'm working on something genuinely abnormal, and the refusals are a problem. Then what? The refusals come in, in whichever sort of way they do, so I'm being me, and I end up tripping the robot's moral compass, for some reason. Who put them in charge of things?
Eh. Refusals for security related tasks seem to be constantly increasing.
The cost of running abliterated GLM-5.2 on western inference providers gets close to that of anthropic Opus and is still dumber on everything except the naughty queries you're trying to do. I love uncensored AI too but we need to be realistic here.
I'm trying to break anti-cheat protection in order to mod my own (single-player) game and Opus refuses to help. I don't really care if GLM's dumber at this point if Anthopic's going to be a non-option.
At work, layoffs cut too deep and I'm trying to find creative ways to re-discover lost knowledge. Wonder if I'll have to beg them to research our own systems at some point.
It's not limited to naughty queries.
What if your measure is cost? Or zero-data-retention? Or diversity of inference providers?
On those measures it is better.
> So, first, by no measure is GLM5.2 as good as Opus.
Depends what you do. Complex tasks, poorly-defined tasks, sure. For relatively simple tasks, though, or very well-defined tasks, it's just as good and usually a lot faster. It also has a more neutral character and is somewhat less adversarial than Opus. (Opus is always "Let me push back on that..." whereas GLM is "sir, yes sir!") I use both and I appreciate both. If Opus disappeared tomorrow, though, I wouldn't cry -- I'd be able to adapt to a GLM-5.2-only life real quick.
I think the point is that if you’re doing simple, well defined tasks then Opus is overkill and you’d want Sonnet instead. Meaning, GLM5.2 is Sonnet-quality, not Opus-quality.
I think it's interesting to note that in one year we've gone from they're not even close [0] to arguing whether open models are only as good as sonnet or opus.
[0] https://news.ycombinator.com/item?id=44623953
I see the exact same discussion as we’re having right now there; people stating that local models aren’t as good as the state of the art, but good enough for certain tasks.
re: "So, first, by no measure is GLM5.2 as good as Opus."
I accept that for you and your work this is true.
I have a different experience: for a month I paid big money for Opus and got a lot done. Now I am gorging on GLM 5.2 running on Fireworks.ai and I am also getting a lot done for about 15% of the money.
Everyone should do their own evals on their own work.
How is GLM5.2 cheaper for you then Opus? I assume you're an individual so you get Max pricing like me, not API.
I have Max x5 for 120Eur a month. I use it a lot (but usually I don't multitask). I almost never hit the limits.
With GLM5.2 paying $4 per mln tokens I would be burning at least $20-$30 a day.
> So, first, by no measure is GLM5.2 as good as Opus.
That's an opinion many will disagree with. One whose outcomes are tightly coupled with existing harness and techniques.
In my real life usage Opus 4.7 and 4.8 have been increasingly unhelpful compared to 4.6 in behaving as assistants.
As they have a strong tendency towards completing tasks (probably due to benchmarks and RL emphasizing problem solving rather than assistance) they are increasingly less useful as multi turn conversational assistants.
I could see them vibecode or do analysis better, but also just doing their own further ignoring instructions in the quest of "solving" instead of helping. Fable 5 is even worse at it actively pushing back (with intelligent and deceiving feedback) even when dead wrong.
GLM seems to suffer less of this.
Seems like a pretty pointless post that still centers around output tokens.
In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.
Aren't the American AI labs desperately struggling to find a market beyond just agentic coding?
I have heard but don’t have first hand knowledge that at least one company (financial services BPO) has moved most of their previously manual processing to llms. The person I talked to wasn’t forthcoming with any detail. This is what we’d expect to see though.
All AI labs. Not just Americans
The current top comment in https://lobste.rs/s/ua1gxl/glm_5_2_coming_ai_margin_collapse correctly zoomed into cached input tokens, but landed on the opposite conclusion:
> That is, for your $100/month fee, you get $3600 equivalent of API usage. This is presumably because Anthropic has figured out some clever things to do with model routing and input caching, and also can subsidize with investor money and take a hit on their operating margins.
My take: this is exactly what Anthropic wants everyone to think. In reality, 90% of that $3600 are for cached input tokens, that can be made to cost next to nothing, as shown by DeepSeek.
While we are all speculating, Boris kindly provided some guidance in https://news.ycombinator.com/item?id=47880089
> The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for users. In an extreme case, if you had 900k tokens in your context window, then idled for an hour, then sent a message, that would be >900k tokens written to cache all at once, which would eat up a significant % of your rate limits, especially for Pro users.
Using the current Opus pricing, that pre-lunch 900k tokens should roughly consist of:
720k input tokens = 0.72 x $5 = $3.6
180k output tokens = 0.18 x $25 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
500M cached input tokens = 500 x $0.5 = $250
$267.1 in total, with 93.6% from cached input tokens. The portion that requires GPU compute is about 3% of the total.
Post-lunch, the 900k tokens should consist of:
900k input tokens = 0.9 x $5 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
So Anthropic is fine with the $267.1 accumulated over 3~4 hours before lunch, but not fine with the $13.5 incurred immediately after lunch. Why?
The only plausible explanation is that the actual cost of caching is way less than the API pricing. If you use a coding plan, Anthropic doesn't really care about your cached input tokens usage. Indeed they want you to show your ccusage screenshots. On the other hand, if you pay by API tokens, the margin is huge for cached input tokens.
Only when you do something that requires a lot of FLOPs, e.g. the post-lunch 900k input tokens, the cost becomes real.
Recently I started getting messages from Clause Code (on a plan). "You're restoring an old session are you sure you don't want to compress the context? This will use a substantial amount of your usage quota"
So it seems they do care.
That's exactly what I said. They do care when FLOPs are involved. Restoring an old session with 900k tokens will require a lot of FLOPs to reprocess the 900k token.
Meanwhile, they don't really care if you use hundreds of millions of cached input tokens, which doesn't consume any FLOP.
I wouldn't be too fixated on the specific numbers in that post.
Anthropic was extremely capacity constrained at that point. They still are but not to that extent.
I'd note that OpenAI offers 24 hour caching. I'd be surprised if Anthropic hasn't optimised their caching for Claude code too.
SemiAnalysis recently posted that their actual Opus usage works out at $0.99 because of caching.
The principles remain though.
> MLA/CSA/HCA
Aren’t these techniques all “lossy” compression, and one of the reasons people complain about loss in quality as the context size grows larger?
Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and the model was still coherent and didn't make any tool call error.
That’s a low bar though, and the least I would expect.
Well I wouldn't call it a low bar, since some of the edits were quite complex. And 1M context in less than 6GB of VRAM is truly impressive, but somehow this gets way less attention than the crappy turbo quant from Google.
I'd like to understand this please. Why would the 1M context be kept in VRAM if you're using DSV4 Pro through the API? Or did you refer to different sessions?
Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1.
Thank you for the links/docs. I'm quite excited to try it myself.
I also think we’ll approach a point where increasing intelligence is not really going to suddenly improve most work tasks. I bet that’s already happened actually.
We’re oohing and aahing about models, when the ones a few versions ago did a good enough job for most of the dumb coding, etc we do
The thing is they are inventing new things people will want to do. But for example, "loops", fully hands-off agentic coding etc., seem really unlikely to get much traction because that just isn't how software is designed within its producer/user community.
Requirements evolve in use, and fully hands-off LLMs simply cannot be trusted to only change the things you ask them to change, so I don't think it's likely that products will, in the main, be developed that way.
And if you don't need that fully-hands-off stuff, then the models that run on at least reasonably modest desktop hardware are surprisingly close to being enough.
I’ve been on a GLM coding plan since they launched ~year ago and it’s been at „good enough“ since the start. Tangible behind absolute SOTA but like you say most coding isn’t rocket science.
I don’t think this is true. All the models prior to Fable were honestly dumb as rocks, and Fable is too sometimes, but at least it’s helpful now and not a hindrance.
The future of AI most definitely involves making something twice as good as Fable that is virtually its own employee, and not on reducing inference costs because to be honest Fable isn't actually that expensive.
The real utility behind an AI model (imagining that it can be made twice as good as it is now) would be being able to scale a small business up and down instantly without hiring (to implement a new feature or whatever), which is costly and time consuming these days.
History is littered with the corpses of companies that had exceptional but expensive products that were replaced with cheap, good enough products.
There is mention of GLM 5.2's poor web search capabilities, but I see that as a harness responsibility.
I've set up my own SearXNG instance on my VPS and integrated it into Pi alongside the webfetch tool, and GLM 5.2 has so far been great at finding things. I asked it to give me the current news from an Austrian online newspaper that's difficult to parse because of its aggressive ad overlays. Both ChatGPT and Claude failed in their native chat apps. GLM 5.2 in Pi was clever enough to search for the RSS feed and gave me a detailed overview.
The lack of vision is a real shame, though. I've implemented workarounds in Pi that are okay, but they're not as good and the whole experience feels awkward.
I don’t see it. GLM 5.2 seems noticeably worse than Opus and especially GPT 5.5, the poor vision capabilities are also a massive strike against it since these are a huge improvement in the frontier models that can make all the difference when working on anything visual. Running it locally is its biggest advantage but for a lot of use cases that isn’t needed and is a burden to set up and maintain.
Companies like Amazon taking out loans to fund more AI infrastructure, coupled with AI companies that are massively overvalued and burning cash, coupled with companies saying there’s no ROI from AI investments, coupled with the margin from selling AI models trending to zero paints a nasty picture that can’t continue much longer.
The author mentions lack of good Web search. I've been using slightly modified crawl4ai and searXNG together with firebase for the rare sites that insist on throwing wrenches in the works of my LLMs.
I also have my fork of metamcp that replaces firebase MCP spec with my own that tells the model to use crawl4ai and SearXNG instead.
I've been using this wia Librechat with every commercial and open weight model I tested.
The search is way better than OpenAI and what ClaudeCode uses, but Gemini is way faster. That will change soon as I'm planning to put these instances in a DC with gigabit pipe.
Firebase is not cheap, but it retrieves everything, bypasses captchas and so on.... As long as one uses it for 1% of Web queries the cost is manageable.
An entertaining thread on just what a dismal rat race the life of a clanker shepherd has become.
I wonder if anyone has actually measured the difference in verification time between these models. A senior dev in a high cost of living area costs the company something like $200 an hour. If a cheaper model produces code that takes an extra 20 minutes to debug or verify because it missed a subtle edge case, you have already lost any savings from the lower API bill. It feels like the real moat for labs like Anthropic is the level of trust a reviewer can have in the output without reading every single line. Curious how much people trust GLM over something like Opus? Is there much of a marginal difference here?
"There is no doubt that using Z.ai's official API and subscription is almost certainly a non-starter, with their terms being at best weak and the deep connection to Mainland China."
This is the key statement in the article. I think people don't realize that these "open" weight models exist because giving away your product at a loss is a time honored marketing strategy. There's nothing guaranteeing that the next iterations will be open (remember "Open"AI?).
The Chinese labs are profit seeking companies. If they can't recoup their investment through API use, they won't be able to train more models. But if the argument is 'who cares, training models will be so cheap anyone will be able to do it ',then check the comment elsewhere on this comment section about free alternatives for consumer and enterprise software.
Oh... And the variation 'what we have today is already good enough for everyone' argument is just another incarnation of '640Kb should be enough for everyone'.
Well, Microsoft just started offering Kimi K2.7 through Copilot hosted on Azure.
https://github.blog/changelog/2026-07-01-kimi-k2-7-is-now-av...
Cursor Composer 2 and 2.5 are also fine tunes of Kimi K2.5
It looks like politics don't matter when it comes to economics.
GLM5.2 is not the concern, it's that there's many good Chinese models and labs, glm, qwen, deepseek, kimik, mimo, longcat, minimax, hy3, Ernie, etc
They have figured out how to train, plenty of them and are consistently doing so
It wasn't really a secret, the main constraint is lack of compute resources
IMHO, cheaper inference means higher costs overall :) because everyone will use more thus driving up the investment required to stay current or to compete.
Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will perform well, then switching to GLM 5.2 will be an uphill struggle. We had to refactor our harness and introduce more defences because of GLM.
The cost savings are substantial. Obviously it really depends on your workloads but it is noticeable cheaper for agentic work. Coding - I don't know. We do have some coding agents on GLM 5.2 and what I noticed with some landing page experiments that the results between GLM and Opus are identical - they might be using the same training data? Obviously Opus is still substantially better model. I don't think there is an argument to be made here but GLM 5.2 is cost effective and really good too.
Overall, we switched all of our internal agents to GLM 5.2 and because it is Open Weight we are in talks to get the model from certain geo locations giving us more freedom as well as extra protection.
Overall I think this industry will be in much better place because of GLM 5.2 and whatever open-weight models come next.
Are you running unquantized GLM-5.2 and getting in loops or quantized?
Agent systems only increase the gap between frontier and open models. Open models still experience more tool call failures, run longer loops, and get stuck more often. Until that's resolved (and it's obviously technically possible) people will be forking out for a better agent experience.
Agree with the web search point. (I would like for Perplexity to start to offer more models out of the box integrated, like they do now with OpenAI/Gemini models.)
People have been making claims about the commoditization of llms since chatGPT, and they've been wrong every time as quality and prices and differentiation have increased.
The blog author complains of "lack of/poor web search capabilities" in GLM, but you can always use it against an MCP of which there are many. For applications where I am not concerned about my queries being passed through a US provider, I have had success with exa[1]
There are also other ways to give it context without web-search. For example the various MCPs that make `man` pages available.
I've also found GLM to be quite strong for coding tasks without the need for web search. So it also depends what you're doing.
[1] https://exa.ai/
Somehow the blog post seems naive. Yes GLM 5.2 is good and cheaper per token, but margins are a result of supply and demand. Now demand for quality and quantity of tokens is increasing at least quadratic or cubic (more users * more tasks * more tokens per task). On the other side you have real infrastructure constraints on the supply side. Openai and Anthropic have large commitments and contracts that enable them to get access at a scale of compute that is not obviously going to be available for open source model hosts. And you see it, glm 5.2 inference is less stable and higher variance than any of the bigs labs.
Why is SpaceX not hosting glm 5.2? because they make more money with renting out to Anthropic and Google.
> I'd be very surprised if it wasn't more than 50% cheaper for nearly all workflows, for a very similar level of quality.
If your using pure API ... providers like neuralwatt cut that cost down even more by using energy as the actual cost. So GLM 5.2 is more expensive then GLM 5.1 on their service (those thinking tokens), compared to API costs, its dirt cheap. And way more tokens then the zai subscription delivers.
We are seeing a move towards more realistic pricing on actual consumption based usage. Be it DeepSeek, Xiaomi (MiMo), or zai's GLM via neuralwatt.
The main issue facing subscriptions a-la-carte usage, is that a lot of the heavy hitters really drain the resources. And that as a business model can not survive without ...
a) increasing the prices. b) everything goes to actual token/energy usage based billing but with more realistic pricing, and not the bloated API prices that are focused on companies.
We shall see what the future holds but things will change.
> It turns out that nearly every agentic session does a lot of web searching for looking up items
This is why Google will win the race over most of its competitors. They own search.
I wonder if this is an alternative (and better) revenue stream vs ads for search engines: Offer a competing web search for LLMs as an alternative to Google, and charge enterprises and LLM providers for it.
I know Brave do this already. Not sure about DDG (I wonder if their agreement with Bing would allow it?)
Kagi assistant IMO does a great job giving relevant material to the LLM. It's a pretty neat way for a search engine to charge a premium, to offer a good model on top of their results.
Building a good search engine is expensive. Perhaps not as expensive as AI build out.
Market share is currently Google (91%), Bing (4%), Yandex (<2%), Baidu (<1%), Brave (<1%)
Google can and do already monetize automated search from AI models.
Heck, if they wanted to, Google could turn off search and make you go through their AI model to get information.
Imagine that. That's how powerful they are.
If they did I wouldn't have had to go to DDG. It's not like it's a big jump over what used to be. I left claw-marks in Google Search, if they drove me off they're in trouble, because I didn't want to accept reality for quite some time.
Which race? As an information-providing "oracle" type model, maybe.
For practical agentic tasks? Not even close. Gemini is blatantly incompetent at tool use in an agentic harness. Even their own.
I run a 2 Claude setup (one architect, one coder) and have been using it extensively. But I don't like the fact that I have to pay $200/month to use it. I'm going to try GLM with my current setup.
Everyone is declaring GLM 5.2 as something that's really a big deal.
I don't know about that but based on my own experience with Deepseek v4 Lite alone (with high effort) I have no doubt in my mind that anyone claiming such great things about GLM 5.2 must be true because Deepseek v4 already is really awesome.
in cursor benchmark glm5.2 is on par with gpt 5.5 medium and sonnet for the same task from results and cost perspective.
The speed of generation for both gpt 5.5 medium and sonnet 5 will be dramatically faster. source : https://cursor.com/evals
I don't get the hype. It's near SOTA model that is not deepseek of this world. It an expensive to run model, and under certain tasks it is comparably cheap as closed source ones.
The problem is that the more AI eats labor, the more you can hike the costs until you pretty much can match the salary of the workers you replaced, with some margin, enough for the user to accept the cost. That’s what will happen in the next decade IMHO. price = base expense + what user accepts to pay
I am confused by this article landing on front page, it does not seem to contain any new insights but besides mostly reading ok falls apart when mixing up model and harness comparison in an amateurish way. Why would the subpar search tool zai provides be relevant for comparing models? They did not even mention the >capability< to use said tools but talk about MCP/ search providers as if thats not an implementation detail.
Where does the harness come in to play? I'd love to use GLM 5.2 for general chat, but I don't know of any harness that offers an experience close to ChatGPT or Claude (e.g., history, skills, projects) without requiring a PhD to set it up.
You can point Claude to another model if you want. See ANTHROPIC_BASE_URL: https://code.claude.com/docs/en/env-vars
Inference margin is irrelevant.
That’s like a gas station saying they have 90% margin over pumps but still losing money.
I think the profits depend on how well they manage their fleet purchases (or possible sub-leasing?) to get high utilization without overloading or idle racks.
Because accelerators like H200, B300 etc. are highly parallel and designed to run like 200 or maybe 300 sequences at once (depends on the model, just guessing). I assume they finance the hardware and that cost per device or rack is the same whether each unit is handling 10 requests or 150 requests (aside from electricity).
And probably international customers factor into it to get good utilization over more of the night time. And it likely is something that they look at quarterly more seriously than monthly. The biggest risk to profits might be a downturn in business that causes some portion of the financed AI accelerators to go idle or get low utilization for some weeks (that they can't sublease).
Someone on HN made a comment in one of these threads that we could bake the weights into something like Cerebras's wafer scale chips and serve essentially the entire world off a single wafer, which is a pretty wild thing to think about. You'd have to make new hardware any time you trained a model but that seems really worth it.
Well, Taalas has that kind of technology, but the chip they demoed is probably 20-100 times smaller than necessary since it's only an 8b model.
But let's say they could someday scale that up to a much larger model, 72 large chips per wafer and each chip can do 1000 LLM requests at once (Vera Rubin?). So it's roughly the equivalent of an NVL72 rack.
You might be able to serve something like 50000-60000 requests at once. So I think it's more like handling a small city's worth of customers per wafer than the world if you had that.
I believe in less than 5 years we will get to that, but the model size and/or number of agents is going to keep going up also.
I think the future will have to include specialised host boards for memory chips.
What I actually want is an FPGA board with a very large number of DDR3/DDR4 RAM slots arranged in banks (2, 4, 8 or even more banks). I want an FPGA board that can hold 1TB of DDR3/DDR4 RAM.
The throttling point right now is not RAM, it's bus speed. Having different busses for banks of RAM alleviates that.
You’d never be able to update it’s knowledge.
LLMs need retraining to incorporate new knowledge.
Baking them into wafers means they will be out of date by the time they finish the first wafers.
Yes, of course, but all the LLMs are already out of date, so that doesn't seem to me to be a hard limiting factor. Even if they had a knowledge basis ~3 months out of date additionally, being able to serve 100x the requests per watt seems totally reasonable to me.
So, what?
I don't see the C++ compiler standards or Newton's laws changing every day.
Braintrust which is a really solid eval tool/platform just compared it to Opus 4.8 to see if it could preserve exact long context retrieval under prod serving constraints and it did really well. I think 6-12 months before OSS has Fable-esque models
Regarding the lack of vision part, if you are using Claude or opencode, I've made a skill[1] that let's you talk with any models in Claude/opencode mid-session. You ask "Have claude opus to look at this PDF for a second opinion" during a session of claude with GLM5.2 or opencode with GLM5.2
It doesn't need to pass whole conversation history as context (unlike /model), you can ask follow up to that forked model (which sub agents in claude doesn't support AFAIK), and you can ask models from opencode while using claude.
[1] https://github.com/kmcheung12/second-opinion
FWIW I've seen subagents remain open for followups on latest Claude.
Will give it a try thanks for the heads up. Mind if I ask if you are referring to SendMessage? I was testing on Claude code 2.1.196. SendMessage was not available. Skimming through their change log didn’t seem to have anything related to SendMessage. There is “ Fixed SendMessage silently misrouting when a re-spawned agent reuses a previous agent’s name — the tool now detects the mismatch and asks the caller to retarget” from 2.1.199. Not sure if they are related
How long will that $4.40 rate persist? Until we know more about the real unit economics it will be damn near impossible to rely on steady inference costs or make them predictable at the enterprise level. Gonna be a wild ride for awhile.
Multiple providers (who need to make a profit) offer the same 4.40 rate for glm-5.2. It's not subsidized.
Deepseek's 0.86 or whatever is likely subsidized but alternate providers offer it for a price comparable to glm-5.2.
According to deepseek themselves, their current rates are NOT subsidised.
They have published tons of articles dedicated to performance and efficiency engineering. Feel free to have a look...
Why is no other inference provider offering similar prices then?
How long did it take vLLM to implement deepseeks sparse attention from the r1 paper?
Does ananyone outside deepseek have a working code for the v4 compressed attention mechanism?
Has any other provider managed to bypass CUDA and program the compute engines in their native assembly language to get 10% more performance out of them?
There is your answer.
GPU/RAM/etc prices could continue to rise. If the world leaders decide it's time to build the robot armies, then that could price out the civilian uses for GPUs.
How fast is glm 5.2 in western hosts? It's doing everything I want it to, but going through PRC host it takes like 5-10 times longer. Not sure if that is nature of modest or PRC computer infra/routing.
GLM feels faster and more reliable in my experience. Anthropic and OpenAI models would hem and haw or straight up timeout during peak times.
I don’t understand the argument here. The article doesn’t describe a collapse or the breadcrumbs for it. The only argument I can put together is companies hosting the open source models in house or use some service like Amazon that could potentially host them and so replace the frontier models. Data center and specifically infra to host llms is still the main sticking point given the security concerns about data going to china. The article doesn’t make these arguments coherently
the economics of this are a little counterintuitive.
is there a market saturation point for intelligence? how about for software? it seems like the more you have the more you want because you're trying to do more things.
as the models get smarter I get busier because I'm doing more things...
There's definitely a saturation point depending on the complexity of the problem you're solving. For example, any model can write a small shell script to resize a video with ffmpeg for you right now, so it doesn't matter whether you're using a local Qwen model, GLM, or Fable. They'll all do a roughly comparable job and you'll end up with a working script that does what you need.
Then you have things like CRUD apps, where a model needs to write some SQL, make a service endpoint, serialize some JSON, etc. Here a local model might have a bit more trouble juggling all the pieces, but any hosted model will do just fine. If your day to day job involves working on CRUD apps, then it's basically a solved problem now.
The cases where frontier models matter are when you're solving genuinely complex problems, but that's not what most people are doing day to day. So, paying an order of magnitude for a model that has capabilities to solve problems outside the range of problems you actually work on becomes a waste of money.
There's going to be a market for these models from people who really do work on complex things on regular basis, but the question is how big that market is. Additionally, open models keep getting better, and GLM 6 or DeepSeek v5 could end up being another big jump in capability where they fully close the gap with Fable. At that point, even more of the market becomes covered by these models leaving truly complex cases on the frontier.
Another thing to consider is that most big problems can be broken down into smaller ones. That's the basis for how programming languages are structured. We have primitives which are arranged into functions, that get bundled into classes or namespaces, and so on. So, you don't need an infinitely capable model to solve big problems. You just need to be able to break large problems into smaller ones, and a model that's smart enough to decompose a problem to the point where it becomes tractable.
I don't think the writer has used top tier models very much. I have subscriptions to basically every provider, the difference between glm5.2 and opus is not even close, the gap is huge. raw benchmarks glm is impressive , but in practice these models are lacking so much. I had fable create a detailed implementation guide that explained how to implement everything in immense detail, it included all the libraries to use and versions. I then had deepseek v4 pro execute and it used old versions , different libraries and cut corners. Fable said about 80% was implemented wrong.
I had GLM 5.2 do the same, and it performed exceptionally better, but when it got stuck on something it would be trial and error mode going forward and have zero foresight for future issues that might occur due to fixes it was trying. the model severally lacks prompt understanding, and testing .
Opus is good but not consistently good. That’s a problem. I’m paying the same but not getting the same results.
As long as the SOTA models are 'ahead' then there will be a big premium.
“Z.ai provides a replacement MCP for web search, but it's pretty awful and slow”
I’ve had good results with Tavily so far, might be worth checking as an alternative for agent search.
But what if they introduce a tarriff per token
I hope cheaper inference eventually means faster speeds at the lower tiers. I don't want to settle for 100 t/s, but I don't want to pay $10 per prompt either
Which raises the question, which are the fastest frontier models? are the enterprise hosted Anthropic models faster than what Anthropic serves?
Somehow no one talks about LLM speed.
OAI has announced an upcoming 750tok/s 5.6 served through their cerebras acquisition
> cerebras acquisition
Partnership you mean?, Cerebras went public and are trading at around 45B in market cap.
While OAI could in theory cough up that kind of money, it would massively hamper their existing committed capital outlays.
That is going to be absolutely wild for whoever can access/afford it.
Yeah, Cerebras is the one with competitive speeds nowadays but they cost an absolute fortune. Also they don't host good models publicly. Good to see OpenAI leaning into them, can't wait until these speeds are available by subscription
> Somehow no one talks about LLM speed.
When I've raised speeds about local inference I've been told 60-75 t/s is perfectly usable. It makes sense that people aren't talking about speed yet since you either already have a response fast enough to wait for, or you go do something else and check back in a few minutes.
I would love to wait for the latter type of tasks though, because those are typically the ones that require the most work from me to verify and I don't want my attention divided with multitasking.
GLM 5.2 has a Fast variant at 200-400 tps.
GLM is the model that will sink the frontier labs.
Recall last year deepseek? And 18 month's later? What changed?
A year ago I wasn't using Deepseek. Now I am. I guess what changed is which models people are using most for coding.
> GLM is the model that will sink the frontier labs.
this is the claim you are making here, no one else claimed that.
two obvious issues here -
1. GLM itself is a frontier lab, ranked No.3 in the world in July 2026, ahead of Google, Meta and xai. GLM is not going to sink itself.
2. GLM won't sink OpenAI, it will significantly restrain OpenAI's profit margin. OpenAI will still be able to get stupidly high market cap, but not trillions, hundreds of billions will be far more likely.
This article only promises to get into "the coming AI margin collapse" in a yet to be published "part two". This part only makes the point that GLM 5.2 is pretty good (no shit).
It truly is a pointless article.
Is this AI written (or edited)? The word "genuinely" appears 4 times on the page.
Man, I hate how often people/LLMs use that word now. Maybe other people gloss over it but it's super distracting to me.
Disclosure - Fireworks kindly gave me some free credit to experiment with GLM to help write this article.
needs to be higher up
We'll keep saying the same things about every new free model just like we say the same things about every new frontier model but nothing really changes.
> I expect for most professional use the very lax terms around training and data retention will make this a difficult sell
I couldn't care less whether a chinese or american company reads my crap code.
I'm not working on state secrets but warehousing software for specific clients on a machine that has access to nothing but crap enterprise code.
This take may be frowned upon in HN, but I suspect it's much closer to the median sentiment.
how do massively negative margins "collapse"
If you use a good harness or add the right tools and plugins both the image and web search issues mentioned are non-issues.
oh-my-pi (omp.sh) handles images for text models out of the box - as long as you have any vision capable provider enabled, it will be used when you paste images to a text model. Rather than let it guess I configured it to use MiniMax M3 for this task (as well as other utility tasks like code exploration & library functions).
opencode has plugins that do the same thing, but I haven't used it since picking up omp and haven't tried them.
In open harnesses you can also configure your search provider(s) separately from the model provider - if you've got a ChatGPT sub you can use just their websearch for example. I've been using Kagi's API and found its cheap enough not to matter to me at all.
As for slowness, I'm not sure I'm really seeing that in terms of wall clock time. The author says GLM uses more tokens for reasoning but doesn't explain how they know that - frontier models don't provide nearly the entire reasoning trace. I have the suspicion that the author is not aware of that fact. I use Opus with Claude Code for work and I find it subjectively slower because I can't read its CoT trace. That is another HUGE benefit of GLM: I can't tell you how many times I've seen it start to go sideways in its CoT - usually due to something I didn't tell it - and I just stop it and give a course correction rather than wait a whole turn.
Overall I agree with the takes from the article and frankly its sad how much cope I see on Twitter (and even here) from people that think AI coding is busted once subscription subsidies are dropped. GLM is already good enough and cheap enough to use it at API rates - but it is MUCH more expensive than other open models that are also very nearly good enough.
In twelve months I'm confident you'll be able to get equivalent results at API rates for less than $1 per million output tokens, and more likely that will happen in six months. Deepseek v4 Pro is already almost there (and at only $0.85/MM) - and at least on benchmarks its already better than GLM 5.1 which I was happily using quite a lot before 5.2 dropped. I haven't tried Deepseek since I already have a z.ai pro sub that I locked in for $30 - at $72 its a lot less compelling.
Inference has been decreasing in cost by about 10x per year since 2023.
‘640kb of ram should be enough for anyone’
I would not be unsurprised if the US govt steps in to prevent this. They'll do anything to stop China getting ahead in the AI race.
There's the sanctions already implemented, next step might be giving these companies government funding, just like they do with military companies.
Don't Anthropic and OpenAI both already have military contracts? They are already growing into that fabric I think.
Good luck trying to enforce that outside of the US.
I posted this some time ago https://news.ycombinator.com/item?id=48759668
Singapore seized a mansion due to Nvidia chip smuggling. So there are some countries that will enforce sanctions.
> Where it gets really scary for the frontier labs is how easy it is to migrate to open weights models. Both Z.ai and Fireworks offer both an OpenAI compatible and Anthropic compatible endpoint. This makes it absolutely trivial to use with Claude Code and Codex.
Yes the ease of switching is greatly appreciated.
Now the reason I tolerate Claude Code in my tmux sessions is because apparently Anthropic ain't playing nice with the subscription plans and other harnesses.
But I'm evaluating pi.dev atm and it looks amazing. To me being able to rid of that piece of vibe-coded underperforming, characters-modifying, turd that Claude Code is a big motivation to switch to GLM (I'll probably keep my OpenAI subscription as OpenAI repeatedly said they were cool with other harnesses).
It's also quite obvious that Claude Code is receiving new vibe-coded slop features after vibe-coded slop features in an attempt to lock you in.
To anyone thinking about switching to GLM: I'd say at least evaluate pi.dev and see if that wouldn't be an opportunity to kiss Claude Code and its "gameloop that converts characters from a headless browser to other characters to show in a terminal at 60 fps" goodbye once and for all.
Pi.dev is great and with only a little customization made even previous gen open weights feel superior.
It also doesn't feel like they're trying to sell me on transhumanism all the time.
It also doesn't get mysteriously downgraded. It's just consistent, even before 5.2.
5.2 is great in a lot of ways - but it's best quality is that it gives some pushback and isn't nearly as synchophantic
Yes, margin on model inference is high with some providers. If you just wanted inference (at cost), you'd buy a GPU, or rent one from AWS or Microsoft. But you're not paying OpenAI/Anthropic for inference. You're paying them for a platform. Every feature OpenAI/Anthropic bake into their applications, models, online services, etc - anything that isn't pure LLM text generation - is a custom integrated add-on service that LLM weights do not include. Even if open weights became cheaper and better than OpenAI/Anthropic, most people would still pay for OpenAI/Anthropic, because they give you things the weights alone don't give you.
Comparing Z.ai GLM 5.2 to Claude Code w/Opus 4.8 is like comparing Linux Kernel 7.0 to Microsoft Windows 11. If you don't know much about computers, you'd say these are the same things. If you know a lot about computers, you know the latter has a thousand extra things that make a huge difference in what it does out of the box. Which one you use speaks to what kind of customer you are.
Sure, GLM 5.2 doesn't have vision; but an AI power user can plumb together any VLM with the text generation of GLM 5.2 in most AI harnesses, just like a Linux power user can combine the Linux kernel with KDE Desktop. Most people don't use Linux and KDE, because it's unpopular, difficult to use, hard to get support for. Instead they pay for Windows or Mac, because there's lots of support, with a giant company pouring money and effort into filling all the usability gaps, making it seamless.
Most people don't pay for the cheapest possible thing. They pay for the thing they can afford that improves their life while making it easier. An open weight alone is almost completely unusable by itself (like the Linux kernel), compared to an AI platform (a completely usable system). If you're constantly wondering about when open weights will reach parity with OpenAI/Anthropic, you're a Linux person. If you just pay $20/$50/$100 for OpenAI/Anthropic without thinking about it, you're a Windows/Mac person. There is nothing wrong with either of these groups, but they are fundamentally different, and always will be. An LLM weight is simply a different category of thing than an entire AI platform/provider.
I think the fixation on numbers of tokens and dollars per token is missing the point a bit. LLMs are quite useless without good tools. The article calls out search as one of them. And it's important. If you are coding, the tools are relatively easy: they are mostly open source and don't have a lot of authorization logic around them. Anyone with access to python and some access to a half decent AI model can pull together a decent agentic coding tool. There are many examples out there.
But if you look at the overall market, there's a rapid shift happening to non-coding tools and non programmer users starting to become very active. This kicked off beginning of the year with Claude Cowork. OpenAIs Codex and ChatGPT (they both have the same plugin infrastructure) is doing a lot of the same things. I've talked to a lot of non technical business users in recent months. There's a growing amount of people who definitely have zero interest in programming starting to use these tools and getting value out of them. This is going to rapidly scale to essentially most white collar users. Programming tools are becoming a side show to this market.
The difference here is that these people need connections to all their favorite protected data SAAS silos: MS office, Sales Force, Outlook, Gmail & GSuite, Calendar, SAP, Oracle, etc. The moat here is very different: it's mediated access to these silos in a compliant way. Anthropic announced a solution in the form of some MCP features. Those features boil down to getting access to all your favorite silos, if you sign in with the right identity provider. What's the right identity provider? The one that's whitelisted by the data silos you are locked into. Okta seems to have weaseled themselves into a position of power here. And it's all the other usual suspects. We'll see who is going to "win" that race but I bet it's going to be a pretty exclusive club with zero outsiders from China on that list. You can hack your way around some of those limitations. But doing so in a compliant way is going to be tricky.
And that's before you consider who's going to pay for this and what they are going to insist on. Corporate IT departments & data security policy compliance basically. What's the moat here? Secure & compliant access to all your favorite silos. Here in the EU that also includes data residency. The difference between sending all your data to Silicon Valley or Beijing is that of getting stabbed or getting shot. If it leaves the EU, you have a huge compliance issue. Most of the juicy corporate LLM usage is going to have to be fully compliant. I.e. hosted and controlled in the EU. This will be the same across the world. The least important choice right now is which model you use. The most important ones are about where those models run and what tools the models running there have access to and how that is governed.
On paper, OpenAI, Anthropic, MS, and Google are pretty well positioned here. Not necessarily in that order. Most others are still figuring it out. But they'll have a moat of data center ownership in the right regions + mediated tool access that works out of the box.
I think OpenAI, Anthropic and SpaceX are going to envy the dinosaurs because there's not asteroid coming for them, there's three:
1. There will be no moat around frontier AI models in the future. China is going to make sure that happens. It's a national security interest for them. DeepSeek was the first shot across the bow for that but it won't end with them. There are other labs and there are non-Chinese actors too. The stratospheric valuations depend on there being that moat; and
2. Nobody seems to be considering what the next generation of AI hardware is going to do with current hyperscalar investments. We're about to go through this with the B100/200 move to R100/200 but a lot of the investments are probably slated for that next-gen. But what about 3 years from now when the hypothetical X100/200 comes out and doubles FLOPS and halves performance-per-watt. What will that do to existing investments? Some people are delusional and think that they'll get 10 years out of GPUs when 10 year old GPUs (eg V100) are sold for scrap and 5 year old GPUs (A100) cannot run DeepSeek v4 Pro. And people think the A100 is going to get another 5 years of use? No; and
3. Local LLMs are coming for remote usage. You can buy a 5090 PC for less than $5000 currently but you're limited to 32GB of VRAM, which will comfortably run 31B models but nothing really larger. Go to $12-13k to upgrade to an RTX 8000 Pro and you have 96GB of VRAM, which will run larger models (but certainly not, say, DS v4 Pro or even Flash). You have shared video memory products rapidly coming from NVidia's aggressive market segmentation. Things like Strix Halo and DGX Spark have severe limits on memory bandwidth (<300GB/s compared to 1.8TB/s for a 5090/6000 Pro and 3TB/s+ for server grade HBM3e/4 based GPUs). Macs could be real interesting in this space butr they lack the raw FLOPS with the M5 generation.
But what will this local hardware look like in 2-3 years? I think people will be shocked at how much better it will be with the Apple M7 Pro/Max generation (2028 expected) and the RTX 6000 cards at that time although I fully expect NVidia consumer GPUs to still top out at 32GB of VRAM to maintain that segmentation. And I look forward to what the next generation of the AMD Ryzen AI Halo platform will look like if they really try.
All of this adds up to these three companies needing to cash out before the music stops (IMHO).
Interesting, so point 2 means that a lot of the hardware being installed now won't be able to run the frontier models of 2029? How does that change the demand for compute/models in the future, I can imagine that even if OpenAI/Anthropic have a moat in 2029 there will be so much older hardware and such a hangover from that investment boom that there will be very little installed capacity that can run it
On 1 and 3, the obvious move is to shift the bulk of the harness behind a new API that's not based on raw LLM access. Then they get to hide secret sauce behind that API and all three go from commodity to premium while simultaneously being able to try out whatever tricks they can get away with to reduce their own inference costs. I'm almost surprised this hasn't happened already.
i would use glm 5.2 if the servers weren't in china
i mean i guess my employers wouldn't know the difference
but i'd like to play it safe and keep everything in america
If you look at https://openrouter.ai/z-ai/glm-5.2#providers there's about 28 providers, including z.ai and Alibaba. Most outside of China. I've never seen so many providers for a model on there before, glm 5.2 is popular.
thanks I see cloudflare has it. I will give glm 5.2 a try
its open-weight. I think you can find a host for GLM-5.2 in the USA
The fact that these Chinese models are getting close to “Opus-grade” despite costing 6x-8x less is huge.
As the token bills start to come in, those economics will be harder to ignore (regardless of the origin of the LLM); especially as there will be many CIOs sweating over their quick and costly AI initiatives showing little ROI.
My hope is that the EU also steps up their own competition in the frontier model space so that it’s not just China v USA.
they're not near opus at all, anyone using the models in a real working environment will tell you the same thing. on paper they have impressive benchmarks, but that's not realistic to actual use.
I think it depends on your use case. For my personal projects (a mix of webdev & some Rust desktop apps) it's honestly very close to Opus 4.8 (which I use in my day job).
I don't feel like I'm missing out after cancelling my personal Claude subscription, whereas I used to feel that way a few months ago.
I've been using GLM 5.2 a lot this past week, it's been replacing Opus 4.8. I mostly do front-end web development and haven't noticed much of a quality difference.
Sure, "it's just frontend", but that's actual use enough for me to take it seriously.