I think an interesting point is that hardware as of today still has no utility value after its reported lifetime has elapsed, which prevents neolabs and smaller labs from getting older HW clusters as the banks are not willing to give out loans against them. There is no agreed upon pricing for "expired" A100 clusters or similar.
This is clearly not true, and we are starting to see compute markets, but only for rental prices/H, not for the hardware itself. I feel like there is some artificial moat being built here to stimulate sales of new hardware, because an H100 at 1/16th the price will have comparable dollar/FLOP as Vera Rubin.
> HW clusters as the banks are not willing to give out loans against them
Which banks have analysts that understand the difference between H100 and A100? Do you have actual experience with being denied a loan based on this or are you just making things up?
Depends on the workload. H100 will never have the network performance of Vera Rubin. There's also token per watt, newer systems will beat the older systems.
It's not clear how much of the latest chips have even made it on-line yet.
The claims of many GW of installed training/inference have come under scrutiny lately. The first VeraRubins aren't even there yet, so it's all GB300 NVL72s as the peak performers and probably <<1GW of those so far. Even xAI Colossus is mostly H200s and B200s.
Electricity costs are also a huge differentiator. When drawing 100kW the difference between >50cents and <10cents per kWh is pretty big! One is almost $0.5M and the other is less than $100k.
The problem, IMO, with open-weight models is that you accustom to the capabilities of frontier models too quickly; and downgrading to an open-weight "frontier minus 2" or "frontier minus 3" model is often painful, since they feel way less useful than their newer closed-weights counterpart. To be honest, I don't know any companies using OW models at a large scale for their operations (agents or chat assistants).
I think this is where Deepseek has nailed the mark; DS4.1 Flash is really, really fast, and really, really cheap. If you give it small, structured goals, it completes them crazy quick, at negligible cost. There's different vectors to differentiate along to stay in the conversation.
I've taken to using them as micro-review subagents at development milestones, where a "frontier - 1" model like Opus or Sol launches 10-15 of them on small review tasks that each run for ~10 minutes. Costs about $1 per cycle, and they usually catch something Astra or Fable didn't. Then the orchestrator validates each claim before passing it back to the planning session so we can fold the findings in.
I don't know about that. The frontier models have all sorts of strange regressions and quirks. They're more "intelligent" but, like hyper-intelligent humans who speak up without any skin in the game, I've found that they have a strong pull to add complexity to anything they touch. Truth is, I don't need frontier-level intelligence to write a CRUD app or debug my config file. The vast majority of software work is basic pattern matching.
Perhaps because they are dumber, they produce better results? IMO an excellent well-tuned harness combined with a "frontier minus x" model produces the highest quality result. DS4.1 and Qwen3.8, far from being a compromise, legit give me better results. For my personal definition of "better".
Closed weight models are not usefully ahead of open weight models for coding anymore. "Fable, write a thousand file codebase from this one sentence prompt" is neat and I bet it required a lot of rollout compute to train, but my actual requirements are more detailed.
I haven’t used open weight models yet, but this would match my intuition given I haven’t experienced a noticeable increase in code quality this calendar year. If open weight models are already on par with January frontier models that’s already enough for me to automate most of the manual parts of my work (I.e. not “architecting”).
And once frontier models can architect (turn business requirements into engineered systems) then I guess no one needs a job because that’s the digital singularity.
> To be honest, I don't know any companies using OW models at a large scale for their operations (agents or chat assistants).
Perhaps they simply don’t advertise it and investors (currently) love companies that spend heavily on frontier models. That said, OW models, especially when combined with RAG, work quite well, and given the current state of the industry, they may be the only sensible way to keep inference costs under control.
Why do you think OAI etc are all up in arms? They hate what’s going on. Most here are delusional.
I work in a very large market cap firm and i’m telling you - more and more managers are pushed to get their teams to use open source and squeeze employees to get the max out of them.
This makes the case that open weight models are not necessarily cheaper than closed weight models
But I'd still rather use them since it's inevitable the unsustainable margins of the closed weight providers lead to enshittification once they stop subsidizing demand
I'd rather spend more today with a workflow whose underlying unit economics are sustainable and don't force me to inevitably look for other solutions once the closed weight providers start optimizing for profit and extracting value
Exactly. If open weight models are not cheaper than the big labs then the economic model of the big labs is in trouble. To survive the big labs need to be charging much much more but the market is dragging everything in the wrong direction for them.
It also further exposes the “we’re profitable this quarter” claims coming from big labs. These statements increasingly look like WeWork “community adjusted EBITDA” nonsense of “we’re profitable if you ignore all the reasons why we’re deeply unprofitable.”
Dark grey text on a black background: it's like they are trying not to let anyone read their article!
In my case they succeeded!
I think an interesting point is that hardware as of today still has no utility value after its reported lifetime has elapsed, which prevents neolabs and smaller labs from getting older HW clusters as the banks are not willing to give out loans against them. There is no agreed upon pricing for "expired" A100 clusters or similar.
This is clearly not true, and we are starting to see compute markets, but only for rental prices/H, not for the hardware itself. I feel like there is some artificial moat being built here to stimulate sales of new hardware, because an H100 at 1/16th the price will have comparable dollar/FLOP as Vera Rubin.
> HW clusters as the banks are not willing to give out loans against them
Which banks have analysts that understand the difference between H100 and A100? Do you have actual experience with being denied a loan based on this or are you just making things up?
https://www.stoaexchange.com/ is an exchange for hardware itself and has market data for prices over time.
It looks like they have a persuasive landing page and a place to insert my email.
Depends on the workload. H100 will never have the network performance of Vera Rubin. There's also token per watt, newer systems will beat the older systems.
It's not clear how much of the latest chips have even made it on-line yet.
The claims of many GW of installed training/inference have come under scrutiny lately. The first VeraRubins aren't even there yet, so it's all GB300 NVL72s as the peak performers and probably <<1GW of those so far. Even xAI Colossus is mostly H200s and B200s.
Electricity costs are also a huge differentiator. When drawing 100kW the difference between >50cents and <10cents per kWh is pretty big! One is almost $0.5M and the other is less than $100k.
The problem, IMO, with open-weight models is that you accustom to the capabilities of frontier models too quickly; and downgrading to an open-weight "frontier minus 2" or "frontier minus 3" model is often painful, since they feel way less useful than their newer closed-weights counterpart. To be honest, I don't know any companies using OW models at a large scale for their operations (agents or chat assistants).
I think this is where Deepseek has nailed the mark; DS4.1 Flash is really, really fast, and really, really cheap. If you give it small, structured goals, it completes them crazy quick, at negligible cost. There's different vectors to differentiate along to stay in the conversation.
I've taken to using them as micro-review subagents at development milestones, where a "frontier - 1" model like Opus or Sol launches 10-15 of them on small review tasks that each run for ~10 minutes. Costs about $1 per cycle, and they usually catch something Astra or Fable didn't. Then the orchestrator validates each claim before passing it back to the planning session so we can fold the findings in.
I don't know about that. The frontier models have all sorts of strange regressions and quirks. They're more "intelligent" but, like hyper-intelligent humans who speak up without any skin in the game, I've found that they have a strong pull to add complexity to anything they touch. Truth is, I don't need frontier-level intelligence to write a CRUD app or debug my config file. The vast majority of software work is basic pattern matching.
Perhaps because they are dumber, they produce better results? IMO an excellent well-tuned harness combined with a "frontier minus x" model produces the highest quality result. DS4.1 and Qwen3.8, far from being a compromise, legit give me better results. For my personal definition of "better".
I’ve found the opposite. Going from Opus 5 to Qwen3.8 flash has been a breath of fresh air.
It was a breath of fresh air for me because it was nice going to a model that is "smart enough".
Closed weight models are not usefully ahead of open weight models for coding anymore. "Fable, write a thousand file codebase from this one sentence prompt" is neat and I bet it required a lot of rollout compute to train, but my actual requirements are more detailed.
That is true if you're using them as chatbots but for something behind a product it's largely OK as long as it meets the requirements.
I haven’t used open weight models yet, but this would match my intuition given I haven’t experienced a noticeable increase in code quality this calendar year. If open weight models are already on par with January frontier models that’s already enough for me to automate most of the manual parts of my work (I.e. not “architecting”).
And once frontier models can architect (turn business requirements into engineered systems) then I guess no one needs a job because that’s the digital singularity.
> To be honest, I don't know any companies using OW models at a large scale for their operations (agents or chat assistants).
Perhaps they simply don’t advertise it and investors (currently) love companies that spend heavily on frontier models. That said, OW models, especially when combined with RAG, work quite well, and given the current state of the industry, they may be the only sensible way to keep inference costs under control.
They’re using it behind the scenes - routing.
Why do you think OAI etc are all up in arms? They hate what’s going on. Most here are delusional.
I work in a very large market cap firm and i’m telling you - more and more managers are pushed to get their teams to use open source and squeeze employees to get the max out of them.
Are they self hosting, running them through a router like openrouter, or buying from the source?
For chat I can see, less so for LLM use in automation for example, there it makes much less of a difference I can imagine.
This is nonsense and regardless your employer doesn’t care.
The style and font choice here is awful. Nearly impossible to read in the daytime.
This makes the case that open weight models are not necessarily cheaper than closed weight models
But I'd still rather use them since it's inevitable the unsustainable margins of the closed weight providers lead to enshittification once they stop subsidizing demand
I'd rather spend more today with a workflow whose underlying unit economics are sustainable and don't force me to inevitably look for other solutions once the closed weight providers start optimizing for profit and extracting value
Exactly. If open weight models are not cheaper than the big labs then the economic model of the big labs is in trouble. To survive the big labs need to be charging much much more but the market is dragging everything in the wrong direction for them.
It also further exposes the “we’re profitable this quarter” claims coming from big labs. These statements increasingly look like WeWork “community adjusted EBITDA” nonsense of “we’re profitable if you ignore all the reasons why we’re deeply unprofitable.”
Then why do people still believe that OpenAI any Anthropic have negative margins
Here he is. Shows up on time.
[flagged]