I think if you use common sense reasoning you can see why the failure mode of AI financing predicted by AI skeptics is unlikely - that is, companies will be swimming in debt and are unable to pay for hardware.
The thing is, these AI GPUs are sold at insane markups - a 5090 was like $1700 MSRP, and a datacenter GPU can't cost like more than triple to make - its just that there's so much demand, that people are paying these markups; thast means these companies are flush with cash, and its in their best interest that their customers don't become insolvent, because then its back to selling GPUs to us rubes, who get sticker shock at 'even' a $1000 price tag.
Therefore they both have the ability and willingness to maintain this forever.
Demand also seems to be there in droves, so the only scenario I can imagine is if datacenter or power infra buildout becomes impossible - plausible, but not what people are worrying about right now.
The demand for shovels in a gold rush is high, but, once we realize the amount of gold available isn't that much, the demand for shovels at any price falls off a cliff?
In this analogy, gold would be compute. So the more accurate analogy, would be that there's plenty of gold underground, and there's plenty demand for it (as of now), it's that extracting it is so expensive that we might go broke before we can get the gold out.
And the point I'm making, is that shovels are sold at min. 5x markup, and the shovel company (Nvidia and others) has no interest in seeing the market collapse, and is willing to use its generous profits to prop it up.
No, in this analogy, gold is the theoretical work that AI can perform. Once we realize it's not that huge a percentage of all work, demand for AI compute dries up.
Compute is the shovel.
You use the shovel (GPUs) to dig for the gold (work AI can do). However, the AI can't do as much as was hoped, so the demand for GPUs dries up.
Nvidia is willing to prop it up, I have no doubt about that, but that doesn’t matter if the market loses faith in the AI narrative. Core weave is a public company, investors can abandon the stock and have already started to do so. Anthropic and Open AI will probably also soon be public. If the market loses faith. It doesn’t matter. And Nvidia may eventually have to decide how much money they are willing to lose on Coreweave.
>...its just that there's so much demand, that people are paying these markups.
The catch is that the vast majority of that demand is from data center projects, not actual AI users. And data center projects are funded by private credit that is speculating about future demand (most of these centers will take years to build out). So a corporate user who would put the hardware into service now has to compete with a data center project that is hoarding GPUs for the future. If those GPUs end up being used AND the operative margins on AI SaaS are high enough, then the speculators may win. If either of those conditions are not met (and both look pretty shaky), then the data center bubble will burst and the GPUs will flood the market at pennies to the dollar.
>I do think there is a real but moderate Jevons effect here: because inference is so cheap now, I’m much more likely to implement marginal automations than I would have been before. But it isn’t unbounded. In practice, it’s maybe 2x or 3x as much code as I would have wanted at the old price, because I still have to specify what to build, review the output, and maintain it. When the price per task drops 20x, and my usage only goes up 2x to 3x, total dollar revenue into the ecosystem (and net GPU demand per user) still collapses by 70% to 85%.
I don't think this checks out. Jevon's paradox doesn't say when electricity becomes 20x cheaper, someone already using electricity will use 20x more on the same thing they already do, or even 20x more at all. It means a combination of more people and more enterprises will use it, and use more of it.
Concretely - at Claude API prices I'm not going to have AI run a bunch of things that I can do without it. But at DeepSeek API prices, I might.
Fundamentally, I think that we aren't even imagining a number of the use cases for AI, in large part because they don't exist yet. But I do think they will eventually exist in the next 20+ years. Right now we are strapping AI to everything, not reimagining life with coherent artificial intelligence. Similarly I couldn't have predicted something like Walmart+ or Amazon Prime back in 1995.
I think your comment is well reasoned, but Jevon's doesn't always happen. As an example, the US installed a lot of fiber when the internet took off, but most people were happy with the bandwidth once they could stream high def video, and now there's a lot of dark fiber.
I have no idea which way it will go. Maybe most people will be satiated once they can generate video on demand. They don't need many tokens to read and write their email, book a flight, or do their homework.
I don't know the whole story, and I'd be interested in the details if you're willing to share.
However, I'm a geek who sits online all day, and I don't pay for the top tier of bandwidth available to me. I also see occasional comments where people are saying they use Sonnet or Luna because they're good enough. Maybe in 20 years people will point to the incompetence and greed in data center build outs too.
They keep talking about gpu depreciation and I pay a fortune to rent A100. We have a policy at our company to never close them because demand is so high that we can't find resources to spin them back up when we need them. So msft doesn't even spend money cooling them.
I think if you use common sense reasoning you can see why the failure mode of AI financing predicted by AI skeptics is unlikely - that is, companies will be swimming in debt and are unable to pay for hardware.
The thing is, these AI GPUs are sold at insane markups - a 5090 was like $1700 MSRP, and a datacenter GPU can't cost like more than triple to make - its just that there's so much demand, that people are paying these markups; thast means these companies are flush with cash, and its in their best interest that their customers don't become insolvent, because then its back to selling GPUs to us rubes, who get sticker shock at 'even' a $1000 price tag.
Therefore they both have the ability and willingness to maintain this forever.
Demand also seems to be there in droves, so the only scenario I can imagine is if datacenter or power infra buildout becomes impossible - plausible, but not what people are worrying about right now.
The demand for shovels in a gold rush is high, but, once we realize the amount of gold available isn't that much, the demand for shovels at any price falls off a cliff?
In this analogy, gold would be compute. So the more accurate analogy, would be that there's plenty of gold underground, and there's plenty demand for it (as of now), it's that extracting it is so expensive that we might go broke before we can get the gold out.
And the point I'm making, is that shovels are sold at min. 5x markup, and the shovel company (Nvidia and others) has no interest in seeing the market collapse, and is willing to use its generous profits to prop it up.
No, in this analogy, gold is the theoretical work that AI can perform. Once we realize it's not that huge a percentage of all work, demand for AI compute dries up.
Compute is the shovel.
You use the shovel (GPUs) to dig for the gold (work AI can do). However, the AI can't do as much as was hoped, so the demand for GPUs dries up.
Nvidia is willing to prop it up, I have no doubt about that, but that doesn’t matter if the market loses faith in the AI narrative. Core weave is a public company, investors can abandon the stock and have already started to do so. Anthropic and Open AI will probably also soon be public. If the market loses faith. It doesn’t matter. And Nvidia may eventually have to decide how much money they are willing to lose on Coreweave.
[flagged]
>...its just that there's so much demand, that people are paying these markups.
The catch is that the vast majority of that demand is from data center projects, not actual AI users. And data center projects are funded by private credit that is speculating about future demand (most of these centers will take years to build out). So a corporate user who would put the hardware into service now has to compete with a data center project that is hoarding GPUs for the future. If those GPUs end up being used AND the operative margins on AI SaaS are high enough, then the speculators may win. If either of those conditions are not met (and both look pretty shaky), then the data center bubble will burst and the GPUs will flood the market at pennies to the dollar.
[flagged]
Good article.
>I do think there is a real but moderate Jevons effect here: because inference is so cheap now, I’m much more likely to implement marginal automations than I would have been before. But it isn’t unbounded. In practice, it’s maybe 2x or 3x as much code as I would have wanted at the old price, because I still have to specify what to build, review the output, and maintain it. When the price per task drops 20x, and my usage only goes up 2x to 3x, total dollar revenue into the ecosystem (and net GPU demand per user) still collapses by 70% to 85%.
I don't think this checks out. Jevon's paradox doesn't say when electricity becomes 20x cheaper, someone already using electricity will use 20x more on the same thing they already do, or even 20x more at all. It means a combination of more people and more enterprises will use it, and use more of it.
Concretely - at Claude API prices I'm not going to have AI run a bunch of things that I can do without it. But at DeepSeek API prices, I might.
Fundamentally, I think that we aren't even imagining a number of the use cases for AI, in large part because they don't exist yet. But I do think they will eventually exist in the next 20+ years. Right now we are strapping AI to everything, not reimagining life with coherent artificial intelligence. Similarly I couldn't have predicted something like Walmart+ or Amazon Prime back in 1995.
I think your comment is well reasoned, but Jevon's doesn't always happen. As an example, the US installed a lot of fiber when the internet took off, but most people were happy with the bandwidth once they could stream high def video, and now there's a lot of dark fiber.
I have no idea which way it will go. Maybe most people will be satiated once they can generate video on demand. They don't need many tokens to read and write their email, book a flight, or do their homework.
Dark fiber has many more layers of incompetence and greed than has to do with people we’re happy with speeds.
I don't know the whole story, and I'd be interested in the details if you're willing to share.
However, I'm a geek who sits online all day, and I don't pay for the top tier of bandwidth available to me. I also see occasional comments where people are saying they use Sonnet or Luna because they're good enough. Maybe in 20 years people will point to the incompetence and greed in data center build outs too.
> For the tech industry, writing code is the main bottleneck of the business.
This couldn’t be further from my experience; producing code has never been the bottleneck.
A good read and a more reasoned version of the arguments Ed Zitron has been making.
They keep talking about gpu depreciation and I pay a fortune to rent A100. We have a policy at our company to never close them because demand is so high that we can't find resources to spin them back up when we need them. So msft doesn't even spend money cooling them.
[flagged]