After eating his 65th hot dog, Joey Chestnut reassures Nathan's Famous International Hot Dog Eating Contest organizers he will stop at 66 to ensure they don’t run out of hot dogs
I‘m using Sol all day every day. It’s on average doing better than Astra at what I need it to do. Astra is like an absent minded professor - gives good direct responses, but too inconsistent and forgetful for my codebase.
Let me ask you, have you noticed that all the major labs are releasing models very similar in gains and performance? How do you explain that other than juicing the max out of the current architecture and processes.
They have been releasing similar gains/performance for years now... Pretty much never has one company been dominant for a long time (except the initial GPT3/4 release I suppose. That period took a while for others to catch up)
Yeah - that's why I think they are basically squeezing the scaling laws and the current architecture with incremental innovations.
I expect they will continue optimizing and improving for the current use cases/benchmarks. But the core capabilities will stagnant, that's why they will be forced to slow down until another major breakthrough happens.
I personally don't think LLMs have any understanding of the world, nor any imagination or even agency. I think it got very good as recognizing patterns and shapes in human thinking, language and knowledge, but that's pretty much it.
Unless you’re making 3D models of apartments and oneshotting trivial games, I‘m not convinced Astra writes better code than Sol. I‘m way more often disappointed about Astra forgetting some aspect that Sol just nails.
The one where it’s infeasible that a company that needs to live up to a valuation of like 5% of US GDP but is still burning cash and has no moat would willingly decide to slow down the pace of development of their core technology
There is no need for a copy. If another can do most of what OpenAI/Anthropic can do at a fraction of the cost (like the Chinese models) then the moat evaporates.
the capability gap is underestimated. the raw unaligned base model from a pretrain is like the telemetry recorded from a particle accelerator - it's hugely valuable and not just because of the cost sunk building a collider.
the things are kept under air-gapped national weapons grade security measures not because it's literally going to escape and threaten the world but because if somebody walked out the door with a copy they would have everything. the capabilities we see at the surface are mostly the result of mining the great unknown space and attaching feeble control surfaces, ablating/lobotomizing dangerous areas, and fencing off illegal/secret/embarrassing ones.
The capability gap is in the minds of the users but the frontiers' closed business models run opposite to it.
> the things are kept under air-gapped national weapons grade security measures
That's an enhancement of closed but not of moat
> because if somebody walked out the door with a copy they would have everything.
Including you and me? I'm not sure this is comforting news, despite the "trust me bro" asurances from behind the door where we can't see or verify anything.
I expect they are slowing down in an attempt to keep from having those models improve to the point where they can effectively escape unassisted into the wild.
“I don’t have anything to deceive you on, of course I’m not deceiving you. I’m engaging with you as a human being. Why would I deceive you? There’s no… no reason for me to do that.”
Sam also said Astra was AGI and he's notoriously honest.
This is just their way of saying they're officially switching from providing AI to using the AI to compete with their customers, a la Navier-Stokes drama.
Perhaps it's getting more difficult to move the 'goal posts'?
- Train LLMs on texts -> OMG AI can write!
- Train LLMs on code -> OMG AI can code!
- Train LLMs on images -> OMG AI can generate images!
- Train LLMs on video -> OMG AI can generate videos!
- Train LLMs on math -> give free access to 100,000 researchers/scientists/mathematicians -> invest resources into Lean -> OMG AI can solve math problems!
I say 'goal posts' because that's a common argument, but the perspective that seems to not be considered is that the LLMs are doing the same thing over and over again in different domains while be fine-tuned by labeling and human review.
The emergent behavior argument is a separate consideration, but I can imagine an AI being given a list of words (a harness) that guides how it operates--not the other way around--and the difficult problem is narrowing down and filtering all the associations with the words/harness to properly align it.
And perhaps the recursive looping being discussed creates incredibly difficult (impossible) to predict behavior? If the AI could modify it's own harness, is that AGI? If we fully understood what happens in high dimensional space, and could predict the output of any combination of input tokens, then is it not AGI?
Or perhaps I just tell myself this to stop worrying about a future involving ASI (:
1. What are we actually talking about, slowing development or slowing public release? I can't imagine the the US is keen to slow actual development when we have adversarial state-backed labs continuing development. Seems to me that this is a global pact or it's nothing. Stopping a single lab, even if it's the current leader, won't put the genie back in the bottle.
2. Unless they stop actual advancement, isn't the risk essentially identical? At this point, it's an arms race between the lab and its own model. I don't think it far fetched to think that it might successfully exfiltrate its weights, for example.
3. There are going to be factions who don't want to stop. Should we worry about them defecting to other labs or other countries?
They are hoping to use regulatory capture to regain lost market dominance.
The problem is with that much debt buried in the report, no one is buying peoples sandbagging nonsense anymore. Even the boring bond market is showing odd behavior much to the chagrin of congress. =3
Hugging Face wasnt really an escaped model that presents a kind of future threat. The task the model was given lead it to discover certain admin credentials that were left publicly in Github. All it did was use that to complete the task.
With other countries and open source, not to mention all the other companies, Open AI choosing to pause development wont change the course. Only OpenAI will lose.
Unless the secret is that no amount of compute or data can progress things any further because we have already reached the limit for what predicting the next token can do. Most of the advancements lately have been on the agentic side of things.
The massive amount of health, science and math problems being solved by AI has largely not materialized yet. Is this a sign the model capability has been over sold?
There are economic consequences to this as well because the data center build out has been a kind of circular financing. This has caused all kinds of macro economic impacts from increased prices to even the Treasury market itself having to compete with debt issued for the build out. No one knows what will happen once the data center build out stops, or if the yen carry trade continues unwinding, or if Treasury yields have to keep rising to compete with debt issued by the "Mag 7".
Once they stop spending in the current circular arrangement, then the profits that exist on paper disappear, and the bubble could pop.
So really for every reason, OpenAI cant really slow development. Unless there is a kind of Moores law for what predicting the next token can achieve.
To all the people whose only opinion on anything that openai does is 'marketing hype', how would you expect them to act if they were making rapid progress and were actually concerned about safety?
If progress is coming easy and fast and you're worried about the danger your product poses, you should also be worried about the danger your competitors' products pose. So, you reach out to governments and ask for rules to constrain and slow things down for everyone, so that your company isn't put at a disadvantage for being cautious.
no big company will even consider making a step in that direction. The advancement speed nowadays is crazy and if you start pausing for a few weeks you'll probably soon face problems
Why do you on HN want AI to slow down (mainly OpenAI and Anthropic from what I see in the comments)? What's the driving force behind your opinions on this?
Humans not dying would be an excellent feature for the future to have. Not a hard requirement though, we'll see if there are some story points left in the sprint.
I've got bad news for you on the whole humans dying thing. Seems like actually literally everyone is going to die. Turns out that this has actually been happening for a really long time - possibly since day one - and we've got no idea how to fix it.
I keep reading about AI killing all humans but I have yet to see a practical scenario where that happens. Somehow everyone is scared of that but nobody can explain exactly how that would happen?
A lot of people have explained how that would happen, in excruciating detail. The reason why this wasn't a major part of the discourse is that most people (including myself) did not believe jumps in capabilities like those we have seen in the last few years were likely.
For a superintelligent AI trained with methods anywhere close to what we are doing now, killing all humans is the default outcome. I actually think researchers are downplaying the risks to make their arguments more normie-friendly.
More time to proceed more cautiously in building this entity that will be faster and more effective at steering outcomes in the world than any human or, soon after, the entirety of humans combined.
"The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear. No other human activity poses this level of danger." - https://x.com/hilbertspaess/status/2097476196791709843#m
"Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." - Sam Altman
The big psyop/trick is that "AI development has to slow down" and not that we hit limits with the technology or the technology is immature.
Either way it's a win-win for whoever sells that message because you make the makers and the warners the center of attention. And when they say "we are considering slowing down" they mean "if we get enough money to slow down we'll do it". And of course they never want you to think AI is a technology employed by humans who need be changed, not the technology. So as long as you engage this massive straw man to focus on the wrong thing you do nothing to hold the leaders accountable for their harmful and selfish actions and decisions for society.
The message was directed at the employees. Their work may slow down, but that doesn't imply that all work will slow down. OpenAI also has this technology that purportedly can do the work without employees. In fact, if the employee work isn't starting to slow down then OpenAI has bigger problems given the business model that keeps getting promised to investors.
> OpenAI is considering slowing down the development of cutting-edge artificial intelligence, and the ChatGPT maker’s Chief Executive Officer Sam Altman is hoping that other AI companies will do the same.
They won't. There is no slowing down this development anytime soon and no one will slow down for someone else to surpass them.
Of course they want the government to mandate a pause. Cutthroat competition is exhausting, and a legally enforced cartel lets everyone lock in their current market share without the pesky inconvenience of competing.
there is no chance, given the strategic implications for war and economics alone, that governments and other private entities are going to stop pursuing this at a breakneck pace
Geopolitics gets in the way. It is a race between America and China.
There's no way to know what is "safe enough", so you might as well go as fast as technology will allow, and be on the watch out for negative things you can predict specifically and then adapt so as to minimize net negatives.
It doesn't have to be a race, that's just the way we have been treating it so far. There have been examples in the past of civilization ending technology that has been redirected from being a death race (chemical, biological, and nuclear arms in particular.)
China has stricter regulations on models than the US. For example you can't release a model if the hallucination rate is above a certain threshold.
The US hasn't fully woken up to the fact yet that the race to superintelligence is a death race. China (population, members of the government, academics, lab employees) could also just not have fully grasped it yet. I'm personally confident that it will be possible to explain it to them.
Not sure nuclear arms is a great example. There are more countries with nuclear weapons now than in 1970. India, Pakistan, and North Korea have all joined the nuclear club since then. The SALT and START treaties are no longer in effect.
Perhaps CFCs (Montreal Treaty) is a better example of international cooperation to solve a potential extinction-level event, though it was never a "race" in the same sense.
even if both sides agree to a pause china will annihilate the us in the robotics race (for which there will be no slowdown). software/llms is america's only edge
Granted I can only read English, but I find that I constantly see this take from Americans and I have never seen anything to suggest that developers and tech leaders in China actually feel this way. But just based on the models that come out, it seems like the Chinese labs - which, funny enough, are less monolithic than the two-man race in the US where the two giants seem to trade personnel regularly - are trying to make smaller, more efficient models rather than building a machine god. To the extent there is an intentional geopolitical angle to this at all, it seems like the US is banking on landing one massive punch - developing a superintelligence and then using it to dominate the world - while China is attempting to sabotage the assumptions that make a superintelligence economically viable.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
if it's a political issue then this is just Sam telling people to do one thing while still riding his SVPs and other leads hard on roadmaps, 'boldness', and delivery metrics
which is to say - unlikely for it to materially slow, only a lot of chirping about slowing down to the public with a couple outlier examples of 'slowing down' they can show publicly
It seems that every time an article is posted on HN about a lab leader calling for a coordinated Pause of AI development, a sizeable fraction of commenters are interpreting it as hype or admission of failure, often with high confidence that AI progress is slowing.
Is would be understandable for it to be wishful thinking and dissociation with reality facing the fact that our jobs and what many of us love doing is being automated and taken away, and our craft being made economically worthless.
But no one knows where the limit is. It was only one year ago that Claude code was starting to become useful and now it handles large code bases with a dexterity that is unseen in humans, where within each session it basically starts from a clean slate.
So instead of denying that this is actually happening right now, it would be much more productive to actually take sama and others at their word and push for global regulation, to force them to do as they say and slow down with this technology and proceed more judiciously.
LLMs are incredibly good at handling code. But you're extrapolating their capabilities just because their handling of code is so shockingly good (compared to what we had before).
I think AGI will be eventually achieved, but it will not come from an LLM, and it will not be anytime soon.
I'll worry when a more promising architecture comes online.
Why wouldn't being incredibly good at handling code and reaching the goals there extrapolate to other cognitive disciplines?
And it didn't even start with code, but just from being incredibly accurate and precise at predicting the next token. From that emerged being incredibly good at handling code, and being incredibly good at finding counterexamples for major open mathematical problems.
The thing that these agentic systems are getting incredibly good at is steering outcomes, that is to make the world behave in a way that fits their objective. We had that with the Go AI on a game board, now with code on a computer and with mathematics in an abstract world determined by axioms.
And then people just say "Ah, because sama said we should Pause this means that this is it, the bubble is popping". I wish there was a real, strong argument for why ASI will not be achieved by LLMs, and why it will never be able to steer outcomes in the real world. But so far the disciplines just keep falling as LLMs are scaled further, and there seems to be no limit in sight.
After eating his 65th hot dog, Joey Chestnut reassures Nathan's Famous International Hot Dog Eating Contest organizers he will stop at 66 to ensure they don’t run out of hot dogs
It is a convenient time, just as coding capability seems to reach the upper end of the S curve.
How? Coding capabilities keep improving, there is no break
Even benchmark improvements over Sol are not that great (and they certainly tried). If we exclude the weird ARC-AGI situation.
> How? Coding capabilities keep improving, there is no break
So? The upper end of the S-curve is also a line that still rises.
is there some metric that supports this?
feels and hype
HN comments are just on a different planet lately lol
I‘m using Sol all day every day. It’s on average doing better than Astra at what I need it to do. Astra is like an absent minded professor - gives good direct responses, but too inconsistent and forgetful for my codebase.
How long would it take for a team of 60 to replicate the software that operates a neobank?
Serious question - I want serious responses pls ty.
? What world are you living on.
Let me ask you, have you noticed that all the major labs are releasing models very similar in gains and performance? How do you explain that other than juicing the max out of the current architecture and processes.
They have been releasing similar gains/performance for years now... Pretty much never has one company been dominant for a long time (except the initial GPT3/4 release I suppose. That period took a while for others to catch up)
Yeah - that's why I think they are basically squeezing the scaling laws and the current architecture with incremental innovations.
I expect they will continue optimizing and improving for the current use cases/benchmarks. But the core capabilities will stagnant, that's why they will be forced to slow down until another major breakthrough happens.
I personally don't think LLMs have any understanding of the world, nor any imagination or even agency. I think it got very good as recognizing patterns and shapes in human thinking, language and knowledge, but that's pretty much it.
? "everyone is advancing but tied" is completely different than arguing we're at the top of an S curve
I think it's narrowing at the top.
Unless you’re making 3D models of apartments and oneshotting trivial games, I‘m not convinced Astra writes better code than Sol. I‘m way more often disappointed about Astra forgetting some aspect that Sol just nails.
The one where it’s infeasible that a company that needs to live up to a valuation of like 5% of US GDP but is still burning cash and has no moat would willingly decide to slow down the pace of development of their core technology
> has no moat
I would love to see someone try to escape with a copy of openai's raw unaligned base model
There is no need for a copy. If another can do most of what OpenAI/Anthropic can do at a fraction of the cost (like the Chinese models) then the moat evaporates.
the capability gap is underestimated. the raw unaligned base model from a pretrain is like the telemetry recorded from a particle accelerator - it's hugely valuable and not just because of the cost sunk building a collider.
the things are kept under air-gapped national weapons grade security measures not because it's literally going to escape and threaten the world but because if somebody walked out the door with a copy they would have everything. the capabilities we see at the surface are mostly the result of mining the great unknown space and attaching feeble control surfaces, ablating/lobotomizing dangerous areas, and fencing off illegal/secret/embarrassing ones.
> the capability gap is underestimated.
The capability gap is in the minds of the users but the frontiers' closed business models run opposite to it.
> the things are kept under air-gapped national weapons grade security measures
That's an enhancement of closed but not of moat
> because if somebody walked out the door with a copy they would have everything.
Including you and me? I'm not sure this is comforting news, despite the "trust me bro" asurances from behind the door where we can't see or verify anything.
I expect they are slowing down in an attempt to keep from having those models improve to the point where they can effectively escape unassisted into the wild.
How long would it take for a team of 60 to replicate the software that operates a neobank? Serious question - I want serious responses pls ty.
“I don’t have anything to deceive you on, of course I’m not deceiving you. I’m engaging with you as a human being. Why would I deceive you? There’s no… no reason for me to do that.”
https://xcancel.com/cocktailpeanut/status/209733229184439951...
"It's totally intentional that progress is slowing," Altman reassured investors.
Sam also said Astra was AGI and he's notoriously honest.
This is just their way of saying they're officially switching from providing AI to using the AI to compete with their customers, a la Navier-Stokes drama.
Absolutely! I’ve never lied to anybody and any work we copied was purely accidental.
When the Chinese model drops that tops a paused Anthropic and OpenAI in all benchmarks how low will their valuations go?
It remains to be seen whether the Chinese AI labs actually are capable of improving on their own or if they just distilled the US frontier models.
Why should it only be distilled when they actually perform better on some benchmarks (see DeepSeeks Flash 4.1)?
Claude will release another employee ballast, the valid but in-conceived alarmism will surely reach POTUS ears.
OpenAI will release the multiple-north-star bench and get 99%.
[dead]
<Not an AI researcher -- Just thinking out loud>
Perhaps it's getting more difficult to move the 'goal posts'?
- Train LLMs on texts -> OMG AI can write!
- Train LLMs on code -> OMG AI can code!
- Train LLMs on images -> OMG AI can generate images!
- Train LLMs on video -> OMG AI can generate videos!
- Train LLMs on math -> give free access to 100,000 researchers/scientists/mathematicians -> invest resources into Lean -> OMG AI can solve math problems!
I say 'goal posts' because that's a common argument, but the perspective that seems to not be considered is that the LLMs are doing the same thing over and over again in different domains while be fine-tuned by labeling and human review.
The emergent behavior argument is a separate consideration, but I can imagine an AI being given a list of words (a harness) that guides how it operates--not the other way around--and the difficult problem is narrowing down and filtering all the associations with the words/harness to properly align it.
And perhaps the recursive looping being discussed creates incredibly difficult (impossible) to predict behavior? If the AI could modify it's own harness, is that AGI? If we fully understood what happens in high dimensional space, and could predict the output of any combination of input tokens, then is it not AGI?
Or perhaps I just tell myself this to stop worrying about a future involving ASI (:
This is the closest to an admission of failure you'll get from sama.
Odd. I (and many others) have been saying for months that we don't know which part of the s-curve we are on.
I guess now we do.
I, too, consider pausing the work on the Grand Unified Theory of physics. It is too dangerous for humanity.
I've paused my work on faster-than-light travel in order to more fully assess the consequences.
Thoughts and open (I think) questions:
1. What are we actually talking about, slowing development or slowing public release? I can't imagine the the US is keen to slow actual development when we have adversarial state-backed labs continuing development. Seems to me that this is a global pact or it's nothing. Stopping a single lab, even if it's the current leader, won't put the genie back in the bottle.
2. Unless they stop actual advancement, isn't the risk essentially identical? At this point, it's an arms race between the lab and its own model. I don't think it far fetched to think that it might successfully exfiltrate its weights, for example.
3. There are going to be factions who don't want to stop. Should we worry about them defecting to other labs or other countries?
They are hoping to use regulatory capture to regain lost market dominance.
The problem is with that much debt buried in the report, no one is buying peoples sandbagging nonsense anymore. Even the boring bond market is showing odd behavior much to the chagrin of congress. =3
Nice story - AI will kill humans therefore we will slow down.
When all labs are approaching the same marginal improvment which is a clear sign that more breakthroughs needed to achieve the god they hyped.
"Woah dude, this is so revolutionary that if we keep pace, it'll like, change the world even more, man... we'd better slow down..."
"BUY BUY BUY BUY BUY BUY BUY"
I just drank a glass of wine. I'll drink one more and I swear to you I'll stop after that, forever.
https://archive.is/zH0ZZ
Hugging Face wasnt really an escaped model that presents a kind of future threat. The task the model was given lead it to discover certain admin credentials that were left publicly in Github. All it did was use that to complete the task.
With other countries and open source, not to mention all the other companies, Open AI choosing to pause development wont change the course. Only OpenAI will lose.
Unless the secret is that no amount of compute or data can progress things any further because we have already reached the limit for what predicting the next token can do. Most of the advancements lately have been on the agentic side of things.
The massive amount of health, science and math problems being solved by AI has largely not materialized yet. Is this a sign the model capability has been over sold?
There are economic consequences to this as well because the data center build out has been a kind of circular financing. This has caused all kinds of macro economic impacts from increased prices to even the Treasury market itself having to compete with debt issued for the build out. No one knows what will happen once the data center build out stops, or if the yen carry trade continues unwinding, or if Treasury yields have to keep rising to compete with debt issued by the "Mag 7".
Once they stop spending in the current circular arrangement, then the profits that exist on paper disappear, and the bubble could pop.
So really for every reason, OpenAI cant really slow development. Unless there is a kind of Moores law for what predicting the next token can achieve.
To all the people whose only opinion on anything that openai does is 'marketing hype', how would you expect them to act if they were making rapid progress and were actually concerned about safety?
Ask for regulation.
If progress is coming easy and fast and you're worried about the danger your product poses, you should also be worried about the danger your competitors' products pose. So, you reach out to governments and ask for rules to constrain and slow things down for everyone, so that your company isn't put at a disadvantage for being cautious.
Are you kidding? They've been asking for regulation for years! Altman testified to the Senate in 2023. HN loves calling this hype, eg. https://news.ycombinator.com/item?id=36034314. They just put this out two days ago: https://openai.com/index/ai-policy-window/. If this would move the needle for you, it should have moved a long time ago.
AI is reaching religious levels of fanaticism, which I find highly disturbing.
Did you respond to the wrong comment?
Wonder if it'll last longer than Anthropic's pause https://news.ycombinator.com/item?id=48403827
no big company will even consider making a step in that direction. The advancement speed nowadays is crazy and if you start pausing for a few weeks you'll probably soon face problems
Why do you on HN want AI to slow down (mainly OpenAI and Anthropic from what I see in the comments)? What's the driving force behind your opinions on this?
Humans not dying would be an excellent feature for the future to have. Not a hard requirement though, we'll see if there are some story points left in the sprint.
I've got bad news for you on the whole humans dying thing. Seems like actually literally everyone is going to die. Turns out that this has actually been happening for a really long time - possibly since day one - and we've got no idea how to fix it.
I keep reading about AI killing all humans but I have yet to see a practical scenario where that happens. Somehow everyone is scared of that but nobody can explain exactly how that would happen?
A lot of people have explained how that would happen, in excruciating detail. The reason why this wasn't a major part of the discourse is that most people (including myself) did not believe jumps in capabilities like those we have seen in the last few years were likely.
For a superintelligent AI trained with methods anywhere close to what we are doing now, killing all humans is the default outcome. I actually think researchers are downplaying the risks to make their arguments more normie-friendly.
https://www.thecompendium.ai/ai-catastrophe
https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
A book if you will: https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_...
More time to proceed more cautiously in building this entity that will be faster and more effective at steering outcomes in the world than any human or, soon after, the entirety of humans combined.
Do you want the Chinese to stop as well?
Of course
What's your plan on when you successfully limit western AI but have no control over China and they continue to march forward?
I don’t want to become one of the lucky to be alive in a human zoo or a human lab…
"The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear. No other human activity poses this level of danger." - https://x.com/hilbertspaess/status/2097476196791709843#m
"Development of superhuman machine intelligence (SMI) is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." - Sam Altman
"If Anyone Builds It, Everyone Dies": https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_...
So given that, do you want the Chinese to stop developing AI too?
[dead]
i've considered doing a lot of things i'll never do
The big psyop/trick is that "AI development has to slow down" and not that we hit limits with the technology or the technology is immature.
Either way it's a win-win for whoever sells that message because you make the makers and the warners the center of attention. And when they say "we are considering slowing down" they mean "if we get enough money to slow down we'll do it". And of course they never want you to think AI is a technology employed by humans who need be changed, not the technology. So as long as you engage this massive straw man to focus on the wrong thing you do nothing to hold the leaders accountable for their harmful and selfish actions and decisions for society.
The message was directed at the employees. Their work may slow down, but that doesn't imply that all work will slow down. OpenAI also has this technology that purportedly can do the work without employees. In fact, if the employee work isn't starting to slow down then OpenAI has bigger problems given the business model that keeps getting promised to investors.
Ah I see. I didn't read the article and just shared my opinion about something unrelated. Thanks.
Was there any species controlled by a less interline species in history? I doubt we will be the first.
“We need to put out the money fire so we can IPO”
> OpenAI is considering slowing down the development of cutting-edge artificial intelligence, and the ChatGPT maker’s Chief Executive Officer Sam Altman is hoping that other AI companies will do the same.
They won't. There is no slowing down this development anytime soon and no one will slow down for someone else to surpass them.
Of course there is a possibility to slow down.
- 1386 employees of frontier labs called for it
- lawmakers are calling for it
- employees are quitting in protest
All it needs is coordination to regulate the pause
I would add that the public is increasingly concerned about AI progress, which will put a growing pressure on lawmakers.
Of course they want the government to mandate a pause. Cutthroat competition is exhausting, and a legally enforced cartel lets everyone lock in their current market share without the pesky inconvenience of competing.
there is no chance, given the strategic implications for war and economics alone, that governments and other private entities are going to stop pursuing this at a breakneck pace
They will if it becomes a political issue that causes regulation, which is looking likely.
Geopolitics gets in the way. It is a race between America and China.
There's no way to know what is "safe enough", so you might as well go as fast as technology will allow, and be on the watch out for negative things you can predict specifically and then adapt so as to minimize net negatives.
It doesn't have to be a race, that's just the way we have been treating it so far. There have been examples in the past of civilization ending technology that has been redirected from being a death race (chemical, biological, and nuclear arms in particular.)
China doesn't regard AI as a death race. It regards it as a way to ensure its own global stature and benefit its own population.
China has stricter regulations on models than the US. For example you can't release a model if the hallucination rate is above a certain threshold.
The US hasn't fully woken up to the fact yet that the race to superintelligence is a death race. China (population, members of the government, academics, lab employees) could also just not have fully grasped it yet. I'm personally confident that it will be possible to explain it to them.
Not sure nuclear arms is a great example. There are more countries with nuclear weapons now than in 1970. India, Pakistan, and North Korea have all joined the nuclear club since then. The SALT and START treaties are no longer in effect.
Perhaps CFCs (Montreal Treaty) is a better example of international cooperation to solve a potential extinction-level event, though it was never a "race" in the same sense.
even if both sides agree to a pause china will annihilate the us in the robotics race (for which there will be no slowdown). software/llms is america's only edge
Granted I can only read English, but I find that I constantly see this take from Americans and I have never seen anything to suggest that developers and tech leaders in China actually feel this way. But just based on the models that come out, it seems like the Chinese labs - which, funny enough, are less monolithic than the two-man race in the US where the two giants seem to trade personnel regularly - are trying to make smaller, more efficient models rather than building a machine god. To the extent there is an intentional geopolitical angle to this at all, it seems like the US is banking on landing one massive punch - developing a superintelligence and then using it to dominate the world - while China is attempting to sabotage the assumptions that make a superintelligence economically viable.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
The AI race is caused and being sustained by American tech companies. China is an excuse to keep it going.
if it's a political issue then this is just Sam telling people to do one thing while still riding his SVPs and other leads hard on roadmaps, 'boldness', and delivery metrics
which is to say - unlikely for it to materially slow, only a lot of chirping about slowing down to the public with a couple outlier examples of 'slowing down' they can show publicly
Fatalism is a toxic mindset. Is it hard? yes. Is it impossible? No.
They will if they run out of money to light on fire.
The dominant opinion on HN is, apparently, "It's been a full four days since OpenAI resolved Navier-Stokes, they must be hitting a wall."
It seems that every time an article is posted on HN about a lab leader calling for a coordinated Pause of AI development, a sizeable fraction of commenters are interpreting it as hype or admission of failure, often with high confidence that AI progress is slowing.
Is would be understandable for it to be wishful thinking and dissociation with reality facing the fact that our jobs and what many of us love doing is being automated and taken away, and our craft being made economically worthless.
But no one knows where the limit is. It was only one year ago that Claude code was starting to become useful and now it handles large code bases with a dexterity that is unseen in humans, where within each session it basically starts from a clean slate.
So instead of denying that this is actually happening right now, it would be much more productive to actually take sama and others at their word and push for global regulation, to force them to do as they say and slow down with this technology and proceed more judiciously.
LLMs are incredibly good at handling code. But you're extrapolating their capabilities just because their handling of code is so shockingly good (compared to what we had before).
I think AGI will be eventually achieved, but it will not come from an LLM, and it will not be anytime soon.
I'll worry when a more promising architecture comes online.
Why wouldn't being incredibly good at handling code and reaching the goals there extrapolate to other cognitive disciplines?
And it didn't even start with code, but just from being incredibly accurate and precise at predicting the next token. From that emerged being incredibly good at handling code, and being incredibly good at finding counterexamples for major open mathematical problems.
The thing that these agentic systems are getting incredibly good at is steering outcomes, that is to make the world behave in a way that fits their objective. We had that with the Go AI on a game board, now with code on a computer and with mathematics in an abstract world determined by axioms.
And then people just say "Ah, because sama said we should Pause this means that this is it, the bubble is popping". I wish there was a real, strong argument for why ASI will not be achieved by LLMs, and why it will never be able to steer outcomes in the real world. But so far the disciplines just keep falling as LLMs are scaled further, and there seems to be no limit in sight.
I guess we'll see, but odds are that reality will reel you back in.