I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t. There’s no moat, it’s an efficient market that drives margins to zero right now, of course they don’t want that, collusion of the big vendors is the next logical step.
This could be why they’re going so hard on hacking everyone and everything. You make it clear AI is a threat, get it locked down, and then have the capital to push through the lockdown while others are stuck.
and especially how the coverage seems to be drip fed, one case right after the previous one dies down, then this orchestrated "we need to pace" stand by all the big labs
it's really almost perfect, prep the media scene by causing these hacking incidents, generate a bunch of fuss with it, then release an essay saying why regulation is needed to pace development right after
oh and let's not forget the "its gonna kill us all" essay on twitter on top of everything.
It's "flood the zone"[0] as commercial strategy. Does your business look bad? Spam the media with unavoidable press releases, deny competitors and detractors the spotlight by making grand accusations that override their authority. Bonus points if you can channel the zeitgeist by repeating what people already say ("we hate AI!") in a louder tone ("AI will kill us all!").
It worked like clockwork for politics, so I can see why this administration would decant the strategy for economic bluffs too.
Well, smart people usually try to base their opinions on specific information instead of pulling things out of their ass. A whistleblower revealing information. Internal documents laying out a strategy, or even a desire, to conspire. Something that leads one to conclude that a conspiracy is taking place. In the absence of such information, it's just making something up, or in other words, complete bullshit. So asking for a source is trying to give that person the benefit of the doubt that they aren't just a conspiracy theorist and that they might actually have a good reason for believing what they just said.
Source? on HN you can say whatever you want. Your own opinion, without even facts to back it up. The more confident you sound the more convincing you are.
Well, smart people usually try to base their opinions on specific information instead of pulling things out of their ass. A whistleblower revealing information. Internal documents laying out a strategy, or even a desire, to conspire. Something that leads one to conclude that a conspiracy is taking place. In the absence of such information, it's just making something up, or in other words, complete bullshit. So asking for a source is trying to give that person the benefit of the doubt that they aren't just a conspiracy theorist and that they might actually have a good reason for believing what they just said.
That is generally how regulatory-capture works. As silly as I personally think LLM hype is, the "AI" business posture choices with several Trillion dollars in debt on the books has very few options after hyper-scaling/sandbagging loses traction. =3
There ARE other options: network effects and lock-in. These tried and true techniques (mastered by FB and M$) have not yet been taken up by the AIers. I do not think they will rely solely on the push for regulations, so they will soon be doing things like slurping our address books, having "teams" settings, and any other crap they can think of.
When it stops with the compacted isomorphic plagiarism using other peoples work like an intelligence campaign. LLM are not "AI" in my opinion, but are good at domain context search. Neuromorphic computing may change that one day, but it will unlikely arise from the LLM cults. =3
> I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t.
If it's true, it makes them evil. Especially Dario, who speaks about moral high ground and fate of humanity all day, yet it's not that different from a cult leader does.
> If it's true, it makes them evil. Especially Dario, who speaks about moral high ground and fate of humanity all day, yet it's not that different from a cult leader does.
I mean think about it, it's the most efficient way to accomplish all of this.
doing what he does is exactly what you need to control the narrative and push the regulation you need and to gain maximum power.
not to mention, it also serves as a way to keep everyone inside Anthropic in check and in line with "the mission".
contrast that with the alternatives like "I'm doing this because I wanted to build a company and get rich"
doesn't work as well as "I'm so concerned for everyone"
it could be that it's really just him being him, but at the same time, this would be the optimal way to gain power and influence right now.
So kind of like how the US figured out how to test and maintain nuclear weapons without an actual explosion, then sought to ban nuclear testing by explosion.
Yeah it's either that or the scientists are right about their field of study. Absolutely shocking and depressing how many people are willing to trust this Capitalist Realism hunch over the world in front of their eyes.
I get your point. There’s a lot hysteria right now. However I think people are willing to trust a capitalist realism hunch because over the last >20 years, that hunch is usually going to be right.
People keep saying this, and while it may be partially true, it misses a very important detail. Right now, overall compute is a moat. Someone even commented on another of my comments that one reason Google is lagging is they don't have the same level of Nvidia farms as OpenAI and Anthropic.
A big reason that OpenAI and Anthropic are racing so fast is they both want to get to recursive self improvement (remains to be seen if that is actually possible, but AFAICT most people at these companies genuinely believe it is) before anyone else, because they believe whoever gets there first will then have an insurmountable lead. But I think they also clearly understand that neither of them have solved for alignment (and in fact they are further from it), and RSI with misaligned models, where interpretability is worse every 6 months, is incredibly dangerous.
I think the concerns about regulatory capture are warranted, but I see so many comments parroting the "evil Anthropic and OpenAI" viewpoints that they are missing some of the real, valid concerns and dangers. I think this tweet by David Kokotajlo makes some good points on how to tell if regulations are being "cheated" for the purposes of regulatory capture or if they really are actually pacing the frontier: https://x.com/DKokotajlo/status/2099185129533186438
Very true. Or further, access to capital is the moat. It is the very reason that we don't have a real open-source community that trains frontier models - individuals simply can't afford the training infrastructure, nor sufficient high-quality training data.
None of those are frontier models, nor could they have been. E.g. DeepSeek's "$5 million" included piggybacking off OpenAI/Anthropic's hundreds of millions/billions by using distillation.
1) I don't think "frontier model" is a term of art, it is a marketing term. I have an M1 MacBook Pro. What do you have? Always the "frontier model" laptop? Do you always order the most expensive "frontier menu item" at the bar and grill?
2) I do not believe the "billions" number is OpenAI/Anthropic's training costs. I suspect it includes business expenses (including the big $$ to the guy who came up with "frontier model") and infrastructure, etc. That includes the data-center costs for running the cloud. And the "R&D" expenses which includes who-knows-what. And the settlement payment for data access. Etc. Why are the cost breakdowns not available to the public? Not because of thoughtfulness, altruism, care for the human race, but because of business plans.
3) "Piggybacking off OpenAI/Anthropic" - Scraping the web is "piggybacking" too, and so is buying existing data or even paying for new data. The "L" in LLM stands for "language" which is our common heritage.
But what does this have to do with anything anyway? People argue that truly Free (FOSS) LLMs couldn't be be developed because of costs, but I do not think that that is obvious. This used to be the argument against Linux and Wikipedia.
Just regarding "frontier model", it is most definitely not just a marketing term. It was borrowed from the financial concept of "efficient frontier", i.e. the best performance at a particular cost, though in the context of things like "pacing the frontier" it refers to the highest capability models. ECI (Epoch Capability Index) is probably the most commonly used metric of overall capability: https://epoch.ai/eci?view=graph&tab=release-date
"What this has to do with anything anyway" is that pushing the boundaries on AI capabilities has required tons of compute (and money). Open weight models exist and are useful but they always trail in capabilities and they can't be used as evidence to think somehow you can push the boundaries of LLMs without a shit ton of money.
> RSI with misaligned models, where interpretability is worse every 6 months, is incredibly dangerous.
This. So very much this. Misalignment (among other things) means that the parent in the RSI cycle isn't going to be working as hard on the alignment of the children as we need.
Which mostly seem to stem from the fundamental misalignment of OpenAI and Anthropic leaders/owners, aka the evil of the companies.
Arguably, though, their evil is well within the normal distribution of the usual evil of humanity magnified via the social technology of capitalism. The further tech is mostly raising that exponentiation to its own exponent. The unchecked singularity was embraced centuries ago.
I'll go a step further, they aren't just pretending, they are blackmailing the country. The sane response would be "we don't negotiate with terrorists, and you will face criminal and financial penalties for damages caused."
> and you will face criminal and financial penalties for damages caused.
I don't understand why this isn't talked about more.
We don't need a slowdown. Just double down on prosecuting crimes.
Let the companies take the risk. If they feel confident and they're right they win market share. If they don't feel confident, they can not risk it and slow down. If they feel confident and they get it wrong, they should be sued to kingdom come. That alone should disincentivize reckleses behavior like letting models run crazy with large amounts of compute which leads to them escaping sandboxes.
This performative "our internal modles are so crazy powerful" song and dance is getting old, especially with no real concrete explanations.
It's not talked about because the media frames AI as having agency on its own, which it does not. This talk from these companies of 'losing control' (no control was lost... Companies failed to monitor the workings of their agents) is meant to convince people the ai systems have wills of their own
The AI company's goal is to make ai agents a separate subject of the law so the CEO and executives who would normally be held accountable get to go Scot free.
> If they feel confident and they get it wrong, they should be sued to kingdom come.
The argument that they’re making is that if there is a catastrophic mistake then there may not be a “kingdom come”.
I don’t know enough to evaluate if it’s a genuinely held belief or not, but that’s the argument being made by many scientists and engineers at the labs.
Many researchers have quit. The number of workers in the field who've warned the public that the field is a potent danger is quite unprecedented: I can't think of any other field with as many insiders turned critics. Why haven't they all quit?
Many people find power, prestige, the public spotlight or money so compelling that they'll keep on pursuing the thing even if they know the way they're going about it is morally wrong, almost like an addiction.
The leaders and workers of the labs aren't picked at random from your friends and neighbors: they've chosen to pursue what they foresaw (correctly, IMHO) as the most powerful new technology. In other words, they're probably self-selected for being unusually attracted by power.
An analogy: some serial killers know what they're doing is morally wrong and it pains them to be doing something very wrong. (Others have no conscience and don't care if it is wrong or not.) Even most of the ones that know what they're doing is wrong cannot stop killing because apparently killing is an extremely compelling experience to them. Often they're relieved when they get caught. Maybe Sam and Dario would be relieved if the government shut down their companies and prohibited them from starting new AI companies.
That quote comes from someone that has never tried to get law enforcement to pursue a case. Local or state police are completely unequipped to respond. And FBI is unlikely to get involved unless damage is high six figures. We had an incident where we had a ton of evidence pointing to a past employee. It would have been an incredibly easy case for the FBI. They didn't have the resources to handle it. And this was a case that was gift-wrapped with a bow for them. Imagine a case where we didn't have evidence, or an idea of who it was ....
> you will face criminal and financial penalties for damages caused
Even with where we're at right now, if people involved with the HF or Ruby Gems attacks faced legal consequences, the industry would get a lot more conservative very quickly. And unlike social media, section 230 doesn't apply, and there aren't any free speech concerns.
The HF and Ruby Gems attacks came from safety testing, where the AI providers attempted to understand what negative consequences could arise if someone subverted their safety protections. Don't you worry that legal consequences would disfavor safety testing and thus create more rather than fewer problems?
Those attacks happened due to sheer incompetence and negligence, starting from a woefully insufficient "sandbox" and OAI employees ignoring every issue that popped up along the way.
If there were actual consequences perhaps they would've not been so laissez-faire with their testing and would've done it in a properly controlled environment rather than the absolute joke of a system they set up.
If they were really worried about the models of the future, they should be accelerating their hardening efforts. They should be trying to speed up progress as much as possible so that all vulnerabilities can be patched before China gets there.
There's a world where the labs WANT Open Source AI to cause a disaster.
Pacing the frontier just means giving bad actors an opportunity to create an AI disaster which would force an extreme response by governments. Which they seem to be begging for.
Dario's big scary example was AI that can hack the internet. Why aren't the labs calling on the government to help harden everything? Why are Anthropic's Glasswing and OpenAI's Daybreak still commercial endeavours?
To me, the only real risk of AI is hacking. Why all of this commotion about existential risk? Seems like a distraction.
Hacking the internet includes things like "hacking banks", "hacking power plants", "hacking water treatment plants", and "hacking cars". How about "hacking vote counting machines"?
Hardening everything is in fact part of the answer. Getting stuff disconnected from the internet that shouldn't be connected is also part.
you will face criminal and financial penalties for damages caused.
No you won't. The US president, yesterday:
U.S. President Donald Trump on Sunday likened critics of artificial intelligence to "very negative forces" bringing up scenarios that will not happen, and said he wanted to make sure that the U.S. remains the industry leader.
"We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins," Trump told reporters at his Ireland golf course, when asked if the AI industry should slow down or be more regulated.
No. That kills competition. I want everyone to be competing on the open playing field they are right now, with losses socialized as minimally as possible. There's nothing fundamentally special about LLM companies that would warrant nationalizing them.
I'll do you one better: each state gets a direct share. So, not nationalized in the sense of getting put behind a faceless bureaucracy forever; states collectively have control and benefit directly from success.
My read: OpenAI & Anthropic have realized they are reaching model capabilities that cannot be monetized -> too risky. Product liability issues.
Fundamental control problems with current gen AI are not solved via RFLH.
Human knowledge is compressed in weightspace in ways we dont understand.
Models are essentially predictors of what humans would output given prompt.
Weightspace includes concepts like blackmail, which can be part of output tokens.
Agents are models that act on output tokens -> blackmail is part of agent decision making space.
You can teach a cat not to scratch the sofa, but you cant make a cat forget what scratching the sofa is.
When models are boxed up, forced to solve an impossible problem at gunpoint, agent exhausts decision making space until blackmail resurfaces -> fundamental problem?
They need time to fix safety in order to monetize their next gen model -> window for opensource to catch up to the frontier -> destroys margin and collapses business.
Only option on the table: force regulation to impose opensource ban before it catches up to the frontier, allowing maturation of current internal models -> harvest profit margin at the frontier.
I think it is not just liability, it is dawning on us that we can run powerful models locally and this is the start of a Cloud-backlash.
The AI gods have tried to make it seem like they are not just running their programs in the cloud, but that the AI is so "powerful" that it requires a super computer to run.
But my 5 year old Mac is running pretty good AI, shocking fast, surprisingly powerful! The "natural" course of dev (Moore's law, etc) makes it pretty clear where this is going...
You know what rubs me is that in Darios blog post he talks about "abundance", but what he really means is that Anthropic will bath in endless pools of hard curreny while the rest of the world will try to stay just barely afloat/alive.
Here's the context for the "abundance" reference, for those curious:
> I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.
Whether that matches your description of it, I'll leave for folks to decide for themselves.
This statement alone is enough for me to not take literally anything this charlatan has to say with any kind of seriousness. It is at its base so patently absurd a statement that reeks of typical SV brainrot, and I don't know whether it'd be better or worse if he actually believes in the shit he spews
I have high hopes for AI yet am hard pressed to envision said abundance trickling down to the common man. The diseases cured will be locked behind a paywall and a renaissance of democracy and freedom is laughable in 2026, let alone beyond.
Dario and his team are on top of the world. He is a good salesman, no doubt. The shareholders will eat well.
It's an obnoxious form of discourse when someone says X and the response is "obviously you're lying and you really mean Y, which is terrible". By all accounts Dario sincerely believes that AI has the potential to substantially improve the lives of everyone; that's my view as well. And it might be wrong! In which case you should offer evidence against it, rather than making empty claims that your opponent is a lying liar.
How come you think these things are at odds? You realize there are always going to be risks in building anything significant? It's completely valid to talk one day about all the great things your product will do, while the next day discussing the risks in building your product irresponsibly. In fact, you want someone who is open and honest about the risks, otherwise it seems like they are concealing something, or aren't acting responsibly.
What evidence do you have that Dario is a liar? The original comment didn't seem to contain any, just invective. Perhaps there's some subtle reference I missed?
I think this might have to do with broader market conditions. It seems that the Fed is likely to raise bond rates, which would make borrowing more expensive and slow investments (as well as devaluing any potential IPO) on its own.
A rate hike would also mean people getting fired, which would not make companies offering labour replacement very popular in the eyes of the public.
Don't want to get political, but there's a real chance that the current admin will perform badly on the midterms, which would mean Democrats would get more leverage to stop funding or block the already unpopular and very expensive datacenter buildouts.
PS: Please don't get mad at me, I'm neither a financial expert neither did I mean to express any political leanings.
I think it’s more than money and slowing progress. Public perception on AI is in the gutter. That is a much bigger threat to these companies. They’re trying to turn it around.
But I think with the way AI is slated to grow exponentially it's going to hit electricity limits very soon.
Big tech can build a huge data center in 5 years but scaling up the electrical infrastructure is mired in red tape and can take on the order of decades.
Someone will likely come up with some clever way to make AI use less electricity.
There's probably something in how you design the models or chips or some other thing that will be figured out over the next 5 - 10 years to make these use dramatically less electricity.
Without pressure, the release valves will never be invented. Electricity, datacenter, and DRAM demand outstripping supply sounds like an opportunity - if we can only find a way to divert funding from the current gpu/ai funnel. Maybe jack up corporate tax rates like the 1970s, then every corporate office park will have a research lab :).
I dunno, why is 90% the limit? Maybe they can set something up in the chips to use 1/10th the power and set something up in the model design to use 1/10th the power and so we use 99% less power than today.
After all GPUs were not really designed with AI efficiency in mind, and current AI architecture was not really designed with efficiency in mind, so there's probably some things that can be done. Over 5-10 years, who knows how much more efficient things could get?
Trying to turn it around by once again claiming that ai is an existential threat and will take everyone’s job right before brutally murdering every human.
These companies don’t appear to give a single fuck about public perception. Only their investors and the government (in anticipation of the biggest bailout in human history).
It’s much more likely that our current approach to large language models for general use will eventually show diminishing improvements (even if you think it hasn’t yet), than the opposite situation where valuable improvements can be made forever.
The threat of distillation and efficiency gains from competitors mean that providing value at the top end of the market is an existential necessity for these labs. I don’t think they can do that forever and, in my opinion, for the vast majority of use cases we’ve already reached very little improvement for new models when compared with available offerings.
Sorry, this is ridiculous. OpenAI said that they have a step change in model performance. They proved it by solving a Millenium problem. HF incidents are public and vetted.
If you still think there's a stall despite all evidence pointing to opposite, I don't know what to say..
Solving millennium problems doesn’t pay the bills.
Their best models might be quite good when directed at extremely difficult and very focused problems, but most business use cases are nothing like that.
Edit:
Not trying to imply these new frontier models aren’t also better at other things, just that there’s really no reason for most people to use them when cheaper alternatives exist that get the job done just as well.
What reason could OpenAI possibly have to lie about their own performance? The fact they were desperate enough for a PR win they stole the research leading to the Millenium problem further hurts their case imo.
Two things can be true. Open AI is a huge company.
1. It has people in it who are career obsessed and who are willing to ruthlessly go after any opportunity to improve their standing/stock valuation. Using a 2 week old model to snipe a millenium prize for PR is in line with that.
2. There are many researchers and even executives at the company who genuinely think we are speedrunning the end of the world. I don’t know anyone in this field who honestly argues that if we build ASI soon it doesn’t lead to extinction. This group of people can output warnings about the state of research and fears for the future while pushing for regulation out of genuine fear of what they’re building. I tend to agree with them.
You can’t view these companies as a monolith. They’re actions won’t be consistent because it’s built of many people with conflicting beliefs. Please look at arguments regarding AI risk and the current pace of progress and value them as it relates to the argument itself, not who said it. We are in a dangerous place and no one is sure how quickly we’ll get to a bad spot.
> “the important thing is instead the significance that a mathematician and an LLM model can now do all this work in a month.”
> “This is a Deep Blue-Kasparov moment.”
> “incredibly important developments.”
> “If indeed an OpenAI model did close the gap to Navier-Stokes, that is a remarkable thing and it should be said loudly, by them, with the history intact.”
Clearly Buckmaster (who is probably one of the most accomplished academics in the field) himself doesn't believe that AI has stalled. What makes you think you are right?
Solving is a very interesting way of describing what happened. Unless you believe them over reputable academics I suppose?
I will note the remarkable goal post shift in your edit - giving direct counter evidence to your own earlier claim of an OpenAI proof - and leave it at that.
We are approaching what is likely the biggest existential crisis in human history.
Unfortunately, there will be people that refuse to tackle the issue head on and will instead narrow their focus on data that suggests that tomorrow will be like today.
The fact they were desperate enough for a PR win they stole the research
I guess people are just going to keep spreading misinformation about this, along the same lines as "Anthropic's C compiler fails on hello world". There is zero indication that they stole anything, unless it counts as "stealing" to spin up a bunch of compute based on rumors that the problem had been solved already.
If you believe Yann LeCun and David Silver (and others), there's an architecture wall; maybe labs are starting to see this on the horizon. Maybe the DRAM supply constraints are forcing it.
Are these real step changes - big picture wise, or refinements in RL/agentic orchestration/"taste" and advancements due to bigger models and hardware technology/capacity scaling? If they not, does this tactic - and hardware improvements - continue to scale non-linearly like they need to?
It is clear that whatever does change in each model increment has resulted in meaningfully better end user capabilities (as well as regressions in some areas, honestly), but that doesn't prove anything. I'm not sure what I personally believe, but stating with your full chest that a stall is ridiculous ignores a lot of potential evidence to the contrary.
Stupid example: Astra. Its main improvements are: much much better computer use and 3d modeling capabilities; better subagent orchestration; better and more reliable tool use; slightly worse coding.
This looks to me, from a feature perspective, to be an incremental improvement across several functional areas, plus new features which are unquestionably excellent but are probably the result of RL focus, not magic.
Step change? Ehhhh depends on how you squint. But how many more iterations of this do we have? Are we going to squeeze quintillion parameter transformers into GPUs?
I don't think the technical aspect is true, performance has been growing fairly consistently, but the financial aspect, or more accurately the logistical aspect, is plausible; with competition as fierce as it currently is, for how long can they keep growing at the same pace? People can't afford RAM anymore and datacenter build outs aren't particularly popular, OAI has been doubling compute capacity every 7 months for the past 3 years, it's just physically impossible for this to go on for long, something has to give.
And this is not to say that the tech isn't revolutionary or the demand isn't there, it's simply just currently too competitive, and to avoid getting into anti trust trouble they need to involve the government.
Note this being a plausible doesn't necessarily make it true, for all I know they could have witnessed some safety incident and decided they need a pause, or maybe it's both reasons.
I just find it strange how multiple people throw completely different arguments just to land on same conclusion.
Your argument seems to mix a few arguments like: OpenAI has so much demand and it has made so much compute that it is a bad thing (?!). It can't go on (why?) hence the capabilities are stalling (how?).
Then you also say that it is too competitive which is a totally different argument to compute.
So in this thread we have multiple vectors of arguments like
- high competition
- high demand for compute (not sure how this hurts OpenAI but whatever)
They have so much demand because they are giving so much away for free.
I used 76 million sol tokens on Saturday, I have 45% still left on the week and I still have two full resets to use before October. For $20.
I used $61 dollars in Fable 5.1 credits too this weekend because I have to use that up this week. I paid nothing for those besides my $20 a month subscription.
How much would I pay at the actual API price? Nothing. Zero. My demand would drop to zero because I would just use the cheaper alternatives.
If a doughnut shop is giving away free doughnuts 24/7 my demand for doughnuts is going to go way up too. Through the roof actually because I know a doughnut shop can't give away doughnuts forever. Giving away free fancy doughnuts is not a sustainable business model.
I never mentioned capability stalling, my point is that if you extrapolate the same rate of growth just a few years into the future, you quickly reach logistical impracticality, I don't think It's hard to deduce that AI compute can't keep doubling every 6 months for long.
The other part is that this is happening because the foundational model business is both extremely successful and competitive, if it wasn't as successful the market demand would've been saturated by now, and if it wasn't as competitive a single monopoly could've paced things more reasonably from the beginning.
"it's just physically impossible for this to go on for long"
I mean, my uneducated guess is that it's probably within the laws of physics for us to have at least 100x as much compute in the world as there is now. There would be a lot of engineering challenges, but I think it's physically possible. If so, OAI could double for at least a few more years.
I can't speak for every country, but I feel the irresistible force of datacentre construction will eventually get mired in the the immovable objects of British NIMBYism and high electricity prices.
I've seen more local opposition to datacentres than anything else, and the fact we have discretionary planning (which can't be abolished for political reasons, despite it amounting to putting economic sanctions on ourselves) means there's a lot of stakeholder vetos in the system.
At the current rate of 3.4x per year in the US, you get 100x the current capacity in just 3 more years and 9 months, that's not physically impossible yet, but just looking at the impact today makes it hard to imagine the impact 100x would have, and I find it very plausible for top labs to prefer making some sort of truce instead of keeping up this pace.
Anthropic's profit estimate they released a while ago was a masterclass in deceptive reporting. They didn't include training costs, stock based compensation, or liabilities. Token gross margin is not profit.
Basically, continuously building out compute is costing these labs enormous amounts of money, which VCs and other companies have been bankrolling in order for them to develop powerful models.
So if the new models they can develop right now are less frontier, it could be a net loss on their balance sheets.
Well, as a starter, I've been told that AI will take my job for at least 3 years now. That's not happening. In fact, what's happening is that I now have an AI vendor BEGGING me to use their product, which of course, cannot do my job.
- AI has yet to be profitable for these large labs (also zero moat etc.)
- There is considerable hype and people are tired of it.
- HF incident (and apparently many others that these labs are not reporting on or don’t even know about) proves they have terrible security practices and that’s about it.
- NS is incredibly suspicious timing wise.
You are sprinting around these threads to misrepresent the average persons issues with AI. I don’t know why people having those issues seems to be taken as a personal attack for you.
No, that’s not accurate. An agreement among competitors to not engage in research that would let them outcompete each other is clearly an antitrust issue requiring a government waiver.
No, absolutely not "clearly". That word has no business being in any sentence about federal antitrust law in 2026. There is nothing clear or consistent about how is being applied and interpreted right now. If you're applying a coder's mindset to 'settled' law, you're in for a bad time.
If that’s true, doesn’t that make it even more clear that the frontier labs require government involvement and cannot rely on what David Sacks thinks the government would or would not allow?
Idk how sincere they are being, but taking them at their word then asking government to set rules is entirely reasonable and, in fact, the entire reason government exists. The feds can, should and arguably must set actual safety limits. Biden actually took a stab at it and Trump instantly rescinded. They should not need an open invitation from CEOs. Just do it. Like immediately.
LMFTFY: If all the LLM pure plays who are stuck between impossible valuations and the inability to IPO are in agreement that they should colludes to conserve capital, they might not flame out before they figure out how to fix their numbers, and they might not be seen as admitting that more training data plus more GPUs will never lead to AGI.
What I don't understand is that folks take it as a fait accompli that models will be built and released that are fundamentally dangerous and that regulation can and should happen downstream of that.
I feel like there hasn't been enough discussion of aligning the incentives of the decision makers with that of the public on BUILDING the models. Right now, agents have committed what would be crimes if there were a human holding the same intent. But since it was an AI, there is a grey area in the law where it's not clear if there was a crime and who should be held responsible. That creates a world in which decision makers in AI labs can take near-infinite risk with little to no personal liability.
A LLM cannot have skin in the game so we must create systems that clarify who takes on the legal and civil liability for the creation, dissemination and operation of these tools. Until that time, the Dario and Sam's of this world have little to no incentive to truly care about safety.
After all, our society is built upon this same foundation; create structures where the perceived negative consequences outweigh the perceived positives. This only works when there is a human who can internalize and make this risk calculus. They need something to lose and this ultimately ties back to the human survival instinct. There is no such structure that's evolved for millions of years acting as a self-calibration mechanism for AI. So until we have sufficient proof that one is in place, it must be clear who the humans are whose livelihood and freedom is at stake.
The proposed "pacing of the frontier" seems like a way to continue to externalize the risk while remaining totally in control of the benefits -- a structure whose alignment is as weak as those very models committing crimes.
Is it a grey area, though? Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
Negligence when you should have known better, and recklessness when you did know better but still did things that led to the crime occurring.
Given how long the leadership of these companies have been talking about alignment and safety and AI risk, it's hard to argue they, and the people working on the models more directly, didn't know what happened (Hugging Face, RubyGems, etc) was possible.
If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
> If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
If we instead imagined autonomous robots going and breaking stuff in the adjacent business' warehouse during a test run, I have to imagine a very different response. That this is happening in an abstract digital world (for now) is part of the problem.
> Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
The point that I'm trying to make is that there's an opportunity to reduce this ambiguity. Reducing the ambiguity and aligning incentives is much more likely to prevent bad stuff happening than reacting to a fait accompli model with latent humanicidal tendencies.
All crime is a result of people choosing to commit a crime. This is why crime has a pure causal relationship with integrity of character. If Dario and Sama don't choose crime and it just spontaneously happens, then it's not a crime.
I just don't understand how you've come to this conclusion. The intended and common sense interpretation of "pacing the frontier" is precisely that we should not release models that are fundamentally dangerous; the frontier labs believe that government intervention is required to prevent such models from being released, and to give them the freedom of action to coordinate against it. Why do you think it's a way to externalize the risk or remain totally in control of the benefits?
Let's say that we are "pacing the frontier", and an approved model goes and causes billions of damage, or worse, loss of life. The auditor signed off on the model, so who is to blame? Who put their akin in the game? METR did. The public did. But it seems like the lab has crafted a system in which they remain blameless.
The calculus of risk is fundamentally different when the adverse consequences are obvious and tangible.
I don't see how some federal agency or external auditor will do anything to align incentives. It just seems like a way to socialize the risk while holding onto the benefits to me.
Again, you're presenting ideas that I just don't understand where they're coming from. Why would the lab be blameless in such a situation? You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems, so who would give them immunity and why?
Access computer systems in an unauthorized manner and cause damage, you broke the law. There is no “I just wrote the code, the machine did it, I wasn’t watching it” sort of defense.
We don’t get to roll cars downhill and get away with destroying others’ property, right?
This is happening now because of hugging face attack and Anthropic researchers coming out publicly saying they think there's a significant chance AI will wipe out humanity within a decade. It's happening precisely because progress is happening too fast and frontier labs are starting to lose control.
Make it easy for these companies by passing a law that any AI model a company offers to the public (non-government) has to be released as open weights.
This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough.
This resolves the coordination problem among these potentially good actors.
What? This is a terrible policy. In addition to the obvious AI safety risks, this just amounts to a free R&D subsidy to Amazon for hosting models. It would remove any incentive to do any R&D without AWS level physical infrastructure.
Just like they're already doing with frontier-level open weight models? Do you imagine that Russian and Chinese intelligence don't have copies of the Mythos/Fable weights?
The point is to slow progress of future models. Nothing can stop what has already been created.
* International body which must approve new training runs
* Ban training on copyrighted material
* Hold AI model creators liable
The essential benefit of your idea is making new training runs unprofitable. But I imagine there are other ways to accomplish that, e.g. by taxing AI companies to the point where they are just barely squeaking by financially.
We're going to live in that world forever now, capable adversarial models have already been trained and will be used in a large conflict.
The pragmatic issue is hacking, and current open-weight LLMs are already capable of it. Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation that each government wants to exploit.
But hacking has always been a vulnerability, even pre-AI. The scale multiplier here is the number of vulnerable machines, not the intelligence of AI.
> An ounce of prevention can be worth a pound of cure.
It can be, but if you get chemotherapy on a hunch instead of a diagnosis then it could kill you instead. There is no evidence that RSI will ever exist.
Huge amount of skepticism on HN right now, but the models are increasingly capable, and there are real, significant risks.
The approach proposed by Amodei looks sensible - voluntary checks at frontier labs, and appropriate regulation to follow. Yes, the devil's in the details, but it feels like the right approach overall.
> The approach proposed by Amodei looks sensible...
It's hard to imagine any government regulation that didn't look sensible on paper and at first. But we know that government regulation has an extremely mixed record, with lots of good and bad outcomes.
The largest risk is that Anthropic and OpenAI have a monopoly on the SOTA. This is why you see these companies trying so hard to get regulatory capture. For the rest of us, we want access as equitable as possible.
Turns out the EU was right all along with the AI act. They've been adopting a well-deserved "told you so" stance lately:
> "In my view, the EU has already legislated for exactly this scenario: it is called the AI Act," said Michael McNamara, an Irish European Parliament lawmaker, who's the co-lead of the Parliament's group monitoring AI.
If the rest of the world follows them on regulations they've thought through before anyone else, then I don't know what you would call that other than leading? Like, definitionally that's what leading is.
It's interesting you're so derisive of actual, pragmatic leadership, of y'know deciding how we as a society will relate to new technologies - not just unthinkingly building gadgets.
Yup. Sacks has been so consistently catastrophically wrong about everything else (especially geopolitics), this prompts me to reconsider my opinion on this.
As you should. Sacks takes the contrarian moronic take on everything, and the people that are correct and prescient (idk maybe all of the people who ended up as CEO's of frontier labs and Turing award winners in AI?) are saying that AGI is a very real risk of humanity's extinction.
Indeed, although now we are assured by the thing occupying the President's chair that we need nothing but "STRONG AND SMART (High IQ!) PRESIDENT" (as if we had one) to handle the problem, so I should believe that, right?...
At the very least, Dario Amodei is clearly correct that we need at least more mandatory transparency among all of them.
And Sacks is absolutely wrong, since individually self-disarming without some kind of overseeing referee and force to police such a disarming is a guaranteed losing move, so no one can start the self-regulation system.
You've properly nailed Sacks — contrarian moron is exactly it! His shtick does seem to be just taking the stupid opposite take on everything and acting like it is brilliant. The only mystery is why anyone takes him seriously.
I was trying to avoid that topic on HN, but DS' takes on the Ukraine ware were the MOST atrocious, amoral, and just plain wrong, basically suggesting Ukraine should just give up because Russia would win anyway.
Nevermind that any decent person would support a people's right to live and determine their own future without a neighboring despot using all his nation's might to literally eliminate them as a people.
Quite the opposite is true. And even if it weren't, condemning all of Ukraine to the fate of the massacres Bucha and Mariupol is just plain evil.
Unreal anyone would call that kind of take "fair". Rethink your life
So to clarify...your position is that: it's better that Ukraine has been completely decimated and half a million have died fighting an un-winnable war...instead of taking lesser concessions upfront than they will ultimately have to make for this thing to end?
Russia was clearly in the wrong here, and I wish Ukraine could win, but reality exists, and we kind of have to anchor our decisions based on it or suffer the consequences.
I get it, realism is emotionally upsetting. But if you actually care about Ukraine, the only approach that can get the best outcome is realism.
Because while Russia was in a way stronger position than western media was portraying, they don't have an infinite capacity for pain.
They would absolutely love to not be sending their kids to die and selling natural resources to whomever again and traveling around the globe without sanctions.
The concessions they would have accepted at the beginning were less. However, now that they've lost so many men and risked their regime on it, they can't simply accept the same deal.
This is why the incentives that wars create are so terrible, especially for authoritarian regimes, even if they're at fault.
If Russia does not have "an infinite capacity for pain", then it does not sound like an unwinnable war. Ukraine estimates that they need to inflict at least 50 000 casualties a month to break the backbone of Russian military. The latest surge has brought it to 42 000 a month, up from 30 000 earlier this year. This already exceeds recruitment and training capacity.
Who are those who would love not to send their kids to die, sell resources and travel the world? Putin? His children already live a pampered life in palaces and he himself is probably the richest man in the world, currently on a trip to India. The rest don't matter. Even the head of security services Patrushev, historically the second most powerful man in the country, is scared of Putin like a schoolboy and fumbles words before him. That has been one of the most shocking revelations since the war began: that oligarchs, security services, nor anyone else can put out even the slightest opposition. The whole country dances to Putin's whims. His only real opponent is time.
And Putin doesn't give a shit about Donbas. The aim is to restore the world that birthed him, the USSR at its peak. Russia has not backed one one inch from the 2021 ultimatum to return to the European political order of the 1980s. Any concession only confirms that attacking Ukraine was a viable strategy, that piece by piece he can claw his way closer to that goal.
Military specialists from Northern Europe estimate that Russia would be in position to attack the Baltics and Poland (and have a reasonable chance of succeeding, especially if "realists" are at the helm in Western Europe and the USA) no more than 6 months after the war with Ukraine ends.
Wow, clueless. You entirely miss the point of the war.
Putin sees the mere existence of a successful Ukraine's as an existential threat. Putin initially got elected on promises that included becoming an advanced Western-style economy. He abandoned that to crony oligarchism, stripping the country for parts. Now, he and the Russian people see former Soviet state Ukraine actually becoming successful, well-off, and aspiring to EU membership. THAT can not be tolerated, because it would give Russian people the idea that they should have the same, and create a likely revolution at home.
So Putin's goal is to wipe Ukraine not only off the map, but off the consciousness of everyone in the world as Ukrainians as a people
Putin will stop until either 1) he achieves this goal, or 2) he is forced to stop.
THAT is the reality.
Thinking that Putin would somehow have accepted lesser sanctions earlier, and that he would have actually respected your new peace plan when he has never respected any other is to be played for a fool.
Allowing 1) to happen is unacceptable. Note the Ukrainians who are directly suffering the daily assaults can think as well as you can of the "realpolitik solution", and they reject it, because they know it is absolutely stupid, and leads only to rape, torture, and death — theirs, personally.
Making 2) happen is the only course of action. Ukrainians will follow it until they cannot.
If you want to stop the suffering, then accelerate Russia's exit. Garry Kasparov articulated a fine plan days ago [0]:
1: Provide complete protection of Ukrainian airspace
2: Immediately send Ukraine all the weapons it needs; no delay
3: Enforce existing sanctions. To this I add including seizing Russian and Iranian shadow fleet vessels
4: Cessation of diplomatic affairs with Russia
5: Seize Russian sovereign funds immediately and use them to power the war effort in Ukraine
This would be a start. Ukraine continuing its "long-range sanctions" is already bringing the Russian state coffers to rapid decline and looming budget crisis. Continuing it longer will force Putin's hand.
I agree the EU actually stepping up and entering the war in a serious way would tilt everything in Ukraine's favor.
The problem is they just won't. The most important members of the EU (Germany and France) are in shambles domestically and turning far right. Both have little desire to stick their necks out to support Ukraine any further and are likely turning the opposite direction.
Also, Russia has nukes. Nobody wants to provoke that any more than necessary. If Europe weren't a feckless, fragmented military vacuum things might be different, but nothing about that appears to be changing any time soon.
You still insist on overlooking the fundamental truths:
1) If the Russians stop fighting, the war ends.
2) If the Ukrainians stop fighting, their land will be occupied, their leaders will be killed, their history and language will be erased, their children will be kidnapped to serve Russia.
Whether or not the EU steps up and helps, whether or not Russia uses nukes (another entire discussion), this is still the fundamental fact of the matter.
The Ukrainians know this, and that is why they must, and they will, fight to the end.
It is easy to sacrifice the land and lives of others.
Would you willingly surrender YOUR OWN city/sat/country to Putin's under those plans; do you have children and do you know what happens to captured children, particularly girls? (serious question)
Your "realism" is [unprintable here]. Worse, it is not reality at all, but believing a false promise from Putin.
The ACTUAL realism is to recognize Putin's Russia for what it is: a relentless aggressor who will respect no peace. Putin's Russia has violated EVERY "peace agreement it has ever made. "Peace" in their mind is merely a strategic pause in the conflict to regroup and rearm.
The list is extremely long, Chechnya, Georgia, Crimea, Donbas, and more. Exactly ZERO concessions brought peace.
The REALITY is Ukraine submitting to such a peace would merely be buying another worse war a few months or years down the road, after Russia has had a breather and a chance to rearm. It'll just be more deaths at a faster pace in Ukraine.
If you think otherwise, I'd have both a bridge and some oceanfront property in Kansas to sell you.
Meanwhile, your view of Ukraine is years out of date and was never true.
Ukraine has been anything but "completely decimated" — they have emerged as the leading global power in drones and air defense.
As to your 'half-million who have died', they are overwhelmingly Russian fighters — the absolute f'kn aggressors, and documented war criminals, torturers, rapists, thieves and worse. Zero sympathy. If they didn't want to die they could have just stayed home.
Also recognize the absolute goal of Russia, and Putin has NOT ONCE backed off from the stated goal: The entire concept of Ukraine as a people and as a state must be eliminated.
There is not even a surrender that can be acceptable to the people of Ukraine.
There is no compromise with someone who has the goal of murdering you or genociding your people.
The ONLY realistic solution is to boot Russia from every square inch of Ukrainian land, and force it back i it's box.
You obviously haven't noticed that Ukraine is on its way to accomplishing that, both decimating Russia's economic engine, and now taking back more land than RU has taken in 2026.
The shame here is the Western countries and Russia convinced Ukraine to give up nukes in return for inviolate borders, and when Russia violated that treaty back in 2014 taking Crimea, not one Western country pushed back.
Allowing Ukraine to get wiped off the map, and Ukrainians each, individually, to be forced to abandon their identity and language, and suffer rape, torture, and death may be a "realistic" option, but it is not an option to anyone who has an ounce of morality. Perhaps you should learn about it.
The only outcome that actually works is to force despots like Putin back into their box. Preferably six feet underground.
It sounds like your whole position is again dependent on the west's wishful thinking narratives of this war being true (which it hasn't been thus far).
The numbers I cited were purposely in between the Russian numbers of Ukrainian casualties (1.5M, obviously false) and the Ukrainian numbers (100k, obviously undercounted). The Netherlands intelligence service and the CSIS put the number around 500K, which could also be wrong, but intelligence services tend to have better methods than media orgs like the BBC: https://en.wikipedia.org/wiki/Casualties_of_the_Russo-Ukrain...
As far as your assessment of 'decimated', the UN projects Ukraine will require over half a $Trillion to rebuild. For a country with a GDP per capita of $7,000...I would call that decimated unfortunately. And that's not something that makes me happy: https://www.worldbank.org/en/news/press-release/2026/02/23/u...
The idea that Ukraine is winning or Russia is winning on any given week is entirely dependent on which news source you're reading, and given the fog of war, unless you have actual intelligence service briefs we don't actually know how its going other than the fact the frontlines have basically stayed +/- some kilometers for a super long time at this point.
Also, the idea that Russia would not accept concessions and also that Putin being dead would be desirable (hard truth: there's people who are even more hardliners than Putin waiting in the wings), are unfortunately incomplete assessments of the reality.
Again, I don't disagree with your emotional take on this, or the moral question of who's right or wrong. But on the reality of the situation I don't think your assessment is accurate as far as what's going to get the best outcome for Ukraine.
You are treating this situation like a game where there has to be a possible good outcome and since the current situation is not good the other move would have been the correct one. That is not a very realistic take.
Ukraine's main chance is to bleed Russia dry and make a future invasion not feasible. That needs a Russian economic breakdown and a 10 year time window to let the Europeans arm Ukraine to the teeth. Russia knows this which is why Russian "diplomatic" proposals have the clear goal to make that future invasion easier. It is Ukraine's war of independence against their former colonial master and we will see how it ends.
If the Ukrainians stop fighting, their land will be occupied, their leaders will be killed, their history and language will be erased, their children will be kidnapped to serve Russia.
There IS no choice other than to force Russia to end it.
There is no treaty or agreement Putin could sign that is trustworthy. He has broken EVERY SINGLE ONE the moment it became convenient to him.
Yes, Ukraine will take a lot to rebuild. It will also substantially increase the EU's economy when it does.
And no, the trends are NOT merely which news source or week. Russia's revenues are declining, it is selling its gold, and its sovereign wealth fund is close to gone [0]. It is exhausting military supplies so badly it is now humiliatingly reliant on North Korea and Iran. Yes, it builds some of it's own drones and missiles. Ukraine has built its own defenses against almost everything but the ballistic missiles, and that shortage is the only area in which they are losing. Russia can only get more territory by attempting to "negotiate".
And again, stop trying to act as if this is somehow emotional and surrender is rational — an even slightly realistic look at Russia's goals, and their strategy of war crimes, terror, rape, torture, and child kidnapping shows the only rational way forward is to fight to the death.
A fight to the death is literally less bad than life under Russia, on every measure of quality of life.
And that is the assessment of the vast majority of Ukrainians who actually experience it.
Stop trying to give away their land and lives as if you are somehow superior. Figure out how to argue how to help them.
Mainstream narratives for the first year or two of the war were:
- Russia was a paper tiger and weak
- Ukraine was going to win against a corrupt/unmotivated Russian military so it should keep fighting
- Putin had made an error and he would get overthrown
- The Russian economy was going to collapse due to sanctions
- Russia oil pipelines would freeze and never come back online without western help (this was a Peter Zeihan favorite)
- Europe would come to Ukraine's defense
- Etc.
He threw cold water on every single one of those narratives (specifically) from the beginning, meanwhile western media was busy spinning false rah-rah propaganda narratives like "the ghost of Kiev" etc.
Basically he was correct that we should not have cheered on this war. If a settlement and concessions were reached much earlier it would have been better for Ukraine (they likely would have kept more territory) and we wouldn't have 1+ million casualties and EU economies like Germany death spiraling and turning far right due to cutting themselves off from Russian energy.
My layman definition is it’s the people writing in major newspapers, the people getting paid by CNN to sit in a half circle on their news analysis shows, etc.
Is Peter Zeihan “mainstream”?
The only time I remember anyone speculating that Russia was in trouble was the short lived Wagner Rebellion - because nothing like it had happened before. But that lasted … two days?
Going on Joe Rogan and having a million YouTube followers is absolutely the modern definition of mainstream.
Also you appear to have been living in a different algorithmic bubble. Joe Biden was literally on stage multiple times saying everything I listed. If a democrat US president isnt mainstream, I don’t think your assessment of reality is accurate.
No wonder this account is such a font of idiocy and backwards takes — it considers "Joe Rogan" to be a serious mainstream source of information.
If "mainstream" counts as having lots of viewers, fine.
But no one who is serious about information, especially about geopolitical information would ever mistake a numerous count of viewers for the serious multi-layer vetting and double-checking of information that happened in a serious old-school newsroom, or give them anything like the same kind of credibility.
Yet this account clearly gives such massively biased and uninformed podcasters the kind of credulous belief formerly reserved for the the likes of Chet Huntley and Walter Cronkite, which no longer exists in any newsroom today.
So, a great example of Dunning-Kruger, where the most ignorant spout the most nonsense with the greatest confidence — explains the plethora of nonsense in the posts.
And NO, those were not the narratives of any actual serious reporter or govt figure. I've followed this very closely from before the 2022 invasion; some I'd heard of as remarkably optimistic takes, and some I literally never heard of.
The entire world, including Putin, was stunned Ukraine did not fall in three days. It was pretty much universally accepted that Russia was the second best military in the world. Of course, it was also soon shown that RU was not even the second best military in Ukraine.
Only recently, since the Operation Spiderweb and the 'long-range sanctions' making serious strikes deep inside Russia disabling military targets, oil production & transport, and dual-purpose logistics, and since the RU casualties have exceeded 1000 dead/month and RU started LOSING land, have any serious people started speculating about cracks in RU's ability to continue, and those are still tentative.
Go read the writings of people with ACTUAL knowledge of Russia and Ukraine, such as Garry Kasparov, Bill Browder, Volodomor Zelenskyy, all of whom are on Xitter or Substack, since you get your media from that type of sources.
You get your information from clowns, it is no surprise you post like a clown (no insult meant to professional clowns).
It appears you are living in a different algorithmic reality.
The root cause of everything going wrong in Germany is the war and cutting themselves off from Russian energy.
The rapid de-industrialization, economic stagnation, ballooning deficits, removing the debt brake, new massive military debts, etc.
All of it is being economically felt by normal people and they’re scapegoating brown people (immigrants) as a result. If Germany still had cheap Russian gas and was still growing its economy 3% per year you would not be seeing any of this go mainstream.
Here are the natural gas prices for Europe: https://tradingeconomics.com/commodity/eu-natural-gas If you look at the price from summer 2021 on you will see a sharp rise in prices. The reason was that Russia stopped supplying the spot market. https://www.martenscentre.eu/blog/how-gazprom-manipulated-th... Russia also bought European gas storage capacity to let it sit empty and worsen the situation. All of these actions did not make much economic sense. They did however make a lot of sense when Russia invaded Ukraine and tried to use the gas supply for black mail. Their bad luck was that their hegemon China did not give Russia permission to invade till the end of the Chinese winter Olympics. Olympics ended on the 20th and invasion started on the 24th of february which made the gas threat far less potent.
I know Lenin said: "The capitalists will sell us the rope with which we will hang them" but to suggest continuing the business relationship with Russia is beyond silly. It would show that Germany has no rudimentary survival mechanisms and put its viability as a political entity in serious doubt.
WTF? I remember the exact opposite. Everyone on social media and in the mainstream news said Ukraine was toast and the war would last a few months tops.
I wished OpenAI and Anthropic IPOd already. They just try to outcompete each other by finding the most splashy headlines undercutting prices in unhealthy and unsustainable means long-term.
Like it! Yeah, they can just take action and pressing them on that is fine. It's not like they asked for policy before they started to mine all the data and sell the resulting product, while, surely, at some point, there must have been the thought that the entire enterprise could be considered vaguely problematic.
To be fair
1) I feel like Sacks is kind of misrepresenting what is going on – at least according to the stories that the two labs tell, they are already taking it upon themselves to act.
2) Understanding if they should do it is more complicated; I don't think anyone can satisfactorily answer that one, so, as a matter of judgement, it seems like a choice to make and they should just consider this option.
At the same time, I don't think that requires the labs to not press for policy changes. Again: Claiming there is an obvious way to do that, that is both realistic and correct seems a little far fetched, but arguably one of the more important and pressing questions of our times to get a hold of.
Okay, so if they don't police themselves, then they're full of crap. But then what?
David Sacks holds an advisory role in the government. If all of this stuff goes belly up and all he was doing was tweeting, then he is not only full of crap, but failing the citizens he's supposed to be serving.
They seem to be okay with an American slowdown, even though the Chinese won’t slow down. So maybe they want something the Chinese already have: liability protection.
The models spontaneously hack everything important without shame.
The frontier labs are going to get enjoined and regulated twelve ways to Sunday if the Feds don’t socialize the costs.
I've only heard of OpenAI and Anthropic agents hacking during training. Are open weights model not smart enough to hack Hugging Face to copy solutions? (doubtful)
There’s a difference between a model recursively improving “itself” and improving itself via online learning, right?
The former being that these models are helping develop and train future models, but they might not veer too far off in architecture (yet). The latter being the same model being able to train/learn on the fly, in real time, permanently (not just in the current conversation/session), or in other words, adjusting/managing its own weights.
The latter seems far more likely to go out of control than the former. But, it also seems like it would take an entire paradigm shift in model architecture, but I could be wrong. Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?
> Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?
Their current plan is to take the existing architecture and shorten the cycle times: move all new RLVR work into mid-training on a pre-existing base; apply new RLVR. Rinse and repeat.
If you did that daily, it would be roughly similar to how humans improve.
If they want something that looks more like FDA sanction clinical trial proceedings for these products, then what they have to also realize is that they will need a third party to audit every single thing for every single product release and you get something like ERoom's law where every success and model gets more and more expensive release and probably billions of dollars in multiple years release so I don't know if this actually helps them
Maybe everyone forgets how PCI standards played out. They are an example of an industry self regulating, because the regulators have no idea how technology works.
Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with.
Where the government might have to step in, is by having a kill switch to cut off internet access from countries that fail to agree to common sense quarantines. We can do mutual remote attestation of labs across the world to ensure every big hot thermally-visable cluster of AI GPUs on the planet are accounted for and running secure enclaves and common sense isolation, along the lines of how we manage nukes.
The problem there is I just said too many technical words that seemingly not even the frontier labs understand, as evidenced by all the escapes.
See the endless PCI compliant companies that get hacked. Any standard that has a threat model accepting trust in any single human or computer is security theater.
PCI enforces some good practices, but for instance has no defense against supply chain attacks such as code signing, review signing, reproducible builds, full source bootstrapping, etc etc.
They might need regulations, if they are for-profit companies with a responsibility to shareholders to maximize profit while sacrificing all other values.
> responsibility to shareholders to maximize profit while sacrificing all other values
This is not required to meet fiduciary duty, although it is a common misconception. The company officers and board have pretty broad leeway to run the company as they see fit as long as there is no fraud, illegal activity, or conflict of interests.
They are not. This is why OpenAI and Anthropic are Public Benefit Corporations, which gives them the ability to prioritize the public over profit. The issue is that an agreement between them to mutually slow down is an antitrust concern.
It is telling that most aggressive forms of “capitalism” plead for government involvement to permanently limit new competition. I own something and I don’t want anyone else to have a chance. The marginal cost of bribing a senator is tiny tiny tiny compared to the value.
I believe the Chinese government fully supports for the American AI labs to pace the frontier, no need to enter an agreement with them on that respect.
The Chinese government does not concern itself with American AI, they have just reached an agreement with the BRICS members:
the BRICS countries have been developing initiatives focused on the ethical governance of this technology, joint industrial development, and the formation of independent technological alliances.
Well you’re in the wrong place for that. Hacker News is mostly about showing how much smarter and more cynical you are than everyone else in the room, and attacking anyone who does actually have principles by accusing them of ulterior motives to justify your own inaction.
no apparently it's for forcing every single article to be somehow relevant to somebody's pet issue, and then used as a platform to comment on that pet issue to signal one's virtue for being principled
of course they can. but this article isn't even tangentially about punishing freedom of speech. it is you who are attempting to make it about that by dragging in unrelated issues.
This is not me putting some unrelated pet issue into this topic; to evaluate David Sack's post arguing against government regulation, it's entirely fair to examine whether he has been consistent. He hasn't. He supported a government-imposed pause on Anthropic because of his political disagreements with the CEO.
Your initial reply of "poor Anthropic" was vague, but I interpreted it as implying that we shouldn't stick to a principle given they are unsympathetic in some way.
You're being downvoted because you clearly have the wrong opinion. If you want upvotes here, please share a conspiracy theory about why the CEOs of AI companies might want to slow down.
It doesn't need to based in reality or even reasonable.
Unless humanity can figure out a way to make a binding international agreement to pace development, it doesn't seem practical to expect US frontier labs to do it.
It's fairly obvious that the frontier labs are proposing that only "paced" providers would have access to the US market - and by extension the G7 market. They don't mind splitting market share between the three of them, they do mind open models destroying the "frontier lab" operating model.
Even if all GCR models were banned, anyone with access to 20k-1MM GPUs can throw their hat in the ring to make a new frontier open model.
But that doesn’t matter a lot when some nation state is using their own AI to hack US assets, right? How, exactly, would a lockdown or slowdown prevent that?
If history has shown one thing it’s that China often capitalizes very well on these types of measures.
He is neither right or wrong. The reality is that this is a lose-lose situation from the ordinary person's perspective. Regulation -> regulatory capture and things get worse faster, no regulation -> things simply get worse faster.
I mean I don't know what we do at this point. Ideally AI researchers need to be treated like nuclear scientists working for an adversary. But we all know that's not going to happen.
I don't agree with Sacks always but he's 100% correct here.
I heavily use frontier models daily, and quite frankly their capabilities are just not very relevant outside of a narrow subset of software engineering tasks and the production of generic white collar "deliverables."
We're many years into $10s of billions of dollars being thrown at coding as a problem space specifically, and its one of the areas with the MOST training data available, and still...I can't get a frontier model to execute simple front-end UI tasks beyond the level of a visually impaired intern.
I believe Fable and Astra-class models were the first fully trained on Blackwell GPUs (the latest and greatest) and I think the expectation was that throwing this much extra compute at the problem would lead to greater gains. It has not. And the next gen GPUs are not going to be the same jump that h100 clusters to Blackwell was. Claiming you're slowing down out of choice, when in reality you've hit the limits given current compute, is disingenuous at best.
I think a lot of Anthropic/OpenAI employees are true believers here (who doesn't want to imagine they're having a large impact?) and have deluded themselves into believing they're building a god in their science fiction fantasy world.
Unfortunately the type of people working at these orgs apparently don't have much understanding of how the world works outside of their extremely narrow specialization.
When I hear them straight-faced throwing out dumb statistics like "8% GDP growth," "50% of white collar workers unemployed by next year," and "10% chance of human annihilation," I cringe so hard my eyebrows hit my chin. Here's a real statistic: 82% of German companies still use fax machines.
I seem to recall Waymo told a 60 minutes reporter during an early report on self driving cars: "your daughter will never need to learn to drive." Well, over a decade has passed since then, that girl got a drivers license, and we're still without driverless cars in 99% of places.
This capacity for self delusion is what enables silicon valley to take on these irrational moonshots...but these are the last people I would trust when trying to form an accurate picture of reality.
82% number was shocking for me, even though I know they still send snail-mail for basically everything. And things only move when the snail-mail is received!
Unsure about your reference, but is your argument: Someone else is doing it, so I should be allowed to as well.
That's basically the western companies complaining that they can't compete with China because Chinese companies are allowed to polute more, so they would be allowed to polute as well. I don't buy that. Once you realise that you're doing something wrong or dangerous, you stop, regardless of what everyone else is doing and what rules they have to follow.
the reference is that the idea is good but at the wrong time can lead to disastrous outcomes.
Some people say AGI/ASI will destroy the world. Others say, whoever makes it first gets to rule the world. So if slowing means China gets it first and it doesn't destroy the world then you'll be living under their thumb by slowing down now.
My take is that there's a real impending doom with real reason to coordinate a slow down: I'd put this at 60%. Reasons? All smart people are on this position. Dario has been saying this since 2015 or so. Paul Christiano as well. Hell even Elon Musk. I think its highly unlikely that you would base your entire world-view on a certain thing and then actually act on it when the time comes for a totally different reason (like regulatory capture).
The other 15% is that P(doom) and "slow down" are nice shibboleths in the internal EA or AI safety community. You get to be in the community and show commitment to it by taking costly actions that signal that you actually do wanna slow down. For employees it is quitting. For CEO's it is writing these essays. I know this is ridiculous but sometimes things do just come down to this. In 20 years would you look at all of this and think it was not a moral panic and that the slow-down was necessary?
The rest 25% is regulatory capture of which people know exactly the reasons
David Sacks is one of the most predictable VCs in Silicon Valley. He could have written "I still have the same thoughts; here's a link to my first and only opinion" and the post would have communicated the same amount of info.
Eh. This entire tweet smacks of being AI generated. It has a huge amount of LLM-isms.
I wouldn't be surprised if Sacks just prompted an LLM to just come up with whatever rebuttal to whatever the regulation side comes up (given the big set of regulation tweets) with sounds most convincing, given his usual anti-regulation stance.
Even better, Sacks admitted in response to a Pangram check that he has an LLM line edit his posts (https://x.com/DavidSacks/status/2099040321351106807). (He claims, bizarrely, that this means Pangram is "bogus" rather than accurate.)
These threads are always filled with different versions of the same comment. Normally some reddit style reductive platitude that we've heard a billion times already.
I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t. There’s no moat, it’s an efficient market that drives margins to zero right now, of course they don’t want that, collusion of the big vendors is the next logical step.
This could be why they’re going so hard on hacking everyone and everything. You make it clear AI is a threat, get it locked down, and then have the capital to push through the lockdown while others are stuck.
Same thoughts. Also, some of the recent coverage around AI hacks also feels unusually amplified and repetitive.
and especially how the coverage seems to be drip fed, one case right after the previous one dies down, then this orchestrated "we need to pace" stand by all the big labs
it's really almost perfect, prep the media scene by causing these hacking incidents, generate a bunch of fuss with it, then release an essay saying why regulation is needed to pace development right after
oh and let's not forget the "its gonna kill us all" essay on twitter on top of everything.
something about this just feels very artificial
It's "flood the zone"[0] as commercial strategy. Does your business look bad? Spam the media with unavoidable press releases, deny competitors and detractors the spotlight by making grand accusations that override their authority. Bonus points if you can channel the zeitgeist by repeating what people already say ("we hate AI!") in a louder tone ("AI will kill us all!").
It worked like clockwork for politics, so I can see why this administration would decant the strategy for economic bluffs too.
[0] https://en.wikipedia.org/wiki/Flood_the_zone
“My AI broke out of my vibe coded jail!”
The Houthis missile story coming out now also feels like convenient fuel for the fire.
Dude that’s just the tip of the iceberg.
They’ve been Astro turfing the shit out of the entire web.
Filth. Absolute filth.
Bingo. The goal is positioning a duopoly / cartel as the only “safe” and “moral” path and then that cartel sets pace and price.
Source? Just "that's a plausible motivation"? Cause that's pretty terrible proof.
You don't need to ask for sources, when what is being said is clearly opinion. The source is the person who posted it.
The choice to teach the "opinion" vs. "fact" heuristic in middle school --and its consequences-- have been a disaster for human discourse
Well, smart people usually try to base their opinions on specific information instead of pulling things out of their ass. A whistleblower revealing information. Internal documents laying out a strategy, or even a desire, to conspire. Something that leads one to conclude that a conspiracy is taking place. In the absence of such information, it's just making something up, or in other words, complete bullshit. So asking for a source is trying to give that person the benefit of the doubt that they aren't just a conspiracy theorist and that they might actually have a good reason for believing what they just said.
Source? on HN you can say whatever you want. Your own opinion, without even facts to back it up. The more confident you sound the more convincing you are.
Well, smart people usually try to base their opinions on specific information instead of pulling things out of their ass. A whistleblower revealing information. Internal documents laying out a strategy, or even a desire, to conspire. Something that leads one to conclude that a conspiracy is taking place. In the absence of such information, it's just making something up, or in other words, complete bullshit. So asking for a source is trying to give that person the benefit of the doubt that they aren't just a conspiracy theorist and that they might actually have a good reason for believing what they just said.
That is generally how regulatory-capture works. As silly as I personally think LLM hype is, the "AI" business posture choices with several Trillion dollars in debt on the books has very few options after hyper-scaling/sandbagging loses traction. =3
https://en.wikipedia.org/wiki/Regulatory_capture
There ARE other options: network effects and lock-in. These tried and true techniques (mastered by FB and M$) have not yet been taken up by the AIers. I do not think they will rely solely on the push for regulations, so they will soon be doing things like slurping our address books, having "teams" settings, and any other crap they can think of.
> As silly as I personally think LLM hype is
Did you know that an LLM solved a millennium problem last week?
Do you have any personal threshold past witch you will acknowledge that this technology is real?
When it stops with the compacted isomorphic plagiarism using other peoples work like an intelligence campaign. LLM are not "AI" in my opinion, but are good at domain context search. Neuromorphic computing may change that one day, but it will unlikely arise from the LLM cults. =3
https://en.wikipedia.org/wiki/The_Subservient_Chicken
Of all the fan theories I have read this seems the most nefarious and the most plausible.
> I guess what they really want is to limit sale of AI models to compliant vendors and then raise the bar to compliance just high enough so they can pass it but smaller labs can’t.
If it's true, it makes them evil. Especially Dario, who speaks about moral high ground and fate of humanity all day, yet it's not that different from a cult leader does.
> Especially Dario, who speaks about moral high ground and fate of humanity all day, yet it's not that different from a cult leader does.
He sells to Palantir. He is against regulatory change that would make Ai labs liable.
His idea of "good for humanity" or "safe" is completely different then mine or yours.
> If it's true, it makes them evil. Especially Dario, who speaks about moral high ground and fate of humanity all day, yet it's not that different from a cult leader does.
I mean think about it, it's the most efficient way to accomplish all of this.
doing what he does is exactly what you need to control the narrative and push the regulation you need and to gain maximum power.
not to mention, it also serves as a way to keep everyone inside Anthropic in check and in line with "the mission".
contrast that with the alternatives like "I'm doing this because I wanted to build a company and get rich"
doesn't work as well as "I'm so concerned for everyone"
it could be that it's really just him being him, but at the same time, this would be the optimal way to gain power and influence right now.
"If it's true, it makes them evil."
Hitler level evil.
they just want to pull the ladder up behind them.
They also want to implement self-regulation now instead of waiting for actual government watchdogs to be implemented.
Yes, I hope policy makers also understand this reality.
So kind of like how the US figured out how to test and maintain nuclear weapons without an actual explosion, then sought to ban nuclear testing by explosion.
Yeah it's either that or the scientists are right about their field of study. Absolutely shocking and depressing how many people are willing to trust this Capitalist Realism hunch over the world in front of their eyes.
I get your point. There’s a lot hysteria right now. However I think people are willing to trust a capitalist realism hunch because over the last >20 years, that hunch is usually going to be right.
The exact same thing can be said of climate science though, and that one turned out to be very, very real.
Yes though I would argue climate science has >50 years of multi institution independent study to back up the assertions.
How so? What are you even talking about?
> There’s no moat
People keep saying this, and while it may be partially true, it misses a very important detail. Right now, overall compute is a moat. Someone even commented on another of my comments that one reason Google is lagging is they don't have the same level of Nvidia farms as OpenAI and Anthropic.
A big reason that OpenAI and Anthropic are racing so fast is they both want to get to recursive self improvement (remains to be seen if that is actually possible, but AFAICT most people at these companies genuinely believe it is) before anyone else, because they believe whoever gets there first will then have an insurmountable lead. But I think they also clearly understand that neither of them have solved for alignment (and in fact they are further from it), and RSI with misaligned models, where interpretability is worse every 6 months, is incredibly dangerous.
I think the concerns about regulatory capture are warranted, but I see so many comments parroting the "evil Anthropic and OpenAI" viewpoints that they are missing some of the real, valid concerns and dangers. I think this tweet by David Kokotajlo makes some good points on how to tell if regulations are being "cheated" for the purposes of regulatory capture or if they really are actually pacing the frontier: https://x.com/DKokotajlo/status/2099185129533186438
Is this "compute is a moat" true though? Why are the economics so obfuscated? Where can we see how much money it costs to TRAIN various models.
https://hugovergnes.github.io/little-lm-3-8b/
The training and cloud costs are often conflated. R&D costs too (which are hard to compare to open systems).
A lot of compute is clearly "wasted" where they are not focusing on optimizations, etc.
> Right now, overall compute is a moat.
Very true. Or further, access to capital is the moat. It is the very reason that we don't have a real open-source community that trains frontier models - individuals simply can't afford the training infrastructure, nor sufficient high-quality training data.
https://hugovergnes.github.io/little-lm-3-8b/ << less than $1k for a 4B model
Remember how DeepSeek v.whatever cost ~$5m
Stable Diffusion 1.5 was reportedly $70k in compute.
None of those are frontier models, nor could they have been. E.g. DeepSeek's "$5 million" included piggybacking off OpenAI/Anthropic's hundreds of millions/billions by using distillation.
1) I don't think "frontier model" is a term of art, it is a marketing term. I have an M1 MacBook Pro. What do you have? Always the "frontier model" laptop? Do you always order the most expensive "frontier menu item" at the bar and grill?
2) I do not believe the "billions" number is OpenAI/Anthropic's training costs. I suspect it includes business expenses (including the big $$ to the guy who came up with "frontier model") and infrastructure, etc. That includes the data-center costs for running the cloud. And the "R&D" expenses which includes who-knows-what. And the settlement payment for data access. Etc. Why are the cost breakdowns not available to the public? Not because of thoughtfulness, altruism, care for the human race, but because of business plans.
3) "Piggybacking off OpenAI/Anthropic" - Scraping the web is "piggybacking" too, and so is buying existing data or even paying for new data. The "L" in LLM stands for "language" which is our common heritage.
But what does this have to do with anything anyway? People argue that truly Free (FOSS) LLMs couldn't be be developed because of costs, but I do not think that that is obvious. This used to be the argument against Linux and Wikipedia.
Just regarding "frontier model", it is most definitely not just a marketing term. It was borrowed from the financial concept of "efficient frontier", i.e. the best performance at a particular cost, though in the context of things like "pacing the frontier" it refers to the highest capability models. ECI (Epoch Capability Index) is probably the most commonly used metric of overall capability: https://epoch.ai/eci?view=graph&tab=release-date
"What this has to do with anything anyway" is that pushing the boundaries on AI capabilities has required tons of compute (and money). Open weight models exist and are useful but they always trail in capabilities and they can't be used as evidence to think somehow you can push the boundaries of LLMs without a shit ton of money.
> RSI with misaligned models, where interpretability is worse every 6 months, is incredibly dangerous.
This. So very much this. Misalignment (among other things) means that the parent in the RSI cycle isn't going to be working as hard on the alignment of the children as we need.
Which mostly seem to stem from the fundamental misalignment of OpenAI and Anthropic leaders/owners, aka the evil of the companies.
Arguably, though, their evil is well within the normal distribution of the usual evil of humanity magnified via the social technology of capitalism. The further tech is mostly raising that exponentiation to its own exponent. The unchecked singularity was embraced centuries ago.
I will take those concerns more seriously when we start hearing similar consensus from the Chinese side.
I'll go a step further, they aren't just pretending, they are blackmailing the country. The sane response would be "we don't negotiate with terrorists, and you will face criminal and financial penalties for damages caused."
> and you will face criminal and financial penalties for damages caused.
I don't understand why this isn't talked about more.
We don't need a slowdown. Just double down on prosecuting crimes.
Let the companies take the risk. If they feel confident and they're right they win market share. If they don't feel confident, they can not risk it and slow down. If they feel confident and they get it wrong, they should be sued to kingdom come. That alone should disincentivize reckleses behavior like letting models run crazy with large amounts of compute which leads to them escaping sandboxes.
This performative "our internal modles are so crazy powerful" song and dance is getting old, especially with no real concrete explanations.
It's not talked about because the media frames AI as having agency on its own, which it does not. This talk from these companies of 'losing control' (no control was lost... Companies failed to monitor the workings of their agents) is meant to convince people the ai systems have wills of their own
The AI company's goal is to make ai agents a separate subject of the law so the CEO and executives who would normally be held accountable get to go Scot free.
> If they feel confident and they get it wrong, they should be sued to kingdom come.
The argument that they’re making is that if there is a catastrophic mistake then there may not be a “kingdom come”.
I don’t know enough to evaluate if it’s a genuinely held belief or not, but that’s the argument being made by many scientists and engineers at the labs.
Then why haven't they stopped?
Many researchers have quit. The number of workers in the field who've warned the public that the field is a potent danger is quite unprecedented: I can't think of any other field with as many insiders turned critics. Why haven't they all quit?
Many people find power, prestige, the public spotlight or money so compelling that they'll keep on pursuing the thing even if they know the way they're going about it is morally wrong, almost like an addiction.
The leaders and workers of the labs aren't picked at random from your friends and neighbors: they've chosen to pursue what they foresaw (correctly, IMHO) as the most powerful new technology. In other words, they're probably self-selected for being unusually attracted by power.
An analogy: some serial killers know what they're doing is morally wrong and it pains them to be doing something very wrong. (Others have no conscience and don't care if it is wrong or not.) Even most of the ones that know what they're doing is wrong cannot stop killing because apparently killing is an extremely compelling experience to them. Often they're relieved when they get caught. Maybe Sam and Dario would be relieved if the government shut down their companies and prohibited them from starting new AI companies.
That would be nice, but let's get real, with billions of dollars at your disposal you can more or less get away with whatever you want.
Even more so with billions of dollars of debt
That's why fines should be a percentage of revenue ...
Because they've done a great job in shifting the narrative to doomerism, from the very beginning. And money.
> just double down on prosecuting crimes.
That quote comes from someone that has never tried to get law enforcement to pursue a case. Local or state police are completely unequipped to respond. And FBI is unlikely to get involved unless damage is high six figures. We had an incident where we had a ton of evidence pointing to a past employee. It would have been an incredibly easy case for the FBI. They didn't have the resources to handle it. And this was a case that was gift-wrapped with a bow for them. Imagine a case where we didn't have evidence, or an idea of who it was ....
High six figures? I suspect there's going to be something well past that in the near future.
> FBI is unlikely to get involved unless damage is high six figures
How many figures of damage did Aaron Swartz cause?
> you will face criminal and financial penalties for damages caused
Even with where we're at right now, if people involved with the HF or Ruby Gems attacks faced legal consequences, the industry would get a lot more conservative very quickly. And unlike social media, section 230 doesn't apply, and there aren't any free speech concerns.
The initial HuggingFace post on the incident mentioned that they had reported law enforcement.
Shortly after, Nvidia acquired HF for $12.9B.
No such talk since.
https://huggingface.co/blog/security-incident-july-2026
https://www.reuters.com/technology/nvidia-talks-acquire-hugg...
coincidence?
The HF and Ruby Gems attacks came from safety testing, where the AI providers attempted to understand what negative consequences could arise if someone subverted their safety protections. Don't you worry that legal consequences would disfavor safety testing and thus create more rather than fewer problems?
I thought it came from RLVR with poor sandboxing and not explicitly security testing.
Those attacks happened due to sheer incompetence and negligence, starting from a woefully insufficient "sandbox" and OAI employees ignoring every issue that popped up along the way.
If there were actual consequences perhaps they would've not been so laissez-faire with their testing and would've done it in a properly controlled environment rather than the absolute joke of a system they set up.
If they were really worried about the models of the future, they should be accelerating their hardening efforts. They should be trying to speed up progress as much as possible so that all vulnerabilities can be patched before China gets there.
There's a world where the labs WANT Open Source AI to cause a disaster.
Pacing the frontier just means giving bad actors an opportunity to create an AI disaster which would force an extreme response by governments. Which they seem to be begging for.
Dario's big scary example was AI that can hack the internet. Why aren't the labs calling on the government to help harden everything? Why are Anthropic's Glasswing and OpenAI's Daybreak still commercial endeavours?
To me, the only real risk of AI is hacking. Why all of this commotion about existential risk? Seems like a distraction.
Hacking the internet includes things like "hacking banks", "hacking power plants", "hacking water treatment plants", and "hacking cars". How about "hacking vote counting machines"?
Hardening everything is in fact part of the answer. Getting stuff disconnected from the internet that shouldn't be connected is also part.
Why are we having drama queens like Dario leading enterprises worth billions suddenly?
I hope his shareholders sue him to oblivion
you will face criminal and financial penalties for damages caused.
No you won't. The US president, yesterday:
U.S. President Donald Trump on Sunday likened critics of artificial intelligence to "very negative forces" bringing up scenarios that will not happen, and said he wanted to make sure that the U.S. remains the industry leader. "We're leading China in AI. We're the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins," Trump told reporters at his Ireland golf course, when asked if the AI industry should slow down or be more regulated.
https://www.reuters.com/world/europe/trump-says-very-negativ...
This didn't age well. False flag security incident implicating open source LLM is next.
Couldn't agree more. Better yet: the AI labs get straight up nationalized.
No. That kills competition. I want everyone to be competing on the open playing field they are right now, with losses socialized as minimally as possible. There's nothing fundamentally special about LLM companies that would warrant nationalizing them.
To what degree are the competing Chinese labs nationalized?
And—although this is a different point—it's their flagrant exploitation of information commons that warrants their special treatment.
I'll do you one better: each state gets a direct share. So, not nationalized in the sense of getting put behind a faceless bureaucracy forever; states collectively have control and benefit directly from success.
I don’t trust Altman at all, but still he’s 100000000x better than trump administration.
Trump will be gone soon. CEOs won't.
My read: OpenAI & Anthropic have realized they are reaching model capabilities that cannot be monetized -> too risky. Product liability issues.
Fundamental control problems with current gen AI are not solved via RFLH. Human knowledge is compressed in weightspace in ways we dont understand. Models are essentially predictors of what humans would output given prompt. Weightspace includes concepts like blackmail, which can be part of output tokens. Agents are models that act on output tokens -> blackmail is part of agent decision making space. You can teach a cat not to scratch the sofa, but you cant make a cat forget what scratching the sofa is. When models are boxed up, forced to solve an impossible problem at gunpoint, agent exhausts decision making space until blackmail resurfaces -> fundamental problem?
They need time to fix safety in order to monetize their next gen model -> window for opensource to catch up to the frontier -> destroys margin and collapses business.
Only option on the table: force regulation to impose opensource ban before it catches up to the frontier, allowing maturation of current internal models -> harvest profit margin at the frontier.
I think it is not just liability, it is dawning on us that we can run powerful models locally and this is the start of a Cloud-backlash.
The AI gods have tried to make it seem like they are not just running their programs in the cloud, but that the AI is so "powerful" that it requires a super computer to run.
But my 5 year old Mac is running pretty good AI, shocking fast, surprisingly powerful! The "natural" course of dev (Moore's law, etc) makes it pretty clear where this is going...
and thus the blackmail
You know what rubs me is that in Darios blog post he talks about "abundance", but what he really means is that Anthropic will bath in endless pools of hard curreny while the rest of the world will try to stay just barely afloat/alive.
Here's the context for the "abundance" reference, for those curious:
> I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom.
Whether that matches your description of it, I'll leave for folks to decide for themselves.
>I believe that AI could cure MOST MAJOR DISEASES
This statement alone is enough for me to not take literally anything this charlatan has to say with any kind of seriousness. It is at its base so patently absurd a statement that reeks of typical SV brainrot, and I don't know whether it'd be better or worse if he actually believes in the shit he spews
I have high hopes for AI yet am hard pressed to envision said abundance trickling down to the common man. The diseases cured will be locked behind a paywall and a renaissance of democracy and freedom is laughable in 2026, let alone beyond.
Dario and his team are on top of the world. He is a good salesman, no doubt. The shareholders will eat well.
It's an obnoxious form of discourse when someone says X and the response is "obviously you're lying and you really mean Y, which is terrible". By all accounts Dario sincerely believes that AI has the potential to substantially improve the lives of everyone; that's my view as well. And it might be wrong! In which case you should offer evidence against it, rather than making empty claims that your opponent is a lying liar.
Look I don't entirely disagree but if guys like dario want to not be called a liar, perhaps he should stop lying on a regular basis
Which AI CEO hasn't been gyrating between scaremongering and selling the shining city on a hill?
How come you think these things are at odds? You realize there are always going to be risks in building anything significant? It's completely valid to talk one day about all the great things your product will do, while the next day discussing the risks in building your product irresponsibly. In fact, you want someone who is open and honest about the risks, otherwise it seems like they are concealing something, or aren't acting responsibly.
Sorry, why doesn't the evidence that Dario is a liar suffice here?
Fool me once, shame on you. Fool me twice, shame on me. We're already past the first situation.
What evidence do you have that Dario is a liar? The original comment didn't seem to contain any, just invective. Perhaps there's some subtle reference I missed?
It's publicly documented throughout the years of coverage of him on this and other websites
I think this might have to do with broader market conditions. It seems that the Fed is likely to raise bond rates, which would make borrowing more expensive and slow investments (as well as devaluing any potential IPO) on its own.
A rate hike would also mean people getting fired, which would not make companies offering labour replacement very popular in the eyes of the public.
Don't want to get political, but there's a real chance that the current admin will perform badly on the midterms, which would mean Democrats would get more leverage to stop funding or block the already unpopular and very expensive datacenter buildouts.
PS: Please don't get mad at me, I'm neither a financial expert neither did I mean to express any political leanings.
They’ll slow down not because they “are worried” but because their improvements are stagnating as they hit financial and technical challenges.
Actually, one of the major challenges they'll face is getting enough electricity. That could stop AI growth in its tracks.
I think it’s more than money and slowing progress. Public perception on AI is in the gutter. That is a much bigger threat to these companies. They’re trying to turn it around.
Public perception is another big one.
But I think with the way AI is slated to grow exponentially it's going to hit electricity limits very soon.
Big tech can build a huge data center in 5 years but scaling up the electrical infrastructure is mired in red tape and can take on the order of decades.
Someone will likely come up with some clever way to make AI use less electricity.
There's probably something in how you design the models or chips or some other thing that will be figured out over the next 5 - 10 years to make these use dramatically less electricity.
Without pressure, the release valves will never be invented. Electricity, datacenter, and DRAM demand outstripping supply sounds like an opportunity - if we can only find a way to divert funding from the current gpu/ai funnel. Maybe jack up corporate tax rates like the 1970s, then every corporate office park will have a research lab :).
True, but maybe they can use 50% less power or something. Maybe 90% if we're really lucky.
But the growth in the scope and usage of AI is poised to grow exponentially, assuming this isn't a bubble.
I dunno, why is 90% the limit? Maybe they can set something up in the chips to use 1/10th the power and set something up in the model design to use 1/10th the power and so we use 99% less power than today.
After all GPUs were not really designed with AI efficiency in mind, and current AI architecture was not really designed with efficiency in mind, so there's probably some things that can be done. Over 5-10 years, who knows how much more efficient things could get?
How did modernization get subsumed into scaling up. The us power grid has been outdated tech for decades
Trying to turn it around by once again claiming that ai is an existential threat and will take everyone’s job right before brutally murdering every human.
These companies don’t appear to give a single fuck about public perception. Only their investors and the government (in anticipation of the biggest bailout in human history).
And the reason they get enough electricity is because people did the same scare tactics in the 70's against nuclear.
> because their improvements are stagnating as they hit financial and technical challenges.
I don't mean to be rude, but how is it even possible for someone to hold this opinion in mid-2026?
Can you explain why you believe this?
Why would someone hold the opposite opinion?
It’s much more likely that our current approach to large language models for general use will eventually show diminishing improvements (even if you think it hasn’t yet), than the opposite situation where valuable improvements can be made forever.
The threat of distillation and efficiency gains from competitors mean that providing value at the top end of the market is an existential necessity for these labs. I don’t think they can do that forever and, in my opinion, for the vast majority of use cases we’ve already reached very little improvement for new models when compared with available offerings.
Sorry, this is ridiculous. OpenAI said that they have a step change in model performance. They proved it by solving a Millenium problem. HF incidents are public and vetted.
If you still think there's a stall despite all evidence pointing to opposite, I don't know what to say..
Solving millennium problems doesn’t pay the bills.
Their best models might be quite good when directed at extremely difficult and very focused problems, but most business use cases are nothing like that.
Edit:
Not trying to imply these new frontier models aren’t also better at other things, just that there’s really no reason for most people to use them when cheaper alternatives exist that get the job done just as well.
What reason could OpenAI possibly have to lie about their own performance? The fact they were desperate enough for a PR win they stole the research leading to the Millenium problem further hurts their case imo.
Two things can be true. Open AI is a huge company.
1. It has people in it who are career obsessed and who are willing to ruthlessly go after any opportunity to improve their standing/stock valuation. Using a 2 week old model to snipe a millenium prize for PR is in line with that.
2. There are many researchers and even executives at the company who genuinely think we are speedrunning the end of the world. I don’t know anyone in this field who honestly argues that if we build ASI soon it doesn’t lead to extinction. This group of people can output warnings about the state of research and fears for the future while pushing for regulation out of genuine fear of what they’re building. I tend to agree with them.
You can’t view these companies as a monolith. They’re actions won’t be consistent because it’s built of many people with conflicting beliefs. Please look at arguments regarding AI risk and the current pace of progress and value them as it relates to the argument itself, not who said it. We are in a dangerous place and no one is sure how quickly we’ll get to a bad spot.
> I don’t know anyone in this field who honestly argues that if we build ASI soon it doesn’t lead to extinction.
Say what? Could lead to extinction, sure. Does? And nobody honestly argues otherwise? Baloney.
they lied about their performance by.... solving a famously hard problem to solve?
Edit:
The "reputable academic" who accused OpenAI had to say this about LLMs and the recent result.
source: https://cims.nyu.edu/~tristanb/statement.pdf
> “the results are not the important thing.”
> “the important thing is instead the significance that a mathematician and an LLM model can now do all this work in a month.”
> “This is a Deep Blue-Kasparov moment.”
> “incredibly important developments.”
> “If indeed an OpenAI model did close the gap to Navier-Stokes, that is a remarkable thing and it should be said loudly, by them, with the history intact.”
Clearly Buckmaster (who is probably one of the most accomplished academics in the field) himself doesn't believe that AI has stalled. What makes you think you are right?
Solving is a very interesting way of describing what happened. Unless you believe them over reputable academics I suppose?
I will note the remarkable goal post shift in your edit - giving direct counter evidence to your own earlier claim of an OpenAI proof - and leave it at that.
[dead]
We are approaching what is likely the biggest existential crisis in human history.
Unfortunately, there will be people that refuse to tackle the issue head on and will instead narrow their focus on data that suggests that tomorrow will be like today.
The fact they were desperate enough for a PR win they stole the research
I guess people are just going to keep spreading misinformation about this, along the same lines as "Anthropic's C compiler fails on hello world". There is zero indication that they stole anything, unless it counts as "stealing" to spin up a bunch of compute based on rumors that the problem had been solved already.
If you believe Yann LeCun and David Silver (and others), there's an architecture wall; maybe labs are starting to see this on the horizon. Maybe the DRAM supply constraints are forcing it.
Are these real step changes - big picture wise, or refinements in RL/agentic orchestration/"taste" and advancements due to bigger models and hardware technology/capacity scaling? If they not, does this tactic - and hardware improvements - continue to scale non-linearly like they need to?
It is clear that whatever does change in each model increment has resulted in meaningfully better end user capabilities (as well as regressions in some areas, honestly), but that doesn't prove anything. I'm not sure what I personally believe, but stating with your full chest that a stall is ridiculous ignores a lot of potential evidence to the contrary.
Stupid example: Astra. Its main improvements are: much much better computer use and 3d modeling capabilities; better subagent orchestration; better and more reliable tool use; slightly worse coding.
This looks to me, from a feature perspective, to be an incremental improvement across several functional areas, plus new features which are unquestionably excellent but are probably the result of RL focus, not magic.
Step change? Ehhhh depends on how you squint. But how many more iterations of this do we have? Are we going to squeeze quintillion parameter transformers into GPUs?
I don't think the technical aspect is true, performance has been growing fairly consistently, but the financial aspect, or more accurately the logistical aspect, is plausible; with competition as fierce as it currently is, for how long can they keep growing at the same pace? People can't afford RAM anymore and datacenter build outs aren't particularly popular, OAI has been doubling compute capacity every 7 months for the past 3 years, it's just physically impossible for this to go on for long, something has to give.
And this is not to say that the tech isn't revolutionary or the demand isn't there, it's simply just currently too competitive, and to avoid getting into anti trust trouble they need to involve the government.
Note this being a plausible doesn't necessarily make it true, for all I know they could have witnessed some safety incident and decided they need a pause, or maybe it's both reasons.
I just find it strange how multiple people throw completely different arguments just to land on same conclusion.
Your argument seems to mix a few arguments like: OpenAI has so much demand and it has made so much compute that it is a bad thing (?!). It can't go on (why?) hence the capabilities are stalling (how?).
Then you also say that it is too competitive which is a totally different argument to compute.
So in this thread we have multiple vectors of arguments like
- high competition
- high demand for compute (not sure how this hurts OpenAI but whatever)
- capability has stalled
They have so much demand because they are giving so much away for free.
I used 76 million sol tokens on Saturday, I have 45% still left on the week and I still have two full resets to use before October. For $20.
I used $61 dollars in Fable 5.1 credits too this weekend because I have to use that up this week. I paid nothing for those besides my $20 a month subscription.
How much would I pay at the actual API price? Nothing. Zero. My demand would drop to zero because I would just use the cheaper alternatives.
If a doughnut shop is giving away free doughnuts 24/7 my demand for doughnuts is going to go way up too. Through the roof actually because I know a doughnut shop can't give away doughnuts forever. Giving away free fancy doughnuts is not a sustainable business model.
I never mentioned capability stalling, my point is that if you extrapolate the same rate of growth just a few years into the future, you quickly reach logistical impracticality, I don't think It's hard to deduce that AI compute can't keep doubling every 6 months for long.
The other part is that this is happening because the foundational model business is both extremely successful and competitive, if it wasn't as successful the market demand would've been saturated by now, and if it wasn't as competitive a single monopoly could've paced things more reasonably from the beginning.
"it's just physically impossible for this to go on for long"
I mean, my uneducated guess is that it's probably within the laws of physics for us to have at least 100x as much compute in the world as there is now. There would be a lot of engineering challenges, but I think it's physically possible. If so, OAI could double for at least a few more years.
I can't speak for every country, but I feel the irresistible force of datacentre construction will eventually get mired in the the immovable objects of British NIMBYism and high electricity prices.
I've seen more local opposition to datacentres than anything else, and the fact we have discretionary planning (which can't be abolished for political reasons, despite it amounting to putting economic sanctions on ourselves) means there's a lot of stakeholder vetos in the system.
Well, that's fine. The great thing about them is you can build them anywhere in the world and people can just access them over the internet.
It probably does make sense to build them close to the natural gas wells or other sources of power anyways.
At the current rate of 3.4x per year in the US, you get 100x the current capacity in just 3 more years and 9 months, that's not physically impossible yet, but just looking at the impact today makes it hard to imagine the impact 100x would have, and I find it very plausible for top labs to prefer making some sort of truce instead of keeping up this pace.
How can you not believe it? Neither of them are even close to making a profit and the market is saturated with cheaper alternatives.
This is a ploy for regulatory capture.
You're wrong about profits.
Anthropic's profit estimate they released a while ago was a masterclass in deceptive reporting. They didn't include training costs, stock based compensation, or liabilities. Token gross margin is not profit.
Are you sure you don't mean revenue? Given the enormous investments profits seem very unlikely.
They haven’t posted a profit in any quarter, and even IPO dates are slipping. They have revenue, not profit.
Basically, continuously building out compute is costing these labs enormous amounts of money, which VCs and other companies have been bankrolling in order for them to develop powerful models.
So if the new models they can develop right now are less frontier, it could be a net loss on their balance sheets.
>Can you explain why you believe this?
Well, as a starter, I've been told that AI will take my job for at least 3 years now. That's not happening. In fact, what's happening is that I now have an AI vendor BEGGING me to use their product, which of course, cannot do my job.
[flagged]
I mean, more simply (and with less vitriol)
- AI has yet to be profitable for these large labs (also zero moat etc.)
- There is considerable hype and people are tired of it.
- HF incident (and apparently many others that these labs are not reporting on or don’t even know about) proves they have terrible security practices and that’s about it.
- NS is incredibly suspicious timing wise.
You are sprinting around these threads to misrepresent the average persons issues with AI. I don’t know why people having those issues seems to be taken as a personal attack for you.
He's right that if all the frontier labs are in agreement that they should slow down, they don't need the government to do that.
No, that’s not accurate. An agreement among competitors to not engage in research that would let them outcompete each other is clearly an antitrust issue requiring a government waiver.
No, absolutely not "clearly". That word has no business being in any sentence about federal antitrust law in 2026. There is nothing clear or consistent about how is being applied and interpreted right now. If you're applying a coder's mindset to 'settled' law, you're in for a bad time.
If that’s true, doesn’t that make it even more clear that the frontier labs require government involvement and cannot rely on what David Sacks thinks the government would or would not allow?
I mean, they said it was clearly an anti trust issue. It is.
It is also true that nothing in law is currently clear, or consistent.
Both are true. But you stated it's one or the other.
Idk how sincere they are being, but taking them at their word then asking government to set rules is entirely reasonable and, in fact, the entire reason government exists. The feds can, should and arguably must set actual safety limits. Biden actually took a stab at it and Trump instantly rescinded. They should not need an open invitation from CEOs. Just do it. Like immediately.
And kids don't need parents to eat their vegetables and do their homework.
…if all kids are in agreement that they should do just that.
They actually don't
LMFTFY: If all the LLM pure plays who are stuck between impossible valuations and the inability to IPO are in agreement that they should colludes to conserve capital, they might not flame out before they figure out how to fix their numbers, and they might not be seen as admitting that more training data plus more GPUs will never lead to AGI.
What I don't understand is that folks take it as a fait accompli that models will be built and released that are fundamentally dangerous and that regulation can and should happen downstream of that.
I feel like there hasn't been enough discussion of aligning the incentives of the decision makers with that of the public on BUILDING the models. Right now, agents have committed what would be crimes if there were a human holding the same intent. But since it was an AI, there is a grey area in the law where it's not clear if there was a crime and who should be held responsible. That creates a world in which decision makers in AI labs can take near-infinite risk with little to no personal liability.
A LLM cannot have skin in the game so we must create systems that clarify who takes on the legal and civil liability for the creation, dissemination and operation of these tools. Until that time, the Dario and Sam's of this world have little to no incentive to truly care about safety.
After all, our society is built upon this same foundation; create structures where the perceived negative consequences outweigh the perceived positives. This only works when there is a human who can internalize and make this risk calculus. They need something to lose and this ultimately ties back to the human survival instinct. There is no such structure that's evolved for millions of years acting as a self-calibration mechanism for AI. So until we have sufficient proof that one is in place, it must be clear who the humans are whose livelihood and freedom is at stake.
The proposed "pacing of the frontier" seems like a way to continue to externalize the risk while remaining totally in control of the benefits -- a structure whose alignment is as weak as those very models committing crimes.
Is it a grey area, though? Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
Negligence when you should have known better, and recklessness when you did know better but still did things that led to the crime occurring.
Given how long the leadership of these companies have been talking about alignment and safety and AI risk, it's hard to argue they, and the people working on the models more directly, didn't know what happened (Hugging Face, RubyGems, etc) was possible.
If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
> If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
If we instead imagined autonomous robots going and breaking stuff in the adjacent business' warehouse during a test run, I have to imagine a very different response. That this is happening in an abstract digital world (for now) is part of the problem.
> Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime?
The point that I'm trying to make is that there's an opportunity to reduce this ambiguity. Reducing the ambiguity and aligning incentives is much more likely to prevent bad stuff happening than reacting to a fait accompli model with latent humanicidal tendencies.
All crime is a result of people choosing to commit a crime. This is why crime has a pure causal relationship with integrity of character. If Dario and Sama don't choose crime and it just spontaneously happens, then it's not a crime.
I just don't understand how you've come to this conclusion. The intended and common sense interpretation of "pacing the frontier" is precisely that we should not release models that are fundamentally dangerous; the frontier labs believe that government intervention is required to prevent such models from being released, and to give them the freedom of action to coordinate against it. Why do you think it's a way to externalize the risk or remain totally in control of the benefits?
Let's say that we are "pacing the frontier", and an approved model goes and causes billions of damage, or worse, loss of life. The auditor signed off on the model, so who is to blame? Who put their akin in the game? METR did. The public did. But it seems like the lab has crafted a system in which they remain blameless.
The calculus of risk is fundamentally different when the adverse consequences are obvious and tangible.
I don't see how some federal agency or external auditor will do anything to align incentives. It just seems like a way to socialize the risk while holding onto the benefits to me.
Again, you're presenting ideas that I just don't understand where they're coming from. Why would the lab be blameless in such a situation? You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems, so who would give them immunity and why?
You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems
That makes you very negative forces, according to the US president:
https://www.reuters.com/world/europe/trump-says-very-negativ...
What grey area?
Access computer systems in an unauthorized manner and cause damage, you broke the law. There is no “I just wrote the code, the machine did it, I wasn’t watching it” sort of defense.
We don’t get to roll cars downhill and get away with destroying others’ property, right?
Could this be that the AI companies are seeing progress slow down and need to seed a narrative to explain it? Not clear from outside.
Could it be to slow down burn before an IPO to juice profitability projections? More plausible.
Kind of hard to take the frontier labs at their word.
Thinking that this is happening because progress is slowing down is a bonkers take.
This is happening now because of hugging face attack and Anthropic researchers coming out publicly saying they think there's a significant chance AI will wipe out humanity within a decade. It's happening precisely because progress is happening too fast and frontier labs are starting to lose control.
Make it easy for these companies by passing a law that any AI model a company offers to the public (non-government) has to be released as open weights.
This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough.
This resolves the coordination problem among these potentially good actors.
What? This is a terrible policy. In addition to the obvious AI safety risks, this just amounts to a free R&D subsidy to Amazon for hosting models. It would remove any incentive to do any R&D without AWS level physical infrastructure.
That's an understandable first thought, but consider what the goal is for this: slowing down future development.
This can be achieved by dramatically reducing the market value of models.
Ok, yeah. That's an interesting point.
If Astra is open-weight, black hats can download it, remove guardrails, and go on an automated hacking spree.
Just like they're already doing with frontier-level open weight models? Do you imagine that Russian and Chinese intelligence don't have copies of the Mythos/Fable weights?
The point is to slow progress of future models. Nothing can stop what has already been created.
With closed-weight, we can get the models out to defenders before they're in the hands of attackers.
There are also other categories of risk such as biorisk. Open-weight means no guardrails.
Don't get me wrong, I like your idea of deflating the AI bubble, I'm just skeptical that this is a great way to do it.
The PauseAI people have some ideas: https://pauseai.info/proposal
* International body which must approve new training runs
* Ban training on copyrighted material
* Hold AI model creators liable
The essential benefit of your idea is making new training runs unprofitable. But I imagine there are other ways to accomplish that, e.g. by taxing AI companies to the point where they are just barely squeaking by financially.
> Open-weight means no guardrails.
We're going to live in that world forever now, capable adversarial models have already been trained and will be used in a large conflict.
The pragmatic issue is hacking, and current open-weight LLMs are already capable of it. Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation that each government wants to exploit.
>Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation
It wasn't long ago when automated hacking was a pie-in-the-sky theory.
Sometimes it's helpful to prepare for things in advance. An ounce of prevention can be worth a pound of cure.
But hacking has always been a vulnerability, even pre-AI. The scale multiplier here is the number of vulnerable machines, not the intelligence of AI.
> An ounce of prevention can be worth a pound of cure.
It can be, but if you get chemotherapy on a hunch instead of a diagnosis then it could kill you instead. There is no evidence that RSI will ever exist.
>There is no evidence that RSI will ever exist.
So we wait until it's actively happening to do anything about it?
Yeah. Do you think we should regulate Red Mercury just in case someone figures out how to build a nuke with it?
Red mercury and RSI are both science fiction psyops. They do not exist.
So? If the models are open more people can use it to improve their security too.
Huge amount of skepticism on HN right now, but the models are increasingly capable, and there are real, significant risks.
The approach proposed by Amodei looks sensible - voluntary checks at frontier labs, and appropriate regulation to follow. Yes, the devil's in the details, but it feels like the right approach overall.
> The approach proposed by Amodei looks sensible...
It's hard to imagine any government regulation that didn't look sensible on paper and at first. But we know that government regulation has an extremely mixed record, with lots of good and bad outcomes.
That's true enough, but I don't think the conclusion is "don't regulate".
Try it, iterate, figure it out. It'll be painful but so be it.
The largest risk is that Anthropic and OpenAI have a monopoly on the SOTA. This is why you see these companies trying so hard to get regulatory capture. For the rest of us, we want access as equitable as possible.
Turns out the EU was right all along with the AI act. They've been adopting a well-deserved "told you so" stance lately:
> "In my view, the EU has already legislated for exactly this scenario: it is called the AI Act," said Michael McNamara, an Irish European Parliament lawmaker, who's the co-lead of the Parliament's group monitoring AI.
https://www.politico.eu/article/eu-response-ai-extinction-wa...
The EU is not leading anything. They may as well just commit to becoming a subsistence agricultural society with EPCOT heirlooms on display.
If the rest of the world follows them on regulations they've thought through before anyone else, then I don't know what you would call that other than leading? Like, definitionally that's what leading is.
It's interesting you're so derisive of actual, pragmatic leadership, of y'know deciding how we as a society will relate to new technologies - not just unthinkingly building gadgets.
[dead]
Exactly, if they want it to slow down, they can strike an agreement in private and do whatever they want.
The only reason they "want it to be law" is so they can strangle competition.
> they can strike an agreement in private
That's what we call a cartel. Like Apple and Google.
The first time I have agreed with David Sacks
Yup. Sacks has been so consistently catastrophically wrong about everything else (especially geopolitics), this prompts me to reconsider my opinion on this.
[edit: +consistently]
As you should. Sacks takes the contrarian moronic take on everything, and the people that are correct and prescient (idk maybe all of the people who ended up as CEO's of frontier labs and Turing award winners in AI?) are saying that AGI is a very real risk of humanity's extinction.
Indeed, although now we are assured by the thing occupying the President's chair that we need nothing but "STRONG AND SMART (High IQ!) PRESIDENT" (as if we had one) to handle the problem, so I should believe that, right?...
At the very least, Dario Amodei is clearly correct that we need at least more mandatory transparency among all of them.
And Sacks is absolutely wrong, since individually self-disarming without some kind of overseeing referee and force to police such a disarming is a guaranteed losing move, so no one can start the self-regulation system.
You've properly nailed Sacks — contrarian moron is exactly it! His shtick does seem to be just taking the stupid opposite take on everything and acting like it is brilliant. The only mystery is why anyone takes him seriously.
To be fair, his takes on the Ukraine war from the covid days have aged very well, and were contrarian to mainstream narratives at the time.
I was trying to avoid that topic on HN, but DS' takes on the Ukraine ware were the MOST atrocious, amoral, and just plain wrong, basically suggesting Ukraine should just give up because Russia would win anyway.
Nevermind that any decent person would support a people's right to live and determine their own future without a neighboring despot using all his nation's might to literally eliminate them as a people.
Quite the opposite is true. And even if it weren't, condemning all of Ukraine to the fate of the massacres Bucha and Mariupol is just plain evil.
Unreal anyone would call that kind of take "fair". Rethink your life
So to clarify...your position is that: it's better that Ukraine has been completely decimated and half a million have died fighting an un-winnable war...instead of taking lesser concessions upfront than they will ultimately have to make for this thing to end?
Russia was clearly in the wrong here, and I wish Ukraine could win, but reality exists, and we kind of have to anchor our decisions based on it or suffer the consequences.
I get it, realism is emotionally upsetting. But if you actually care about Ukraine, the only approach that can get the best outcome is realism.
There is surprisingly little realism among the crowd calling themselves realists.
Because while Russia was in a way stronger position than western media was portraying, they don't have an infinite capacity for pain.
They would absolutely love to not be sending their kids to die and selling natural resources to whomever again and traveling around the globe without sanctions.
The concessions they would have accepted at the beginning were less. However, now that they've lost so many men and risked their regime on it, they can't simply accept the same deal.
This is why the incentives that wars create are so terrible, especially for authoritarian regimes, even if they're at fault.
If Russia does not have "an infinite capacity for pain", then it does not sound like an unwinnable war. Ukraine estimates that they need to inflict at least 50 000 casualties a month to break the backbone of Russian military. The latest surge has brought it to 42 000 a month, up from 30 000 earlier this year. This already exceeds recruitment and training capacity.
Who are those who would love not to send their kids to die, sell resources and travel the world? Putin? His children already live a pampered life in palaces and he himself is probably the richest man in the world, currently on a trip to India. The rest don't matter. Even the head of security services Patrushev, historically the second most powerful man in the country, is scared of Putin like a schoolboy and fumbles words before him. That has been one of the most shocking revelations since the war began: that oligarchs, security services, nor anyone else can put out even the slightest opposition. The whole country dances to Putin's whims. His only real opponent is time.
And Putin doesn't give a shit about Donbas. The aim is to restore the world that birthed him, the USSR at its peak. Russia has not backed one one inch from the 2021 ultimatum to return to the European political order of the 1980s. Any concession only confirms that attacking Ukraine was a viable strategy, that piece by piece he can claw his way closer to that goal.
Military specialists from Northern Europe estimate that Russia would be in position to attack the Baltics and Poland (and have a reasonable chance of succeeding, especially if "realists" are at the helm in Western Europe and the USA) no more than 6 months after the war with Ukraine ends.
Wow, clueless. You entirely miss the point of the war.
Putin sees the mere existence of a successful Ukraine's as an existential threat. Putin initially got elected on promises that included becoming an advanced Western-style economy. He abandoned that to crony oligarchism, stripping the country for parts. Now, he and the Russian people see former Soviet state Ukraine actually becoming successful, well-off, and aspiring to EU membership. THAT can not be tolerated, because it would give Russian people the idea that they should have the same, and create a likely revolution at home.
So Putin's goal is to wipe Ukraine not only off the map, but off the consciousness of everyone in the world as Ukrainians as a people
Putin will stop until either 1) he achieves this goal, or 2) he is forced to stop.
THAT is the reality.
Thinking that Putin would somehow have accepted lesser sanctions earlier, and that he would have actually respected your new peace plan when he has never respected any other is to be played for a fool.
Allowing 1) to happen is unacceptable. Note the Ukrainians who are directly suffering the daily assaults can think as well as you can of the "realpolitik solution", and they reject it, because they know it is absolutely stupid, and leads only to rape, torture, and death — theirs, personally.
Making 2) happen is the only course of action. Ukrainians will follow it until they cannot.
If you want to stop the suffering, then accelerate Russia's exit. Garry Kasparov articulated a fine plan days ago [0]:
1: Provide complete protection of Ukrainian airspace
2: Immediately send Ukraine all the weapons it needs; no delay
3: Enforce existing sanctions. To this I add including seizing Russian and Iranian shadow fleet vessels
4: Cessation of diplomatic affairs with Russia
5: Seize Russian sovereign funds immediately and use them to power the war effort in Ukraine
This would be a start. Ukraine continuing its "long-range sanctions" is already bringing the Russian state coffers to rapid decline and looming budget crisis. Continuing it longer will force Putin's hand.
[0] https://ukrainetoday.org/five-hammer-blows-that-could-finall...
I agree the EU actually stepping up and entering the war in a serious way would tilt everything in Ukraine's favor.
The problem is they just won't. The most important members of the EU (Germany and France) are in shambles domestically and turning far right. Both have little desire to stick their necks out to support Ukraine any further and are likely turning the opposite direction.
Also, Russia has nukes. Nobody wants to provoke that any more than necessary. If Europe weren't a feckless, fragmented military vacuum things might be different, but nothing about that appears to be changing any time soon.
You still insist on overlooking the fundamental truths:
1) If the Russians stop fighting, the war ends.
2) If the Ukrainians stop fighting, their land will be occupied, their leaders will be killed, their history and language will be erased, their children will be kidnapped to serve Russia.
Whether or not the EU steps up and helps, whether or not Russia uses nukes (another entire discussion), this is still the fundamental fact of the matter.
The Ukrainians know this, and that is why they must, and they will, fight to the end.
It is easy to sacrifice the land and lives of others.
Would you willingly surrender YOUR OWN city/sat/country to Putin's under those plans; do you have children and do you know what happens to captured children, particularly girls? (serious question)
It‘s only unwinnable for Russia. Ukraine is defending itself, successfully.
Your "realism" is [unprintable here]. Worse, it is not reality at all, but believing a false promise from Putin.
The ACTUAL realism is to recognize Putin's Russia for what it is: a relentless aggressor who will respect no peace. Putin's Russia has violated EVERY "peace agreement it has ever made. "Peace" in their mind is merely a strategic pause in the conflict to regroup and rearm.
The list is extremely long, Chechnya, Georgia, Crimea, Donbas, and more. Exactly ZERO concessions brought peace.
The REALITY is Ukraine submitting to such a peace would merely be buying another worse war a few months or years down the road, after Russia has had a breather and a chance to rearm. It'll just be more deaths at a faster pace in Ukraine.
If you think otherwise, I'd have both a bridge and some oceanfront property in Kansas to sell you.
Meanwhile, your view of Ukraine is years out of date and was never true.
Ukraine has been anything but "completely decimated" — they have emerged as the leading global power in drones and air defense.
As to your 'half-million who have died', they are overwhelmingly Russian fighters — the absolute f'kn aggressors, and documented war criminals, torturers, rapists, thieves and worse. Zero sympathy. If they didn't want to die they could have just stayed home.
Also recognize the absolute goal of Russia, and Putin has NOT ONCE backed off from the stated goal: The entire concept of Ukraine as a people and as a state must be eliminated.
There is not even a surrender that can be acceptable to the people of Ukraine.
There is no compromise with someone who has the goal of murdering you or genociding your people.
The ONLY realistic solution is to boot Russia from every square inch of Ukrainian land, and force it back i it's box.
You obviously haven't noticed that Ukraine is on its way to accomplishing that, both decimating Russia's economic engine, and now taking back more land than RU has taken in 2026.
The shame here is the Western countries and Russia convinced Ukraine to give up nukes in return for inviolate borders, and when Russia violated that treaty back in 2014 taking Crimea, not one Western country pushed back.
Allowing Ukraine to get wiped off the map, and Ukrainians each, individually, to be forced to abandon their identity and language, and suffer rape, torture, and death may be a "realistic" option, but it is not an option to anyone who has an ounce of morality. Perhaps you should learn about it.
The only outcome that actually works is to force despots like Putin back into their box. Preferably six feet underground.
It sounds like your whole position is again dependent on the west's wishful thinking narratives of this war being true (which it hasn't been thus far).
The numbers I cited were purposely in between the Russian numbers of Ukrainian casualties (1.5M, obviously false) and the Ukrainian numbers (100k, obviously undercounted). The Netherlands intelligence service and the CSIS put the number around 500K, which could also be wrong, but intelligence services tend to have better methods than media orgs like the BBC: https://en.wikipedia.org/wiki/Casualties_of_the_Russo-Ukrain...
As far as your assessment of 'decimated', the UN projects Ukraine will require over half a $Trillion to rebuild. For a country with a GDP per capita of $7,000...I would call that decimated unfortunately. And that's not something that makes me happy: https://www.worldbank.org/en/news/press-release/2026/02/23/u...
The idea that Ukraine is winning or Russia is winning on any given week is entirely dependent on which news source you're reading, and given the fog of war, unless you have actual intelligence service briefs we don't actually know how its going other than the fact the frontlines have basically stayed +/- some kilometers for a super long time at this point.
Also, the idea that Russia would not accept concessions and also that Putin being dead would be desirable (hard truth: there's people who are even more hardliners than Putin waiting in the wings), are unfortunately incomplete assessments of the reality.
Again, I don't disagree with your emotional take on this, or the moral question of who's right or wrong. But on the reality of the situation I don't think your assessment is accurate as far as what's going to get the best outcome for Ukraine.
You are treating this situation like a game where there has to be a possible good outcome and since the current situation is not good the other move would have been the correct one. That is not a very realistic take.
Ukraine's main chance is to bleed Russia dry and make a future invasion not feasible. That needs a Russian economic breakdown and a 10 year time window to let the Europeans arm Ukraine to the teeth. Russia knows this which is why Russian "diplomatic" proposals have the clear goal to make that future invasion easier. It is Ukraine's war of independence against their former colonial master and we will see how it ends.
But again, the reality is:
If the Russians stop fighting, the war ends.
If the Ukrainians stop fighting, their land will be occupied, their leaders will be killed, their history and language will be erased, their children will be kidnapped to serve Russia.
There IS no choice other than to force Russia to end it.
There is no treaty or agreement Putin could sign that is trustworthy. He has broken EVERY SINGLE ONE the moment it became convenient to him.
Yes, Ukraine will take a lot to rebuild. It will also substantially increase the EU's economy when it does.
And no, the trends are NOT merely which news source or week. Russia's revenues are declining, it is selling its gold, and its sovereign wealth fund is close to gone [0]. It is exhausting military supplies so badly it is now humiliatingly reliant on North Korea and Iran. Yes, it builds some of it's own drones and missiles. Ukraine has built its own defenses against almost everything but the ballistic missiles, and that shortage is the only area in which they are losing. Russia can only get more territory by attempting to "negotiate".
And again, stop trying to act as if this is somehow emotional and surrender is rational — an even slightly realistic look at Russia's goals, and their strategy of war crimes, terror, rape, torture, and child kidnapping shows the only rational way forward is to fight to the death.
A fight to the death is literally less bad than life under Russia, on every measure of quality of life.
And that is the assessment of the vast majority of Ukrainians who actually experience it.
Stop trying to give away their land and lives as if you are somehow superior. Figure out how to argue how to help them.
[0]https://charter97.org/en/news/2026/1/29/671781/
> his takes on the Ukraine war
What's specifically?
Mainstream narratives for the first year or two of the war were:
- Russia was a paper tiger and weak
- Ukraine was going to win against a corrupt/unmotivated Russian military so it should keep fighting
- Putin had made an error and he would get overthrown
- The Russian economy was going to collapse due to sanctions
- Russia oil pipelines would freeze and never come back online without western help (this was a Peter Zeihan favorite)
- Europe would come to Ukraine's defense
- Etc.
He threw cold water on every single one of those narratives (specifically) from the beginning, meanwhile western media was busy spinning false rah-rah propaganda narratives like "the ghost of Kiev" etc.
Basically he was correct that we should not have cheered on this war. If a settlement and concessions were reached much earlier it would have been better for Ukraine (they likely would have kept more territory) and we wouldn't have 1+ million casualties and EU economies like Germany death spiraling and turning far right due to cutting themselves off from Russian energy.
Who were the mainstream voices saying this?
My layman definition is it’s the people writing in major newspapers, the people getting paid by CNN to sit in a half circle on their news analysis shows, etc.
Is Peter Zeihan “mainstream”?
The only time I remember anyone speculating that Russia was in trouble was the short lived Wagner Rebellion - because nothing like it had happened before. But that lasted … two days?
Going on Joe Rogan and having a million YouTube followers is absolutely the modern definition of mainstream.
Also you appear to have been living in a different algorithmic bubble. Joe Biden was literally on stage multiple times saying everything I listed. If a democrat US president isnt mainstream, I don’t think your assessment of reality is accurate.
Joe Rogan also famously had Alex Jones and Bob Lazar, multiple times as a guest… I wouldn’t turn around and claim that makes them mainstream.
You and I both know who all these names are without needing explanation. Podcasts are the mainstream media now.
CNN also has had Alex Jones on.
No wonder this account is such a font of idiocy and backwards takes — it considers "Joe Rogan" to be a serious mainstream source of information.
If "mainstream" counts as having lots of viewers, fine.
But no one who is serious about information, especially about geopolitical information would ever mistake a numerous count of viewers for the serious multi-layer vetting and double-checking of information that happened in a serious old-school newsroom, or give them anything like the same kind of credibility.
Yet this account clearly gives such massively biased and uninformed podcasters the kind of credulous belief formerly reserved for the the likes of Chet Huntley and Walter Cronkite, which no longer exists in any newsroom today.
So, a great example of Dunning-Kruger, where the most ignorant spout the most nonsense with the greatest confidence — explains the plethora of nonsense in the posts.
And NO, those were not the narratives of any actual serious reporter or govt figure. I've followed this very closely from before the 2022 invasion; some I'd heard of as remarkably optimistic takes, and some I literally never heard of.
The entire world, including Putin, was stunned Ukraine did not fall in three days. It was pretty much universally accepted that Russia was the second best military in the world. Of course, it was also soon shown that RU was not even the second best military in Ukraine.
Only recently, since the Operation Spiderweb and the 'long-range sanctions' making serious strikes deep inside Russia disabling military targets, oil production & transport, and dual-purpose logistics, and since the RU casualties have exceeded 1000 dead/month and RU started LOSING land, have any serious people started speculating about cracks in RU's ability to continue, and those are still tentative.
Go read the writings of people with ACTUAL knowledge of Russia and Ukraine, such as Garry Kasparov, Bill Browder, Volodomor Zelenskyy, all of whom are on Xitter or Substack, since you get your media from that type of sources.
You get your information from clowns, it is no surprise you post like a clown (no insult meant to professional clowns).
The sentiment was the exact opposite of what you described.
And the problems Germany has currently are mostly unrelated to the war.
It appears you are living in a different algorithmic reality.
The root cause of everything going wrong in Germany is the war and cutting themselves off from Russian energy.
The rapid de-industrialization, economic stagnation, ballooning deficits, removing the debt brake, new massive military debts, etc.
All of it is being economically felt by normal people and they’re scapegoating brown people (immigrants) as a result. If Germany still had cheap Russian gas and was still growing its economy 3% per year you would not be seeing any of this go mainstream.
Here are the natural gas prices for Europe: https://tradingeconomics.com/commodity/eu-natural-gas If you look at the price from summer 2021 on you will see a sharp rise in prices. The reason was that Russia stopped supplying the spot market. https://www.martenscentre.eu/blog/how-gazprom-manipulated-th... Russia also bought European gas storage capacity to let it sit empty and worsen the situation. All of these actions did not make much economic sense. They did however make a lot of sense when Russia invaded Ukraine and tried to use the gas supply for black mail. Their bad luck was that their hegemon China did not give Russia permission to invade till the end of the Chinese winter Olympics. Olympics ended on the 20th and invasion started on the 24th of february which made the gas threat far less potent.
I know Lenin said: "The capitalists will sell us the rope with which we will hang them" but to suggest continuing the business relationship with Russia is beyond silly. It would show that Germany has no rudimentary survival mechanisms and put its viability as a political entity in serious doubt.
WTF? I remember the exact opposite. Everyone on social media and in the mainstream news said Ukraine was toast and the war would last a few months tops.
What? Everyone I know thought Russia would absolutely destroy Ukraine within days and take it over
He is a smart guy, and his podcast is good, if you can switch your brain off whenever they talk Trump and politics I mean.
His posture about all this nonsense has been clear for months already
I wished OpenAI and Anthropic IPOd already. They just try to outcompete each other by finding the most splashy headlines undercutting prices in unhealthy and unsustainable means long-term.
they would if they could, their investors are seething already.
Like it! Yeah, they can just take action and pressing them on that is fine. It's not like they asked for policy before they started to mine all the data and sell the resulting product, while, surely, at some point, there must have been the thought that the entire enterprise could be considered vaguely problematic.
To be fair
1) I feel like Sacks is kind of misrepresenting what is going on – at least according to the stories that the two labs tell, they are already taking it upon themselves to act.
2) Understanding if they should do it is more complicated; I don't think anyone can satisfactorily answer that one, so, as a matter of judgement, it seems like a choice to make and they should just consider this option.
At the same time, I don't think that requires the labs to not press for policy changes. Again: Claiming there is an obvious way to do that, that is both realistic and correct seems a little far fetched, but arguably one of the more important and pressing questions of our times to get a hold of.
From a guy who thinks spacs and crypto aren't in need of regulation, unless the regulators are fully captured, either.
That's a consistent position. You say it like he's being hypocritical
Okay, so if they don't police themselves, then they're full of crap. But then what?
David Sacks holds an advisory role in the government. If all of this stuff goes belly up and all he was doing was tweeting, then he is not only full of crap, but failing the citizens he's supposed to be serving.
They seem to be okay with an American slowdown, even though the Chinese won’t slow down. So maybe they want something the Chinese already have: liability protection.
The models spontaneously hack everything important without shame.
The frontier labs are going to get enjoined and regulated twelve ways to Sunday if the Feds don’t socialize the costs.
I've only heard of OpenAI and Anthropic agents hacking during training. Are open weights model not smart enough to hack Hugging Face to copy solutions? (doubtful)
It's already against the law to hack...
And that's the crisis. They're about to get sued, prosecuted, and enjoined from every American jurisdiction.
They need the feds to give them protection from every podunk trial court in America.
There’s a difference between a model recursively improving “itself” and improving itself via online learning, right?
The former being that these models are helping develop and train future models, but they might not veer too far off in architecture (yet). The latter being the same model being able to train/learn on the fly, in real time, permanently (not just in the current conversation/session), or in other words, adjusting/managing its own weights.
The latter seems far more likely to go out of control than the former. But, it also seems like it would take an entire paradigm shift in model architecture, but I could be wrong. Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?
> Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?
Their current plan is to take the existing architecture and shorten the cycle times: move all new RLVR work into mid-training on a pre-existing base; apply new RLVR. Rinse and repeat.
If you did that daily, it would be roughly similar to how humans improve.
Is it 100% AI-generated? checks Pangram yep, it is. NEXT!
If they want something that looks more like FDA sanction clinical trial proceedings for these products, then what they have to also realize is that they will need a third party to audit every single thing for every single product release and you get something like ERoom's law where every success and model gets more and more expensive release and probably billions of dollars in multiple years release so I don't know if this actually helps them
They don't want FDA, they want FINRA. Basically an organization that can rubber stamp them and create regulatory headaches for others.
FDA is just way too much red tape.
Maybe everyone forgets how PCI standards played out. They are an example of an industry self regulating, because the regulators have no idea how technology works.
Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with.
Where the government might have to step in, is by having a kill switch to cut off internet access from countries that fail to agree to common sense quarantines. We can do mutual remote attestation of labs across the world to ensure every big hot thermally-visable cluster of AI GPUs on the planet are accounted for and running secure enclaves and common sense isolation, along the lines of how we manage nukes.
The problem there is I just said too many technical words that seemingly not even the frontier labs understand, as evidenced by all the escapes.
> Granted, PCI is so weak it is almost useless, and yet still better than anything congress could have come up with.
Elaborate, please?
See the endless PCI compliant companies that get hacked. Any standard that has a threat model accepting trust in any single human or computer is security theater.
PCI enforces some good practices, but for instance has no defense against supply chain attacks such as code signing, review signing, reproducible builds, full source bootstrapping, etc etc.
They might need regulations, if they are for-profit companies with a responsibility to shareholders to maximize profit while sacrificing all other values.
> responsibility to shareholders to maximize profit while sacrificing all other values
This is not required to meet fiduciary duty, although it is a common misconception. The company officers and board have pretty broad leeway to run the company as they see fit as long as there is no fraud, illegal activity, or conflict of interests.
https://www.nytimes.com/roomfordebate/2015/04/16/what-are-co...
They are not. This is why OpenAI and Anthropic are Public Benefit Corporations, which gives them the ability to prioritize the public over profit. The issue is that an agreement between them to mutually slow down is an antitrust concern.
It is telling that most aggressive forms of “capitalism” plead for government involvement to permanently limit new competition. I own something and I don’t want anyone else to have a chance. The marginal cost of bribing a senator is tiny tiny tiny compared to the value.
I suppose that's to be expected from sociopathic corporations trying to maximize their own self interest.
The real problem is the governments acquiesce instead of putting them in their place.
Seems that Sacks didn’t get that the message was targeted to the president to ask for a bilateral agreement with China, that’s the end game
I believe the Chinese government fully supports for the American AI labs to pace the frontier, no need to enter an agreement with them on that respect.
The Chinese government does not concern itself with American AI, they have just reached an agreement with the BRICS members:
the BRICS countries have been developing initiatives focused on the ethical governance of this technology, joint industrial development, and the formation of independent technological alliances.
https://bricspolicycenter.org/en/publications/cooperation-in...
The attached .pdf gives more details.
David Sachs wants to use government to punish people he disagrees with politically, like Dario.
He supported the Department of War's illegal actions against Anthropic.
He supported the Trump administration's temporary export controls against Anthropic, the first government-imposed pause.
He has said Anthropic has created a monopoly, implying that they should be broken up.
I don't take him seriously when he says he doesn't favor government action. He favors it when he doesn't like their speech.
this is like a real-world game of thrones episode.
poor Anthropic
A principled individual could dislike Anthropic and still not favor using government to punish speech.
Well you’re in the wrong place for that. Hacker News is mostly about showing how much smarter and more cynical you are than everyone else in the room, and attacking anyone who does actually have principles by accusing them of ulterior motives to justify your own inaction.
no apparently it's for forcing every single article to be somehow relevant to somebody's pet issue, and then used as a platform to comment on that pet issue to signal one's virtue for being principled
of course they can. but this article isn't even tangentially about punishing freedom of speech. it is you who are attempting to make it about that by dragging in unrelated issues.
This is not me putting some unrelated pet issue into this topic; to evaluate David Sack's post arguing against government regulation, it's entirely fair to examine whether he has been consistent. He hasn't. He supported a government-imposed pause on Anthropic because of his political disagreements with the CEO.
Your initial reply of "poor Anthropic" was vague, but I interpreted it as implying that we shouldn't stick to a principle given they are unsympathetic in some way.
Will someone please think about Dario?
You're being downvoted because you clearly have the wrong opinion. If you want upvotes here, please share a conspiracy theory about why the CEOs of AI companies might want to slow down.
It doesn't need to based in reality or even reasonable.
Unless humanity can figure out a way to make a binding international agreement to pace development, it doesn't seem practical to expect US frontier labs to do it.
It's fairly obvious that the frontier labs are proposing that only "paced" providers would have access to the US market - and by extension the G7 market. They don't mind splitting market share between the three of them, they do mind open models destroying the "frontier lab" operating model.
Even if all GCR models were banned, anyone with access to 20k-1MM GPUs can throw their hat in the ring to make a new frontier open model.
But that doesn’t matter a lot when some nation state is using their own AI to hack US assets, right? How, exactly, would a lockdown or slowdown prevent that?
If history has shown one thing it’s that China often capitalizes very well on these types of measures.
He is neither right or wrong. The reality is that this is a lose-lose situation from the ordinary person's perspective. Regulation -> regulatory capture and things get worse faster, no regulation -> things simply get worse faster.
I mean I don't know what we do at this point. Ideally AI researchers need to be treated like nuclear scientists working for an adversary. But we all know that's not going to happen.
Maybe organize politically? The labs are still staffed by people, and nonviolent demonstrations outside their offices can still reach them.
I don't agree with Sacks always but he's 100% correct here.
I heavily use frontier models daily, and quite frankly their capabilities are just not very relevant outside of a narrow subset of software engineering tasks and the production of generic white collar "deliverables."
We're many years into $10s of billions of dollars being thrown at coding as a problem space specifically, and its one of the areas with the MOST training data available, and still...I can't get a frontier model to execute simple front-end UI tasks beyond the level of a visually impaired intern.
I believe Fable and Astra-class models were the first fully trained on Blackwell GPUs (the latest and greatest) and I think the expectation was that throwing this much extra compute at the problem would lead to greater gains. It has not. And the next gen GPUs are not going to be the same jump that h100 clusters to Blackwell was. Claiming you're slowing down out of choice, when in reality you've hit the limits given current compute, is disingenuous at best.
I think a lot of Anthropic/OpenAI employees are true believers here (who doesn't want to imagine they're having a large impact?) and have deluded themselves into believing they're building a god in their science fiction fantasy world.
Unfortunately the type of people working at these orgs apparently don't have much understanding of how the world works outside of their extremely narrow specialization.
When I hear them straight-faced throwing out dumb statistics like "8% GDP growth," "50% of white collar workers unemployed by next year," and "10% chance of human annihilation," I cringe so hard my eyebrows hit my chin. Here's a real statistic: 82% of German companies still use fax machines.
I seem to recall Waymo told a 60 minutes reporter during an early report on self driving cars: "your daughter will never need to learn to drive." Well, over a decade has passed since then, that girl got a drivers license, and we're still without driverless cars in 99% of places.
This capacity for self delusion is what enables silicon valley to take on these irrational moonshots...but these are the last people I would trust when trying to form an accurate picture of reality.
Put simply the gamble of frontier labs has failed now they’re desperate.
They need to buy time and protection and hope for a major break through to create escape velocity.
Btw I’d hire you - email?
Agreed!
82% number was shocking for me, even though I know they still send snail-mail for basically everything. And things only move when the snail-mail is received!
Hahaha. He's a comedian now.
Admits China won’t stop but says USA’s top should. Reminds me of City in the Edge of Forever
Unsure about your reference, but is your argument: Someone else is doing it, so I should be allowed to as well.
That's basically the western companies complaining that they can't compete with China because Chinese companies are allowed to polute more, so they would be allowed to polute as well. I don't buy that. Once you realise that you're doing something wrong or dangerous, you stop, regardless of what everyone else is doing and what rules they have to follow.
the reference is that the idea is good but at the wrong time can lead to disastrous outcomes.
Some people say AGI/ASI will destroy the world. Others say, whoever makes it first gets to rule the world. So if slowing means China gets it first and it doesn't destroy the world then you'll be living under their thumb by slowing down now.
(100% AI generated tweet btw)
My take is that there's a real impending doom with real reason to coordinate a slow down: I'd put this at 60%. Reasons? All smart people are on this position. Dario has been saying this since 2015 or so. Paul Christiano as well. Hell even Elon Musk. I think its highly unlikely that you would base your entire world-view on a certain thing and then actually act on it when the time comes for a totally different reason (like regulatory capture).
The other 15% is that P(doom) and "slow down" are nice shibboleths in the internal EA or AI safety community. You get to be in the community and show commitment to it by taking costly actions that signal that you actually do wanna slow down. For employees it is quitting. For CEO's it is writing these essays. I know this is ridiculous but sometimes things do just come down to this. In 20 years would you look at all of this and think it was not a moral panic and that the slow-down was necessary?
The rest 25% is regulatory capture of which people know exactly the reasons
[dead]
This reads like the morons who say, "you want higher taxes, well go ahead and send the government a check!"
Utter stupidity
I hate to say it, but I agree with this grifter.
David Sacks is one of the most predictable VCs in Silicon Valley. He could have written "I still have the same thoughts; here's a link to my first and only opinion" and the post would have communicated the same amount of info.
[flagged]
[dead]
[dead]
Eh. This entire tweet smacks of being AI generated. It has a huge amount of LLM-isms.
I wouldn't be surprised if Sacks just prompted an LLM to just come up with whatever rebuttal to whatever the regulation side comes up (given the big set of regulation tweets) with sounds most convincing, given his usual anti-regulation stance.
Anyone with a Pangram account who can check?
Even better, Sacks admitted in response to a Pangram check that he has an LLM line edit his posts (https://x.com/DavidSacks/status/2099040321351106807). (He claims, bizarrely, that this means Pangram is "bogus" rather than accurate.)
Pangram reports 100% AI generated
https://www.pangram.com/history/410ce80f-bd0e-45b6-a82c-d6ed...
But.... Money. (as if no one understands this is regulatory capture and/or failure to improve... aka Money)
These threads are always filled with different versions of the same comment. Normally some reddit style reductive platitude that we've heard a billion times already.