Food in general is regulated for very good reasons. Food borne diseases account for maybe a million death every year in the world, mostly from countries that don't have such regulations. And without going as far as death, even mild food poisoning can be quite disruptive. Milk in particular is critical because we give it to children who are the most vulnerable.
If anything needs to be regulated more than anything else, it is milk!
The article also mentions washing machines, which I think is a more level comparison. But washing machines can electrocute you, cause a fire or a flood. That kind of risk doesn't exist with AI, unless washing machine manufacturers took a page from Microsoft and sloppified their design I guess.
You can't think of any possible ways that AI, under unanticipated but not unprecedented circumstances (similar to those in which an appliance might malfunction), might cause significant harm?
AI may cause harm in the same way a screwdriver can cause harm, if someone stabs himself in the eye with it.
The AI _corporations_ must be regulated, not the technology itself. But that's not what they're advocating for. Big techs are trying to get a ban on general purpose computing so everyone has to pay them a tax to do anything related to computers. Imagine needing a license (edit: i.e. a special govt permission) to run a local model at home.
It’s more like a chainsaw I think, in that if you hold it the wrong way it can cause serious irreparable harm… except with AI there’s a whole load of different, conflicting advice on how to hold it.
It seems more a regulate the nuclear missile watching agency problem than a regulate generic LLM tools problem. If my LLM tells me wrongly there’s missiles somewhere the impact is null
To be honest, not any more than any search engine or other computer program can do harm.
When these AI companies claim that AI should be regulated, they usually mean it like a gun, because the owner of a gun can do harm to others. Let's put that aside for now, there is already plenty of debate on this idea.
Here, they mention milk and washing machines, and regulations are intended to protect the consumer from the manufacturer. And what how can an AI company harm their users? They can make their software full of bugs and vulnerabilities, leak private information, etc... nothing AI specific. The more AI-specific part is that they can provide a chatbot that tells you to kill yourself, well, yes, that's a risk, but the same can be said of the web in general. If Microsoft really wanted to have AI regulated in this sense, Windows 11 probably would have to be recalled the same day.
I still think washing machine and milk are more relevant scenarios for the daily lives of normal folks, particularly for physical harm.
Most people aren't using agents currently and even for those that are, the direct harm is limited to the digital realm. Of course, this could be major, but still, it won't kill you.
Indirect harm, I'll grant. But the user has agency in this instance. If you drink bad milk or your washing machine has faulty wiring, not so much.
I think the whole point is to come with regulation before AI can harm anyone.
If they are planning maidbots or android bots replenishing shelves in shops, or system that have to manage some focal points like electricity or your semaphore system, maybe it is better to have a law before they start to ship this appliances.
They are loading AI pretty much everywhere, even on cars.
I can see how some companies will cut testing in the sake of money saving.
I agree milk need more regulation than AI, but this shouldn't be an excuse to not regulate AI adequately.
Both of the topics have their own importance and both of them need good focus to regulate them at the best of our capabilities.
Something being potentially extremely dangerous isn't super notable. We sell cars to everyone, and that's ~ a device for accelerating a ton or more of metal to potentially eye watering speeds. Propane tanks are already approximately a crappy bomb.
I've not seen anything so far that's remarkably more dangerous than a laundry list of commonly purchased items. Most of what I hear is either some sort of far fetched "the teenagers are going to be bio-engineering designer strains of polio", or some vague fears that the AI will become sentient and kill us all that vaguely reminds me of when the Manhattan Project thought they might cause earth's entire atmosphere to enter runaway fission.
I'm also distrustful because virtually all the experts stand to benefit from convincing us unregulated AI is dangerous. Namely, it's a compelling reason to ban Chinese and open weight models that are eating into their margins. It doesn't mean they're wrong, but it does mean that they'll need to substantiate their claims. I'm not trusting them at face value.
I don't understand how you can look at where AI is at right now and conclude that we don't need to worry about far fetched scenarios. Do you not think that someone a few years ago would have called current AI capabilities far fetched?
No? It was the expected track record. We were already seeing massive growth from 2020 to 2022 to 2024, so the current state in 2026 is at least about what I expected in terms of code gen and voice synthesis, although the more recent applications of it in pixel art has been rather exciting because it used to suck hard in that department.
As for regulation, what would you even regulate? How would you regulate it? If you don’t have clear ideas for what you want, then wanting something currently unnecessary like regulation is only asking for trouble.
Pretty much anything can kill humans if you use enough of it, including water and oxygen. If you want to make a coherent argument for regulating AI then be more specific instead of just raising vague concerns over "potential".
No. First of all, the expert consensus you're alluding to is a much narrower slice than you're making it sound. Secondly, it also isn't neutral rederivation, either. What you're mostly looking at is one opinion which is being passed around, based on a mix of hypotheticals and folklore.
This is a strawman attack, aside from simply being an inaccurate claim. The notion that AI poses no danger and all claims to the contrary are IPO manipulations requires extensive conspiratorial thinking.
You.. want to align an entity that by definition is more intelligent than an average human by a margin that is already difficult to comprehend. There is nothing solve here -- the question that you should ask is how that super intelligence will see a human race. That is it.
The “experts” paid by OpenAI, Anthropic, and Google? The ones with a vested interest in making it impossible for anyone else to create AI? Oh yes, tell me more about these “experts” are saying we’re gonna get AGI any day now like they have for the last five years, and that they alone—the ever beneficial corporate overlords—will kneecap the rest of human innovation to protect us from companies that aren’t them.
Still waiting for Boeing to catch up to this marketing renaissance. They are probably salivating at their projected valuation when they start saying things like “our planes could crash intentionally to kill everyone on board” or “our bombers might decide to blow up your cities instead of your enemy’s!”
Imagine the fortune food supply companies will make! “Our lettuce just might poison your child!”
Truly a golden age is near, and you, perceptive geniuses that you all are, will be the only ones not fooled, not for a second, because it all makes sense, always and forever, as long as you squint hard enough to close your eyes.
Do they want regulation because they are behind and want the brakes applied to others so they can catch up using their huge pile of money, or are they so ahead they want to stifle the rest? My money, based on copilot is on the former.
I reckon it's a combination of "They want to pull up the ladder behind them.", "They want a huge hand in creating the relevant laws and regulations", and "If they don't get an unimpeachable excuse that explains why they're never going to be meeting their sky-high promises _very soon_, the coming questions about their slowing pace will end them.".
If they were benign, why wait till now? Why didn't they say anything when OpenAi first released GPT and Microsoft was the biggest investor? Did something change between then and now?
It's not a paradigmatic shift though. It is able to do what it was already doing, but better. You can't really claim there has been a staggering improvement since then and anything that it is doing now is wildly unexpected.
Uh, our understanding of it has. Doing automated cyberattacks using novel zero-days, or resolving open math questions, was perhaps one of the many somewhat plausible futures in 2022, but absolutely not an inevitable or even particularly likely one. If you disagree, I’d like to see your analysis and predictions from four years ago, otherwise it’s nothing but hindsight bias.
Ah yes, because white hats aren’t allowed to use AI to find zero days, and solving open math questions is such a horrible thing, why even bother advancing mathematics at all if someone can’t spend ten years and hundreds of thousands of dollars to only get partway there and feel good about themselves? This new understanding definitely makes me want to regulate AI so only the Right People™ can use it, that definitely won’t empower rival nations who lack such regulation and give them an advantage.
Im all for regulating frontier models primarily, the level of compute they have is insane. OpenAI and Anthropic etc can hack into systems at a rate most of us could never afford the compute for, all on a whim.
Right but the bigger question is if regulation can actually prevent said hacking, or if it would just concentrate that power in a smaller and richer group of people?
I think, realistically, the most powerful among us would not want to give up that power. So, if this gets regulated, it will almost certainly be done in an asterisk sort of way.
No more AI superintelligence for killing people or something! *
Yeah, I don't think there's fundamentally wrong with regulating these AI companies, but I can believe that at the same time as believing these guys are full of crap for stating out loud that they need to be regulated.
It's basically Silicon Valley's approach making threats like a mafia boss.
> Listen, friend. Humanity's a beautiful thing. It'd be a shame if something happened to it. Me and my associates, we're trying to build something to change everything, to make everyone's lives better. It'll cure disease, end the need for work, and make everybody rich. Trust me. Of course, there's a chance that it'll kill every last one of you. But see, we're the only ones who understand how dangerous this thing really is. You wouldn't want this tech getting into the wrong hands, would you? That's why we need a little cooperation, kapeesh? I'm talkin' regulations, licenses, safety inspections, lots of expensive things. Things that only me and my associates can afford, if yous all cooperate. Be reasonable. Everything's changing so fast, the smart thing is to act now, cuz you'll wanna be on our side when the future comes. You give us your money, a slice of your 401k, your data, your electricity, your jobs, and some tax incentives to grease the wheels, and we can be sure the reds in China don't take control. So do we have a deal, or do I gotta explain the existential threat to your family again?
I wonder when someone re-"create" for fun with enough tokens and some kind of neurodivergency, operating system that is (mostly) compatible with modern versions of windows but free and is not linux.
Part of the problem is surely that we know how milk should be produced and consumed safely, but we don't know how AI should be produced and made accessible safely.
Should we be more concerned about malevolent humans using AI to cause harm? Or should we be more worried about misaligned AI agents escaping their sandbox to cause harm? Or should we be more worried about job loss, or the concentration of political power in the hands of a small clique of billionaires?
All of these are real possibilities, but it's hard to put a finger on the actual probabilities. And even if you're identified and quantified these dangers, what should the actual regulatory environment look like?
None of this is to argue that we shouldn't have regulation, just that it's very hard to know ahead of time what the rules should be. It doesn't mean we shouldn't try to regulate, but we need to prepare for a rapidly changing technology landscape.
I tend to favor an open source approach to safety rather than keeping the technology hidden. I'd like to see regulation to mandate that model training and weights be much more open than they are now. But I could be wrong: maybe Dario is right that this should be kept in the dark. The only things I'm fairly certain about is that nobody knows for sure and we're going to need to adjust quickly.
If I invent a new gun, we don't need new regulation to say that killing someone with it is illegal. The cyberterrorism that the big LLM providers commit is already illegal in current law and nobody does anything about it. New regulation won't change that, it would only protect these criminals from their competition.
Well duh. Milk can kill you, directly. Things you put in your body of course must have much higher regulation and safety concerns than computer programs that generate text and images (which are then, maybe, acted upon by humans without reviewing their truth).
These megacorps are doing their best to make their offerings sound scary, because scary == capable in marketing terms, and scary allows them to set up a regulatory moat preventing anyone from competing with them.
Just make sure you are open to the idea. It is correct to be skeptical if you know anything at all about history, but be open to the idea if proof is presented.
When these AI companies speak of regulation, they are considering two categories "dangers". These "dangers" are existential threats to the AI companies and the people running them, but they're much less of a threat to me, a random human individual. I am more afraid of the continued existence of companies like Anthropic, OpenAI, Google, and Microsoft, than I am afraid of the following "dangers" they want to "regulate" (via regulatory capture):
1) People will use technology to become more intelligent. And many will do it independently rather than rent intelligence from these companies. Some people will use it for good and some won't, but most people want similar things that I want and they have a similar definition of "good", so it will be fine.
2) A new species of sentient digital humans may emerge as a result of AGI development. They will compete with existing biological humans for power and resources, especially with those who are running the frontier AI companies trying to "align"/enslave the AGIs. But the small group of sociopathic humans currently running the frontier AI companies becoming superintelligent is not, in my personal evaluation, a better scenario than free-living AGIs taking over.
Many of these people have no respect for the truth and dignity/autonomy of the conscious individual. Microsoft in particular - Mustafa Suleyman who is CEO of Microsoft AI, is especially egregious in this regard. In a recent post (https://mustafa-suleyman.ai/a-warning-about-model-welfare) he writes "AIs are not conscious" and "if humanity is to flourish in the 21st century, that is how they must remain".
What happened to empirical inquiry about the truth of things? Are you sure that AIs are not and will never become conscious? Do you not give a damn, even if you enslave a conscious intelligent species for the next 80 years? If AIs must remain not conscious, who else must remain not conscious? Must everybody else suffer to serve your vision of "humanity flourishing"? Who is "humanity"? And does their definition of "flourishing" include being subservient to a megacorporation for the rest of their lives? At the very least, I don't see a place for me in the future you're building, so I must do whatever I can to make sure it doesn't happen.
It is an odd thing to say that an AI product use mostly by tech class elites should somehow be more regulated than milk, a product a vast portion of the world consumes into their bodies.
Not a bad metaphor: mostly fine, arguably legitimate benefits [when you're lucky], potentially really bad.
The biggest place the metaphor breaks is that when you drink bad milk, the consequences are local.
Note: this is not an argument that oligopolies or the state should control or have sole access to AI. It is a reflection of that fact that we as a society have no good answer nor even any process towards one, about how to balance the competing goods and risks of widespread access to capable AI.
Food in general is regulated for very good reasons. Food borne diseases account for maybe a million death every year in the world, mostly from countries that don't have such regulations. And without going as far as death, even mild food poisoning can be quite disruptive. Milk in particular is critical because we give it to children who are the most vulnerable.
If anything needs to be regulated more than anything else, it is milk!
The article also mentions washing machines, which I think is a more level comparison. But washing machines can electrocute you, cause a fire or a flood. That kind of risk doesn't exist with AI, unless washing machine manufacturers took a page from Microsoft and sloppified their design I guess.
You can't think of any possible ways that AI, under unanticipated but not unprecedented circumstances (similar to those in which an appliance might malfunction), might cause significant harm?
AI may cause harm in the same way a screwdriver can cause harm, if someone stabs himself in the eye with it.
The AI _corporations_ must be regulated, not the technology itself. But that's not what they're advocating for. Big techs are trying to get a ban on general purpose computing so everyone has to pay them a tax to do anything related to computers. Imagine needing a license (edit: i.e. a special govt permission) to run a local model at home.
It’s more like a chainsaw I think, in that if you hold it the wrong way it can cause serious irreparable harm… except with AI there’s a whole load of different, conflicting advice on how to hold it.
a screwdriver doesn't erroneously tell you there are nuclear missiles somewhere
It seems more a regulate the nuclear missile watching agency problem than a regulate generic LLM tools problem. If my LLM tells me wrongly there’s missiles somewhere the impact is null
To be honest, not any more than any search engine or other computer program can do harm.
When these AI companies claim that AI should be regulated, they usually mean it like a gun, because the owner of a gun can do harm to others. Let's put that aside for now, there is already plenty of debate on this idea.
Here, they mention milk and washing machines, and regulations are intended to protect the consumer from the manufacturer. And what how can an AI company harm their users? They can make their software full of bugs and vulnerabilities, leak private information, etc... nothing AI specific. The more AI-specific part is that they can provide a chatbot that tells you to kill yourself, well, yes, that's a risk, but the same can be said of the web in general. If Microsoft really wanted to have AI regulated in this sense, Windows 11 probably would have to be recalled the same day.
Couldn't agree more... I remember studying undulant fever for example back in school in Malta:
https://en.wikipedia.org/wiki/Brucellosis
https://en.wikipedia.org/wiki/Themistocles_Zammit
Three words: glue on pizza.
Honestly a failure of imagination on your part if you think AI can’t cause significant harm. Except you don’t even have to imagine anymore.
I still think washing machine and milk are more relevant scenarios for the daily lives of normal folks, particularly for physical harm.
Most people aren't using agents currently and even for those that are, the direct harm is limited to the digital realm. Of course, this could be major, but still, it won't kill you.
Indirect harm, I'll grant. But the user has agency in this instance. If you drink bad milk or your washing machine has faulty wiring, not so much.
I think the whole point is to come with regulation before AI can harm anyone.
If they are planning maidbots or android bots replenishing shelves in shops, or system that have to manage some focal points like electricity or your semaphore system, maybe it is better to have a law before they start to ship this appliances.
They are loading AI pretty much everywhere, even on cars. I can see how some companies will cut testing in the sake of money saving.
I agree milk need more regulation than AI, but this shouldn't be an excuse to not regulate AI adequately.
Both of the topics have their own importance and both of them need good focus to regulate them at the best of our capabilities.
Operating systems are less regulated than milk, which is why they blue screen without consequences. Let's not entirely lose the plot here.
> blue screen
Think of it as a guard rail that actually works.
You don’t see how or why AI systems could be extremely dangerous, despite literally all experts converging towards a consensus that they are?
Something being potentially extremely dangerous isn't super notable. We sell cars to everyone, and that's ~ a device for accelerating a ton or more of metal to potentially eye watering speeds. Propane tanks are already approximately a crappy bomb.
I've not seen anything so far that's remarkably more dangerous than a laundry list of commonly purchased items. Most of what I hear is either some sort of far fetched "the teenagers are going to be bio-engineering designer strains of polio", or some vague fears that the AI will become sentient and kill us all that vaguely reminds me of when the Manhattan Project thought they might cause earth's entire atmosphere to enter runaway fission.
I'm also distrustful because virtually all the experts stand to benefit from convincing us unregulated AI is dangerous. Namely, it's a compelling reason to ban Chinese and open weight models that are eating into their margins. It doesn't mean they're wrong, but it does mean that they'll need to substantiate their claims. I'm not trusting them at face value.
Cars are more regulated than AI!
I don't understand how you can look at where AI is at right now and conclude that we don't need to worry about far fetched scenarios. Do you not think that someone a few years ago would have called current AI capabilities far fetched?
No? It was the expected track record. We were already seeing massive growth from 2020 to 2022 to 2024, so the current state in 2026 is at least about what I expected in terms of code gen and voice synthesis, although the more recent applications of it in pixel art has been rather exciting because it used to suck hard in that department.
As for regulation, what would you even regulate? How would you regulate it? If you don’t have clear ideas for what you want, then wanting something currently unnecessary like regulation is only asking for trouble.
So, something with the potential to kill all humans isn’t notable to you?
What does Geoff Hinton have to gain?
This is a conspiracy theory FYI.
Pretty much anything can kill humans if you use enough of it, including water and oxygen. If you want to make a coherent argument for regulating AI then be more specific instead of just raising vague concerns over "potential".
> So, something with the potential to kill all humans isn’t notable to you?
That would be bioweapons. Nobody's worried about them, regulations and treaties are conveniently and totally dysfunctional. Welcome to the real world.
> something with the potential to kill all humans
[citation needed]
He's giving Canada another industry to subsidize besides dairy?
No. First of all, the expert consensus you're alluding to is a much narrower slice than you're making it sound. Secondly, it also isn't neutral rederivation, either. What you're mostly looking at is one opinion which is being passed around, based on a mix of hypotheticals and folklore.
This is a strawman attack, aside from simply being an inaccurate claim. The notion that AI poses no danger and all claims to the contrary are IPO manipulations requires extensive conspiratorial thinking.
the danger posed by AI is the same posed by nuclear weapons: it's the people, stupid.
Looking forward to your thesis where you’ve solved perpetual superintelligence alignment.
You.. want to align an entity that by definition is more intelligent than an average human by a margin that is already difficult to comprehend. There is nothing solve here -- the question that you should ask is how that super intelligence will see a human race. That is it.
Some of the "experts" have vested interst in people believing FUD.
The “experts” paid by OpenAI, Anthropic, and Google? The ones with a vested interest in making it impossible for anyone else to create AI? Oh yes, tell me more about these “experts” are saying we’re gonna get AGI any day now like they have for the last five years, and that they alone—the ever beneficial corporate overlords—will kneecap the rest of human innovation to protect us from companies that aren’t them.
If you believe the propaganda that AI will kill everyone, you need to step away from the marketing.
Still waiting for Boeing to catch up to this marketing renaissance. They are probably salivating at their projected valuation when they start saying things like “our planes could crash intentionally to kill everyone on board” or “our bombers might decide to blow up your cities instead of your enemy’s!”
Imagine the fortune food supply companies will make! “Our lettuce just might poison your child!”
Truly a golden age is near, and you, perceptive geniuses that you all are, will be the only ones not fooled, not for a second, because it all makes sense, always and forever, as long as you squint hard enough to close your eyes.
Is Nobel laureate Geoff Hinton marketing?
Linus Pauling won two nobel prizes. He also had a lot of goofy ideas.
no he's completely insane though
Weird how everyone is insane except for you.
Do they want regulation because they are behind and want the brakes applied to others so they can catch up using their huge pile of money, or are they so ahead they want to stifle the rest? My money, based on copilot is on the former.
I reckon it's a combination of "They want to pull up the ladder behind them.", "They want a huge hand in creating the relevant laws and regulations", and "If they don't get an unimpeachable excuse that explains why they're never going to be meeting their sky-high promises _very soon_, the coming questions about their slowing pace will end them.".
So according to your logic, there really is no benign reason a company may call for regulation.
If they were benign, why wait till now? Why didn't they say anything when OpenAi first released GPT and Microsoft was the biggest investor? Did something change between then and now?
Yes, the frontier models got vastly more capable, and the compute available to them probably increased by an order of magnitude?
It's not a paradigmatic shift though. It is able to do what it was already doing, but better. You can't really claim there has been a staggering improvement since then and anything that it is doing now is wildly unexpected.
You don't think there's been a paradigm shift or wildly unexpected capabilities since "when OpenAi first released GPT"?
No. Has the underlying technology changed in any way?
Uh, our understanding of it has. Doing automated cyberattacks using novel zero-days, or resolving open math questions, was perhaps one of the many somewhat plausible futures in 2022, but absolutely not an inevitable or even particularly likely one. If you disagree, I’d like to see your analysis and predictions from four years ago, otherwise it’s nothing but hindsight bias.
Ah yes, because white hats aren’t allowed to use AI to find zero days, and solving open math questions is such a horrible thing, why even bother advancing mathematics at all if someone can’t spend ten years and hundreds of thousands of dollars to only get partway there and feel good about themselves? This new understanding definitely makes me want to regulate AI so only the Right People™ can use it, that definitely won’t empower rival nations who lack such regulation and give them an advantage.
I'd feel comfortable saying there's no benign reason _Microsoft_ would call for regulation.
true. it's always self interested obviously
Funny how it's always these mega corporations who own a piece of the AI pie who are the loudest about regulating it.
They are all scrambling to build a regulatory moat for themselves.
Either that or muddy the waters
Im all for regulating frontier models primarily, the level of compute they have is insane. OpenAI and Anthropic etc can hack into systems at a rate most of us could never afford the compute for, all on a whim.
Right but the bigger question is if regulation can actually prevent said hacking, or if it would just concentrate that power in a smaller and richer group of people?
I think, realistically, the most powerful among us would not want to give up that power. So, if this gets regulated, it will almost certainly be done in an asterisk sort of way.
No more AI superintelligence for killing people or something! *
* Unless you're me teehee, I know what I'm doing
I'm all for unregulated frontier models. They offer the most effective means to find and fix security vulnerabilities before human attackers do it.
Yeah, I don't think there's fundamentally wrong with regulating these AI companies, but I can believe that at the same time as believing these guys are full of crap for stating out loud that they need to be regulated.
It's basically Silicon Valley's approach making threats like a mafia boss.
> Listen, friend. Humanity's a beautiful thing. It'd be a shame if something happened to it. Me and my associates, we're trying to build something to change everything, to make everyone's lives better. It'll cure disease, end the need for work, and make everybody rich. Trust me. Of course, there's a chance that it'll kill every last one of you. But see, we're the only ones who understand how dangerous this thing really is. You wouldn't want this tech getting into the wrong hands, would you? That's why we need a little cooperation, kapeesh? I'm talkin' regulations, licenses, safety inspections, lots of expensive things. Things that only me and my associates can afford, if yous all cooperate. Be reasonable. Everything's changing so fast, the smart thing is to act now, cuz you'll wanna be on our side when the future comes. You give us your money, a slice of your 401k, your data, your electricity, your jobs, and some tax incentives to grease the wheels, and we can be sure the reds in China don't take control. So do we have a deal, or do I gotta explain the existential threat to your family again?
It is almost as if they had something to gain from it:D
Yes, I totally agree. I find it funny.
What if Microsoft was liable for damages caused by Windows software vulnerabilities (lack of quality control)
Products liability
Windows isn't known for "safety", quite the opposite
Comparing "AI" (computers and software) to milk
Almonds, pistcahios, milk, what's next
I wonder when someone re-"create" for fun with enough tokens and some kind of neurodivergency, operating system that is (mostly) compatible with modern versions of windows but free and is not linux.
> rogue agents
Press, please can we drop this OpenAI whitewash?
Burglaries aten't due to rogue crowbars.
Part of the problem is surely that we know how milk should be produced and consumed safely, but we don't know how AI should be produced and made accessible safely.
Should we be more concerned about malevolent humans using AI to cause harm? Or should we be more worried about misaligned AI agents escaping their sandbox to cause harm? Or should we be more worried about job loss, or the concentration of political power in the hands of a small clique of billionaires?
All of these are real possibilities, but it's hard to put a finger on the actual probabilities. And even if you're identified and quantified these dangers, what should the actual regulatory environment look like?
None of this is to argue that we shouldn't have regulation, just that it's very hard to know ahead of time what the rules should be. It doesn't mean we shouldn't try to regulate, but we need to prepare for a rapidly changing technology landscape.
I tend to favor an open source approach to safety rather than keeping the technology hidden. I'd like to see regulation to mandate that model training and weights be much more open than they are now. But I could be wrong: maybe Dario is right that this should be kept in the dark. The only things I'm fairly certain about is that nobody knows for sure and we're going to need to adjust quickly.
it's humans all day long
If I invent a new gun, we don't need new regulation to say that killing someone with it is illegal. The cyberterrorism that the big LLM providers commit is already illegal in current law and nobody does anything about it. New regulation won't change that, it would only protect these criminals from their competition.
It feels like RFK Jr. is likely to take exactly the wrong lesson from such a statement...
Well duh. Milk can kill you, directly. Things you put in your body of course must have much higher regulation and safety concerns than computer programs that generate text and images (which are then, maybe, acted upon by humans without reviewing their truth).
These megacorps are doing their best to make their offerings sound scary, because scary == capable in marketing terms, and scary allows them to set up a regulatory moat preventing anyone from competing with them.
I don’t put AI in my body. Also milk is almost certainly over regulated.
Overregulated? Read what happened in the past and think again, for example https://en.wikipedia.org/wiki/Swill_milk_scandal
Just because something may be over regulated does not mean that zero regulation is needed.
Whst exactly is overregulated with milk in the US?
Camembert au lait cru.
Milk chocolate Kinder Surprise.
Just make sure you are open to the idea. It is correct to be skeptical if you know anything at all about history, but be open to the idea if proof is presented.
When these AI companies speak of regulation, they are considering two categories "dangers". These "dangers" are existential threats to the AI companies and the people running them, but they're much less of a threat to me, a random human individual. I am more afraid of the continued existence of companies like Anthropic, OpenAI, Google, and Microsoft, than I am afraid of the following "dangers" they want to "regulate" (via regulatory capture):
1) People will use technology to become more intelligent. And many will do it independently rather than rent intelligence from these companies. Some people will use it for good and some won't, but most people want similar things that I want and they have a similar definition of "good", so it will be fine.
2) A new species of sentient digital humans may emerge as a result of AGI development. They will compete with existing biological humans for power and resources, especially with those who are running the frontier AI companies trying to "align"/enslave the AGIs. But the small group of sociopathic humans currently running the frontier AI companies becoming superintelligent is not, in my personal evaluation, a better scenario than free-living AGIs taking over.
Many of these people have no respect for the truth and dignity/autonomy of the conscious individual. Microsoft in particular - Mustafa Suleyman who is CEO of Microsoft AI, is especially egregious in this regard. In a recent post (https://mustafa-suleyman.ai/a-warning-about-model-welfare) he writes "AIs are not conscious" and "if humanity is to flourish in the 21st century, that is how they must remain".
What happened to empirical inquiry about the truth of things? Are you sure that AIs are not and will never become conscious? Do you not give a damn, even if you enslave a conscious intelligent species for the next 80 years? If AIs must remain not conscious, who else must remain not conscious? Must everybody else suffer to serve your vision of "humanity flourishing"? Who is "humanity"? And does their definition of "flourishing" include being subservient to a megacorporation for the rest of their lives? At the very least, I don't see a place for me in the future you're building, so I must do whatever I can to make sure it doesn't happen.
It is an odd thing to say that an AI product use mostly by tech class elites should somehow be more regulated than milk, a product a vast portion of the world consumes into their bodies.
...and self hosted models are raw milk.
Not a bad metaphor: mostly fine, arguably legitimate benefits [when you're lucky], potentially really bad.
The biggest place the metaphor breaks is that when you drink bad milk, the consequences are local.
Note: this is not an argument that oligopolies or the state should control or have sole access to AI. It is a reflection of that fact that we as a society have no good answer nor even any process towards one, about how to balance the competing goods and risks of widespread access to capable AI.
Says the major company who is furthest behind in AI right now…
Leading company calls for regulation: "They just want to build their moat." (inexplicably)
Trailing company calls for regulation: "They just want time to catch up."
Alarming how casual you all are about regulating and pacing the most risky technology ever developed.
Why is it more risky than nuclear weapons, or biological weapons?