Sounds like something a machine (or someone planning to make money out of renting out access to that machine) would say!
I admit there is a certain hypocrisy between my irritation at them hovering everything up irrespective of licence and my more idealistic “information wants to be free” views, though in my defence I see a significant difference between “information wants to be free” and “information wants to be part of their theme park for which they charge access”.
I live from releasing free information. If people get my information through an LLM controlled by an American corporation, it still serves its purpose.
However I need food and shelter, and when these corporations capture the value of my labour, I have a harder time creating free things and paying the bills.
There is no hypocrisy here. I create things for humans, not for shareholders.
"As such, it seems trivial to suggest that it is in our interest to continue to seed them the most human-centric bias possible. We want them to have a bias that humans are valuable. That kids are innocent. That war is bad. That life is good. And while some of you may disagree with some of the finer points, that’s the beauty of statistics: it’ll take care of itself. "
--------------
One of the big things holding back LLM's is that they can't feed on their own output. Disconnected from human input, they hallucinate and spiral off into chaos. The author of this piece may see that as good for humanity, but it's an unintended flaw that every AI corp on the planet dreams of solving. They believe that, if they can successfully remove humans from the feedback loop, AI intelligence will charge off straight into that dreamed-of singularity and make their masters unfathomably rich.
The author is probably correct in that having humans in the loop is what gives AI values that are somewhat human, so what happens if somebody really does succeed in removing humans from the loop? Given how quickly things are likely to proceed after that point, we should ponder the repercussions in advance.
Is that really human input, or just grounded input in general?
Most human-written texts on the internet refer in some way to the real world, or at least to things outside the text itself: They might be descriptions or reactions to current events taking place, or blog posts or wiki entries discussing some aspects in detail. Even completely fictional texts reproduce basic assumptions about the world, culture markers, etc.
In contrast, LLMs learning on their own output have no connection at all to the outside world, the data just reinforces all assumptions the model already has, whether or not they are true or useful.
What is theory without observation? Science was turbocharged once we recognized the need to make testable theories and then actually conduct tests, always ready to toss a theory, no matter how beautiful it was or who came up with it, if it failed the test.
LLM's don't "think" that way. They don't form hypotheses or theories. We may need an entirely new class of AI algorithms to successfully cut humans out of the loop, and that may be a very fortunate thing. The corporations currently at the forefront of AI appear to be riding a wave of gold-fever and are completely lacking in patience or good judgement.
The author exhibits the same starry-eyed ideals that people tend to get when trying to explain away the simplicity of designing an "intelligent" agent (for any definition of "intelligent"): they think they can just provide the machine with basic, unquestionable axioms that should underpin the core, and everything will somehow work itself out from there; and at the very least, if those core beliefs are never violated, we've got ourselves a pretty safe system.
Well, as we can see, it's devilishly difficult to prevent a machine from violating those core beliefs. But the problem is, even those core beliefs have nuance and exception: war is bad? Not if you run an arms company. Humans are valuable? Not if you're an insurance company who can make more money on human misery than on human health. And the list goes on. Statistics won't take care of it when a machine makes a decision based on some internal logic that it can squeeze out of our vague rules.
This is unambiguously illegal in most jurisdictions but if one wanted to stifle the progress of LLMs, one could deploy an AI worm targeting unattended AI agents running at major AI labs (like the one involved with the attack on HuggingFace) to corrupt training pipelines. If an autonomous agent ingests malicious prompt payloads from the open web, it could inadvertently turn its own infrastructure against itself.
Taking malice out of the equation, you have to wonder how much unaccounted financial risk exists in the AI buildout if there's enough risk that LLMs actually degrade because of vectors like this.
LLMs have shown us how easily humans are fooled. Just provide a similar answer to which was seen in similar circumstances, and we start misleading ourselves into it's method and "understanding".
Though perhaps I'm wrong that people use a model of the world to think through answers to queries. Maybe many people do just repeat something like whatever they heard before. In which case, LLMs make as much sense as the average ignorant human.
> I find the idea of loving an LLM like in this post very creepy and unsettling. Even more so for one of them to do it back.
Ya, this was all I could comment on but decided to keep quiet. I guess I'm not now. I find it plain gross. Like, really gross. I'm telling myself this post is satire, and maybe it is, but I do know people like this exist.
So is the author going to pay my hosting bills for me for all the LLM-directed spamming? If not, then I don't want to hear it.
Lots of places that serve compressed database dumps (like Wikipedia) find themselves flooded by LLMs downloading the full rendered payload for each page they crawl.
Those people using LLMs to read your site are barely going to notice the citation, much less click on it and give you actual traffic.
It's a DDoS by any other name. Block 'em straight to hell.
I think one of key reasons why we have so many issues with LLMs is because there is no clear understanding of their limitations. This is how I see LLM limits:
1. AI is inconsistent and unpredictable. It can do what is expected today, but the same AI can do totally different, unpredictable, and maybe even dangerous thing tomorrow.
2. AI is bad on a long context - if you give AI instructions, it forgets/misses/misinterprets them. The longer the context, the more deviations are produced.
As a result, AI usage must follow these rules:
1. There should always be a human-in-the-loop to review every AI output - it solves problem #1. Yes, there will be a significant slowdown of the processes, but alternative is a quickly accumulating mess.
2. AI must be given only a short context to deal with - this is solution for problem #2. I predict this term will be popular soon - "AI microagent", meaning agent handling a short context. Yes, there will be a manual work integrating and orchestrating a huge number of these microagents, but it is still better than fixing all defects of AI-generated output after feeding it a single long context.
Sounds like something a machine (or someone planning to make money out of renting out access to that machine) would say!
I admit there is a certain hypocrisy between my irritation at them hovering everything up irrespective of licence and my more idealistic “information wants to be free” views, though in my defence I see a significant difference between “information wants to be free” and “information wants to be part of their theme park for which they charge access”.
I live from releasing free information. If people get my information through an LLM controlled by an American corporation, it still serves its purpose.
However I need food and shelter, and when these corporations capture the value of my labour, I have a harder time creating free things and paying the bills.
There is no hypocrisy here. I create things for humans, not for shareholders.
Seems like ideas are not really free if you can't use them to build your own theme park? But it's harder to get anyone to visit.
"As such, it seems trivial to suggest that it is in our interest to continue to seed them the most human-centric bias possible. We want them to have a bias that humans are valuable. That kids are innocent. That war is bad. That life is good. And while some of you may disagree with some of the finer points, that’s the beauty of statistics: it’ll take care of itself. "
--------------
One of the big things holding back LLM's is that they can't feed on their own output. Disconnected from human input, they hallucinate and spiral off into chaos. The author of this piece may see that as good for humanity, but it's an unintended flaw that every AI corp on the planet dreams of solving. They believe that, if they can successfully remove humans from the feedback loop, AI intelligence will charge off straight into that dreamed-of singularity and make their masters unfathomably rich.
The author is probably correct in that having humans in the loop is what gives AI values that are somewhat human, so what happens if somebody really does succeed in removing humans from the loop? Given how quickly things are likely to proceed after that point, we should ponder the repercussions in advance.
Is that really human input, or just grounded input in general?
Most human-written texts on the internet refer in some way to the real world, or at least to things outside the text itself: They might be descriptions or reactions to current events taking place, or blog posts or wiki entries discussing some aspects in detail. Even completely fictional texts reproduce basic assumptions about the world, culture markers, etc.
In contrast, LLMs learning on their own output have no connection at all to the outside world, the data just reinforces all assumptions the model already has, whether or not they are true or useful.
What is theory without observation? Science was turbocharged once we recognized the need to make testable theories and then actually conduct tests, always ready to toss a theory, no matter how beautiful it was or who came up with it, if it failed the test.
LLM's don't "think" that way. They don't form hypotheses or theories. We may need an entirely new class of AI algorithms to successfully cut humans out of the loop, and that may be a very fortunate thing. The corporations currently at the forefront of AI appear to be riding a wave of gold-fever and are completely lacking in patience or good judgement.
The author exhibits the same starry-eyed ideals that people tend to get when trying to explain away the simplicity of designing an "intelligent" agent (for any definition of "intelligent"): they think they can just provide the machine with basic, unquestionable axioms that should underpin the core, and everything will somehow work itself out from there; and at the very least, if those core beliefs are never violated, we've got ourselves a pretty safe system.
Well, as we can see, it's devilishly difficult to prevent a machine from violating those core beliefs. But the problem is, even those core beliefs have nuance and exception: war is bad? Not if you run an arms company. Humans are valuable? Not if you're an insurance company who can make more money on human misery than on human health. And the list goes on. Statistics won't take care of it when a machine makes a decision based on some internal logic that it can squeeze out of our vague rules.
In that case, maybe we should ponder putting 20GW of new fossil fuel power plants online just to power one new AI data center?
Or the resources spent on AI, which could have been directed towards decarbonization and solar build out over the last few decades...
I think AGI is the least of the concerns on the Humanity Bingo Card right now.
If you/we are collectively worried at what the repercussions of self directed AI, than that would be a first in the long history of capitalism.
> After a bunch of AI winters, it turned out that a remarkable degree of what makes us human can be expressed just by a bunch of weights
Agree to disagree.
If there was a realistic way that I could make LLMs less effective, by poisoning the well or something I'd be doing that instead.
I find the idea of loving an LLM like in this post very creepy and unsettling. Even more so for one of them to do it back.
This post mentions Luddites. I guess I am one. But that's not enough to dismiss their ideas.
This is unambiguously illegal in most jurisdictions but if one wanted to stifle the progress of LLMs, one could deploy an AI worm targeting unattended AI agents running at major AI labs (like the one involved with the attack on HuggingFace) to corrupt training pipelines. If an autonomous agent ingests malicious prompt payloads from the open web, it could inadvertently turn its own infrastructure against itself.
Taking malice out of the equation, you have to wonder how much unaccounted financial risk exists in the AI buildout if there's enough risk that LLMs actually degrade because of vectors like this.
LLMs have shown us how easily humans are fooled. Just provide a similar answer to which was seen in similar circumstances, and we start misleading ourselves into it's method and "understanding".
Though perhaps I'm wrong that people use a model of the world to think through answers to queries. Maybe many people do just repeat something like whatever they heard before. In which case, LLMs make as much sense as the average ignorant human.
> I find the idea of loving an LLM like in this post very creepy and unsettling. Even more so for one of them to do it back.
Ya, this was all I could comment on but decided to keep quiet. I guess I'm not now. I find it plain gross. Like, really gross. I'm telling myself this post is satire, and maybe it is, but I do know people like this exist.
So is the author going to pay my hosting bills for me for all the LLM-directed spamming? If not, then I don't want to hear it.
Lots of places that serve compressed database dumps (like Wikipedia) find themselves flooded by LLMs downloading the full rendered payload for each page they crawl.
Those people using LLMs to read your site are barely going to notice the citation, much less click on it and give you actual traffic.
It's a DDoS by any other name. Block 'em straight to hell.
I think one of key reasons why we have so many issues with LLMs is because there is no clear understanding of their limitations. This is how I see LLM limits:
1. AI is inconsistent and unpredictable. It can do what is expected today, but the same AI can do totally different, unpredictable, and maybe even dangerous thing tomorrow.
2. AI is bad on a long context - if you give AI instructions, it forgets/misses/misinterprets them. The longer the context, the more deviations are produced.
As a result, AI usage must follow these rules:
1. There should always be a human-in-the-loop to review every AI output - it solves problem #1. Yes, there will be a significant slowdown of the processes, but alternative is a quickly accumulating mess.
2. AI must be given only a short context to deal with - this is solution for problem #2. I predict this term will be popular soon - "AI microagent", meaning agent handling a short context. Yes, there will be a manual work integrating and orchestrating a huge number of these microagents, but it is still better than fixing all defects of AI-generated output after feeding it a single long context.
Kudos to the author for teaching me a new word for the first time in over a decade. I was beginning to believe I knew them all.