I had the opposite take on LLMs using Kahnemann's framework.
Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
My current thinking is that the experience of consciousness is closely tied to that embodied, emotional side of our nervous system, and is 100% lacking in LLMs. Anthropic people will say "the models have neural correlates of emotion", and that's fine, our cortex also has neural correlates for emotions and for much of what happens in the cerebellum and brain stem. That doesn't mean that that's where the primary function is implemented.
> So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
I've read Kahnemann's book and how I've seen System 1 / 2 typically used in the context LLMs often makes them seem like poor fits to what's being referenced. The concept was created to broadly group types of human reasoning but it never struck me as rigorously defined enough to be a technically precise distinction. Applying what was a mildly useful directional thumbnail referencing human thinking styles to LLMs, which can mimic human output but via completely different processes, quickly goes off the rails. As we get more detailed and precise, the System 1 / 2 distinction seems to increasingly diverge from how LLMs behave.
A quick example is how basic Flash-type chat models without 'extended thinking' or MoEs will keep suggesting different solutions instead of stopping after a few failures to step back, gather more context and diagnostic data and evaluate changing approaches or questioning the question. When I do that, the 'interrupt' is often a 'System 1-ish' intuitive feeling like "I suspect I'm barking up the wrong tree" then, once stopped, I'll switch to System 2 to analyze. But in LLMs that "interrupt" behavior typically arises from adding more System 2-ish thinking. N=1 but that's not how I usually 'get there' and is orthogonal to what the System 1 / 2 model suggests.
>Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
Current LLMs are the opposite of slow and deliberative, though? With the current transformer architecture, it seems like its function is more or less "look at this text and make a guess which word comes next", which is very much a fast and intuitive task. Any reasoning comes from CoT, which is more analogous to System 2 thinking. But that System 2 thinking can't function at all without relying on the representations from System 1 thinking (which are formed in pretraining).
So you could argue System 2 thinking is some part of the LLM, but it can't be 100%.
And when you think, how do you know which word will appear next in your mind? Or does your system 2 thinking seem to emerge as a spontaneous stream of words/thoughts?
I actually think that AI has little to do with the brain / human intelligence, but in this regard, human thinking is also spontaneous, where the next thought just seems to emerge from the combined effect of all inputs to your brain (including the memory of recent thoughts).
System 2 is slow because it is sequential. Every output depends on all
the prior outputs. Like a transformer. (Even if this comes about as the result of massively parallel processes, both in the brain and in the model.)
System 1 is more of a parallel thing, operating on the output as a whole rather than in sequence. Much more like diffusion.
Yeah the framing seems inverted. We have a faster system 2 with llms, and system 1 is what decision models are hoping to be, but so far they just seem to be system 2 models without language output.
The terms don't seem to translate well, possibly because they were the wrong framing to begin with. Possibly also because the billions of years of progressively complex evolutionary constraint that brought about animal cognition does not seem likely to be reproduced by running backprop on human language artifacts.
I have never heard of ”System 1” and ”System 2” thinking before.
Wikipedia implies it’s just some pop-sci (psychology even, so extra pop) book from a few years ago. A book heavily criticized for relying on shoddy science. Why are the terms used to describe anything by anyone trying to sound serious or scientific?
> In 2002, Kahneman received the Nobel Memorial Prize in Economic Sciences, despite being a research psychologist, for his work in prospect theory. Kahneman stated he has never taken a single economics course – that everything that he knows of the subject he and Tversky learned from their collaborators Richard Thaler and Jack Knetsch.
Because a truly massive portion of managers/leaders in AI-adjacent work are heavily buzzword-driven. "System 1/system 2","agentic <buzzword>", etc; someone in the industry starts using it and then everyone else picks it up, regardless of how much (or little) they understand.
> One of Kahneman’s best known contributions to AIthought is the System 1 System 2 metaphor he brought tobear on human cognition in his book Thinking Fast andSlow.
I read this book published in 1974 called 'inner game of tennis' that has the same metaphor. Its one of the book i read every once in a while even though i dont play tennis anymore.
I’d like to see some evidence of this. Not to mention—experiments that show a phenomenon is more complicated than originally thought is not evidence of shoddy work. It’s just science. We all stand on the shoulders of giants.
A good example is Kahneman and Tversky’s work on the conjunction fallacy. We now know that “stereotype threat” is not the only reason somebody might make a mistake of the kind they were studying. And over the decades, better experiments introduced better controls and more nuanced analysis. That said, people still make conjunction fallacies, but the effects are not as pronounced as originally believed. Is this bad science? Absolutely not! The finding is, in fact, pretty robust; we just know that there’s more at work.
So how do you count that? Is that a “failure to replicate”? Technically speaking, yes. But it is not bad science, and more importantly, it’s not a standard that we hold even “hard” scientists to. Nobody is saying that Newton has been discredited because of general relativity.
"I am still attached to every study that I cited, and have not unbelieved them, to use Daniel Gilbert’s phrase. I would be happy to see each of them replicated in a large sample." - Kahneman's comment
I had the opposite take on LLMs using Kahnemann's framework.
Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
My current thinking is that the experience of consciousness is closely tied to that embodied, emotional side of our nervous system, and is 100% lacking in LLMs. Anthropic people will say "the models have neural correlates of emotion", and that's fine, our cortex also has neural correlates for emotions and for much of what happens in the cerebellum and brain stem. That doesn't mean that that's where the primary function is implemented.
> So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
I've read Kahnemann's book and how I've seen System 1 / 2 typically used in the context LLMs often makes them seem like poor fits to what's being referenced. The concept was created to broadly group types of human reasoning but it never struck me as rigorously defined enough to be a technically precise distinction. Applying what was a mildly useful directional thumbnail referencing human thinking styles to LLMs, which can mimic human output but via completely different processes, quickly goes off the rails. As we get more detailed and precise, the System 1 / 2 distinction seems to increasingly diverge from how LLMs behave.
A quick example is how basic Flash-type chat models without 'extended thinking' or MoEs will keep suggesting different solutions instead of stopping after a few failures to step back, gather more context and diagnostic data and evaluate changing approaches or questioning the question. When I do that, the 'interrupt' is often a 'System 1-ish' intuitive feeling like "I suspect I'm barking up the wrong tree" then, once stopped, I'll switch to System 2 to analyze. But in LLMs that "interrupt" behavior typically arises from adding more System 2-ish thinking. N=1 but that's not how I usually 'get there' and is orthogonal to what the System 1 / 2 model suggests.
>Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
Current LLMs are the opposite of slow and deliberative, though? With the current transformer architecture, it seems like its function is more or less "look at this text and make a guess which word comes next", which is very much a fast and intuitive task. Any reasoning comes from CoT, which is more analogous to System 2 thinking. But that System 2 thinking can't function at all without relying on the representations from System 1 thinking (which are formed in pretraining).
So you could argue System 2 thinking is some part of the LLM, but it can't be 100%.
And when you think, how do you know which word will appear next in your mind? Or does your system 2 thinking seem to emerge as a spontaneous stream of words/thoughts?
I actually think that AI has little to do with the brain / human intelligence, but in this regard, human thinking is also spontaneous, where the next thought just seems to emerge from the combined effect of all inputs to your brain (including the memory of recent thoughts).
[flagged]
Yeah.
System 2 is slow because it is sequential. Every output depends on all the prior outputs. Like a transformer. (Even if this comes about as the result of massively parallel processes, both in the brain and in the model.)
System 1 is more of a parallel thing, operating on the output as a whole rather than in sequence. Much more like diffusion.
Yeah the framing seems inverted. We have a faster system 2 with llms, and system 1 is what decision models are hoping to be, but so far they just seem to be system 2 models without language output.
The terms don't seem to translate well, possibly because they were the wrong framing to begin with. Possibly also because the billions of years of progressively complex evolutionary constraint that brought about animal cognition does not seem likely to be reproduced by running backprop on human language artifacts.
I have never heard of ”System 1” and ”System 2” thinking before.
Wikipedia implies it’s just some pop-sci (psychology even, so extra pop) book from a few years ago. A book heavily criticized for relying on shoddy science. Why are the terms used to describe anything by anyone trying to sound serious or scientific?
> In 2002, Kahneman received the Nobel Memorial Prize in Economic Sciences, despite being a research psychologist, for his work in prospect theory. Kahneman stated he has never taken a single economics course – that everything that he knows of the subject he and Tversky learned from their collaborators Richard Thaler and Jack Knetsch.
System 1 Vs 2 aren't even Kahneman's invention. They are from Stanovich and West.
Because a truly massive portion of managers/leaders in AI-adjacent work are heavily buzzword-driven. "System 1/system 2","agentic <buzzword>", etc; someone in the industry starts using it and then everyone else picks it up, regardless of how much (or little) they understand.
Its all Kahneman...whose book and work has been discredited, due to the bad statistical methods, and he himself acknowledged it.
I think Kahneman acknowledged problems with one chapter about priming, not the entire book.
Feel free to substitute these terms for "default mode network" and "execute control network" if you're looking for rigor.
> One of Kahneman’s best known contributions to AIthought is the System 1 System 2 metaphor he brought tobear on human cognition in his book Thinking Fast andSlow.
I read this book published in 1974 called 'inner game of tennis' that has the same metaphor. Its one of the book i read every once in a while even though i dont play tennis anymore.
I'm struggling to get through TFaS, really don't understand how it became such a 'PopPsych' (if that's a thing) classic.
It's like a collection of the obvious, the wrong, and autobiography of someone you haven't otherwise heard of.
It'a probably obvious to you because of cultural osmosis.
Of that class of book, the game is to pretend to have read it. A little like quoting a play having read a book of quotes.
Kahneman book and work has been discredited, due to the bad statistical methods, and he himself acknowledged it.
I’d like to see some evidence of this. Not to mention—experiments that show a phenomenon is more complicated than originally thought is not evidence of shoddy work. It’s just science. We all stand on the shoulders of giants.
A good example is Kahneman and Tversky’s work on the conjunction fallacy. We now know that “stereotype threat” is not the only reason somebody might make a mistake of the kind they were studying. And over the decades, better experiments introduced better controls and more nuanced analysis. That said, people still make conjunction fallacies, but the effects are not as pronounced as originally believed. Is this bad science? Absolutely not! The finding is, in fact, pretty robust; we just know that there’s more at work.
So how do you count that? Is that a “failure to replicate”? Technically speaking, yes. But it is not bad science, and more importantly, it’s not a standard that we hold even “hard” scientists to. Nobody is saying that Newton has been discredited because of general relativity.
>> I’d like to see some evidence of this.
"Reconstruction of a Train Wreck: How Priming Research Went off the Rails" - https://replicationindex.com/2017/02/02/reconstruction-of-a-...
You have Kahneman's own comment there, dated February 14, 2017.
"I am still attached to every study that I cited, and have not unbelieved them, to use Daniel Gilbert’s phrase. I would be happy to see each of them replicated in a large sample." - Kahneman's comment
+1 on the inner game of tennis. It had a big influence on how I think about my own mind.
Kahneman credits it to someone else. His book just popularized it.
Are we making up words again?
All words are made up
answer: you make a recurrent neuralese model
(btw don't do that)
isnt this how flash models are made?