When I read titles like these, I expect a relatively low value article.
This has probably been the highest value such article I've read in years. You've phrased item 3 better than I have been trying to for years after my own journey in realizing this.
Dang these are really not at all common advice/reflections.
> 1. The importance of understanding your own incentive structure, and not believing everything you think.
I think certainly your actions are maybe that paramount to the world because it has so many actors, but they are certainly important to you. The actions that we take on a daily basis nudge nudge nudge us to becoming something else. I think its something worth focusing on but not to the point it makes you anxious.
> The importance of understanding your own incentive structure, and not believing everything you think.
There is an implication from this one that the article doesn't get into, which is whether you've picked a strategy of fail-safe or fail-disastrously. Once someone makes the leap to realising that their own mind is unreliable and it is entirely possible that their thoughts and memories are just wrong they have to start seriously asking what happens if they set out to do something, achieve it and then discover that it was a bad idea from the start. It is one of those ideas that sounds very simple to incorporate into your life and turns out to be hard.
IMO the people who tend towards pacifism and non-violence have internalised the idea because the violent ones just do so much more harm than good while thinking they are virtuous.
> The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
He's technically right, but our instincts are not suitable for a complex 8-billion person global economy. We have a bunch of instincts calibrated so that a group of ~200 monkeys probably raises >200 descendants on average, just. We need some sort of principle where we expect people to take evidence-based action rather than always just picking what feels good in the moment.
It doesn't have to be a reason-emotion dichotomy, but there has to be something where effective people > destructive people in situations where our instincts don't trigger or push people in predictably bad directions.
> 3. The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
Guessing: Essentially that's psychology. And Human psychology is itself emergent behavior cannot be directly traced back to neuroscience. Logic, that is another thing, which primarily is used for understanding and reconstruct retainable after emotional experiences.
It's not psychology, it's just wrong. You can point to the parts of the brain (more accurately the body and brain) that handle reason and emotions, and they're physically distinct.
Also, reason is essentially a combination of pattern recognition and abstraction, and emotion is a set of spontaneous limbic responses.
Which is not to say there's no relationship between them, or that the grey area where they overlap isn't unusually interesting.
But it's a wild overreach to say the distinction is a cultural construct and not grounded in (largely) mammalian biology.
Points 1 and 2 are also questionable. They're not novel opinions, they have philosophical names (more or less consequentialism vs deontology, with some wrinkles), and they can be - and have been - hotly debated.
I don't see them as evidence of maturity or of immaturity, more of a considered moral position that not everyone agrees with.
Being able to detach from your thinking while you are thinking, that's difficult and indeed very useful. I found that more often in llm than human beings.
The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
This is a nice, thoughtful piece, but the final point oversteps while chasing a noble heuristic, I think. To touch on a few fields off the top of my head:
1. All serious cognitive philosophy since at least Kant has supported the idea of separate faculties -- in his case, specifically the four chambers [`--` quantitative sensitivity, `-+` qualitative perceptivity, `+-` causal judgement, `++` modal reason].[1] This is also at the root of the induction vs. deduction [vs. abduction] debate, which had been raging for at least 2,200 years before Kant.
2. Even our pre-gallilean understanding of neuroscience arrived at the same conclusion, though in hilariously abstract terms: 'fast thinking' vs. 'slow thinking'. Popularized primarily by a popsci book (?) that frames it like this: "System 1" is fast, instinctive and emotional; "System 2" is slower, more deliberative, and more logical.
3. Psychologically the clearest paradigm is Freud's `- Id`/`+ Ego`/`* Super-Ego`, which is not nearly as discredited as his famous theories about parental lust would imply. If you asked your psychiatrist(/local research psychologist) about it today, they'd ofc use less medieval words for the same concepts -- roughly 'unconscious', 'cognition', and 'executive', in order.[2]
4. In cognitive science more broadly, a lot of the binaries discussed on Wikipedia's Mathematical Model article[3] are getting at this delineation.
5. Eastern philosophy has plenty of stuff on this dichotomy; in fact, I'd argue their discussion of it is predicated upon its existence and the various harmful states it can create if left unexamined. See, for example, https://en.wikipedia.org/wiki/Buddhist_logico-epistemology
On a 'logical' level, the case for a separation of at least two types of thinking seems clear: any cognitive activity can either call the 'lightweight' model to guess what's happening (and be right 95% of the time), or call the 'superheavy' model to iteratively reason through webs of beliefs.
Sure, any call probably uses more than 0.0001% of both so they're not mutually exclusive, but I think the qualitative diffence is clear.
The sentiment is true in a bit more subtle of a way, I will admit... It's hard to articulate in brief (/at all), but I've become fascinated with the way the presence of the higher faculties sometimes seems to completely transform the lower ones, unlocking whole new affordances. Like the existence of a new structure of a higher level of complexity completely changing the behavior of nearby cellular automata. If you'll excuse the anthropomorphization of abstract patterns for moment: it reminds me of how a competent leader can unlock whole new affordances for the individuals on their team without actually directly changing the individuals themselves.
Like, consider how we have have tons of both instincts & intuitions related to language use -- speaking, writing, listening, etc. Chimpanzees & bonobos share many of our incredibly advanced intuitive abilities (e.g. manual dexterity, social 'theory of mind'/empathy, etc.), but they are categorically unable to pick those up. In this way the super-ego didn't add funtionality to our species, and it didn't even really multiply it geometrically (?) -- it seems much closer to an exponentiation of capability.[4]
TL;DR: Acknowledging the fallibility and inherently-subconscious basis of reason is a noble goal, but abandoning the Idea altogether is misleading. IMHO!
[4]: Here's to hoping that this fourth rung is the last one, at least for a while! Cause if there's a fifth one, and AI discovers and produces it... yikes. Truely alien ASI would follow, rather than just ASI-as-huge-but-fundamentally-grokkable-AGI.)
> The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
Hard disagree with part of this. Reason is an intellectual construct, that has a basis in logic. (Neuroscience not finding a division: sure. Just like electrical engineering doesn't see a dichotomy between Microsoft Word and Firefox.)
Reasoning isn't separated from emotion because reasoning needs inputs. Some inputs are secondary: the result of other reasoning. Some are non-negotiable facts from the world. But some primary inputs that go into reasoning are determined by values and desires. E.g. "I feel it would be nice obtain such and such a result in the world" (hard reasoning on how to get it ensues).
I don’t have the link right now, but I recall a study of some sort that examined brain activity and decision making. The “emotion” part lit up first, then the “reasoning” part.
Essentially, we make decisions based on emotion, then come up with an explanation for the decision that was already made. We form a memory that says the explanation was why we made the decision, but it was actually the emotion. In other words, by default, we’re rationalizing, not rational.
I think the issue is precisely the rationalising part.
Let's assume I have pre-decided something: if the fruit weighs over 100g, I'll buy it, otherwise I won't.
I can then measure and find out. My emotions -- based on looking at the fruit -- may mislead me here.
Now what about "likely to weigh more than 100g"?
Here it's really easy to lie to yourself.
But if you set conditions for what counts as likely before you know what they'll be used for, and you stick to them, you can make a decision based on reason.
There is no way to decide 100g is the threshold divorced from emotion, however -- unless it's rationally derived from something else emotionally based.
The "rationalisation" bit comes from us not making decisions for the reasons we think we are -- or in the way we claim to be.
When I read titles like these, I expect a relatively low value article.
This has probably been the highest value such article I've read in years. You've phrased item 3 better than I have been trying to for years after my own journey in realizing this.
Thank you.
Dang these are really not at all common advice/reflections.
> 1. The importance of understanding your own incentive structure, and not believing everything you think.
I think certainly your actions are maybe that paramount to the world because it has so many actors, but they are certainly important to you. The actions that we take on a daily basis nudge nudge nudge us to becoming something else. I think its something worth focusing on but not to the point it makes you anxious.
Thanks for the thoughtful piece.
> The importance of understanding your own incentive structure, and not believing everything you think.
There is an implication from this one that the article doesn't get into, which is whether you've picked a strategy of fail-safe or fail-disastrously. Once someone makes the leap to realising that their own mind is unreliable and it is entirely possible that their thoughts and memories are just wrong they have to start seriously asking what happens if they set out to do something, achieve it and then discover that it was a bad idea from the start. It is one of those ideas that sounds very simple to incorporate into your life and turns out to be hard.
IMO the people who tend towards pacifism and non-violence have internalised the idea because the violent ones just do so much more harm than good while thinking they are virtuous.
> The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
He's technically right, but our instincts are not suitable for a complex 8-billion person global economy. We have a bunch of instincts calibrated so that a group of ~200 monkeys probably raises >200 descendants on average, just. We need some sort of principle where we expect people to take evidence-based action rather than always just picking what feels good in the moment.
It doesn't have to be a reason-emotion dichotomy, but there has to be something where effective people > destructive people in situations where our instincts don't trigger or push people in predictably bad directions.
Greybeard is not sexist.
One of rare articles here that is actually worth reading.
1st one requires that you actually care
2nd one relevance depends on what are you doing, not everyone works on airplane software
3rd one, novelty for me
Here, you can find https://en.wikipedia.org/wiki/Theory_of_constructed_emotion to be useful in articulating the following.
> 3. The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
Guessing: Essentially that's psychology. And Human psychology is itself emergent behavior cannot be directly traced back to neuroscience. Logic, that is another thing, which primarily is used for understanding and reconstruct retainable after emotional experiences.
It's not psychology, it's just wrong. You can point to the parts of the brain (more accurately the body and brain) that handle reason and emotions, and they're physically distinct.
Also, reason is essentially a combination of pattern recognition and abstraction, and emotion is a set of spontaneous limbic responses.
Which is not to say there's no relationship between them, or that the grey area where they overlap isn't unusually interesting.
But it's a wild overreach to say the distinction is a cultural construct and not grounded in (largely) mammalian biology.
Points 1 and 2 are also questionable. They're not novel opinions, they have philosophical names (more or less consequentialism vs deontology, with some wrinkles), and they can be - and have been - hotly debated.
I don't see them as evidence of maturity or of immaturity, more of a considered moral position that not everyone agrees with.
3. Emotion motivates everything that we do. It's just a question of what emotions are you considering and what emotions are you prioritizing.
Reasoning is taking all emotions into context and focusing on actions that align with certain emotions.
Being able to detach from your thinking while you are thinking, that's difficult and indeed very useful. I found that more often in llm than human beings.
> Monocausal determinism is an illusion, and largely does not exist outside of computer debugging.
Also: quite often does not exist inside computer debugging, either.
1. All serious cognitive philosophy since at least Kant has supported the idea of separate faculties -- in his case, specifically the four chambers [`--` quantitative sensitivity, `-+` qualitative perceptivity, `+-` causal judgement, `++` modal reason].[1] This is also at the root of the induction vs. deduction [vs. abduction] debate, which had been raging for at least 2,200 years before Kant.
2. Even our pre-gallilean understanding of neuroscience arrived at the same conclusion, though in hilariously abstract terms: 'fast thinking' vs. 'slow thinking'. Popularized primarily by a popsci book (?) that frames it like this: "System 1" is fast, instinctive and emotional; "System 2" is slower, more deliberative, and more logical.
3. Psychologically the clearest paradigm is Freud's `- Id`/`+ Ego`/`* Super-Ego`, which is not nearly as discredited as his famous theories about parental lust would imply. If you asked your psychiatrist(/local research psychologist) about it today, they'd ofc use less medieval words for the same concepts -- roughly 'unconscious', 'cognition', and 'executive', in order.[2]
4. In cognitive science more broadly, a lot of the binaries discussed on Wikipedia's Mathematical Model article[3] are getting at this delineation.
5. Eastern philosophy has plenty of stuff on this dichotomy; in fact, I'd argue their discussion of it is predicated upon its existence and the various harmful states it can create if left unexamined. See, for example, https://en.wikipedia.org/wiki/Buddhist_logico-epistemology
On a 'logical' level, the case for a separation of at least two types of thinking seems clear: any cognitive activity can either call the 'lightweight' model to guess what's happening (and be right 95% of the time), or call the 'superheavy' model to iteratively reason through webs of beliefs.
Sure, any call probably uses more than 0.0001% of both so they're not mutually exclusive, but I think the qualitative diffence is clear.
The sentiment is true in a bit more subtle of a way, I will admit... It's hard to articulate in brief (/at all), but I've become fascinated with the way the presence of the higher faculties sometimes seems to completely transform the lower ones, unlocking whole new affordances. Like the existence of a new structure of a higher level of complexity completely changing the behavior of nearby cellular automata. If you'll excuse the anthropomorphization of abstract patterns for moment: it reminds me of how a competent leader can unlock whole new affordances for the individuals on their team without actually directly changing the individuals themselves.
Like, consider how we have have tons of both instincts & intuitions related to language use -- speaking, writing, listening, etc. Chimpanzees & bonobos share many of our incredibly advanced intuitive abilities (e.g. manual dexterity, social 'theory of mind'/empathy, etc.), but they are categorically unable to pick those up. In this way the super-ego didn't add funtionality to our species, and it didn't even really multiply it geometrically (?) -- it seems much closer to an exponentiation of capability.[4]
TL;DR: Acknowledging the fallibility and inherently-subconscious basis of reason is a noble goal, but abandoning the Idea altogether is misleading. IMHO!
[1]: http://staffweb.hkbu.edu.hk/ppp/ksp1/toc.html
[2]: https://en.wikipedia.org/wiki/Cognitive_therapy#Basis
[3]: https://en.wikipedia.org/wiki/Mathematical_model#Classificat...
[4]: Here's to hoping that this fourth rung is the last one, at least for a while! Cause if there's a fifth one, and AI discovers and produces it... yikes. Truely alien ASI would follow, rather than just ASI-as-huge-but-fundamentally-grokkable-AGI.)
The PopSci book you're thinking of is "Thinking, Fast and Slow" by Daniel Kahneman
> The dichotomy between reason and emotion is a cultural construct, and neither grounded in neuroscience nor in logic.
Hard disagree with part of this. Reason is an intellectual construct, that has a basis in logic. (Neuroscience not finding a division: sure. Just like electrical engineering doesn't see a dichotomy between Microsoft Word and Firefox.)
Reasoning isn't separated from emotion because reasoning needs inputs. Some inputs are secondary: the result of other reasoning. Some are non-negotiable facts from the world. But some primary inputs that go into reasoning are determined by values and desires. E.g. "I feel it would be nice obtain such and such a result in the world" (hard reasoning on how to get it ensues).
I don’t have the link right now, but I recall a study of some sort that examined brain activity and decision making. The “emotion” part lit up first, then the “reasoning” part.
Essentially, we make decisions based on emotion, then come up with an explanation for the decision that was already made. We form a memory that says the explanation was why we made the decision, but it was actually the emotion. In other words, by default, we’re rationalizing, not rational.
I think the issue is precisely the rationalising part.
Let's assume I have pre-decided something: if the fruit weighs over 100g, I'll buy it, otherwise I won't.
I can then measure and find out. My emotions -- based on looking at the fruit -- may mislead me here.
Now what about "likely to weigh more than 100g"?
Here it's really easy to lie to yourself.
But if you set conditions for what counts as likely before you know what they'll be used for, and you stick to them, you can make a decision based on reason.
There is no way to decide 100g is the threshold divorced from emotion, however -- unless it's rationally derived from something else emotionally based.
The "rationalisation" bit comes from us not making decisions for the reasons we think we are -- or in the way we claim to be.