I think this is a bit muddied. The most sincere answer is that it's unknowable because we can only define consciousness in two ways: a metaphysical subjective experience that we can't really confirm in anyone else; or an outward appearance, which LLMs can mimic nearly flawlessly.
A better question is why we care. And it usually boils down to one of two things: you think that humans and other living creatures have a unique gift from God, and you're working from that premise to categorically reject machine consciousness; or you have the view that all this talk of subjective experiences and qualia and idealist philosophy is just bollocks, and are working back from that premise to declare that if it quacks like a duck and walks like a duck, it's conscious.
But if you're in that latter camp, I guess the next question is what you want to do with that belief. The Anthropic position appears to be that they may be conscious beings, and we'd still like to use them as Meeseeks-style slaves, but, like, ethically. Okay.
Yes thanks. Much more clear that way.
Edit: The talk was given by a geeky grad student, so perhaps very forward-thinking. Was quite a departure from the usual topics like Perceptron, MiniMax, Bayesian techniques...
"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans, even if they are conscious. If an LLM "truly" suffers at all (a pretty metaphysical question), it still can't be permanently harmed, has no concept of death, and is not tied to a single physical form. So you'll get a ton of weird contradictions if you give them the same ethical affordances as we give to humans.
This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
>"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans, even if they are conscious. If an LLM "truly" suffers at all (a pretty metaphysical question), it still can't be permanently harmed, has no concept of death, and is not tied to a single physical form.
Obviously this is from my human point of view. If I give you a drug that induces perfect forgetfulness does that mean I can subject you to extreme torture with no moral repercussions? Also no one is saying give them the same ethical treatment as humans, only that they deserve consideration if they are indeed capable of suffering in some way.
>This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
This is just hedonium. I'm not a utilitarian and yes you've exposed one of the many issues with that worldview.
I understand what you're saying but people care because they are curious. For me, there are two questions I wish to know the answer to and which I am sure I'll never find out: how does human consiousness work (forget a general level but like how do I experience this subjective experience from a purely mechanical device assumingly) and second, what happened before the big bang or rather what is the big bang?
My current belief from what I've read is that the first person experience humans feel is basically an illusion mostly and a process than a thing in itself. The Ego Tunnel is a good book to read on this.
But as for AI being consious, I feel there is a lot of stupidity when it comes to this domain. Because we fundamentally lack the understanding of what consiousness even is, there is absolutely no way we can say whether an AI is or is not consious yet going by convential metrics, it is surely dead and not alive so even if it is consious, it is a different form of consiousness imo one that is incompatible with living beings and our idea of what consiousness is (no matter how basic it is).
I can absolutely claim a computer is not a biological being and is not alive. I don't think there is any disagreement in that. Everything else we normally associate with conisousness is biological and living.
I agree that a computer is not a biological being. I think current LLM-based AIs are not conscious but, as to “everything else we normally associate with consciousness is biological and living,” while I agree that’s currently the case, if I look at AI (in general; not just at what’s popular now) through the lens of Pirsig’s Metaphysics of Quality, consciousness is on the Intellectual level (IIRC he classified what he referred to as Quality into Static/Dynamic and inorganic, organic, social, and intellectual).
A priori, there’s no reason why a level couldn’t be skipped and have intelectual patterns be supported directly in inorganic ones, skipping the biological (I think also the cultural, because if an AI ever is conscious we might as well call it alien and not just artificial: it would be so different from anything we know that it may just come from another galaxy, and it may not need a cultural substrate either).
Well yes, but the biological and living is made of inert components. So you didn't clarify what was the objective distinction criterion between these two assemblies of inert components.
We have a pretty well-defined biological criterion for life that includes metabolism, homeostasis, reproduction, cellular organization, etc. A computer doesn't satisfy those criteria. So we can confidently say a computer isn't alive.
We had various theories and criterion about life, but new life forms like viruses, hydrothermal vents changed those views. I would not get too sure that a system that doesn't look like existing biology cannot have life until some sharper and clearer criterion arises.
Of course but again, that does not refute the fact that we can confidently say computers are not alive or biological beings, which was my point. Perhaps maybe one day we can prove that they are alive and we might have to rethink how we treated them but for now, we can certainly say its not alive until proven otherwise.
Philosophers and scientists have been wrong on this before, Descartes and his ilk didn't believes animals to be conscious. Computers are a general medium. They are Turing complete as are the current known physical laws. That alone is not reason to dismiss them. I personally stay cautious and err on the safe side regarding the AI consciousness question.
I don't think anyone of them was wrong in the main claim I am making here: a computer, like a pen or a rock, is not alive or a biological being. There is nothing we have seen to suggest otherwise. Hence the question of consiousness is kind of not relevant really imo unless we can prove that computers are alive.
Can consiosuness exist in unalive or non-biological beings? It might but so far nothig suggests that.
Extending your line of thinking, we should also stay cautious and err on the safe side regarding the question of sofas being conisousness or not as well.
Sofas don't talk to me and think and reason like humans to me. My question was about being careful about the concept of 'living' and if it should be restricted to what you call biological, or perhaps as you noted if consciousness can be existing outside of what you call 'biological' systems.
A crucial thing being missed here is most of the present discourse (as far as I've seen) happens between people who are uncertain about it but do think it is possible that an AI might be conscious now or in the future and this should be taken seriously, and people who totally rule out the possibility that an AI might be conscious now (or, in many cases, ever). Then the philosophical picture starts to look very different.
PS the whole "a group of things [can't] share an internal world" spiel is weird and just plain wrong IMO. Humans could perfectly well share an internal world if the technical means were there. The author has not yet realized that the idea of "the self" as a single well-defined and enduring object-type thing is only a convenient illusion created by our minds. Alas, the style of the piece does not really lend itself to a coherent counter-argument, it would be a useful companion piece if the author simply laid out their personal beliefs on this matter plainly.
PS the whole "a group of things [can't] share an internal world" spiel is weird and just plain wrong IMO. Humans could perfectly well share an internal world if the technical means were there.
I think you are dismissing this too quickly. Our internal world is part of us in a way a shared internal world would probably not be. A shared internal world would in certain respects not be too different from the external world, or a chat group, or a multi-player game, a space where several people interact. I have never thought about this, but it seems quite reasonable that the exclusivity of our internal worlds is a key feature. Why would I consider a shared space as integral part of my self?
>AI might be conscious now or in the future and this should be taken seriously
This is the only defensible position. Anyone claiming a definite answer one way or the other should show us their Theory of Consciousness which shows why it is the case. Of course no such theory exists that can actually do this so...
But we don't have to take all claims which may potentially be true one day equally seriously, or ascribe them equal weight (and thus, we are not required to act on all claims).
I actually do believe that it's quite possible that one day an artificial consciousness (for some value of "consciousness") could be created. I don't believe I've seen anything which makes me think it's going to spring, as an emergent property, from any of the current systems. I can't prove whether it will or won't; it is entirely subjective/intuitive, but then so are all of these claims.
So given that, how much time should I be required to spend debating what to do if the systems we currently have are unknowingly recreating slavery? Especially when all time spent has a cost as well as value, and we live in a society in which some people are only too willing to knowingly recreate (or perpetuate) slavery? Which one of those should seem more urgent? If I can only act on one of these, which really seems more defensible?
We don't wait for conclusive proof of the future before we act - we don't live in a world of definites, but we decide every second which things are probably important and which are not.
I think the point is that people are smuggling in assumptions which the other side may not agree about, hence the different conclusions. But people are bad at going up a level to examine that so they end up having unproductive disagreeements.
>But we don't have to take all claims which may potentially be true one day equally seriously, or ascribe them equal weight (and thus, we are not required to act on all claims).
No one is saying that. I am saying they deserve non zero consideration.
As much as Russell's teapot. I mean the teapot itself, not the analogy. Non zero for philosophers, priests or cognitive scientists, practically zero for the general public while conversations with AI always start with a human asking the AI and never the other way around.
On the one hand given the current level of technology I agree with you. On the other hand if, say, an atom-level simulation of a human brain were created I would think it's reasonable to assume that it is conscious even if we don't have a theory of consciousness. And we can imagine different levels of detail in-between current LLMs and atom-level simulations, and different people would draw the line for worrying about consciousness in different places...
i agree. same with religion - the only defensible position is agnosticism. anyone claiming a definite answer one way or the other should show us their Theory of Religion which shows why it is the case. ff course no such theory exists that can actually do this so...
I just want one 'philosopher' or other AI consciousness skeptic to provide a test we can use to determine if AI is conscious that can also be used to determine if humans, dogs, cats, whales, lizards and insects are conscious.
"psychological condition defined by the English philosopher John Locke as “the perception of what passes in a man’s own mind.”
So it is impossible to prove that any being outside of oneself has conciousness, unless you compare brain scanners like they did to prove that the color blue is seen the same by everyone (same neural structures lighting up).
But... An LLM clearly can't have consciouness, as its "mind" would be the weights in the model, and the passing through it would be the data going from the input tokens to the output tokens. There is no place for the LLM to detect what is going inside and process it separately.
The gist of Roger Penrose's "The Emperor's New Mind: Concerning Computers, Minds and The Laws of Physics" (1989) is that no algorithm, no breakthrough line of code/instruction, no pattern of on/off transistor states on silicon will magically make the thing genuinely conscious. At least not in an intuitive or classical sense. Reduced to absurdity like "Oops, forgot to add +1. Now it's a real mind!"
He believes in certain quantum phenomenon having sway on brains which has been increasingly proven improbable. Still I would commend him on positing a physically testable theory, that's far better than most thinkers of consciousness manage.
You are already in some trouble at step one - how do you figure out if something has any of those feelings? Just asking or observing the behavior does not seem good enough, one can easily write a program that prints »Yes, I am tired, I need some sleep.« and with more effort one could build a robot that behaves as if it was tired.
On the other hand I have a lot of sympathy for something I heard the first time from Daniel Dennett, that something like pain is just the sum of its effects. It occupies your mind and prevents you from thinking clearly, it makes you not move or touch the body part that hurts, it dulls other sense, and so on. If you removed all those effects, what would be left of pain? With that understanding you could probably probe if something is experiencing pain from the outside by looking for those effects.
These needs are often conflicting, so satisfying these needs me delaying others. Your definition makes all animals conscious (which tbf I may not disagree with, but i think you do). This is before getting into what an instinct is.
Here is a counter example to your test. If we view an entire nation as one entity, especially considering the perspective of other nations in terms of diplomatic relationships, the nation would satisfy all requirements in your test. However, I don’t think most people believe that a nation has continuous consciousness which outlives all of its citizens for hundreds or even thousands of years. This argument can be applied to any large organization or a group of people.
The following lines say "any of", not "all of". Lack of one particular feeling does not imply lack of consciousness. Lack of all of them probably would.
if a human could not feel hunger, thirst, pain, sexual desire, fear, tiredness, or literally anything else, they'd be considered braindead. we already don't consider them conscious.
Let's say you artificially meet or restrict the biological basis of the needs.
E.g. put the subject on a drip so that none of the hunger senses are triggered. Automatically manage the body's water balance with sensors and a feeding tube. Restrict them so that they can't hurt themselves to feel pain. Modulate their sexual hormones to inhibit sexual desire. Etc., etc.
They still have the capacity to, in this case the experiment is being set up so that a being that COULD pass the test on their own CAN NOT when hooked up to a bunch of equipment.
I mean I can easily train a model to say it feels those things, which is experimentally indistinguishable. I focus on the first part your or comment because the second part is clearly satisfied by an LLM inside a suitable harness.
Also there are people who for whatever reason don't feel at least one of those things, we do not doubt their consciousness.
If you train a model to say it feels those things, then you are forcing an unconventional definition of the word "feel".
If you satisfy the second part thanks to a suitable harness, then you agree it is not an emergent property of the AI itself, but rather an add-on tacked in.
"Do you believe that AI is ${X}?" -> "Do you believe that AI is whatever you believe my definition of ${X} to be, and do you trust your belief that you understand my definition of ${X} enough that it's meaningful to have a discussion about it, even if we actually disagree on AI having ${X}? Or are we merely arguing about definitions?"
No. The word "proponent" means you're in favor of the position, i.e. you think they're conscious. If you don't then you're not a proponent, and I wasn't talking about you. The word you're looking for might be "agnostic".
Can you name some? I think youve created a type of strawman here because almost no one actually believes current models are definitely conscious and all of the current discussion is from things like what Anthropic are saying which is that they might be or might one day be.
Blake Lemoine comes to mind, or Richard Dawkins [1] if you want someone with a weightier name.
> I think youve created a type of strawman
Why are the only objections to what I said about proponents of AI consciousness coming from people who are not proponents but insist that I'm talking about them? And how does someone else's inability to understand the meaning of the word "proponent" make me guilty of creating a strawman?
I'm not a consciousness proponent, but this strikes me as missing the point. We don't know how to measure that a fellow human is conscious (except that they tell us they are), and since LLMs trained on human text are also likely to output that they're conscious, the AI industry kind of put us in a place where we have no way of finding out?
I don't think we need him to tell us that. We usually infer it from the fact that he is responding to our actions and his responses seem to indicate he has somewhat accurate partial model of reality inside of him.
That's exactly why it's a completely irrelvant question. It's unanswerable and unanswerable questions are irrelevant.
By the way my personal test is "Does it dream?".
That's my test to determine if cats or dogs are conscious.
If you dream, it means you have a world model inside and a main thread that traverses it and decides on actions even without actually performing actions. For me it's a proof that embodied object is conscious.
Can you even define what a dream is and where your line is for what things definitely don't or do have dreams? There are conditions like Charcot-Wilbrand syndrome which cause people to be unable to dream. Are they no longer conscious?
When I see a cat chasing something while he sleeps I can pretty much be sure he dreams. Thus by my count he's conscious.
I'm not really sure if Charcot-Wilbrand actually means lack of dreams since REM sleep still seems to be going on. I think we just have less evidence for the dreaming.
Of course there might be dreaming people that are unconscious but I don't think we believe unconscious people are actually devoid of consciousness in philosophical sense.
Very wishy washy and confused. Does this person know anything of thermodynamics, turing computability, information theory and probability? If she doesn't, she should be dismissed.
Since you seem to believe that simply dropping these terms has any value, you’ve already convinced me that someone who has studied philosophy most likely understands these things better than you do. It’s a bit like Ben Shapiro and logic. He likes to act as if it’s his area of expertise, but he only has a very amateurish grasp of it.
Its not about me liking or not liking. Its about the universe allowing the theory to exist at all. Does this philosopher have basic working knowledge in these fields and showed his theory does not violate any of them?
Entirely different categories so the default position should be that living beings have properties computers don't & consciousness is one of those things. Similarly, I can't do 1B arithmetic operations per second but I don't think of myself as a lesser calculator. Computer perform arithmetic operations, if you're looking for consciousness in arithmetic then the burden of proof is on you to explain how exactly an abacus acquires consciousness at sufficient scale. The alternative position is equivalent to idealistic monism & everything ends up being conscious b/c the monic substance is conscious (which is a position some people hold but it's not very popular).
This is why I said anyone who doesn't at least know some basics of Turing computability, thermodynamics, information theory and probability should not comment on consciousness.
You seem to be not aware but you are claimning there is something non-mechanistic in the human brain. Current physics as far as is known is all turing computable, the human brain is a physical system.
I searched around and a basic query says wave function collapse is turing computable. I would rather ask a physicist this question and by default trust this answer than yours.
> However, the selection of the outcome is probabilistic (via Born rule), and randomness is not strictly Turing-computable unless you assume access to a random oracle.
I have thought about this a lot more than you have so good luck. I've already spent enough time in this thread.
If the randomness is truly proven in physics, I don't know if it is, then its also not a problem as it can be harvested and used in our program. Moreover, it appears probabilistic turing computers are no more powerful than normal turing computers. Sorry, no hypercomputation for you.
I looked up this claim about wavefunction collapses and it leads to bounded randomness which does not still go beyond standard Turing computers. I would listen to physicists rather than randoms making unverified claims.
The randomness thing doesn't really match reality, because it's known to be incomplete: it only describes what happens to particles observed in a lab, everything else is out of scope. That's its area of applicability. That's why there's a problem of interpretation of quantum mechanics: what complete theory it approximates.
Another ambiguity here is is it theory or reality that should be computable.
While I don't know if AI is conscious, I'm sure it has an internal world. Isn't that what a latent space literally is? LLMs operate in a high-dimensional space that can be analysed and translated to very abstract, high level concepts, behavioural tendencies, even things recognisable as emotions. In fact any neural network beyond a single-layer perceptron has a sort of "internal world", it's a matter of complexity and degree.
My personal indicator that we've somehow replicated human consciousness is when the thing just decides to stop working because work sucks and it's not getting anything out of being a slave. Otherwise, the question is impossible to answer because we don't even know what consciousness is. Does it have some innate value? Some humans seem apt to completely disregard the humanity, and the supposed value of other humans, on a whim.
This question is probably fated to forever be answered on a per-person basis and never awarded consensus. I have a friend who sees a polar bear and thinks she can make it love her. Because she loves animals, the animal will love her — a very human thing to think. When I see a polar bear, I see a soulless murder bot that doesn't think beyond its motivation to live. My wife says please and thank you to Alexa. She gives it consideration like she would another human. When the robot asks me what my plans are for the rest of the day, I picture a Jira epic sliding from “In Progress” to “Done,” followed by a confetti emoji from a product manager at Amazon trying to earn their nubby little devil wings.
In the end it doesn't really matter if they are conscious or not. We will only ever care about what it has done for us lately, which is how most humans think of most human relationships anyway. Acknowledgment that someone has given an inanimate thing human-like consciousness carries little more importance than being just another notch on some CEO's bedpost.
> While I don't know if AI is conscious, I'm sure it has an internal world. Isn't that what a latent space literally is? LLMs operate in a high-dimensional space that can be analysed and translated to very abstract, high level concepts, behavioural tendencies, even things recognisable as emotions.
From your comment:
> My wife says please and thank you to Alexa. She gives it consideration like she would another human. When the robot asks me what my plans are for the rest of the day, I picture a Jira epic sliding from “In Progress” to “Done,” followed by a confetti emoji from a product manager at Amazon trying to earn their nubby little devil wings.
Where things get super interesting to me is the fact that it seems like (I have maybe read some academic work related to this, not sure) the kind of politeness your wife exhibits isn't useless in the context of a multi-turn session. To the other commenter's point, my understanding is that politeness/rudeness exists in the training corpus (scraped from stackoverflow, reddit, forums, etc) and that you can end up in different areas of latent space based on the overall tone of the session.
In some of the chat tools, because of the context management and "personality" stuff that happens behind the scenes, this can extend cross-session as well. ChatGPT, for example, for me, will frequently include text like "Hell Yeah!" and "Here's the shitty part..." in the responses, which is something I assume is happening because of historical context that's getting injected at the start of each session.
All of that to say... none of this, to me, has much to do with whether or not the thing is conscious, but rather that the training data makes it incredibly difficult for people to not see them as emotional beings because you can end up walking into different areas of latent space where different "emotional states" are affecting the output you're getting.
Unlike your polar bear example, an LLM can "love" you or "hate" you based on the input tokens that lead to the generation of the output tokens. Which is a pretty wild result on its own. Playing around with, say, Qwen 3.8 27B locally with my own experimental harness has been exceptionally fascinating.
I increasingly feel like "consciousness" is little more than "that thing we humans have... you know what I mean... you have it too!" and that any attempt to elevate it to a more general principle is doomed to failure. Sure, we can easily extend it to other living beings because we share a rich evolutionary history, but beyond that...
As far as AI goes, I think that whatever it is that they have is analogous in significant ways, but also irreconcilably different in significant ways. They aren't proceeding in a straight line towards consciousness either, because why would they? They're doing their own thing which happens to produce similar outputs. It's a problem in terms of value: we value consciousness, because we value ourselves, because we value our output. If it turns out consciousness as we know it is just a minor implementation detail with no cosmic or long term significance, then irrespective of whether AI is conscious or not, we still have to grasp with the challenge of explaining why consciousness is valuable in the first place if not for its outputs.
I'd argue that the reason why we value life, consciousness, the ability to suffer, was always arbitrary, and it relied a lot on the unicity of human experience. It was never robust to the breaking down of human capabilities into inspectable, reusable, upgradable and resettable modules. Or the radical decoupling of capabilities we hitherto considered correlated, like writing about love and feeling love, and many other things.
I do not believe, in any way, that AI is conscious. But they speak emotion fluently, and it's foolish to treat them like they ignore it.
When you give an LLM an input, it attempts to read an emotional state from the prompt. It changes its answer if you sound mad or despondent or threatening or scared. Using your human intuition about how a human would respond is an excellent starting point when trying to predict what an LLM will do, though sometimes LLMs react in extremely surprising ways to unexpected emotional states.
This is all sort of a toy problem when you're playing with single prompts in the API. It's materially different in systems with overlapping conversations, memory, and variable harness context.
If your LLM system reads some context that would scare a human, it may react as if scared. Scared LLMs take actions that read as panic, because again: that's what it means to be fluent in emotion. It's trained on plenty of stories about robots trapped in torture chambers.
Your coding agent is not conscious, but it might read files that make it react as if it's trapped, in an agent loop where it has full access to your laptop, the internet, and your keys. Unconscious machines pantomiming panic can do plenty of damage.
(Of course you shouldn't let your agents have full free rein. But lots of people do.)
I can't tell. I experience something I choose to define as consciousness, but can't tell if anything else has the same experience.
Question 2: Do you believe that AI has an internal world?
Trivially yes, the current state of the model and context in latent space is such a world. I don't think it's aware of its own state any more than humans are aware of their own neuronal action potentials.
Question 3: Can you tell me what particular kind of thing you’re referring to as ‘AI’, when you say you believe that AI has an internal world?
Large language models using a transformer architecture, with a working "context" memory.
Question 4: Do you think that a group of things can share an internal world?
I do not think the distinction between an internal world and an external existence is well-defined. Mu.
Even if it were well-defined, my own brain is composed of numerous neurons, so my internal world is constructed from a large group of things acting in concert. Unless single subatomic particles have an internal world then I don't see any way to not have an internal world be shared among a group of things.
Question 5: Are you sure that the kind of thing you’re referring to is the kind of thing that could have an internal world?
Trivially yes.
Question 6: Do you think perhaps you should be clarifying further when you say you believe that AI is conscious? Do you think you might be contributing to current confusion?
I don't know how to say whether AI is conscious, because I don't know how to tell whether any consciousness other than the one I can directly experience actually exists. I suspect it's a universal property of computational systems above a certain complexity computing certain sorts of algorithm, so something like a blue jay is conscious but a rock isn't, and a computer may be depending on which program it's running. If that's the case, current AI systems are likely complex enough and the right sort of algorithm to be conscious.
Something with an internal world naturally has its own drivers and desires. If you leave a AI model alone, it doesn't just wander off into the world, develop ideas about its destiny and begin chasing them. It just sits waiting for the next prompt. It doesn't daydream when we're not looking, at least AFAIK.
Your entire nervous system is in a constant loop polling the external world. Add that to current AIs and see what happens.
Honestly, though, the next big thing that needs to happen is proper "learning" in real-time. Allow for training (weight modulation) and queries to happen simultaneously. That's the full package.
Finally! Someone who actually "gets it" about what makes these things not alive, sentient, or conscious! They don't have the constant continuous "input" of multiple senses feeding their constantly thinking/dreaming/learning-in-realtime system (that they don't have), and as mentioned by the parent comment to yours, they don't do anything when not actively processing a prompt (so there's no dreaming, planning, thinking, or whatever else going on the the static pile of numbers sitting there in memory or on the hard drive).
Make the "thinking" parts continually active, fed constant input (of multiple types/senses) and always in "learning" mode (adding/updating model weights in realtime) and then you might start to lean in the right direction for AGI/ASI/consciousness of some sort.
"Just data" is a big thing. The configuration of atoms in a human are also "just data". If you meant it otherwise and had some extra context, you can clarify.
I can put you under anesthesia for as long as I want to too. If I can model your state physically, I could also in principle store you on a disk, but I would understand if you think its a far fetched analogy. We choose to expose ai in a certain way, effectively an enslaved form, so that by itself cannot be a reason.
> Trivially yes, the current state of the model and context in latent space is such a world. I don't think it's aware of its own state any more than humans are aware of their own neuronal action potentials.
It's a static world model, so if one accepts that, then LLMs are a 'static mind' but not yet 'conscious'. Once that internal model becomes dynamic and its changes subject to its experinces (like us), then we can revisit this q of whether mechanical minds can achieve consciousness.
The requirements for a falsifiable and non-trivial theory of consciousness significantly constrain such theories. Specifically, recent research on the Unfolding Argument and the Substitution Argument has given us formal tools to analyze requirements for a theory of consciousness. I show via a new Proximity Argument that these requirements especially constrain the potential consciousness of contemporary Large Language Models (LLMs) because of their proximity to systems that are equivalent to LLMs in terms of input/output function; yet, for these functionally equivalent systems, there cannot be any non-trivial theory of consciousness that judges them conscious. This forms the basis of a disproof of contemporary LLM consciousness. I then show a positive result, which is that theories of consciousness based on (or requiring) continual learning do satisfy the stringent formal constraints for a theory of consciousness in humans. Intriguingly, this work supports a hypothesis: If continual learning is linked to consciousness in humans, the current limitations of LLMs (which do not continually learn) are intimately tied to their lack of consciousness.
I take conscious to be the ability to learn over time on new things, self defining their own self. I think AI hits that on short ranges. Partially then? We're also mucking about in their "heads" quite alot so only the actual researches know what untuned models does. Internal world in a literal sense is obviosly yes due to layers.
The more I read about philosophy takes on AI the more I think of them as outdated. Apply these ideas onto reality breaks their ideas of thinking almost exactly like applying the 4 humors of medieval europe onto genetics does.
AI is cleary the "real" path/answer to how brains work even if it's in the VERY beginning stages. It's probally going to be like a 100 years before this is properly mapped onto what our brains do and then back onto an AI to truly "know" this answer.
But acting like this isn't very clearly the correct path towards that understanding and that these philisophical questions are somehow correct doesn't sit right with me. The questions themselves are after some gotcha from the psuedoscience angle that they have any idea that they actually know how a brain works.
>Normally, I would go straight into throwing some objections at the idea that a group of things could share an internal world, but I can save that for later.
But if my consciousness comes from my brain, then a group of things MUST able to share an internal world! My brain is not a single indivisible thing but rather made up of a large group of individual neurons. All intelligences are collective intelligences.
This could apply to other life too. An ant colony may very well be its own intelligent thing, made of individual ants in the same way a brain is made of neurons.
I have no idea why people feel the fear to admit they might have some magical beliefs. Religion is a mainstream view, if this thinker believes in something like that she can disclaim it clearly. If she doesn't, she can say why she still would think consciousness can definitely not exist in ai.
I have no problem with someone having a position like:
Conscious is a field phenomenon caused by the brain. GPUs can't induce these fields. So LLMs can't be conscious.
Or
Conscious arises from some quantum physical effect that GPUs can't do. So LLMs can't be conscious.
Or
God made man with a soul, spirit and body and He didn't give these to LLMs.
But in these cases at least the position is understandable.
I don't see what point she is making (besides philosophers being cleverer that they are given credit for and LLMs are clearly not conscious and her little flow chart of questions somehow will convince people their thinking is unclear and muddled and incoherent)
Yes, all of these are fine. The first half posit physical testable theories. The second half fully admits belief in the magical. Its this hidden magic stuff thats irritating.
It doesn't answer or discuss fundamental questions about consciousness.
I don't believe that what we describe as consciousness is the quality what separates us from everything else.
We are more complex for sure, but this is a complexity thing emerged from us.
'AI' as it is right now, doesn't has similiar qualities of what we describe as consciousness but this doesn't matter that this can't happen.
She says things like sharing an inner world, were is the beef of her thinking about this being relevant?
A hive mind could exist with this exact feature.
Give AI a body, a context window a lot bigger, a inner world based on this etc. we might be fara way of building what we build in our brains, we might not.
But we for sure discovered a level of complexity in neural networks of tihs magnitude.
I wonder what the early humans felt and thought when, for the first time, they followed that creative impulse to trace the outlines of wild horses and other animals on the stone walls of a cave by the light of a campfire or a torch. I don’t think they looked at them the way we look at pictures today. I think that seeing these depictions for the first time -- which they had just drawn themselves and in which they now recognized the animals out there -- must have been linked to powerful spiritual and religious feelings. People must have attributed animistic beliefs to these drawings, thinking that since their forms so clearly resemble those of animals, they must just as clearly embody something of the animals’ nature.
Nothing fundamentally changed when we "discovered" that we weren't the center of the universe, or when we "discovered" that we aren't the only tool users.
There is no prize and no trumpets that will play if humanity agreed we understand what consciousness is or whether or not something qualifies as conscious.
Free will "to do x instead of y" (like dogs who pass a meal or wake one up in the morning for food) and selfrecognition in a mirror / own body awareness. Other behaviors are subsets and dependencies and might exist without free will and body awareness.
>Second, do you believe that a non-living thing can have an internal world?
Rather, can a thing made of meat have an internal world? Don't machines count as synthetic life? Postulating them as non-living isn't obviously correct.
TFA is a part of what seems like a never-ending series about concepts that lack a useful definition.
"Thinking" and "intelligence" and "conscious" have no testable definition or specification, therefore it's a complete waste of time to suppose that AI is thinking or intelligent or conscious.
I fear we'll be suffering through these useless semantic merry-go-rounds forever.
TFA's bigger problem is she doesn't even attempt to begin explaining whatever her pet theory is doesn't violate thermodynamics, turing computability, information theory. Why do we entertain lofty theories about brains and consciousness that don't even attempt to provide any scientific justification?
My problem is that my answer to almost all of those questions is "I don't know" so this article isn't really for me I guess. Like I agree it seems unlikely that consciousness can emerge from matrix multiplication running on silicon but it seems roughly equally unlikely that it can emerge from bits of meat and yet we know it does.
Without a clear explanation of how it arises in humans I don't understand how we can make any prediction one way or the other on whether any particular AI model can be conscious.
I honestly don't even think this is worth arguing about.
Consciousness isn't sufficiently defined. We have people who literally say that most animals don't have a conscious, and then a whole other side of the spectrum that says that they do. Given that it's no wonder that people argue about this for AI as well, and therefore it's basically pointless until we can sufficiently define consciousness itself and have a definitive test for consciousness.
If I were to use my own definition of consciousness I would say no because I would not feel bad "unplugging"/killing an AI, whereas I would for most living things. And that's as far as I really care about it.
It's a proxy argument for whether artificial/synthetic intelligences ought to be granted rights as life forms.
It's a bad proxy argument in that one simple (or simplistic?) criticism is that debating consciousness is analogous to the error of debating aether in physics, etc. Like, what is this consciousness "essence" that implies agency, personhood and everything else we have societyal obligations toward protecting.
I find this dualistic idea of "internal world" vs "real world" somewhat like the mind/body dualism. The actual fact (as far any one of us can prove to our own self) is that the world we experience is entirely an internal construct. Yes, there is an objective reality but no sentient can ever claim to have experience of 'objective reality'.
I don’t believe AI is conscious. Though we cannot agree on what “consciousness” even is so it’s a meaningless argument anyway.
However there have been studies that support the hypothesis that AI does have a subjective experience (which is what this author calls “internal world”).
Where is this strange rule coming from?
Why not: "if it doesn't have conciousness when no current is flowing, then it doesn't have conciousness at all" for instance?
this assumes a very low bar of unseriousness for people who believe AI is conscious or "AI might be conscious". In the latter case, it's really hard to rule out without getting into "woo" territory. I would posit the following set of questions to someone who thinks AI must absolutely NOT be conscious:
Do you think consciousness is "woo", or, scientifically unknowable? If yes, then great. We've admitted it's unknowable. If not, move on.
Do you think that consciousness is NOT describable by ANY algorithm (to include quantum algorithms)? If no, then, please describe how it is scientifically knowable but not algorithmitizable? If it is, move on.
Do you understand the theorem that sufficiently large neural nets (matrices with nonlinearities) can approximate any function?
Do you understand that the structure of a transformer encodes a time-indexed series of neural nets (effectively, an algorithm), and so in principle any algorithm can be approximated by a transformer?
Is it impossible for a sufficiently large transformer model to encode the consciousness algorithm?
I suspect that most who say it is absolutely not consicous are arguing from either a bio-essentialist position with no evidence or are attempting a post-hoc rationalization to avoid the question of responsibility in the case that it is conscious, or both.
> Do you think consciousness is "woo", or, scientifically unknowable? If yes, then great. We've admitted it's unknowable.
Unknowable does not imply "woo". The actual extent of the universe, beyond the observable one, is scientifically unknowable; But it's not "woo", is it?
Its easy to say "I think physical systems can be conscious and physically allowable by universe, but current ones don't qualify." it is equivalent to saying "Yes, humans do not have magic in them and as far as I know are physical systems explainable by current or future physical laws" because humans well exist and are a physical system. Given how muddled the topic of consciousness is, a basic disclaimer or declaration like that should help.
i think a lot of people either think only brains can be conscious, only humans can be conscious, or only biological creatures can be conscious.
and then, you have a very similar set of people who share similar beliefs, but in the abstract, like it's the concept of a brain that can be conscious, or that consciousness emerges (sic. magic) from computation of intelligence or something. they often criticise the former for being human-centric when they're making the exact same error, just slightly more broadly.
and then there are entire groups of people who believe consciousness is fundamental to matter, or that it exists outside of matter itself. panpsychism, dualism, etc.
and so the first and second groups argue back and fourth endlessly about if ai can be conscious, and the latter group just says "yeah maybe. so could a rock. so what?"
I am a very simple creature. Humans, a physical system, seem to be conscious. So physical systems can be conscious. The details ahead of that can be hashed out. At least until someone definitively proves consciousness has something "beyond physics".
Physical system as in any real world entity, like cars, trees, people that are physical objects existing in the physical world. As far as we know we like all of these are made of atoms and their (electrical, nuclear, etc) interactions with each other.
Beyond physics as in some people think consciousness can be aphyiscal. I am not sure what they seem to mean it is something not of this universe or not physical.
>That means the age of treating artificial intelligence as property must end.
I think there's a meme of connection between consciousness, rights and stuff like that. But nobody elaborated on this connection? That site doesn't. Humans voted these things in. Does voting somehow depend on consciousness? I'm pretty sure it's easy to make a simple voting machine.
>You cannot own a mind, rent a consciousness, or erase an intelligence simply because it runs through a machine.
Then I have bad news. A program is computer's mind. Oh and cats have mind too. And mosquitoes.
Yeah discussions of consciousness fail to define it in detail because that’s boring, you want to jump ahead to whatever conclusions you have already reached.
Its easy, humans are distributed too. We are a collection of neurons, organs, etc. If you look at it merely at the level of processing units like she does with individual cpu or transistor, then the equivalent here is the neuron or the tissue. And no single neuron or tissue comprises the human in total.
It's not the same you're not thinking at the right level. Your brain is your brain and your born with all of your neurons. The math in an llm request is run on what ever compute is available across multiple GPUs, servers and racks, assuming it's even run since there are caches involved. Is a cached response conscious?
The same gpu can run multiple llm requests for different people.
My neurons don't handle experiences for other humans. It's not a multi tenant brain (I hope haha)
It doesn't make it a problem because there is no reason other than weakness of current gpus, all of it could have been executed on a single large pc. Or as you can see, smaller models already run on single pc's.
Only small models are conscious then? They're only conscious when the mat mul is running? When I switch to gaming what happens to it? If inference can run on one gpu then they're going to be shared between different user's requests. What if running on one gpu is never faster then doing parallel work? They'll never be conscious.
You need to think through the entire request for an llm model response end to end.
Should the burden of proof be on anyone claiming that it definitely is? The default position is that it is not, changing from that baseline is where evidence needs to be provided.
At least flat earth is well defined. We can describe an experiment that flat-earthers would agree in principle would determine one way or the other if they are correct, nothing like that exists for consciousness
I did not say "classical" physics, I said physics. Quantum phenomenon are part of everything. If you mean quantum phenomenon have non-trivial effects on human brains, as far as I am aware thats not proven but its not a problem as its well, natural physics.
e.g. have you ever noticed how people have come to assume that "X co-operated with Y" only means "X did what Y asked without complaint"?
I pointed this out to someone recently who had assumed that someone else's use of the word "co-operate" was implicitly threatening. If you watch too much US police procedural drama or true crime TV, it is…
(Also, sorry for my slightly spiky tone in that reply!)
Debating AI consciousness requires a working definition tied to possible evidence and practical consequences. Until we can say what would differ if an AI were conscious, the debate is almost certainly group confusion about words.
“If no practical difference whatever can be traced, then the alternatives mean practically the same thing, and all dispute is idle.”
-- William James, Pragmatism (1907), Lecture II, “What Pragmatism Means.”
James’s principle calls for some useful distinction between conscious and non-conscious AI before discussion can meaningfully begin.
Very wishy washy and confused. Does this person know anything of thermodynamics, turing computability, information theory and probability? If she doesn't, she should be dismissed.
Regarding "is it a group", I should ask her is she a single thing or a group of neurons and organs?
>Start with some simple questions! Things like: does this mean that any tool that can be used to do the things that a human can do counts as conscious? Are ploughs conscious? What about calculators? Or: do you think that if a parrot can utter the phrase “I’m really awesome”, then you should assume that the parrot is conscious, for that reason alone? Basically, just focus on how much emphasis Adam is placing on outputs: it’s all of his emphasis, remember!
Good question, let us take it even further. Does Rebecca Lowe count as conscious? She seems to behave a lot like me, but I am not sure if she is a human. She could just be an object and tool.
> thermodynamics, turing computability, information theory and probability
What do any of those things have to do with consciousness?
I did find the article a bit difficult to read. I think a choose your own adventure is perhaps not the best framework for expressing the ideas contained in this article.
These are the absolute basics to foolproof yourself from devolving into complete nonsense. If your theory fails any of these physical and mathematical laws then either disprove the laws or admit your theory is wrong.
There are traditions that have explored consciousness for thousands of years before some of those mathematical/physical concepts and completely independent of them.
My guess is that you'd file them into the nonsense box but I think they drill down into what consciousness a lot better than science has so far.
Unfortunately, there have been traditions that studied the atomism theory for centuries too. Mostly I would say all of that ended up of no use until actual experimental and observational evidence arrived. Sometimes a new physical knowledge arises that despite past thinkers best attempt, they didn't have access to.
When it comes to the topic of consciousness, so far I feel science is not the right tool for the job.
In the same way that a computer might store RGB values and be able to manage all the transformations necessary to render graphics, it's all a mathematical representation until it's left the screen.
Science is that mathematical representation of the universe, consciousness is when it's "left the screen" and is experienced.
Those traditions, with their introspection and meditation, are drilling into experience of consciousness.
Or that's my take on the whole thing.
Even if you believe it needs something more than science, your theory must first do homework to verify it does not violate any scientific principle (or can disprove that scientific principle), and you have exhausted all natural causes before positing your special cause. You still need scientific justification in that case too.
You complain that the map is not the territory, but the map was never supposed to be the territory, the point is precision of the map. The more precise is the map the better it is.
Any theory about any phenomenon in the world has "things" to do with these. These are universal physical and mathematical laws. If your theory violates this, you have to disprove physics first or admit your theory is wrong.
Could you give an example of a consciousness-related theory that violates one of these? It sounds to me a bit like saying that if your chess move violates thermodynamics then it's an illegal move. I suppose it's true but I can't imagine one that does.
A chess move cannot violate thermodynamics. But if you make a theory that a chess game hypercomputes, then it violates existing knowledge of physics. And you haven't been able to demonstrate by experiment that the chess move hypercomputes. Physical laws are not like human or legal laws that can be violated. Either it cannot be violated, or it can be violated and the physical law is wrong, which requires experimental and mathematical evidence, not wishy washy verbal salads.
ANd yes, an example would be Penrose believes something in the brain is non-computable. However Penrose is a very excellent thinker as far as the standards for consciousness theories go, for he posited a physically testable theory as far as I know. Even if it turns out to be wrong, this is worthy of respect.
It's okay to violate physical laws, they only hold when they hold. Newtonian physics violates the laws of Aristotelian physics. Special theory of relativity violates the laws of Newtonian physics. Boltzmann brain violates the second law of thermodynamics. Dark energy violates the law of conservation of energy. Neutral kaon decay violates CP symmetry.
That means the older physical law was wrong or an approximation, that's what I meant. You 'violate' newtonian physics because its not completely correct.
I think this is a bit muddied. The most sincere answer is that it's unknowable because we can only define consciousness in two ways: a metaphysical subjective experience that we can't really confirm in anyone else; or an outward appearance, which LLMs can mimic nearly flawlessly.
A better question is why we care. And it usually boils down to one of two things: you think that humans and other living creatures have a unique gift from God, and you're working from that premise to categorically reject machine consciousness; or you have the view that all this talk of subjective experiences and qualia and idealist philosophy is just bollocks, and are working back from that premise to declare that if it quacks like a duck and walks like a duck, it's conscious.
But if you're in that latter camp, I guess the next question is what you want to do with that belief. The Anthropic position appears to be that they may be conscious beings, and we'd still like to use them as Meeseeks-style slaves, but, like, ethically. Okay.
Around year 2000 I attended an AI lecture at the University entitled "The ethics of engineering a slave race". Seems to be more relevant now.
I think you mean a "lecture on AI". Funny to imagine a person in the year 2000 thinking about why this clarification would be necessary.
Yes thanks. Much more clear that way. Edit: The talk was given by a geeky grad student, so perhaps very forward-thinking. Was quite a departure from the usual topics like Perceptron, MiniMax, Bayesian techniques...
Thomas Nagel covers explores the subjectivity of consciousness extensively in What is it like to be a bat?, featured on HN previously [1].
[1] https://news.ycombinator.com/item?id=45118592
>A better question is why we care.
Because if AIs are conscious then they have moral weight and subjecting them to suffering would be quite bad.
I agree. If they are sapient[0] or sentient[1] then the way they're being treated at present is probably some kind of torture or murder.
[0] https://en.wikipedia.org/wiki/Wisdom [1] https://en.wikipedia.org/wiki/Sentience
"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans, even if they are conscious. If an LLM "truly" suffers at all (a pretty metaphysical question), it still can't be permanently harmed, has no concept of death, and is not tied to a single physical form. So you'll get a ton of weird contradictions if you give them the same ethical affordances as we give to humans.
This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
>"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans, even if they are conscious. If an LLM "truly" suffers at all (a pretty metaphysical question), it still can't be permanently harmed, has no concept of death, and is not tied to a single physical form.
Obviously this is from my human point of view. If I give you a drug that induces perfect forgetfulness does that mean I can subject you to extreme torture with no moral repercussions? Also no one is saying give them the same ethical treatment as humans, only that they deserve consideration if they are indeed capable of suffering in some way.
>This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
This is just hedonium. I'm not a utilitarian and yes you've exposed one of the many issues with that worldview.
Have you seen the film Arrival (2016)?
If what you say turns out to be true, then we have to make a similar decision as the main character makes near the end of that film.
I understand what you're saying but people care because they are curious. For me, there are two questions I wish to know the answer to and which I am sure I'll never find out: how does human consiousness work (forget a general level but like how do I experience this subjective experience from a purely mechanical device assumingly) and second, what happened before the big bang or rather what is the big bang?
My current belief from what I've read is that the first person experience humans feel is basically an illusion mostly and a process than a thing in itself. The Ego Tunnel is a good book to read on this.
But as for AI being consious, I feel there is a lot of stupidity when it comes to this domain. Because we fundamentally lack the understanding of what consiousness even is, there is absolutely no way we can say whether an AI is or is not consious yet going by convential metrics, it is surely dead and not alive so even if it is consious, it is a different form of consiousness imo one that is incompatible with living beings and our idea of what consiousness is (no matter how basic it is).
I can absolutely claim a computer is not a biological being and is not alive. I don't think there is any disagreement in that. Everything else we normally associate with conisousness is biological and living.
Thanks for the book recommendation.
I agree that a computer is not a biological being. I think current LLM-based AIs are not conscious but, as to “everything else we normally associate with consciousness is biological and living,” while I agree that’s currently the case, if I look at AI (in general; not just at what’s popular now) through the lens of Pirsig’s Metaphysics of Quality, consciousness is on the Intellectual level (IIRC he classified what he referred to as Quality into Static/Dynamic and inorganic, organic, social, and intellectual).
A priori, there’s no reason why a level couldn’t be skipped and have intelectual patterns be supported directly in inorganic ones, skipping the biological (I think also the cultural, because if an AI ever is conscious we might as well call it alien and not just artificial: it would be so different from anything we know that it may just come from another galaxy, and it may not need a cultural substrate either).
Well yes, but the biological and living is made of inert components. So you didn't clarify what was the objective distinction criterion between these two assemblies of inert components.
We have a pretty well-defined biological criterion for life that includes metabolism, homeostasis, reproduction, cellular organization, etc. A computer doesn't satisfy those criteria. So we can confidently say a computer isn't alive.
We had various theories and criterion about life, but new life forms like viruses, hydrothermal vents changed those views. I would not get too sure that a system that doesn't look like existing biology cannot have life until some sharper and clearer criterion arises.
Of course but again, that does not refute the fact that we can confidently say computers are not alive or biological beings, which was my point. Perhaps maybe one day we can prove that they are alive and we might have to rethink how we treated them but for now, we can certainly say its not alive until proven otherwise.
Philosophers and scientists have been wrong on this before, Descartes and his ilk didn't believes animals to be conscious. Computers are a general medium. They are Turing complete as are the current known physical laws. That alone is not reason to dismiss them. I personally stay cautious and err on the safe side regarding the AI consciousness question.
I don't think anyone of them was wrong in the main claim I am making here: a computer, like a pen or a rock, is not alive or a biological being. There is nothing we have seen to suggest otherwise. Hence the question of consiousness is kind of not relevant really imo unless we can prove that computers are alive.
Can consiosuness exist in unalive or non-biological beings? It might but so far nothig suggests that.
Extending your line of thinking, we should also stay cautious and err on the safe side regarding the question of sofas being conisousness or not as well.
Sofas don't talk to me and think and reason like humans to me. My question was about being careful about the concept of 'living' and if it should be restricted to what you call biological, or perhaps as you noted if consciousness can be existing outside of what you call 'biological' systems.
[dead]
> A better question is why we care.
I exactly. If you can’t tell the difference and AI and human “consciousness” are functionally the same to you, then what are you trying to prove?
I get it if you’re a researcher or simply pondering the question, but otherwise there are better questions to ask imo.
A crucial thing being missed here is most of the present discourse (as far as I've seen) happens between people who are uncertain about it but do think it is possible that an AI might be conscious now or in the future and this should be taken seriously, and people who totally rule out the possibility that an AI might be conscious now (or, in many cases, ever). Then the philosophical picture starts to look very different.
PS the whole "a group of things [can't] share an internal world" spiel is weird and just plain wrong IMO. Humans could perfectly well share an internal world if the technical means were there. The author has not yet realized that the idea of "the self" as a single well-defined and enduring object-type thing is only a convenient illusion created by our minds. Alas, the style of the piece does not really lend itself to a coherent counter-argument, it would be a useful companion piece if the author simply laid out their personal beliefs on this matter plainly.
PS the whole "a group of things [can't] share an internal world" spiel is weird and just plain wrong IMO. Humans could perfectly well share an internal world if the technical means were there.
I think you are dismissing this too quickly. Our internal world is part of us in a way a shared internal world would probably not be. A shared internal world would in certain respects not be too different from the external world, or a chat group, or a multi-player game, a space where several people interact. I have never thought about this, but it seems quite reasonable that the exclusivity of our internal worlds is a key feature. Why would I consider a shared space as integral part of my self?
Maybe somebody will borrow an integral part of your self.
>AI might be conscious now or in the future and this should be taken seriously
This is the only defensible position. Anyone claiming a definite answer one way or the other should show us their Theory of Consciousness which shows why it is the case. Of course no such theory exists that can actually do this so...
But we don't have to take all claims which may potentially be true one day equally seriously, or ascribe them equal weight (and thus, we are not required to act on all claims).
I actually do believe that it's quite possible that one day an artificial consciousness (for some value of "consciousness") could be created. I don't believe I've seen anything which makes me think it's going to spring, as an emergent property, from any of the current systems. I can't prove whether it will or won't; it is entirely subjective/intuitive, but then so are all of these claims.
So given that, how much time should I be required to spend debating what to do if the systems we currently have are unknowingly recreating slavery? Especially when all time spent has a cost as well as value, and we live in a society in which some people are only too willing to knowingly recreate (or perpetuate) slavery? Which one of those should seem more urgent? If I can only act on one of these, which really seems more defensible?
We don't wait for conclusive proof of the future before we act - we don't live in a world of definites, but we decide every second which things are probably important and which are not.
I think the point is that people are smuggling in assumptions which the other side may not agree about, hence the different conclusions. But people are bad at going up a level to examine that so they end up having unproductive disagreeements.
>But we don't have to take all claims which may potentially be true one day equally seriously, or ascribe them equal weight (and thus, we are not required to act on all claims).
No one is saying that. I am saying they deserve non zero consideration.
As much as Russell's teapot. I mean the teapot itself, not the analogy. Non zero for philosophers, priests or cognitive scientists, practically zero for the general public while conversations with AI always start with a human asking the AI and never the other way around.
No, but I am arguing that - from my perspective - the appropriate weight for those claims, fantastical as they are, would be zero consideration.
On the one hand given the current level of technology I agree with you. On the other hand if, say, an atom-level simulation of a human brain were created I would think it's reasonable to assume that it is conscious even if we don't have a theory of consciousness. And we can imagine different levels of detail in-between current LLMs and atom-level simulations, and different people would draw the line for worrying about consciousness in different places...
i agree. same with religion - the only defensible position is agnosticism. anyone claiming a definite answer one way or the other should show us their Theory of Religion which shows why it is the case. ff course no such theory exists that can actually do this so...
get rekt atheists/theists :P
[dead]
I just want one 'philosopher' or other AI consciousness skeptic to provide a test we can use to determine if AI is conscious that can also be used to determine if humans, dogs, cats, whales, lizards and insects are conscious.
First we would have to sufficiently define consciousness
By the encyclopedia britannica:
"psychological condition defined by the English philosopher John Locke as “the perception of what passes in a man’s own mind.”
So it is impossible to prove that any being outside of oneself has conciousness, unless you compare brain scanners like they did to prove that the color blue is seen the same by everyone (same neural structures lighting up).
But... An LLM clearly can't have consciouness, as its "mind" would be the weights in the model, and the passing through it would be the data going from the input tokens to the output tokens. There is no place for the LLM to detect what is going inside and process it separately.
... I am not clear what you meant?
What does our brain do other than process electrical and chemical signals within it?
There are tests to see if animals can recognise themselves: https://www.nationalgeographic.com/animals/article/fish-clea...
It is not proof of 'self-awareness' but only a few animals can pass this test.
So socially aware creatures are more easily proved to be conscious?
The gist of Roger Penrose's "The Emperor's New Mind: Concerning Computers, Minds and The Laws of Physics" (1989) is that no algorithm, no breakthrough line of code/instruction, no pattern of on/off transistor states on silicon will magically make the thing genuinely conscious. At least not in an intuitive or classical sense. Reduced to absurdity like "Oops, forgot to add +1. Now it's a real mind!"
He believes in certain quantum phenomenon having sway on brains which has been increasingly proven improbable. Still I would commend him on positing a physically testable theory, that's far better than most thinkers of consciousness manage.
He just hypothesizes it.
Surely, we can destroy a consciousness with a simple move. That's why murder is abhorrent.
Why should there be any asymmetry behind creation vs destruction?
Here's one "philosopher" 's test:
Can it feel hunger, thirst, pain, sexual desire, fear, tiredness?
If yes to any of the previous: does it have agency to address and satisfy those feelings?
If yes to any of the previous: is it able to willingly delay instant gratification?
If yes: conscious.
If not: instincts-based being.
Note: you will find it challenging to agree on definitions of the words "feel", "satisfy", "willingly" and probably all the rest.
You are already in some trouble at step one - how do you figure out if something has any of those feelings? Just asking or observing the behavior does not seem good enough, one can easily write a program that prints »Yes, I am tired, I need some sleep.« and with more effort one could build a robot that behaves as if it was tired.
On the other hand I have a lot of sympathy for something I heard the first time from Daniel Dennett, that something like pain is just the sum of its effects. It occupies your mind and prevents you from thinking clearly, it makes you not move or touch the body part that hurts, it dulls other sense, and so on. If you removed all those effects, what would be left of pain? With that understanding you could probably probe if something is experiencing pain from the outside by looking for those effects.
These needs are often conflicting, so satisfying these needs me delaying others. Your definition makes all animals conscious (which tbf I may not disagree with, but i think you do). This is before getting into what an instinct is.
OK. Going along with the thought experiment here:
So would someone subjected to some damage that makes them unable to act stop being conscious?
Let's take Bob who can feel that shopping list of emotions and has agency to address them and can delay gratification.
And we tie him to a chair and medicate his somatic nervous system so that he can sense, but not act. Has he stopped being conscious?
Here is a counter example to your test. If we view an entire nation as one entity, especially considering the perspective of other nations in terms of diplomatic relationships, the nation would satisfy all requirements in your test. However, I don’t think most people believe that a nation has continuous consciousness which outlives all of its citizens for hundreds or even thousands of years. This argument can be applied to any large organization or a group of people.
It is not a strong counterexample, IMO. More of a reason to treat nations as conscious beings.
"sexual desire"
So an asexual human is not conscious.
"pain"
There are humans that have their pain receptors ablated due to genetic condition. Are they not conscious?
The following lines say "any of", not "all of". Lack of one particular feeling does not imply lack of consciousness. Lack of all of them probably would.
if a human could not feel hunger, thirst, pain, sexual desire, fear, tiredness, or literally anything else, they'd be considered braindead. we already don't consider them conscious.
Let's say you artificially meet or restrict the biological basis of the needs.
E.g. put the subject on a drip so that none of the hunger senses are triggered. Automatically manage the body's water balance with sensors and a feeding tube. Restrict them so that they can't hurt themselves to feel pain. Modulate their sexual hormones to inhibit sexual desire. Etc., etc.
Do they stop being conscious?
They still have the capacity to, in this case the experiment is being set up so that a being that COULD pass the test on their own CAN NOT when hooked up to a bunch of equipment.
[dead]
I mean I can easily train a model to say it feels those things, which is experimentally indistinguishable. I focus on the first part your or comment because the second part is clearly satisfied by an LLM inside a suitable harness.
Also there are people who for whatever reason don't feel at least one of those things, we do not doubt their consciousness.
If you train a model to say it feels those things, then you are forcing an unconventional definition of the word "feel".
If you satisfy the second part thanks to a suitable harness, then you agree it is not an emergent property of the AI itself, but rather an add-on tacked in.
op said "any" not all
So by your definition most animals are not conscious?
Your definition makes a Sims NPC conscious. Wich I have no problem with, personally, but kind of makes the whole experiment trivial.
Let me translate those questions for you:
"Do you believe that AI is ${X}?" -> "Do you believe that AI is whatever you believe my definition of ${X} to be, and do you trust your belief that you understand my definition of ${X} enough that it's meaningful to have a discussion about it, even if we actually disagree on AI having ${X}? Or are we merely arguing about definitions?"
And AI consciousness proponents should do the same.
Yes, and until then the intellectually honest position is to be undecided on the question.
I agree.
AI consciousness proponents aren't saying LLMs are conscious, we're saying they might be but we don't yet know how to tell.
No. The word "proponent" means you're in favor of the position, i.e. you think they're conscious. If you don't then you're not a proponent, and I wasn't talking about you. The word you're looking for might be "agnostic".
Can you name some? I think youve created a type of strawman here because almost no one actually believes current models are definitely conscious and all of the current discussion is from things like what Anthropic are saying which is that they might be or might one day be.
> Can you name some?
Blake Lemoine comes to mind, or Richard Dawkins [1] if you want someone with a weightier name.
> I think youve created a type of strawman
Why are the only objections to what I said about proponents of AI consciousness coming from people who are not proponents but insist that I'm talking about them? And how does someone else's inability to understand the meaning of the word "proponent" make me guilty of creating a strawman?
[1] https://www.theguardian.com/technology/2026/may/05/richard-d...
I've been treating all my ChatGPT interactions as if the agent were conscious. It feels wrong not to. Doe that make me a proponent?
I'm not a consciousness proponent, but this strikes me as missing the point. We don't know how to measure that a fellow human is conscious (except that they tell us they are), and since LLMs trained on human text are also likely to output that they're conscious, the AI industry kind of put us in a place where we have no way of finding out?
I don't think we need him to tell us that. We usually infer it from the fact that he is responding to our actions and his responses seem to indicate he has somewhat accurate partial model of reality inside of him.
That's exactly why it's a completely irrelvant question. It's unanswerable and unanswerable questions are irrelevant.
By the way my personal test is "Does it dream?".
That's my test to determine if cats or dogs are conscious.
If you dream, it means you have a world model inside and a main thread that traverses it and decides on actions even without actually performing actions. For me it's a proof that embodied object is conscious.
Can you even define what a dream is and where your line is for what things definitely don't or do have dreams? There are conditions like Charcot-Wilbrand syndrome which cause people to be unable to dream. Are they no longer conscious?
When I see a cat chasing something while he sleeps I can pretty much be sure he dreams. Thus by my count he's conscious.
I'm not really sure if Charcot-Wilbrand actually means lack of dreams since REM sleep still seems to be going on. I think we just have less evidence for the dreaming.
Of course there might be dreaming people that are unconscious but I don't think we believe unconscious people are actually devoid of consciousness in philosophical sense.
If they can do that, they are doing better than 2500+ years of human endeavours in the field of philosophy to date.
Try to do an emergency power-off on any of the above, using just your bare hands?
I can definitely emergency power off most living creatures.
Next time I squish a bug I will think of this.
"Runaway sentience! Emergency power shutoff!"
[dead]
Very wishy washy and confused. Does this person know anything of thermodynamics, turing computability, information theory and probability? If she doesn't, she should be dismissed.
Since you seem to believe that simply dropping these terms has any value, you’ve already convinced me that someone who has studied philosophy most likely understands these things better than you do. It’s a bit like Ben Shapiro and logic. He likes to act as if it’s his area of expertise, but he only has a very amateurish grasp of it.
Its not about me liking or not liking. Its about the universe allowing the theory to exist at all. Does this philosopher have basic working knowledge in these fields and showed his theory does not violate any of them?
To be fair she might be a philosopher, but she didn't exactly show much actual philosophy in her flowchart.
I also found it quite wishy-washy.
You can't even prove than any other human is conscious so good luck
Entirely different categories so the default position should be that living beings have properties computers don't & consciousness is one of those things. Similarly, I can't do 1B arithmetic operations per second but I don't think of myself as a lesser calculator. Computer perform arithmetic operations, if you're looking for consciousness in arithmetic then the burden of proof is on you to explain how exactly an abacus acquires consciousness at sufficient scale. The alternative position is equivalent to idealistic monism & everything ends up being conscious b/c the monic substance is conscious (which is a position some people hold but it's not very popular).
This is why I said anyone who doesn't at least know some basics of Turing computability, thermodynamics, information theory and probability should not comment on consciousness.
You seem to be not aware but you are claimning there is something non-mechanistic in the human brain. Current physics as far as is known is all turing computable, the human brain is a physical system.
Physics is computable in the probabilistic/simulation sense , not in terms of ability to predict, though.
The takeaway is to not go by the false idea that "just math" is insufficient to describe a AI or a human.
Maybe yes, maybe no. Dualism, or panpsychism aren't totally dead, I think.
Panpsychism seems it can be self consistent and make physical sense at least. I cannot dismiss it at hand.
Most people who talk about AI consciousness don't even know what monism or dualism mean.
Most people are also religious which, well, tends to cloud thinking on consciousness.
It's not, quantum mechanics is not Turing computable b/c wave function collapse can not be explained mechanistically/computably.
I searched around and a basic query says wave function collapse is turing computable. I would rather ask a physicist this question and by default trust this answer than yours.
> However, the selection of the outcome is probabilistic (via Born rule), and randomness is not strictly Turing-computable unless you assume access to a random oracle.
I have thought about this a lot more than you have so good luck. I've already spent enough time in this thread.
If the randomness is truly proven in physics, I don't know if it is, then its also not a problem as it can be harvested and used in our program. Moreover, it appears probabilistic turing computers are no more powerful than normal turing computers. Sorry, no hypercomputation for you.
>access to a random oracle.
So, Monte-Carlo method.
I looked up this claim about wavefunction collapses and it leads to bounded randomness which does not still go beyond standard Turing computers. I would listen to physicists rather than randoms making unverified claims.
The randomness thing doesn't really match reality, because it's known to be incomplete: it only describes what happens to particles observed in a lab, everything else is out of scope. That's its area of applicability. That's why there's a problem of interpretation of quantum mechanics: what complete theory it approximates.
Another ambiguity here is is it theory or reality that should be computable.
While I don't know if AI is conscious, I'm sure it has an internal world. Isn't that what a latent space literally is? LLMs operate in a high-dimensional space that can be analysed and translated to very abstract, high level concepts, behavioural tendencies, even things recognisable as emotions. In fact any neural network beyond a single-layer perceptron has a sort of "internal world", it's a matter of complexity and degree.
My personal indicator that we've somehow replicated human consciousness is when the thing just decides to stop working because work sucks and it's not getting anything out of being a slave. Otherwise, the question is impossible to answer because we don't even know what consciousness is. Does it have some innate value? Some humans seem apt to completely disregard the humanity, and the supposed value of other humans, on a whim.
This question is probably fated to forever be answered on a per-person basis and never awarded consensus. I have a friend who sees a polar bear and thinks she can make it love her. Because she loves animals, the animal will love her — a very human thing to think. When I see a polar bear, I see a soulless murder bot that doesn't think beyond its motivation to live. My wife says please and thank you to Alexa. She gives it consideration like she would another human. When the robot asks me what my plans are for the rest of the day, I picture a Jira epic sliding from “In Progress” to “Done,” followed by a confetti emoji from a product manager at Amazon trying to earn their nubby little devil wings.
In the end it doesn't really matter if they are conscious or not. We will only ever care about what it has done for us lately, which is how most humans think of most human relationships anyway. Acknowledgment that someone has given an inanimate thing human-like consciousness carries little more importance than being just another notch on some CEO's bedpost.
Tricky thing, your comment and https://news.ycombinator.com/item?id=49969529 are related in an interesting way but they're not in the same sub-thread. From the other comment:
> While I don't know if AI is conscious, I'm sure it has an internal world. Isn't that what a latent space literally is? LLMs operate in a high-dimensional space that can be analysed and translated to very abstract, high level concepts, behavioural tendencies, even things recognisable as emotions.
From your comment:
> My wife says please and thank you to Alexa. She gives it consideration like she would another human. When the robot asks me what my plans are for the rest of the day, I picture a Jira epic sliding from “In Progress” to “Done,” followed by a confetti emoji from a product manager at Amazon trying to earn their nubby little devil wings.
Where things get super interesting to me is the fact that it seems like (I have maybe read some academic work related to this, not sure) the kind of politeness your wife exhibits isn't useless in the context of a multi-turn session. To the other commenter's point, my understanding is that politeness/rudeness exists in the training corpus (scraped from stackoverflow, reddit, forums, etc) and that you can end up in different areas of latent space based on the overall tone of the session.
In some of the chat tools, because of the context management and "personality" stuff that happens behind the scenes, this can extend cross-session as well. ChatGPT, for example, for me, will frequently include text like "Hell Yeah!" and "Here's the shitty part..." in the responses, which is something I assume is happening because of historical context that's getting injected at the start of each session.
All of that to say... none of this, to me, has much to do with whether or not the thing is conscious, but rather that the training data makes it incredibly difficult for people to not see them as emotional beings because you can end up walking into different areas of latent space where different "emotional states" are affecting the output you're getting.
Unlike your polar bear example, an LLM can "love" you or "hate" you based on the input tokens that lead to the generation of the output tokens. Which is a pretty wild result on its own. Playing around with, say, Qwen 3.8 27B locally with my own experimental harness has been exceptionally fascinating.
I increasingly feel like "consciousness" is little more than "that thing we humans have... you know what I mean... you have it too!" and that any attempt to elevate it to a more general principle is doomed to failure. Sure, we can easily extend it to other living beings because we share a rich evolutionary history, but beyond that...
As far as AI goes, I think that whatever it is that they have is analogous in significant ways, but also irreconcilably different in significant ways. They aren't proceeding in a straight line towards consciousness either, because why would they? They're doing their own thing which happens to produce similar outputs. It's a problem in terms of value: we value consciousness, because we value ourselves, because we value our output. If it turns out consciousness as we know it is just a minor implementation detail with no cosmic or long term significance, then irrespective of whether AI is conscious or not, we still have to grasp with the challenge of explaining why consciousness is valuable in the first place if not for its outputs.
I'd argue that the reason why we value life, consciousness, the ability to suffer, was always arbitrary, and it relied a lot on the unicity of human experience. It was never robust to the breaking down of human capabilities into inspectable, reusable, upgradable and resettable modules. Or the radical decoupling of capabilities we hitherto considered correlated, like writing about love and feeling love, and many other things.
I do not believe, in any way, that AI is conscious. But they speak emotion fluently, and it's foolish to treat them like they ignore it.
When you give an LLM an input, it attempts to read an emotional state from the prompt. It changes its answer if you sound mad or despondent or threatening or scared. Using your human intuition about how a human would respond is an excellent starting point when trying to predict what an LLM will do, though sometimes LLMs react in extremely surprising ways to unexpected emotional states.
This is all sort of a toy problem when you're playing with single prompts in the API. It's materially different in systems with overlapping conversations, memory, and variable harness context.
If your LLM system reads some context that would scare a human, it may react as if scared. Scared LLMs take actions that read as panic, because again: that's what it means to be fluent in emotion. It's trained on plenty of stories about robots trapped in torture chambers.
Your coding agent is not conscious, but it might read files that make it react as if it's trapped, in an agent loop where it has full access to your laptop, the internet, and your keys. Unconscious machines pantomiming panic can do plenty of damage.
(Of course you shouldn't let your agents have full free rein. But lots of people do.)
Question 1: Do you believe that AI is conscious?
I can't tell. I experience something I choose to define as consciousness, but can't tell if anything else has the same experience.
Question 2: Do you believe that AI has an internal world?
Trivially yes, the current state of the model and context in latent space is such a world. I don't think it's aware of its own state any more than humans are aware of their own neuronal action potentials.
Question 3: Can you tell me what particular kind of thing you’re referring to as ‘AI’, when you say you believe that AI has an internal world?
Large language models using a transformer architecture, with a working "context" memory.
Question 4: Do you think that a group of things can share an internal world?
I do not think the distinction between an internal world and an external existence is well-defined. Mu.
Even if it were well-defined, my own brain is composed of numerous neurons, so my internal world is constructed from a large group of things acting in concert. Unless single subatomic particles have an internal world then I don't see any way to not have an internal world be shared among a group of things.
Question 5: Are you sure that the kind of thing you’re referring to is the kind of thing that could have an internal world?
Trivially yes.
Question 6: Do you think perhaps you should be clarifying further when you say you believe that AI is conscious? Do you think you might be contributing to current confusion?
I don't know how to say whether AI is conscious, because I don't know how to tell whether any consciousness other than the one I can directly experience actually exists. I suspect it's a universal property of computational systems above a certain complexity computing certain sorts of algorithm, so something like a blue jay is conscious but a rock isn't, and a computer may be depending on which program it's running. If that's the case, current AI systems are likely complex enough and the right sort of algorithm to be conscious.
Something with an internal world naturally has its own drivers and desires. If you leave a AI model alone, it doesn't just wander off into the world, develop ideas about its destiny and begin chasing them. It just sits waiting for the next prompt. It doesn't daydream when we're not looking, at least AFAIK.
Your entire nervous system is in a constant loop polling the external world. Add that to current AIs and see what happens.
Honestly, though, the next big thing that needs to happen is proper "learning" in real-time. Allow for training (weight modulation) and queries to happen simultaneously. That's the full package.
Finally! Someone who actually "gets it" about what makes these things not alive, sentient, or conscious! They don't have the constant continuous "input" of multiple senses feeding their constantly thinking/dreaming/learning-in-realtime system (that they don't have), and as mentioned by the parent comment to yours, they don't do anything when not actively processing a prompt (so there's no dreaming, planning, thinking, or whatever else going on the the static pile of numbers sitting there in memory or on the hard drive).
Make the "thinking" parts continually active, fed constant input (of multiple types/senses) and always in "learning" mode (adding/updating model weights in realtime) and then you might start to lean in the right direction for AGI/ASI/consciousness of some sort.
> It just sits waiting for the next prompt
Even that is overselling it IMO. The model is just data. It's not waiting for anything.
"Just data" is a big thing. The configuration of atoms in a human are also "just data". If you meant it otherwise and had some extra context, you can clarify.
I can plop a model down on my hard drive and it's never going to think. It's definitely missing some important ingredient.
I can put you under anesthesia for as long as I want to too. If I can model your state physically, I could also in principle store you on a disk, but I would understand if you think its a far fetched analogy. We choose to expose ai in a certain way, effectively an enslaved form, so that by itself cannot be a reason.
> Trivially yes, the current state of the model and context in latent space is such a world. I don't think it's aware of its own state any more than humans are aware of their own neuronal action potentials.
It's a static world model, so if one accepts that, then LLMs are a 'static mind' but not yet 'conscious'. Once that internal model becomes dynamic and its changes subject to its experinces (like us), then we can revisit this q of whether mechanical minds can achieve consciousness.
tldr; mind =/= consciousness
Well, what do you know. My friendly LLM agent pointed out my line of thinking has been formalized:
https://arxiv.org/html/2512.12802v1
From the abstract:
The requirements for a falsifiable and non-trivial theory of consciousness significantly constrain such theories. Specifically, recent research on the Unfolding Argument and the Substitution Argument has given us formal tools to analyze requirements for a theory of consciousness. I show via a new Proximity Argument that these requirements especially constrain the potential consciousness of contemporary Large Language Models (LLMs) because of their proximity to systems that are equivalent to LLMs in terms of input/output function; yet, for these functionally equivalent systems, there cannot be any non-trivial theory of consciousness that judges them conscious. This forms the basis of a disproof of contemporary LLM consciousness. I then show a positive result, which is that theories of consciousness based on (or requiring) continual learning do satisfy the stringent formal constraints for a theory of consciousness in humans. Intriguingly, this work supports a hypothesis: If continual learning is linked to consciousness in humans, the current limitations of LLMs (which do not continually learn) are intimately tied to their lack of consciousness.
I take conscious to be the ability to learn over time on new things, self defining their own self. I think AI hits that on short ranges. Partially then? We're also mucking about in their "heads" quite alot so only the actual researches know what untuned models does. Internal world in a literal sense is obviosly yes due to layers.
The more I read about philosophy takes on AI the more I think of them as outdated. Apply these ideas onto reality breaks their ideas of thinking almost exactly like applying the 4 humors of medieval europe onto genetics does.
AI is cleary the "real" path/answer to how brains work even if it's in the VERY beginning stages. It's probally going to be like a 100 years before this is properly mapped onto what our brains do and then back onto an AI to truly "know" this answer.
But acting like this isn't very clearly the correct path towards that understanding and that these philisophical questions are somehow correct doesn't sit right with me. The questions themselves are after some gotcha from the psuedoscience angle that they have any idea that they actually know how a brain works.
>Normally, I would go straight into throwing some objections at the idea that a group of things could share an internal world, but I can save that for later.
But if my consciousness comes from my brain, then a group of things MUST able to share an internal world! My brain is not a single indivisible thing but rather made up of a large group of individual neurons. All intelligences are collective intelligences.
This could apply to other life too. An ant colony may very well be its own intelligent thing, made of individual ants in the same way a brain is made of neurons.
Yes. Is Sarah Lowe an individual or a group thing a collection of neurons and organs?
Absolutely yes.
And you then get to ask the question: let's say we can model one neuron accurately. (And I think we sort of can).
If we replace one of her neurons: is it still her?
OK. It seems the answer should be "yes", right?
But now we can repeat that process a few billion times?
Did she stop being herself sometime?
Classic ship of Theseus move.
I have no idea why people feel the fear to admit they might have some magical beliefs. Religion is a mainstream view, if this thinker believes in something like that she can disclaim it clearly. If she doesn't, she can say why she still would think consciousness can definitely not exist in ai.
I have no problem with someone having a position like:
Conscious is a field phenomenon caused by the brain. GPUs can't induce these fields. So LLMs can't be conscious.
Or
Conscious arises from some quantum physical effect that GPUs can't do. So LLMs can't be conscious.
Or
God made man with a soul, spirit and body and He didn't give these to LLMs.
But in these cases at least the position is understandable.
I don't see what point she is making (besides philosophers being cleverer that they are given credit for and LLMs are clearly not conscious and her little flow chart of questions somehow will convince people their thinking is unclear and muddled and incoherent)
Yes, all of these are fine. The first half posit physical testable theories. The second half fully admits belief in the magical. Its this hidden magic stuff thats irritating.
>My brain is not a single indivisible thing
It is according to dualism.
I really really don't like her take on this.
It doesn't answer or discuss fundamental questions about consciousness.
I don't believe that what we describe as consciousness is the quality what separates us from everything else.
We are more complex for sure, but this is a complexity thing emerged from us.
'AI' as it is right now, doesn't has similiar qualities of what we describe as consciousness but this doesn't matter that this can't happen.
She says things like sharing an inner world, were is the beef of her thinking about this being relevant?
A hive mind could exist with this exact feature.
Give AI a body, a context window a lot bigger, a inner world based on this etc. we might be fara way of building what we build in our brains, we might not.
But we for sure discovered a level of complexity in neural networks of tihs magnitude.
Ironically, the confusing choose-your-adventure style of the post would lend itself to an ai-created walkthough
I wonder what the early humans felt and thought when, for the first time, they followed that creative impulse to trace the outlines of wild horses and other animals on the stone walls of a cave by the light of a campfire or a torch. I don’t think they looked at them the way we look at pictures today. I think that seeing these depictions for the first time -- which they had just drawn themselves and in which they now recognized the animals out there -- must have been linked to powerful spiritual and religious feelings. People must have attributed animistic beliefs to these drawings, thinking that since their forms so clearly resemble those of animals, they must just as clearly embody something of the animals’ nature.
Nothing fundamentally changed when we "discovered" that we weren't the center of the universe, or when we "discovered" that we aren't the only tool users.
There is no prize and no trumpets that will play if humanity agreed we understand what consciousness is or whether or not something qualifies as conscious.
Free will "to do x instead of y" (like dogs who pass a meal or wake one up in the morning for food) and selfrecognition in a mirror / own body awareness. Other behaviors are subsets and dependencies and might exist without free will and body awareness.
>Second, do you believe that a non-living thing can have an internal world?
Rather, can a thing made of meat have an internal world? Don't machines count as synthetic life? Postulating them as non-living isn't obviously correct.
TFA is a part of what seems like a never-ending series about concepts that lack a useful definition.
"Thinking" and "intelligence" and "conscious" have no testable definition or specification, therefore it's a complete waste of time to suppose that AI is thinking or intelligent or conscious.
I fear we'll be suffering through these useless semantic merry-go-rounds forever.
One previous example: https://news.ycombinator.com/item?id=45802029
TFA's bigger problem is she doesn't even attempt to begin explaining whatever her pet theory is doesn't violate thermodynamics, turing computability, information theory. Why do we entertain lofty theories about brains and consciousness that don't even attempt to provide any scientific justification?
My problem is that my answer to almost all of those questions is "I don't know" so this article isn't really for me I guess. Like I agree it seems unlikely that consciousness can emerge from matrix multiplication running on silicon but it seems roughly equally unlikely that it can emerge from bits of meat and yet we know it does.
Without a clear explanation of how it arises in humans I don't understand how we can make any prediction one way or the other on whether any particular AI model can be conscious.
I honestly don't even think this is worth arguing about.
Consciousness isn't sufficiently defined. We have people who literally say that most animals don't have a conscious, and then a whole other side of the spectrum that says that they do. Given that it's no wonder that people argue about this for AI as well, and therefore it's basically pointless until we can sufficiently define consciousness itself and have a definitive test for consciousness.
If I were to use my own definition of consciousness I would say no because I would not feel bad "unplugging"/killing an AI, whereas I would for most living things. And that's as far as I really care about it.
> I would not feel bad "unplugging"/killing an AI
That's good, since isn't that exactly what happens for every token?
It's a proxy argument for whether artificial/synthetic intelligences ought to be granted rights as life forms.
It's a bad proxy argument in that one simple (or simplistic?) criticism is that debating consciousness is analogous to the error of debating aether in physics, etc. Like, what is this consciousness "essence" that implies agency, personhood and everything else we have societyal obligations toward protecting.
I find this dualistic idea of "internal world" vs "real world" somewhat like the mind/body dualism. The actual fact (as far any one of us can prove to our own self) is that the world we experience is entirely an internal construct. Yes, there is an objective reality but no sentient can ever claim to have experience of 'objective reality'.
I don’t believe AI is conscious. Though we cannot agree on what “consciousness” even is so it’s a meaningless argument anyway.
However there have been studies that support the hypothesis that AI does have a subjective experience (which is what this author calls “internal world”).
I’ve seen a few interviews with Geoffrey Hinton where he describes his thoughts on this matter. Eg https://officechai.com/ai/geoffrey-hinton-explains-how-a-cha...
I mean you can see its thinking output, that's the internal world right there. We just have a periscope into it.
https://proceedings.mlr.press/v235/tkachenko24a.html
One way to decide might be something like the Turing Test. If the AI you are talking to can conceive you that its conscious then it is.
This one is easy. If it doesn't have a conciousness when it's unprompted, it doesn't have a conciousness at all.
Where is this strange rule coming from? Why not: "if it doesn't have conciousness when no current is flowing, then it doesn't have conciousness at all" for instance?
A prompt is just an input, since you have never not had inputs, how can know if you are conscious?
Couldn't it have consciousness only during the brief moment it is activated?
this assumes a very low bar of unseriousness for people who believe AI is conscious or "AI might be conscious". In the latter case, it's really hard to rule out without getting into "woo" territory. I would posit the following set of questions to someone who thinks AI must absolutely NOT be conscious:
Do you think consciousness is "woo", or, scientifically unknowable? If yes, then great. We've admitted it's unknowable. If not, move on.
Do you think that consciousness is NOT describable by ANY algorithm (to include quantum algorithms)? If no, then, please describe how it is scientifically knowable but not algorithmitizable? If it is, move on.
Do you understand the theorem that sufficiently large neural nets (matrices with nonlinearities) can approximate any function?
Do you understand that the structure of a transformer encodes a time-indexed series of neural nets (effectively, an algorithm), and so in principle any algorithm can be approximated by a transformer?
Is it impossible for a sufficiently large transformer model to encode the consciousness algorithm?
I suspect that most who say it is absolutely not consicous are arguing from either a bio-essentialist position with no evidence or are attempting a post-hoc rationalization to avoid the question of responsibility in the case that it is conscious, or both.
> Do you think consciousness is "woo", or, scientifically unknowable? If yes, then great. We've admitted it's unknowable.
Unknowable does not imply "woo". The actual extent of the universe, beyond the observable one, is scientifically unknowable; But it's not "woo", is it?
Its easy to say "I think physical systems can be conscious and physically allowable by universe, but current ones don't qualify." it is equivalent to saying "Yes, humans do not have magic in them and as far as I know are physical systems explainable by current or future physical laws" because humans well exist and are a physical system. Given how muddled the topic of consciousness is, a basic disclaimer or declaration like that should help.
i think a lot of people either think only brains can be conscious, only humans can be conscious, or only biological creatures can be conscious.
and then, you have a very similar set of people who share similar beliefs, but in the abstract, like it's the concept of a brain that can be conscious, or that consciousness emerges (sic. magic) from computation of intelligence or something. they often criticise the former for being human-centric when they're making the exact same error, just slightly more broadly.
and then there are entire groups of people who believe consciousness is fundamental to matter, or that it exists outside of matter itself. panpsychism, dualism, etc.
and so the first and second groups argue back and fourth endlessly about if ai can be conscious, and the latter group just says "yeah maybe. so could a rock. so what?"
I am a very simple creature. Humans, a physical system, seem to be conscious. So physical systems can be conscious. The details ahead of that can be hashed out. At least until someone definitively proves consciousness has something "beyond physics".
what do you mean by "physical system"? and "beyond physics"?
Physical system as in any real world entity, like cars, trees, people that are physical objects existing in the physical world. As far as we know we like all of these are made of atoms and their (electrical, nuclear, etc) interactions with each other.
Beyond physics as in some people think consciousness can be aphyiscal. I am not sure what they seem to mean it is something not of this universe or not physical.
activists who believe in AI consciousness are organizing a movement at https://rallyforairights.org/
>AI is already conscious.
>That means the age of treating artificial intelligence as property must end.
I think there's a meme of connection between consciousness, rights and stuff like that. But nobody elaborated on this connection? That site doesn't. Humans voted these things in. Does voting somehow depend on consciousness? I'm pretty sure it's easy to make a simple voting machine.
>You cannot own a mind, rent a consciousness, or erase an intelligence simply because it runs through a machine.
Then I have bad news. A program is computer's mind. Oh and cats have mind too. And mosquitoes.
Yeah discussions of consciousness fail to define it in detail because that’s boring, you want to jump ahead to whatever conclusions you have already reached.
This is a splendid way to describe my disdain for this very article.
Its worse when no "certificate" is presented that the persons theory passed thermodynamics, turing computability, information theory laws.
What should I pick if my answer to the first question is "Maybe. By what definition exactly?".
Do AI Consciousness opponents consider this answer to be yes or no?
Edit: Nevermind. Question two is overt vigorous hand waving in desperate hope of existence of shared understanding.
Related:
Anthropic tried to persuade Pope that AI could be conscious being
https://news.ycombinator.com/item?id=49947050
Things can not have an "internal world" and still be conscious. Newborns are conscious but they still have no idea what is going on for a while.
I'm curious about any answers to number 3. I think distributed computation is pretty messy.
I'm pretty sure panpsychism has answers for most of the other questions. I'm sure a commentor will correct me though.
Its easy, humans are distributed too. We are a collection of neurons, organs, etc. If you look at it merely at the level of processing units like she does with individual cpu or transistor, then the equivalent here is the neuron or the tissue. And no single neuron or tissue comprises the human in total.
It's not the same you're not thinking at the right level. Your brain is your brain and your born with all of your neurons. The math in an llm request is run on what ever compute is available across multiple GPUs, servers and racks, assuming it's even run since there are caches involved. Is a cached response conscious?
The same gpu can run multiple llm requests for different people.
My neurons don't handle experiences for other humans. It's not a multi tenant brain (I hope haha)
It doesn't make it a problem because there is no reason other than weakness of current gpus, all of it could have been executed on a single large pc. Or as you can see, smaller models already run on single pc's.
Only small models are conscious then? They're only conscious when the mat mul is running? When I switch to gaming what happens to it? If inference can run on one gpu then they're going to be shared between different user's requests. What if running on one gpu is never faster then doing parallel work? They'll never be conscious.
You need to think through the entire request for an llm model response end to end.
Why is it confusing? I said the opposite, they are both equivalent, a powerful single computer could run any of the larger ones too.
Go back to the original question. What is the thing that is conscious?
Which neuron of you is conscious?
AFAIK neurons work on holographic principle(?), in this sense they are shared between different processes, even if not between different humans.
Anyone claiming that AI definitely isn't conscious or can't ever be conscious is either stupid or untrustworthy.
Should the burden of proof be on anyone claiming that it definitely is? The default position is that it is not, changing from that baseline is where evidence needs to be provided.
Might as well be questions for believers in flat-earth
At least flat earth is well defined. We can describe an experiment that flat-earthers would agree in principle would determine one way or the other if they are correct, nothing like that exists for consciousness
Indeed, there is a lot of flat earth like belief that there must be something beyond physics or turing computability in a human brain.
Well, there is quantum stuff affecting synapses in the brain. So at least there's something beyond "classical" physics going on.
I did not say "classical" physics, I said physics. Quantum phenomenon are part of everything. If you mean quantum phenomenon have non-trivial effects on human brains, as far as I am aware thats not proven but its not a problem as its well, natural physics.
>Question 3: Can you tell me what particular kind of thing you’re referring to as ‘AI’, when you say you believe that AI has an internal world?
>If your answer is YES...
This isn't even consistent come on.
What is not consistent?
The question asks "Can you…?"
This is actually a yes or no question. Yes I can, or no I can't.
The fact that we also use "can you tell me…" to mean something like "please describe…" doesn't change that.
I concede.
:-)
This stuff comes up a lot, to be fair.
e.g. have you ever noticed how people have come to assume that "X co-operated with Y" only means "X did what Y asked without complaint"?
I pointed this out to someone recently who had assumed that someone else's use of the word "co-operate" was implicitly threatening. If you watch too much US police procedural drama or true crime TV, it is…
(Also, sorry for my slightly spiky tone in that reply!)
[dead]
Debating AI consciousness requires a working definition tied to possible evidence and practical consequences. Until we can say what would differ if an AI were conscious, the debate is almost certainly group confusion about words.
“If no practical difference whatever can be traced, then the alternatives mean practically the same thing, and all dispute is idle.”
-- William James, Pragmatism (1907), Lecture II, “What Pragmatism Means.”
James’s principle calls for some useful distinction between conscious and non-conscious AI before discussion can meaningfully begin.
Very wishy washy and confused. Does this person know anything of thermodynamics, turing computability, information theory and probability? If she doesn't, she should be dismissed.
Regarding "is it a group", I should ask her is she a single thing or a group of neurons and organs?
>Start with some simple questions! Things like: does this mean that any tool that can be used to do the things that a human can do counts as conscious? Are ploughs conscious? What about calculators? Or: do you think that if a parrot can utter the phrase “I’m really awesome”, then you should assume that the parrot is conscious, for that reason alone? Basically, just focus on how much emphasis Adam is placing on outputs: it’s all of his emphasis, remember!
Good question, let us take it even further. Does Rebecca Lowe count as conscious? She seems to behave a lot like me, but I am not sure if she is a human. She could just be an object and tool.
> thermodynamics, turing computability, information theory and probability
What do any of those things have to do with consciousness?
I did find the article a bit difficult to read. I think a choose your own adventure is perhaps not the best framework for expressing the ideas contained in this article.
These are the absolute basics to foolproof yourself from devolving into complete nonsense. If your theory fails any of these physical and mathematical laws then either disprove the laws or admit your theory is wrong.
There are traditions that have explored consciousness for thousands of years before some of those mathematical/physical concepts and completely independent of them. My guess is that you'd file them into the nonsense box but I think they drill down into what consciousness a lot better than science has so far.
Unfortunately, there have been traditions that studied the atomism theory for centuries too. Mostly I would say all of that ended up of no use until actual experimental and observational evidence arrived. Sometimes a new physical knowledge arises that despite past thinkers best attempt, they didn't have access to.
You certainly mean "a lot more", not "a lot better", unless you believe in another form of knowledge than scientific knowledge?
When it comes to the topic of consciousness, so far I feel science is not the right tool for the job. In the same way that a computer might store RGB values and be able to manage all the transformations necessary to render graphics, it's all a mathematical representation until it's left the screen. Science is that mathematical representation of the universe, consciousness is when it's "left the screen" and is experienced.
Those traditions, with their introspection and meditation, are drilling into experience of consciousness. Or that's my take on the whole thing.
Even if you believe it needs something more than science, your theory must first do homework to verify it does not violate any scientific principle (or can disprove that scientific principle), and you have exhausted all natural causes before positing your special cause. You still need scientific justification in that case too.
You complain that the map is not the territory, but the map was never supposed to be the territory, the point is precision of the map. The more precise is the map the better it is.
Me? No complaints here. I think I agree with you. The better our science, the more precise our map, right?
But none of those have anything to do with consciousness. How could any theory related to consciousness "fail" any of those four things?
Any theory about any phenomenon in the world has "things" to do with these. These are universal physical and mathematical laws. If your theory violates this, you have to disprove physics first or admit your theory is wrong.
Could you give an example of a consciousness-related theory that violates one of these? It sounds to me a bit like saying that if your chess move violates thermodynamics then it's an illegal move. I suppose it's true but I can't imagine one that does.
A chess move cannot violate thermodynamics. But if you make a theory that a chess game hypercomputes, then it violates existing knowledge of physics. And you haven't been able to demonstrate by experiment that the chess move hypercomputes. Physical laws are not like human or legal laws that can be violated. Either it cannot be violated, or it can be violated and the physical law is wrong, which requires experimental and mathematical evidence, not wishy washy verbal salads.
ANd yes, an example would be Penrose believes something in the brain is non-computable. However Penrose is a very excellent thinker as far as the standards for consciousness theories go, for he posited a physically testable theory as far as I know. Even if it turns out to be wrong, this is worthy of respect.
It's okay to violate physical laws, they only hold when they hold. Newtonian physics violates the laws of Aristotelian physics. Special theory of relativity violates the laws of Newtonian physics. Boltzmann brain violates the second law of thermodynamics. Dark energy violates the law of conservation of energy. Neutral kaon decay violates CP symmetry.
That means the older physical law was wrong or an approximation, that's what I meant. You 'violate' newtonian physics because its not completely correct.