There is evidence that awareness is an emergent property from sensory experience. And consciousness is an emergent property of language that has grammatical meaning for self and other.
How do you know the sensation of a red photon hitting a cone cell, transduced to the optic nerve through ion junctions and processed by pyramidal neurons, is any more or less real than the excitation of electrons in a doped silicon junction activating the latent space of the "red" thought vector? Cause we are made of meat?
You’re arguing against the opposite of my position. I am arguing that LLMs have a reasonable basis to be seen as conscious because there is nothing special about biological neural networks.
Sensory data is a specific data set that corresponds to phenomena in the world. But to say that LLMs don’t have senses merely because they are linguistic or computational doesn’t follow when they can take in data from the world that similarly reflects something about the world.
They don't have senses because they don't have a body. It's just a program. Do weights on a hard drive have consciousness? Does my installation of starcraft have consciousness? It doesn't make any sense.
Bodies aren’t necessary for senses. I can send a picture to Claude. I can send a series of pictures. That’s usually called a sense of vision. I could connect it to a pressure sensor and that would be touch.
The weights on your hard drive might have consciousness if they can respond to stimuli in ways other conscious brains do. That’s the whole point of the Turing test, it’s a criteria for when the threshold of reasonable interpretation is crossed.
I’m not sure I fully understand the distinction you’re making, or if I do I’m not sure I agree. Concretely, I agree that these are very different mechanisms. Abstractly… I agree that an LLM cannot be burned. I’m not sure I agree, though, that there is a significant conceptual difference between thermoreceptors in the skin causing action potentials to make their way up the spinal cord to the brain is all that different than reading a temperature sensor over I2C and turning it into input tokens.
Edit: what they don’t have, obviously, is a hard-coded twitch response, where the brain itself is largely bypassed and muscles react to massive temperature differentials independently of conscious thought. But I don’t think that defines consciousness either. Ants instinctively run away from flames too.
The human Brain is a neural network. Your sense of “knowing what warmth is” reduces down to the weights of connections between neurons in an analog of LLMs. What is different about the human brain that warrants saying that the same emergent characteristics for one network are inaccessible to another?
You really don't think there's an experiential difference between putting your hand on a hot stove, versus reading the text "the stove is 200c, and will hurt if you touch it"?
That’s not a charitable reading of my argument. All senses are of different “kind”. That doesn’t mean that the lack of a sense of touch means that LLMs can never have senses. That is a very broad claim.
We don't have a way of measuring "cognitive awareness" though. We have a way of measuring electrical impulses, and how they behave in response to various treatments (eg anaesthetics or magnetic fields), but we can't objectively measure whether the system is aware at all.
We can measure electrical spikes, and we can ask the system to reply what it experiences when various spikes occur. Guess what: we can do that with ANNs now too.
It'd be one thing if this were all a philosophical discussion, but in this thread so many folks are making very firm statements about the nature of reality we have no means to back up.
LLMs have no self, sensory experience, or experience of any kind. The idea doesn't even really make sense. Even if it did, the closest analogy to biological "experience" for an LLM would be the training process, since training at least vaguely resembles an environment where the model is receiving stimuli and reacting to it (i.e. human lived experience) - inference is just using the freeze-dried weights as a lookup table for token statistics. It's absurd to think that such a thing is conscious.
What is different about the human neural network? People have given LLMs sensors and they respond to stimuli. The sense of self can be expressed as a linguistic artifact that results in an emergent pattern recognition of distinct entities. For example, merely my saying I am sitting under the tree with a friend I have encountered the self as a pointer to me as the speaker. There is evidence from early childhood development that language acquisition correlates to awareness of the self as distinct from other. And there is evidence from anthropology indicating that language structures shape exactly what the self is perceived to be.
Your best argument is that the weights are set because that means it’s not a system that can self reflect and alter the experience. But I don’t see why that is necessary to have an experience. It seems that I can sense a light and feel its warmth regardless of whether my neurons change. One experience being identical to another doesn’t mean neither was an experience.
The sense of self may be an emergent property of the grammatical structure of language and the operations of memory. If an LLM, by necessity, operates with the linguistics of “you” and “me” and “others”. And documents that in a memory system and can reliably identify itself as a discrete entity from you and others then on what basis would we say it doesn’t have a sense of self?
> Can such an algorithm reason about itself in relation to others?
No, but an LLM doesn't do that either. An LLM is an algorithm to generate text output which can simulate how humans describe reasoning about themselves in relation to others. Humans do that by using words to describe what they internally experienced. LLMs do it by calculating the statistical weight of linguistic symbols based on a composite of human-generated text samples in its training data.
LLMs never experienced what their textual output is describing. It's more similar to a pocket calculator calculating symbols in relation to other symbols, except scaled up massively.
Toddlers learn over the course of several years of observing training data and for the first few years misspeak about themselves and others. What’s the difference?
That they do it at all is the point and is what separates then from MP3 encoding algorithms. The "how" doesn't seem to me to be as important as you're suggesting.
You asked a hypothetical above about a different algorithm and now we've ascertained the reasons why that was reductive.
> LLMs never experienced ...
What is experience beyond taking input from the world around you and holding an understanding of it?
> The "how" doesn't seem to me to be as important as you're suggesting.
When the question is understanding the true nature of what is occurring (eg "is an LLM conscious"), the "how it works" is critical. For example, the 1700s "Mechanical Turk" automaton which appeared to play chess (https://en.wikipedia.org/wiki/Mechanical_Turk). Royal courts and their advisors accepted that it played chess after glancing at the complex gearing inside the cabinet. Had they taken the time to examine how the internal gearing worked in greater detail, they would have arrived at a more accurate understanding of the device's true nature.
> That they do it at all is the point
True in some cases but not others, especially when external appearances can be deceiving. The Mechanical Turk was: 1. Designed to deceive, and 2. Not able to mechanistically play chess. Conversely, LLMs were not intentionally designed to deceive but they can still be misleading because they're a novel system which: 1. Manipulates linguistic symbols in highly complex ways, and 2. Can instantly access vast quantities of detailed information pre-trained into it's relational database that's been indexed across thousands of dimensions. And these abilities are not only novel but can be highly useful for some real-world tasks. This makes LLMs uniquely challenging for humans to reason about because LLMs are specifically tuned to generate output which closely mirrors the exact ways humans assess intelligence (and consciousness). We couldn't have designed a system to be more ideal at playing Turing's 'Imitation Game' and convincing humans they are human-like if we'd intentionally tried to.
In fact, I've previously described LLMs as accidentally being "the most perfectly deceptive magic trick ever" (while I'm a technologist professionally, I've spent quite few years designing actual magic tricks as a hobby). Designers of magical illusions joke that "the perfect floating lady trick" would actually be able to do useful things like replace a forklift, since it could float anything, anywhere instead of just appearing to violate physics. LLMs actually can really do useful things and replace some human labor but that fact doesn't mean they have all the abilities and traits of humans nor that they internally function in similar ways.
> You asked a hypothetical above...
That wasn't me, it was another poster.
> What is experience beyond taking input from the world around you and holding an understanding of it?
In the view of many leading philosophers of mind (Dennett, Chalmers, Nagle, etc) "Experiencing" is quite a bit more than just sensing, processing and recording. They use the term "Qualia" (https://plato.stanford.edu/entries/qualia/) which is what they're talking about when they ask "what is it like to be a..." (wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat)? While the philosophical debate around why material reductionism can't explain human consciousness is fascinating, we don't need to go there to understand "what it is like to be an LLM" because we already know the answer: it's not like anything - there is no there there.
First, it's obvious we can't trust what the LLM's textual output says when it's asked "what is it like to be you" because it's an 'imitation machine' trained on 100% human sample text. The algorithms were designed, tuned and tested to generate text output which most plausibly simulates how a composite human would respond to the input (including the invisible system prompt instructing: "You are a Large Language Model, not a human"). We even added a tiny degree of random variability to the processing of the statistical weights because we found that makes the simulation seem a bit more plausibly like what a composite human would say. In short, the 'self-reported' textual output of a system purpose-built to generate plausible human-like textual output can't be trusted any more than a study of pathological liars can trust self-reported data from their study population.
Fortunately, with LLMs we can directly look under the hood at how it works and the entire specialty of Mechanistic Interpretability exists to do exactly that (https://towardsdatascience.com/mechanistic-interpretability-...). So we know with certainty that, despite what they may say, LLMs do not experience qualia in the way that humans and even other mammals do (which we have insight on from 'looking under the biological hood' with fMRI, surgical and brain injury studies).
Then the only question left is whether to redefine "consciousness" in some new way very different from "human consciousness" or "consciousness in mammals" (the only examples we've had until very recently). Personally, I think it makes little sense to radically redefine consciousness to include statistical algorithms running billions of matrix multiplications on a massive database of human-generated text. The term "consciousness", as vague and poorly defined as it is, was created to refer to human, or at least biological, consciousness. I'd be fine with creating a new term to refer to whatever novel traits of LLMs someone wants to quantify but they should leave the term "consciousness" out of it because the poor thing's already barely useful and stretching it further will just leave it broken and devoid of any meaning.
How are you sure it doesn’t reason about itself? The grammar of languages encode the concepts of self and others. LLMs operate with those grammar structures and do so in increasingly accurate ways. Why would we say humans that exhibit the same behavior are inherently more likely to be conscious?
By the laws of physics, it's pretty clear we don't. The same chemical and electromagnetic interactions that drive everything around us are active in our brains, causing us to do things and feel things. We feel like we're in control of it, we feel like there's something there riding around inside. We grant that other people have the same magic, because I clearly do. But rocks, trees, LLMs, those are not people and clearly, clearly not conscious because they don't have our magic.
Hard disagree. We reliably operate with the concept of a self that’s distinct from others. The chemical and physical processes change in response to stimulus.
Indeed. We assume a lot, because we don't know. We don't have have settled, universal definitions of what consciousness means. But that also means that while we like to rule out consciousness in other things, we don't have a clear basis for doing so.
I'll bite that bullet. In fact I contend the idea that "humans and maybe some animals are conscious, but other things are not" is the special pleading stand. Why are the oscillating fundamental fields over here (brains) special, but the oscillations over there (computers, oceans, rocks) not? If they are, where do you draw the line? It smacks of "babies dont feel pain" (widely believed until the 80s! the 1980s!) sort of reasoning.
Actually I don't really have any problems with panpsychism. It's a pretty uncommon perspective, but when discussing conscious machines, it at least presents a consistent criteria for consciousness.
It's not an ad hominem. In fact, it's perhaps the most good faith interpretation of your words possible. Ad hominem would be calling you stupid because you obviously know that you have a self and only your own stupidity could explain your inability to see how your self is generalisable. When you go around pretending you genuinely think maybe humans don't have selves, really the only way to take you seriously is to think that maybe you're a p-zombie.
