As Majromax stated, if the game is framed such that losing vs MAD have the same penalty, then it is not at all representative of reality. It is set up to be a "you miss 100% of the shots you don't take" game.
As an important aside, I hope that everyone is aware that as of 2025, the annual US military funding law (the NDAA) established a prohibition by Congress prohibiting AI from automating nuclear launch. This was already US policy and is common sense; now it's also part of federal law.
This is strong evidence against LLMs experiencing qualia. (I know that that topic often gets people laughed out of the room but please don't jump on me for engaging in that debate. When we can collect evidence and be able to show it to people.)
How did you jumped to qualia from consciousness? I can see how this is a strong evidence against LLM being conscious, but to my mind it doesn't imply in any way or form that they do not experience qualia.
Or... well, ok, maybe they can't experience if they are not conscious? I see how this can be argued, but I still do not agree. I'm sure qualia is created not by consciousness (I would notice if it was), and I'm sure it is created not for consciousness specifically, it must have some other uses too.
No disagreements with what you said in your first paragraph.
> I'm sure qualia is created not by consciousness
Whether or not qualia is created by consciousness, I don't see how we necessarily can tell one way or the other. We don't exactly have great introspective tools to do such self analysis, not to mention what we think we feel is often illusory/not reflective of reality.
Yeah, I think you are right. "The content of consciousness" and "consciousness" are different things, and I mixed them up, when writing that sentence. I can watch the content, but not other parts that may be producing it.
Moreover, thinking about it, I come to a conclusion, that if I cannot reflect on qualia creation, then it is a (weak) evidence for qualia created by consciousness. I suppose the consciousness is harder to reflect on than other things, hard to map it into states of the content of consciousness. Like, I can reflect on my vision and see some hints on how I get these wonderful pictures, despite it being definitely not consciousness, I can reflect on how I produce or decode language. And to my mind it is because consciousness was devised to reflect on these things, so I could report on my observations to others. But to reflect on itself is a wholly different matter.
I would not deny it. The reason people accept current evidence is, after all, because they can relate to the experience of qualia, even if there's no complete objective understanding of it yet.
How can qualia be experienced if not from a conscious observer? It's the same question as asking if a tree falls in the woods with no one to hear or know about it.
People have different notions of what “qualia” means. To me, the experience of qualia is the perception of qualities, or of the texture, of inner processings of the brain. Not consciously experiencing them doesn’t mean that these qualities aren’t there, just like not consciously experiencing sounds wouldn’t mean that the sounds aren’t there, and may be unconsciously processed.
I don’t think that “experience as such” makes any sense. Experience is always of something. And that in turn implies that the something that is being experienced also exists independently of it being experienced.
You're saying the discovery that humans can process language without being conscious "couldn't possibly" inform the debate about LLMs? When that debate is literal predicated on the assumption that the ability to process language implies consciousness?
This is a counter example to the fundamental assumption of that argument. Without that, you are left with something like "if we ignore their ability to to process language, do we have any reason to suppose that LLMs (as opposed to, say, a spread sheet or stats package) are conscious?"
Sorry to hear that someone rudely thinks that basic logic is "Nuts".
> When that debate is literal (sic) predicated on the assumption that the ability to process language implies consciousness?
This is an incoherent claim. Debates are between people with differing claims and often differing assumptions; they aren't "predicated" on some assumption or another--that's a category mistake.
Someone can easily argue that LLMs are conscious (or have qualia--that was the disputed claim, and they aren't the same thing) without the strong claim that the ability to process language entails consciousness ... perhaps it is the processing of language together with other features that they think indicates consciousness. For instance, George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them.
If LLMs did not process language as well as they do, we would not be having the argument.
The only reason we are having the argument at all is that people see LLMs responding appropriately to language, and _from_that_ conclude that LLMs may be conscious. You even sneak this in yourself when you say "George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them" -- in other words, they wouldn't have had judgements in the first place if the LLMs had not "said things to them".
I don't see how that follows. The brain could be experiencing all sorts of things while processing, but simply not record it, and so of course the person will have no recollection of experiencing anything.
I broke my leg recently. Shortly after that I've lost my consciousness. It was very painful, the body reacted with a lot of adrenaline, and after a several minutes when adrenaline was drained away my consciousness was drained too.
I experienced something like this several times, though not to the point of fainting. But this time was special in other way too: I had friends near me, they observed me through all the process and we could compare our observations later. It seems, that my memory stopped recording before I fainted. I was still operating to some extent, but I couldn't remember a thing. When asked something I grunted in answer. When one of my friends insisted that I stand up and come to a better place to sit down, I actually stand up and did several steps before stopping and slowly (and carefully) sank to the ground. (An interesting observation, my controls over my body were weakening, but I was still using them for what they worth. It fits with all other similar experiences: the limbs and all the muscles seem to be losing their strength, and it takes a lot of will to make them work.)
On the light of this, I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state. Did I feel myself extremely bad at the time? Or maybe I didn't feel anything? My friends are sure that the former statement is true, but they may be mistaken by my outside looks. I personally don't remember. Up to some point I remember that I felt really bad, but the next thing I remember I look at the sky and I'm surprised by what I see (I was not in a place I expected to be). And at that moment I was pretty ok already, no more adrenaline issues, just my leg was aching.
Was I experiencing qualia is another interesting question. I'm pretty sure I was, but I'd like to hear an argument for the opposite.
Maybe it was a different part of your nervous system experiencing them, akin to a BIOS versus the operating system. The brain is a very complex and fractal thing, it is entirely possible that a more basal part of "you" took over for a very traumatic part of your life, very similar, but not exactly, to those with multiple personality disorder act.
> I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state.
I’m not one of these proponents, but to play the devil’s advocate: The fact that you can’t remember it doesn’t necessarily imply that you didn’t fully consciously experience it at the time.
I feel like it might be strengthened by a machine that actually does implement the core of the argument (even passes the turing test?) but is also just matrix multiplication. Previously the idea of a device that could respond to input with human-like output was a fantasy, now it's reality which removes one of the arguments against the Chinese Room on the basis of plausibility.
But yes, I would like to see a modern philosophers take on it.
I'm more interested in whether the uninformed intuitions have changed, whether someone who used an LLM before graduating from school and entering university has a more forgiving stance than an "ignorant" college entrant from ten years ago. From my experience of the field, I would really doubt many philosophers of mind have changed their view based on this. The major questions of interest concern qualia, it ha been so for many years.
Plausibility was never an argument against Searle's Chinese Room Argument ... that's a very deep confusion, since Searle was arguing against the possibility of Strong AI, whereas advocates of Strong AI of course thought it was plausible and that's why they were working to create it. That you conflate the Chinese Room Argument with "the Chinese Room" is one aspect of the confusion.
It’s absolutely about the plausibility of mechanistic intentionality, given the implausibility of intentionality where the person in the experiment doesn’t understand chinese
Searle's argument may well be about the plausibility of "mechanistic intentionalty", whatever exactly that means ("mechanistic" just sounds like bigotry to me ... we are all mechanisms and Searle didn't say otherwise, just that we are meat mechanisms and not purely syntactic mechanisms ... and his argument was intended as a logical proof, not just a plausibility argument), but that's not what the previous comments were about. Apparently you mechanistically see the word "plausibility" and think that any and all statements about it refer to the same thing.
There are arguments made specifically about the implausibility of such a room making the argument itself invalid. Or I guess I should say “we’re”, because that position is now much less tenable with LLMs.
"For [John Searle's] part, he has one argument, the Chinese Room, and he has been trotting it out, basically unchanged, for fifteen years. It has proven to be an amazingly popular number among the non-experts, in spite of the fact that just about everyone who knows anything about the field dismissed it long ago. It is full of well-concealed fallacies. By Searle’s own count, there are over a hundred published attacks on it. He can count them, but I guess he can’t read them, [...]"
Every significant argument has plenty of detractors, that doesn’t mean they’re right. Every argument I’ve heard to the contrary is unconvincing to me, and to the majority:
Do you prefer that we not even try to spread beyond our one planet, when an entire galaxy, or maybe even the neighboring ones, might be in reach if we try? What if someday at the very end of the lifetime of our sun and similar stars, we look back, and regret not trying?
> Do you prefer that we not even try to spread beyond our one planet, when an entire galaxy, or maybe even the neighboring ones, might be in reach if we try? What if someday at the very end of the lifetime of our sun and similar stars, we look back, and regret not trying?
The timeline you are speaking of is in billions of year. Yes, in that timescale, it definitely makes sense to try.
This very century, there are very serious scientific concerns about the continued comfortable habitability of Earth and the ensuing geopolitical instability caused by the accumulation of greenhouse gases in the atmosphere, the sixth mass extinction event, etc. We have solutions to mitigate those to some degree, but this requires very significant resource allocation to that goal, and so far it seems possible that we would fall short.
I don't think there should be zero resources allocated to space exploration, but it's at least reasonable to question whether we have our priorities set right.
Exactly what I mean. People who obsess over space are so egotistical they can’t imagine someone not thinking it’s “the best”. It’s interesting, but no more important than any other science or field of study.
And no, I’d prefer we take care of our home planet and make it a sustainable place to live first. It’s ludicrous to think we have any chance of surviving elsewhere if we can’t stop ourselves from making our natural habitat unlivable.
Yes we can do both. I still think it’s stupid to place human spaceflight on a pedestal. It’s putting the cart before the horse.
I don't agree with you. An endeavour like this is bringing humanity closer together. After the moon landing people all over the world said "we landed on the moon", not "the US landed on the moon". After they released the photograph of earth as seen from the moon people started to see earth differently, and a flurry of environment protection laws followed.
I too wish we would all hold hands and sing kumbaya and collectively decide to simply make things better here on Earth but sadly it seems far off.
The best alternative demonstrated so far seems to be for some of us to push the limits of what is possible and watch all boats rise with the tide. Better this than slinging Tomahawks at school children.
> People who obsess over space are so egotistical they can’t imagine someone not thinking it’s “the best”
What? No I don't. Some people want to spend money on sports or making our planet sustainable before doing anything else. I think that's fine. It doesn't interest me, however, so I'm looking at this other thing. We can coexist without having to insult each other.
> It’s putting the cart before the horse
How many times in history did necessity create efficiencies versus ambition?
Fair, my first point was poorly worded. People who think space is the best and can’t imagine anyone else thinking it’s not the best are still self centered and lacking in empathy imo.
> How many times in history did necessity create efficiencies versus ambition?
Literally all the time? There’s another saying, “necessity is the mother of invention”.
There have always been lame people like this and for all of human history we have continued to explore and expand. I think at this point those people can safely be ignored.
But that was exactly the point I was responding to, no? If NASA was fine with skipping the astronauts, then they would just send it unmanned, not find a random volunteer.
especially not one that may chicken out ( "very not brave" ) and destroy the cabin from the inside out by any means necessary (bashing at walls, pissing in cracks, etc.)
As an important aside, I hope that everyone is aware that as of 2025, the annual US military funding law (the NDAA) established a prohibition by Congress prohibiting AI from automating nuclear launch. This was already US policy and is common sense; now it's also part of federal law.
10 USC Ch. 24: NUCLEAR POSTURE https://uscode.house.gov/view.xhtml?path=/prelim@title10/sub...
FY2025 NDAA, Section 1638 https://agora.eto.tech/instrument/1740
reply