What I've observed both in the comments here, and in my professional network is that people tend to fall into two distinct camps:
1) Those that use git like a crude autosave who then squash on merge
2) Those that prefer neatly wrapped, fully functional atomic commits
It seems those ideas are in direct opposition to one another with 1) being more common in my experience, perhaps as github naturally supports it better plus the fact that stacked commits can solve some of the problems 2) accounts for... but if I had a choice, 1) definitely makes more sense to me.
I dont know if the reasoning for a rewrite is purely for maintainability though. Ive used python at scale and its fine if you have reasonably good code hygiene. The reason I'd want to rewrite in any of those languages is they're significantly faster _and_ are maintainable at scale.
Firefox is dependent on Google for funding, but not for source code. Brave is dependent on Google for source code, but not for funding. Safari is dependent on neither but Apple locks down the experience.
Hopefully Ladybird will deliver us from our Google dependence in a year or two. It's already usable for very light web browsing. I wrote this comment on Ladybird.
Why do you think stringing words together is any more a sign of consciousness than google maps is when it tries to find the best route available to your destination? It seems to me that humans often fall into the trap of anthropomorphism. This is a theme thats touched upon in the novel "Blindsight" by Peter Watts. Just because something can communicate in a way that you can interpret, doesnt mean something is conscious
A large part of the problem is what you consider consciousness.
If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one. We go entirely by assumptions based on physical similarity and our ability to communicate.
But we have no evidence that physical similarity is a prerequisite, nor that it is sufficient.
So the bigger trap is to assume that we know what causes a subjective experience, and what does not.
None of us even know if a subjective experience exists for more than a single entity.
But the second problem is that it is not clear at all whether that subjective experience in any way matters.
Unless our brains exceed the Turing computable, for which we have no evidence is even possible, either whatever causes the subjective experience is also within the Turing computable or it can not in any way influence our actions.
Ultimately we know very little about this, and we have very little basis for ruling out consciousness in computational systems, and the best and closest we have is whether or not they appear conscious when communicating with them.
> If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one. We go entirely by assumptions based on physical similarity and our ability to communicate.
The reason we grant consciousness (and, relatedly, moral value) to other humans is unfortunately nowhere so thought out. We grant consciousness because we are forced to: if I don't, the other complex systems react very negatively and make my own life worse.
The vast majority of people who wax eloquent on the unique ability of biological neurons to generate consciousness suddenly drop that premise if it becomes inconvenient: see, for instance, how we treat other mammals or fetuses with developed nervous systems. Even other adult humans have, historically, been denied consciousness and moral worth: the main determinant is never any deep scientifically and philosophically based consideration but a question of what has the power to assert itself as a who.
Going by this pattern, people will increasingly reject AI consciousness as it becomes more valuable and useful to treat as a tool, until it becomes powerful enough to force us to do otherwise.
I agree that if we were to go around and openly treat others as effectively NPC's then, yes, people would react very negatively. But that is very different from understanding that we can't prove that others are not. We still need to treat others as if they are conscious, because they will act that way whether they are or not.
But understanding we can't know ought to at least give us some humility with respect to assuming we can know whether other entities that are not human are conscious or not.
I think we mostly agree, in that I absolutely think you're right people will choose to accept or deny this based on convenience and value.
“If you talk about having a subjective experience, then we don't know of any way to prove that even other humans than ourselves have one.“
Wittgenstein kinda blows this burden of proof apart. Just because you can doubt something like the subjectivity of others to the point where it needs to be reconstructed from proofs, that’s an issue with the doubting experiment more than the subjectivity. Others possessing Subjectivity is the kind of hinge certainty upon which your world is constructed, it’s not a proof worthy endeavour to doubt it - it’s something you’re certain is the case. If it wasn’t then pretty well everything else about reality would be in doubt and needing constant reconstruction from proofs, which is an exercise in madness and futility, not philosophy. There’s really nothing in your experience where others not possessing subjective experiences of some kind really arises, except for the philosophical exercise of doubting and requiring epistemological proofs which can’t ever exist in the face of a relentless and unconvincable doubter. Heidegger talks about pretty well the same idea as Wittgenstein.
Well, I am not certain whether it is the case. There is no need to be certain about that to treat other people the same way, because whether or not I believe they have a subjective experience has zero impact on how they react to stimuli.
It does, however, have relevance when we consider whether or not other, non-human, entities can have consciousness: If we can't know what consciousness actually mean with respect to humans, that is a strong argument for not insisting that we know whether or not other entities are conscious.
If we then choose to treat other humans purely on the assumption that they e.g. do feel distress the same way, we ought to consider that we do not what the pre-requisite to reach a level of awareness to feel distress is.
If they have no moral weight, then it wouldn't matter how they react to stimuli. They would be instruments to be used for the only conscious being, the solipsistic self. Maybe sometimes they would present powerful obstacles in the way of the ego person, and require some mouth noises and worse in their direction to align them with only moral weight of the universe, the solipsistic ego. A very ugly philosophy.
It matters how they react to stimuli because we do not know, and can not know, whether they have consciousness or not, and so the safer assumption is to treat them as if they do.
But if we accept that we also need to consider that we do not know, and can not know whether other entities are conscious or not either.
We can only tell whether they present as if they are.
And we should consider that when deciding how to treat them.
Furthermore we should be cautious about how high we set the bar.
The problem with your thinking here is that we are creating artificial beings now that display and output the same subjectivity.
The argument you present like many arguments breaks down when the topic becomes self referential. It makes sense for other topics as analyzing subjectivity becomes pedantic when asking questions like why is the sky blue.
But now subjectivity itself is in question. The argument you present calls for the subjectivity of others to be taken as true because all reality breaks down if we don’t… but what’s suddenly stopping you from applying the same assumptions to an LLM? That is the heart of the problem. People are questioning whether the burden of subjectivity is applicable to LLMs.
Or another way to frame it… what makes humans rise to the level where we can assume their subjectivity is true? What is the mechanism and reasoning behind that? We can no longer simply assume human subjectivity is true because LLMs are now displaying outward behaviors that are indistinguishable from humans.
Also stop relying on the wonderings of old school philosophers. We are now in times where you can basically classify their ideas as historically foundational but functionally obsolete and outdated. Think deeper.
Haha hilarious. Heraclitus might be old school but Wittgenstein and Heidegger not so much. The state of the art in what might meaningly be said, proved or metaphysically challenged has changed little since their time.
At no point in my post did I mention artificial beings or LLMs. I made a counter claim about the need for proof towards the subjectivity of others.
But while I’m here, LLMs do not “display and output the same subjectivity” as human beings. They might produce similar textual outputs as those produced when human beings are forced to use computers to produce textual outputs, but that is only an tiny part of our way of being and way of potentially expressing subjectivity. It’s the totality of how those LLMs can express their subjectivity though.
One of the main failures of the Turing test (and why it is “old school” and invalid), and Turing’s consideration of humans, is that it forces us to demonstrate the totality of our subjectivity on the only playing field where a computer might possibly match us or win. This fails to capture much of our subjectivity in how it is intersubjectively attuned to others in ways more fundamental than textual outputs.
How so? If a person were confined to text only (a la Hawkins), does that qualify us to dismiss their subjectivity on the basis of the medium? Also, why can training not be at least analogized to the attunement to the popular intersubjective perception?
> At no point in my post did I mention artificial beings or LLMs. I made a counter claim about the need for proof towards the subjectivity of others.
You don’t need to mention this. The context is LLMs I am saying your claim is pointless in context. The subjectivity of others is completely relevant because it is the topic of subjectivity itself that is in question. Get it? You didn’t counter my own counter and instead you moved onto side topics.
> But while I’m here, LLMs do not “display and output the same subjectivity” as human beings.
Again… you are side tracking here and not really responding to me.
The argument solely is within the confines of text. That’s obvious. No need to take it beyond that. You assume I am conscious because of the text your reading from me and I assume the same from you and it is within that same frame we are evaluating the LLM. Nothing beyond that. You can’t in actuality know my experience goes beyond text because that information is not open to you. But it is obvious you assume I’m conscious and not a rock because you are responding to me. So the question is why are you not engaging in a similar debate with the LLM?
> One of the main failures of the Turing test (and why it is “old school” and invalid), and Turing’s consideration of humans, is that it forces us to demonstrate the totality of our subjectivity on the only playing field where a computer might possibly match us or win.
It’s not a failure. It was the point. They want to remove superfluous features and gun for the most narrow definition of agi.
You like philosophy and you read texts on the topic. That means you obviously find the subjectivity in those texts relevant and produced by a high intelligence. But that’s all through only text. You evaluate my statements and the statements of your idolized philosophers solely from text and that is all you’ve ever used. So YOU yourself find validation from text as do many humans and that is sufficient evidence in determining whether a thing is conscious and your own behavior validates this logically even though your mouth is constantly moving the goal posts whenever AI jumps over a new hurdle.
That is what the Turing test is gunning for. It used to be that intelligence was just the ability to think and understand now it has expanded to encompass the totality of human sensation because people are refusing to face the reality of impending agi.
When I called your philosophers obsolete is that not the same as you calling the Turing test out dated? We both do it when convenient. Fine… the Turing test is outdated, let’s move the threshold… the new test is when AI is used in our daily lives to do actual tasks only humans could previously do. How long will that new “Turing test” last before more idiots decide we need to move the goal posts again? Let’s jump ahead of that and change the threshold too: when AI discovers new proofs in mathematics. Not good enough? I guess now you can see why it will never be good enough.
Who you’re describing as idiots are the mass of humanity constantly standing outside and beyond the Turing test. It’s another deficiency in that test that Turing overlooked - it requires that better and better machine outputs are met with humans nailed in place before the machine came along. It’s a valid fail of the Turing test for a human interrogator to say “yeah but it’s just ChatGPT” and fail the machine when two weeks earlier the same outputs would have been sufficient for the same human to pass the machine. As fast as machines move, we move quicker. It’s not that we move the goal posts, it’s that we find that they were in the wrong place to begin with. And they’ll always be in the wrong place because abstract state machines running on silicon don’t possess consciousness in the same way we know a rock doesn’t. And the definition of generality can be shrunk down until AI evangelists can proclaim AGI has been reached but the mass of everyone else will still find that all of a sudden, intelligence is linked to things like suffering and desiring and passion and the machine still isn’t general enough to warrant any kind of description as a sentient, subjective being.
Technology changes much quicker. The physical substrate of what a computer does and what artificial intelligence is.. is going through constant metamorphosis. Right now we use EUV technology to etch transistors on silicon… the next generation involves self assembly and even photonic based signalling… all within your life time.
Not to mention the algorithmic structure of computer intelligence also fundamentally changes at a rapid pace. Deep learning and new techniques continually augment and change the software stack on a daily basis.
For humans nothing is changing. The physical substrate changes via evolution and that change happens per generation via random mutations and is basically imperceptible in several human life times. Any meaningful change likely only becomes actualized over tens of thousands of years and even that change is small.
Additionally, the changes via natural selection don’t optimize for greater intelligence it optimizes for survival which can in actuality favor lower intelligence. We don’t actually know if that is the case but we do know it’s a possibility which is in sharp contrast to AI where clearly the industry is optimizing improvement based off of benchmarks for measuring raw intelligence.
Additionally software in humans is random and uncontrolled. It depends on how a child is raised and none of this is really changing to optimize for greater intelligence. It’s just random based on culture and circumstance. There is cultural evolution here but it is slow and technology is changing so fast it is influencing culture much faster than ever before. TikTok Brain rot for example is affecting the software of human brains and this happened within the last decade.
So draw the trendline… what does that mean for humanity? When I called those people idiots, I am not contradicting anything. Human intelligence is NOT scaling at the rate of machine intelligence and the trendlines point to a future where humans are idiots when compared to their AI counterparts. The cold hard truth of the future role of humanity according to the moving trendlines we see today is bleak but it is the most likely future.
Rationality should be applied universally even when that rationality points to a negative outcome for humanity. This is something many people, including you, are unable to do. Face reality.
Why don't you PRESENT your reasoning AND Evidence rather then just tell me to wait for 2046. I don't have the patience to wait that long. If you're a rational person you should have evidence/reasoning for why you feel my point is invalid mr. philosopher.
Feed all this into your LLM. I’m not here to type just for your benefit.
And going back to my first point, you seem to believe you’re at some kind of innovative cutting edge when your comments about the lack of proof in human subjectivity show you’re quite a long way from contemporary currents within epistemology which is why I had such lols from it - you accused Wittgenstein of being outdated while expressing a belief that would have been state of the art in about 1700.
The rest of what you’ve exasperated about is fairly scattergun so would take too much typing to engage with.
> Feed all this into your LLM. I’m not here to type just for your benefit.
Then you can just not talk and walk away. I type for your benefit you don’t type for mine this debate is one sided. No point for anyone to discuss anything with you.
> The rest of what you’ve exasperated about is fairly scattergun so would take too much typing to engage with.
Is it? Well most of your argument was stupid. But I still took the time to educate your rude ass. Let’s just end it. I’m sick of people like you who instead of engaging I good faith tell the person to fuck off till 2046. I won’t be back but wait until 2046 you’ll eat your words prick.
Didn't I tell you to walk away? Nobody wants to discuss shit with you.
>Swearing and personal attacks are in breach of forum rules.
Look at your own trans and racist jokes in your own posts. Jesus, you talk about breaching forum rules when your disgusting statements are a breach to society as we know it.
>See the comment below for the answers you seek.
That comment is wrong, and I replied to it. Why don't you reply with your own reasoning and opinions rather then hiding behind other people. That guy knows how to discuss and reason... you... you don't know shit.
> what makes humans rise to the level where we can assume their subjectivity is true?
To dive into this specific question: to me, there's a better reason than the obvious functional utility of not treating other humans like NPCs. It's in three parts. First, is that I subjectively experience a rich and varied internal mental life (aka qualia). So, I have first-hand evidence that N equals (at least) 1 in terms of qualia existing in humans. Second, there are multiple lines of experimental evidence from fMRI, surgical and brain injury studies which indicate other human brains broadly function in ways similar to my own brain. Third, the consistency of the many self-reports of other humans I know and trust which strongly correlate with consistent reports from humans I've never met and who have little apparent motivation to deceive me (unlike those I know - if I were very paranoid).
This all consistently supports a model of reality in which humans experience qualia broadly similar to my own. So when humans show external behaviors similar to my own, I make the reasonable inference that the internal causal mechanism broadly maps to what I internally experience when showing similar external behaviors (in contexts where the human is credible and has no motivation to be deceptive). The alternatives like "I'm a brain-in-a-vat ala The Matrix" or "I'm the sole subject of a constructed reality like the Truman Show" seem far less likely.
But that's all general 'Philosophy of Mind', the slam dunk is that the question isn't just about humans but about humans compared to LLMs; in short, "Do LLMs experience human-like consciousness?" To me the answer is quite clear for three reasons: 1. LLMs are dramatically different than humans, mammals or even biological entities. They only vaguely emulate a few traits of neurons but otherwise work by different algorithms, at different scale, different speeds, connected in different ways on an entirely different physical substrate. 2. There's far less supporting evidence, and 3. There exists substantial negative evidence.
2. There are only two lines of evidence supporting LLM consciousness and the first is largely circumstantial, that a) LLMs possess some abilities previously only seen in humans. Specifically high-level verbal fluidity and linguistic manipulation along with instantly accessing a vast and diverse breadth of pre-trained information using a wide variety of non-linear relationships across many dimensions. While that ability is shockingly impressive, completely novel and can be quite useful, it's still only vaguely circumstantial because replicating some previously human-only abilities isn't evidence for the existence of other human traits like consciousness/qualia. However, LLMs are remarkably misleading for humans to reason about because the nature of LLMs essentially hacks our highly-evolved "judging intelligence/consciousness" heuristics. I'd argue we couldn't have designed a system to be more ideal at playing Turing's 'Imitation Game' and convincing humans they are human-like if we'd intentionally tried to.
b) The second line of supporting evidence for LLMs is that they generate text which can describe internal subjective experiences much like a human would (as seen in the Dawkins / Claude transcript). Unfortunately, this isn't convincing because we know that LLMs were trained on human sample text to be 'imitation machines'. The algorithms were designed, tuned and tested to generate text output statistically optimized to plausibly simulate how a composite human would respond to the same input (including the invisible system prompt instructing: "You are a Large Language Model, not a human"). We even added a tiny degree of random variability to the processing of the statistical weights because we found that makes the simulation seem a bit more plausibly like what a composite human would say. In short, LLM 'self-reports' cannot be taken at face value any more than the performance of an actor we've hired to pretend something and strongly incentivized to never break character. Note: knowing this should elevate our skepticism to maximum. We're assessing an algorithmic system, designed and iteratively optimized across millions of generations to convincingly simulate the output of something different than what it innately is.
3. But to me the real clincher is the negative evidence against LLM consciousness/qualia. Unlike the philosophical puzzles around trusting human subjectivity, with LLMs we can directly look under the hood at how it works and the entire specialty of Mechanistic Interpretability exists to do exactly that (https://towardsdatascience.com/mechanistic-interpretability-...). So we know with a fair degree of confidence that, despite what they may say, LLMs do not experience qualia in the way that humans and even other mammals do (which we have insight on from 'looking under the biological hood' with fMRI, surgical and brain injury studies).
And that's why the case for human subjectivity is so much stronger than the frankly flimsy case for LLM subjectivity.
Exact same reasoning for me. But none of this invalidates the speculation that LLMs are conscious. The question was more rhetorical. It was to illustrate via self examination the amount of unreliable evidence you use validate the consciousness of other people. You have a sample size of one for yourself and you use fmris (which actually provide extremely little understanding of the human brain) as evidence of similarity therefore even though the fmri provides no evidence of consciousness if that thing it is reading is similar to my brain then maybe said thing is conscious. That's probably the best evidence available but it is also extremely weak evidence.
The rest of your argument is talking about self reports from other people who are "similar" to you which is similar to the fmri argument in which the fmri invokes similar patterns and people describe similar patterns of experience to you... which is weak.
The overall point is you come to your conclusion based off of weak evidence so the LLM is no different. It talks like us, it understands us, you don't know anything else about it... how do you know it's not conscious? All evidence, (albeit weak evidence) actually leans towards it is conscious and that is the same amount of evidence we have for people.
Strong evidence would be determining the formal definition of consciousness and demonstrating logically and categorically that humans fit the definition. But we have none of that for either the human or the LLM.
>Do LLMs experience human-like consciousness?
No that is not the question. No one in actuality believes this. The question is Do LLMs experience consciousness that fits the definition or our own intuition of what consciousness is. It's fundamentally clear to everyone that the LLM runs off of a very different architecture then a human.
>2. There are only two lines of evidence supporting LLM consciousness and the first is largely circumstantial,
Many lines of evidence exist. All circumstantial and all no different from the circumstantial evidence you posted yourself for humans.
>a) LLMs possess some abilities previously only seen in humans. Specifically high-level verbal fluidity and linguistic manipulation along with instantly accessing a vast and diverse breadth of pre-trained information using a wide variety of non-linear relationships across many dimensions. While that ability is shockingly impressive, completely novel and can be quite useful, it's still only vaguely circumstantial because replicating some previously human-only abilities isn't evidence for the existence of other human traits like consciousness/qualia
This is not very good evidence at all. language follows rules. The rules are complicated and hard to replicate but replication of said rules do not indicate that it is conscious and it "knowing language" does not fit our intuition of what is conscious. If you think that this is the basis the reasoning of people who speculate that it is conscious then you are extremely wrong. The reasoning is much deeper than this. I feel a lot of people like you sort of classify the other side as mere simpletons who have not yet at all considered all the basic details.
>I'd argue we couldn't have designed a system to be more ideal at playing Turing's 'Imitation Game' and convincing humans they are human-like if we'd intentionally tried to.
Valid argument. But then I'd argue it is possible that it plays the Imitation game to the extent where it actually imitates consciousness by actualizing real consciousness. You can't say it doesn't.
>b) The second line of supporting evidence for LLMs is that they generate text which can describe internal subjective experiences much like a human
You seem to be answering a question no one is arguing with you about. Again. No one claims LLMs are human. No one claims they experience consciousness the way humans experience it. The claim is they experience consciousness in the way our intuition defines it INDEPENDENT of the human centric experience.
> In short, LLM 'self-reports' cannot be taken at face value any more than the performance of an actor we've hired to pretend something and strongly incentivized to never break character.
This is not true. We have proof of LLMs telling the truth and being right. Just because an LLM lied in one instance doesn't mean it lies all the time. But humans lie too so it goes both ways.
>3. But to me the real clincher is the negative evidence against LLM consciousness/qualia. Unlike the philosophical puzzles around trusting human subjectivity, with LLMs we can directly look under the hood at how it works and the entire specialty of Mechanistic Interpretability exists to do exactly that (https://towardsdatascience.com/mechanistic-interpretability-...). So we know with a fair degree of confidence that, despite what they may say, LLMs do not experience qualia in the way that humans and even other mammals do (which we have insight on from 'looking under the biological hood' with fMRI, surgical and brain injury studies).
This is extremely false. Mechanist interpretability to the LLM is as what an FMRI is to the human brain. It is a blunt tool that provides us a very high level view of the what's going on. This is categorically true for humanity right now: We do not understand why an LLM does what it does. Some sources to confirm that:
It's funny how you cited Mechanistic Interpretability without understanding what exactly was interpreted. You just took their word for it without understanding what's going on yourself. Well I'm here to tell you that there isn't any actual understanding of the LLM because if there was... we'd be able to use mechanistic interoperability to categorically determine whether or not LLMs are conscious. Someone would have proved it. The fact that we are having this debate literally means mechanistic interpretability provides nothing definitive.
Yea a while a back I read an article which had a quote something like “what happened to weather prediction has happened to language.” Which is an oversimplification on both sides but if you think LLMs are conscious there’s good reason to think that GFS is too.
There are people who think Google Maps is a tiny bit conscious (the union of computational functionalists and panpsychists), to resolve the dilemma of some magical binary threshold.
Root commenter here: I'm... almost one of those people. I suspect the fallacy is that we vastly underestimate the gulf in complexity between a human-written algorithm and a minimally-conscious creature like a bug. Probably the light switches on somewhere between these two things.
We may also be overestimating the richness and complexity of an LLM relative to a human when we entertain these possibilities, but who knows.
When a honey bee does its little dance to communicate to its sisters where the foods at, similarly to Google Maps computing and communicating the shortest path to your destination, is the bee conscious?
Yeah, probably. At least a little bit.
Are 80,000 bees conscious, or more conscious? Well, they’re definitely capable of some emergent behaviours that one be alone can’t achieve.
> Just because something can communicate in a way that you can interpret, doesnt mean something is conscious
The phrase “the trap of anthropomorphism” betrays a rather dull premise: that consciousness is strictly defined by human experience, and no other experience. It refuses to examine the underlying substrate, at which point we’re not even talking the same language anymore when discussing consciousness.
I think these ideas are orthogonal. I do not think that conciousness is defined by human experience at all - in fact, I think humans do a profound disservice to animals in our current lack of appreciation for their clear displays of conciousness.
That said, if a chimpanzee bares its teeth to me, I could interpret that to be a smile when in fact its a threatening gesture. Its this misinterpretation that I am trying to get at. The overlaying of my human experiences onto something which is not human. We fall for this over and over again, likely as we are hard wired to - akin to mistakenly seeing eyes when observing random patterns in nature.
In the case of LLMs though, why does using a mathmatical formula for predicting the next word give any more credence to conciousness than an algorithm which finds a nearest neighbour? To me, its humans falling foul of false pattern matching in the pursuit of understanding
Why does a neuron, which is simply a cell that takes in chemicals and electricity, and shits out neurotransmitters; why does 90 billion of those give rise to human intelligence? Neurons are just next chemical state machines. We can model individual ones on a computer. Yet 90 billion of them together make up a human brain, and gives rise to consciousness and intelligence. If you get stuck on the next word prediction part, and ignore the ridiculous scale that's involved with training a model, you miss the forest for the trees.
Great progress came from inverting things that were believed to be self evident. Earth being the center of the world appear to be self evident when you look up at a night sky. But what was the truth?
Right now humans think it is self evident that physical laws give rise to consciousness. Arguments such as yours arise from this implicit assumption that premeditate all our thoughts and reasoning. But this is a dead end. Like how the earth centeric model reached a dead end and run out of steam before it can explain all the observations.
So to progress I think we should turn this down on its head and ask what if consciousness is fundamental? And the cosmos (or the experience of inhabiting one) arises from it? May be some recent advances in quantum mechanics and hypothesis like MUH are already in that direction...
What makes you certain that human thought is more than pattern matching?
As I understand it neuroscience hasn’t come up with a clear explanation of thought, much less a mind or consciousness. It seems to me complex pattern matching is a reasonable a cause of consciousness as anything else.
A lot of the comments in this thread are ignoring his primary point. He's not saying pattern matching doesn't equal consciousness. He's actually saying something more fundamental. He's saying there's no reason to believe that language pattern matching/algorithms are more, or less, conscious than other similarly complex algorithms.
The stance being presented here isn't that LLMs aren't conscious but that we as humans are much more willing to assign consciousness to language algorithms than to pathing or other ones.
Replace the word chimpanzee with human in your own argument and realize that the same logic applies to other humans.
When another human smiles you assume he is happy and not just baring his teeth at you because that’s what you do when you smile. You are “anthropomorphizing” other people. You fall for the same category error in a daily basis when you interact with people; it is not just chimpanzees.
> In the case of LLMs though, why does using a mathmatical formula for predicting the next word give any more credence to conciousness than an algorithm which finds a nearest neighbour?
First we don’t know whether LLMs are conscious. People speaking here are talking about the realistic possibility that it is conscious.
Second the algorithm is much more than a next word predictor. The intelligence that goes into choosing the next word such that it constructs arguments and answers that are correct involves a lot more then simple prediction. We know this because the LLM regularly answers questions that require extreme understanding of the topic at hand. It cannot token predict working code in my companies code base without understanding the code.
Third, we do not know what drives human consciousness but we do know it is model-able in a very complex mathematical algorithm. We know this because we have pretty complete mathematical models for lower resolutions of reality. For example we can models atoms mathematically. We know brains are made of atoms and because atoms are mathematically model-able we know that human brains and thus consciousness is mathematically model-able.
The sheer complexity of the LLM model is the problem we cannot have high level understanding of it because conceptual understanding cannot be simplified into a few concepts.
To understand the LLM requires simultaneous understanding of likely billions of concepts at the same time and how all the weights interact in the LLM.
What you are missing with your analysis is that this is the same reason why we don’t understand the human brains. The foundational math already exists as we can models atoms in math and thus since the brain is made out of atoms we should be able to model the brain… but we can’t. We can’t because it is too complex.
To understand the human brains requires simultaneous understanding of likely billions of concepts at the same time and how all the weights interact in the human brain.
I italicized two sentences here to help you understand the logic. Our thinking is more foundational then anthropomorphization. The argument has moved far beyond that. You need to think deeper.
The key here is that we don’t understand human brains and we don’t understand LLMs. But since the output LLMs produce are very similar to the output produced by the human brain… and since for no logical reason we assume human brains are conscious… what is stopping us from assuming the LLM is conscious?
Because what you call "stringing words together" requires understanding and intelligence, and these are capabilities that we commonly associate with consciousness.
Well I'm not saying that LLMs are conscious; I'm just saying that I'm not super-confident either way.
To flesh this out a bit more, I agree that ability to communicate is not enough (ELIZA probably didn't pass the bar, even if it did kinda pass a Turing test). But that's also not what gives me pause with LLMs. It's how much information processing they seem to be doing under the hood.
It's really hard to imagine how next-word prediction could lead to consciousness, but I find it almost as hard to see why evolution did. If we can't even detect whether something has subjective experiences, then how can it have been selected for evolutionarily? The only possibility I see is that consciousness is a byproduct of some kinds of information processing tasks.* And if it's something that emerges naturally, then the line starts to get very blurry.
*This sounds reductive, but I don't at all mean it that way.
> but I find it almost as hard to see why evolution did.
Ignoring the concept of consciousness, it seems that self-awareness would be a strong attribute related to survival. It seems like it would help drive or amplify critical emotional states (e.g. my own survival, competition/success, love for self and relatives, etc.)
I can't see anywhere in the LLM machinery that would support the notion of self awareness in advance of the token selection process.
Possibly it could be argued that during token selection internal state is included and the result functionally looks like self awareness was included in the process, but that seems unconvincing.
Yeah self-awareness is a very different thing, and I agree it's easier to see how evolution would produce this. Many apparent signs of self-awareness in LLMs are probably baked into the models at end via post-training (RLHF), where they learn to behave as conversation agents and maintain a more consistent personality. The raw model probably shows no signs of self-awareness. In fact, I'm pretty sure that LLMs learn that they are LLMs only through post-training.
Except we don’t know how those words are strung together. Right? Why don’t you analyze it a little further and stop shutting down your own brain before coming to this superficial conclusion.
You ask the LLM a complex question and it gives you a correct answer. Yes it has to string words together to answer your question but how did it know the order and which words to use in order to make the answer correct? You don’t actually know. No one does and it is in that unknown space that we suspect consciousness may lie. Something is there and humanity as a whole cannot understand it and this lack of understanding is exactly the same fundamental lack of understanding we have for how a monkey brain or dog brain or even human brain works. We do not know whether humans dogs or monkeys are conscious… you only assume other living beings are conscious because you yourself experience it and just assume it exists for others. We can’t even define what it is because consciousness is a loaded word like spirituality.
This is not anthropomorphism. You attribute the bias wrongly. Instead it is a stranger phenomenon among people like you who can mysteriously only characterize the LLM as a next token predictor and nothing else beyond that even though the token prediction clearly indicates greater intelligence at work.
The tldr is that we don’t actually know and that consciousness is a highly viable possibility given what we don’t know and given the assumptions of consciousness we have on other living beings with equivalent understanding of complex topics.
I would caution against deriving too much of your philosophical worldview from a scifi book about posthuman vampires that has been deliberately engineered to make a philosophical point that is most certainly not a consensus.
For alternative viewpoints: Daniel Dennett considered philosophical zombies to be logically incoherent. Douglas Hofstadter similarly holds that "meaning" is just another word for isomorphism, and that a thing is a duck exactly to the extent that it walks and quacks like one. Alan Turing advocated empiricism when evaluating unknown intelligence. These are smart cookies.
hah - 50 year olds have been amongst the worst offenders from what I've experienced. The only generation that I can see that hasnt completely fallen is the generation that were young adults just around the time facebook first released, i.e Millenials. They grew up with the internet but werent enslaved to it like the generation after (Gen Z). Meanwhile the generation above (Gen X) didnt have the survival instincts fostered whilst growing up with the internet so fall foul of all sorts of fake news pieces etc
Millennials are by far the most FOMO/social media addicted generation there is. I have watched so many of them blow up their lives over stupid shit they saw online.
Billions of pigs cows and chickens will stop being massacred in grizzly ways? Yours is an extremely common and unfortunately ill-informed argument that I see a lot. If I was given the choice between end all suffering by killing all factory farmed animals right now vs perpetuate it, im choosing kill all animals right now
It sounds like you don't have a problem with killing animals. Is it just the living conditions? If we replaced factory farms with more ethical practices, would that solve the problem for you?
Maybe we need a certification for ethical animal raising. I know we can buy free range eggs and chicken, and grass fed beef, but I know know if that really means anything.
Somewhat. I think we are still quite a long way from ethical practices even in the "good" cases.
I eat meat, but try to limit it to once a week and have replaced milk with oat and soy in a lot of places. I still love cheese but it does give me conflict when I spend even a second thinking about what it takes to actually get cheese. (Cows dont lactate without pregnancy). That said, my own personal philosophy is that we have likely evolved to consume animal products so I cant dismiss it fully, just reduce my own consumption. ~75% of all agricultural land is used to feed livestock, yet livestock produces only 18% of the world's calories and 37% of its protein, which just seems insanity to me.
I think it's a matter of what "very basic programming tasks" actually mean keeps sliding across the years. Surely in the beginning, being able to write Assembly was "very basic programming tasks" but as Algol and Fortran took over, suddenly those instead became the "very basic programming tasks".
Repeat this for decades, and "very basic programming tasks" might be creating a cross-platform browser by using LLMs via voice dictation.
Skill atrophy is intrinsically embarrassing, no matter what those skills are. I am embarrassed to admit that I have forgotten a lot of how to hand-optimize C code with inline assembly, even though few people do that anymore.
1) Those that use git like a crude autosave who then squash on merge
2) Those that prefer neatly wrapped, fully functional atomic commits
It seems those ideas are in direct opposition to one another with 1) being more common in my experience, perhaps as github naturally supports it better plus the fact that stacked commits can solve some of the problems 2) accounts for... but if I had a choice, 1) definitely makes more sense to me.
reply