We are constantly fed a version of AI that looks, sounds and acts suspiciously like us. It speaks in polished sentences, mimics emotions, expresses curiosity, claims to feel compassion, even dabbles in what it calls creativity.

But what we call AI today is nothing more than a statistical machine: a digital parrot regurgitating patterns mined from oceans of human data (the situation hasn’t changed much since it was discussed here five years ago). When it writes an answer to a question, it literally just guesses which letter and word will come next in a sequence – based on the data it’s been trained on.

This means AI has no understanding. No consciousness. No knowledge in any real, human sense. Just pure probability-driven, engineered brilliance — nothing more, and nothing less.

So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure. It doesn’t hunger, desire or fear. And because there is no cognition — not a shred — there’s a fundamental gap between the data it consumes (data born out of human feelings and experience) and what it can do with them.

Philosopher David Chalmers calls the mysterious mechanism underlying the relationship between our physical body and consciousness the “hard problem of consciousness”. Eminent scientists have recently hypothesised that consciousness actually emerges from the integration of internal, mental states with sensory representations (such as changes in heart rate, sweating and much more).

Given the paramount importance of the human senses and emotion for consciousness to “happen”, there is a profound and probably irreconcilable disconnect between general AI, the machine, and consciousness, a human phenomenon.

https://archive.ph/Fapar

    • hendrik@palaver.p3x.de
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      1
      ·
      edit-2
      2 days ago

      That is a good take. Sadly suffers from some of the same shortcomings as OP’s article, mainly shitting on statistics, since not just LLMs run on maths but humans as well and the entire universe… But looking at it this way explains a lot of things. Why it blabbers and repeats a lot, why lots of people tell me how good it is at programming and I think it sucks… I’ll have to bookmark this for the next person who doesn’t believe me.

      • msage@programming.dev
        link
        fedilink
        arrow-up
        5
        ·
        1 day ago

        It does go a bit overboard, but the main point is pretty simple:

        LLMs don’t understand anything.

        People will believe anything when it’s convenient for them.

        Or lock themselves into a garden, from which they can not escape, apparently.

        • hendrik@palaver.p3x.de
          link
          fedilink
          English
          arrow-up
          4
          arrow-down
          1
          ·
          edit-2
          1 day ago

          I mainly love the picture with the mentalist and us in the audience. And the dynamics at play. That’s a really good figure and space to wander in and draw parallels. A lot about LLMs is a con.

          I don’t think the part with “LLMs don’t understand anything.” is factually correct, though. They’re specifically designed to generalize. That is “understand” things and not memorize them. There are a bunch of papers about it and it’s the main difference and advancement between the chatbots of the 1980s, and today.

          Yeah, and people love to see what they want to see, or anticipate to see or whatever.

          • msage@programming.dev
            link
            fedilink
            arrow-up
            2
            ·
            21 hours ago

            “Understanding” as much as “”“reasoning”“” still sound like mentalist tricks, trying to amthropomorphize the LLM.

            Grouping words together based on its usage is far from intelligent.

            • hendrik@palaver.p3x.de
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              20 hours ago

              Right. I’ll see if I write a longer article some day and post it in this community. All the antropomorphization is kind of an issue in my opinion. And we humans are programmed to see faces, intent and such things. I mean that’s why illusions work. Or religion. Or mentalists… But it’s also not the opposite. The grouping words together on usage … that’s markov-chain chatbots from the 1980s. But that’s also not what happens here. Modern LLMs are a different beast. They’re specifically designed to do more than that. And they do. We have some understanding. But it’s a long story and very technical. And I believe we have to give a good definition of words like “understanding” or “reasoning” first. Seems lots of people deduct what they mean by looking at humans. And even with humans it’s more complicated than just saying they “reason”. In my experience a lot of being human includes not being reasonable. And we even need tools like maths and logic if we want to reason properly and find something that is true. And by seeing how illusions etc are effective on us, we can even see how smoke and shadows are a part of how we operate.