I do not fear these LLMs gaining sentience.

I do fear what happens when the text regurgitating machine sounds sentient enough to convince the average person, and tech companies start selling it as such.

It’s because the LLM is incapable of understanding symbols, so it couldn’t even understand the chessboard and the images that represented the pieces. This capability for abstract thinking is the thing that human brains do incredibly well (sometimes too well, then you get pareidolia), but is completely outside the bounds of what an LLM is or ever will be able to do.

I’m sure they’ve digested every public piece of chess notation ever written but they have no capacity for comprehension and are programs that emit text shaped like chess notation if you make that request of them.

when people here call it a text extrusion machine thats literally what it is. In fact it doesnt even look at text, it looks at tokens. And there are a limited number of tokens (llama uses a vocabulary size of about 32k i think). It takes all of the previously entered input and output, turns it into tokens, and then each token “attends” (is multiplied by with some coefficient) to each other token. Then it all goes through more gigantic layers of matrix multiplication and at the end you have the statistically most likely next token. Then it does the whole thing again recursively until it reaches what it decides is the end of the output. It may also not decide and would need to be cut off.

So its not really looking at the game. It is in a way but it doesnt really know the rules, its just producing the next most likely token which is not necrssarily the next best move or even next correct move.

An LLM can summarize the rules of chess, because it predicts the sequence of words needed to create that with incredible accuracy. This is why it’s so weird when it goes wrong, because if one part of it is off then it throws the rest of the work it’s doing out of balance.

But all it is doing is a statistical analysis of all the writing it’s has been trained on and determining the best next word to use (some later models do them in groups and out of order).

That doesn’t tell it fuck-all about how to make a chess move. It’s not ingesting information in a way that lets it create a model to tell you what the next best chess move is, how to solve linear algebra, or any other activity that requires procedural thought.

It’s just a chatterbox that tells you whatever you want to hear. No wonder the chuds love it

If I say, “Knight to B4,” does that sound like something a person playing chess might say? Then it did it’s job.

Think of an LLM as an actor. You don’t hire someone to act as a grandmaster in a movie based on their skill at chess, they might not even know how to play, but if they deliver the lines in a convincing way, that’s what you’re looking for. There’s chess AIs that are incredibly good at chess, because that’s what they’re designed for and trained on. That’s why this is a very silly test, it’s like testing a fish on its tree-climbing ability, the only thing sillier than this test is that people are surprised by it.

Disregarding the /s bc i want to rant

I guess if you described board states in language and got them to recognize chess board states from images (by describing them in language), and trained them on real games, you could probably make a really inefficient chess bot.

But that said, you could use an “agentic” model with an mcp to route queries about chess to an api that links the LLM to an actual chess bot.

Then it’d just be like going to the chess bot website and entering the board states to get the next move. No magic involved, just automated interaction with an api. The hype and fear and mysticism around llms bugs me. The concepts behind how they work aren’t hard, just convoluted

A solution for chess exists, but the space is too big to calculate with current technology

Well no, it’s not a memorization game. Part of a grandmaster’s strategy is deciding when to go “off book” and cause their opponent to have to reason through a position. An attribute of a chess engine like Stockfish is its “depth” which is a measurement of how many permutations it searches through in a tree of possibilities. You get some ridiculous number of permutations very quickly on a chess board.

That’s not to say that a competitor doesn’t do anything assload of memorization of the “correct” moves as proven in landmark games. But you don’t just memorize chess and solve it as such like you can do with tic tac toe. Unrelated but I think a spectrum is fun: tic tac toe, solved, memorizable. Connect 4, solved, unmemorizable. Checkers, surprisingly solved, in your dreams. Chess, unsolved.

Yes, chess can be solved by simply knowing every possible board state. However there’s like 10^50 possible positions (we think, it’s actually unknown how many possible legal positions there are) and storing that amount of information would require more than the sun’s volume in hard drives

No offense, but you’re wrong about this.

Machine learning does have valid use cases, and chess (and go and other board games) is one of them. The thing about chess is that there’s a definitive win state that the AI is trying to reach. This is a huge difference from language and image models, which require human input to tell them if they’re any good or not, and feeding the output back into it makes it more and more gibberish. With chess AI, the goal isn’t to play like a human, but to win, which means it can judge it’s own output against that metric and train off of that, with no need for human games at all. You can start it off playing random nonsense moves, and then let it run, and it’ll play millions of games getting a little better with each one, as fast as the hardware allows. The end result is something much, much better than what any human or brute force algorithm can achieve. Speaking as a go player, AI has completely revolutionized the way we play the game, and I believe the chess world has had a similar experience.

Having said that, there have been some problems with go AI. A while back, somebody discovered a trick that anybody could use to beat otherwise unbeatable AI. It involved intentionally letting a group get surrounded with no way to live, and then surrounding the group surrounding that group in order to kill it. It was a nonsense strategy that any human player would catch on to and subvert, but because it was a bad strategy, the AI never tried it and so it wasn’t in its training data. This served as an important reminder that the AI isn’t perfect and isn’t actually thinking.

However, without exploits like that, nobody, not even the top professionals, have any chance whatsoever of beating a top AI. And that only started being the case with go relatively recently, because the brute force algorithms weren’t good enough but the machine learning algorithms were a huge leap forward, and they’re getting better and better.

I’m as much of an AI skeptic as the next person, but a W is a W.

If I didn’t have an argument with a pro-“AI” (it’s not AI, I refuse to call it that) person in my fucking post history about just this fucking issue, maybe I’d be more willing to agree with you here. But no, the people who keep trying to get me to use so-called “AI” seem to believe that it can reason, or, at least, that it can be convinced to reason. So yes, I will use this article to “hate on AI”, because the “AI” lovers seem to believe that chatGPT should be capable of something like this. When clearly, fucking obviously, it isn’t. It isn’t those of us who hate so-called “AI” that are trying to claim that these text predictors can reason, it’s the people who like them and want to force me to use them that make this claim.

We’re hating on AI right now.

we are, you are correct. but singularity next week or w/e

Grok are you okay? Your replies to prompts are getting lazy.

Is this cuz what’s been going on with your dad?

+2