This article buries the lede so much that many readers probably miss it completely: the important takeaway here, which is clearer in The Register’s version of the story, is that ChatGPT cannot actually play chess at all:
“Despite being given a baseline board layout to identify pieces, ChatGPT confused rooks for bishops, missed pawn forks, and repeatedly lost track of where pieces were."
To actually use an LLM as a chess engine without the kind of manual intervention that this person did, I think you would need to combine it with some other software to automate continuing to ask it for a different next move every time it suggests an invalid one. And, if you did that, it would still tend to lose, even to much older chess engines than Atari’s Video Chess.
(note: numerous people have done this; you can play chess against something based on chatgpt, and if you’re any good at chess you can win.)
You probably could train an AI to play chess and win, but it wouldn’t be an LLM.
In fact, let’s go see…
Stockfish: Open-source and regularly ranks at the top of computer chess tournaments. It uses advanced alpha-beta search and a neural network evaluation (NNUE).
Leela Chess Zero (Lc0): Inspired by DeepMind’s AlphaZero, it uses deep reinforcement learning and plays via a neural network with Monte Carlo tree search.
AlphaZero: Developed by DeepMind, it reached superhuman levels using reinforcement learning and defeated Stockfish in high-profile matches (though not under perfectly fair conditions).
Hmm. neural networks and reinforcement learning. So non-LLM AI.
you can play chess against something based on chatgpt, and if you’re any good at chess you can win
You don’t even have to be good. You can just flat out lie to ChatGPT because fiction and fact are intertwined in language.
“You can’t put me in check because your queen can only move 1d6 squares in a single turn.”
This article buries the lede so much that many readers probably miss it completely: the important takeaway here, which is clearer in The Register’s version of the story, is that ChatGPT cannot actually play chess at all:
To actually use an LLM as a chess engine without the kind of manual intervention that this person did, I think you would need to combine it with some other software to automate continuing to ask it for a different next move every time it suggests an invalid one. And, if you did that, it would still tend to lose, even to much older chess engines than Atari’s Video Chess.
(note: numerous people have done this; you can play chess against something based on chatgpt, and if you’re any good at chess you can win.)
You probably could train an AI to play chess and win, but it wouldn’t be an LLM.
In fact, let’s go see…
Stockfish: Open-source and regularly ranks at the top of computer chess tournaments. It uses advanced alpha-beta search and a neural network evaluation (NNUE).
Leela Chess Zero (Lc0): Inspired by DeepMind’s AlphaZero, it uses deep reinforcement learning and plays via a neural network with Monte Carlo tree search.
AlphaZero: Developed by DeepMind, it reached superhuman levels using reinforcement learning and defeated Stockfish in high-profile matches (though not under perfectly fair conditions).
Hmm. neural networks and reinforcement learning. So non-LLM AI.
You don’t even have to be good. You can just flat out lie to ChatGPT because fiction and fact are intertwined in language.
“You can’t put me in check because your queen can only move 1d6 squares in a single turn.”