>Unless you think that there is some fundamental reason why LLMs will never be able to play chess competently, and I doubt there is, then it seems that we could with the right prompts implement some sort of chess AI using an LLM.
You can play a good game of chess (or poker for that matter) with GPT.
There's also some work going on in the eleuther ai discord training LLMs specifically for chess to see how they shape up. They're using the pythia models. so far:
I've found they fall apart after a couple of moves and lose track of the game.
Edit: This might not be the case anymore it seems, my below point doesn't actually contradict you, seems it matters a lot how you tell the model your moves.
Also saying things like "move my rightmost pawn" completely confuses them.
The token model of LLMs doesn't map well into how human experience the world of informational glyphs. Left and right is a intrinsic quality of our vision system. An LLM has to map the idea of left and right into symbols via text and line breaks.
I do think it will be interesting as visual input and internal graphical output is integrated with text based LLMs as that should help correct their internal experience to be based closer to what we as humans experience.
" An LLM has to map the idea of left and right into symbols via text and line breaks."
Oh yeah that's i suggested it :)
I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better
>I do wonder though if we give the LLMs enough examples of texts with people describing their relative spatial position to each other and things will it eventually "learn" to work things these out a bit better
GPT-4's spatial position understanding is actually really good all things considered. By the end, 4 was able to construct an accurate maze just from feedback about the current position and possible next moves after each move by GPT-4.
A funny thing GPT-4 is unusually good at is giving driving directions. This shouldn't work, and of course isn't 100% right, but… it's kind of right.
Bard can answer questions like this, but I think it actually uses the Maps API. (It certainly says that's what it's doing.)
On the other hand, every chatbot including GPT-4 is both unable to do ASCII art and unable to tell it can't do it. (Bard always shows you `cowsay` and tells you it's what you asked for, no matter what it was supposed to be.)
Probably. But what seems much more interesting is to have a spatial model pre-seeded in the LLM, so that it "attaches" language to that as part of its training. Ditto for other models of the world we want the language module to be able to draw on and reason with.
You can play a good game of chess (or poker for that matter) with GPT.
https://twitter.com/kenshinsamurai9/status/16625105325852917...
https://arxiv.org/abs/2308.12466
There's also some work going on in the eleuther ai discord training LLMs specifically for chess to see how they shape up. They're using the pythia models. so far:
Pythia 70M, est ELO 1050
Pythia 160M, est ELO 1370