I'm quite sure that you could use a LLM to play chess and probably even successful, but you need to train it on chess notation of games instead of a pile of fanfiction and other copyright infringements. I have considered trying that but was turned off by how inaccessible LLM training is and how difficult it would be to get a sufficient amount of games written in proper chess notation. Obviously this would not be a real LLM, as it does not "speak", but I was curious how well this would work utilizing the same technique.