Yup. Give a Markov chain multi-billion parameters and you can get some surprisingly cogent results.
I will freely admit that current LLM architectures include several innovations that make them not actually Markov chains, but it's still statistics and linear algebra. I don't know what thought is, but I'm quite unconvinced that LLMs (or any current generative AI architecture) is doing it.