Someone made a GPT-like chatbot that runs locally on Raspberry Pi, and you can too
(www.xda-developers.com)
Hopefully we see more specific hardware for this. Like extension cards with pretty much just tensor cores and their own ram.