▲ 277 ▼ Someone made a GPT-like chatbot that runs locally on Raspberry Pi, and you can too (www.xda-developers.com) submitted 2 years ago by throws_lemy@lemmy.nz to c/technology@lemmy.world 30 comments fedilink hide all child comments
[–] Flumpkin@slrpnk.net 0 points 2 years ago (1 child) How fast are they with a good GPU? permalink fedilink source parent hideshow 2 child comments replies: [–] DarkThoughts@fedia.io 0 points 2 years ago (1 child) Have you missed the first part where I explained that I couldn't get it to run through my GPU? I would only have a 6650 XT anyway but even that would be significantly faster than my CPU. How far I can't say exactly without experiencing it though, but I suspect with longer chats and consequently larger context sizes it would still be too slow to be really usable. Unless you're okay waiting for ages for a response. permalink fedilink source parent hideshow 2 child comments replies: [–] Flumpkin@slrpnk.net 1 point 2 years ago Sorry, I'm just curious in general how fast these local LLMs are. Maybe someone else can give some rough info. permalink fedilink source parent
[–] DarkThoughts@fedia.io 0 points 2 years ago (1 child) Have you missed the first part where I explained that I couldn't get it to run through my GPU? I would only have a 6650 XT anyway but even that would be significantly faster than my CPU. How far I can't say exactly without experiencing it though, but I suspect with longer chats and consequently larger context sizes it would still be too slow to be really usable. Unless you're okay waiting for ages for a response. permalink fedilink source parent hideshow 2 child comments replies: [–] Flumpkin@slrpnk.net 1 point 2 years ago Sorry, I'm just curious in general how fast these local LLMs are. Maybe someone else can give some rough info. permalink fedilink source parent
[–] Flumpkin@slrpnk.net 1 point 2 years ago Sorry, I'm just curious in general how fast these local LLMs are. Maybe someone else can give some rough info. permalink fedilink source parent