277
Someone made a GPT-like chatbot that runs locally on Raspberry Pi, and you can too (www.xda-developers.com)
submitted 2 years ago by to c/technology@lemmy.world
30 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 28 points 2 years ago (1 child)

Can we have smaller more domain specific models. that shouldn't require more than casual hardware. like a small model for coding, one for medicine, one for history, and so on. ???

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 14 points 2 years ago (1 child)

    Check out hugging face! Honestly fine tunned models for specific domains seems very popular (if for nothing else because training smaller models is just easier!).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Unfortunately the roleplaying chatbot type models are typically fairly sizeable / demanding. I'm curious how this will develop with more specific AI hardware though, like extension cards with primarily tensor cores + their own ram, so that you don't have to use your GPU for that. If we can drag down the price for such hardware then locally run models could become much more viable and mainstream.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -5 points 2 years ago (1 child)

    Dude sorry to say but roleplay is not equally important as medicine or coding XD

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    For me they are. I have no use for medicine or coding bots.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 2 years ago (2 children)

    but you have the use for the very software you're using daily or medicine developments.

    I play D&D from time to time, but saying that roleplaying is more important than medicine is just nuts.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 2 years ago

    Not wanting to be mean, I just find the thought of people talking to robots a bit strange, and use them as tools only. Not sure what "roleplay" means, if it is some "fantasy DND generator" still you could say this may be better done by humans to keep that grey matter running.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (1 child)

    Not so much for the latter but I'm pretty specifically talking about my personal use case here. lol "Roleplaying" in this scenario isn't really referring to actual tabletop type RPGs btw. It's the LLM roleplaying specific characters or personas that you then chat with in specific (or not so specific) scenarios. Although that same tech is also experimented with to be used in video games for NPCs. But who knows. A specifically trained model could potentially make a half decent dungeon master too.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 years ago

    There also a huge amount of training, medical and otherwise, that’s done through role-playing. I could definitely see medical students getting use out of learning telemedicine with LLMs that were ultimately adapted from TTRPGs character generator schemas.

  • permalink
  • fedilink
  • source
  • parent