328
any cool ideas what i could do with termux? (lemmy.ml)
submitted 2 years ago by to c/linux@lemmy.ml
113 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 23 points 2 years ago (2 children)

Compile llama.cpp, download a small GGML LLM model and you will have a quite intelligent assiatant running into your phone.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 9 points 2 years ago (2 children)

    Would that actually be decent? Even 6b models feel way too rudimentary after experiencing 33+b models and/or chatgpt. I haven't tried those really scaled down and optimized models, though!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 2 years ago

    They're decent for text completion purposes, e.g. generating some corpspeak for an email, or generating some "wikipedia"-like text. You have to know how to write good prompts, don't try to treat it like ChatGPT.

    For example if i want to know about the history of Puerto Rico I would put:

    "The history of puerto rico starts in about 480BC when"

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (1 child)

    I got llama to compile but it crashes every time I tried running it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (2 children)

    Do we have any llama apk/deb/flatpack/appimage where you just choose the llm to use?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 years ago

    lamacpp is kind of this. you give it a path to a model and a prompt. download a smallish model GGUF file by thebloke. lamacpp main exec has an interactive mode too which can function like a chatbot. i never built it on android though maybe you need android NDK

  • permalink
  • fedilink
  • source
  • parent