▲ 649 ▼ ‘Sputnik moment’: $1tn wiped off US stocks after Chinese firm unveils AI chatbot (www.theguardian.com) submitted 2 years ago by zdhzm2pgp@lemmy.ml to c/technology@lemmy.ml 191 comments fedilink hide all child comments
[–] Phoenicianpirate@lemm.ee 23 points 2 years ago (5 children) So can I have a private version of it that doesn't tell everyone about me and my questions? permalink fedilink source parent hideshow 10 child comments replies: [–] SpaceRanger@lemmy.world 26 points 2 years ago (1 child) Checkout ollama. https://ollama.com/library/deepseek-r1 permalink fedilink source parent hideshow 2 child comments replies: [–] Phoenicianpirate@lemm.ee 2 points 2 years ago Thank you very much. I did ask chatGPT was technical questions about some... subjects... but having something that is private AND can give me all the information I want/need is a godsend. Goodbye, chatGPT! I barely used you, but that is a good thing. permalink fedilink source parent [–] Mongostein@lemmy.ca 4 points 2 years ago (2 children) Yeah, but you have to run a different model if you want accurate info about China. permalink fedilink source parent hideshow 4 child comments replies: [–] Phoenicianpirate@lemm.ee 5 points 2 years ago (1 child) Yeah but China isn't my main concern right now. I got plenty of questions to ask and knowledge to seek and I would rather not be broadcasting that stuff to a bunch of busybody jackasses. permalink fedilink source parent hideshow 2 child comments replies: [–] Mongostein@lemmy.ca -1 points 2 years ago I agree. I don’t know enough about all the different models, but surely there’s a model that’s not going to tell you “<whoever’s> government is so awesome” when asking about rainfall or some shit. permalink fedilink source parent [–] Alsephina@lemmy.ml 2 points 2 years ago Unfortunately it's trained on the same US propaganda filled english data as any other LLM and spits those same talking points. The censors are easy to bypass too. permalink fedilink source parent [–] lambda@programming.dev 4 points 2 years ago Yep, lookup ollama permalink fedilink source parent [–] MetalMachine@feddit.nl 3 points 2 years ago Yes permalink fedilink source parent [–] tooclose104@lemmy.ca 2 points 2 years ago (2 children) Can someone with the knowledge please answer this question? permalink fedilink source parent hideshow 4 child comments replies: [–] TonyTonyChopper@mander.xyz 8 points 2 years ago (1 child) Yes, you can run a downgraded version of it on your own pc. permalink fedilink source parent hideshow 2 child comments replies: [–] tooclose104@lemmy.ca 5 points 2 years ago Apparently phone too! Like 3 cards down was another post linking to instructions on how to run it locally on a phone in a container app or termux. Really interesting. I may try it out in a vm on my server. permalink fedilink source parent [–] boomzilla@programming.dev 5 points 2 years ago* (last edited 2 years ago) I watched one video and read 2 pages of text. So take this with a mountain of salt. From that I gathered that deepseek R1 is the model you interact with when you use the app. The complexity of a model is expressed as the number of parameters (though I don't know yet what those are) which dictate its hardware requirements. R1 contains 670 bn Parameter and requires very very beefy server hardware. A video said it would be 10th of GPUs. And it seems you want much of VRAM on you GPU(s) because that's what AI crave. I've also read 1BN parameters require about 2GB of VRAM. Got a 6 core intel, 1060 6 GB VRAM,16 GB RAM and Endeavour OS as a home server. I just installed Ollama in about 1/2 an hour, using docker on above machine with no previous experience on neural nets or LLMs apart from chatting with ChatGPT. The installation contains the Open WebUI which seems better than the default you got at ChatGPT. I downloaded the qwen2.5:3bn model (see https://ollama.com/search) which contains 3 bn parameters. I was blown away by the result. It speaks multiple languages (including displaying e.g. hiragana), knows how much fingers a human has, can calculate, can write valid rust-code and explain it and it is much faster than what i get from free ChatGPT. The WebUI offers a nice feedback form for every answer where you can give hints to the AI via text, 10 score rating thumbs up/down. I don't know how it incooperates that feedback, though. The WebUI seems to support speech-to-text and vice versa. I'm eager to see if this docker setup even offers APIs. I'll probably won't use the proprietary stuff anytime soon. permalink fedilink source parent
[–] SpaceRanger@lemmy.world 26 points 2 years ago (1 child) Checkout ollama. https://ollama.com/library/deepseek-r1 permalink fedilink source parent hideshow 2 child comments replies: [–] Phoenicianpirate@lemm.ee 2 points 2 years ago Thank you very much. I did ask chatGPT was technical questions about some... subjects... but having something that is private AND can give me all the information I want/need is a godsend. Goodbye, chatGPT! I barely used you, but that is a good thing. permalink fedilink source parent
[–] Phoenicianpirate@lemm.ee 2 points 2 years ago Thank you very much. I did ask chatGPT was technical questions about some... subjects... but having something that is private AND can give me all the information I want/need is a godsend. Goodbye, chatGPT! I barely used you, but that is a good thing. permalink fedilink source parent
[–] Mongostein@lemmy.ca 4 points 2 years ago (2 children) Yeah, but you have to run a different model if you want accurate info about China. permalink fedilink source parent hideshow 4 child comments replies: [–] Phoenicianpirate@lemm.ee 5 points 2 years ago (1 child) Yeah but China isn't my main concern right now. I got plenty of questions to ask and knowledge to seek and I would rather not be broadcasting that stuff to a bunch of busybody jackasses. permalink fedilink source parent hideshow 2 child comments replies: [–] Mongostein@lemmy.ca -1 points 2 years ago I agree. I don’t know enough about all the different models, but surely there’s a model that’s not going to tell you “<whoever’s> government is so awesome” when asking about rainfall or some shit. permalink fedilink source parent [–] Alsephina@lemmy.ml 2 points 2 years ago Unfortunately it's trained on the same US propaganda filled english data as any other LLM and spits those same talking points. The censors are easy to bypass too. permalink fedilink source parent
[–] Phoenicianpirate@lemm.ee 5 points 2 years ago (1 child) Yeah but China isn't my main concern right now. I got plenty of questions to ask and knowledge to seek and I would rather not be broadcasting that stuff to a bunch of busybody jackasses. permalink fedilink source parent hideshow 2 child comments replies: [–] Mongostein@lemmy.ca -1 points 2 years ago I agree. I don’t know enough about all the different models, but surely there’s a model that’s not going to tell you “<whoever’s> government is so awesome” when asking about rainfall or some shit. permalink fedilink source parent
[–] Mongostein@lemmy.ca -1 points 2 years ago I agree. I don’t know enough about all the different models, but surely there’s a model that’s not going to tell you “<whoever’s> government is so awesome” when asking about rainfall or some shit. permalink fedilink source parent
[–] Alsephina@lemmy.ml 2 points 2 years ago Unfortunately it's trained on the same US propaganda filled english data as any other LLM and spits those same talking points. The censors are easy to bypass too. permalink fedilink source parent
[–] tooclose104@lemmy.ca 2 points 2 years ago (2 children) Can someone with the knowledge please answer this question? permalink fedilink source parent hideshow 4 child comments replies: [–] TonyTonyChopper@mander.xyz 8 points 2 years ago (1 child) Yes, you can run a downgraded version of it on your own pc. permalink fedilink source parent hideshow 2 child comments replies: [–] tooclose104@lemmy.ca 5 points 2 years ago Apparently phone too! Like 3 cards down was another post linking to instructions on how to run it locally on a phone in a container app or termux. Really interesting. I may try it out in a vm on my server. permalink fedilink source parent [–] boomzilla@programming.dev 5 points 2 years ago* (last edited 2 years ago) I watched one video and read 2 pages of text. So take this with a mountain of salt. From that I gathered that deepseek R1 is the model you interact with when you use the app. The complexity of a model is expressed as the number of parameters (though I don't know yet what those are) which dictate its hardware requirements. R1 contains 670 bn Parameter and requires very very beefy server hardware. A video said it would be 10th of GPUs. And it seems you want much of VRAM on you GPU(s) because that's what AI crave. I've also read 1BN parameters require about 2GB of VRAM. Got a 6 core intel, 1060 6 GB VRAM,16 GB RAM and Endeavour OS as a home server. I just installed Ollama in about 1/2 an hour, using docker on above machine with no previous experience on neural nets or LLMs apart from chatting with ChatGPT. The installation contains the Open WebUI which seems better than the default you got at ChatGPT. I downloaded the qwen2.5:3bn model (see https://ollama.com/search) which contains 3 bn parameters. I was blown away by the result. It speaks multiple languages (including displaying e.g. hiragana), knows how much fingers a human has, can calculate, can write valid rust-code and explain it and it is much faster than what i get from free ChatGPT. The WebUI offers a nice feedback form for every answer where you can give hints to the AI via text, 10 score rating thumbs up/down. I don't know how it incooperates that feedback, though. The WebUI seems to support speech-to-text and vice versa. I'm eager to see if this docker setup even offers APIs. I'll probably won't use the proprietary stuff anytime soon. permalink fedilink source parent
[–] TonyTonyChopper@mander.xyz 8 points 2 years ago (1 child) Yes, you can run a downgraded version of it on your own pc. permalink fedilink source parent hideshow 2 child comments replies: [–] tooclose104@lemmy.ca 5 points 2 years ago Apparently phone too! Like 3 cards down was another post linking to instructions on how to run it locally on a phone in a container app or termux. Really interesting. I may try it out in a vm on my server. permalink fedilink source parent
[–] tooclose104@lemmy.ca 5 points 2 years ago Apparently phone too! Like 3 cards down was another post linking to instructions on how to run it locally on a phone in a container app or termux. Really interesting. I may try it out in a vm on my server. permalink fedilink source parent
[–] boomzilla@programming.dev 5 points 2 years ago* (last edited 2 years ago) I watched one video and read 2 pages of text. So take this with a mountain of salt. From that I gathered that deepseek R1 is the model you interact with when you use the app. The complexity of a model is expressed as the number of parameters (though I don't know yet what those are) which dictate its hardware requirements. R1 contains 670 bn Parameter and requires very very beefy server hardware. A video said it would be 10th of GPUs. And it seems you want much of VRAM on you GPU(s) because that's what AI crave. I've also read 1BN parameters require about 2GB of VRAM. Got a 6 core intel, 1060 6 GB VRAM,16 GB RAM and Endeavour OS as a home server. I just installed Ollama in about 1/2 an hour, using docker on above machine with no previous experience on neural nets or LLMs apart from chatting with ChatGPT. The installation contains the Open WebUI which seems better than the default you got at ChatGPT. I downloaded the qwen2.5:3bn model (see https://ollama.com/search) which contains 3 bn parameters. I was blown away by the result. It speaks multiple languages (including displaying e.g. hiragana), knows how much fingers a human has, can calculate, can write valid rust-code and explain it and it is much faster than what i get from free ChatGPT. The WebUI offers a nice feedback form for every answer where you can give hints to the AI via text, 10 score rating thumbs up/down. I don't know how it incooperates that feedback, though. The WebUI seems to support speech-to-text and vice versa. I'm eager to see if this docker setup even offers APIs. I'll probably won't use the proprietary stuff anytime soon. permalink fedilink source parent