263
Rsync author responds to online outrage about his usage of LLMs (medium.com)
submitted 2 months ago by to c/programming@programming.dev
235 comments fedilink hide all child comments
 

Seems like he's been pushed into using LLMs as a way to cope with the deluge of LLM-generated security reports.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 months ago (2 children)

What hardware that needs? My issue with running local models was that it's too much of a resource hog to be able to do gamedev on the same machine, and any sensible model needs pretty expensive hardware to just get a server for it. Especially with current prices.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 2 months ago (1 child)

    64GB unified memory. I run it (and a lot more) on a dgx spark, but a Mac mini would suffice also.

    You could prob run 4-bit version on a RTX card with 32g. Maybe even 24g. Like a 5090 or 4090 or such.

    So much info out there.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago (1 child)

    Mac Minis top out at 48GB and are 1.8k when configured like that. It's going to be at least $2k to buy anything that has a hope of running it at a reasonable speed.

    Running local isn't free, but at least it's just a single upfront payment.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •