777
Generative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project. (www.theatlantic.com)
submitted 1 week ago by to c/technology@lemmy.world
211 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 152 points 1 week ago (3 children)

But it’s so good at programming if you already know how to program! Surely that’s worth burning the planet and crashing the world economy??

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 60 points 1 week ago (3 children)

    Actually still no

    https://github.com/JustVugg/colibri

    Everyone was desperate to be first because capitalism. But we are getting good models without the insane build out requirement. Which will be hilarious to leave the cunts holding the bag. Not that the planet is better for it in the end.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 32 points 1 week ago (1 child)

    ~1 token per second (storage bound gen4 nvme)... Some of us have places to be.

    Don't get me wrong. Its impressive that it can run at all, but honestly the usecase is exceedingly narrow. You'd have better results with a structured quantized gpu-only gemma or qwen workflow. Quality over quantity, rely on validation and a structured process: lots of cross-model review and iteration loops with spec and test driven dev. You could probably get a working alpha by the time colibri set up the environment.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 1 week ago* (3 children)

    Yeah I'm just beginning my local AI journey on a 5080, tried Qwen3.6 27b Q4 and was getting like 1tps because of the vram overflow. Ran it over night at it was still chewing on generating a prompt for a sub agent when I got up in the middle of the night until it simply ended in some kind of "fetch failure" lol. I think I gave it something too large to tackle, but either way 1tps is kinda garbage.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 14 points 1 week ago (1 child)

    You could use the 35B MoE model, tune it a little bit and get much better results. I have a 5060 ti and 70-80 tok/s are the norm

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 week ago (1 child)

    That's what I generally use. I wanted to see if I could use the 27b to "review" what the 35b put out. The 35b has been working pretty well, but it's not very thorough. I asked it to make a program and then 27b was like "this is a skeleton, there are folders but no contents." Lol

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 1 week ago (1 child)

    These models generally are not capable enough to do one-shot vibe coding. They are pretty good as coding assistants if you tell them exactly what you want and let them focus on a specific aspect/part of code, not the whole thing at once.

    Using an agent framework (I like Kilo on VSCode but there are many others) you can start with a planning session to let the model find out what you want to build. Then you let it write that gist into AGENTS.md and double check if that is what you want. AGENTS.md will be loaded into the context automatically so the model has a solid base of understanding for everything you do afterwards. Once you have that, building in vertical slices on top of that skeleton is much easier. Another neat trick is to ask the model a few questions about the current code base (if there is any...) at the beginning of a session, e.g. "How does feature x work in the current code? ". This primes the model for what you are about to do. All this is obviously a bit more work than just vibe coding away, but it lets you keep in control of the code and helps in being alert for errors these models (and all LLMs in general) will inevitably produce.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 1 week ago

    Thanks for the tips! What I had started doing the other day was having one session where it reviewed the code and created a document explaining what the program did in technical detail and then in another session I asked it to review that document before attempting anything with the program. Agent programs and harnesses are the next thing I need to start learning for sure.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 1 week ago (1 child)

    Is that quantized? 4 bit Qwen 3.6 can get 22tps on a 1060.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 11 points 1 week ago* (1 child)

    The engine is a single C file (c/glm.c, ~2,400 lines)

    That file is almost 6k lines. The style also makes my eyes bleed. Why do people pretend stuffing 6k lines of code with almost no whitespace and meaningless variable names into a single file is a good thing? I've seen this a lot recently

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 1 week ago (1 child)

    Assuming the vars are all really short, it sounds like the same idea behind Webpack's (et al) minification & mangling to achieve tiny performance gains everywhere it can. Which might mean there's a dev version that isn't squashed and ugly, but doesn't make it to us..?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 1 week ago

    Actually, it's better at programming if you don't know how to program.

    For example, someone who didn't know how to program sent me a patch to my code to 'fix' a problem they encountered. It was changed to silently swallow the error that really really would have needed to be fixed, but they were so enthusiastic that the problem looked to go away that they wouldn't let me know actually actionable debug information and just whined that I wouldn't take Claude's perfectly good fix for it. It is much better at satisfying folks that don't know how to look for better.

    To satisfy my standards, I would have had to extract more debugging info, probably construct a test case tickling that exact situation making an expected error to ensure it won't just throw it away, and add it to the suite and then ask it to spin until the test case passed.

    But alas, I'm neck deep in merge requests from folks that were formerly intimidated by coding and are now complaining that I'm not accepting more of their changes and more quickly because "all" I have to do is review the code which is obviously so much easier than writing it... It's like the "I have an app idea, all I need is for you to do it", but on steriods....

  • permalink
  • fedilink
  • source
  • parent